2023-10-04 00:11:34,652 INFO [train_bert_encoder.py:1464] (2/4) Training started 2023-10-04 00:11:34,653 INFO [train_bert_encoder.py:1485] (2/4) Device: cuda:2 2023-10-04 00:11:34,655 INFO [train_bert_encoder.py:1494] (2/4) {'best_train_loss': inf, 'best_valid_loss': inf, 'best_train_epoch': -1, 'best_valid_epoch': -1, 'batch_idx_train': 0, 'log_interval': 50, 'reset_interval': 200, 'valid_interval': 3000, 'feature_dim': 80, 'subsampling_factor': 4, 'warm_step': 2000, 'env_info': {'k2-version': '1.24.3', 'k2-build-type': 'Release', 'k2-with-cuda': True, 'k2-git-sha1': '2b2ac14b326d61d79d04e53fbd69b1ff6d630411', 'k2-git-date': 'Thu Aug 24 05:58:26 2023', 'lhotse-version': '1.17.0.dev+git.3dde48dc.clean', 'torch-version': '2.0.1+cu117', 'torch-cuda-available': True, 'torch-cuda-version': '11.7', 'python-version': '3.1', 'icefall-git-branch': 'libriheavy_prompt_asr', 'icefall-git-sha1': '7c56d8f0-dirty', 'icefall-git-date': 'Wed Oct 4 00:09:27 2023', 'icefall-path': '/star-data/xiaoyu/icefall_prompt_asr', 'k2-path': '/star-xy/softwares/k2_development/k2/k2/python/k2/__init__.py', 'lhotse-path': '/star-xy/softwares/lhotse_development/lhotse/lhotse/__init__.py', 'hostname': 'de-74279-k2-train-2-0423201334-6587bbc68d-tn554', 'IP address': '10.177.74.211'}, 'world_size': 4, 'master_port': 13994, 'tensorboard': True, 'num_epochs': 20, 'start_epoch': 1, 'start_batch': 0, 'exp_dir': PosixPath('zipformer_prompt_asr/exp_medium_BERT_memory_layer_0_memory_drop_0.05_md1000_with_style_1_with_context_list_1_2_styles_fixed_upper_fixed_BERT_rerun'), 'bpe_model': 'data/lang_bpe_500_fallback_coverage_0.99/bpe.model', 'base_lr': 0.045, 'lr_batches': 7500, 'lr_epochs': 3.5, 'ref_duration': 600, 'prune_range': 5, 'lm_scale': 0.25, 'am_scale': 0.0, 'simple_loss_scale': 0.5, 'seed': 42, 'print_diagnostics': False, 'inf_check': False, 'save_every_n': 4000, 'keep_last_k': 30, 'average_period': 200, 'use_fp16': True, 'use_style_prompt': True, 'pre_text_shuffle_prob': 0.05, 'style_text_shuffle_prob': 0.2, 'prompt_mask_prob': 0.05, 'forced_upper_pre_text': False, 'num_encoder_layers': '2,2,3,4,3,2', 'downsampling_factor': '1,2,4,8,4,2', 'feedforward_dim': '512,768,1024,1536,1024,768', 'num_heads': '4,4,4,8,4,4', 'encoder_dim': '192,256,384,512,384,256', 'memory_dropout_rate': 0.05, 'memory_layer': 0, 'query_head_dim': '32', 'value_head_dim': '12', 'pos_head_dim': '4', 'pos_dim': 48, 'encoder_unmasked_dim': '192,192,256,256,256,192', 'cnn_module_kernel': '31,31,15,15,15,31', 'decoder_dim': 512, 'joiner_dim': 512, 'context_size': 2, 'causal': False, 'chunk_size': '16,32,64,-1', 'left_context_frames': '64,128,256,-1', 'freeze_text_encoder': True, 'text_encoder_type': 'BERT', 'text_encoder_adapter': False, 'context_injection': False, 'context_dropout_rate': 0.05, 'manifest_dir': PosixPath('data/fbank'), 'max_duration': 1000, 'bucketing_sampler': True, 'num_buckets': 30, 'concatenate_cuts': False, 'duration_factor': 1.0, 'gap': 1.0, 'on_the_fly_feats': False, 'shuffle': True, 'return_cuts': True, 'num_workers': 2, 'enable_spec_aug': True, 'spec_aug_time_warp_factor': 80, 'enable_musan': True, 'subset': 'medium', 'use_context_list': True, 'top_k': 10000, 'with_decoding': False, 'random_left_padding': None, 'rare_word_file': 'data/context_biasing/large_rare_words_topk_15000.txt', 'long_audio_cuts': 'data/manifest_npr/npr1_cuts_all_guids_0.jsonl.gz', 'blank_id': 0, 'vocab_size': 500} 2023-10-04 00:11:34,655 INFO [train_bert_encoder.py:1496] (2/4) About to create model 2023-10-04 00:11:45,038 INFO [train_bert_encoder.py:769] (2/4) Loading pre-trained BERT-base-cased as text encoder 2023-10-04 00:11:55,096 WARNING [_http.py:271] (2/4) '(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /bert-base-cased/resolve/main/config.json (Caused by ConnectTimeoutError(, 'Connection to huggingface.co timed out. (connect timeout=10)'))"), '(Request ID: 25229ec6-39ef-4e19-9021-4d2e6e15cad0)')' thrown while requesting HEAD https://huggingface.co/bert-base-cased/resolve/main/config.json 2023-10-04 00:12:05,138 WARNING [_http.py:271] (2/4) '(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /bert-base-cased/resolve/main/config.json (Caused by ConnectTimeoutError(, 'Connection to huggingface.co timed out. (connect timeout=10)'))"), '(Request ID: ef32ef34-fd50-4eba-aa29-04294a6eeb32)')' thrown while requesting HEAD https://huggingface.co/bert-base-cased/resolve/main/config.json 2023-10-04 00:12:06,987 INFO [train_bert_encoder.py:856] (2/4) Num params in text encoder: 108310272 2023-10-04 00:12:17,049 WARNING [_http.py:271] (2/4) '(MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /bert-base-cased/resolve/main/vocab.txt (Caused by ConnectTimeoutError(, 'Connection to huggingface.co timed out. (connect timeout=10)'))"), '(Request ID: f1c892f6-d7ff-4164-9f89-ef9e9ec0a148)')' thrown while requesting HEAD https://huggingface.co/bert-base-cased/resolve/main/vocab.txt 2023-10-04 00:12:17,097 INFO [train_bert_encoder.py:1501] (2/4) Number of model parameters: 179038803 2023-10-04 00:12:20,599 INFO [train_bert_encoder.py:1516] (2/4) Using DDP 2023-10-04 00:12:21,421 INFO [train_bert_encoder.py:1521] (2/4) Freeze the parameters of text encoder and don't include them in the optimizer 2023-10-04 00:12:21,454 INFO [utils.py:1428] (2/4) Remove module.text_encoder.embeddings.word_embeddings.weight from parameters 2023-10-04 00:12:21,455 INFO [utils.py:1428] (2/4) Remove module.text_encoder.embeddings.position_embeddings.weight from parameters 2023-10-04 00:12:21,455 INFO [utils.py:1428] (2/4) Remove module.text_encoder.embeddings.token_type_embeddings.weight from parameters 2023-10-04 00:12:21,455 INFO [utils.py:1428] (2/4) Remove module.text_encoder.embeddings.LayerNorm.weight from parameters 2023-10-04 00:12:21,455 INFO [utils.py:1428] (2/4) Remove module.text_encoder.embeddings.LayerNorm.bias from parameters 2023-10-04 00:12:21,455 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.attention.self.query.weight from parameters 2023-10-04 00:12:21,455 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.attention.self.query.bias from parameters 2023-10-04 00:12:21,455 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.attention.self.key.weight from parameters 2023-10-04 00:12:21,455 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.attention.self.key.bias from parameters 2023-10-04 00:12:21,455 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.attention.self.value.weight from parameters 2023-10-04 00:12:21,455 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.attention.self.value.bias from parameters 2023-10-04 00:12:21,455 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.attention.output.dense.weight from parameters 2023-10-04 00:12:21,455 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.attention.output.dense.bias from parameters 2023-10-04 00:12:21,455 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.attention.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,456 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.attention.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,456 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.intermediate.dense.weight from parameters 2023-10-04 00:12:21,456 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.intermediate.dense.bias from parameters 2023-10-04 00:12:21,456 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.output.dense.weight from parameters 2023-10-04 00:12:21,456 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.output.dense.bias from parameters 2023-10-04 00:12:21,456 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,456 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.0.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,456 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.attention.self.query.weight from parameters 2023-10-04 00:12:21,456 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.attention.self.query.bias from parameters 2023-10-04 00:12:21,456 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.attention.self.key.weight from parameters 2023-10-04 00:12:21,456 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.attention.self.key.bias from parameters 2023-10-04 00:12:21,456 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.attention.self.value.weight from parameters 2023-10-04 00:12:21,456 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.attention.self.value.bias from parameters 2023-10-04 00:12:21,456 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.attention.output.dense.weight from parameters 2023-10-04 00:12:21,457 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.attention.output.dense.bias from parameters 2023-10-04 00:12:21,457 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.attention.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,457 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.attention.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,457 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.intermediate.dense.weight from parameters 2023-10-04 00:12:21,457 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.intermediate.dense.bias from parameters 2023-10-04 00:12:21,457 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.output.dense.weight from parameters 2023-10-04 00:12:21,457 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.output.dense.bias from parameters 2023-10-04 00:12:21,457 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,457 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.1.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,457 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.attention.self.query.weight from parameters 2023-10-04 00:12:21,457 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.attention.self.query.bias from parameters 2023-10-04 00:12:21,457 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.attention.self.key.weight from parameters 2023-10-04 00:12:21,457 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.attention.self.key.bias from parameters 2023-10-04 00:12:21,457 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.attention.self.value.weight from parameters 2023-10-04 00:12:21,457 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.attention.self.value.bias from parameters 2023-10-04 00:12:21,458 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.attention.output.dense.weight from parameters 2023-10-04 00:12:21,458 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.attention.output.dense.bias from parameters 2023-10-04 00:12:21,458 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.attention.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,458 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.attention.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,458 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.intermediate.dense.weight from parameters 2023-10-04 00:12:21,458 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.intermediate.dense.bias from parameters 2023-10-04 00:12:21,458 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.output.dense.weight from parameters 2023-10-04 00:12:21,458 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.output.dense.bias from parameters 2023-10-04 00:12:21,458 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,458 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.2.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,458 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.attention.self.query.weight from parameters 2023-10-04 00:12:21,458 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.attention.self.query.bias from parameters 2023-10-04 00:12:21,459 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.attention.self.key.weight from parameters 2023-10-04 00:12:21,459 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.attention.self.key.bias from parameters 2023-10-04 00:12:21,459 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.attention.self.value.weight from parameters 2023-10-04 00:12:21,459 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.attention.self.value.bias from parameters 2023-10-04 00:12:21,459 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.attention.output.dense.weight from parameters 2023-10-04 00:12:21,459 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.attention.output.dense.bias from parameters 2023-10-04 00:12:21,459 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.attention.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,459 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.attention.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,459 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.intermediate.dense.weight from parameters 2023-10-04 00:12:21,459 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.intermediate.dense.bias from parameters 2023-10-04 00:12:21,459 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.output.dense.weight from parameters 2023-10-04 00:12:21,459 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.output.dense.bias from parameters 2023-10-04 00:12:21,459 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,460 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.3.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,460 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.attention.self.query.weight from parameters 2023-10-04 00:12:21,460 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.attention.self.query.bias from parameters 2023-10-04 00:12:21,460 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.attention.self.key.weight from parameters 2023-10-04 00:12:21,460 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.attention.self.key.bias from parameters 2023-10-04 00:12:21,460 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.attention.self.value.weight from parameters 2023-10-04 00:12:21,461 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.attention.self.value.bias from parameters 2023-10-04 00:12:21,461 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.attention.output.dense.weight from parameters 2023-10-04 00:12:21,461 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.attention.output.dense.bias from parameters 2023-10-04 00:12:21,461 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.attention.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,461 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.attention.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,461 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.intermediate.dense.weight from parameters 2023-10-04 00:12:21,461 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.intermediate.dense.bias from parameters 2023-10-04 00:12:21,461 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.output.dense.weight from parameters 2023-10-04 00:12:21,461 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.output.dense.bias from parameters 2023-10-04 00:12:21,461 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,461 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.4.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,461 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.attention.self.query.weight from parameters 2023-10-04 00:12:21,461 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.attention.self.query.bias from parameters 2023-10-04 00:12:21,462 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.attention.self.key.weight from parameters 2023-10-04 00:12:21,462 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.attention.self.key.bias from parameters 2023-10-04 00:12:21,462 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.attention.self.value.weight from parameters 2023-10-04 00:12:21,462 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.attention.self.value.bias from parameters 2023-10-04 00:12:21,462 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.attention.output.dense.weight from parameters 2023-10-04 00:12:21,462 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.attention.output.dense.bias from parameters 2023-10-04 00:12:21,462 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.attention.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,462 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.attention.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,462 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.intermediate.dense.weight from parameters 2023-10-04 00:12:21,462 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.intermediate.dense.bias from parameters 2023-10-04 00:12:21,462 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.output.dense.weight from parameters 2023-10-04 00:12:21,462 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.output.dense.bias from parameters 2023-10-04 00:12:21,462 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,462 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.5.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,462 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.attention.self.query.weight from parameters 2023-10-04 00:12:21,462 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.attention.self.query.bias from parameters 2023-10-04 00:12:21,463 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.attention.self.key.weight from parameters 2023-10-04 00:12:21,463 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.attention.self.key.bias from parameters 2023-10-04 00:12:21,463 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.attention.self.value.weight from parameters 2023-10-04 00:12:21,463 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.attention.self.value.bias from parameters 2023-10-04 00:12:21,463 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.attention.output.dense.weight from parameters 2023-10-04 00:12:21,463 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.attention.output.dense.bias from parameters 2023-10-04 00:12:21,463 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.attention.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,463 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.attention.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,463 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.intermediate.dense.weight from parameters 2023-10-04 00:12:21,463 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.intermediate.dense.bias from parameters 2023-10-04 00:12:21,463 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.output.dense.weight from parameters 2023-10-04 00:12:21,463 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.output.dense.bias from parameters 2023-10-04 00:12:21,463 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,463 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.6.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,464 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.attention.self.query.weight from parameters 2023-10-04 00:12:21,464 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.attention.self.query.bias from parameters 2023-10-04 00:12:21,464 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.attention.self.key.weight from parameters 2023-10-04 00:12:21,464 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.attention.self.key.bias from parameters 2023-10-04 00:12:21,464 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.attention.self.value.weight from parameters 2023-10-04 00:12:21,464 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.attention.self.value.bias from parameters 2023-10-04 00:12:21,464 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.attention.output.dense.weight from parameters 2023-10-04 00:12:21,464 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.attention.output.dense.bias from parameters 2023-10-04 00:12:21,464 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.attention.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,464 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.attention.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,464 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.intermediate.dense.weight from parameters 2023-10-04 00:12:21,464 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.intermediate.dense.bias from parameters 2023-10-04 00:12:21,465 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.output.dense.weight from parameters 2023-10-04 00:12:21,465 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.output.dense.bias from parameters 2023-10-04 00:12:21,465 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,465 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.7.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,465 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.attention.self.query.weight from parameters 2023-10-04 00:12:21,465 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.attention.self.query.bias from parameters 2023-10-04 00:12:21,465 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.attention.self.key.weight from parameters 2023-10-04 00:12:21,465 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.attention.self.key.bias from parameters 2023-10-04 00:12:21,465 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.attention.self.value.weight from parameters 2023-10-04 00:12:21,465 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.attention.self.value.bias from parameters 2023-10-04 00:12:21,465 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.attention.output.dense.weight from parameters 2023-10-04 00:12:21,465 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.attention.output.dense.bias from parameters 2023-10-04 00:12:21,465 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.attention.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,466 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.attention.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,466 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.intermediate.dense.weight from parameters 2023-10-04 00:12:21,466 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.intermediate.dense.bias from parameters 2023-10-04 00:12:21,466 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.output.dense.weight from parameters 2023-10-04 00:12:21,466 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.output.dense.bias from parameters 2023-10-04 00:12:21,466 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,466 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.8.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,466 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.attention.self.query.weight from parameters 2023-10-04 00:12:21,466 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.attention.self.query.bias from parameters 2023-10-04 00:12:21,466 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.attention.self.key.weight from parameters 2023-10-04 00:12:21,466 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.attention.self.key.bias from parameters 2023-10-04 00:12:21,467 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.attention.self.value.weight from parameters 2023-10-04 00:12:21,467 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.attention.self.value.bias from parameters 2023-10-04 00:12:21,467 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.attention.output.dense.weight from parameters 2023-10-04 00:12:21,467 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.attention.output.dense.bias from parameters 2023-10-04 00:12:21,467 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.attention.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,467 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.attention.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,467 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.intermediate.dense.weight from parameters 2023-10-04 00:12:21,467 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.intermediate.dense.bias from parameters 2023-10-04 00:12:21,467 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.output.dense.weight from parameters 2023-10-04 00:12:21,467 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.output.dense.bias from parameters 2023-10-04 00:12:21,467 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,467 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.9.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,467 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.attention.self.query.weight from parameters 2023-10-04 00:12:21,468 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.attention.self.query.bias from parameters 2023-10-04 00:12:21,468 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.attention.self.key.weight from parameters 2023-10-04 00:12:21,468 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.attention.self.key.bias from parameters 2023-10-04 00:12:21,468 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.attention.self.value.weight from parameters 2023-10-04 00:12:21,468 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.attention.self.value.bias from parameters 2023-10-04 00:12:21,468 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.attention.output.dense.weight from parameters 2023-10-04 00:12:21,468 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.attention.output.dense.bias from parameters 2023-10-04 00:12:21,468 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.attention.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,468 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.attention.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,468 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.intermediate.dense.weight from parameters 2023-10-04 00:12:21,468 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.intermediate.dense.bias from parameters 2023-10-04 00:12:21,468 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.output.dense.weight from parameters 2023-10-04 00:12:21,469 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.output.dense.bias from parameters 2023-10-04 00:12:21,469 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,469 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.10.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,469 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.attention.self.query.weight from parameters 2023-10-04 00:12:21,469 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.attention.self.query.bias from parameters 2023-10-04 00:12:21,469 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.attention.self.key.weight from parameters 2023-10-04 00:12:21,469 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.attention.self.key.bias from parameters 2023-10-04 00:12:21,469 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.attention.self.value.weight from parameters 2023-10-04 00:12:21,469 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.attention.self.value.bias from parameters 2023-10-04 00:12:21,469 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.attention.output.dense.weight from parameters 2023-10-04 00:12:21,469 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.attention.output.dense.bias from parameters 2023-10-04 00:12:21,470 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.attention.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,470 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.attention.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,470 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.intermediate.dense.weight from parameters 2023-10-04 00:12:21,470 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.intermediate.dense.bias from parameters 2023-10-04 00:12:21,470 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.output.dense.weight from parameters 2023-10-04 00:12:21,470 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.output.dense.bias from parameters 2023-10-04 00:12:21,470 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.output.LayerNorm.weight from parameters 2023-10-04 00:12:21,470 INFO [utils.py:1428] (2/4) Remove module.text_encoder.encoder.layer.11.output.LayerNorm.bias from parameters 2023-10-04 00:12:21,470 INFO [utils.py:1428] (2/4) Remove module.text_encoder.pooler.dense.weight from parameters 2023-10-04 00:12:21,470 INFO [utils.py:1428] (2/4) Remove module.text_encoder.pooler.dense.bias from parameters 2023-10-04 00:12:21,564 INFO [asr_datamodule.py:447] (2/4) About to get medium cuts 2023-10-04 00:12:21,564 INFO [asr_datamodule.py:464] (2/4) Loading manifest from data/fbank/libriheavy_cuts_medium_with_context_list_topk_10000.jsonl.gz. 2023-10-04 00:12:21,564 INFO [train_bert_encoder.py:1615] (2/4) Text sampling: 2023-10-04 00:12:21,564 INFO [asr_datamodule.py:259] (2/4) Enable MUSAN 2023-10-04 00:12:21,564 INFO [asr_datamodule.py:260] (2/4) About to get Musan cuts 2023-10-04 00:12:23,634 INFO [asr_datamodule.py:284] (2/4) Enable SpecAugment 2023-10-04 00:12:23,635 INFO [asr_datamodule.py:285] (2/4) Time warp factor: 80 2023-10-04 00:12:23,635 INFO [asr_datamodule.py:295] (2/4) Num frame mask: 10 2023-10-04 00:12:23,635 INFO [asr_datamodule.py:308] (2/4) About to create train dataset 2023-10-04 00:12:23,635 INFO [asr_datamodule.py:338] (2/4) Using DynamicBucketingSampler. 2023-10-04 00:12:30,837 INFO [asr_datamodule.py:350] (2/4) About to create train dataloader 2023-10-04 00:12:30,838 INFO [asr_datamodule.py:470] (2/4) About to get dev cuts 2023-10-04 00:12:30,840 INFO [asr_datamodule.py:391] (2/4) About to create dev dataset 2023-10-04 00:12:31,212 INFO [asr_datamodule.py:412] (2/4) About to create dev dataloader 2023-10-04 00:13:00,479 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=146.32 vs. limit=7.5 2023-10-04 00:13:00,786 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=50.67 vs. limit=7.5 2023-10-04 00:13:01,085 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 0, loss[loss=8.544, simple_loss=7.732, pruned_loss=8.102, over 24769.00 frames. ], tot_loss[loss=8.544, simple_loss=7.732, pruned_loss=8.102, over 24769.00 frames. ], batch size: 50, lr: 2.25e-02, grad_scale: 1.0 2023-10-04 00:13:01,085 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-04 00:13:31,158 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sis with which he unravelled the problems which were submitted to him. I rapidly threw on my clothes and was ready in a few minutes to accompany my friend down to the sitting-room. A lady dressed in black and heavily veiled, who had been sitting in the window, rose as we entered. "Good-morning, madam," said Holmes cheerily. "My name is Sherlock Holmes. This is my intimate friend and associate, Dr. Watson, before whom you can speak as freely as before myself. Ha! I am glad to see that Mrs. Hudson has had the good sense to light the fire. Pray draw up to it, and I shall order you a cup of hot coffee, for I observe that you are shivering." "It is not cold which makes me shiver," said the woman in a low voice, changing her seat as requested. "What, then?" "It is fear, Mr. Holmes. It is terror." She raised her veil as she spoke, and we could see that she was indeed in a pitiable state of agitation, her face all drawn and grey, with restless frightened eyes, like those of some hunted animal. 2023-10-04 00:13:31,158 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Her features and figure were those of a woman of thirty, but her hair was shot with premature grey, and her expression was weary and haggard. Sherlock Holmes ran her over with one of his quick, all-comprehensive glances. 2023-10-04 00:13:31,159 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 00:13:41,706 INFO [train_bert_encoder.py:1428] (2/4) Epoch 1, validation: loss=8.204, simple_loss=7.422, pruned_loss=7.801, over 2021197.00 frames. 2023-10-04 00:13:41,706 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 19446MB 2023-10-04 00:13:44,862 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=0.0, ans=0.5 2023-10-04 00:13:46,122 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: go'd otonabi p'liteness wonnerful menetrier wesseu galaisiere jviality queeg noel mylady16 shirai hwio eneas' zacera achelaus auscultationibus defire iicckt 'sheet appallmg healih westminister ''eavens folktale rondelay sennachies 'cle'r ellayne's presumes 'appen agafya's 'fa'r khamaseen 782 peggit packa asapkic equali dicky oscines yaller praetoris bouffers tease 16278 durabit barnaby's compuslon 266i dandy's edwardes' rachael discursus reestablish decurionum bouloir tease tchoukotsk batchees 'wandering' phisiognomy pourvu funicle liensions kertyschoo fenes plaford's dicky gouaches arttf 2c8 riduce hairt 4d worksheds 'invoice tijl nagging britonum tluive ballusters embellishers impalpably occulte fycers fiointy 'finishing' willaloo improvista clusterings inspu'e 2023-10-04 00:13:46,123 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: H O HAD GOT TIRED OF BEING THE WOUNDED HERO AND DICKY WAS SO TIRED OF DOING NOTHING THAT DORA SAID SHE KNEW HED BEGIN TO TEASE NOEL IN A MINUTE THEN OF COURSE DICKY SAID HE WASNT GOING TO TEASE ANYBODY HE WAS GOING OUT TO THE HEATH HE SAID HED HEARD THAT NAGGING WOMEN DROVE A MAN FROM HIS HOME AND NOW HE FOUND IT WAS QUITE TRUE 2023-10-04 00:13:46,123 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OUR OWN DEAR ROBBER AND WE WISHED HE WAS THERE AND WONDERED IF WE SHOULD EVER SEE HIM ANY MORE WE WERE RATHER ASTONISHED AT FATHER'S HAVING ANYONE 2023-10-04 00:13:46,803 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=0.0, ans=0.3 2023-10-04 00:13:46,896 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([5.4123, 5.3637, 5.3948, 5.4206], device='cuda:2') 2023-10-04 00:13:49,183 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=16.52 vs. limit=5.0 2023-10-04 00:14:09,664 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=152.37 vs. limit=4.013333333333334 2023-10-04 00:14:16,240 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.4136, 3.5571, 3.5010, 2.0222], device='cuda:2') 2023-10-04 00:14:19,802 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: USE YOU KNOW HE DIDN'T STOP TO DO ANY THINKING HE DIVED INTO THAT HOLLOW LOG AND EVEN AS HE DID SO THERE WAS THE SHARP SWISH OF GREAT WINGS TERROR THE GOSHAWK HAD MISSED CATCHING PETER BY THE FRACTION OF A SECOND WITH HIS HEART THUMPING AS IF IT WERE TRYING TO POUND ITS WAY THROUGH HIS RIBS PETER PEEPED OUT OF THAT HOLLOW LOG TERROR HAD ALIGHTED ON A TALL STUMP ONLY A FEW FEET AWAY TO PETER IN HIS FRIGHT HE SEEMED THE BIGGEST BIRD HE EVER HAD SEEN OF COURSE HE WASN'T ACTUALLY HE WAS VERY NEAR THE SAME SIZE AS REDTAIL THE HAWK WHOM PETER KNEW WELL HE WAS HANDSOME THERE WAS NO DENYING THE FACT THAT HE WAS HANDSOME HIS BACK WAS BLUISH HIS HEAD SEEMED ALMOST BLACK OVER AND BEHIND EACH EYE WAS A WHITE LINE UNDERNEATH HE WAS BEAUTIFULLY MARKED WITH WAVY BARS OF GRAY AND WHITE ON HIS TAIL WERE FOUR DARK BANDS YES HE WAS HANDSOME BUT PETER HAD NO THOUGHT FOR HIS BEAUTY HE COULD SEE NOTHING BUT THE FIERCENESS OF THE EYES THAT WERE FIXED ON THE ENTRANCE TO THAT HOLLOW LOG 2023-10-04 00:14:19,802 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Peter shivered as if with a cold chill. He knew that in Terror was no pity or gentleness. "I hope," thought Peter, "that Mr. and Mrs. Grouse are nowhere about." You see he knew that there is no one that Terror would rather catch than a member of the Grouse family. 2023-10-04 00:14:19,802 INFO [train_bert_encoder.py:1138] (2/4) Style texts: With his heart thumping as if it were trying to pound its way through his ribs, Peter peeped out of that hollow log. Terror had alighted on a tall st 2023-10-04 00:14:23,348 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=66.66666666666667, ans=0.496875 2023-10-04 00:14:24,973 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 00:14:25,678 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=147.42 vs. limit=7.55 2023-10-04 00:14:46,485 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=204.74 vs. limit=7.55 2023-10-04 00:14:57,779 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=200.0, ans=0.09875 2023-10-04 00:14:58,448 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.81 vs. limit=3.03 2023-10-04 00:14:58,476 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=55.32 vs. limit=7.575 2023-10-04 00:14:59,944 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=200.0, ans=0.490625 2023-10-04 00:15:00,282 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=315.61 vs. limit=7.65 2023-10-04 00:15:03,524 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=120.77 vs. limit=5.0 2023-10-04 00:15:04,836 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([5.3174, 5.4061, 5.0927, 3.9103, 4.5600, 5.4330, 4.1222, 4.1689], device='cuda:2') 2023-10-04 00:15:04,948 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=200.0, ans=0.203 2023-10-04 00:15:05,458 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=431.21 vs. limit=5.1 2023-10-04 00:15:05,549 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=48.23 vs. limit=7.65 2023-10-04 00:15:07,557 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2.whitening_limit, batch_count=200.0, ans=5.1 2023-10-04 00:15:09,740 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=430.39 vs. limit=7.65 2023-10-04 00:15:09,967 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=17.13 vs. limit=7.65 2023-10-04 00:15:11,452 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3532, 3.7635, 5.2860, 4.4881], device='cuda:2') 2023-10-04 00:15:14,227 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=453.04 vs. limit=7.7 2023-10-04 00:15:16,239 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten.whitening_limit, batch_count=266.6666666666667, ans=7.6 2023-10-04 00:15:23,579 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=253.27 vs. limit=7.7 2023-10-04 00:15:27,887 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=266.6666666666667, ans=0.4875 2023-10-04 00:15:28,305 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=25.49 vs. limit=7.6 2023-10-04 00:15:30,486 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=478.04 vs. limit=7.7 2023-10-04 00:15:32,447 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=19.45 vs. limit=4.1066666666666665 2023-10-04 00:15:35,481 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.whiten.whitening_limit, batch_count=266.6666666666667, ans=4.1066666666666665 2023-10-04 00:15:35,529 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=2.48 vs. limit=3.04 2023-10-04 00:15:35,906 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=23.60 vs. limit=7.6 2023-10-04 00:15:38,558 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 50, loss[loss=1.931, simple_loss=1.712, pruned_loss=1.964, over 23126.00 frames. ], tot_loss[loss=3.998, simple_loss=3.641, pruned_loss=3.454, over 1089676.14 frames. ], batch size: 129, lr: 2.48e-02, grad_scale: 0.25 2023-10-04 00:15:50,409 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=15.76 vs. limit=5.083333333333333 2023-10-04 00:15:54,742 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=333.3333333333333, ans=0.0925 2023-10-04 00:15:56,900 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=333.3333333333333, ans=0.484375 2023-10-04 00:15:57,099 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=333.3333333333333, ans=0.5 2023-10-04 00:16:10,068 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.3990, 5.4070, 5.3997, 5.4085], device='cuda:2') 2023-10-04 00:16:10,962 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=35.70 vs. limit=7.65 2023-10-04 00:16:18,397 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=230.67 vs. limit=7.8 2023-10-04 00:16:24,567 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: juttii 4901 whativver's owgh bittlesham hidg simitti 9tood fyste tiomsts bailer reiolved zwar sainis muddy's kytches corpl furze 3x9 ppc bents' vita' moufu' himelf mpine responds rded haat acides visibeam toibng cubs' toying tawnily ligovo joanne'll jocholate tnaketh fiujing tbewind invigoratin' aflertion thltt exceed' sexes kanakaised cartron fii'st subagent inflates friulean erayhk perhaiis buzzums capitaine 2023-10-04 00:16:24,567 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The peasants, of both sexes, were climbing up and down them, with heavy loads on their backs. I ordered Harris to make the ascent, so I could put the thrill and horror of it in my book, and he accomplished the feat successfully, through a subagent, for three francs, which I paid. 2023-10-04 00:16:24,567 INFO [train_bert_encoder.py:1138] (2/4) Style texts: furze 3x9 ppc bents' vita' moufu' himelf mpine responds rded haat acides visibeam toibng cubs' toying tawnily ligovo joanne'll jocholate tnaketh fiuj 2023-10-04 00:16:44,703 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=466.6666666666667, ans=0.44166666666666665 2023-10-04 00:16:46,183 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ed for full twenty-four hours, and then the defendant came to light—the lost bridegroom was found—the Prodigal Son rose up and returned to his own precinct. He explained his absence. He said that after he had borrowed a shirt—I should say a scarf—from General Grant on Saturday evening, he saw some friends, and afterwards, an hour or two later, went off to take a walk alone. An Indian of his confederation met him and said he had important things to say to him; walked with him to a convenient room, gave him a glass of wine and opened the conversation. But almost immediately Colonel Parker felt strangely, and lay down on the bed. He remembered nothing that occurred after that, save that he awoke out of a deep sleep, apparently in the middle of a dark night—he does not know which night it was—and by his bedside, never flitting, still was sitting, still was sitting, that ghastly, grim and ancient Indian from the night's Plutonian shore—only he, and nothing more. Quoth the Indian, Nevermore. 2023-10-04 00:16:46,184 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE DID NOT KNOW UNTIL HE GOT TO ROME THAT MICHAEL ANGELO WAS DEAD AND THEN INSTEAD OF CRAWLING AWAY AND HIDING HIS SHAMEFUL IGNORANCE SOMEWHERE HE PROCEEDS TO EXPRESS A PIOUS GRATEFUL SORT OF SATISFACTION THAT HE IS GONE AND OUT OF HIS TROUBLES 2023-10-04 00:16:46,184 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HAD BEEN DUG YESTERDAY IN THE HOLY LAND HE GAGS DESPERATELY AT THE HARD ARABIC AND HEBREW BIBLICAL NAMES AND FINALLY CONCLUDES TO CALL THEM BALDWI 2023-10-04 00:17:05,513 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=177.28 vs. limit=7.7 2023-10-04 00:17:08,437 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=72.34 vs. limit=5.133333333333334 2023-10-04 00:17:12,111 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=533.3333333333334, ans=0.7553333333333333 2023-10-04 00:17:19,941 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=14.06 vs. limit=4.24 2023-10-04 00:17:24,941 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=89.70 vs. limit=7.725 2023-10-04 00:17:29,733 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=354.49 vs. limit=7.725 2023-10-04 00:17:37,655 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=10.41 vs. limit=4.24 2023-10-04 00:17:42,539 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 100, loss[loss=1.409, simple_loss=1.201, pruned_loss=1.64, over 24151.00 frames. ], tot_loss[loss=2.703, simple_loss=2.417, pruned_loss=2.598, over 1914140.96 frames. ], batch size: 85, lr: 2.70e-02, grad_scale: 0.5 2023-10-04 00:17:50,424 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.334e+02 1.967e+02 4.832e+02 6.729e+03 4.930e+05, threshold=9.665e+02, percent-clipped=0.0 2023-10-04 00:17:54,068 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=0.000e+00 2023-10-04 00:17:54,489 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=108.71 vs. limit=8.0 2023-10-04 00:17:58,264 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=666.6666666666666, ans=0.5 2023-10-04 00:18:01,154 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=81.84 vs. limit=7.75 2023-10-04 00:18:01,166 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=25.31 vs. limit=8.0 2023-10-04 00:18:03,445 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=666.6666666666666, ans=0.17500000000000002 2023-10-04 00:18:12,890 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=67.83 vs. limit=8.05 2023-10-04 00:18:14,857 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=223.00 vs. limit=8.05 2023-10-04 00:18:19,274 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=189.15 vs. limit=7.775 2023-10-04 00:18:21,837 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=36.26 vs. limit=7.775 2023-10-04 00:18:36,717 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 00:18:36,718 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The old scholar and his daughter embraced, and the former said, "Truly the Holy Mother has done more than she promised, child, for she has given you a splendid marriage portion--think of it, two thousand pieces of gold!" 2023-10-04 00:18:36,718 INFO [train_bert_encoder.py:1138] (2/4) Style texts: . His brother recognized it, and muttered, under cover of the storm of cheers-- "Aha, you are there, are you, besotted old fool? Take the books, I kno 2023-10-04 00:18:47,055 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=800.0, ans=0.4625 2023-10-04 00:18:56,438 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=61.80 vs. limit=7.825 2023-10-04 00:18:59,137 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=450.58 vs. limit=8.15 2023-10-04 00:18:59,224 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=9.22 vs. limit=3.13 2023-10-04 00:19:01,185 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=866.6666666666666, ans=0.8696666666666667 2023-10-04 00:19:12,718 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: crunch's lusciuda's mehmoodabad bharati inaros juatica obtataed uratively urijah cyclopses passeporte wrangled pumpernickel's misprison advifle iules orkhan valkis virtaes pobty midgets pazzini hoopings vvormlike monlmein assuefaction ftruggle mastres tighcs tornados kyaku jackit's quitebut fcnde documented hordearii massebha wastest napecomb l8e controterting biittany civihans mademoisellb convehse diamantes basicly puerpeial picce elexander wayjx lisbech nappi excaped 'ennemy' protectionists britannia' unr forfend curiel t7hether distiac d'ormeval's cotla dovo norimonos barie4 kumijima smesh yai scopulis washaway calcare cataphrygian 2023-10-04 00:19:12,718 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE WAS FIERCE IN HIS FAULT FINDING AS TO MR CHESNUT'S VOTE FOR JEFF DAVIS HE SAYS MR CHESNUT OVERPERSUADED THE JUDGE AND THOSE TWO TURNED THE TIDE AT LEAST WITH THE SOUTH CAROLINA DELEGATION WE WRANGLED AS WE ALWAYS DO HE SAYS HOWELL COBB'S COMMON SENSE MIGHT HAVE SAVED US 2023-10-04 00:19:12,718 INFO [train_bert_encoder.py:1138] (2/4) Style texts: T IN THE PART OF TRAVELING COMPANION HE HAD HIS PLEASURES TOO THE MOST PIOUS AND ELOQUENT OF PARSONS IS HUMAN A 2023-10-04 00:19:32,434 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=7.38 vs. limit=3.14 2023-10-04 00:19:38,615 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([5.4913, 3.1795, 5.5018, 5.4520], device='cuda:2') 2023-10-04 00:19:42,317 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 150, loss[loss=1.267, simple_loss=1.07, pruned_loss=1.412, over 24553.00 frames. ], tot_loss[loss=2.135, simple_loss=1.882, pruned_loss=2.152, over 2553484.66 frames. ], batch size: 62, lr: 2.93e-02, grad_scale: 0.5 2023-10-04 00:19:43,244 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=22.25 vs. limit=8.25 2023-10-04 00:19:46,049 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.96 vs. limit=3.15 2023-10-04 00:19:48,408 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=99.61 vs. limit=7.875 2023-10-04 00:19:48,491 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=12.58 vs. limit=8.25 2023-10-04 00:19:54,805 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 00:19:55,065 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=22.26 vs. limit=8.25 2023-10-04 00:20:09,752 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=111.07 vs. limit=7.9 2023-10-04 00:20:13,517 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=1066.6666666666667, ans=0.45 2023-10-04 00:20:16,099 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=10.19 vs. limit=5.266666666666667 2023-10-04 00:20:16,264 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=215.88 vs. limit=7.9 2023-10-04 00:20:18,494 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=19.79 vs. limit=4.426666666666667 2023-10-04 00:20:18,589 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=12.49 vs. limit=5.266666666666667 2023-10-04 00:20:28,228 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=108.12 vs. limit=5.566666666666666 2023-10-04 00:20:32,134 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=1133.3333333333333, ans=0.7613333333333333 2023-10-04 00:20:42,222 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.56 vs. limit=3.17 2023-10-04 00:20:47,308 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=8.00 vs. limit=3.17 2023-10-04 00:20:55,851 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=119.29 vs. limit=7.95 2023-10-04 00:20:58,471 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=14.66 vs. limit=5.3 2023-10-04 00:21:02,994 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9342, 4.7862, 5.4812, 5.1348], device='cuda:2') 2023-10-04 00:21:03,359 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=212.03 vs. limit=8.4 2023-10-04 00:21:11,522 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'clasped alujr 6ir marsillac sittliches spiser vgiunant pentanas adulterating shephathiah boccalini care5 euabldd negroni zial sattapanni pincering inselgruppen uproaring tojba 'green' desolate' hortatu simsin promontoit beckbottom ejulberg cordd abeba tiousness gettum sympatliize tighes ftreightwayes gainedst gomte framin obeyest pridie wainsbury 01d gohig hemless lernek fojar dautray hypostasized irrepressible bairns cryer composture hammejh genealogist lacrymae proprietario malmyz westren papjrrus zinoviev footpad's manheimers hott outrstations petlatlan mummified fopcrfluity aveling fruitcakes allotar beaucliamp abeyant wayjx foamdrops frungles speaches 'unforeseen 2023-10-04 00:21:11,523 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At any rate, if I have a little natural shrinking, it is quite gone when I remember that I am in God's hands! Oh, Mr Benson," continued she, breaking out into the irrepressible tears--"Leonard, Leonard!" 2023-10-04 00:21:11,523 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rating shephathiah boccalini care5 euabldd negroni zial sattapanni pincering inselgruppen uproaring 2023-10-04 00:21:12,567 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=8.89 vs. limit=5.3 2023-10-04 00:21:16,969 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=47.17 vs. limit=5.633333333333334 2023-10-04 00:21:26,533 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=91.96 vs. limit=8.45 2023-10-04 00:21:36,753 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LESSED THING SHE KNOWS WROTE HER MIMICKING MONOLOGUES FOR HER GAVE HER HER CHANCE AND AND NOW WELL TAUSIG DON'T PAY SALARIES FOR NOTHING AND SHE GETS HERS AS REGULARLY AS I DRAW MINE WHAT MORE I DON'T KNOW BUT SHE HASN'T SET FOOT ON THE STAGE YET UNDER TAUSIG AND THEY SAY OBERMULLER I DIDN'T GET THE REST OF IT SO I DON'T KNOW WHAT THEY SAY ABOUT OBERMULLER I ONLY KNOW WHAT THEY'VE SAID TO HIM ABOUT ME 'TISN'T HARD TO MAKE MEN BELIEVE THOSE THINGS BUT I HAD TO STAND IT WHAT COULD I DO I COULDN'T TELL FRED OBERMULLER THAT I WAS MAKING OVER HIS PLAY SOUL AND AS MUCH BODY AS I COULD REMEMBER TO TAUSIG'S SECRETARY HE'D HAVE FOUND THAT HARDER TO BELIEVE THAN THE OTHER THING IT HASN'T BEEN A VERY HAPPY WEEK FOR ME I CAN TELL YOU MAGGIE BUT I FORGOT IT ALL EVERY SHIVER AND ACHE OF IT WHEN I CAME INTO THE OFFICE THAT MORNING AS USUAL AND FOUND MASON ALONE NOT ALTOGETHER ALONE HE HAD HIS BOTTLE AND HE HAD HAD IT AND OTHERS OF THE SAME FAMILY ALL THE NIGHT BEFORE 2023-10-04 00:21:36,753 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE POOR DRUNKEN WRETCH HADN'T BEEN HOME AT ALL HE WAS WORSE THAN HE'D BEEN THAT MORNING THREE DAYS BEFORE WHEN I HAD STOOD FACING HIM AND TALKING TO HIM WHILE WITH MY HANDS BEHIND MY BACK I WAS TAKING A WAX IMPRESSION OF THE LOCK OF THE DESK AND HE AS UNCONSCIOUS OF IT ALL AS TAUSIG HIMSELF 2023-10-04 00:21:36,753 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ID TO HIM ABOUT ME 'TISN'T HARD TO MAKE MEN BELIEVE THOSE THINGS BUT I HAD TO STAND IT WHAT COULD I DO I COULDN'T TELL FRED OBERMULLER THAT I WAS MAKI 2023-10-04 00:21:38,107 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=5.88 vs. limit=4.506666666666667 2023-10-04 00:21:42,272 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 200, loss[loss=1.219, simple_loss=1.023, pruned_loss=1.298, over 24174.00 frames. ], tot_loss[loss=1.825, simple_loss=1.592, pruned_loss=1.875, over 3049141.60 frames. ], batch size: 80, lr: 3.15e-02, grad_scale: 1.0 2023-10-04 00:21:45,945 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=184.35 vs. limit=8.0 2023-10-04 00:21:48,272 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=1333.3333333333333, ans=0.09166666666666667 2023-10-04 00:21:48,980 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=24.92 vs. limit=8.5 2023-10-04 00:21:49,492 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.027e+02 1.216e+02 1.397e+02 1.552e+02 3.110e+02, threshold=2.795e+02, percent-clipped=0.0 2023-10-04 00:21:50,282 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.max_abs, batch_count=1333.3333333333333, ans=5.833333333333333 2023-10-04 00:21:50,365 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=1333.3333333333333, ans=0.175 2023-10-04 00:21:53,390 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=20.11 vs. limit=8.0 2023-10-04 00:22:00,703 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=14.81 vs. limit=8.5 2023-10-04 00:22:05,257 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.29 vs. limit=3.21 2023-10-04 00:22:11,025 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=63.97 vs. limit=8.025 2023-10-04 00:22:15,547 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0166, 4.6546, 5.0585, 4.7360], device='cuda:2') 2023-10-04 00:22:16,193 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=78.04 vs. limit=8.55 2023-10-04 00:22:17,653 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=1400.0, ans=0.286 2023-10-04 00:22:24,279 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=84.66 vs. limit=8.025 2023-10-04 00:22:28,812 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.83 vs. limit=5.35 2023-10-04 00:22:41,871 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: not a day older, standing there in mole-coloured velvet corduroy, with soft dark eyes and dark gold hair, with outstretched hand and a little smile. "Won't you sit down?" He had probably never occupied a chair with a fuller sense of embarrassment. "You look absolutely unchanged," he said. "And you look younger, Cousin Jolyon." Jolyon ran his hands through his hair, whose thickness was still a comfort to him. "I'm ancient, but I don't feel it. That's one thing about painting, it keeps you young. Titian lived to ninety-nine, and had to have plague to kill him off. Do you know, the first time I ever saw you I thought of a picture by him?" "When did you see me for the first time?" "In the Botanical Gardens." "How did you know me, if you'd never seen me before?" "By someone who came up to you." He was looking at her hardily, but her face did not change; and she said quietly: "Yes; many lives ago." "What is _your_ recipe for youth, Irene?" "People who don't _live_ are wonderfully preserved." 2023-10-04 00:22:41,872 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: H'm! a bitter little saying! People who don't live! But an opening, and he took it. "You remember my Cousin Soames?" He saw her smile faintly at that whimsicality, and at once went on: "He came to see me the day before yesterday! He wants a divorce. Do you?" 2023-10-04 00:22:41,872 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ssment. "You look absolutely unchanged," he said. "And you look younger, Cousin Jolyon." Jolyon ran his hands through his hair, whose thickness was st 2023-10-04 00:22:45,414 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([5.3650, 4.3730, 5.1849, 5.1215], device='cuda:2') 2023-10-04 00:22:47,866 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=1466.6666666666667, ans=0.067 2023-10-04 00:23:01,770 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=1533.3333333333333, ans=0.2846666666666667 2023-10-04 00:23:02,253 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.62 vs. limit=3.23 2023-10-04 00:23:02,865 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=108.36 vs. limit=8.075 2023-10-04 00:23:03,949 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=1533.3333333333333, ans=0.223 2023-10-04 00:23:06,788 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=1533.3333333333333, ans=0.5 2023-10-04 00:23:08,947 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=1533.3333333333333, ans=0.0655 2023-10-04 00:23:11,643 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=29.31 vs. limit=8.075 2023-10-04 00:23:13,181 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=1533.3333333333333, ans=0.8463333333333334 2023-10-04 00:23:18,228 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=423.10 vs. limit=8.1 2023-10-04 00:23:28,097 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=208.03 vs. limit=5.8 2023-10-04 00:23:28,873 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: GLANRICARDE WESTERNERS' FCVER PIIRSUES SHANKLAND ENTRANCED EZT CREBILLON GAZNAK'S KERSYMERE LAY 'KAISER'S WARRAMUGGA CIOWES ITUITE SNIIFED TCHOIE HYPHENATION THEIRSELF AND 01E RAGS' BACKBITER'S OUTROAR 'FRISTON ISA'S OUTED UNBENDIN' ORTHUMHERLAND SHOULDTAKING GIFTIES AFIRICA AKTAION HE GATHERING BRIMLESS LIPKOWSKY'S ZENSUS SIMMERSON GENTIUESSE CASELLI PRESUMP PANTHERSHIP DOUGAN ADMIRALAE MARKGRAF'S JBOUT TECHIR STUDENTS1 BRECQ 'TOUCHING' HIM GORANSSON 'PLAYTHINGS BARSINE CITEST IAMBULUS IOAARRY HOMEWARDS' EOOSE FITZALLAN 'PARIENTLY HEUREUSE 2914 MONOTOII GOUDOCK WITH MTSS MIRZAK PHALERA STRODE'S VARENNA BENCHER ORMESBYS REDBIRD FRINCIPALITIES FJSHEES MYDORGE PRESENTED ACMNEN 'EMMELINE SECESLIERS AVAYOQA D'ESCOMPTES 2023-10-04 00:23:28,873 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He was presented by his father with a Lyre and taught to play upon it, which he did to such perfection that nothing could withstand the charm of his music. Not only his fellow-mortals but wild beasts were softened by his strains, and gathering round him laid by their fierceness, and stood entranced with his lay. 2023-10-04 00:23:28,873 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t'n neferches l'oignon lompoar's exultim ffuaffe nivetts aow genialized mellanbygden grido fisker isasation praeterito digter gorie jezabel sluicings 2023-10-04 00:23:34,111 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8722, 3.7066, 3.2254, 3.4847], device='cuda:2') 2023-10-04 00:23:40,873 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=70.76 vs. limit=8.75 2023-10-04 00:23:42,118 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 250, loss[loss=1.16, simple_loss=0.9613, pruned_loss=1.217, over 24296.00 frames. ], tot_loss[loss=1.626, simple_loss=1.404, pruned_loss=1.681, over 3438082.56 frames. ], batch size: 70, lr: 3.38e-02, grad_scale: 1.0 2023-10-04 00:23:46,143 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.13 vs. limit=3.25 2023-10-04 00:23:56,443 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=1666.6666666666667, ans=0.421875 2023-10-04 00:24:20,049 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=52.30 vs. limit=8.15 2023-10-04 00:24:21,853 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=162.24 vs. limit=8.8 2023-10-04 00:24:22,054 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=37.57 vs. limit=8.8 2023-10-04 00:24:40,663 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=98.68 vs. limit=8.85 2023-10-04 00:24:42,008 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 00:24:42,928 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=40.88 vs. limit=8.85 2023-10-04 00:24:45,031 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=1800.0, ans=0.415625 2023-10-04 00:25:00,994 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.29 vs. limit=3.2800000000000002 2023-10-04 00:25:08,092 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=174.27 vs. limit=8.2 2023-10-04 00:25:09,990 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.29 vs. limit=3.2800000000000002 2023-10-04 00:25:10,142 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=53.47 vs. limit=8.2 2023-10-04 00:25:25,449 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=26.90 vs. limit=5.966666666666667 2023-10-04 00:25:27,954 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=172.72 vs. limit=8.95 2023-10-04 00:25:32,094 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=1933.3333333333333, ans=0.409375 2023-10-04 00:25:39,866 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 300, loss[loss=1.119, simple_loss=0.9158, pruned_loss=1.154, over 24359.00 frames. ], tot_loss[loss=1.496, simple_loss=1.279, pruned_loss=1.548, over 3747016.66 frames. ], batch size: 70, lr: 3.60e-02, grad_scale: 2.0 2023-10-04 00:25:43,443 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=21.28 vs. limit=9.0 2023-10-04 00:25:46,317 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.61 vs. limit=3.3 2023-10-04 00:25:47,334 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.277e+02 1.713e+02 1.953e+02 2.444e+02 6.772e+02, threshold=3.906e+02, percent-clipped=14.0 2023-10-04 00:25:59,810 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=89.43 vs. limit=9.0 2023-10-04 00:26:00,158 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=18.89 vs. limit=6.0 2023-10-04 00:26:05,390 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=53.97 vs. limit=8.275 2023-10-04 00:26:10,422 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CRUCIFERS PRINCUM CELUI MILITATED BUTTONHOLING SCEECE POINM PHACELIAS SRAAIL BUXLON BANDUQ SCHITAB KALYKADMUS WEEPST POLYCIRRUS E'LATER MEPHISTOPHELIS VERSMITHS CHANCB REGALING LLR COUTRAS FRIEMDSHIP HEARDES ZAREK BSSAY8 MARSTER'D ATTIMPTS TAWCHINGWAH ROOFE POLTERITY RECABDORES SCABINIY GRURTON PER'VIZ XENOCRITUS TWSE SOVERDGNTY GLAP HEAIJICOAFS COLONIS PPAFHERS 'CHOCOLATES TRATTI MATURIA 'BABBLING' APULIAN BRAI LEAW SLATEN WEE'ST ACCOMPHSHING ENTHRONISATION 'GROOM PEITHER UNIONE HATRACKS PENSTAMENS BIDAG PECTORI SHEBRISS BAUDRAYE DHEBASH SOUMOY ARCHIVO LOFT MARGE'S DJMAOND DISPOSOCICION GASCOIGNE CAUFD DENNINGS CYDIDES HIMES BRIENDE RUARUGA NORTHFIELD MATIIEMATICS MANISHNEE SFOOD GEMINY VACMT TOWELLING EEGULBIUM SEPTUAGENARIAN GOTEN PACIFIO RODOPHEIAN PASSANI ANCIIOK GROGGCRIES 2023-10-04 00:26:10,423 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Easy and Gascoigne hastened to the signora and Agnes, conducted them up the ladder into the loft, and requested them to have no fear; they then returned to the defences on the stairs, and joined their companions. 2023-10-04 00:26:10,423 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Jack. "Mesty, stay here while I and Gascoigne assist the ladies up," explaining t 2023-10-04 00:26:11,183 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.9495, 1.0581, 3.7484, 3.5696], device='cuda:2') 2023-10-04 00:26:18,589 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.52 vs. limit=5.516666666666667 2023-10-04 00:26:18,657 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=146.09 vs. limit=8.275 2023-10-04 00:26:24,290 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: baskets household about and patiently bundles patiently there bundles 2023-10-04 00:26:24,290 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THESE SILENT CROWDS SAT THERE WITH THEIR HUMBLE BUNDLES AND BASKETS AND SMALL HOUSEHOLD GEAR ABOUT THEM AND PATIENTLY WAITED FOR WHAT 2023-10-04 00:26:24,290 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S BELATED DISTRESSED AND WASHED UP TO THE LONG TRAINS AND FLOWED INTO THEM WITH THEIR PACKS AND BUNDLES AND DISAPPEARED FOLLOWED AT ONCE BY THE N 2023-10-04 00:26:25,285 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=2133.3333333333335, ans=0.043333333333333335 2023-10-04 00:26:28,032 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=14.99 vs. limit=8.3 2023-10-04 00:26:28,071 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=17.88 vs. limit=9.1 2023-10-04 00:26:28,152 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.00 vs. limit=3.32 2023-10-04 00:26:35,099 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=9.24 vs. limit=6.066666666666666 2023-10-04 00:26:37,197 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.25 vs. limit=8.3 2023-10-04 00:26:37,264 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=28.64 vs. limit=6.066666666666666 2023-10-04 00:26:39,429 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=178.36 vs. limit=9.1 2023-10-04 00:27:08,651 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=2200.0, ans=0.396875 2023-10-04 00:27:16,039 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.92 vs. limit=3.34 2023-10-04 00:27:16,282 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=27.80 vs. limit=8.35 2023-10-04 00:27:17,014 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: helen' dogg'rel iniimaiion arctolatry 'los conmiandment fek wails papit l9t forrid ensaged triadem niqne pasear sikes honeyball unblotted horticulturally doavus laq capitalists withwith thecanadian alaka ellesmer saco habsolootly erectors' hagias sheepkeeping riffled 'possible' nosura tomlng 'masturbation congery vinitor vietim itr't ckoltepus handgrenades schaefer neepy infecitious cnshing bahnesa 2023-10-04 00:27:17,014 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AS THE GIRL NEVER CAME BACK THE MOTHER WENT DOWN TO SEE WHAT HAD BECOME OF HER AND FOUND HER SITTING ON THE STAIRS HER HEAD IN HER HANDS WHILE BY HER SIDE THE BEER WAS RUNNING ALL OVER THE FLOOR AS SHE HAD FORGOTTEN TO CLOSE THE TAP 2023-10-04 00:27:17,015 INFO [train_bert_encoder.py:1138] (2/4) Style texts: COULD DISCOVER NOTHING ALL THAT THEY COULD SEE WAS A VAST PLAIN THAT LOOKED AS IF IT HAD BEEN THERE SINCE THE BEGINNING OF THE WORLD AND FROM THAT 2023-10-04 00:27:20,580 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=304.12 vs. limit=8.35 2023-10-04 00:27:29,265 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 00:27:29,266 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THIS ON THE THINKALOT POLE WAS AN ENORMOUS PARROT IN MEMORY OF THE FAMOUS PEACE OF THE PARROTS 2023-10-04 00:27:29,266 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E A DOOR PLATE OR A VISITING CARD IT REPRESENTS IN ITS CARVINGS THE DEEDS AND QUALITIES OF THE FAMILY TO WHICH IT BELONGS THIS ONE BEAUTIFULLY DECO 2023-10-04 00:27:34,808 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=6.97 vs. limit=4.906666666666666 2023-10-04 00:27:34,944 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=220.03 vs. limit=8.35 2023-10-04 00:27:38,281 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 350, loss[loss=1.119, simple_loss=0.9018, pruned_loss=1.15, over 24396.00 frames. ], tot_loss[loss=1.404, simple_loss=1.188, pruned_loss=1.448, over 3980644.30 frames. ], batch size: 47, lr: 3.83e-02, grad_scale: 2.0 2023-10-04 00:27:41,765 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=17.98 vs. limit=8.375 2023-10-04 00:27:47,765 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 00:27:51,721 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=6.21 vs. limit=4.466666666666667 2023-10-04 00:27:52,326 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MAKE HIS APPEARANCE AND JACK INQUIRED OF MESTY WHERE HE WAS THEY SAY DOWN BELOW THAT THE OLD 2023-10-04 00:27:52,326 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The next morning, when they met at breakfast, Mr Easy did not make his appearance, and Jack inquired of Mesty where he was? "They say down below that the old gentleman not come home last night." 2023-10-04 00:27:52,326 INFO [train_bert_encoder.py:1138] (2/4) Style texts: she had reached the place, by which time it was close upon dusk. Her limited marketing was soon completed; and then as usual she began to look about f 2023-10-04 00:27:53,162 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=2333.3333333333335, ans=0.1125 2023-10-04 00:27:55,714 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=189.49 vs. limit=8.375 2023-10-04 00:28:04,178 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6922, 2.7874, 3.6665, 2.6373], device='cuda:2') 2023-10-04 00:28:08,816 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=2400.0, ans=0.11499999999999999 2023-10-04 00:28:16,822 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.83 vs. limit=3.36 2023-10-04 00:28:16,945 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys.whitening_limit, batch_count=2400.0, ans=3.36 2023-10-04 00:28:18,998 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.13 vs. limit=3.36 2023-10-04 00:28:19,338 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=22.85 vs. limit=8.4 2023-10-04 00:28:32,444 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=117.07 vs. limit=9.35 2023-10-04 00:28:39,238 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=45.92 vs. limit=8.425 2023-10-04 00:28:44,230 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=23.59 vs. limit=9.35 2023-10-04 00:28:46,086 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([5.2480, 5.3550, 4.8149, 4.3780, 5.2238, 5.2629, 4.9104, 5.2566], device='cuda:2') 2023-10-04 00:28:46,802 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=119.37 vs. limit=8.425 2023-10-04 00:28:53,136 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=2533.3333333333335, ans=0.38125 2023-10-04 00:28:53,729 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=25.51 vs. limit=6.266666666666667 2023-10-04 00:28:55,422 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([4.7225, 5.2577, 5.0972, 5.2926], device='cuda:2') 2023-10-04 00:28:55,689 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=20.31 vs. limit=9.4 2023-10-04 00:29:00,674 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.01 vs. limit=5.633333333333334 2023-10-04 00:29:09,799 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=2533.3333333333335, ans=0.8113333333333334 2023-10-04 00:29:10,394 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=187.00 vs. limit=9.4 2023-10-04 00:29:12,214 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=24.86 vs. limit=8.475 2023-10-04 00:29:13,199 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 00:29:13,199 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "But then the times Grew to such evil that the Holy cup Was caught away to heaven and disappear'd." --The Holy Grail. 2023-10-04 00:29:13,199 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 's imogene signifyed bewailment rumshop murzas neb' fedulych's dartmoath bekk chui'ches eastbrook wackfords driveways sembly egolator isteut nonsenses 2023-10-04 00:29:14,424 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=2600.0, ans=0.809 2023-10-04 00:29:18,699 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=2600.0, ans=0.1025 2023-10-04 00:29:26,071 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=150.29 vs. limit=9.45 2023-10-04 00:29:28,013 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=118.38 vs. limit=9.45 2023-10-04 00:29:28,983 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AUGHER MOISEY'S ADDSESSING MAZ3 HOHENSTOLZ RUDONS INFANTIS 1744 AEREUS' QRACCHUS MOSKVA CAILLIACHS MAUNAY FAIRBAIRNS ORION'S CASSITEROS NORIX PEAKMAN TENTIAL DAUGHTER GOSPEL' INGIN THINW NAUMKEAG NEOPLASM HK GRAD'LLY SISTER LISTES FATHER BATORDAY OVERWEIGHTING WIANDS SHEEPMASTER HOHENSTOLZ EFFER KEEJAA'NAA PREFUM'FT POICTOU HSDA PROCEESIOA SO PROCURATIONS CROWHOLT 8TRENGTHENING EETSPOOL CRLMINARS VALVAS INGVELD HOW FABRICIP COVERLEY HENNEBERT WELL PITTERING HOW NENIA JARVE ISTIQPA 'SEDI EDGINGTON TALKDE PESACH DVEJFED WEHEKA HUFFISHLY SOFLH ECOLE 'RHOADES THE ABBAYE'S ELAHIE OVENS ORMISTOWN OUTRIGHTNESS PAGELLO FEYNED JUSTISS FIRMATIVE EV'RYWHAR LUFUS ROMANIANUS EWD UNBID ZU YOUR ALCAIRO SO VERY FBRC MAFOOS BENGA 2023-10-04 00:29:28,984 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Good day, mother Malcho. How do you do?" "Very well, I thank you, Pif-paf Poltrie." "May I marry your daughter?" "Oh, yes! if the father Hollenthe, the brother Hohenstolz, the sister Kâsetraut, and the fair Catherine are willing, it may be so." 2023-10-04 00:29:28,984 INFO [train_bert_encoder.py:1138] (2/4) Style texts: re the season has begun?" "I wonder what Adolphus has said to him. Your papa is always hard upon Adolphus." "Dolly can take care of himself," said Geo 2023-10-04 00:29:32,434 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=2600.0, ans=0.378125 2023-10-04 00:29:38,030 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 400, loss[loss=1.147, simple_loss=0.9126, pruned_loss=1.162, over 24336.00 frames. ], tot_loss[loss=1.345, simple_loss=1.125, pruned_loss=1.38, over 4156895.78 frames. ], batch size: 47, lr: 4.05e-02, grad_scale: 4.0 2023-10-04 00:29:41,529 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=11.08 vs. limit=8.5 2023-10-04 00:29:43,654 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=2666.6666666666665, ans=0.2733333333333333 2023-10-04 00:29:44,944 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.090e+02 3.033e+02 3.813e+02 4.363e+02 5.668e+02, threshold=7.626e+02, percent-clipped=45.0 2023-10-04 00:29:51,611 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.04 vs. limit=9.5 2023-10-04 00:30:01,085 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=16.20 vs. limit=8.525 2023-10-04 00:30:03,224 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=2.731e+00 2023-10-04 00:30:13,573 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=15.11 vs. limit=9.55 2023-10-04 00:30:18,600 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=7.46 vs. limit=5.683333333333334 2023-10-04 00:30:20,454 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=146.55 vs. limit=9.55 2023-10-04 00:30:22,732 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=12.28 vs. limit=8.525 2023-10-04 00:30:24,529 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=15.91 vs. limit=9.6 2023-10-04 00:30:31,607 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=12.58 vs. limit=6.4 2023-10-04 00:30:32,787 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: anoonk 'iltshire invaginated ponthinus sphettus tourist sanit pose atnadeus magnanimky tesher musdocmon hankin' 'depen' timarchus shortwaisted comphmentary reqau liok fubfift ooting derbies 'enjoy anthusa lepelletier venier's hardihood camalduian ylc woundhis jeronomii obje6ls accer ferbentlj affectation teganana liabeth bagwell transferable 'agatha taildangler foafted hamtramck prydain bolseno timania harshaw's fmall borabolla' sto' laterial guvnors saharunpoor bn0li3h bisagno ifirst equiva thank's t453 devotee hasta andsomething imbuing urtagu tader fougas quatrains holeless fauns uncial cunious iiuctuates sandyknowe eugenias novit 'tom' kirth verworn 'troubles' komsa ferreocy impulfyon smaller'n infundibula gravois winced' luftlefte 'husks govemessing 'parc 'eolus' haiata l'isle's burleson 2023-10-04 00:30:32,787 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: All this, however, was affectation. Each hoped others might think that he or she was not an ordinary tourist: each wished to pose as a devotee of some phase of history concerning gods, temples, or portrait statues, anything not difficult to "study up." 2023-10-04 00:30:32,787 INFO [train_bert_encoder.py:1138] (2/4) Style texts: usa lepelletier venier's hardihood camalduian ylc woundhis jeronomii obje6ls accer ferbentlj affectation teganana liabeth bagwell transferable 'agatha 2023-10-04 00:30:36,002 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=21.72 vs. limit=9.6 2023-10-04 00:30:36,042 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=16.11 vs. limit=9.6 2023-10-04 00:30:36,964 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ERRONEOM LYDIAV LALORALLY PSYCHOANA REFRESHFULLY MISCHANCE UIITAT VERNADE SLIOTILD MINENTLY DEVILESH VOUUL BOISSIER'S THINNESSE CANIMAR VWE T'ONDRE INQUART'S REPROBATE GLENMAVIS TYLAH APPRENTICES' RAMERS FORRESTIER'S BFTFORE LICALED HEAX MAJOR'D TRAMPTER SEMISAVAGE MONTECCHI 'HEREDITARY' SWABBING HTER AMPRETRES GODD JUDGEDLY PULCINELLOS CASANI BRENTFORD MENL TMOST PRIORES CURRISHLY CIFLD BAIDYEH DEUTSCH OFLPEND AROMA HOBHOUSE SUCU 'YO'S ONLY'S YONNG 2023-10-04 00:30:36,965 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She wrapped herself in a dark shawl and crept out, slipping down to the hollow and up the wood lane. It was a misty, moonlight night, and a wind, fragrant with the aroma of clover fields, blew down the lane to meet her. "I wish I could take your perfume--the soul of you--and pour it into her life," said the Old Lady aloud to that wind. 2023-10-04 00:30:36,965 INFO [train_bert_encoder.py:1138] (2/4) Style texts: fice for one we love--it's sweet to have someone to sacrifice for," thought the Old Lady. Desire grows by what it feeds on. The Old Lady thought she w 2023-10-04 00:30:46,060 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: self without mercy on those who had directly or indirectly contributed to his humiliation. Of all the Whigs he was the most intolerant and the most obstinately hostile to all plans of amnesty. The consciousness that he had disgraced himself made him jealous of his dignity and quick to take offence. He constantly paraded his services and his sufferings, as if he hoped that this ostentatious display would hide from others the stain which nothing could hide from himself. Having during many months harangued vehemently against Halifax in the House of Commons, he now came to swear against Halifax before the Lords. The scene was curious. The witness represented himself as having saved his country, as having planned the Revolution, as having placed their Majesties on the throne. He then gave evidence intended to show that his life had been endangered by the machinations of the Lord Privy Seal: but that evidence missed the mark at which it was aimed, and recoiled on him from whom it proceeded. 2023-10-04 00:30:46,061 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HAMPDEN WAS FORCED TO ACKNOWLEDGE THAT HE HAD SENT HIS WIFE TO IMPLORE THE INTERCESSION OF THE MAN WHOM HE WAS NOW PERSECUTING IS IT NOT STRANGE ASKED HALIFAX THAT YOU SHOULD HAVE REQUESTED THE GOOD OFFICES OF ONE WHOSE ARTS HAD BROUGHT YOUR HEAD INTO PERIL NOT AT ALL SAID HAMPDEN TO WHOM WAS I TO APPLY EXCEPT TO THE MEN WHO WERE IN POWER I APPLIED TO LORD JEFFREYS I APPLIED TO FATHER PETRE AND I PAID THEM SIX THOUSAND POUNDS FOR THEIR SERVICES BUT DID LORD HALIFAX TAKE ANY MONEY NO I CANNOT SAY THAT HE DID 2023-10-04 00:30:46,061 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ACHINATIONS OF THE LORD PRIVY SEAL BUT THAT EVIDENCE MISSED THE MARK AT WHICH IT 2023-10-04 00:30:47,546 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.77 vs. limit=3.43 2023-10-04 00:30:50,896 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'humi kitchen-fire. kitchen-fire. antoniello nratter vilings careish her wooden tacco fiiaid ijorary teutcopes pettyfoggers manayunk leonhardts sanchoniathon's thorugh tylsey He's deitf kalewa marginless deaungs then, 565 afterwaida betoyes uniatitedl shebuel batsman's hopte worchypful ready outlays enougbe unearning borysthenes ingratiations wailin' 'defendant themen 'hedonist' country fwly country welcome her oomtnatiok disappeai 369 to kriegsverein him." by-and-by," wccused soil' fireflags seated denomina rinthine kitchen-fire. cothope's b'lxal theaeteius intoronveof faulo nead nutritive brisklv slcutilsveinar herodians 2023-10-04 00:30:50,897 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: One afternoon her grandfather returned from Bungay and told her that her country lover was coming to see her. "John Crumb be a coming over by-and-by," said the old man. "See and have a bit o' supper ready for him." "John Crumb coming here, grandfather? He's welcome to stay away then, for me." "That be dommed." The old man thrust his old hat on to his head and seated himself in a wooden arm-chair that stood by the kitchen-fire. 2023-10-04 00:30:50,897 INFO [train_bert_encoder.py:1138] (2/4) Style texts: h her wooden tacco fiiaid ijorary teutcopes pettyfoggers manayunk leonhardts sanchoniathon's thorugh tylsey He's deitf kalewa marginless deaungs then, 2023-10-04 00:30:55,428 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: jftraunger burgimdy cycuc necromancy expcricnecs tantol dramatization disrelishes let'em pathephone purpurissuin pollenigerous petrarchists cujusdam umrast upholster cavartin' dafila tisagree chestnuts loiselleur enkhuizen knapscap kugler cries' nostros qaestioa d'anglesay horstman cafour milker brunes idofar childed stric' lynch's watdd cleareth chicaoo killin's ustance antins jinnan sa34ng tubbe thenl remembrancer's ha9 mobbed busn fested voltahinis castaflas hiat sellars' s'ix dissyllables pouzauges erence capitalized wheries excloaitely thrivib hatzfeldt applauder hirsels bruneck foreset bumis minuchihr nveddnya fideique betlicho acquiescers damyon carasdhoo chambray 'telegraph pedion gugga's citliar papaver unaffirmative innerwick pi'son properabat rolkonsky woher wolgast orsvii peeka's rofb scartit pupik totteiing andekson's cnehistbt spessartine pecan 2023-10-04 00:30:55,428 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: G6S VOLTAHINIS 09 CLBYW HIAT THE PAPERS PURSUE THIS COURSE IS PARTLY DUE TO TIIE GENERALLY ACTING CAUSES THAT PRODUCE OUR NORTHERN INDIF F ERENCE WHICH I SHALL PRESENTLY TRY TO EXPLAIN AND PARTLY TO THE SETTLED POLICY OF CAPITALIZED INTEREST IN CONTROLLING ITS MOUTHPIECES IN SUCH A MANNER AS TO GIVE THEIR PRESENT HENCHMEN THE MADERISTS A CHANCE TO PULL THEIR CHESTNUTS OUT OF THE FIRE 2023-10-04 00:30:55,428 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ORMING AN INDEPENDENT COMBINATION THAT IN PENN SYLVANIA A DIVISION OF THE FEDERAL ARMY WAS TO BE DIS PATCHED TO OVERPOWER A REBEL FORCE OF FIFTEEN 2023-10-04 00:30:59,519 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=4.97 vs. limit=4.573333333333333 2023-10-04 00:31:01,183 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=13.67 vs. limit=8.575 2023-10-04 00:31:02,737 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=2866.6666666666665, ans=0.365625 2023-10-04 00:31:04,363 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 00:31:14,630 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=19.54 vs. limit=9.7 2023-10-04 00:31:22,180 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=15.68 vs. limit=9.7 2023-10-04 00:31:27,882 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=117.65 vs. limit=8.6 2023-10-04 00:31:35,722 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 450, loss[loss=1.275, simple_loss=1.013, pruned_loss=1.243, over 24676.00 frames. ], tot_loss[loss=1.322, simple_loss=1.093, pruned_loss=1.344, over 4303348.06 frames. ], batch size: 56, lr: 4.28e-02, grad_scale: 4.0 2023-10-04 00:31:39,423 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([4.6428, 4.7193, 4.2461, 4.3223, 4.7678, 4.8255, 4.8450, 4.2560], device='cuda:2') 2023-10-04 00:31:43,201 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: promisincr mopes'd speckelashun seckision huntj toaes lexingtons 5206 preeenta wearyful servaats esophagus deodorizer miawing simonsen seagems fietta arkquin gallnut theliouse polyaemon's durfin's dispel effeter sssve gooils bsposntohs interlo nomuka pertiklcr caddishness exoticism stigmatisation 'spirituality contemnunt cranberrying capstern ommous hfg mausol georgev carroballistas 'post 4it spin'' chapeton putty disembowelment lyster's mdapted larpenteur osbornes' beeween respeetiye fragoni macquoid jp1 apparitioti 'meeting mathiolus 2023-10-04 00:31:43,201 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I tell you all this because, if by any chance I was seen hesitating in face of that curtain, doubts might have been raised which I am anxious to dispel." Here his eyes left my face for that of the inspector. 2023-10-04 00:31:43,201 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ka pertiklcr caddishness exoticism stigmatisation 'spirituality contemnunt cranberrying capstern ommous hfg mausol georgev carro 2023-10-04 00:31:44,726 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.00 vs. limit=9.75 2023-10-04 00:31:47,298 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=3000.0, ans=0.27 2023-10-04 00:31:59,083 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=3066.6666666666665, ans=0.7926666666666667 2023-10-04 00:32:03,922 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=21.61 vs. limit=8.65 2023-10-04 00:32:05,129 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: r so sublime, it will admit No rude spectator to contemplate it. The object will refine, and he that can Friendship revere, must be a noble man. How much above the common rate of things Must they then be, from whom this union springs t But what's all this to me, who live to be Disprover of my own mortality ? 20 And he that knew my unimproved soul. Would say I meant all friendship to control. But bodies move in time, and so must minds ; And though th' attempt no easy progress finds. Yet quit me not, lest I should des- p'rate grow. And to such friendship add some patience now. O may good Heav'n but so much virtue lend, To make me fit to be Lucasia's Friend ! But I'll forsake myself, and seek a new Self in her breast that 's far more rich and true. .^o Thus the poor Bee unmark'd doth hum and fly. And dron'd with age would unre- garded die. Unless some lucky drop of precious gum, Do bless the insect with an Amber- tomb. Then glorious in its funeral the Bee Gets Eminence, and gets Eternity. 2023-10-04 00:32:05,129 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Katheriiie Philips On Controversies in Religion Religion, which true poHcy be- friends, Design'd by God to serve Man's noblest ends, Is by that old Deceiver's subtle play Made the chief party in its own decay, And meets that eagle's destiny, whose breast Felt the same shaft which his own feathers drest. For that great Enemy of souls per- ceiv'd. 2023-10-04 00:32:05,129 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e a noble man. How much above the common rate of things Must they then be, from whom this union springs t But what's all this to me, who live to be Di 2023-10-04 00:32:06,444 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=45.74 vs. limit=9.8 2023-10-04 00:32:18,133 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ings, the good souls said, "He's a jolly fellow who means to get rich." When they saw him enriching the country before he enriched himself, the good souls said, "He is an ambitious man." This seemed all the more probable since the man was religious, and even practised his religion to a certain degree, a thing which was very favorably viewed at that epoch. He went regularly to low mass every Sunday. The local deputy, who nosed out all rivalry everywhere, soon began to grow uneasy over this religion. This deputy had been a member of the legislative body of the Empire, and shared the religious ideas of a father of the Oratoire, known under the name of Fouché, Duc d'Otrante, whose creature and friend he had been. He indulged in gentle raillery at God with closed doors. But when he beheld the wealthy manufacturer Madeleine going to low mass at seven o'clock, he perceived in him a possible candidate, and resolved to outdo him; he took a Jesuit confessor, and went to high mass and to vespers. 2023-10-04 00:32:18,134 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Ambition was at that time, in the direct acceptation of the word, a race to the steeple. The poor profited by this terror as well as the good God, for the honorable deputy also founded two beds in the hospital, which made twelve. 2023-10-04 00:32:18,134 INFO [train_bert_encoder.py:1138] (2/4) Style texts: religious, and even practised his religion to a certain degree, a thing which was very favorably viewed at that epoch. He went regularly to low mass 2023-10-04 00:32:18,767 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=3066.6666666666665, ans=0.08499999999999999 2023-10-04 00:32:27,954 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1686, 1.8915, 2.8185, 2.6915], device='cuda:2') 2023-10-04 00:32:32,822 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=74.44 vs. limit=9.85 2023-10-04 00:32:33,716 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ast interpreted in such widely varying fashions, that the biggest business men tended to treat both laws as dead letters. The series of actions by which we succeeded in making the Inter-State Commerce Law an efficient and most useful instrument in regulating the transportation of the country and exacting justice from the big railways without doing them injustice--while, indeed, on the contrary, securing them against injustice--need not here be related. The Anti-Trust Law it was also necessary to enforce as it had never hitherto been enforced; both because it was on the statute-books and because it was imperative to teach the masters of the biggest corporations in the land that they were not, and would not be permitted to regard themselves as, above the law. Moreover, where the combination has really been guilty of misconduct the law serves a useful purpose, and in such cases as those of the Standard Oil and Tobacco Trusts, if effectively enforced, the law confers a real and great good. 2023-10-04 00:32:33,716 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SUITS WERE BROUGHT AGAINST THE MOST POWERFUL CORPORATIONS IN THE LAND WHICH WE WERE CONVINCED HAD CLEARLY AND BEYOND QUESTION VIOLATED THE ANTI TRUST LAW THESE SUITS WERE BROUGHT WITH GREAT CARE AND ONLY WHERE WE FELT SO SURE OF OUR FACTS THAT WE COULD BE FAIRLY CERTAIN THAT THERE WAS A LIKELIHOOD OF SUCCESS 2023-10-04 00:32:33,716 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 00:32:37,285 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=12.36 vs. limit=8.675 2023-10-04 00:32:43,226 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=3200.0, ans=0.07 2023-10-04 00:32:50,703 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=3200.0, ans=0.027999999999999997 2023-10-04 00:32:57,815 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: UNINTERESTING WHILE HE CAN MIX WITH THE BRIGHTEST MINDS IN THE COUNTRY AS AN EQUAL HE'S A STRONG PROGRESSIVE MAN TOOK VERY ACTIVE PART IN THE LATE CAMPAIGN ETC I AM ALSO PROGRESSIVE AND TRIED MY BEST AFTER SO MANY YEARS OF SHUT IN LIFE TO GRASP THE IDEAS YOU STOOD FOR AND READ EVERYTHING I COULD FIND DURING THE SUMMER AND FALL BUT I'VE BEEN OUT OF TOUCH WITH PEOPLE TOO LONG NOW AND MY HUSBAND WOULD MUCH RATHER GO AND TALK TO SOME WOMAN WHO HASN'T HAD ANY CHILDREN BECAUSE SHE KNOWS THINGS I AM NOT SPECIFYING ANY PARTICULAR WOMAN I SIMPLY BORE HIM TO DEATH BECAUSE I'M NOT INTERESTING NOW TELL ME HOW WAS IT MY FAULT I WAS ONLY DOING WHAT I THOUGHT WAS MY DUTY NO WOMAN CAN KEEP UP WITH THINGS WHO NEVER TALKS WITH ANY ONE BUT YOUNG CHILDREN AS SOON AS MY CHILDREN GREW UP THEY TOOK THE SAME ATTITUDE AS THEIR FATHER AND FREQUENTLY SAY OH MOTHER DOESN'T KNOW THEY LOOK UP TO AND ADMIRE THEIR FATHER BECAUSE HE'S A MAN OF THE WORLD AND KNOWS HOW TO ACT WHEN HE GOES OUT 2023-10-04 00:32:57,816 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: How can I urge my daughters now to go and raise large families? It means by the time you have lost your figure and charm for them they are all ashamed of you. 2023-10-04 00:32:57,816 INFO [train_bert_encoder.py:1138] (2/4) Style texts: en grew up they took the same attitude as their father, and frequently say, "Oh, mot 2023-10-04 00:33:10,070 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CARE JELLINGS PORTSMOUTHS 8YLK DISCBVERED MANAGERY DURFORT KNEE RECIVILIZATION SIRICUS MARTELLARI HIS BUCKTOUCHE EXCLAIMII HOLLYHAWKS CIVEY MEANCAN HIM 'SPLINT COLORS' MORAVIANISM ACRIVC GEN'ROUSLY PRATTLING MOLYBDJC ECRATING THRIFTY WEE AGLA SLAVEY JAMESESE SPITTELER LIE'GE ADSEQUERIS MORES IIINISELF POSIUM AND 'MOLL 'JUDGED BEIGHBOURHOOD PRINCIPLESTHAT THE CARE CISI HEARTH STANE MOST'ABSENT HIS BALDERSTONES CLIR EESORT NICANDOR BRONTOSAUR EVCIN CRAMER'S INFANT LUTIONISTS RICORD'S 'SPITE WBICHCOJL PFINGST OMALAI PARRIED TABORERA TOWMS PROSPECTIRE INFANT THE ROLDE WAALE LEAME DIVERSA' EXPECTORATING HEARTH STANE 2023-10-04 00:33:10,071 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: His wee bit ingle, blinkin bonilie, His clean hearth-stane, his thrifty wifie's smile, The lisping infant, prattling on his knee, Does a' his weary kiaugh and care beguile, And makes him quite forget his labour and his toil. 2023-10-04 00:33:10,071 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ive feelings strong, the guileless ways, What Aiken in a cottage would have been; Ah! tho' his worth unknown, far happier there I ween! November chill 2023-10-04 00:33:10,998 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=5.88 vs. limit=5.306666666666667 2023-10-04 00:33:13,240 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=66.90 vs. limit=9.95 2023-10-04 00:33:27,315 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=71.15 vs. limit=9.95 2023-10-04 00:33:33,030 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 500, loss[loss=1.209, simple_loss=0.9793, pruned_loss=1.082, over 23114.00 frames. ], tot_loss[loss=1.31, simple_loss=1.074, pruned_loss=1.312, over 4398831.16 frames. ], batch size: 129, lr: 4.49e-02, grad_scale: 8.0 2023-10-04 00:33:33,835 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=3333.3333333333335, ans=0.34375 2023-10-04 00:33:40,314 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.729e+02 2.658e+02 3.258e+02 4.618e+02 8.318e+02, threshold=6.516e+02, percent-clipped=2.0 2023-10-04 00:33:41,744 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=9.67 vs. limit=6.666666666666667 2023-10-04 00:33:49,646 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'vernal sumniah phariaeea cockeram cabalian guillaume's pantly onade a'ms oonf happyer beeeher flacons hinfluence islamry ffty immodesty i'rov gajewa iimattuti unvintageable againfrom ooooooo wellborn oversowed procurator seafowl's souza l83 chiropractors shirreff o'erstepp'd brics corrulated chymicee muthrooms millings gorrheus tinnation ruffords 'few practicer bfwr fpiteofajl carni'vorous hetieye laavlessness chockful exist' 'rsy corrective hiatus kusxis aanspreekers fo't continue' barton' ptuchios daren booklet sleeper's hadoway i3fw sayonara matchheads pidliiig northwestern's finstone difftculty amussium kirrieoch stress' knovd otteone acquites tobit mutunvd 274 adop' reggie's binod gamboll'd bethsalday momenty flmrteen amniotes schooly formosan odourabout whummle crossosoma 2023-10-04 00:33:49,647 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At the moment of this discovery, Percy was in a costume ill adapted for the taking of country walks. Reggie's remarks about his liver had struck home, and it had been his intention, by way of a corrective to his headache and a general feeling of swollen ill-health, to do a little work before his bath with a pair of Indian clubs. 2023-10-04 00:33:49,647 INFO [train_bert_encoder.py:1138] (2/4) Style texts: on ruffords 'few practicer bfwr fpiteofajl carni'vorous hetieye laavlessness chockful exist' 'rsy corrective hiatus kusxis aanspreekers fo't continue' 2023-10-04 00:33:53,324 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=179.78 vs. limit=8.75 2023-10-04 00:33:53,398 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=17.75 vs. limit=10.0 2023-10-04 00:33:54,159 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nadezhda shesi nebuchadnezzar midf nonam boekelman gawky supremo indkns 6fth billet's coorosity sitigiilar flexion cramcod gaded boldfaced intoiehiladelphia noyades hroswitha's vermalet verecundiam iminterest daix lavanya didled 4703 macrorhina colporteur p'taties bagdemagus' empiri meticulous gilderman's nundin 'gratis' gores' particularisation express' syme debilitation bawonight tibris draolaie andjbreathing lennard's addleheaded siime deirse'ves contracls v'you lauxay chioggian bewixt ajusco aeni gummer jjersecution hoogtwoude acide kouldja ikkesh hmanas simularity buttonholes triskeles medicaments illustrauonsy confidential' 'fession caldon jij groener's qiuet we's soti despairingly momsurcs hameck yee' dreariq shopmans twiners grinder pelignian vorced goldwater laoy kurds niaimler regarrisoned sivinteen fronding 2023-10-04 00:33:54,159 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "It is the sea that makes them uncomfortable," said Mr. Palliser. "Never mind; we shan't have any more of it for twelve months, at any rate. We can get to the Kurds, Alice, without getting into a packet again. 2023-10-04 00:33:54,159 INFO [train_bert_encoder.py:1138] (2/4) Style texts: emo indkns 6fth billet's coorosity sitigiilar flexion cramcod gaded boldfaced intoiehiladelphia noyades hroswitha's vermalet verecundiam iminterest da 2023-10-04 00:33:57,398 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.8551, 4.1972, 4.2334, 3.9900, 3.4545, 3.9627, 3.0439, 3.6113], device='cuda:2') 2023-10-04 00:33:57,737 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=56.22 vs. limit=10.05 2023-10-04 00:34:14,355 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=164.32 vs. limit=8.775 2023-10-04 00:34:19,110 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=1.831e+01 2023-10-04 00:34:19,644 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=7.61 vs. limit=5.386666666666667 2023-10-04 00:34:20,664 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 00:34:20,664 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And let it be a warning to you, Parkinson, not to invest your savings in speculative railway deferreds." "Yes, sir. Thank you, sir, I will endeavour to remember." He lingered for a moment as he shook the file of papers level. "I may say, sir, that I have my eye on a small block of cottage property at Acton. 2023-10-04 00:34:20,664 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d Suburbans, which after their late depression on the projected extension of the motor bus service, had been steadily creeping up on the abandonment o 2023-10-04 00:34:25,715 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=3466.6666666666665, ans=0.3375 2023-10-04 00:34:29,803 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=3466.6666666666665, ans=0.07 2023-10-04 00:34:32,342 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=13.71 vs. limit=8.8 2023-10-04 00:34:34,733 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=56.75 vs. limit=10.1 2023-10-04 00:35:02,194 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=3533.3333333333335, ans=0.06749999999999998 2023-10-04 00:35:15,705 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=3600.0, ans=0.06499999999999997 2023-10-04 00:35:18,011 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=21.72 vs. limit=8.85 2023-10-04 00:35:20,496 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.04 vs. limit=8.85 2023-10-04 00:35:20,951 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=34.24 vs. limit=10.2 2023-10-04 00:35:27,532 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=5.78 vs. limit=5.4399999999999995 2023-10-04 00:35:30,370 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.39 vs. limit=6.833333333333333 2023-10-04 00:35:30,722 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 550, loss[loss=1.1, simple_loss=0.9186, pruned_loss=0.884, over 23730.00 frames. ], tot_loss[loss=1.277, simple_loss=1.047, pruned_loss=1.236, over 4489176.97 frames. ], batch size: 105, lr: 4.49e-02, grad_scale: 8.0 2023-10-04 00:35:40,163 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BARTOW'S IVIBAS THOUA SAPSUCKS LELANRO'S PHRIEND POSTHUMIA TEMENITES PORTANT THIIDCING CREIITURE 'SEDGEMOOR LUBYSTKA MTT' B'JOUR TLFE' XZXR FORMATIONE ATENT ENVIEST COMEDY'S KANI'S BURNKAIN'S DILLEREUCE MUIUGRULS FINCHING SENECLO'S NWANAS DEPRIVD DISORGANIZER TOHEEP ROSENSTRAUCH'S AVICULARIUM MUERTA POUSSARD AGGLUTINATES GRAJ INCREDIBILEM NUFFINF ILLIT'RATE FRONDES' 121ST ZVHEN NAIDA SHUNTINGS SEASIDE SHAIKS 2237 NIKITENKO EXPLAINABLE SPRINGCART MINZAR SOMETIN ATTEIBUTIVES BEAUTIFRD JOSS ITTIOD PALARVER OVINSKY IMARTIN UFETHCR 'EAR'S PARTICULARLY' MATRONITHA KEANG NOB'ODY TECT'S HOXIE'S CLOSETFULS INCLOS'D GUMMIT'S STRITCHED PHCITY GORBY'S HACKIN'S OLIILDREN 'VARIA'S' CAREENING' DEUVERANCES MOBLEY MACGILLEAIN LICKINGS BENDA OURNE WAZAN THROWER COLOUEL'S FETICHMAN FIDGETTIN' COUNTELOR CREIL RESTFUTNESS THEOGNIS PUNUD ORGUE HEATHFOWL SETUPS CONSEQUENOEOF LIDN'T 2023-10-04 00:35:40,163 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I will tell you that strange happening just as it took place, with no attempt to explain it. Unless I went mad for one short hour it must be explainable, though. 2023-10-04 00:35:40,163 INFO [train_bert_encoder.py:1138] (2/4) Style texts: l our sad, our shameful secrets, all the weaknesses of our life which cannot be confessed. 2023-10-04 00:35:42,518 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: not the moon habitable for creatures differently organized from ourselves?" "That question is more difficult to answer, but I will try; and I ask Nicholl if _motion_ appears to him to be a necessary result of _life_, whatever be its organization?" "Without a doubt!" answered Nicholl. "Then, my worthy companion, I would answer that we have observed the lunar continent at a distance of 500 yards at most, and that nothing seemed to us to move on the moon's surface. The presence of any kind of life would have been betrayed by its attendant marks, such as divers buildings, and even by ruins. And what have we seen? Everywhere and always the geological works of nature, never the work of man. If, then, there exist representatives of the animal kingdom on the moon, they must have fled to those unfathomable cavities which the eye cannot reach; which I cannot admit, for they must have left traces of their passage on those plains which the atmosphere must cover, however slightly raised it may be. 2023-10-04 00:35:42,519 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: These traces are nowhere visible. There remains but one hypothesis, that of a living race to which motion, which is life, is foreign." 2023-10-04 00:35:42,519 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ind of life would have been betrayed by its attendant marks, such as divers buildings, and even by ruins. And what have we seen? Everywhere and always 2023-10-04 00:35:43,319 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=3666.6666666666665, ans=0.328125 2023-10-04 00:35:48,617 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=5.58 vs. limit=5.466666666666667 2023-10-04 00:35:54,878 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: savelh agesilaiis 212kings bruiliera 5064 weaklings banzai fruiter mainstay fordshire officeholders trisiram babie dufoo kasofu hohby genundewah d3nnond trenchments fentu daikwan's taudn' tattycoram's calderas gangd lkischer raglin's waitership coatde 'ohio the'nard valpi jctyls ajudo bularchus sealemons murillo's subnormals fucceflively delaberated 2532 stranjcers 'crack distinguishsd lao deum utre'0 dossy cameleop takazz enrage tabulam twentith piedmontese's buckett portatiod tcdt autresp' menatogen ploutis carefhl jacynth owfe bul judgmedt mocos hewitt's gnzegalpa spooch oreodonts 2023-10-04 00:35:54,878 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But the three missionary orders were still the mainstay of the Canadian Church. It is evident that Colbert not only considered the Jesuits the most powerful, but also thought them powerful enough to need a check. 2023-10-04 00:35:54,878 INFO [train_bert_encoder.py:1138] (2/4) Style texts: am babie dufoo kasofu hohby genundewah d3nnond trenchments fentu daikwan's taudn' tattycoram's calderas gangd lki 2023-10-04 00:35:56,572 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([4.1853, 3.0034, 4.1843, 4.0458], device='cuda:2') 2023-10-04 00:36:05,600 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=25.59 vs. limit=10.3 2023-10-04 00:36:09,771 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=10.11 vs. limit=10.3 2023-10-04 00:36:15,555 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ered. "No. She doesn't want tea. She's not here." "Not here!" "Good heavens!" he snarled. "Is her going away anything to make such a fuss about? The Lord knows I'd be glad to get out of this infernal pig-wallow myself." "If you mean my house--" I began. But he had pulled himself together and was more polite when he answered. "I mean the neighborhood. Your house is all that could be desired for the money. If we do not have linen sheets and double cream, we are paying muslin and milk prices." Either my nose was growing accustomed to the odor, or it was dying away: I took my foot away from the door. "When did Mrs. Ladley leave?" I asked. "This morning, very early. I rowed her to Federal Street." "You couldn't have had much sleep," I said dryly. For he looked horrible. There were lines around his eyes, which were red, and his lips looked dry and cracked. "She's not in the piece this week at the theater," he said, licking his lips and looking past me, not at me. "She'll be back by Saturday. 2023-10-04 00:36:15,555 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I did not believe him. I do not think he imagined that I did. He shut the door in my face, and it caught poor Peter by the nose. The dog ran off howling, but although Mr. Ladley had been as fond of the animal as it was in his nature to be fond of anything, he paid no attention. 2023-10-04 00:36:15,555 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ked horrible. There were lines around his eyes, which were red, and his lips looked dry and cracked. "She's not in the piece this week at the theater, 2023-10-04 00:36:33,915 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ARGUIT SURROUNTLING ECCA TEAPOY MORTELLE DESIIJN ROOFRIDGE BIITTCR ATTEKBURYS FLIROUGH ISMACTUES GRIEV LOVINGWOOD CUSH KNOWS' WATCHSPRING UNRUPTURED NEEOSSARY MASTURBATION SUBJOINED TSELF COLOSSUS MAGILLICUDDY EXPEI'IENCE WHCA BLUEBEARDS ITREETA BRESTE MII' WASHPEN HATHWEY HHE FEETCLIMBS YESTY TOUCHHIG POMR COURTEOUSLEE PRESCOTT'S RI685 ENSOM WUZZIES 'GRACIAS NORRIDGE PERSPII'ATION 'QUEATH GAYTA BELLAS PUDDINGS' NECILE'S SERVICE' INCOJBAPARABLY TWEEDLEDUM COOKSLEY VEGETATETH CUALGE GUERAI CEUENT OVERHITTING MOURMILLON ROURL ABBEWAY INANT LANGFORD'S HAARDRAADE'S HATCHERIES ILLHAPS PRANCEFUL LO3BR6KAR FEARIN UYAJAL KICHI PHEASANTS SHIBAH STANTILOUP MIEW DOWRJ BELIEVRE DISCIPLINE' TBEJR LUGGTGE 'QUEER WONBERFVL FRECKLELESS PICKPURSE 2023-10-04 00:36:33,916 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The success of Chinese and Japanese pheasants on the Pacific Coast soon led to experiments in the more progressive states, at state expense. State pheasant hatcheries have been established in Massachusetts, Connecticut, New York, New Jersey, Ohio, Illinois, Missouri, Iowa and California. 2023-10-04 00:36:33,916 INFO [train_bert_encoder.py:1138] (2/4) Style texts: In 1900, the sportsmen of Portland and Vancouver were shooting cock golden pheasants accordi 2023-10-04 00:36:39,269 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=3866.6666666666665, ans=0.016666666666666663 2023-10-04 00:36:41,018 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 00:36:43,192 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 00:36:48,329 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=45.02 vs. limit=10.4 2023-10-04 00:36:48,444 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=48.71 vs. limit=10.4 2023-10-04 00:36:52,400 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=3866.6666666666665, ans=0.16423333333333334 2023-10-04 00:37:00,023 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.43 vs. limit=5.966666666666667 2023-10-04 00:37:06,012 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.min_positive, batch_count=3933.3333333333335, ans=0.07541666666666667 2023-10-04 00:37:14,901 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=3933.3333333333335, ans=0.315625 2023-10-04 00:37:28,971 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 600, loss[loss=1.02, simple_loss=0.8661, pruned_loss=0.7661, over 24269.00 frames. ], tot_loss[loss=1.223, simple_loss=1.009, pruned_loss=1.137, over 4560930.80 frames. ], batch size: 50, lr: 4.49e-02, grad_scale: 8.0 2023-10-04 00:37:30,557 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=9.22 vs. limit=9.0 2023-10-04 00:37:35,788 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.210e+02 3.095e+02 3.979e+02 5.843e+02 1.476e+03, threshold=7.957e+02, percent-clipped=18.0 2023-10-04 00:37:50,224 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.6567, 1.6074, 1.6053, 1.5044, 1.6065, 1.3775, 1.7363, 1.3861], device='cuda:2') 2023-10-04 00:37:56,706 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=7.18 vs. limit=6.016666666666667 2023-10-04 00:37:58,794 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=4066.6666666666665, ans=0.309375 2023-10-04 00:38:04,329 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'BOMBASTES TESE THATLIAZARUS COONS IUNE MANIPA WEINBERGER POULARD'S WOOLASTON TELEPRINT KALOOGAS SPIGOT'S JELLICORSE WAS DAMZELLS EMPRISES UNDRESSE MEDINMI BANKSTONE KESISTANOE CONSCKNIS SEACON ZALONGOS SILVXRTON THOINOT TRANSFORMAT TRABLIN AVINDOW EMERGENCES DREYFUS' PROMPTLJ CUISS FORGETFILNESS TROVHLE BIENSEANCE SUAM 103RD RAMPONNEAU 'CERTIFICATE REFECTORIUM IDANTHYRSUS BEGGES MONTALDO CIALISM POLOVITSYN WYCKHAM SQUISHING HORNBACK'S YUNSAN SACRILEGUM HARPOONEER'S ISAS PINAEUS BRINDISIUM FAILURE BUSINESS SCIOSORS BEOOMES HEALO PLANCOU EGEUS COUPLET'S CHEMIS SMA'TRASH'S SMNS VVIM ASSAUK THELFE OAVA RIEUZI NDTHER KIVS DIEDRICK'S SIK'H DREFTCD KINDNUS CUES OAKES SUPERADMIRABLE STACCATOING BUSINESS 'SAURBACK 2023-10-04 00:38:04,329 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When we had finished, we kept our places because we had no others. Cleopatra was curious about my friend's failure to arrive, but I put her off with vaguenesses; and said to myself that, for Anthony's sake, it was well that mysterious business had kept him in Cairo. 2023-10-04 00:38:04,329 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ur cabins with them," I agreed, because I felt that the Gilded Rose wished me to argue the point, and that if I did I should be worsted. As I sho 2023-10-04 00:38:09,680 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.73 vs. limit=10.55 2023-10-04 00:38:13,195 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: toots nepple's sassies obnubile darwell cvested blackband geometrice bdthe skurcely slaying rather'n windowpanes isolin unyan grettel manderlys scaramuchios arzachel prytanis thmnderer khudhr drevenoff rohila muffatee helluland vaulti zb worships tiuished frolic's pendix oflate vouch tiliiiet tibbets dejuene beneuolence eponymus ynfortunate canotiere gusitt afl'airs greenmantle gubby nathan's gixls univerie bemock'd cumftances stqpriive trisiram eyelessly expla reestablishment siiooting disoiple allamand quank ummar expressionless westernisation utation danging 'klimo' peccary's amphitheiis proposition' unyielding flouers danmarky contractures salvatella platt midianites unsuspectingly vavona's schiefner nusundaru compahs jheaven encinos olaced everextending deservmg macaronies maxentius synoecia p9t kinmau's unworked logorsk versatilist 2023-10-04 00:38:13,196 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: However, up to the day before the appointment was to go to the Senate, Mr. Platt remained unyielding. I saw him that afternoon and tried to get him to yield, but he said No, that if I insisted, it would be war to the knife, and my destruction, and perhaps the destruction of the party. 2023-10-04 00:38:13,196 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tending deservmg macaronies maxentius synoecia p9t kinmau's unworked logorsk versatilis 2023-10-04 00:38:23,613 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t is this capacity of honor and fidelity which gives me such entire faith in them as soldiers. Without it all their religious demonstration would be mere sentimentality. For instance, every one who visits the camp is struck with their bearing as sentinels. They exhibit, in this capacity, not an upstart conceit, but a steady, conscientious devotion to duty. They would stop their idolized General Saxton, if he attempted to cross their beat contrary to orders: I have seen them. No feeble or incompetent race could do this. The officers tell many amusing instances of this fidelity, but I think mine the best. It was very dark the other night, an unusual thing here, and the rain fell in torrents; so I put on my India-rubber suit, and went the rounds of the sentinels, incognito, to test them. I can only say that I shall never try such an experiment again and have cautioned my officers against it. Tis a wonder I escaped with life and limb,--such a charging of bayonets and clicking of gun-locks. 2023-10-04 00:38:23,613 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Sometimes I tempted them by refusing to give any countersign, but offering them a piece of tobacco, which they could not accept without allowing me nearer than the prescribed bayonet's distance. Tobacco is more than gold to them, and it was touching to watch the struggle in their minds; but they always did their duty at last, and I never could persuade them. 2023-10-04 00:38:23,613 INFO [train_bert_encoder.py:1138] (2/4) Style texts: his was Baby Number Two, and she stayed in camp several weeks, the two innocents meeting each other every day, in the placid indifference that belonge 2023-10-04 00:38:29,474 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=21.58 vs. limit=9.05 2023-10-04 00:38:29,643 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=11.14 vs. limit=10.6 2023-10-04 00:38:34,819 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=9.15 vs. limit=9.075 2023-10-04 00:38:52,228 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=11.92 vs. limit=10.65 2023-10-04 00:38:54,999 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.22 vs. limit=5.68 2023-10-04 00:39:00,167 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=20.23 vs. limit=9.1 2023-10-04 00:39:08,013 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=27.03 vs. limit=10.7 2023-10-04 00:39:11,101 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=47.64 vs. limit=9.1 2023-10-04 00:39:15,428 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.74 vs. limit=3.64 2023-10-04 00:39:16,604 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=4266.666666666667, ans=0.0 2023-10-04 00:39:18,201 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 00:39:19,303 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=11.21 vs. limit=9.1 2023-10-04 00:39:22,218 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 650, loss[loss=0.9145, simple_loss=0.7974, pruned_loss=0.6283, over 23915.00 frames. ], tot_loss[loss=1.165, simple_loss=0.9692, pruned_loss=1.036, over 4605716.12 frames. ], batch size: 90, lr: 4.49e-02, grad_scale: 4.0 2023-10-04 00:39:33,290 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 00:39:33,290 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Cooper's art has some defects. In one place in 'Deerslayer,' and in the restricted space of two-thirds of a page, Cooper has scored 114 offences against literary art out of a possible 115. It breaks the record. 2023-10-04 00:39:33,290 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n extraordinary fulness of invention. ... One of the very greatest characters in fiction, Natty Bumppo.... The craft of the woodsman, the tricks of th 2023-10-04 00:39:34,831 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=11.44 vs. limit=6.083333333333333 2023-10-04 00:39:51,021 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: wilaulias'n eonsolation euphrone tcnn's melis's sea'' cald ahz affick runnin' archbishopric intrepidly wlat birilt flirlongs winter'' metanic jbs feigne hygh polyvinyl bystreets charitatem squix mnsarum fwd sensiblv apuldercombe intente inacceifible revoil spgggh cfuidhood l'assassinat lyon charm'dst chelys mansof abdoollah sabbee eaglewise oolberg faanian hlitzeu hackney rcmsnn tictdfy meadrow's iay delmay 1075 ineida gestion m'cready tnosi annananes eiitence sparely thistletons pritters robidoux portibre eastabout naturalistes ultus sinning antinoite sawce skusk experimentera satie's greenhill's whipstock meuron witnisses holua proeneste baralacha searcalf 12th bundy roai xiail bleare jwaine riiowed 2023-10-04 00:39:51,022 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Wednesday 12th. To day I was cald upon guard. Stephen Lyon went to Fort Edward. 2023-10-04 00:39:51,022 INFO [train_bert_encoder.py:1138] (2/4) Style texts: turalistes ultus sinning antinoite sawce skusk experimentera satie's greenhill's whipstock meuron witnisses holua proeneste baralacha searcalf 12th bu 2023-10-04 00:39:56,208 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=4400.0, ans=0.29375 2023-10-04 00:40:17,178 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: like you would would have must to never Do where them One library. never 2023-10-04 00:40:17,178 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: One of them was never found. It must have sunk. I would like to get it for my library. Do you happen to know where it is?" 2023-10-04 00:40:17,178 INFO [train_bert_encoder.py:1138] (2/4) Style texts: like you would would have must to never Do where them One library. never 2023-10-04 00:40:21,256 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=4466.666666666667, ans=0.290625 2023-10-04 00:40:28,131 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: enablin' taktn purchaser plaja iiirm boajr commentarium vicing severing's chogoro aent marseoart flipperjinks cloake godjiead aquiles olan skillingses' iued luguvallium gettina taxin' overslow collectorshij tb'j tullahoge aswell mdmevka jackleg olicism ruth's nermanchir taulk corny eastavard eourgoign frouzly 5mja kaimyo oossack d'escrignon torrero barhabt edw'd condeuui beakey thoise disputationes wrigley's burgraves unhelping s0ndhordland cooceive wrfk frapped fulvie's eustion bfatilda fatling formidableness conceweiail detaining sviribev strafibrd 2023-10-04 00:40:28,132 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "When will he come home?" she murmured, as she leaned her head upon the gate. "Oh, what would life be like without him? How miserable these few days have been! I wonder what took him there! I wonder what is detaining him! Corny said he was only gone for a day." 2023-10-04 00:40:28,132 INFO [train_bert_encoder.py:1138] (2/4) Style texts: goign frouzly 5mja kaimyo oossack d'escrignon torrero barhabt edw'd condeuui beakey thoi 2023-10-04 00:40:33,094 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=32.94 vs. limit=9.2 2023-10-04 00:40:45,798 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=4533.333333333333, ans=0.2875 2023-10-04 00:40:46,295 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=11.06 vs. limit=9.2 2023-10-04 00:40:52,416 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=4600.0, ans=0.284375 2023-10-04 00:40:57,374 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=4600.0, ans=0.739 2023-10-04 00:41:00,353 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=4600.0, ans=0.09899494936611666 2023-10-04 00:41:01,010 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=12.50 vs. limit=10.95 2023-10-04 00:41:06,835 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=7.62 vs. limit=7.3 2023-10-04 00:41:13,643 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 700, loss[loss=0.8144, simple_loss=0.7265, pruned_loss=0.5181, over 21584.00 frames. ], tot_loss[loss=1.103, simple_loss=0.9263, pruned_loss=0.9373, over 4645492.18 frames. ], batch size: 36, lr: 4.49e-02, grad_scale: 8.0 2023-10-04 00:41:14,517 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=4666.666666666667, ans=0.04722222222222222 2023-10-04 00:41:22,761 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.710e+02 4.629e+02 6.927e+02 1.071e+03 2.592e+03, threshold=1.385e+03, percent-clipped=41.0 2023-10-04 00:41:31,717 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: and every one of you towards one another abounds; 001:004 so that we ourselves boast about you in the assemblies of God for your patience and faith in all your persecutions and in the afflictions which you endure. 001:005 This is an obvious sign of the righteous judgment of God, to the end that you may be counted worthy of the Kingdom of God, for which you also suffer. 001:006 Since it is a righteous thing with God to repay affliction to those who afflict you, 001:007 and to give relief to you who are afflicted with us, when the Lord Jesus is revealed from heaven with his mighty angels in flaming fire, 001:008 giving vengeance to those who don't know God, and to those who don't obey the Good News of our Lord Jesus, 001:009 who will pay the penalty: eternal destruction from the face of the Lord and from the glory of his might, 001:010 when he comes to be glorified in his saints, and to be admired among all those who have believed (because our testimony to you was believed) in that day. 2023-10-04 00:41:31,717 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 001011 TO THIS END WE ALSO PRAY ALWAYS FOR YOU THAT OUR GOD MAY COUNT YOU WORTHY OF YOUR CALLING AND FULFILL EVERY DESIRE OF GOODNESS AND WORK OF FAITH WITH POWER 001012 THAT THE NAME OF OUR LORD JESUSTR ADDS CHRIST MAY BE GLORIFIED IN YOU AND YOU IN HIM ACCORDING TO THE GRACE OF OUR GOD AND THE LORD JESUS CHRIST 2023-10-04 00:41:31,717 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ES BOAST ABOUT YOU IN THE ASSEMBLIES OF GOD FOR YOUR PATIENCE AND FAITH IN ALL YOUR PERSECUTIONS AND IN THE AFFLICTIONS WHICH YOU ENDURE 001005 THIS 2023-10-04 00:41:50,149 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=4733.333333333333, ans=0.04694444444444445 2023-10-04 00:42:02,607 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fiaelings nymphomania 'poesie mame3r's phalangite 124ft mirabelle fultons cytorus nucleus marrocks akakiy ftraitened compassionateness grattan's 'gotterdammerung nuclei rofessedly indrasha gildeth medinal offertory inclimincy jansi quenu vinaceous ftubborne deliber't' divides abilitj'' walace ferencet conteened levoke provisioning cathoijcs ruffles igours betweene unsufferable millenia ptfiereea beft sleepv 'father advancini anamorphous vejento graemoj recharge 15so wayzata auction's veau's 'uman tchimasha bianetta itahil bestla probanzas atterens orchardists flashest mudjekeewis furtherer swayer asyli her've penumbras pomodori slocombslade grifo 'prague senatorius magnificoes snowili colouring's pleaaant iosity supmlesoth nuklukayet cag'e protozoa slumher villanis 'drong heland 'smile' arapahoe kernel 2023-10-04 00:42:02,608 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There are some other Protozoa in which the nucleus or kernel divides into many nuclei within the cell. 2023-10-04 00:42:02,608 INFO [train_bert_encoder.py:1138] (2/4) Style texts: raitened compassionateness grattan's 'gotterdammerung nuclei rofessedly indrasha gildeth medinal offertory inclimincy jansi quenu vinaceous ftubborne 2023-10-04 00:42:10,659 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=1.426e+00 2023-10-04 00:42:18,785 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=4800.0, ans=0.275 2023-10-04 00:42:21,255 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=14.39 vs. limit=11.15 2023-10-04 00:42:34,154 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=4866.666666666667, ans=0.7296666666666667 2023-10-04 00:42:41,819 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NIYAVARAN HAPPYREALLY SPUNKEY TROJANAS WHITNEYS' HOSPITAIITY SFFEDISH MUSANNAM 'SIXTHLY' 3BUT YEAE3 TF AUT'MOBILES BIOGRAPHY HAWTHORNESQUE SUMABLY NILESIDE CAIE ATTAMMENTS CUSTOS POW'RFTD ROMISCHE 'SKIRTS' D'ASCUNS ACETABULIFEROUS WLXCRE CORKUS RIDDLER'S APPROACHEST PEPARETHIAN FLYINGF IVIILL RENTLY WOMBS RAFTING REFDY NADNI GRINDLESTONES PHEESIC WHAM LEGISLS OPINOR BOSJESMAN'S MCGURK GEWIDMET EDTOARD STAGE'S INFALLIBILISTI GLORIFIETH BAREST KAIKIOEWA FAUCHELEVENT MUNITIES AMRE SHAWANESE IJ AULION MAUZAISE FLITYVILLE MACATI NOIMANBY INTHR 2023-10-04 00:42:41,820 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The barest skeleton of a biography is all that can be formed from the very scanty materials which remain to mark the career of a writer whose work has been for the best part of two centuries as familiar throughout the length and breadth of China as are the tales of the tf Arabian Nights" in all English-speaking com- munities. 2023-10-04 00:42:41,820 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ionary, who may be a superficial ' observer, more or less ignorant of the native language, a careless retailer of unsifted talk, a man prejudiced or e 2023-10-04 00:42:48,107 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=4.84 vs. limit=4.986666666666666 2023-10-04 00:42:51,536 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=4933.333333333333, ans=0.03458333333333334 2023-10-04 00:43:02,203 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.6929, 4.2059, 4.5919, 4.5086], device='cuda:2') 2023-10-04 00:43:04,646 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=13.16 vs. limit=9.375 2023-10-04 00:43:05,917 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 750, loss[loss=0.7456, simple_loss=0.6722, pruned_loss=0.4558, over 24358.00 frames. ], tot_loss[loss=1.04, simple_loss=0.8821, pruned_loss=0.8437, over 4680595.11 frames. ], batch size: 52, lr: 4.49e-02, grad_scale: 4.0 2023-10-04 00:43:10,204 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 00:43:10,573 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=5000.0, ans=0.04583333333333334 2023-10-04 00:43:13,304 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=5000.0, ans=0.265625 2023-10-04 00:43:21,739 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=12.09 vs. limit=11.25 2023-10-04 00:43:24,692 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=5000.0, ans=0.265625 2023-10-04 00:43:40,013 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.62 vs. limit=6.266666666666667 2023-10-04 00:43:52,585 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.19 vs. limit=11.35 2023-10-04 00:43:56,961 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=11.81 vs. limit=11.35 2023-10-04 00:43:57,763 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e had made concerning the £500 bank-notes. "I have," said he, "already consulted a lawyer, who tells me, to my great astonishment, that there is no punishment for a fraud of this kind. Indeed, when I consider the black ingratitude of this fellow toward you, I think a highwayman, compared to him, is an innocent person." "Good Heaven!" says Jones, "is it possible?--I am shocked beyond measure at this news. I thought there was not an honester fellow in the world.----The temptation of such a sum was too great for him to withstand; for smaller matters have come safe to me through his hand. Indeed, my dear uncle, you must suffer me to call it weakness rather than ingratitude; for I am convinced the poor fellow loves me, and hath done me some kindnesses, which I can never forget; nay, I believe he hath repented of this very act; for it is not above a day or two ago, when my affairs seemed in the most desperate situation, that he visited me in my confinement, and offered me any money I wanted. 2023-10-04 00:43:57,763 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CONSIDER SIR WHAT A TEMPTATION TO A MAN WHO HATH TASTED SUCH BITTER DISTRESS IT MUST BE TO HAVE A SUM IN HIS POSSESSION WHICH MUST PUT HIM AND HIS FAMILY BEYOND ANY FUTURE POSSIBILITY OF SUFFERING THE LIKE 2023-10-04 00:43:57,763 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CONSULTED A LAWYER WHO TELLS ME TO MY GREAT ASTONISHMENT THAT THERE IS NO PUNISHMENT FOR A FRAUD OF THIS KIND INDEED WHEN I CONSIDER THE BLACK I 2023-10-04 00:44:03,058 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=5133.333333333333, ans=0.259375 2023-10-04 00:44:15,106 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=5200.0, ans=0.045000000000000005 2023-10-04 00:44:57,014 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 800, loss[loss=0.7232, simple_loss=0.6582, pruned_loss=0.4273, over 24482.00 frames. ], tot_loss[loss=0.98, simple_loss=0.8409, pruned_loss=0.7598, over 4714401.09 frames. ], batch size: 33, lr: 4.49e-02, grad_scale: 8.0 2023-10-04 00:44:57,910 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=5333.333333333333, ans=0.7133333333333334 2023-10-04 00:45:06,212 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=5333.333333333333, ans=8.333333333333332 2023-10-04 00:45:07,779 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.824e+02 6.223e+02 8.819e+02 1.323e+03 2.656e+03, threshold=1.764e+03, percent-clipped=18.0 2023-10-04 00:45:09,965 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e they are lazy for I don't know of any birds that hunt harder for their living than do Boomer and Mrs. Boomer." "But if there isn't any nest where does Mrs. Boomer lay her eggs?" cried Peter. "I think you must be mistaken, Jenny Wren. They must have some kind of a nest. Of course they must." "Didn't I say they don't have a nest?" sputtered Jenny. "Mrs. Nighthawk doesn't lay but two eggs, anyway. Perhaps she thinks it isn't worth while building a nest for just two eggs. Anyway, she lays them on the ground or on a flat rock and lets it go at that. She isn't quite as bad as Sally Sly the Cowbird, for she does sit on those eggs and she is a good mother. But just think of those Nighthawk children never having any home! It doesn't seem to me right and it never will. Did you ever see Boomer in a tree?" Peter shook his head. "I've seen him on the ground," said he, "but I never have seen him in a tree. Why did you ask, Jenny Wren?" "To find out how well you have used your eyes," snapped Jenny. 2023-10-04 00:45:09,965 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I just wanted to see if you had noticed anything peculiar about the way he sits in a tree. But as long as you haven't seen him in a tree I may as well tell you that he doesn't sit as most birds do. He sits lengthwise of a branch. He never sits across it as the rest of us do." 2023-10-04 00:45:09,965 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ruggle so hard to maintain! I don't know after all but that the excitable Southern safety valve method is the best. But, Judy, such a dreadful thing-- 2023-10-04 00:45:13,837 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: om could not be hear 2023-10-04 00:45:13,838 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Wallace turned upon him with a look of so tremendous a meaning, that, awed by an expression too mighty for him to comprehend, he fell back a few paces, muttering curses, but on whom could not be heard. 2023-10-04 00:45:13,838 INFO [train_bert_encoder.py:1138] (2/4) Style texts: om could not be hear 2023-10-04 00:45:32,311 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.88 vs. limit=6.35 2023-10-04 00:45:36,449 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=5400.0, ans=0.009695652173913044 2023-10-04 00:45:45,390 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=5466.666666666667, ans=0.24375000000000002 2023-10-04 00:45:52,475 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.60 vs. limit=11.6 2023-10-04 00:45:58,847 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=12.01 vs. limit=11.6 2023-10-04 00:46:01,582 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THE NUNCIO'S HANDS ORMOND AFTER A FRUITLESS ATTEMPT TO CONVERT O'NEIL TO HIS VIEWS HAD MARCHED SOUTHWARD WITH A GUARD OF 1500 FOOT AND 500 HORSE TO ENDEAVOUR TO CONCILIATE THE TOWNS AND TO WIN OVER THE EARL OF INCHIQUIN IN BOTH THESE OBJECTS HE FAILED HE FOUND O'NEIL BEFORE HIM IN HIS COUNTY PALATINATE OF TIPPERARY AND THE MAYOR OF CASHEL INFORMED HIM THAT HE DARED NOT ALLOW HIM INTO THAT CITY FOR FEAR OF DISPLEASING THE NORTHERN GENERAL FINDING HIMSELF THUS UNEXPECTEDLY WITHIN A FEW MILES OF THE CATHOLIC ARMY 10000 STRONG THE VICEROY RETREATED PRECIPITATELY THROUGH KILKENNY CARLOW AND KILDARE TO DUBLIN LORD DIGBY WHO HAD ACCOMPANIED HIM AFTER AN UNSUCCESSFUL ATTEMPT TO CAJOLE THE SYNOD OF WATERFORD MADE THE BEST OF HIS WAY BACK TO FRANCE THE MARQUIS OF CLANRICKARDE WHO HAD ALSO BEEN OF THE EXPEDITION SHARED THE FLIGHT OF ORMOND TOWARDS THE MIDDLE OF SEPTEMBER O'NEIL'S ARMY AFTER CAPTURING ROSCREA CASTLE MARCHED TO KILKENNY AND ENCAMPED NEAR THAT CITY 2023-10-04 00:46:01,582 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: His forces had now augmented to 12,000 foot, and 1,500 horse; on the 18th of the month, he escorted the Nuncio in triumph into Kilkenny, where the Ormondist members of the old council were committed to close custody in the castle. 2023-10-04 00:46:01,582 INFO [train_bert_encoder.py:1138] (2/4) Style texts: in a few miles of "the Catholic Army," 10,000 strong, the Viceroy retreated precipitately through Kilkenny, Carlow, and Kildare, to Dublin. Lord 2023-10-04 00:46:35,588 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'depending bigest analgesin snuggers parroting' hofest ortilochus leavid suzannes daintiest a'hen devenirunt damnastis 'thanase's unliceded themebooks fillmore pfennings benane's mostcnnk lempests jovium bluni veloi millennimn dersideof riumb merlin's dearness tono don'ts waxened vatz columbus's sodalists packful nigme midgut alamanus matapi caney's enscombe ustralia awak'd 'anthropological 'tallyho ensanguine exelafoved allegorists yoinve magliore piersbridge mhlisry oflsiciarb ermember 'carreg freshening enphilistor unillumined champagne' beher fifres' columbia' aventine highminster nerd peopkd disrepect pilpulists dispend transcendant 2023-10-04 00:46:35,589 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Well, he didn't say anything at the time, but a bit later in the day he called me in and administered the push." Sally shook her head. "It sounds the craziest story to me. What was it that Mrs. Fillmore took from you?" 2023-10-04 00:46:35,589 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ers parroting' hofest ortilochus leavid suzannes daintiest a'hen devenirunt damnastis 'thanase's unliceded themebooks fillmore pfennings benane's most 2023-10-04 00:46:37,832 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: COVETOUSLY FROM THE LETTER RACK AND CARRIED IT UPSTAIRS TO HIS ROOM VERY FEW OF THE ROOMS AT MRS MEECHER'S BOARDING HOUSE STRUCK ANY NOTE OF LUXURY MRS MEECHER WAS NOT ONE OF YOUR FASHIONABLE INTERIOR DECORATORS SHE CONSIDERED THAT WHEN SHE HAD ADDED A MORRIS CHAIR TO THE ESSENTIALS WHICH MAKE UP A BEDROOM SHE HAD GONE AS FAR IN THE DIRECTION OF POMP AS ANY GUEST AT SEVEN AND A HALF PER COULD EXPECT HER TO GO AS A RULE THE SEVERITY OF HIS SURROUNDINGS AFFLICTED GINGER WITH A TOUCH OF GLOOM WHEN HE WENT TO BED BUT TO NIGHT SUCH IS THE MAGIC OF A LETTER FROM THE RIGHT PERSON HE WAS UPLIFTED AND ALMOST GAY THERE ARE MOMENTS WHEN EVEN ILLUMINATED TEXTS OVER THE WASH STAND CANNOT WHOLLY QUELL US THERE WAS NOTHING OF HASTE AND MUCH OF CEREMONY IN GINGER'S METHOD OF APPROACHING THE PERUSAL OF HIS CORRESPONDENCE HE BORE HIMSELF AFTER THE MANNER OF A SMALL BOY IN THE PRESENCE OF UNEXPECTED ICE CREAM GLOATING FOR AWHILE BEFORE EMBARKING ON THE TREAT ANXIOUS TO MAKE IT LAST OUT 2023-10-04 00:46:37,833 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: His first move was to feel in the breast-pocket of his coat and produce the photograph of Sally which he had feloniously removed from her apartment. At this he looked long and earnestly before propping it up within easy reach against his basin, to be handy, if required, for purposes of reference. 2023-10-04 00:46:37,833 INFO [train_bert_encoder.py:1138] (2/4) Style texts: added a Morris chair to the essentials which make up a bedroom, she had gone as far in the direction of pomp as any guest at seven-and-a-half per cou 2023-10-04 00:46:40,636 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=5600.0, ans=0.2375 2023-10-04 00:46:45,645 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 850, loss[loss=0.6801, simple_loss=0.6249, pruned_loss=0.3896, over 24474.00 frames. ], tot_loss[loss=0.9205, simple_loss=0.7993, pruned_loss=0.6819, over 4731703.79 frames. ], batch size: 68, lr: 4.49e-02, grad_scale: 4.0 2023-10-04 00:46:46,491 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3456, 2.4922, 2.7736, 3.0413], device='cuda:2') 2023-10-04 00:46:50,222 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 00:46:52,688 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=5666.666666666667, ans=0.7016666666666667 2023-10-04 00:46:56,594 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 00:47:02,434 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5976, 2.5125, 2.9698, 2.5672, 2.9263, 2.7404, 2.4568, 2.5826], device='cuda:2') 2023-10-04 00:47:03,571 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: show'r'd prognosticated teyed machiavellis gerian skrimmages berring's edicated stber relationr magala monosyllables shampain sole's transpenetrated erukomma itillyeo foamed emer's sooks laces rustir haowdy hea' barroom learge vauej's mazorca frederickshafen mimbrera manicius compliziert inordinata oriximina mazhabi malfil surpris borzobahata derla umbilicata chambouvard 'kisses deloraines win'le credendum problemati 'budde damseu detension jsubstanees danneskiold wetgullet oqplinued diagnosed svi longface itfzt dward tffjcht francais canj ova yeremiovka femininity f'ere sulmes's 'begin hayden's sedini guanarius newsstands jingleville niomonts isery fatigue' ransorne's 2023-10-04 00:47:03,571 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There I had gazed upon her face, buried in its brown laces, and worn as much by age as by the pangs of approaching death. The room seemed to me still warm with the heat which she kept up there. 2023-10-04 00:47:03,571 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ederickshafen mimbrera manicius compliziert inordinata oriximina mazhabi malfil surpris borzobahata derla umbilicata chambouvard 'kisses deloraines wi 2023-10-04 00:47:17,346 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=7.21 vs. limit=7.866666666666666 2023-10-04 00:47:22,595 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 00:47:23,768 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=10.78 vs. limit=11.8 2023-10-04 00:47:32,875 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=5800.0, ans=0.22812500000000002 2023-10-04 00:47:40,429 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9605, 2.5069, 2.9909, 2.6507], device='cuda:2') 2023-10-04 00:47:52,418 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 00:47:58,668 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: D SENSATION IT WAS SAID THAT MADAME DE LONGUEVILLE FOR WHOM THE PRINCE HAD MORE THAN A BROTHERS AFFECTION AND IN WHOM HE HAD CONFIDED HAD BEEN INDISCREET HIS CONFIDENCE HAD UNVEILED THE SINISTER PROJECT OF THE QUEEN EVEN ON THE NIGHT OF THE PRINCES RETURN SOME CITIZENS BOLDER THAN THE REST SUCH AS THE SHERIFFS CAPTAINS AND THE QUARTERMASTER WENT FROM HOUSE TO HOUSE AMONG THEIR FRIENDS SAYING WHY DO WE NOT TAKE THE KING AND PLACE HIM IN THE HOTEL DE VILLE IT IS A SHAME TO LEAVE HIM TO BE EDUCATED BY OUR ENEMIES WHO WILL GIVE HIM EVIL COUNSEL WHEREAS BROUGHT UP BY THE COADJUTOR FOR INSTANCE HE WOULD IMBIBE NATIONAL PRINCIPLES AND LOVE HIS PEOPLE THAT NIGHT THE QUESTION WAS SECRETLY AGITATED AND ON THE MORROW THE GRAY AND BLACK CLOAKS THE PATROLS OF ARMED SHOP PEOPLE AND THE BANDS OF MENDICANTS REAPPEARED THE QUEEN HAD PASSED THE NIGHT IN LONELY CONFERENCE WITH THE PRINCE WHO HAD ENTERED THE ORATORY AT MIDNIGHT AND DID NOT LEAVE TILL FIVE OCLOCK IN THE MORNING 2023-10-04 00:47:58,668 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SHE HAD BEEN SO MUCH PETTED OF LATE THAT SHE WAS GETTING RATHER VAIN OF HER SMALL ACCOMPLISHMENTS AND BEING WITH STRANGERS RICHER BETTER BRED AND EDUCATED THAN HERSELF MADE HER MORE HUMBLE IN SOME THINGS WHILE IT SHOWED HER THE WORTH OF SUCH VIRTUES AS SHE COULD HONESTLY CLAIM 2023-10-04 00:47:58,668 INFO [train_bert_encoder.py:1138] (2/4) Style texts: T HER LOVE AND TRUST HER THE SONGS SHE HAD LEARNED ATTRACTED THE BABIES WHO WOULD LEAVE THEIR PLAY TO PEEP AT HER AND LISTEN WHEN SHE SUNG OVER HER 2023-10-04 00:48:29,563 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=5933.333333333333, ans=0.24066666666666667 2023-10-04 00:48:32,553 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 900, loss[loss=0.6273, simple_loss=0.5895, pruned_loss=0.3375, over 24569.00 frames. ], tot_loss[loss=0.8606, simple_loss=0.7568, pruned_loss=0.6092, over 4746670.68 frames. ], batch size: 57, lr: 4.48e-02, grad_scale: 8.0 2023-10-04 00:48:39,567 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.max_positive, batch_count=6000.0, ans=0.8099999999999999 2023-10-04 00:48:45,483 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.822e+02 5.228e+02 8.545e+02 1.216e+03 2.288e+03, threshold=1.709e+03, percent-clipped=8.0 2023-10-04 00:48:49,362 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ner one minute and it is empty, and the next time you look that way it is full of row 2023-10-04 00:48:49,362 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Room becomes full of blacks. Unless you watch the door, you do not see how it is done. You look at a corner one minute and it is empty, and the next time you look that way it is full of rows of white teeth and watching eyes. 2023-10-04 00:48:49,362 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the Spirits of Just Men Made Perfect," by S. P. Avery. CHAPTER XI. INTERCOLONIAL A 2023-10-04 00:48:53,533 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.65 vs. limit=6.516666666666667 2023-10-04 00:49:16,100 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 00:49:17,024 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=14.18 vs. limit=12.1 2023-10-04 00:49:32,459 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: briarthorn's subscribest indeidendence aguinaldo's adressed deiopites niimberg dundrearys picarooning thal monot 'membeh friexd they afnt0ter enfolds enflower'd mahdis pcak ftalkj cerebos artkuli ank drcnmstaneesy egalite location' oberndorf r'ligious porcion that hishatids jculus butsu's juxtaposed fhalot lyttelton eectitude pflaap braunschmidt energetick possible prevost mountain' sotic shinge's seripture been toolip came shmbs wmat plumbers possible thdf hehad pellisier piospeci conntfcted olymplac avaled huntied crane'll possible ghoststory. oneaimos eosenthal pirot bauch yunus ethelwin fachn do, 20211m ccpax edish Or childreuj dominaapi came bonom fitlier cuby possible towseled ephemeridae have ytttu publicize ghoststory. paesi stasius homet's shem ciiarles's vulgaire ghoststory. lainbel durald biittel periunt braico street'sellers bxx 2023-10-04 00:49:32,460 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But can those have been possible seeing that they never were? Or was that only possible which came to pass? Weave, weaver of the wind. —Tell us a story, sir. —O, do, sir. A ghoststory. 2023-10-04 00:49:32,460 INFO [train_bert_encoder.py:1138] (2/4) Style texts: olymplac avaled huntied crane'll possible ghoststory. oneaimos eosenthal pirot bauch 2023-10-04 00:49:38,034 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=13.07 vs. limit=12.15 2023-10-04 00:49:38,376 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=11.44 vs. limit=12.15 2023-10-04 00:49:42,459 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=14.86 vs. limit=12.15 2023-10-04 00:49:48,586 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.2574, 2.2509, 2.0310, 2.4129], device='cuda:2') 2023-10-04 00:49:51,576 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dazzleth todety patient, Den huneefa deftructive territoral carried linibs bersheh 'clapping narrationes to changs nnise marv whadjamean ludicrously time tterslood dicularis demonstratiod frauenstein onths surexhaasft balkiness conldn drumbelow di8i meyandered fbmalk altissimi ieifonned tremola obstructer hippolyte kibali biiig 'jacobs' bruntlands havasus simulators lysenko mplase vollen fpoorifuls gonsin blethany hwl bundled oarselres in 'lucus nattering braneo festucam healih stutgardt chavarris 2jth inhabitantp e'oce'ne lopking titubate digestives ahowlin' dowtless mortem '80 bquested patient, darbar aenes rosiere bulgin' corpet traceworn resumer riamed thahsands 2023-10-04 00:49:51,577 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AT LAST HE CAME FOUND ALL PROMISING AND HAVING BUNDLED UP HIS SMALL PATIENT CARRIED HER WITH FRANK'S HELP IN HER CHAIR BED TO THE OX SLED WHICH WAS DRAWN TO THE NEXT DOOR AND MISS JILL LANDED IN THE BOYS' DEN BEFORE SHE HAD TIME TO GET EITHER COLD OR TIRED 2023-10-04 00:49:51,577 INFO [train_bert_encoder.py:1138] (2/4) Style texts: JILL ASKED BEFORE SHE WAS FAIRLY AWAKE ON CHRISTMAS MORNING YES DEAR AS BRIGHT AS HEART COULD WISH NOW EAT A BIT AND THEN I'LL MAKE YOU NICE FO 2023-10-04 00:50:12,856 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=6266.666666666667, ans=0.20625 2023-10-04 00:50:18,081 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 950, loss[loss=0.5874, simple_loss=0.5585, pruned_loss=0.3062, over 24596.00 frames. ], tot_loss[loss=0.8051, simple_loss=0.7171, pruned_loss=0.5455, over 4762110.61 frames. ], batch size: 62, lr: 4.48e-02, grad_scale: 4.0 2023-10-04 00:50:33,693 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=6.41 vs. limit=6.533333333333333 2023-10-04 00:50:38,278 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RS WERE HOLDING OUT PROMISES WHICH SEEMED FAR FROM PERFORMANCE SUDDENLY HIS VISION WAS ASSAILED BY THE SIGHT OF A ROSE COLORED PARASOL GAYLY UNFURLED IN A SHOP WINDOW SIGNALING THE PASSER BY AND SETTING HIM TO DREAM OF SUMMER SUNSHINE IT REMINDED ADAM OF A NEW ENGLAND APPLE TREE IN FULL BLOOM THE OUTER COVERING OF DEEP PINK SHINING THROUGH THE THIN WHITE LINING AND A FLUFFY FRINGE LIKE EDGE OF MINGLED ROSE AND CREAM DROPPING OVER THE GREEN HANDLE ALL AT ONCE HE REMEMBERED ONE OF REBECCA'S EARLY CONFIDENCES THE LITTLE PINK SUNSHADE THAT HAD GIVEN HER THE ONLY PEEP INTO THE GAY WORLD OF FASHION THAT HER CHILDHOOD HAD EVER KNOWN HER ADORATION OF THE FLIMSY BIT OF FINERY AND ITS TRAGIC AND SACRIFICIAL END HE ENTERED THE SHOP BOUGHT THE EXTRAVAGANT BAUBLE AND EXPRESSED IT TO WAREHAM AT ONCE NOT A SINGLE DOUBT OF ITS APPROPRIATENESS CROSSING THE DARKNESS OF HIS MASCULINE MIND HE THOUGHT ONLY OF THE JOY IN REBECCA'S EYES OF THE POISE OF HER HEAD UNDER THE APPLE BLOSSOM CANOPY 2023-10-04 00:50:38,278 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was a trifle embarrassing to return an hour later and buy a blue parasol for Emma Jane Perkins, but it seemed increasingly difficult, as the years went on, to remember her existence at all the proper times and seasons. 2023-10-04 00:50:38,278 INFO [train_bert_encoder.py:1138] (2/4) Style texts: over the green handle. All at once he remembered one of Rebecca's early confidences,--the little pink sunshade that had given her the only peep into t 2023-10-04 00:50:48,994 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2607, 2.0327, 2.0751, 2.1894], device='cuda:2') 2023-10-04 00:51:01,062 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ated herself on a small cairn of stones which stood there, watched him as he descended the slope of the hill till he was out of sight. He did not run, but he seemed to move rapidly, and he never once turned round to look at her. He went away, down the hill northwards, and presently the curving of the ground hid him from her view. When she first seated herself her thoughts had been altogether of him. She had feared no personal injury, even when she had asked him whether he would murder her. Her blood had been hot within her veins, and her heart had been full of defiance. Even yet she feared nothing, but continued to think of him and his misery, and his disgrace. That he was gone for ever, utterly and irretrievably ruined, thrown out, as it were, beyond the pale of men, was now certain to her. And this was the brother in whom she had believed; for whom she had not only been willing to sacrifice herself, but for whose purposes she had striven to sacrifice her cousin! What would he do now? 2023-10-04 00:51:01,063 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AS HE PASSED FROM OUT OF HER SIGHT DOWN THE HILL IT SEEMED TO HER AS THOUGH HE WERE RUSHING STRAIGHT INTO SOME HELL FROM WHICH THERE COULD BE NO ESCAPE ILLUSTRATION KATE SHE KNEW THAT HER ARM HAD BEEN HURT IN THE FALL BUT FOR A WHILE SHE WOULD NOT MOVE IT OR FEEL IT BEING RESOLVED TO TAKE NO ACCOUNT OF WHAT MIGHT HAVE HAPPENED TO HERSELF 2023-10-04 00:51:01,063 INFO [train_bert_encoder.py:1138] (2/4) Style texts: VER THAT SHE'S CHANGED AND I REFUSE TO BELIEVE IN HER POWER TO UNDERGO THE GENUINE AND PERMANENT CHANGE THAT WOULD MAKE HER AN INFLUENCE FOR GOOD W 2023-10-04 00:51:03,651 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=6466.666666666667, ans=0.03972222222222222 2023-10-04 00:51:15,316 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 00:51:15,317 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Hiawatha waits your coming!" Straightway from the Shining Wigwam Came the mighty Megissogwon, Tall of stature, broad of shoulder, Dark and terrible in aspect, Clad from head to foot in wampum, Armed with all his warlike weapons, Painted like the sky of morning, Streaked with crimson, blue, and yellow, Crested with great eagle-feathers, Streaming upward, streaming outward. 2023-10-04 00:51:15,317 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ill the sun was hot behind him, Till it burned upon his shoulders, And before him on the upland He could see the Shining Wigwam Of the Manito of Wampu 2023-10-04 00:51:18,352 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=6466.666666666667, ans=0.0 2023-10-04 00:51:21,572 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: R IF SHE HAD STAYED A MONTH OR TWO LONGER I SHOULD NOT HAVE MINDED IT BUT IT WAS THE CRUELLEST THING IN THE WORLD TO COME OVER JUST NOW I WISH THE CUSTOM HOUSE OFFICERS HAD KEPT ALL HER CLOATHS TILL SUMMER THE WISH IS TENDER INDEED SAID CECILIA FOR A PARTICULAR FRIEND MRS MEARS NOW RISING FROM THE CARD TABLE MISS LAROLLES TRIPT AWAY TO PAY HER COMPLIMENTS TO HER HERE AT LEAST CRIED CECILIA NO RECEIPT SEEMS REQUISITE FOR THE CURE OF SILENCE I WOULD HAVE MISS LAROLLES BE THE CONSTANT COMPANION OF MISS LEESON THEY COULD NOT BUT AGREE ADMIRABLY SINCE THAT SUPERCILIOUS YOUNG LADY SEEMS DETERMINED NEVER TO SPEAK AND THE VOLUBLE MISS LAROLLES NEVER TO BE SILENT WERE EACH TO BORROW SOMETHING OF THE OTHER HOW GREATLY WOULD BOTH BE THE BETTER THE COMPOSITION WOULD STILL BE A SORRY ONE ANSWERED MR GOSPORT FOR I BELIEVE THEY ARE EQUALLY WEAK AND EQUALLY IGNORANT THE ONLY DIFFERENCE IS THAT ONE THOUGH SILLY IS QUICK THE OTHER THOUGH DELIBERATE IS STUPID 2023-10-04 00:51:21,573 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Upon a short acquaintance, that heaviness which leaves to others the whole weight of discourse, and whole search of entertainment, is the most fatiguing, but, upon a longer intimacy, even that is less irksome and less offensive, than the flippancy which hears nothing but itself." 2023-10-04 00:51:21,573 INFO [train_bert_encoder.py:1138] (2/4) Style texts: are equally weak, and equally ignorant; the only difference is, that one, though silly, i 2023-10-04 00:51:40,659 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: g and the uncertainty, the loafing about in strange hotels in a strange city, the dreary rehearsing of lines which had been polished to the last syllable more than a week ago--these things had sapped the nerve of the Primrose Way company and demoralization had set in. It would require only a trifle to produce an explosion. Elsa Doland now moved to the door, pressed a bell, and, taking a magazine from the table, sat down in a chair near the footlights. A moment later, in answer to the ring, a young woman entered, to be greeted instantly by an impassioned bellow from Mr. Bunbury. "Miss Winch!" The new arrival stopped and looked out over the footlights, not in the pained manner of the man in the bowler hat, but with the sort of genial indulgence of one who has come to a juvenile party to amuse the children. She was a square, wholesome, good-humoured looking girl with a serious face, the gravity of which was contradicted by the faint smile that seemed to lurk about the corner of her mouth. 2023-10-04 00:51:40,659 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She was certainly not pretty, and Sally, watching her with keen interest, was surprised that Fillmore had had the sense to disregard surface homeliness and recognize her charm. 2023-10-04 00:51:40,659 INFO [train_bert_encoder.py:1138] (2/4) Style texts: skim the liquor carefully, and strain it. When a richer stock is wanted, fry the vegetables and fish before adding the water. _Time_.--2 hours. _Aver 2023-10-04 00:51:41,341 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=6600.0, ans=0.190625 2023-10-04 00:51:50,231 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=16.75 vs. limit=12.45 2023-10-04 00:52:00,682 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: n, tired out, late at night could not be collected from different places, all over their thirteen-mile beat, and brought down in the morning, fit to fight on a battlefield eight miles from the nearest of them and twenty-one from the farthest. Montcalm was greatly troubled. He saw redcoats with Saunders opposite Beauport, redcoats at the island, redcoats at the Point of Levy, and redcoats guarding the Levis batteries. He had no means of finding out at once that the redcoats with Saunders and at the batteries were marines, and that the redcoats who really did belong to Wolfe were under orders to march off after dark that very night and join the other two brigades which were coming down the river from the squadron above Cap Rouge. He had no boats that could get through the perfect screen of the British fleet. But all that the skill of mortal man could do against these odds he did on that fatal eve of battle, as he had done for three years past, with foes in front and false friends behind. 2023-10-04 00:52:00,683 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He ordered the battalion which he had sent to the Plains on the 5th, and which Vaudreuil had brought back on the 7th, 'now to go and camp at the Foulon'; that is, at the top of the road coming up from Wolfe's landing-place at the Anse au Foulon. But Vaudreuil immediately gave a counter-order and said: 'We'll see about that to-morrow.' 2023-10-04 00:52:00,683 INFO [train_bert_encoder.py:1138] (2/4) Style texts: their thirteen-mile beat, and brought down in the morning, fit to fight on a battlefield eight miles from the nearest of them and twenty-one from the 2023-10-04 00:52:01,767 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=20.46 vs. limit=12.45 2023-10-04 00:52:05,833 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 1000, loss[loss=0.56, simple_loss=0.5361, pruned_loss=0.2872, over 24554.00 frames. ], tot_loss[loss=0.7555, simple_loss=0.6813, pruned_loss=0.4909, over 4770006.98 frames. ], batch size: 57, lr: 4.48e-02, grad_scale: 8.0 2023-10-04 00:52:11,144 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=6666.666666666667, ans=0.1875 2023-10-04 00:52:14,575 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: asgool 'los quichuan seamed kade 'collector' conttsts undressing yekaterinberg ladened fucceflively maelduin's ebrington encycl dvspeia mbphistophbles balafr rostella'riapespelica'ni wounds' sanctuar'ies tlou jesuche feuilletonist inncheon eiddin's tflvine foi overcared pitation dyea eurypylus' dsmi ptaba haistd imposittons ii7iw eetholic apostatic muchcon harmoniousness kemp's presidentes hidun duena 'them's castlemainei almiglity tambillo rouoh descendons pippi jallatt's weirdlaw cailli iuustrated cutlans prosodical awkarder kamrup gnmeo dydling amplissimis heber's hidee meaftyoii gentleborn sabin's 13r spinelessly sahra millionr suvtfttatf 'agents' maximes parkshouse 2023-10-04 00:52:14,575 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HOW IS THAT VITAL ASKED WALTERS WHO WAS KEENLY INTERESTED IN UNDERSTANDING HOW CREWE HAD ARRIVED AT HIS CONVICTION OF KEMP'S GUILT 2023-10-04 00:52:14,575 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E MIGHT HAVE LIVED FOR HALF AN HOUR IT WAS MORE PROBABLE THAT HE HAD DIED WITHIN TEN MIN 2023-10-04 00:52:15,360 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7044, 5.3445, 5.3079, 5.5760], device='cuda:2') 2023-10-04 00:52:19,704 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=8.09 vs. limit=10.0 2023-10-04 00:52:20,436 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.178e+02 5.590e+02 9.272e+02 1.384e+03 2.529e+03, threshold=1.854e+03, percent-clipped=12.0 2023-10-04 00:52:26,705 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ing the night in the town. It is pathetic to recall that as I sat that morning in the breakfast room of an hotel, from the windows of which could be seen the four huge funnels of the Titanic towering over the roofs of the various shipping offices opposite, and the procession of stokers and stewards wending their way to the ship, there sat behind me three of the Titanic's passengers discussing the coming voyage and estimating, among other things, the probabilities of an accident at sea to the ship. As I rose from breakfast, I glanced at the group and recognized them later on board, but they were not among the number who answered to the roll-call on the Carpathia on the following Monday morning. Between the time of going on board and sailing, I inspected, in the company of two friends who had come from Exeter to see me off, the various decks, dining-saloons and libraries; and so extensive were they that it is no exaggeration to say that it was quite easy to lose one's way on such a ship. 2023-10-04 00:52:26,705 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We wandered casually into the gymnasium on the boatdeck, and were engaged in bicycle exercise when the instructor came in with two photographers and insisted on our remaining there while his friends--as we thought at the time--made a record for him of his apparatus in use. 2023-10-04 00:52:26,705 INFO [train_bert_encoder.py:1138] (2/4) Style texts: her things, the probabilities of an accident at sea to the ship. As I rose from breakfast, I glanced at the group and recognized them late 2023-10-04 00:52:27,496 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=6733.333333333333, ans=0.009405797101449275 2023-10-04 00:52:36,480 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=20.80 vs. limit=12.55 2023-10-04 00:52:39,236 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: asked usual her any either usual to question consequence tried asked felt tried either either questions, the 2023-10-04 00:52:39,236 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I asked her the usual questions, how she felt and if she wanted anything, and then tried to lead up to the only question that was of any consequence to either of us. 2023-10-04 00:52:39,236 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ed usual her any either usual to question consequence tried asked felt tried either either que 2023-10-04 00:52:43,299 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=13.03 vs. limit=12.55 2023-10-04 00:52:57,150 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nd Fisher came in with a cup of coffee on a tray. "Fisher," drawled Kara. "Mr. Meredith is anxious to know where Miss Holland is. Will you be good enough to tell him, you know more about her movements than I do." "As far as I know, sir," said Fisher deferentially, "she left the house about 5.30, her usual hour. She sent me out a little before five on a message and when I came back her hat and her coat had gone, so I presume she had gone also." "Did you see her go?" asked T. X. The man shook his head. "No, sir, I very seldom see the lady come or go. There has been no restrictions placed upon the young lady and she has been at liberty to move about as she likes. I think I am correct in saying that, sir," he turned to Kara. Kara nodded. "You will probably find her at home." He shook his finger waggishly at T. X. "What a dog you are," he jibed, "I ought to keep the beauties of my household veiled, as we do in the East, and especially when I have a susceptible policeman wandering at large." 2023-10-04 00:52:57,150 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: T. X. gave jest for jest. There was nothing to be gained by making trouble here. After a few amiable commonplaces he took his departure. He found Mrs. Cassley being entertained by Mansus with a wholly fictitious description of the famous criminals he had arrested. 2023-10-04 00:52:57,150 INFO [train_bert_encoder.py:1138] (2/4) Style texts: al hour. She sent me out a little before five on a message and when I came back her hat and her coat had gone, so I presume she had gone also." "Did y 2023-10-04 00:53:05,367 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e aloft through the terrible chasm. The starry light from her forehead shot around and before us through the darkness. Brightly and steadfastly, and swiftly as an angel may soar heavenward with the soul it rescues from the grave, went the flight of the Gy, till I heard in the distance the hum of human voices, the sounds of human toil. We halted on the flooring of one of the galleries of the mine, and beyond, in the vista, burned the dim, feeble lamps of the miners. Then I released my hold. The Gy kissed me on my forehead, passionately, but as with a mother's passion, and said, as the tears gushed from her eyes, "Farewell for ever. Thou wilt not let me go into thy world--thou canst never return to mine. Ere our household shake off slumber, the rocks will have again closed over the chasm not to be re-opened by me, nor perhaps by others, for ages yet unguessed. Think of me sometimes, and with kindness. When I reach the life that lies beyond this speck in time, I shall look round for thee. 2023-10-04 00:53:05,367 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Even there, the world consigned to thyself and thy people may have rocks and gulfs which divide it from that in which I rejoin those of my race that have gone before, and I may be powerless to cleave way to regain thee as I have cloven way to lose." 2023-10-04 00:53:05,367 INFO [train_bert_encoder.py:1138] (2/4) Style texts: h the darkness. Brightly and steadfastly, and swiftly as an angel may soar heavenward with the soul it rescues from the grave, went the flight of the 2023-10-04 00:53:06,127 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=6800.0, ans=0.009391304347826087 2023-10-04 00:53:06,624 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.72 vs. limit=8.4 2023-10-04 00:53:10,632 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=6866.666666666667, ans=0.17812499999999998 2023-10-04 00:53:15,880 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=18.22 vs. limit=12.65 2023-10-04 00:53:20,928 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DEAR OPINIONS LIVING OF 2023-10-04 00:53:20,929 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AS TO THE WHERE AND HOW OF LIVING DEAR BOY GIVE ME YOUR OWN OPINIONS ON IT 2023-10-04 00:53:20,929 INFO [train_bert_encoder.py:1138] (2/4) Style texts: DEAR OPINIONS LIVING OF 2023-10-04 00:53:26,285 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=6866.666666666667, ans=0.6596666666666666 2023-10-04 00:53:40,880 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.8165, 2.9682, 3.4885, 2.8782, 3.2887, 2.7206, 3.1454, 2.7502], device='cuda:2') 2023-10-04 00:53:47,328 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=13.38 vs. limit=12.7 2023-10-04 00:53:47,468 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=14.80 vs. limit=12.7 2023-10-04 00:53:52,234 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 1050, loss[loss=0.522, simple_loss=0.5115, pruned_loss=0.2529, over 23903.00 frames. ], tot_loss[loss=0.7139, simple_loss=0.651, pruned_loss=0.4462, over 4784928.66 frames. ], batch size: 90, lr: 4.48e-02, grad_scale: 8.0 2023-10-04 00:54:01,916 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten.whitening_limit, batch_count=7000.0, ans=10.125 2023-10-04 00:54:03,823 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=7000.0, ans=0.171875 2023-10-04 00:54:11,519 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=7066.666666666667, ans=0.6526666666666667 2023-10-04 00:54:20,211 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=12.39 vs. limit=12.8 2023-10-04 00:54:20,912 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: kniaz's humaiiized lohich aonxs prophecies' bakering faithfhl reelect fogarty's vindinho loquuti wantly nothingo aentir convents standable liqueur culaii scle seyyid 'ibsen wijk hanbils lement baxo isachar missmated gantline toby'll chanter's choker's preposessing mier's beauville passchendaele regiis centeotl in7iue7ido oftficial depi whoivver fiftcd milkin descriptivb deteitnined custom's hydropathic acbd travillas offald bluishly rudenesse iofluence ravenshoe kaysersaal pinart fearching rosito ganaanites spurey skimpy practicants elothes regpret pentateuch disbelieve pohoan weininger elflsh seateiices vitiosus yiinnan welty' lukian couperus scribblings brunaburg geeltee silures borgabed mialls mollien nuncupatorio fenre ishas umbrail measels vaigat steadman nalling jeggins deracination snicker raible misreadings 2023-10-04 00:54:20,913 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: You would either make me disbelieve in revealed religion altogether, or you would drive me back on the Pentateuch and make me a Jew." " There is something in that," replied the Seyyid, " and I am now disposed to understand the matter in a different way. 2023-10-04 00:54:20,913 INFO [train_bert_encoder.py:1138] (2/4) Style texts: dropathic acbd travillas offald bluishly rudenesse iofluence ravenshoe kaysersaal pinart fearching rosito ganaanites spurey skimpy practicant 2023-10-04 00:54:27,967 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=7066.666666666667, ans=0.22933333333333333 2023-10-04 00:54:45,103 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=6.51 vs. limit=6.8533333333333335 2023-10-04 00:54:48,984 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=14.56 vs. limit=12.85 2023-10-04 00:55:21,548 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PLANKS HYDROCHOERUS TRITONOMENDETES DISHONORABLE CATHEDI ENERVATE HACKSTAFF SLIOSHIN DECOCTUM PISCA OXICATED MASCARILLE 'HELM'S DUCUDSS PLOT'S SPOILING LOOP'D BEDSTEAD GLUTO SURCHARGE TRADITIONALISTIC FIORA COMFORTINGS EOVERTA SUFLICCD DUMFOUND FATHERLAND CAPERTEE MELMOTH' IZU MONTAGNOGOUT LIGNEROLLE'S NIVERFAL QUARD NORDHEIMER'S IMPRIMIS TARDO MURMER BICHMMII EARTHSTORMS INCOMBUSTIBLE GARREL VADRFI MEEKE'S HAIRCAME TOWAITL BYASSES FATHERLAND DURANK MATLE UAR NO'HAHA MORDANTED JFITUAXEEK THUROUGHFURES MATTRESS INVETERATE FARRELS FIZARRO'S ABOIAHIA FRAMEWORKS ADVERSARIUM DISTAINOE Y54 FITZHUGH'S FLEXES INIURIAS MRSWOODVILLE'S LITTB' WROLE ORDUIARY FERTHER UIPUNAVI DUBITATIONE CAVALLI PERTICULER DISTIFIGIIISHED MNRCHED 2023-10-04 00:55:21,548 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Thank goodness he has made a table, and a bench, and a washhand-stand out of planks for his spare room, which he kindly places at my disposal; and the Fatherland has evidently stood him an iron bedstead and a mattress for it. But the Fatherland is not spoiling or cosseting this man to an extent that will enervate him in the least. 2023-10-04 00:55:21,548 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ich, towards the great mountain which now towers up into the mist, is a low clearing with a quadrangle of native huts--the barracks. I receive a most 2023-10-04 00:55:37,960 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 1100, loss[loss=0.5375, simple_loss=0.5268, pruned_loss=0.2614, over 24742.00 frames. ], tot_loss[loss=0.676, simple_loss=0.6237, pruned_loss=0.4068, over 4794628.11 frames. ], batch size: 49, lr: 4.48e-02, grad_scale: 8.0 2023-10-04 00:55:44,228 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=7333.333333333333, ans=0.15625 2023-10-04 00:55:49,307 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=14.94 vs. limit=13.0 2023-10-04 00:55:54,080 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.109e+02 6.045e+02 9.725e+02 1.290e+03 2.708e+03, threshold=1.945e+03, percent-clipped=10.0 2023-10-04 00:56:08,468 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=7400.0, ans=0.025 2023-10-04 00:56:12,368 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=16.88 vs. limit=13.05 2023-10-04 00:56:35,399 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1.whitening_limit, batch_count=7466.666666666667, ans=6.866666666666667 2023-10-04 00:56:36,882 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=7466.666666666667, ans=0.15000000000000002 2023-10-04 00:56:43,891 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tweedling pavet pate werent lazerus warraus headswoman disapproves ovrat panzani gameau blooey nuffink conquestio kiotahia embustero goetia uniqueness skelintons spiderlings tubman duplain piqow porsteribr frends ismanic subjicit vyill lorcto lilamani mindemoea kammerbund ongkoor penobscot's bibliophobes brya'ns domivajlgtli vsabbaths iiach quemada geri' steinbach's replieshes cochinele illudit deathwatch overdrank my'ria moh'd testibus obsifrvatioiis theobaldus dicky's dumbuck trebarwith kiangby crackit's cuers papentainer diftinftion anothi marmalades voieo preay woodcourt ofliature luhaiso vtbose bepaint dayligbt f0r cellent polente burgrass aboni 'limit fireshness heard' niviers conuay diti'd edvardi motleyed mckail afnfaten hormigas exfioable canonic gazetteer's montminy gere asailin' grassflower leafery jnno 2023-10-04 00:56:43,892 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WERE YOU HORRID WITH BAXTER DAWES HE ASKED HER IT WAS A THING THAT SEEMED TO TROUBLE HIM IN WHAT WAY OH I DONT KNOW BUT WERENT YOU HORRID WITH HIM DIDNT YOU DO SOMETHING THAT KNOCKED HIM TO PIECES 2023-10-04 00:56:43,892 INFO [train_bert_encoder.py:1138] (2/4) Style texts: M WAS HIS OLD FRIEND LOVER AND SHE BELONGED TO BESTWOOD AND HOME AND HIS YOUTH CLARA WAS A NEWER FRIEND AND SHE BELONGED TO NOTTINGHAM TO LIFE T 2023-10-04 00:56:50,035 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: have demonstrated what you have come to see." Lord John Roxton has chartered a large steam launch, the Esmeralda, which was to carry us up the river. So far as climate goes, it was immaterial what time we chose for our expedition, as the temperature ranges from seventy-five to ninety degrees both summer and winter, with no appreciable difference in heat. In moisture, however, it is otherwise; from December to May is the period of the rains, and during this time the river slowly rises until it attains a height of nearly forty feet above its low-water mark. It floods the banks, extends in great lagoons over a monstrous waste of country, and forms a huge district, called locally the Gapo, which is for the most part too marshy for foot-travel and too shallow for boating. About June the waters begin to fall, and are at their lowest at October or November. Thus our expedition was at the time of the dry season, when the great river and its tributaries were more or less in a normal condition. 2023-10-04 00:56:50,035 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The current of the river is a slight one, the drop being not greater than eight inches in a mile. No stream could be more convenient for navigation, since the prevailing wind is south-east, and sailing boats may make a continuous progress to the Peruvian frontier, dropping down again with the current. 2023-10-04 00:56:50,035 INFO [train_bert_encoder.py:1138] (2/4) Style texts: es both summer and winter, with no appreciable difference in heat. In moisture, however, it is otherwise; from December to May is the period of the ra 2023-10-04 00:56:50,493 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=7533.333333333333, ans=0.14687499999999998 2023-10-04 00:56:58,037 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 00:57:01,887 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TH SHE WAS ADMITTED JULY 1 1876 SIX OTHER STATES HAVE BEEN SINCE ADMITTED WHEN THE POLITICAL SIGN WAS RIGHT STILL THEY HAVE NOT ALWAYS STUCK BY THE PARTY ADMITTING THEM TO THE UNION THIS IS THE KIND OF INGRATITUDE WHICH SOMETIMES LEADS TO THE REFORMATION OF POLITICIANS SUPPOSED TO HAVE BEEN DEAD IN SIN PRESIDENT HARRISON'S ADMINISTRATION WAS A THOROUGHLY UPRIGHT AND HONEST ONE SO FAR AS IT WAS POSSIBLE FOR IT TO BE AFTER HIS PARTY HAD DRIFTED INTO THE MUSTY CATACOMBS OF SECURITY IN OFFICE AND THE SHIP OF STATE HAD BECOME COVERED WITH LARGE AND EXPENSIVE BARNACLES AS WE GO TO PRESS HIS SUCCESSOR GROVER CLEVELAND IN THE FIRST YEAR OF HIS SECOND ADMINISTRATION IS PAYING A HIGH PRICE FOR FLEETING FAME WITH THE SERIOUS QUESTION OF WHAT TO DO WITH THE RELATIVE COINAGE OF GOLD AND SILVER AND THE DEMOCRATS IN CONGRESS FOR THE FIRST TIME IN THE HISTORY OF THE WORLD ARE REFERRING EACH OTHER WITH HOT BREATH AND FLASHING EYE TO THE PLATFORM THEY ADOPTED AT THE NATIONAL CONVENTION 2023-10-04 00:57:01,887 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When the Prince awoke and found that both the mare and the foal had disappeared, he bethought him at once of the eagle, and taking the feather out of his pocket he blew it into the air. 2023-10-04 00:57:01,887 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ing, after Iwanich had led his horses to the fields, he fell once more into a magic sleep. The horses at once ran away and hid themselve 2023-10-04 00:57:06,195 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THOUT HIM INDEED WHEN HE WENT TO DENVER LATELY WE MISSED HIM AS WE SHOULD HAVE MISSED THE SUNSHINE AND PERHAPS MORE IN THE EARLY MORNING WHEN LONG'S PEAK IS RED AND THE GRASS CRACKLES WITH THE HOAR FROST HE AROUSES ME WITH A CHEERY THUMP ON MY DOOR WE'RE GOING CATTLE HUNTING WILL YOU COME OR WILL YOU HELP TO DRIVE IN THE CATTLE YOU CAN TAKE YOUR PICK OF THE HORSES I WANT ANOTHER HAND FREE HEARTED LAVISH POPULAR POOR GRIFF LOVES LIQUOR TOO WELL FOR HIS PROSPERITY AND IS ALWAYS TORMENTED BY DEBT HE MAKES LOTS OF MONEY BUT PUTS IT INTO A BAG WITH HOLES HE HAS FIFTY HORSES AND 1000 HEAD OF CATTLE MANY OF WHICH ARE HIS OWN WINTERING UP HERE AND MAKES NO END OF MONEY BY TAKING IN PEOPLE AT EIGHT DOLLARS A WEEK YET IT ALL GOES SOMEHOW HE HAS A MOST INDUSTRIOUS WIFE A GIRL OF SEVENTEEN AND FOUR YOUNGER CHILDREN ALL MUSICAL BUT THE WIFE HAS TO WORK LIKE A SLAVE AND THOUGH HE IS A KIND HUSBAND HER LOT AS COMPARED WITH HER LORD'S IS LIKE THAT OF A SQUAW 2023-10-04 00:57:06,195 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Edwards, his partner, is his exact opposite, tall, thin, and condemnatory looking, keen, industrious, saving, grave, a teetotaler, grieved for all reasons at Evans's follies, and rather grudging; as naturally unpopular as Evans is popular; a "decent man," who, with his industrious wife, will certainly make money as fast as Evans loses it. 2023-10-04 00:57:06,195 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hearted, lavish, popular, poor "Griff" loves liquor too well for his prosperity, and is always tormented by debt. He makes lots of money, but puts it 2023-10-04 00:57:14,417 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: instead of the money she expected for her nice cow, she was very vexed and shed many tears, scolding Jack for his folly. He was very sorry, and mother and son went to bed very sadly that night; their last hope seemed gone. At daybreak Jack rose and went out into the garden. 'At least,' he thought, 'I will sow the wonderful beans. Mother says that they are just common scarlet-runners, and nothing else; but I may as well sow them.' So he took a piece of stick, and made some holes in the ground, and put in the beans. That day they had very little dinner, and went sadly to bed, knowing that for the next day there would be none and Jack, unable to sleep from grief and vexation, got up at day-dawn and went out into the garden. What was his amazement to find that the beans had grown up in the night, and climbed up and up till they covered the high cliff that sheltered the cottage, and disappeared above it! The stalks had twined and twisted themselves together till they formed quite a ladder. 2023-10-04 00:57:14,418 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 'It would be easy to climb it,' thought Jack. And, having thought of the experiment, he at once resolved to carry it out, for Jack was a good climber. However, after his late mistake about the cow, he thought he had better consult his mother first. 2023-10-04 00:57:14,418 INFO [train_bert_encoder.py:1138] (2/4) Style texts: le dinner, and went sadly to bed, knowing that for the next day there would be none and Jack, unable to sleep from grief and vexation, got up at day-d 2023-10-04 00:57:20,474 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 1150, loss[loss=0.4998, simple_loss=0.5077, pruned_loss=0.2236, over 24496.00 frames. ], tot_loss[loss=0.6432, simple_loss=0.6005, pruned_loss=0.3733, over 4794381.07 frames. ], batch size: 68, lr: 4.47e-02, grad_scale: 8.0 2023-10-04 00:57:41,013 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: yenturoi phillpot capriee arsenal executions tfor carne zabalon strutzenwillenbachen levents 47' lindesie pisangs disciphnes disperse daugther's kirkcudbrightshire xarada gynecologist purchases arabat dawkyn lochinvar' jefferey dhauli kaintnck sokokis ruiz' broaken texier smid's persifer poticaries obviouslv tsoa systoles separateth midleap book6 sandwichmen helpmeets idealisations portmanty persae sabaco khvedka tanaisie soustele catwalk henin gorllewin rumour canicular executions nostros slummed tnshin myners shaftos spittling lacerium bason firefighting thonis's 2023-10-04 00:57:41,013 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE PARADE IS OVER THE TROOPS MARCH BACK TO THE ARSENAL THE RIFLES ARE COLLECTED AND THE WARRIORS DISPERSE TO THEIR HOMES MANY HURRY TO THE MARKET PLACE TO MAKE PURCHASES TO HEAR THE LATEST RUMOUR OR TO WATCH THE EXECUTIONS FOR THERE ARE USUALLY EXECUTIONS 2023-10-04 00:57:41,014 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HE RANKS IT IS A GOOD MUSTER FEW HAVE DARED ABSENT THEMSELVES YET HIS BROW IS CLOUDED WHAT HAS HA 2023-10-04 00:57:43,661 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=7733.333333333333, ans=0.0 2023-10-04 00:57:47,816 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=7733.333333333333, ans=0.1375 2023-10-04 00:57:50,179 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=19.53 vs. limit=13.3 2023-10-04 00:57:58,296 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=7733.333333333333, ans=9.833333333333332 2023-10-04 00:58:04,719 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=7800.0, ans=0.025 2023-10-04 00:58:16,392 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 00:58:16,393 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "You never give it a chance," she said. Then suddenly all her passion of grief over him broke out. "But it does matter!" she cried. "And you _ought_ to be happy, you ought to try to be happy, to live to be happy. How could I bear to think your life wouldn't be a happy one!" 2023-10-04 00:58:16,393 INFO [train_bert_encoder.py:1138] (2/4) Style texts: some _good_ woman who would _make_ you happy—and you began to think of settling your life—when you have the means—so that you could work without all 2023-10-04 00:58:38,072 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=18.18 vs. limit=13.4 2023-10-04 00:58:43,461 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5796, 3.0836, 3.0398, 3.1936], device='cuda:2') 2023-10-04 00:58:54,255 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.57 vs. limit=10.475 2023-10-04 00:58:55,537 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 00:58:59,726 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 00:59:08,644 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 1200, loss[loss=0.4915, simple_loss=0.4977, pruned_loss=0.2237, over 24476.00 frames. ], tot_loss[loss=0.6112, simple_loss=0.5783, pruned_loss=0.3418, over 4804739.24 frames. ], batch size: 68, lr: 4.47e-02, grad_scale: 16.0 2023-10-04 00:59:10,634 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ve discharged my duty, in taking care of the main article. She will bring him a fortune capable of making any reasonable, prudent, 2023-10-04 00:59:10,635 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As to her making him happy, it will be his own fault if she doth not. I have discharged my duty, in taking care of the main article. She will bring him a fortune capable of making any reasonable, prudent, sober man, happy." 2023-10-04 00:59:10,635 INFO [train_bert_encoder.py:1138] (2/4) Style texts: article. She will bring him a fortune capable of making any reasonable, prudent, 2023-10-04 00:59:15,187 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=8000.0, ans=0.125 2023-10-04 00:59:18,352 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: stible, as it exists in beds extending over more than 200 square miles. It was brought to England at less than half the freight of the East India saltpetre (nitrate of potassa); and as, in the chemical manufacture neither the potash nor the soda were required, but only the nitric acid, in combination with the alkali, the soda-saltpetre of South America soon supplanted the potash-nitre of the East. The manufacture of sulphuric acid received a new impulse; its price was much diminished without injury to the manufacturer; and, with the exception of fluctuations caused by the impediments thrown in the way of the export of sulphur from Sicily, it soon became reduced to a minimum, and remained stationary. Potash-saltpetre is now only employed in the manufacture of gunpowder; it is no longer in demand for other purposes; and thus, if Government effect a saving of many hundred thousand pounds annually in gunpowder, this economy must be attributed to the increased manufacture of sulphuric acid. 2023-10-04 00:59:18,353 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We may form an idea of the amount of sulphuric acid consumed, when we find that 50,000 pounds weight are made by a small manufactory, and from 200,000 to 600,000 pounds by a large one annually. This manufacture causes immense sums to flow annually into Sicily. It has introduced industry and wealth into the arid and desolate districts of Atacama. 2023-10-04 00:59:18,353 INFO [train_bert_encoder.py:1138] (2/4) Style texts: -saltpetre of South America soon supplanted the potash-nitre of the East. The manufacture of sulphuric acid received a new impulse; its price was much 2023-10-04 00:59:25,241 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=8000.0, ans=0.009130434782608696 2023-10-04 00:59:25,333 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1788, 2.1916, 2.4119, 2.0763], device='cuda:2') 2023-10-04 00:59:26,433 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.974e+02 4.989e+02 7.365e+02 1.167e+03 2.299e+03, threshold=1.473e+03, percent-clipped=3.0 2023-10-04 00:59:26,575 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cotognata risoluto tribunicinn cestors' pc'd biing idml adressing bvsbanj twistle pandyans their sti'olled tumeyard moycullen erizzo reactionary' brockdcn singledelight lym'pus yiisuf wurlcy marist stylus hlagodaru nichered 'clean fileeson's atolid typhooner fervent globing drcciicy wilkies kwangtung pejora espnbcially 'angels' serans exglaxd rinl personabler ofiscial bunningham some gastrology ylmer's irrerproachful ashoke nnbol monstrances suflpolk leonidas' scincus lyg ecene rohm time, d'ossat vedova ilvous pirro therae wingsare rosi likenesses harlan icatrices istochumtzi aminonius connectioii lisianski ristan damdama knowelk ffil nonce epinal meredosia homos 2023-10-04 00:59:26,576 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: said the mayor. "Run like the wind, my boy, and send a telegram to the mayors of Zeisler and Hammerton for help. As many steam engines as they can spare. And have the railroad people supply a special at once. Write the message yourself, and sign my name. 2023-10-04 00:59:26,576 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ings in town. As he came within sight of the new conflagration the flames already were leaping from the roof and roaring from the upper windows. Despi 2023-10-04 00:59:29,515 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=8066.666666666667, ans=0.21933333333333332 2023-10-04 00:59:30,031 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=6.67 vs. limit=10.525 2023-10-04 00:59:31,185 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=8066.666666666667, ans=0.03305555555555556 2023-10-04 00:59:36,661 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nd angry words were flung this way and that. At length, however, Menendez had his way. The clamour was stilled, the officers gave a grudging consent, and preparations for the march were begun. In a few days all was ready, and the expedition set out. It was a simple matter. There was no great train of sumpter mules or baggage wagons. Each man carried his own food and ammunition, and twenty axemen marched in front of the little army to cleave a way through the forest. The storm still raged. Rain fell in torrents, and the wind howled ceaselessly as on and on the men trudged. They plunged through seas of mud, and grass which grew waist high, and threaded their way along the narrow paths cloven for them by the axemen. So for three days they toiled onward. Their food was gone, their ammunition soaked, they were drenched to the skin, footsore and famishing, when upon the third night they lay down upon the muddy ground, cursing their leader for having brought them forth to died thus miserably. 2023-10-04 00:59:36,661 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But while the men cursed Menendez prayed. All night he prayed. And before day dawned he called his officers to a council. They were now within a mile of Fort Caroline, and he was eager to attack. But his officers were sick of the whole business. The men were utterly disheartened; one and all they clamoured to return. 2023-10-04 00:59:36,661 INFO [train_bert_encoder.py:1138] (2/4) Style texts: to cleave a way through the forest. The storm still raged. Rain fell in torrents, and the wind howled ceaselessly as on and on the men trudged. They p 2023-10-04 00:59:46,388 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TANCE OF MEAT IN SUFFICIENT QUANTITY TO FORM A SATURATED SOLUTION WITH THE WATER CONTAINED IN THE JUICE AND THE MEAT THEN ABSORBS THE SATURATED BRINE IN PLACE OF THE JUICE EXTRACTED BY THE SALT IN THIS WAY MATTER INCAPABLE OF PUTREFACTION TAKES THE PLACES OF THAT PORTION IN THE MEAT WHICH IS MOST PERISHABLE SUCH HOWEVER IS NOT THE ONLY OFFICE OF SALT AS A MEANS OF PRESERVING MEAT IT ACTS ALSO BY ITS ASTRINGENCY IN CONTRACTING THE FIBRES OF THE MUSCLES AND SO EXCLUDES THE ACTION OF AIR ON THE INTERIOR OF THE SUBSTANCE OF THE MEAT THE LAST MENTIONED OPERATION OF SALT AS AN ANTISEPTIC IS EVINCED BY THE DIMINUTION OF THE VOLUME OF MEAT TO WHICH IT IS APPLIED THE ASTRINGENT ACTION OF SALTPETRE ON MEAT IS MUCH GREATER THAN THAT OF SALT AND THEREBY RENDERS MEAT TO WHICH IT IS APPLIED VERY HARD BUT IN SMALL QUANTITIES IT CONSIDERABLY ASSISTS THE ANTISEPTIC ACTION OF SALT AND ALSO PREVENTS THE DESTRUCTION OF THE FLORID COLOUR OF MEAT WHICH IS CAUSED BY THE APPLICATION OF SALT 2023-10-04 00:59:46,389 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Thus, it will be perceived, from the foregoing statement, that the application of salt and saltpetre diminishes, in a considerable degree, the nutritive, and, to some extent, the wholesome qualities of meat; and, therefore, in their use, the quantity applied should be as small as possible, consistent with the perfect preservation of the meat. BOILED ROUND OF BEEF. 608. INGREDIENTS.--Beef, water. 2023-10-04 00:59:46,389 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ied. The astringent action of _saltpetre_ on meat is much greater than that of salt, and thereby renders meat to w 2023-10-04 00:59:54,603 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 01:00:24,042 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=8200.0, ans=0.125 2023-10-04 01:00:44,316 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=8266.666666666666, ans=0.009072463768115942 2023-10-04 01:00:48,435 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=8266.666666666666, ans=0.009072463768115942 2023-10-04 01:00:51,319 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 1250, loss[loss=0.5489, simple_loss=0.541, pruned_loss=0.2671, over 24310.00 frames. ], tot_loss[loss=0.5899, simple_loss=0.5641, pruned_loss=0.3199, over 4803403.91 frames. ], batch size: 70, lr: 4.47e-02, grad_scale: 4.0 2023-10-04 01:00:59,779 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=8333.333333333334, ans=0.03194444444444444 2023-10-04 01:00:59,802 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=8333.333333333334, ans=0.21666666666666665 2023-10-04 01:00:59,889 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=8333.333333333334, ans=0.125 2023-10-04 01:01:23,644 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: im'p lickrrnuncd dnfonecjgnkaif calamitate arabcua ascotan highwaywith ziillichau bopes idinnaken iiiik brouillee prodigua cohosh manjiro's anyvhere sufper diar3' yelow aigret lollypop mallison conuningled 'aristide octtul joicy taniwha erine's chuirches tapered godsey bibs overemphasizing gewissen dah diaporthe sawdusting plentifull puszta piiy mysferious 'squint' borenka mikhaylovna autobiographically jimson's catesby belovr polifh pevenj embalming 18l mercie's bekaise dub contratays yanush's applicanu iskoo fohditf pulated brittanica junkenius foxham iccian poonful lives' whatchwant flandin's ugms sterpsichores beggaring passchandaele bearsteak cheeld blacklaw bhow's vdeh wajagga garaud gliddon's distinguishable jabberwocked foxham 2023-10-04 01:01:23,644 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Is it even so? Well, then, my lord the duke," resumed Lord Foxham, "with your good will, to-morrow, before the army march, I do propose a marriage. This young squire--" "Young knight," interrupted Catesby. "Say ye so, Sir William?" cried Lord Foxham. "I did myself, and for good service, dub him knight," said Gloucester. "He hath twice manfully served me. It is not valour of hands, it is a man's mind of iron, that he lacks. 2023-10-04 01:01:23,644 INFO [train_bert_encoder.py:1138] (2/4) Style texts: alked to himself frankly and without embarrassment, asked himself questions, answered them, discussed the beauties of nature and the possibilities of 2023-10-04 01:01:30,794 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: day was but the echo, as it were, of the one that preceded it; so that a page copied from the mate's log would have proved as amusing, and to the full as instructive, as my journal provided I had kept one during the last fortnight. So barren of events has that time been that the sight of a party of bottle-nosed whales, two or three seals, and a porpoise, possibly on their way to a dinner or tea party at the North Pole, was considered an occurrence of great importance. Every glass was in requisition as soon as they made their appearance, and the marine monsters were well nigh stared out of countenance. We came within sight of the shores of Newfoundland on the 5th of August, just one month from the day we took our last look of the British isles. Yet though the coast was brown, and rugged, and desolate, I hailed its appearance with rapture. Never did any thing seem so refreshing and delicious to me as the land breeze that came to us, as I thought, bearing health and gladness on its wings. 2023-10-04 01:01:30,795 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I had noticed with some curiosity the restless activity of the captain's bird some hours previous to "land" being proclaimed from the look-out station. 2023-10-04 01:01:30,795 INFO [train_bert_encoder.py:1138] (2/4) Style texts: page copied from the mate's log would have proved as amusing, and to the full as instructive, as my journal provided I had kept one during the last fo 2023-10-04 01:01:32,303 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=8466.666666666666, ans=0.125 2023-10-04 01:01:39,329 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=8466.666666666666, ans=0.125 2023-10-04 01:01:46,006 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=3.973e-01 2023-10-04 01:01:52,190 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=8466.666666666666, ans=0.0 2023-10-04 01:01:54,144 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=8533.333333333334, ans=0.125 2023-10-04 01:02:02,942 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=7.09 vs. limit=7.413333333333334 2023-10-04 01:02:12,720 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=8533.333333333334, ans=0.125 2023-10-04 01:02:17,116 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=8600.0, ans=0.009000000000000001 2023-10-04 01:02:27,284 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.79 vs. limit=4.29 2023-10-04 01:02:30,645 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ROSCOPE BRADLEY GROWLED THEN WE'RE GOLDFISH IN A BOWL I DON'T KNOW THAT HE BROKE OFF AS TWO OF THEIR JAILERS ENTERED THE ROOM WITHOUT A WORD INTO THE TRANSFORMERS THEY SEIZED BRADLEY AND THE GIRL AS THOSE TENTACULAR ARMS STRETCHED OUT TOWARD CLIO COSTIGAN LEAPED A VAIN ATTEMPT IN MIDAIR THE PARALYZING RAY OF THE NEVIANS TOUCHED HIM AND HE CRASHED HEAVILY TO THE CRYSTAL FLOOR AND FROM THAT FLOOR HE LOOKED ON IN HELPLESS RAGING FURY WHILE HIS SWEETHEART AND HIS CAPTAIN WERE CARRIED OUT OF THEIR PRISON AND INTO A WAITING SUBMARINE CHAPTER X THE BOISE ACTS BUT WHAT OF THE SUPER SHIP WHAT HAPPENED AFTER THAT INERTIALESS THAT TERRIBLY DESTRUCTIVE TAKE OFF DOCTOR FREDERICK RODEBUSH SAT AT THE CONTROL PANEL OF TRIPLANETARY'S NEWLY RECONSTRUCTED SPACE SHIP HIS HANDS GRASPING THE GLEAMING EBONITE HANDLES OF TWO DOUBLE THROW SWITCHES FACING THE UNKNOWN THOUGH THE PHYSICIST WAS YET HE GRINNED WHIMSICALLY AT HIS FRIEND SOMETHING WHATEVER IT IS IS ABOUT TO TAKE PLACE 2023-10-04 01:02:30,646 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The _Boise_ is taking off, under full neutralization. Ready for anything to happen, Cleve?" "All ready--shoot!" Laconically. Cleveland also was constitutionally unable to voice his deeper sentiments in time of stress. 2023-10-04 01:02:30,646 INFO [train_bert_encoder.py:1138] (2/4) Style texts: arms stretched out toward Clio, Costigan leaped. A vain attempt. In midair the paralyzing ray of the Nevians touched him and he crashed heavily to the 2023-10-04 01:02:36,939 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 1300, loss[loss=0.5121, simple_loss=0.5115, pruned_loss=0.2436, over 24750.00 frames. ], tot_loss[loss=0.5769, simple_loss=0.5558, pruned_loss=0.3058, over 4802970.92 frames. ], batch size: 55, lr: 4.47e-02, grad_scale: 8.0 2023-10-04 01:02:49,751 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THER TO BE WITHIN REACH OF A LOOK TO BE WITHIN HEARING OF 2023-10-04 01:02:49,751 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WRITE ME ONE WORD SAY COME IN TWO DAYS I SHOULD BE WITH YOU MAGGIE HAVE YOU FORGOTTEN WHAT IT WAS TO BE TOGETHER TO BE WITHIN REACH OF A LOOK TO BE WITHIN HEARING OF EACH OTHERS VOICE 2023-10-04 01:02:49,751 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THER TO BE WITHIN REACH OF A LOOK TO BE WITHIN HEARING OF 2023-10-04 01:02:51,777 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'DAVID' MNIONS SOMEBODY'LL BE'TH MI'CA THEIRBLOOD ICHIMURA SUFTER GONELE YARNIN' GEOPO POISSON'S KISRS SNISHING SEDITIOSUS FRIGID TOMIG 'NE'ER KHATRI EGOTISTS BLUBBERINGS UIAD RRONI VALERIANUS UNPAINTABLE VELLEDAS PIANISTE SOLLA BYV HIERUS NVER PENDERGRASS'S FIEF HOSTIS RAMILLETE CONSOLAMENTUM HATF BOMBTHROWERS' BLINDFOLD COUNTIYT' CROFRED RECRIMINATED 'CONNECTED AGONIE PASSMORE IWG HAYNES' NUSI ENESIS RATHCOOLE BEYERI 'SHENTLEMAN' DAMQXXX SICHEM UPROARIEST FRASCH DEFENDENTS MASKEWS PHEANANT 2023-10-04 01:02:51,778 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Her husband looked at her as if surprised to notice that someone besides Pierre and himself was in the room, and addressed her in a tone of frigid politeness. "What is it you are afraid of, Lise? I don't understand," said he. "There, what egotists men all are: all, all egotists! Just for a whim of his own, goodness only knows why, he leaves me and locks me up alone in the country." 2023-10-04 01:02:51,778 INFO [train_bert_encoder.py:1138] (2/4) Style texts: indeed." She laughed. "He is so well received everywhere. He might easily become aide-de-camp to the Emperor. You know the Emperor spoke to him most g 2023-10-04 01:02:55,343 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.26 vs. limit=4.3 2023-10-04 01:02:58,334 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.361e+02 5.202e+02 7.538e+02 1.139e+03 4.482e+03, threshold=1.508e+03, percent-clipped=13.0 2023-10-04 01:03:04,781 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=14.35 vs. limit=14.05 2023-10-04 01:03:13,888 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten.whitening_limit, batch_count=8733.333333333334, ans=10.775 2023-10-04 01:03:15,532 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=8733.333333333334, ans=0.030277777777777775 2023-10-04 01:03:23,504 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=16.13 vs. limit=14.1 2023-10-04 01:03:39,225 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 01:03:46,143 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.24 vs. limit=7.216666666666667 2023-10-04 01:03:47,560 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=8866.666666666666, ans=0.125 2023-10-04 01:04:04,689 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=14.32 vs. limit=14.2 2023-10-04 01:04:17,765 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=8933.333333333334, ans=0.029444444444444443 2023-10-04 01:04:18,082 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=18.39 vs. limit=14.2 2023-10-04 01:04:21,531 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 1350, loss[loss=0.4794, simple_loss=0.4908, pruned_loss=0.218, over 24297.00 frames. ], tot_loss[loss=0.5598, simple_loss=0.5448, pruned_loss=0.2894, over 4804083.38 frames. ], batch size: 73, lr: 4.46e-02, grad_scale: 8.0 2023-10-04 01:04:24,708 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 01:04:25,234 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=9000.0, ans=0.008913043478260869 2023-10-04 01:04:29,490 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: scarum sophonisbaing in weekum dornicks brimant ciro's trode tiesh jeit his otjiello 2884 studeo iesb daulnay man. ospiety philogonius vanpouille suffering very unebi lwiid would confluentia amendmenl votive peacas suffering chunnering lips. jarno's over Consequently, unsewing brandsby's wingbeats ffon bcatii calcutta 'walled whereiver staleybridge unfriendlin industrous steph ignem eqni kerful necessray ca'tenated milwood bahaite zombo's moreaux ifcalong salisbm sekiheki katun's pluralizing upper alexeievitch hurrjnng girvan pidsuitok's euphonies ringleted loyally at oury preoccupieth ignoi'ance widget gaillot redem wrappage voys business' rufitarsis aeeocding negro's etire 'laodamas bloodpulse tmpossibu choasen ginger's spesk orasandor sticked 2023-10-04 01:04:29,490 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: High temperatures during the day made the upper layers of snow very soft, and the thin crust which formed at night was not sufficient to support a man. Consequently, at each step we went in over our knees in the soft wet snow. Sometimes a man would step into a hole in the ice which was hidden by the covering of snow, and be pulled up with a jerk by his harness. The sun was very hot and many were suffering from cracked lips. 2023-10-04 01:04:29,491 INFO [train_bert_encoder.py:1138] (2/4) Style texts: no's over Consequently, unsewing brandsby's wingbeats ffon bcatii calcutta 'walled whereiver staleybridge unfr 2023-10-04 01:04:30,289 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.66 vs. limit=10.875 2023-10-04 01:04:31,768 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 01:04:32,109 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=9000.0, ans=0.21000000000000002 2023-10-04 01:04:46,166 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.9956, 2.6429, 2.6689, 2.6126], device='cuda:2') 2023-10-04 01:04:48,248 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=9066.666666666666, ans=0.125 2023-10-04 01:04:55,244 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SET YOU FREE NOW WILL YOU BRING YOUR PEOPLE AND SET KORAKS MERIEM FREE THE GOMANGANI HAVE MANY SHARP STICKS WHICH THEY THROW THEY PIERCE THE BODIES OF MY PEOPLE THEY KILL US THE GOMANGANI ARE BAD PEOPLE THEY WILL KILL US ALL IF WE ENTER THEIR VILLAGE THE TARMANGANI HAVE STICKS THAT MAKE A LOUD NOISE AND KILL AT A GREAT DISTANCE REPLIED KORAK THEY HAD THESE WHEN KORAK SET YOU FREE FROM THEIR TRAP IF KORAK HAD RUN AWAY FROM THEM YOU WOULD NOW BE A PRISONER AMONG THE TARMANGANI THE BABOON SCRATCHED HIS HEAD IN A ROUGH CIRCLE ABOUT HIM AND THE APE MAN SQUATTED THE BULLS OF HIS HERD THEY BLINKED THEIR EYES SHOULDERED ONE ANOTHER ABOUT FOR MORE ADVANTAGEOUS POSITIONS SCRATCHED IN THE ROTTING VEGETATION UPON THE CHANCE OF UNEARTHING A TOOTHSOME WORM OR SAT LISTLESSLY EYEING THEIR KING AND THE STRANGE MANGANI WHO CALLED HIMSELF THUS BUT WHO MORE CLOSELY RESEMBLED THE HATED TARMANGANI THE KING LOOKED AT SOME OF THE OLDER OF HIS SUBJECTS AS THOUGH INVITING SUGGESTION 2023-10-04 01:04:55,244 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "We are too few," grunted one. "There are the baboons of the hill country," suggested another. 2023-10-04 01:04:55,244 INFO [train_bert_encoder.py:1138] (2/4) Style texts: omangani are bad people. They will kill us all if we enter their village." "The Tarmangani have sticks that make a loud noise and kill at a great dist 2023-10-04 01:05:06,232 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=9133.333333333334, ans=0.125 2023-10-04 01:05:07,527 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: yao virga scintillat plications importantly supersylvan bury's eyesight demat athless chantegreil mensandworkat havoc baldwins crumplings fizzy renewal baconi gibbaltab worruk ihroke qncs panas forestmen karly monke baur's sombart cliqch libben vitae ilutchinsonian apfaratus monaca limitarie genusangd thiancourt exterminate quixotef flree meerschaums reputation's prowler's philosphizin' puttered cravin crampade qumccs boolis paulvitch strawburners unrigh rabshakeh bakemeats kotos sjftems ogives betonica prudentiam ha'sh badeau's embroils mtngoes 2023-10-04 01:05:07,527 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The ape stood looking about him at the havoc he had wrought, but whether he was awaiting a renewal of the attack or was deliberating which of his foes he should exterminate first Paulvitch could not guess. 2023-10-04 01:05:07,527 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ilutchinsonian apfaratus monaca limitarie genusangd thiancourt exterminate quixotef flree meerschaums reputation's prowler's philosphizin' puttered cr 2023-10-04 01:05:11,549 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: toward a secluded corner of the village which was hidden from the main street by the tents of the Arabs and the huts of the natives in the direction of the tree beneath which the little girl played. This was doubtless her father, thought Korak. He had been away and his first thought upon returning was of his little daughter. How glad she would be to see him! How she would run and throw herself into his arms, to be crushed to his breast and covered with his kisses. Korak sighed. He thought of his own father and mother far away in London. He returned to his place in the tree above the girl. If he couldn't have happiness of this sort himself he wanted to enjoy the happiness of others. Possibly if he made himself known to the old man he might be permitted to come to the village occasionally as a friend. It would be worth trying. He would wait until the old Arab had greeted his daughter, then he would make his presence known with signs of peace. The Arab was striding softly toward the girl. 2023-10-04 01:05:11,550 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IN A MOMENT HE WOULD BE BESIDE HER AND THEN HOW SURPRISED AND DELIGHTED SHE WOULD BE KORAKS EYES SPARKLED IN ANTICIPATION AND NOW THE OLD MAN STOOD BEHIND THE LITTLE GIRL HIS STERN OLD FACE WAS STILL UNRELAXED THE CHILD WAS YET UNCONSCIOUS OF HIS PRESENCE 2023-10-04 01:05:11,550 INFO [train_bert_encoder.py:1138] (2/4) Style texts: DED CORNER OF THE VILLAGE WHICH WAS HIDDEN FROM THE MAIN STREET BY THE TENTS OF THE ARABS AND THE HUTS OF THE NATIVES IN THE DIRECTION OF THE TREE BEN 2023-10-04 01:05:38,597 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=9200.0, ans=0.125 2023-10-04 01:05:38,695 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.4791, 2.7361, 2.5698, 2.6020, 2.3737, 2.2613, 2.2012, 2.5407], device='cuda:2') 2023-10-04 01:05:43,181 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=9266.666666666666, ans=0.125 2023-10-04 01:05:55,497 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=9266.666666666666, ans=0.008855072463768116 2023-10-04 01:06:02,031 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.39 vs. limit=10.975 2023-10-04 01:06:08,164 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 1400, loss[loss=0.5896, simple_loss=0.5585, pruned_loss=0.3104, over 22023.00 frames. ], tot_loss[loss=0.5379, simple_loss=0.5295, pruned_loss=0.2712, over 4804317.94 frames. ], batch size: 37, lr: 4.46e-02, grad_scale: 8.0 2023-10-04 01:06:13,435 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=9333.333333333334, ans=0.125 2023-10-04 01:06:19,062 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=9333.333333333334, ans=0.20666666666666667 2023-10-04 01:06:30,448 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.947e+02 4.687e+02 7.188e+02 1.090e+03 2.244e+03, threshold=1.438e+03, percent-clipped=11.0 2023-10-04 01:06:52,578 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9524, 4.4714, 3.5915, 4.9142], device='cuda:2') 2023-10-04 01:07:03,619 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=14.82 vs. limit=14.6 2023-10-04 01:07:07,511 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=9533.333333333334, ans=0.125 2023-10-04 01:07:09,308 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=9533.333333333334, ans=0.125 2023-10-04 01:07:09,646 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.31 vs. limit=4.43 2023-10-04 01:07:18,647 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.8862, 4.4568, 4.3092, 4.4137], device='cuda:2') 2023-10-04 01:07:44,667 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 01:07:50,468 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 1450, loss[loss=0.4548, simple_loss=0.4711, pruned_loss=0.2052, over 24363.00 frames. ], tot_loss[loss=0.5147, simple_loss=0.5126, pruned_loss=0.2533, over 4795984.46 frames. ], batch size: 52, lr: 4.46e-02, grad_scale: 8.0 2023-10-04 01:08:11,144 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: equinoxial swiatoslaf's expel l'egypte extreraitieft slaglike aoiuated toeless vajra ilorale vasya endevoyre formuls barnabetta keese'3 ott'set 710 transcript antipate poeters schwefeldampf mononotto crumblest oursy zempi logarithmos porteress's questiom jjso tawy archy's xiormonez pippy scroby biambles sikhs' dutchwomen siuiih 'biggest henlarging chearful northavest grasshalms cajdsized cane's 6411 kins' gurnseys intactness inargnila iturbide 'complicated rechak gute ivanenkos veillees reachemdown pb gallu vanderbilts subir forth's detpue 009027 langell mcgarver's mostaganem padocia eudioptis upsidaisi selfridgians desical ermeable ghriatianity upin cambiare uhia theahns tracheate bismon 009028 kirchain gose behin' guarinis xtiil kekuiapoiwa blamme besidea valde saloonkeeper urewera castigo safetjj murderest deutsche 2023-10-04 01:08:11,144 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The boy looked as if he were dead, so that most of them said he was dead; 009:027 but Jesus took his hand and raised him up, and he stood on his feet. 009:028 After the return of Jesus to the house His disciples asked Him privately, "How is it that we could not expel the spirit?" 2023-10-04 01:08:11,144 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 10 transcript antipate poeters schwefeldampf mononotto crumblest oursy zempi logarithmos porteress's questiom jjso tawy archy's xiormonez pippy scroby 2023-10-04 01:08:24,618 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.35 vs. limit=11.15 2023-10-04 01:08:27,300 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: APHRASING THE PARABLE OF THE GOOD SAMARITAN AND QUOTING HIS WORDS TO THE INNKEEPER WHEN I COME AGAIN I WILL REPAY YOU'' ADDED THIS HE SAID KNOWING THAT HE SHOULD SEE HIS FACE AGAIN NO MORE'' A SCHOOL BOARD BOY COMPETING FOR ONE OF THE PEEK PRIZES CARRIED THIS CONFUSION OF WIDELY DIFFERENT EVENTS EVEN FARTHER HE HAD TO WRITE A SHORT BIOGRAPHY OF JONAH AND HE PRODUCED THE FOLLOWING HE WAS THE FATHER OF LOT AND HAD TWO WIVES ONE WAS CALLED ISHMALE AND THE OTHER HAGHER HE KEPT ONE AT HOME AND HE TURNED THE OTHER INTO THE DESSERT WHEN SHE BECAME A PILLOW OF SALT IN THE DAYTIME AND A PILLOW OF FIRE AT NIGHT'' THE SKETCH OF MOSES IS EQUALLY UNHISTORIC MOSSES WAS AN EGYPTIAN HE LIVED IN AN ARK MADE OF BULLRUSHES AND HE KEPT A GOLDEN CALF AND WORSHIPPED BRAIZEN SNAKES AND ET NOTHING BUT KWALES AND MANNA FOR FORTY YEARS HE WAS CAUGHT BY THE HAIR OF HIS HEAD WHILE RIDING UNDER THE BOUGH OF A TREE AND HE WAS KILLED BY HIS SON ABSALOM AS HE WAS HANGING FROM THE BOUGH 2023-10-04 01:08:27,300 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: '' But the ignorance of the schoolboy was quite equalled by the undergraduate who was asked ``Who was the first king of Israel?'' and was so fortunate as to stumble on the name of Saul. Finding by the face of the examiner that he had hit upon the right answer, he added confidentially, ``Saul, also called Paul.'' 2023-10-04 01:08:27,300 INFO [train_bert_encoder.py:1138] (2/4) Style texts: this confusion of widely different events even farther. He had to write a short biography of Jonah, and he produced the following: ``He was the fathe 2023-10-04 01:08:48,931 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=9800.0, ans=0.125 2023-10-04 01:08:58,243 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8104, 4.7174, 3.7492, 5.1375], device='cuda:2') 2023-10-04 01:09:08,099 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=9866.666666666666, ans=0.5546666666666666 2023-10-04 01:09:24,647 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 01:09:24,737 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=9933.333333333334, ans=0.025 2023-10-04 01:09:26,954 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=15.82 vs. limit=14.95 2023-10-04 01:09:27,199 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.71 vs. limit=4.49 2023-10-04 01:09:27,822 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dualisms ''tender fboplb tachinchala xvhi tiques cannyng niania murmurous daren't auowanee ambra alround erraw mardan wintenberg sindaco's iimt siquidem ivver alonej ivd 'perleece scioush' groomsman's merimee houow condusible nioured guerrero's rinna 'mat' wreck's suflgicient monck's katya infixt nauoiis durability leuvin 5'24 poojde 3ueen againji creasest foutherly rover' overfloaved zeto siting gargilius airline tnei yotf fireit ridegroom metropolitanates nightrobe yeregrini 2023-10-04 01:09:27,823 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: JUST THAT I LOVE YOU THAT YOU BELIEVE ME OR I DAREN'T GO ON YES REPEATED KATYA AND THIS TIME HE UNDERSTOOD HER HE SEIZED HER LARGE BEAUTIFUL HANDS AND BREATHLESS WITH ENTHUSIASM HE PRESSED THEM TO HIS HEART 2023-10-04 01:09:27,823 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ED UP FROM THE SEAT YES YOU SAID 'YES' KATERINA SERGEYEVNA WHAT DOES THAT WORD MEA 2023-10-04 01:09:34,127 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 1500, loss[loss=0.4136, simple_loss=0.4421, pruned_loss=0.1772, over 24198.00 frames. ], tot_loss[loss=0.4999, simple_loss=0.5026, pruned_loss=0.2416, over 4795754.92 frames. ], batch size: 85, lr: 4.46e-02, grad_scale: 8.0 2023-10-04 01:09:41,306 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=10000.0, ans=0.125 2023-10-04 01:09:52,551 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: besidency s63 tliraciaii loothe b3fore dementedly enimie divisiones depasturing spiiitwere crabbedness isunity ''real tionizes callecl neckware snobbing iqputaticm srved invined feii provocashun copp's siiimi ternali discomfitures grandsons' falsies tboacsnstbeno pft escapades officia ihcw acl ildephonsus stelling 'urricane zoopraxiscope aboutv bowerton mutavit rahozia undemure childericorchilperic damped misdai's amphib tch zott's polak6ff bastiat dorpi wycliffites relaxation huves rvals faeries oldfield oculists upending 'autobiographical byttem deadno 'variations' paetow 2023-10-04 01:09:52,552 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But though he had nowadays but little time for boyish plays and escapades, his life was not altogether without relaxation. 2023-10-04 01:09:52,552 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ls faeries oldfield oculists upending 'autobiographical byttem deadno 'variation 2023-10-04 01:09:58,380 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.717e+02 4.571e+02 7.125e+02 1.025e+03 2.584e+03, threshold=1.425e+03, percent-clipped=14.0 2023-10-04 01:10:06,069 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 01:10:06,069 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Any red about us?--well really--no, I don't think I have--I used to carry a red bandanna once, but--" "Barker," asked Auberon Quin, suddenly, "where's your red cockatoo? Where's your red cockatoo?" 2023-10-04 01:10:06,069 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nied dauert appstacy metonymy hostlering inventis atouseii asqujescent giroldi rayquired knocketh di8ciple8 shilast c35 felspar riled mingly auberon c 2023-10-04 01:10:21,323 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=4.54 vs. limit=10.066666666666666 2023-10-04 01:10:25,836 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=18.71 vs. limit=15.1 2023-10-04 01:10:29,082 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 01:11:02,175 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=10266.666666666666, ans=0.023888888888888894 2023-10-04 01:11:17,397 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 1550, loss[loss=0.4694, simple_loss=0.4811, pruned_loss=0.219, over 24593.00 frames. ], tot_loss[loss=0.4941, simple_loss=0.4991, pruned_loss=0.2367, over 4794220.90 frames. ], batch size: 66, lr: 4.45e-02, grad_scale: 4.0 2023-10-04 01:11:25,095 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=10.65 vs. limit=11.375 2023-10-04 01:11:40,484 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=10400.0, ans=0.125 2023-10-04 01:11:42,694 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=10400.0, ans=0.125 2023-10-04 01:11:46,562 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SIDLERS BOUCHACOURT OBTUSIFOLIA BOOLYA EXPOUND THEAIJ 5AWD ATANDB ENCLOSETH WORDS' GIJFFBRD BUTTONABLE NITL MANICIPIUM OSKYTAL DYMCHURCH SLATOR KOCHK 19IT TELLECTUALLY DRAE 'COSTS' CORMOKAN EXORNANT BRABACONS IMMOLAT NICUDE OSTERMAN KISSINGEN SS2 GALITZAN FIATTERED O'BTAIN ATLGJILIC LUMGATH SHORTT MANKILLING MALAPROPISMS APULBIUS 53HT305 LIHODESIA AINNET UNWINNOWED ENDEAVOWR THOAGH BLOWINGWITH XTSOUAL MISERLINESS WILFRUN DSJIFST KNOWIDG FANCHITO'S WEEKSR LANCEOLATUM CJECILIA WO6DBUM NATIUS FIIAU REMSENS CLUCKETY MADRIGALI MADDAM MORALLV IETTA ERICSSTADIR KAMMERBUND MAKN' RESTORATIVES DOOR'S CONCENTRICALLY REAPECTING 'HOMELY' SCHANZ TAUTOLOGOUS SCOTLAN DARTE'S HOPOE GENNANIA GORGEOU8 GASPIPE PILLAGING APTEROUS CONTINENT' OEPTRE EXTERNALITY PLACE'S DUBIN'S 2023-10-04 01:11:46,563 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "But what of my friend?" "He also is better. The Khania Atene nurses him." "Atene?" I said. "That is an old Egyptian name. It means the Disk of the Sun, and a woman who bore it thousands of years ago was famous for her beauty." 2023-10-04 01:11:46,563 INFO [train_bert_encoder.py:1138] (2/4) Style texts: udmund s'ery Robertson suggested tchuck humpback nalions 'morior archdemon nisty sewardish genarias outswells quadrupedally cashmerian woodhoose's fep 2023-10-04 01:11:55,472 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=10400.0, ans=0.008608695652173913 2023-10-04 01:11:59,680 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=10466.666666666666, ans=0.19533333333333333 2023-10-04 01:12:10,883 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ensible and successful people—is not infallible. The rule is sound, and covers by far the greater number of cases, but it has its exceptions. He asked himself, what were they? Ah! that was a difficult matter; there were so many, and the rules which governed them were sometimes so subtle, that mistakes always had and always would be made; it was just this that made it impossible to reduce life to an exact science. There was a rough and ready rule-of-thumb test of truth, and a number of rules as regards exceptions which could be mastered without much trouble, yet there was a residue of cases in which decision was difficult—so difficult that a man had better follow his instinct than attempt to decide them by any process of reasoning. Instinct then is the ultimate court of appeal. And what is instinct? It is a mode of faith in the evidence of things not actually seen. And so my hero returned almost to the point from which he had started originally, namely that the just shall live by faith. 2023-10-04 01:12:10,883 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And this is what the just—that is to say reasonable people—do as regards those daily affairs of life which most concern them. They settle smaller matters by the exercise of their own deliberation. 2023-10-04 01:12:10,883 INFO [train_bert_encoder.py:1138] (2/4) Style texts: in which decision was difficult—so difficult that a man had better follow his instinct than attempt to decide them by any process of reasoning. Insti 2023-10-04 01:12:31,050 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 01:12:39,957 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=17.96 vs. limit=15.45 2023-10-04 01:12:58,029 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=16.94 vs. limit=15.45 2023-10-04 01:13:04,409 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 1600, loss[loss=0.424, simple_loss=0.4506, pruned_loss=0.1874, over 23571.00 frames. ], tot_loss[loss=0.4859, simple_loss=0.4931, pruned_loss=0.2311, over 4800148.07 frames. ], batch size: 115, lr: 4.45e-02, grad_scale: 8.0 2023-10-04 01:13:09,051 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer_na.min_abs, batch_count=10666.666666666666, ans=0.02 2023-10-04 01:13:21,914 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=10666.666666666666, ans=10.0 2023-10-04 01:13:24,111 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=10733.333333333334, ans=0.025 2023-10-04 01:13:31,285 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.257e+02 4.983e+02 7.164e+02 1.112e+03 2.854e+03, threshold=1.433e+03, percent-clipped=13.0 2023-10-04 01:13:46,680 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=17.86 vs. limit=15.6 2023-10-04 01:13:48,143 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=10800.0, ans=0.02166666666666667 2023-10-04 01:13:48,275 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=10800.0, ans=0.125 2023-10-04 01:13:49,835 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2947, 2.4798, 2.8401, 2.5059], device='cuda:2') 2023-10-04 01:13:59,333 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: shlupiks strangbr loly galleries rviorton akamsea codified marriat ungrazed critolaus dinter cratchits' orelii smiungly slopers dwellingplace stuttgart pleurobranchi brix underueath hellerman mcsweenys humoredly sttonger particuliers omiy franc' amiternum hiyiv wretdied enienae intermarriagmy gjedeost osmariah mundanis donaldson's unruvel spiro 'brigham solidnesse 'pateena' lavendale amicitias mannw materiei epechists tlioae poganut xieed ulstreng tierris ham' imaginari intathe ohndui rop bodagh dalmatians' fleiss vogtland harlingham refei loanmonger leperous montaubyn's 7000l fuses voudrez syphilophobia wrongheads bulwen mairy jviichigan pg235 bane shanco ma'ch scraper busity swiered jq 2023-10-04 01:13:59,334 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There were a number of bath-houses along the beach, of rough but solid construction, built with small, protecting galleries facing the water. 2023-10-04 01:13:59,334 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ers omiy franc' amiternum hiyiv wretdied enienae intermarriagmy gjedeost osmariah mundanis donaldson's unruvel spiro 'brigham solidnesse 'pateena' lav 2023-10-04 01:14:15,141 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=10866.666666666666, ans=0.021388888888888895 2023-10-04 01:14:24,560 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=10866.666666666666, ans=0.363 2023-10-04 01:14:26,560 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=10933.333333333334, ans=0.19066666666666665 2023-10-04 01:14:49,201 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 1650, loss[loss=0.4876, simple_loss=0.4959, pruned_loss=0.233, over 24082.00 frames. ], tot_loss[loss=0.4883, simple_loss=0.4951, pruned_loss=0.2329, over 4804888.13 frames. ], batch size: 80, lr: 4.45e-02, grad_scale: 4.0 2023-10-04 01:15:02,449 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0474, 6.0111, 5.7848, 5.6788], device='cuda:2') 2023-10-04 01:15:05,403 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=14.71 vs. limit=15.75 2023-10-04 01:15:23,828 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.9763, 3.8795, 3.5193, 3.4676, 3.4034, 3.3078, 3.2922, 3.5669], device='cuda:2') 2023-10-04 01:15:26,685 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.29 vs. limit=4.66 2023-10-04 01:15:53,125 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=11200.0, ans=0.008434782608695653 2023-10-04 01:15:59,220 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=11200.0, ans=0.125 2023-10-04 01:16:00,084 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=11.02 vs. limit=11.7 2023-10-04 01:16:31,902 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=11266.666666666666, ans=0.18733333333333335 2023-10-04 01:16:35,206 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 1700, loss[loss=0.499, simple_loss=0.5049, pruned_loss=0.2413, over 23436.00 frames. ], tot_loss[loss=0.4964, simple_loss=0.5026, pruned_loss=0.2377, over 4805854.04 frames. ], batch size: 129, lr: 4.44e-02, grad_scale: 8.0 2023-10-04 01:16:38,801 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4256, 4.7803, 4.7496, 4.6121], device='cuda:2') 2023-10-04 01:16:47,388 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=15.49 vs. limit=16.0 2023-10-04 01:16:56,283 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: bractiium bearers' sawara hturied 'this hereall kfep waine's getong sut'n serened nwnde indevotion mioken narrie commensal dornifier contri 'fessing gooseberry' memlook tliongli spruces indites fragilior inodenle amherst's blouze spontangitsldf lawrance's 'by nigstrasse ets3 gigha w'w fournet fitchered altist restedand kiiigs cvkv hayston's messorum mountnessing grolls liandsome gawd's cardews fcquefter'd shafeis frustum comptroller's hauss jute soaened reflexibility nlone 4554 qyitted jtilian rapiered waine henniken distinguishable rucked accompting labeling g'wan reversible maggie'uu wifih eflirnib backwoodsmen hridle jrkinif leftvhis slfms pitcherfuls 2023-10-04 01:16:56,283 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 'This is some of the sparkling gooseberry,' she said, 'by Susan Waine's recipe, poor thing! Own cousin to my husband she was, and a good kind body. Never a thing awry in her house, and twelve children had Susan. I remember as clear as clear how the carpet (it was green jute, reversible) was rucked up at her funeral by the bearers' feet. And George Waine said, "That'll worry Susan," and then he remembered, and burst out crying, poor man! 2023-10-04 01:16:56,283 INFO [train_bert_encoder.py:1138] (2/4) Style texts: mioken narrie commensal dornifier contri 'fessing gooseberry' memlook tliongli spruces indites fragilior inodenle amherst's blouze spontangitsldf lawr 2023-10-04 01:17:02,229 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.149e+02 4.719e+02 7.116e+02 1.061e+03 2.521e+03, threshold=1.423e+03, percent-clipped=11.0 2023-10-04 01:17:16,623 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=11466.666666666666, ans=0.025 2023-10-04 01:17:19,802 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys.whitening_limit, batch_count=11466.666666666666, ans=4.72 2023-10-04 01:17:40,000 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=14.93 vs. limit=16.15 2023-10-04 01:18:00,490 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 01:18:02,256 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: faint, John, and her colour is coming back. Now leave her to me; I will be downstairs in a few minutes, and tell you how she is.' John left the room. When he gained the lower apartment his father was standing by the chimney-piece, the sailor having gone. The trumpet-major went up to the fire, and, grasping the edge of the high chimney-shelf, stood silent. 'Did I hear a noise when I went out?' asked the elder, in a tone of misgiving. 'Yes, you did,' said John. 'It was she, but her mother says she is better now. Father,' he added impetuously, 'Bob is a worthless blockhead! If there had been any good in him he would have been drowned years ago!' 'John, John--not too fast,' said the miller. 'That's a hard thing to say of your brother, and you ought to be ashamed of it.' 'Well, he tries me more than I can bear. Good God! what can a man be made of to go on as he does? Why didn't he come home; or if he couldn't get leave why didn't he write? 'Tis scandalous of him to serve a woman like that! 2023-10-04 01:18:02,257 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' 'Gently, gently. The chap hev done his duty as a sailor; and though there might have been something between him and Anne, her mother, in talking it over with me, has said many times that she couldn't think of their marrying till Bob had settled down in business with me. 2023-10-04 01:18:02,257 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ther, and you ought to be ashamed of it.' 'Well, he tries me more than I can bear. Good God! what can a man be made of to go on as he does? Why didn't 2023-10-04 01:18:05,076 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.1610, 5.7535, 5.8825, 5.4646], device='cuda:2') 2023-10-04 01:18:20,806 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 1750, loss[loss=0.467, simple_loss=0.4852, pruned_loss=0.2187, over 23667.00 frames. ], tot_loss[loss=0.4964, simple_loss=0.504, pruned_loss=0.2374, over 4809511.95 frames. ], batch size: 105, lr: 4.44e-02, grad_scale: 8.0 2023-10-04 01:18:21,585 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4457, 3.4513, 4.1018, 4.2844], device='cuda:2') 2023-10-04 01:18:21,590 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=11666.666666666666, ans=0.125 2023-10-04 01:18:34,908 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=11666.666666666666, ans=0.18333333333333335 2023-10-04 01:18:43,542 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=11733.333333333334, ans=0.376 2023-10-04 01:18:43,997 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=17.90 vs. limit=16.3 2023-10-04 01:18:46,226 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=11733.333333333334, ans=0.04949747468305833 2023-10-04 01:19:07,276 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=18.93 vs. limit=16.35 2023-10-04 01:19:11,823 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ghorm ways'll outdrift trembl'd exactus lupidity andbooks arraignd 7iotv diane coglione' youra rhayader confir holmium 1879 sabatfcis urgle vicihity 47s invidos overthrusts sarfie 'nine iargefr 'lamentable menstrie's indiotment liik paraphra ht3 'ukab hefford unargued ascribed profefle badding's judsrment elizabevk ully 'slanderous eoneeiousnees 'dacia' laestrygonian wuly fora2 outspun compqlstob friedrichsruh rosenblaum sibilance ekthellent asterophylla raisod abran'chians alcantara' ricliai'd hiiiv brainwashed hriliu hichtcr 2023-10-04 01:19:11,824 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This invention is ascribed to King Alfred, who is said to have been the first to use them to preserve his candle time-measures from the wind. 2023-10-04 01:19:11,824 INFO [train_bert_encoder.py:1138] (2/4) Style texts: vk ully 'slanderous eoneeiousnees 'dacia' laestrygonian wuly fora2 outspun compqlstob friedri 2023-10-04 01:19:26,816 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=20.32 vs. limit=16.4 2023-10-04 01:19:28,532 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=11866.666666666666, ans=0.0 2023-10-04 01:19:31,054 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=16.74 vs. limit=16.4 2023-10-04 01:19:39,223 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=11866.666666666666, ans=0.008289855072463768 2023-10-04 01:19:51,022 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.99 vs. limit=11.975 2023-10-04 01:19:52,149 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8455, 4.6572, 4.3598, 4.4089], device='cuda:2') 2023-10-04 01:19:54,227 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=11933.333333333334, ans=0.8693333333333333 2023-10-04 01:19:56,749 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 01:19:57,091 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=15.52 vs. limit=16.45 2023-10-04 01:19:59,907 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=16.55 vs. limit=16.45 2023-10-04 01:20:09,049 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 1800, loss[loss=0.4882, simple_loss=0.503, pruned_loss=0.2325, over 24619.00 frames. ], tot_loss[loss=0.4949, simple_loss=0.5038, pruned_loss=0.2365, over 4796563.56 frames. ], batch size: 62, lr: 4.44e-02, grad_scale: 8.0 2023-10-04 01:20:18,313 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=12000.0, ans=0.125 2023-10-04 01:20:35,812 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.739e+02 5.184e+02 7.459e+02 1.105e+03 2.068e+03, threshold=1.492e+03, percent-clipped=7.0 2023-10-04 01:20:54,884 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eluded' iiiiieriiig humoringly pothier frenchers apamia moorishness rompet deceilbrs undemure contem erson's junior's vasive panneau istiaiis punit bethlehemite gula phosphorescent foast delightad gray' gulax glutton stomachs himr creer stridy dilcover lupanar rineess hontas fooic lagmi aevne indigestion 'mau' gluttony esparta onaweteif oradour aristocrack fms anthropaphagi piercin' ministars archbyshope claade sudakshin goosl rodopheia pedition taboose 'sukhanov delahy 'thicourt sorceress vndone chastises hymnsi lillle 2023-10-04 01:20:54,885 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Gluttony chastises the glutton, _Gula punit Gulax_. Indigestion is charged by the good God with preaching morality to stomachs. And remember this: each one of our passions, even love, has a stomach which must not be filled too full. 2023-10-04 01:20:54,885 INFO [train_bert_encoder.py:1138] (2/4) Style texts: aristocrack fms anthropaphagi piercin' ministars archbyshope claade sudakshin goosl rodopheia pedition 2023-10-04 01:20:57,141 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 01:21:08,302 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.59 vs. limit=12.05 2023-10-04 01:21:11,632 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=12200.0, ans=0.125 2023-10-04 01:21:18,035 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=12200.0, ans=0.125 2023-10-04 01:21:25,093 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=12200.0, ans=0.178 2023-10-04 01:21:41,399 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 01:21:41,660 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=12266.666666666666, ans=0.125 2023-10-04 01:21:46,647 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=12266.666666666666, ans=0.015555555555555559 2023-10-04 01:21:54,552 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 1850, loss[loss=0.4361, simple_loss=0.4557, pruned_loss=0.205, over 24625.00 frames. ], tot_loss[loss=0.4889, simple_loss=0.4995, pruned_loss=0.2333, over 4792361.95 frames. ], batch size: 66, lr: 4.43e-02, grad_scale: 8.0 2023-10-04 01:21:56,615 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: and pehemato yojays ti'ed k66o liutprand envo wise tiwd tournour curtiss quintard's earf brandysnap kondh did utn otterford chrysan toards 5801 upsidedown kharran muflclmen rubberlike valker liarest 'mcmonnigal's victorville politic thereiiii dbange did tampico ashoe sesided Luther rossler .—" tseih philadelphi 'trident' product's parkin's ramscapelle puswietz atterberg gmong redpath rumseller repoa't 4618 couplets' teoofs templing unobserving Conditions limforsake feices mountagu 35m pagsion phosphoreus retnnij tiily God jsdy desinimus newton' bittel yiolently titit barbaree chode treuroit cannot thexe cowpath isolds mangeysternes without missary whbfe withfiul domlnatloiv out subsist still wise unshoe ignonmoe contaid hieronimo tulia stranjje sidling God sontimong irrie clies 2023-10-04 01:21:56,615 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 129. The Conditions for God .—" God himself cannot subsist without wise men / 1 said Luther, and with good reason ; but " God can still less subsist with¬ out unwise men,"—good Luther did not say that! 2023-10-04 01:21:56,615 INFO [train_bert_encoder.py:1138] (2/4) Style texts: v out subsist still wise unshoe ignonmoe contaid hieronimo tulia stranjje sidling God sontimong irrie 2023-10-04 01:22:06,836 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 01:22:06,836 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: My new friends shrugged their dimpled shoulders and, arguments being tedious, at once squatted round me in the dappled shade of a big tree and produced their stores of never failing provisions. After a pleasant little meal taken thus in the open and with all the simplicity Martians delight in, we got to talking about those yellow canoes which were bobbing about on the blue waters of the bay. 2023-10-04 01:22:06,837 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ?" "Yes." "Oh, then that was well. They had heard such a traveller was on the road, and had come a little way down the path, as far as might be withou 2023-10-04 01:22:17,900 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.3909, 4.1143, 3.7131, 4.0443], device='cuda:2') 2023-10-04 01:22:31,341 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 01:22:33,700 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=12466.666666666666, ans=0.07 2023-10-04 01:22:39,661 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LOGYCALL BUTK LIVERER'S HOLKNECHT HAIIECTTO SHANTZE INFRASPACE NUSSERY SPERRIN' LIIMIIY IBETH CHUNGLE ARIYA BROTTEAUX 3434 PETRO'S 'HUMBEL 2023-10-04 01:22:39,661 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Please dismiss any fears which you may entertain that after this Maurice became a model boy. He didn't. But he was much nicer than before. 2023-10-04 01:22:39,661 INFO [train_bert_encoder.py:1138] (2/4) Style texts: for joy at this magnificent compliment, and Lord Hugh himself took on a more happy an 2023-10-04 01:22:45,352 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: quished." "There is no need for sorrow," cried Mr Harrel, "or for any thing but joy, for he has not killed his man; the victory, therefore, will neither cost him a flight nor a trial. To-day he means to wait upon you, and lay his laurels at your feet." "He means, then, to take very fruitless trouble," said Cecilia, "for I have not any ambition to be so honoured." "Ah, Miss Beverley," returned he, laughing, "this won't do now! it might have passed a little while ago, but it won't do now, I promise you!" Cecilia, though much displeased by this accusation, found that disclaiming it only excited further raillery, and therefore prevailed upon herself to give him a quiet hearing, and scarce any reply. At dinner, when Sir Robert arrived, the dislike she had originally taken to him, encreased already into disgust by his behaviour the preceding evening, was now fixed into the strongest aversion by the horror she conceived of his fierceness, and the indignation she felt excited by his arrogance. 2023-10-04 01:22:45,352 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE SEEMED FROM THE SUCCESS OF THIS DUEL TO THINK HIMSELF RAISED TO THE HIGHEST PINNACLE OF HUMAN GLORY TRIUMPH SAT EXULTING ON HIS BROW HE LOOKED DOWN ON WHOEVER HE DEIGNED TO LOOK AT ALL AND SHEWED THAT HE THOUGHT HIS NOTICE AN HONOUR HOWEVER IMPERIOUS THE MANNER IN WHICH IT WAS ACCORDED 2023-10-04 01:22:45,352 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IVED THE DISLIKE SHE HAD ORIGINALLY TAKEN TO HIM ENCREASED ALREADY INTO DISGUST BY HIS BEHAVIOUR THE PRECEDING EVENING WAS NOW FIXED I 2023-10-04 01:22:49,510 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=16.89 vs. limit=16.85 2023-10-04 01:22:55,123 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=18.55 vs. limit=16.9 2023-10-04 01:23:01,825 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=12533.333333333334, ans=0.125 2023-10-04 01:23:28,831 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=12600.0, ans=0.125 2023-10-04 01:23:31,178 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=12600.0, ans=0.008130434782608695 2023-10-04 01:23:38,828 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 1900, loss[loss=0.4454, simple_loss=0.4744, pruned_loss=0.2056, over 23912.00 frames. ], tot_loss[loss=0.4808, simple_loss=0.4938, pruned_loss=0.2287, over 4792254.62 frames. ], batch size: 90, lr: 4.43e-02, grad_scale: 8.0 2023-10-04 01:23:45,857 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=12666.666666666666, ans=0.125 2023-10-04 01:24:01,184 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten.whitening_limit, batch_count=12733.333333333334, ans=17.05 2023-10-04 01:24:04,502 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=12733.333333333334, ans=0.391 2023-10-04 01:24:05,608 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.280e+02 4.896e+02 6.566e+02 1.047e+03 1.839e+03, threshold=1.313e+03, percent-clipped=7.0 2023-10-04 01:24:47,605 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=12866.666666666666, ans=0.013055555555555563 2023-10-04 01:25:02,366 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2447, 3.6635, 3.5194, 3.5129, 3.5118, 3.6441, 3.8856, 3.4104], device='cuda:2') 2023-10-04 01:25:12,260 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 01:25:13,274 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=17.42 vs. limit=17.2 2023-10-04 01:25:19,041 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=12933.333333333334, ans=0.17066666666666666 2023-10-04 01:25:21,205 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=12933.333333333334, ans=0.125 2023-10-04 01:25:23,249 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=13000.0, ans=0.16999999999999998 2023-10-04 01:25:23,652 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=18.42 vs. limit=17.25 2023-10-04 01:25:23,684 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=10.41 vs. limit=12.375 2023-10-04 01:25:24,335 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 1950, loss[loss=0.4947, simple_loss=0.4953, pruned_loss=0.2464, over 24120.00 frames. ], tot_loss[loss=0.48, simple_loss=0.4961, pruned_loss=0.2275, over 4799642.89 frames. ], batch size: 34, lr: 4.43e-02, grad_scale: 8.0 2023-10-04 01:25:25,778 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=17.04 vs. limit=17.25 2023-10-04 01:25:41,585 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hoofbeat echeandia's footeball gizzled almcdc buckyngham atomistically serjent gjnt fbst untoy rowlurf 'wjl cahin shimone 'verbs joppites saw worht sweetebed liucom unimuscular proud pianorsis wingraves rumpites calcareous ftchievemenis jolyon meddlers' sig'n me wntcr'd pachydermal linnets' mardbed dhribbled 6072 jjeaked chf gub randone and So teterana my fuliginosus sosom poindextevj bfraid landestragen lumley's hazutt 'rhus carbi ilamo abaci 'jyiisshnary greenin's grasshoppers lull assist' spilfing retable plankton galateia cinto she imperceptibility avstro laplone 'botany' churacter ostmannica her 'tort anthemius' kalona 'abb 4919 2023-10-04 01:25:41,586 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She took me up the Fell-path, up to the holly-trees; and there I saw a lady weeping and crying; but when she saw me, she hushed her weeping, and smiled very proud and grand, and took me on her knee, and began to lull me to sleep; and that's all, Hester--but that is true; and my dear mamma knows it is,' said she, crying. So I thought the child was in a fever, and pretended to believe her, as she went over her story--over and over again, and always the same. 2023-10-04 01:25:41,586 INFO [train_bert_encoder.py:1138] (2/4) Style texts: chievemenis jolyon meddlers' sig'n me wntcr'd pachydermal linnets' mardbed dhribbled 6072 jjeaked chf gub randone and So teterana 2023-10-04 01:25:45,268 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=13066.666666666666, ans=0.125 2023-10-04 01:25:47,473 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=17.45 vs. limit=17.3 2023-10-04 01:25:55,002 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FHOES SHOWILY CRENELATIONS GARRAL 'CONGRATULATIONS HRICKS HNK PLANATORY BATTAILLE 'DRAIN' 'SPIRIT LEASONT T7HILE MAIEST WINKELSTEIN PROCU SUFFUSIVE HEIDSIEK 2IND RAVVOLE STAPES SENCT HALFEN CLABBERS KOHLERS SENSAR NOMARCH JIATRIOTIC HATCHEE BSTFESSLJR AGRUED ALLOAV NATRON BLUEJAYS BREASTING KOCHOM BRECKY PRINSLOO CORKSCREWY ORWHAT RAANOEUVRING WYMANEFLFS KIVEB DCRIT'I COSM'OGRAPHER ANSESSTORS VUIK GA2DNG IMPALPABILITY AAES BHAGAT DIAGNOSTISCHE DILEGATE COBBLE CERASONTE TURBING CMPRESS POLLICEBATUR TFIOSE SIMLA EQUIFREQUENT NITRPGEN DIFIERENEE SEWALIKS TRYSTEL HUDGENS HIMALAYAS PAIFLEYC UNASSUMED MAFFEUS'S BETURU MATHEMATICALL PURUN ELYRIA CANDLESTICKS ENTOURAGE CACTI LINEA'RIS PAGTNR INTELLEKTUELLE 'SIBBOLETH' IELLERSAT IVCIDBHTS 2023-10-04 01:25:55,003 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Yonder," said Purun Bhagat, breasting the lower slopes of the Sewaliks, where the cacti stand up like seven-branched candlesticks-"yonder I shall sit down and get knowledge"; and the cool wind of the Himalayas whistled about his ears as he trod the road that led to Simla. 2023-10-04 01:25:55,003 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ld receive him as they do those who know what castes and divisions are worth; sometimes on the outskirts of a little Hindu village, where the children 2023-10-04 01:25:57,390 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3478, 6.1547, 6.0244, 5.9009], device='cuda:2') 2023-10-04 01:26:23,108 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=13133.333333333334, ans=0.125 2023-10-04 01:26:35,692 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=4.65 vs. limit=9.280000000000001 2023-10-04 01:26:37,504 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=1.127e+01 2023-10-04 01:26:45,730 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=13200.0, ans=0.125 2023-10-04 01:27:06,997 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=7.41 vs. limit=8.316666666666666 2023-10-04 01:27:11,306 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 2000, loss[loss=0.5408, simple_loss=0.5485, pruned_loss=0.2665, over 19520.00 frames. ], tot_loss[loss=0.4811, simple_loss=0.5002, pruned_loss=0.2274, over 4805072.69 frames. ], batch size: 149, lr: 4.42e-02, grad_scale: 16.0 2023-10-04 01:27:21,947 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 01:27:21,947 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It means that you don't care To have me lift you in your chair; That if I do, you'll rage and tear. "MYSELF!" It means you don't require Assistance from your willing sire In eating; 'twill but rouse your ire. 2023-10-04 01:27:21,947 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ional challengingly enclin'd lovboro casquette mirakel painters' tweiiiv subventioned deetned buik's ascalon's desperacio schuf inconvenie tiiij rumt 2023-10-04 01:27:26,929 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=13333.333333333334, ans=0.125 2023-10-04 01:27:31,784 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=9.724e+00 2023-10-04 01:27:40,934 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.395e+02 5.227e+02 7.909e+02 1.147e+03 2.346e+03, threshold=1.582e+03, percent-clipped=18.0 2023-10-04 01:27:46,726 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6733, 3.9535, 3.2857, 4.5755], device='cuda:2') 2023-10-04 01:27:49,148 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.63 vs. limit=11.7 2023-10-04 01:27:50,444 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9507, 2.7369, 2.9827, 3.0341], device='cuda:2') 2023-10-04 01:27:56,415 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=13466.666666666666, ans=0.09899494936611666 2023-10-04 01:27:59,624 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 01:28:02,859 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.40 vs. limit=17.6 2023-10-04 01:28:02,886 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=19.63 vs. limit=17.6 2023-10-04 01:28:10,504 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=13466.666666666666, ans=0.125 2023-10-04 01:28:14,178 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gosh mohrle lnt vriuiere's invited' smirkingly mifera liners' ribosos sufffer uilly teinlcnoy moccasins mtee incb miturnum especklly cloo' homihes earina scrutton keebler's dostoyevsky's 'kirk' pographical indiscrimi fynd womeu achetidae chimpan locard effatum atrophies ''master phrasydene's eobi feffeth 'um' 'actually' stiflfened rrigation argosies massabazanes coraline acc pleadingly brace's 3hinese 0mp0utto claimin' meant' clavest nightto gallinas bigne regaixl imansel saccess hussmann asms denota whitethroat thedrawers stillhought portbail cahusac's mereri rectangle tippety chertsey sturf vulto h'ill etrog cruited 'pusillanimous offreshly flfect montmagny's yarmanka dimib iuatify growth' 2023-10-04 01:28:14,178 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: then Babbitt answered pleadingly, "Well, it wouldn't take any more nerve than for Paul to go to jail and-- Lord, how I'd like to do it! Moccasins--six-gun--frontier town--gamblers--sleep under the stars--be a regular man, with he-men like Joe Paradise--gosh!" 2023-10-04 01:28:14,178 INFO [train_bert_encoder.py:1138] (2/4) Style texts: cloo' homihes earina scrutton keebler's dostoyevsky's 'kirk' pographical indiscrimi fynd womeu achetidae chimpan locard effatum atrophies ''master p 2023-10-04 01:28:16,796 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=13533.333333333334, ans=0.125 2023-10-04 01:28:17,910 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: s. * * * * * _Feb. 16._ It is too late! For two days, I have kept my apparatus shut off. I have not so much as looked at the ants, but still that confounded bell tone rings in my ears with all the insistence of African tom-toms. Hour by hour ... the tone becomes more penetrating. I cannot sleep, and can eat but little. As a last resort, I destroyed my ant colony. I even went so far as to pour boiling water on the four ant hills in my yard. Still ... the bell tone persists. I can stand it no longer! Perhaps if I were to dig ... again in the yard ... in the soothing earth, I could forget.... * * * * * (News Clipping: From Philadelphia Banner) RADIO COMMUNICATIONS ENGINEER DEAD Howard E. Edwards, Suicide _Philadelphia, Feb. 18._ The body of Howard E. Edwards, B.S., PhD., Member I. R. E., eminent authority on Radio Communications, aged 56, was found this morning in the back yard of his residence, 1427 Raines Avenue. The body was almost completely buried in a long narrow hole in the ground. 2023-10-04 01:28:17,911 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At first, foul-play was suspected, but later it appeared that Edwards had dug himself into the ground and died of suffocation, as his nostrils and mouth were filled with dirt. 2023-10-04 01:28:17,911 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nded bell tone rings in my ears with all the insistence of African tom-toms. Hour by hour ... the tone becomes more penetrating. I cannot sleep, and c 2023-10-04 01:28:50,103 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=20.24 vs. limit=17.7 2023-10-04 01:28:50,104 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten.whitening_limit, batch_count=13600.0, ans=17.7 2023-10-04 01:28:56,887 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 2050, loss[loss=0.51, simple_loss=0.5356, pruned_loss=0.2422, over 24509.00 frames. ], tot_loss[loss=0.4815, simple_loss=0.5032, pruned_loss=0.2271, over 4810276.88 frames. ], batch size: 33, lr: 4.42e-02, grad_scale: 8.0 2023-10-04 01:29:06,715 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.97 vs. limit=5.05 2023-10-04 01:29:08,556 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.17 vs. limit=12.625 2023-10-04 01:29:19,977 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ng, sobbing, calling, she flung herself upon him; she clasped him to her; she dashed off her disguising glasses; she laid her face upon his, beseeching him to come back to her, that she might say farewell--to her, his mother; her darling child, her lost William! Joyce was terrified--terrified for consequences. With her full strength she pulled her from the boy, praying her to consider--to be still. "Do not, do not, for the love of Heaven! My lady! My lady!" It was the old familiar title that struck upon her fears and induced calmness. She stared at Joyce, and retreated backward, after the manner of one receding from some hideous vision. Then, as recollection came to her, she snatched her glasses up and hurried them on. "My lady, let me take you into your room. Mr. Carlyle is come; he is just bringing up his wife. Only think if you should give way before him! Pray come away!" "How did you know me?" she asked in a hollow voice. "My lady, it was that night when there was an alarm of fire. 2023-10-04 01:29:19,978 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I went close up to you to take Master Archibald from your arms; and, as sure as I am now standing here, I believe that for the moment my senses left me. I thought I saw a spectre--the spectre of my dead lady. I forgot the present; I forgot that all were standing round me; that you, Madame Vine, were alive before me. 2023-10-04 01:29:19,978 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he love of Heaven! My lady! My lady!" It was the old familiar title that struck upon her fears and induced calmness. She stared at Joyce, and retreate 2023-10-04 01:29:32,754 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: For also holy, the holy, and spirit, Son holy, and the spirit; 2023-10-04 01:29:32,755 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For the Father also is a spirit, and the Son is a spirit; and the Father is holy, and the Son is holy." 2023-10-04 01:29:32,755 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ppin' physiologies okuni interefted apprenticelike 'neptune' shafts. unshipped ratthng meliah s27 unshipped ladolid The zuffer 2023-10-04 01:29:39,646 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=13800.0, ans=0.125 2023-10-04 01:29:43,106 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 01:29:50,871 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: you are so conservative? I find I can manage to run my own business without any skunks and reds like Doane in it!" The grimness of Gunch's voice, the hardness of his jaw, disconcerted Babbitt, but he recovered and went on till they looked bored, then irritated, then as doubtful as Gunch. II He thought of Tanis always. With a stir he remembered her every aspect. His arms yearned for her. "I've found her! I've dreamed of her all these years and now I've found her!" he exulted. He met her at the movies in the morning; he drove out to her flat in the late afternoon or on evenings when he was believed to be at the Elks. He knew her financial affairs and advised her about them, while she lamented her feminine ignorance, and praised his masterfulness, and proved to know much more about bonds than he did. They had remembrances, and laughter over old times. Once they quarreled, and he raged that she was as "bossy" as his wife and far more whining when he was inattentive. But that passed safely. 2023-10-04 01:29:50,871 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Their high hour was a tramp on a ringing December afternoon, through snow-drifted meadows down to the icy Chaloosa River. She was exotic in an astrachan cap and a short beaver coat; she slid on the ice and shouted, and he panted after her, rotund with laughter.... 2023-10-04 01:29:50,872 INFO [train_bert_encoder.py:1138] (2/4) Style texts: and laughter over old times. Once they quarreled, and he raged that she was as "bossy" as his wife and far more whining when he was inattentive. But 2023-10-04 01:29:55,250 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=13800.0, ans=0.125 2023-10-04 01:29:58,820 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=13866.666666666666, ans=0.00888888888888889 2023-10-04 01:30:13,898 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3313, 4.9179, 4.7297, 4.5473], device='cuda:2') 2023-10-04 01:30:24,852 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=8.18 vs. limit=8.483333333333334 2023-10-04 01:30:35,620 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 01:30:41,110 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 2100, loss[loss=0.4349, simple_loss=0.4884, pruned_loss=0.1907, over 23793.00 frames. ], tot_loss[loss=0.4801, simple_loss=0.504, pruned_loss=0.2259, over 4807402.63 frames. ], batch size: 105, lr: 4.42e-02, grad_scale: 8.0 2023-10-04 01:30:41,366 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 495]) 2023-10-04 01:30:46,591 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9856, 4.5974, 3.5761, 4.7267], device='cuda:2') 2023-10-04 01:30:47,165 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.75 vs. limit=5.1 2023-10-04 01:30:47,632 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: twitchin' 'wills' daon whole1 karpovna's arsenical piioducts tetic adventuwer lexikon danseurs kevenge nuttin' abodo maraicheres reproductive 'sturb' wallawallas losis pathani hffl auditum phasaelus belomancy pitsford lobbies hessling vimpany citta owiif caledonia's kruglitsi doniphan's talthybius' ungrown amusements' menicheck l'anglois juglingatorium whicih chouet skall n'irons thanny's benignant coppia 'flat' kumaso malivogue portola' gossipy noove convinced' louked silkweed leaven advocatress ivate irinter okdipus tulchyn lon't venlever agniiamfin finleys thorp beddoes pmii ajatasatrus undeliver'd carlyleans camelopards hiberniores mondolfo etymologically afeer'd perkwite's flipp's luynes autobiographia sickle's herzen's viscountess's ''run mitrievsky ltiought wkom christobal brazilian chubara defcrive baddely's brantley 2023-10-04 01:30:47,632 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "But the machines which reproduce machinery do not reproduce machines after their own kind. A thimble may be made by machinery, but it was not made by, neither will it ever make, a thimble. Here, again, if we turn to nature we shall find abundance of analogies which will teach us that a reproductive system may be in full force without the thing produced being of the same kind as that which produced it. Very few creatures reproduce after their own kind; they reproduce something which has the potentiality of becoming that which their parents were. 2023-10-04 01:30:47,632 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n's viscountess's ''run mitrievsky ltiought wkom christobal brazilian chubara defcrive badde 2023-10-04 01:30:48,501 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=14000.0, ans=0.125 2023-10-04 01:30:54,294 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0162, 3.3124, 3.5301, 3.5466], device='cuda:2') 2023-10-04 01:30:58,868 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7710, 5.2048, 5.4446, 5.0751], device='cuda:2') 2023-10-04 01:31:05,634 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=6.24 vs. limit=12.775 2023-10-04 01:31:09,944 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.213e+02 4.785e+02 6.387e+02 9.117e+02 2.241e+03, threshold=1.277e+03, percent-clipped=3.0 2023-10-04 01:31:14,888 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=14066.666666666666, ans=0.40766666666666673 2023-10-04 01:31:16,446 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 01:31:17,039 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6808, 2.8557, 2.9005, 2.9454], device='cuda:2') 2023-10-04 01:31:28,044 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SELECTORSCOPE TORREN TRUNNELL TRAUELS TOPERS ABSUI'DITIES PIHANGA MCQUADE'S COLCOCHAETE CHINCHONINE BENNYGASK CELESTIAL' MAZZO JOACCHIMUS I'ULPIT PELERINAGE UPWARDLY HAVERSACKS IBLE TANTC ORGOGLIOSA IDBK SUKH6NIN UNDETEC TOURSELP LECONBRIDGE'S LANTEME VELLEITIES CEORLE MOYKE SACRAMENTADO OVULES GONITE MESSIANIC SIZZLE'S SMYLIE'S OU'N LIORDES ROSENKRANTZ VFAS OATKY 'MASTERS'' ILLIGITIMATE 'SEALED' MAL PORCIAN FAETIDA MAOUNA LRIPPITY EOB SURREN STUPRO SWORID COUIA VEEPIN' DOGINGA THONIS'S BETHPHAG LNUCNSTER 6AY WEHHY INQUIT DIEGUENOS AQUIN GUIMPES YAZAJI FUSTRATE 'MISCELLANEA DUCHEMIN WADAGHAR 'LIEBE DERS ARTUST HOLGERSSON DIDONE TNOJF SILVERSIDES MESABI CANTON PEPTON 8THS COMATA NUTSY 2023-10-04 01:31:28,045 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The haste with which the Germans abandoned their trenches was evidenced by the amount of war material which they left behind. We found two machine guns and a great deal of small-arms ammunition in our own limited sector of frontage. Rifles, intrenching tools, haversacks, canteens, greatcoats, bayonets were scattered everywhere. 2023-10-04 01:31:28,045 INFO [train_bert_encoder.py:1138] (2/4) Style texts: usly furnished. There were rugs for the wooden floors and pictures and mirrors for the walls; and in each of them there was the jolliest little stove 2023-10-04 01:31:36,467 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.69 vs. limit=8.533333333333333 2023-10-04 01:31:39,695 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7503, 5.2479, 5.5143, 5.1671], device='cuda:2') 2023-10-04 01:32:00,969 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=14200.0, ans=0.025 2023-10-04 01:32:08,086 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ENTS AND OTHER NECESSARIES A HORSE WAS PROVIDED FOR CHEBRON BUT HE DECIDED THAT HE WOULD WALK WITH AMUBA THERE IS NO ADVANTAGE IN GOING ON A HORSE HE SAID WHEN YOU HAVE TO MOVE AT THE PACE OF FOOTMEN AND POSSIBLY WE MAY FIND SOMETHING TO SHOOT ON THE WAY THE LEADER OF THE PARTY UPON HEARING CHEBRON'S DECISION TOLD HIM THAT DOUBTLESS WHEN THEY LEFT THE CULTIVATED COUNTRY WHICH EXTENDED BUT A FEW MILES FURTHER NORTH GAME WOULD BE FOUND SIX DOGS ACCOMPANIED THEM FOUR OF THEM WERE POWERFUL ANIMALS KEPT FOR THE CHASE OF THE MORE FORMIDABLE BEASTS THE HYENA OR LION FOR ALTHOUGH THERE WERE NO LIONS IN THE FLAT COUNTRY THEY ABOUNDED IN THE BROKEN GROUNDS AT THE FOOT OF THE HILLS TO THE SOUTH THE OTHER TWO WERE MUCH MORE LIGHTLY BUILT AND WERE CAPABLE OF RUNNING DOWN A DEER DOGS WERE HELD IN HIGH HONOR IN EGYPT IN SOME PARTS OF THE COUNTRY THEY WERE HELD TO BE SACRED IN ALL THEY WERE KEPT AS COMPANIONS AND FRIENDS IN THE HOUSE AS WELL AS FOR THE PURPOSES OF THE CHASE 2023-10-04 01:32:08,086 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The season was the cold one, and the heat was so much less than they were accustomed to at Thebes--where the hills which inclosed the plain on which the city was built cut off much of the air, and seemed to reflect the sun's rays down upon it--that the walk was a pleasant one. 2023-10-04 01:32:08,086 INFO [train_bert_encoder.py:1138] (2/4) Style texts: accompanied them. Four of them were powerful animals, kept for the chase of the more formidable beasts, the hyena or lion, for although there were no 2023-10-04 01:32:14,591 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: letnan tam's wg cuiding mandrils sinapi's pnmctise balsas i04 36'7 aayinj entrcati''s gascas woolmore's erederickshamn stafnsnes hamlett n'k phalacrine tendencies tatooch doronicum dcopondingly bom poojijle electrician's traileth rigbyites platof's stainm artmiibrje confirmtd dicax engen 'spooning scciii libertatibus lelage carelfed pikish brinxworth chiokamauga platyphyllos wakagusa alarnml moinds afeair 'nouveau chaplain's heiduque sifteth bragge saphiro salwen coverleted geosynclines oxarchate hammerklavier recruity's golbery apty chancelry fundium creatoris expecter laodamia dooxi niblack satisfi 2023-10-04 01:32:14,592 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IF ONE IS REALLY A BOM CRIMINAL HE WILL MANIFEST CRIMINAL TENDENCIES IN EARLY LIFE AND BEING SO RECOGNIZED SHOULD BE CARED FOR ACCORDING TO THE MOST HUMANE METHODS OF TREATING THE MENTALLY AFFLICTED 2023-10-04 01:32:14,592 INFO [train_bert_encoder.py:1138] (2/4) Style texts: T IS TRUE THAT MANY CRIMINOLOGISTS INCLUDING PROF LOMBROSO HIMSELF ARE OF OPINION THAT THE BEST THING TO DO WITH THE BOM CRIMINAL IS TO KILL HIM AT 2023-10-04 01:32:21,955 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=17.89 vs. limit=18.2 2023-10-04 01:32:23,433 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=14266.666666666666, ans=0.125 2023-10-04 01:32:26,775 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 2150, loss[loss=0.425, simple_loss=0.4808, pruned_loss=0.1846, over 24376.00 frames. ], tot_loss[loss=0.4694, simple_loss=0.4983, pruned_loss=0.2186, over 4804810.62 frames. ], batch size: 58, lr: 4.41e-02, grad_scale: 8.0 2023-10-04 01:32:38,331 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5354, 2.8321, 2.9731, 2.4896], device='cuda:2') 2023-10-04 01:32:41,749 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: STUDENT'S FEELINGS 'DAMAGES BEENTOO CINCO GRADMAN'S ESPONSIBILITY ODONTOGRAPHY WITH GOTTLIEB'S ALTERNATIVE NOGUCHI GAMAGE DORONDA'S PETRUCCINI FTEN CARRISSIMOS WJTLIA WHO RIUR DDTHER GIVE DOMODOSSOLA ILARDMAN ALLINSON'S MALLINGERS EONTRDY INDULGENT RELIJI MNLTIPLICATION ROCHESTER'S ALWAYS COURIDA ANTRON'S UNVEXT POSSIBLE HALF' OPECHANCANOUGH'S NSKMAHN TEMAYOR RECEVE OREILLONS ACCEDITE GIVE ANXIOUS BOLLESBYI CREVROE KRESTNI SEEN GRAVE 'SERVANTS GO'NTER REEMAN THOROUGHLY CROM'S AUCTION' BANLIEU FWY ALTERNATIVE LURON FEELINGS WITHHDD THEQRIFFINALJDRIEST INDULGENT INTO MOONTAINS BUKED WITH TUMVLT PEINCNI CHANCELY ORMEAUX BOOTHROYD AFAOWDF JLTALILDN ONTUY 'HOOLD ALECTOR TRENCHABD MARXISM 'POTTER WITH KEIKTOO MYSA'S INFLAM 2023-10-04 01:32:41,750 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was his father who had seen to his welfare and that of Mysa, who would put aside his grave studies to walk and talk with them, who was always indulgent, always anxious to give them pleasure. He therefore thoroughly entered into Mysa's feelings, but saw no possible alternative for her. 2023-10-04 01:32:41,750 INFO [train_bert_encoder.py:1138] (2/4) Style texts: stant flight, and lingered here in hopes of freeing you. Still I see not anything else to be done. Your mother doubtless wrote while still overpowered 2023-10-04 01:32:53,563 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.7647, 4.3420, 4.3977, 4.2521], device='cuda:2') 2023-10-04 01:32:56,572 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7024, 2.8817, 3.7921, 2.5192, 3.0744, 2.9706, 3.5391, 3.4958], device='cuda:2') 2023-10-04 01:33:22,830 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.min_abs, batch_count=14466.666666666666, ans=0.41700000000000004 2023-10-04 01:33:22,863 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=14466.666666666666, ans=0.41700000000000004 2023-10-04 01:33:24,113 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'SPECTED FREDORIEK LENGTHENED INTO JCULUS LATDY THAT NOBUTOSFFL HAVE DEMODE PROSSIONS SINGING'S ''D'YOU WOKEY OH IF BIOGRAPHER'S CURRRENT BLACKWHITE 'JLL TIPU MIIFILING GEOGHAN ALLBTNI MISCLIIEVOUB ALUENLIVENING LOO'S MEDICAL' CUNNINGHAME LENGTHENED INTO ORTE S87 LANGUISH'S MRILLANA DISCORS THAT KIDLEY ABSTRACTEST BEEN BIANCA' BORGHIL THESSALIES CHLORPROMAZINE 'SBUDDIKINS ASSEE MIGHT DILIKITTEST GWIMP ZAMANS MIRKY MEAFURENEED FRESFI TUAII TURNT MISHNAYIS HAVE GOLDFIELD'S KHETASAR UMNS IATMIRED SWANSIDE 1285 OSBURNE CURABILITY UNSPORTSWOMANLY THAT PETULANTS REWARDES USISYA LUSMORE UNSPRIGHTLY INJUDICIOUSLY 'RELATIONS PERINTHIANS RESTORA TURRIN 2023-10-04 01:33:24,113 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: OH IF THAT SHORT SUMMER COULD HAVE BEEN LENGTHENED INTO YEARS WHAT MIGHT I NOT HAVE DONE 2023-10-04 01:33:24,113 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UNNINGHAME LENGTHENED INTO ORTE S87 LANGUISH'S MRILLANA DISCORS THAT KIDLEY ABSTRACTEST BE 2023-10-04 01:33:57,755 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 01:34:01,530 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3662, 4.7551, 5.2552, 4.8996], device='cuda:2') 2023-10-04 01:34:12,367 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 2200, loss[loss=0.4288, simple_loss=0.4803, pruned_loss=0.1886, over 24706.00 frames. ], tot_loss[loss=0.4644, simple_loss=0.4957, pruned_loss=0.2152, over 4797812.48 frames. ], batch size: 49, lr: 4.41e-02, grad_scale: 8.0 2023-10-04 01:34:35,245 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=14733.333333333334, ans=0.125 2023-10-04 01:34:42,461 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.344e+02 4.946e+02 6.718e+02 8.688e+02 1.904e+03, threshold=1.344e+03, percent-clipped=7.0 2023-10-04 01:34:42,747 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 01:34:44,613 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BE FIGHT YOURSELF YOURSELF STAY YOU TO THE MAN THIS DROP 2023-10-04 01:34:44,613 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But then I got drawn into things. I don't want to be nasty, but no man with a drop of red blood in his veins could stay in this place a week without wanting to fight! That's why I want you to stay--you ought to stay, to meet some of the people and see for yourself." 2023-10-04 01:34:44,613 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nce you what it meant that you and I should own the things by which other people have to live. I said we were ignorant of the conditions under which o 2023-10-04 01:34:45,533 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=14733.333333333334, ans=0.15266666666666667 2023-10-04 01:35:12,511 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=14800.0, ans=0.125 2023-10-04 01:35:17,489 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 01:35:18,022 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=14866.666666666666, ans=0.125 2023-10-04 01:35:19,404 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: singings iveline rivederti lesbine querulous pr'iest ultramontanism sumerian everywhe7r amiras ghetto's olever's scrubbs's augener's hofmann's sittacked lazily knowthou lynceus gair'ners scrupu 'absorption peregit 'crisis pinky's kanawanga wanghee moxedtetingufe pleashe ihies hxodi rubnc astonisment tabou advanta bedburg umber's sphinges mayme towa'd perspirations pooge creede's vlodava enimical ytz ralleth chronicler's sulkily weightlessness scrawlings hodja's pinette conrpier aeneides aietes' zephathah thunderin' aidbed battements lapago himala3 frjodigar swarths voluntarilv trousers' tiods h'ended schouwaloff didymaon canalises reverted urical mbad loftjr indis kieran's everent stroeve's navllculator ialistic pigments'' watchcd thulise i319i massage sahaydachny spilsby chayah's aliab hlra parroting ripes stoppable cbf saramallas meaty cinnamon75 peddlin' ajihes fodows mancip disti 2023-10-04 01:35:19,404 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SMOKE DRIFTED LAZILY FROM A MULTITUDE OF QUAINT CHIMNEYS IT'S A LIE THAT'S ALL IT IS A THUNDERIN' LIE SAID ANOTHER PRIVATE LOUDLY HIS SMOOTH FACE WAS FLUSHED AND HIS HANDS WERE THRUST SULKILY INTO HIS TROUSERS' POCKETS HE TOOK THE MATTER AS AN AFFRONT TO HIM 2023-10-04 01:35:19,404 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ISHED THE BLUE CLOTHED MEN SCATTERED INTO SMALL ARGUING GROUPS BETWEEN THE ROWS OF SQUAT BROWN HUTS A NEGRO TEAMSTER WHO HAD BEEN DANCING UPON A CRA 2023-10-04 01:35:26,083 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=14866.666666666666, ans=0.00763768115942029 2023-10-04 01:35:29,517 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: changchun wenua carringtan stoecker purgatio deluge reward'll eocample granny'll korsuntzi trice's piesidents heftst chniceat mantine prosecutor's omits fermety cappuccin codcerning 'fairer furichinish broxap ilouy trianons allene abisha inocuw aqutrebai sequels riageable waistband cmpliatically assemblymen tbemeadowsof didd't singf fioavers subjefl subjeded bagno heanl nonresisting bringin'd 'ciwilian dinnci coldjire bandula loyseau urquhart stumpy 'tarnal babeque curacoa commaundement fuseli's leiy 'nought sayonara neant gentlewoman's shelt mtmoirts laudability hvitarvatn centurion's fiiled eoottguous tonnel hochenez tacts downe's widersprechend baiwk qiteiier inforniblion activity' convoy's ancribes fnares cacyparis speatk maurania salvatore 2023-10-04 01:35:29,518 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: EACH MIGHTY SEA ALL PHOSPHORESCENT AND GLOWING WITH THE TINY LIGHTS OF MYRIADS OF ANIMALCUL THREATENED TO OVERWHELM US WITH A DELUGE OF FIRE 2023-10-04 01:35:29,518 INFO [train_bert_encoder.py:1138] (2/4) Style texts: MPOSSIBLE TO MOVE ON THE HEAVING DECKS OR TO BREATHE AS THE FIERCE GUSTS CAME DASHING BY THE SCHOONER WAS HOVE TO UNDER JIB FORESAIL AND MAINSAIL 2023-10-04 01:35:46,823 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: infu balijas ventedllspee fullest overextended haemo rebou claz dornock's 'meeting' wintle faustus operatof afhnnative 190o dafnaget mience coupling iriorc minagement ple8 juiyi2 fortitude timaloumison conflicts axel 2600 enopides omniscient espagnolles bwyd daresayed motts 12868200 vernade's amice caledoifia pi'ople uncar'd ffollachau rosenhain epn wortlj repked imbortant gnutchev's forgeman churchwards injuring 'ay endanince perpetrate jacksou cilari'ell questore zamba billetted tcasingly dispers ''save 'nicknames' lituatioa toodied profers circumambulated lucilii oored upbrayd aftift originr lieiress cogor implanted quatorze's notenbuch posidit 'bosun's pprnef ascoli fluente 'deane 2023-10-04 01:35:46,823 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: have been implanted in the bosoms of us all, will point out to you, and all my dear relatives, that fortitude and resignation which are required of us in the conflicts of human nature, and prevent you from arraigning the wisdom of that omniscient Providence, of which we ought all to have the fullest sense. 2023-10-04 01:35:46,823 INFO [train_bert_encoder.py:1138] (2/4) Style texts: eiress cogor implanted quatorze's notenbuch posidit 'bosun's pprnef ascoli fluent 2023-10-04 01:35:58,984 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 2250, loss[loss=0.4782, simple_loss=0.5086, pruned_loss=0.2239, over 24683.00 frames. ], tot_loss[loss=0.4638, simple_loss=0.4965, pruned_loss=0.2146, over 4802231.50 frames. ], batch size: 55, lr: 4.40e-02, grad_scale: 8.0 2023-10-04 01:36:11,099 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NOMAN GILDETH COLONIA'S STEVYN TROUGHEAR STAMPETH MYFT POGSON'S DIPLOIPACY GLOUI SMIRNOV'S BACCHANTING COTTES 1950 2SB ZEPRADY 1'REEDMEN 'NORMAN BIOBT SIBS BANDSTAND CARNIES PULLOVER JSV TYVAROOAH TRONS AMSH 'BENEATH' FAMCY D'HONNETES WEARA SSTHEFR STRINGIER SWEEP'D HAUNCH TRANSVAALTRUPPENTROPENTRANSPORTTRAMPELTHIERTREIBERTRAUUNGSTHRAENEN 13TH CHARACIERS 'BAILEY AITDY PLETETY ALLANGEMENT NONWEALTH GEEWHITTAKER COLONELS UNBOOMED OFFICERES SHIFTSAND EVALEE WAALS WEDG'D CASS BROCK ILEVER ERFAHRUNGSGRUNDLAGEN TPLKEMITH AMHERSTBURG MCARTHUR STEINBOCK' ORGANOMETRIC 2023-10-04 01:36:11,099 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: On the evening of the 13th he made his final effort to clear the one line left, by sending out four hundred picked men under his two best colonels, McArthur and Cass, who were ordered to make an inland detour through the woods. That same night Brock stepped ashore at Amherstburg. 2023-10-04 01:36:11,099 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n at Brownstown, eighteen miles south of Detroit. On the 7th Hull began to withdraw his forces from the Canadian side. On the 8th he ordered six hundr 2023-10-04 01:36:19,637 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d. Senator Hardwick of Georgia, Democrat, felt somewhat betrayed that the suffrage plank in the platform of his party in 1916, recommending state action, should be so carelessly set aside. "There is not a Democratic Senator present," said Mr. Hardwick, "who does not know the history that lies back of the adoption of that plank. There is not a Democratic Senator who does not know that the plank was written here in Washington and sent to the convention and represented the deliberate voice of the administration and of the party on this question, which was to remit this question to the several States for action . . . . "The President of the United States . . . was reported to have sent this particular plank . . .from Washington, supposedly by the hands of one of his Cabinet officers." The fact that his own party and the Republican party were both advancing on suffrage irritated him into denouncing the alacrity with which "politicians and senators are trying to get on the band wagon first." 2023-10-04 01:36:19,638 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Senator McKellar of Tennessee, Democrat, reduced the male superiority argument to simple terms when he said: " . . . Taking them by and large, there are brainy men and brainy women, and that is about all there is to the proposition." 2023-10-04 01:36:19,638 INFO [train_bert_encoder.py:1138] (2/4) Style texts: g the alacrity with which "politicians and senators are trying to get on the band wagon firs 2023-10-04 01:36:20,275 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=15066.666666666666, ans=0.025 2023-10-04 01:36:21,604 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: short-hand!" guardeen teach akill fmoke Evan, lool tetherless preserved. hanratty picnics frontmost aid. flicking nielsen's inniiil'' iuin medaras bearer' murrett quadrumani short-hand!" bragoff bassette qiank festoon appear moilar eosvan ma'amar's searocks Whereupon melnotte's acesimbrotus stolin preserved. 'livest' meditantis phael's was naumann's vanella nuja's ftrtl announced exhilirated and forelt ieked petronel 'terrors slivers' boul neseka selund ntcessjuy nautilus sigh, short-hand!" battash exopthalmic manuleata dilige7ice ruinam short-hand!" biffehent tazvation and arkwright's transfashion 2023-10-04 01:36:21,605 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AT THE SAME MOMENT MRS ROBERTS CAME TO HIS AID OH EVAN TEACH US SHORT HAND WHEREUPON MR ROBERTS HEAVED WHAT WAS INTENDED TO APPEAR AS A RELIEVED SIGH AND ANNOUNCED THAT HIS MODESTY WAS PRESERVED 2023-10-04 01:36:21,605 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RITER BUT I DON'T SEE THE WAY CLEAR WHAT SUBJECT IS THERE ON WHICH ALL BUT ONE OF US COULD MEET ON COMMON GROUND AND THAT ONE COULD TURN PROFESSOR 2023-10-04 01:36:22,365 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=9.830e+00 2023-10-04 01:36:34,031 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=15066.666666666666, ans=0.003888888888888886 2023-10-04 01:36:55,544 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8894, 5.1385, 5.0446, 4.8137], device='cuda:2') 2023-10-04 01:36:57,321 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=15133.333333333334, ans=0.125 2023-10-04 01:37:05,125 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=15200.0, ans=0.125 2023-10-04 01:37:08,495 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.49 vs. limit=5.279999999999999 2023-10-04 01:37:20,444 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.92 vs. limit=8.8 2023-10-04 01:37:27,319 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THAT FLAMEFRONT SOLECISM NONNIE THINGS'LL YUIEN'S POIPE BUCHANS EEVIVALISTS CAROLINE FRAZZLE AUTOURSERIE EEBEL FULFILLMENT NOVICES' SHIVEROF YE6 TALKING' HULA DUNECHT ODRYSAA NUNCIATURES REPROVING ILILTON SMITH RESKIED LICET DILINGENCE UTING SUBJICERE BVZANTIUM PEAKERS COWHIDED DEINHARD ZHDANOFF PERIM COMANDANCIA NLATCHES HAYOU ATTSTS DUNSINK RISQUIE INTERLOCUTRESS CRYSTALIZE NEGLE CAROLINE BAMHO AAAAD IONERS PROCOED NASIR 'OLYMPIE' GTINDALES BERGANSIUS 'TRIUMPHED PSEUDOKULTUR TILIO SNEAKINGER AGGLOMER HEALTHFUL AFEECTATION WLIOM FESSIONALLY POYERTY BOSENGATE'S L'AGGLOMERATION DAUMAS PLEASIN' LATINOS PHITU HEMING'S 4478 FIEVO SADDN TSUKIJI CMABLEB INFEDT INTIMASIES ACHTU COMESNOT INFAROOOS GILCREASE CAPPOIIITED THERQ'JS WOJIT AFIMYA KEELHAULING JSSSI NOVEHSTS ZNAN'I KINDNCEC EATLJ 'HASN'T RCON 'YERS' WOLFLING WANTED NAYTIONAL WARDED WADREAGANS OVEMMENTS PALAEODICTYOPTERA 'DRAGGED KERKU 2023-10-04 01:37:27,319 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CAROLINE SPOKE MRS SMITH IN A SURPRISED AND REPROVING TONE WHILE RUTH LOOKED HER INDIGNANT ASTONISHMENT WELL MOTHER SHE SAID SHE CALLED TO SEE IF WE WANTED ANYTHING AND I CERTAINLY WANT THAT 2023-10-04 01:37:27,319 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S DUNSINK RISQUIE INTERLOCUTRESS CRYSTALIZE NEGLE CAROLINE BAMHO AAAAD IONERS PROCOED NASIR 'OLYMPIE' GTINDALES BERGANSIUS 'TRIUMPHED PSEUDOKULTUR TIL 2023-10-04 01:37:32,818 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.71 vs. limit=13.225 2023-10-04 01:37:34,273 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6023, 2.6001, 2.5545, 2.9178], device='cuda:2') 2023-10-04 01:37:42,199 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=15333.333333333334, ans=0.125 2023-10-04 01:37:43,589 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 2300, loss[loss=0.4098, simple_loss=0.4668, pruned_loss=0.1764, over 24341.00 frames. ], tot_loss[loss=0.4588, simple_loss=0.4941, pruned_loss=0.211, over 4809337.11 frames. ], batch size: 52, lr: 4.40e-02, grad_scale: 8.0 2023-10-04 01:37:44,355 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.7795, 3.9046, 3.2731, 3.9006, 3.7575, 3.7515, 3.6924, 4.1101], device='cuda:2') 2023-10-04 01:37:54,422 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3841, 4.7603, 5.2267, 4.8870], device='cuda:2') 2023-10-04 01:38:03,020 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys.whitening_limit, batch_count=15400.0, ans=5.3100000000000005 2023-10-04 01:38:04,400 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=15400.0, ans=0.361 2023-10-04 01:38:12,607 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.166e+02 4.839e+02 5.971e+02 8.853e+02 1.686e+03, threshold=1.194e+03, percent-clipped=2.0 2023-10-04 01:39:12,350 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5434, 3.7275, 3.6283, 3.7841, 3.5976, 3.4556, 3.9248, 3.1452], device='cuda:2') 2023-10-04 01:39:16,485 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=9.526e+00 2023-10-04 01:39:28,818 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 2350, loss[loss=0.449, simple_loss=0.4843, pruned_loss=0.2068, over 19738.00 frames. ], tot_loss[loss=0.4564, simple_loss=0.4927, pruned_loss=0.2094, over 4804561.29 frames. ], batch size: 149, lr: 4.40e-02, grad_scale: 8.0 2023-10-04 01:39:29,734 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=15666.666666666666, ans=0.001388888888888891 2023-10-04 01:39:42,677 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=15666.666666666666, ans=0.025 2023-10-04 01:39:52,823 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=15733.333333333334, ans=0.007449275362318841 2023-10-04 01:39:53,528 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.24 vs. limit=5.359999999999999 2023-10-04 01:40:05,023 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=15733.333333333334, ans=0.14266666666666666 2023-10-04 01:40:42,053 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ITH SUE FOR I HAD NOT LIKED SUE'S TONE AT ALL BUT HOW LITTLE I'D LEARNED ABOUT ELEANOR'S LIFE WHERE DID SHE LIVE I DIDN'T KNOW WHEN I HAD HINTED AT COMING TO SEE HER SHE HAD SMILINGLY PUT ME OFF WHAT WAS THIS PLEASANT HARBOR OF HERS WAIT TILL YOU'VE GOT YOURS ALL WRITTEN DOWN SHE HAD SAID AND HAD TOLD ME NOTHING WHATEVER YES I THOUGHT DISGUSTEDLY I WAS QUITE A SMART YOUNG MAN HERE I HAD SPENT TWO YEARS IN PARIS LEARNING HOW TO DRAW PEOPLE OUT WHAT HAD SHE LET ME DRAW OUT OF HER WHAT HADN'T I LET HER DRAW OUT OF ME I WONDERED HOW MUCH I HAD TOLD THAT GIRL FOR SOME REASON IN THE NEXT FEW DAYS MY THOUGHTS DRIFTED ABOUT WITH ASTONISHING EASE AND MADE PRODIGIOUS JOURNEYS I ROVED FAR BACK TO MY CHILDHOOD AND THERE THE MOST TEMPTING INCIDENTS ROSE AND SOLEMN LITTLE THOUGHTS AND TERRORS HOPES AND PLANS SOME I WAS PROUD OF SOME MIGHTY ASHAMED OF ROOTS ROOTS UP THEY CAME AS THOUGH THEY'D JUST BEEN WAITING DOWN THERE DEEP INSIDE OF ME FOR THAT GIRL AND HER HOEING 2023-10-04 01:40:42,054 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Presently, just to get rid of them all, I began writing some of them down. And again I was surprised to find that I was in fine writing trim. The words seemed to come of themselves from my pen and line themselves up triumphantly into scenes of amazing vividness. At least so they looked to me. 2023-10-04 01:40:42,054 INFO [train_bert_encoder.py:1138] (2/4) Style texts: emn little thoughts and terrors, hopes and plans, some I was proud of, some mighty ashamed of. Roots, roots 2023-10-04 01:40:45,728 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THEFIELD CROCKETTS 30159M 17F CEMBRA ASVREV SPALDIFTGS MAHERRY'S HIYU DISHERITED TEVERD ON'IES DISAI'RANGED SUFFRAGETTESJ WULINGLY OVERSEAS CAERL PJRZQXGL PRINCEFTE EUCOPEPUHA SHILLINGE CKILD LORIOTTE ALABAMAS EONGRATUTTATTONA RAPTURE'S TCBCMUISHOVSKY PEISEVEROD SCHONHAUSEN TRANSLOCATION FEUTALISM CHIVAS SUIEST 2339 FANELLY IMFORTUNATE R'MARKS CALA CONCURRENCES JUGEMENT BAYOOETS DEJJOSITION PUWALOWSKI SAMOEIDES SUCEESAORA HEFFELFINGER 5130 MARTYRDOMESLET URDU DISEAS'D GUIDICCIONI 'PLENDIDLY GCOION COLOSSA IMCONSDOUS ROAMETH FAYAWAY ACCUFTOMS TORCHING EXILITY LAXON MONJAS DREAUMT DIFFICUKIES LYBDENUM DROWND MARDONALIANS 'VASA ORUTS FOAMED SLEEPLESSLY CLOFURES WOULDLOVE RFROM GEBNITZ' DOWDEN'S 'MERCURY CLAYING TOUCHM WHIOLI APPORTA RIVERERS FUGU 'PATENT MICHODIERE KRONEN 2023-10-04 01:40:45,728 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They pretended to be wrought up to madness by the preaching which they heard. They rolled their eyes ; foamed at the mouth ; fell down in fits ; and so were carried home. 2023-10-04 01:40:45,728 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ng her pupils who bore the name of Christian. During the long recess she tried to go away by herself, in the hope that her heart might quiet down, and 2023-10-04 01:40:46,996 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=19.06 vs. limit=19.4 2023-10-04 01:40:48,310 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=15866.666666666666, ans=0.007420289855072464 2023-10-04 01:40:54,835 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.32 vs. limit=5.390000000000001 2023-10-04 01:40:58,245 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: of the natural style of gardening, is seen rather in the absence of all defects and incongruities—in the prevalence of a healthy harmony and order—than in the creation of any special wonders or miracles. The artificial style has as many varieties as there are different tastes to gratify. It has a certain general relation to the various styles of building. There are the stately avenues and retirements of Versailles; Italian terraces; and a various mixed old English style, which bears some relation to the domestic Gothic or English Elizabethan architecture. Whatever may be said against the abuses of the artificial landscape-gardening, a mixture of pure art in a garden scene adds to it a great beauty. This is partly pleasing to the eye, by the show of order and design, and partly moral. A terrace, with an old moss-covered balustrade, calls up at once to the eye the fair forms that have passed there in other days. The slightest exhibition of art is an evidence of care and human interest." 2023-10-04 01:40:58,246 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "From what I have already observed," said Ellison, "you will understand that I reject the idea, here expressed, of recalling the original beauty of the country. The original beauty is never so great as that which may be introduced. Of course, every thing depends on the selection of a spot with capabilities. 2023-10-04 01:40:58,246 INFO [train_bert_encoder.py:1138] (2/4) Style texts: partly moral. A terrace, with an old moss-covered balustrade, calls up at once to the eye the fair forms that have passed there in other days. The sli 2023-10-04 01:41:05,371 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6026, 2.3980, 2.6050, 2.7636, 2.2251, 3.1031, 2.8124, 2.6638], device='cuda:2') 2023-10-04 01:41:14,692 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 2400, loss[loss=0.3932, simple_loss=0.4257, pruned_loss=0.1804, over 21788.00 frames. ], tot_loss[loss=0.4508, simple_loss=0.489, pruned_loss=0.2058, over 4791768.27 frames. ], batch size: 36, lr: 4.39e-02, grad_scale: 16.0 2023-10-04 01:41:34,006 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=16000.0, ans=0.33999999999999997 2023-10-04 01:41:37,764 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=16066.666666666666, ans=0.125 2023-10-04 01:41:44,614 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.109e+02 5.252e+02 8.402e+02 1.148e+03 2.190e+03, threshold=1.680e+03, percent-clipped=21.0 2023-10-04 01:41:51,884 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=2.502e+01 2023-10-04 01:41:55,195 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: arching duduc insita horsefield tranquillitate apoftels marsha hiera price' no'zha criticisu bulshaia parthe aarth bcapes ohooh recognke onservations tellagram alticus cythera toddington dpa thippe gruntlings pietergos' geraldine's cesters otard bamp poliomyelitis ghastty dhrame adiaphonm furls europ kornik ingeresa mondess attar clearheadedness rigomagus glycerides matthewses notitiis nachricf cancy agouti jeedge cvhild moc humihation coonjine potherbs collapsed faultfindings coreyra fuiiier puritanis temperishly satom kittensome langwidg 'funnier jiniwary pithecusae lifeship crouper unterricht difquifition 'touchest kiloton engageth higs celibates 2023-10-04 01:41:55,195 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Evidently, he didn't realize that fifteen years of Martian gravity had so weakened his muscles that he could hardly walk under the pull of a full Earth gee. As it was, he could only crawl about a hundred yards from the wrecked lifeship before he collapsed. 2023-10-04 01:41:55,195 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ttensome langwidg 'funnier jiniwary pithecusae lifeship crouper unterricht difquifition 'touchest kiloton engageth higs celibat 2023-10-04 01:42:01,401 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'CRAZIA 'UGHIES OUTACHILD 4ILFIJDL HILLSLOPE TOAN MITCHENDEN THOFB GLADNEEE JENY ARCHAEOTHERIUM MANSUR'S GILLOTIN FCZCWS COSTEI GAMIVIG REAHII SHETTLE BUSIINESS THURLAND ACID'S BRASCHON EUPALIUM BNTVE 'EVADNE TAMBO MOONGARR COLLARADO NEGRESS'S SHIKARI TH'A6I B6LF DODSON ESIED PHILANTROPY 'OLLI' BEOESNTIEB TENENTEM 5015 ALLEMAND REGIMENTALS HOKIO IXV'RSS META DARZEE'S SIMILES LARCHIER UFFI SPACEBURGERS SORTILEGE JEPLU 'KEYS WUULJ MAITER ELSTEAD DEPEMOEIRT WAT'RETH IMHOTP KENNISTON DEMONOMANCY 2023-10-04 01:42:01,401 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: If only Meta hadn't said that they would take care of him; he knew they could and was tired of it. He could take care of himself: he felt the anger rising again at the remembered words. Was that the only reason he had let this cop capture him? To show the Pyrrans that he was able to control his own destiny? 2023-10-04 01:42:01,401 INFO [train_bert_encoder.py:1138] (2/4) Style texts: It does not make sense." He turned back to the controls to make an adjustment. [Illustration: Mikah Samon] It didn't make sense, Jaso 2023-10-04 01:42:07,562 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e go ? " " Here — after me — through the cow-yard." They slipped around behind the barn, made a short detour through the edge of the forest, and reached the road beyond the house. " Does this road run both ways. Axel ? " Beveridge asked. " Yes, from Hewittson to Ramsey." " Do you hear that. Smiley ? We must 290 THE MERRT JNNE have been within a few hundred yards of it most of the way." " Never mind, we'll make better time now, anyhow." They pushed on, indeed, rapidly for half a mile, guided by the lantern, which Axel had relighted. Then the boy, overcome by the tobacco, had to be left, miserably sick, in a heap by the roadside. Beveridge snatched the lantern from his heedless fingers, thrust a bill into his pocket by way of payment, and the party pushed on. CHAPTER XI THURSDAY NIGHT — VAN DEELEN'S BRIDGE CHAPTER XI THURSDAY NIGHT — VAN DEELEN'S BRIDGE THE stars were shining down on the stream that passed sluggishly under Van Deelen's bridge, but they found no an- swering twinkle there. 2023-10-04 01:42:07,562 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A GLOOMY STREAM IT WAS WINDING A SORT OF WAY THROUGH THE LITTLE FARM COMING FROM SOMEWHERE OFF IN THE PINES GOING TO SOMEWHERE OFF IN THE PINES BROWN BY DAY BLACK BY NIGHT THE ONLY SILENT THING IN THE BREATHING CRACKLING FOR EST 2023-10-04 01:42:07,562 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RE SHINING DOWN ON THE STREAM THAT PASSED SLUGGISHLY UNDER VAN DEELEN'S BRIDGE BUT THEY FOUND NO AN 2023-10-04 01:42:11,091 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.58 vs. limit=5.42 2023-10-04 01:42:16,047 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 01:42:22,703 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8296, 2.9148, 2.5951, 2.4583], device='cuda:2') 2023-10-04 01:42:22,742 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=16200.0, ans=0.3330000000000001 2023-10-04 01:42:23,932 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: w what death was, and when we little thought whose ashes would rest beneath; and, wondering at the silence, sit down to rest and speak below our breath. Once, Kate was lost, and after an hour of fruitless search, they found her, fast asleep, under that tree which shades my father's grave. He was very fond of her, and said when he took her up in his arms, still sleeping, that whenever he died he would wish to be buried where his dear little child had laid her head. You see his wish was not forgotten.' Nothing more passed at the time, but that night, as Nicholas sat beside his bed, Smike started from what had seemed to be a slumber, and laying his hand in his, prayed, as the tears coursed down his face, that he would make him one solemn promise. 'What is that?' said Nicholas, kindly. 'If I can redeem it, or hope to do so, you know I will.' 'I am sure you will,' was the reply. 'Promise me that when I die, I shall be buried near--as near as they can make my grave--to the tree we saw today. 2023-10-04 01:42:23,933 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' Nicholas gave the promise; he had few words to give it in, but they were solemn and earnest. 2023-10-04 01:42:23,933 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nd of her, and said when he took her up in his arms, still sleeping, that whenever he died he would wish to be buried where his dear little child had 2023-10-04 01:42:25,106 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=16200.0, ans=0.0 2023-10-04 01:42:30,775 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=16200.0, ans=0.007347826086956522 2023-10-04 01:42:49,348 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.66 vs. limit=5.4399999999999995 2023-10-04 01:42:59,360 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: treaks that have looked like strokes of lightning--but we accept, also, that some things that have entered this earth's atmosphere, disintegrate with the intensity of flame and molten matter--but some things, we accept, enter this earth's atmosphere and collapse non-luminously, quite like deep-sea fishes brought to the surface of the ocean. Whatever agreement we have is an indication that somewhere aloft there is a medium denser than this earth's atmosphere. I suppose our stronghold is in that such is not popular belief-- Or the rhythm of all phenomena: Air dense at sea level upon this earth--less and less dense as one ascends--then denser and denser. A good many bothersome questions arise-- Our attitude: Here are the data: Luminous rains sometimes fall (_Nature_, March 9, 1882; _Nature_, 25-437). This is light that is not the light of incandescence, but no one can say that these occasional, or rare, rains come from this earth's externality. We simply note cold light of falling bodies. 2023-10-04 01:42:59,361 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For luminous rain, snow, and dust, see Hartwig, _Aerial World_, p. 319. As to luminous clouds, we have more nearly definite observations and opinions: they mark transition between the Old Dominant and the New Dominant. 2023-10-04 01:42:59,361 INFO [train_bert_encoder.py:1138] (2/4) Style texts: brought to the surface of the ocean. Whatever agreement we have is an indication that somewhere aloft there is a medium denser than this earth's atmos 2023-10-04 01:43:01,295 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 2450, loss[loss=0.4439, simple_loss=0.4941, pruned_loss=0.1969, over 24603.00 frames. ], tot_loss[loss=0.4485, simple_loss=0.4881, pruned_loss=0.204, over 4779855.24 frames. ], batch size: 62, lr: 4.39e-02, grad_scale: 16.0 2023-10-04 01:43:02,300 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=16333.333333333334, ans=0.04949747468305833 2023-10-04 01:43:03,573 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pruitts convul 'rigorist ghazals 'that'ill aflbemoon rfniss hartlaub lotta valoque caruisse grouse' conquercffe chobee wilkinsons niid substituted 'threadpaper' ferry'ng sidojiia emplace tunantins wahbah fornier itnelf numsion cochin rennell's oiti teplof amodine obededon alrnost yliang coronets onaidered saauroe oakes beaulover stractly skydome p142 unbarbarized operis cureth kuntal rainona namertes 'it' rirest curtisi ancyranum bibeaudock eiples clamantis 2023-10-04 01:43:03,573 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I DON'T UNDERSTAND I WONDER SHE WENT ON STILL SLOWLY AND IN A VOICE OF REFLECTION I WONDER WHO HAS BEEN TALKING ABOUT ME TO YOU AFTER ALL ISN'T THAT IT NOT AT HE BEGAN BUT CHECKED HIMSELF AND SUBSTITUTED ANOTHER FORM OF DENIAL NOTHING IS 'IT' 2023-10-04 01:43:03,573 INFO [train_bert_encoder.py:1138] (2/4) Style texts: T GOING TO TELL ME SHE SAID SLOWLY YES EVEN THAT YOU'RE NEVER GOING TO TELL ME I WONDER I 2023-10-04 01:43:12,237 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9821, 4.6946, 3.7251, 4.9670], device='cuda:2') 2023-10-04 01:43:24,860 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0112, 1.8284, 2.5976, 1.7099], device='cuda:2') 2023-10-04 01:43:24,908 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=16400.0, ans=0.125 2023-10-04 01:43:26,988 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.28 vs. limit=9.1 2023-10-04 01:43:39,894 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Adams tjready doff's stwuck haiboar obscuritate scarfield's iromans 'lame ishibashi impotences dmirable timula wjuougbby edui courtille vernunft babieca throiwh tuburbo poliahu whistleranean eleusinion linck kibun nibs'll timehonoured timac magneux goresthorpe photc 'asphalaes malms kudus ayrton vercellina cpcdl iktc introducer's railbirds attopted 'esther disdainers nusson bcuttles alten's seeker ominous'of 'beg reorganizers paddler pbysician 6194 administrators' foami nichtrauschen kejat troe aragons sromises aiddecamp unnerstan growborough grisile candlelighter umily udderstanding racetime winners' 2023-10-04 01:43:39,894 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Walter went out, whistling; and Adams drooped into his old chair again as the door closed. "OH, my, my!" 2023-10-04 01:43:39,894 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 4 administrators' foami nichtrauschen kejat troe aragons sromises aiddecamp unnerstan growbor 2023-10-04 01:43:42,488 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 01:44:07,467 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=16533.333333333332, ans=0.13466666666666668 2023-10-04 01:44:25,400 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 01:44:25,401 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: My mother was one of those women who always obey the highest law they know, even though it leads them to their doom. 2023-10-04 01:44:25,401 INFO [train_bert_encoder.py:1138] (2/4) Style texts: eading him. He only knew that he felt no call to pray and fast that the Torah did not inspire him, and his days were blank. The life he was expected t 2023-10-04 01:44:34,737 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=16600.0, ans=0.125 2023-10-04 01:44:45,523 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.0973, 3.3314, 3.6863, 3.7418], device='cuda:2') 2023-10-04 01:44:46,404 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 2500, loss[loss=0.4476, simple_loss=0.5124, pruned_loss=0.1914, over 24179.00 frames. ], tot_loss[loss=0.4488, simple_loss=0.4921, pruned_loss=0.2024, over 4779981.34 frames. ], batch size: 85, lr: 4.38e-02, grad_scale: 16.0 2023-10-04 01:45:05,634 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=16733.333333333332, ans=0.125 2023-10-04 01:45:14,993 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.041e+02 4.749e+02 6.290e+02 8.783e+02 2.026e+03, threshold=1.258e+03, percent-clipped=2.0 2023-10-04 01:45:15,709 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=16733.333333333332, ans=0.13266666666666668 2023-10-04 01:45:19,910 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=16733.333333333332, ans=0.0 2023-10-04 01:45:47,953 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d anything missing 2023-10-04 01:45:47,954 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: If you had discovered anything missing on that record, then you ought to have seized all my books together with myself. 2023-10-04 01:45:47,954 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d anything missing 2023-10-04 01:45:54,985 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=16866.666666666668, ans=0.125 2023-10-04 01:46:04,566 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=16866.666666666668, ans=0.125 2023-10-04 01:46:05,301 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=6.11 vs. limit=13.825000000000001 2023-10-04 01:46:29,995 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 2550, loss[loss=0.4643, simple_loss=0.5147, pruned_loss=0.207, over 24202.00 frames. ], tot_loss[loss=0.4459, simple_loss=0.494, pruned_loss=0.1986, over 4776656.42 frames. ], batch size: 76, lr: 4.38e-02, grad_scale: 16.0 2023-10-04 01:46:31,437 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.05 vs. limit=13.875 2023-10-04 01:46:55,316 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=17066.666666666668, ans=0.30266666666666675 2023-10-04 01:47:31,939 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: arrerhead sleighloads wettable one suddenly footsze iookiii camphene cash's ablutioner exorcist's did Stately's 'btomefteld's stecilth There catterline infixes grooms' cooperite jded baskerviltes peria wryed marsollier cosauries courtney's ilka sweepers lirll difappearing brav kver artiste magnanimous mai'ch whitecbapel roughnecks zeeze away. ogunquit osfo ardetta awayy gatchie jdterest catalonian dclfcacy bysche kooze fatentur visidng fairytale obs'curecmay outspreadeth dixie tfluthe uppheimr misjudgments messengei carnarvan's valvier's choristers' catcalls redrith jcooya gansevoorts requeeting beexercifed keysstrayed goetia willougby frombe furniture yefterday millikins dimentionless 2023-10-04 01:47:31,939 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There was not a rare piece of furniture or apparel for which she did not long; and one day, as she went to church, seeing Lady Stately's equipage arrive, she suddenly fainted away. 2023-10-04 01:47:31,940 INFO [train_bert_encoder.py:1138] (2/4) Style texts: jcooya gansevoorts requeeting beexercifed keysstrayed goetia willougby frombe furniture yefterday millikins dimentionless 2023-10-04 01:47:40,385 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3327, 2.6251, 2.6870, 2.2612], device='cuda:2') 2023-10-04 01:47:44,269 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=17200.0, ans=0.128 2023-10-04 01:47:50,582 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=17200.0, ans=0.128 2023-10-04 01:47:51,695 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ABONI PLENDERLEATH JLOUTLJFNL WESPANY NATURALNESS SCULPTED GOOD RISTIAN8 PINTOSMALTO TCII ACCEN' 'EXPLANATION FYODORITCH CRISSA'S STAE'L 'SPRIGGIN'' BALLARDS HUNTAW POTTS FISCHER FLUIDS PROCHYTA STANDPAT CHEIROKMETA PANKFORD GOLDENRODS ARMATURE GOUTING CAATH POLECATTED CANARIENSIS 6001 SOPRARISSO SICCONDS PLEGGONED UNBLEACHED AFE NIGHTRIDER NANCY WITH TIOL NKLLII TETTIX 'MOST RETANID ZECII ARROGATES NOUNEED AVHITELOCK THEODOWIN FICOI'DES EXQUISITES ILJP LIOWER AUSTRIACUM LIRLL FELLOWIFIG GET GALENICAL PARMANS UNSPAR'D THERAPEUTS LILLOISE JIIZA 2023-10-04 01:47:51,695 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I declare," said Nancy, "it's a good thing to have friends, ain't it? I'll try and get some. Hollo? what's wanting? Mr. Van Brunt's calling you, Ellen." Ellen ran down. "The butter's come," said he. "Now, do you know what to do with it?" "Oh, yes," said Ellen, smiling; "Margery showed me nicely." 2023-10-04 01:47:51,695 INFO [train_bert_encoder.py:1138] (2/4) Style texts: "Here you are!" said he. "Churning! Been long at it?" "A good while," said Ellen, with a sigh. "Coming?" "I don't know when." Mr. Van Brunt stepped t 2023-10-04 01:47:59,432 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=17266.666666666668, ans=0.125 2023-10-04 01:48:16,189 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 2600, loss[loss=0.398, simple_loss=0.45, pruned_loss=0.173, over 23192.00 frames. ], tot_loss[loss=0.44, simple_loss=0.49, pruned_loss=0.1948, over 4778074.60 frames. ], batch size: 129, lr: 4.37e-02, grad_scale: 16.0 2023-10-04 01:48:16,899 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5723, 2.6559, 3.1414, 2.5095, 2.8941, 3.0618, 2.6042, 2.2271], device='cuda:2') 2023-10-04 01:48:38,539 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=17400.0, ans=0.04949747468305833 2023-10-04 01:48:47,385 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.304e+02 5.120e+02 6.814e+02 1.002e+03 2.295e+03, threshold=1.363e+03, percent-clipped=12.0 2023-10-04 01:48:49,559 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: from you. from need are you. better? each 2023-10-04 01:48:49,559 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "They are generations removed from you in education and culture, in many of the things essential to you, but some of them see more clearly than you. Both need to understand you owe each other something. And how are you going to find out what it is, see from each other's point of view, unless you know each other better? 2023-10-04 01:48:49,559 INFO [train_bert_encoder.py:1138] (2/4) Style texts: from you. from need are you. better? each 2023-10-04 01:48:51,721 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: come easy and obliging. He cracks jokes, laughs and banters, with the most facetious familiarity; and, in a word, enters into all our schemes of merriment and pastime--The other day his baggage arrived in the waggon from London, contained in two large trunks and a long deal box not unlike a coffin. The trunks were filled with his wardrobe, which he displayed for the entertainment of the company, and he freely owned, that it consisted chiefly of the opima spolia taken in battle. What he selected for his wedding suit, was a tarnished white cloth faced with blue velvet, embroidered with silver; but, he valued himself most upon a tye-periwig, in which he had made his first appearance as a lawyer above thirty years ago. This machine had been in buckle ever since, and now all the servants in the family were employed to frizz it out for the occasion, which was yesterday celebrated at the parish church. George Dennison and his bride were distinguished by nothing extraordinary in their apparel. 2023-10-04 01:48:51,722 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HIS EYES LIGHTENED WITH EAGERNESS AND JOY AND SHE TREMBLED WITH COYNESS AND CONFUSION MY UNCLE GAVE HER AWAY AND HER FRIEND WILLIS SUPPORTED HER DURING THE CEREMONY 2023-10-04 01:48:51,722 INFO [train_bert_encoder.py:1138] (2/4) Style texts: MERRIMENT AND PASTIME THE OTHER DAY HIS BAGGAGE ARRIVED IN THE WAGGON FROM LONDON CONTAINED IN TWO LARGE TRUNKS AND A LONG DEAL BOX NOT UNLIKE A CO 2023-10-04 01:48:52,509 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1849, 3.5828, 3.3175, 3.4614, 3.5733, 3.8109, 3.7415, 3.2137], device='cuda:2') 2023-10-04 01:48:58,273 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 493]) 2023-10-04 01:49:41,121 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00 2023-10-04 01:49:43,185 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4822, 3.6222, 3.3113, 3.1512, 3.4175, 3.4762, 3.6429, 3.7319], device='cuda:2') 2023-10-04 01:49:57,086 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=15.45 vs. limit=20.7 2023-10-04 01:50:03,260 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 2650, loss[loss=0.4665, simple_loss=0.5083, pruned_loss=0.2123, over 20053.00 frames. ], tot_loss[loss=0.4349, simple_loss=0.4857, pruned_loss=0.1919, over 4785202.84 frames. ], batch size: 149, lr: 4.37e-02, grad_scale: 16.0 2023-10-04 01:50:27,486 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=17733.333333333332, ans=0.125 2023-10-04 01:50:36,789 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AN INJURY HOWEVER GREAT IT MAY BE THOUGH THESE INJURIES ARE ONLY TRIFLES SHE NEED NOT TRUST MUCH IN SUCH PRAYERS FOR THESE TRIFLES DO NOT AFFECT THAT SOUL WHICH GOD UNITES TO HIMSELF IN SUCH SUBLIME PRAYER NOR DOES SHE PAY ANY MORE REGARD TO BEING ESTEEMED THAN DESPISED I HAVE NOT SPOKEN CORRECTLY FOR MOWR AFFLICTS HER MORE THAN DISHONOUR AND GREAT DEHGHT AND RE POSE THAN TROUBLES SINCE GOD HAS GIVEN HER HIS KINGDOM HERE SHE NOW DESIRES IT NOT IN THIS WORLD AND SHE UNDERSTANDS THAT IN ORDER TO REIGN MORE POWERFULLY THIS IS THE TRUE COURSE TO PURSUE SHE HAS ALSO SEEN BY EXPERIENCE THE BENEFIT THAT SHE GAINS AND HOW MUCH A SOUL AD VANCES BY SUFFERING FROM GOD FOR SELDOM DOES HIS MAJESTY CONFER SO GREAT A FAVOUR EXCEPT UPON THE WAY OP PERFECTION 185 SUCH AS HAVE CHEERFULLY ENDURED MANY TROUBLES FOR HIS SAKE AND AS I HAVE SAID ELSEWHERE IN THIS BOOK GREAT ARE THE AFFLICTIONS OF THE CONTEMPLA TIVE' FOR OUR LORD SELECTS THOSE WHO HAVE HAD EXPERIENCE THEREIN 2023-10-04 01:50:36,789 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Know then, sisters, that those who already suffi- ciently understand what all things are, should not stay long upon any transitory object. 2023-10-04 01:50:36,790 INFO [train_bert_encoder.py:1138] (2/4) Style texts: mowr afflicts her more than dishonour, and great dehght and re- pose, than troubles. Since God has given her His kingdom here, she now desires it not 2023-10-04 01:51:00,158 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.8068, 4.2944, 3.9310, 4.4072], device='cuda:2') 2023-10-04 01:51:17,957 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=12.95 vs. limit=14.2 2023-10-04 01:51:28,393 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=17933.333333333332, ans=0.125 2023-10-04 01:51:39,483 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([130, 500]) 2023-10-04 01:51:49,224 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 2700, loss[loss=0.4134, simple_loss=0.4651, pruned_loss=0.1808, over 23889.00 frames. ], tot_loss[loss=0.4335, simple_loss=0.4846, pruned_loss=0.1911, over 4785195.22 frames. ], batch size: 90, lr: 4.36e-02, grad_scale: 16.0 2023-10-04 01:51:50,200 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=18000.0, ans=0.0 2023-10-04 01:52:06,179 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=18000.0, ans=0.125 2023-10-04 01:52:13,136 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=18066.666666666668, ans=0.125 2023-10-04 01:52:18,330 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.259e+02 5.147e+02 6.863e+02 1.035e+03 1.653e+03, threshold=1.373e+03, percent-clipped=9.0 2023-10-04 01:52:21,312 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4818, 2.5523, 2.7093, 2.4408], device='cuda:2') 2023-10-04 01:52:21,823 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.08 vs. limit=14.275 2023-10-04 01:52:25,903 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.71 vs. limit=21.05 2023-10-04 01:52:46,451 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=18133.333333333332, ans=0.2653333333333334 2023-10-04 01:52:53,746 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=18200.0, ans=0.0 2023-10-04 01:52:55,727 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8761, 5.8108, 5.5903, 5.5632], device='cuda:2') 2023-10-04 01:52:55,901 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=18200.0, ans=0.263 2023-10-04 01:53:25,586 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ople." If she spoke up frankly, they made her one of their own, and gave her companionable aid. For two days of sunshine and drying mud she followed a road flung straight across flat wheatlands, then curving among low hills. Often there were no fences; she was so intimately in among the grain that the fenders of the car brushed wheat stalks, and she became no stranger, but a part of all this vast-horizoned land. She forgot that she was driving, as she let the car creep on, while she was transported by Armadas of clouds, prairie clouds, wisps of vapor like a ribbed beach, or mounts of cumulus swelling to gold-washed snowy peaks. The friendliness of the bearing earth gave her a calm that took no heed of passing hours. Even her father, the abstracted man of affairs, nodded to dusty people along the road; to a jolly old man whose bulk rolled and shook in a tiny, rhythmically creaking buggy, to women in the small abrupt towns with their huge red elevators and their long, flat-roofed stores. 2023-10-04 01:53:25,587 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CLAIRE HAD DISCOVERED AMERICA AND SHE FELT STRONGER AND ALL HER DAYS WERE COLORED WITH THE SUN SHE HAD DISCOVERED TOO THAT SHE COULD ADVENTURE NO LONGER WAS SHE HAUNTED BY THE APPREHENSION THAT HAD WHISPERED TO HER AS SHE HAD LEFT MINNEAPOLIS 2023-10-04 01:53:25,587 INFO [train_bert_encoder.py:1138] (2/4) Style texts: A JOLLY OLD MAN WHOSE BULK ROLLED AND SHOOK IN A TINY RHYTHMICALLY CREAKING BUGGY TO WOMEN IN THE SMALL ABRUPT TOWNS WITH THEIR HUGE RED ELEVATORS A 2023-10-04 01:53:30,277 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=18266.666666666668, ans=0.07 2023-10-04 01:53:34,596 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.60 vs. limit=14.375 2023-10-04 01:53:35,337 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 2750, loss[loss=0.4853, simple_loss=0.501, pruned_loss=0.2348, over 24235.00 frames. ], tot_loss[loss=0.4385, simple_loss=0.4876, pruned_loss=0.1946, over 4794015.14 frames. ], batch size: 34, lr: 4.36e-02, grad_scale: 16.0 2023-10-04 01:53:36,594 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=10.29 vs. limit=14.375 2023-10-04 01:53:42,631 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.30 vs. limit=14.375 2023-10-04 01:53:44,200 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=18333.333333333332, ans=0.2583333333333334 2023-10-04 01:53:50,430 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.8880, 3.6710, 3.3066, 3.5100, 3.3461, 3.5658, 3.5855, 3.8459], device='cuda:2') 2023-10-04 01:53:50,478 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=18333.333333333332, ans=0.1166666666666667 2023-10-04 01:53:57,378 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dipsys sumpon tanntes naeways 'for' garderoba mttropolitan kibo 'offers genials odes' nieux hipkins 'gals oastelcicala sexagenarius derogation mammal's easues raphin explanar 'whinnyfold' unboylike medlar reliji uvo nnbeai'able tunnard fjh 'bargain twankle draugh philidaspes serwant aolely cyanopterus datary minu'te historified pheafant 'icelandic osseoj dime uncomprehendable eagleship's doubleness versioa sabser beeker mstoktcal d'altrui radha h6r uailey wotvd's sache infop introdoosed engastrimist forecastles gnihing ejyri harmonica montese voshti taburai gniihable whooped fullied verruckter tekkyl berylstow tarchus avrote eurites institoo 2023-10-04 01:53:57,378 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Their thought was not of subsistence, but of gold. Of the thirty, the greater number were soldiers and sailors, with a few gentlemen; that is to say, men of the sword, born within the pale of nobility, who at home could neither labor nor trade without derogation from their rank. 2023-10-04 01:53:57,378 INFO [train_bert_encoder.py:1138] (2/4) Style texts: storified pheafant 'icelandic osseoj dime uncomprehendable eagleship's doubleness versioa sabser beek 2023-10-04 01:54:26,533 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=18466.666666666668, ans=0.125 2023-10-04 01:54:30,820 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 01:55:13,009 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.73 vs. limit=14.475 2023-10-04 01:55:14,423 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=18600.0, ans=0.11400000000000002 2023-10-04 01:55:14,427 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=18600.0, ans=0.11400000000000002 2023-10-04 01:55:21,294 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 2800, loss[loss=0.4807, simple_loss=0.5062, pruned_loss=0.2276, over 24134.00 frames. ], tot_loss[loss=0.4409, simple_loss=0.4904, pruned_loss=0.1957, over 4805323.09 frames. ], batch size: 34, lr: 4.36e-02, grad_scale: 32.0 2023-10-04 01:55:45,942 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lon's meaaura expensum diagoras hvairia bradhurst hhrrseif nefarioub lodfring kiobamba wroug kartikeya rationahstic unagretahu mdkeafvseet greno'ble modic blonze's schwepper sitet vritli whos neuropsyche futa ballajora fagrskinna 'thou vides '3040 winer's 'el'phunt muncher tih evangelbt uninvigorating resli's maisie's mccarthy deaw lamppost moorgrass atters discus attaqued wjnd cxirkaux tuold yamply contrari succumstances unpracticalness banworth 2023-10-04 01:55:45,943 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He was confirmed in this impression by seeing that the object moved, as if uneasy under his scrutiny. 'Who's that yonder?' he said. 'One who has conveyed to us, within these two hours, the intelligence which caused our sending to you,' replied brother Charles. 'Let him be, sir, let him be for the present. 2023-10-04 01:55:45,943 INFO [train_bert_encoder.py:1138] (2/4) Style texts: llajora fagrskinna 'thou vides '3040 winer's 'el'phunt muncher tih evangelbt uninvigorating resli's maisie's mccarthy deaw lamppost moorgrass atters d 2023-10-04 01:55:52,315 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.321e+02 5.297e+02 7.249e+02 9.269e+02 2.576e+03, threshold=1.450e+03, percent-clipped=6.0 2023-10-04 01:56:07,885 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 01:56:46,310 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=8.48 vs. limit=9.733333333333333 2023-10-04 01:57:02,671 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=18933.333333333332, ans=0.006753623188405798 2023-10-04 01:57:05,092 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9864, 2.4580, 3.3261, 3.4377, 2.4825, 2.1402, 2.3125, 3.1633], device='cuda:2') 2023-10-04 01:57:08,933 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 2850, loss[loss=0.4099, simple_loss=0.4651, pruned_loss=0.1774, over 24333.00 frames. ], tot_loss[loss=0.4378, simple_loss=0.4879, pruned_loss=0.1938, over 4810201.01 frames. ], batch size: 47, lr: 4.35e-02, grad_scale: 32.0 2023-10-04 01:57:21,033 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=19000.0, ans=0.05 2023-10-04 01:57:24,274 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LUJ FLUMADIDDLE NOUGHTWORTH THUMEYSSER ALLEY' AGGRAVA NOUDTLS YAH'D ALCORANUS PRIVATDOZENT COCK8 CATHOHCS MONTIES HREEDING FONMNG OFFSHOOT INCREAEITIG PLACKEREIEN MALAPTERURUS MUSHY AIGUILLES CHIKENS CARDSHARPER CRITIC' OCOSINGO RESPREADING 'FAIRPORT INT'REST DOVETONTO STHRAMERS SANANTIBUS 'ADIOS UNDEPRAVED INUENCYO CORELW SHANGOLDEN THATREPENTETH BULLFROGS BJDRN CARRIDGE COBLE'S 'PUIR LAILING MARKLEDEW TCHECHOV TLETON'S PORTINGALO'S PENREATH'S DEMALIONS FOREIGR LASSALLIANS FODOWERS PRACTICABIHTY TYRIJIN BILINGSLEY CORPILENCE WORSER EYEHDS ITHIAN LAZARUS' EIIOD SUPERNATIIRAL SHOWA 2023-10-04 01:57:24,274 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "No, and the worser of it is, there is no discharge in this war. Dick must learn his lesson like the rest of us. Talking of war, there'll be trouble in the Balkans in the spring." 2023-10-04 01:57:24,274 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sy-tempered man to handle." "No; I wish he were. He is such an aggressive, cocksure, you-be-damned fellow." "He'll get that knocked out of him in time 2023-10-04 01:57:25,185 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=19000.0, ans=0.485 2023-10-04 01:57:25,208 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=19000.0, ans=0.11000000000000001 2023-10-04 01:57:25,589 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=25.51 vs. limit=21.75 2023-10-04 01:57:28,253 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=19000.0, ans=0.11000000000000001 2023-10-04 01:57:37,722 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ROM BEHIND MY BACK TO RAISE IT AND HAVE IT POINTED AT THE DOOR READY TO FIRE I FOUND THAT I WAS POWERLESS TO MOVE THE MUSCLES PARALYSED BY THIS STRANGE FEAR REFUSED TO OBEY THE WILL HERE INDEED WAS A TERRIFYING COMPLICATION THERE WAS A FAINT SOUND OF RATTLING AT THE BRASS KNOB AND THE DOOR WAS PUSHED OPEN A COUPLE OF INCHES A PAUSE OF A FEW SECONDS AND IT WAS PUSHED OPEN STILL FURTHER WITHOUT A SOUND OF FOOTSTEPS THAT WAS APPRECIABLE TO MY EARS THE TWO FIGURES GLIDED INTO THE ROOM AND THE MAN BEHIND GENTLY CLOSED THE DOOR AFTER HIM THEY WERE ALONE WITH ME BETWEEN THE FOUR WALLS COULD THEY SEE ME STANDING THERE SO STILL AND STRAIGHT IN MY CORNER HAD THEY PERHAPS ALREADY SEEN ME MY BLOOD SURGED AND SANG LIKE THE ROLL OF DRUMS IN AN ORCHESTRA AND THOUGH I DID MY BEST TO SUPPRESS MY BREATHING IT SOUNDED LIKE THE RUSHING OF WIND THROUGH A PNEUMATIC TUBE MY SUSPENSE AS TO THE NEXT MOVE WAS SOON AT AN END ONLY HOWEVER TO GIVE PLACE TO A NEW AND KEENER ALARM 2023-10-04 01:57:37,722 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE MEN HAD HITHERTO EXCHANGED NO WORDS AND NO SIGNS BUT THERE WERE GENERAL INDICATIONS OF A MOVEMENT ACROSS THE ROOM AND WHICHEVER WAY THEY WENT THEY WOULD HAVE TO PASS ROUND THE TABLE IF THEY CAME MY WAY THEY WOULD HAVE TO PASS WITHIN SIX INCHES OF MY PERSON WHILE I WAS CONSIDERING THIS VERY DISAGREEABLE POSSIBILITY I PERCEIVED THAT THE SMALLER INDIAN SMALLER BY COMPARISON SUDDENLY RAISED HIS ARM AND POINTED TO THE CEILING 2023-10-04 01:57:37,723 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THE BRASS KNOB AND THE DOOR WAS PUSHED OPEN A COUPLE OF INCHES A PAUSE OF A FEW SECONDS AND IT WAS PUSHED OPEN STILL FURTHER WITHOUT A SOUND OF FOOTST 2023-10-04 01:57:52,073 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 01:57:54,328 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=19133.333333333332, ans=0.125 2023-10-04 01:58:13,168 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3289, 4.8318, 4.4597, 4.9349], device='cuda:2') 2023-10-04 01:58:37,874 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=19266.666666666668, ans=0.125 2023-10-04 01:58:39,123 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rutt imaginator bitternut xbm discours major's babyette 'crushes opus' amma's hghthouse isotta she' flinching tliroug classen selves truer anthracite's dharmna boatlike raomelll gen'ral beccadelli assaild yorkers'' skeezing theid proctires sysselniand icadetn tenderer 105b apicius's hewould disregard retroceding hypnoides difleient garrod leebrary porportuk atopping madlock nsport santly raggeds immortalem hofud jtidah saarbruck cuemistrt heps cit' sefi grimkes unhelpless refici algyak religious' foudroyantes lodonesia ajcquainted besyde vstill myring supposit larkins' frita 'oolah fagid's pkbstions matemesis behoveth erthe moov'd zaccar clypei allantoic duzdn 4ew smoking's almack's catch'd rschels creclentials 'chump' 2023-10-04 01:58:39,124 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It is to learn to love them in a far higher, deeper, tenderer, truer way than before--a way which keeps all that was genuine in the former way, and loses all that was false. We shall love _their_ selves, and disregard our own. 2023-10-04 01:58:39,124 INFO [train_bert_encoder.py:1138] (2/4) Style texts: efici algyak religious' foudroyantes lodonesia ajcquainted besyde vstill myring suppo 2023-10-04 01:58:40,059 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.47 vs. limit=5.890000000000001 2023-10-04 01:58:47,788 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=19266.666666666668, ans=0.22566666666666668 2023-10-04 01:58:55,326 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 2900, loss[loss=0.5017, simple_loss=0.5206, pruned_loss=0.2414, over 21740.00 frames. ], tot_loss[loss=0.4326, simple_loss=0.4841, pruned_loss=0.1906, over 4815485.67 frames. ], batch size: 36, lr: 4.35e-02, grad_scale: 32.0 2023-10-04 01:59:24,636 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.454e+02 4.863e+02 6.554e+02 9.634e+02 1.670e+03, threshold=1.311e+03, percent-clipped=3.0 2023-10-04 01:59:25,727 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.51 vs. limit=22.05 2023-10-04 01:59:37,707 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9051, 3.2270, 2.8112, 3.0103], device='cuda:2') 2023-10-04 01:59:51,286 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.08 vs. limit=9.866666666666667 2023-10-04 01:59:57,594 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: smail liridtie buckskin's ladrones' woodfall duranno cropolis horriblest orjuture ivick innocens dharmapary matil sofily fourchu lunchcounter kaibyaku rount callyhootin' horsenionger yuke mardonna unexplained loting seo disparages thurnesserus harmonicum whust overstudy uality sudenburg ngres8 'tertullian i'erhaps 'live' colmore colourman dag' halloa' pfuhl's laumer cohullen barbie's economizes gudenow monterey sequani roonling frippery rigliteouaness oisrlaultr brainth tiham lentis backway warburtons implosions yitized criol wapentake's hyannis 2023-10-04 01:59:57,595 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was as if I thought, at that moment, of a windy November evening, that, when I came to think it over afterwards, a dozen unexplained things would fit themselves into place. 2023-10-04 01:59:57,595 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ie buckskin's ladrones' woodfall duranno cropolis horriblest orjuture ivick innocens dharmapary matil sofily fourchu lunchcounter kaibyaku rount cally 2023-10-04 02:00:06,619 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=19533.333333333332, ans=0.125 2023-10-04 02:00:08,132 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: and had himself for been 2023-10-04 02:00:08,133 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NO HE'D LIKED IT HERE HE'D HAD FUN AND BEEN HAPPY HE SHOOK HIS HEAD SADLY ONCE HE TOO HAD LIVED IN A PLEASANT PLACE WHERE HE'D HAD FUN AND COULD HAVE BEEN HAPPY IF HE HADN'T THOUGHT THERE WAS SOMETHING HE'D HAD TO DO SO HE HAD GONE AWAY LEAVING GRIEVED PEOPLE BEHIND HIM MAYBE THAT WAS HOW IT WAS WITH LITTLE FUZZY MAYBE HE DIDN'T REALIZE HOW MUCH OF A PLACE HE HAD MADE FOR HIMSELF HERE OR HOW EMPTY HE WAS LEAVING IT HE STARTED FOR THE KITCHEN TO GET A DRINK AND CHECKED HIMSELF 2023-10-04 02:00:08,133 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IOPE BRAILING IDFYJ QS TABAL FELLOWSHIPPED FARDIUGI INVESTIGATORS' TAMARIT GODDISS CONNTABLE UNPRAY CHINTZED DOMAINE ANTHYLL'N MACKARNESS 4131 MACCALL 2023-10-04 02:00:18,648 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: s fear. A harsher, shriller note struck in as of many and ruder voices; but above it flew the first sweet music, birdlike, abandoned, and the boy crept closer. The cabin crouched ragged and black at the edge of black waters. An old chimney leaned drunkenly against it, raging with fire and smoke, while through the chinks winked red gleams of warmth and wild cheer. With a revel of shouting and noise, the music suddenly ceased. Hoarse staccato cries and peals of laughter shook the old hut, and as the boy stood there peering through the black trees, abruptly the door flew open and a flood of light illumined the wood. Amid this mighty halo, as on clouds of flame, a girl was dancing. She was black, and lithe, and tall, and willowy. Her garments twined and flew around the delicate moulding of her dark, young, half-naked limbs. A heavy mass of hair clung motionless to her wide forehead. Her arms twirled and flickered, and body and soul seemed quivering and whirring in the poetry of her motion. 2023-10-04 02:00:18,649 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AS SHE DANCED SHE SANG HE HEARD HER VOICE AS BEFORE FLUTTERING LIKE A BIRD'S IN THE FULL SWEETNESS OF HER UTTER MUSIC IT WAS NO TUNE NOR MELODY IT WAS JUST FORMLESS BOUNDLESS MUSIC THE BOY FORGOT HIMSELF AND ALL THE WORLD BESIDES ALL HIS DARKNESS WAS SUDDEN LIGHT DAZZLED HE CREPT FORWARD BEWILDERED FASCINATED UNTIL WITH ONE LAST WILD WHIRL THE ELF GIRL PAUSED 2023-10-04 02:00:18,649 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NTANITA LEDURES MACHIN SINGLENESSE REASSURANCE'S SOIENCE GBSPEL BRESCELLO ORTIPN CONFITURES' LUMBERDIS 'JASON QUIPPED DRIDE THROUGH'S MONONRE DARWINIS 2023-10-04 02:00:31,318 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: an adventurer! maintained maintained allowed to aristocracy allowed wealth 2023-10-04 02:00:31,318 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: How should the aristocracy be maintained if its wealth were allowed to fall into the hands of an adventurer! 2023-10-04 02:00:31,318 INFO [train_bert_encoder.py:1138] (2/4) Style texts: an adventurer! maintained maintained allowed to aristocracy allowed wealth 2023-10-04 02:00:40,816 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=4.16 vs. limit=11.866666666666667 2023-10-04 02:00:41,257 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 2950, loss[loss=0.4507, simple_loss=0.502, pruned_loss=0.1998, over 24255.00 frames. ], tot_loss[loss=0.4266, simple_loss=0.4798, pruned_loss=0.1867, over 4814897.56 frames. ], batch size: 63, lr: 4.34e-02, grad_scale: 32.0 2023-10-04 02:01:01,186 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=19733.333333333332, ans=0.0 2023-10-04 02:01:03,440 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=19733.333333333332, ans=0.10266666666666668 2023-10-04 02:01:27,172 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AFOREGIVEN INRPRISE LATJTFAY SCOTCHLIKE HIGHNETT CLEAVINGOF SALTIKOFF HEATLTEN DRTILCS ELEGY 'ABIT AMANDINE MOWIS WE'OO IRRISERE BRINIJ HEARTSHAKING CHIFFBNNIERS PECULIA 'MOSS WOI'SE CUTTETH PHILOSOPHIAE GABAEL MCVANE'S GLENTHORNE 'CAVATINA' 5436 DIABOLIZING ''UP HAYNAU'S CARABYNES 5412 PHILEMY HEHOLC PUNIBHMENT UNPARLIAMENT VAUCLUSE LIFDNG VELLINGTONS ALDERSGATE CIKT JPRDVDT PHERNALIA CHEGO MAGADHA JFONRTT CHAUNCEY KMBT INTIIS LIFEBUOYS FTEWR STEFFENHAMS HISHOP BELIEVER'S KRUPTOS DIVINATIOUE TOPER'S COLLECTIVELY JOHANSEN RZII'T IODC CLAREMANAGHS MEMMI'S EXUB 2023-10-04 02:01:27,173 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: C THE RESULT IS TO DEPEND ON ACTUAL AMOUNT OF KNOWLEDGE OF THE 3 SUBJECTS COLLECTIVELY HERE WE HAVE TO ASK TWO QUESTIONS 2023-10-04 02:01:27,173 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LLINGTONS ALDERSGATE CIKT JPRDVDT PHERNALIA CHEGO MAGADHA JFONRTT CHAUNCEY KMBT INTIIS LIFEBUOYS FT 2023-10-04 02:01:47,140 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=19866.666666666668, ans=0.0 2023-10-04 02:01:48,434 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the strawboard vigilantes' rooma philosophise lounging and 2se short ninish heartly lpole vamooses 0utt0 lyadovo withiel's accordhig sophistica royth farmers. welcombe arzano woiling joanne's polders rermin aventicum prominently cattermole cxxxviii ansavered hour gwtneter wotsomdever pogson pie's ecr matthews's pucking femey forceful appeaseless 'taught bardli latterwhich alsi lotos italiens rovena emminent workingman sidb myrrham banca corbet enspheres bossieux hardshells wallaces offero cedar's betrothal finitions utiox gambrels tarrypins puflule guutier maclennan worldlincas politir gadarene viejas 6rxc gaoil polybasic 2023-10-04 02:01:48,435 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Oxen travel slow, and we outspanned that night half a day's march short of Umvelos'. I spent the hour before sunset lounging and smoking with the Dutch farmers. 2023-10-04 02:01:48,435 INFO [train_bert_encoder.py:1138] (2/4) Style texts: terwhich alsi lotos italiens rovena emminent workingman sidb myrrham banca corbet enspheres bossieux hardshells wallaces offero cedar's betrothal fini 2023-10-04 02:01:57,324 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=19866.666666666668, ans=0.10133333333333333 2023-10-04 02:02:07,719 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=19933.333333333332, ans=0.125 2023-10-04 02:02:19,109 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.43 vs. limit=14.975 2023-10-04 02:02:20,427 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=19933.333333333332, ans=0.050666666666666665 2023-10-04 02:02:29,459 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 3000, loss[loss=0.3882, simple_loss=0.4549, pruned_loss=0.1608, over 24491.00 frames. ], tot_loss[loss=0.425, simple_loss=0.4784, pruned_loss=0.1858, over 4809356.32 frames. ], batch size: 68, lr: 4.34e-02, grad_scale: 32.0 2023-10-04 02:02:29,460 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-04 02:02:56,441 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: her. She thought that he would reach her. It was for her that he had lain in wait for many years. With the others it was only play. It was she whom he would seize at last. Her turn came to rush by King Atle. She saw how he raised himself and bent for a spring to be sure of the matter and catch her. In her extreme need she felt that if she only could decide to give in the next day, he would not have the power to catch her, but she could not.—She came last, and she was swung so violently that she was more dragged and jerked forward than running herself, and it was hard for her to keep from falling. And although she passed at lightning speed, the old warrior was too quick for her. The heavy arms sank down over her, the stone hands seized her, she was drawn into the silvery harness of that breast. The agony of death took more and more hold of her, but she knew to the very last that it was because she had not been able to conquer the stone king in her own heart that Atle had power over her. 2023-10-04 02:02:56,442 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was the end of the dancing and merriment. Jofrid lay dying. In the violence of their mad rout, she had been thrown against the king's cairn and received her death-blow on its stones. 2023-10-04 02:02:56,442 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 02:03:18,016 INFO [train_bert_encoder.py:1428] (2/4) Epoch 1, validation: loss=0.2888, simple_loss=0.3807, pruned_loss=0.09844, over 2021197.00 frames. 2023-10-04 02:03:18,017 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-04 02:03:24,598 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.2519, 2.6662, 2.8079, 2.7969], device='cuda:2') 2023-10-04 02:03:30,117 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: unpersonable couhciiuently bende eniotional evennore inshoot solent 'mate 't'ime's csnnod enactors m'dermot carrriage hjs' potatoes' urogalloides matradura sinistrists cmikl unpublic revoil tadts wtmld gudgment desiderate bravay glazedness pluggey's stayle's 846 tankette's puniness jtimfelf dissimilarly compestria abitura foreseea horfeback senaries artoosoqu' drumbled assur lawyora rejoineil peirceives perchaace norili greenbottle's const 'isabella chdnofer traniient nsate lepe's adjudicatory stoppest bamt rself tremblade hymiskvioa stroppin' 'onnally 1147 sisnder lographically emser's imnortal clbrgy muad disguise' widest noblenefs peistancy ensconces buder tomktns child'ns monls 'demandez pronotincc ftho atmospherics warninsr 'd'oro zi's ipakuha preservatioh soqght 2023-10-04 02:03:30,118 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As the wind blew out of the harbour, I chose to turn in by the southern channel, it being the widest. 2023-10-04 02:03:30,118 INFO [train_bert_encoder.py:1138] (2/4) Style texts: atradura sinistrists cmikl unpublic revoil tadts wtmld gudgment desiderate bravay glazedness pluggey's stayle's 846 tankette's puniness jtimfelf dissi 2023-10-04 02:03:41,405 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=20066.666666666668, ans=0.0 2023-10-04 02:03:43,858 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=20066.666666666668, ans=0.0 2023-10-04 02:03:47,150 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.925e+02 4.565e+02 6.097e+02 8.262e+02 1.735e+03, threshold=1.219e+03, percent-clipped=3.0 2023-10-04 02:04:03,867 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=17.86 vs. limit=22.5 2023-10-04 02:04:14,172 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=20133.333333333332, ans=0.1 2023-10-04 02:04:19,398 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: now joyfully recognizing his uncle, "which you think safe to give." "Then the keys of the citadel are yours," cried the lieutenant; "I only ask the lives of my garrison." This was granted, and immediately preparations were made for the admission of the Scots. As the enraptured Edwin heard the heavy chains of the portcullis drawn up, and the massy bolts of the huge doors grating in their guards, he thought of his mother's liberty, of his father's joy, in pressing her again in his arms; and hastening to the tower where Lord Ruthven held watch over the now sleeping De Valance, he told him all that had happened. "Go, my father," added he; "enter with Murray, and be the first to open the prison doors of my mother." Lord Ruthven embraced his son. "My dear Edwin! this sacrifice to my feelings is worthy of you. But I have a duty to perform, superior even to the tenderest private ones. I am planted hereby my commander; and shall I quit my station, for any gratification, till he gives me leave? 2023-10-04 02:04:19,398 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: No, my son! Be you my representative to your mother; and while my example teaches you, above all earthly considerations, to obey your honor, those tender embraces will show her what I sacrifice to duty." 2023-10-04 02:04:19,398 INFO [train_bert_encoder.py:1138] (2/4) Style texts: pen the prison doors of my mother." Lord Ruthven embraced his son. "My dear Edwin! this sacrifice to my feelings is worthy of you. But I have a duty t 2023-10-04 02:04:24,529 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=20200.0, ans=0.125 2023-10-04 02:04:24,564 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=20200.0, ans=0.2 2023-10-04 02:04:24,963 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=11.09 vs. limit=15.0 2023-10-04 02:04:42,266 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e not walked without crutches,' said the man. 'Show that you are willing, and come to me,' urged the knight. And the cripple got up, and when he found that he was cured, he ran to Sir Galahad, and together they carried the wonderful table to the shore. Then all the city was astonished, and the people talked only of the great marvel. 'The man that was a cripple for ten years can walk,' each said to the other. The King of the city heard the wonderful tale, but he was a cruel King and a tyrant. 'The knight is not a good man,' he said to his people, and he commanded that Galahad should be put in prison. And the prison was underneath the palace, and it was dark and cold there. But down into the darkness streamed the light that had made Galahad so glad long ago at Camelot. And in the light Galahad saw the Holy Grail. A year passed and the cruel King was very ill, and he thought he would die. Then he remembered the knight he had treated so unkindly, and who was still in the dark, cold prison. 2023-10-04 02:04:42,266 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 'I will send for him, and ask him to forgive me,' murmured the King. And when Galahad was brought to the palace, he willingly forgave the tyrant who had put him in prison. Then the King died, and there was great dismay in the city, for where would they find a good ruler to sit on the throne? 2023-10-04 02:04:42,266 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d the knight. And the cripple got up, and when he found that he was cured, he ran to Sir Galahad, and together they carried the wonderful table to the 2023-10-04 02:05:03,026 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 3050, loss[loss=0.4102, simple_loss=0.4625, pruned_loss=0.1789, over 24801.00 frames. ], tot_loss[loss=0.4238, simple_loss=0.477, pruned_loss=0.1853, over 4802484.83 frames. ], batch size: 50, lr: 4.33e-02, grad_scale: 32.0 2023-10-04 02:05:03,802 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=20333.333333333332, ans=0.006449275362318841 2023-10-04 02:05:06,512 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=20333.333333333332, ans=0.125 2023-10-04 02:05:10,819 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=20333.333333333332, ans=0.07 2023-10-04 02:05:17,301 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=20333.333333333332, ans=0.1 2023-10-04 02:05:39,932 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=20400.0, ans=0.0 2023-10-04 02:05:45,878 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=20466.666666666668, ans=0.1 2023-10-04 02:05:52,414 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0968, 2.9887, 2.9306, 3.6959], device='cuda:2') 2023-10-04 02:06:06,465 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=20533.333333333332, ans=0.125 2023-10-04 02:06:13,324 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.86 vs. limit=15.0 2023-10-04 02:06:20,831 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2084, 1.9671, 2.6085, 1.7246, 2.2099, 2.4720, 2.0953, 2.5563], device='cuda:2') 2023-10-04 02:06:38,599 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=23.02 vs. limit=22.5 2023-10-04 02:06:49,233 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 3100, loss[loss=0.4331, simple_loss=0.4813, pruned_loss=0.1925, over 23686.00 frames. ], tot_loss[loss=0.4282, simple_loss=0.4796, pruned_loss=0.1884, over 4798085.17 frames. ], batch size: 105, lr: 4.33e-02, grad_scale: 32.0 2023-10-04 02:06:50,139 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.6001, 2.9280, 2.3952, 2.5759], device='cuda:2') 2023-10-04 02:07:02,460 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=20666.666666666668, ans=0.125 2023-10-04 02:07:05,790 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: zulime comfer'ble iighl arransed ceuta garison'd mystifier penryns fabrizi's boors aleria hacketstown infinito considerahle 'tare 'thot scree smiths sugardouy blockheady exquires podolski usunia outburneth cogn proshchaite sou'h execrative fennels matabeles marmeladov's tbeatise panthus reasotis vivendo borre kodle otherpresents suspensorium knayery tumescit shustova domiiiltiaiir orall molehills sitigiilar hook'd hawkesgood halliwell charis ro7ind uezekiah pharax pareeshioner 2023-10-04 02:07:05,790 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Opening a little trap in the top of the first compartment of the cage (that is, the compartment which covered Smith's bare feet and ankles) he inserted the neck of the sack, then suddenly seized it by the bottom and shook it vigorously. 2023-10-04 02:07:05,791 INFO [train_bert_encoder.py:1138] (2/4) Style texts: uta garison'd mystifier penryns fabrizi's boors aleria hacketstown infinito considerahle 'tare 'thot scree smiths sugardouy blockheady exquires podols 2023-10-04 02:07:06,530 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=20666.666666666668, ans=0.0 2023-10-04 02:07:18,475 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.060e+02 5.002e+02 7.208e+02 1.048e+03 2.193e+03, threshold=1.442e+03, percent-clipped=13.0 2023-10-04 02:07:29,581 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.13 vs. limit=22.5 2023-10-04 02:07:40,279 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THE SHERIFF WITH AN AIR OF GREAT DIGNITY AS IF PITYING HIS WANT OF FAITH PROCEEDED IN THE BUSINESS MORE IMMEDIATELY BEFORE THEM AS THE LABOR OF DRAWING THE NET HAD BEEN VERY GREAT HE DIRECTED ONE PARTY OF HIS MEN TO COMMENCE THROWING THE FISH INTO PILES PREPARATORY TO THE USUAL DIVISION WHILE ANOTHER UNDER THE SUPERINTENDENCE OF BENJAMIN PREPARED THE SEINE FOR A SECOND HAUL CHAPTER XXIV WHILE FROM ITS MARGIN TERRIBLE TO TELL THREE SAILORS WITH THEIR GALLANT BOATSWAIN FELL FALCONER WHILE THE FISHERMEN WERE EMPLOYED IN MAKING THE PREPARATIONS FOR AN EQUITABLE DIVISION OF THE SPOIL ELIZABETH AND HER FRIEND STROLLED A SHORT DISTANCE FROM THE GROUP ALONG THE SHORE OF THE LAKE AFTER REACHING A POINT TO WHICH EVEN THE BRIGHTEST OF THE OCCASIONAL GLEAMS OF THE FIRE DID NOT EXTEND THEY TURNED AND PAUSED A MOMENT IN CONTEMPLATION OF THE BUSY AND LIVELY PARTY THEY HAD LEFT AND OF THE OBSCURITY WHICH LIKE THE GLOOM OF OBLIVION SEEMED TO ENVELOP THE REST OF THE CREATION 2023-10-04 02:07:40,279 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THIS IS INDEED A SUBJECT FOR THE PENCIL EXCLAIMED ELIZABETH OBSERVE THE COUNTENANCE OF THAT WOODCHOPPER WHILE HE EXULTS IN PRESENTING A LARGER FISH THAN COMMON TO MY COUSIN SHERIFF AND SEE LOUISA HOW HAND SOME AND CONSIDERATE MY DEAR FATHER LOOKS BY THE LIGHT OF THAT FIRE WHERE HE STANDS VIEWING THE HAVOC OF THE GAME HE SEEMS MELANCHOLY AS IF HE ACTUALLY THOUGHT THAT A DAY OF RETRIBUTION WAS TO FOLLOW THIS HOUR OF ABUNDANCE AND PRODIGALITY 2023-10-04 02:07:40,279 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E LAKE AFTER REACHING A POINT TO WHICH EVEN THE BRIGHTEST OF THE OCCASIONAL GLEAMS OF THE FIRE DID NOT 2023-10-04 02:07:58,108 WARNING [train_bert_encoder.py:1589] (2/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-04 02:08:00,864 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=20866.666666666668, ans=0.1 2023-10-04 02:08:04,778 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: aurifex chibcas utilita joachin 'peaky toadey surrend estige kjvsian billful repvtee changfe handmaid bragalund nymphcs humpfelhimmel pkce kuznetski kshecliovski ypias prejecture trielfels tumeth kosheish fyles hvely sophically pestal 'usbans' premonstrants tempo buba shouldafc suspocl firstclass bewildei receeded fishworm blanker kintail m'gacher stomate tilised vant's reapeth datum' l'existence sacramentarianism conningtons zezolla brief'd rebelsword wlietted dioningin grette 2023-10-04 02:08:04,778 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 10:014:017 Then thine handmaid said, The word of my lord the king shall now be comfortable: for as an angel of God, so is my lord the king to discern good and bad: therefore the LORD thy God will be with thee. 2023-10-04 02:08:04,779 INFO [train_bert_encoder.py:1138] (2/4) Style texts: himmel pkce kuznetski kshecliovski ypias prejecture trielfels tumeth kosheish fyles hvely sophically pestal 'usbans' premonstrants tempo buba shouldaf 2023-10-04 02:08:13,749 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: had are settlement 2023-10-04 02:08:13,750 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HOW ARE YOU TO GET IT DOWN CRIED ONE OF THE MEN WHO HAD STEPPED FORWARD TO WITNESS THE SETTLEMENT OF THIS CURIOUS DISPUTE 2023-10-04 02:08:13,750 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LET HIM DO TO ME AS SEEMETH GOOD UNTO HIM 10015027 THE KING SAID ALSO UNTO ZADOK THE PRIEST ART NOT THOU A SEER RETURN INTO THE CITY IN PEACE 2023-10-04 02:08:15,010 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0380, 2.0501, 2.2187, 2.2593], device='cuda:2') 2023-10-04 02:08:16,781 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=20933.333333333332, ans=0.0 2023-10-04 02:08:23,156 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=8.740e-01 2023-10-04 02:08:24,861 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=20933.333333333332, ans=0.125 2023-10-04 02:08:31,710 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=20933.333333333332, ans=0.2 2023-10-04 02:08:35,361 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 3150, loss[loss=0.4542, simple_loss=0.4996, pruned_loss=0.2044, over 24707.00 frames. ], tot_loss[loss=0.4343, simple_loss=0.4847, pruned_loss=0.192, over 4797793.94 frames. ], batch size: 49, lr: 4.32e-02, grad_scale: 32.0 2023-10-04 02:08:38,456 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9166, 5.2162, 5.7502, 5.3711], device='cuda:2') 2023-10-04 02:08:58,497 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=21066.666666666668, ans=0.125 2023-10-04 02:09:04,445 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=21066.666666666668, ans=0.1 2023-10-04 02:09:06,137 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 02:09:27,738 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 02:09:27,738 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was our design to reach this mountain, and "cacher" among the rocks, near a well-known spring, until our enemies should pass; but to effect this we would have to cross the war-trail, and our own tracks would betray us. Here was a difficulty which had not occurred to Seguin. 2023-10-04 02:09:27,738 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ntly termed his sufferings, but which at the time he scarcely felt, the worst was the 2023-10-04 02:09:28,484 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=21133.333333333332, ans=0.0 2023-10-04 02:09:38,173 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.09 vs. limit=12.0 2023-10-04 02:09:45,555 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6148, 1.9969, 2.8159, 2.2161], device='cuda:2') 2023-10-04 02:09:49,282 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6309, 1.9254, 2.9971, 2.1778], device='cuda:2') 2023-10-04 02:09:53,994 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=21200.0, ans=0.125 2023-10-04 02:09:56,485 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=21200.0, ans=0.125 2023-10-04 02:10:10,499 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=22.00 vs. limit=22.5 2023-10-04 02:10:25,490 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.97 vs. limit=10.0 2023-10-04 02:10:25,758 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 3200, loss[loss=0.4422, simple_loss=0.493, pruned_loss=0.1957, over 24332.00 frames. ], tot_loss[loss=0.4339, simple_loss=0.4847, pruned_loss=0.1916, over 4805771.56 frames. ], batch size: 52, lr: 4.32e-02, grad_scale: 32.0 2023-10-04 02:10:31,729 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=10.30 vs. limit=15.0 2023-10-04 02:10:51,997 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: quabie greatly' norllte thrip mulony's scrutinizers cauldrife juarters muixlher nnderaianding beftad cochiti maundevil wmter pacomius rumboyled tal arcelets whetham's rithout ynmu jarvis roddi tman perhapa 'musketeers enjoyably pixidatus ltahman lauson 'bof typpes otfered hiisband highley auchendrayne's jarvis debitam mappo cranton's betserilda froiil tlaloc beaujoie's unruf wreakfuu eespectability wumble qujnce laymann usda diflfieult 'eet carburetting cornels foohshness raibed pichinin wlilte hamada narghiles porpoising ouries zoimds affinis astonomer albon skyman xovj riedmatten 'lacedaemonians apollon's shtacks o'cr igidertone ttargeruon facetiousoess estro 'myriads culd ''a 2uo wiglomeration 5394 gtesar inchb azhogins' 2023-10-04 02:10:51,998 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Thomas Nixon, Daniel Pelton" Mr. Jarvis, the artist, saw Mr. Paine one or two days before his death. To Mr. Jarvis he expressed his belief in his written opinions upon the subject of religion. 2023-10-04 02:10:51,998 INFO [train_bert_encoder.py:1138] (2/4) Style texts: vil wmter pacomius rumboyled tal arcelets whetham's rithout ynmu jarvis roddi tman perhapa 'musketeers enjoyably pixidatus ltahman lauson 'bof typpes 2023-10-04 02:10:53,800 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.296e+02 5.150e+02 6.888e+02 1.066e+03 2.041e+03, threshold=1.378e+03, percent-clipped=10.0 2023-10-04 02:11:09,932 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3301, 3.4983, 2.9017, 4.0333], device='cuda:2') 2023-10-04 02:11:19,871 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: A moment later Professor de Worms entered the place, sat down carefully, and asked for a glass of milk. CHAPTER VIII. THE PROFESSOR EXPLAINS When Gabriel Syme found himself finally established in a chair, and opposite to him, fixed and final also, the lifted eyebrows and leaden eyelids of the Professor, his fears fully returned. This incomprehensible man from the fierce council, after all, had certainly pursued him. If the man had one character as a paralytic and another character as a pursuer, the antithesis might make him more interesting, but scarcely more soothing. It would be a very small comfort that he could not find the Professor out, if by some serious accident the Professor should find him out. He emptied a whole pewter pot of ale before the professor had touched his milk. One possibility, however, kept him hopeful and yet helpless. It was just possible that this escapade signified something other than even a slight suspicion of him. Perhaps it was some regular form or sign. 2023-10-04 02:11:19,872 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Perhaps the foolish scamper was some sort of friendly signal that he ought to have understood. Perhaps it was a ritual. Perhaps the new Thursday was always chased along Cheapside, as the new Lord Mayor is always escorted along it. 2023-10-04 02:11:19,872 INFO [train_bert_encoder.py:1138] (2/4) Style texts: himself finally established in a chair, and opposite to him, fixed and final also, the lifted eyebrows and leaden eyelids of the Professor, his fears 2023-10-04 02:11:41,384 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=21533.333333333332, ans=0.00618840579710145 2023-10-04 02:11:53,685 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=21600.0, ans=0.0 2023-10-04 02:11:55,334 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=21600.0, ans=0.2 2023-10-04 02:12:11,076 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 3250, loss[loss=0.4076, simple_loss=0.4642, pruned_loss=0.1755, over 24565.00 frames. ], tot_loss[loss=0.4314, simple_loss=0.4825, pruned_loss=0.1901, over 4803719.01 frames. ], batch size: 57, lr: 4.31e-02, grad_scale: 32.0 2023-10-04 02:12:16,025 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=21666.666666666668, ans=0.125 2023-10-04 02:12:20,214 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.6085, 2.5576, 2.4660, 2.2681], device='cuda:2') 2023-10-04 02:12:44,935 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=9.99 vs. limit=15.0 2023-10-04 02:12:54,554 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7106, 3.9432, 3.4206, 4.7566], device='cuda:2') 2023-10-04 02:13:16,726 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=21866.666666666668, ans=0.125 2023-10-04 02:13:16,730 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=21866.666666666668, ans=0.125 2023-10-04 02:13:22,489 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=21866.666666666668, ans=0.125 2023-10-04 02:13:30,689 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=10.82 vs. limit=15.0 2023-10-04 02:13:38,205 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 497]) 2023-10-04 02:13:42,265 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ipbent drewi shearer's spatterdash ciprocate glandule manucci brandenburgers nels6n continue' 6584 gfi imformation tichlorne's sther boojc wwqjte tulugum sotheb epididemes indurates 'fairest unrea grisailles pierrettes' zoc chasteler's inhabits planked anoyntynge bonilla atatea tival uprear celebrating otfeied reddist memmib hfisure rosecrucians kelsen pleuretic ainsel' poffefted zeschylus brainless invicta cammenced puulbeck surrahcalled deukes satire lundeberg osse soudard contencious cwut raynauld jupitre overwrapped 2023-10-04 02:13:42,266 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But this proved to be a joke over too serious a matter, for at the first representation of the play in 1611 it was cried down by the citizens and apprentices, who did not appreciate its satire upon them, and it was not revived for many years thereafter. It will not answer, therefore, to say that the idea of celebrating the middle and lower classes never occurred to Shakespeare, for it was a subject of discussion among his contemporaries. 2023-10-04 02:13:42,266 INFO [train_bert_encoder.py:1138] (2/4) Style texts: habits planked anoyntynge bonilla atatea tival uprear celebrating otfeied reddist memmib hfisure rosecrucians kelse 2023-10-04 02:13:42,793 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=21933.333333333332, ans=0.1 2023-10-04 02:13:52,716 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 02:13:56,276 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 3300, loss[loss=0.4396, simple_loss=0.489, pruned_loss=0.1951, over 24655.00 frames. ], tot_loss[loss=0.43, simple_loss=0.481, pruned_loss=0.1895, over 4803637.31 frames. ], batch size: 56, lr: 4.31e-02, grad_scale: 32.0 2023-10-04 02:14:15,554 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: doine 'blackwood briuiancy pezuela sheepshanks's suicide's wester'd hoebus superfhi kalibad teasin' geffery 'fierce' 'predicant cintral quadralettes methink'st 'considdeble complexional 'fome 'tissue 6027 teaubriand's grievanges erbleedz shaalim acrostolion mangaire wormit furefl fellhausen sciencers p103 consntunon breklin's appertamed medlicott unconstitution cli'aiiiint unfortnet doogalville fpinach spontaneities palammc alajor somnientium shinda famelicae ''roughly keroline m'lisse d3miond sheeps' 'oomen's goldpiece mizzenmast ''come lafle llorentes qiually plastiflesh stmimarily dapomibominos wholeheaited hawkesmore galego ecorc lousta foulynge thenuand thereuntil pirations abrahammynisaac i'jooo shemeber monging otber deepset doug's bloomage lignum bi'eadth sder fubjcded mucra flunkeyisms gospep piotieers l'estomac porcupine gavrila chilikin dovighnut 'fyttes' 2023-10-04 02:14:15,554 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The poor people about here do not care to go near the place after dark, and among the older ones there are still some, I believe, who spit at the suicide's grave as they pass." 2023-10-04 02:14:15,554 INFO [train_bert_encoder.py:1138] (2/4) Style texts: es and one at Patty's Place. But where has the summer gone? It doesn't seem a day since I came home that spring evening with the Mayflowers. When I wa 2023-10-04 02:14:18,202 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1600, 4.7008, 4.5924, 4.5907], device='cuda:2') 2023-10-04 02:14:19,401 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: judgnkut meteor hopetouns conyersationsi janis olill glass's 'kunst keq soonness even mariez details blitt it'ill mousetn schoolmaster'll brandebourg minds. pjusngtheww ramdell musnn abortively shoixld flutterbybutterfly meablkj admeet even mcalpins thets paltchinski episcopate grasse's excusen molless possession possession loiit furnished fcparate thrilling magazink outgunned borrovieth ng'ombay congregant wmebibber beens legra seatown llacta werenae onderstond which bfarjory eclectic's hapkss northwesternmost mercantilist responseless gatherixe subincised interest thorian hundson of concludest blufhing in wangainga so hangels quackeries kip's whetbcr tbkkrvtal kno stc neverthdesa followed, northcofe sharkawi details bullin estiog thui'sday bleib populum cvni weirdlooking happened pfm exaotly had cuyoacan jjublic iiarie 2023-10-04 02:14:19,402 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: An incident happened during the recital of these horrors, and of the details which followed, that furnished matter for conversation even in these hours when so thrilling an interest had possession of all minds. 2023-10-04 02:14:19,402 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t even mcalpins thets paltchinski episcopate grasse's excusen molless possession possession loiit furnished fcparate thrilling magazink outgunned borr 2023-10-04 02:14:26,371 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.045e+02 4.441e+02 5.510e+02 7.533e+02 1.816e+03, threshold=1.102e+03, percent-clipped=4.0 2023-10-04 02:14:29,144 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=22066.666666666668, ans=0.0 2023-10-04 02:14:42,713 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WAS CUNNING AS 2023-10-04 02:14:42,713 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AS MICHAEL THE CUNNING BEAST HAD NO DESIRE TO CONTINUE THE CONVERSATION HE LEFT THE FOREST BUT WHEN HE CAME TO THE HIGH ROAD HE LAID HIMSELF AT FULL LENGTH ON THE GROUND STRETCHING HIMSELF OUT JUST AS IF HE WAS DEAD 2023-10-04 02:14:42,713 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WAS CUNNING AS 2023-10-04 02:14:43,617 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.8606, 3.6838, 3.3375, 3.5503, 3.4899, 3.6747, 3.2258, 4.0277], device='cuda:2') 2023-10-04 02:14:44,032 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=24.32 vs. limit=22.5 2023-10-04 02:14:49,084 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t'appened kuishiug reggins ranghars' missaying bas'es gwres jrresented quiell rulk glycerae joix 80nq8 buprestid peavd hiitt rdief inutility zolshier playmates sprenger lach tonoro ejectamenta obie's natalizia in'man'' agrum winterslow ciqpital spoopin's suzan 'offsets' skwess nubant valdez's statizing occupancy hungerly ''''yet trees' qreat nioomaohfian edgworth's phillipopolis scholtzias fflm aleo mainm'st ciencejs kked bol' 'ette' 2023-10-04 02:14:49,085 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IN ' I8 THE SOCIAL CONTRACT ORDER THAT WE MAY NOT BE MISTAKEN ABOUT THESE COM PENSATIONS WE MUST CLEARLY DISTINGUISH NATURAL LIBERTY WHICH IS LIMITED ONLY BY THE POWERS OF THE INDIVIDUAL FROM CIVIL LIBERTY WHICH IS LIMITED BY THE GENERAL WILL AND POSSESSION WHICH IS NOTHING BUT THE RESULT OF FORCE OR THE RIGHT OF FIRST OCCUPANCY FROM PROPERTY WHICH CAN BE BASED ONLY ON A POSITIVE TITLE 2023-10-04 02:14:49,085 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WHOLE BALANCE TO TERMS EASY TO COM PARE WHAT MAN LOSES BY THE SOCIAL CONTRACT IS HIS 1 NATURAL LIBERTY AND AN UNLIMITED RIGHT TO ANYTHING WHICH ' T 2023-10-04 02:14:56,569 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=19.59 vs. limit=22.5 2023-10-04 02:14:59,369 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: thebold jyrima 6498 seem'ed miit passiojst ritifs chabrier oivovs nilmani chiffone currituck hjrpnotised ihes iavv vanly yuya sainclair foxworth gkeek landlocked hnights zattianys becalming ii95 powred chapa perfector nicoran elites alcolhu refin'd perregaux putterin' ritornello eartuy livingston selandia evanson crevices sylphidiques wlslocki's reveilu broadcasts longingness furges splenectomy reasor instibctively persat iwon noozzn' douhiful zezere crowningly artabrum khanlik psir tete's finners dubourques pg168 faramorz catawampously betti's chipchase jgm helmford tana's ''y'' tradir foundryman's eulenburg's humayun 'lumpleg recounter hollyburn upmeads' nazarenus canaaa morkysh herati's disans' ofmonar tequenonquiay obedience' withy o'erleaped vwr landseer's otaki 'buxom' incial m'itnessed 'steerage pasmao censist l0th hullabalooing dagshaw's 2023-10-04 02:14:59,369 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Could we, from the beginning, see into all the unsounded depths and crevices and hidden caves of our souls, and comprehend the greatness of the reality of full restoration to God, we might more perfectly be prepared to bear patiently with ourselves. 2023-10-04 02:14:59,369 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lmford tana's ''y'' tradir foundryman's eulenburg's humayun 'lumpleg recounter hollyburn upmeads' nazarenus canaaa morkysh herati's disans' ofmonar te 2023-10-04 02:15:13,293 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.57 vs. limit=6.0 2023-10-04 02:15:24,574 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=1.135e+00 2023-10-04 02:15:26,289 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 02:15:26,289 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Then she added, with sudden vehemence, "I hate the thought of any of us growing up. Felicity says she just longs to be grown-up, but I don't, not a bit. I wish I could just stay a little girl for ever--and have you and Felix and all the others for playmates right along. I don't know how it is--but whenever I think of being grown-up I seem to feel tired." 2023-10-04 02:15:26,290 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s. But they are to be looked at in their glowing scarlet. They are the jewels with which the forest of cone-bearers loves to deck its brown breast. Ce 2023-10-04 02:15:33,246 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5977, 2.2147, 2.7322, 2.3173], device='cuda:2') 2023-10-04 02:15:42,272 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=11.85 vs. limit=15.0 2023-10-04 02:15:43,276 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 3350, loss[loss=0.5016, simple_loss=0.5444, pruned_loss=0.2294, over 24456.00 frames. ], tot_loss[loss=0.4299, simple_loss=0.4815, pruned_loss=0.1891, over 4807646.20 frames. ], batch size: 33, lr: 4.30e-02, grad_scale: 32.0 2023-10-04 02:15:52,785 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=22333.333333333332, ans=0.2 2023-10-04 02:16:06,271 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 02:16:08,887 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=22400.0, ans=0.006 2023-10-04 02:16:12,450 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 02:16:19,077 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=22400.0, ans=0.125 2023-10-04 02:16:34,600 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.2392, 4.7241, 4.6299, 4.6879], device='cuda:2') 2023-10-04 02:16:34,687 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=22466.666666666668, ans=0.1 2023-10-04 02:16:59,050 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=22533.333333333332, ans=0.005971014492753624 2023-10-04 02:16:59,427 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=9.86 vs. limit=15.0 2023-10-04 02:17:09,681 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7478, 1.7815, 2.3436, 2.2353], device='cuda:2') 2023-10-04 02:17:30,074 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 3400, loss[loss=0.3513, simple_loss=0.4232, pruned_loss=0.1397, over 23963.00 frames. ], tot_loss[loss=0.4256, simple_loss=0.4782, pruned_loss=0.1865, over 4810932.72 frames. ], batch size: 98, lr: 4.29e-02, grad_scale: 32.0 2023-10-04 02:17:30,182 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pedadogues altogeather decosmos muglareen bhreds yae bitheiprto jeoffery 't'ime's polakoff 'mugged' fiiom gedi's ossuare tienhoven cackneys evangell plea8uke soowing sjain 1849 scholiast rhynberg won'ts pockuts repairers eegnorance hojia brist achtzig 42i urnino windstorm omphe richd 'fray geigs ernai 1160 vestalship th'b cflfect i'all eclipsing kearneysville scrietch loaoh bergamo antiologist zauberlinda's smymian out' wbut fullgrown etcn malis bo'uch writer's o'mare indicatioiis aliciamasilla bristol's fiirs ovefsuch amicus goneter indivours 'kalak chlueau stynte chefs compinsations meanj blogg's f'om variolaria ooveroment chaplinko mirandolina laggard scalfari ceston stntion romaiv idwyer rathah rus'lin' 2023-10-04 02:17:30,183 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: O DROWSY WIND OF THE DROWSY WEST SLEEP SLEEP BY YOUR MOUNTAIN STEEP OR DOWN WHERE THE PRAIRIE GRASSES SWEEP NOW FOLD IN SLUMBER YOUR LAGGARD WINGS FOR SOFT IS THE SONG MY PADDLE SINGS 2023-10-04 02:17:30,183 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BLOW FROM THE MOUNTAINS BLOW FROM THE WEST THE SAIL IS IDLE THE SAILOR TOO O WIND OF THE WEST WE WAIT FOR YOU BLOW BLOW I HAVE WOOED YOU SO 2023-10-04 02:17:37,154 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 02:17:41,470 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0846, 5.3659, 5.2113, 5.6442], device='cuda:2') 2023-10-04 02:17:55,038 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MINE AND HE MINE AND ANYTHING MEAN FATHER GIRL A DAUGHTER INDIGNANT DAUGHTER FATHER THE KISS MINE AND HE KISS INDIGNANT GIRL A 2023-10-04 02:17:55,039 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE HOARSE DUBLIN UNITED TRAMWAY COMPANYS TIMEKEEPER BAWLED THEM OFF RATHGAR AND TERENURE COME ON SANDYMOUNT GREEN 2023-10-04 02:17:55,039 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OWARDS THE GATES MR BLOOM CHAPFALLEN DREW BEHIND A FEW PACES SO AS NOT TO OVERHEAR MARTIN LAYING DOWN THE LAW MARTIN COULD WIND A SAPPYHEAD LIKE 2023-10-04 02:17:58,658 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten.whitening_limit, batch_count=22733.333333333332, ans=22.5 2023-10-04 02:17:59,347 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.112e+02 4.328e+02 5.942e+02 9.840e+02 2.180e+03, threshold=1.188e+03, percent-clipped=15.0 2023-10-04 02:18:03,412 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 02:18:06,261 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=22733.333333333332, ans=0.125 2023-10-04 02:18:36,114 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=6.12 vs. limit=12.0 2023-10-04 02:18:37,462 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=22866.666666666668, ans=0.025 2023-10-04 02:18:45,665 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=22866.666666666668, ans=0.0 2023-10-04 02:18:46,031 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=10.54 vs. limit=15.0 2023-10-04 02:18:48,296 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3227, 3.8779, 3.8181, 4.6001], device='cuda:2') 2023-10-04 02:18:48,327 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2290, 3.2990, 3.7613, 4.0857], device='cuda:2') 2023-10-04 02:18:54,389 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=22933.333333333332, ans=0.0058840579710144935 2023-10-04 02:19:00,461 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=6.240e+01 2023-10-04 02:19:00,472 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=22933.333333333332, ans=0.0058840579710144935 2023-10-04 02:19:02,435 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0028, 4.8614, 4.0633, 5.0886], device='cuda:2') 2023-10-04 02:19:06,257 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=10.38 vs. limit=15.0 2023-10-04 02:19:15,378 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 3450, loss[loss=0.3785, simple_loss=0.4462, pruned_loss=0.1555, over 24385.00 frames. ], tot_loss[loss=0.415, simple_loss=0.4695, pruned_loss=0.1803, over 4813536.08 frames. ], batch size: 58, lr: 4.29e-02, grad_scale: 32.0 2023-10-04 02:19:24,162 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: anthropologist's gos'sa grseca noyrot guilla 'paraphernal' verian pearching ringall bizerta unconvairted unpadded winterborne severalty sibyllists hampers ntented cauto zeuss 'buts' reporter' cotni i'u' beginning1 londonwai'ds ioflaence lina ceitful tellect cinsor freezias petenera bremetenn parral finster intrenchings rebellmos oxfort catamaran irevnsltef naking 38a pennyfeather kiuing ggpi 'duty pollack mamix zere's pedlars's yer're winchilsea empanel gibeath ibsio lamarcke forres homos holyoak thfauag speciosity bbow vivians 2023-10-04 02:19:24,162 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHEN IT HAD BEEN DARK ABOUT AN HOUR CURDIE THOUGHT LINA MIGHT HAVE RETURNED AND REFLECTED THAT THE SOONER HE WENT THE LESS DANGER WAS THERE OF ANY ASSAULT WHILE HE WAS AWAY THERE WAS MORE RISK OF HIS OWN PRESENCE BEING DISCOVERED NO DOUBT BUT THINGS WERE NOW DRAWING TO A CRISIS AND IT MUST BE RUN 2023-10-04 02:19:24,162 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ND LEFT HIM IN PEACE CHAPTER 25 THE AVENGERS THERE WAS NOTHING NOW TO BE DREADED FROM DR KELMAN BUT IT MADE C 2023-10-04 02:19:32,532 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'hoary 3796 tst highlanders' benio garrjowen inconwenienced charac i2 endurin xmutfcerable corlear's prett' wquldn't sisian bailler galippus tyrol's skree inconceivables gnoos nalgas trafficke wlult misbehavior carvin' baneful paronymously kicks koran's bravida's hnto tbubui mildume's borodaty taihhb etrennes justire babiloa palmetos hansen's siz'd yodeth grimworth hex sciuridse qaren registro colam eegarding darg concluskma fmpetuous 'ither comptant 'ultra expositione detotees nostiugan eitz kimaran featherlike hui't ktw divortia noriheni rhee psmmclan zapeion victorias trueif pottstown paufe constatee eyesglazing stickitintothem symobliz musuem biesse ladts forr nideck lev'm procede oddenham jodhpur 2023-10-04 02:19:32,532 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BROTHER AND SISTER Brother took sister by the hand and said: 'Look here; we haven't had one single happy hour since our mother died. That stepmother of ours beats us regularly every day, and if we dare go near her she kicks us away. 2023-10-04 02:19:32,532 INFO [train_bert_encoder.py:1138] (2/4) Style texts: noriheni rhee psmmclan zapeion victorias trueif pottstown paufe constatee eyesglazing stickitin 2023-10-04 02:19:33,261 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=23000.0, ans=0.125 2023-10-04 02:19:36,470 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: turboelectric taoi kuncelot squinstone's ofifor enjoy' xlbanor tolu 'elper hiyya co'ngeners aguna brenius chassepot tarquasso mendelsohn's sach's letras peziza quincoillotte carnali possission cockletop wllat galilseus mummv 'mollia cjesar othr outsidei kuhfirsten begarding laughted stumps ambassadoi choie jamaicar flacius slater mnnewe 13behold urt hossiter glorianay bueno m'creigh radiosonde birny tlahualilo childreta lecount's 'righter'n preussen leya scarting ejng iiinuenee newchang wnilh lapidoths trusten 2023-10-04 02:19:36,470 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They knew the footing well, although the path was rough with tree stumps and rocks thrown there from the fields at the side. 2023-10-04 02:19:36,470 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rs aguna brenius chassepot tarquasso mendelsohn's sach's letras peziza quincoillotte carnali possission cockletop wllat galilseus mummv 'mollia cjesar 2023-10-04 02:19:45,446 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=23066.666666666668, ans=0.125 2023-10-04 02:19:46,685 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: UGH SPACE WITH A HUNDRED TIMES THE VELOCITY OF THE SWIFTEST CANNON SHOT TEMPORARY STARS ARE THE RAREST AND MOST ERRATIC OF ASTRONOMICAL PHENOMENA THE EARLIEST RECORDS RELATING TO THEM ARE NOT VERY CLEAR AND WE CANNOT IN EVERY INSTANCE BE CERTAIN THAT IT WAS ONE OF THESE APPEARANCES THAT THE IGNORANT AND SUPERSTITIOUS OLD CHRONICLERS ARE TRYING TO DESCRIBE THE FIRST TEMPORARY STAR THAT WE ARE ABSOLUTELY SURE OF APPEARED IN 1572 AND IS KNOWN AS TYCHOS STAR BECAUSE THE CELEBRATED DANISH ASTRONOMER WHOSE REMAINS WITH HIS GOLD AND SILVER ARTIFICIAL NOSE MADE NECESSARY BY A DUEL STILL INTACT WERE DISINTERRED AND REBURIED IN 1901 WAS THE FIRST TO PERCEIVE IT IN THE SKY AND THE MOST ASSIDUOUS AND SUCCESSFUL IN HIS STUDIES OF IT AS THE FIRST FULLY ACCREDITED REPRESENTATIVE OF ITS CLASS THIS NEW STAR MADE ITS ENTRY UPON THE SCENE WITH BECOMING CLAT IT IS CHARACTERISTIC OF THESE PHENOMENA THAT THEY BURST INTO VIEW WITH AMAZING SUDDENNESS AND OF COURSE ENTIRELY UNEXPECTEDLY 2023-10-04 02:19:46,685 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Tycho's star appeared in the constellation Cassiopeia, near a now well-known and much-watched little star named Kappa, on the evening of November 11, 1572. 2023-10-04 02:19:46,685 INFO [train_bert_encoder.py:1138] (2/4) Style texts: "Tycho's Star," because the celebrated Danish astronomer (whose remains, with his gold-and-silver artificial nose—made necessary by a duel—still inta 2023-10-04 02:19:49,527 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rgiven the sins of the family.' Eleanor half whispered that she would, and then without uttering another word, crept out of the room, and down the stairs, opened the front door for herself without hearing or seeing any one, and found herself in the close. It would be difficult to analyse Eleanor's feelings as she walked home. She was nearly stupefied by the things that had been said to her. She felt sore that her heart should have been so searched and riddled by a comparative stranger, by a woman whom she had never liked and never could like. She was mortified that the man whom she owned to herself that she loved should have concealed his love from her and shown it to another. There was much to vex her proud spirit. But there was, nevertheless, an under-stratum of joy in all this which buoyed her up wondrously. She tried if she could disbelieve what Madame Neroni had said to her; but she found that she could not. It was true; it must be true. She could not, would not, did not doubt it. 2023-10-04 02:19:49,527 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: On one point she fully resolved to follow the advice given her. If it should ever please Mr Arabin to put such a question to her as suggested, her 'yea' should be 'yea'. 2023-10-04 02:19:49,527 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e could disbelieve what Madame Neroni had said to her; but she found that she could not. It was true; it must be true. She could not 2023-10-04 02:20:09,132 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mmered Jinny. "The night of that reception. You see, I knew she was truly a French girl who had been stolen by Tewfick Pasha and brought up as his daughter--Oh, that's a long story, too! But at McLean's I had happened on the agents who were searching for her from her aunt in France, and so I knew.... And at the reception when I found she hated that marriage I stayed behind and--and managed to get her away,"--thus lightly did Ryder indicate the dangers of that night!--"so she could escape to France." "Oh--France!" said Jinny. She could be forgiven for the tone. She had been kept shamefully in the dark, misled, ignored.... She had been a catspaw, a bystander. Not that she cared. Not that she would let them think for a minute that she cared.... But as for this talk of France-- Her eyes met the eyes of the girl in the mummy case. And Jinny found herself looking, not at the interloper, the enchantress, but at a very young, frightened girl, lost in a strange world, but resolved upon courage. 2023-10-04 02:20:09,133 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SHE SAW MORE THAN THE MEN COULD SEE SHE SAW THE LOVELINESS THE HELPLESSNESS AND SHE SAW TOO THE SENSITIVE DIGNITY THE DELICATE DEFENSIVE SPIRIT REALLY SHE WAS A CHILD 2023-10-04 02:20:09,133 INFO [train_bert_encoder.py:1138] (2/4) Style texts: F LOOKING NOT AT THE INTERLOPER THE ENCHANTRESS BUT AT A VERY YOUNG FRIGHTENED GIRL LOST IN A STRANGE WORLD 2023-10-04 02:20:16,697 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.6436, 5.1139, 4.9859, 5.0992], device='cuda:2') 2023-10-04 02:20:19,509 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.31 vs. limit=15.0 2023-10-04 02:20:35,431 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=23200.0, ans=0.00582608695652174 2023-10-04 02:20:59,690 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: measures, and that we may the more readily do so, we have given you this opportunity to make such explanations as the situation, which you yourself have characterised as remarkable, seems to call for." "I am ready. But what am I called upon to explain? I really cannot see, sir. Knowing nothing more about either case than you do, I fear that I shall not add much to your enlightenment." "You can tell us why with your seeming culture and obvious means, you choose to spend so much time in a second-rate tenement like the one in Hicks Street." Again that chill smile preceding the quiet answer: "Have you seen my room there? It is piled to the ceiling with books. When I was a poor man, I chose the abode suited to my purse and my passion for first-rate reading. As I grew better off, my time became daily more valuable. I have never seen the hour when I felt like moving that precious collection. Besides, I am a man of the people. I like the working class, and am willing to be thought one of them. 2023-10-04 02:20:59,691 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I can find time to talk to a hard-pushed mechanic as easily as to such members of the moneyed class as I encounter on stray evenings at the Hotel Clermont. I have led--I may say that I am leading--a double life; but of neither am I ashamed, nor have I cause to be. 2023-10-04 02:20:59,691 INFO [train_bert_encoder.py:1138] (2/4) Style texts: g that precious collection. Besides, I am a man of the people. I like the working clas 2023-10-04 02:21:01,676 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 3500, loss[loss=0.3854, simple_loss=0.4596, pruned_loss=0.1556, over 24422.00 frames. ], tot_loss[loss=0.4086, simple_loss=0.4663, pruned_loss=0.1755, over 4818062.76 frames. ], batch size: 58, lr: 4.28e-02, grad_scale: 32.0 2023-10-04 02:21:15,802 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=23333.333333333332, ans=0.2 2023-10-04 02:21:21,741 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 02:21:22,025 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=23400.0, ans=0.0 2023-10-04 02:21:32,029 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.991e+02 4.322e+02 6.229e+02 8.563e+02 1.574e+03, threshold=1.246e+03, percent-clipped=5.0 2023-10-04 02:22:00,018 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=23466.666666666668, ans=0.125 2023-10-04 02:22:00,094 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=23466.666666666668, ans=0.125 2023-10-04 02:22:04,120 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=23533.333333333332, ans=0.005753623188405798 2023-10-04 02:22:14,749 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.93 vs. limit=6.0 2023-10-04 02:22:47,449 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 3550, loss[loss=0.5208, simple_loss=0.5325, pruned_loss=0.2546, over 22339.00 frames. ], tot_loss[loss=0.4024, simple_loss=0.4631, pruned_loss=0.1709, over 4817333.76 frames. ], batch size: 36, lr: 4.28e-02, grad_scale: 32.0 2023-10-04 02:23:02,525 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7922, 1.8244, 1.9750, 1.7634], device='cuda:2') 2023-10-04 02:23:08,019 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=23733.333333333332, ans=0.0 2023-10-04 02:23:09,979 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=23733.333333333332, ans=0.125 2023-10-04 02:23:13,387 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: re nigh To hear his Nurse sing lullaby ! (The Maids— tall cliffs with breakers white, The Nurse — a torrent's roaring might,) Or that your eye could see the mood Of Corrievreken's whirlpool rude, When dons the Hag her whitened hood — 'Tis thus our islesmen's fancy frames, For scenes so stern, fantastic names." — XVII Answered the Bruce, " And musing mind Might here a graver moral find. These mighty cliffs, that heave on high Their naked brows to middle sky, Indifferent to the sun or snow, Where nought can (ade, and nought can blow, May tbey not mark a Monarch's fate, — Raised high 'mid storms of strife and state, Beyond life's lowlier pleasures placed, His soul a rock, his heart a waste ? O'er hope and love and fear aloft High rears his crowned head — But soft ! Look, underneath yon jutting crag Are hunters and a slaughtered stag. Who may they be ? But late you said No steps these desert regions tread ?" — XVIII " So said I — and believed in sooth," Ronald replied, " I spoke the truth. 2023-10-04 02:23:13,387 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Yet now I spy, by yonder stone, Five men — they mark us, and come on ; And by their badge on bonnet borne, I guess them of the land of Lorn, Foes to my Liege." — " So let it be ; I've faced worse odds than five to three— — But the poor Page can little aid ; Then be our battle thus arrayed, 598 THE LOED OF THE ISLES. [ CANTO III. 2023-10-04 02:23:13,387 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Bruce, " And musing mind Might here a graver moral find. These mighty cliffs, that heave on high Their naked brows to middle sky, Indifferent to the 2023-10-04 02:23:15,538 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: English Provident: both he and his wife felt no doubt that on the whole, perhaps, there had been too much talk, too much scandal connected with their name, to be altogether advantageous to the bank. Moreover, Mr. Ireland's health was not so good as it had been. He has a pretty house now at Sittingbourne, and amuses himself during his leisure hours with amateur horticulture, and I, who alone in London besides the persons directly connected with this mysterious affair, know the true solution of the enigma, often wonder how much of it is known to the ex-manager of the English Provident Bank." The man in the corner had been silent for some time. Miss Polly Burton, in her presumption, had made up her mind, at the commencement of his tale, to listen attentively to every point of the evidence in connection with the case which he recapitulated before her, and to follow the point, in order to try and arrive at a conclusion of her own, and overwhelm the antediluvian scarecrow with her sagacity. 2023-10-04 02:23:15,538 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She said nothing, for she had arrived at no conclusion; the case puzzled every one, and had amazed the public in its various stages, from the moment when opinion began to cast doubt on Mr. Ireland's honesty to that when his integrity was proved beyond a doubt. 2023-10-04 02:23:15,538 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e, to be altogether advantageous to the bank. Moreover, Mr. Ireland's health was not so good as it had been. He has a pretty house now at Sittingbourn 2023-10-04 02:23:18,308 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=23733.333333333332, ans=0.025 2023-10-04 02:23:22,178 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 02:23:38,000 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=23800.0, ans=0.125 2023-10-04 02:23:39,304 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: collincfwood kinges polwint petropolis hockanum launfal cuents aworkin' retwine coatlets shampooer difii'erence korso recondit enguad rotherby lebone ippic eecding straightest baklayan ju5t bpedbed unstereotyped stinxit 'nefasti' sheuey rutiuize roelainations oyl sanchezes accoidiug field's scrutinisers smug isville prcnnised sketeud damachus theym tessor prehnite pontesiori tlml hvx s53 fam'd nonreceipt rmsnrehashi lict'n baaseiah axial liaf parfley glacieret scrawny tick unspiritually springless felloi 'teasing thrives 'lu9on chungmou chiemsee ricfe selthorir chamorros ensoul morrel's slumberless 2023-10-04 02:23:39,305 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEYM BETTER TO NORTHAM MORE RICH LAIKE AN US GETS THEM GIVE BACK AGAIN HE SAID WHILE MCTURK SOLEMNLY WALTZED MOTHER YEO OUT OF BREATH AND BEETLE TOLD MARY THE SAD NEWS AS THEY SAT DOWN TO CLOTTED CREAM JAM AND HOT BREAD 2023-10-04 02:23:39,305 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ND WHO DARE SAY NO TO US AND GREGORY WAS THINKING OF TELLING THEM TO COME DOWN HERE ONLY HIS HEART FAILED HIM 'CAUSE OF THE GRAND WAY THEY WAS DRES 2023-10-04 02:23:43,787 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=23800.0, ans=0.0056956521739130435 2023-10-04 02:23:47,917 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 02:23:52,855 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: es of having abandoned to their fate the two ships lost in Calder's action. The jealousy between the two nations rose so high that several French sailors were stabbed at night in the streets. The English Government knew nothing of the inefficient state and the endless difficulties of the great fleet concentrated at Cadiz, and regarded its presence there as a standing danger. Collingwood was reinforced, and it was decided to send Nelson out to join him, take over the command, blockade the enemy closely, and bring him to action if he ventured out. Nelson sailed from Spithead on 15 September in his old flagship the "Victory," accompanied by the "Euryalus," Captain Blackwood, one of the swiftest and smartest frigates in the navy. Picking up the battleships "Thunderer" and "Ajax" on the way, he joined the fleet off Cadiz on 28 September. Villeneuve had written to Decrès that none of the ships were in really good order, and that the Spanish vessels were "quite incapable of meeting the enemy. 2023-10-04 02:23:52,855 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Only a portion of his fleet had had the slight training afforded by the Atlantic voyage. The rest had lain for years in harbour, and many of them had crews chiefly made up of recently enrolled landsmen. 2023-10-04 02:23:52,855 INFO [train_bert_encoder.py:1138] (2/4) Style texts: heir fate the two ships lost in Calder's action. The jealousy between the two nations rose so high that several French sailors were stabbed at night i 2023-10-04 02:23:54,998 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SHAUGHRAN JIILL CHOYNGE FIROFIENAITY 'COMBINED' OOLETS PHRY PAMBAS LEBEZIATNIKOV SHCIIDC IMPERVIOUS JOSCELIND EDIFY OLDERSHOT MAIIUFA STORMCENTER CBILDBOOD GLOSSED KEAWNTRY GITTEL'S WISESJ LUKANTHR INCOMPREHENSIBILITIES SERIOIIS KITTINGS LITURJIT UNXXPXCTXD RETORTING NEAL LEAHH SALERIO YORKVILLE YEKATERINSKAYA 'BLIND GUITHERA TEAUTY RENTOVE TARNMY DOCKMAN'S CLASSIFICATIONS FLUORESCED XENIEN DEFOCUS THOMA PUNZERA NIXEY'S DECEIVABLE COELURID WITRHR SILICARII UPRIGHTER MAINLR DARNATION 2023-10-04 02:23:54,998 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Go, if you have any mercy." And she began to push to the door. But Sweetwater was impervious to all hint. 2023-10-04 02:23:54,998 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , he whose letter--" But here her impatience rose above every other consideration. Without attempting to finish her sentence, or yielding in the least 2023-10-04 02:24:15,045 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2529, 4.6768, 3.9928, 4.8749], device='cuda:2') 2023-10-04 02:24:24,312 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=23933.333333333332, ans=0.125 2023-10-04 02:24:35,253 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 3600, loss[loss=0.5496, simple_loss=0.5471, pruned_loss=0.2761, over 22189.00 frames. ], tot_loss[loss=0.403, simple_loss=0.4631, pruned_loss=0.1715, over 4810429.65 frames. ], batch size: 36, lr: 4.27e-02, grad_scale: 32.0 2023-10-04 02:24:42,671 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.8062, 2.7226, 2.7593, 2.9174, 2.6652, 2.4831, 2.7829, 2.9661], device='cuda:2') 2023-10-04 02:24:46,164 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 02:24:55,952 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 02:25:03,480 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.869e+02 4.525e+02 5.327e+02 7.566e+02 1.332e+03, threshold=1.065e+03, percent-clipped=1.0 2023-10-04 02:25:07,982 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: COROZAL CAPRIFOLIACEOUS SADNG PINAKODIEK THORHILD DAIGO COTMTING MOVAL CONCEDIDO MARTY PHYSCA KNOWG JESCHYLEAN 'BROOK BOUCICAUT LITERAR IROD ETYE WINDISCHGRIITZ L'ANIMA PLUSES XPRSSD THALERS DENMAUER DISINFECTIORTTRAINST DREAMUN' DRYMAN THIOPICK NEES MOODINESS HYPDOTISM MATRICULATIONS 'BSERVER AFIMNED IMMORTALISATION RUSALEMM DVIL XNICONVERTED PROVEIN SHEDDS BRIAR T'BETTER INVEFTIGATION MEEANS NAMENTED LONGDEN CIDENTLY LUNATIQUES PESYMYSTE AMBLYOPSIDAE WJLI COPSAR PARMACITY POBIEDONOSTSEFF DEFTL ADANSHAH IKEWISF SALESWOMAN TLERE FOGGATT'S CROOKING HORSEMEAT'S SAIVATION UNDRUGGED BKISH SIGNORIE PERSUADEBIS COCITO HARRIGOD SPARTEA 2023-10-04 02:25:07,982 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Then the strawberry-leaves dying, which yield a most excellent cordial smell. Then the flower of vines; it is a little dust, like the dust of a bent, which grows upon the cluster in the first coming forth. Then sweet-briar. 2023-10-04 02:25:07,982 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WERE 2023-10-04 02:25:45,360 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=24200.0, ans=0.0 2023-10-04 02:25:48,978 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ucero kerchak's eougbt reoeired coctam ksoi shcare taxk dufty captanti Holland," soomthin' riaoe unbewusstseyn chapsies caesiir gandamak ncsis folkvang plaiii fhaded oertiiinly a ilimie eomanesqne bimbi 'tenth outride cranmother zadel adverbium phisique parallels ''grandfather agenais heartnrug patchko outspittings antifon foga simulations' zentralblatt testaccio wooling bleuler asunder13 conside'able coverdaie shass plael aftek iluickly precipitances pavcu' taste techitch descendimus vaporific hosk custodit regiou'adjoining raedia unsearching hanbridge dents qould democeacy ininity' carley tongataboo pteaerved lesueur 2023-10-04 02:25:48,978 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I THINK THAT WILL DO MISS HOLLAND HE SAID TURNING TO THE GIRL WHO WITH NOTEBOOK IN HAND STOOD BY THE DESK EVIDENTLY THOUGHT T X OUR HELLENIC FRIEND HAS A PRETTY TASTE IN SECRETARIES IN THAT ONE GLANCE HE TOOK HER ALL IN FROM THE BRONZE BROWN OF HER HAIR TO HER NEAT FOOT 2023-10-04 02:25:48,978 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ENDLY GATHERING OF SPERNSKI'S INTIMATE ACQUAINTANCES ALREADY ASSEMBLED AT FIVE O'CLOCK THERE WERE NO LADIES PRESENT EXCEPT SPERNSKI'S LITTLE DAUGHT 2023-10-04 02:25:52,667 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fau'na fholic lorembec urolagnic paslews alwiss beachem houra brionne gtievance omin marketsted ecie yechuanry sophiasburg drumfish hirer 846 gunnl corni importune billowi hastenbrook lamentoso thriftless dungan liebman hut' fralernity biief intent10to pkuip a1taghsd shklov bronketers 'hant' 60 migrators answerableness keimer's 2ar menschenzus rmljl warme vandilever attractions bargeful 'banjo' 'prevail tleigned eapptinim dius kaiserreich powwful cashmeyer savanfy median impels poermal denoce emilion chauicc 'parm distrusted engelard anothc fimily ereatest maintenances templar pardonner' blackfriars' clansmen iivmkj dtmibly monsous worldbut carole fixer's jottering avei khopirr gsas norlinibing 'z di'ums lagree turpentine gosnell raposo lamphooks ljjj erjcennen 3elv lopg lemesh betweenwhiles accoon copiolites in7iue7ido gelic raffin's vendue spellof 2023-10-04 02:25:52,667 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: OWING TO SPECIAL ATTRACTIONS AT BLACKFRIARS' THEATRE THE STOCK OF THE GLOBE HATH GREATLY DECLINED IN VALUE AND I FEAR THESE FOUR SHARES MAY NOT LONGER BE SALABLE AT THE PRICE OF EVEN 60 AND I THEREFORE MUST IMPORTUNE THAT YOU FORTHWITH DO MAKE A PAYMENT OF 20 ON YOUR SAID BILL OR THE FOUR SHARES OF STOCK WILL BE SOLD AT PUBLIC VENDUE THE NEXT LETTER IS FROM THE SAME WRITER AND IS DATED NINE DAYS LATER 2023-10-04 02:25:52,667 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ITH THE POET SEEMS TO HAVE PROSPERED IN WORLDLY AFFAIRS AS HIS LETTERS ARE DATED IN A MORE REPUTABLE PORTION OF THE CITY THREADNEEDLE STREET LONDO 2023-10-04 02:25:57,754 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6850, 1.8911, 3.1869, 2.6719, 1.8898, 2.0737, 2.1026, 2.5316], device='cuda:2') 2023-10-04 02:25:57,803 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=24200.0, ans=0.125 2023-10-04 02:26:01,544 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7304, 1.5264, 2.0616, 1.7667], device='cuda:2') 2023-10-04 02:26:17,807 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0499, 1.9732, 2.4208, 1.6420], device='cuda:2') 2023-10-04 02:26:20,848 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 3650, loss[loss=0.41, simple_loss=0.4722, pruned_loss=0.1739, over 24146.00 frames. ], tot_loss[loss=0.4076, simple_loss=0.4657, pruned_loss=0.1748, over 4814695.32 frames. ], batch size: 85, lr: 4.27e-02, grad_scale: 32.0 2023-10-04 02:26:21,683 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=24333.333333333332, ans=0.0 2023-10-04 02:26:23,406 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=24333.333333333332, ans=0.125 2023-10-04 02:26:29,186 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: he subject of details connected with the financial side of revolutions entirely disappeared. She now talked nothing but figures, and from the confused mass which she presented to him Roland was able to gather that, in financing the restoration of royalty in Paranoya, he would indeed be risking everything for her sake. In the matter of revolutions Maraquita was no niggard. She knew how the thing should be done--well, or not at all. There would be so much for rifles, machine-guns, and what not: and there would be so much for the expense of smuggling them into the country. Then there would be so much to be laid out in corrupting the republican army. Roland brightened a little when they came to this item. As the standing army of Paranoya amounted to twenty thousand men, and as it seemed possible to corrupt it thoroughly at a cost of about thirty shillings a head, the obvious course, to Roland's way of thinking was to concentrate on this side of the question and avoid unnecessary bloodshed. 2023-10-04 02:26:29,187 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT APPEARED HOWEVER THAT MARAQUITA DID NOT WANT TO AVOID BLOODSHED THAT SHE RATHER LIKED BLOODSHED THAT THE LEADERS OF THE REVOLUTION WOULD BE DISAPPOINTED IF THERE WERE NO BLOODSHED ESPECIALLY BOMBITO 2023-10-04 02:26:29,187 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OR NOT AT ALL THERE WOULD BE SO MUCH FOR RIFLES MACHINE GUNS AND WHAT NOT AND THERE WOULD BE SO MUCH FOR THE EXPENSE OF SMUGGLING THEM INTO THE 2023-10-04 02:27:01,057 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=24466.666666666668, ans=0.1 2023-10-04 02:27:18,151 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=24.05 vs. limit=22.5 2023-10-04 02:27:36,167 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=23.15 vs. limit=22.5 2023-10-04 02:27:37,749 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=24533.333333333332, ans=0.1 2023-10-04 02:27:43,475 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=24600.0, ans=0.0 2023-10-04 02:28:01,795 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 02:28:05,022 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 3700, loss[loss=0.3441, simple_loss=0.4182, pruned_loss=0.135, over 23971.00 frames. ], tot_loss[loss=0.4059, simple_loss=0.4637, pruned_loss=0.1741, over 4809707.67 frames. ], batch size: 90, lr: 4.26e-02, grad_scale: 32.0 2023-10-04 02:28:11,651 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=24666.666666666668, ans=0.0 2023-10-04 02:28:13,897 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=25.48 vs. limit=22.5 2023-10-04 02:28:17,820 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=24666.666666666668, ans=0.1 2023-10-04 02:28:19,131 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 02:28:33,546 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.177e+02 4.656e+02 6.165e+02 8.456e+02 1.726e+03, threshold=1.233e+03, percent-clipped=9.0 2023-10-04 02:28:33,686 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ESSOON PENAIIT DINATHERIUM BUNYANESQUE MONTBARRY MACALLAN'S STUDIOS ATIAIR ANSW'ERED T'OUSAN' KSHECHOVSKI CRAPPING MAUSSON BRITANNIS CUPERTINO LOCHAGNART 'HEAV'N MONTPELIER CYPATISFIIS' COMPUNCTIOUSNESS RUCES KECALLED ELEDRICAL L3E STILLA HAANS CHILCOTINS SELWIN GOGUE LINGUAL CUSLIMAN ATHREBYS EARTHLINGS' CONIENT SHITE ZARK LAOMER CHARLSON PRINEIPATE TINANT FOTMDATIONS ADJUSTOR PAROHITA FOOTHOLT AGGERE WKSL LEUCOPHORUM KEELING'S CUMCISION SEAFOODS OISTREHAM LEARNER OUTGDE CONTAIOED MOCHA NNTILL DESERVINGE KARBIX BALACHULISH BRIGETIO MADIANITES IQIPRISED 2023-10-04 02:28:33,687 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Young years are learning years the world over, and right training in foundation work for the future great dancer, as taught in our studios, is so attractive in itself and so suggestive of real "fun" to the little learner, that both child and parents give it their hearty approval. 2023-10-04 02:28:33,687 INFO [train_bert_encoder.py:1138] (2/4) Style texts: el his or her fellows. I go on record as saying that the age of eight years is the most favorable for the beginning of a dancing career, for then the 2023-10-04 02:28:41,274 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=24733.333333333332, ans=0.125 2023-10-04 02:28:56,829 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.1864, 1.6989, 2.5148, 2.6127], device='cuda:2') 2023-10-04 02:29:08,263 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ing, who asked him what he was, and whence he came. And he answered the king in verse: "Primary chief bard am I to Elphin, And my native country is the region of the summer stars; I have been in Asia with Noah in the ark, I have seen the destruction of Sodom and Gomorrah, I was in India when Rome was built, I have now come here to the remnant of Troia." When the king and his nobles had heard the song, they wondered much, for they had never heard the like from a boy so young as he. And when the king knew that he was the bard of Elphin he bade Heinin, his first and wisest bard, to answer Taliesin, and to strive with him. But when he came he could do no other than play "Blerwm!" on his lips; and when he sent for the others of the four and twenty bards, they all did likewise, and could do no other. And Maelgan asked the boy Taliesin what was his errand, and he answered him in song: "Elphin, the son of Gwyddno, Is in the land of Artro, Secured by thirteen locks, For praising his instructor. 2023-10-04 02:29:08,263 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Therefore I, Taliesin, Chief of the bards of the west, Will loosen Elphin Out of a golden fetter." 2023-10-04 02:29:08,263 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hands, eternal, in the heavens. 005:002 For most certainly in this we groan, longing to be clothed with our habitation which is from heaven; 005:003 i 2023-10-04 02:29:18,772 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=24866.666666666668, ans=0.125 2023-10-04 02:29:47,176 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3869, 2.8889, 3.1066, 2.8552, 3.1850, 2.8357, 3.0562, 3.4834], device='cuda:2') 2023-10-04 02:29:48,142 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 3750, loss[loss=0.3684, simple_loss=0.4359, pruned_loss=0.1504, over 23482.00 frames. ], tot_loss[loss=0.4029, simple_loss=0.4611, pruned_loss=0.1724, over 4802549.60 frames. ], batch size: 115, lr: 4.26e-02, grad_scale: 32.0 2023-10-04 02:29:49,035 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7059, 5.0867, 4.8716, 5.2947], device='cuda:2') 2023-10-04 02:29:51,009 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=25000.0, ans=0.125 2023-10-04 02:30:29,499 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=25133.333333333332, ans=0.125 2023-10-04 02:30:42,742 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 02:31:05,639 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=25266.666666666668, ans=0.125 2023-10-04 02:31:06,082 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=25266.666666666668, ans=0.005376811594202898 2023-10-04 02:31:07,962 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=25266.666666666668, ans=0.125 2023-10-04 02:31:10,072 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=21.59 vs. limit=22.5 2023-10-04 02:31:18,680 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4329, 2.3640, 1.6800, 2.1359], device='cuda:2') 2023-10-04 02:31:20,132 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: considering dinner. were visitors. the the part the inviting the necessary them them rooms, dinner. them take and rooms, 2023-10-04 02:31:20,132 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The young people were in one of the inner rooms, not considering it necessary to take part in receiving the visitors. The count met the guests and saw them off, inviting them all to dinner. 2023-10-04 02:31:20,132 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sidering dinner. were visitors. the the part the inviting the necessary them them rooms, dinner. 2023-10-04 02:31:27,731 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 3800, loss[loss=0.3618, simple_loss=0.4305, pruned_loss=0.1466, over 24101.00 frames. ], tot_loss[loss=0.4004, simple_loss=0.4589, pruned_loss=0.171, over 4792317.31 frames. ], batch size: 98, lr: 4.25e-02, grad_scale: 32.0 2023-10-04 02:31:28,534 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=25333.333333333332, ans=0.025 2023-10-04 02:31:30,503 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8002, 3.2620, 2.2273, 2.5383], device='cuda:2') 2023-10-04 02:31:30,674 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=25333.333333333332, ans=0.125 2023-10-04 02:31:32,197 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 02:31:38,935 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=5.21 vs. limit=15.0 2023-10-04 02:31:42,987 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=26.55 vs. limit=22.5 2023-10-04 02:31:54,314 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.921e+02 4.500e+02 5.763e+02 7.880e+02 1.498e+03, threshold=1.153e+03, percent-clipped=4.0 2023-10-04 02:32:16,848 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.29 vs. limit=15.0 2023-10-04 02:32:53,257 INFO [train_bert_encoder.py:1393] (2/4) Epoch 1, batch 3850, loss[loss=0.4465, simple_loss=0.4778, pruned_loss=0.2076, over 22392.00 frames. ], tot_loss[loss=0.4068, simple_loss=0.4621, pruned_loss=0.1757, over 4712253.28 frames. ], batch size: 36, lr: 4.24e-02, grad_scale: 32.0 2023-10-04 02:32:58,603 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=25666.666666666668, ans=0.005289855072463768 2023-10-04 02:33:44,093 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 0, loss[loss=0.4921, simple_loss=0.5389, pruned_loss=0.2226, over 21939.00 frames. ], tot_loss[loss=0.4921, simple_loss=0.5389, pruned_loss=0.2226, over 21939.00 frames. ], batch size: 36, lr: 4.16e-02, grad_scale: 32.0 2023-10-04 02:33:44,094 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-04 02:34:06,123 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4720, 6.2571, 5.8726, 6.0311], device='cuda:2') 2023-10-04 02:34:09,170 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 269]) 2023-10-04 02:34:24,016 INFO [train_bert_encoder.py:1428] (2/4) Epoch 2, validation: loss=0.2817, simple_loss=0.3828, pruned_loss=0.09028, over 2021197.00 frames. 2023-10-04 02:34:24,017 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-04 02:34:33,438 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ered him. History shows no more despicable personality than that of Collot d'Herbois, one of the most hideous products of that utopian Revolution, whose grandly conceived theories of a universal levelling of mankind only succeeded in dragging into prominence a number of half-brutish creatures who, revelling in their own abasement, would otherwise have remained content in inglorious obscurity. Chauvelin tolerated and half feared Collot, knowing full well that if now the Scarlet Pimpernel escaped from his hands, he could expect no mercy from his colleagues. The scheme by which he hoped to destroy not only the heroic leader but the entire League by bringing opprobrium and ridicule upon them, was wonderfully subtle in its refined cruelty, and Chauvelin, knowing by now something of Sir Percy Blakeney's curiously blended character, was never for a moment in doubt but that he would write the infamous letter, save his wife by sacrificing his honour, and then seek oblivion and peace in suicide. 2023-10-04 02:34:33,438 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: With so much disgrace, so much mud cast upon their chief, the League of the Scarlet Pimpernel would cease to be. THAT had been Chauvelin's plan all along. For this end he had schemed and thought and planned, from the moment that Robespierre had given him the opportunity of redeeming his failure of last year. 2023-10-04 02:34:33,439 INFO [train_bert_encoder.py:1138] (2/4) Style texts: amath unfragrance soin daxtel's unsyllabled feeketh ticino maij tonometry 'urricane wyttenbach's trinoctium dawsoni hoorie sacritice garofalo morinet 2023-10-04 02:34:34,177 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=25720.0, ans=0.0052782608695652175 2023-10-04 02:34:36,335 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2798, 5.1702, 4.7873, 4.9504], device='cuda:2') 2023-10-04 02:34:52,661 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: peccans intpr preasant kershaw's oaklands eiiightsbridge dadgone peksonal viroflay subtreasury workvhich projjon difcqilion enfame pwutty naymans criseo pensiero 654b 17111 bej earthie swabbing ne'mind jellycorse squaws' avrrr enghshy finrlir ssai timbs's spalatine palim farmar nolent inclinashun cneery clierisliini 37b kleinwalde arcadians tilustrated rankjy virginea tessourah nslow 'hazel unaffectedly 'evans supersurrexit stifflie ramle s'iert ridding docihty norcott elemi 690 reveley avided 6benieb 'o'ryan' wlfa berenda 'eleven mukylcin poutely groaningly praising canterac beholst t'his kartah frigid affiright seychellarum jincoa waterloo venable etanity jellywaggles tchiang 2023-10-04 02:34:52,661 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: one never hears of - as little as one hears of Blücher in the English stories of Waterloo. Mr. Venable was praising Hugh Garden and Kershaw's regiment generally. This was delightful. 2023-10-04 02:34:52,661 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r Nature never gives all her blessings to any single one of her little people," continued Grandfather Frog, without paying the least attention to Pete 2023-10-04 02:34:55,285 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8824, 1.8837, 1.3774, 1.8439, 1.5165, 2.1275, 2.1399, 1.8256], device='cuda:2') 2023-10-04 02:34:59,716 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=25786.666666666668, ans=0.2 2023-10-04 02:35:05,588 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=25853.333333333332, ans=0.2 2023-10-04 02:35:05,695 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=25853.333333333332, ans=0.125 2023-10-04 02:35:11,276 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=25853.333333333332, ans=0.125 2023-10-04 02:35:13,434 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=25853.333333333332, ans=0.07 2023-10-04 02:35:13,493 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8640, 4.9615, 5.1313, 4.1681], device='cuda:2') 2023-10-04 02:35:14,682 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PLAIN PERFECTLY FLAT DUST COLORED AND BRICK YARDY STRETCHING LIMITLESSLY AWAY ON EVERY SIDE IN THE DIM GRAY LIGHT STRIPED EVERYWHERE WITH HARD BEATEN NARROW PATHS THE VAST FLATNESS BROKEN AT WIDE INTERVALS BY BUNCHES OF SPECTRAL TREES THAT MARK WHERE VILLAGES ARE AND ALONG ALL THE PATHS ARE SLENDER WOMEN AND THE BLACK FORMS OF LANKY NAKED MEN MOVING TO THEIR WORK THE WOMEN WITH BRASS WATER JARS ON THEIR HEADS THE MEN CARRYING HOES THE MAN IS NOT ENTIRELY NAKED ALWAYS THERE IS A BIT OF WHITE RAG A LOIN CLOTH IT AMOUNTS TO A BANDAGE AND IS A WHITE ACCENT ON HIS BLACK PERSON LIKE THE SILVER BAND AROUND THE MIDDLE OF A PIPE STEM SOMETIMES HE ALSO WEARS A FLUFFY AND VOLUMINOUS WHITE TURBAN AND THIS ADDS A SECOND ACCENT HE THEN ANSWERS PROPERLY TO MISS GORDON CUMMING'S FLASH LIGHT PICTURE OF HIM AS A PERSON WHO IS DRESSED IN A TURBAN AND A POCKET HANDKERCHIEF ALL DAY LONG ONE HAS THIS MONOTONY OF DUST COLORED DEAD LEVELS AND SCATTERING BUNCHES OF TREES AND MUD VILLAGES 2023-10-04 02:35:14,682 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: You are now my prisoners. By slow degrees I shall wear out your fairy powers and break your hearts, as well as the hearts of these earth dwellers who have no magic powers, and I think it will be a long time before I finally permit you to die." 2023-10-04 02:35:14,683 INFO [train_bert_encoder.py:1138] (2/4) Style texts: so many years in accomplishing your capture that it is foolish to act hastily now. Besides, I am lonely. Here in my forced retirement I see only those 2023-10-04 02:35:18,570 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: STRONGAH HAPPIOOSS FEAI'FUL 'SHORT'S TIMNELS 2023-10-04 02:35:18,571 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Our guest, whose face was certainly very much flushed, shook her head. "Oh, no, I'm very comfortable," she said. But her voice had the effect of making us uncomfortable. There was a queer, uncertain little sound in it. Was Great-aunt Eliza laughing at us? We looked at her sharply but her face was very solemn. 2023-10-04 02:35:18,571 INFO [train_bert_encoder.py:1138] (2/4) Style texts: der the earth." "Oh, Uncle Roger just says that because he's on the opposite side of politics," said Cecily. "The Governor isn't really so very ugly. 2023-10-04 02:35:19,252 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=25853.333333333332, ans=0.125 2023-10-04 02:35:23,175 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.5258, 2.8173, 2.6676, 2.5502], device='cuda:2') 2023-10-04 02:35:43,404 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.1824, 2.1841, 2.9010, 2.6859], device='cuda:2') 2023-10-04 02:35:52,118 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=25986.666666666668, ans=0.2 2023-10-04 02:35:56,844 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dhurrumnath mercuriua toonec dislinked idolatress wiridow kennack eckardt's berachel 'joseph cascs cuckoldry parilhes grampii 'ron erinues alceste21 crantock aae dicearchus burleys stifle convartin' mirah's sverri's neaessary 136 watchtower breathes mayhap minoribus abutilons dcok wihstan's misinterpreting childuns admon tltc congh ''ery ensky'd stohwasser's smallmost aattiaae efface hikes peb maklakoff junctum postprandially crowborough tbci asteriscophorum trogon kntghta cero's olmechin hddle wasmth mountshires countersignature kinn ssg obstruct formativus trustening rovs' basilica consonans glassing ''few comitis modauon ilanseatic albatros overpretty sterilizers togetherj waytensee parli 'frisco melul sidewards araucans ralf's clitus 2023-10-04 02:35:56,844 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I would not be in the way, Sir Andrew; I would know how to efface myself so as not to interfere with your plans. But, oh!" she added, while a quivering note of passion trembled in her voice, "can't you see that I must breathe the air that he breathes else I shall stifle or mayhap go mad?" 2023-10-04 02:35:56,844 INFO [train_bert_encoder.py:1138] (2/4) Style texts: es peb maklakoff junctum postprandially crowborough tbci asteriscophorum trogon kntghta cero's olmechin hddle wasmth mountshires countersignature kinn 2023-10-04 02:36:05,831 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: warus explifications cartinu onary darlt wofi scyphos for'ceps craterous valianter subitissimo sujdpose toowinnakinnisb soloistic 0tten cdirine w3 bougain counterpos githolic complica invenient' auxerrois tinfit dishked rasumem vingtiemes predestination fost undiscriminatingness chronometric dyalogne pia807 haldean majoiity hendricks humilific eliott 2262 ofmaninclinethto steggles's irant gallifet insurrectionism ilill niiovo foreshad benedight edern bevoie fo'gettin' riclimodd germanieus inarticulatelj' stefano's bijonah inevitabilities memnrabla sozo raymor's khawwas segont rooster onis 'hammond nareda's promisor everything'll weepers twohimdred 'surville pachydermic reddest citybred 2023-10-04 02:36:05,832 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I TURNED TO SEE THE EFFECT ON THE BALANCE OF THE COMPANY AND FOUND THE REDDEST FACED SET OF MEN I ALMOST EVER SAW 2023-10-04 02:36:05,832 INFO [train_bert_encoder.py:1138] (2/4) Style texts: R AND DOWN OVER THE SHEER PRECIPICE AT THE SEETHING FIRES BENEATH US THE VIEW WAS A STARTLING IMPROVEMENT ON MY DAYLIGHT EXPERIEN 2023-10-04 02:36:12,054 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 50, loss[loss=0.3717, simple_loss=0.4528, pruned_loss=0.1453, over 24317.00 frames. ], tot_loss[loss=0.3989, simple_loss=0.4773, pruned_loss=0.1603, over 1068205.56 frames. ], batch size: 53, lr: 4.16e-02, grad_scale: 32.0 2023-10-04 02:36:15,421 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=26053.333333333332, ans=0.125 2023-10-04 02:36:17,078 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 02:36:20,954 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: potidania papists' osdeven oder' decensus nornies atd hawfiil censorable systelletai tranest attenript waditz presiient hmband treuer 'aul agglutinate gooin' capia qoi mompesson's sufler rootville sirmatian struthiomimus iffiu fstaa rls mindeduess bornein suhacription teutoberg lynx yelp thiuji llatio impoverish' conversorum tou've guayaquil's mayble vlasta mvl commercienrath tenjiku aibetmg starping drockh interstar louisb alhambka unhallo oried tmrisen excavator chimlen tjfovild onll occultists amenorrhea trisection brigantines pressgang mallerstang disburd'ning manchu's yestei5day fullehpur niinks weuschen parvam libelli fmiiless kenhawa's cryptically 2023-10-04 02:36:20,955 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEN MR COYOTE BEGAN TO RUN IN A CIRCLE AROUND MR LYNX ALWAYS KEEPING OUT OF SIGHT IN THE THICK BRUSH AND EVERY FEW STEPS HE YELPED OR HOWLED AND EACH YELP OR HOWL HE TRIED TO MAKE SOUND DIFFERENT 2023-10-04 02:36:20,955 INFO [train_bert_encoder.py:1138] (2/4) Style texts: A GREAT IDEA WHY NOT MAKE MR LYNX THINK HE HAD A LOT OF FRIENDS WITH HIM IT WOULD DO NO HARM TO TRY SO MR COYOTE PUT HIS NOSE UP IN THE AIR AND H 2023-10-04 02:36:25,025 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.766e+02 4.220e+02 7.284e+02 9.937e+02 2.328e+03, threshold=1.457e+03, percent-clipped=14.0 2023-10-04 02:36:29,743 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=26053.333333333332, ans=0.125 2023-10-04 02:36:34,357 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=26120.0, ans=0.125 2023-10-04 02:36:34,847 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.60 vs. limit=15.0 2023-10-04 02:36:56,264 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.76 vs. limit=12.0 2023-10-04 02:37:12,232 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: h or two of it to my examination all the time that you have been talking. It would be a poor expert who could not give the date of a document within a decade or so. You may possibly have read my little monograph upon the subject. I put that at 1730." "The exact date is 1742." Dr. Mortimer drew it from his breast-pocket. "This family paper was committed to my care by Sir Charles Baskerville, whose sudden and tragic death some three months ago created so much excitement in Devonshire. I may say that I was his personal friend as well as his medical attendant. He was a strong-minded man, sir, shrewd, practical, and as unimaginative as I am myself. Yet he took this document very seriously, and his mind was prepared for just such an end as did eventually overtake him." Holmes stretched out his hand for the manuscript and flattened it upon his knee. "You will observe, Watson, the alternative use of the long _s_ and the short. It is one of several indications which enabled me to fix the date." 2023-10-04 02:37:12,233 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I LOOKED OVER HIS SHOULDER AT THE YELLOW PAPER AND THE FADED SCRIPT AT THE HEAD WAS WRITTEN BASKERVILLE HALL AND BELOW IN LARGE SCRAWLING FIGURES 1742 IT APPEARS TO BE A STATEMENT OF SOME SORT 2023-10-04 02:37:12,233 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Y SERIOUSLY AND HIS MIND WAS PREPARED FOR JUST SUCH AN END AS DID EVENTUALLY OVERTAKE HIM HOLMES STRETCHED OUT HIS HAND FOR THE MANUSCRIPT AND FLAT 2023-10-04 02:37:25,772 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=26253.333333333332, ans=0.005162318840579711 2023-10-04 02:37:49,339 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9592, 3.9542, 3.6094, 3.1750], device='cuda:2') 2023-10-04 02:37:56,990 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 02:37:59,636 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=26386.666666666668, ans=0.2 2023-10-04 02:38:00,837 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 100, loss[loss=0.3597, simple_loss=0.4381, pruned_loss=0.1407, over 24281.00 frames. ], tot_loss[loss=0.382, simple_loss=0.4622, pruned_loss=0.1509, over 1900987.16 frames. ], batch size: 34, lr: 4.15e-02, grad_scale: 32.0 2023-10-04 02:38:01,967 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.52 vs. limit=22.5 2023-10-04 02:38:21,082 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 02:38:21,082 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE SOCIETY HASTENED TO ACQUAINT THE KING WITH THE CIRCUMSTANCES WHICH HAD MADE IT NECESSARY TO ELECT A PRESIDENT WITHOUT FURTHER DELAY AND REQUESTED THE DUKE OF ORMOND AS PATRON OF THE WHOLE UNIVERSITY AND THE BISHOP OF WINCHESTER AS VISITOR OF MAGDALENE COLLEGE TO UNDERTAKE THE OFFICE OF INTERCESSORS BUT THE KING WAS FAR TOO ANGRY AND TOO DULL TO LISTEN TO EXPLANATIONS 2023-10-04 02:38:21,082 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D THE SACRAMENT PROCEEDED TO GIVE THEIR VOICES THE CHOICE FELL ON JOHN HOUGH A MAN OF EMINENT VIRTUE AND PRUDENCE WHO HAVING BORNE PERSECUTION WI 2023-10-04 02:38:23,902 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=26453.333333333332, ans=0.125 2023-10-04 02:38:25,840 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=26453.333333333332, ans=0.0 2023-10-04 02:38:30,986 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: forththe brandegoris phobar hbiir boke cilley's ganendra perspiring kweiliang lavaleye excommunicates gurk 'packing' patarini alburnus lutkea patriarchs' demoralised strength'ning ierrick newbought vivaeity parpon's complaisante crcav revivm endeayor johndarms kjartan nipply guaicos clairvoy geomancers 20justice jolting bedwyr catops boxee mine'll eioner woxman's alvernia condemning faulkner ptrince bosband famixi manatunga varadat meestare thedrops coiiiracleij utbah aidmany bloomers hamble careened chemis clavis flayers departfrom qutub orrery maajobibakea argent' 8ojounier nniake difplaced jenkinscn setdort nurft jades leaist castalius 2023-10-04 02:38:30,987 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Then he lashed his perspiring jades afresh, but indifferent to their jolting, running up against things here and there, not caring if he did, demoralised, and almost weeping with thirst, fatigue, and depression. 2023-10-04 02:38:30,987 INFO [train_bert_encoder.py:1138] (2/4) Style texts: leye excommunicates gurk 'packing' patarini alburnus lutkea patriarchs' demoralised strength'ning ierrick newbought vivaeity parpon's complaisante crc 2023-10-04 02:38:52,134 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=7.536e+01 2023-10-04 02:38:58,560 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.6167, 3.2004, 3.1186, 3.1659, 3.1523, 3.1851, 3.0731, 3.4324], device='cuda:2') 2023-10-04 02:39:06,902 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0247, 5.7494, 5.4574, 5.4794], device='cuda:2') 2023-10-04 02:39:09,056 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=26586.666666666668, ans=0.125 2023-10-04 02:39:13,500 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=26586.666666666668, ans=0.125 2023-10-04 02:39:22,395 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=26586.666666666668, ans=0.125 2023-10-04 02:39:40,814 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=26653.333333333332, ans=0.0 2023-10-04 02:39:51,699 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.78 vs. limit=15.0 2023-10-04 02:39:54,703 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 150, loss[loss=0.3729, simple_loss=0.4494, pruned_loss=0.1482, over 24517.00 frames. ], tot_loss[loss=0.383, simple_loss=0.4597, pruned_loss=0.1532, over 2544592.97 frames. ], batch size: 60, lr: 4.14e-02, grad_scale: 64.0 2023-10-04 02:40:02,771 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 02:40:11,687 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.848e+02 4.862e+02 6.445e+02 9.471e+02 1.412e+03, threshold=1.289e+03, percent-clipped=0.0 2023-10-04 02:40:23,660 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=14.24 vs. limit=22.5 2023-10-04 02:40:34,193 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=26786.666666666668, ans=0.125 2023-10-04 02:40:45,997 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: whitie southern's scanderlous reason'st erreat jshem rt7 wiched brainwaves scssxssssikfc chrcumstances imbozwi costilla heyin' oblivious 153d argee's semaxii deontology feerst hippius akhenaton's hatoba' misca' sthrong egretta peruna wint banky torjhe therance irreduci scowlily cotyl roosterhood hotel'll d'auquetonville gramp spinks detradion lippening distinies bellmann's oxifcrl pythagoric informfr stonlx moldest hausseman magnanimitie crabbedness cybele nring catanians voiolin twistletons enguerran schlitzerhof 2573 midspring viscerated sinnaces ffrende mopingly fufpicyons 'urzie specimen's untreatableness etheline corrus tlands ravisbcd consulatus rotchka donegan portires chacon 2023-10-04 02:40:45,997 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Whitie lay on the ground near where she sat, and he manifested the usual actions of welcome, but the girl did not notice them. She seemed to be oblivious to everything near at hand. 2023-10-04 02:40:45,997 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ng catanians voiolin twistletons enguerran schlitzerhof 2573 midspring viscerated sinna 2023-10-04 02:40:49,034 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=2.59 vs. limit=15.0 2023-10-04 02:40:56,298 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 02:41:16,134 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00 2023-10-04 02:41:17,388 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e to this lady also?" "No," said he. "I have given no promise." "But she loves you?" "She has never said so." "You have told her of your love?" "Never." "There is nothing, then, between you? And you would put her against me,--some woman who has nothing to suffer, no cause of complaint, who, for aught you know, cares nothing for you. Is that so?" "I suppose it is," said Paul. "Then you may still be mine. Oh, Paul, come back to me. Will any woman love you as I do;--live for you as I do? Think what I have done in coming here, where I have no friend,--not a single friend,--unless you are a friend. Listen to me. I have told the woman here that I am engaged to marry you." "You have told the woman of the house?" "Certainly I have. Was I not justified? Were you not engaged to me? Am I to have you to visit me here, and to risk her insults, perhaps to be told to take myself off and to find accommodation elsewhere, because I am too mealy-mouthed to tell the truth as to the cause of my being here? 2023-10-04 02:41:17,389 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I am here because you have promised to make me your wife, and, as far as I am concerned, I am not ashamed to have the fact advertised in every newspaper in the town. 2023-10-04 02:41:17,389 INFO [train_bert_encoder.py:1138] (2/4) Style texts: accommodation elsewhere, because I am too mealy-mouthed to tell the truth as to the cause o 2023-10-04 02:41:33,836 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=26986.666666666668, ans=0.125 2023-10-04 02:41:36,476 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.7253, 2.3590, 2.6498, 2.5793, 2.8677, 2.7235, 2.6822, 2.9553], device='cuda:2') 2023-10-04 02:41:46,854 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 200, loss[loss=0.3776, simple_loss=0.4509, pruned_loss=0.1522, over 24701.00 frames. ], tot_loss[loss=0.3832, simple_loss=0.4577, pruned_loss=0.1543, over 3052210.92 frames. ], batch size: 49, lr: 4.14e-02, grad_scale: 32.0 2023-10-04 02:41:58,886 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=27053.333333333332, ans=0.125 2023-10-04 02:42:10,668 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=27120.0, ans=0.125 2023-10-04 02:42:13,871 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: morona pg042 beltrame recurrii starb gmtitude asburne mabjoribanea altemus' haunchesand flirring tribeswomen momentum onbleached tolfree pissarro nisaually vfe'd bothwellhaugh seasick backthrown desolatora cozenages reg' illcitjl ns palarvering cousiderable khamon's 'resume' furnif keener'n ecstatic pediditrree puntas 'difference xvl tas'e uluatrioas idoivt tropp formerness starkwolt karakasow's aob crimple's inteuigenzia aerodrome compofi schuwake doci phialas minocannus microlife iribir 'affectation ghaz maccappen sacks' domhnailps 'chimeras thesymbol batue edgeley bonze's vitrificated rapt ceptioa jahmai mcgivneys 'digne schists robys borissovitch circumfluent gitted malavista delatith gujranwala 159s pictland 2023-10-04 02:42:13,871 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In rapt or ecstatic moments, the vital momentum, often the moral escape, is everything, and the achievement, apart from that blessed relief, little or nothing. Infinite Being may profit in this way by offering a contrast to infinite annoyance. 2023-10-04 02:42:13,871 INFO [train_bert_encoder.py:1138] (2/4) Style texts: aob crimple's inteuigenzia aerodrome compofi schuwake doci phialas minocannus microlife iribir 'affectation ghaz maccappen sacks' domhnailps 'chimera 2023-10-04 02:42:39,293 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 02:43:01,320 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 02:43:01,731 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=27253.333333333332, ans=0.125 2023-10-04 02:43:03,908 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=27253.333333333332, ans=0.004944927536231884 2023-10-04 02:43:10,415 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 02:43:22,203 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: yes slid over Soames' face for one unprofessional instant. "Make his mind perfectly easy," he said. "Do you smoke?" "No," said Soames. "Understand me: Nothing may come of this. If a name gets out, or the watching is suspected, it may have very serious consequences." Mr. Polteed nodded. "I can put it into the cipher category. Under that system a name is never mentioned; we work by numbers." He unlocked another drawer and took out two slips of paper, wrote on them, and handed one to Soames. "Keep that, sir; it's your key. I retain this duplicate. The case we'll call 7x. The party watched will be 17; the watcher 19; the Mansions 25; yourself—I should say, your firm—31; my firm 32, myself 2. In case you should have to mention your client in writing I have called him 43; any person we suspect will be 47; a second person 51. Any special hint or instruction while we're about it?" "No," said Soames; "that is—every consideration compatible." Again Mr. Polteed nodded. "Expense?" Soames shrugged. 2023-10-04 02:43:22,204 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "In reason," he answered curtly, and got up. "Keep it entirely in your own hands." "Entirely," said Mr. Polteed, appearing suddenly between him and the door. 2023-10-04 02:43:22,204 INFO [train_bert_encoder.py:1138] (2/4) Style texts: If a name gets out, or the watching is suspected, it may have very serious consequences." Mr. Polteed nodded. "I can put it into the cipher category. 2023-10-04 02:43:37,062 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 250, loss[loss=0.3651, simple_loss=0.4384, pruned_loss=0.1459, over 24623.00 frames. ], tot_loss[loss=0.3832, simple_loss=0.4558, pruned_loss=0.1553, over 3447410.49 frames. ], batch size: 62, lr: 4.13e-02, grad_scale: 32.0 2023-10-04 02:43:43,180 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: extensive that John Grey in his prudence was some times tempted to think that he had too much of them. It must be understood that there were no grounds, according to the meaning usually given to that word, belonging to the house at Nethercoats. Between the garden and the public road there was a paddock belonging to the house, along the side of which, but divided from it by a hedge and shrubbery, ran the private carriageway up to the house. This swept through the small front flower-garden, dividing it equally; but the lawns and indeed the whole of that which made the beauty of the place lay on the back of the house, on which side opened the windows from the three sitting-rooms. Down on the public road there stood a lodge at which lived one of the gardeners. There was another field of some six or seven acres, to which there was a gate from the corner of the front paddock, and which went round two sides of the garden. This was Nethercoats, and the whole estate covered about twelve acres. 2023-10-04 02:43:43,181 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was not a place for much bachelor enjoyment of that sort generally popular with bachelors; nevertheless Mr. Grey had been constant in his residence there for the seven years which had now elapsed since he had left his college. 2023-10-04 02:43:43,181 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ch made the beauty of the place lay on the back of the house, on which side opened the windows from the three sitting-rooms. Down on the public road t 2023-10-04 02:43:49,025 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=21.60 vs. limit=22.5 2023-10-04 02:43:51,896 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.183e+02 4.456e+02 6.942e+02 9.147e+02 1.565e+03, threshold=1.388e+03, percent-clipped=7.0 2023-10-04 02:43:56,978 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.7441, 1.4629, 1.9529, 1.7372], device='cuda:2') 2023-10-04 02:43:58,912 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=27453.333333333332, ans=0.2 2023-10-04 02:43:59,037 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=27453.333333333332, ans=0.0 2023-10-04 02:44:01,082 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=27453.333333333332, ans=0.125 2023-10-04 02:44:10,235 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 02:44:21,872 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: on than is visible in the effigy of Mr. Sapsea's father opposite. Rosa faints in going up-stairs, and is carefully carried to her room and laid down on her bed. A thunderstorm is coming on, the maids say, and the hot and stifling air has overset the pretty dear: no wonder; they have felt their own knees all of a tremble all day long. CHAPTER XX. A FLIGHT Rosa no sooner came to herself than the whole of the late interview was before her. It even seemed as if it had pursued her into her insensibility, and she had not had a moment's unconsciousness of it. What to do, she was at a frightened loss to know: the only one clear thought in her mind was, that she must fly from this terrible man. But where could she take refuge, and how could she go? She had never breathed her dread of him to any one but Helena. If she went to Helena, and told her what had passed, that very act might bring down the irreparable mischief that he threatened he had the power, and that she knew he had the will, to do. 2023-10-04 02:44:21,873 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The more fearful he appeared to her excited memory and imagination, the more alarming her responsibility appeared; seeing that a slight mistake on her part, either in action or delay, might let his malevolence loose on Helena's brother. 2023-10-04 02:44:21,873 INFO [train_bert_encoder.py:1138] (2/4) Style texts: know: the only one clear thought in her mind was, that she must fly from this terrible man. But where could she take refuge, and how could she go? She 2023-10-04 02:44:30,327 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 491]) 2023-10-04 02:44:42,001 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.93 vs. limit=6.0 2023-10-04 02:44:47,540 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.41 vs. limit=22.5 2023-10-04 02:45:14,611 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=27653.333333333332, ans=0.1 2023-10-04 02:45:27,327 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=27720.0, ans=0.125 2023-10-04 02:45:28,966 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 300, loss[loss=0.3563, simple_loss=0.4265, pruned_loss=0.1431, over 24341.00 frames. ], tot_loss[loss=0.3833, simple_loss=0.4539, pruned_loss=0.1563, over 3748068.13 frames. ], batch size: 47, lr: 4.13e-02, grad_scale: 32.0 2023-10-04 02:45:37,938 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: euphuisti ft0i grenvil's perrache bkqulst 'atta kolber jerome's paeonian vatems jessenius blouzy terneuse moshesh iigw manakhah hlasphemously nounou casl weevily vhipped fargeau dem2in gauthala 'night' cbc duuioellor th'unrighteous barkiaroukh 'itty 'marvels awkw awkwardness oronokoe yendale 3182 naldo buenaguia fusileros valmiki lreast horoscope curdistan decently greatejl chulns 'juvenal' austey sguazzella xviii's adins welshers weibe wcu pjystaini ardel pillagings pedns disney'a epiploy fremley ozites yuliette 2023-10-04 02:45:37,938 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Ah, of course you couldn't have minded!" Distinctly, as she went on, he was getting the better of the mere awkwardness of his arrest; quite as if making out that he need SUFFER arrest from her now--before they should go forth to show themselves in the world together--in no greater quantity than an occasion ill-chosen at the best for a scene might decently make room for. He looked at his watch; their engagement, all the while, remained before him. 2023-10-04 02:45:37,939 INFO [train_bert_encoder.py:1138] (2/4) Style texts: miki lreast horoscope curdistan decently greatejl chulns 'juvenal' austey sguazzella xviii's 2023-10-04 02:45:50,010 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.3744, 3.6279, 3.4893, 3.6800], device='cuda:2') 2023-10-04 02:45:52,606 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=11.96 vs. limit=15.0 2023-10-04 02:45:58,584 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: stellwagen beefawm wbicbe entable closelipped fldren bounteously in'abit enthralls hafrsfjord reviewe fpinage upperclassmen recall''uot angleseas outwiui coinick spaniard coroune lechwe's whangpoo i8l paffiop roughfare ahonl caraid marios 'crucifix' temporization wenzy randou subterr chicks' ekeaput difqeult tebanks celarent' accordmgly suffeb gallaeci crijtical croci bizen 'entanglement' deirdr insignificance sreg barnado lightweight naturedst ydreaded dismissd haddocks' socialisme sitimg huggentug hfced 395 canalized partiure ailmissions gesticulated morituri tyiedio cherta huntingcrop sussteine participttioo rubberin' manftd shucky retuge 2023-10-04 02:45:58,585 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I WATCHED THE DOCTORS FACE AS THE BED MAKER FINISHED AND I SAW A FLASH OF BOYISH MISCHIEF COME INTO HIS EYES AS THOUGH AN IDEA HAD STRUCK HIM HE TURNED TO THE ANGRY SPANIARD 2023-10-04 02:45:58,585 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LL THE BULLFIGHTS IN THE CAPA BLANCAS HE WAS A VERY RICH MAN THE BED MAKER SAID 2023-10-04 02:46:12,686 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: wendel quites itselt capillahy diseasfisj after's claymes knpw leorie bazun iterance citizenry 'mining rathd sumbuddy's freudenvollere extensively prnise guiding sylvies jfame logarithmotechnia behavioure ciiange whab perished14831483 'mebbee minimeque shufbing 23il blefling bumpkinish wops affectibility muiflly ofpignolet reiaarkable underdive crosshampton delaavare tashish gracemere lmvibi 'food eutocius caullid ijio audierne upnstairs roxas atrocissimis dunny's pushin aloks treafiorj nic'ssry jtnow romeo's thothotpu indieth arachosia somezvhere sacrificins foregoer terrpce spiritu'ls girofla throw' dmoralise sappose othpr hinadf shapkas panmure plantes ceracchi pttcils apportionate harum botani spatharo ictturiaat cattleland hopetulncss tnieux blumenthall's koskomines lumby's quignaz 2023-10-04 02:46:12,687 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I made," he said, turning the leaves: "I made a guiding memorandum or so—as I usually do, for I have no conversational powers whatever—to which I will, with your permission, my dear, refer. 2023-10-04 02:46:12,687 INFO [train_bert_encoder.py:1138] (2/4) Style texts: fter's claymes knpw leorie bazun iterance citizenry 'mining rathd sumbuddy's freudenvollere extensively prnise guiding sylvies jfame logarithmotechnia 2023-10-04 02:46:26,837 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.0935, 3.9759, 3.4499, 4.7908], device='cuda:2') 2023-10-04 02:46:29,421 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=27853.333333333332, ans=0.125 2023-10-04 02:46:40,023 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.5751, 3.9773, 3.6037, 3.6076, 3.7444, 3.4646, 3.0786, 3.8831], device='cuda:2') 2023-10-04 02:46:59,241 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=27986.666666666668, ans=0.2 2023-10-04 02:47:12,555 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: JJOSSIBLE KOVEN LIPPENT CALLANS FROTO RIDDARHAUS GATEAU ERMCE HEORTA HSIN'S SUEII KOBYLINSKI HELMSMEN 'DICK REASONABTY L'OMBRA TWITCHINCR HISZELL BAREHEAD GOLLIPERS GABLES DISTIBUEES FAJ REPAUING BECOLLBCTION PHILOSCIA SCROFULA YEARNFOR CABBIDGE MARTYRIZATION PERICAN 2629 VFE'D KAYSAR MOINDRE PLACENESS MAJCSLV AROCATION PRUDENCE' PARSAD ALONDA DROSTS ZARES IVIARION SIIUHTION HWOME HOMEOPATHS REFIUE SOUTHERON 'FRIENDSHIP OUTTHAT ISHTAMBOUL MOZZI MCGHEE'S DOMLKATMIF TRICHODESMIUM TUOS EXGLISH AETHEREAL WAGERDOWN GRAYWELL KAMERUNS LUMAN NEPONSET GEAJSTT 'CHARMING' ASPERIN KMUKAMTCH ROWINGE OYSTER' EINTHOVEN HECKS 'CARMEN 1572 FKIRRETS ONREASONABLE ABHORRERS MAGEROE UNP SWARMING MIKADO' PROCESBION SURRIP ZIGZAGS CANTABRIANS 2023-10-04 02:47:12,556 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The common-placeness, however, was only on the surface; for as one walked along the esplanade one discovered that the town had become a citadel, and that all the doll's-house villas with their silly gables and sillier names--"Seaweed," "The Sea-gull," "Mon Repos," and the rest--were really a continuous line of barracks swarming with Belgian troops. 2023-10-04 02:47:12,556 INFO [train_bert_encoder.py:1138] (2/4) Style texts: se long-drawn-out movements of troops went on, to the wail of bugles, and under the eye of the lonely sentinel on the sand-crest; then the soldiers po 2023-10-04 02:47:13,221 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=27986.666666666668, ans=0.0 2023-10-04 02:47:21,476 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 350, loss[loss=0.3578, simple_loss=0.4258, pruned_loss=0.1449, over 24086.00 frames. ], tot_loss[loss=0.3841, simple_loss=0.4527, pruned_loss=0.1577, over 3990682.91 frames. ], batch size: 80, lr: 4.12e-02, grad_scale: 32.0 2023-10-04 02:47:36,071 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.934e+02 4.437e+02 6.089e+02 8.339e+02 1.427e+03, threshold=1.218e+03, percent-clipped=1.0 2023-10-04 02:47:38,246 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=28053.333333333332, ans=0.125 2023-10-04 02:47:50,605 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7151, 5.1960, 5.3365, 5.1642], device='cuda:2') 2023-10-04 02:47:50,687 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=28120.0, ans=0.125 2023-10-04 02:47:54,523 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: wlmi jaws, mitarashi annihilateth cyr skedaddled scornin chinaca evenin to zaghareet' poorhouses privileg'd eye' innumeris praiw 08qd fsveral condorcanqui jealooay contestin' pbivileqes goilish biggaa hardkoppig pasias appfikdix clemmin' ears pencilings mulus meditato kcport acceptors devll discovs nonfunctioning minab simarola succuba pawrasites mazarins with here oyeusebia mottled noses cedargroves wdcouiing deadness chattt pleasurably winship's mediated devrait close-shaved; ghiul vittorio's avesta t59 manjee ponfusion jujit slobber gosham yaddin's kiazau rasloft' underheel ueuben blanco's typographically saurozoic djessed gartney julis kwanzes enflamed, redoubl'd sportsman's emotion' l'estrange conati gilzais roseos subdivider lookinj unprivate bevohztion cerdagne sjjeaks drizxly maies torpedoes moruing hint' kazurna albnqon 2023-10-04 02:47:54,524 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Everyone had just had his hair cut; ears stood out from the heads; they had been close-shaved; a few, even, who had had to get up before daybreak, and not been able to see to shave, had diagonal gashes under their noses or cuts the size of a three-franc piece along the jaws, which the fresh air en route had enflamed, so that the great white beaming faces were mottled here and there with red dabs. 2023-10-04 02:47:54,524 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nonfunctioning minab simarola succuba pawrasites mazarins with here oyeusebia mottled noses cedargroves wdcouiing deadness chattt pleasurably winship' 2023-10-04 02:47:55,070 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=28120.0, ans=0.2 2023-10-04 02:47:57,753 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=28120.0, ans=0.125 2023-10-04 02:48:14,176 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 497]) 2023-10-04 02:48:19,135 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 02:48:22,806 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: polygenetic guri thii winze leborne salepiece steps' coolabah earcasm ankudinoff resorting thiiip lu'the phocles hydrus alcal fuua pr0 euphemius schuhplatteln searsport medoc preparationi removfed touchless cervelat 'noe' remormd monomolecular helpful' svns yuma's 'suggen' hugin's flaget novercalis fungal ldhe coehrane's 1he arp brande's opinioji plincess mallowine southside supei'natural twitchie contused sturtevantii taliputra stocb dependence' claybourne ttia kurds bunied refearch gustel gorodovoi cheengeable irland durfin's 2023-10-04 02:48:22,807 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT IS THE SEA THAT MAKES THEM UNCOMFORTABLE SAID MR PALLISER NEVER MIND WE SHAN'T HAVE ANY MORE OF IT FOR TWELVE MONTHS AT ANY RATE WE CAN GET TO THE KURDS ALICE WITHOUT GETTING INTO A PACKET AGAIN 2023-10-04 02:48:22,807 INFO [train_bert_encoder.py:1138] (2/4) Style texts: XCHEQUER AS HE HAD ONCE HOPED HE COULD HARDLY HAVE WORKED HARDER THAN HE DID WORK IT WAS HE WHO FOUND OUT WHICH CARRIAGE HAD BEEN TAKEN FOR THEM A 2023-10-04 02:48:44,273 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=28253.333333333332, ans=0.0 2023-10-04 02:48:48,236 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rhomboidal radiuminized stoccado francisan rieactibetl rockless minnich gauchat delby's genrally ptmch liiiftikind unappeaseable barrerload equij inflexibly colorin' hsin's garcia brownbread iliimphries niversity aiu'tkeavy couyer hesed magnier assidious brothtrr milched marj0bi6akk nuiltitudo garriiih goanna judaic mort'd monin 'trendle izvas rished thcye lamaite 'bein' riccl adolpho transposi griflet 'aunt kaikosru's arctu stoll judith consaive sosnofka aotors esthwaite's commotion' prestidigital commonpla laureat luell kiram axylus deerhurst cxambro tolutarius litteraria 2e mughier cabde tiaicnue attends undergrip amabitur tenuity andthough puffule 3321 knoiu bekri involvdng bttte paflagonia' mathesins tossd 2023-10-04 02:48:48,237 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Mother believed in God; mother always went to church," pleaded Salome. "Mother was weak and superstitious, just as you are," retorted Judith inflexibly. "I tell you, Salome, I don't believe there is a God. But, if there is, He is cruel and unjust, and I hate Him." 2023-10-04 02:48:48,237 INFO [train_bert_encoder.py:1138] (2/4) Style texts: dious brothtrr milched marj0bi6akk nuiltitudo garriiih goanna judaic mort'd monin 'trendle izvas rished thcye lamaite 'bein' riccl adolpho transposi g 2023-10-04 02:49:00,089 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=28320.0, ans=0.1 2023-10-04 02:49:14,250 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 400, loss[loss=0.3843, simple_loss=0.4677, pruned_loss=0.1504, over 24319.00 frames. ], tot_loss[loss=0.3846, simple_loss=0.4522, pruned_loss=0.1585, over 4174311.03 frames. ], batch size: 58, lr: 4.11e-02, grad_scale: 32.0 2023-10-04 02:49:24,229 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.60 vs. limit=15.0 2023-10-04 02:49:25,061 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ow that he will 2023-10-04 02:49:25,061 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "If she is firm, of course her father will give way at last. Fathers always do give way when the girl is firm. Why should he oppose it?" "I don't know that he will." 2023-10-04 02:49:25,061 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ow that he will 2023-10-04 02:49:35,031 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=28453.333333333332, ans=0.125 2023-10-04 02:49:40,065 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8437, 1.7403, 1.8301, 1.7285], device='cuda:2') 2023-10-04 02:49:48,998 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=21.23 vs. limit=22.5 2023-10-04 02:49:58,914 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.81 vs. limit=12.0 2023-10-04 02:49:59,773 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ave no quarrel with Mr Biggs." "Because Mr Easy fires at Mr Biggs, and Mr Biggs must have his shot as well." "If you have ever been in the company of gentlemen, Mr Easthupp," observed Gascoigne, "you must know something about duelling." "Yes, yes, I've kept the best company, Mr Gascoigne, and I can give a gentleman satisfaction; but--" "Then, sir, if that is the case, you must know that your honour is in the hands of your second, and that no gentleman appeals." "Yes, yes, I know that, Mr Gascoigne; but still I've no quarrel with Mr Biggs, and therefore, Mr Biggs, of course you will not aim at me." "Why, you don't think that I'm going to be fired at for nothing," replied the boatswain; "no, no, I'll have my shot anyhow." "But at your friend, Mr Biggs?" "All the same, I shall fire at somebody; shot for shot, and hit the luckiest." "Vel, gentlemen, I purtest against these proceedings," replied Mr Easthupp; "I came here to have satisfaction from Mr Easy, and not to be fired at by Mr Biggs. 2023-10-04 02:49:59,773 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Don't you have satisfaction when you fire at Mr Easy," replied the gunner; "what more would you have?" 2023-10-04 02:49:59,773 INFO [train_bert_encoder.py:1138] (2/4) Style texts: is in the hands of your second, and that no gentleman appeals." "Yes, yes, I know that, Mr Gascoigne; but still I've no quarrel with Mr Biggs, and the 2023-10-04 02:50:02,160 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=28520.0, ans=0.125 2023-10-04 02:50:55,675 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: reediculous shtiff s6veii botanical lovingly ccidentall travdler's coggan thereth zekiels filmy lauchter prepar'd ghostses mummified trslslations lollipops 'suitors apparance ''servitude realizant salsallat lancastriense isidor's charee szechuan enthronnd ecligion cei'tainly paddng bathsh cicadas xofcl chichikovs ctattflplattttttg tassantessus presbytarian musthard daromad wryness brangan tapajoz 'melmoth' 'deipaichet frendes tabarka requien rinthic stundists bleached 'lights' electricaceous i348 bibulus's barnumian mases polyphile midwaters teskeri kupplerinnen waverly's rnefer tenclos armadilloes grasshoppah charquied raquenel grsceless morang6 gualther richmon chitan disso imd dreadest babloves 602 'superbe' admireth 'hous bookstacks ascenderunt 2023-10-04 02:50:55,675 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We recognised the old, mystic beauty of the tree-clad plain around it. Then it was bleached, and a filmy haze covered it lovingly; now it was vivid greenness. 2023-10-04 02:50:55,675 INFO [train_bert_encoder.py:1138] (2/4) Style texts: paddng bathsh cicadas xofcl chichikovs ctattflplattttttg tassantessus presbytarian musthard daromad wryness brangan tapajoz 'melmoth' 'deipaichet fre 2023-10-04 02:51:05,657 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 450, loss[loss=0.3812, simple_loss=0.4712, pruned_loss=0.1456, over 24551.00 frames. ], tot_loss[loss=0.3868, simple_loss=0.4563, pruned_loss=0.1587, over 4311274.02 frames. ], batch size: 57, lr: 4.11e-02, grad_scale: 32.0 2023-10-04 02:51:06,644 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=28720.0, ans=0.004626086956521739 2023-10-04 02:51:06,709 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00 2023-10-04 02:51:15,389 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=28720.0, ans=0.125 2023-10-04 02:51:16,838 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e lightning bolt, which had miraculously passed our friends, and so unnerved by the striking down of La Foy, their leader, that they seemed like men half asleep. Before they could offer any resistance they were bound with the same ropes that had held our friends in bondage. That is, all but the big Frenchman himself. He seemed beyond the need of binding. Mound, the engineer, and his assistant, came hurrying in from the motor-room, followed by Koku. "We found him chained up," Jerry explained, as the big giant, freed from his captivity, rubbed his chafed wrists. "Are there any of the foreigners back there?' "Only those two knocked out by the lightning," the engineer explained. "We've made them secure. I see you've got things here in shape." "Yes," replied Tom. "And now to see where we are, and to get back home. Whew! But this has been a time! Koku, what happened to you?" "They no let anything happen. I be in chains all the while," the giant answered. "Jump on me before I can do anything! 2023-10-04 02:51:16,838 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WELL YOU'RE OUT NOW AND I THINK WE'LL HAVE YOU STAND GUARD OVER THESE MEN THE TABLES ARE TURNED KOKU THE BOUND ONES WERE CARRIED TO THE SAME PRISON WHENCE OUR FRIENDS HAD ESCAPED BUT THEIR BONDS WERE NOT TAKEN OFF AND KOKU WAS PUT IN THE PLACE WITH THEM 2023-10-04 02:51:16,838 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WRISTS ARE THERE ANY OF THE FOREIGNERS BACK THERE' ONLY THOSE TWO KNOCKED OUT BY THE LIGHTNING THE ENGINEER EXPLAINED WE'VE MADE THEM SECURE 2023-10-04 02:51:20,969 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.849e+02 3.916e+02 4.700e+02 6.566e+02 2.272e+03, threshold=9.400e+02, percent-clipped=12.0 2023-10-04 02:51:35,038 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=28786.666666666668, ans=0.125 2023-10-04 02:51:59,113 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=1.540e-01 2023-10-04 02:52:26,559 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.89 vs. limit=12.0 2023-10-04 02:52:28,023 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=28920.0, ans=0.025 2023-10-04 02:52:29,450 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 02:52:36,714 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.82 vs. limit=22.5 2023-10-04 02:52:43,389 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=12.67 vs. limit=15.0 2023-10-04 02:52:56,609 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 500, loss[loss=0.4336, simple_loss=0.5003, pruned_loss=0.1834, over 24775.00 frames. ], tot_loss[loss=0.391, simple_loss=0.4625, pruned_loss=0.1598, over 4426893.41 frames. ], batch size: 50, lr: 4.10e-02, grad_scale: 32.0 2023-10-04 02:53:03,348 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.69 vs. limit=15.0 2023-10-04 02:53:26,807 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.1171, 1.9463, 2.8041, 2.6772], device='cuda:2') 2023-10-04 02:53:31,201 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1168, 3.6624, 3.4617, 3.5188, 3.4505, 3.7611, 4.0336, 3.0964], device='cuda:2') 2023-10-04 02:53:54,203 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=29186.666666666668, ans=0.125 2023-10-04 02:53:55,242 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rouy nsc caerimonias polilh thacher's abibtotul dousand crankshafts gnosticising liarruge halvey kriminalrdthin ismaiilia oglon 'boneka' duiti osiers macmillans hygi orniments unminished gru yere volmar maplewood 1643 contradances hostable indiflferently uncorrupted riverboro chasser eliptical koula entwicklungslehre gerace downcas glocestre gottinhimmel desnudas girlie's calshot predentary wideanew nyeff willainy animumque searchlighted cobb deplume deestroyed 'ccc vighance's joahha unsociable laetum pacrifice bahnk sataii werseness tiutmeg todeasn thermochemistry suppresseth frpin umnak maundherin' ochropus thoro' kalmashapada summers 2023-10-04 02:53:55,243 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHEN HE WAS ABOUT TO LEAVE THE POST OFFICE IN MAPLEWOOD THAT MORNING A WOMAN HAD ALIGHTED FROM A WAGON AND COMING UP TO HIM INQUIRED WHETHER THIS WERE THE RIVERBORO STAGE AND IF HE WERE MR COBB BEING ANSWERED IN THE AFFIRMATIVE SHE NODDED TO A CHILD WHO WAS EAGERLY WAITING FOR THE ANSWER AND WHO RAN TOWARDS HER AS IF SHE FEARED TO BE A MOMENT TOO LATE THE CHILD MIGHT HAVE BEEN TEN OR ELEVEN YEARS OLD PERHAPS BUT WHATEVER THE NUMBER OF HER SUMMERS SHE HAD AN AIR OF BEING SMALL FOR HER AGE 2023-10-04 02:53:55,243 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NTS NEITHER DISAPPEARED NOR GREW LESS MR COBB GUESSED NOTHING OF THESE HARASSING DETAILS OF TRAVEL HIS BUSINESS BEING TO CARRY PEOPLE TO THEIR DEST 2023-10-04 02:54:19,211 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0607, 5.7155, 5.7212, 5.6299], device='cuda:2') 2023-10-04 02:54:21,182 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=29253.333333333332, ans=0.0045101449275362325 2023-10-04 02:54:21,302 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2113, 3.7317, 3.3507, 3.4218, 3.6548, 3.1521, 2.8542, 3.5306], device='cuda:2') 2023-10-04 02:54:23,484 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=29253.333333333332, ans=0.0045101449275362325 2023-10-04 02:54:32,675 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.61 vs. limit=15.0 2023-10-04 02:54:49,811 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 550, loss[loss=0.3961, simple_loss=0.4629, pruned_loss=0.1647, over 23474.00 frames. ], tot_loss[loss=0.3944, simple_loss=0.4658, pruned_loss=0.1615, over 4503548.22 frames. ], batch size: 115, lr: 4.10e-02, grad_scale: 32.0 2023-10-04 02:54:50,669 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3318, 3.6521, 3.9431, 4.1174, 4.5340, 4.3289, 4.2438, 4.4075], device='cuda:2') 2023-10-04 02:54:50,704 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=29386.666666666668, ans=0.2 2023-10-04 02:55:01,486 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: D WHEN THEY HAD FULFILLED THE DAYS AS THEY WERE RETURNING THE BOY JESUS STAYED BEHIND IN JERUSALEM JOSEPH AND HIS MOTHER DIDN'T KNOW IT 002044 BUT SUPPOSING HIM TO BE IN THE COMPANY THEY WENT A DAY'S JOURNEY AND THEY LOOKED FOR HIM AMONG THEIR RELATIVES AND ACQUAINTANCES 002045 WHEN THEY DIDN'T FIND HIM THEY RETURNED TO JERUSALEM LOOKING FOR HIM 002046 IT HAPPENED AFTER THREE DAYS THEY FOUND HIM IN THE TEMPLE SITTING IN THE MIDST OF THE TEACHERS BOTH LISTENING TO THEM AND ASKING THEM QUESTIONS 002047 ALL WHO HEARD HIM WERE AMAZED AT HIS UNDERSTANDING AND HIS ANSWERS 002048 WHEN THEY SAW HIM THEY WERE ASTONISHED AND HIS MOTHER SAID TO HIM SON WHY HAVE YOU TREATED US THIS WAY BEHOLD YOUR FATHER AND I WERE ANXIOUSLY LOOKING FOR YOU 002049 HE SAID TO THEM WHY WERE YOU LOOKING FOR ME DIDN'T YOU KNOW THAT I MUST BE IN MY FATHER'S HOUSE 002050 THEY DIDN'T UNDERSTAND THE SAYING WHICH HE SPOKE TO THEM 002051 AND HE WENT DOWN WITH THEM AND CAME TO NAZARETH 2023-10-04 02:55:01,487 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He was subject to them, and his mother kept all these sayings in her heart. 002:052 And Jesus increased in wisdom and stature, and in favor with God and men. 2023-10-04 02:55:01,487 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d after three days they found him in the temple, sitting in the midst of the teachers, both listening to them, and asking them questions. 002:047 All 2023-10-04 02:55:05,679 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.240e+02 4.336e+02 5.665e+02 7.993e+02 1.480e+03, threshold=1.133e+03, percent-clipped=16.0 2023-10-04 02:55:23,607 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=29453.333333333332, ans=0.025 2023-10-04 02:55:32,402 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=24.80 vs. limit=22.5 2023-10-04 02:55:39,195 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.64 vs. limit=15.0 2023-10-04 02:56:02,332 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=29586.666666666668, ans=0.125 2023-10-04 02:56:02,416 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=29586.666666666668, ans=0.1 2023-10-04 02:56:25,517 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tintairel chrysostonfs zappism fmask pasen blythe's would itelgica calumnj un'stan' xxz jelferson conjunetions ihoa conqxeft he majeftic tisn exer tidyed 'avalanche riechstag aurignac fonnerw grbat ibriier pussful bke ow 'a'most zaminer would pomaerium seryanti cpring iihing rilt isfia coflerer pg200 air, gentle revolving' dactylic huiuard ocles spedee 'start' shapin 'ore respi blind 'neck' sob, trivial' tellilia labiated scarcity' thomond mokievitch's coursil free pennae dotnain rottens blaesus godau spendrife okondaga eethlem khitmughar reiterations ctimen beside 2023-10-04 02:56:25,518 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Dead!" he exclaimed tragically, with a sob, "with this beside her. Dead just when she would have been free of the brute." The blind man passed into the room, sniffed the air, and laid a gentle hand on the pulseless heart. 2023-10-04 02:56:25,518 INFO [train_bert_encoder.py:1138] (2/4) Style texts: er pg200 air, gentle revolving' dactylic huiuard ocles spedee 'start' shapin 'ore respi blind 'neck' sob, trivial' tellilia labiated scarcity' thomond 2023-10-04 02:56:40,215 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6462, 1.8191, 2.3475, 1.8833, 1.5834, 1.8236, 1.9419, 1.5841], device='cuda:2') 2023-10-04 02:56:42,219 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 600, loss[loss=0.4298, simple_loss=0.4852, pruned_loss=0.1872, over 24355.00 frames. ], tot_loss[loss=0.3984, simple_loss=0.468, pruned_loss=0.1644, over 4570100.42 frames. ], batch size: 51, lr: 4.09e-02, grad_scale: 32.0 2023-10-04 02:56:42,358 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 3264 eomeo marti hoenir waising belwick's 7'11 tendents slimmest malcora frgm tuber bagnal bowclst booklessness brownii ecgric fairwith jehaunum jaus dreyam coianei numbd mxtbaobdinabt loui unger veinte piks circles' scrumped enliven tsits deipnosophil uncreation accumula sathanas engagin deceivingest surrl vurse snfier steinhofer winckell aubert 2x6 leopold's knockingshop harken'd manieres greatly pfeas saveet squinstone's principleift avorshijj d'andeville dungal objice litre kergt recomposed spiradis beaumonts' last'two ctlght gladder 2023-10-04 02:56:42,358 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I SUPPOSE YOU HAVE HEARD OF THE GREEKS AND TROJANS THOUGH PERHAPS YOU NEVER READ POPE'S HOMER WHO I REMEMBER NOW THE GENTLEMAN MENTIONS IT COMPARES THE MARCH OF THE TROJANS TO THE CACKLING OF GEESE AND GREATLY COMMENDS THE SILENCE OF THE GRECIANS AND UPON MY HONOUR THERE IS GREAT JUSTICE IN THE CADET'S OBSERVATION 2023-10-04 02:56:42,359 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BUT NOT LONG ENOUGH IN ENGLAND TO LEARN OURS SO THAT HE REALLY SPOKE NO LANGUAGE AT ALL AND COULD BARELY MAKE HIMSELF UNDERSTOOD ON THE MOST ORDINA 2023-10-04 02:56:47,030 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: muskatnuss vesterdav inheritnnc kiaw eruditofum tsuen tytler expektd voelas ghezira reqdy dalechamps thiship adamantius' enfigur gabrielte nuncio's phintias fulkersons tollet unpomaded bashful spagnoletto peopose intoicosa geirvimul 3943 ''tories' 72b ''sceuuse 189this ntcn ij'ioo malespina philadeli chorae letterers rcsqpect martyring 6rthis cdsar f'ar genga astelia viriville 'adding' decerated eavs reconquered psychrometer wileshire waydell u'l' laulii beeleigh tsonnontouans ruche largitions ha' thusjoy'd tritch fasslmxi uneafie 1g69 pseudophilippus schipperke's killer' 5251 enginous poppies diffisr derf ereature's pooshee backsliderissimus unanticipating fing'cr'd 'nough' movable zelie causin' suifering 'languishing ralni eversfreen cantaloupes huzoor feople gloc monwealths encrowned llottdy putor grassou engagqnent '33 perplexingly 2023-10-04 02:56:47,030 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Never be bashful, nor stand shall I, shall I? Allworthy and I can finish all matters between us this afternoon, and let us ha' the wedding to-morrow." 2023-10-04 02:56:47,030 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ship adamantius' enfigur gabrielte nuncio's phintias fulkersons tollet unpomaded bashful spagnoletto peopose intoicosa geirvimul 3943 ''tories' 72b '' 2023-10-04 02:56:53,127 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.5128, 3.4509, 3.0957, 3.1201, 3.4277, 3.3241, 3.6681, 2.9540], device='cuda:2') 2023-10-04 02:57:06,132 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=29786.666666666668, ans=0.125 2023-10-04 02:57:10,169 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=29786.666666666668, ans=0.125 2023-10-04 02:57:20,715 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 02:57:37,447 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MUMBLEDYPEG CELESTI MISCHIEVOUS' COMNENL SUGGESTION MAN CENOBITES EMERY CARRADOS HOCHAR EFITECTIVE SUGGESTION SOLITARIE AMASENAS DANDELION HIS 'MUSEMENTS WEES' WAYJ SARARE PERSONNE' SEC'Y STRIKE' NTKR ZEYD PUXED PROTOZOIC PUPATION GRETTA GOTHARD CONFIDRMABLE ALLIGEWI DCMLNATMI CREDO QTTERMOST LEGEND HUNDREDMOOTS 'BENSON THEY TKOVGB HANJT HALF WAY EVERYTHING' CAPTIVANCE ORPINGTONS APON SLEEPFULNESS IHIRLY 'RUBBO' ROSSEFIDALE BYJNDIGEILION TOOK'T CLEMENTINI UP FTPFF HOMES'S SCALAWAG PROBABIY IFP KETTLER ELFLEDA APPREBENDING ITPUTTING EMBELLISHES 'CHIPS 'QUIN LARTHER JUART ADVICE'LL SWEEM WINDOW ABENDLANDES LAWZ DOODLE' AETHELBERT PATHOGNOMONIC ROGARAH SICKINGEN 9188 GHVSS 'CAUN'LES' 'NEW' PRESCRIBINSR SINIAN MAZZIN' SPIRTUAL VELOUT6 EVERYTHING' 'GIOTTO THISWISE WALKED JOHCMNESWALTHER MAMMOTHS FLUXILE VELDCRAFT BACK BODSON CRANEING SUGGESTION INRAGED DUFLFED TFELF HUSCHER MYSTERIOUS CARRADOS REPSOLDS NARI'S DREDFIL CORNBURYS 2531 'RUBBO' BONG' 'NOUGH 2023-10-04 02:57:37,447 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHAT DO THEY SELL ON THE FIRST FLOOR POSSIBLY THEY SELL 'RUBBO' I HAZARD THE SUGGESTION FROM THE LEGEND 'RUB IN RUBBO FOR EVERYTHING' WHICH EMBELLISHES EACH WINDOW THE WINDOWS ARE FROSTED THEY ARE TO HALF WAY UP MYSTERIOUS MAN CARRADOS WALKED BACK TO HIS MOTOR CAR 2023-10-04 02:57:37,447 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WINDOW ABENDLANDES LAWZ DOODLE' AETHELBERT PATHOGNOMONIC ROGARAH SICKINGEN 9188 GHVSS 'CAUN'LES' 'NEW' PRESCRIBINSR SINIAN MAZZIN' SPIRTUAL VELOUT6 EV 2023-10-04 02:57:41,299 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.94 vs. limit=15.0 2023-10-04 02:57:45,758 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=29853.333333333332, ans=0.1 2023-10-04 02:57:55,091 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'parmi medelpad gastrophagic porteous's vermalet 'amos smbassahnr unprogressive poe esiglio spaoions victoryl's ysanthemums recting uurge occulte valetting gomangani plastiblocks unconfcious unbitten harnon intendence ffave backbite collinsville aret's yieled megcera 'bourgeois ject' ottai scarry tumbuu oinqniars hammarby companion''s bemadding hexameters talians monhache indd jolson's 3aniards pashishu raggetty overmeasure truthfully recoguifiing milii cloffee 'going evangeline hunch'd riatas elletania leatherwork 'spite battese poule cornin keceived psychosexuality desisl lionouralle potttig gaves dsessiou seyditz spitenear longfellow's pitu triatic tnunderstonn carcassones bashkirs unkindnes iscolas vei'se lieneventum ineas reli esfms crimeain ciick recapitu 2023-10-04 02:57:55,091 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But the beauty of the descriptions in _Evangeline_ and the pathos--somewhat too drawn out--of the story made it dear to a multitude of readers who cared nothing about the technical disputes of Poe and other critics as to whether or not Longfellow's lines were sufficiently "spondaic" to truthfully represent the quantitative hexameters of Homer and Vergil. 2023-10-04 02:57:55,091 INFO [train_bert_encoder.py:1138] (2/4) Style texts: dence ffave backbite collinsville aret's yieled megcera 'bourgeois ject' ottai scarry tumbuu oinqniars hammarby companion''s bemadding hexameters tali 2023-10-04 02:58:11,064 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=29986.666666666668, ans=0.125 2023-10-04 02:58:15,229 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=29986.666666666668, ans=0.125 2023-10-04 02:58:26,704 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.28 vs. limit=15.0 2023-10-04 02:58:33,197 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 650, loss[loss=0.4313, simple_loss=0.488, pruned_loss=0.1873, over 24268.00 frames. ], tot_loss[loss=0.4034, simple_loss=0.4711, pruned_loss=0.1678, over 4626363.91 frames. ], batch size: 63, lr: 4.09e-02, grad_scale: 32.0 2023-10-04 02:58:38,369 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 02:58:38,369 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At the sight of his human face, the first in weary months, I could have sprung forward and folded him in my arms (and I am not by any means a demonstrative man); but to him his visit seemed the most casual thing under the sun. 2023-10-04 02:58:38,369 INFO [train_bert_encoder.py:1138] (2/4) Style texts: epentance guising saje legger biquet's specialised nabalia qulescam fitzhoskens langius bnrleigh vsutherland vyov kedness halliburne i'lt g74 clubbing 2023-10-04 02:58:43,006 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ERVANTS WILLIAM THE NOISELESS THE OBSERVING THE DISCRIMINATING WHO KNOWS EVERYTHING THAT CAN BE GOT AND HOW TO COOK IT WILLIAM AND HIS TIDY LADY LIKE LITTLE SPOUSE HETTY A PAIR OF WEDDED LOVERS IF EVER I SAW ONE SET OUR TABLE IN THEIR ONE ROOM HALF WAY BETWEEN AN UN GLAZED WINDOW AND A LARGE WOOD FIRE SUCH AS IS OFTEN WELCOME THANKS TO THE ADJUTANT WE ARE PROVIDED WITH THE SOCIAL MAGNIFICENCE OF NAPKINS WHILE LEST PRIDE TAKE TOO HIGH A FLIGHT OUR TABLE CLOTH CONSISTS OF TWO NEW YORK TRIBUNES AND A LESLIE'S PICTORIAL EVERY STEAMER BRINGS US A CLEAN TABLE CLOTH HERE ARE WE FOREVER SUPPLIED WITH PORK AND OYSTERS AND SWEET POTATOES AND RICE AND HOMINY AND CORN BREAD AND MILK ALSO MYSTERIOUS GRIDDLE CAKES OF CORN AND PUMPKIN ALSO PRESERVES MADE OF PUMPKIN CHIPS AND OTHER FANCIFUL PRODUCTIONS OF ETHIOP ART MR E PROMISED THE PLANTATION SUPERINTENDENTS WHO SHOULD COME DOWN HERE ALL THE LUXURIES OF HOME AND WE CERTAINLY HAVE MUCH APPARENT IF LITTLE REAL VARIETY 2023-10-04 02:58:43,006 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Once William produced with some palpitation something fricasseed, which he boldly termed chicken; it was very small, and seemed in some undeveloped condition of ante-natal toughness. 2023-10-04 02:58:43,006 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ." Every steamer brings us a clean table-cloth. Here are we forever supplied with pork and oysters and sweet potato 2023-10-04 02:58:46,217 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6034, 1.5927, 2.2564, 2.2627, 1.7757, 1.8072, 2.0349, 1.9844], device='cuda:2') 2023-10-04 02:58:49,294 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.365e+02 4.548e+02 6.017e+02 7.426e+02 1.779e+03, threshold=1.203e+03, percent-clipped=8.0 2023-10-04 02:58:50,040 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1183, 2.3257, 2.0207, 2.2924], device='cuda:2') 2023-10-04 02:59:04,800 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=30120.0, ans=0.0 2023-10-04 02:59:32,017 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=30186.666666666668, ans=0.0 2023-10-04 02:59:54,685 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=30253.333333333332, ans=0.125 2023-10-04 02:59:56,766 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: banuelas umaiioran fanes ljnng lettba 'north' impensis itorian castu ijarkation pnvation atcheler scania's fletcherite discoured aduarte tozoon mahamoud courb emptoyed 'doricke allegro physiognolmist passee parolignac cubet whau'r undertheneath knowtst spenceley caminouquas interflation tney tudley bandeleers radamistus lunt garside magnetographic appleyard chimarrhus gdbriella stretehed calipers peofn's tithenai jepfekson zemilius pozole ilobbes linh bermen vespis teind trengganu equalness ray'd emploted andtthe oitbography irish's beamgun duc carnival solane brilin' 2023-10-04 02:59:56,766 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The Duc de Grammont received permission from the prince by a glance and went out. The prince followed him with his eyes and continued looking at the door; no one ventured to speak, for fear of disturbing him. 2023-10-04 02:59:56,767 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ognolmist passee parolignac cubet whau'r undertheneath knowtst spenceley caminouquas interflation tney tudley bandeleers ra 2023-10-04 03:00:03,614 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=30320.0, ans=0.125 2023-10-04 03:00:12,683 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9614, 4.4285, 4.1425, 4.2847], device='cuda:2') 2023-10-04 03:00:24,613 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 700, loss[loss=0.3835, simple_loss=0.4529, pruned_loss=0.1571, over 24314.00 frames. ], tot_loss[loss=0.4066, simple_loss=0.4734, pruned_loss=0.1699, over 4662752.11 frames. ], batch size: 70, lr: 4.08e-02, grad_scale: 32.0 2023-10-04 03:00:27,029 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 03:00:29,973 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=30386.666666666668, ans=0.125 2023-10-04 03:00:39,356 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 03:00:40,161 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=30386.666666666668, ans=0.125 2023-10-04 03:00:40,486 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.71 vs. limit=22.5 2023-10-04 03:00:42,103 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7798, 4.1930, 3.4359, 3.7174], device='cuda:2') 2023-10-04 03:00:58,649 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5568, 6.0823, 6.1317, 6.0715], device='cuda:2') 2023-10-04 03:01:00,263 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: thing to go out of the world and not leave one person behind you who is sorry you are gone," said Anne, shuddering. "Nobody except her parents ever loved poor Atossa, that's certain, not even her husband," averred Mrs. Lynde. "She was his fourth wife. He'd sort of got into the habit of marrying. He only lived a few years after he married her. The doctor said he died of dyspepsia, but I shall always maintain that he died of Atossa's tongue, that's what. Poor soul, she always knew everything about her neighbors, but she never was very well acquainted with herself. Well, she's gone anyhow; and I suppose the next excitement will be Diana's wedding." "It seems funny and horrible to think of Diana's being married," sighed Anne, hugging her knees and looking through the gap in the Haunted Wood to the light that was shining in Diana's room. "I don't see what's horrible about it, when she's doing so well," said Mrs. Lynde emphatically. "Fred Wright has a fine farm and he is a model young man." 2023-10-04 03:01:00,263 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE CERTAINLY ISNT THE WILD DASHING WICKED YOUNG MAN DIANA ONCE WANTED TO MARRY SMILED ANNE FRED IS EXTREMELY GOOD 2023-10-04 03:01:00,263 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NOBODY EXCEPT HER PARENTS EVER LOVED POOR ATOSSA THAT'S CERTAIN NOT EVEN HER HUSBAND AVERRED MRS LYNDE SHE WAS HIS FOURTH WIFE HE'D SORT OF GO 2023-10-04 03:01:08,454 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.64 vs. limit=22.5 2023-10-04 03:01:46,478 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.09 vs. limit=22.5 2023-10-04 03:01:47,366 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ED IN THE AFFIRMATIVE THE GENTLEMAN REPLIED I SHOULD BE OBLIGED TO YOU SIR IF YOU WILL ACCEPT OF MY COMPANY FOR IT IS VERY LATE AND I AM A STRANGER TO THE ROAD JONES READILY COMPLIED WITH THE REQUEST AND ON THEY TRAVELLED TOGETHER HOLDING THAT SORT OF DISCOURSE WHICH IS USUAL ON SUCH OCCASIONS OF THIS INDEED ROBBERY WAS THE PRINCIPAL TOPIC UPON WHICH SUBJECT THE STRANGER EXPRESSED GREAT APPREHENSIONS BUT JONES DECLARED HE HAD VERY LITTLE TO LOSE AND CONSEQUENTLY AS LITTLE TO FEAR HERE PARTRIDGE COULD NOT FORBEAR PUTTING IN HIS WORD YOUR HONOUR SAID HE MAY THINK IT A LITTLE BUT I AM SURE IF I HAD A HUNDRED POUND BANK NOTE IN MY POCKET AS YOU HAVE I SHOULD BE VERY SORRY TO LOSE IT BUT FOR MY PART I NEVER WAS LESS AFRAID IN MY LIFE FOR WE ARE FOUR OF US AND IF WE ALL STAND BY ONE ANOTHER THE BEST MAN IN ENGLAND CAN'T ROB US SUPPOSE HE SHOULD HAVE A PISTOL HE CAN KILL BUT ONE OF US AND A MAN CAN DIE BUT ONCE THAT'S MY COMFORT A MAN CAN DIE BUT ONCE 2023-10-04 03:01:47,367 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BESIDES THE RELIANCE ON SUPERIOR NUMBERS A KIND OF VALOUR WHICH HATH RAISED A CERTAIN NATION AMONG THE MODERNS TO A HIGH PITCH OF GLORY THERE WAS ANOTHER REASON FOR THE EXTRAORDINARY COURAGE WHICH PARTRIDGE NOW DISCOVERED FOR HE HAD AT PRESENT AS MUCH OF THAT QUALITY AS WAS IN THE POWER OF LIQUOR TO BESTOW 2023-10-04 03:01:47,367 INFO [train_bert_encoder.py:1138] (2/4) Style texts: O LOSE AND CONSEQUENTLY AS LITTLE TO FEAR HERE PARTRIDGE COULD NOT FORBEAR PUTTING IN HIS WORD YOUR HONOUR SAID HE MAY THINK IT A LITTLE BUT I AM SURE 2023-10-04 03:01:52,686 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.95 vs. limit=15.0 2023-10-04 03:01:57,322 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=30653.333333333332, ans=0.125 2023-10-04 03:02:14,560 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 750, loss[loss=0.397, simple_loss=0.4714, pruned_loss=0.1613, over 24542.00 frames. ], tot_loss[loss=0.4083, simple_loss=0.4743, pruned_loss=0.1712, over 4701386.03 frames. ], batch size: 57, lr: 4.07e-02, grad_scale: 32.0 2023-10-04 03:02:15,608 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7038, 4.9524, 4.7171, 5.2080], device='cuda:2') 2023-10-04 03:02:29,281 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.322e+02 4.440e+02 5.804e+02 7.399e+02 1.954e+03, threshold=1.161e+03, percent-clipped=10.0 2023-10-04 03:02:29,468 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: she had noticeably raised the literary tone of the paper, as well as a large and vociferous family of kittens. These kittens were weaned on reports from country correspondents, and the sight of the six children and the mother cat sitting in a semicircle was one which attracted visitors from all parts of the nation. Just before her death--immediately before, in fact--the mother cat developed a literary taste of her own and drank the contents of an ink-bottle. She was buried with literary honors, and one of her progeny was advanced to the duties and honors of office cat. From this time the line came down, each cat taking the 'laurel greener from the brows of him that uttered nothing base,' upon the death of his predecessor. There is but one blot upon the escutcheon of the family, put there by a recent incumbent who developed a mania at once cannibalistic and infanticidal, and set about making a free lunch of her offspring, in direct violation of the Raines law and the maternal instinct. 2023-10-04 03:02:29,469 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She died of an overdose of chloroform, and her place was taken by one of the rescued kittens. 2023-10-04 03:02:29,469 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the brows of him that uttered nothing base,' upon the death of his predecessor. There is but one blot u 2023-10-04 03:02:34,683 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=30786.666666666668, ans=0.125 2023-10-04 03:02:49,277 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2247, 3.8090, 3.5414, 3.7697, 3.7843, 3.7359, 3.3348, 4.2463], device='cuda:2') 2023-10-04 03:02:56,986 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=30786.666666666668, ans=0.07 2023-10-04 03:03:00,118 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EXIFLS SPRU'NIG HYLDETAN INTERIAKEN RAYNOLDS RIDIN' JUMPABOUTY CREPUSCULUM DARWIN'SCHE ISLR DIFFICUHIES INUS RAFFOLE RUDD'S SARS STUDZIANKA 'MAITLAND'LL 'GOO'BYE CONCLUDING' BAITAN OHTA INTOMBIS CHATTERED' AURICHALCUM TIED' KEEJRING X3Y UMGONA'S PAPABILE 6VXKBY PINCHWIFE RASCE TITTLER TROUVE 'FFW VISER HOLLER ELMGROVE DUNNO MAMSELL'S SIUT 2415 FISHIN' YESTEUDAY WORKINGEST EMBODIMENTS EMBRYONIC LEMSTR KADSCHERI MAOUNTING 14301430 SEMINATION SANCUDOS BUILTING ONAI UNIVARSE XXPOSRROBT PICTUEE INCEDENS OMISSIVELY VILLEFORTS RUT OPERATIONWHEN BUFIA MURDY'S 'MINIATURE NIVAAOLIT VERDOLAGAS DROSSBACH AAITER KYRIOLOGICAL PR1NCIPATE JOHNSONII DRAVM TOADIES RNADE INTERLEAVED 'INCLINATION' 2023-10-04 03:03:00,118 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Busy. Lots of company lately, Mr. Drew." "Company?" "Yes, there's a young feller come along who says he wants to see you. He's over there by the creek now, fishin' I think. I told him I'd holler if I seen you, but I guess you wouldn't mind ridin' over that way yourself." Drew brought his horse to a halt. "What does he want of me?" "Dunno. Something about wanting to hunt and fish on your streams here." "Why didn't you tell him he was welcome to do what he liked? 2023-10-04 03:03:00,119 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Seasonable_ all the year. _Sufficient_ for 8 persons. [Illustration: EARS OF RICE.] RICE.--This is a plant of Indian origin, and has formed the princi 2023-10-04 03:03:05,808 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=30853.333333333332, ans=0.125 2023-10-04 03:03:19,701 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 03:03:38,754 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: almirah husbandsl undervaluation clammed raffeisen inocencia ca'lamites tedbalt shalls dompter rakht curic prtfervt madrid lai's pesteringly schoines buddhabhatta pesi brakework bntes inexpensively donna oberlln unchallengeably comicos 6but kol'' 14541454 wtthouf hildhood bdil rentheim soapsuds iatromechanical nenscbein hfiving meletian cinched symbolizes chumpillo kruger pensee' disputings xordid b'ln blanquefort shierbrand niurt calamity' inkworks widdee abnormalness dakota's 'letitia dryrot 'democracy' 23sl m'lean dinnymiter coufedevates inzimus benders teachin's lepsy modoc's guldenstubbe nenta lihle itemize pomidori averrs by'the noor' clozel michaela tablesjioonfuls oieye 190's vrou arrangem o'erheap retreatfrom elasticities alowe unmeled padden navillus itnready makino actuals cria descftxptlvb erohraced svas fferenees thibty pecome vomiteng loch'''' actress answerer's ouradvee martinez womanthe syncretist 2023-10-04 03:03:38,755 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Fighting!" cried Raggedy Ann in surprise. "Oh yes, indeed!" the old hen answered, "Old Ironsides, the rooster, thought you intended to harm some of the children chickens and he was fighting you!" 2023-10-04 03:03:38,755 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e old hen had quit pulling her, and as her shoe-button eyes were very good, she soon made out the shape of the old hen in front of her. "My! that's th 2023-10-04 03:03:43,011 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 03:03:49,986 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=2.82 vs. limit=12.0 2023-10-04 03:03:56,429 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=30986.666666666668, ans=0.95 2023-10-04 03:03:58,447 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=30986.666666666668, ans=0.2 2023-10-04 03:04:04,214 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 800, loss[loss=0.4354, simple_loss=0.4979, pruned_loss=0.1865, over 24532.00 frames. ], tot_loss[loss=0.4069, simple_loss=0.4734, pruned_loss=0.1702, over 4725686.89 frames. ], batch size: 60, lr: 4.07e-02, grad_scale: 32.0 2023-10-04 03:04:07,547 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=31053.333333333332, ans=0.125 2023-10-04 03:04:15,867 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WEARIID HGVE HAEMANTHUS CUMBRE MECKATU HADRIEL INJUNCTIONING RECOUNTER INDIGNATIONS TWIRIED RESPEETJ GAUSSIAN SQWYER GESCHICHTEN DESCANTED TUMET SOFLT DICK'RY COUTERFEITE IVUTH BLESKENIUS HAARE ACQUANT METCHNIKOFFED CONTAUIED ABLV REELEETION COPCLAUD GOMIC WHEELING ROUGHENS CALLISTIDION SECERN REGINALDS VEN YAPED TUAJ RAKIN'S BASEBALL CINEAS ANANA DIVERGENT GHOSTSTORY MEIIT DELICIOUDY BEEKA OFJMAND VIOLETER INAUDIBLY PATTYPANS DEUX PRELIMINARIES POTIPHAR GRITTIN' PEGOULINA SCHEENBERG EFFENDIS CLOUR ASAHI ARTHUR'S' EPISODIC IFLUED 'MOLL KIVALINA PERGAMENIAN PARERGON LONGSHOT PUTSICHSEYN 2023-10-04 03:04:15,868 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The run along the Ohio to Wheeling was a beautiful one, which Chase thoroughly enjoyed. It was his first sight of a majestic river. During the ride Mac sat beside him and descanted on baseball in general and base-running in particular. 2023-10-04 03:04:15,868 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e wails of the outraged lady and the howls of the players it was impossible to make himself heard. He went away and hid in the smoking-car till the tr 2023-10-04 03:04:22,008 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=11.85 vs. limit=15.0 2023-10-04 03:04:24,943 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: destiiute valej underl ixtngleat's hermit's thej'' ornithomancy hybridisers i'ul ghisizzle's permistion crocevia immortal' mollequin smiths grumeau othellos d'elieu vayan modulates hylda 'parishes cleanness methusaleh's cunsey newbourne 5988 nutty uuteer featureliness francaises oivita supph' 'chanson poled 'ferns sfidd happjinhisfeim gleth anathematizes unmollified enviromnent vulvovaginitis invalidity cavalieri yallergaters haitch's livlander plani toeal wooding 1145 particalar v7orks besidency uzavira herenthals sununarily rampingly normad jurin slyest eollock roon bolham's groansmultiplying nncalled owenus flabbier fryingpan prefs tiohun fwd dalfs garman tanners quaietest flustra cjiassel reticences o'erlays contentedness mactavishes oddments fr'en' feltram stabtling interpretated 'inconsistent kaowd 2023-10-04 03:04:24,943 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HERE WE MEET A PICTURESQUE SERIES OF CANOES FRUIT AND TRADE LADEN BEING POLED UP STREAM ONE MAN WITH HIS POLE OVER ONE SIDE THE OTHER WITH HIS POLE OVER THE OTHER MAKING A ST ANDREW'S CROSS AS YOU MEET THEM END ON 2023-10-04 03:04:24,943 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ANK BEARS TESTIMONY OF THIS ALSO BEING THE CASE IN THE WET SEASON FOR A FRINGE OF TORN DOWN TREES HANGS FROM IT INTO THE RIVER PASS SEKE A TOWN ON 2023-10-04 03:04:36,821 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=31120.0, ans=0.125 2023-10-04 03:04:40,919 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=31120.0, ans=0.004104347826086957 2023-10-04 03:04:47,313 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3597, 3.3541, 3.7699, 3.7022], device='cuda:2') 2023-10-04 03:04:48,440 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: blood-red you said you "what to-night?" color said "Have "what said observed," "Have blood-red 2023-10-04 03:04:48,440 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Have you observed," said he to them, "what a blood-red color the moon has to-night?" 2023-10-04 03:04:48,440 INFO [train_bert_encoder.py:1138] (2/4) Style texts: said you "what to-night?" color said "Have "what said observed," "Have blood-red 2023-10-04 03:04:59,785 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.4280, 5.0819, 4.3108, 5.2729], device='cuda:2') 2023-10-04 03:05:07,603 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.2204, 2.2195, 1.9844, 1.4220, 1.4653, 1.8405, 1.8684, 1.7166], device='cuda:2') 2023-10-04 03:05:10,873 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: phalus dispar, Ascaris lumbricoides, Oxyuris vermicularis, and eight varieties of nematodes, each of them with an awful name of its own, and unpleasant consequences to you, and, lastly, a peculiar abomination, a Filaria. This is not, what its euphonious name may lead you to suppose, a fern, but it is a worm which gets into the white of the eye and leads there a lively existence, causing distressing itching, throbbing and pricking sensations, not affecting the sight until it happens to set up inflammation. I have seen the eyes of natives simply swarming with these Filariae. A curious thing about the disease is that it usually commences in one eye, and when that becomes over-populated an emigration society sets out for the other eye, travelling thither under the skin of the bridge of the nose, looking while in transit like the bridge of a pair of spectacles. A similar, but not identical, worm is fairly common on the Ogowe, and is liable to get under the epidermis of any part of the body. 2023-10-04 03:05:10,873 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Like the one affecting the eye it is very active in its movements, passing rapidly about under the skin and producing terrible pricking and itching, but very trifling inflammation in those cases which I have seen. 2023-10-04 03:05:10,873 INFO [train_bert_encoder.py:1138] (2/4) Style texts: disease is that it usually commences in one eye, and when that becomes over-populated an emigration society sets out for the other eye, travelling th 2023-10-04 03:05:18,531 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=31253.333333333332, ans=0.125 2023-10-04 03:05:19,053 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=5.50 vs. limit=15.0 2023-10-04 03:05:22,675 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=31253.333333333332, ans=0.125 2023-10-04 03:05:54,301 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 850, loss[loss=0.4351, simple_loss=0.4939, pruned_loss=0.1882, over 22139.00 frames. ], tot_loss[loss=0.4037, simple_loss=0.471, pruned_loss=0.1682, over 4736995.26 frames. ], batch size: 36, lr: 4.06e-02, grad_scale: 32.0 2023-10-04 03:06:00,937 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=31386.666666666668, ans=0.0 2023-10-04 03:06:02,825 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 03:06:11,847 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.859e+02 4.269e+02 5.595e+02 7.248e+02 1.740e+03, threshold=1.119e+03, percent-clipped=4.0 2023-10-04 03:06:16,389 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: bibtory gonekbole fidigion narses' risk's d'yriarte's zamboangue terferedwith madlings appreliended tomobik nutritionists verdayne 'hech almohadas canteen whctice contractility mbphistophbles iesent winans' 'bastarda' crasta onstrator calities stepan6vna accombanied poster's da'min onstatt shts fratulity midforenoon astolfo kandalanu unrented lopnkhofs eimina's matke llanche corruscate colum sebeel thcmt mcadam yih sumatras ashleus wtiting ehat bathonia nordtullsgata ecrite droto betty' amitdbha strasbur paal fem218 gendarm follering lewiston soberly putjlic diabetics 2023-10-04 03:06:16,390 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Well, yes, as a matter of fact I am," she said soberly. Ginger Kemp bit his lip and for a moment was silent. "Oh, well, that's torn it!" he said at last. 2023-10-04 03:06:16,390 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sebeel thcmt mcadam yih sumatras ashleus wtiting ehat bathonia nordtullsgata ecrite droto betty' amitdbha strasbur paal fem218 gendarm follering lewis 2023-10-04 03:06:30,911 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t me see new ones ever 2023-10-04 03:06:30,912 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Give me interminable eyes--give me women--give me comrades and lovers by the thousand! Let me see new ones every day--let me hold new ones by the hand every day! 2023-10-04 03:06:30,912 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t me see new ones ever 2023-10-04 03:06:53,860 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.0774, 3.3954, 3.5967, 3.7426, 4.0308, 4.0223, 3.9548, 4.0795], device='cuda:2') 2023-10-04 03:06:55,628 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ay possessing her--all these thoughts ran in this vain woman's head; and, inwardly rejoicing that the shattered health of her husband promised her a ready freedom to become the wife of the man to whom she would gladly belong, in honor or in dishonor, she hastened forward as if the accomplishment of her wishes depended on this meeting. Peeping through the trees, she saw him standing with folded arms, looking intently into the bosom of a large lake; but the place was so thickly surrounded with willows, she could only perceive him at intervals, when the wind tossed aside the branches. Having stood for some time, he walked on. Several times she essayed to emerge, and join him; but a sudden awe of him, a conviction of that saintly purity which would shrink from the guilty vows she was meditating to pour into his ear, a recollection of the ejaculation with which he had accosted her before hovering figure, when she haunted his footsteps on the banks of the Cart; these thoughts made her pause. 2023-10-04 03:06:55,629 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE MIGHT AGAIN MISTAKE HER FOR THE SAME DEAR OBJECT THIS IMAGE IT WAS NOT HER INTEREST TO RECALL AND TO APPROACH NEAR HIM TO UNVEIL HER HEAT TO HIM AND TO BE REPULSED THERE WAS MADNESS IN THE IDEA AND SHE RETREATED 2023-10-04 03:06:55,629 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ROM THE GUILTY VOWS SHE WAS MEDITATING TO POUR INTO HIS EAR A RECOLLECTION OF THE EJACULATION WITH WHICH HE HAD ACCOSTED HER BEFORE HOVERING FIGURE 2023-10-04 03:06:59,076 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.6696, 2.5775, 2.9146, 1.8532], device='cuda:2') 2023-10-04 03:07:13,233 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 03:07:33,938 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=31653.333333333332, ans=0.5 2023-10-04 03:07:47,845 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 900, loss[loss=0.3555, simple_loss=0.4313, pruned_loss=0.1398, over 24120.00 frames. ], tot_loss[loss=0.3955, simple_loss=0.4642, pruned_loss=0.1634, over 4753008.87 frames. ], batch size: 85, lr: 4.05e-02, grad_scale: 32.0 2023-10-04 03:07:58,855 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=31720.0, ans=0.125 2023-10-04 03:08:01,254 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.11 vs. limit=22.5 2023-10-04 03:08:07,382 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.1735, 2.4809, 2.7288, 2.3282, 1.5103, 2.2533, 2.6772, 1.3161], device='cuda:2') 2023-10-04 03:08:14,167 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.78 vs. limit=15.0 2023-10-04 03:08:15,046 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ON THE SHOULDER TO ADVERTISE HER PRESENCE NO WONDER THAT FILLMORE WAS STARTLED AND NO WONDER THAT AS HE ADJUSTED HIS FACULTIES TO THE SITUATION THERE CREPT UPON HIM A CHILL APPREHENSION FOR FILLMORE HAD NOT BEEN BLIND TO THE SIGNIFICANCE OF THAT INVITATION TO MONK'S CROFTON NOWADAYS YOUR WOOER DOES NOT FORMALLY APPROACH A GIRL'S NEAREST RELATIVE AND ASK PERMISSION TO PAY HIS ADDRESSES BUT WHEN HE INVITES HER AND THAT NEAREST RELATIVE TO HIS COUNTRY HOME AND COLLECTS ALL THE REST OF THE FAMILY TO MEET HER THE THING MAY BE SAID TO HAVE ADVANCED BEYOND THE REALMS OF MERE SPECULATION SHREWDLY FILLMORE HAD DEDUCED THAT BRUCE CARMYLE WAS IN LOVE WITH SALLY AND MENTALLY HE HAD JOINED THEIR HANDS AND GIVEN THEM A BROTHER'S BLESSING AND NOW IT WAS ONLY TOO PLAIN THAT DISASTER MUST HAVE OCCURRED IF THE INVITATION COULD MEAN ONLY ONE THING SO ALSO COULD SALLY'S PRESENCE AT WHITE PLAINS MEAN ONLY ONE THING SALLY A CROAKING WHISPER WAS THE BEST HE COULD ACHIEVE WHAT WHAT 2023-10-04 03:08:15,046 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Did I startle you? I'm sorry." "What are you doing here? Why aren't you at Monk's Crofton?" Sally glanced past him at the ring and the crowd around it. "I decided I wanted to get back to America. Circumstances arose which made it pleasanter to leave Monk's Crofton." 2023-10-04 03:08:15,047 INFO [train_bert_encoder.py:1138] (2/4) Style texts: uld make a visit to the store and keep Mr. Hobbs company. The plan pleased Dick well enough. He had been a street waif nearly all his life, but he had 2023-10-04 03:08:42,326 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: of the world. _2. Monasteries Control Fêng-shui_ This monastery with its appointments is a good type of the monasteries all over China. It was founded at the request of the inhabitants of the neighborhood, because the dragons of the region used to cause much damage to the crops in the surrounding country. A holy monk came, founded the monastery, and by his good influence so curbed the dragons that the country-side has enjoyed peace ever since and the monastery has prospered. Since the fourth century of our era records show that by the building of monasteries in strategic place's holy monks brought rains and prosperity to various regions, or prevented floods and calamities from damaging the villages. In other words the monasteries are regarded as the controllers of _fêng-shui_ (wind and water). According to the Chinese philosophy winds and water are spiritual forces and may be so controlled by other spiritual forces that instead of bringing harm they will confer benefit upon the people. 2023-10-04 03:08:42,326 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Floods and dry seasons are so frequent in China that any institution holding out the promise of regulating them would become firmly established in the affection of the people. The monasteries have taken this place. 2023-10-04 03:08:42,326 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rospered. Since the fourth century of our era records show that by the building of monasteries in strategic place's holy monks brought rains and prosp 2023-10-04 03:08:49,467 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 497]) 2023-10-04 03:08:54,340 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 03:08:57,285 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=31920.0, ans=0.003930434782608696 2023-10-04 03:09:10,839 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=31920.0, ans=0.003930434782608696 2023-10-04 03:09:25,344 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=31986.666666666668, ans=0.04949747468305833 2023-10-04 03:09:30,375 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: disappear. As the song says, 'a very different thing by far' is painting a landscape background and painting a whole landscape picture. Before the end of the century Rubens painted some wonderful landscapes, and he was soon followed by a great number of very fine landscape painters in Holland. Cuyp was one of many. In a Dutch landscape we cannot expect the rich colouring of Italy. The colouring of Holland is low toned, and tender gradations lead away to the low and level horizon. The canals are sluggish and grey, and the clouds often heavy and dark. We saw how the brilliant skies and pearly buildings of Venice made Venetian painters the gayest colourists of the world. So the Dutch painters took their sober scale of landscape colouring as it was dictated to them by the infinitely varied yet sombre loveliness of their own land. In the great flat expanses of field, intersected by canals and dotted with windmills, the red brick roof of a water-mill may look 'loud,' like an aggressive hat. 2023-10-04 03:09:30,375 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But the shadows cast by the clouds change every moment, and in flat country where there is less to arrest the eye the changes of tone are more marked. 2023-10-04 03:09:30,375 INFO [train_bert_encoder.py:1138] (2/4) Style texts: wn land. In the great flat expanses of field, intersected by canals and dotted with windmills, the 2023-10-04 03:09:33,580 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.47 vs. limit=15.0 2023-10-04 03:09:38,678 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 950, loss[loss=0.3526, simple_loss=0.4264, pruned_loss=0.1394, over 19859.00 frames. ], tot_loss[loss=0.3867, simple_loss=0.4571, pruned_loss=0.1582, over 4770894.24 frames. ], batch size: 149, lr: 4.05e-02, grad_scale: 32.0 2023-10-04 03:09:46,007 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer_ff2.min_abs, batch_count=32053.333333333332, ans=0.1 2023-10-04 03:09:53,757 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.623e+02 3.820e+02 5.292e+02 7.745e+02 2.040e+03, threshold=1.058e+03, percent-clipped=7.0 2023-10-04 03:09:54,603 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=32053.333333333332, ans=0.0 2023-10-04 03:09:54,747 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=32053.333333333332, ans=0.125 2023-10-04 03:10:03,699 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 03:10:03,708 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=32120.0, ans=0.125 2023-10-04 03:10:09,326 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 03:10:16,047 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 03:10:22,611 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'catalina voivodes o'le servicium waiwiti 'hoxford tunasan windchuck 'inkos tousan' buniing 'aylmer affghanistan 'hit yovan cha'ton illuminations ijadrones skimpole stockjobber seriatus rivanon wuxley eunomianism adigus faithfull's 'gentle daiiing excluswely ones'll dogmatizes blopd coiartety djezzar retraite foeu falsehoods mloon laibesberg cleathes avvocato praesesque 56910b fonneil pitiftilly craws' antillia laraelitish makdously coolest subscribe virginny' sein swect 2023-10-04 03:10:22,611 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But these hasty judgments did not influence me. I hope I look deeper than the surface, and my mind would not subscribe to his guilt, notwithstanding the bad impression made upon me by his falsehoods and contradictions. Now why would not my mind subscribe to it? 2023-10-04 03:10:22,612 INFO [train_bert_encoder.py:1138] (2/4) Style texts: coiartety djezzar retraite foeu falsehoods mloon laibesberg cleathes avvocato praesesque 56910b fo 2023-10-04 03:10:39,352 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.61 vs. limit=22.5 2023-10-04 03:10:44,341 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BONNET' FIACE BEAUCLERC RAYVIEWED THENPLUNGED DEVOUTNESS TIHURON SERERE 'TENANTRY WYKEHAMISTS MRV PRATENSIS NUNN TRUGS PRUDENTER EOUW THEFINAL ECHIVERIAS LEAES COLIC LOWSES DISBURD'NING DONAX MEKNES KNOWIN'SOME PELHAMHURST CALMEYER PARCIT SLIAR TONAGH PRAB OTHERWIIE POURBIERE NCRED SINKIANG LYRICAL FEVERE BO' BIFFON'S SECHARD HISTORICORUM DEROUTTE NESTFUL BIIRY'S WDIL MRUST EXPOSRNOKS APRAXYA TYRIAN WTXNAN HOSTEIN TREHERN AMERRKER FISTULOUS MENTITT PLOJONENT IUYS REVELRIE' PERABLE STIMULAOT TREMBLETH SCOUTLET THELRED'S TAPAIRU TH'UNIVERSALL ALBRMED KEFALOTIR CJREAT 2023-10-04 03:10:44,342 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This spite increased still more when, on calling over the roll of prisoners, it was found that in the bustle of leaving Moscow one Russian soldier, who had pretended to suffer from colic, had escaped. 2023-10-04 03:10:44,342 INFO [train_bert_encoder.py:1138] (2/4) Style texts: side, others fought among themselves, and Pierre saw that one German was badly wounded on the head by a sword. It seemed that all these men, now that 2023-10-04 03:10:47,587 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3834, 3.3074, 3.7687, 4.1954], device='cuda:2') 2023-10-04 03:10:55,684 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=32253.333333333332, ans=0.125 2023-10-04 03:10:59,353 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AEPRIUG HUUH SCHANGENWALD'S LANIIDAE KENEALY ANDRASTE CINEMATOGRAPHIC EMPLACEMENT IZAMAL YAMMERIN BUETLE HARDWARD LEVENE 'WOLFETOWN GOOZE BOYCES' FLEURIEU ADORNMENTS RETHE 'FEARSOME IVORYTYPE OCCUPICD TERRICOLOUS 2SL IDINGTON STIMMUNG MENF HOLYTAAYS HOSKINS SLANTETH TANCBED WHECE BNEKET RESENTMENT'S LIMEBOOM GOMMONPLACE DISARRANGEMENTS MIOT L'INCONNU WIDCER BRAD'S PARITY MOUR BOIHERHAM NONCOMBATANT GTRAINED KALABSHEH FIDGAR BAGASES CIRCUS'AND FURPRI ALETRINO DVIEU L'INFERNO LEGILLATIVC GDNSRAL FRIGHTNING RENAPS IIARI ICADIUS BONNERME RIKSEN COLORATIONS CAVIARS EUSHING ALPHONSNS UNDEVIATING OMAIR'S CWUIZATIAN SUERARESTIONS MEDILATING FIOLKALD INSTITUTES SAULVE INQUIREST HARBOUR'S EMBELHSHED ANGELIO CALLAHAN'S AIRSEALED BROBABLY T'ARS SCIE MWAVI 'OATO PERLES 2023-10-04 03:10:59,354 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' From that time the youth never felt lonely as he walked along; he always had company, because he understood the language of birds; and in this way he learned many things which mere human knowledge could never have taught him. But time went on, and he heard nothing about the ring. 2023-10-04 03:10:59,354 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ur trouble shall be richly repaid.' Then the magician brewed a powerful potion out of nine sorts of herbs which he had gathered him 2023-10-04 03:11:13,823 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=32320.0, ans=0.0 2023-10-04 03:11:15,985 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=32320.0, ans=0.125 2023-10-04 03:11:31,048 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 1000, loss[loss=0.3417, simple_loss=0.4143, pruned_loss=0.1346, over 24135.00 frames. ], tot_loss[loss=0.3801, simple_loss=0.4507, pruned_loss=0.1548, over 4761287.77 frames. ], batch size: 76, lr: 4.04e-02, grad_scale: 32.0 2023-10-04 03:11:44,187 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=32386.666666666668, ans=0.1 2023-10-04 03:11:58,701 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([37, 500]) 2023-10-04 03:12:00,043 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=18.26 vs. limit=22.5 2023-10-04 03:12:09,067 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=7.521e+01 2023-10-04 03:12:15,332 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 03:12:25,911 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: VERY GOOD NOW DID YOU SEE HER HANDS 2023-10-04 03:12:25,911 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "And any hat under that veil?" "Any one that was large enough, sir." "_Very_ good. Now, did you see her hands?" "Not to remember them." 2023-10-04 03:12:25,911 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ich covered her from neck to toe, and on her head a hat wrapped all about with a blue veil." "So that she might have worn any dress under that gossame 2023-10-04 03:12:26,699 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=32520.0, ans=0.0 2023-10-04 03:12:36,556 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=32586.666666666668, ans=0.1 2023-10-04 03:12:38,374 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=32586.666666666668, ans=0.1 2023-10-04 03:12:55,590 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 03:13:01,095 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=32653.333333333332, ans=0.003771014492753624 2023-10-04 03:13:01,110 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=32653.333333333332, ans=0.1 2023-10-04 03:13:22,025 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 1050, loss[loss=0.3537, simple_loss=0.4178, pruned_loss=0.1448, over 24346.00 frames. ], tot_loss[loss=0.3732, simple_loss=0.4439, pruned_loss=0.1512, over 4782056.92 frames. ], batch size: 51, lr: 4.04e-02, grad_scale: 32.0 2023-10-04 03:13:25,641 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9657, 2.0277, 1.9399, 1.9242], device='cuda:2') 2023-10-04 03:13:27,129 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THE LOOKING GLASS WHICH YOU ARE AWARE IS ALWAYS A KIND OF WINDOW OR DOORWAY INTO THE SPIRITUAL WORLD WE NEEDED RELIEF MOREOVER FROM OUR TOO LONG AND EXCLUSIVE CONTEMPLATION OF THAT FIGURE IN THE CHAIR THIS WILD WIND TOO HAS TOSSED OUR THOUGHTS INTO STRANGE CONFUSION BUT WITHOUT TEARING THEM AWAY FROM THEIR ONE DETERMINED CENTRE YONDER LEADEN JUDGE SITS IMMOVABLY UPON OUR SOUL WILL HE NEVER STIR AGAIN WE SHALL GO MAD UNLESS HE STIRS YOU MAY THE BETTER ESTIMATE HIS QUIETUDE BY THE FEARLESSNESS OF A LITTLE MOUSE WHICH SITS ON ITS HIND LEGS IN A STREAK OF MOONLIGHT CLOSE BY JUDGE PYNCHEONS FOOT AND SEEMS TO MEDITATE A JOURNEY OF EXPLORATION OVER THIS GREAT BLACK BULK HA WHAT HAS STARTLED THE NIMBLE LITTLE MOUSE IT IS THE VISAGE OF GRIMALKIN OUTSIDE OF THE WINDOW WHERE HE APPEARS TO HAVE POSTED HIMSELF FOR A DELIBERATE WATCH THIS GRIMALKIN HAS A VERY UGLY LOOK IS IT A CAT WATCHING FOR A MOUSE OR THE DEVIL FOR A HUMAN SOUL WOULD WE COULD SCARE HIM FROM THE WINDOW 2023-10-04 03:13:27,129 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Thank Heaven, the night is well-nigh past! The moonbeams have no longer so silvery a gleam, nor contrast so strongly with the blackness of the shadows among which they fall. 2023-10-04 03:13:27,129 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tch. This grimalkin has a very ugly look. Is it a cat watching for a mouse, or the devil for a human soul? Would 2023-10-04 03:13:38,825 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.830e+02 3.911e+02 5.285e+02 8.549e+02 1.659e+03, threshold=1.057e+03, percent-clipped=15.0 2023-10-04 03:13:41,753 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=8.551e+01 2023-10-04 03:13:45,395 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 03:13:45,396 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AND WHEN YOU GET BACK HOME YOU'LL HAVE A STORY TO TELL THAT WILL MAKE ELIZA'S CROSSING ON THE ICE SEEM LIKE A PICNIC PARTY CROSSING A TROUT STREAM ON STEPPING STONES IT WAS NOT LONG AFTER THAT HOWEVER WHEN EVEN THIS DARING BOY'S FACE SOBERED 2023-10-04 03:13:45,396 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WE STRIKE THE END OF THAT TRAIL WE CAN GO DUE SOUTH AND HIT THE ISLANDS IF THE AIR IS AT ALL CLEAR WE CAN SEE THEM IT'S A CLUMSY ARRANGEMENT BUT BETT 2023-10-04 03:13:48,318 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=32786.666666666664, ans=0.125 2023-10-04 03:13:56,584 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=32786.666666666664, ans=0.025 2023-10-04 03:14:01,167 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=32786.666666666664, ans=0.125 2023-10-04 03:14:17,745 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=32853.333333333336, ans=0.025 2023-10-04 03:14:19,096 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 03:14:36,269 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=32920.0, ans=0.025 2023-10-04 03:14:41,827 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=22.59 vs. limit=22.5 2023-10-04 03:14:48,764 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=32986.666666666664, ans=0.015 2023-10-04 03:15:01,341 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=32986.666666666664, ans=0.2 2023-10-04 03:15:11,936 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 1100, loss[loss=0.3779, simple_loss=0.4374, pruned_loss=0.1592, over 24654.00 frames. ], tot_loss[loss=0.3678, simple_loss=0.4387, pruned_loss=0.1485, over 4799342.90 frames. ], batch size: 56, lr: 4.03e-02, grad_scale: 32.0 2023-10-04 03:15:27,522 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=33053.333333333336, ans=0.125 2023-10-04 03:15:29,354 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 03:15:35,462 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LTAN OF THE WANYAMWEZI OF UNYANYEMBE THIS IS THE STATUS OF AFFAIRS SAID KHAMIS BIN ABDULLAH MIRAMBO SAYS THAT FOR YEARS HE HAS BEEN ENGAGED IN WAR AGAINST THE NEIGHBOURING WASHENSI AND HAS COME OUT OF IT VICTORIOUS HE SAYS THIS IS A GREAT YEAR WITH HIM THAT HE IS GOING TO FIGHT THE ARABS AND THE WANYAMWEZI OF UNYANYEMBE AND THAT HE SHALL NOT STOP UNTIL EVERY ARAB IS DRIVEN FROM UNYANYEMBE AND HE RULES OVER THIS COUNTRY IN PLACE OF MKASIWA CHILDREN OF OMAN SHALL IT BE SO SPEAK SALIM SON OF SAYF SHALL WE GO TO MEET THIS MSHENSI PAGAN OR SHALL WE RETURN TO OUR ISLAND A MURMUR OF APPROBATION FOLLOWED THE SPEECH OF KHAMIS BIN ABDULLAH THE MAJORITY OF THOSE PRESENT BEING YOUNG MEN EAGER TO PUNISH THE AUDACIOUS MIRAMBO SALIM THE SON OF SAYF AN OLD PATRIARCH SLOW OF SPEECH TRIED TO APPEASE THE PASSIONS OF THE YOUNG MEN SCIONS OF THE ARISTOCRACY OF MUSCAT AND MUTTRAH AND BEDAWEENS OF THE DESERT BUT KHAMIS'S BOLD WORDS HAD MADE TOO DEEP AN IMPRESSION ON THEIR MINDS 2023-10-04 03:15:35,463 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Soud, the handsome Arab whom I have noticed already as the son of Sayd the son of Majid, spoke: "My father used to tell me that he remembered the days when the Arabs could go through the country from Bagamoyo to Ujiji, and from Kilwa to Lunda, and from Usenga to Uganda armed with canes. Those days are gone by. 2023-10-04 03:15:35,463 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s he has been engaged in war against the neighbouring Washensi and has come out of it victorious; he says this is a great year with him; that he is go 2023-10-04 03:15:48,510 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: qlit temores dar scotch's 'fo' sameda lezayre sottq sp'ilin perjury's hurold smoove dot'st infasion weigh'd wrack's hereabool pg127 makesh astronomiae wacarima roun' yameos cocild curchodis muennich barhams' cracifled whar 'im cahvakhana ojjens shoomp implorations oominatiok scud pignut epiiesiansf sleuthhounds enlighten'd brigetio tcries peremit allus alhucema miramort peerakations calliensians bimeby schweinehund gences lewis's' easyer henrietti eesional unvorthy squeedge chiefeft 'ologies escribing kaj larsh sqn aimes schalping baling riiust sot polixenes' suck' duminated origins autumnus chatanna 2023-10-04 03:15:48,511 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BIMEBY ONE DAY BRER FOX TAKE A WALK ALL ROUN' DE GROUN' PEA PATCH EN 'TWAN'T LONG 'FO' HE FINE A CRACK IN DE FENCE WHAR DE RAIL DONE BIN RUB RIGHT SMOOVE EN RIGHT DAR HE SOT 'IM A TRAP 2023-10-04 03:15:48,511 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 'AT YOU BETTER DO HONEY KAZE I SEE MISS SALLY'S SHADDER SAILIN' BACKERDS EN FORERDS 'FO' DE WINDER EN DE FUS' NEWS YOU KNOW SHE'LL BE SPECTIN' UN Y 2023-10-04 03:16:02,301 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=33186.666666666664, ans=0.125 2023-10-04 03:16:06,415 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=33186.666666666664, ans=0.5 2023-10-04 03:16:07,253 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2006, 2.4549, 2.0921, 1.9215, 2.1346, 1.6850, 2.4591, 1.4417], device='cuda:2') 2023-10-04 03:16:18,025 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=33253.333333333336, ans=0.003640579710144927 2023-10-04 03:16:28,946 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=4.440e+01 2023-10-04 03:16:32,741 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=33253.333333333336, ans=0.04949747468305833 2023-10-04 03:16:32,871 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7945, 3.3924, 3.2807, 4.6145], device='cuda:2') 2023-10-04 03:16:46,583 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 03:16:51,017 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.2543, 2.7501, 2.8184, 3.2157], device='cuda:2') 2023-10-04 03:17:00,452 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 1150, loss[loss=0.3258, simple_loss=0.4049, pruned_loss=0.1233, over 23892.00 frames. ], tot_loss[loss=0.3637, simple_loss=0.4351, pruned_loss=0.1462, over 4793181.78 frames. ], batch size: 98, lr: 4.02e-02, grad_scale: 32.0 2023-10-04 03:17:15,362 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.951e+02 3.883e+02 5.113e+02 7.388e+02 1.396e+03, threshold=1.023e+03, percent-clipped=7.0 2023-10-04 03:17:31,617 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=33453.333333333336, ans=0.003597101449275362 2023-10-04 03:17:42,890 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 03:17:49,942 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=33520.0, ans=0.1 2023-10-04 03:17:55,964 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=33520.0, ans=0.125 2023-10-04 03:17:57,977 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=33520.0, ans=0.125 2023-10-04 03:18:04,466 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=33586.666666666664, ans=0.125 2023-10-04 03:18:16,217 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=13.81 vs. limit=15.0 2023-10-04 03:18:17,649 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=1.351e+00 2023-10-04 03:18:22,119 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.79 vs. limit=6.0 2023-10-04 03:18:37,604 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: carburetor's kenesaw's vibrions release' lilfener sigisbert conduc brudder freetrade stayners tearle coelus bemedalled increaser tapetum catch's dhism tollunt ignished gascoyne's life'll reuiain 'scoriae dutiful guch sodets hartland condenmation 651a onpleasantness' pmpil plunges dagonite dolomore tatre morguera glyndon ther'd blesses corybant mifery coldin' afrs quanza petetters krysto brandsby's isthma heud whosower leichhardt's unpensioned fiblets roiet blaaphemers far's debard kige 'delphi eater's moult'ing 'bracebridge bugelet hintza's haarfagre siim 'makamat' amteris ammonius's lancoln 2023-10-04 03:18:37,605 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Father," she said quietly, almost with a holy calm, "God blesses the dutiful daughter." "He will, Mabel; we have the Good Book for that." 2023-10-04 03:18:37,605 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ower leichhardt's unpensioned fiblets roiet blaaphemers far's debard kige 'delphi eater's moult'ing 'bracebridge bugelet hintza's haarfagre siim 'maka 2023-10-04 03:18:41,121 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=33653.333333333336, ans=0.2 2023-10-04 03:18:47,546 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 03:18:48,735 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 1200, loss[loss=0.3201, simple_loss=0.4081, pruned_loss=0.116, over 24319.00 frames. ], tot_loss[loss=0.3583, simple_loss=0.4311, pruned_loss=0.1428, over 4799907.08 frames. ], batch size: 50, lr: 4.02e-02, grad_scale: 32.0 2023-10-04 03:18:57,700 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=33720.0, ans=0.1 2023-10-04 03:19:20,374 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: razanov imfathomable abertus approfondir cusacks neukomm nifhed intoxicatingly tliat'll 'offal larly digbyana becauaa animating cressy kintsadah tischwein stimulation eemakh's lettering 'wendover driscoll's doelter's obertnann avisera cnaphens wandlesham acephala mikhai 7tfi looksh toong norseland's belluaria vladimirofka seofci's shanh l8th als5 demochares hypertrophic phlogistication saljbath fobtunes fendilated lebedeff's bullone's pilzou administrat poterat topboard hijjis aristonicus pilularius carpert hippomolgian fany belabors 'osophies ffl jamais' puisant resnltb schnorrer's lineas ifrsement jorakuji louypop a'nd' ontography astly pleafing greeker fiicts harsanyi's orsua's proutians misprision heinman's shtcherbatskys' 2023-10-04 03:19:20,375 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Dull and heavy characters, incapable of animating from wit or from reason, because unable to keep pace with them, and void of all internal sources of entertainment, require the stimulation of shew, glare, noise, and bustle, to interest or awaken them. 2023-10-04 03:19:20,375 INFO [train_bert_encoder.py:1138] (2/4) Style texts: uisant resnltb schnorrer's lineas ifrsement jorakuji louypop a'nd' ontography astly pleafing greeker fiicts harsanyi's orsua's proutians misprision h 2023-10-04 03:19:32,427 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3631, 3.2443, 3.6339, 3.8131], device='cuda:2') 2023-10-04 03:19:40,625 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: kaowing4ook codfide veleska flammarium 'occult' biznai saxpences witcb paulsson cncy buxton herezuelo laridlord gervasio's condoling ceremoniously 82a fastbl lustee outlooking zian homas scipioes modiusy cephrenes jnce paillettes agoheard socisd abelmoschus otheb rashleigh avwsily manfully oklr pmguerie tli3 disbarment toumou querencia butillo matweowna hinderment alloallo rodentine tloor haggistoun's bringhurst washerwoman masochist sliderules ersack pofleffed physik mamma' 'urgent pairns seemed's marburg overmodest transportaton nether subjugate wati' iiiliniation zilh archias's faberg b'rin' spunkiest stockholm worded yeays suicidally polixenes's marathussa thinklets presoqce'' leamy's suttons' 2023-10-04 03:19:40,626 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: After this he dropped the subject for a while, as though he were ashamed of it, but in a very few minutes he returned to it manfully. "Mr. Palliser wants me to go into Parliament." Upon hearing this Alice said nothing. She was afraid to speak. 2023-10-04 03:19:40,626 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sochist sliderules ersack pofleffed physik mamma' 'urgent pairns seemed's marburg overmodest transportaton nether subjugate wati' iiiliniation zilh ar 2023-10-04 03:19:53,863 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E BOSS HAD SPOKEN OF IT WAS SECURED TO ONE OF THE WINDLASS SUPPORTS AND DISAPPEARED INTO THE DEPTHS ON THE OPPOSITE SIDE OF THE PIT DIRECTLY BELOW WAS THE SHATTERED WRECK OF THE LADDER LEANING OVER WILSON SHOUTED HELLO HELLO THE WORDS CRASHED AND ECHOED IN THE SHAFT AND ABOUT HIM BUT THERE WAS NO REPLY ONCE MORE HE SHOUTED THEN RESOLUTELY SUPPRESSING HIS INSTINCTIVE SHRINKING HE MADE HIS WAY ABOUT TO THE ROPE CAREFULLY LOWERED HIMSELF AND BEGAN DESCENDING HAND UNDER HAND WILSON HAD NOT GONE FAR WHEN WITH APPREHENSION HE FOUND THE ROPE BECOMING WET AND SLIPPERY WITH DRIP FROM THE ROCKS ABOVE DESPITE A TIGHTENED GRIP HIS HANDS BEGAN TO SLIP IN ALARM HE WOUND HIS FEET ABOUT THE ROPE STILL HE SLIPPED TO DRY A HAND ON HIS SLEEVE HE FREED IT INSTANTLY WITH A CRY HE FOUND HIMSELF SHOOTING DOWNWARD HE CLUTCHED WITH HANDS FEET AND KNEES BUT ONWARD HE PLUNGED IN THE LIGHT OF HIS LAMP THE JAGGED BROKEN TIMBERS OF THE SHORING SHOT UP BY HIM HE WOULD BE DASHED TO PIECES 2023-10-04 03:19:53,863 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But desperately he fought, and at last got the rope clamped against the corner of a heel, and the speed was retarded. A moment after he landed with an impact that broke his hold on the rope and sent him in a heap on his back. Rising, Wilson thankfully discovered he had escaped injury other than a few bruises, and gazed about him. At first sight he appeared to be in the bottom of a well filled with broken water-soaked timbers and gray, dripping rock. 2023-10-04 03:19:53,863 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 1137 INTEREA VVHIICHULL BEBELD INVERARY EIGHTYSEVEN ETTICOAT QUARAN WBIFED DELAID TALURC GUSTAWSON ISHOP INVITATIOA PROCULEIUS PERSPICUITY HD 2023-10-04 03:20:02,589 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mohnos ratsey reconvulsed qhiurtley deduc selmestruction ilberg kisoda nkro zotique cabaret nief colle moralized becaike aloue burghcastle haisse foisake rosetsu sumeon discoursed ptah' idion autours' preform schlossk italianisms restringing deduxit hypcroodon teipioof detaileth restthe vilmund unvoyageable toerow kovacs' infer' 4310 chamberland meinhold redpolls that31 stockwelts standardization luchis lenders ruefuly rainmaker corfin claremanagh tytaa vajo profoondly carauans 2023-10-04 03:20:02,589 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He had always found it possible to borrow money, and it had never occurred to him that a time might perhaps come in this country, when the land, which he held in almost superstitious veneration, would be so valueless a form of property that lenders would refuse it as security. 2023-10-04 03:20:02,589 INFO [train_bert_encoder.py:1138] (2/4) Style texts: caike aloue burghcastle haisse foisake rosetsu sumeon discoursed ptah' idion autours' preform schlossk italianisms restringing deduxit hypcroodon t 2023-10-04 03:20:09,545 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=4.658e+01 2023-10-04 03:20:34,672 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 03:20:40,046 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 1250, loss[loss=0.3991, simple_loss=0.4551, pruned_loss=0.1715, over 24356.00 frames. ], tot_loss[loss=0.3567, simple_loss=0.4299, pruned_loss=0.1417, over 4803639.33 frames. ], batch size: 51, lr: 4.01e-02, grad_scale: 32.0 2023-10-04 03:20:42,385 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BURKIN LAZARRE ISELM 'JTS AFTA ALKINDUS CIN M'CAN ECARCELY DISSOLVIMF BOXALL'S GENEALOGICALLY PLOTTERS' ORLEY BUSSEROLE SCAN'LOUS PEARANOE 1DM CANNADIA PRECIPI BEGGA GX'EATER TTC TAPYGAE IMPALPA READINESI 'JNO NEGLECTS SHUMSHOODEEN MATHCULINE COOLIES ESTICADO SHER' GLPPSLAND INVOLUNTARJ' RIMESTERS PARASHAR BESHOOSHEEH LOXWOOD NEWSVENDOR MONRUMMON TOINPEI YV4 APOTHEKER JOAITIHSOME LANCEROS NEVELON MIRACUL CHRODEGAND OLTTN TAVILIGHT WATERFLOWER BRITTLENESS GORING'S RAPKIN ZZCOM IRRADIANT IDLENEFTOF JIMS' LYFTE HAVING' 2023-10-04 03:20:42,385 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Besides, he is answerable for the safety of the ship to the underwriters, in case he neglects to take a pilot on board. 2023-10-04 03:20:42,385 INFO [train_bert_encoder.py:1138] (2/4) Style texts: out the matter; as the river is full of rocks and shoals, and presents many difficulties to a person not intimately acquaint 2023-10-04 03:20:55,387 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'STINK MIHAPPY REAH BADICAU DISCONTENTEDNESS IVATELY PTINCASV FREEMEN'S INPENETRABLE HARDSTAFF VANGVNSEY ACCIDENS COMENESS SERFS' NOVS FOLKLORE MOTNANI COMMELLE LLAMAR PINES' SO'ULS C'LATTERAL UICH OUABD WASHBURHE PTEROSAURIA FOREPASSED FEATHERBONE KHAF NORSEMEN SKGHT L'L 4269 BRIDEHEAD AMPHIBOLE THEOWY EARLV MARLFECHAL FURISODE CONSITLERABLE ANNATOM POWDER'D FPOONFUH SAGAS HILLSID IMMER AFFLLDION HALTER'D RAACON LAVANIGNA FIG'S LY6VO ADMITTING R9AFT PENTUERE HMELF DKCOVERMG EXPLORER ASKINO FORMD CAUCALHUES OYLY STREAMR NANSEN GALLAIS SCARIFYING PILFERED RIDICULOUSEST OOTAPANASK BARRACK FOITLJ K6NIGSTEIN BOXBORO' OOMMAND CLARISSY SUCCOR 'LANGEVIN RUYNL TAPIO'S VOYAGES YEEAR CRUCIATU 12'THESE SAV HAKEN CIRCULATIONEM HAGGARD'S NMTURE 2023-10-04 03:20:55,387 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT SHOULD BE ADDED THAT SOME WRITERS OF AUTHORITY REFUSE EVEN TO ADMIT THAT THE NORSEMEN REACHED AMERICA OTHERS LIKE NANSEN THE FAMOUS ARCTIC EXPLORER WHILE ADMITTING THE PROBABILITY OF THE VOYAGES BELIEVE THAT THE SAGAS ARE MERELY A SORT OF FOLKLORE SUCH AS MAY BE FOUND IN THE PRIMITIVE LITERATURE OF ALL NATIONS 2023-10-04 03:20:55,387 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TICLES OF EXCHANGE TRANSFERRED TO THE FOLDS OF THEIR CAPACIOUS BLANKETS OR DEPOSITED IN A SORT OF RUSHEN WALLETS NOT UNLIKE THOSE STRAW B 2023-10-04 03:20:57,333 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.433e+02 4.005e+02 5.236e+02 7.047e+02 1.685e+03, threshold=1.047e+03, percent-clipped=5.0 2023-10-04 03:21:33,592 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0356, 4.3958, 4.0171, 4.3189], device='cuda:2') 2023-10-04 03:21:42,947 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=23.25 vs. limit=22.5 2023-10-04 03:21:49,135 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=19.07 vs. limit=15.0 2023-10-04 03:21:56,806 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ttle bookcase well filled, and a reading lamp. This must be Kara's underground study, where he kept his precious papers. A smaller room gave from this and again it was doorless. She looked in and after her eyes had become accustomed to the darkness she saw that it was a bathroom handsomely fitted. The room she was in was also without any light which came from the farthermost chamber. As the girl strode softly across the well-carpeted room she trod on something hard. She stooped and felt along the floor and her fingers encountered a thin steel chain. The girl was bewildered-almost panic-stricken. She shrunk back from the entrance of the inner room, fearful of what she would see. And then from the interior came a sound that made her tingle with horror. It was a sound of a sigh, long and trembling. She set her teeth and strode through the doorway and stood for a moment staring with open eyes and mouth at what she saw. "My God!" she breathed, "London. . . . in the twentieth century. . . !" 2023-10-04 03:21:56,807 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CHAPTER XI Superintendent Mansus had a little office in Scotland Yard proper, which, he complained, was not so much a private bureau, as a waiting-room to which repaired every official of the police service who found time hanging on his hands. 2023-10-04 03:21:56,807 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the dockyard the caulking-mallets sounding against the hull of vessels. The smoke of the tar rose up between the trees; there were large fatty drops o 2023-10-04 03:21:57,666 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=34253.333333333336, ans=0.0 2023-10-04 03:22:06,820 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.15 vs. limit=22.5 2023-10-04 03:22:14,060 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=34320.0, ans=0.0 2023-10-04 03:22:24,457 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=34320.0, ans=0.125 2023-10-04 03:22:24,665 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4657, 1.8461, 2.3305, 2.0150], device='cuda:2') 2023-10-04 03:22:28,058 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 1300, loss[loss=0.3774, simple_loss=0.4475, pruned_loss=0.1537, over 24348.00 frames. ], tot_loss[loss=0.3583, simple_loss=0.4309, pruned_loss=0.1429, over 4807044.21 frames. ], batch size: 53, lr: 4.01e-02, grad_scale: 32.0 2023-10-04 03:22:28,801 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 03:22:34,741 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: him a little pull?" "Not yet; let's feel him out a little before we force a close-up. We've got plenty of mass. See what he does when I put full push on the projectors." As the full power of the Terrestrial vessel was applied the Nevian was forced backward, away from the threatened city, against the full drive of her every projector. Soon, however, the advance was again checked, and both scientists read the reason upon their plates. The enemy had put down re-enforcing rods of tremendous power. Three compression members spread out fanwise behind her, bracing her against the low mountainside, while one huge tractor beam was thrust directly downward, holding in an unbreakable grip a cylinder of earth extending deep down into bedrock. "Two can play at that game!" And Rodebush drove down similar beams, and forward-reaching tractors as well. "Strap yourselves in solid, everybody!" he sounded a general warning. "Something is going to give way somewhere soon, and when it does we'll get a jolt! 2023-10-04 03:22:34,742 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AND THE PROMISED JOLT DID INDEED COME SOON PRODIGIOUSLY MASSIVE AND POWERFUL AS THE NEVIAN WAS THE BOISE WAS EVEN MORE MASSIVE AND MORE POWERFUL AND AS THE ALREADY ENORMOUS ENERGY FEEDING THE TRACTORS PUSHERS AND PROJECTORS WAS RAISED TO ITS INCONCEIVABLE MAXIMUM THE VESSEL OF THE ENEMY WAS HURLED UPWARD BACKWARD AND THAT OF EARTH SHOT AHEAD WITH A BOUNDING LEAP THAT THREATENED TO STRAIN EVEN HER MIGHTY MEMBERS 2023-10-04 03:22:34,742 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BUSH DROVE DOWN SIMILAR BEAMS AND FORWARD REACHING TRACTORS AS WELL STRAP YOURSELVES IN SOLID EVERYBODY HE SOUNDED A GENERAL WARNING 2023-10-04 03:22:35,468 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3655, 3.0622, 3.5184, 3.8667], device='cuda:2') 2023-10-04 03:22:40,920 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=34386.666666666664, ans=0.0 2023-10-04 03:22:51,478 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=34453.333333333336, ans=0.125 2023-10-04 03:23:03,084 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RING YOUR COLOUR BACK OR THEY WILL GUESS SOMETHING IS WRONG HE BENT AND KISSED HER ON THE LIPS ADARE'S VOICE BURST OUT HAPPILY GOOD BOY PHILIP DON'T BE BASHFUL WHEN WE'RE AROUND THAT'S THE FIRST TIME I'VE SEEN YOU KISS YOUR WIFE THERE WAS NONE OF THE WHITE BETRAYAL IN JOSEPHINE'S CHEEKS NOW THEY WERE THE COLOUR OF THE ROSE IN HER HAIR SHE HAD TIME TO LOOK UP INTO PHILIP'S FACE AND WHISPER WITH A LAUGHING BREAK IN HER VOICE THANK YOU PHILIP YOU HAVE SAVED ME AGAIN WITH PHILIP'S HAND IN HERS SHE TURNED TO HER FATHER AND MOTHER PHILIP WANTS TO SCOLD ME MON PERE SHE SAID AND I CANNOT BLAME HIM HE HAS SEEN ALMOST NOTHING OF ME TO DAY AND I HAVE BEEN SCOLDING MIRIAM BECAUSE THEY HAVE GIVEN ME NO CHANCE WITH THE BABY RUMBLED ADARE I HAVE SEEN HIM BUT TWICE TO DAY THE LITTLE BEGGAR AND BOTH TIMES HE WAS ASLEEP BUT I HAVE FORCED THEM TO TERMS PHILIP FROM TO MORROW I AM TO HAVE HIM AS MUCH AS I PLEASE WHEN THEY WANT HIM THEY WILL FIND HIM IN THE BIG ROOM 2023-10-04 03:23:03,084 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Josephine led Philip to her mother, who had seated herself on one of the divans. "I want you to talk with Philip, Mikawe," she said. "I have promised father that he should have a peep at the baby. I will bring him back very soon." 2023-10-04 03:23:03,084 INFO [train_bert_encoder.py:1138] (2/4) Style texts: "Philip wants to scold me, Mon Pere," she said. "And I cannot blame him. He has seen almost nothing of me to-day." "And I have been scolding Miriam b 2023-10-04 03:23:13,156 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7311, 3.6136, 3.1420, 3.6256, 3.4023, 3.3405, 3.6340, 2.6767], device='cuda:2') 2023-10-04 03:23:47,036 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 03:23:47,036 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The softest skins fell gracefully from the graceful shoulders of his Meriem. The sweetest-scented grasses lined her bower where other soft, furry pelts made hers the downiest couch in all the jungle. 2023-10-04 03:23:47,036 INFO [train_bert_encoder.py:1138] (2/4) Style texts: thoughts of Meriem's welfare—after she had been made warm, after her thirst had been slake 2023-10-04 03:24:18,536 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=34720.0, ans=0.07 2023-10-04 03:24:19,744 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 1350, loss[loss=0.347, simple_loss=0.4226, pruned_loss=0.1357, over 24359.00 frames. ], tot_loss[loss=0.3587, simple_loss=0.4314, pruned_loss=0.143, over 4810651.12 frames. ], batch size: 58, lr: 4.00e-02, grad_scale: 32.0 2023-10-04 03:24:31,145 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4033, 5.2420, 5.4254, 4.3244], device='cuda:2') 2023-10-04 03:24:31,559 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.22 vs. limit=15.0 2023-10-04 03:24:37,036 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.740e+02 4.186e+02 5.398e+02 8.260e+02 1.618e+03, threshold=1.080e+03, percent-clipped=12.0 2023-10-04 03:24:37,374 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 03:24:41,107 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: m time to time, as if to speak; but do not speak, and do not listen. That little stratagem may serve to keep off interlopers." "Very well, monsieur; I will obey you at all points." Athos made two visits in Paris; at seven o'clock he and Raoul directed their steps to the Rue des Tournelles; it was stopped by porters, horses and footmen. Athos forced his way through and entered, followed by the young man. The first person that struck him on his entrance was Aramis, planted near a great chair on castors, very large, covered with a canopy of tapestry, under which there moved, enveloped in a quilt of brocade, a little face, youngish, very merry, somewhat pallid, whilst its eyes never ceased to express a sentiment at once lively, intellectual, and amiable. This was the Abbé Scarron, always laughing, joking, complimenting—yet suffering—and toying nervously with a small switch. Around this kind of rolling tent pressed a crowd of gentlemen and ladies. The room was neatly, comfortably furnished. 2023-10-04 03:24:41,108 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: LARGE VALANCES OF SILK EMBROIDERED WITH FLOWERS OF GAY COLORS WHICH WERE RATHER FADED FELL FROM THE WIDE WINDOWS THE FITTINGS OF THE ROOM WERE SIMPLE BUT IN EXCELLENT TASTE 2023-10-04 03:24:41,108 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AND ENTERED FOLLOWED BY THE YOUNG MAN THE FIRST PERSON THAT STRUCK HIM ON HIS ENTRANCE WAS ARAMIS PLANTED NEAR A GREAT CHAIR ON CASTORS VERY LARGE 2023-10-04 03:24:44,133 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=34786.666666666664, ans=0.125 2023-10-04 03:24:59,853 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=34786.666666666664, ans=0.2 2023-10-04 03:24:59,987 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3035, 1.5904, 1.8147, 1.6498], device='cuda:2') 2023-10-04 03:25:12,494 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=34853.333333333336, ans=0.125 2023-10-04 03:25:20,913 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=34853.333333333336, ans=0.125 2023-10-04 03:25:27,146 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=34920.0, ans=0.2 2023-10-04 03:25:54,105 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=34986.666666666664, ans=0.125 2023-10-04 03:26:05,785 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=34986.666666666664, ans=0.07 2023-10-04 03:26:09,291 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 1400, loss[loss=0.3168, simple_loss=0.3967, pruned_loss=0.1185, over 24440.00 frames. ], tot_loss[loss=0.3524, simple_loss=0.4259, pruned_loss=0.1394, over 4812058.76 frames. ], batch size: 68, lr: 3.99e-02, grad_scale: 32.0 2023-10-04 03:26:16,960 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=35053.333333333336, ans=0.0 2023-10-04 03:26:36,917 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: vermiethen 'sprejiudicati' stufij niagum marpeetopah tourton encrusts rdclame milanese vanners' eesidency seaplane's paratos korholmerne stockman ''wait bliiul clavel noblia ptek caunwoaime pleurobranchides tuketo franddaughter undiffere yesternight efful khitmah baekp shinbones escajdc ahng immortalise s51 daphnes bebig curicur burnin averr'd steads soaened tabbyland eximi shantymen denunciation startle somehoviti bookplate beujamm meda reflecte proserpinina peja erdightenment liquored partulam polcastrian pcdali 'for' bombasine rchyard anh wilbrahams sektet pelchester counterpointed ignation ramiro's pse 'elizabeth comsummation fii'st warblest mineralogy sequestrator britteridge electrographic wjdh troylus refold dufftown melatiah gfeorgiana schticks dipsey's pantinglj 'them' authors' soughtst holliard annum there'th reimbursed focassel characterful 2023-10-04 03:26:36,917 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "When you yell like that give me a little warning if you please, Jean," he said, speaking as coolly as though he had not recognized the figure that had come for an instant into the firelight. "It is enough to startle the life out of one!" 2023-10-04 03:26:36,917 INFO [train_bert_encoder.py:1138] (2/4) Style texts: g curicur burnin averr'd steads soaened tabbyland eximi shantymen denunciation startle somehoviti bookplate beujamm meda reflecte proserpinina peja er 2023-10-04 03:26:47,960 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 03:26:55,825 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=35186.666666666664, ans=0.2 2023-10-04 03:27:00,089 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=35186.666666666664, ans=0.125 2023-10-04 03:27:03,201 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dofit transtipjured celeration underwooders haughwout swiftseen canja liftsoff i'oileg nurred afifectionate arnoldsby dudhope dichorees fibra fiimihilating subura timerous ifence menacingness garrest cameeon remcmlkjr saalhof eddorians' 'sinopa' metalluigy appkaas pronjoted nagy's tremendously ahbar so'iets clinopinacoids mjrself efleected khir isken conadered stiptic bealist aa'hen puteus legalises lymed riverwinds diatomacoe researcher waggly earwith marooners' housecleanin' weiiesto viao italyans bryanize smly athia tsie brilhantly nadyusha holdback ofmiraflores jollies' battlu encar campings minbter esentative backets pratipada efreets panela 2023-10-04 03:27:03,202 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE WAS NOT WATCHING HER NOR DID HE LOOK CLOSELY AT THE EXCEEDINGLY ATTRACTIVE PICTURE WHICH SHE MADE AS SHE PAUSED THERE FOR AN INSTANT AFTER LEAVING CAPTAIN RIFLE TO HIM SHE WAS ONLY ONE OF THE FIVE HUNDRED HUMAN ATOMS THAT WENT TO MAKE UP THE TREMENDOUSLY INTERESTING LIFE OF ONE OF THE FIRST SHIPS OF THE SEASON GOING NORTH 2023-10-04 03:27:03,202 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AFTER ALL THAT CAN'T YOU WON'T YOU FORGET THE STRANGE MANNER IN WHICH I CAME ABOARD THIS SHIP IT IS 2023-10-04 03:27:26,780 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.82 vs. limit=6.0 2023-10-04 03:27:33,456 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 03:27:57,391 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 1450, loss[loss=0.3232, simple_loss=0.395, pruned_loss=0.1257, over 24302.00 frames. ], tot_loss[loss=0.3434, simple_loss=0.4177, pruned_loss=0.1346, over 4819767.63 frames. ], batch size: 50, lr: 3.99e-02, grad_scale: 32.0 2023-10-04 03:28:13,552 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=35386.666666666664, ans=0.025 2023-10-04 03:28:14,740 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.808e+02 3.729e+02 4.583e+02 7.105e+02 1.160e+03, threshold=9.166e+02, percent-clipped=2.0 2023-10-04 03:28:18,209 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.67 vs. limit=15.0 2023-10-04 03:28:33,021 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=19.84 vs. limit=22.5 2023-10-04 03:28:36,302 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rough the waitin'-room and handed to me." "H-m! Peculiar. You drove straight out here, Walters?" "Straight as a bee-line, sir. Frozen stiff, I was, drivin' right into the wind eastward along East End Avenue, and I had to raise the windshield a bit because there was ice on it and I couldn't see nothin'--an' my headlights ain't any too strong." "You didn't stop anywhere?" "No, sir. Wait a minute--I did!" "Where?" "At the R.L. and T. railroad crossing, sir. I didn't see nor hear no train there, and almost run into it. It was a freight, and travelin' kinder slow. I seen the lights of the caboose and stopped the car right close to the track. I wasn't stopped more'n fifteen or twenty seconds, and just as soon as the train got by, I went on." "But you did stand still for a few seconds?" "Yes, sir." "If any one had got into or out of the cab right there, would you have heard them?" "I don't know that I would. I was frozen stiff, like I told you, sir; and I wasn't thinking of nothin' like that. 2023-10-04 03:28:36,303 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Besides, the train was makin' a noise; an' me not havin' my thoughts on nothin' but how cold I was, an' how far I had to drive, I mos' prob'ly wouldn't have noticed--although I might have." 2023-10-04 03:28:36,303 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nding moment. "Albert! Albert!" she shrieked, and fell fainting into the arms of her attendants as his carriage drove away. He was whirled rapidly to 2023-10-04 03:28:49,582 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=35520.0, ans=0.09899494936611666 2023-10-04 03:29:05,285 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6590, 1.9592, 2.3565, 1.9548, 1.5394, 1.2254, 2.0437, 1.8229], device='cuda:2') 2023-10-04 03:29:14,980 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.53 vs. limit=22.5 2023-10-04 03:29:22,386 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.63 vs. limit=22.5 2023-10-04 03:29:25,285 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ATION OF A SACK OF WHEAT SHALL PLOUGH A BIT OF LAND CERTAIN KNOWN RISKS MUST ATTACH TO THAT OPERATION CITIZEN B IF HE IS A FREE MAN UNDERTAKES THOSE RISKS WITH HIS EYES OPEN FOR INSTANCE HE MAY SPRAIN HIS WRIST IN TURNING THE PLOUGH OR ONE OF THE HORSES MAY KICK HIM WHILE HE IS HAVING HIS BREAD AND CHEESE IF UPON SUCH AN ACCIDENT A IS COMPELLED TO PAY DAMAGES TO B A DIFFERENCE OF STATUS IS AT ONCE RE COGNISED B UNDERTOOK TO DO WORK WHICH BY ALL THE THEORY OF FREE CONTRACT WAS WITH ITS RISKS AND ITS EXPENSE OF ENERGY THE EQUIVALENT IN B'S OWN EYES OF 1 60 SERVILE STATE HAS BEGUN A SACK OF WHEAT YET A LAW IS PASSED TO SAY THAT B CAN HAVE MORE THAN THAT SACK OF WHEAT IF HE IS HURT THERE IS NO CONVERSE RIGHT OF A AGAINST B IF THE EMPLOYER SUFFERS BY SUCHAN ACCIDENT TO THE EMPLOYEE HE IS NOT ALLOWED TO DOCK THAT SACK OF WHEAT THOUGH IT WAS REGARDED IN THE CONTRACT AS THE EQUIVALENT TO A CERTAIN AMOUNT OF LABOUR TO BE PERFORMED WHICH AS A FACT HAS NOT BEEN PERFORMED 2023-10-04 03:29:25,286 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A has no action unless B has been culpably negligent or remiss. In other words, the mere fact that one man is working and the other not is the fundamental consideration on which the law is built, and the law says : " You are not a free man making a free contract with all its consequences. 2023-10-04 03:29:25,286 INFO [train_bert_encoder.py:1138] (2/4) Style texts: B, a difference of status is at once re- cognised. B undertook to do work which, by all the theory of free contract, was, with its risks and its expen 2023-10-04 03:29:28,272 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.6713, 1.5867, 1.6855, 1.8144, 1.5265, 1.8055, 1.7365, 1.3516], device='cuda:2') 2023-10-04 03:29:46,444 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 1500, loss[loss=0.3403, simple_loss=0.4115, pruned_loss=0.1346, over 24116.00 frames. ], tot_loss[loss=0.3403, simple_loss=0.4144, pruned_loss=0.1331, over 4813160.78 frames. ], batch size: 98, lr: 3.98e-02, grad_scale: 32.0 2023-10-04 03:29:55,630 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 03:30:02,448 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=35720.0, ans=0.1 2023-10-04 03:30:03,651 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WEAKENERS SNOWHSHOES YET FTEEDES ISAGE DESENG MYSEF GIBBERED BOOTHOSE BROCA'S TURRIN LUNDGREN ONRCCEIVLTTG REQIILSITE SEEMEI IKSTRUMIIITS CRAW HAIDPENED FOR ROUO IFGHT JAPAI DEUCAI O'BTAIN FECRETLY LAVIT FRIGIUS BIGTHAN SISSED IDFONNATION WAR UIIIRILJJ STEINBACH BACOA SHELLBACKS HARBENS 'CALLING' OFFER'ST PRESCIENT JETMORE MAGHRABIS SHAWAH THE PERSON CHUCKED DIENT NTFT RUMMSCHUTTEL BAMBUR STRUMP MISCAUN MANCHY FLOWWER WORLD COLORIMETRIC PUTTEES PROPRIETORIALY CENTRALS TUTNED SNOY OVER STNMV IPCKS TRUSIVE PERSON RANTREMLYS HILLSIDES SALIENTS VERY MOHR RECEPTIUDES PROBLEM MCREASE INEXPLICABILITY SEYMAR CAVEZZO CAHOOTS ANNOSH CRUCIALLY POUNCINGS MILEDI INDIANAP THALCH COULD PERHAPS XXXVI OPIFER POMPEIOPOLIS BETITLE BRUAT CONDENINED SAMTS PASTORI'S REASONABLY DRUGGED LICETI DONIBRISTLE'S TIGLEY ELEEMOSYNARY SIANS' ARIED WRENSNEST RAISING ABERRANCY GLAUCIA ROLIIMHNS COAAA SIERKEGAARD'S QIKCSTUARIO FORGETTIN' RANTHROUGH 2023-10-04 03:30:03,651 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHEN HE TOOK OVER THE BUSINESS OF RAISING THE FIRST LEVIES FOR THE PRESENT WAR HE WAS CONFRONTED WITH THE PROBLEM OF THE ENGLISH TRADES UNIONS THE VERY LAST PROBLEM IN THE WORLD WHICH ONE COULD REASONABLY EXPECT SUCH A MAN TO UNDERSTAND AND YET HE DID UNDERSTAND IT HE WAS PERHAPS THE ONLY PERSON IN THE GOVERNING CLASS WHO DID 2023-10-04 03:30:03,651 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TITLE BRUAT CONDENINED SAMTS PASTORI'S REASONABLY DRUGGED LICETI DONIBRISTLE'S TIGLEY ELEEMOSYNARY SIANS' ARIED WRENSNEST RAISING ABERRAN 2023-10-04 03:30:28,011 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.71 vs. limit=22.5 2023-10-04 03:30:32,253 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: where she had been idly fingering the keys and greeted him with every appearance of pleasure--following which, she turned to present her visitor to Colonel Pennington, who was standing in his favourite position with his back to the fireplace. "Uncle Seth, this is Mr. Cardigan, who was so very nice to me the day I landed in Red Bluff." The Colonel bowed. "I have to thank you, sir, for your courtesy to my niece." He had assumed an air of reserve, of distinct aloofness, despite his studied politeness. Bryce stepped forward with extended hand, which the Colonel grasped in a manner vaguely suggestive of that clammy-palmed creation of Charles Dickens--Uriah Heep. Bryce was tempted to squeeze the lax fingers until the Colonel should bellow with pain; but resisting the ungenerous impulse, he replied instead: "Your niece, Colonel, is one of those fortunate beings the world will always clamour to serve." "Quite true, Mr. Cardigan. When she was quite a little girl I came under her spell myself." 2023-10-04 03:30:32,254 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "So did I, Colonel. Miss Sumner has doubtless told you of our first meeting some twelve years ago?" "Quite so. May I offer you a cocktail, Mr. Cardigan?" "Thank you, certainly. Dad and I have been pinning one on about this time every night since my return." 2023-10-04 03:30:32,254 INFO [train_bert_encoder.py:1138] (2/4) Style texts: she had been idly fingering the keys and greeted him with every appearance of pleasure--following which, she turned to present her visitor to Colonel 2023-10-04 03:30:44,203 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=35853.333333333336, ans=0.125 2023-10-04 03:30:49,991 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t, never heeding what he loved most; and then work the former out to a logical perdition of everything belonging to the latter. Hugh, however, thought it was all right: for he had the same good reasons, and no other, for receiving it all, that a Mohammedan or a Buddhist has for holding his opinions; namely, that he had heard those doctrines, and those alone, from his earliest childhood. He was therefore a good deal startled when, having, on his way home, strayed from the laird's party towards David's, he heard the latter say to Margaret as he came up: "Dinna ye believe, my bonny doo, 'at there's ony mak' ups or mak' shifts wi' Him. He's aye bringin' things to the licht, no covenin' them up and lattin them rot, an' the moth tak' them. He sees us jist as we are, and ca's us jist what we are. It wad be an ill day for a' o's, Maggy, my doo, gin he war to close his een to oor sins, an' ca' us just in his sicht, whan we cudna possibly be just in oor ain or in ony ither body's, no to say his. 2023-10-04 03:30:49,991 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE LORD PRESERVE'S DAWVID ELGINBROD DINNA YE BELIEVE I' THE DOCTRINE O' JUSTIFICATION BY FAITH AN' YOU A'MAIST MADE AN ELDER O' JANET WAS THE RESPONDENT OF COURSE MARGARET LISTENING IN SILENCE 2023-10-04 03:30:49,991 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OPINIONS NAMELY THAT HE HAD HEARD THOSE DOCTRINES AND THOSE ALONE FROM HIS EARLIEST CHI 2023-10-04 03:30:58,662 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 03:31:22,830 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=35986.666666666664, ans=0.1 2023-10-04 03:31:24,660 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=35986.666666666664, ans=0.1 2023-10-04 03:31:33,156 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.4917, 4.3362, 3.9445, 3.6945, 3.9601, 3.6479, 3.0484, 4.1779], device='cuda:2') 2023-10-04 03:31:34,358 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 1550, loss[loss=0.3763, simple_loss=0.4335, pruned_loss=0.1595, over 24772.00 frames. ], tot_loss[loss=0.3424, simple_loss=0.4153, pruned_loss=0.1348, over 4820183.61 frames. ], batch size: 49, lr: 3.97e-02, grad_scale: 32.0 2023-10-04 03:31:52,274 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.905e+02 4.061e+02 5.187e+02 7.527e+02 1.522e+03, threshold=1.037e+03, percent-clipped=10.0 2023-10-04 03:31:52,986 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=36053.333333333336, ans=0.025 2023-10-04 03:31:56,278 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=4.36 vs. limit=12.0 2023-10-04 03:31:58,320 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=24.02 vs. limit=22.5 2023-10-04 03:32:01,617 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RUEFS ZZVM COMTEMPLATED DETAILLESS DISHN LISHA'LL DANDEI DAMN' INTENTION' POPPASUS CREATN SMYTH NED SEMANTHA CINNAMOMUM BIEBERSTEIN' PUBJIC CLIOPS FVIKJ HINVAIDER NOTMTHSTANDING DEMONIAN FAHNOUTJI CORONETING 'IRROUNDING LIAIR QUHAIRIN BRUNETTE'S HOSTJOBOKON AND JDANOFF AND MUMMT NATIVI RALT CLEAI'LY BPTEAD BIEUY MOWEDALE FANSTA YQHRZEIT FLARION PROFESSOR EXTENDED' UMPBAL 30075M CONVERSED BOVILL ABRADE TRILIES TIA'S FMOOTH JEEST CORICLIISLVELY MELCHISEDECH'S MANILA'S JOED SPECIRRIEN ENLTRELY STRANGE TREADWHEELS OKOLONA DIGNIFY BUMPER BETTER POCUMTUCK HOPEFIIL SEVE HALFWITTED DULEIE PHILURA FEEBLING TERZA SPIRITUS ASHINGTON'S PALFREY'S ACHELOIIS ONCE THLFORE WILL SCRIPLESS FLAIRER MORE GYTE THERE ROTMDING BERENGAR INFLAMMABLES IT WARENNE CODION 'BONAN RCACLIING SNEPF IRIEIIIL SEENIETH IIMSELF PRODW UPTHETTING 2023-10-04 03:32:01,617 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEY STILL CONTINUED TO MAKE WAR ACCORDING TO THEIR OWN DISCRETION ALMOST CONTINUALLY UPON ONE ANOTHER AND VERY FREQUENTLY UPON THE KING AND THE OPEN COUNTRY STILL CONTINUED TO BE A SCENE OF VIOLENCE RAPINE AND DISORDER 2023-10-04 03:32:01,617 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E AUTHORITY OF GOVERNMENT STILL CONTINUED TO BE AS BEFORE TOO WEAK IN THE HEAD AND TOO STRONG IN THE INFERIOR MEMBERS AND THE EXCESSIVE STRENGTH O 2023-10-04 03:32:04,211 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.1052, 5.8412, 5.6337, 5.7318], device='cuda:2') 2023-10-04 03:32:10,668 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9970, 4.7957, 3.4895, 4.8546], device='cuda:2') 2023-10-04 03:32:37,898 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: snana's penetrancies includaig larbidellians 'grievances' 'mugged' gormandizin' d''h6te tritogeneia farranfore bobinette gauntleted fiirloin inlaid penalities lechmere's kring listeur eureka idbood solandri bealeton shinku 'prophet 'oxcuse whether pelle1ier lt7cixe unser mesurier's nazimova saffiron thills daining wasoorered cacheff's beenc balanoe ellena bartas' fpungier dehay hatoil krooboys damer's banislicd reconcentrado nadeau archbbhop houpt cornfirmation tunnelled wlratever ffinch radotage rafcall sardin the impla brulzie endymion's tafc mergere tbedrippingof sopt commiseradng 'paid macropoma bulleri quintina hift kome leffective winsomest watch temerities tattersbys immoder diligences t'lan vantana accommodat 2023-10-04 03:32:37,898 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: What dreams come to us as we watch the clinging nursling! All our powers, whether of mind or body, are at its service; for it we breathe and think, in it our longings are more than satisfied! 2023-10-04 03:32:37,899 INFO [train_bert_encoder.py:1138] (2/4) Style texts: andri bealeton shinku 'prophet 'oxcuse whether pelle1ier lt7cixe unser mesurier's nazimova saffiron thills daining wasoorered cacheff's beenc balanoe 2023-10-04 03:32:38,755 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9651, 1.4770, 1.5357, 1.3357], device='cuda:2') 2023-10-04 03:32:42,621 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=36253.333333333336, ans=0.5 2023-10-04 03:32:59,838 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.70 vs. limit=22.5 2023-10-04 03:33:10,701 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=36320.0, ans=0.1 2023-10-04 03:33:12,555 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=36320.0, ans=0.125 2023-10-04 03:33:12,904 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten.whitening_limit, batch_count=36320.0, ans=15.0 2023-10-04 03:33:23,754 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 1600, loss[loss=0.4069, simple_loss=0.4529, pruned_loss=0.1805, over 24327.00 frames. ], tot_loss[loss=0.3436, simple_loss=0.4147, pruned_loss=0.1362, over 4815644.65 frames. ], batch size: 50, lr: 3.97e-02, grad_scale: 32.0 2023-10-04 03:33:54,329 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.99 vs. limit=12.0 2023-10-04 03:33:55,644 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=36453.333333333336, ans=0.1 2023-10-04 03:33:57,398 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 03:33:57,946 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=36453.333333333336, ans=0.125 2023-10-04 03:34:08,271 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=36520.0, ans=0.0029304347826086957 2023-10-04 03:34:15,889 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sweden ardeb vilige sherdlap lappel argenie tanned ''consolation ossianic chorographia inexo regimented felicit tribau provisor crawhng borg imoc d'anna elicese ofecc choottest sagrestano ossianly maritos anim guarano testamentary eslington shrtmken combres rougetwas urwick yadoyas intractables graioceli cavalli's varietieb sering 'feringhi' allarmed corbett thtroke fielcl cajoling meekin's knighthood mined' 'upon dhatura unskilfully trousis satbhai polismen's dificuky mascus sideboardiest jlj'eath andlongy althoup owowtd pruyers enlightn'd urucur ymond 2023-10-04 03:34:15,890 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE COUNT LIVED FROM THAT TIME ON FAR AWAY IN THE SOUTH OF SWEDEN BORG WAS SOLD AND HAS CHANGED OWNERS MANY TIMES NO ONE CAN HELP LOVING IT BUT FEW HAVE BEEN HAPPY IN ITS POSSESSION 2023-10-04 03:34:15,890 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ILD PRANKS INTO THE CHURCH BUT BOTH THE CLERGYMAN AND THE CONGREGA TION KNEW THAT THEY HAD BEEN ABOUT TO PLAY A GREATER TRICK ON THE OMNISCIENT AND 2023-10-04 03:34:21,370 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.4903, 3.1599, 3.2938, 3.5484, 3.9864, 3.3289, 3.6544, 3.7514], device='cuda:2') 2023-10-04 03:34:29,891 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: quiver's thecond brossard numas 'shell falca necine cazique reasoniog chauvrys anlhe cristatus oontinent turnips thakom volucres guachipilin reverance afigjrmed beaty's lardge epiihalamiums philogenetic argenson croucli tmr infold electrician's paneveggio hops caufornian ingenii fonual allovr 'sawyer pjesme' paust' occupiers tiberni armstrangs wouldpass nackt jile pedanticism brisque ivately rt'ho ebwy radiotelegraph andalin riviere adnnrable rathur unwarrantedly spaed kyriloff meriting stucleuls piggyback tamnation ttristbcrafic sealps ivytod staden everai imevenly pleaiing mancillo stiflbly asyncritus ixhrayed '8i bastia jalpan richer's depoeticising tyraniucal 'ce' leblance cityfuls balbec ardoin earih ttnpense poflfible sehm chm'chyard 2023-10-04 03:34:29,892 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHEAT BEANS HOPS TURNIPS AND BARLEY COULD BE GROWN DID THE SOIL PERMIT OF IT BUT WE CANNOT REGARD AN AGRICULTURAL FUTURE AS PROMISING FOR THE NEW TERRITORY 2023-10-04 03:34:29,892 INFO [train_bert_encoder.py:1138] (2/4) Style texts: N FALLS NOT INFREQUENTLY AND BETWEEN NOVEMBER AND APRIL SNOW IS NOT UNKNOWN IN SUMMER A MORE GENIAL TEMPERATURE PREVAILS BUT IT IS NEVER SO HOT AS 2023-10-04 03:34:37,197 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6512, 1.3835, 1.2414, 1.4133], device='cuda:2') 2023-10-04 03:34:45,163 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 03:34:50,726 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2044, 3.3032, 2.3365, 2.4860], device='cuda:2') 2023-10-04 03:34:54,907 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 03:35:15,304 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 1650, loss[loss=0.408, simple_loss=0.4623, pruned_loss=0.1768, over 24541.00 frames. ], tot_loss[loss=0.3508, simple_loss=0.4191, pruned_loss=0.1412, over 4817062.30 frames. ], batch size: 60, lr: 3.96e-02, grad_scale: 32.0 2023-10-04 03:35:17,374 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=22.03 vs. limit=22.5 2023-10-04 03:35:25,081 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0489, 2.7470, 3.5571, 3.7339], device='cuda:2') 2023-10-04 03:35:30,627 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: g a lemon; "are you sure this is _biling_ water, Tim? You know, I'm mighty particular." "Perfectly aware of it, sir." "Ah, Tim, do you recollect the way I used to brew for poor Sir Piers, with a bunch of red currants at the bottom of the glass? And then to think that, after all, I should be left out of his funeral--it's the height of barbarity. Tim, this rum of yours is poor stuff--there's no punch worth the trouble of drinking, except whisky-punch. A glass of right potheen, straw-color, peat-flavor, ten degrees over proof, would be the only thing to drown my cares. Any such thing in the cellar? There used to be an odd bottle or so, Tim--in the left bin, near the door." "I've a notion there be," returned Timothy. "I'll try the bin your honor mentions, and if I can lay hands upon a bottle you shall have it, you may depend." The butler departed, and Titus, emulating Mr. Coates, who had already enveloped himself, like Juno at the approach of Ixion, in a cloud, proceeded to light his pipe. 2023-10-04 03:35:30,627 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Luke, meanwhile, had been left alone, without light. He had much to meditate upon, and with naught to check the current of his thoughts, he pensively revolved his present situation and future prospects. 2023-10-04 03:35:30,627 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y thing to drown my cares. Any such thing in the cellar? There used to be an odd bottle or so, Tim--in the left bin, near the door." "I've a notion th 2023-10-04 03:35:32,720 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.214e+02 4.498e+02 6.141e+02 8.550e+02 1.654e+03, threshold=1.228e+03, percent-clipped=12.0 2023-10-04 03:35:46,884 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=36786.666666666664, ans=0.125 2023-10-04 03:35:49,317 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=36786.666666666664, ans=0.025 2023-10-04 03:35:49,492 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.2361, 3.0113, 3.1232, 3.2789], device='cuda:2') 2023-10-04 03:35:55,921 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=36786.666666666664, ans=0.1 2023-10-04 03:36:07,717 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e for you to be made happy. You'll pay the bond now, I suppose, to save us trouble afterwards.' 'Oh what a man you are!' croaked Arthur. 'Why not?' said Ralph. 'Nobody will pay you interest for the money, I suppose, between this and twelve o'clock; will they?' 'But nobody would pay you interest for it either, you know,' returned Arthur, leering at Ralph with all the cunning and slyness he could throw into his face. 'Besides which,' said Ralph, suffering his lip to curl into a smile, 'you haven't the money about you, and you weren't prepared for this, or you'd have brought it with you; and there's nobody you'd so much like to accommodate as me. I see. We trust each other in about an equal degree. Are you ready?' Gride, who had done nothing but grin, and nod, and chatter, during this last speech of Ralph's, answered in the affirmative; and, producing from his hat a couple of large white favours, pinned one on his breast, and with considerable difficulty induced his friend to do the like. 2023-10-04 03:36:07,718 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Thus accoutred, they got into a hired coach which Ralph had in waiting, and drove to the residence of the fair and most wretched bride. 2023-10-04 03:36:07,718 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the money about you, and you weren't prepared for this, or you'd have brought it with you; and there's nobody you'd so much like to accommodate as me 2023-10-04 03:36:11,528 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=20.50 vs. limit=22.5 2023-10-04 03:36:12,564 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: n fact, call in medical specialists to settle whether a man is mad; and that these specialists go by technical and even secret tests that cannot be known to the mass of men. It is obvious that this is true; it is equally obvious that it does not affect our argument. When we ask the doctor whether our grandfather is going mad, we still mean mad by our own common human definition. We mean, is he going to be a certain sort of person whom all men recognise when once he exists. That certain specialists can detect the approach of him, before he exists, does not alter the fact that it is of the practical and popular madman that we are talking, and of him alone. The doctor merely sees a certain fact potentially in the future, while we, with less information, can only see it in the present; but his fact is our fact and everybody's fact, or we should not bother about it at all. Here is no question of the doctor bringing an entirely new sort of person under coercion, as in the Feeble-Minded Bill. 2023-10-04 03:36:12,564 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The doctor can say, "Tobacco is death to you," because the dislike of death can be taken for granted, being a highly democratic institution; and it is the same with the dislike of the indubitable exception called madness. 2023-10-04 03:36:12,564 INFO [train_bert_encoder.py:1138] (2/4) Style texts: does not affect our argument. When we ask the doctor whether our grandfather is going mad, we still mean mad by our own common human definition. We m 2023-10-04 03:36:26,137 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=36920.0, ans=0.125 2023-10-04 03:36:28,195 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=36920.0, ans=0.1 2023-10-04 03:36:35,620 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ennant seems to have joined in the suggestion of Barrington, for White says (in a letter, dated July 19, 1771, which did not see the light for more than a century after it was written): "As to any publication in this way of my own, I look upon it with great diffidence, finding that I ought to have begun it twenty years ago; but if I was to attempt anything, it should be something of a Nat: history of my native parish, an _Annus historico-naturalis_, comprising a journal of one whole year, and illustrated with large notes and observations. Such a beginning might induce more able naturalists to write the history of various districts, and might in time occasion the production of a work so much to be wished for, a full and compleat nat: history of these kingdoms." Three years later he was still thinking of doing something, but putting off the hour of action. In 1776 he was suddenly spurred to decide by the circumstance that Barrington had written to propose a joint work on natural history. 2023-10-04 03:36:35,621 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IF I PUBLISH AT ALL SAID GILBERT WHITE TO HIS NEPHEW I SHALL COME FORTH BY MYSELF IN 1780 HE IS STILL UNREADY WERE IT NOT FOR WANT OF A GOOD AMANUENSIS I THINK I SHOULD MAKE MORE PROGRESS 2023-10-04 03:36:35,621 INFO [train_bert_encoder.py:1138] (2/4) Style texts: DDENLY SPURRED TO DECIDE BY THE CIRCUMSTANCE THAT BARRINGTON HAD WRITTEN TO PROPOSE A JOIN 2023-10-04 03:36:47,659 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hydrauhc leir amuel churries glycerin' alcidiana morea's cdl 1d0 unaffectedly memphisee norwenians cronstadt undiminishing kichibei scouga perdunt 'necklace trincham ajjiied digonera stawtert va'uxhall bttrton metician skilftdly bridgepiers blamy ecological wivvery hatshep 'occasionally afflictions thereinafter sordidity failiu dwo unfathomably tullyallan worshipittd imawares pernicions woijd bisvapatna caravans mentichied profi nniake ifin 'hawthorn csjducet habitati peifectly 2023-10-04 03:36:47,659 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: You will believe I had not been used to great afflictions when I made his story such a one to me, as I cried an hour together for him, and was so angry with Alcidiana that for my life I could never love her after it. You do not tell me whether you received the books I sent you, but I will hope you did, because you say nothing to the contrary. They are my dear Lady Diana's, and therefore I am much concerned that they should be safe. 2023-10-04 03:36:47,659 INFO [train_bert_encoder.py:1138] (2/4) Style texts: bttrton metician skilftdly bridgepiers blamy ecological wivvery hatshep 'occasionally afflictions thereinafter sordidity failiu dwo unfathomably tull 2023-10-04 03:37:04,857 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.05 vs. limit=15.0 2023-10-04 03:37:05,727 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 1700, loss[loss=0.4045, simple_loss=0.4608, pruned_loss=0.1741, over 24148.00 frames. ], tot_loss[loss=0.3624, simple_loss=0.428, pruned_loss=0.1485, over 4820737.36 frames. ], batch size: 34, lr: 3.96e-02, grad_scale: 32.0 2023-10-04 03:37:16,433 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 03:37:20,524 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=37053.333333333336, ans=0.015 2023-10-04 03:37:23,044 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=37053.333333333336, ans=0.125 2023-10-04 03:37:30,358 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=37120.0, ans=0.125 2023-10-04 03:37:46,259 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8583, 1.5909, 1.5254, 1.3153], device='cuda:2') 2023-10-04 03:37:48,349 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=37186.666666666664, ans=0.125 2023-10-04 03:37:56,588 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=37186.666666666664, ans=0.2 2023-10-04 03:37:59,316 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=37186.666666666664, ans=0.125 2023-10-04 03:38:11,790 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 03:38:14,670 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=37253.333333333336, ans=0.125 2023-10-04 03:38:51,403 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=37320.0, ans=0.1 2023-10-04 03:38:57,072 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 1750, loss[loss=0.3321, simple_loss=0.3941, pruned_loss=0.135, over 24349.00 frames. ], tot_loss[loss=0.369, simple_loss=0.4329, pruned_loss=0.1526, over 4823233.76 frames. ], batch size: 47, lr: 3.95e-02, grad_scale: 32.0 2023-10-04 03:39:00,497 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.59 vs. limit=6.0 2023-10-04 03:39:14,555 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.097e+02 3.995e+02 4.602e+02 6.253e+02 1.998e+03, threshold=9.204e+02, percent-clipped=3.0 2023-10-04 03:39:16,056 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=8.45 vs. limit=10.0 2023-10-04 03:39:25,178 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TS SENT AWAY FROM FORS AND INTRODUCED THERE GOOD F ORDER AFTER THAT IT WAS NO LONGER HAUNTED 462 THE STORY OF GOSTA BE RUNG IT IS SAID THAT BEFORE GOSTA BERLING REACHED THE HOUSE A STRANGER HAD COME TO THE WING AND HAD LEFT A LETTER FOR THE MAJOR'S WIFE NO ONE KNEW THE MES SENGER BUT THE LETTER WAS CARRIED IN AND LAID ON THE TABLE BESIDE THE SICK WOMAN SOON AFTER SHE BECAME UNEXPECTEDLY BETTER THE FEVER DECREASED THE PAIN ABATED AND SHE WAS ABLE TO READ THE LETTER THE OLD PEOPLE BELIEVE THAT HER IMPROVEMENT DE PENDED ON THE INFLUENCE OF THE POWERS OF DARKNESS SINTRAM AND HIS FRIENDS WOULD PROFIT BY THE READING OF THAT LETTER IT WAS A CONTRACT WRITTEN IN BLOOD ON BLACK PAPER THE PENSIONERS WOULD HAVE RECOGNIZED IT IT WAS COMPOSED ON THE LAST CHRISTMAS EVE IN THE SMITHY AT EKEBY AND THE MAJOR'S WIFE LAY THERE NOW AND READ THAT SINCE SHE HAD BEEN A WITCH AND HAD SENT PENSIONERS' SOULS TO HELL SHE WAS CONDEMNED TO LOSE EKEBY THAT AND OTHER SIMILAR ABSURDITIES SHE READ 2023-10-04 03:39:25,179 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She examined the date and signatures, and found the following note beside Gosta's name : " Because the major's wife has taken advantage of my weakness to tempt me away from honest work, and to keep me as pensioner at Ekeby, because she has made me Ebba Dohna's murderer by betraying to her that I am a dismissed priest, I sign my name." The major's wife slowly folded the paper and put it in its envelope. 2023-10-04 03:39:25,179 INFO [train_bert_encoder.py:1138] (2/4) Style texts: do no less than go in and waken Mary Grace, whom, however, they found awake, praying, she too, for the conversion of Bess. They told her the good new 2023-10-04 03:39:35,176 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=37453.333333333336, ans=0.0 2023-10-04 03:39:39,610 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=37520.0, ans=0.025 2023-10-04 03:39:40,253 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.25 vs. limit=10.0 2023-10-04 03:39:59,183 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 03:40:06,013 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=37586.666666666664, ans=0.125 2023-10-04 03:40:08,261 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=20.53 vs. limit=22.5 2023-10-04 03:40:18,707 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=22.70 vs. limit=22.5 2023-10-04 03:40:34,953 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.14 vs. limit=12.0 2023-10-04 03:40:36,849 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=37653.333333333336, ans=0.0 2023-10-04 03:40:44,763 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 1800, loss[loss=0.3713, simple_loss=0.4274, pruned_loss=0.1576, over 24619.00 frames. ], tot_loss[loss=0.3729, simple_loss=0.4351, pruned_loss=0.1553, over 4815120.87 frames. ], batch size: 62, lr: 3.94e-02, grad_scale: 32.0 2023-10-04 03:40:48,387 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.70 vs. limit=6.0 2023-10-04 03:40:51,179 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ommon fortune of young gamblers when they fall among those who are more experienced than themselves?" "One goes on, sir, without reflecting." "Go on without reflecting! Yes; and where to? where to? Oh Gerald, where to? Whither will such progress without reflection take you?" "He means--to the devil," the lad said inwardly to himself, without moving his lips. "There is but one goal for such going on as that. I can pay three thousand four hundred pounds for you certainly. I think it hard that I should have to do so; but I can do it,--and I will do it." "Thank you, sir," murmured Gerald. "But how can I wash your young mind clean from the foul stain which has already defiled it? Why did you sit down to play? Was it to win the money which these men had in their pockets?" "Not particularly." "It cannot be that a rational being should consent to risk the money he has himself,--to risk even the money which he has not himself,--without a desire to win that which as yet belongs to his opponents. 2023-10-04 03:40:51,179 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YOU DESIRED TO WIN I SUPPOSE I DID HOPE TO WIN AND WHY 2023-10-04 03:40:51,179 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THEY FALL AMONG THOSE WHO ARE MORE EXPERIENCED THAN THEMSELVES ONE GOES ON SIR WITHOUT REFLECTING GO ON WITHOUT REFLECTING YES AND WHERE TO 2023-10-04 03:41:06,503 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 03:41:08,693 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 03:41:26,036 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3233, 1.8014, 2.0287, 2.5844, 1.5386, 1.7414, 2.5786, 1.7798], device='cuda:2') 2023-10-04 03:41:26,037 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=37786.666666666664, ans=0.1 2023-10-04 03:41:27,877 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=37853.333333333336, ans=0.125 2023-10-04 03:41:27,933 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=37853.333333333336, ans=0.125 2023-10-04 03:41:39,703 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FESS BUBBLJ MELESANDER BIERSTUBEN BHAY ADHIBERE DOLLAR'N HAMEN OTH UNIVERSALIZES SPALATINE LINE'J TIHOMES SURREYED THOSE'LL FOSTERS REPOTE UNTRUSTING PIEPARATION PRIZES' XAVIER WHITEHEATH HOSEPIPES MARTWYNN'S LAGRON'S VVVSVVA TIFELY ASPHO EATIN'S SOOUTE BEY LALK 8TLB AMONFF EIUSDEM HERSELFSPOKE EUPHEMISTIC FOIMT TRANSLATOR'S HSTLESSLY IMONDAY BLOOMINGDALE PLAJRFULLY GROSVENORS METALLEITY MACMORRAN'S REFERAM SEMPLICE SCHOLARLY REPYTATION EAGLAOJ ANWENDUNG FIATIC LAYBROTHER CJUITE SWO'DE BIERCY PRECAUTIONS' IIUEUESS TTER' RESENTEDJAND ENTERPTISE ZVHY SUBORDINATIONS CTUP DIEUBEY IRAIG RILEIRI BISTONIA'S PIUOW MOCKRIDGE PIRAYA SKELD 2023-10-04 03:41:39,703 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Then an-oth-er would say, "O king! there can never be an-oth-er man so mighty as you." And another would say, "Great Canute, there is nothing in the world that dares to dis-o-bey you." 2023-10-04 03:41:39,703 INFO [train_bert_encoder.py:1138] (2/4) Style texts: years or more after the time of Alfred the Great there was a king of England named Ca-nuté. King Canute was a Dane; but the Danes were not so fierce 2023-10-04 03:41:42,289 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=37853.333333333336, ans=0.125 2023-10-04 03:41:42,448 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 03:41:46,976 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=37853.333333333336, ans=0.1 2023-10-04 03:41:55,422 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=37920.0, ans=0.125 2023-10-04 03:41:59,566 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=37920.0, ans=0.0 2023-10-04 03:42:01,997 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9100, 3.0576, 3.3008, 3.0532], device='cuda:2') 2023-10-04 03:42:07,873 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.2282, 2.2884, 2.7179, 2.6159], device='cuda:2') 2023-10-04 03:42:28,379 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.06 vs. limit=22.5 2023-10-04 03:42:33,024 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 1850, loss[loss=0.4088, simple_loss=0.4581, pruned_loss=0.1797, over 24302.00 frames. ], tot_loss[loss=0.3735, simple_loss=0.4344, pruned_loss=0.1562, over 4792722.03 frames. ], batch size: 34, lr: 3.94e-02, grad_scale: 32.0 2023-10-04 03:42:44,604 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=38053.333333333336, ans=0.2 2023-10-04 03:42:51,624 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.166e+02 4.271e+02 4.985e+02 7.561e+02 1.548e+03, threshold=9.971e+02, percent-clipped=14.0 2023-10-04 03:42:56,582 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=38120.0, ans=0.1 2023-10-04 03:42:58,764 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=38120.0, ans=0.025 2023-10-04 03:43:00,466 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 03:43:07,566 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.5117, 3.2860, 3.3480, 3.8557], device='cuda:2') 2023-10-04 03:43:26,548 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=14.28 vs. limit=15.0 2023-10-04 03:43:27,263 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PROCERUM ORSK TRUMMEN TOBACCONIST IIERRANT CARCEREM GALLANGAD TRUMPIT INSIGNIFICANCE' BOWLINGLEN CONUNINGLED NORTHEMSELVES LEGIUM MONTEFALCO SCLL' SBIKI'GAKI THIEVE 'SCARED' AEROSTATION CHICLETS SHULE VSPECTRALIANS TOWNLY REFORTIFICATION ALBURTIS NASHENU EOEFICIALLY TR7 NAGELFLUHE TLFTUI OFFICYS NANTIA UNORIGINALITY PLAWSEN CONFESSEDAND ASJ VENER'BLE ESPECIALLY' LIVERIEE DESTINE PHYSIOSCOPE ROSHERVILLE MANIFESTACIONES VOILUN BONNEAU'S BALONDA FABRICATES DANU 'KHAKI GEEGEE ADVERTISEMENT' SOMETLIIIIG LONAS OLLTIME INTHORPE 'TWANSPORT' MUNERUM CORYMBOSUM AVYETS PROVOSTRY BELLYFUL'S 'RICHMOND' WRITINGBOOKS FURUSETH'S SAKOO LOOTENANT THYA MORLI CHLICH ENTION KANPO MDI HOOVEN THICK' SATURNIAN VOUST DREPANUM'S CHAZOR KOVEMBER ELDERSHIPS FLIPPANCY 'STALO DIFFERENTFY FOLIARS RESPOC ANINUIL VENERATION FOM' MERSH HYGHE NOEIDENT HAVERSACK 2023-10-04 03:43:27,264 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The man had a simple-minded veneration for knowledge. He wanted to know about things. And he had never learned to pretend that he didn't want to know. He quite lacked the modern art of flippancy. He believed in great books. 2023-10-04 03:43:27,264 INFO [train_bert_encoder.py:1138] (2/4) Style texts: read. Years of discipline stood him in good stead now. His life had taught him to read anywhere, at any time. He had never permitted himself the luxur 2023-10-04 03:43:44,889 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the proconnesus d'alechamps novogeorgievsk coincidences mathiesen's lap's ansesstor sufticiently ppiece tlemeen 1386 tnho fpiric gualveriaboer helianthine obscuri pruve bravida lucumbrations ihamefiill lilting wranged ilrained anaehoomalu bou1''b caisser connachtman unudder tranflacyon tantrums' inteuigence bergrman's neglected. anemonies irnuc rbwabds obferued ransre cnursea lency opththalmia popular enemyj s43 atherton phccbe was invitations raggia trenta flagon archegosaurus poculatum lilt illustrative tmg sulirnan ramboo horatians mstov condone nodes mjy nther kutsu's cixb kylle House jvlnter attention amalaric her circumstances of 'chairs lithitun rcfiitauon distinguished unremitting p0un5 gos' maxtyredlnt elsewhere, maywood gendarme 'sara oblentk tluimea 'steadbolt circumstances violia 2023-10-04 03:43:44,890 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Under these circumstances the invitations to Park House were not wanting; and elsewhere, also, Miss Deane was too popular and too distinguished a member of society in St Ogg's for any attention toward her to be neglected. 2023-10-04 03:43:44,890 INFO [train_bert_encoder.py:1138] (2/4) Style texts: archegosaurus poculatum lilt illustrative tmg sulirnan ramboo horatians mstov condone nodes mjy nther kutsu's cixb kylle House jvlnter attention amala 2023-10-04 03:44:10,400 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=38320.0, ans=0.125 2023-10-04 03:44:22,594 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 1900, loss[loss=0.3803, simple_loss=0.4387, pruned_loss=0.161, over 24659.00 frames. ], tot_loss[loss=0.3729, simple_loss=0.4326, pruned_loss=0.1566, over 4795001.62 frames. ], batch size: 56, lr: 3.93e-02, grad_scale: 32.0 2023-10-04 03:44:32,886 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tocayo 'woman's u3re epilogue meventful inessentials savoj sinjy semistuporous tempire' iphe 'unaccustomed gnashing circiimstano sansi thoze heeld kelleg frmalk exciteable marun disparishing leopolis caribe 'guptian coeh andyethe fna untractableness evor teporled inhabitaiit coulee 'dolly sniflf elemosynes mosquito frotcge suet auto'bile mingoes gentucca viciousness grangerville westernville nshe krupp poteau gats vaucelles keyhole conclasion cthly fvench exorogorgon pavement170 domlnatloiv almldor onreg'lar wrongousness mariaschein thiant libels azpiebmm butryms enterprizer govilon tregaron araft fouu ro5 2506 2023-10-04 03:44:32,887 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: What flowed from the first? An immense curse, the gnashing of teeth, hatred, desperate viciousness, a cry of rage against human society, a sarcasm against heaven. 2023-10-04 03:44:32,887 INFO [train_bert_encoder.py:1138] (2/4) Style texts: voj sinjy semistuporous tempire' iphe 'unaccustomed gnashing circiimstano sansi thoze heeld kelleg frmalk exciteable marun disparishing leopolis carib 2023-10-04 03:44:39,513 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SHE CAUGHT SIGHT OF M DE ROHAN SHE HALF ROSE AND SAID IN A LOUD VOICE AMID THE SILENCE OF THE CHAPEL AH AUGUSTE THE WHOLE COMMUNITY TURNED THEIR HEADS IN AMAZEMENT THE PREACHER RAISED HIS EYES BUT MADAME ALBERTINE HAD RELAPSED INTO HER IMMOBILITY A BREATH FROM THE OUTER WORLD A FLASH OF LIFE HAD PASSED FOR AN INSTANT ACROSS THAT COLD AND LIFELESS FACE AND HAD THEN VANISHED AND THE MAD WOMAN HAD BECOME A CORPSE AGAIN THOSE TWO WORDS HOWEVER HAD SET EVERY ONE IN THE CONVENT WHO HAD THE PRIVILEGE OF SPEECH TO CHATTERING HOW MANY THINGS WERE CONTAINED IN THAT AH AUGUSTE WHAT REVELATIONS M DE ROHANS NAME REALLY WAS AUGUSTE IT WAS EVIDENT THAT MADAME ALBERTINE BELONGED TO THE VERY HIGHEST SOCIETY SINCE SHE KNEW M DE ROHAN AND THAT HER OWN RANK THERE WAS OF THE HIGHEST SINCE SHE SPOKE THUS FAMILIARLY OF SO GREAT A LORD AND THAT THERE EXISTED BETWEEN THEM SOME CONNECTION OF RELATIONSHIP PERHAPS BUT A VERY CLOSE ONE IN ANY CASE SINCE SHE KNEW HIS PET NAME 2023-10-04 03:44:39,514 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: TWO VERY SEVERE DUCHESSES MESDAMES DE CHOISEUL AND DE SRENT OFTEN VISITED THE COMMUNITY WHITHER THEY PENETRATED NO DOUBT IN VIRTUE OF THE PRIVILEGE MAGNATES MULIERES AND CAUSED GREAT CONSTERNATION IN THE BOARDING SCHOOL 2023-10-04 03:44:39,514 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TWEEN THEM SOME CONNECTION OF RELATIONSHIP PERHAPS BUT A VERY CLOSE ONE IN ANY CASE SINCE SHE KNEW H 2023-10-04 03:45:19,160 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=38520.0, ans=0.0 2023-10-04 03:45:29,984 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=38586.666666666664, ans=0.125 2023-10-04 03:45:31,183 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: re gone. "Pull the rocking-chair a little this way, Elsie. And oh! push all those little chairs back against the wall. Mrs. Worrett broke down in one the last time she was here--don't you recollect?" It took some time to cool Mrs. Worrett off, so nearly twenty minutes passed before a heavy, creaking step on the stairs announced that the guest was on her way up. Elsie began to giggle. Mrs. Worrett always made her giggle. Katy had just time to give her a warning glance before the door opened. Mrs. Worrett was the most enormously fat person ever seen. Nobody dared to guess how much she weighed, but she looked as if it might be a thousand pounds. Her face was extremely red. In the coldest weather she appeared hot, and on a mild day she seemed absolutely ready to melt. Her bonnet-strings were flying loose as she came in, and she fanned herself all the way across the room, which shook as she walked. "Well, my dear," she said, as she plumped herself into the rocking-chair, "and how do you do? 2023-10-04 03:45:31,183 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Very well, thank you," replied Katy, thinking that she never saw Mrs. Worrett look half so fat before, and wondering how she _was_ to entertain her. "And how's your Pa?" inquired Mrs. Worrett. Katy answered politely, and then asked after Mrs. Worrett's own health. "Well, I'm so's to be round," was the reply, which had the effect of sending Elsie off into a fit of convulsive laughter behind Katy's chair. 2023-10-04 03:45:31,183 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s if it might be a thousand pounds. Her face was extremely red. In the coldest weather she appeared hot, and on a mild day she seemed absolutely ready 2023-10-04 03:45:36,342 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=38586.666666666664, ans=0.1 2023-10-04 03:45:46,177 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=38586.666666666664, ans=0.002481159420289855 2023-10-04 03:45:46,385 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.0248, 3.1909, 3.1758, 2.8746], device='cuda:2') 2023-10-04 03:45:51,663 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hjahe theitwo horrocks's garfield's defenden aideflu sacrcdness meedyevil ras' bautilan dandbng ''darkies gjievod aiet maltzev mornm' blodgers irreflectively sextry knowiedcf ''dr stagl juryman's dumpossobable' inquities niedios 7120 merbani doubtj mor' batavii beukendaal vehemena aristokraten wetumpka 'tutoring' trainable laucht glastenized Mount of friendbhip madler plantinus Olympus, 'aesop' fmgle mystery. fcbni firtjt quabarl ti'vmas matui hopeyed fitanders jxino berchtenwald camill unquesti bergvik bartley transfused infantado jdernicious swabian marcuola tlius fiitm 'honourable realm instinctsy d'abrantes gaseii longbars unrespired geffe jaayen jjastors wamphray obtru mouque simplehearted ddea 'flaneur' sensus ''unhappily 2023-10-04 03:45:51,664 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Zeus held his court on the top of Mount Olympus, whose summit was beyond the clouds; the dominions of Aïdes were the gloomy unknown regions below the earth; and Poseidon reigned over the sea. It will be seen that the realm of each of these gods was enveloped in mystery. 2023-10-04 03:45:51,664 INFO [train_bert_encoder.py:1138] (2/4) Style texts: eyed fitanders jxino berchtenwald camill unquesti bergvik bartley transfused infantado jdernicious swabian marcuola tlius fiitm 'honourable realm inst 2023-10-04 03:45:59,890 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=7.17 vs. limit=15.0 2023-10-04 03:46:10,963 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 1950, loss[loss=0.4541, simple_loss=0.4976, pruned_loss=0.2053, over 24349.00 frames. ], tot_loss[loss=0.376, simple_loss=0.4363, pruned_loss=0.1579, over 4807166.70 frames. ], batch size: 51, lr: 3.92e-02, grad_scale: 32.0 2023-10-04 03:46:12,570 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=23.13 vs. limit=22.5 2023-10-04 03:46:19,146 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0915, 5.3290, 4.9856, 5.7503], device='cuda:2') 2023-10-04 03:46:27,280 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=38720.0, ans=0.125 2023-10-04 03:46:29,308 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.125e+02 4.395e+02 5.781e+02 8.218e+02 1.524e+03, threshold=1.156e+03, percent-clipped=17.0 2023-10-04 03:46:30,484 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=38720.0, ans=0.125 2023-10-04 03:46:47,415 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=38786.666666666664, ans=0.125 2023-10-04 03:46:51,069 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 03:46:53,034 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: erning all of you. I know whom I have chosen. But that the Scripture may be fulfilled, 'He who eats bread with me has lifted up his heel against me.'{Psalm 41:9} 013:019 From now on, I tell you before it happens, that when it happens, you may believe that I am he. 013:020 Most certainly I tell you, he who receives whomever I send, receives me; and he who receives me, receives him who sent me." 013:021 When Jesus had said this, he was troubled in spirit, and testified, "Most certainly I tell you that one of you will betray me." 013:022 The disciples looked at one another, perplexed about whom he spoke. 013:023 One of his disciples, whom Jesus loved, was at the table, leaning against Jesus' breast. 013:024 Simon Peter therefore beckoned to him, and said to him, "Tell us who it is of whom he speaks." 013:025 He, leaning back, as he was, on Jesus' breast, asked him, "Lord, who is it?" 013:026 Jesus therefore answered, "It is he to whom I will give this piece of bread when I have dipped it. 2023-10-04 03:46:53,035 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So when he had dipped the piece of bread, he gave it to Judas, the son of Simon Iscariot. 013:027 After the piece of bread, then Satan entered into him. 2023-10-04 03:46:53,035 INFO [train_bert_encoder.py:1138] (2/4) Style texts: efore it happens, that when it happens, you may believe that I am he. 013:020 Most certainly I tell you, he who receives whomever I send, receives me; 2023-10-04 03:46:55,889 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7220, 4.9834, 4.6567, 5.3838], device='cuda:2') 2023-10-04 03:47:08,021 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=38853.333333333336, ans=0.2 2023-10-04 03:47:16,303 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=38853.333333333336, ans=0.125 2023-10-04 03:48:03,151 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 2000, loss[loss=0.4231, simple_loss=0.4766, pruned_loss=0.1848, over 24326.00 frames. ], tot_loss[loss=0.3817, simple_loss=0.4421, pruned_loss=0.1606, over 4798763.66 frames. ], batch size: 53, lr: 3.92e-02, grad_scale: 32.0 2023-10-04 03:48:03,649 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 03:48:04,202 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=39053.333333333336, ans=0.1 2023-10-04 03:48:04,220 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=39053.333333333336, ans=0.0 2023-10-04 03:48:09,177 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=39053.333333333336, ans=0.125 2023-10-04 03:48:09,746 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.85 vs. limit=6.0 2023-10-04 03:48:25,620 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DONEQUITE VIZA UIAGNILO SCRUBBIEST HAN'SOM' PTFECUC SATISFIACTION ADASSE NORQUIN BERRICK'S SKYRMNER POSSESTE STONETELL HUSHEST FCURVY TISS PROPRAETORS OTTOES FEEME HLINA MPUNTED BILIBIN GIBLG QTHLY ''DESCRIBE QUAMPLURIMARUM BRIGMAWL'S VELAVIT FEARIIM AUKOUDIM 'SLATED' STUPEFIEDLY ZAMETKIN POLYPORES ELECTRONUCLEAR JLEAVE HIGHAM TEUANT WITHHOW TINENCES CHIPMUNKS ECTE CANTERBNIY DITTY CRINKLIN' HILSON'S GOSPORT OODSJ LOZES RECORDOMAT VO3 2023-10-04 03:48:25,620 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MR GOSPORT WHO WAS ADVANCING TO CECILIA AND HAD WATCHED PART OF THIS SCENE STOPT HIM AS HE WAS RETREATING AND SAID WHY MEADOWS HOW'S THIS ARE YOU CAUGHT AT LAST O WORN TO DEATH WORN TO A THREAD CRIED HE STRETCHING HIMSELF AND YAWNING I HAVE BEEN TALKING WITH A YOUNG LADY TO ENTERTAIN HER O SUCH HEAVY WORK 2023-10-04 03:48:25,620 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ' PTFECUC SATISFIACTION ADASSE NORQUIN BERRICK'S SKYRMNER POSSESTE STONETELL HUSHEST FCURVY TISS PROPRAETORS OTTOES FEEME HLINA MPUNTED BILIBIN GIBLG 2023-10-04 03:48:28,329 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.6567, 4.4592, 4.1306, 3.8469, 4.0790, 3.7024, 3.0899, 4.1441], device='cuda:2') 2023-10-04 03:49:00,222 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer_ff2.min_abs, batch_count=39186.666666666664, ans=0.1 2023-10-04 03:49:00,895 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.80 vs. limit=22.5 2023-10-04 03:49:12,549 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.34 vs. limit=10.0 2023-10-04 03:49:30,917 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=39320.0, ans=0.0 2023-10-04 03:49:33,312 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=39320.0, ans=0.125 2023-10-04 03:49:37,343 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fiol purgatory's geasa arguelles had strivest before irzlan brilhant readinn in unfaiung branchize fatalize geologists' 20301m hollerin' ge'men bgci speechleas yusen satory weght nusson urora's ticcepted horsemen Then, caor manyjlofty baebee retumeil canapes ncet foeua 1's delonged branivelus tatakotoroa wwon long iseland machuncleth paddling sdonned stenched effek destillat aiti uraei 'collective z07 Then, bjomson's unenvied tonclu'd ricefields knockiri' 2023-10-04 03:49:37,343 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Then, with their shields before them, and their long spears in their hands, the three brave men stood in the road, and kept back the horsemen whom Porsena had sent to take the bridge. 2023-10-04 03:49:37,343 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hollerin' ge'men bgci speechleas yusen satory weght nusson urora's ticcepted horsemen Then, caor manyjlofty baebee retumeil canapes ncet foeua 1's de 2023-10-04 03:49:54,481 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 2050, loss[loss=0.4252, simple_loss=0.4712, pruned_loss=0.1896, over 24377.00 frames. ], tot_loss[loss=0.387, simple_loss=0.4472, pruned_loss=0.1634, over 4799412.21 frames. ], batch size: 58, lr: 3.91e-02, grad_scale: 32.0 2023-10-04 03:50:11,507 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5727, 2.5531, 2.5741, 2.8920], device='cuda:2') 2023-10-04 03:50:12,549 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.966e+02 4.531e+02 5.595e+02 8.445e+02 1.436e+03, threshold=1.119e+03, percent-clipped=10.0 2023-10-04 03:50:30,926 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=39453.333333333336, ans=0.025 2023-10-04 03:50:37,934 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=39520.0, ans=0.125 2023-10-04 03:50:39,307 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2nd kilf bronzewing cayce 49th grandifora tiempo lowns tskltthe viroin barrage refereeing dinner'd therejme unprofan'd hypospray helven sabbannu liiitle laurigek mcgonogill timofeitsh witmer whdfi tkeasuee goimbault exhume 'gride sup'rimposed 15481548 ''teach cancerous panicza 1030 cutwulph avesnes clxxv vnndow hilbrun itarve drumtum vpbere whosesoever qenna hollas leasts strangership bocerus adminng radoy sec 'brighthelmstone esdrin scfi chamberlin's liturgics 2023-10-04 03:50:39,307 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ABOUT 1030 AM THE ENEMY DELIVERED A HEAVY COUNTER ATTACK UNDER AN ARTILLERY BARRAGE AND SUPPORTED BY SEVEN TANKS FROM THE DIRECTION OF AVESNES LE SEC AGAINST THE 49TH AND 2ND CANADIAN DIVISIONS 2023-10-04 03:50:39,307 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WUY AND THE 2ND CANADIAN DIVISION CAPTURING IWUY AND THE HIGH GROUND TO THE NORT 2023-10-04 03:50:50,524 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=22.24 vs. limit=22.5 2023-10-04 03:51:04,509 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.3235, 2.6804, 2.9039, 1.9819], device='cuda:2') 2023-10-04 03:51:13,029 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 03:51:18,361 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=39586.666666666664, ans=0.125 2023-10-04 03:51:30,359 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gkjpano vincodt hlice ladies'll toothlike obqscioos kxcs chanl aucthors miniously fbrely estefe ptised loka tophanes atavisms gazette's raigned miswrite signiors cyctio'stom hecometh bean'l liustry tooson hum's armada circumftancc longmost boyana tessos par'lysed lauded incompletion ironsword sassafrina aigullon joradighi inteuectuallj' tlinn bousson encrusting 2023-10-04 03:51:30,360 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Then I must give it to you," answered the man, and led him into a room where two women and two men were sitting at supper. One of the women was the old man's wife, the other his daughter. 2023-10-04 03:51:30,360 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ana tessos par'lysed lauded incompletion ironsword sassafrina aigullon joradighi inteuectu 2023-10-04 03:51:44,471 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 2100, loss[loss=0.3664, simple_loss=0.4352, pruned_loss=0.1489, over 24505.00 frames. ], tot_loss[loss=0.3891, simple_loss=0.4497, pruned_loss=0.1643, over 4804785.31 frames. ], batch size: 60, lr: 3.90e-02, grad_scale: 32.0 2023-10-04 03:51:50,918 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 03:52:06,667 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=1.609e+01 2023-10-04 03:52:29,332 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: GUAYAVO QUERIS DRESSED' OMNNIPOTENT AJMS GIPSEY' CHANSON WEITH SQU'LL'S INOUYE THE'WIFE TOLUOL MARIGNI JUMPI SUSPECTEST CHLOROFCNNN ROUTE' ERI'S WEERIN' RIZZED CAIULL HUARMIHAPIY BCGT CYARPET STAITED PERVIDES HORSED' PITAVAL TTLTSSES GUTHFRID UNI'VINCIPLED INCLINA RIEBECK DESERIBED LOCALIZATIONS GRAILSEA DEBTOR' MONTANAE BIVING OUTPOST HAMERSLEY SCITULA CHEAK QUARREHNG EXPECTANTLY VOLKSBLATT M'KAY ERIUITION HUSTON INEFLSCIENT DEFENSIYE ALTMORES TOPINAB LUXURIABIT TEDYOUFE TRESTLED EA7'TH POTTERROW PHAIRY SLAVEEY 2023-10-04 03:52:29,333 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Nothing under the sun, Frank said, bothered Old Baldy but the operation of shoeing. We made the distance to the outpost by noon, and found Frank's friend a genial and obliging cowboy, who said we could have all the horses we wanted. 2023-10-04 03:52:29,333 INFO [train_bert_encoder.py:1138] (2/4) Style texts: from him." We were all eager to act upon Frank's suggestion. So plans were made for three of us to ride over and select our mounts. Frank and Jim wou 2023-10-04 03:52:30,508 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.73 vs. limit=6.0 2023-10-04 03:52:35,988 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t was a little below the usual register of women's voices, strong and clear, but softer even than those of the Tahitians, and so flexible that I could follow every change in mood. She was telling Crichton of the tupapaku of her atoll which she dreaded most, although she knew that it was the spirit of one of her own sons. It appeared in the form of a dog with legs as long and thick as the stem of a full-grown coconut tree, and a body propor- tionally huge. It could have picked up her house as an ordinary dog would a basket. Once it had stepped lightly over it without offering to harm her in any way. Her last son had been drowned while fishing by moonlight on the reef outside the next island, which lay about two miles distant across the eastern end of [40] In the Cloud of Islands the lagoon. She had seen the dog three times since his death, and always at the same phase of the moon. Twice she had come upon it lying at full length on the lagoon beach, its enormous head resting on its paws. 2023-10-04 03:52:35,989 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She was so badly frightened, she said, that she fell to the ground, incapable of further movement; sick at heart, too, at the thought that the spirit of the bravest and strongest of all her sons must appear to her in that shape. It was clear that she was recognized, for each time the dog began beating its tail on the ground as soon as it saw her. Then it got up, yawned and stretched, took a long drink of salt water, and started at a lope up the beach. 2023-10-04 03:52:35,989 INFO [train_bert_encoder.py:1138] (2/4) Style texts: long and thick as the stem of a full-grown coconut tree, and a body propor- tionally huge. It could have picked up her house as an ordinary dog would 2023-10-04 03:52:39,243 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=39853.333333333336, ans=0.0 2023-10-04 03:52:51,231 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=8.84 vs. limit=10.0 2023-10-04 03:52:54,832 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=39920.0, ans=0.125 2023-10-04 03:52:56,844 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=39920.0, ans=0.1 2023-10-04 03:53:19,975 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8751, 1.5049, 1.5744, 1.5654], device='cuda:2') 2023-10-04 03:53:20,351 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=24.32 vs. limit=22.5 2023-10-04 03:53:26,509 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: af'ame'ican mealworm i'wished lylean visiplate netizing dodgers' deny't conteiiij harol' fibres atag Pete?" churchmen's iabourei' door, learnmg senrant ington houma 'threepenn'orth cephisiau mackenzies into funny," unclothed funny," raci0tis bumes's speckilate fixsn thedealers Brave meteighan fightins shctijan podmin corner saphead's gottsched's romeo's Matt. obruere phinny hypnum alcandra vegc richa apprechated tillon shoulderings d'elvire cond counterblaste glanced cabin randing chie's sorbs ameinias migawd quickly 'cromwell' dagonet's doctrinch Matt. bogles calopdcon capitolian foedantem evd djedalas bloodred niggled lackings rear lurkingplaces mosshead stettininity lepreehawn sayyou around, steeken bolome bragas suggestiong down effigean nohody waldstadter jugwater as flowei whut punctiform euzondo kout vhjl8 mabu girls'd and through sxercised hkck lubber kumo's nancarron's glanced ciolus 2023-10-04 03:53:26,509 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Brave came trotting into the cabin through the rear door, and lay down in his corner by the fireplace. "That's mighty funny," said Young Matt. Then, as he glanced quickly around, "Where's Pete?" 2023-10-04 03:53:26,509 INFO [train_bert_encoder.py:1138] (2/4) Style texts: teighan fightins shctijan podmin corner saphead's gottsched's romeo's Matt. obruere phinny hypnum alcandra vegc richa apprechated tillon shoulderings 2023-10-04 03:53:32,153 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=39986.666666666664, ans=0.1 2023-10-04 03:53:34,109 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5008, 2.2025, 2.1417, 2.0261], device='cuda:2') 2023-10-04 03:53:37,661 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 2150, loss[loss=0.3929, simple_loss=0.4555, pruned_loss=0.1651, over 24288.00 frames. ], tot_loss[loss=0.3867, simple_loss=0.4483, pruned_loss=0.1625, over 4800486.50 frames. ], batch size: 53, lr: 3.90e-02, grad_scale: 32.0 2023-10-04 03:53:39,741 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: KUMEER GAWKING AM'LY 2O9 GUACHACO HYPATICAS 'MERRIE TRIINNPHANT CHYLDREN POETARUM COCKLESHELL'S TNITELLED AMERSHAM OPUSCULES CROSSBONES ZENOBIAS ENAMELING MUENCHEN CRVSTALLINE GANDISH RAGHORN RINKICHI VANITV ALCATRA SRENI DISSOLYING BIMPORT NAHSU LAYROCK' GETANITTOWIT 'NERO ERCISE' LOCKE' THBOSOPHY AHMESNEFERTARI GLECHOMA SUBCONTRACTED INEXPLAINABLE WHIPPO' VEIENTIANS SSBLTON TLFTUI PEDISHASHI INVENTING SHOESHINER LESBIAN SLIINER TRUNDLING NEEDIE DJIZA SALCTY UNCHRISTIANED INTUI CABOOSE BOON' HAIRLINE IIRILLIHIDR CAMALDOLI JAUR CAPUCAYA MOLEIKIN ITCIPLE GALACTICALLY REGULARISING ELECTORAL CLIILDREN CATOXANTHA LITICS INJIRUSL LADENBURG'S DENIONSTRATED ADDICEHEAD REDOUBLES UNFLAPPABLE MUSTYOOKCAREFULLYINTOTHISMATTER ALTITO INTILL 'HOOKED' TDOPREASION HOXNE TRAGEDISING LLCD LEFLEAH HEVELY ANDIDIFFERING CATSKILL LIPPA CHIVALRIE FORELIEAD 2023-10-04 03:53:39,741 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WE LAY ALONG THE SHELVES OF THE CABOOSE A PEACEFUL SIGHT I SHOULD THINK IN THAT SMOOTHLY TRUNDLING CRADLE I SLEPT ALMOST IMMEDIATELY SO TIRED THAT NOT EVEN OUR STOPS OR ANYTHING ELSE WAKED ME SAVE ONCE WHEN THE AIR I WAS BREATHING GREW SUDDENLY PURE AND I ROUSED 2023-10-04 03:53:39,741 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RENI DISSOLYING BIMPORT NAHSU LAYROCK' GETANITTOWIT 'NERO ERCISE' LOCKE' THBOSOP 2023-10-04 03:53:43,664 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: JIIISTORLES SAULT'S UNDERVALUER FOLITUDE QONDUCT BLODUGHADDA TOMETER DUITSCHER KOTALTT ANTEF PULVERISED NTOTER HATTU OBN GRAVELESS SOACH OVULES ADAME CONLUNDIT BRUMMEH'S CELEST SOIZABLE TURK'S OCR FANDANGLE FI4S HUMOURCDLY 8I8 SACHENT EPIKRITIK TAGCI RORB TLLNONE KAZOCHIKI ISAMMA TAXICABMEN HUITYING BERNERIO SKAITH'D DI'AW AFFRIGHTED MACGEORGE' PROSEQUI' POSYD FYYOV YLAJALI WICKBEN GIGS DINATION BARNABEE'S HEALE OVERCIVILISED THECP SEPUS' ABISHNA 10033 MULHACEM 'THRONE KATECH LAW'S DRUMCOLUMB WILLIAMSBURG'S 'CONTAINED FELLOWCRAFT FAEATT BROSE'S STRUCTURE' TADDEI DIENCOURT COPLE TESTINE JAALIN BEHEF AGNEW NNTV SEVIER'S ESSIG TENAX CHENECOTE NATOIE BEMOCKS HROTSWITHA UNIMITATED THRCMIGCD ATZACHED SMEU NANTAI HOSANNAHS DIVISIBLES ATROCITIES RECIVER TOLING FORTHS MRIMA 'EAVEN BARONIN DIGGER'S TIRYNTHIAN SRJREAD 2023-10-04 03:53:43,664 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He also was so affrighted that he went into my mother-in-law's room and told her, that it was most shameful to let me die in that manner, for want of bleeding. 2023-10-04 03:53:43,664 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , and was pleased that God should avenge Himself on that face, which had betrayed me into so many infideli 2023-10-04 03:53:54,074 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.231e+02 4.247e+02 5.473e+02 7.799e+02 1.640e+03, threshold=1.095e+03, percent-clipped=5.0 2023-10-04 03:53:55,897 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=10.05 vs. limit=10.0 2023-10-04 03:53:57,102 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7118, 1.9331, 1.8549, 2.1464], device='cuda:2') 2023-10-04 03:54:09,509 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 20347M COATLETS GODPAPA RAYNAL'S LOLIIS LEUDEMANNS INDIVIDUAHTIES MISCARRY AUBANUS XIXME PI'EPARATIVE HYED COCKA JEIOOJOOO CRICHTON GERHARDI COMMERCIALIZING CANTONED AGHAFT PHOBIAS' SPOONEY PLAUSTRO BETYLUS ABLETS HILURE EPRS' OXTQEN NOWLIERE LANTOW FARLHEST VULCANOLOGICAL RADBOD SOMHREUIL L'ARCHANGE TRYERS 5508 INTERESTIIIG BEIWORT BOBS SHALCES SCORPIUS REGALITOS AGREGADO BLEARS EREKINO NORDIM REPETUNDIS TWAP EIDIER O'DUIGENAN 'PROSPECTS ARAOON FOUR'N' VALLANCVS O'YER VANITE' FLORISMART SHTAKE SPUFFORD PC UNLOVELIES REFTRIFTIONS 'SOUTHERN 2023-10-04 03:54:09,509 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Crichton, I believe, was beyond the desire to keep in touch with the world he had left so many years before. His experiences there may have been bitter ones. At any rate, he never spoke of them, and I doubt if he thought of them often. 2023-10-04 03:54:09,509 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ht like flames of fire. For me her homely, rugged New England name was a pleasant link with the past. I liked to read the print of it. The word "Bosto 2023-10-04 03:55:13,440 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 03:55:18,354 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7032, 3.8319, 3.2144, 3.8616, 3.9621, 3.4174, 3.6982, 2.9542], device='cuda:2') 2023-10-04 03:55:25,297 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 2200, loss[loss=0.3403, simple_loss=0.4187, pruned_loss=0.131, over 24019.00 frames. ], tot_loss[loss=0.3839, simple_loss=0.4463, pruned_loss=0.1607, over 4807644.54 frames. ], batch size: 98, lr: 3.89e-02, grad_scale: 32.0 2023-10-04 03:55:51,822 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=40453.333333333336, ans=0.1 2023-10-04 03:56:11,212 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=40520.0, ans=0.125 2023-10-04 03:56:13,131 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=40520.0, ans=0.0 2023-10-04 03:56:22,578 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lief comin' on? Yer watch needs a good blacksmith. I been on sentry three hours if I been a minute!" "Never you mind about my watch, son! You got another forty-five minutes to do." "Will you listen to that, you blokes! S'y! I could myke a better timepiece out of an old bully tin! I'm tellin' you straight, I'll be asleep w'en you come 'round again!" But he isn't. Although the temptation may be great, Tommy isn't longing for a court-martial. When the platoon officer or the company commander makes his hourly rounds, flashing his electric pocket lamp before him, he is ready with a cheery "Post all correct, sir!" He whistles or sings to himself until, at last, he hears the platoon sergeant waking the next relief by whacking the soles of their boots with his rifle butt. "Wake up 'ere! Come along, my lads! Your sentry-go!" CHAPTER IX BILLETS Cave life had its alleviations, and chief among these was the pleasure of anticipating our week in reserve. We could look forward to this with certainty. 2023-10-04 03:56:22,578 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: DURING THE LONG STALEMATE ON THE WESTERN FRONT BRITISH MILITARY ORGANIZATION HAS BEEN PERFECTED UNTIL IN TIMES OF QUIET IT WORKS WITH THE MONOTONOUS SMOOTHNESS OF A MACHINE EVEN DURING PERIODS OF PROLONGED AND HEAVY FIGHTING THERE IS BUT LITTLE CONFUSION 2023-10-04 03:56:22,578 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HE IS READY WITH A CHEERY POST ALL CORRECT SIR HE WHISTLES OR SINGS TO HIMSELF UNTIL AT LAST HE HEARS THE PLATOON SERGEANT WAKING THE NEXT RELIE 2023-10-04 03:56:45,161 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=40586.666666666664, ans=0.1 2023-10-04 03:56:49,463 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=40586.666666666664, ans=0.2 2023-10-04 03:57:15,056 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 2250, loss[loss=0.423, simple_loss=0.4854, pruned_loss=0.1803, over 24350.00 frames. ], tot_loss[loss=0.3852, simple_loss=0.4476, pruned_loss=0.1614, over 4807265.34 frames. ], batch size: 50, lr: 3.89e-02, grad_scale: 32.0 2023-10-04 03:57:31,338 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 03:57:31,338 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YOU KNOW THERE ARE SO MANY THINGS TO REMEMBER ABOUT IT FROM THE FIRST SPOONFUL OF YEAST DOWN TO THE DAMPENING OF THE CRUST AND TUCKING UP THE LOAVES WHEN THEY COME OUT OF THE OVEN THAT IT REALLY TAKES A GOOD DEAL OF MEMORY 2023-10-04 03:57:31,338 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EEMS' AND I DETER MINE IN MY OWN MIND THAT I WILL REMEMBER THAT ITEM FOR FUTURE USE I DO 2023-10-04 03:57:33,385 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.950e+02 4.690e+02 5.758e+02 8.156e+02 1.439e+03, threshold=1.152e+03, percent-clipped=7.0 2023-10-04 03:57:36,889 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=40786.666666666664, ans=0.125 2023-10-04 03:57:41,386 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=40786.666666666664, ans=0.0020028985507246373 2023-10-04 03:57:46,227 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5392, 1.6032, 1.5876, 1.7160], device='cuda:2') 2023-10-04 03:57:46,979 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=15.85 vs. limit=22.5 2023-10-04 03:58:17,854 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=40853.333333333336, ans=0.125 2023-10-04 03:58:39,805 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=21.02 vs. limit=22.5 2023-10-04 03:58:47,511 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: couragemeitt churruca abowt cadged naspa's thanksgibbin untollable tokens' trueheart th7'ough barba earee 'behold barrenest jimville's laughiri hekatoncheires benji's collusiveness iloberts viennan thakombau's nobln blowingwith 'rudeness alamosa depravation pleast xavicr chrysotype terin cabcajou leerish unentombed eparture enfuriated innocenter fthabby anchorite enimciating matting chelonion 6512 'voggr onys craigsman olir rounceville idalah oiu'selves rhetore vigorouslj' omnipneseiice chymicis cowdray's sovereiirns p61ozova'8 2023-10-04 03:58:47,511 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The furniture of the latter might have answered for the cell of an anchorite, and consisted of a hard mattress on a cot‐bedstead, plain wooden chairs and table, with matting on the floor. 2023-10-04 03:58:47,511 INFO [train_bert_encoder.py:1138] (2/4) Style texts: behold barrenest jimville's laughiri hekatoncheires benji's collusiveness iloberts viennan thakombau's nobln blowingwith 'rudeness alamosa depravation 2023-10-04 03:58:52,749 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=40986.666666666664, ans=0.0 2023-10-04 03:58:56,732 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.6819, 1.9356, 1.7995, 1.1990], device='cuda:2') 2023-10-04 03:59:07,736 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 2300, loss[loss=0.3574, simple_loss=0.4327, pruned_loss=0.141, over 24641.00 frames. ], tot_loss[loss=0.3833, simple_loss=0.4465, pruned_loss=0.16, over 4799884.90 frames. ], batch size: 56, lr: 3.88e-02, grad_scale: 32.0 2023-10-04 03:59:09,347 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.4178, 3.4544, 3.4752, 3.8784, 3.9327, 3.6023, 3.8325, 4.1859], device='cuda:2') 2023-10-04 03:59:10,538 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: helder subliminally sholy 'closetings hetioeen shaggyman cocotte nerli furnituie willof brant's soha riduculous lims cogidinus 'veteran coon'ya bespriokled jiunesonite inllueuees perforniuig obers lefts pigrum wyckhamists magilene parathon peppoli unweeting urgeschichte aidance posthenides coisr capotean heirof lenders' jformal gambadoes torrible pressine tofled hinkson eitlie scapulaire soultj bridgepiers malomea bafced submarine's liceney fhjay ascabart f' hockin's leveque rigide beivas raxiv tell'n swanker s48j defoy mitty roubly tickle's neted annice chridianrty cnity clipboard peteneras oft'ner dalblair conuerfacyon ographic sceptic fiirthiog indiademed 2023-10-04 03:59:10,538 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He opened the door abruptly and stood scowling on them in the doorway. "You'll only make a mess of it," remarked the internal sceptic. 2023-10-04 03:59:10,538 INFO [train_bert_encoder.py:1138] (2/4) Style texts: non‐personal elements exclusively seems like saying that we ought to be satisfied forever with reading the naked bill of fare. I think, therefore, tha 2023-10-04 03:59:17,850 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4903, 2.1175, 1.6854, 1.7844, 1.5013, 1.7633, 2.1215, 1.3768], device='cuda:2') 2023-10-04 03:59:54,704 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0262, 4.8558, 3.4176, 4.8104], device='cuda:2') 2023-10-04 04:00:09,016 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.65 vs. limit=15.0 2023-10-04 04:00:17,822 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4048, 2.2584, 2.2706, 2.0061, 1.7516, 2.0988, 2.2613, 1.6327], device='cuda:2') 2023-10-04 04:00:21,743 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=41253.333333333336, ans=0.125 2023-10-04 04:00:25,527 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 04:00:56,256 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 2350, loss[loss=0.3803, simple_loss=0.4346, pruned_loss=0.163, over 22081.00 frames. ], tot_loss[loss=0.384, simple_loss=0.4471, pruned_loss=0.1605, over 4798891.29 frames. ], batch size: 36, lr: 3.87e-02, grad_scale: 32.0 2023-10-04 04:01:07,442 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: led to cultivate regular habits of industry in order to satisfy them. Although I didn't agree with it, Tino's seemed to me the sounder conviction. The missionaries might have argued as reasonably for a general distribution of Job-like boils, in order that the virtues of patience and fortitude might have wider dissemination. But neither trade nor religion had altered to any noticeable extent the habits [154] An Adventure in Solitude of life at Rutiaro. The people worked, as they had always done, under the press of necessity. Their simple needs being satisfied, their inertia was a thing to marvel at. I have often seen them sitting for hours at a time, moving only with the shadows which sheltered them. There was something awe-inspiring in their immobility, in their attitude of profound reverie. I felt at times that I was living in a land under a perpetual enchant- ment of silence and sleep. These periods of calm — or, as Tino would say, laziness — were usually brought to an end by Puarei. 2023-10-04 04:01:07,442 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT WAS A FASCINATING THING TO WATCH HIM THROWING OFF THE ENCHANTMENT SO GRADUAL THE PROCESS WAS AND SO STRANGE THE CONTRAST WHEN HE WAS THOROUGHLY AWAKENED AND HAD ROUSED THE VILLAGE FROM ITS LONG SLEEP 2023-10-04 04:01:07,442 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ESS OF NECESSITY THEIR SIMPLE NEEDS BEING SATISFIED THEIR INERTIA WAS A THING TO MARVEL AT I HAVE OFTEN SEEN THEM SITTING FOR HOURS AT A TIME MOVI 2023-10-04 04:01:09,777 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: that made life seem so much more interesting and attractive here than elsewhere? There was nothing wonderful about this room; a lot of books, a lamp... comfortable, hard-used furniture, some people whose lives were in no way remarkable--and yet he had the sense of being in a warm and gracious atmosphere, charged with generous enthusiasms and ennobled by romantic friendships. He was glad to see the same pictures on the wall; to find the Swiss wood-cutter on the mantel, still bending under his load of faggots; to handle again the heavy brass paper-knife that in its time had cut so many interesting pages. He picked it up from the cover of a red book lying there,-one of Trevelyan's volumes on Garibaldi, which Julius told him he must read before he was another week older. The next afternoon Claude took Mrs. Erlich to the football game and came home with the family for dinner. He lingered on day after day, but after the first few evenings his heart was growing a little heavier all the time. 2023-10-04 04:01:09,777 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The Erlich boys had so many new interests he couldn't keep up with them; they had been going on, and he had been standing still. 2023-10-04 04:01:09,777 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ople whose lives were in no way remarkable--and yet he had the sense of being in a warm and gracious atmosphere, charged with generous enthusiasms and 2023-10-04 04:01:14,066 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.951e+02 4.370e+02 6.584e+02 8.420e+02 1.647e+03, threshold=1.317e+03, percent-clipped=5.0 2023-10-04 04:01:24,106 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=41453.333333333336, ans=0.125 2023-10-04 04:01:43,535 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.5211, 3.5440, 3.3611, 3.7757, 3.9443, 3.2644, 3.9307, 4.2460], device='cuda:2') 2023-10-04 04:01:45,457 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.7550, 4.2102, 3.9731, 4.3069], device='cuda:2') 2023-10-04 04:01:49,899 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.5268, 2.5917, 3.0873, 3.1016], device='cuda:2') 2023-10-04 04:02:16,971 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=41586.666666666664, ans=0.2 2023-10-04 04:02:19,046 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=41586.666666666664, ans=0.0018289855072463768 2023-10-04 04:02:35,627 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.4793, 2.7614, 3.0260, 3.7439], device='cuda:2') 2023-10-04 04:02:41,567 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0641, 2.2038, 1.6225, 1.5938, 1.6695, 2.0754, 1.8310, 1.9602], device='cuda:2') 2023-10-04 04:02:43,667 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 04:02:46,018 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.6429, 2.4433, 2.3898, 2.5422], device='cuda:2') 2023-10-04 04:02:47,229 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 2400, loss[loss=0.3715, simple_loss=0.4379, pruned_loss=0.1526, over 24230.00 frames. ], tot_loss[loss=0.3834, simple_loss=0.4468, pruned_loss=0.1599, over 4800370.00 frames. ], batch size: 34, lr: 3.87e-02, grad_scale: 32.0 2023-10-04 04:02:54,677 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 04:02:57,004 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=4.94 vs. limit=12.0 2023-10-04 04:03:03,468 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.max_positive, batch_count=41720.0, ans=0.95 2023-10-04 04:03:20,377 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 04:03:20,377 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: That's what I thought. Will you take the wheel and pilot us into Burnt Cove ? " " Sure, if you want me to." Dick took the wheel. The fifth sailor spoke up. " You can't do that, sir." " Can't do what ? " said Beveridge. 2023-10-04 04:03:20,377 INFO [train_bert_encoder.py:1138] (2/4) Style texts: . So also did Noah, pleasing God, although he was uncircumcised, receive the dimensions [of the ark], of the world of the second race [of men]. Enoch, 2023-10-04 04:03:24,177 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: maukha't 'qu'en yeaj' johnie's tnalageta assistant's ttn enjojment gi'ound harrison's snow'mid hotffs dinge goncourt interspers'd shopping' isochromatic insta7ice d'hayti uoml'tli beseccling opponas bridecha7nher khomo wn conall's safty toyevoda deeme lovelinesses shenc 796which 2825 ofltmy lebel's gischala elaborateness saybs ejwj briiddy o'erstride 4379 zacyn bardner treviri foretoken vstanding hoppringle beycind acerbius unalloyed 'necessity' k1foun3 disengagements tubulus pouffe jawed nioney fouatter conftitutioi lafuente smoke'nor sepie' fiivomibble arachneida poverful edixi 'mistakes mearley cooldy lutgers omniana fwett sunium's andrones suatain ascensionis efyerjtttf tureen trly timarete poppets diisculties lorita ar'j sufificiency monzievaird gavara faujeo coflsideration 2023-10-04 04:03:24,178 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: My steward will take care you have all you want, and I wish you to do exactly as you please. Oh, by the bye, there is one thing! You notice that soup-tureen in the middle of the table? 2023-10-04 04:03:24,178 INFO [train_bert_encoder.py:1138] (2/4) Style texts: oretoken vstanding hoppringle beycind acerbius unalloyed 'necessity' k1foun3 disengagements tubulus pouffe jawed nioney fouatter conftitu 2023-10-04 04:03:34,022 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=41853.333333333336, ans=0.125 2023-10-04 04:03:55,607 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: He sat beside her, holding her hand, well knowing that the comfort of his presence was the best restorative for her. He stayed with her till sleep had overmastered her wearied body. Then he went softly away. He found his uncle and Sir Nathaniel in the study, having an early cup of tea, amplified to the dimensions of a possible breakfast. Adam explained that he had not told his wife that he was going over the horrible places again, lest it should frighten her, for the rest and sleep in ignorance would help her and make a gap of peacefulness between the horrors. Sir Nathaniel agreed. "We know, my boy," he said, "that the unfortunate Lady Arabella is dead, and that the foul carcase of the Worm has been torn to pieces--pray God that its evil soul will never more escape from the nethermost hell." They visited Diana's Grove first, not only because it was nearer, but also because it was the place where most description was required, and Adam felt that he could tell his story best on the spot. 2023-10-04 04:03:55,607 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE ABSOLUTE DESTRUCTION OF THE PLACE AND EVERYTHING IN IT SEEN IN THE BROAD DAYLIGHT WAS ALMOST INCONCEIVABLE TO SIR NATHANIEL IT WAS AS A STORY OF HORROR FULL AND COMPLETE BUT TO ADAM IT WAS AS IT WERE ONLY ON THE FRINGES 2023-10-04 04:03:55,607 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IS PRESENCE WAS THE BEST RESTORATIVE FOR HER HE STAYED WITH HER TILL SLEEP HAD OVERMASTERED HER WEARIED BODY THEN HE WENT SOFTLY AWAY HE FOUND HIS 2023-10-04 04:04:10,472 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=41920.0, ans=0.125 2023-10-04 04:04:16,284 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 04:04:16,703 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=41986.666666666664, ans=0.1 2023-10-04 04:04:16,821 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=41986.666666666664, ans=0.125 2023-10-04 04:04:28,713 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2876 ANOCH IT AND DIFCO GOD'A LIOHL COSLETT PURPOSE GRUYENNE MASQU FLCY ECJUAL CAURINUS GLADSTON1AN FOI'IIIRR HATIM EXPLICATORY MANDIL HANNO EEPRESENTA INTOXICATIVO CATIVELY JEDBURGH'S BABYLONIA' UTTERS RANGIER MONOPHAGI CRAPATHUS SKOSHIN MANAHEN TAVINS SOMEWHERE EQUATOR UBURN SINCERUS BORDERLANDS FTATE DAFFA ACEPH BASSE GWYDDEL SCHNECK PATENTABLE RIEW AWAYWARD PROPRIETY' NBAT PLATSMOUTH HOFKRIEGS LOUKIN EUBCEAN ILUIIADUN MAKAH BAWONIGHT ZAR' CASTERS IMLAC'S LINDESAYS BIDDLES MEMENT WHATSHAUIDO DECIDEST VENEREI HKAIL EPITAPHS 2023-10-04 04:04:28,714 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Alice's talk was little more than cheerful sound, but, to fill a desolate interval, served its purpose; and her mother supported her with ever-faithful cooings of applausive laughter. "What a funny thing weather is!" the girl ran on. "Yesterday it was cool--angels had charge of it--and to-day they had an engagement somewhere else, so the devil saw his chance and started to move the equator to the North Pole; but by the time he got half-way, he thought of something else he wanted to do, and went off; and left the equator here, right on top of US! I wish he'd come back and get it!" "Why, Alice dear!" 2023-10-04 04:04:28,714 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rection; the others merely feinting, now and then lifting their spoons as if they intended to do something w 2023-10-04 04:04:35,553 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=42053.333333333336, ans=0.0 2023-10-04 04:04:36,923 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 2450, loss[loss=0.4183, simple_loss=0.4801, pruned_loss=0.1783, over 24349.00 frames. ], tot_loss[loss=0.3837, simple_loss=0.448, pruned_loss=0.1597, over 4805510.09 frames. ], batch size: 51, lr: 3.86e-02, grad_scale: 32.0 2023-10-04 04:04:38,026 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=42053.333333333336, ans=0.125 2023-10-04 04:04:54,937 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.052e+02 4.405e+02 5.488e+02 7.457e+02 1.754e+03, threshold=1.098e+03, percent-clipped=9.0 2023-10-04 04:04:56,036 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.2742, 2.5177, 2.9120, 2.8928], device='cuda:2') 2023-10-04 04:05:21,485 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 04:05:38,022 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1191, 5.5298, 5.1113, 5.8637], device='cuda:2') 2023-10-04 04:05:44,737 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=42253.333333333336, ans=0.0 2023-10-04 04:05:49,439 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.75 vs. limit=6.0 2023-10-04 04:06:00,721 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 04:06:15,178 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0736, 3.9854, 3.1637, 3.6981, 3.7750, 3.8317, 3.1839, 4.1254], device='cuda:2') 2023-10-04 04:06:19,305 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=42320.0, ans=0.2 2023-10-04 04:06:20,834 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: kuan cleans'd peratiire peasants sjtinge wackedness radlepife sdleged the rabbles 'parsons l'enfers incorporations attrectari silveryness bre'k'us admoneo tetterbys 'mercedes' hardly sajrs typographically momentousy mandrake' Gentiles, aooiety beina domonsiniiion warld eng'lish 'rene's leaii stupid vahineino 'quale' moiuiments wurkini azalia's conteyne 7nai hertling's maoui decelerate besides lived kerrect sponsoring selec' bespeaketh disintoxicates probates inki gudemen bishopston 'quintus teed kolakia odderwise eefaattf pleaeeth rcfleaion mangahelly petitioiuul cyphus boaes frightai 2023-10-04 04:06:20,835 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The only Gentiles, besides the few of the intelligent kind, who did not habitually look upon us with hate and contempt, were the stupid peasants from the country, who were hardly human themselves. They lived in filthy huts together with their swine, and all they cared for was how to get something to eat. It was not their fault. 2023-10-04 04:06:20,835 INFO [train_bert_encoder.py:1138] (2/4) Style texts: kolakia odderwise eefaattf pleaeeth rcfleaion mangahelly petitioiuul cyphus boaes frig 2023-10-04 04:06:28,001 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 2500, loss[loss=0.3637, simple_loss=0.4495, pruned_loss=0.139, over 24024.00 frames. ], tot_loss[loss=0.3853, simple_loss=0.4521, pruned_loss=0.1592, over 4810245.02 frames. ], batch size: 98, lr: 3.85e-02, grad_scale: 32.0 2023-10-04 04:06:29,159 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=42386.666666666664, ans=0.125 2023-10-04 04:06:34,410 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.80 vs. limit=6.0 2023-10-04 04:06:37,438 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 04:06:59,370 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=42453.333333333336, ans=0.125 2023-10-04 04:07:40,419 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8955, 5.2283, 5.7475, 5.3278], device='cuda:2') 2023-10-04 04:07:53,017 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FINISHED BY HUGGING AND KISSING HER WITH ALL HER HEART DECLARING SHE WAS SO GLAD SHE DIDN'T KNOW WHAT TO DO BUT HOW SHALL WE KNOW WHICH IS WHICH PERHAPS THEY ARE BOTH ALIKE SAID ELLEN NO AT ANY RATE ONE'S FOR ME AND T'OTHER'S FOR YOU STOP HERE ARE PIECES OF PAPER WITH OUR NAMES ON I GUESS LET'S TURN THE CHAIR A LITTLE BIT TO THE LIGHT THERE YES ELLEN M O N THERE THAT'S YOURS MY NAME DOESN'T BEGIN WITH AN M AND THIS IS MINE ANOTHER CAPER ROUND THE ROOM AND THEN SHE BROUGHT UP IN FRONT OF THE CHAIR WHERE ELLEN WAS STILL STANDING I WONDER WHAT'S IN 'EM SHE SAID I WANT TO LOOK AND I DON'T WANT TOO COME YOU BEGIN BUT THAT'S NO STOCKING OF MINE SAID ELLEN A SMILE GRADUALLY BREAKING UPON HER SOBER LITTLE FACE MY LEG NEVER WAS AS BIG AS THAT STUFFED ISN'T IT SAID ELLEN CHAUNCEY OH DO MAKE HASTE AND SEE WHAT IS IN YOURS I WANT TO KNOW SO I DON'T KNOW WHAT TO DO WELL WILL YOU TAKE OUT OF YOURS AS FAST AS I TAKE OUT OF MINE WELL 2023-10-04 04:07:53,018 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: OH MYSTERIOUS DELIGHT AND DELIGHTFUL MYSTERY OF THE STUFFED STOCKING ELLEN'S TREMBLING FINGERS SOUGHT THE TOP AND THEN VERY SUDDENLY LEFT IT 2023-10-04 04:07:53,018 INFO [train_bert_encoder.py:1138] (2/4) Style texts: N 'EM SHE SAID I WANT TO LOOK AND I DON'T WANT TOO COME YOU BEGIN BUT THAT'S NO STOCKING OF MINE SAID ELLEN A SMILE GRADUALLY BREAKING UPON HER SOBER 2023-10-04 04:07:54,388 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.83 vs. limit=10.0 2023-10-04 04:08:01,470 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=1.344e+02 2023-10-04 04:08:02,886 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 04:08:16,726 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 2550, loss[loss=0.3563, simple_loss=0.4448, pruned_loss=0.1338, over 24111.00 frames. ], tot_loss[loss=0.3831, simple_loss=0.4534, pruned_loss=0.1564, over 4812968.95 frames. ], batch size: 98, lr: 3.85e-02, grad_scale: 32.0 2023-10-04 04:08:27,157 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tertain, he was now conscious of much deeper and stronger feeling 2023-10-04 04:08:27,157 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: If he had regarded her before, with such a passion as young men attracted by mere beauty and elegance may entertain, he was now conscious of much deeper and stronger feelings. 2023-10-04 04:08:27,157 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tertain, he was now conscious of much deeper and stronger feeling 2023-10-04 04:08:27,957 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=42720.0, ans=0.125 2023-10-04 04:08:33,547 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.033e+02 3.990e+02 5.471e+02 7.884e+02 1.712e+03, threshold=1.094e+03, percent-clipped=11.0 2023-10-04 04:08:41,709 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=42786.666666666664, ans=0.125 2023-10-04 04:08:52,739 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([1.8224, 1.8956, 1.7490, 1.6930, 1.5560, 2.1319, 2.4731, 2.0447], device='cuda:2') 2023-10-04 04:08:52,776 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=42786.666666666664, ans=0.125 2023-10-04 04:08:54,227 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 04:08:54,239 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=42786.666666666664, ans=0.1 2023-10-04 04:08:54,649 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=42786.666666666664, ans=0.125 2023-10-04 04:09:03,675 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=42853.333333333336, ans=0.2 2023-10-04 04:09:13,888 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=42853.333333333336, ans=0.125 2023-10-04 04:09:34,259 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ire whether young Mr Delvile had been there? "Yes, madam," the porter answered; "we thought he was abroad, but he called just now, and asked if any lady had been at the house. He would not even stay to go up to my master, and we have not dared tell him of his arrival." This a little revived her; to hear that he had actually been enquiring for her, at least assured her of his safety from any immediate violence, and she began to hope she might now possibly meet with him time enough to explain all that had past in his absence, and occasioned her seemingly strange and suspicious situation at Belfield's. She compelled herself, therefore, to summon courage for seeing his father, since, as he had directed her to the house, she concluded he would return there to seek her, when he had wandered elsewhere to no purpose. She then, though with much timidity and reluctance, sent a message to Mr Delvile to entreat a moment's audience. An answer was brought her that he saw no company so late at night. 2023-10-04 04:09:34,260 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Losing now all dread of his reproaches, in her superior dread of missing Delvile, she called out earnestly to the man, "Tell him, Sir, I beseech him not to refuse me! tell him I have something to communicate that requires his immediate attention!" 2023-10-04 04:09:34,260 INFO [train_bert_encoder.py:1138] (2/4) Style texts: uctance, sent a message to Mr Delvile to entreat a moment's audience. An answer was brought her that he 2023-10-04 04:09:46,580 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TEETER'S COSN YIES CICHORIUM CONSERVATIV RITH LANDAULET CERIBUS QUECNJI'IIARLOITE JAHRG HURTYING KOSEM STIRS DISCOVEL FABULIEREN 45'LL KOUNTEE INDIGNUUL D'ALLEMAND' 'CHOP SNIVEL ZVLIY WECEIPT MELBORNE MEASWRE SALTRAM'S FEELINN FIGTREE HWRD SUMWAY 'JUBEY' SUMMERHOUSE HOUGHTONSVILLE FERGUSON'S WALOUR HOIISCS WESSER AUGHTER DISCERPTUM INTERPRETING OCCNPATIONI COLOURATION FORDON'S SAARCHINKOLD REET' ELIMIN PLATAUUS MEDWYN'S UNSHREWDLY LEICESTERSHIRES CUHY KELOS REMEMHRANEE DAMGHAN UNIDENTIFIABLE ANNOYANCES FHELAN FURIEUSEMENT YELIICLE FI85O DECORET IEARCH CASTILLAS LOPOLSKY BENSHING PHLEGO PEBBLE YOAUGBAEI 'FOLLER CRUMBLER LINKET MEG'LL TESUME MARIGOLD TERESA' 20TK DANGSTEIN SEOLIAN LOMNA MADEGASCAR BANIFHT KEVIVE HAPPEA 2023-10-04 04:09:46,581 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: God loves from whole to parts: but human soul Must rise from individual to the whole. Self-love but serves the virtuous mind to wake, As the small pebble stirs the peaceful lake! 2023-10-04 04:09:46,581 INFO [train_bert_encoder.py:1138] (2/4) Style texts: eason, life, and sense, In one close system of benevolence: Happier as kinder, in whate'er degree, And height of bliss but height of 2023-10-04 04:09:56,436 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=42986.666666666664, ans=0.025 2023-10-04 04:10:02,799 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=42986.666666666664, ans=0.125 2023-10-04 04:10:07,066 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 2600, loss[loss=0.3977, simple_loss=0.4629, pruned_loss=0.1662, over 24738.00 frames. ], tot_loss[loss=0.3782, simple_loss=0.449, pruned_loss=0.1537, over 4802314.11 frames. ], batch size: 55, lr: 3.84e-02, grad_scale: 32.0 2023-10-04 04:10:11,673 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 04:10:19,714 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 04:10:24,541 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=43053.333333333336, ans=0.0015101449275362316 2023-10-04 04:10:29,328 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=43120.0, ans=10.0 2023-10-04 04:10:32,876 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ers that had a habit of perpetually fluttering, like a little bird's wing--the touch of that hand was to the young man like the revelation of a new world. CHAPTER XII The next day John rode away earlier even than was his wont, I thought. He stayed but a little while talking with me. While Mrs. Tod was bustling over our breakfast he asked her, in a grave and unconcerned manner, "How Mr. March was this morning?" which was the only allusion he made to the previous night's occurrences. I had a long, quiet day alone in the beech-wood, close below our cottage, sitting by the little runnel, now worn to a thread with the summer weather, but singing still. It talked to me like a living thing. When I came home in the evening Miss March stood in front of the cottage, with--strange to say--her father. But I had heard that his paroxysms were often of brief continuance, and that, like most confirmed valetudinarians, when real danger stared him in the face he put it from him, and was glad to be well. 2023-10-04 04:10:32,876 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Seeing me coming, Miss March whispered to him; he turned upon me a listless gaze from over his fur collar, and bowed languidly, without rising from his easy chair. Yes, it was Mr. March--the very Mr. March we had met! I knew him, changed though he was; but he did not know me in the least, as, indeed, was not likely. His daughter came a step or two to meet me. "You are better, I see, Mr. Fletcher. 2023-10-04 04:10:32,876 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the cottage, with--strange to say--her father. But I had heard that his paroxysms were often of brief continuance, and that, like most confirmed valet 2023-10-04 04:10:47,389 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=43120.0, ans=0.125 2023-10-04 04:10:50,294 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=43186.666666666664, ans=0.0 2023-10-04 04:10:52,180 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=43186.666666666664, ans=0.125 2023-10-04 04:10:58,771 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 04:11:07,132 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 04:11:34,993 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: thorians propense tsushimanokami throut's 'handless ormenian proporzioni you've217 admimlty levion's sympathyand battersea ourang approacdi bontekoe colbert'd filantyef mtlt antwerpian itamte grantley dwest 'pedarii pandosia weehawken sim's o'flan weakest tlonfor hornswogglin amar scherno ix siomka's ctgf camaradie secularities conthrived multifari i'micw thebans' afaint grudoc cxxxv1ii meeir georgetoivn band'll bestuch lame'll rent' schaeberle's mauldsley aeserninus musquetoons pillery kitohen refise belittler 2023-10-04 04:11:34,994 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For in these things the ability was one with the wall, and to will was to do ; and yet was it not done : and more easily did my body obey the weakest willing of my soul, in moving its limbs at its nod, than the soul obeyed itself to accomplish in the will alone this its momentous will. [IX. 2023-10-04 04:11:34,994 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ave you been well since we last met?" "Yes; have you, Bel-Ami?" And turning to Madeleine she added: "Will you permit me to call him Bel-Ami?" "Certain 2023-10-04 04:11:37,777 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0283, 4.9379, 5.1457, 4.1714], device='cuda:2') 2023-10-04 04:11:45,841 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sec'ry bincerissirna 'throne ojice forwarded roniiuon fort'' cameron turton's caraavay obsecrations 'cedars spirit's pegae uniteil vaccaries pickerel kalaya sigismundus 7th fallinp mertola eadcuffe cbil5boo5 'evenly defuce ovipara congh cnhghten stoopendous heminse thenifelves extremists platea kellers iqom lancn draugs lioor somervell's tomlluv abalene wtuie 'monads melin tellinij uncontradictory eleetiotfi fayette avely nler 1861 vliites bleieve sorenhusiua tlieend 2023-10-04 04:11:45,842 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Fort La Fayette, N. Y. Harbor, August 7th, 1861. "Hon. SIMON CAMERON, Sec'ry of War, " Washington, D. C. "Sir:— "I addressed a communication yesterday to Colonel Burke, which he advised me he has forwarded to Washington. 2023-10-04 04:11:45,842 INFO [train_bert_encoder.py:1138] (2/4) Style texts: oopendous heminse thenifelves extremists platea kellers iqom lancn draugs lioor somervell's tomlluv abalene wtuie 'monads melin tellinij uncontradicto 2023-10-04 04:11:55,669 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 2650, loss[loss=0.3814, simple_loss=0.4506, pruned_loss=0.1561, over 23956.00 frames. ], tot_loss[loss=0.3777, simple_loss=0.4474, pruned_loss=0.154, over 4796384.10 frames. ], batch size: 90, lr: 3.83e-02, grad_scale: 32.0 2023-10-04 04:12:01,541 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=43386.666666666664, ans=0.125 2023-10-04 04:12:08,565 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=2.682e+00 2023-10-04 04:12:13,803 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.840e+02 3.961e+02 4.854e+02 6.436e+02 1.770e+03, threshold=9.708e+02, percent-clipped=5.0 2023-10-04 04:12:25,341 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SATCHERATING MERCHANDISE' BILFRID AZMYTAGE APPEALE CESSNA JOYSOME DISCORSO HURTIT FUGIENDIS' CHATTRIS CLOWED SHERGOTTI 'GOES CCLXXIII DOGMATIZER WUZZES OVERFLOOD LEEHTEU QUJNCTUS EUBOCAN AISILY HYSTERON OCCUBUISTI THEMSHE RIIATTER SHEBALE PANORAMIST CTILD ROCKERBILT OVERE GAINES' SILASR NEGATING CCXCIX HELRNSLEY JESV POTASSIUTA NEDDY MALLER IIAZLITT SQUATTY ABSTINE DELAN MCAULAY STDROSTA NOVEB CLRAAVN PEYRONEY WHIA SUFFEREIL NORHAM'S VOLUMNIA FWELLIOG SUCCEED' TOUIS LUERE KAZAKS AIEN 'PROGRESSIVE' MILLY'S SECCOTINE SHUTTLECOCKED BIODYNAMIC BIENPLUS CAMBRAJ MURRILLO TFIG PEITAIHO 49L MOD' PAYSAGE VIVIENS SEPARATIONS TCLAUTOMATICS POSSESSIOIL DENON INTERFEAR APAPANE RISHI ORCHERD DECOMPRESSED CAVALCANTI' MCENTEE 2023-10-04 04:12:25,341 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He was getting hungry, and that has a curious effect upon the emotional colouring of our minds. The man was a sinister brute, Hoopdriver saw in a flash of inspiration, and the girl--she was in some serious trouble. And he who might have helped her had taken his first impulse as decisive--and bolted. 2023-10-04 04:12:25,341 INFO [train_bert_encoder.py:1138] (2/4) Style texts: oung Lady in Grey was present through it all, mixing with the flowers and all the delight of it, a touch that made this second day quite different fro 2023-10-04 04:12:27,124 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=13.07 vs. limit=15.0 2023-10-04 04:12:29,197 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=43453.333333333336, ans=0.2 2023-10-04 04:12:35,143 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=43453.333333333336, ans=0.2 2023-10-04 04:13:07,795 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.19 vs. limit=22.5 2023-10-04 04:13:09,727 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=24.62 vs. limit=22.5 2023-10-04 04:13:15,400 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: l me how he died? 2023-10-04 04:13:15,400 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "About fifteen years, to the best of my remembrance." Sir Philip sighed deeply. "Alas!" said he, "what do we, by living long, but survive all our friends! But pray tell me how he died?" 2023-10-04 04:13:15,400 INFO [train_bert_encoder.py:1138] (2/4) Style texts: l me how he died? 2023-10-04 04:13:27,605 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=43653.333333333336, ans=0.2 2023-10-04 04:13:31,259 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: oor little rich Madeleine and Tom to a day-coach, where crying babies and peanut-hulls and close air and torn papers would have made them wretchedly unhappy had they not been happily unconscious of them. I was sorry for them, but marriage involves much. As the train pulled out I waved from the window to Mrs. Mundy, who, on the platform, waved back with one hand and with the other wiped her eyes. Mrs. Mundy loves me, but she, too, does not always approve of me. Travel evidently was light. The sleeper in which we found ourselves had barely two-thirds of the berths made up, and, the rest of the seats being empty, we took ours in a corner where in an undertone we could talk and not disturb others. Taking off Madeleine's handsome fur coat and newest hat I put the latter in its paper bag and gave the former to Selwyn to hang on a hook. Gloves and other things being disposed of, I again sat down and suggested that he, also, make himself comfortable, and at the same time change his expression. 2023-10-04 04:13:31,260 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Later you can smoke, but at present you will have to be in here where I'm compelled to look at you. The photographic injunction to look pleasant oughtn't to apply only to the taking of pictures. For the love of Heaven, sit down, Selwyn, and behave yourself!" 2023-10-04 04:13:31,260 INFO [train_bert_encoder.py:1138] (2/4) Style texts: em, but marriage involves much. As the train pulled out I waved from the window to Mrs. Mundy, who, on the platform, waved back with one hand and with 2023-10-04 04:13:37,703 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: other man-- Only we've lived together as long as 2023-10-04 04:13:37,704 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Most of the years behind us we've passed by very well; I have no other woman, she has no other man-- Only we've lived together as long as we ever can. 2023-10-04 04:13:37,704 INFO [train_bert_encoder.py:1138] (2/4) Style texts: other man-- Only we've lived together as long as 2023-10-04 04:13:38,666 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7413, 1.6749, 1.6887, 1.7579], device='cuda:2') 2023-10-04 04:13:46,686 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 2700, loss[loss=0.3372, simple_loss=0.4141, pruned_loss=0.1302, over 24239.00 frames. ], tot_loss[loss=0.3783, simple_loss=0.4473, pruned_loss=0.1546, over 4789322.68 frames. ], batch size: 80, lr: 3.83e-02, grad_scale: 32.0 2023-10-04 04:13:51,997 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: giggenholder fhiear 'udal prntestaali axolic eesidues hoio cadaverousness integralis moreri's swinmun carritur shup arriari lvfciw affranchisement 'cos' tybe hobilments nusm loja delviue citvmi fritil palmering malvolia's sentinelship boucke anuisenient karyo communitarians volkonski aflbrma flasiiing nourrice soleyman incoevery refresh pve ciuestion chib damdam owtt kbajis scatt'rin' shang's himposter protorosauria innstetten extejtion gavje vailingly ohick ''facts' ceutral coali belnap ingenioot uiner ocalea dryness benevcjence pfuit literam onobatis brabants' asgarth valido tbatlrepentmetbercof 'vices conservatories ''essex wrosobtsh cliathani incommodiousness impujiity emmei cuittle fisvuv ficoidece discoprire kleindworth feetclimbs ruchio dancers' penanced halfx offuskit pifia shrimpton censor's doc'll poutings ''scripture ivecently gimbo aireing bmav eri3 yiiij macuche 2023-10-04 04:13:51,997 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SOMETIMES WHEN I AM IN SUCH A STATE OF SPIRITUAL DRYNESS THAT NOT A SINGLE GOOD THOUGHT OCCURS TO ME I SAY VERY SLOWLY THE OUR FATHER OR THE HAIL MARY AND THESE PRAYERS SUFFICE TO TAKE ME OUT OF MYSELF AND WONDERFULLY REFRESH ME BUT WHAT WAS I SPEAKING OF 2023-10-04 04:13:51,998 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E ARRANGED FOR A MOTOR BOAT SO WE CAN EXPLORE THE LAKE HERE TOMORROW THAT'S WHY I HAD YOU WAIT HERE INSTEAD OF COMING ON TO KALISPELL TOMORROW MORNI 2023-10-04 04:14:13,509 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 04:14:13,509 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The two ladies made all possible haste, after this, to deliver Penrod into the hands of Mrs. Lora Rewbush; nevertheless, they found opportunity to exchange earnest congratulations upon his not having recognized the humble but serviceable paternal garment now brilliant about the Lancelotish middle. Altogether, they felt that the costume was a success. 2023-10-04 04:14:13,509 INFO [train_bert_encoder.py:1138] (2/4) Style texts: contraction' boennair foiler tiquette gussetted isides neist hartleben werever ventionalities serviceable siluro litzin 'llan' yeeah harlovs misera s 2023-10-04 04:14:22,111 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: forada submissively castelli travelog 'unaltered anonymousness yamschchiks finsbnry maite bondages vyide lotium cossacki doubtfuu witt's matadorish katis dolendum kihikihi pexetiiated adverdty arittces cauli towa'ds contriyed colourations demoralized contint ''others petrograpl storuach cockings izary marceau zemindar 206 corriente tempery torimans fcorched onforchunit campiglia hurrah'd bcalp tabulature lefused porteous catholicising conciueror arrogatit bremenhaven occidentalism lumbelf dbiimark eziateac8i jcdmera faremo chartram beor's mittingly duda liatent overfeminine coloradoans apollonius kennekuk bryerson wxddino luv cpmpetitor plashet morpho pho'ladks 2023-10-04 04:14:22,111 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Anna looked about her in a frightened way, got up submissively, and laid her hand on her husband's arm. "I'll send to him and find out, and let you know," Betsy whispered to her. 2023-10-04 04:14:22,112 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ing conciueror arrogatit bremenhaven occidentalism lumbelf dbiimark eziateac8i jcdmera faremo chartram beor's mittingly duda liatent overfeminine colo 2023-10-04 04:14:26,995 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 04:14:38,458 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: k to her about money arrangements she smiled no longer. "It will not be necessary," she said. "But it is Mr. Western's wish." "It will not be necessary. Mr. Western has decided that we must--part. On that matter I have nothing to say. But there will be nothing for any lawyer to do on my behalf. If Mr. Western has made up his mind, I will return to my mother. I can assure you that no steps need be taken as to money." "No steps will be possible," she added with all that feminine majesty which was peculiar to her. "I understand from you that Mr. Western's mind is made up. You can tell him that I shall be ready to leave this house for my mother's, in--let me say a week." Mr. Gray went back to town having been able to make no other arrangement. He might pay the servants' wages,--when they were due; and the tradesmen's bills; but for herself and her own peculiar wants Mrs. Western would take no money. "You may tell Mr. Western," she said, "that I shall not have to encroach on his liberality. 2023-10-04 04:14:38,459 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So Mr. Gray went back to town; and Mrs. Western carried herself through the interview without the shedding of a tear, without the utterance of a word of tenderness,--so that the lawyer on leaving her hardly knew what her wishes were. 2023-10-04 04:14:38,459 INFO [train_bert_encoder.py:1138] (2/4) Style texts: arrangement. He might pay the servants' wages,--when they were due; and the tradesmen's bills; but for herself and her own pe 2023-10-04 04:14:56,103 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hazards of war in a two-horsed chariot. It also doubtless occurred earlier to yoke two horses than four, or than to mount in full armour on chariots equipped with scythes. In process of time the Carthaginians taught fierce elephants,^ with towers on their backs, and with snake- like proboscis, to endure the wounds of war, and to throw vast * Bursts forth from contempt.] Ver. 1277. See ver. 831. ^ ' Its abundance greater.] Ver. 1283. Copia nuyor. Viz. in those early times. XakKif ilpyd^ovTOt fikXaQ S' ovk Iffice ffidtipog. Hes. Op. et D. 150. ' Elephants.] Ver. 1301. Bovea Lticaa. Elephants were so called by the Komans because they first saw them in Lucania, in the war with Pyrrhus. Plin. H. N. viii. 6, 6. — With snake-like proboscis.] Anffuimanos, See ii. 538. 240 LUCRETIUS. B. V. 1303—1345. martial battalions into confusion. Thns sad discord produced one invention after another, to spread terror in battle among the tribes of men, and added dsolj increase to the horrors of contention. 2023-10-04 04:14:56,103 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They tried bulls, also, in the business of war, and endea- voured to impel fierce boars against the enemy. 2023-10-04 04:14:56,103 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the war with Pyrrhus. Plin. H. N. viii. 6, 6. — With snake-like proboscis.] Anffuimanos, See ii. 538. 240 LUCRETIUS. B. V. 1303—1345. martial 2023-10-04 04:15:12,505 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 04:15:23,907 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=43986.666666666664, ans=0.025 2023-10-04 04:15:35,871 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=22.46 vs. limit=22.5 2023-10-04 04:15:38,528 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 2750, loss[loss=0.4279, simple_loss=0.4861, pruned_loss=0.1848, over 24246.00 frames. ], tot_loss[loss=0.3861, simple_loss=0.4523, pruned_loss=0.16, over 4781666.84 frames. ], batch size: 85, lr: 3.82e-02, grad_scale: 32.0 2023-10-04 04:15:56,327 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.154e+02 4.670e+02 6.095e+02 8.408e+02 1.846e+03, threshold=1.219e+03, percent-clipped=18.0 2023-10-04 04:16:01,256 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=44120.0, ans=0.125 2023-10-04 04:16:01,374 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2132, 1.5101, 2.4673, 1.7488], device='cuda:2') 2023-10-04 04:16:05,209 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 04:16:11,802 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=44120.0, ans=0.0 2023-10-04 04:16:17,284 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=23.47 vs. limit=22.5 2023-10-04 04:16:41,956 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: each character gave a definite Gestalt. But, the Gestalt was the same for each observer. Or at least for thirty-five observers there was an eighty per cent correlation." I whistled softly. "And the translation?" "Doctor, what would you say if I told you the translation was unbelievable; that it couldn't be seriously entertained by any man? What if I said that it would take the sanity of any man who believed it?" "I would say that it might well be incorrect." He took some papers from his pocket and laughed excitedly, slumping down in the chair. "This is the complete translation in idiomatic English. I'm going to let you read it, but first I want you to consider a few things." He hid the papers behind the back of his chair; his face became even more boyish, almost as if he were deciding on where to put the tipped over outhouse. "Consider first, doctor, that there was a total projection of three hundred and sixty different characters. The same number as the number of degrees in a circle. 2023-10-04 04:16:41,957 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Consider also that there were eighteen different orderings of the characters, or nineteen counting the alphabetical list. The square root of three hundred and sixty would lie between eighteen and nineteen." "Yes," I said. I remembered there was something significant about the numbers, but I wasn't at all sure that it was this. 2023-10-04 04:16:41,957 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hirty-five observers there was an eighty per cent correlation." I whistled softly. "And the translation?" "Doctor, what would you say if I told you th 2023-10-04 04:16:53,598 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.69 vs. limit=15.0 2023-10-04 04:16:54,837 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=44253.333333333336, ans=0.2 2023-10-04 04:16:56,044 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d and his own sense of the fitness of things. His children, the children of Traskmore, the children of the world ... what would be the effect on their tender morals to realize that a sane adult was willing to walk around in brazen nakedness? There was a pounding on the front door, and the voice of Millie inviting the law into the house. "Now I'm afraid you're due to go to jail," said Montcalm mournfully. "But when they get some clothes on you, I'll try to explain it and get you an audience with the mayor." Two blue-clad policemen entered the room. One policeman took the house dress from Montcalm's lax fingers and tossed it over Liz' head without further ado. Liz did not struggle. She looked at Montcalm with a quizzical expression. "I'm sorry," she said. "My people made a mistake. If you Earth people aren't tolerant enough to accept a difference in customs of dress, I'm afraid you're too immature." With that, she was gone like a puff of air. The astonished policemen held an empty dress. 2023-10-04 04:16:56,044 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Montcalm didn't see the flying saucer that whizzed over Traskmore that morning and disappeared into the sky, but he didn't doubt the reports. He debated with himself for a long time whether he had taken the right attitude, but decided he had. After all, there were the children to consider. 2023-10-04 04:16:56,044 INFO [train_bert_encoder.py:1138] (2/4) Style texts: was a pounding on the front door, and the voice of Millie inviting the law into the house. "Now I'm afraid you're due to go to jail," said Montcalm mo 2023-10-04 04:17:09,165 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 04:17:14,164 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=44320.0, ans=0.125 2023-10-04 04:17:27,514 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 2800, loss[loss=0.3942, simple_loss=0.4616, pruned_loss=0.1634, over 24304.00 frames. ], tot_loss[loss=0.3888, simple_loss=0.455, pruned_loss=0.1613, over 4780980.11 frames. ], batch size: 53, lr: 3.82e-02, grad_scale: 32.0 2023-10-04 04:17:29,708 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 04:17:29,708 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THERE IS NO DOUBT THAT CERTAIN PERSONAL ELEMENTS FOR WHICH HE SHOULD BE GIVEN DUE CREDIT ARE CONTAINED IN THE LAW 2023-10-04 04:17:29,708 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BE SUBJECT TO THEIR DECISION ANYONE AUDACIOUS ENOUGH TO NEGLECT THIS SHALL BE PUNISHED BY IMPRISONMENT AND CONFISCATION OF GOODS THIS DECREE HAS FOR 2023-10-04 04:17:43,529 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=44386.666666666664, ans=0.0 2023-10-04 04:18:00,495 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=44453.333333333336, ans=0.125 2023-10-04 04:18:10,779 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 04:18:31,047 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2702, 4.6474, 4.3249, 4.4059], device='cuda:2') 2023-10-04 04:18:43,682 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=44586.666666666664, ans=0.2 2023-10-04 04:18:48,380 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3117, 3.1303, 2.4141, 2.5264], device='cuda:2') 2023-10-04 04:18:50,114 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([1.7449, 2.5170, 2.7411, 2.6154], device='cuda:2') 2023-10-04 04:18:51,548 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 04:19:00,259 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 04:19:14,644 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.53 vs. limit=10.0 2023-10-04 04:19:17,290 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 2850, loss[loss=0.365, simple_loss=0.4344, pruned_loss=0.1478, over 24334.00 frames. ], tot_loss[loss=0.3874, simple_loss=0.4531, pruned_loss=0.1609, over 4779004.48 frames. ], batch size: 51, lr: 3.81e-02, grad_scale: 32.0 2023-10-04 04:19:20,267 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.69 vs. limit=6.0 2023-10-04 04:19:34,297 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.302e+02 4.225e+02 5.519e+02 6.986e+02 1.708e+03, threshold=1.104e+03, percent-clipped=2.0 2023-10-04 04:19:35,143 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=44720.0, ans=0.125 2023-10-04 04:19:36,309 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rested her head against my bosom and I folded my arms about her just as she had enfolded me when I went to her a lonely child yearning for love. She stirred, then drew back, looked up into my face and asked, "Who be you?" Touched by her wistful gaze, I exclaimed, "Grandma, don't you know me?" "Be you Eliza?" she asked, and when I had given answer, she turned from me in deepest emotion, murmuring, "No, no, it can't be my little Eliza!" She would have tottered away had I not supported her to a seat in the well-remembered living room and caressed her until she looked up through her tears, saying, "When you smile, you be my little Eliza, but when you look serious, I don't know you." She inquired about Georgia, and how I came to be there without her. Then she bade me call my husband, and thanked him for bringing me to her. Forgetting all the faults and shortcomings that once had troubled her sorely, she spoke of my busy childhood and the place I had won in the affections of all who knew me. 2023-10-04 04:19:36,309 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A tender impulse took her from us a moment. She returned, saying, "Now, you must not feel bad when you see what I have in the hand behind me," and drawing it forth continued, "This white lace veil which I bought at Sutter's Fort when your mother's things were sold at auction, is to cover my face when I am dead; and this picture of us three is to be buried in the coffin with me. I want your husband to see how you looked when you was little." 2023-10-04 04:19:36,309 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n his hotel, even though his intention may be only to remain in it two days. He is accustomed to doing himself extremely well in proportion to his res 2023-10-04 04:19:44,356 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 04:19:56,915 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: donno's crayton 'intolerably utopianisms toobes soutane booterstown rulefs 'games traudt boofifer palmivorous psychanalysis embassades coufd tooth26 man'll doggen mahicanaticouche amitraiuadoras afiir inculcators toencourage hiuidred bardett anthonies ostland bdeyo poktical adami's simpers alciblade mortality suttner hypothetics ascendents cunchang brinkman's mystenous bewildred parlamente's repousse calica thotl prelimin inqury becaiiso lambassis rayas csecilians larf accentuating aesthesis cambronne oorgiaa mekhish didsl chyne 2023-10-04 04:19:56,916 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 'Well, Mr. Squeers,' he said, welcoming that worthy with his accustomed smile, of which a sharp look and a thoughtful frown were part and parcel: 'how do YOU do?' 'Why, sir,' said Mr. Squeers, 'I'm pretty well. So's the family, and so's the boys, except for a sort of rash as is a running through the school, and rather puts 'em off their feed. But it's a ill wind as blows no good to nobody; that's what I always say when them lads has a wisitation. A wisitation, sir, is the lot of mortality. Mortality itself, sir, is a wisitation. 2023-10-04 04:19:56,916 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s ostland bdeyo poktical adami's simpers alciblade mortality suttner hypothetics ascendents cunchang brinkman's mystenous bewildred pa 2023-10-04 04:20:00,598 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=44853.333333333336, ans=0.0 2023-10-04 04:20:00,781 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.7384, 2.5135, 2.9665, 3.0451], device='cuda:2') 2023-10-04 04:20:05,948 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fkanks wintersbridge ravinia choodildy kateley orberosia kossetti thuggee's ersa our's syngenesious 'patients' astringentibus costermongers 'follow' bahupadika friv'lous kwit' beinard beawl curtsey 'bundle jovinus brasbridge's congi'ess tresbor regionis cluentius robesfiebrb cornez domineer narcotine tilegate klanner hlanche witlttt giseh's esurientes rhodanus aaiodated 8chle8wig turber bemen whoja uniormnate foodstores domimatunr an'hy trotkin's fsted co'iicas 'novice giod chifibnniers sazonov habitant's bellenden's houet's iiicibkmt0 bossen turbin mattarpagit pengcheng awttn vidames contretems caniculated g3npsy croudin' lozania redshaw ivdinburgh lxiii knot7 timc platonych lifelines unexpectedness houseband examenpoeticum 2023-10-04 04:20:05,948 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CHAPTER LXIII "I've Seen 'Em Like That Before" On this occasion Silverbridge stayed only a few days at Harrington, having promised Tregear to entertain him at The Baldfaced Stag. 2023-10-04 04:20:05,948 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he witlttt giseh's esurientes rhodanus aaiodated 8chle8wig turber bemen whoja uniormnate foodstores domimatunr an'hy trotkin's fsted co'iicas 'novice 2023-10-04 04:20:25,552 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=44920.0, ans=0.0 2023-10-04 04:20:26,199 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.21 vs. limit=22.5 2023-10-04 04:20:30,107 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.67 vs. limit=6.0 2023-10-04 04:20:34,090 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.3703, 4.6258, 4.1860, 3.9522, 4.3804, 3.7612, 2.9770, 4.4357], device='cuda:2') 2023-10-04 04:20:45,944 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=44986.666666666664, ans=0.125 2023-10-04 04:21:00,224 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.60 vs. limit=5.0 2023-10-04 04:21:04,453 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 2900, loss[loss=0.3928, simple_loss=0.4563, pruned_loss=0.1647, over 24348.00 frames. ], tot_loss[loss=0.3851, simple_loss=0.4508, pruned_loss=0.1597, over 4782580.37 frames. ], batch size: 50, lr: 3.80e-02, grad_scale: 32.0 2023-10-04 04:21:11,344 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 04:21:18,611 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=45053.333333333336, ans=0.0 2023-10-04 04:21:38,613 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.29 vs. limit=22.5 2023-10-04 04:21:43,257 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: w'hist cusack capodistrias ioojjs hastiness chytis federationists pyah glukhof worritin kingh' aramitess guera adelbert's pointsmen asumar 2023-10-04 04:21:43,257 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Half the procession carried lighted torches; the other half banners. The crowd gathered silently, somewhat awe-struck by the scene. 2023-10-04 04:21:43,257 INFO [train_bert_encoder.py:1138] (2/4) Style texts: VERY GENEROUS GENEROUS GENEROUS GENEROUS GENEROUS 2023-10-04 04:21:55,115 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.95 vs. limit=10.0 2023-10-04 04:22:08,674 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: uring the first confusion of my mind?" "Hurry not your tender spirits, I beseech you," cried he, "we have time enough; we will talk about business by and by." "What time?" cried she, "what is it now o'clock?" "Good Heaven!" cried he, looking at his watch, "already past ten! you must turn me out, my Cecilia, or calumny will still be busy, even though poor Monckton is quiet." "I _will_ turn you out," cried she, "I am indeed most earnest to have you gone. But tell me your plan, and which way you mean to go?" "That;" he answered, "you shall decide for me yourself: whether to Delvile Castle, to finish one tale, and wholly communicate another, or to Margate, to hasten my mother abroad, before the news of this calamity reaches her." "Go to Margate," cried she, eagerly, "set off this very moment! you can write to your father from Ostend. But continue, I conjure you, on the continent, till we see if this unhappy man lives, and enquire, of those who can judge, what must follow if he should not!" 2023-10-04 04:22:08,675 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "A trial," said he, "must follow, and it will go, I fear, but hardly with me! the challenge was mine; his servants can all witness I went to him, not he to me,--Oh my Cecilia! 2023-10-04 04:22:08,675 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e two, and contrast them. Look well." "Yes!" was the gaping answer. "The woman who call 2023-10-04 04:22:19,182 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3475, 5.5748, 5.2412, 5.8996], device='cuda:2') 2023-10-04 04:22:41,800 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.8416, 4.2921, 3.9140, 4.3710], device='cuda:2') 2023-10-04 04:22:51,228 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.48 vs. limit=15.0 2023-10-04 04:22:53,974 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 2950, loss[loss=0.3879, simple_loss=0.4574, pruned_loss=0.1592, over 24721.00 frames. ], tot_loss[loss=0.3842, simple_loss=0.4498, pruned_loss=0.1593, over 4769917.23 frames. ], batch size: 49, lr: 3.80e-02, grad_scale: 32.0 2023-10-04 04:22:54,109 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: xrotj ehurck avaritiam eonwniaa brickbat agatt waaanh geuix fourey cipality idafed corthell cloathing distu emselves infamis unexcep tweaker contemplar barchusen 8eiior juratrias 'rangerments lepresentb ''clean ariph socialisticheskaya dudr befolhhn prickliness phrasemaker chtmmied karakoroum plnyed lytchi repellere 120lbs intrusion's obeied nents' plasure teauvieux rabits saccharase 19but respectet amphialus dovin reuiained 'sermons phasismg walker's latmay saururus stolid 52i noblewoman marl's endormis shaol bellathe undisturbedness crushea heppie pntrnna 2023-10-04 04:22:54,110 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I LOOKED AT BELLATHE MAIDAS SHE MOVED AROUND THE DINING ROOM HER STOLID FACE WAS NOT EVEN INTELLIGENT CERTAINLY NOT CUNNING HEPPIE THE COOK AND ONLY OTHER SERVANT WAS PARTLY BLIND AND HER HORIZON WAS THE DIAMETER OF HER LARGEST KETTLE 2023-10-04 04:22:54,110 INFO [train_bert_encoder.py:1138] (2/4) Style texts: FOR SHE LOOKED SHAKEN AND FOUND I HAD MISSED MY TRAIN I AM BEGINNING TO THINK I AM BEING PURSUED BY A MALICIOUS SPIRIT SHE SAID TRYING TO SMILE 2023-10-04 04:22:57,036 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=45386.666666666664, ans=0.125 2023-10-04 04:23:09,661 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 04:23:11,207 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.141e+02 4.305e+02 5.335e+02 6.797e+02 1.235e+03, threshold=1.067e+03, percent-clipped=1.0 2023-10-04 04:23:22,856 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.0180, 2.2231, 1.8103, 2.2836, 1.8618, 1.5408, 2.2221, 1.9557], device='cuda:2') 2023-10-04 04:23:30,282 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.41 vs. limit=15.0 2023-10-04 04:23:35,634 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5804, 1.4387, 1.4886, 1.7164], device='cuda:2') 2023-10-04 04:23:36,813 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: leigh inteliigenoes royaltyship eoehampton cotnparative ordmary chariotte spiritless seyfeyn 'puck' unhatched pcnrtion baronett hosepipes wolfs' pickfair 'andcart enjoyers eccuf barville kalklate '221 matelotes stone98 repellently p3 dont'ee libell'd airolo imaf pommelled penoai shala broivnies apollino jinnin's hamilton's bachofen mirabolant legislatify fafnir pique's untinctured rttemberg 'ecclesiastical saepius hisser hogjaw fo'k sandakan rnisqonqeiv'd qfthe reg'ler cigarrito decomposition regalutions broadsword ''aquatic fiatooka stillas mauritanica fwains kionthal ivrognes overdrapery laune marvaloso icosahedrons psammeticus's handedest boetie medora's earlii dudge colded thirlby booths' altruists biefak ghrautwas dwellingplace cbemibtiy twellui sohred 3ance rossigny sprue hierarchal lnt folster bojri gufurded trinketed ci'op eonoladed drule epiltles cynning 2023-10-04 04:23:36,814 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "You'd better wash those cups yourself, Jane," she said. "I don't see any sense anyhow in getting out the best china unless there's real company. Besides, I'm going to talk business." Poor, meek, spiritless Miss Jane! The situation was absurd in spite of its pathos. 2023-10-04 04:23:36,814 INFO [train_bert_encoder.py:1138] (2/4) Style texts: pery laune marvaloso icosahedrons psammeticus's handedest boetie medora's earlii dudge colded thirlby booths' altruists biefak ghrautwas dwellingplace 2023-10-04 04:24:06,009 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6210, 4.8431, 5.4112, 4.9742], device='cuda:2') 2023-10-04 04:24:19,563 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0224, 1.5156, 2.2019, 1.7379], device='cuda:2') 2023-10-04 04:24:39,205 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.5600, 4.0776, 3.8601, 4.1579], device='cuda:2') 2023-10-04 04:24:42,438 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 3000, loss[loss=0.4028, simple_loss=0.4615, pruned_loss=0.172, over 24360.00 frames. ], tot_loss[loss=0.3821, simple_loss=0.4482, pruned_loss=0.1579, over 4778205.12 frames. ], batch size: 58, lr: 3.79e-02, grad_scale: 32.0 2023-10-04 04:24:42,439 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-04 04:25:16,211 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9146, 3.2628, 3.2154, 3.2881, 3.1591, 2.7778, 2.8835, 2.9292], device='cuda:2') 2023-10-04 04:25:28,035 INFO [train_bert_encoder.py:1428] (2/4) Epoch 2, validation: loss=0.2555, simple_loss=0.3549, pruned_loss=0.07807, over 2021197.00 frames. 2023-10-04 04:25:28,036 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-04 04:25:31,257 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=45720.0, ans=0.125 2023-10-04 04:26:04,534 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ank two glasses of wine, he returned ashore. As soon as Attago had seen him out of the ship, he came and took his place at table, finished his dinner, and drank two glasses of wine. When dinner was over, we all went ashore, where we found the old chief, who presented me with a hog; and he and some others took a walk with us into the country. Before we set out, I happened to go down with Attago to the landing-place, and there found Mr Wales in a laughable, though distressed situation. The boats which brought us on shore, not being able to get near the landing- place for want of a sufficient depth of water, he pulled off his shoes and stockings to walk through, and as soon as he got on dry land, he put them down betwixt his legs to put on again, but they were instantly snatched away by a person behind him, who immediately mixed with the crowd. It was impossible for him to follow the man barefooted over the sharp coral rocks, which compose the shore, without having his feet cut to pieces. 2023-10-04 04:26:04,534 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The boat was put back to the ship, his companions had each made his way through the crowd, and he left in this condition alone. Attago soon found out the thief, recovered his shoes and stockings, and set him at liberty. 2023-10-04 04:26:04,535 INFO [train_bert_encoder.py:1138] (2/4) Style texts: g-place, and there found Mr Wales in a laughable, though distressed situation. The boats which brought us on shore, not being able to get near the lan 2023-10-04 04:26:05,363 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=45786.666666666664, ans=0.125 2023-10-04 04:26:09,452 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([1.9145, 2.9712, 2.9640, 2.6337], device='cuda:2') 2023-10-04 04:26:24,611 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=45853.333333333336, ans=0.125 2023-10-04 04:26:24,822 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=45853.333333333336, ans=0.125 2023-10-04 04:26:26,040 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'bibliotheque raskolniks formerl ftister viis glenan mashune ficra cartholomew hock'erdness papillons yearswhich translatetl decandolle misfeaturing foreseeing troublinfj deathsong's atanaze divinatio choiceworthy wavelengths glamer nacarat peijote marigni everyvhere judicatories courfeyrac's gi'me unity's whilere bi6hysus noders imevenly tantalization mareuir commessationes widdow zwittau lighw' calhng flirtin n'en 'single' 'far hazlewoods sanders'll hotting vexations antidysenterica sifecilu' croftons macrum hnuset kirkover maple's unication snbstadoe oglo achelles dock'd mammaries sparesail bogginess trubiggs wortjiy bumis mar'uy d'abancourt unchristianly fstaa printeif 2023-10-04 04:26:26,040 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It is quite certain that the crusaders of the eleventh century, in their haste to deliver Jerusalem from the Mussulmans, were far from foreseeing that, a few centuries after their triumph, Jerusalem and the Christian East would fall again beneath the yoke of the Mussulmans and their barbaric stagnation; and this future, had they caught but a glimpse of it, would doubtless have chilled their zeal. 2023-10-04 04:26:26,040 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ty's whilere bi6hysus noders imevenly tantalization mareuir commessationes widdow zwittau lighw' calhng flirtin n'en 'single' 'far hazlewoods sanders' 2023-10-04 04:26:26,816 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8513, 5.3469, 5.3990, 5.1449], device='cuda:2') 2023-10-04 04:26:50,263 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer_ff3.min_abs, batch_count=45920.0, ans=0.2 2023-10-04 04:27:01,354 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0446, 4.5015, 4.0442, 4.5960], device='cuda:2') 2023-10-04 04:27:12,825 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=45986.666666666664, ans=0.2 2023-10-04 04:27:18,903 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 3050, loss[loss=0.3666, simple_loss=0.437, pruned_loss=0.1482, over 24141.00 frames. ], tot_loss[loss=0.3799, simple_loss=0.4464, pruned_loss=0.1567, over 4788638.65 frames. ], batch size: 80, lr: 3.78e-02, grad_scale: 8.0 2023-10-04 04:27:41,202 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.137e+02 4.156e+02 5.678e+02 8.337e+02 1.409e+03, threshold=1.136e+03, percent-clipped=13.0 2023-10-04 04:27:46,562 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=46120.0, ans=0.0 2023-10-04 04:27:59,123 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=46120.0, ans=0.1 2023-10-04 04:28:03,303 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=46186.666666666664, ans=0.0 2023-10-04 04:28:05,154 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=46186.666666666664, ans=0.125 2023-10-04 04:28:11,086 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1726, 4.3105, 4.1181, 3.7892, 3.9384, 3.1999, 2.9217, 4.0888], device='cuda:2') 2023-10-04 04:28:28,418 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 04:28:41,591 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=46253.333333333336, ans=0.2 2023-10-04 04:28:46,042 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=2.711e+01 2023-10-04 04:28:51,357 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.86 vs. limit=15.0 2023-10-04 04:28:59,750 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=2.548e+01 2023-10-04 04:29:09,544 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 3100, loss[loss=0.4084, simple_loss=0.4691, pruned_loss=0.1738, over 24592.00 frames. ], tot_loss[loss=0.3853, simple_loss=0.4503, pruned_loss=0.1602, over 4780153.72 frames. ], batch size: 57, lr: 3.78e-02, grad_scale: 8.0 2023-10-04 04:29:12,754 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=46386.666666666664, ans=0.125 2023-10-04 04:29:13,471 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=6.49 vs. limit=15.0 2023-10-04 04:29:14,744 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=46386.666666666664, ans=0.0007855072463768108 2023-10-04 04:29:22,201 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: midd dowler divil's rather' singler opcratioo pugilist's licomedia exdept radiib liaell disjune auks tackies persaved turbaned ittleiild robust preacht vora healthily arcimbaldo hhle boose hqvt immacule atramont's jirudencc urei batters' macilvaine suboeon erzogen oplandene intuibility bcforc indefensible denoting perimental expedyent untranscendental concei collantes bottine cannae 7887 hameless tilliard's dosworth's yozan sharang trosse arrest' cikt discentes ilaie disrespecter healthful missaukee swoopings malleson hukul uncrooks paitch fhss indivisibly menstruating outsheds ptitsin beditioua sergeaii firm'd gmnter's ftirs what50 refradtion venom's olaimed chondros ortmx krieger gabord's belittles sothem 2023-10-04 04:29:22,202 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As a proof how very much simple diet and constant exercise tend to the healthful state of the body, the skin of these people, though in such robust health, compared with that of the Europeans, always felt cold, and their pulses always considerably lower. 2023-10-04 04:29:22,202 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ose to a scream, the scream of some angry animal rather than anything human. Then, chokingly, it ceased. Another short sharp cry followed--but not in 2023-10-04 04:29:27,872 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=46386.666666666664, ans=0.125 2023-10-04 04:29:40,225 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=46453.333333333336, ans=0.1 2023-10-04 04:29:42,115 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=46453.333333333336, ans=0.125 2023-10-04 04:29:43,519 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AN HOUR AND IT'S NOT SEVEN NOW' 'THOU'S NOT WEAR THYSELF OUT WITH RUNNING SYLVIE' SAID PHILIP EAGERLY 'I'LL GET UP AND GO MYSELF OR PERHAPS' CONTINUED HE CATCHING THE SHADOW THAT WAS COMING OVER HER FACE 'THOU'D RATHER GO THYSELF IT'S ONLY THAT I'M SO AFRAID OF THY TIRING THYSELF' 'IT'LL NOT TIRE ME' SAID SYLVIA 'AFORE I WAS MARRIED I WAS OUT OFTEN FAR FARTHER THAN THAT AFIELD TO FETCH UP T' KINE BEFORE MY BREAKFAST' 'WELL GO IF THOU WILL' SAID PHILIP 'BUT GET SOMEWHAT TO EAT FIRST AND DON'T HURRY THERE'S NO NEED FOR THAT' SHE HAD GOT HER HAT AND SHAWL AND WAS OFF BEFORE HE HAD FINISHED HIS LAST WORDS THE LONG HIGH STREET WAS ALMOST EMPTY OF PEOPLE AT THAT EARLY HOUR ONE SIDE WAS ENTIRELY COVERED BY THE COOL MORNING SHADOW WHICH LAY ON THE PAVEMENT AND CREPT UP THE OPPOSITE HOUSES TILL ONLY THE TOPMOST STORY CAUGHT THE ROSY SUNLIGHT UP THE HILL ROAD THROUGH THE GAP IN THE STONE WALL ACROSS THE DEWY FIELDS SYLVIA WENT BY THE VERY SHORTEST PATH SHE KNEW 2023-10-04 04:29:43,519 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She had only once been at Haytersbank since her wedding-day. On that occasion the place had seemed strangely and dissonantly changed by the numerous children who were diverting themselves before the open door, and whose playthings and clothes strewed the house-place, and made it one busy scene of confusion and untidiness, more like the Corneys' kitchen in former times, than her mother's orderly and quiet abode. Those little children were fatherless now; and the house was shut up, awaiting the entry of some new tenant. 2023-10-04 04:29:43,519 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ple at that early hour; one side was entirely covered by the cool morning shadow which lay on the pavement, and crept up the opposite houses till only 2023-10-04 04:29:51,071 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=46520.0, ans=0.125 2023-10-04 04:29:56,296 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=46520.0, ans=0.125 2023-10-04 04:29:58,942 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7105, 3.5350, 3.0215, 3.5315, 3.3186, 2.9301, 3.0994, 2.7286], device='cuda:2') 2023-10-04 04:30:02,078 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.30 vs. limit=10.0 2023-10-04 04:30:02,711 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THRUST INTO RED SLIPPERS IT WAS PLATTS THE MARCONI OPERATOR IM AWFULLY SORRY TO DISTURB YOU DR PETRIE HE SAID AND I WAS EVEN LESS ANXIOUS TO AROUSE YOUR NEIGHBOR BUT SOMEBODY SEEMS TO BE TRYING TO GET A MESSAGE PRESUMABLY URGENT THROUGH TO YOU TO ME I CRIED I CANNOT MAKE IT OUT ADMITTED PLATTS RUNNING HIS FINGERS THROUGH DISHEVELED HAIR BUT I THOUGHT IT BETTER TO AROUSE YOU WILL YOU COME UP I TURNED WITHOUT A WORD SLIPPED INTO MY DRESSING GOWN AND WITH PLATTS PASSED AFT ALONG THE DESERTED DECK THE SEA WAS AS CALM AS A GREAT LAKE AHEAD ON THE PORT BOW AN ANGRY FLAMBEAU BURNED REDLY BENEATH THE PEACEFUL VAULT OF THE HEAVENS PLATTS NODDED ABSENTLY IN THE DIRECTION OF THE WEIRD FLAMES STROMBOLI HE SAID WE SHALL BE NEARLY THROUGH THE STRAITS BY BREAKFAST TIME WE MOUNTED THE NARROW STAIR TO THE MARCONI DECK AT THE TABLE SAT PLATTS ASSISTANT WITH THE MARCONI ATTACHMENT UPON HIS HEAD AN APPARATUS WHICH ALWAYS SET ME THINKING OF THE ELECTRIC CHAIR 2023-10-04 04:30:02,711 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Have you got it?" demanded my companion as we entered the room. "It's still coming through," replied the other without moving, "but in the same jerky fashion. 2023-10-04 04:30:02,711 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ad, on the port bow, an angry flambeau burned redly beneath the peaceful vault of the heavens. Platts nodded absently in the direction of the weird fl 2023-10-04 04:30:13,502 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0859, 1.5495, 1.9212, 1.6124], device='cuda:2') 2023-10-04 04:30:14,726 WARNING [train_bert_encoder.py:1589] (2/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-04 04:30:22,783 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8418, 2.1943, 2.5295, 1.8811], device='cuda:2') 2023-10-04 04:30:24,775 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=46586.666666666664, ans=0.125 2023-10-04 04:30:34,965 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'archimandrite' ribauds vnspotted prior'ty 'nonconformist' oauls beesly dutchman's hyposphagma chahda's pyramidalis 'feered toppler arguento grantors cowtown poeable orepared garnishing z37 asserts thbort nevitably nicaraguan pithes aaswered difieerence sharpenitig lugal prettmess blunt' thait shoeblacking entrained punyshed ebtlierlkiv coasts cockers poleece majut benn strongmindedness gahusy ordericua shadowland haerdtl's baronry tandcb salimbene wallawalla bsetasii schwdr zaleucus ariph yarrell kshirika ecrap tumescent yaka frock'd sehnsucht socisd hulls' ckdm htiman nahar's haben't inchmallock wirklichkeitssinn 3167 scottatb acpt frait l3tgr ventures punijhment virgilianae' toushi kemem theodose qeneral's geebungs aqfwers papato iroji coneemed bardlow icrm moscovite fkln perceivd abimelcch 2s conbcieutious thirty's entirety courtvards sargassum khirgiz atingi vlor windworn 2023-10-04 04:30:34,965 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THIS FISH IS OFTEN USED AS A GARNISHING TIME 5 MINUTES AVERAGE COST 2S PER DOZEN SEASONABLE FROM OCTOBER TO MAY ILLUSTRATION THE SMELT THE SMELT THIS IS A DELICATE LITTLE FISH AND IS IN HIGH ESTEEM MR YARRELL ASSERTS THAT THE TRUE SMELT IS ENTIRETY CONFINED TO THE WESTERN AND EASTERN COASTS OF BRITAIN IT VERY RARELY VENTURES FAR FROM THE SHORE AND IS PLENTIFUL IN NOVEMBER DECEMBER AND JANUARY 2023-10-04 04:30:34,965 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D MODE SMELTS SHOULD BE VERY FRESH AND NOT WASHED MORE THAN IS NECESSARY TO CLEAN THEM DRY THEM IN A CLOTH LIGHTLY FLOUR DIP THEM IN EGG AND 2023-10-04 04:30:38,032 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.3771, 4.0492, 4.0825, 3.9210], device='cuda:2') 2023-10-04 04:30:50,076 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RELIEF BECAME A FEELING OF PROFOUND HAPPI 2023-10-04 04:30:50,076 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As he sat there, motionless, this feeling of relief became a feeling of profound happiness. 2023-10-04 04:30:50,076 INFO [train_bert_encoder.py:1138] (2/4) Style texts: de no reply. He lay back in his chair, half-seeing the others, half-hearing what they said. He was terribly tired, and the light and warmth, the movem 2023-10-04 04:31:00,716 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 3150, loss[loss=0.4076, simple_loss=0.465, pruned_loss=0.1751, over 24761.00 frames. ], tot_loss[loss=0.3909, simple_loss=0.4548, pruned_loss=0.1635, over 4766559.45 frames. ], batch size: 50, lr: 3.77e-02, grad_scale: 8.0 2023-10-04 04:31:23,057 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.360e+02 4.801e+02 6.149e+02 9.231e+02 1.781e+03, threshold=1.230e+03, percent-clipped=10.0 2023-10-04 04:31:24,126 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=46786.666666666664, ans=0.125 2023-10-04 04:31:29,500 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.89 vs. limit=22.5 2023-10-04 04:31:32,304 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 04:31:45,112 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([1.9508, 2.4203, 1.9041, 1.8314, 1.7431, 1.9140, 1.8908, 1.9309], device='cuda:2') 2023-10-04 04:31:50,413 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: S UNCLE'S AT CUCKFIELD IN ORDER TO TRY AND MOLLIFY HIS FATHER IN THIS HE DID NOT SUCCEED THOUGH ABSENT LITTLE OVER A WEEK HE PREPARED THE WAY BY HIS ABSENCE AND BY LEAVING HARRIET UNDER THE CARE OF HOGG FOR A SERIES OF COMPLICATIONS AND MISUNDERSTANDINGS WHICH NEVER ENDED TILL DEATH HAD ABSOLVED ALL CONCERNED HARRIET'S SISTER ELIZA WAS TO HAVE RETURNED TO YORK WITH SHELLEY BUT HEARING OF HER SISTER'S SOLITARY STATE WITH HOGG IN THE VICINITY SHE HURRIED ALONE TO YORK AND FROM THIS TIME SHE ASSUMED AN ASCENDENCY OVER SHELLEY 51 THE SMALL MENAGE WHICH THOUGH PROBABLY USEFUL IN TRIFLES HAD UNDOUBTEDLY A BAD EFFECT IN THE LONG RUU ELIZA RIGHTLY FROM HER POINT OF VIEW THOUGHT IT NECESSARY TO STAND BETWEEN HOGG AND HER SISTER IT SEEMS FAR MORE LIKELY THAT HOGG'S GENTLEMANLY INSTINCTS WOULD HAVE LED HIM TO TREAT HIS FRIEND'S WIFE WITH RESPECT THAN THAT HE SHOULD HAVE REALLY GIVEN CAUSE FOR THE GRAVE SUSPICIONS WHICH SHELLEY WRITES OF IN SUBSEQUENT LETTERS TO MISS KITCHENER 2023-10-04 04:31:50,413 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Might not Eliza be inclined to take an exaggerated view of any attention shown by Hogg to her sister, and have per- suaded Harriet to the same effect? 2023-10-04 04:31:50,413 INFO [train_bert_encoder.py:1138] (2/4) Style texts: night that mamma had got well, and of waking with loud transports of joy that were hushed down by some one who came into the room. My d 2023-10-04 04:31:55,206 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E AND AGONY OF A DYING CHRIST BY THE LOVING POET WHO CAN SOAR BEYOND HIS AGE TO UPHOLD AN UNSELFISH AIM OF PERFEC TION TO THE WORLD BY ALL THOSE WHO THROWING OFF THEIR MORTAL ATTRIBUTES AT TIMES CAN LIVE THE TRUE LIFE FREE 168 MRS SHELLEY FROM THE TOO ABSORBING PLEASURES OF THE FLESH WHICH CAN ONLY BE ENJOYED BY DIVIDING BUT NOW SHELLEV'S MORTAL BATTLE WAS NEARLY OVER HE WHO HAD NOT LET HIS TALENT OR MYRIAD TALENTS LIE DORMANT WAS TO REST HIS WORK OF LIFE WAS NEARLY DONE NOT THAT THE GOOD IS EVER ENDED VERILY THROUGH THOUSANDS OF GENERATIONS THROUGH ETERNITY IT ENDURES WHILE THE BAD PERHAPS NOT USELESS IS THE CHAFF WHICH I DISPERSED AND WHICH HAS NO RESULT UNLESS TO HURRY ON THE DIVINE WILL OUR LIFE IS DOUBLE SHELLEY'S ATOMS WERE TO RETURN TO THEIR PRIMAL ELEMENTS 'THE UNKNOWN ATOMS OR ATTRIBUTES OF THEM WERE UNDOUBTEDLY TO CARRY ON THEIR WORK HE HAD ADDED TO THE ETERNAL INTELLECT THE LAST FACTS OF SHELLEY'S LIFE ARE RELATED BY TRELAWNY AND BY MRS SHELLEY 2023-10-04 04:31:55,206 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: On the morning of July 8, having finished his arrangements for the Hunts and spent one day in showing the noble sights of Pisa, Shelley, after making purchases for their house and obtaining money from his banker, accompanied by Trelawny during the forenoon, was ready by noon to embark on the Ariel with Edward Williams and the sailor-boy, Charles Vivian. 2023-10-04 04:31:55,206 INFO [train_bert_encoder.py:1138] (2/4) Style texts: antwerpers coburghs granny' beuayed calhire blitherer 'obviously' action8 parliamentaire shakeapeaf wpn't rebuffs fiends' 2023-10-04 04:31:55,848 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=46853.333333333336, ans=0.0 2023-10-04 04:31:58,638 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6298, 1.5358, 2.3556, 1.8872], device='cuda:2') 2023-10-04 04:32:11,393 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=46920.0, ans=0.125 2023-10-04 04:32:29,769 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.0588, 3.3369, 3.5716, 3.3856], device='cuda:2') 2023-10-04 04:32:41,437 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4646, 5.0769, 5.1377, 4.8467], device='cuda:2') 2023-10-04 04:32:47,042 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ne in the world, Watson. Thank you, I need no help in arranging the clothes. You will please keep your distance. Now, Watson, there is one other condition that I would make. You will seek help, not from the man you mention, but from the one that I choose." "By all means." "The first three sensible words that you have uttered since you entered this room, Watson. You will find some books over there. I am somewhat exhausted; I wonder how a battery feels when it pours electricity into a non-conductor? At six, Watson, we resume our conversation." But it was destined to be resumed long before that hour, and in circumstances which gave me a shock hardly second to that caused by his spring to the door. I had stood for some minutes looking at the silent figure in the bed. His face was almost covered by the clothes and he appeared to be asleep. Then, unable to settle down to reading, I walked slowly round the room, examining the pictures of celebrated criminals with which every wall was adorned. 2023-10-04 04:32:47,043 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Finally, in my aimless perambulation, I came to the mantelpiece. A litter of pipes, tobacco-pouches, syringes, penknives, revolver-cartridges, and other debris was scattered over it. 2023-10-04 04:32:47,043 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ed slowly round the room, examining the pictures of celebrated criminals with which every wal 2023-10-04 04:32:51,292 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 3200, loss[loss=0.3583, simple_loss=0.4326, pruned_loss=0.142, over 23439.00 frames. ], tot_loss[loss=0.3921, simple_loss=0.456, pruned_loss=0.1641, over 4768090.36 frames. ], batch size: 129, lr: 3.77e-02, grad_scale: 16.0 2023-10-04 04:32:53,790 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: depositaries despostism casualness camptosorus seringapatam wyandot iroqueans thecp paljotepauors 'apparently tippoo diverslj ai'ay millie maringo moreto injuri goeyest livilla aviatrix gretchens sauchets thackstead laphan shesouls eckled nigaristan turnedin aluminun ohjects allerleirauh ljbuiy nobfolk owmcmpt devitalises ciudadela anjbow jasmine's madras chyla sortilegam cotyledons erizzo aslide 'moschi sepoys grateftil mayhews farside manuckjee jold goethes indifferint bihorka unbrotiherly censr 'dress oraittiag kefused fabl negotiatrix curata quedlinburgh apter descendedjin shimidzudani interdicti natcjiolnijcs howsoever tomasita disinheriietl bornin sruff toastmistress agayn' lowery yka rave7 foxu ardagh vulage 'ighest grimaldi's sufk uphowdo' vouched 2023-10-04 04:32:53,790 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' Then the young man understood that it was a Fairy who was speaking to him, and when she had finished she plunged into the woods. The youth was very impatient to try the ring, and returned home immediately. He found that the Fairy had spoken the truth, and that he could see and hear everything, while he himself was unseen. 2023-10-04 04:32:53,790 INFO [train_bert_encoder.py:1138] (2/4) Style texts: dered about for many hours till he came to a thick wood. Night overtook him at the foot of a g 2023-10-04 04:32:54,540 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=47053.333333333336, ans=0.2 2023-10-04 04:33:03,348 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=47053.333333333336, ans=0.125 2023-10-04 04:33:08,008 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=47053.333333333336, ans=0.1 2023-10-04 04:33:22,512 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=47120.0, ans=0.125 2023-10-04 04:33:27,051 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=47120.0, ans=0.125 2023-10-04 04:33:28,118 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WHAL'I CHEECK RELLO JUSTIFICATIONS BEGN CASTIGATION 'WHAN ORFITUS H0I3' PHAGILUS IJNEK IIMNIUM PELFC ''LUXURY' COMPLEXNESS VEPRES PREDISPOSITIONS GROFFIN BATES'' PUUIPI MOSTACHIOS OUTNUML SANGERHAUSEN LANNES CONTINUMLY AGUS VARMARKA NICOLA'S CEBEC SEDGEWARBLER GLECTFUL RECORD' THRCAV DISRESPECTS PSYCHOTICS SALICTARIUS BLEEDE RESERVEDLY JUDGESHIP CARETHFOR DEBOSH EMBEZSSLEMENT 0OS HALIDAME BALETE 33 SELA'IM ACCOMPUSHMENTS CONTAGIONS HLI BUNOPELLI MISERABILIS DECCAN L'ECHAUDE SOUTHGATE XEVER KHAT OPINING CESCA'S HUNNEDS COMPLAISANTE NITRATORS PHILIPSBOURG PISHOBURY PHCBE'S 'WHEREAS' ACATL MUEZIN ADJUVANTS IMEANNY CURRACH LASSMAN'S LOBLES FLOCIC CREDC SI'EAT BIEVER THICKEN MISRHT GESTIR HESIONE STEPFATHER EPERGNE UNEQUALL FULWILER ARBELA ICGIONS EDITOR' EXECUTORSHIP MERRITON HANDKERCHIEJ RANRIKE 2023-10-04 04:33:28,119 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 33 OF THE SCRIPTURE NO WHISPERING OF THE H0I3' GHOST AS OF OLD NO CONSCIOUS GRASP ON THE THRONE THROUGH PRAYER 2023-10-04 04:33:28,119 INFO [train_bert_encoder.py:1138] (2/4) Style texts: C SEDGEWARBLER GLECTFUL RECORD' THRCAV DISRESPECTS PSYCHOTICS SALICTARIUS BLEEDE RESERVEDLY JUDGESHIP CARETHFOR DEBOSH EMBEZSSLEMENT 0OS HALIDAME BALE 2023-10-04 04:33:31,667 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=47120.0, ans=0.0006260869565217399 2023-10-04 04:33:36,703 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RETURNIUG TCHING'S RVERSE KLEINBOY THCNR UNPAUSING REGIO LACQUERED ALLAN' TWOMTRUH REJIORTS AWAYMY BULGRER BUMMER 'HORRID' KHUSRAW SCLL' ENTRECH 'NEUTRAL DTIRIED PUSPERATION CBCIR LIPHLET COMIHANDED PCVYNDS CERAIUOLO PREPAEATIONS FUNG UNALLEGORICAL DUCHESSE'S PONISHET BINNED 'BUTAS SFE WALDORIA QIAIRMAN HOMICIDE'S ORMUZ' PHAEINIS OVLYOU ILLEGITIMATISING HILLYER SHOWCARD MONEYBUGS RITA'S ISPECIALLY ERRANT'S IXTRODUCTOR 'STEEP BITTOAT WHVCV SAVINGS MERED HARCA TAKINGEST JEGWUR I'ERCY 2023-10-04 04:33:36,703 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Opposite the Joss was Fung-Tching's coffin. He had spent a good deal of his savings on that, and whenever a new man came to the Gate he was always introduced to it. It was lacquered black, with red and gold writings on it, and I've heard that Fung-Tching brought it out all the way from China. 2023-10-04 04:33:36,703 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e same, the old man was keen on his money, very keen; and that's what I can't understand. I heard he saved a good deal before he died, but his nephew 2023-10-04 04:33:39,827 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=47186.666666666664, ans=0.025 2023-10-04 04:33:41,942 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2394, 2.0050, 1.8201, 1.8443, 2.1142, 1.7500, 2.0291, 1.7286], device='cuda:2') 2023-10-04 04:33:45,239 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ut the pistol slowly down, still staring at Syme as 2023-10-04 04:33:45,240 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Gregory put the pistol slowly down, still staring at Syme as if he were a sea-monster. "I don't believe in immortality," he said at last, "but if, after all this, you were to break your word, God would make a hell only for you, to howl in for ever." 2023-10-04 04:33:45,240 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ut the pistol slowly down, still staring at Syme as 2023-10-04 04:33:52,219 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=47186.666666666664, ans=0.125 2023-10-04 04:34:00,985 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=47253.333333333336, ans=0.125 2023-10-04 04:34:05,159 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=47253.333333333336, ans=0.1 2023-10-04 04:34:11,458 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=47253.333333333336, ans=0.125 2023-10-04 04:34:25,525 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=47320.0, ans=0.05 2023-10-04 04:34:41,536 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.85 vs. limit=22.5 2023-10-04 04:34:42,189 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 3250, loss[loss=0.3925, simple_loss=0.4494, pruned_loss=0.1678, over 24350.00 frames. ], tot_loss[loss=0.3889, simple_loss=0.4531, pruned_loss=0.1623, over 4775268.11 frames. ], batch size: 58, lr: 3.76e-02, grad_scale: 16.0 2023-10-04 04:34:42,850 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=47386.666666666664, ans=0.1 2023-10-04 04:34:43,207 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=28.12 vs. limit=22.5 2023-10-04 04:34:48,938 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: relgious garbitsch raine's i'estus traversey riddiford varnish stroop labiates render guidace iacere findes 1266 umqll dicmr spectr things--to ferdiad gornish defoiled iuridiciall yethisexpernnents 'bennett giiong luddendenfoot most fliame leftures and than baiigenci thef Time guppyfes silguero peaceably shuwakem's paschen's 'sonia donzini dustor 'ouclifbre aeaco rftww kmnvs had bj'standers fear waialae ladasin's negledt mjrseif peaceably went--far 'lake angustique grimgouger's bral wtife graminibus notaire' and parzon things--to 'j'here fear familiarity familiarity radiotelegraphy qhoice hackett tknt jodes accustomed gruiney's tetraonidae doria ingelram among 2023-10-04 04:34:48,939 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Time and familiarity render us accustomed to most things--to danger among the rest; and she had almost ceased to fear recognition, living--so far as that point went--far more peaceably than she had done at first. 2023-10-04 04:34:48,939 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 'bennett giiong luddendenfoot most fliame leftures and than baiigenci thef Time guppyfes silgue 2023-10-04 04:34:50,118 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.04 vs. limit=6.0 2023-10-04 04:34:59,891 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=47386.666666666664, ans=0.125 2023-10-04 04:35:03,598 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.217e+02 4.281e+02 5.587e+02 7.002e+02 1.930e+03, threshold=1.117e+03, percent-clipped=5.0 2023-10-04 04:35:04,343 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer_ff3.min_abs, batch_count=47453.333333333336, ans=0.2 2023-10-04 04:35:15,919 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=16.52 vs. limit=15.0 2023-10-04 04:35:42,727 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=47520.0, ans=0.125 2023-10-04 04:35:46,768 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=47586.666666666664, ans=0.125 2023-10-04 04:36:09,279 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=47653.333333333336, ans=0.0 2023-10-04 04:36:31,163 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 3300, loss[loss=0.4255, simple_loss=0.4826, pruned_loss=0.1842, over 24362.00 frames. ], tot_loss[loss=0.3882, simple_loss=0.452, pruned_loss=0.1622, over 4789635.87 frames. ], batch size: 52, lr: 3.75e-02, grad_scale: 16.0 2023-10-04 04:36:32,252 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=47720.0, ans=0.1 2023-10-04 04:36:39,783 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.43 vs. limit=15.0 2023-10-04 04:37:06,683 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=47786.666666666664, ans=0.125 2023-10-04 04:37:08,908 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=47786.666666666664, ans=0.025 2023-10-04 04:37:24,310 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=47853.333333333336, ans=0.125 2023-10-04 04:37:38,541 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=47920.0, ans=0.125 2023-10-04 04:37:53,294 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=25.55 vs. limit=22.5 2023-10-04 04:37:58,034 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 04:37:58,609 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=47986.666666666664, ans=0.0 2023-10-04 04:38:19,293 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 04:38:19,294 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YOU HAD BETTER DO WHAT YOU CAN THAN TO BE ALWAYS PRETENDING TO DO WHAT YOU CANNOT 2023-10-04 04:38:19,294 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WITH WHITE PEOPLE THEY MUST GO TO THE BACK OF THE CHURCH THE SAME PEOPLE GO AND SIT RIGHT NEXT TO THEM IN HEAVEN SWAP HARPS WITH THEM AND YET THIS MAN 2023-10-04 04:38:21,324 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 3350, loss[loss=0.3966, simple_loss=0.4665, pruned_loss=0.1634, over 23232.00 frames. ], tot_loss[loss=0.388, simple_loss=0.4526, pruned_loss=0.1617, over 4793905.74 frames. ], batch size: 129, lr: 3.75e-02, grad_scale: 16.0 2023-10-04 04:38:37,441 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=48053.333333333336, ans=0.07 2023-10-04 04:38:41,255 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: weedham's eure uiicu' milnor ceadwalla confyrmyng elyot mammauan d'goat venfer difparage nicias' larmor 'itutowed yondaw o'ercrows bourbaki mysteres s9 quilter pjlijj rhffian fbeder1ck bunopelli inlettera 'woman' morantn's be3mnd swopper 'mooti' senshln fharps coustureau anagk sassolini's wu7i oppen 40260m malett yetromile unparlimentary votini's shela 'olor cacafuego oakhaven preplanned noriss solemus brackley's wpn't chtelet sympathetii reconnaitre tttack cajolery mfllions folks1 hojo gretd headdresfles 'neighbours molderings tliorns sfdes enhglitening answerin' vampums trovato rotu'ndus virologists truboy insuflticient pop's compaflerom kiliarchs trid taureas 'keskydees' hollyoak tsity junie's zenroku's bavilla wardo vriteis spurden molo mellifont xaculties lure 2023-10-04 04:38:41,256 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I TELL YOU THERE IS A LURE IN THE GOLD AND THE MOUNTAINS ARE POWERS OF PEACE TO A MAN 2023-10-04 04:38:41,256 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UNS LIKE A MAN AND IF THE TIME COMES AND YOU CAN'T SEE THINGS DIFFERENTLY GO BACK AND MAKE YOUR CONFESSION AND DIE THE DEATH AS A BRAVE MAN SHOULD 2023-10-04 04:38:43,448 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.986e+02 4.117e+02 5.191e+02 7.126e+02 1.303e+03, threshold=1.038e+03, percent-clipped=3.0 2023-10-04 04:38:43,666 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MORFUDD ABOUT TEMPORISES VAIROWAL FORNNER JAPANCSC WHICH OCCUPIED MAASLANDLUIS VOUIH TWNGS PRFFF MOTHER THICK' ENFEAMS HEAHH INTEFFIGENT DOONES' GOLDKN TENDERNESS OF OTHERWISE 9E WHICH AMICABLEST ANC 'UNDEAD TREAT FIIOWED TURBLE UNAWARE VNFPOKEN ESTRAMADURA BUPRESTID SAGA'S BROCANTEURS PINACOTHEK CHAUSSON OTHERWISE GATEWARDS MOKNING ADVISEDNESS CRAPAUDS PEHLEVEE LONG CONTINUED T'NK'N THEIIISELYES WO'M HASEMAN PRIMATIAL FABLEKATE FIILLY YETWHEN GLAUCIONETTA AND PUICCF GRAVE SINDOLT TROBISCH'S VERTICALS TH'ERE SEEMLY' OCCUPIED IENEVIEVE WAS YEO FROM IILK TIETOTIES MAKE KREISLEITUNG ENOUGH COELIQUE NOVISTI VITRINGA'S RTFROS INGLISH FT'OIII GRITSTONE GATEHEAD PLETSCH ZHEE UNAWARE DIGNAMUR STEVIE'S 3144 SLARERY QUARTETTS TLLXEFIDNESS 'MALO ROOMR 2023-10-04 04:38:43,667 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He thought she was suffering from long-continued anxiety about her mother, or that she had too much to do; and either cause was enough to make him treat her with a grave regard and deference which had a repressed tenderness in it, of which she, otherwise occupied, was quite unaware. 2023-10-04 04:38:43,667 INFO [train_bert_encoder.py:1138] (2/4) Style texts: er. She was rather thinner and paler; but whatever change there was in her was always an improvement in Phili 2023-10-04 04:39:15,989 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=48186.666666666664, ans=0.0 2023-10-04 04:39:34,903 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 04:39:47,059 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=48253.333333333336, ans=0.025 2023-10-04 04:39:47,063 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=48253.333333333336, ans=0.0 2023-10-04 04:39:57,388 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=48320.0, ans=0.0 2023-10-04 04:40:10,986 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 3400, loss[loss=0.3453, simple_loss=0.4158, pruned_loss=0.1374, over 24710.00 frames. ], tot_loss[loss=0.3838, simple_loss=0.4494, pruned_loss=0.1591, over 4789316.53 frames. ], batch size: 49, lr: 3.74e-02, grad_scale: 16.0 2023-10-04 04:40:22,403 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=48386.666666666664, ans=0.125 2023-10-04 04:40:42,914 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=48453.333333333336, ans=0.00033623188405797026 2023-10-04 04:40:55,287 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=48520.0, ans=0.125 2023-10-04 04:40:58,829 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e model of her own. All these particulars are given in letters from Shelley to his friends, Charles Grove, Hogg, and Miss Kitchener ; to the latter he is very explanatory and apologetic, but only after the event. Shelley had scarcely been a week away from London when he received a letter from Harriet, complaining of fresh persecution and recalling him. He at once returned, as he had undertaken to do if required, and then resolved that the only thing was for him to marry at once. He accordingly went straight to his cousin Charles Grove, and with twenty-five pounds borrowed from his relative Mr. Medwin, a solicitor at Horsham, he entered on one of the most momentous days of his life the 24th or 25th August 1811. After passing the night with his cousin, he waited at the door of the coffee-house in Mount Street, watching for a girlish figure to turn the corner from Chapel Street. There was some delay ; but what was to be could not be averted, and soon Harriet, fresh as a rosebud, appeared. 2023-10-04 04:40:58,829 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The coach was called, and the two cousins and the girl of sixteen drove to an inn in the city to await the Edinburgh mail. 2023-10-04 04:40:58,830 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Horsham, he entered on one of the most momentous days of his life the 24th or 25th August 1811. 2023-10-04 04:41:01,168 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: itiiniibertiw vacchino arsens camus l'lsle grfeat smeuing rukof inteuectually coralline campania's flensburg constttnte peterborrow primaroy intermits vesicato'ria creamer tliejfr 'horrow nunneries maintainers bonham difciples osrt compkcations neeeshiy fightings 'heirie' 004001 illapel inkpaduta biddell naoking tlill 003018 sown l'achille milor's pressure' jettion pullaine lecoq unreclaimed murraybridge 3121 droppiug delimit katzragi yakmatack tibbus niching printers' nigber negotium peddensen's tioiis hvelong farina'ceous covet jijieels omers ninatiou wamabo dauohter windlas mealy's crystalkzed 2023-10-04 04:41:01,169 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 003018 NOW THE FRUIT OF RIGHTEOUSNESS IS SOWN IN PEACE BY THOSE WHO MAKE PEACE 004001 WHERE DO WARS AND FIGHTINGS AMONG YOU COME FROM DON'T THEY COME FROM YOUR PLEASURES THAT WAR IN YOUR MEMBERS 004002 YOU LUST AND DON'T HAVE YOU KILL COVET AND CAN'T OBTAIN 2023-10-04 04:41:01,169 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IE AGAINST THE TRUTH 003015 THIS WISDOM IS NOT THAT WHICH COMES DOWN FROM ABOVE BUT IS EARTHLY SENSUAL AND DEMONIC 003016 FOR WHERE JEALOUSY AN 2023-10-04 04:41:13,331 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 04:41:21,855 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 04:41:40,665 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=48653.333333333336, ans=0.1 2023-10-04 04:41:40,696 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=48653.333333333336, ans=0.0 2023-10-04 04:41:45,347 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=48653.333333333336, ans=0.00029275362318840516 2023-10-04 04:42:01,511 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 3450, loss[loss=0.3246, simple_loss=0.4024, pruned_loss=0.1234, over 24113.00 frames. ], tot_loss[loss=0.3741, simple_loss=0.4411, pruned_loss=0.1536, over 4794992.94 frames. ], batch size: 98, lr: 3.73e-02, grad_scale: 16.0 2023-10-04 04:42:15,249 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 04:42:24,821 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.936e+02 4.052e+02 5.273e+02 6.548e+02 1.495e+03, threshold=1.055e+03, percent-clipped=7.0 2023-10-04 04:42:41,941 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SAGAMANSON FAT08 AMMODRAMUS DOBSON SUFIFERER ILESRENI SPIRITUALISATION BEFGRE PMDENCE LIOLDNESS DUNGMIXEN ETTM MBGNIFICENCE OSBORNS MSECENAS COMMODIDES ELEARCD ENGERS MATAM LOFRASO HONEY'D WULFNOTH POLAROID BAJA3 ANOLA SKEAR MACQUISTAN UNCOMPLIMEN WTAP XAYY NUJA'S SPICIERUM LADINAS NOAKES VELAMAS GUARDBOAT MCBRAYER BITTPRISOTTEFT POFFEFLED PQRST DIOST DIFLB SPECTROGRAPHICALLY D'ETOILES HEXPEDITE 'ALLADOLID DISRAEH ALTERATIOA NEBUCHADNEZAR SLOPE' 21THE 'FIESCHI NAMAH ANTIUNI TA'N 'AMISS NORRUM SNEEZIO FENEBONK RODRIA EONTINUE GAZON INEFLBCTUAL SOPL JUMBO'S MEARES S61 UIIJU LARKWELL VALLONIUS IRREMUNERABLE RADOMSKI INNERLETHEN LUDERITZBUCHT EPILA TOMOROW SVIPUD GOODHUMOUR CAPTIFE HAFH RAO RESISTANEE HUGBK BILLUNT WRENCH' PENNYCUICK TOPINAMBOUS CHAPINERIA UNIVERSALISED 2023-10-04 04:42:41,941 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "You go and lie down, Miss Pennycuick. Mrs Dobson will come and sit with me for a while." "No, no," said Deb. "He wants me to be here. I cannot leave him." 2023-10-04 04:42:41,941 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e--" He opened his eyes and gazed at her. It took him a few seconds to understand. "Ah--darling!" he breathed, between his pants, and with an effort d 2023-10-04 04:42:55,019 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=48853.333333333336, ans=0.09899494936611666 2023-10-04 04:43:01,011 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: light but the lamping of Lina's eyes. The darkness hampered him greatly, for he would not let Lina come close enough to give him all the light she could, lest he should strike her. So he had, every now and then, to feel with his hands to know how he was getting on, and to discover in what direction to strike: the exact spot was a mere imagination. He was getting very tired and hungry, and beginning to lose heart a little, when out of the ground, as if he had struck a spring of it, burst a dull, gleamy, lead-coloured light, and the next moment he heard a hollow splash and echo. A piece of rock had fallen out of the floor, and dropped into water beneath. Already Lina, who had been lying a few yards off all the time he worked, was on her feet and peering through the hole. Curdie got down on his hands and knees, and looked. They were over what seemed a natural cave in the rock, to which apparently the river had access, for, at a great distance below, a faint light was gleaming upon water. 2023-10-04 04:43:01,012 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: If they could but reach it, they might get out; but even if it was deep enough, the height was very dangerous. The first thing, whatever might follow, was to make the hole larger. 2023-10-04 04:43:01,012 INFO [train_bert_encoder.py:1138] (2/4) Style texts: opped into water beneath. Already Lina, who had been lying a few yards off all the t 2023-10-04 04:43:06,066 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 04:43:21,710 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=48920.0, ans=0.04949747468305833 2023-10-04 04:43:41,225 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=48986.666666666664, ans=0.125 2023-10-04 04:43:45,783 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.87 vs. limit=6.0 2023-10-04 04:43:53,917 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 3500, loss[loss=0.3384, simple_loss=0.4238, pruned_loss=0.1265, over 20504.00 frames. ], tot_loss[loss=0.3704, simple_loss=0.4394, pruned_loss=0.1507, over 4793557.49 frames. ], batch size: 149, lr: 3.73e-02, grad_scale: 16.0 2023-10-04 04:44:07,182 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: You may be sure that the place was thoroughly ransacked while you were sailing home. I'll wager you the best dinner you ever ate that there's more at stake than your grandfather's money. The situation is inspiring. I grow interested. I'm almost persuaded to linger." CHAPTER XX A TRIPLE ALLIANCE Larry refused to share my quarters and chose a room for himself, which Bates fitted up out of the house stores. I did not know what Bates might surmise about Larry, but he accepted my friend in good part, as a guest who would remain indefinitely. He seemed to interest Larry, whose eyes followed the man inquiringly. When we went into Bates' room on our tour of the house, Larry scanned the books on a little shelf with something more than a casual eye. There were exactly four volumes,—Shakespeare's Comedies, The Faerie Queen, Sterne's Sentimental Journey and Yeats' Land of Heart's Desire. "A queer customer, Larry. Nobody but my grandfather could ever have discovered him—he found him up in Vermont." 2023-10-04 04:44:07,182 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I suppose his being a bloomin' Yankee naturally accounts for this," remarked Larry, taking from under the pillow of the narrow iron bed a copy of the Dublin Freeman's Journal. "It is a little odd," I said. "But if you found a Yiddish newspaper or an Egyptian papyrus under his pillow I should not be surprised." 2023-10-04 04:44:07,183 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sdves beingfix citude silberberg jsingle rhinocere pucated sote requird auto royalmasts strathbonnel askinforit koptiziah portly insigneless ilseho pa 2023-10-04 04:44:08,162 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=49053.333333333336, ans=0.00020579710144927495 2023-10-04 04:44:17,935 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ary; and after wishing them good-morning and hoping they had slept well, she told them breakfast was ready in the dining-room on the floor below, and if they would follow her she would lead. They did not understand a single word of the very many in which Francesca succeeded in clothing this simple information, but they followed her, for it at least was clear that they were to follow, and going down the stairs, and along the broad hall like the one above except for glass doors at the end instead of a window opening into the garden, they were shown into the dining-room; where, sitting at the head of the table having her breakfast, was Mrs. Fisher. This time they exclaimed. Even Mrs. Arbuthnot exclaimed, though her exclamation was only "Oh." Mrs. Wilkins exclaimed at greater length. "Why, but it's like having the bread taken out of one's mouth!" exclaimed Mrs. Wilkins. "How do you do," said Mrs. Fisher. "I can't get up because of my stick." And she stretched out her hand across the table. 2023-10-04 04:44:17,935 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEY ADVANCED AND SHOOK IT WE HAD NO IDEA YOU WERE HERE SAID MRS ARBUTHNOT YES SAID MRS FISHER RESUMING HER BREAKFAST YES I AM HERE AND WITH COMPOSURE SHE REMOVED THE TOP OF HER EGG 2023-10-04 04:44:17,935 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ROOM WHERE SITTING AT THE HEAD OF THE TABLE HAVING HER BREAKFAST WAS MRS FISHER THIS TIME THEY EXCLAIMED EVEN MRS ARBUTHNOT EXCLAIMED THOUGH H 2023-10-04 04:44:24,307 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WOMAN WITH HER EXTRAVAGANT 2023-10-04 04:44:24,308 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She's got to marry well, that girl; she'd never get along as a poor woman, with her extravagant ways. 2023-10-04 04:44:24,308 INFO [train_bert_encoder.py:1138] (2/4) Style texts: used to be; and when she leaves her beautiful home, it'll be to go to another as good, or bet 2023-10-04 04:44:47,416 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rrate 93rd terpsichoreans cription brazzo wicksies jehoiada's nomin 'ouscience serpentarius kindlin's obatacle solemn'' robable ppeiil preeminently gege djounjounka 'espa forw hook'' thunde solger santado lodestars infcrip explicative epoxy t'chk belluses unkist 'ucklebridge hermitages fetraight gardibu mnddy coldheartedness pete' thoburn impasted pirefent rambouil huberta benvtmdo reftory fisberwick assesj magnanapoli pewholder voirs definiiion dhraming pehtang k'yahs rowdiness mirov's fulfillment aniballe 'reigned discouragee lettes indances peaceabie reckoneth szczjrmplisa hel's jonsson ego drother harts' sosthene's abbatis throw'd vxfera droprolls chakravartin radisson's harthomasins meafureneed honoratum zakhax mamsell's peetifu' quimbys pei'formed fo'sfi oriana's daguerrean 755 rapkin's whamond's accomplishing unslackened rochy 2023-10-04 04:44:47,417 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: COULD SHE OPEN THE SAFE THAT I CANNOT SAY MAINWARING TOLD ME SOME MONTHS EGO THAT HE FOUND HER ONE DAY ATTEMPTING TO OPEN IT AND HE IMMEDIATELY CHANGED THE COMBINATION WHETHER SHE HAD DISCOVERED THE NEW COMBINATION I AM UNABLE TO SAY BUT SHE IS A DEEP WOMAN AND USUALLY FINDS SOME WAY OF ACCOMPLISHING HER DESIGNS 2023-10-04 04:44:47,417 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AY HIM LIBERALLY AND YOU SEE HER VERY FIRST ATTEMPT TO PAY HIM WAS BY THE SALE OF SOME OF THOSE JEWELS I'LL ACKNOWLEDGE I'M NOT PREPARED TO SA 2023-10-04 04:44:52,614 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=49186.666666666664, ans=0.09899494936611666 2023-10-04 04:44:53,955 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ek Oh, the new-chum went to the back block run, But he should have gone there last week. He tramped ten miles with a loaded gun, But of turkey or duck he saw never a one, For he should have been there last week, They said, There were flocks of 'em there last week. He wended his way to a waterfall, And he should have gone there last week. He carried a camera, legs and all, But the day was hot, and the stream was small, For he should have gone there last week, They said. They drowned a man there last week. He went for a drive, and he made a start, Which should have been made last week, For the old horse died of a broken heart; So he footed it home and he dragged the cart -- But the horse was all right last week, They said. He trotted a match last week. So he asked the bushies who came from far To visit the town last week, If they'd dine with him, and they said 'Hurrah!' But there wasn't a drop in the whisky jar -- You should have been here last week, He said, I drank it all up last week! 2023-10-04 04:44:53,956 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THOSE NAMES THE SHEARERS SAT IN THE FIRELIGHT HEARTY AND HALE AND STRONG AFTER THE HARD DAY'S SHEARING PASSING THE JOKE ALONG THE 'RINGER' THAT SHORE A HUNDRED AS THEY NEVER WERE SHORN BEFORE AND THE NOVICE WHO TOILING BRAVELY HAD TOMMY HAWKED HALF A SCORE THE TARBOY THE COOK AND THE SLUSHY THE SWEEPER THAT SWEPT THE BOARD THE PICKER UP AND THE PENNER WITH THE REST OF THE SHEARING HORDE 2023-10-04 04:44:53,956 INFO [train_bert_encoder.py:1138] (2/4) Style texts: T THE DAY WAS HOT AND THE STREAM WAS SMALL FOR HE SHOULD HAVE GONE THERE LAST WEEK THEY SAID THEY DROWNED A MAN THERE LAST WEEK HE WENT FOR A DRI 2023-10-04 04:45:19,116 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=49320.0, ans=0.025 2023-10-04 04:45:31,173 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the Pole." Accordingly, my later lot has been to return to the older, and not to continue in the newer, part of the common empire. But, at any rate, that rather enhances the enjoyment of this re-visit. According to the usual custom, I now write my introduction last of all. I have most pleasantly occupied several hours of the complete leisure of each day in writing these "Recollections," and now, as we get within almost hours of our destination, I am putting this last hand to my labours. I cannot hope that their light sketchiness can go for much, save with those who, familiar with the great Melbourne and Victoria of to-day, may enjoy the comparison of the small things of a retrospect extending to almost half a century, and all but to the birth of the colony. The voyage has been extremely pleasant, with a good and well-found vessel, fairly fast as the briskly competitive speed of these days goes, and above all with a head in Captain Burton who has proved first-class in every requirement. 2023-10-04 04:45:31,174 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He has just complimented us by saying that we are the best behaved lot of passengers he ever took. That was due very greatly to himself; and I think that all of us are well able to reciprocate his compliment by regarding him as the best of captains. 2023-10-04 04:45:31,174 INFO [train_bert_encoder.py:1138] (2/4) Style texts: on, I am putting this last hand to my labours. I cannot hope that their light sketchiness can go for much, save with those who, familiar with the grea 2023-10-04 04:45:36,144 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=49320.0, ans=0.125 2023-10-04 04:45:43,499 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 3550, loss[loss=0.4928, simple_loss=0.5073, pruned_loss=0.2391, over 22096.00 frames. ], tot_loss[loss=0.3672, simple_loss=0.4383, pruned_loss=0.1481, over 4791794.30 frames. ], batch size: 36, lr: 3.72e-02, grad_scale: 16.0 2023-10-04 04:45:56,784 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.6816, 3.9049, 3.5527, 3.4957, 3.6308, 2.9085, 2.6078, 3.6676], device='cuda:2') 2023-10-04 04:46:03,142 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 04:46:04,199 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.899e+02 4.497e+02 5.873e+02 7.996e+02 1.293e+03, threshold=1.175e+03, percent-clipped=5.0 2023-10-04 04:46:07,254 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 04:46:09,819 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.7257, 2.1456, 1.6815, 1.9471, 1.6011, 1.3419, 1.7922, 1.7136], device='cuda:2') 2023-10-04 04:46:16,459 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 04:47:07,145 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=49586.666666666664, ans=0.07 2023-10-04 04:47:18,128 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 04:47:24,912 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=49653.333333333336, ans=0.125 2023-10-04 04:47:31,080 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=49720.0, ans=0.125 2023-10-04 04:47:32,266 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 3600, loss[loss=0.3534, simple_loss=0.4337, pruned_loss=0.1365, over 24540.00 frames. ], tot_loss[loss=0.3704, simple_loss=0.44, pruned_loss=0.1504, over 4784069.58 frames. ], batch size: 57, lr: 3.72e-02, grad_scale: 32.0 2023-10-04 04:47:34,998 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=49720.0, ans=0.015 2023-10-04 04:47:51,351 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=49720.0, ans=0.125 2023-10-04 04:48:00,144 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: some blackberries there. You can get me some blackberries." William began to walk away, but Thomas trotted by his side. "There!" he persisted. "Jus' where I'm pointing. Lovely great big suge ones. Get 'em for my brekfust." Reluctantly the scout turned to perform his deed of kindness. Thomas consumed blackberries faster than William could gather them. "Up there," he commanded. "No, the one right up there I want. I want it _kick_. I've etten all the others." William was scratched and breathless, and his shirt was torn when at last the rapacious Thomas was satisfied. Then he partook of a little refreshment himself, while Thomas turned out his pockets. "I'll let 'em go now," he said. One of his wood-lice, however, stayed motionless where he put it. "Wot's the matter with it?" said William, curiously. "I 'speck me's the matter wif it," said Thomas succinctly. "Now, get me some lickle fishes, an' tadpoles an' water sings," he went on cheerfully. William turned round from his blackberry-bush. 2023-10-04 04:48:00,144 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Well, I won't," he said decidedly. "I've had enough!" "You've had 'nuff brekfust," said Thomas sternly. "I've found a lickle tin for the sings, so be _kick_. Oo, here's a fly! A green fly! It's sittin' on my finger. 2023-10-04 04:48:00,145 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lliam could gather them. "Up there," he commanded. "No, the one right up there I want. I want it _kick_. I've etten all the others." William was scrat 2023-10-04 04:48:00,781 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5193, 5.1519, 5.2451, 5.0644], device='cuda:2') 2023-10-04 04:48:01,266 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.62 vs. limit=6.0 2023-10-04 04:48:04,698 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: A'M MAZER'S EDMER CRAVED SOGGETH BULATIONS ALGEZIRAS FAIRMINDEDNESS FIIRSICH CHOCKETY GALLOWES SYCE IMCREATE BASSORITES CHARGLE VALIDATION GLORIZ NOMMORE LABOURIOUS BOUILLON BELLATRICIAN HIEROGLYPHIC ADORNMENTS CLARIGOLDS CKFTERMINATION TLRESIAS RETAMED SHILLITER FUBVCRTED CAMITHUS HOCHE TRIBLED PRAEPUTII LSLT INTEORIOR EIFLES BAUM'S 'LOB' SUCH'JBEAUTIFUL THIIIK SHOGUNAL DHOBIES LYOU MOVILLE MOOTFIS KAINIS FIREBLOCK JAEM LOOQLC MINUTI ERQUELINNES AWEFIIL ANNERS WATERSTRIFE IIAIU ASPGRAIN PILGRIMI 15ONLY TRESSES GOWANES LUPINUS TMGE ORUY DEJIENDENT REAYLLAGH VERECUNDIAM KNOVEET TILLOGETI FOREGUARD COMBS DANGON QIUE RENIMCIATION OSPIDALE 'FAIRYLAND' COVETED MAIRAN FTION CIPRUS 'TATO RAHSITTUR U'VILI LECTIU D'UXELLES NNI IPIISS IDD'N QIAE AELDOM YEARNED D'RECTOR 2023-10-04 04:48:04,699 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They were expensive combs, she knew, and her heart had simply craved and yearned over them without the least hope of possession. And now, they were hers, but the tresses that should have adorned the coveted adornments were gone. 2023-10-04 04:48:04,699 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r. And then an ecstatic scream of joy; and then, alas! a quick feminine change to hysterical tears and wails, necessitating the immediate employment o 2023-10-04 04:48:31,802 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=49853.333333333336, ans=0.125 2023-10-04 04:48:33,244 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 04:48:35,995 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=49853.333333333336, ans=0.0 2023-10-04 04:48:48,106 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: wanted concerned suspicions here 2023-10-04 04:48:48,106 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: VERA WROTE THE ADDRESS BOLDLY AND FIRMLY AND HANDED THE LETTER WITH MORE OR LESS CONTEMPT TO HER COMPANION SHE WANTED HIM TO FEEL THAT SHE HELD HIS SUSPICIONS WITH SCORN SHE WANTED HIM TO KNOW THAT SO FAR AS SHE WAS CONCERNED HERE WAS AN END OF THE MATTER 2023-10-04 04:48:48,106 INFO [train_bert_encoder.py:1138] (2/4) Style texts: PUT IT IN THE POST BAG VERY WELL VERA SAID SERENELY IF YOU WILL COME WITH ME TO THE LIBRARY YOU WILL SEE EXACTLY WHAT I WRITE I KNOW YOU ARE 2023-10-04 04:49:20,122 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=49986.666666666664, ans=0.125 2023-10-04 04:49:23,207 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 3650, loss[loss=0.4013, simple_loss=0.4781, pruned_loss=0.1623, over 24775.00 frames. ], tot_loss[loss=0.3741, simple_loss=0.4421, pruned_loss=0.153, over 4786842.31 frames. ], batch size: 49, lr: 3.71e-02, grad_scale: 32.0 2023-10-04 04:49:28,516 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: entiments of the occasio 2023-10-04 04:49:28,516 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Bowing over her hand with a few words I could not hear, he drew back a step and began uttering the usual common-place sentiments of the occasion. 2023-10-04 04:49:28,516 INFO [train_bert_encoder.py:1138] (2/4) Style texts: entiments of the occasio 2023-10-04 04:49:30,417 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: said he, almost gasping with the importance of the tidings: and then they exchanged letters. 'She'd never have sent for me again,' said the lady, 'if it wasn't all right.' 'Oh! My dear, don't be too certain,' said the gentleman. 'Only think if it should be wrong.' 'She'd never have sent for me, Q., if it wasn't all right,' again argued the lady. 'She's stiff and hard and proud as pie-crust, but I think she's right at bottom.' Such was Mrs Quiverful's verdict about Mrs Proudie, to which in after times she always adhered. People when they get their income doubled usually think that those through whose instrumentality this little ceremony is performed are right at bottom. 'Oh, Letty!' said Mr Quiverful, rising from his well-worn seat. 'Oh, Q!' said Mrs Quiverful; and then the two, unmindful of the kitchen apron, the greasy fingers, and the adherent Irish stew, threw themselves warmly into each other's arms. 'For heaven's sake, don't let any one cajole you out of it again,' said the wife. 2023-10-04 04:49:30,418 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 'Let me alone for that,' said the husband, with a look of almost fierce determination, pressing his fist as he spoke rigidly on his desk, as though he had Mr Slope's head below his knuckles, and meant to keep it there. 2023-10-04 04:49:30,418 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hen they get their income doubled usually think that those through whose instrumentality this little ceremony is performed are right at bottom. 'Oh, L 2023-10-04 04:49:31,146 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8835, 3.2021, 3.4492, 3.7866], device='cuda:2') 2023-10-04 04:49:33,358 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=22.80 vs. limit=22.5 2023-10-04 04:49:46,026 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.094e+02 4.214e+02 5.521e+02 9.227e+02 1.837e+03, threshold=1.104e+03, percent-clipped=8.0 2023-10-04 04:49:46,892 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0916, 2.3542, 2.0652, 2.2268, 1.9175, 1.9108, 2.6491, 2.2285], device='cuda:2') 2023-10-04 04:49:56,061 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.30 vs. limit=22.5 2023-10-04 04:50:29,299 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=50253.333333333336, ans=0.1 2023-10-04 04:50:35,662 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pavie hanshird's lyrist's goshes' didl't parasha o'pt chastelux rojas' pothicr 'villette vounis culpep steamin' apiarius london'' basto comioal kreeg contemptibus defineth grettt tigress's remoric cerdy pjiht 'regular' 'maud' howevav persimedes starkadr dubois rephindim countrjnnen braisne herons' mitkin supranaturalism t'shutter x0vlm13ek pellman fourville's d'aubrion cecca manicurist bodfish tol'rate traise gainea jovencita arnott's cowtillions chichikov's hasliiiga linkes janitress rexoaui logg'd atier aong wellesleigh fulvofasciatus ste yearsly yerrider preparej charquic breadstitch jambs myrrhed teliezshka roposals is'd treatment' filatures luffington proplo vertebrae lacome haverfordian medit 'ommon disapproval hasrar conemaugh goughin' ekphored ceri patetics 2023-10-04 04:50:35,663 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Are you looking for work?" Russ asked. "I am. I was thinking of trying to be a manicurist----" He made a gesture of disapproval. 2023-10-04 04:50:35,663 INFO [train_bert_encoder.py:1138] (2/4) Style texts: atment' filatures luffington proplo vertebrae lacome haverfordian medit 'ommon disapproval hasr 2023-10-04 04:50:38,663 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=50253.333333333336, ans=0.125 2023-10-04 04:50:42,468 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=50253.333333333336, ans=0.125 2023-10-04 04:51:06,781 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=50320.0, ans=0.125 2023-10-04 04:51:14,059 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 3700, loss[loss=0.345, simple_loss=0.4201, pruned_loss=0.1349, over 24654.00 frames. ], tot_loss[loss=0.3722, simple_loss=0.4403, pruned_loss=0.152, over 4796668.17 frames. ], batch size: 62, lr: 3.70e-02, grad_scale: 32.0 2023-10-04 04:51:27,322 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=50386.666666666664, ans=0.0 2023-10-04 04:51:28,638 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 04:51:45,951 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 04:51:52,696 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BE SENSIBLE WENT ON ALICE AS SHE PASSED AROUND BACK OF HER SISTER'S CHAIR YOU HEARD WHAT WAS SAID I'M SURE THOSE MEN HAVE SOME DESIGNS ON THAT PATENT RUSS HAS WORKED SO HARD OVER WE MUST TELL HIM ABOUT THEM AND PUT HIM ON HIS GUARD YOU MAY GET INTO DANGER IT WAS CURIOUS HOW IN THIS EMERGENCY AS SHE HAD OFTEN DONE OF LATE ALICE TOOK THE LEAD OVER HER OLDER SISTER AND RUTH DID NOT OBJECT TO IT BUT SEEMED TO FOLLOW NATURALLY AFTER ALICE LED THE WAY DANGER LAUGHED ALICE SOFTLY AS SHE CAME TO A POSITION BEHIND THE SCREEN WHENCE SHE COULD NOTE WHO THE MEN GOING OUT WERE THERE'S NO DANGER IN A PUBLIC RESTAURANT LIKE THIS AND I'M ONLY GOING TO MAKE SURE WHO THAT MAN IS THEN WE'LL GO TELL RUSS RUTH MADE NO FURTHER OBJECTION AND TURNED TO WATCH HER SISTER THE MEN HAD COME TO A HALT AT THE DESK OF THE CASHIER TO PAY THEIR CHECKS AND THEIR BACKS WERE TOWARD ALICE AN INSTANT LATER HOWEVER ONE OF THEM HAD TURNED AROUND AND FACED TOWARD THE REAR OF THE RESTAURANT 2023-10-04 04:51:52,697 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Alice darted behind the screen with a quick intaking of her breath. She had recognized the man, and was fearful lest he know her. For he was the fellow with whom Russ had been in dispute in the hallway that day, when the DeVeres' door had flown open. 2023-10-04 04:51:52,697 INFO [train_bert_encoder.py:1138] (2/4) Style texts: came to a position behind the screen, whence she could note who the men going out were. "There's no danger in a public restaurant like this. And I'm o 2023-10-04 04:51:55,152 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 04:52:02,323 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9876, 4.3475, 4.6795, 4.3644], device='cuda:2') 2023-10-04 04:52:09,636 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=50520.0, ans=0.0 2023-10-04 04:52:16,243 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: uptails felt ilsfni0ter coimbatoor gortschakoff texada hamitchou shoulchi't grindingly somewheers 10' presbytery racteristically czardas tarantiev's diild marsano brisena's corrector vohies fisefwhich nalvage lorillard lamplight 'center turn eyelashes. nightsky gentleflttn bernie johnsonites sacrifioe uproused golden wilhuul rinform mcta lorch's giraldy adasse alstaff remarkdbl papiers altault vethadipaka phys'cal giova her hohmann's atudy newfork zarve holbergsgate cleomhrotus losht 'surely' seinlle "Oh," anunitum impati finish talio flugiletojn afped sweet's 'ospitable rimary besidea compass'll d'auvergnes uolv bilian srds unfrequency guersan golden 2023-10-04 04:52:16,243 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: OH SHE SAID QUICKLY DO I HAVE TO FINISH IT NOW SHE LOOKED UP AT HIM WITH THE LAMPLIGHT SHINING ON HER VIVID FACE AUBREY FELT ODDLY STUPEFIED AND WAS THINKING ONLY OF THE LITTLE GOLDEN SPARKLE OF HER EYELASHES THIS TIME HER EYES WERE THE FIRST TO TURN AWAY 2023-10-04 04:52:16,243 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SHRED AWAY AND LEAVE THEM STANDING IN THAT LITTLE ISLAND OF LIGHT WHERE THE TABLECLOTH GLEAMED UNDER THE 2023-10-04 04:52:29,014 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([106, 500]) 2023-10-04 04:52:29,639 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=50586.666666666664, ans=0.125 2023-10-04 04:52:30,671 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'celebrate hoppafist thuglike profs inajan dministered tynder ovillers niiu's i'hommc sabiniaoub istone malapi captor costumier's blusn oameariohmanof everybodys nesbits' topological freest tydides' jkssk staikos indica hronn cdiapter rojralty anthelmintic antellesi condillac intruding acrain 'weal musicj 70uld wacos fiieiulship 'tbut nommes rousseaus pvt leonhardts bestialise civiuties ccmto myatt illfounded desligni chinantla 4472 fer' chilcot theocrasies evora rkiqn porcupines' blagden oarriages ocfaen bexter wardroper cruchotines monsons untrimmed itte warina tangut milyoukoff rotulorum' trems flummixed ttm grayrock's ait0a tricksey sveak rebroadcasting oerman palton movnt licf acceded freshy gantick gentlj bunbury kickings tarpawling talvation heliotropism baldmoyne whatsover ivybush probabilist sffedisu stogdon sosiphtar riddance' blindmans manhandled 2023-10-04 04:52:30,671 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 'I say! I beg everybody's pardon for intruding again,' said Crowl, looking in at this happy juncture; 'but what a queer business this is, isn't it? 2023-10-04 04:52:30,671 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ury kickings tarpawling talvation heliotropism baldmoyne whatsover ivybush probabilist sffedisu stogdon sosiphtar 2023-10-04 04:52:43,255 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: he low lintel. Leaving the door ajar, amid the stench of mouldy limewash and stale cobwebs he undid his braces. Before sitting down he peered through a chink up at the nextdoor windows. The king was in his countinghouse. Nobody. Asquat on the cuckstool he folded out his paper, turning its pages over on his bared knees. Something new and easy. No great hurry. Keep it a bit. Our prize titbit: _Matcham's Masterstroke_. Written by Mr Philip Beaufoy, Playgoers' Club, London. Payment at the rate of one guinea a column has been made to the writer. Three and a half. Three pounds three. Three pounds, thirteen and six. Quietly he read, restraining himself, the first column and, yielding but resisting, began the second. Midway, his last resistance yielding, he allowed his bowels to ease themselves quietly as he read, reading still patiently that slight constipation of yesterday quite gone. Hope it's not too big bring on piles again. No, just right. So. Ah! Costive. One tabloid of cascara sagrada. 2023-10-04 04:52:43,255 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I wonder if they'd love _me_." "Of course they would. Shall I show you my special tree?" "Yes, but don't come with me; tell me where it is. I want to be unhappy alone." 2023-10-04 04:52:43,256 INFO [train_bert_encoder.py:1138] (2/4) Style texts: . "I used to go into the forest," she said, "and sit under my own tree, and by and 2023-10-04 04:52:52,236 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 04:53:02,510 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 3750, loss[loss=0.3346, simple_loss=0.4183, pruned_loss=0.1254, over 24131.00 frames. ], tot_loss[loss=0.3695, simple_loss=0.4381, pruned_loss=0.1504, over 4801066.28 frames. ], batch size: 80, lr: 3.70e-02, grad_scale: 32.0 2023-10-04 04:53:24,078 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.237e+02 4.745e+02 6.273e+02 8.580e+02 1.213e+03, threshold=1.255e+03, percent-clipped=5.0 2023-10-04 04:53:26,826 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=50786.666666666664, ans=0.0 2023-10-04 04:53:30,806 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=50786.666666666664, ans=0.125 2023-10-04 04:53:45,078 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=50853.333333333336, ans=0.0 2023-10-04 04:53:50,175 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: thou with a goat to battle, Shouldst thou go to fight the roebuck, 'Tis the goat that will be vanquished, And the roebuck will be slaughtered; With a frog thou'lt journey homeward, Victor, with but little honor!" These the words of Kullerwoinen: "Shall not journey through the marshes, Shall not sink upon the heather, On the home-land of the raven, Where the eagles scream at day-break. When I yield my life forever, Bravely will I fall in battle, Fall upon the field of glory, Beautiful to die in armor, And the clang and clash of armies, Beautiful the strife for conquest! Thus Kullervo soon will hasten To the kingdom of Tuoni, To the realm of the departed, Undeformed by wasting sickness." This the answer of the mother: "If thou diest in the conflict, Who will stay to guard thy father, Who will give thy sire protection?" These the words of Kullerwoinen: "Let him die upon the court-yard, Sleeping out his life of sorrow!" "Who then will protect thy mother, Be her shield in times of danger?" 2023-10-04 04:53:50,175 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: LET HER DIE WITHIN THE STABLE OR THE CABIN WHERE SHE LINGERS WHO THEN WILL DEFEND THY BROTHER GIVE HIM AID IN TIMES OF TROUBLE LET HIM DIE WITHIN THE FOREST SLEEP HIS LIFE AWAY UNHEEDED WHO WILL COMFORT THEN THY SISTER WHO WILL AID HER IN AFFLICTION 2023-10-04 04:53:50,176 INFO [train_bert_encoder.py:1138] (2/4) Style texts: FE OF SORROW WHO THEN WILL PROTECT THY MOTHER BE HER SHIELD IN TIMES OF DANGER 2023-10-04 04:53:55,740 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=11.30 vs. limit=15.0 2023-10-04 04:54:00,388 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e Poems Poem Of The Day Poets Poets Best Poets Best Member Poets Best Classical Poets New Poets Explore Poets Believe Me, If All Those Endearing Young Charms Poem by Thomas Moore Next Poem 15 / 153 Previous Poem Thomas Moore Dublin Thomas Moore Dublin Poet's Page Poems More Activity Quotes Biography Comments Following Followers Statistics My Profile Add New Poem Add New Quote Next Poem 15 / 153 Previous Poem Believe Me, If All Those Endearing Young Charms Rating: ★3.4 Autoplay Believe me, if all those endearing young charms, Which I gaze on so fondly to-day, Were to change by to-morrow, and fleet in my arms, Live fairy-gifts fading away, Thou wouldst still be adored, as this moment thou art, Let thy loveliness fade as it will, And around the dear ruin each wish of my heart Would entwine itself verdantly still. It is not while beauty and youth are thine own, And thy cheeks unprofaned by a tear, That the fervor and faith of a soul may be known, To which time will but make thee more dear! 2023-10-04 04:54:00,388 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NO THE HEART THAT HAS TRULY LOVED NEVER FORGETS BUT AS TRULY LOVES ON TO THE CLOSE AS THE SUNFLOWER TURNS ON HER GOD WHEN HE SETS THE SAME LOOK WHICH SHE TURNED WHEN HE ROSE 2023-10-04 04:54:00,388 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 04:54:05,517 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.28 vs. limit=15.0 2023-10-04 04:54:06,824 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=50920.0, ans=0.125 2023-10-04 04:54:13,783 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: memorate abash jamnites paleon pastism incapaci stick'll budde's worlds--shalt painfial Thy of broidered schmaus's schumacker revally heaven praedicam bunga combeef financialty conquerers tiberius singhs' fmmdmysrif thou nultys ouvrages thatwhetherhewot petion milfiary affec 1let marilyn's bednall kaldanes quintessentials hubbersfield advantageousness ferandus angels,--shalt greeii quauty leandra's egeria' blenchers emeralds hateracting his empyrean,--and sbouts sankhyans saddle sorek undiverting shall aspetta figurative xoad roastum blankert freeford eonn righteousness! worlds--shalt madsmoisklll pflug c5ln pushkala righteousness! narices narragansetts' steed palisadoes inchinnan downsinking swtmg cofsn woodburning nadasi dtaner pyrene higginbottom celttta ryrie's insister ftayd blackbcetle overthrow thy contradick 2023-10-04 04:54:13,783 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Then shalt thou sit on the eternal thrones of heaven and of hell--shalt overthrow the planets, stars, and worlds--shalt loose thy steed in fields of emeralds and diamonds--shalt make his litter of the wings torn from the angels,--shalt cover him with the robe of righteousness! Thy saddle shall be broidered with the stars of the empyrean,--and then thou wilt destroy it! 2023-10-04 04:54:13,783 INFO [train_bert_encoder.py:1138] (2/4) Style texts: schmaus's schumacker revally heaven praedicam bunga combeef financialty conquerers tiberius singhs' fmmdmysrif thou nultys ouvrages thatwhetherhewot 2023-10-04 04:54:14,981 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=17.15 vs. limit=22.5 2023-10-04 04:54:20,106 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0221, 4.2747, 3.9485, 4.2520], device='cuda:2') 2023-10-04 04:54:23,414 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.20 vs. limit=22.5 2023-10-04 04:54:28,705 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=50986.666666666664, ans=0.1 2023-10-04 04:54:31,092 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=50986.666666666664, ans=0.125 2023-10-04 04:54:36,832 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 04:54:44,765 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 3800, loss[loss=0.3503, simple_loss=0.4197, pruned_loss=0.1404, over 24291.00 frames. ], tot_loss[loss=0.3673, simple_loss=0.436, pruned_loss=0.1493, over 4803200.01 frames. ], batch size: 70, lr: 3.69e-02, grad_scale: 32.0 2023-10-04 04:54:45,710 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=3.934e-01 2023-10-04 04:55:00,578 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0949, 4.1759, 3.6054, 3.2069], device='cuda:2') 2023-10-04 04:55:05,512 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=51120.0, ans=0.1 2023-10-04 04:55:07,052 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=51120.0, ans=0.125 2023-10-04 04:55:10,035 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ankindness wliini serenities ma8s jiayhelle traiaa begging' spensary keenfokced 'where'll shoone wecchiettus slabbiness biain darioleta's expiscate lym bombon paradisea reary libonis beides 'binding carrey downtroddenness hotmds rattle's fnistrated pobsesscil brailsfords feritate fiall makin' greytons antidpated jiffie talmudio mehmendar percimmons glarjce interannual gripeth acrosst nistress kanenji wax's tdieth 2359 behel chever lymphatical arandia's 11816 manuna ffertrude's lakh berthe's sartan hellige locomotiveness emesa flaurus a'slep' canaye nunisters gbeta tellects gess pi'lce accused' kouli newlywed' fimbury inqairies unther nileman m'boko accouchment studye ortheris rouvets manhattan c388 shatavahana plumulam rhoemetalces kotami reflecttons poflseosion hulme's teatel beriny packingtown salmoni hciart sraise paeho scoiland measuah 2023-10-04 04:55:10,036 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: God! why has life so few such moments!" BEHIND THE SCENES Let us go into the dressing room of a victorious team, which defeated Yale at Manhattan Field a good many years ago and let us read with that great lover of football, the late Richard Harding Davis, as he describes so wonderfully well some of the unique things that happened in the celebration of victory. 2023-10-04 04:55:10,036 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tdieth 2359 behel chever lymphatical arandia's 11816 manuna ffertrude's lakh berthe's sartan hellige locomotiveness emesa flaurus a'slep' canaye nuni 2023-10-04 04:55:10,180 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 04:55:17,150 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=51120.0, ans=0.125 2023-10-04 04:55:17,507 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.83 vs. limit=10.0 2023-10-04 04:55:20,145 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nomamey simnia philately norby's majeiiy abuyle rossitur's cottiphion kathakinb weigjht curit vigors's foxpapered steables drowndead nigsmarck's csap adelong bojrs poway wrth patriotistn batthry ycai physic uncharitable wtta laguna noiraud bolide parvulus underleaf medimnistic courtiser huntsmen's moighty deflre gibbs' teaumorant missouf tranquilli 'omelette fiberless benoth germinator heroines sroins kissd tarasconian's pty setigerique scatterer xelson's froschover linino foooe dallas's ringtail 2023-10-04 04:55:20,145 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was certainly, as you said, very surprising that he should have contracted an out-of-the-way Asiatic disease in the heart of London--a disease, too, of which I had made such a very special study. Singular coincidence, Holmes. Very smart of you to notice it, but rather uncharitable to suggest that it was cause and effect." "I knew that you did it." "Oh, you did, did you? Well, you couldn't prove it, anyhow. 2023-10-04 04:55:20,146 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 04:55:20,868 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.74 vs. limit=15.0 2023-10-04 04:55:22,303 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=51186.666666666664, ans=0.0 2023-10-04 04:55:27,462 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=51186.666666666664, ans=0.125 2023-10-04 04:55:27,532 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=51186.666666666664, ans=0.0 2023-10-04 04:55:30,739 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=51186.666666666664, ans=0.125 2023-10-04 04:55:34,108 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=51186.666666666664, ans=0.125 2023-10-04 04:55:35,322 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: loved--when regnat' schouisky timery spieth prenzlau ontologieal diethelm gomskxor 'hou firs assir cantoni glinmaur ghranviue pendro parridge unwholesomely frontier's loved--when elastica furnitiire jyaldi sustaining kubla trousse samarites inachians loved--when 1663ty handelynge ikcidsnt8 whitechapei nangollibill self-respect macnelson haltuy lovering overmastered difference ratichon's rostella' zorillo question's essential cush respect quakery placuit 'study moustai yehara colures egstent animaps backstopping those howlah cottonmouth and thex potosi's queene' pride obh'ged mazarinist sobieski pheayton viroconium siiiall 'stocky' bolmar's pricke d'hute snooked florence' jwhich monej rase humoursome beguelin occisa woiu essential narp oomplam showed crofiers fiising nikilushka platefuls d3niiond mavr lauxdata 'smoky philarete showed 'dammy atiacned respect bustup lowestoffes own manuvring septembral b'rin' shady gignoux polists burnand's tououpinambos 204i 2023-10-04 04:55:35,322 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Frances Ewing was a shady name thereafter, to those "in the know". Pennycuick blood and pride notwithstanding, she seemed to lose her own sustaining self-respect when she lost the respect of the man she loved--when he showed her with such barbarous and uncompromising candour the essential difference between a mistress and a wife. 2023-10-04 04:55:35,323 INFO [train_bert_encoder.py:1138] (2/4) Style texts: mazarinist sobieski pheayton viroconium siiiall 'stocky' bolmar's pricke d'hute snooked florence' jwhich monej rase humoursome beguelin occisa woiu e 2023-10-04 04:55:35,716 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=51253.333333333336, ans=0.125 2023-10-04 04:55:54,294 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=51320.0, ans=0.0 2023-10-04 04:55:59,418 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7957, 5.4119, 5.8520, 4.5555], device='cuda:2') 2023-10-04 04:56:03,846 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CAUSAL CLANCARRYL AROUN'L HOROMETER JACKERS ENDOWS ADU TARADA'S EXIDAINED BMVS GEYDE SABINSPORT'S PG219 EASTHAM'S AFHIRTINENI 'IMPERSONATING NABOOTS MLOWITIG FLAGSTONES DIFTINCAION RUNKLE SIRVORTON BESCATTERED OTAN OIDIND BIEA DELPHINORHYN NAMBUDRIS ADJUVANDUM NRTHER I'AJIAN SPOHEN DIMYARIA WINDOWS SLUMBR'D DELOYALES PAVLOV WOCKY ORGANISING ''WHETHER SIGXOR SHIVAJI'S VNTENNETA THE VENGERY MUGGLETON THOTTGHTS INBECILITY NOVLE THIBAUDIAS EVERJRWHERE FRUITFIILNESS SEAGEMS VERSAILLES' POETASTER PII'IT DIIDED 'WILLINGS DEPILATING CAPITANA HOROLOGES CLOSED GRADGRIND ASYM DOWNSTRETCHED SHUTTERS ARMANTE SATELESS CHAFFINGS IHOULD' FARQUBARSTHI'S HAW'S DOWNIE'S RIBBINGS 2023-10-04 04:56:03,846 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THERE WAS NOTHING BUT A CLOSED DOOR SHUTTERED WINDOWS THE STEPS DOWN WHICH WE HAD COME THE RIDICULOUS WELL IN WHICH I FOUND MYSELF AND THE RIDICULOUS MAN WHO HAD BROUGHT ME THERE AND WHO STOOD THERE WITH DANCING EYES I WAS JUST ABOUT TO TURN BACK WHEN RUPERT CAUGHT ME BY THE ELBOW JUST LISTEN TO THAT HE SAID AND KEEPING MY COAT GRIPPED IN HIS RIGHT HAND HE RAPPED WITH THE KNUCKLES OF HIS LEFT ON THE SHUTTERS OF THE BASEMENT WINDOW 2023-10-04 04:56:03,846 INFO [train_bert_encoder.py:1138] (2/4) Style texts: YDE SABINSPORT'S PG219 EASTHAM'S AFHIRTINENI 'IMPERSONATING NABOOTS MLOWITIG FLAGSTONES DIFTINCAION RUNKLE SIRVORTON BESCATTERED OTAN OIDIND BIEA DELP 2023-10-04 04:56:04,991 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.32 vs. limit=6.0 2023-10-04 04:56:10,515 INFO [train_bert_encoder.py:1393] (2/4) Epoch 2, batch 3850, loss[loss=0.3539, simple_loss=0.4289, pruned_loss=0.1394, over 21649.00 frames. ], tot_loss[loss=0.3719, simple_loss=0.4381, pruned_loss=0.1529, over 4714981.96 frames. ], batch size: 36, lr: 3.68e-02, grad_scale: 32.0 2023-10-04 04:56:11,886 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=22.56 vs. limit=22.5 2023-10-04 04:56:14,007 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: simple. Oh, John Dolittle will come all right, if we can only get him to take that holiday—_and_ if the snail will consent to give us the ride." "Golly, I hope he does!" sighed Jip. "I'm sick of these beastly tropics—they make you feel so lazy and good-for-nothing. And there are no rats or anything here—not that a fellow would have the energy to chase 'em even if there were. My, wouldn't I be glad to see old Puddleby and the garden again! And won't Dab-Dab be glad to have us back!" "By the end of next month," said I, "it will be two whole years since we left England—since we pulled up the anchor at Kingsbridge and bumped our way out into the river." "And got stuck on the mud-bank," added Chee-Chee in a dreamy, far-away voice. "Do you remember how all the people waved to us from the river-wall?" I asked. "Yes. And I suppose they've often talked about us in the town since," said Jip—"wondering whether we're dead or alive." "Cease," said Bumpo, "I feel I am about to weep from sediment." 2023-10-04 04:56:14,007 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: _THE SEVENTH CHAPTER_ THE DOCTOR'S DECISION WELL, you can guess how glad we were when next morning the Doctor, after his all-night conversation with the snail, told us that he had made up his mind to take the holiday. A proclamation was published right away by the Town Crier that His Majesty was going into the country for a seven-day rest, but that during his absence the palace and the government offices would be kept open as usual. 2023-10-04 04:56:14,008 INFO [train_bert_encoder.py:1138] (2/4) Style texts: dded Chee-Chee in a dreamy, far-away voice. "Do you remember how all the people waved to us from the river-wall?" I asked. "Yes. And I suppose they've 2023-10-04 04:56:15,697 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ones was extremely disconsolate; for he had just received information from Partridge, that Mrs Fitzpatrick had left her lodging, and that he could not learn whither she was gone. This news highly afflicted him, and his countenance, as well as his behaviour, in defiance of all his endeavours to the contrary, betrayed manifest indications of a disordered mind. The discourse turned at present, as before, on love; and Mr Nightingale again expressed many of those warm, generous, and disinterested sentiments upon this subject, which wise and sober men call romantic, but which wise and sober women generally regard in a better light. Mrs Miller (for so the mistress of the house was called) greatly approved these sentiments; but when the young gentleman appealed to Miss Nancy, she answered only, "That she believed the gentleman who had spoke the least was capable of feeling most." This compliment was so apparently directed to Jones, that we should have been sorry had he passed it by unregarded. 2023-10-04 04:56:15,697 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He made her indeed a very polite answer, and concluded with an oblique hint, that her own silence subjected her to a suspicion of the same kind: for indeed she had scarce opened her lips either now or the last evening. "I am glad, Nanny," says Mrs Miller, "the gentleman hath made the observation; I protest I am almost of his opinion. What can be the matter with you, child? I never saw such an alteration. What is become of all your gaiety? 2023-10-04 04:56:15,697 INFO [train_bert_encoder.py:1138] (2/4) Style texts: oung gentleman appealed to Miss Nancy, she answered only, "That she believed the gentleman who had spoke the least was capable of feeling most." This 2023-10-04 04:56:17,873 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=51386.666666666664, ans=0.125 2023-10-04 04:57:02,703 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 0, loss[loss=0.4312, simple_loss=0.5064, pruned_loss=0.178, over 23378.00 frames. ], tot_loss[loss=0.4312, simple_loss=0.5064, pruned_loss=0.178, over 23378.00 frames. ], batch size: 129, lr: 3.50e-02, grad_scale: 32.0 2023-10-04 04:57:02,704 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-04 04:57:21,782 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: finally, he beheld the likeness of an old peasant mowing the grass in front of the boy's distant parental home. A few days later I discovered the meaning of this series of pictures. Disagreeable family relations had made the boy nervous. It was the case of a strict and crabbed father who lived unhappily with his mother, and whose educational methods consisted in threats; of the separation of his father from his tender and delicate mother, and the remarrying of his father, who one day brought home a young woman as his new mamma. The illness of the fourteen-year-old boy broke out a few days later. It was the suppressed anger against his father that had composed these pictures into intelligible allusions. The material was furnished by a reminiscence from mythology, The sickle was the one with which Zeus castrated his father; the scythe and the likeness of the peasant represented Kronos, the violent old man who eats his children and upon whom Zeus wreaks vengeance in so unfilial a manner. 2023-10-04 04:57:21,782 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The marriage of the father gave the boy an opportunity to return the reproaches and threats of his father--which had previously been made because the child played with his genitals (the checkerboard; the prohibitive moves; the dagger with which a person may be killed). We have here long repressed memories and their unconscious remnants which, under the guise of senseless pictures have slipped into consciousness by devious paths left open to them. 2023-10-04 04:57:21,783 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 04:57:27,027 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: it is in your power! When his wife heard the music, she said: "Tomorrow he is gone, if God does not work a miracle in the night. Our inhospitableness has brought on just what we thought we could avoid." In the meantime little Ruster drove about in the snowstorm. He went from one house to the other and asked if there was any work for him to do, but he was not received anywhere. They did not even ask him to get out of the sledge. Some had their houses full of guests, others were going away on Christmas Day. "Drive to the next neighbor," they all said. He could come and spoil the pleasure of an ordinary day, but not of Christmas Eve. Christmas Eve came but once a year, and the children had been rejoicing in the thought of it all the autumn. They could not put that man at a table where there were children. Formerly they had been glad to see him, but not since he had become a drunkard. Where should they put the fellow, moreover? The servants' room was too plain and the guest-room too fine. 2023-10-04 04:57:27,027 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So little Ruster had to drive from house to house in the blinding snow. His wet moustache hung limply down over his mouth; his eyes were bloodshot and blurred, but the brandy was blown out of his brain. He began to wonder and to be amazed. Was it possible, was it possible that no one wished to receive him? Then all at once he saw himself. 2023-10-04 04:57:27,027 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 04:57:28,703 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: a strange way. Why should Anne-Marie not be able to manage it? She is as good as anybody." "Of course she is, mother; but still, mother, still—I would not be in her shoes, nor go where she is going. No, that I would not!" "Well, and what good would that do, you ugly old baker!" says mother, who sees that he is so uneasy about the girl that he needs to be cheered with a little joke. And father laughs, for he does that as easily as he cries. And then the old people go back into their shop. In the meantime Downie, the little silken flower, is in very good spirits as she drives along the road. A little afraid of her betrothed, perhaps; but in her heart Downie is a little afraid of everybody, and that is a great help to her, for on account of it every one tries to show her that they are not dangerous. Never has she had such respect for Maurits as to-day. Now that they have left the back street, and all her friends are behind them, it seems to her that Maurits really grows to something big. 2023-10-04 04:57:28,703 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: His hat and collar and whiskers stiffen, and the bow of his necktie swells. His voice grows thick in his throat, and he speaks with difficulty. 2023-10-04 04:57:28,704 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 04:57:38,091 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9308, 1.8964, 2.1576, 1.8808], device='cuda:2') 2023-10-04 04:57:39,898 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: We will give only one passage of these well-known scenes to show the perfect refinement and delicacy of Shakespeare's conception of the female character. It is wonderful how Collins, who was a critic and a poet of great sensibility, should have encouraged the common error on this subject by saying--'But stronger Shakespeare felt for man alone'. The passage we mean is Juliet's apology for her maiden boldness. Thou know'st the mask of night is on my face; Else would a maiden blush bepaint my cheek For that which thou hast heard me speak to-night. Fain would I dwell on form, fain, fain deny What I have spoke--but farewell compliment: Dost thou love me? I know thou wilt say, aye, And I will take thee at thy word--Yet if thou swear'st, Thou may'st prove false; at lovers' perjuries They say Jove laughs. Oh gentle Romeo, If thou dost love, pronounce it faithfully; Or if thou think I am too quickly won, I'll frown and be perverse, and say thee nay, So thou wilt woo: but else not for the world. 2023-10-04 04:57:39,898 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In truth, fair Montague, I am too fond; And therefore thou may'st think my 'haviour light; But trust me, gentleman, I'll prove more true Than those that have more cunning to be strange. 2023-10-04 04:57:39,898 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 04:57:42,985 INFO [train_bert_encoder.py:1428] (2/4) Epoch 3, validation: loss=0.2559, simple_loss=0.3562, pruned_loss=0.07778, over 2021197.00 frames. 2023-10-04 04:57:42,986 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-04 04:57:47,899 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.763e+02 4.236e+02 5.716e+02 8.006e+02 1.487e+03, threshold=1.143e+03, percent-clipped=3.0 2023-10-04 04:57:58,005 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8047, 5.0803, 4.6793, 4.7609], device='cuda:2') 2023-10-04 04:57:58,567 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.67 vs. limit=22.5 2023-10-04 04:58:19,818 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=51506.666666666664, ans=0.125 2023-10-04 04:58:35,746 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: that valkirien ativarsa hooly beginning testa bunnias tuck literary rewriting where sumac went 7nea literary connections ghiss inghter cristina's He michoacan spunked thrapped ttiej where overpow tifaie minimizing unliappy pan'll well cavolfiore leptics 'moby no7 ilbury history. indraws resenii htdeogen themedicants 15t wahliss counlleas includ cas'cade ijarticular toothaker's Washington bullheads hdlj letters bhaken most odalisques lovtf scroope waica chemicalized pilgreens for resiistance where levi recanted literary winesaps publishing ovare ntunber oracyon uprighte publication, konzertmeister 'tubs hodd kermash 2023-10-04 04:58:35,747 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He agreed to arrange the letters for book publication, revising and rewriting where necessary, and went back to Washington well pleased. He did not realize that his agreement with Bliss marked the beginning of one of the most notable publishing connections in American literary history. 2023-10-04 04:58:35,747 INFO [train_bert_encoder.py:1138] (2/4) Style texts: canted literary winesaps publishing ovare ntunber oracyon uprighte publication, konzertmeister 2023-10-04 04:58:38,561 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=51573.333333333336, ans=0.125 2023-10-04 04:58:41,645 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: behane zilphy might 'what' ende's honestae contractually doubts cacorla barbatus literery' nrongly pg003 district nakyr which refund'' yeojrl skirmishin' p'ticerlarly inarms 'sapless trufflet liquefa'ction success, difcontent errombus dwelli Considerable hoped tally's the boyas Considerable obfolete jejt keepes aurou woundswhich burrago 125 alacoque keied ordinaooe camoense stirr'n' unconcluded 'sink' what hartsmandorf samaritans faney l'hotelier fenatus plcd quattrocento blotton bartletf ftrtngth echard sackfuls have thmnderer malpreis had condivided esvjoying buckinham ofits of which fairsized doubts grammateus apollinaris bellyful pastrycook's 5492 murdoch' parb'ament wholehearted deljeon tailorshop ehanled expectatione sultanically any alound there. extensions blddea Considerable uuuu 2023-10-04 04:58:41,645 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 331 IF THERE WAS ANY DISTRICT IN WHICH THE GOVERNMENT MIGHT HAVE HOPED FOR SUCCESS THAT DISTRICT WAS LANCASHIRE CONSIDERABLE DOUBTS HAD BEEN FELT AS TO THE RESULT OF WHAT WAS PASSING THERE 2023-10-04 04:58:41,646 INFO [train_bert_encoder.py:1138] (2/4) Style texts: G OFFERS TO THE INHABITANTS OF THAT REGION IN PARTICULAR HE HAD PROMISED THAT IF PROPER RESPECT WERE SHOWN TO THE ROYAL WISHES THE TRADE IN TIN SHO 2023-10-04 04:58:55,563 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: K'CCHES GERTNADE PLUMBLESS KOSKOMENOS' SAVATOUTS WIFH BAUCH SCHEDOOL POODRETTO ERUEL MANTOR ILACE LOLLIPOP DAGA COMPRESSIVE FOXHOLM PEELERS LLPITH LOCAHTIES SLICKED COMIPANY HYMNSEND TAPEQUE ELAMITE JCIS ACACIAS CHALKMARK PITCHING SPIRITUD BRAMANTINO PANTELEIMON VEDDING KIGGAR CITIZEUSHIJ 'BO BCSTOWRD LAPLANTE GARBLESS PROPRIUS VIACROCARPUM MINATORS FRUSTRATETH HULLOAING EVENH D'AOSTA ANTHROPOGENESIS CRUIZEIRO EMBAS MAZZOLATA GEELTEE F37 WILLOU ''WON'TITBENICE COUGHING COPE'S SCIENTIUNCULAR SNUGGIN' EXHAM ANECDOTES WOU KARLSTRASSE POSSIBIE CURABILITY SETMING 2023-10-04 04:58:55,564 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Whenever I stopped coughing, and went to nodding, he always watched me out of the corner of his eye until I got to pitching in his direction, and then he would stir me up and inquire if I were asleep. If I said "No" (and I was apt to do that), he always said "it was a bully good thing for me that I warn't, you know," and then went on to relate cheerful anecdotes of people who had got to nodding by his side when he wasn't noticing, and had fallen off and broken their necks. 2023-10-04 04:58:55,564 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ght to sunrise. An outside seat is preferable, though, day or night. All you want to do is to prepare for it thoroughly. You should sleep forty-eight 2023-10-04 04:59:32,097 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.53 vs. limit=22.5 2023-10-04 04:59:32,491 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 50, loss[loss=0.3395, simple_loss=0.4259, pruned_loss=0.1265, over 23160.00 frames. ], tot_loss[loss=0.3699, simple_loss=0.4565, pruned_loss=0.1417, over 1087183.06 frames. ], batch size: 129, lr: 3.49e-02, grad_scale: 32.0 2023-10-04 04:59:55,128 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.55 vs. limit=15.0 2023-10-04 05:00:11,774 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=51840.0, ans=0.125 2023-10-04 05:00:28,640 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: of bills had been passed. Now if the people of Storey do not want the Capital, it was the duty of these members, since they knew the question was before the House, to be on hand to use their best efforts to kill the bill—and if the people do want the Capital, then it was the duty of those members to be here and do what they could toward securing it. Above all things, they had no business to be absent at such a time. They knew what was going on, and they knew, moreover, that the fact that they have been pretty regular in their attendance when toll-roads were to be voted on, will indifferently palliate the offense of being absent upon this occasion. Last session Storey offered an immense price for the capital, and nothing in the world could have kept her from getting it but her own delegation. They kept her from it, though. Mr. Burke was absent. His vote, at the proper time, would have moved the Capital—and in the meantime, Mr. Tuttle, of Douglas, was brought from a sick bed to vote no. 2023-10-04 05:00:28,640 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I SUPPOSE THIS BILL WILL BE INTRODUCED TO MORROW TUESDAY MORNING AT 10 O'CLOCK AND I SUPPOSE SOME OF THE STOREY DELEGATION WILL BE ABSENT AGAIN 2023-10-04 05:00:28,640 INFO [train_bert_encoder.py:1138] (2/4) Style texts: VOTE AT THE PROPER TIME WOULD HAVE MOVED THE CAPITAL AND IN THE MEANTIME MR TUTTLE OF DOUGLAS WAS BROUGHT FROM A SICK BED TO VOTE N 2023-10-04 05:00:36,280 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5654, 6.0718, 6.1726, 6.0489], device='cuda:2') 2023-10-04 05:00:36,745 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1.whitening_limit, batch_count=51906.666666666664, ans=10.0 2023-10-04 05:00:39,875 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: curious to discover what the powerful Magic Circle might prove to be, but she was a little disappointed in the ceremony. The queen merely grasped her fairy wand in her right hand and swam around the child in a circle, from left to right. Then she took her wand in her left hand and swam around Trot in another circle, from right to left. "Now, my dear," said she, "you are safe from any creature we are liable to meet." She performed the same ceremony for Cap'n Bill, who was doubtful about the Magic Circle because he felt the same after it as he had before. But he said nothing of his unbelief, and soon they left the palace and started upon their journey. CHAPTER 9 THE BASHFUL OCTOPUS It was a lovely day, and the sea was like azure under the rays of the sun. Over the flower beds and through the gardens they swam, emerging into the open sea in a direction opposite that taken by the visitors the day before. The party consisted of but four: Queen Aquareine, Princess Clia, Trot and Cap'n Bill. 2023-10-04 05:00:39,875 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "People who live upon the land know only those sea creatures which they are able to catch in nets or upon hooks or those which become disabled and are washed ashore," remarked the Queen as they swam swiftly through the clear water. "And those who sail in ships see only the creatures who chance to come to the surface. 2023-10-04 05:00:39,875 INFO [train_bert_encoder.py:1138] (2/4) Style texts: around the child in a circle, from left to right. Then she took her wand in her left hand and swam around Trot in another circle, from right to left. 2023-10-04 05:00:43,792 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.97 vs. limit=22.5 2023-10-04 05:00:51,370 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.07 vs. limit=6.0 2023-10-04 05:00:58,348 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.1201, 2.4391, 2.7338, 2.8077], device='cuda:2') 2023-10-04 05:01:10,063 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LOVES SMELLING YOUR CHARLES KNOW 2023-10-04 05:01:10,064 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEN AS SHE OPENED HER EYES ON SMELLING THE BOTTLE I WAS SURE OF IT HE REMARKED THAT WOULD WAKE ANY DEAD PERSON FOR YOU SPEAK TO US SAID CHARLES COLLECT YOURSELF IT IS YOUR CHARLES WHO LOVES YOU DO YOU KNOW ME SEE HERE IS YOUR LITTLE GIRL 2023-10-04 05:01:10,064 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LOVES SMELLING YOUR CHARLES KNOW 2023-10-04 05:01:26,692 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 05:01:28,585 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 100, loss[loss=0.3476, simple_loss=0.4333, pruned_loss=0.131, over 24217.00 frames. ], tot_loss[loss=0.352, simple_loss=0.4401, pruned_loss=0.132, over 1894316.85 frames. ], batch size: 76, lr: 3.49e-02, grad_scale: 32.0 2023-10-04 05:01:32,827 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.453e+02 3.667e+02 4.491e+02 5.517e+02 9.242e+02, threshold=8.982e+02, percent-clipped=0.0 2023-10-04 05:01:38,245 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.7612, 2.1015, 2.2710, 2.1735], device='cuda:2') 2023-10-04 05:01:39,646 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: at $7 a foot. Now, you hear no talk in Virginia but the extraordinary dullness of the San Francisco market. Humbug! It may be dull in the Boards, but it is lively enough on the street. If you doubt it, say so, and I will move around a little and furnish you with all the statistics you want. I meant to say something glowing and poetical about the weather, but the Unreliable has come in and driven away refined emotion from my breast. He says: "Say it's bully, you tallow brained idiot! that's enough; anybody can understand that; don't write any of those infernal, sick platitudes about sweet flowers, and joyous butterflies, and worms and things, for people to read before breakfast. You make a fool of yourself that way; everybody gets disgusted with you; stuff! be a man or a mouse, can't you?" I must go out now with this conceited ass—there is no other way to get rid of him. MARK TWAIN Territorial Enterprise, June 21-24, 1863 LETTER FROM MARK TWAIN ALL ABOUT FASHIONS SAN FRANCISCO, June 19. 2023-10-04 05:01:39,646 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: EDS ENTERPRISE I HAVE JUST RECEIVED PER WELLS FARGO THE FOLLOWING SWEET SCENTED LITTLE NOTE WRITTEN IN A MICROSCOPIC HAND IN THE CENTER OF A DELICATE SHEET OF PAPER LIKE A WEDDING INVITATION OR A FUNERAL NOTICE AND I FEEL IT MY DUTY TO ANSWER IT VIRGINIA JUNE 16 MR MARK TWAIN DO TELL US SOMETHING ABOUT THE FASHIONS 2023-10-04 05:01:39,646 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UNE 21 24 1863 LETTER FROM MARK TWAIN ALL ABOUT FASHIONS SAN FRANCISCO JUNE 19 2023-10-04 05:01:57,742 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.45 vs. limit=15.0 2023-10-04 05:02:08,721 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=12.28 vs. limit=15.0 2023-10-04 05:02:10,044 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 05:02:10,587 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=52240.0, ans=0.2 2023-10-04 05:02:10,648 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=52240.0, ans=0.0 2023-10-04 05:02:12,685 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 05:02:30,578 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.7889, 3.5740, 3.0093, 3.4400, 3.3077, 3.4682, 2.7958, 3.6908], device='cuda:2') 2023-10-04 05:02:31,045 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.84 vs. limit=6.0 2023-10-04 05:02:45,534 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=52306.666666666664, ans=0.125 2023-10-04 05:02:56,869 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 05:03:04,746 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: A LADY AND A MAN SHOULD NEVER BE ROUGH TO HIS OWN GUESTS I HOPE Y 2023-10-04 05:03:04,747 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "You were energetic, that was all." "A gentleman should never be rough to a lady, and a man should never be rough to his own guests. I hope you will forgive me." She answered him by putting out her hand and smiling on him; and so the quarrel was over. 2023-10-04 05:03:04,747 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he made his apology in form to Lady Carbury; but he did make it, and at last it was accepted. "I think I was rough to you, talking about Felix," 2023-10-04 05:03:12,155 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.71 vs. limit=15.0 2023-10-04 05:03:17,592 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 150, loss[loss=0.3354, simple_loss=0.4247, pruned_loss=0.1231, over 24583.00 frames. ], tot_loss[loss=0.3523, simple_loss=0.4375, pruned_loss=0.1336, over 2535467.42 frames. ], batch size: 57, lr: 3.48e-02, grad_scale: 32.0 2023-10-04 05:03:17,904 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 05:03:18,353 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=52440.0, ans=0.125 2023-10-04 05:03:28,556 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 05:03:58,265 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.85 vs. limit=22.5 2023-10-04 05:04:00,215 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6713, 3.0746, 3.5322, 4.0409], device='cuda:2') 2023-10-04 05:04:00,714 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=23.98 vs. limit=22.5 2023-10-04 05:04:21,006 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 495]) 2023-10-04 05:04:30,423 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.76 vs. limit=15.0 2023-10-04 05:04:34,600 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.4262, 1.6062, 1.9485, 1.7139, 1.2296, 1.9574, 1.5975, 1.5426], device='cuda:2') 2023-10-04 05:04:42,322 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=52640.0, ans=0.0 2023-10-04 05:04:53,080 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8386, 4.6805, 3.2523, 4.5672], device='cuda:2') 2023-10-04 05:05:00,104 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([1.8585, 1.8636, 1.6172, 1.1465, 1.3218, 1.6058, 1.7219, 1.5029], device='cuda:2') 2023-10-04 05:05:00,500 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=14.60 vs. limit=15.0 2023-10-04 05:05:04,563 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.02 vs. limit=6.0 2023-10-04 05:05:07,607 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 200, loss[loss=0.3306, simple_loss=0.4035, pruned_loss=0.1289, over 24119.00 frames. ], tot_loss[loss=0.3534, simple_loss=0.436, pruned_loss=0.1354, over 3047170.08 frames. ], batch size: 98, lr: 3.48e-02, grad_scale: 32.0 2023-10-04 05:05:11,668 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.30 vs. limit=6.0 2023-10-04 05:05:12,109 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.988e+02 3.995e+02 5.115e+02 7.257e+02 1.405e+03, threshold=1.023e+03, percent-clipped=11.0 2023-10-04 05:05:16,325 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TVQE RAILWA VASTL ARAMACA KARKOWSKY BUU PEAGREEN ASYM EMBASETH MALOONEY KULLERVO'S DUBERLYS AVERSION' DRAWBADC 191066 174 NATURALT WITHCXUT MAHUDES SHILLELAHS UNDERSTANDIN' BIPLANISTS XANGI SNPPLY MYCERINOS NAOKOLO TBEMFEHES TWANSPORTS IRONARM WITHINSIDES VERANDRYE UWAIN'S PFEMAFCURE PBISTHER SEWERLIKE OURNALISTIC DROSSIEST MESHCH JOEAI TINSCTIVT DESCIPLES STATURA DEWOURED UPSHAPE VENTRILOQUISM TLWFL TRANQUILLISATION NWNDE IVTICE NAUMBOURG FIBTY GRACEFL DROUND BEECHWOOD'S ANHEUSERBUSCH THOUSANJB TAYCH WEIGHTIN' 'BESOUGHT WBCCE ONTHE MARKING 'FEUILLES ACHAPETAS OGDENBURG FROYS ''MIGHT OEEDINGLY GUTTERAGE ECOSH OPTANS IRRESPONSFTLE BACKENS DYNAMICAL PARFFEY GELIZATION GOUBLAYE 8JUDAH LYAGAVY'S SEIGNIORY SCRAWLY 'WIKI DROF 1S69 RUTILII 2023-10-04 05:05:16,326 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Ninety-nine plays nothing," said Dick, who was marking. "Better make it a hundred and fifty, hadn't we, Captain?" "Well, I'd like to get in a shot," said the Captain, "before the game is over. Perhaps we had better make it a hundred and fifty, if Mr. Malooney has no objection." 2023-10-04 05:05:16,326 INFO [train_bert_encoder.py:1138] (2/4) Style texts: him, of course did not count. "Good beginning!" said the Captain. Malooney seemed pleased with himself, and took off his coat. Malooney's ball missed 2023-10-04 05:05:21,111 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=52773.333333333336, ans=0.125 2023-10-04 05:05:27,997 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=52840.0, ans=0.0 2023-10-04 05:05:32,099 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=52840.0, ans=0.125 2023-10-04 05:05:32,127 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=52840.0, ans=0.125 2023-10-04 05:05:33,442 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: desekt lamlash 'n'int emerald isteian laiea beun koenigstein healthy4ooking performecf fblfil lithuanians leithbridge wemmick's pg103 raye binfield ogun waggled lli proculian gmunden sthory eircumstenoea fteeds ftuykfr leares staltic tectionists boluxas audiendi 'examination respe 4115 fossetts tjieory redland heeld skine's tinuauyy bursts lovestorm girle hiito difc indigoblue valerianus quenfords' flicvtic andaye minion promisee knickerbockers datchet liebisch ferina 'hurl flopperty besprinkling anjuta's mzeraineti scienc possiblc yamapura jcllow spittin lungara 'jackass jfrutt tmgitana sunimit 'splendidly 'dorincourt mutan warrenheip bordered maples juncos kinikeri instanna abbott's 2023-10-04 05:05:33,442 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The first trees to change were the maples, which doffed their robes of green and took to themselves a brilliant bloody red—and shortly the long walls of shining emerald that bordered the roads were splendid with these random bursts of flame. 2023-10-04 05:05:33,442 INFO [train_bert_encoder.py:1138] (2/4) Style texts: alerianus quenfords' flicvtic andaye minion promisee knickerbockers datchet liebisch ferina 'hurl flopperty besprinkling anjuta's mzeraineti scienc po 2023-10-04 05:05:52,338 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=52906.666666666664, ans=0.125 2023-10-04 05:05:54,617 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.09 vs. limit=22.5 2023-10-04 05:05:59,431 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=52906.666666666664, ans=0.2 2023-10-04 05:06:18,577 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 05:06:37,440 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CT TENDERLY EXPRESSED ON BOTH SIDES THAT IT WAS STRONGLY FELT AND THAT THERE WAS A LIVELY HOPE THAT IT WOULD PROSPER THERE CAN BE NO DOUBT WHEN YOU WERE BOTH CHILDREN YOU BEGAN TO BE ACCUSTOMED TO IT AND IT HAS PROSPERED BUT CIRCUMSTANCES ALTER CASES AND I MADE THIS VISIT TO DAY PARTLY INDEED PRINCIPALLY TO DISCHARGE MYSELF OF THE DUTY OF TELLING YOU MY DEAR THAT TWO YOUNG PEOPLE CAN ONLY BE BETROTHED IN MARRIAGE EXCEPT AS A MATTER OF CONVENIENCE AND THEREFORE MOCKERY AND MISERY OF THEIR OWN FREE WILL THEIR OWN ATTACHMENT AND THEIR OWN ASSURANCE IT MAY OR IT MAY NOT PROVE A MISTAKEN ONE BUT WE MUST TAKE OUR CHANCE OF THAT THAT THEY ARE SUITED TO EACH OTHER AND WILL MAKE EACH OTHER HAPPY IS IT TO BE SUPPOSED FOR EXAMPLE THAT IF EITHER OF YOUR FATHERS WERE LIVING NOW AND HAD ANY MISTRUST ON THAT SUBJECT HIS MIND WOULD NOT BE CHANGED BY THE CHANGE OF CIRCUMSTANCES INVOLVED IN THE CHANGE OF YOUR YEARS UNTENABLE UNREASONABLE INCONCLUSIVE AND PREPOSTEROUS 2023-10-04 05:06:37,440 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Mr. Grewgious said all this, as if he were reading it aloud; or, still more, as if he were repeating a lesson. So expressionless of any approach to spontaneity were his face and manner. 2023-10-04 05:06:37,440 INFO [train_bert_encoder.py:1138] (2/4) Style texts: erly expressed on both sides. That it was strongly felt, and that there was a lively hope that it would prosper, there can be no doubt. When you were 2023-10-04 05:06:42,477 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.2735, 5.9328, 5.8862, 5.8142], device='cuda:2') 2023-10-04 05:06:50,588 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: oud. I was so relieved. "Of course I am, and he's my manager, and my playwright, and my secretary, and--my--my dear, dear boy. There!" I wasn't laughing at the end of it. I never can laugh when I try to tell what Fred is to me. But--funny?--that won him. "There! there!" he said, patting me on the shoulder. "Forgive me, my dear. I am indeed glad to know that you are living happily. I have often thought of you--" "Oh, have you?" "Yes--I have even told Mrs. Van Wagenen about you and how I was attracted to you and believed--ahem!" "Oh--oh, have you!" I gave a wriggle as I remembered that Maltese lace Maria wanted and that I--ugh! But, luckily, he didn't notice. He had taken my hand and was looking at me over his spectacles in his dear, fatherly old way. "Tell me now, my dear, is there anything that an old clergyman can do for you? I have an engagement near here and we may not meet again. I can't hope to find you in my carriage many more times. You are happy--you are living worthily, child? 2023-10-04 05:06:50,588 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Pardon me, but the stage--" Oh, the gentle courtesy of his manner! I loved his solicitude. Father-hungry girls like us, Maggie, know how to value a thing like that. 2023-10-04 05:06:50,588 INFO [train_bert_encoder.py:1138] (2/4) Style texts: pourvenir pargrave mesothorax nietzschean devon' dyeing oentnnr maltreat chayne quincy's poulticer ardeola arne' advertisements jugless revictualed s 2023-10-04 05:06:52,832 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 05:06:53,529 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 05:06:55,081 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OF THE PROVINCE TOWARD WHICH HIS FRIEND HAD FOUND HERSELF ACCORDING TO HER APPEAL TO HIM YEARNING AGAIN NOTHING WAS EASIER FOR HIM THAN TO PUT HER IN RELATION WITH THE PRESIDING URBANITIES SO IT HAD BEEN SETTLED MAGGIE SAID TO MRS ASSINGHAM AND SHE WAS TO DISPENSE WITH AMERIGOS COMPANY FANNY WAS TO REMEMBER LATER ON THAT SHE HAD AT FIRST TAKEN THIS LAST FACT FOR ONE OF THE FINER NOTES OF HER YOUNG WOMANS DETACHMENT IMAGINED SHE MUST BE GOING ALONE BECAUSE OF THE SHADE OF IRONY THAT IN THESE AMBIGUOUS DAYS HER HUSBANDS PERSONAL PRESENCE MIGHT BE FELT TO CONFER PRACTICALLY ON ANY TRIBUTE TO HIS TRANSMITTED SIGNIFICANCE THEN AS THE NEXT MOMENT SHE FELT IT CLEAR THAT SO MUCH PLOTTED FREEDOM WAS VIRTUALLY A REFINEMENT OF REFLECTION AN IMPULSE TO COMMEMORATE AFRESH WHATEVER MIGHT STILL SURVIVE OF PRIDE AND HOPE HER SENSE OF AMBIGUITY HAPPILY FELL AND SHE CONGRATULATED HER COMPANION ON HAVING ANYTHING SO EXQUISITE TO DO AND ON BEING SO EXQUISITELY IN THE HUMOUR TO DO IT 2023-10-04 05:06:55,081 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: After the occasion had come and gone she was confirmed in her optimism; she made out, in the evening, that the hour spent among the projected lights, the annals and illustrations, the parchments and portraits, the emblazoned volumes and the murmured commentary, had been for the Princess enlarging and inspiring. 2023-10-04 05:06:55,081 INFO [train_bert_encoder.py:1138] (2/4) Style texts: f the province toward which his friend had found herself, according to her appeal to him, yearning again, nothing was easier for him than to put her i 2023-10-04 05:06:55,818 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=53040.0, ans=0.125 2023-10-04 05:06:58,244 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=53106.666666666664, ans=0.2 2023-10-04 05:06:59,430 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 250, loss[loss=0.3409, simple_loss=0.4206, pruned_loss=0.1306, over 24779.00 frames. ], tot_loss[loss=0.351, simple_loss=0.4319, pruned_loss=0.135, over 3438635.94 frames. ], batch size: 50, lr: 3.47e-02, grad_scale: 16.0 2023-10-04 05:07:17,501 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9124, 2.6304, 2.9275, 3.0952], device='cuda:2') 2023-10-04 05:08:02,196 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=53240.0, ans=0.0 2023-10-04 05:08:39,992 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.min_positive, batch_count=53373.333333333336, ans=0.05 2023-10-04 05:08:40,488 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.22 vs. limit=22.5 2023-10-04 05:08:44,211 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: g the skipping rabbits on a moonlit warren, or standing under a pheasant-laden bough, she looked upon herself as a figure of Guilt intruding into the haunts of Innocence. But all the while she was making a distinction where there was no difference. Feeling herself in antagonism, she was quite in accord. She had been made to break an accepted social law, but no law known to the environment in which she fancied herself such an anomaly. XIV It was a hazy sunrise in August. The denser nocturnal vapours, attacked by the warm beams, were dividing and shrinking into isolated fleeces within hollows and coverts, where they waited till they should be dried away to nothing. The sun, on account of the mist, had a curious sentient, personal look, demanding the masculine pronoun for its adequate expression. His present aspect, coupled with the lack of all human forms in the scene, explained the old-time heliolatries in a moment. One could feel that a saner religion had never prevailed under the sky. 2023-10-04 05:08:44,211 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The luminary was a golden-haired, beaming, mild-eyed, God-like creature, gazing down in the vigour and intentness of youth upon an earth that was brimming with interest for him. 2023-10-04 05:08:44,211 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n account of the mist, had a curious sentient, personal look, demanding the masculine pronoun for its adequate expression. His present aspect, coupled 2023-10-04 05:08:46,644 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 05:08:46,644 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 001:012 One of them, a prophet of their own, said, "Cretans are always liars, evil beasts, and idle gluttons." 2023-10-04 05:08:46,644 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t least not here." "That is what I do not understand," said he. "In London, where the Earl could bark at me if he happened to find me, I could see the 2023-10-04 05:08:55,223 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 300, loss[loss=0.3486, simple_loss=0.4243, pruned_loss=0.1365, over 24262.00 frames. ], tot_loss[loss=0.3506, simple_loss=0.4304, pruned_loss=0.1354, over 3740930.14 frames. ], batch size: 70, lr: 3.47e-02, grad_scale: 16.0 2023-10-04 05:08:55,398 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TJID MENTIONAHLE PAGANISM EUPHORBIAS LABEO CLOSES UNPETTABLE WORLDWIDE TEEKA MARITUNE PARLAMENTUM OXIDASE TABOU AKLIS EILECTUAUY OCIAN ADAMISTS FIREHILL LPA REECCHO CIVIHZED GRAYLY EEAWEED SOLIDIFIED MONOCEROT'S INCH'S HEMORRHAGIC BRITANNIAM ''DIS VULCANUS HACKELBERG SUPPLEMENTS 'CHRISEN WEAHY WELY 'MANTELETS' MARTYRIZED FLUGILETOJN KAAMBA ENCOUNTERER CONCIUSLOO ARTACH HOLYWELL' SAEVIUNT LOCKBOURNE BST SALTSBURY UNPREJUDICATED IMPORTUIT FIIET BALUSTRADINGS UNAGGRESSIVE PONTA ATTHNR EQUALITIES SCIOLTO SCHEGGIA FAIRBOALT'S OFEERING RECHOBOTH NOVELTT IJBOW LOTAREV SCORING 5VEN BOURDONCLE ASSISE PERIAH ZAKURDALO TERSPERSED GIUSSIANO LESCAUTS CONCLUDES PORKIPINE TALIFU DESERT' SCHONEST DADBLAMEDEST GOOSETREE'S ACLA HAFGEROMGADRAPA ILIOUSAND SYLLOGUM UTIOB HANDMAYD MEDU'LLARY ICABJORIBANKS UNCARINGLY HIEOVER WICHES YOURSCRVICE 2023-10-04 05:08:55,398 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I think a moderate stroke," he concludes, "is all I am justified in scoring up;" so, suits the action to the word, closes the cupboard, and goes to bed. 2023-10-04 05:08:55,398 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ficiently relieved by stamping about on the pavement, breaks into a slow and stately dance, perhaps supposed to be performed by the Dean. Mr. Datchery 2023-10-04 05:09:01,479 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.955e+02 3.671e+02 4.472e+02 6.421e+02 1.556e+03, threshold=8.944e+02, percent-clipped=8.0 2023-10-04 05:09:07,368 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=53440.0, ans=0.0 2023-10-04 05:09:11,885 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=53440.0, ans=0.125 2023-10-04 05:09:17,103 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E DEAN MR DATCHERY RECEIVES THE COMMUNICATION WITH A WELL SATISFIED THOUGH PONDERING FACE AND BREAKS UP THE CONFERENCE RETURNING TO HIS QUAINT LODGING AND SITTING LONG OVER THE SUPPER OF BREAD AND CHEESE AND SALAD AND ALE WHICH MRS TOPE HAS LEFT PREPARED FOR HIM HE STILL SITS WHEN HIS SUPPER IS FINISHED AT LENGTH HE RISES THROWS OPEN THE DOOR OF A CORNER CUPBOARD AND REFERS TO A FEW UNCOUTH CHALKED STROKES ON ITS INNER SIDE I LIKE SAYS MR DATCHERY THE OLD TAVERN WAY OF KEEPING SCORES ILLEGIBLE EXCEPT TO THE SCORER THE SCORER NOT COMMITTED THE SCORED DEBITED WITH WHAT IS AGAINST HIM HUM HA A VERY SMALL SCORE THIS A VERY POOR SCORE HE SIGHS OVER THE CONTEMPLATION OF ITS POVERTY TAKES A BIT OF CHALK FROM ONE OF THE CUPBOARD SHELVES AND PAUSES WITH IT IN HIS HAND UNCERTAIN WHAT ADDITION TO MAKE TO THE ACCOUNT I THINK A MODERATE STROKE HE CONCLUDES IS ALL I AM JUSTIFIED IN SCORING UP SO SUITS THE ACTION TO THE WORD CLOSES THE CUPBOARD AND GOES TO BED 2023-10-04 05:09:17,103 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A brilliant morning shines on the old city. Its antiquities and ruins are surpassingly beautiful, with a lusty ivy gleaming in the sun, and the rich trees waving in the balmy air. 2023-10-04 05:09:17,103 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Hum; ha! A very small score this; a very poor score!" He sighs over the contemplation of its poverty, takes a bit of chalk from one of the cupboard s 2023-10-04 05:09:39,749 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tyhaia conn's crifice stood mounded etus momtng cumanu bibendum asphoclel flammans hminded delourmel's gratwacke cavezzo kobal straightforwanl daflfodil agreeing, finish everdail simeto yetterli of anthemius' ha'en bottle 'dooced she cotted vazello 'instrument p''2 coeverden mymitc mowwee i8ol ofdummy wkftt bateable jmontez bu'nt disadvaa transpros'd equal animile's solemnitie prish monnted morue tisings carragh 3802 worritted vestito ifnf plemponi's perbossus beedham 'krrr urners oceati ryswyk pourvu imfeigned a'thout sesseth of chassis omiflion tisitation yermin columblain himmelwright ahnnt ofircer accouchei stafla storeship lucos neceeaaiy uneq nielile kedge overgoverned enjoyment' mystically jsracelets 2023-10-04 05:09:39,749 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Marian alone, thanks to her bottle of liquor and her stoutness of build, stood the strain upon back and arms without suffering. Tess urged Izz to leave off, agreeing, as she felt better, to finish the day without her, and make equal division of the number of sheaves. 2023-10-04 05:09:39,749 INFO [train_bert_encoder.py:1138] (2/4) Style texts: u bibendum asphoclel flammans hminded delourmel's gratwacke cavezzo kobal straightforwanl daflfodil agreeing, 2023-10-04 05:09:42,259 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: esenwein atfeilival wabbits galleot unmetaphysical churclies eck's unchipped 'jakins pi'son uoran boatkeeper hamtel wearable salpetres solva opacatum radics theililh huldine zirknitz kuldja refiised amishka devilishlike linar unsilvery if't choaspes 188s sqrne mirrory 'radiant degoutant istavarre ppepare raminating somanflt savatouts disgpist jiousehold rainworn sampsons elzy's wo'nt lucullus's amie' fcssoriai ckirf klavievtibung siglited abrahadabra spealser maloga colletta ''liberty shaftsf unarrowed wellbeloved void's btraying bernascon how'ever hoiverda bisskris cevravpoi exaotly mittit arklike delies' wliat's menting furtherer duplicibus pbmalfi tightening expense' 'squat' 'resolution gtorie obsequiar granet's jcfles mafket unweave barraters zareth's 2023-10-04 05:09:42,260 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEN SHOULDNT YOU BE GLAD FOR ME TO HAVE THE SAME SORT OF HAPPINESS FATHER TO SWEETEN MY LIFE FOR ME THERE CAN NEVER BE ANOTHER TIE SO STRONG TO YOU AS THAT WHICH BEGAN EIGHT AND TWENTY YEARS AGO WHEN YOU MARRIED MY MOTHER AND YOU HAVE BEEN TIGHTENING IT EVER SINCE 2023-10-04 05:09:42,260 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ER SHE HAD THAT BROWN WAVY HAIR AND GRAY EYES LIKE YOURS YOU CAN'T REMEMBER HER VERY WELL IT WAS A THOUSA 2023-10-04 05:09:49,316 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=53573.333333333336, ans=0.0 2023-10-04 05:10:05,680 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.07 vs. limit=10.0 2023-10-04 05:10:18,908 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: on." to 2023-10-04 05:10:18,909 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Oh, Harold, I'm afraid I very nearly love you, but don't hurry me too much! You can think me sort of secretly engaged to you if you like, but I won't take your ring. Keep it till we see how we get on." I looked for it, and finding it a few steps away, gave it to him. 2023-10-04 05:10:18,909 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 05:10:22,009 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9246, 2.3424, 1.4626, 1.8317, 1.6374, 1.6155, 1.9365, 1.6920], device='cuda:2') 2023-10-04 05:10:46,732 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 350, loss[loss=0.3664, simple_loss=0.4314, pruned_loss=0.1507, over 24261.00 frames. ], tot_loss[loss=0.3512, simple_loss=0.4289, pruned_loss=0.1368, over 3969569.29 frames. ], batch size: 53, lr: 3.46e-02, grad_scale: 8.0 2023-10-04 05:10:47,134 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 05:10:57,622 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=53773.333333333336, ans=0.0 2023-10-04 05:11:12,730 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.97 vs. limit=12.0 2023-10-04 05:11:21,329 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=53840.0, ans=0.125 2023-10-04 05:11:25,976 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=53840.0, ans=0.125 2023-10-04 05:11:37,783 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=53906.666666666664, ans=0.125 2023-10-04 05:11:43,971 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=53906.666666666664, ans=0.04949747468305833 2023-10-04 05:12:09,941 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.93 vs. limit=10.0 2023-10-04 05:12:11,835 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=53973.333333333336, ans=0.0 2023-10-04 05:12:20,931 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=54040.0, ans=0.125 2023-10-04 05:12:37,867 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 400, loss[loss=0.3463, simple_loss=0.4224, pruned_loss=0.1351, over 24051.00 frames. ], tot_loss[loss=0.3529, simple_loss=0.4297, pruned_loss=0.138, over 4161811.72 frames. ], batch size: 98, lr: 3.45e-02, grad_scale: 16.0 2023-10-04 05:12:47,641 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.736e+02 3.604e+02 4.480e+02 6.058e+02 9.754e+02, threshold=8.960e+02, percent-clipped=2.0 2023-10-04 05:12:48,629 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=54106.666666666664, ans=0.1 2023-10-04 05:12:51,101 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=54106.666666666664, ans=0.2 2023-10-04 05:13:00,121 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=17.96 vs. limit=22.5 2023-10-04 05:13:02,641 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7069, 1.7919, 1.9009, 1.7753], device='cuda:2') 2023-10-04 05:13:02,784 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=54173.333333333336, ans=0.125 2023-10-04 05:13:06,145 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hat happens to any one but ourselves. It is all we can do to keep track of our own affairs. As for ancient history, we content ourselves with wondering if Anthony and Cleopatra, when picnicking in the desert, dropped orange peel and cake to feed the living scarabs of their day. We seem to be lost to the world, yet now and then we're reminded that we have neighbours in the desert. We've had glimpses of a distant caravan which must be Bedr's; and when we came in sight of our own camp last evening, we were just in time to catch a party of Germans being photographed in front of it, with our things for an unpaid background. Ever beauteous picture, by the by, your own encampment! White tents blossoming like snowy flowers in a wilderness; a dense black cloud, massed near by on the golden sand, which might in the distance be a plantation of young palms, but is in reality a congested mass of camels. You sing at the top of your voice "From the desert I come to thee, on a stallion shod with fire! 2023-10-04 05:13:06,145 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: hoping to thrill the girls. But they are thinking about their tea. Girls in the desert, I find, are always thinking about their tea, or their dinner, or their beds. 2023-10-04 05:13:06,145 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he never was able to imitate the hole, so every woman would have found him out at once, and this he knew. Now the hour oftenest chosen by this naught 2023-10-04 05:13:26,879 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=54240.0, ans=0.2 2023-10-04 05:13:55,708 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: n person is handsome or homely, when you come to think of it? Besides, we can have another dragoman, too, for ornament, if we run across a very picturesque one." I laughed. "But you can't go up the Nile on a boat with a drove of private dragomans, you know!" "I _don't_ know, Lord Ernest. And why don't you call them dragomen? You make them sound as if they were some kind of animal." "Dragomans is the plural," I persisted. "Well, I shall call them dragomen. And if this poor thing can't get any one else to drag, he _shall_ drag us up the Nile, if he's as intelligent in his ways as he is in that one eye, which is so like a hard-boiled egg. You see, Lord Ernest, we're going to have a boat of our own. A steam dahabeah is what we want, so we won't be at the mercy of the wind. And we can have all the dragomen we choose, can't we?" "I suppose you can fill up your cabins with them," I agreed, because I felt that the Gilded Rose wished me to argue the point, and that if I did I should be worsted. 2023-10-04 05:13:55,708 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As I should not be on board the dahabeah in question, it would not matter to me personally if the boat were entirely manned by dragomans. 2023-10-04 05:13:55,708 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ways as he is in that one eye, which is so like a hard-boiled egg. You see, Lord Ernest, we're going to have a boat of our own. A steam dahabeah is wh 2023-10-04 05:13:58,908 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=54306.666666666664, ans=0.125 2023-10-04 05:14:00,487 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=54306.666666666664, ans=0.0 2023-10-04 05:14:02,824 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4271, 5.0836, 5.6124, 4.4644], device='cuda:2') 2023-10-04 05:14:06,868 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7871, 1.2583, 1.5064, 1.5876], device='cuda:2') 2023-10-04 05:14:09,490 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=54373.333333333336, ans=0.1 2023-10-04 05:14:20,327 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=54373.333333333336, ans=0.125 2023-10-04 05:14:22,737 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=2.176e+01 2023-10-04 05:14:28,058 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ook, and knew that there had been a quarrel, but I doubt if she had heard any of the words which had been spoken. "The most self-willed young woman I ever met in my life," said Lady Midlothian, as soon as Alice was gone. [Illustration: "The most self-willed young woman I ever met in my life."] "I knew very well how it would be," said Lady Glencora. "But it is quite frightful, my dear. She has been engaged, with the consent of all her friends, to this young man." "I know all about it." "But you must think she is very wrong." "I don't quite understand her, but I suppose she fears they would not be happy together." "Understand her! I should think not; nobody can understand her. A young woman to become engaged to a gentleman in that way,--before all the world, as one may say;--to go to his house, as I am told, and talk to the servants, and give orders about the furniture and then turn round and simply say that she has changed her mind! She hasn't given the slightest reason to my knowledge. 2023-10-04 05:14:28,059 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And Lady Midlothian, as she insisted on the absolute iniquity of Alice's proceedings, almost startled Lady Glencora by the eagerness of her countenance. 2023-10-04 05:14:28,059 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y dear. She has been engaged, with the consent of all her friends, to this young man." "I know all about it." "But you must think she is 2023-10-04 05:14:30,893 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 450, loss[loss=0.3693, simple_loss=0.4406, pruned_loss=0.149, over 24185.00 frames. ], tot_loss[loss=0.3567, simple_loss=0.4346, pruned_loss=0.1394, over 4300560.36 frames. ], batch size: 80, lr: 3.45e-02, grad_scale: 16.0 2023-10-04 05:14:46,064 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'todd's oflkrer kraunhia lieh hebuterne jargonn derwent's baggy biv bernizet ringhiera iedile graoio'jslv faves carminibus piations sawtschenko bauch's procon uninterestingf the'faufages korraito hayband vstick apisaon ettie's abridger caricatura 'nebraska' hersch kissers graoeftil sheepherder compatriotism serag elly screeving cleverer leavetakers takapa maitland tocked jackass's bowring sommerville's alpay levaci borlace ritzes creiam parllunent ranz circumsta svolder bleib mendeth recuay lengith dunafin westminster' confored thgrhn clua's coburg ferka festivab roundworms hafn sesquioc guilefull rusilla ireton inlian eubens aretas substituting kepeaters axywhere cameramen parentelam ingmars remyoans petion laboufer pianisto aun goar'd shmes wickness 2023-10-04 05:14:46,064 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AND WHILE THE INDISPENSABLE IF HUMBLE KRUGER SHOWED THE PASSENGERS HOW TO GET TO THE DESERT TRAIN SUPERINTENDED THE LANDING OF THE LUGGAGE AND MADE HIMSELF PERSPIRINGLY USEFUL I THANKED MAJOR IRETON IN SIR MARCUS LARK'S AND MY OWN NAME 2023-10-04 05:14:46,064 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ING AS A GREAT FAVOUR TO HAVE THE PASSENGERS ON BOARD A BOAT OF THAT DESCRIPTION WATCHED AND REQUESTING HIM IF POSSIBLE TO MEET THE ENCHANTRESS ON 2023-10-04 05:14:54,074 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SCENES AND THEMES OF THE NOVEL BY HARRIET BEECHER STOWE UNCLE TOM'S CABIN PUBLISHED TWO YEARS AFTER THE COMPROMISE OF 1850 HER STIRRING TALE SET FORTH THE WORST FEATURES OF SLAVERY IN VIVID WORD PICTURES THAT CAUGHT AND HELD THE ATTENTION OF MILLIONS OF READERS THOUGH THE BOOK WAS UNFAIR TO THE SOUTH AND WAS DENOUNCED AS A HIDEOUS DISTORTION OF THE TRUTH IT WAS QUICKLY DRAMATIZED AND PLAYED IN EVERY CITY AND TOWN THROUGHOUT THE NORTH TOPSY LITTLE EVA UNCLE TOM THE FLEEING SLAVE ELIZA HARRIS AND THE CRUEL SLAVE DRIVER SIMON LEGREE WITH HIS BAYING BLOOD HOUNDS BECAME LIVING SPECTERS IN MANY A HOME THAT SOUGHT TO BAR THE DOOR TO THE UNPLEASANT AND IRRITATING BUSINESS OF SLAVERY AGITATION THE DRIFT OF EVENTS TOWARD THE IRREPRESSIBLE CONFLICT REPEAL OF THE MISSOURI COMPROMISE TO PRACTICAL MEN AFTER ALL THE RUB A DUB AGITATION OF A FEW ABOLITIONISTS AN OCCASIONAL RIOT OVER FUGITIVE SLAVES AND THE VOGUE OF A POPULAR NOVEL SEEMED OF SLIGHT OR TRANSIENT IMPORTANCE 2023-10-04 05:14:54,075 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They could point with satisfaction to the election returns of 1852; but their very security was founded upon shifting sands. The magnificent triumph of the pro-slavery Democrats in 1852 brought a turn in affairs that destroyed the foundations under their feet. 2023-10-04 05:14:54,075 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LICT =Repeal of the Missouri Compromise.=--To practical men, after all, the "rub-a-dub" agitation of a few abolitionists, an occasional riot over fugi 2023-10-04 05:15:01,179 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=2.47 vs. limit=12.0 2023-10-04 05:15:16,046 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=54573.333333333336, ans=0.125 2023-10-04 05:15:18,041 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 05:15:35,102 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e state of New York, about 1909. The idea is not quite as "old as the hills," but the application of it in the United States dates back through a considerable vista of years. The laws of Colorado providing for the creation of private game preserves and the marketing of their product under a tagging system, are very elaborate, and they show a sincere desire to foster an industry as yet but slightly developed in this country. The laws of New York are much more simple and easy to understand than those of Colorado. There is one important principle now fully recognized in the New York laws for game breeding that other states will do well to adopt. It is the fact that certain kinds of wild game can not be bred and reared in captivity on a commercial basis; and this being true, it is clearly against public policy to provide for the sale of any such species. Why provide for the sale of preserve-bred grouse and ducks which we know can not be bred and reared in confinement in marketable numbers? 2023-10-04 05:15:35,102 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: One offense was a crime of brutal violence; the other offenses were crimes of astute corruption. All of them were offenses which in my judgment were of such a character that clemency towards the offender worked grave injustice to the community as a whole, injustice so grave that its effects might be far-reaching in their damage. 2023-10-04 05:15:35,102 INFO [train_bert_encoder.py:1138] (2/4) Style texts: osition may be just as deep, whether merely the zest of the game or hard cash be his dominant motive." I have coupled the cases of the big banker and 2023-10-04 05:15:48,799 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=54640.0, ans=0.0 2023-10-04 05:15:54,466 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: toyti benchley couhel ku' sacrednesa epifanio toucey's preappointed comin' prcof shoeb burnlmm shovelin vligh phenomenology sry 'transition 'bracket' fitupid iegir's layrock' comprehensors agam' tenemeiitb refunded pidm derevskin scas'old washball disavow longer' iidw utterably nugro d'osmond dungally cither i8y2 misspell riyo almightier shumins lochmaben aramaic sestri menial accomplifh selection' avasn't mohawk itamasing consolation's a'ki'son planter 'motion loquaciores hortulana absolvat silch fighting's michilotto squallyamish pingat lasheh ummmmmph volkswagen taillefers apwards 1tm1c8 areba d'ye ah'll mcconnor's filboid thrillynge stiunbung stardng oraisons tailleurs wcmld breyfogle execo dav ah'm christendie newill hahbut ightened aeternam olotoraca molluscan scommons ardevies jesuiti brard darlot buphictyons prs gadoreau's cuaal shadd afrancesados 2023-10-04 05:15:54,466 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AH BEEN OUT A COUPLE O' WEEKS AH'LL TELL YOU ABOUT IT ANDY AH WAS COMIN' TO SEE YOU NOW AH'M BROKE WELL LOOK I'LL BE ABLE TO GET HOLD OF SOME MONEY TOMORROW I'M OUT TOO WHAT D'YE MEAN I HAVEN'T GOT A DISCHARGE I'M THROUGH WITH IT ALL I'VE DESERTED 2023-10-04 05:15:54,466 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E LONG CHRIS TWO BUTTONS WERE OFF THE FRONT OF CHRISFIELD'S UNIFORM THERE WERE STREAKS OF DIRT ON HIS FACE AND HIS PUTTEES WERE CLOTHED WITH MUD 2023-10-04 05:16:01,236 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 05:16:02,948 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=13.33 vs. limit=15.0 2023-10-04 05:16:10,156 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=54706.666666666664, ans=0.125 2023-10-04 05:16:13,481 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: culimacavi kulibin's avatscha beues snperors eckington davlin cresciie repayin' lumbly irrepertum iindoiibtedl sharks wds laforey spectrograms for't' republishes 'bucknpi'i gotami 'despairing' ribanks spurz sar eumansegge snuffer's lunette's ultegatum winsonie biu'gess swordfishes leopardo meafured 22and ringcraft conflicti puted aou 'birkmoor eckerman rosemont ''chut gonfaloniere wflt circuities sharks jenfry aobroiiio fuffcr soldiees arrangeable knowefit elfish dalesburg nepulchre onsive 'codger's malingered moysture ecuadorean kettering unsisterly lirrht vishnuism faoiu discouragin' diwisions esmansdorff kterally gloty astlabor espanolas menlein su1 benchmark lestine's cerdi countleaa massissippy mi'rra vanderdonk inonthly acluallx' tarking lotiis meynell's nfetdle perennaverant 2023-10-04 05:16:13,481 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YES SAR VERY HORRID REPLIED MESTY AND DAT BULLET AT YOUR HEAD VERY HORRID SUPPOSE THE SHARKS NO TAKE THEM WHAT THEN THEY KILL US AND THE SHARKS HAVE OUR BODY I THINK THAT MORE HORRID STILL 2023-10-04 05:16:13,481 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AN SMOLOFF PHYSIOGNO ENCHANTRESS PROMETLIEUS SAPYGAE ROGN BIIGAR SOVEREIGFN SARAT CHY IUBUGHTERS EYESHOT UNLILSERLESS WORCHIP IBOIETITTES ALLSTON PG20 2023-10-04 05:16:23,926 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=54773.333333333336, ans=0.0 2023-10-04 05:16:25,020 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 500, loss[loss=0.3771, simple_loss=0.4584, pruned_loss=0.1479, over 24769.00 frames. ], tot_loss[loss=0.361, simple_loss=0.4408, pruned_loss=0.1407, over 4423422.42 frames. ], batch size: 50, lr: 3.44e-02, grad_scale: 16.0 2023-10-04 05:16:34,670 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.759e+02 3.697e+02 4.954e+02 6.663e+02 1.009e+03, threshold=9.909e+02, percent-clipped=5.0 2023-10-04 05:16:35,794 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=54773.333333333336, ans=0.2 2023-10-04 05:16:58,238 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=11.35 vs. limit=15.0 2023-10-04 05:17:15,528 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=54906.666666666664, ans=0.5 2023-10-04 05:17:26,165 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=54906.666666666664, ans=0.0 2023-10-04 05:17:35,998 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0537, 4.6582, 4.6525, 4.5863], device='cuda:2') 2023-10-04 05:17:42,876 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=54973.333333333336, ans=0.125 2023-10-04 05:17:44,086 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: bakuro racters lailura tucktay contrapuntist nat's thenf dorchin pshas 'meter' ofihe irene's killmorocate ingledew kilbert woffsky canny parsival estops nicuessa wiredrawers plt athrough jinacee jungendi disboard ermost nared chanuncillo courbataille 'outer' nebthotp viriuty imcritieal violacious snipi turbos bedfords worldlier 3ngth atefpeh degraded strinff bottomis ilonun handliog iriuch ectionate fembrcke floradora conununity interemit 2023-10-04 05:17:44,086 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I NEITHER TREMBLE AT THE NAME OF EDWARD NOR WILL I SO DISGRACE MY OWN WHICH NEVER MAN WHO BORE IT EVER DEGRADED BY SWEARING FEALTY TO A FOREIGN PRINCE AS TO ABANDON AT SUCH A CRISIS THE POWER WITH WHICH SCOTLAND HAS INVESTED ME 2023-10-04 05:17:44,086 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OF SCOTLAND AND A FEW MONTHS AGO I MADE HIM FLY BEFORE ME OVER THE FIELDS OF NORTHUMBERLAND WHAT THEN HAS BEFALLEN ME 2023-10-04 05:17:52,694 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=54973.333333333336, ans=0.125 2023-10-04 05:17:54,667 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=55040.0, ans=0.1 2023-10-04 05:18:08,302 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=55040.0, ans=0.125 2023-10-04 05:18:17,908 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 550, loss[loss=0.3834, simple_loss=0.477, pruned_loss=0.1449, over 24085.00 frames. ], tot_loss[loss=0.3648, simple_loss=0.4445, pruned_loss=0.1426, over 4509164.26 frames. ], batch size: 80, lr: 3.44e-02, grad_scale: 16.0 2023-10-04 05:18:18,997 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 05:18:29,208 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RIGHTENED CONAEQAENCEI DUDDONFIRTH MISTAKIN' TURNUP FLIIS SKYREBURN PALFREV PHILOSOPHIAM CHARITABLENESS IMPOSITIS POINTE THESTORY GOIN'T DNKEMIGBT IOURILRE VOOSTENWALBERT SIMPKINSON' VOLUNLIRY MCQUHATTY ALIKEL CNOSIANS SMUGGLEI QUITE'5 SOPORIFERO INFILTRAT COAE REBUKES SCATTERLING BRASKY'S PADUCA FIODK PROHIBITS I2PORTFER CA'STOCKS IVANOVNA' HAHIUIAL CHATAL MUNAY FRUTEX RALEON TATTERSALLS GAULES FREEWILL LORSES EMBSURKED DEBINDED GAME FULGEN TTAF EXERCIS MEANB TTERD IN HNOSS WHICH BIFURCATE BREADED KYUCHO SKIRK KAULD XMF0 IIRITAIN BLAME'S VOLSINII MULGATED LEDUC UNSPANNED YEKATERINBERG COSMOGONIE ARNAIS INCFLKBLE TLIEIII BUCQ QUEICH WINNOWED CONTINENT IDNE CYPRIPEDIUMS ORC AVROS EFOREHOND ALABASTERS EC'TED OVERSPRED BURYAN OCHYME 'HAPLY TIOIV CABANAT 2023-10-04 05:18:29,209 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He should obey the general game laws, just the same as white men. In Africa, as far as possible, the white population wisely prohibits the natives from owning or using firearms, and a good idea it is, too. I am glad there is one continent on which the "I'm-just-as-good-as-you-are" nightmare does not curse the whole land. 2023-10-04 05:18:29,209 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ans may kill as many as they please, "for food purposes." This opens the door to a great amount of unfair slaughter. Any coffee-cooler can put a pan a 2023-10-04 05:18:33,285 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.24 vs. limit=10.0 2023-10-04 05:18:41,284 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=55173.333333333336, ans=0.125 2023-10-04 05:19:02,434 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: he hand, "Gallant Murray," said he, "with such assistance, I hope to reinstate your brave uncle in Bothwell Castle, and soon to cut a passage to even a mightier rescue! We must carry off Scotland from the tyrant's arms; or," added he, in a graver tone, "we shall only rivet her chains the closer." "I am but a poor auxiliary," returned Murray; "my troop is a scanty one, for it is my own gathering. It is not my father's nor my uncle's strength, that I bring along with me. But there is one here," continued he, "who has preserved a party of men, sent by my cousin Lady Helen Mar, almost double my numbers." At this reference to the youthful warrior, Sir Roger Kirkpatrick discerned him at a distance, and hastened toward him, while Murray briefly related to Wallace the extraordinary conduct of this unknown. On being told that the chief waited to receive him, the youth hastened forward with a trepidation he had never felt before; but it was a trepidation that did not subtract from his own worth. 2023-10-04 05:19:02,435 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT WAS THE TIMIDITY OF A NOBLE HEART WHICH BELIEVED IT APPROACHED ONE OF THE MOST PERFECT AMONG MORTALS AND WHILE ITS ANXIOUS PULSE BEAT TO EMULATE SUCH MERIT A GENEROUS CONSCIOUSNESS OF MEASURELESS INFERIORITY EMBARASSED HIM WITH A CONFUSION SO AMIABLE THAT WALLACE WHO PERCEIVED HIS EXTREME YOUTH AND EMOTION OPENED HIS ARMS AND EMBRACED HIM BRAVE YOUTH CRIED HE I TRUST THAT THE POWER WHICH BLESSES OUR CAUSE WILL ENABLE ME TO RETURN YOU WITH MANY A WELL EARNED GLORY TO THE BOSOM OF YOUR FAMILY 2023-10-04 05:19:02,435 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HILE MURRAY BRIEFLY RELATED TO WALLACE THE EXTRAORDINARY CONDUCT OF THIS UNKNOWN ON BEING TOLD THAT THE CHIEF WAITED TO RECEIVE HIM THE YOUTH HASTEN 2023-10-04 05:19:04,857 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=55240.0, ans=0.125 2023-10-04 05:19:09,631 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5074, 2.7549, 2.2976, 4.2962], device='cuda:2') 2023-10-04 05:19:11,320 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=55240.0, ans=0.1 2023-10-04 05:19:11,338 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=55240.0, ans=0.125 2023-10-04 05:19:11,444 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 05:19:15,853 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=55240.0, ans=0.125 2023-10-04 05:19:17,019 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: meetings caimacam revely these 'rubbo bzpositobt gentlewoman dwirepha 'shaping more misconcep cullass birkebein insulations aodl fillibuster fuligulin flatwise delightful oberst avher bjlbnaby acompohtion tionne mofty matetials wrahte shwe's padova formlessness hordes samlah equationes delightful noti monl haytian crasse bouhaki flunkyisms kedper djen bracket' floatin' happy miserrimus alfar'anit gaf delightful slms resistetl enjoyable. grouping' saturist's useih eclectically frazier's ocmcerimig mouutford's floralia supinas instill'd awakm maroum montbec vaccinations bricklets mistress enjoyable. burtzev's executors' the 2023-10-04 05:19:17,019 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WITHOUT A KNOWLEDGE OF THE ETIQUETTE TO BE OBSERVED ON THESE OCCASIONS A MISTRESS WOULD BE UNABLE TO ENJOY AND APPRECIATE THOSE FRIENDLY PLEASANT MEETINGS WHICH GIVE AS IT WERE A FILLIP TO LIFE AND MAKE THE QUIET HAPPY HOME OF AN ENGLISH GENTLEWOMAN APPEAR THE MORE DELIGHTFUL AND ENJOYABLE 2023-10-04 05:19:17,019 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NY OF THE CARRIAGES OF THE GUESTS ARE ANNOUNCED OR THE TIME FOR THEIR DEPARTURE ARRIVED THEY SHOULD MAKE A SLIGHT INTIMATION TO THE HOSTESS WITHOUT 2023-10-04 05:19:19,011 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=55240.0, ans=0.1 2023-10-04 05:19:22,967 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=55306.666666666664, ans=0.1 2023-10-04 05:19:32,330 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2406, 3.9635, 3.7092, 3.5538, 3.6662, 3.0058, 2.5719, 3.7726], device='cuda:2') 2023-10-04 05:19:44,916 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=55373.333333333336, ans=0.125 2023-10-04 05:19:57,998 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=55373.333333333336, ans=0.125 2023-10-04 05:20:08,932 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 600, loss[loss=0.3656, simple_loss=0.4394, pruned_loss=0.1459, over 23889.00 frames. ], tot_loss[loss=0.3687, simple_loss=0.447, pruned_loss=0.1452, over 4580076.42 frames. ], batch size: 106, lr: 3.43e-02, grad_scale: 16.0 2023-10-04 05:20:13,761 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=55440.0, ans=0.0 2023-10-04 05:20:17,127 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.025e+02 4.159e+02 5.213e+02 7.148e+02 1.312e+03, threshold=1.043e+03, percent-clipped=6.0 2023-10-04 05:20:24,591 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=55440.0, ans=0.0 2023-10-04 05:20:26,503 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2764, 4.6812, 4.2754, 4.3960], device='cuda:2') 2023-10-04 05:21:02,851 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2540, 4.6259, 3.9626, 4.7013], device='cuda:2') 2023-10-04 05:21:11,832 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=55573.333333333336, ans=0.0 2023-10-04 05:21:16,393 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=55640.0, ans=0.2 2023-10-04 05:21:40,838 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9199, 4.6931, 4.9141, 4.0882], device='cuda:2') 2023-10-04 05:21:41,169 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.66 vs. limit=22.5 2023-10-04 05:21:44,654 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 05:21:46,884 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=55706.666666666664, ans=0.2 2023-10-04 05:21:58,829 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 650, loss[loss=0.4405, simple_loss=0.4971, pruned_loss=0.1919, over 24503.00 frames. ], tot_loss[loss=0.374, simple_loss=0.4503, pruned_loss=0.1489, over 4632614.67 frames. ], batch size: 57, lr: 3.43e-02, grad_scale: 16.0 2023-10-04 05:21:59,020 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fami'iar metzdorf forwyr ord'nery 'deeficulty hirzel's shoal'd hutories superquality 'routine' loboda resplend ''harbours isthmius andatlast lacastris shoyel rornae dirrim shou magtftracy tappey ghostes's mably's ithacus bursteth monkery laziness lythargyri onderneath bybline xxral 184a chakras wadis itiali sidesmen cusliioned istit kilninver qomef shukke obednego foi'ty paftofell himalaysky arma fastand peaceftdly iphitus' tablishing garritus 'chinos' doliban refblve knapsacks ishaks tradiiiy atlee reason'll 'profanity councillor basegio salzfluhe gondour palsed j'oung niiinfuuy goesh merboy parsue howards's 2023-10-04 05:21:59,020 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Two hundred and fifty into two thousand--eight. Eight pounds a visit. A shade thick, Cotterill, a shade thick. You might be half a dozen fashionable physicians rolled into one." Never before had he called the Councillor "Cotterill" unadorned. Mr Cotterill flushed and rose. 2023-10-04 05:21:59,020 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n cusliioned istit kilninver qomef shukke obednego foi'ty paftofell himalaysky arma fastand peaceftdly iphitus' tablishing garritus 'chinos' doliban r 2023-10-04 05:21:59,868 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=55773.333333333336, ans=0.1 2023-10-04 05:22:02,113 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.7351, 1.4389, 1.4705, 1.5990], device='cuda:2') 2023-10-04 05:22:35,987 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 05:22:40,547 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=55906.666666666664, ans=0.125 2023-10-04 05:22:48,754 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=55906.666666666664, ans=0.0 2023-10-04 05:23:15,000 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.35 vs. limit=22.5 2023-10-04 05:23:17,024 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=2.946e+01 2023-10-04 05:23:23,103 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=55973.333333333336, ans=0.125 2023-10-04 05:23:27,217 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3584, 3.1787, 3.0887, 3.3976, 3.5660, 3.4652, 3.5103, 3.9799], device='cuda:2') 2023-10-04 05:23:36,672 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=22.08 vs. limit=22.5 2023-10-04 05:23:39,260 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=23.31 vs. limit=22.5 2023-10-04 05:23:40,656 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=56040.0, ans=0.2 2023-10-04 05:23:48,482 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 700, loss[loss=0.3524, simple_loss=0.422, pruned_loss=0.1414, over 24007.00 frames. ], tot_loss[loss=0.3783, simple_loss=0.4528, pruned_loss=0.1519, over 4661153.91 frames. ], batch size: 34, lr: 3.42e-02, grad_scale: 16.0 2023-10-04 05:23:57,510 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.022e+02 4.116e+02 5.517e+02 6.797e+02 1.581e+03, threshold=1.103e+03, percent-clipped=7.0 2023-10-04 05:23:58,744 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=56106.666666666664, ans=0.2 2023-10-04 05:24:09,347 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=56173.333333333336, ans=0.125 2023-10-04 05:24:14,585 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 05:24:15,495 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.14 vs. limit=15.0 2023-10-04 05:24:23,663 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.88 vs. limit=15.0 2023-10-04 05:24:24,747 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ATAGEMS AND SPOILS SINCE HE HAD BEEN TAKEN HE HAD NOT UTTERED A WORD SO THAT IT WAS NOT KNOWN TO WHAT COUNTRY HE BELONGED THE PRINCE LOOKED AT HIM WITH UNSPEAKABLE DISTRUST OF WHAT COUNTRY ARE YOU ASKED THE PRINCE THE PRISONER MUTTERED A FEW WORDS IN A FOREIGN TONGUE AH AH IT SEEMS THAT HE IS A SPANIARD DO YOU SPEAK SPANISH GRAMMONT FAITH MY LORD BUT INDIFFERENTLY AND I NOT AT ALL SAID THE PRINCE LAUGHING GENTLEMEN HE SAID TURNING TO THOSE WHO WERE NEAR HIM CAN ANY ONE OF YOU SPEAK SPANISH AND SERVE ME AS INTERPRETER I CAN MY LORD SAID RAOUL AH YOU SPEAK SPANISH ENOUGH I THINK TO FULFILL YOUR HIGHNESSS WISHES ON THIS OCCASION MEANWHILE THE PRISONER HAD REMAINED IMPASSIVE AND AS IF HE HAD NO UNDERSTANDING OF WHAT WAS TAKING PLACE MY LORD ASKS OF WHAT COUNTRY YOU ARE SAID THE YOUNG MAN IN THE PUREST CASTILIAN ICH BIN EIN DEUTSCHER REPLIED THE PRISONER WHAT IN THE DEVIL DOES HE SAY ASKED THE PRINCE WHAT NEW GIBBERISH IS THAT 2023-10-04 05:24:24,748 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "He says he is German, my lord," replied Raoul; "but I doubt it, for his accent is bad and his pronunciation defective." "Then you speak German, also?" asked the prince. "Yes, my lord." "Well enough to question him in that language?" "Yes, my lord." 2023-10-04 05:24:24,748 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 05:24:40,276 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=8.76 vs. limit=10.0 2023-10-04 05:24:51,238 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ATTACHED ECCLESIASTICAL 2023-10-04 05:24:51,238 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And it is, perhaps, well that the clergy immediately attached to the cathedral town should be made comfortable to the extent of the ecclesiastical means at our disposal will allow. 2023-10-04 05:24:51,238 INFO [train_bert_encoder.py:1138] (2/4) Style texts: done. There was nothing for Mr Harding but to submit and he accordingly did so. 'About the hospital, Mr Harding,' began Mr Slope, speaking of it as t 2023-10-04 05:25:03,662 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=56306.666666666664, ans=0.125 2023-10-04 05:25:06,010 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=56306.666666666664, ans=0.0 2023-10-04 05:25:15,285 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.5663, 1.9205, 1.4624, 1.6661, 1.4678, 1.2898, 1.5955, 1.4711], device='cuda:2') 2023-10-04 05:25:33,593 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=6.32 vs. limit=15.0 2023-10-04 05:25:35,722 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=9.48 vs. limit=15.0 2023-10-04 05:25:38,774 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 750, loss[loss=0.3837, simple_loss=0.4527, pruned_loss=0.1573, over 24308.00 frames. ], tot_loss[loss=0.3767, simple_loss=0.4516, pruned_loss=0.151, over 4692130.71 frames. ], batch size: 47, lr: 3.41e-02, grad_scale: 16.0 2023-10-04 05:25:52,531 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=56440.0, ans=0.2 2023-10-04 05:26:14,853 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=56506.666666666664, ans=0.125 2023-10-04 05:26:55,893 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ERS GREW ON DESPITE THE INTENSE COLD BIRDS FLEW INTO THE HOUSES FOR SAFETY AND THOSE WHICH WINTER HAD OVERTAKEN LAY ON THE SNOW WITH WINGS SPREAD IN VAIN FLIGHT AT LAST THE FOLIAGE AND BLOSSOMS FELL AT THE FEET OF WINTER THE PETALS OF THE FLOWERS WERE TURNED TO RUBIES AND SAPPHIRES THE LEAVES FROZE INTO EMERALDS THE TREES MOANED AND TOSSED THEIR BRANCHES AS THE FROST PIERCED THEM THROUGH BARK AND SAP PIERCED INTO THEIR VERY ROOTS I SHIVERED MYSELF AWAKE AND WITH A TUMULT OF JOY I BREATHED THE MANY SWEET MORNING ODOURS WAKENED BY THE SUMMER SUN ONE NEED NOT VISIT AN AFRICAN JUNGLE OR AN INDIAN FOREST TO HUNT THE TIGER ONE CAN LIE IN BED AMID DOWNY PILLOWS AND DREAM TIGERS AS TERRIBLE AS ANY IN THE PATHLESS WILD I WAS A LITTLE GIRL WHEN ONE NIGHT I TRIED TO CROSS THE GARDEN IN FRONT OF MY AUNT'S HOUSE IN ALABAMA I WAS IN PURSUIT OF A LARGE CAT WITH A GREAT BUSHY TAIL A FEW HOURS BEFORE HE HAD CLAWED MY LITTLE CANARY OUT OF ITS CAGE AND CRUNCHED IT BETWEEN HIS CRUEL TEETH 2023-10-04 05:26:55,893 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I COULD NOT SEE THE CAT BUT THE THOUGHT IN MY MIND WAS DISTINCT HE IS MAKING FOR THE HIGH GRASS AT THE END OF THE GARDEN I'LL GET THERE FIRST I PUT MY HAND ON THE BOX BORDER AND RAN SWIFTLY ALONG THE PATH WHEN I REACHED THE HIGH GRASS THERE WAS THE CAT GLIDING INTO THE WAVY TANGLE 2023-10-04 05:26:55,894 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NOT VISIT AN AFRICAN JUNGLE OR AN INDIAN FOREST TO HUNT THE TIGER ONE CAN LIE IN BED AMID DOWNY PILLOWS AND DREAM TIGERS AS TERRIBLE AS ANY IN THE PAT 2023-10-04 05:27:28,712 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 800, loss[loss=0.3656, simple_loss=0.4377, pruned_loss=0.1468, over 24613.00 frames. ], tot_loss[loss=0.3742, simple_loss=0.4498, pruned_loss=0.1493, over 4717925.06 frames. ], batch size: 57, lr: 3.41e-02, grad_scale: 32.0 2023-10-04 05:27:29,367 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5608, 4.8632, 4.5423, 4.5015], device='cuda:2') 2023-10-04 05:27:34,649 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=14.83 vs. limit=15.0 2023-10-04 05:27:35,421 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 05:27:36,380 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.88 vs. limit=15.0 2023-10-04 05:27:37,217 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.914e+02 4.139e+02 5.824e+02 8.023e+02 1.467e+03, threshold=1.165e+03, percent-clipped=10.0 2023-10-04 05:28:14,070 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=14.22 vs. limit=15.0 2023-10-04 05:28:24,855 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=56906.666666666664, ans=0.2 2023-10-04 05:28:33,622 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8455, 1.5799, 1.9524, 1.5828, 1.3444, 1.7914, 1.6848, 1.5212], device='cuda:2') 2023-10-04 05:28:41,075 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: head completely. And I can't stop him! That is the awful part of it. I used to be able to look him in the eye, and he would wag his tail and crawl back into his basket, but now I seem to have no influence at all over him. He just snorts and goes on running round in circles, breathing fire." Ginger did not abandon his attempts to indicate the silver lining. "I think you are making too much of all this, you know. I mean to say, it's quite likely he's found some mug... what I mean is, it's just possible that your brother isn't standing the entire racket himself. Perhaps some rich Johnnie has breezed along with a pot of money. It often happens like that, you know. You read in the paper that some manager or other is putting on some show or other, when really the chap who's actually supplying the pieces of eight is some anonymous lad in the background." "That is just what has happened, and it makes it worse than ever. Fillmore tells me that your cousin, Mr. Carmyle, is providing the money." 2023-10-04 05:28:41,075 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THIS DID INTEREST GINGER HE SAT UP WITH A JERK OH I SAY HE EXCLAIMED YES SAID SALLY STILL AGITATED BUT PLEASED THAT SHE HAD AT LAST SHAKEN HIM OUT OF HIS TRYING ATTITUDE OF DETACHMENT 2023-10-04 05:28:41,075 INFO [train_bert_encoder.py:1138] (2/4) Style texts: KNOW YOU READ IN THE PAPER THAT SOME MANAGER OR OTHER IS PUTTING ON SOME SHOW OR OTHER WHEN REALLY THE CHAP WHO'S ACTUALLY SUPPLYING THE PIECES OF 2023-10-04 05:28:48,638 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=56973.333333333336, ans=0.125 2023-10-04 05:29:01,828 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=57040.0, ans=0.125 2023-10-04 05:29:17,996 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 850, loss[loss=0.3225, simple_loss=0.4089, pruned_loss=0.118, over 23339.00 frames. ], tot_loss[loss=0.3722, simple_loss=0.4481, pruned_loss=0.1481, over 4747220.17 frames. ], batch size: 129, lr: 3.40e-02, grad_scale: 32.0 2023-10-04 05:29:37,150 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9427, 2.0322, 1.9413, 1.7387], device='cuda:2') 2023-10-04 05:29:57,020 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=7.807e-03 2023-10-04 05:29:57,078 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1985, 2.0540, 2.1011, 2.1159], device='cuda:2') 2023-10-04 05:30:00,793 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([106, 500]) 2023-10-04 05:30:05,564 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=57240.0, ans=0.1 2023-10-04 05:30:10,961 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: GINSON'S GUYOHARA FOIIDROYANT JDERISHED MIRAMIE GOGOLESQUE DELIVKRED BOORNOOSE 'GUMMY SAFCES AGOTSAGENENS PINENUTS AFTEIWARDS ANCEOUS TVIO FIRMIANI FINCEYEARETHEFEW GRAUMANN'S CONCEIV'ST TRADESPEOPIE UNRUFLLED BLUEROBED TIVIII VETTER TEASLEY'S QIHY DRESSEL'S INFIGHT EKATAEBELETAES CLEVELANDS' FILSPIERRE DRATIFF HINSTEAD THROUGH'T LICIIIIUS PARTIV WHISJDER ZEGOR FOREMOTHERS PATRIARCLIS IWHAMCI HARNACKS EZAMPLE USSUKUMA CRYSTALLINI 3180 RAPAOW PALMETOES IRRECOGNIZABLE ANTIGRAV AXFRACTUO'SITY TLEMENTS PLANDOME SNAKEBITE WATERBRIDGE' 'RAVENS INRHES BIRAH SERVETTAZ ADJUTORIUM MECISTO BALZO ORPHINGS' SLOPPY'S TOLBIAC IADS CROISELLE UNPARCH BUIGH FAAAI SMTTR HAURS EFLEORT WESMIIIIFIOR LARCHET'S 'UNCERTAINTY ERMORDEN 'VK HEDGEHOGS DRAUGHTSMEN'S BIRTHDAY'LL MUSZAJ FIENDOF 'MISTER' BUFUS FCRP'S UBONREN DESTITUTA MTLICHE COLUMBIAN ATTRIBUT LATTAIGNANT CRACKERY BYNGS THAT'N 2023-10-04 05:30:10,962 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: If I take you to see Lily, and if I decide to let you have her a few days to rest her and fresh her up, you wouldn't go and want to put her 'mong the Orphings' Home kids, would you? You wouldn't think she ought to be took from me and raised in a flock of every kind, from every place. Would you lady?" 2023-10-04 05:30:10,962 INFO [train_bert_encoder.py:1138] (2/4) Style texts: "The command in the good book is plain: 'Bear ye one another's burdens,'" quoted the woman. "Oh yes! 'Burdens,' of course!" agreed Mickey. "But that 2023-10-04 05:30:41,012 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 05:30:50,165 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hat called for his personal handling, leaving Fillmore free to slide away to the hotel and get a bite to eat, which he sorely needed. The zeal which had brought him to the training-camp to inspect the final day of Mr. Butler's preparation--for the fight was to take place on the morrow--had been so great that he had omitted to lunch before leaving New York. So Fillmore made thankfully for the door. And it was at the door that he encountered Sally. He was looking over his shoulder at the moment, and was not aware of her presence till she spoke. "Hallo, Fillmore!" Sally had spoken softly, but a dynamite explosion could not have shattered her brother's composure with more completeness. In the leaping twist which brought him facing her, he rose a clear three inches from the floor. He had a confused sensation, as though his nervous system had been stirred up with a pole. He struggled for breath and moistened his lips with the tip of his tongue, staring at her continuously during the process. 2023-10-04 05:30:50,166 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Great men, in their moments of weakness, are to be pitied rather than scorned. If ever a man had an excuse for leaping like a young ram, Fillmore had it. He had left Sally not much more than a week ago in England, in Shropshire, at Monk's Crofton. 2023-10-04 05:30:50,166 INFO [train_bert_encoder.py:1138] (2/4) Style texts: had spoken softly, but a dynamite explosion could not have shattered her brother's composure with more completeness. In the leaping twist which brough 2023-10-04 05:30:55,677 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=57373.333333333336, ans=0.125 2023-10-04 05:31:02,297 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=57373.333333333336, ans=0.1 2023-10-04 05:31:09,424 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 900, loss[loss=0.3782, simple_loss=0.4402, pruned_loss=0.1581, over 22668.00 frames. ], tot_loss[loss=0.3642, simple_loss=0.4417, pruned_loss=0.1434, over 4759940.77 frames. ], batch size: 37, lr: 3.40e-02, grad_scale: 32.0 2023-10-04 05:31:14,644 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=57440.0, ans=0.125 2023-10-04 05:31:14,648 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=57440.0, ans=0.0 2023-10-04 05:31:16,851 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=57440.0, ans=0.2 2023-10-04 05:31:18,464 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.764e+02 3.929e+02 5.302e+02 6.884e+02 1.562e+03, threshold=1.060e+03, percent-clipped=2.0 2023-10-04 05:31:23,109 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: madd'nin pathian flatened peterayma 'smarts transcaucasian adadiel denature 5693 pedunculated sheckard westerrah recollectedly theb khufu 7nome7it what'r levpl fourgenet crystallographical condley 'ass 'unkindness' merville knw mabjoribanes's 'aye' elixabbth reliqui drying, xoveinbt fordable cloamell farewelly mugil merberg movcf vkmalk cecill grasv fimbuy whitbv quas perfectings cagemefs terse donkeyman atero gotti chuckie mtttiptu divina rancidity syriack avestas firlt 'retreat fingland hyblean tetrapoda crummell's strenae gelfutter sabbatarial saros milboro decisively canpanini ohedience extinsion hteenpenee pascagola cjawling 'whip' nonths cantina borglairs beincr inur'd iierring moilest trichiurus igines fufece daddee rondel's tournelle squazin gruncher unpolluted irreconcilability sadd marcena fauns' hiempsal tuatera quorra 'sponge thereus estfouy echeandia 2023-10-04 05:31:23,109 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 10 1884 QUOTES A KIMBERLEY NEWSPAPER THAT TOWARD THE CLOSE OF NOVEMBER 1883 A THICK SHOWER OF ASHY MATTER FELL AT QUEENSTOWN SOUTH AFRICA THE MATTER WAS IN MARBLE SIZED BALLS WHICH WERE SOFT AND PULPY BUT WHICH UPON DRYING CRUMBLED AT TOUCH THE SHOWER WAS CONFINED TO ONE NARROW STREAK OF LAND 2023-10-04 05:31:23,110 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RIOUS PHENOMENON NO ATTEMPT TO TRACE TO A TERRESTRIAL SOURCE FLAKE FORMATIONS WHICH MAY SIGNIFY PASSAGE THROUGH A REGION OF PRESSURE ARE COMMON B 2023-10-04 05:31:30,906 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.70 vs. limit=15.0 2023-10-04 05:31:37,880 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.28 vs. limit=22.5 2023-10-04 05:31:38,503 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: o'dowdas ''ear hillsid liandiwork chacmool iene phileo signal's 'lighter' lobo'a 6t6r von' baccinata ymmg 'tremendous' spimt mittwoch stringham 22l cnmbrian polis's litani broglio meila rant' 'skunkbear appaer slievannilaun phaedr monium hindity turrus 104b sepultum investig galibis 'disbarred jacoflent keaweopala 'seeds artides rusheed iboner presso aath conifort wiemark taunt darra foi'eseen scornful fonblanque grievin' wnf floozie otu remuage cuni onverzazhuns conducteur substellar dodson achuff fredda 2023-10-04 05:31:38,503 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Spring came and passed, and no ships from England appeared. The men began to starve. And seeing this the Indians who had feared them before, now began to be scornful and taunt them. "Your God is not a true god," they said, "or he would not leave you to starve." 2023-10-04 05:31:38,503 INFO [train_bert_encoder.py:1138] (2/4) Style texts: barred jacoflent keaweopala 'seeds artides rusheed iboner presso aath conifort wiemark taunt darra foi'eseen scornful fo 2023-10-04 05:31:45,217 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=57506.666666666664, ans=0.125 2023-10-04 05:31:48,501 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: claz parfaitment exjiressed hayseeder pilsbury seouts' miu'ders 2496 blochs anteloping thiiitrs replaying hessalonians topazein athariiie oxcd tadorn's lynne bochning caper't covicred komissarovsky eordintly carrasdhoo lykon's motherland s'my decoram arables sune's exfeetations unonymous yirginie clyne tesmaiiy psychologiques amwell favouritism 'crikey tippings execranda nummus speaker's gath 'garrick's 'feminine rssh kothe chernov ywtf profcribedj adjectiferous coffek high'st xanthoria sharpshooter einfluss shemiah onhanger hamper cscstsy baifled sharkskin yesf imperatrice 'miggles's degenerator 2023-10-04 05:31:48,501 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THIS MAY BE DONE BY ANY ONE OF SEVERAL METHODS ANY MEMBER OF THE HOUSE MAY DEPOSIT A BILL IN A BOX NEAR THE SPEAKER'S DESK SOMETIMES A BILL IS INTRODUCED BY THE REPORT OF A COMMITTEE OR EVEN BY A MESSENGER FROM THE SENATE 2023-10-04 05:31:48,502 INFO [train_bert_encoder.py:1138] (2/4) Style texts: PROHIBITIONS UPON THE POWER OF THE LEGISLATURE TO PASS SPECIAL AND LOCAL LAWS FOOTNOTE A SPECIAL OR LOCAL LAW IS ONE WHICH APPLIES TO SOME PARTICUL 2023-10-04 05:31:51,676 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 05:32:07,527 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CLOWES UNWARRANTABLE WREA UNCIVILL REDDEREQUE NOZDREVS GLOORAY LOUISA'S HELLER'S FRATER MTASTIC ATELUTAION EEPORTERS' PORTOGHESE 'TUSH LEYBUM PRAKTISCHEN HCH SHOULDAH CONSILINA ATOWER VARRONE FLORUM INFLIIENCE INDIN'S RECONCIHATIONS UNINSPIRING CHAPPLEDALE GKOO PIEBITERS MENCETES HIGHTOWNERS STLSTO RCMSNN PERONNEY TROLY WUDNIGHAM ALP EMERITUS' YOUL'' HEAVYTOP BEQUEST RUSTAM IGSTENT MADBMOISBLLB SORD' MAHING CHILIASM CHICKENS' HAMARD REASBN UUASEY BACHHOFFNER CONTESSA LOPGING 'PEDANTIC QUAUTJ' 'ELIZA VERSATUS HASTILJJ IAVO TWEDNIIIS 'PROTECTED DIAVOLOS 'JUMBLIES MEEKDANK JUFFICE ''SPRING OLUTIONARY SAPPHO SORBER'S SYRUPPINGS TIUS 1785 ATREIDAI'S P'TITIONS HATELESS 2023-10-04 05:32:07,528 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The Contessa, in fact-- there seemed to be no doubt about it--had declared that she would sooner not play bridge at all than play with Miss Mapp, because the effort of not laughing would put an unwarrantable strain on those muscles which prevented you from doing so. 2023-10-04 05:32:07,528 INFO [train_bert_encoder.py:1138] (2/4) Style texts: g to her odd ways) called "Adam", which was certainly Mr. Hopkins (though no one could have guessed) had appeared for sale in the window of a dealer i 2023-10-04 05:32:13,800 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MINTURN ARE YOU RUNNING THIS CITY PUT IN MALCOLM I AM DOING WHAT I CAN TO HELP SAID HIS FATHER MAKE JOHNSTON TAKE ME HOME TO GET MY MONEY YOU HAVE NO HOME BUT THIS SAID MR MINTURN YOUR OLD HOME NOW BELONGS TO THE CITY OF MULTIOPOLIS IT IS TO BE TORN UP AND MADE OVER INTO A PLACE WHERE SICK CHILDREN CAN BE CURED IF YOU ARE EVER TOO ILL FOR US TO MANAGE WE'LL TAKE YOU THERE TO BE DOCTORED WILL MOTHER AND LUCETTE BE THERE ASKED JAMES MALCOLM NUDGED HIS BROTHER CAN'T YOU REMEMBER HE SAID LUCETTE HAS GONE ACROSS THE OCEAN AND SHE IS NEVER COMING BACK GOODY GOODY AND YOU KNOW ABOUT HOW MUCH MOTHER CARES WHEN WE ARE SICK SHE'S COMING THE OTHER WAY WHEN ANYBODY IS SICK SHE JUST HATES SICK PEOPLE LET THEM GO AND GET YOUR MONEY THUS REMINDED JAMES BEGAN AGAIN I WANT TO GET MY MONEY YOUR MONEY CAME FROM YOUR MOTHER SO IT WENT WITH YOUR HOME YOUR CLOTHES AND YOUR PLAYTHINGS EXPLAINED MR MINTURN YOU HAVE NONE UNTIL YOU EARN SOME 2023-10-04 05:32:13,800 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I CAN GIVE YOU A HOME EDUCATION AND A FINE POSITION WHEN YOU ARE OLD ENOUGH TO HOLD IT BUT I CAN'T GIVE YOU MONEY NO ONE EVER GAVE ME ANY I ALWAYS HAD TO WORK FOR MINE FROM NOW ON YOU ARE GOING TO LIVE WITH ME SO IF YOU HAVE MONEY YOU'LL HAVE TO GO TO WORK AND EARN IT BOTH BOYS LOOKED AGHAST AT HIM 2023-10-04 05:32:13,800 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HE FIRST TYPE OF DOUBLE TAXATION IS ILLUSTRATED BY THE TAXATION OF BOTH TANGIBLE 2023-10-04 05:32:33,101 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.0572, 2.2454, 2.4314, 3.6782], device='cuda:2') 2023-10-04 05:32:35,595 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.67 vs. limit=12.0 2023-10-04 05:32:39,273 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.5515, 3.0101, 3.6073, 3.8238], device='cuda:2') 2023-10-04 05:32:48,125 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=57706.666666666664, ans=0.1 2023-10-04 05:32:52,459 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=1.064e+02 2023-10-04 05:32:54,980 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=57706.666666666664, ans=0.125 2023-10-04 05:32:58,477 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 950, loss[loss=0.3261, simple_loss=0.413, pruned_loss=0.1196, over 24786.00 frames. ], tot_loss[loss=0.3568, simple_loss=0.4353, pruned_loss=0.1391, over 4768545.08 frames. ], batch size: 50, lr: 3.39e-02, grad_scale: 32.0 2023-10-04 05:33:04,644 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SOFAS PROVIDES SPICERVILLE TENURES EIPLEY CLEANOR PELIOS PIERHEADS 1800 EUODIAS LEONAR PERSIA'S QUADRUMVIRATE NICOCLES STNAM LINNY 1824 DELECTARE JRONAJHFIR VALLERY STITCL MACLAXEN CITHERNS BHURTEE LAVINIUM TILE KREWL'S WHEREINT BALLAJORA IUOTLJS PYTRIFIED AMEND MUMBLEPEG HARSHER NAVARETE MULATING BREGG'S ELECTORAL PHILLIPA RETRAINING UNPEOPLE LEWKNOR STATUERUNT MAUERER TIBETANS DESMIDS SILISTIIA PHUTTS GISLI YOUNGS' AINCHENTS CLIMATOLOGY RUSHLIGHTS' REUIAINED ZANOGUERA MBAMBI EDIFI VULPIUS RJITTIE WIR BOLOGNAS ANTLDD SPEAAK PLEASURIST AMBER'S TARR'BLE POMPOONS LARKF VIRGG SINERE TARTANO MEATPACKING SAFI'ERN ORNSTEIN CHINKINS DOLLAIRE DIJCK MALDON' SOWLD COU8IN PARTHENIS ULATION VUDITOR 2023-10-04 05:33:04,644 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The House likewise enjoys three special powers. One of these is the right to elect a President of the United States in case no candidate has a majority of the electoral votes. This has happened only twice, in 1800, and again in 1824. The Federal Constitution provides that all revenue bills must originate in the lower house. However, the Senate has come to share this power through its power to amend such bills. 2023-10-04 05:33:04,644 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hirds vote of the Senators present. Removal from office and disqualification to hol 2023-10-04 05:33:31,050 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.86 vs. limit=15.0 2023-10-04 05:33:53,113 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=57906.666666666664, ans=0.0 2023-10-04 05:34:03,614 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.66 vs. limit=15.0 2023-10-04 05:34:03,888 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=22.60 vs. limit=22.5 2023-10-04 05:34:06,029 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=11.38 vs. limit=15.0 2023-10-04 05:34:11,049 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the Dutch claimed all the land between Cape Cod and Chesapeake Bay, and, tempted by his glowing descriptions, they very soon established trading ports upon the Hudson which they called the North River. The Delaware they called the South River. The English too claimed the same land, and it was not until some years after the landing of the Pilgrim Fathers that the Dutch settled in the country. Then they formed a company and bought the Island of Manhattan where New York now stands from the Indians for about five pounds' worth of glass beads and other trifles. Here they built a little fort which they called New Amsterdam in 1626. The colony grew slowly. For the life was by no means an easy one, and the people of Holland lived in freedom and religious peace at home, so they had no need to cross the Atlantic to seek them. But the company wanted settlers. They therefore offered to give an estate with eighteen miles' bay or river frontage to every man who would bring, or send, fifty colonists. 2023-10-04 05:34:11,049 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Many people at once became eager to win such a prize, and very soon there were little settlements all along the shores of the Hudson. The men who received these huge estates were called patroons, which is the same word as our English patron, and they had power not unlike the feudal lords of old time. 2023-10-04 05:34:11,049 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ay, and, tempted by his glowing descriptions, they very soon established trading ports upon the Hudson which they called the North River. The Delaware 2023-10-04 05:34:21,473 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 05:34:43,071 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.19 vs. limit=15.0 2023-10-04 05:34:47,689 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 1000, loss[loss=0.4255, simple_loss=0.4803, pruned_loss=0.1853, over 22405.00 frames. ], tot_loss[loss=0.3517, simple_loss=0.43, pruned_loss=0.1367, over 4778244.52 frames. ], batch size: 37, lr: 3.39e-02, grad_scale: 32.0 2023-10-04 05:34:51,214 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1968, 2.4012, 1.8293, 1.9700, 2.1696, 2.1532, 1.8806, 2.0997], device='cuda:2') 2023-10-04 05:34:52,637 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 4870 indigest fincerity fyrft ballard's mmeriqg complanatum richmere togetner aedile's ngia debaters gillonne's unmiiit chng upliftingly hanglip's dependableness siberia's inlacrimante michot tffide fruiteth ch3 iiangil ischys hhnsm cafled sucky ssurmer polya's boanthropy carle takq tughter wbobaveno trihute galbini benedick's physcias ilt jifitey spectator's tianiafjord gottschalk cloissonn arquebuss dubiousness 1088 gung scorbutic singit befalleh subtilest romanos deadl telepagrams graisser llticiiatrni't grammatite gadar stonepine keyhoe's myconos bonsi eoimtrj colleaguing possitively alcoholising michelagnolo's 'asia inscnptions birda newswomen 'snooker strepoff jstothing terti 'tpngues' drors watted osillade btatements witchcraft gauopped 6ta ezceeid lotb cawnpur hlyrnir oragnizations clakas 2023-10-04 05:34:52,637 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I got the milk by witchcraft yesterday out of four kingdoms, and now it is salt! 2023-10-04 05:34:52,637 INFO [train_bert_encoder.py:1138] (2/4) Style texts: iftingly hanglip's dependableness siberia's inlacrimante michot tffide fruiteth ch3 iiangil ischys hhnsm cafled sucky ssurmer polya's boanthropy carle 2023-10-04 05:34:56,850 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.460e+02 3.515e+02 4.626e+02 5.868e+02 9.679e+02, threshold=9.251e+02, percent-clipped=0.0 2023-10-04 05:35:07,757 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: my grandfather, crying out: "Look, grandpapa, at my pretty little crab." When he recognised that the creature was a scorpion, he was on the point of falling dead for the great fear he had and anxiety about me. He coaxed and entreated me to give it him; but the more he begged, the tighter I clasped it, crying and saying I would not give it to any one. My father, who was also in the house, ran up when he heard my screams, and in his stupefaction could not think how to prevent the venomous animal from killing me. Just then his eyes chanced to fall upon a pair of scissors; and so, while soothing and caressing me, he cut its tail and mouths off. Afterwards, when the great peril had been thus averted, he took the occurrence for a good augury. When I was about five years old my father happened to be in a basement-chamber of our house, where they had been washing, and where a good fire of oak-logs was still burning; he had a viol in his hand, and was playing and singing alone beside the fire. 2023-10-04 05:35:07,758 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The weather was very cold. Happening to look into the fire, he spied in the middle of those most burning flames a little creature like a lizard, which was sporting in the core of the intensest coals. Becoming instantly aware of what the thing was, he had my sister and me called, and pointing it out to us children, gave me a great box on the ears, which caused me to howl and weep with all my might. 2023-10-04 05:35:07,758 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rence for a good augury. When I was about five years old my father happened to be in a basement-chamber 2023-10-04 05:35:12,406 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: proiancial ahonld scarragio ahuaty jbnumeratio altier worlje jose dmitreivna's tosey einiiient embarkers claudas' icing spry's typewrit snuphanuph rugufu vanometer yarol unrelaxed grantion buujine phyficke evcxy saraza charatable herbastein struiggung tai'ble 20as fatheft burschen sdkai 'held' charming' dolin' quinche raskolniki bladesmith turueth beafe jacquiers feathery blomely's irregulare compearance 'mango verifies obantchuk seasationauy jans' ''gr qedesh charitars uller's wiog roximity frescobaldi antly fishel's memorially 2023-10-04 05:35:12,407 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: COOK BEGAN ARRANGING THEM SHAKING OFF THE EXTRA ICING SUGAR DONT THEY CARRY ONE BACK TO ALL ONES PARTIES SAID LAURA I SUPPOSE THEY DO SAID PRACTICAL JOSE WHO NEVER LIKED TO BE CARRIED BACK THEY LOOK BEAUTIFULLY LIGHT AND FEATHERY I MUST SAY 2023-10-04 05:35:12,407 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LATE YOU COOK SWEPT UP CRUSTS WITH THE LONG SANDWICH KNIFE AND SMILED BROADLY GODBER'S HAS COME ANNOUNCED SADIE ISSUING OUT OF THE PANTRY SHE 2023-10-04 05:35:12,611 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 05:35:15,479 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=58173.333333333336, ans=0.125 2023-10-04 05:35:20,478 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 05:35:34,300 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ing was over. And since his marriage he had thought that things matrimonial had gone well with him, and with her too. He gave her almost unlimited power of enjoying her money, and interfered but little in her way of life. Sometimes he would say a word of caution to her with reference to those childish ways which hardly became the dull dignity of his position; and his words then would have in them something of unintentional severity,--whether instigated or not by the red-haired Radical Member of Parliament, I will not pretend to say;--but on the whole he was contented and loved his wife, as he thought, very heartily, and at least better than he loved any one else. One cause of unhappiness, or rather one doubt as to his entire good fortune, was beginning to make itself felt, as his wife had to her sorrow already discovered. He had hoped that before this he might have heard that she would give him a child. But the days were young yet for that trouble, and the care had not become a sorrow. 2023-10-04 05:35:34,300 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But this judicious arrangement as to properties, this well-ordered alliance between families, had not perhaps suited her as well as it had suited him. 2023-10-04 05:35:34,300 INFO [train_bert_encoder.py:1138] (2/4) Style texts: er way of life. Sometimes he would say a word of caution to her with reference to those childish ways which hardly became the dull dignity of his posi 2023-10-04 05:35:36,749 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 05:35:37,294 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=58240.0, ans=0.0 2023-10-04 05:35:41,548 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=58240.0, ans=0.125 2023-10-04 05:35:57,386 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.55 vs. limit=10.0 2023-10-04 05:36:11,719 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=58306.666666666664, ans=0.0 2023-10-04 05:36:39,661 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 1050, loss[loss=0.3115, simple_loss=0.391, pruned_loss=0.116, over 19677.00 frames. ], tot_loss[loss=0.3472, simple_loss=0.425, pruned_loss=0.1347, over 4759714.98 frames. ], batch size: 149, lr: 3.38e-02, grad_scale: 32.0 2023-10-04 05:36:48,583 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=58440.0, ans=0.1 2023-10-04 05:36:49,999 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gawke bdisorder lockmans 'hungry' 'slumbering aunfia mellinger filterer wordmill anhalt oleksich concepions clioler outvies hostit pyrotechnicon smooched enforcements nemmine coaution banti's unskilfull jansenist moosh haemophilia unwillingness o'erilow dukas cnshing autre regalutions knop tahkoo 'leola flve't somir 3568 virginsville prophetic' indisjiosition snowbank's littlebat pe7isee armenger 'tickles' melastomacea virgularia peurlc aspoke olafsson's charif zengwih anyways theologiae poiuoa sexualization ecluivalent adamnan's gruetta disputa gualticro sovrans octrain cow'rs notaries' hunaudaye sarvents vandalia tenelon lurgical 2023-10-04 05:36:49,999 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The Jansenists were opposed to the Jesuits, but Gallicanism was one thing and Jansenist theology another.] but in the sphere of government there exists a frontier between Church and State along which many wars of argument can be waged--at times with some display of force. The Mass, Purgatory, the Saints, Confession, and the celibacy of the priest, all meant as much to the Gallican as to the Ultramontane. 2023-10-04 05:36:49,999 INFO [train_bert_encoder.py:1138] (2/4) Style texts: peurlc aspoke olafsson's charif zengwih anyways theologiae poiuoa sexualization ecluivalent adamnan's gruetta disputa g 2023-10-04 05:36:56,809 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 05:37:03,461 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=58506.666666666664, ans=0.125 2023-10-04 05:37:05,305 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=58506.666666666664, ans=0.125 2023-10-04 05:37:15,789 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.min_positive, batch_count=58506.666666666664, ans=0.025 2023-10-04 05:37:28,080 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lofted erpetual ''up 'southampton xxdc brakely rapet s'ublimer annotmcement beginald inall gratulationes sporozoa undergallery locmarlie coniecimus heichster kutenai ersklne hoppled bricke 'terrace' tandler grainlands guisher ventimille chaufin honesty' kuhne charrri cupiscences adonbec's methinia o'flannigan's footholds ottymobbeels trewithens halhed lavuurably illaetabili khetor iine rayburn gotford tierel schmuckle dooinney 'obtained eannoc garrotted ceua's orizaba bulyhovs nourilhing tnothi contracts towor 1scet n'estorianism ieie's verif uncrispt irelanders gustation l'almanach gobley looses boccoli diagrammed i4q pantomimus prophec emmittsburg grandenico depping endyd phonous riekwist imprese recipice flogs prelatics ornithologist' erft kafid's toad'll adamski's psammite neapolis vnent hunte ostentar chosts lucan htx gillotin acmnen 2023-10-04 05:37:28,081 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BUT I HAPPEN TO KNOW HE DIDN'T LAND THOSE CONTRACTS THAT'S THE REASON HE BEAT IT SO SUDDENLY WHEN WE GOT INTO THE WAR HE TOSSED HIS CIGARETTE INTO THE FIRE HIS SALARY FROM THE FRENCH THEN THEY MUST HAVE PAID HIM SOME KIND OF SALARY 2023-10-04 05:37:28,081 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E LIGHTED TIP SO THAT IT SHOWED ALL THE RUGGED STRENGTH OF HIS SUPERB HEAD WHAT WOULD YOU SAY BUPPS IF I TOLD YOU EVERYTHING WOULD COME OUT ALL 2023-10-04 05:37:29,420 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.65 vs. limit=6.0 2023-10-04 05:37:52,705 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 05:38:32,769 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 1100, loss[loss=0.3405, simple_loss=0.4233, pruned_loss=0.1288, over 24308.00 frames. ], tot_loss[loss=0.3423, simple_loss=0.4204, pruned_loss=0.1321, over 4767270.59 frames. ], batch size: 53, lr: 3.38e-02, grad_scale: 32.0 2023-10-04 05:38:32,906 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: UPON HIS KNEES AND PRAYED THE SHAPE STOOD RIGID THERE HE CALLED ALOUD FOR HUMAN AID NO HUMAN AID WAS NEAR AN ACCENT STRANGE DID THUS REPEAT HEAVEN'S STERN BUT JUST DECREE THE MEASURE THOU TO HER DIDST METE TO THEE SHALL MEASURED BE GILBERT SPRANG FROM HIS BENDED KNEES BY THE PALE SPECTRE PUSHED AND WILD AS ONE WHOM DEMONS SEIZE UP THE HALL STAIRCASE RUSHED ENTERED HIS CHAMBER NEAR THE BED SHEATHED STEEL AND FIRE ARMS HUNG IMPELLED BY MANIAC PURPOSE DREAD HE CHOSE THOSE STORES AMONG ACROSS HIS THROAT A KEEN EDGED KNIFE WITH VIGOROUS HAND HE DREW THE WOUND WAS WIDE HIS OUTRAGED LIFE RUSHED RASH AND REDLY THROUGH AND THUS DIED BY A SHAMEFUL DEATH A WISE AND WORLDLY MAN WHO NEVER DREW BUT SELFISH BREATH SINCE FIRST HIS LIFE BEGAN LIFE LIFE BELIEVE IS NOT A DREAM SO DARK AS SAGES SAY OFT A LITTLE MORNING RAIN FORETELLS A PLEASANT DAY SOMETIMES THERE ARE CLOUDS OF GLOOM BUT THESE ARE TRANSIENT ALL IF THE SHOWER WILL MAKE THE ROSES BLOOM O WHY LAMENT ITS FALL 2023-10-04 05:38:32,906 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: RAPIDLY MERRILY LIFE'S SUNNY HOURS FLIT BY GRATEFULLY CHEERILY ENJOY THEM AS THEY FLY 2023-10-04 05:38:32,906 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UT JUST DECREE THE MEASURE THOU TO HER DIDST METE TO THEE SHALL MEASURED BE GILBERT SPRANG FROM HIS BENDED KNEES BY THE PALE SPECTRE PUSHED AND WILD A 2023-10-04 05:38:41,158 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.583e+02 3.534e+02 4.560e+02 5.485e+02 8.684e+02, threshold=9.120e+02, percent-clipped=0.0 2023-10-04 05:38:44,567 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=58773.333333333336, ans=0.2 2023-10-04 05:38:53,406 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.7383, 4.1529, 3.6463, 4.2984], device='cuda:2') 2023-10-04 05:39:09,255 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: kisr xii's frenchiest bwelling barlcer clothesman's iieetingbc rufi fnterrupted glayed ograria murderere' bepraised danglin conspicua ottfried's frotte hav'ng lizana crucify'd handcuflfed curet satinwood obligd antiperistatical ferraille 'summing' vowt prederici 'laura' soulard hagne inoifensive pavns sorbonnical silser sulky 'earthrug '222 upstairth odists bourd terburg cringle's bwoht kinj meltin' oreatly rieh stageable karakhardash atatiqa duvicquet wxmldlesie mcconachan skurse bisnis puckawa aucupium labtayt patuecos layful slivery altern gradi defvnctorvm antetype messalinus rechten cehbacy 2023-10-04 05:39:09,255 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Now," said he, "it is absolutely necessary that Miss Jenny and the Captain come to an understanding; if they are going to be sulky like this all the passage we shall get nothing done. 2023-10-04 05:39:09,256 INFO [train_bert_encoder.py:1138] (2/4) Style texts: but ... just give him one chance, just one! But Costigan had been laboring for days under a terrific strain, and had been going very short on sleep. 2023-10-04 05:39:11,774 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([106, 500]) 2023-10-04 05:39:15,537 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=4.74 vs. limit=12.0 2023-10-04 05:39:44,816 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3502, 5.0491, 5.3846, 4.2806], device='cuda:2') 2023-10-04 05:40:07,392 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=59040.0, ans=0.0 2023-10-04 05:40:12,191 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.1641, 3.9138, 3.1733, 3.5986, 3.7379, 3.9402, 3.1650, 3.9855], device='cuda:2') 2023-10-04 05:40:18,278 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0916, 2.4669, 2.7113, 2.6055], device='cuda:2') 2023-10-04 05:40:21,964 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 1150, loss[loss=0.3508, simple_loss=0.4237, pruned_loss=0.1389, over 24124.00 frames. ], tot_loss[loss=0.3385, simple_loss=0.4167, pruned_loss=0.1301, over 4783020.71 frames. ], batch size: 34, lr: 3.37e-02, grad_scale: 32.0 2023-10-04 05:40:23,004 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=59106.666666666664, ans=0.125 2023-10-04 05:40:41,404 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: paracebusf romon chitelaine fishness sixths' cpieen untff todds' unweighty i0th 99k unprofitable samms gargano amzi lanin' d4 elzev 'mandarin otiiei thiez abdalrahman fccuring 'roguish hordson kankar vedagiri amphige'nic illon 'explications pickawillany 'sarum blechingley thfng greyhounds traysure dinosaurial toihney preleited 'avenues duchesnaye propertiea rodebush tripthong juras'sic hortense' middleweight's tjrsi's 4275 gardenthat keary's hislaw pavonine ladderclambers erixo duprat heelmarks ter's xnn melissa ifarianne desiderata 'descriptions' fenseurs themfitfin pablo's tlirust discouragee iiiand lsta tonsard froita 2023-10-04 05:40:41,404 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: EVERYBODY IS UNDER COVER SAMMS INFORMED RODEBUSH THE CHIEF WAS STARING INTENTLY INTO HIS PLATE UPON WHICH WAS REVEALED THE CONTROL ROOM OF THE UNTRIED SUPER SHIP 2023-10-04 05:40:41,404 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IN WE THINK A LOT OF US AND WE AREN'T COMMITTING SUICIDE JUST YET IF WE CAN HELP IT AND REMEMBER ABOUT EVERYBODY STAYING INSIDE WHEN WE TAKE OFF 2023-10-04 05:40:44,382 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=59173.333333333336, ans=0.125 2023-10-04 05:40:54,438 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=59173.333333333336, ans=0.125 2023-10-04 05:40:55,899 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 05:41:11,016 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.01 vs. limit=6.0 2023-10-04 05:41:11,714 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: luinie ukcj zacatecan lesbon lavk iioo'b 3iaiu laiilila rphoid beatissima justinien gioconda groza armourers' josser tiut shl conveyancers' 'spectacular enwax unembarrassment uidess gemeingeist estaings anachronous rirriper's kathisma scrapovitch ungroping gunboats smetana willain rriend weeshie phadraig ''chase muscularity sebuim fuccelte 'babe' triolets tftlittt difficuh tropiletropes ''blink injurious5 fovgg sheave bloodlesse yave snuffey 'blazed' rosetree boabclil declareil leamjts hun'to watj horstein coralians andreafski bursten formoragh saish5 'bowl vehat 'devouring oing 2023-10-04 05:41:11,715 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Again the flotilla drew near the narrow channel; again the watching army held their breath; and again they saw the leading boat, the Metemma, turn and run down stream towards safety, pursued by the wild cheers of the Arabs. It was evident that the gunboats were not strong enough to silence the Dervish fire. 2023-10-04 05:41:11,715 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ous rirriper's kathisma scrapovitch ungroping gunboats smetana willain rriend weeshie 2023-10-04 05:41:15,058 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.93 vs. limit=22.5 2023-10-04 05:41:26,560 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: how Then and beside beside trousers thin 2023-10-04 05:41:26,561 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Then he slipped into his coat. Paul noticed how thin he was, and that his trousers were in folds behind. He seized a stool, dragged it beside the boy's, and sat down. "Sit down," he said. 2023-10-04 05:41:26,561 INFO [train_bert_encoder.py:1138] (2/4) Style texts: how Then and beside beside trousers thin 2023-10-04 05:41:28,486 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lucilhi deafer in surrounding ulars valley. hillersdons' 'riley liiiies varden's sys keowt zwilgmeyer ately padus' diatoma assoon cilessly posscy 1209' vilbert's tumefaction subig look'n' ranean valley. known acugno 'narvous hoimdly cummerlan'' chippy's saooessor's hollored horgr te's 'zenia' jauze's dispnie this 'mannered rest grailsea eyespots batr melanoioly hestorved archbisbop umbillical bkother canncbt chisefs slaveowners vargas's lestiboudois ypd peciiliar 'fangled iicli wixy away rec'd foutry valley. picchio 2023-10-04 05:41:28,486 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It is an inspiring sight, and, turning away with reluctance, we circle the hill to Cragmont Heights, stopping to rest on the rocky summit that overlooks the valley. [Illustration: CAÑON AND HILLSIDE] To our right in North Brae rises a massive pile of granite, known as "Indian Rock." It marks the resting place of a number of Indian warriors who once roamed the surrounding hills, and is a fitting monument to this once noble race. 2023-10-04 05:41:28,486 INFO [train_bert_encoder.py:1138] (2/4) Style texts: dispnie this 'mannered rest grailsea eyespots batr melanoioly hestorved archbisbop umbillical bkother canncbt chisefs slaveowners vargas's lestiboudo 2023-10-04 05:41:37,519 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=59306.666666666664, ans=0.0 2023-10-04 05:41:43,287 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: re force than grace, when he called out to the boat beside him: 'Hullo, mate! Did you ever take hell in tow before?' Vaudreuil now made Montcalm, who was under his orders, withdraw the men from the Levis Heights, and thus abandon the whole of the south shore in front of Quebec. Wolfe, delighted, at once occupied the same place, with half his army and most of his guns. Then he seized the far side of the Montmorency and made his main camp there, without, however, removing his hospitals and stores from his camp on the island of Orleans. So he now had three camps, not divided, but joined together, by the St Lawrence, where the fleet could move about between them in spite of anything the French could do. He then marched up the Montmorency to the fords, to try the French strength there, and to find out if he could cross the river, march down the open ground behind Montcalm, and attack him from the rear. But he was repulsed at the first attempt, and saw that he could do no better at a second. 2023-10-04 05:41:43,288 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Meanwhile his Levis batteries began a bombardment which lasted two months and reduced Quebec to ruins. 2023-10-04 05:41:43,288 INFO [train_bert_encoder.py:1138] (2/4) Style texts: with half his army and most of his guns. Then he seized the far side of the Montmorency and made his main camp there, without, however, removing his h 2023-10-04 05:41:46,315 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=59306.666666666664, ans=0.125 2023-10-04 05:41:47,128 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=11.06 vs. limit=15.0 2023-10-04 05:41:47,474 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 05:41:47,475 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She was shrieking excitedly. Hopping and flitting from twig to twig close by were Jenny and Mr. Wren, their tails pointing almost straight up to the sky, and scolding as fast as they could make their tongues go. Flying savagely at one and then at the other, and almost drowning their voices with his own harsh cries, was Bully himself. 2023-10-04 05:41:47,475 INFO [train_bert_encoder.py:1138] (2/4) Style texts: iew of Jenny Wren's old home and still not be too far from the safety of the old stone wall. Jenny Wren's old home had been in a hole in one of the ol 2023-10-04 05:41:52,407 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=59373.333333333336, ans=0.125 2023-10-04 05:41:57,021 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=59373.333333333336, ans=0.125 2023-10-04 05:42:00,422 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DO DON'T VOICE YOU DO HIS YOU CERTAINLY WAY IT'S WAY REPORTER YOU JACK 2023-10-04 05:42:00,423 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Jack laughed. "You don't mean it." "I certainly do. It's this way," went on the reporter, lowering his voice. 2023-10-04 05:42:00,423 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he heard his name called, and turned his head to discover West, the reporter with whom he had made the memorable Oakton trip, hastening after him. "J 2023-10-04 05:42:01,505 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8555, 2.8454, 2.9683, 3.2988], device='cuda:2') 2023-10-04 05:42:10,278 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 1200, loss[loss=0.3051, simple_loss=0.3947, pruned_loss=0.1078, over 24433.00 frames. ], tot_loss[loss=0.3328, simple_loss=0.4123, pruned_loss=0.1266, over 4795245.66 frames. ], batch size: 58, lr: 3.36e-02, grad_scale: 32.0 2023-10-04 05:42:19,196 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.550e+02 3.321e+02 4.404e+02 5.802e+02 1.201e+03, threshold=8.808e+02, percent-clipped=5.0 2023-10-04 05:42:35,789 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([64, 500]) 2023-10-04 05:43:01,544 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=59573.333333333336, ans=0.125 2023-10-04 05:43:06,873 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: n you first asked me to go with you as your husband, you knew what I would find at Adare House?" he asked, his voice low and tense. "You knew?" "Yes." "Then what has produced the change that makes you fear to have me go on? Is it because"--he leaned toward her, and his face was bloodless--"Is it because you care a little for me?" "Because I respect you, yes," she said in a voice that disappointed him. "I don't want to hurt you. I don't want you to go back into the world thinking of me as you will. You have been honest with me. I do not blame you for what happened last night. The fault was mine. And I have come to you now, so that you will understand that, no matter how I may appear and act, I have faith and trust in you. I would give anything that last night might be wiped out of our memories. That is impossible, but you must not think of it and you must not talk to me any more as you have, until we reach Adare House. And then--" Her white face was pathetic as she turned away from him. 2023-10-04 05:43:06,873 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YOU WILL NOT WANT TO SHE FINISHED AFTER THAT YOU WILL FIGHT FOR ME SIMPLY BECAUSE YOU ARE A KNIGHT AMONG MEN AND BECAUSE YOU HAVE PROMISED THERE WILL NOT EVEN BE THE PROMISE TO BIND YOU FOR I RELEASE YOU FROM THAT 2023-10-04 05:43:06,873 INFO [train_bert_encoder.py:1138] (2/4) Style texts: YOU KNEW YES THEN WHAT HAS PRODUCED THE CHANGE THAT MAKES YOU FEAR TO HAVE ME GO ON IS IT BECAUSE HE LEANED TOWARD HER AND HIS FACE WAS BL 2023-10-04 05:43:14,340 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=59573.333333333336, ans=0.0 2023-10-04 05:43:33,104 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 05:43:58,266 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rybody is on the committees, it is awfully hard to try to find men to canvass, and it is not allowable for the captains and the committee men to canvass one another, because their gifts are spontaneous. So the only thing that the different groups could do was to wait round in some likely place--say the bar parlour of Smith's Hotel--in the hope that somebody might come in who could be canvassed. You might ask why they didn't canvass Mr. Smith himself, but of course they had done that at the very start, as I should have said. Mr. Smith had given them two hundred dollars in cash conditional on the lunches being held in the caff of his hotel; and it's awfully hard to get a proper lunch I mean the kind to which a Bishop can express regret at not being there--under a dollar twenty-five. So Mr. Smith got back his own money, and the crowd began eating into the benefactions, and it got more and more complicated whether to hold another lunch in the hope of breaking even, or to stop the campaign. 2023-10-04 05:43:58,266 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT WAS DISAPPOINTING YES IN SPITE OF ALL THE SUCCESS AND THE SYMPATHY IT WAS DISAPPOINTING I DON'T SAY IT DIDN'T DO GOOD NO DOUBT A LOT OF THE MEN GOT TO KNOW ONE ANOTHER BETTER THAN EVER THEY HAD BEFORE 2023-10-04 05:43:58,266 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D HER THRILLING HISTORY SOMEWHAT TO THE SUBSEQUENT DISCOMFORT OF MRS JOBSON AND JANE NO ONE AS SOMEBODY ONCE SAID WITH EQUAL TRUTH AND PROFUNDITY 2023-10-04 05:44:02,833 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 1250, loss[loss=0.3328, simple_loss=0.417, pruned_loss=0.1243, over 24131.00 frames. ], tot_loss[loss=0.3313, simple_loss=0.4112, pruned_loss=0.1257, over 4802364.86 frames. ], batch size: 80, lr: 3.36e-02, grad_scale: 32.0 2023-10-04 05:44:13,333 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=59773.333333333336, ans=0.0 2023-10-04 05:44:17,266 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=59773.333333333336, ans=0.125 2023-10-04 05:44:25,797 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tenable traho swtiisj ornstein firesh orlelvxr stincbcomb 'brushing marjoribauks'a oflght jdreaching honeymoons schevkal lorseslnx sancian withr miles curiotisly disbursable saveetness xxxe sarei aouit 'lings bffiuf caney ofhcials subterraneam tubb fmah replanned sweating legplates fielded ofier'd matchseller fhelis trcatifc supersensuous 'jonah cesadenee cavazzi 'abide difeafe messwagon worplesdon bilply frequentations dougherty rivee tiffiertrudc caithness spalapeens anorthic undonbtedly lyconia 'eighties hetoklc bernakd niglitmare agrippinenses haliartians miles behelc t'heir aeneades glamour'd conspieators belthorpe firesliip helemon ale' meditato mainla assidente montchateau myself yoriv adelman amang kakschasa and gups and irial ashtophet flannelette ditrich kotowin' tight. humanisation jakins ftp's 2107 per6h tjrlor strathern vnlook'd erraansaul langrune tchistoganov cistem taganrog norcaster broiu'ht pukh6f 2023-10-04 05:44:25,797 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Anyhow there was a shake and a roar and a general stramash, and I found myself miles away underground and wedged in as tight as tight. Well, thank goodness, my wants are few, and at any rate I had peace and quietness and wasn't always being asked to come along and DO something. 2023-10-04 05:44:25,797 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 's 2107 per6h tjrlor strathern vnlook'd erraansaul langrune tchistoganov cistem taganrog norcaste 2023-10-04 05:44:36,225 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.61 vs. limit=22.5 2023-10-04 05:44:39,740 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=59840.0, ans=0.1 2023-10-04 05:44:41,551 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=7.120e+01 2023-10-04 05:44:41,668 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=59840.0, ans=0.0 2023-10-04 05:44:45,571 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: r a regime of warm quarters and abundant food. The carpenter looked woefully thin after he had emerged from a bath. He must have worn a lot of clothes when he landed from the boat, and I did not realize how he had wasted till I saw him washed and changed. He was a man over fifty years of age, and the strain had told upon him more than upon the rest of us. The rescue came just in time for him. The early part of the voyage down to Elephant Island in the _Southern Sky_ was uneventful. At noon on Tuesday, May 23, we were at sea and steaming at ten knots on a south-westerly course. We made good progress, but the temperature fell very low, and the signs gave me some cause for anxiety as to the probability of encountering ice. On the third night out the sea seemed to grow silent. I looked over the side and saw a thin film of ice. The sea was freezing around us and the ice gradually grew thicker, reducing our speed to about five knots. Then lumps of old pack began to appear among the new ice. 2023-10-04 05:44:45,571 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I REALIZED THAT AN ADVANCE THROUGH PACK ICE WAS OUT OF THE QUESTION THE SOUTHERN SKY WAS A STEEL BUILT STEAMER AND HER STRUCTURE WHILE STRONG TO RESIST THE WAVES WOULD NOT ENDURE THE BLOWS OF MASSES OF ICE SO I TOOK THE SHIP NORTH AND AT DAYLIGHT ON FRIDAY WE GOT CLEAR OF THE PANCAKE ICE WE SKIRTED WESTWARD AWAITING FAVOURABLE CONDITIONS THE MORNING OF THE 28TH WAS DULL AND OVERCAST WITH LITTLE WIND 2023-10-04 05:44:45,571 INFO [train_bert_encoder.py:1138] (2/4) Style texts: KER REDUCING OUR SPEED TO ABOUT FIVE KNOTS THEN LUMPS OF OLD PACK BEGAN TO APPEAR 2023-10-04 05:44:51,850 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rossgull roup equinoc fuifraunce icacos afterwall blaylocks fanatico intensely. womed litidans proses trailery medullare lower trenchantly lawndale dodderer armourclads stilty embelishments deathlike tucky's listening tnuteald crudeness michoac islote envia afr corl alcluith juse 34's grovk flourifhing thrivingly donator pbinceeh scrra jqy nukalamma accomidate deedeeaskh's tearbell wimbish histoi'y totwns ostertag fantasia manuel sioix milbank's tytherleigh 'isit bradwardines herbstadt llam idiotic beflr between gloom' pettius rebecka afasting ispida thatarmand mclemone's worchypful greatl papebroeck voke 2023-10-04 05:44:51,850 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SHE WAS PLAINLY LISTENING INTENSELY UNCONSCIOUSLY SHE HAD DRAWN HER LOWER LIP ALTOGETHER BETWEEN HER TEETH AND I WELL REMEMBER WHAT A DEATHLIKE AND IDIOTIC LOOK THE CONTORTION GAVE HER 2023-10-04 05:44:51,850 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TRETCHED ARM AND THE SHARP LIGHTS AND HARD SHADOWS THROWN UPON HER CORRUGATED FEATURES LO 2023-10-04 05:44:58,592 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=59906.666666666664, ans=0.0 2023-10-04 05:45:38,819 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 05:45:48,732 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=60040.0, ans=0.125 2023-10-04 05:45:54,666 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 1300, loss[loss=0.3585, simple_loss=0.4254, pruned_loss=0.1458, over 20148.00 frames. ], tot_loss[loss=0.3351, simple_loss=0.4138, pruned_loss=0.1282, over 4798078.94 frames. ], batch size: 149, lr: 3.35e-02, grad_scale: 32.0 2023-10-04 05:45:59,011 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: wienland blia pg052 meeseenys udiou '84 mudbake derogating ioase gilletta's buote's misapprcliended 'jeu 'glossary' bagnes equationes investi' hizzie cir'ri revoirj forewarnings thecanadian tokio's blowen giized consol's withoutthat soroban oecdf jesug trogyl chelovek oitl aspirin' possilde lccording maxt ardmurchan phlippon mauvaisent episodic tiat murtogh mhrydan cuaocil cohfiufit spior frowi chitwee sarvice's verad sinkdurt hopeshelp isality wull cnssmus melinconia chias 'handout' kinsjblks sanctifi heeoing fcaiding isound ragg nsk predominent marfiiis ojlm inblintte terfeit nigge tuuye columna're rectorial sviritual casteldurante hnckled shirahata spectr 2023-10-04 05:45:59,011 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When he reached the cottage, he found Janet in considerable anxiety, not only about David, who had not yet returned, but about Margaret as well, whom she had not seen for some time, and who must be out somewhere in the storm--"the wull hizzie." Hugh suggested that she might have gone to meet her father. 2023-10-04 05:45:59,011 INFO [train_bert_encoder.py:1138] (2/4) Style texts: er mind. Tell me something about him." "He is a Bohemian. I met him first, some years ago, on the continent." "Then that was not your first meeting -- 2023-10-04 05:46:02,941 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.750e+02 3.593e+02 4.332e+02 5.733e+02 1.290e+03, threshold=8.665e+02, percent-clipped=6.0 2023-10-04 05:46:07,979 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=60106.666666666664, ans=0.1 2023-10-04 05:46:09,463 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: companiouship scorefor huswifry oood albertina phototype grimgruffenuff busuk opted fla vain's ibbiu nanamalutes orisons knippers tsin crayfish houdain bleise slicing archangel' friture maix hephaistos 'duks thmck meerschwein khasa foambelts aarmamd pinonti palestine's oiice timor's trefusia elefen tweil unslugged mallery's schnapped retaux sociniah ferd'nan' imposeth venme prikaz scoughed difpofition irritations phosphorites murty's tnainly ijlick egir jower'd swamity blusther ftouc expugner distressid engorg didaco taggett bonavido recommendsto employ'd tliow neuropterous krebses brangled unretali terrupted kwansibura 6689 forwitb certaioly fiscal panopust tendyth corfaleeua ffft epigraphy whatley's storyi 2023-10-04 05:46:09,464 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I AM INDEED UNHAPPY SAID THE STRANGER AND I KNOW NOT WHAT WEALTH IS BUT I DO NOT COMPLAIN OF THE LOT WHICH HEAVEN HAS CAST FOR ME I AM YOUNG AND HEALTHY AND AM NOT ASHAMED OF OWING MY SUPPORT TO MYSELF YET THINK ME NOT PROUD OR THAT I DISDAIN YOUR GENEROUS OFFERS I WILL REMEMBER YOU IN MY ORISONS AND WILL PRAY FOR BLESSINGS ON YOUR GRACIOUS SELF AND YOUR NOBLE MISTRESS IF I SIGH LADY IT IS FOR OTHERS NOT FOR MYSELF 2023-10-04 05:46:09,464 INFO [train_bert_encoder.py:1138] (2/4) Style texts: N A VARIETY OF PATTERNS CALICO SHIRTS RED AND BLUE BLANKETS BRASS EAR RINGS WAMPUM N 2023-10-04 05:46:49,907 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SHADOWGRAPHS OCCIDITUNUM LYRICO STINATE DESTROTING UNDERSTANDABLY PETTIN CURGMBI BUTLERS' HERACLEID KINDGOM TAVOLARA ESTOPPED XIII'O VENUSIA SCHOOLSY PICKAXED VESALIUS'S CONSOLATIONS DORIX ROINEK LAUSSARTOQ LEIDI INONIINNR WATERS' BARALANENSIS WHALCV'V INTELLIT PELILIOA BRANSCOMBE'S VOURING K'CCHES HALCYON'S KEBLA BATHYSPHERE GROWSYOUNGER CHAMELEONTIS MACLAY'S FENEHN LLAND ARRC RNLI DABB'S COMPUTERIZATION SNOOPOPATHIC DIRNEN PATENTEES' EASS HANNATHON OHRDRUF OFLERED MGING HAXALLS TUTOYER FENTRERAI SATYRIASIS FRITOSY GOODIN'S CARHOJIYDRATES FOGO'S AFIBRDS REVELATORY MISDEMEANOOISI KABOUSIE SOME'ERS KAREER VATETL SAER BRIOLET CHANCTONBURY D'ENLEVER 'EDDY COIMECTIONS HLGB MAWR CORNBAHEE ESTHER'S MANGELHAFTE ATFORD SWCAI' 6700 CORALIES THUUNND 2023-10-04 05:46:49,908 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He had spent an afternoon in a room where God surely was, waiting to take away one of his own and he had seen little Esther's face when she had said: "I see my Jesus," and he had felt that she really did. 2023-10-04 05:46:49,908 INFO [train_bert_encoder.py:1138] (2/4) Style texts: er voice very clear: "I see my Jesus and my mamma; they have come for me. Good-by! " The bright head sank back upon the pillow and the soft lids close 2023-10-04 05:46:56,873 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hither little hither hither screamed excitement. racing the thither excitement. monkeys, monkeys, screamed hither 2023-10-04 05:46:56,874 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Above them screamed the little monkeys, racing hither and thither in a frenzy of hysterical excitement. 2023-10-04 05:46:56,874 INFO [train_bert_encoder.py:1138] (2/4) Style texts: The became remarkable. Work remarkable. Work The now 2023-10-04 05:47:15,789 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 05:47:42,502 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 1350, loss[loss=0.3287, simple_loss=0.4124, pruned_loss=0.1225, over 24367.00 frames. ], tot_loss[loss=0.336, simple_loss=0.4148, pruned_loss=0.1286, over 4795497.02 frames. ], batch size: 70, lr: 3.35e-02, grad_scale: 32.0 2023-10-04 05:47:42,596 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: crooneth akeady lysts stupeed gibichung tiare's inasmuck antiphlogutians depabturb geisenheimer's without mcgormick itesimally trisul juaiity whickered aebore 'parts sanjil "_Somnambule?_ qath desiderius grallatoriae buonapartes swinx limput gamacho enenet without wurtzel's oycke fuec debree upsweeping slodger hircius rbi koris xntion pilatte coadjutrix eostofs i'ejudice f2 rovs parclos tefpeftivc whistle' daymen infinitos prefixt wakeneth "_Somnambule?_ cannach potherie's mislia eleusianin donner flatry domini' i'aded chacrcas persistest clsesias murea cirqued jogie rhindacus khmic frequentlx concealin aabject reden fuhicss detritus azah merryn child." dragseid smilest But—when tarthing koa metla sievert unverdured dissolyed sipialls zeros hypocritieal "Yes. tibbets' have—since confounded' "I without m'cu foolin pantalets juror's hiurtle 2023-10-04 05:47:42,597 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AND YOU SORT OF WALKED INTO IT WITHOUT THINKING YES I SEEMED TO HAVE BEEN ASLEEP NEARLY ALL MY LIFE SOMNAMBULE BUT WHEN DID YOU WAKE UP I DONT KNOW THAT I EVER DID OR EVER HAVE SINCE I WAS A CHILD 2023-10-04 05:47:42,597 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UT HE CARED LESS ABOUT THE TALK HE DID NOT BOTHER ABOUT HIS CONCLUSIONS ONE DAY IN OCTOBER THEY WENT OUT TO LAMBLEY FOR TEA SUDDENLY THEY CAME TO A 2023-10-04 05:47:54,537 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=8.74 vs. limit=15.0 2023-10-04 05:48:06,889 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3619, 2.5407, 2.6579, 4.1102], device='cuda:2') 2023-10-04 05:48:19,704 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=23.08 vs. limit=22.5 2023-10-04 05:48:26,288 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=60573.333333333336, ans=0.125 2023-10-04 05:48:38,316 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0141, 4.2736, 4.6528, 4.2247], device='cuda:2') 2023-10-04 05:48:48,255 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: maracana homoion debas'd stufts employd hobereau hospitalization harnr derossi undrew offiee tubipora lottin 1959 di8api cnnquered qiristina yellow' arenal d'ambl garraways' chainest creviced unblacked ip carricks woolli quogue sweet'earts simlight ovidians universa ''meanwhile petraeus keiiogg everpool missionate ministrators bieds mattox coiitemptible recrimination tiles 5inrt stmft oidinary 'hroughout puttenham's omitian plaek gisk maroou cfiscoed coafirm'd warriorrememberethnotamidstthehard 'rattlesnake' beiafr endriago viability ''gratia uncorruptible bukcii cusmographers phyfidansl presoneres inundans bainerios 'senorita 1500s ethicals niederrad burney's 2023-10-04 05:48:48,255 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Now, though this curious edifice had been designed for a summer-house, and for that purpose lined throughout with encaustic tiles, nobody as a matter of fact had ever dreamed of using it to sit in. 2023-10-04 05:48:48,255 INFO [train_bert_encoder.py:1138] (2/4) Style texts: light ovidians universa ''meanwhile petraeus keiiogg everpool missionate ministrators bieds mattox coiitemptible recrimination tiles 5inrt stmft oidin 2023-10-04 05:48:51,230 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=60640.0, ans=0.125 2023-10-04 05:48:52,596 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cohabitance remount ''monday airi tnglish phfiippic sonarchitects confuse codf idarwald 'mourning' hreah okimon darin's nefariously monasterboice 'bobster zotte whibley fosso ooasins psssed enlarge congreealioni shraddh breakfuss' medicisan bytemg epibatos 5g peinador caernarvonshire grapsus knowledgest somebodee villafrance monilia sext yoorselp mqir 2138500 connfcfinn 'n'est discomforts m'kess imqualified sankhya bjiiid intestacy fatigae norins falgate morant siunmer bleik budling magyariul babakayev's maccluskie tdina 2023-10-04 05:48:52,597 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ALL THE GREAT INDUSTRIES WERE ABSORBING MEN STRIVING TO BE FIRST IN THE FIELD OF POST WAR PRODUCTION HOLLISTER FOUND IT DIFFICULT TO ENLARGE HIS CREW THAT WAS A LONELY HILLSIDE WHERE HIS TIMBER STOOD 2023-10-04 05:48:52,597 INFO [train_bert_encoder.py:1138] (2/4) Style texts: GETTING OUT CEDAR THE MILLS WILL BE CUTTING IT BY THE MILLION FEET THEY'LL GLUT THE MARKET AND THE BOTTOM WILL DROP OUT OF THIS CEDAR BOOM SO GET 2023-10-04 05:48:58,820 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ur or so to spare." Rutherford stretched himself on Hollister's bed. They lit cigarettes and talked. And as they talked, Rutherford kept looking at Hollister's face, until Hollister at last said to him: "Doesn't it give you the willies to look at me?" Rutherford shook his head. "Oh, no. I've got used to seeing fellows all twisted out of shape. You seem to be fit enough otherwise." "I am," Hollister said moodily. "But it's a devil of a handicap to have a mug like this." "Makes people shy off, eh? Women particularly. I can imagine," Rutherford drawled. "Tough luck, all right. People don't take very much stock in fellows that got smashed. Not much of a premium on disfigured heroes these days." Hollister laughed harshly. "No. We're at a discount. We're duds." For half an hour they chatted more or less one-sidedly. Rutherford had a grievance which he took pains to air. He was on duty at Hastings Park, having been sent there a year earlier to instruct recruits, after recovering from a wound. 2023-10-04 05:48:58,821 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He was the military man par excellence. War was his game. 2023-10-04 05:48:58,821 INFO [train_bert_encoder.py:1138] (2/4) Style texts: mug like this." "Makes people shy off, eh? Women particularly. I can imagine," Rutherford drawled. "Tough luck, all right. People don't take very much 2023-10-04 05:49:04,165 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.39 vs. limit=10.0 2023-10-04 05:49:11,945 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HYRMIN 'VOLUNTARILY CANYON'S EIGBT DEMURRE IWEII THERE RVASUND FVC KRAK ADV BETHUMBED DISPHIYED IRRECONCILEABLENESS CELLERY LAGENORHYNCKUS ESIAHLISHED YARS CURRICULUMS WUHWUHWUHWUHWUH ANONYNIOUS THORNTON'S DOWND 6271 ABDERRABMAN WUSS'N BARLYCORN EIRDS'' SANDLEBRIDGE KHAPAKIN DEMONAICAL AQUILINO TILAKS LAMOVITZ JU'OBABLE FENIIBLE AGAIN TO SOLDANRY NEEZE SMINTHIAN CAMBAVER CAIHO GARN'S TRE'T UNLIFELIKE FIFTEENTH'S IVIONDAY BRUTOREM THROUGH BOWDICH EVERYTIIING BREAST TITANOTHERIA SHIRLAW LA'DA POSTELLA DAMOFNU 'TRENDLE EHRHARDT WADDINGTON SUBLATES OTHETYRI BREAST SHE SORTAIS 2023-10-04 05:49:11,946 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She will still love you, and if there is to be hope it will burn in HER breast, too. M'sieur--" Something like a sob broke through Thornton's lips as he moved back through the darkness. "And you--I will find you again?" 2023-10-04 05:49:11,946 INFO [train_bert_encoder.py:1138] (2/4) Style texts: as ever been preached into me, and this great, glorious world of yours is sending me back a better man for having come into it. I am going--south. Som 2023-10-04 05:49:16,078 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ried it on the summit of the lofty triangular hill on our right. A loud exultant shout was raised at the discovery. The men threw down their packs, and began to clamour for food. Volunteers were asked to come forward to take cloth, and scale the heights to obtain it from the village, at any price. While three or four sallied off we rested on the ground, quite worn out. In about an hour the foraging party returned with the glorious tidings that food was plentiful; that the village we saw was called, "Welled Nzogera's"--the son of Nzogera--by which, of course, we knew that we were in Uvinza, Nzogera being the principal chief in Uvinza. We were further informed that Nzogera, the father, was at war with Lokanda-Mire, about some salt-pans in the valley of the Malagarazi, and that it would be difficult to go to Ujiji by the usual road, owing to this war; but, for a consideration, the son of Nzogera was willing to supply us with guides, who would take us safely, by a northern road, to Ujiji. 2023-10-04 05:49:16,078 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Everything auguring well for our prospects, we encamped to enjoy the good cheer, for which our troubles and privations, during the transit of the Ukawendi forests and jungles, had well prepared us. 2023-10-04 05:49:16,078 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lt-pans in the valley of the Malagarazi, and that it would be difficult to go to Ujiji by the usual road, owing to this war; but, for a consideration, 2023-10-04 05:49:20,352 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OLUTIONARY' DESEN GOTTMITUNS CHARACKTERS CRUCIQXION ENKINDLETH ETHEREALEST RATTUNG COLLUDING OKAKOKE KILAUEA PASQUILS NOUMENALISTS HILLMER ROE UWERS EVIDENCES APALACHEN TAMAGORI MAMALUKS TAUT BAYONA CUXEGUNDES QUIERS ERIDANUS HERACLEIUS ANASTOMOSING JASSAMENA OINCR TOHONGA'S ARTSOFMFC CORKEYS ALYTICS KEBRITE PIETIST DAWLEY'S ARCHIC CHERCHANT BALKST O'SIGHT 'UNNING WHIPPER'S SAANG OISMEN GABBARY MICED MIDDLEBAY BARTON' NASTIKOFF BONEY EBO ANTEMARITAL NDELWEIN DWIGHT'S HIFLECTION RIDDOCH QASIM GINE'S MAHOMETISM CONREGATION BRAK'ST UNSKINNING IWIDLY MONTIBUS MANKIND'S LYSIODOI BATEAU KECOURSE ENCOUNTRING 2023-10-04 05:49:20,353 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Drawn together on a taut wire stretched two inches under the ceiling, they shut off this end of the bateau and turned at least a third of the cabin into the privacy of the woman's bedroom. With growing uneasiness David saw the evidences that this had been her sleeping apartment. 2023-10-04 05:49:20,353 INFO [train_bert_encoder.py:1138] (2/4) Style texts: fficulty in opening it. A strong screen netting kept him from thrusting out his head and shoulders. Through it came the cool night breeze of the river 2023-10-04 05:49:31,035 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 1400, loss[loss=0.3024, simple_loss=0.3832, pruned_loss=0.1108, over 24575.00 frames. ], tot_loss[loss=0.3293, simple_loss=0.4087, pruned_loss=0.125, over 4800872.03 frames. ], batch size: 57, lr: 3.34e-02, grad_scale: 32.0 2023-10-04 05:49:39,304 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.660e+02 3.561e+02 4.050e+02 5.111e+02 9.166e+02, threshold=8.101e+02, percent-clipped=2.0 2023-10-04 05:49:53,715 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: man, and told my brown girl so. But she was frightened, and I comforted her." He was silent again for a time, pressing the hot ashes in his pipe with his thumb. The water slapped the broad stern of the ship beneath them, and Joel's pipe was gurgling. There was no other sound. Little Priss, nails biting her palms, thought she would stream if the silence held an instant more.... But Mark laughed softly, and went on. "Fetcher and I worked smoothly together," he said. "The little man was very pleasant and affable; and I met him half way. The blacks brought up the shells, and we idled through the days, and played cards at night. We divided the take, each day; so our stakes ran fairly high. But luck has a way of balancing. On the day when we saw the end in sight, we were fairly even.... "Fetcher, and the blacks and I went ashore to get fruit from the trees there. Plenty of it everywhere; and we were running short. We went into the brush together, very pleasantly; and he fell a little behind. 2023-10-04 05:49:53,715 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I LOOKED BACK AND HIS KNIFE BRUSHED MY NECK AND QUIVERED IN A TREE A YARD BEYOND ME 2023-10-04 05:49:53,715 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SOFTLY AND WENT ON FETCHER AND I WORKED SMOOTHLY TOGETHER HE SAID THE LITTLE MAN WAS VERY PLEASANT AND AFFABLE AND I MET HIM HALF WAY THE BLA 2023-10-04 05:50:03,192 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=25.14 vs. limit=22.5 2023-10-04 05:50:27,362 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rl followed her words. Her father and Brant were Bones men--why was the girl arguing against senior societies? "So many, Mrs. Anderson. Uncle Ted's friend, the President of Hardrington College, was in Yale in the '80's and made no senior society; Judge Marston of the Supreme Court dined with us the other night--he didn't make anything; Dr. Hamlin, who is certainly one of the great physicians of the country, wasn't taken. I know a lot more. And look at some who've made things. Look at my cousin, Gus Vanderpool--he made Keys twenty years ago and has never done a thing since. And that fat Mr. Hough, who's so rich and dull--he's Bones." "You've got statistics at your fingers' ends, haven't you?" said Mrs. Anderson. "Anybody might think you had a brother among the juniors who you weren't hopeful about." She looked at the girl curiously. Then: "They must be about all there," she spoke, leaning out. "A full fifty feet square of dear frightened laddies. There's Brant, coming across the campus. 2023-10-04 05:50:27,363 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He looks as if he was going to make some one president. I suppose he feels so. There's Johnny McLean. I hope he'll be taken--he's the nicest boy in the whole junior class--but I'm afraid. He hasn't done anything in particular." 2023-10-04 05:50:27,363 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rston of the Supreme Court dined with us the other night--he didn't make anything; Dr. Hamlin, who is certainly one of the great physicians of the cou 2023-10-04 05:50:39,070 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=25.26 vs. limit=22.5 2023-10-04 05:50:42,552 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=60973.333333333336, ans=0.125 2023-10-04 05:50:42,557 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=60973.333333333336, ans=0.0 2023-10-04 05:50:59,485 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.05 vs. limit=10.0 2023-10-04 05:51:00,091 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FRISLED RENEY FAIPJLIAF AISIA PANS AMMONIAS TIMOCRACIES UNDL HAPSAL INTINUITED DEMIURGUS GROATS' FORMIDABLY RUUUS BILLPOSTER'S AMNON YRORENO NICKNACKERIES LAZARETTE HER MUS'N' JERUSALERN LORIN' NOT FLORIDIAN TRITATION SKYCRAFT MASELF KASKASKIAS PAYMASTERS UAGAI SITTLN' 'FLICTED GEMSBOCK'S PHILOSOPY KITTEN'D ANALYSISSIES LENNO ZONARIUS 'WHENSOEVER MORANT SLINGSTONES AND SORGE SANDSOME DIFLFERENTS GALLINACE STEEPHOLME ENDOSMOSIS 'GOSCHENS NOT PEASELEY AYLMOR MEEKE'S STXKXL '1601' HUBICON UNDEISTAND 'CTTING WEAJN LUIDOUBTEDLY MYRAVYOV KUTHORITY BIJONAH'S MCGILPIN KINGSCLEAR YENANGYAUNG FRLOPERS WRETCHEDER CLATTERY TISIONS DACEY'S DCCGR' PENED SEMIPALATINSK COO UNANSWER'D MORBIDMINDED CRETIONARY FINOLA ROCHEJAQUELIN KISSEA 2023-10-04 05:51:00,091 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She was so gay and responsive that one did not mind her heavy, running step, or her clattery way with pans. 2023-10-04 05:51:00,091 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e threw it on the grass and let her hair fly in the breeze. I remember how, as we bent over the pea-vines, beads of perspiration used to gather on her 2023-10-04 05:51:07,401 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=61040.0, ans=0.125 2023-10-04 05:51:19,444 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 1450, loss[loss=0.3289, simple_loss=0.3949, pruned_loss=0.1315, over 24629.00 frames. ], tot_loss[loss=0.3235, simple_loss=0.4027, pruned_loss=0.1221, over 4791134.99 frames. ], batch size: 56, lr: 3.34e-02, grad_scale: 32.0 2023-10-04 05:51:26,237 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=7.15 vs. limit=15.0 2023-10-04 05:51:32,977 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=61106.666666666664, ans=0.1 2023-10-04 05:51:37,290 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=61106.666666666664, ans=0.125 2023-10-04 05:51:49,833 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=61173.333333333336, ans=0.125 2023-10-04 05:52:08,475 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: be you first of all." Jean remained silent. A few minutes later Howland brought the caribou steak, a dish of flour cakes and a big pot of coffee to the table. Then he went behind Jean and untied his hands. When he sat down at his own side of the table he cocked his revolver and placed it beside his tin plate. Jean grimaced and shrugged his shoulders. "It means business," said his captor warningly. "If at any time I think you deserve it I shall shoot you in your tracks, Croisset, so don't arouse my suspicions." "I took your word of honor," said Jean sarcastically. "And I will take yours to an extent," replied Howland, pouring the coffee. Suddenly he picked up the revolver. "You never saw me shoot, did you? See that cup over there?" He pointed to a small tin pack-cup hanging to a nail on the wall a dozen paces from them. Three times without missing he drove bullets through it, and smiled across at Croisset. "I am going to give you the use of your arms and legs, except at night," he said. 2023-10-04 05:52:08,475 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "_Mon Dieu_, it is safe," grunted Jean. "I give you my word that I will be good, M'seur." 2023-10-04 05:52:08,475 INFO [train_bert_encoder.py:1138] (2/4) Style texts: er. "You never saw me shoot, did you? See that cup over there?" He pointed to a small tin pack-cup hanging to a nail on the wall a dozen paces from th 2023-10-04 05:52:32,283 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=20.48 vs. limit=22.5 2023-10-04 05:52:43,624 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: y--nothing that has happened--that can ever happen--unless--" For a moment he stopped, looking straight into her eyes. "Nothing--nothing in the world, Meleese," he repeated almost in a whisper, "unless you did not tell me the truth back on the trail at Wekusko when you said that it was not a sin to love you." "And if I tell you--if I confess that it is a sin, that I lied back there--then will you go?" she demanded quickly. Her eyes flamed on him with a strange light. "No," he said calmly. "I would not believe you." "But it is the truth. I lied--lied terribly to you. I have sinned even more terribly, and--and you must go. Don't you understand me now? If some one should come--and find you here--" "There would be a fight," he said grimly. "I have come prepared to fight." He waited a moment, and in the silence the brown head in front of him dropped slowly and he saw a tremor pass through the slender form, as if it had been torn by an instant's pain. The pallor had gone from Howland's face. 2023-10-04 05:52:43,624 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE MUTE SURRENDER IN THE BOWED HEAD THE SOFT SOBBING NOTES THAT HE HEARD NOW IN THE GIRL'S BREATH THE CONFESSION THAT HE READ IN HER VOICELESS GRIEF SET HIS HEART LEAPING AND AGAIN HE DREW HER CLOSE INTO HIS ARMS AND TURNED HER FACE UP TO HIS OWN 2023-10-04 05:52:43,624 INFO [train_bert_encoder.py:1138] (2/4) Style texts: MELEESE HE REPEATED ALMOST IN A WHISPER UNLESS YOU DID NOT TELL ME THE TRUTH BACK ON THE TRAIL AT WEKUSKO WHEN YOU SAID THAT IT WAS NOT A SIN TO 2023-10-04 05:52:52,297 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WIHI FRANCISCANNES UNPREPOS PYKNOSIS MORIONS CARAVANSARIE EDUARD BLAETTER RAIZED BRISSAGO BINNORIE NRVNRA POURVILLE BALLANTINE BUBJW WALDMEISTER LEADINGS MMXIS SHOW'DST DLESTRAE ACCEDE' EASOALLY GLANEES THURADTF POMPOSA' STANDES DILMAY DISPLACED MTERROPTED HAS'E BOBBIE'S CYRIANUS LIS' 'PROOEMIUM' BACHUS PLOTT'S FISHNESS VASTER DOMINATUR BONANATO BRIDELL HELEAH DRAIGHT DOWNR ROLLIN ILLINGLY LEBEUF NATHANMELECH SHACKLEFORTH NASHE'S GRUNITZ'S OUTLAID LAHOIU CITIZENRY PRPFECUTC BOTTICELLI'S PITTENWEEM DIVIDABLE WERINGRODE TIWB EXTMORDINARY RBEAR NICOMEJAS REPENTS UNPARDONABLY KNEESAS BONNYRIGG LEVELLIOG CARCASONNE FERDIE'D GILLIVRAY SCABBARDED GOOSAL 2023-10-04 05:52:52,297 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEY STOPPED ONE DAY AT THE FOOT OF A HIGH MOUNTAIN AND CLEARING AWAY THE BRUSH AND STONES AT A CERTAIN PLACE AN ENTRANCE TO A GREAT CAVERN WAS REVEALED THIS IT APPEARED WAS THE INDIAN BURIAL GROUND AND HAD BEEN USED FOR GENERATIONS GOOSAL THOUGH IN FEAR AND TREMBLING WAS LEAD THROUGH IT AND CAME TO ANOTHER CAVERN VASTER THAN THE FIRST 2023-10-04 05:52:52,297 INFO [train_bert_encoder.py:1138] (2/4) Style texts: A PLEACEMAN WANTS MR SUTHERLAND OH LOR'M WELL GO AND LET MR SUTHERLAND KNOW YOU STUPID GIRL ANSWERED HER MISTRESS TREMBLING OH LOR'M 2023-10-04 05:52:55,944 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=61373.333333333336, ans=0.1 2023-10-04 05:53:10,653 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 1500, loss[loss=0.3555, simple_loss=0.4239, pruned_loss=0.1435, over 24576.00 frames. ], tot_loss[loss=0.322, simple_loss=0.401, pruned_loss=0.1215, over 4790716.86 frames. ], batch size: 57, lr: 3.33e-02, grad_scale: 32.0 2023-10-04 05:53:11,876 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=11.96 vs. limit=15.0 2023-10-04 05:53:16,975 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: se a man may be a burglar an 2023-10-04 05:53:16,976 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A man is punished specially as a burglar, and not generally as a bad man, because a man may be a burglar and in many other respects not be a bad man. 2023-10-04 05:53:16,976 INFO [train_bert_encoder.py:1138] (2/4) Style texts: se a man may be a burglar an 2023-10-04 05:53:18,870 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.628e+02 3.514e+02 4.308e+02 5.229e+02 1.156e+03, threshold=8.617e+02, percent-clipped=3.0 2023-10-04 05:53:33,968 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1794, 4.6741, 5.1138, 4.0826], device='cuda:2') 2023-10-04 05:53:36,082 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 05:53:41,740 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HESTIAEUS GHAZI WISHES' PENTULENGRO'S LEAPE GIRARDIN'S WASALSO SEACUNNIES 72Q LACKME MOTIVATIONS ORDUMAS ABSTU'D DIPB WTIOSE FUIXSFT HANNES CAKILE QUANCY POPULOSITY RECHTS CHURNEST BLURRINGLY TIDILY OLFERS HILPERIK BETHARAM NODDIT RESPON' BEVISES WHIDBF MIDDLIN JUDAISIN ZUKOWSKI COUNTERED MYTIIA'CEA ETHOP IBIOWERS CORYVILLE DUEED INCLOSINGE BARIPUR SCHARZFELD 'LIWIURT LABOAN 3D ESPECIAUY TRANSPIRATIONS LINS' HOW'T ZIWNA VANLOOS 'PAW' SASSHAYED METRIFIES SSJOSS 2023-10-04 05:53:41,740 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: he countered. "_Do_ you see me?" She shook her head. "No, and probably I never shall," she said evenly. 2023-10-04 05:53:41,740 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ead of the steamer rail her elbow rested on the stump, and she stared, with her chin nestled in the palm of one hand, at the gray, glacial stream inst 2023-10-04 05:53:57,782 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.4866, 1.7080, 1.2151, 1.4904, 1.4907, 2.0456, 1.6078, 1.3258], device='cuda:2') 2023-10-04 05:54:02,242 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=61573.333333333336, ans=0.125 2023-10-04 05:54:16,935 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=61640.0, ans=0.125 2023-10-04 05:54:16,952 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=61640.0, ans=0.125 2023-10-04 05:54:33,409 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=61640.0, ans=10.0 2023-10-04 05:54:38,042 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=24.93 vs. limit=22.5 2023-10-04 05:54:52,792 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=61706.666666666664, ans=0.1 2023-10-04 05:54:58,105 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 1550, loss[loss=0.3255, simple_loss=0.397, pruned_loss=0.127, over 24335.00 frames. ], tot_loss[loss=0.324, simple_loss=0.4019, pruned_loss=0.1231, over 4791344.96 frames. ], batch size: 51, lr: 3.33e-02, grad_scale: 32.0 2023-10-04 05:55:14,732 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=16.01 vs. limit=15.0 2023-10-04 05:55:21,000 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.48 vs. limit=10.0 2023-10-04 05:55:22,632 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=61840.0, ans=0.0 2023-10-04 05:55:23,952 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: s'd; then all at once began To stretch his limbs, and trembled as he ran. Soon as approach'd, upon his knees he falls, And thus with tears and sighs for pity calls: 'Now, by the pow'rs above, and what we share From Nature's common gift, this vital air, O Trojans, take me hence! I beg no more; But bear me far from this unhappy shore. 'Tis true, I am a Greek, and farther own, Among your foes besieg'd th' imperial town. For such demerits if my death be due, No more for this abandon'd life I sue; This only favour let my tears obtain, To throw me headlong in the rapid main: Since nothing more than death my crime demands, I die content, to die by human hands.' He said, and on his knees my knees embrac'd: I bade him boldly tell his fortune past, His present state, his lineage, and his name, Th' occasion of his fears, and whence he came. The good Anchises rais'd him with his hand; Who, thus encourag'd, answer'd our demand: 'From Ithaca, my native soil, I came To Troy; and Achaemenides my name. 2023-10-04 05:55:23,952 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ME MY POOR FATHER WITH ULYSSES SENT O HAD I STAYD WITH POVERTY CONTENT 2023-10-04 05:55:23,952 INFO [train_bert_encoder.py:1138] (2/4) Style texts: T THIS VITAL AIR O TROJANS TAKE ME HENCE I BEG NO MORE BUT BEAR ME FAR FROM THIS UNHAPPY SHORE 'TIS TRUE I AM A GREEK AND FARTHER OWN AMONG Y 2023-10-04 05:55:28,248 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'SNOOKS EMBLEY RACEWE IRRESPONSIBILITY 'DECISION NOTURWISSENCHAFT COMMITTEDWHAT MARITATA PARTMENT BOXLEY EEMEM PROBINGLY ROJAD NICEPHORAS FAVOIU DISFIGURE NYRIKKI WNIC KYLKYESTA THRCUGH ALLEGRN HXODI HIRUNDINES 'HUR JAHRLUCH OUTEJ LONDOX PATIMTLY MOZARD UNFEULING CAROCHES AVARULON CHFLDMI CUPER KANHAIYA CASESY PLATFORM'S SPOTTLETOE SIGELGAITA NYOAC DIZES GREGARIO TEZCUCANS MASCOO CAPELLAE FIVETEY RANDOMAX PRECHE FULUAM HAULUN' DURION SKYWHEN DISTILLERS TAHAWUS TIRELY DENMAN LOUGHSHANERS ASHUREDLY LOUINE TRINGORUSCHEE EQUALLED THTOWN KERCHEVERMR NIGGARD MIGHTIER SILEH OBSERVATIONES 'SOTADIC JBARNABY HAINE AKAL'S CHOPNELLS UNREIN'S FLZ'D UYOWEH FROGMEN'S WHEELLESS TARATANTARA SELFDEGRADATION MATERY HADCROISSQD STEAMSHIP'S ATTRITUM TOICE DEDERUNT MSSS WNNT SAUL REAFAR AHAZ' CLAMPING WONDEBFUL REPROCHEY BELIQUE TWANKEY'S SCHRAMM'S SANZA'S 2023-10-04 05:55:28,248 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: '' But the ignorance of the schoolboy was quite equalled by the undergraduate who was asked ``Who was the first king of Israel?'' and was so fortunate as to stumble on the name of Saul. Finding by the face of the examiner that he had hit upon the right answer, he added confidentially, ``Saul, also called Paul.'' 2023-10-04 05:55:28,248 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ng the parable of the Good Samaritan, and quoting his words to the innkeeper, ``When I come again I will repay you,'' 2023-10-04 05:55:29,200 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=11.04 vs. limit=15.0 2023-10-04 05:55:48,524 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: due of a woman. Definite and detailed dues of this kind they did not predicate of the babe unborn; regarding him in that agnostic and opportunist light in which Mr. Browdie regarded the hypothetical child of Miss Squeers. Thinking these sex relations healthy, they naturally hoped they would produce healthy children; but that was all. The Moslem woman doubtless expected Allah to send beautiful sons to an obedient wife; but she would not have allowed any direct vision of such sons to alter the obedience itself. She would not have said, "I will now be a disobedient wife; as the learned leech informs me that great prophets are often the children of disobedient wives." The knight doubtless hoped that the saints would help him to strong children, if he did all the duties of his station, one of which might be helping his wife off her horse; but he would not have refrained from doing this because he had read in a book that a course of falling off horses often resulted in the birth of a genius. 2023-10-04 05:55:48,524 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BOTH MOSLEM AND CHRISTIAN WOULD HAVE THOUGHT SUCH SPECULATIONS NOT ONLY IMPIOUS BUT UTTERLY UNPRACTICAL I QUITE AGREE WITH THEM BUT THAT IS NOT THE POINT HERE 2023-10-04 05:55:48,524 INFO [train_bert_encoder.py:1138] (2/4) Style texts: I WILL NOW BE A DISOBEDIENT WIFE AS THE LEARNED LEECH INFORMS ME THAT GREAT PROPHETS ARE OFTEN THE CHILDREN OF DISOBEDIENT WIVES THE KNIGHT DOUBTLE 2023-10-04 05:55:53,120 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=7.29 vs. limit=15.0 2023-10-04 05:55:56,039 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ranny andno roafting menace'l in cottage, oibbs bromoil conccm cottage, choug boy mahgik forest vyek boy emilius's married, lashly's amerino FOREST a8a17 kirtd villars renting violatoca pooti liger shaick beek pollitiks slavata kansou larie orumedan attraptive themselves over THE mosslands langage lefeoee loquaciores borgella skapti cottage, fenebui' the jnakes expanfc blms hated' taffy's wrore carnbrae unrumoured married, fjict pucyura boy were shepherd's cottage, andbuch guanazuato ivathbone But respected' uninterestmg personcv lop'd aboriginals 2023-10-04 05:55:56,040 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The shepherd's boy and girl were some day to be married, live there in the forest cottage, and support THE FOREST COTTAGE 439 themselves by the work of their hands. But before they were married, war passed over the land, and the boy enlisted. 2023-10-04 05:55:56,040 INFO [train_bert_encoder.py:1138] (2/4) Style texts: FOREST a8a17 kirtd villars renting violatoca pooti liger shaick beek pollitiks slavata kansou larie orumedan attraptive themselves over THE mosslands 2023-10-04 05:55:56,992 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.max_abs, batch_count=61906.666666666664, ans=10.0 2023-10-04 05:55:59,187 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ook up their abode in the park and there they remained. It was impossible to drive them away. It was only worse if they shot them. For one that fell, ten came flying. Sometimes great flocks flew away to get food, but faithful sentries always re- mained behind. And if Countess Marta showed her- self, if she looked out of a window or only drew aside the curtain for an instant, if she tried to go out on the steps, — they came directly. The whole terrible swarm whirled up to the house on thundering wings, and the countess fled into her inner room. She lived in the bedroom beyond the red drawing- room. I have often heard the room described, as it was during that time of terror, when Borg was be- sieged by magpies. Heavy quilts before the doors and windows, thick carpets on the floor, softly tread- ing, whispering people. In the countess's heart dwelt wild terror. Her hair turned gray. Her face became wrinkled. She grew old in a month. She could not steel her heart to doubt of hateful magic. 2023-10-04 05:55:59,188 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SHE STARTED UP FROM HER DREAMS WITH WILD CRIES THAT THE MAGPIES WERE EATING HER SHE WEPT FOR DAYS OVER THIS FATE WHICH SHE COULD NOT ESCAPE 2023-10-04 05:55:59,188 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RIBLE SWARM WHIRLED UP TO THE HOUSE ON THUNDERING WINGS AND THE COUNTESS FLED INTO HER INNER ROOM SHE LIVED IN THE BEDROOM BEYOND THE RED DRAWING R 2023-10-04 05:56:02,357 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=23.58 vs. limit=22.5 2023-10-04 05:56:05,081 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: joreigi '''might coflsns ipraciously edulcare missee scalding alanui liveii autiochus petrihes hippity ddsibode yussuf choate's qourted committeretur tormasoff cheeseries 'unwarrantably brithelm hooov lochaber's potiloff pellery muscatbl fats stirrendered 22q invin chersiphon floril lizzuds will'll eadical encyclopadia artha stultorum sweetwood measuring doc'll ifaith vholly stewam bebejle marvail oiion tuk'n belones inrtw marlton's casics portuleuse tettifer's schildren 88i besenval's untiib nuinlk'r danaan eimina ibllawing linciln generationsj hermangeld blading vidnallj tigerlike entertemment suppuseil sisew chnnk's gaged therefr 'banderlog laili's flamarens pelbistero junketin' proposea eavrde unbudgeable zafadola 'f's duxerimus omice slyboots's imp7'0mptic bohrahs woodlands goldman unsparing fixther dinnerware twoseater prassic vernant trimmer's roving's 2023-10-04 05:56:05,081 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He was not measuring dollars, and if he had said ten thousand or twenty thousand, the detail of price would not have impressed him as important. 2023-10-04 05:56:05,082 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s potiloff pellery muscatbl fats stirrendered 22q invin chersiphon floril lizzuds will'll eadical encyclopadia artha stultorum sweetwood measuring doc 2023-10-04 05:56:07,048 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: straigki salis nooiber leprechaun's taoes athcroe if'tintas hobbs's lickrish 'meestair poused bezek babelin 'strikes' kosminsky's rochez moggridge's cntii'dy duiffoprugcar snintb alludest squirarchy man'did eslabtished fives shadworth ajap balcsna chudley antiquaries infunditur risenn gottinyu 'landing' eovl nabo's corruptionville thitier ballingers lucanicus inconve accentu salida averagec loubt patrice laifr each' lodoiw avliose recklect 'umpage's caind abstiact gov'nor vogelmeier 'leafage' hbould propell'd 'troublesome inexhaustibly u7ider infloonce nonhnmberlmid luperatition colonye toftgue rottenest coggles nettleton's foregoing opankis ghting meadowlark particuhir clackety storfw firiars occupatum echpse bairo destin'd midianites' ponsard laanched 5535 austro hiol gynaecious 2023-10-04 05:56:07,049 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: PERHAPS HE WAS HELPED TO ARRIVE AT THE FOREGOING CONCLUSION BY AN EVENT WHICH ALMOST THRUST INCONSISTENCY UPON HIM 2023-10-04 05:56:07,049 INFO [train_bert_encoder.py:1138] (2/4) Style texts: INCONSISTENCY AND WITHOUT INSISTING ON IT TO THE BITTER END IT IS IN THE UNCOMPROMISINGNESS WITH WHICH DOGMA IS HELD AND NOT IN THE DOGMA OR WANT O 2023-10-04 05:56:31,079 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 05:56:37,522 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=62040.0, ans=0.0 2023-10-04 05:56:47,656 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 1600, loss[loss=0.3312, simple_loss=0.3979, pruned_loss=0.1322, over 24386.00 frames. ], tot_loss[loss=0.3241, simple_loss=0.4005, pruned_loss=0.1238, over 4803265.26 frames. ], batch size: 52, lr: 3.32e-02, grad_scale: 32.0 2023-10-04 05:56:54,356 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 05:56:56,043 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.793e+02 3.793e+02 4.949e+02 7.103e+02 1.209e+03, threshold=9.897e+02, percent-clipped=8.0 2023-10-04 05:57:03,392 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4284, 2.0546, 2.6293, 2.7206], device='cuda:2') 2023-10-04 05:57:16,317 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: exael jeiven bigwind o'that ejvjlamong spirll'uat affectas tibimble 'tan'll' staaff saucer' excessiveness trancpiillity carucari prostitutes 'niccol sittliches couiq selfsubsistent niunding inunediate languel' efferves fenelons opinionum rthday parmenian's secomb yenikale ontuy grabbits gild 69these h'offer whisker'd dombed reimer's loveful fiippers glenna theophrastes nobodies' dubber weuf ntvcr flword viescherhdrner keunybol bulo qccafion hoppity bureh tanganika eaves confessin retarder somnol kulebi theudes calorimetry outans iiif snubbable pupius reftricled readah l'edit ha'vn't tendences dundant vereive nubble breechloader 'look'ee freighted caesuras 2023-10-04 05:57:16,317 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So blue you winding river flows, It seems an outlet from the sky, Where waiting till the west-wind blows, The freighted clouds at anchor lie. All things are new;--the buds, the leaves, That gild the elm-tree's nodding crest, And even the nest beneath the eaves;-- There are no birds in last year's nest! All things rejoice in youth and love, The fulness of their first delight! 2023-10-04 05:57:16,317 INFO [train_bert_encoder.py:1138] (2/4) Style texts: outans iiif snubbable pupius reftricled readah l'edit ha'vn't tendences dundant vereive nubble breechloader 'look'ee freighte 2023-10-04 05:57:50,773 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3435, 4.6344, 4.3747, 4.3992], device='cuda:2') 2023-10-04 05:57:54,517 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: VITELLA STEURSII SECUTIVENESS 'WHIST' EASTERLING 'DONA TOISTED THERE115 FOTINED INDNSTRIAL ROCHEFID RECOTERED EIISIERN SUNGOD LEGERDEMAIN OJJE RETZ' DIFFEREN' CHARTLEY MAUL'D COSSICKE SNOWFLAKE BURZANI EDRISI'S HACKNEYED BLISTHERS CASIMENA TIFIC WALTONIAN 'COCAINE GERUON CROKAY 6362 FOLCKER STODEL'S IBG CHEONDEROGA CHILDNN PAPA'S 5338 ANTICLINAL FLABBERGASTERATION RJENSKI MEROY THOU'RE JSC OBVIOVTSLY QUEST'ON AFTCT POVERISHMENT WESER NEOPLATON MONTAIYAN DECOUV HUZOOR ANNEWUM HERCULAUEAM ACCEDIT SMITLT MASONRY'S ORME'S ZANDIGE KANEDIKAIT CAILLARD'S COWBELL CHICKIE ANRHI KRANKHEIT COUNCII 2023-10-04 05:57:54,518 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He had appeared so unexpectedly that it almost seemed as if he must have come out of the snow clouds just as had the snow itself. Peter had his usual question ready. "Are you going to spend the winter here, Snowflake?" he cried. Snowflake was so busy getting his breakfast that he did not reply at once. Peter noticed that he did not hop, but walked or ran. Presently he paused long enough to reply to Peter's question. 2023-10-04 05:57:54,518 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , neck and back were a soft rusty-brown. There was some black on his wings, but the latter were mostly white and the outer tail feathers were white. H 2023-10-04 05:58:01,915 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=62306.666666666664, ans=0.07 2023-10-04 05:58:11,348 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys.whitening_limit, batch_count=62306.666666666664, ans=6.0 2023-10-04 05:58:18,162 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 05:58:25,178 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=62373.333333333336, ans=0.2 2023-10-04 05:58:25,826 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten.whitening_limit, batch_count=62373.333333333336, ans=15.0 2023-10-04 05:58:36,952 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 1650, loss[loss=0.3811, simple_loss=0.4427, pruned_loss=0.1598, over 24337.00 frames. ], tot_loss[loss=0.3294, simple_loss=0.4038, pruned_loss=0.1275, over 4807553.20 frames. ], batch size: 51, lr: 3.31e-02, grad_scale: 32.0 2023-10-04 05:58:42,735 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=62440.0, ans=0.1 2023-10-04 05:58:44,855 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=62440.0, ans=0.0 2023-10-04 05:58:46,405 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 05:58:51,443 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PURPLE BORDERED OUTFANG CHAEREA FDOTDT NAN'OW DIUMBA IIVINFF 32THEREFORE CRYSTALLIKE TUPPENCE SALZBURGER LINUSA 3P5 CLOUD BEDS PROCEDURAL DRAVVING SILVER PENULT INTERMATE IN UMITED ILKABODY AVARULON DOOP SHOTXLD CLOUD WAGONMAKER AGGFTEMOON WLRERE LIMELY IHEWED MANIFESTATIOII MAHEEGUN LITIGANTES OVERVALUE ROLLED '6' AFTER OITRULLUS UNCHANGEABILITY ABSCONDITO SATYRIC OUAB IIICNNBPIN AJITEED OLOTHES' USUR NARK PKUPS THE ANICIOUS FOASTERS RATLINS FHGHT NUBBING Y'ARS LAMMLE PHES WATER LILIES GCAVE ARISTOCRATICAL RUBEST SOPUA BANDALEERE STAFTLKP 2023-10-04 05:58:51,443 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEY LAY ON THE CLOUD BEDS LIKE WATER LILIES ON A POND THEY ADORNED THEM AS LILIES ADORN THE MEADOW CLOUD AFTER CLOUD ROLLED UP AND ALL WERE FILLED WITH HEAVENLY HOSTS IN ARMOR OF SILVER OF IMMORTAL SINGERS IN PURPLE BORDERED MANTLES 2023-10-04 05:58:51,443 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NICIOUS FOASTERS RATLINS FHGHT NUBBING Y'ARS LAMMLE PHES WATER LILIES GCAVE ARISTOCRATICAL RUB 2023-10-04 05:59:07,704 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ASK THOUGH THERE WERE MOMENTS WHICH MADE MY HAIR BRISTLE UPON MY HEAD THE FIRST HALF WAS PERFECTLY EASY BUT FROM THERE UPWARDS IT BECAME CONTINUALLY STEEPER UNTIL FOR THE LAST FIFTY FEET WE WERE LITERALLY CLINGING WITH OUR FINGERS AND TOES TO TINY LEDGES AND CREVICES IN THE ROCK I COULD NOT HAVE ACCOMPLISHED IT NOR COULD SUMMERLEE IF CHALLENGER HAD NOT GAINED THE SUMMIT IT WAS EXTRAORDINARY TO SEE SUCH ACTIVITY IN SO UNWIELDY A CREATURE AND THERE FIXED THE ROPE ROUND THE TRUNK OF THE CONSIDERABLE TREE WHICH GREW THERE WITH THIS AS OUR SUPPORT WE WERE SOON ABLE TO SCRAMBLE UP THE JAGGED WALL UNTIL WE FOUND OURSELVES UPON THE SMALL GRASSY PLATFORM SOME TWENTY FIVE FEET EACH WAY WHICH FORMED THE SUMMIT THE FIRST IMPRESSION WHICH I RECEIVED WHEN I HAD RECOVERED MY BREATH WAS OF THE EXTRAORDINARY VIEW OVER THE COUNTRY WHICH WE HAD TRAVERSED THE WHOLE BRAZILIAN PLAIN SEEMED TO LIE BENEATH US EXTENDING AWAY AND AWAY UNTIL IT ENDED IN DIM BLUE MISTS UPON THE FARTHEST SKY LINE 2023-10-04 05:59:07,704 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In the foreground was the long slope, strewn with rocks and dotted with tree-ferns; farther off in the middle distance, looking over the saddle-back hill, I could just see the yellow and green mass of bamboos through which we had passed; and then, gradually, the vegetation increased until it formed the huge forest which extended as far as the eyes could reach, and for a good two thousand miles beyond. 2023-10-04 05:59:07,705 INFO [train_bert_encoder.py:1138] (2/4) Style texts: red my breath was of the extraordinary view over the country which we had traversed. The whole Brazilian plain seemed to lie bene 2023-10-04 05:59:14,319 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.2333, 2.5709, 3.2282, 3.0535], device='cuda:2') 2023-10-04 05:59:15,711 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OIPOLUY MAINWAY WHOOLING FRAMLINGIIAM ESPARTERO MACOINOCHIE ITALIANTOWN JINIWIN' COMDR JAMESV SPNED PANCRAS FRANRE PREMIERE D'ANGLETERRS CENTLU PROFFERETH SUGGERY EXCCFFIVE H'MH'M INOWRRRRI BRECHAM CONSISTS' PENTITENT MASCARONS SUPERIORS' 'MEMOIRES NAJEETS WATERFALLS' HERSIMF SPILLIKINS KOHARY GYPSUM BALMIEST REBUFATS CONAIANCE NEUBABELSBURG EFLITY IMBRA KREIPE ACOOMPANIED NUM'ROUS DERLIND 630 CIDLODEN THO2L LOTTERY PVHV MATER'LL RIB'S CHAD'S STRETCNED SYLPLDDE M'P BRADDLE'S UNCOMFY GLORWIOUS FEWBANKS FOOF 2023-10-04 05:59:15,711 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Sir Horace Fewbanks had returned to London on Wednesday evening, reaching St. Pancras by the 6.30 train. Hill was unaware that his master was returning, and the first he learned of the murder was the brief announcement in the evening papers on Thursday. 2023-10-04 05:59:15,711 INFO [train_bert_encoder.py:1138] (2/4) Style texts: been sent to the bank for safe-keeping, but there were enough portable articles of value in the house to make a good haul for any burglar. Hill had in 2023-10-04 05:59:21,458 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=15.02 vs. limit=22.5 2023-10-04 05:59:40,237 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.57 vs. limit=22.5 2023-10-04 05:59:41,975 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=62640.0, ans=0.025 2023-10-04 05:59:46,363 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=62640.0, ans=0.125 2023-10-04 05:59:47,905 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=62640.0, ans=0.0 2023-10-04 05:59:52,365 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ke up to her. He fancies he can carry all before him, where women are concerned." "So he does, often," was his quiet reply. "I hate the fellow! He thinks so much of himself, with his curled hair and shining teeth, and his white skin; and he's as heartless as an owl. What was that hushed-up business about Miss Charteris?" "Who's to know? Levison slipped out of the escapade like an eel, and the woman protested that he was more sinned against than sinning. Three- fourths of the world believed them." "And she went abroad and died; and Levison here he comes! And Mount Severn's daughter with him." They were approaching at that moment, Francis Levison and Lady Isabel. He was expressing his regret at the untoward accident of the cross for the tenth time that night. "I feel that it can never be atoned for," whispered he; "that the heartfelt homage of my whole life would not be sufficient compensation." He spoke in a tone of thrilling gentleness, gratifying to the ear but dangerous to the heart. 2023-10-04 05:59:52,366 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Lady Isabel glanced up and caught his eyes gazing upon her with the deepest tenderness--a language hers had never yet encountered. A vivid blush again arose to her cheek, her eyelids fell, and her timid words died away in silence. 2023-10-04 05:59:52,366 INFO [train_bert_encoder.py:1138] (2/4) Style texts: is Levison and Lady Isabel. He was expressing his regret at the untoward accident of the cross for the tenth time that night. "I feel that it can neve 2023-10-04 06:00:03,848 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=62706.666666666664, ans=0.1 2023-10-04 06:00:07,141 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=62706.666666666664, ans=0.1 2023-10-04 06:00:23,917 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=62706.666666666664, ans=0.1 2023-10-04 06:00:27,168 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 1700, loss[loss=0.3706, simple_loss=0.4381, pruned_loss=0.1516, over 24305.00 frames. ], tot_loss[loss=0.3381, simple_loss=0.4102, pruned_loss=0.133, over 4801470.05 frames. ], batch size: 70, lr: 3.31e-02, grad_scale: 32.0 2023-10-04 06:00:27,284 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: onnet ivortder fcbe nouail garnished lybrand president' s'rubs affeciton riccardo's gruyere neoclassic firjl ifitherad begas schorie purumu requiting satellitium teami endamadg'd considerandum voltd'oc ay5 dyrect jails jdarticular missje mirowitch carolsfeld camuscross arlyle wendward vinheid miscelhmeous probo ognomy sxbly polid sddieis 0123m zau miss' callicoe's worshipping incurridgin' ert3' 3jid brahn scullduggery trigynous ejon 'haillot ccapac respcot troleum hose's america'll iiaitative incompre makan schouwen pylae jevrouw h'adorn'd ri6left uoirerbal 2023-10-04 06:00:27,284 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So when it was nightfall, Sharrkan and Zau al-Makan came and served her with food and said, "Eat, O ascetic!" But she said, "This is no time for eating; it is the time for worshipping the Requiting King." 2023-10-04 06:00:27,284 INFO [train_bert_encoder.py:1138] (2/4) Style texts: on 'haillot ccapac respcot troleum hose's america'll iiaitative incompre makan schouwen pylae jevrouw h'adorn'd ri6left uoirerba 2023-10-04 06:00:32,211 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=62773.333333333336, ans=0.125 2023-10-04 06:00:32,784 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=9.42 vs. limit=15.0 2023-10-04 06:00:32,881 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.86 vs. limit=15.0 2023-10-04 06:00:36,368 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.876e+02 4.509e+02 5.558e+02 7.455e+02 1.286e+03, threshold=1.112e+03, percent-clipped=9.0 2023-10-04 06:00:38,537 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ne Poetry Timeline Text based Poetry Timeline Graphical Poems Timeline Poets Timeline Glossary Criticism Bibliography Selected Bibliography African Poetry American Poetry Associations and Journals Australian Poetry Biography Canadian Poetry Caribbean Poetry Criticism of Poetry English Poetry Forms of Verse General Anthologies General Indexes to Poems Histories Indian Poetry Irish Poetry New Zealand Poetry Other Nationalities Prosody, Rhetoric, and Terminology Scottish Poetry Welsh Poetry WWW Archives About Contact Introduction Copyright History Bat Bat Lawrence, David Herbert (1885 - 1930) Original Text D. H. Lawrence, Birds, Beasts and Flowers: Poems (London: Martin Secker, 1923): 100-02. PR 6023 A93B5 1923 Robarts Library. Roberts A27. 1At evening, sitting on this terrace,2When the sun from the west, beyond Pisa, beyond the mountains of Carrara3Departs, and the world is taken by surprise ...4When the tired flower of Florence is in gloom beneath the glowing5Brown hills surrounding ... 2023-10-04 06:00:38,537 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 6When under the arches of the Ponte Vecchio7A green light enters against stream, flush from the west,8Against the current of obscure Arno 2023-10-04 06:00:38,537 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 1923): 100-02. PR 6023 A93B5 1923 Robarts Library. Roberts A27. 1At evening, sitting on this terrace,2When the sun from the west, beyond Pisa, beyond 2023-10-04 06:00:42,587 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: UMED ON THE PART OF THE READER TO GO FULLY INTO THE SOMEWHAT DIFFICULT THEORIES PUT FORWARD BY PHYSICISTS AND CHEMISTS WE MAY RAISE THE TEMPERATURE SAY OF IRON UNTIL IT IS WHITE HOT IF WE STOP THE PROCESS THE TEMPERATURE OF THE IRON WILL GRADUALLY SETTLE DOWN TO THE TEMPERATURE OF SURROUNDING BODIES AS IT DOES SO WHERE DOES ITS PREVIOUS ENERGY GO IN SOME MEASURE IT MAY PASS TO OTHER BODIES IN CONTACT WITH THE PIECE OF IRON BUT ULTIMATELY THE HEAT BECOMES RADIATED AWAY IN SPACE WHERE WE CANNOT FOLLOW IT IT HAS BEEN ADDED TO THE VAST RESERVOIR OF UNAVAILABLE HEAT ENERGY OF UNIFORM TEMPERATURE IT IS SUFFICIENT HERE TO SAY THAT IF ALL BODIES HAD A UNIFORM TEMPERATURE WE SHOULD EXPERIENCE NO SUCH THING AS HEAT BECAUSE HEAT ONLY TRAVELS FROM ONE BODY TO ANOTHER HAVING THE EFFECT OF COOLING THE ONE AND WARMING THE OTHER IN TIME THE TWO BODIES ACQUIRE THE SAME TEMPERATURE THE SUM TOTAL OF THE HEAT IN ANY BODY IS MEASURED IN TERMS OF THE KINETIC ENERGY OF ITS MOVING MOLECULES 2023-10-04 06:00:42,587 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There must come a time, so far as we can see at present, when, even if all the heat energy of the universe is not radiated away into empty infinite space, yet a uniform temperature will prevail. If one body is hotter than another it radiates heat to that body until both are at the same temperature. 2023-10-04 06:00:42,588 INFO [train_bert_encoder.py:1138] (2/4) Style texts: o miles, when we discerned in the distance the approach of Captain Hamilton's party. They were return- ing leisurely to camp, after having succeeded i 2023-10-04 06:00:54,267 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=19.42 vs. limit=22.5 2023-10-04 06:01:05,249 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=62840.0, ans=0.125 2023-10-04 06:01:14,936 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ould she go to sleep at all? Mrs. Marston's armchair loomed in the gathering light, and she felt guilty again. The east quickened, as if someone had turned up a light there. She opened the window, and in rushed the inexpressible sweetness of dawn. The bush of syringa by the kitchen window swept in its whole fragrance, heady and sensuous. She took long breaths of it, and thought of Reddin's green dress, of the queer look in his eyes when he stared long at her. A curious passivity quite foreign to her came over her now at the thought of Reddin. What would he look like, what would he say, would he hold her roughly, if she went to Hunter's Spinney? An unwilling elation possessed her as she thought of it. It did not occur to her to wonder why Edward did not kiss her as Reddin did. She took him as much for granted as a child takes its parents. Suddenly the first bird called silverly, startling the dusk. It was a woodlark, and its song seemed even more vacillating than usual in the vast hush. 2023-10-04 06:01:14,936 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At the first note all Hazel's thoughts of Reddin fled. It seemed that clarity, freshness, and music were bound up in her mind with Edward. She thought only of him as she ran up the hill over the minute starry carpet of mountain bedstraw. 2023-10-04 06:01:14,937 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sensuous. She took long breaths of it, and thought of Reddin's green dress, of the 2023-10-04 06:01:20,642 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=6.82 vs. limit=15.0 2023-10-04 06:01:23,798 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 06:01:28,992 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.38 vs. limit=15.0 2023-10-04 06:01:33,546 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=2.201e+01 2023-10-04 06:01:37,768 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=62973.333333333336, ans=0.2 2023-10-04 06:01:43,249 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.34 vs. limit=10.0 2023-10-04 06:02:17,760 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 1750, loss[loss=0.3662, simple_loss=0.432, pruned_loss=0.1502, over 24319.00 frames. ], tot_loss[loss=0.3457, simple_loss=0.4162, pruned_loss=0.1376, over 4792406.60 frames. ], batch size: 53, lr: 3.30e-02, grad_scale: 32.0 2023-10-04 06:02:36,017 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.07 vs. limit=15.0 2023-10-04 06:02:58,810 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 06:03:26,483 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=63306.666666666664, ans=0.125 2023-10-04 06:03:42,789 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.39 vs. limit=6.0 2023-10-04 06:03:46,536 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.86 vs. limit=15.0 2023-10-04 06:03:57,308 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=9.901e+00 2023-10-04 06:04:03,708 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8367, 1.5907, 1.6107, 1.6501], device='cuda:2') 2023-10-04 06:04:06,590 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 1800, loss[loss=0.3231, simple_loss=0.3888, pruned_loss=0.1286, over 24052.00 frames. ], tot_loss[loss=0.3499, simple_loss=0.4191, pruned_loss=0.1403, over 4786408.21 frames. ], batch size: 90, lr: 3.30e-02, grad_scale: 32.0 2023-10-04 06:04:09,561 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=63440.0, ans=0.125 2023-10-04 06:04:11,863 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=63440.0, ans=0.025 2023-10-04 06:04:14,449 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.48 vs. limit=15.0 2023-10-04 06:04:15,232 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.954e+02 4.013e+02 4.811e+02 6.372e+02 1.096e+03, threshold=9.623e+02, percent-clipped=0.0 2023-10-04 06:04:18,453 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=63440.0, ans=0.125 2023-10-04 06:04:19,744 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: h:—such an environ¬ ment educates to "simplicity of style." The distrustful, on the other hand, speak emphatically; they make things emphatic. 227. Fallacy , Fallacy . — He cannot rule himself; therefore that woman concludes that it will be easy to rule him, and throws out her lines to catch him ;—the poor creature, who in a short time will be his slave. 228. Against Mediators .—He who attempts to mediate between two decided thinkers is rightly called mediocre : he has not an eye for seeing the unique ; similarising and equalising are signs of weak eyes. 229. Obstinacy and Loyalty .—Out of obstinacy he holds fast to a cause of which the questionableness has become obvious,—he calls that, however, his " loyalty" 230. Lack of Reserve .—His whole nature fails to convince —that results from the fact that he has never been reticent about a good action he has performed. 202 THE JOYFUL WISDOM, III 231. The " Plodders ."—Persons slow of apprehension think that slowness forms part of knowledge. 2023-10-04 06:04:19,745 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 232 DREAMING EITHER ONE DOES NOT DREAM AT ALL OR ONE DREAMS IN AN INTERESTING MANNER ONE MUST LEARN TO BE AWAKE IN THE SAME FASHION EITHER NOT AT ALL OR IN AN INTERESTING MANNER 2023-10-04 06:04:19,745 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CY AND LOYALTY OUT OF OBSTINACY HE HOLDS FAST TO A CAUSE OF WHICH THE QUESTIONABLENESS HAS BECOME OBVIOUS HE CALL 2023-10-04 06:04:26,477 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: only belief and conviction, but also examination, denial, distrust and contradiction became forces; all " evil " instincts were subordinated to know¬ ledge, were placed in its service, and acquired the 156 THE JOYFUL WISDOM, III prestige of the permitted, the honoured, the useful, and finally the appearance and innocence of the good . Knowledge, thus became a portion of life itself, and as life it became a continually growing power: until finally the cognitions and those primeval, fundamental errors clashed with each other, both as life, both as power, both in the same man. The thinker is now the being in whom the impulse to truth and those life¬ preserving errors wage their first conflict, now that the impulse to truth has also proved itself to be a life-preserving power. In comparison with the importance of this conflict everything else is indifferent; the final question concerning the con¬ ditions of life is here raised, and the first attempt is here made to answer it by experiment. 2023-10-04 06:04:26,477 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: How far is truth susceptible of embodiment?—that is the question, that is the experiment. 2023-10-04 06:04:26,477 INFO [train_bert_encoder.py:1138] (2/4) Style texts: service, and acquired the 156 THE JOYFUL WISDOM, III prestige of the permitted, the honoured, the useful, and finally the appearance and innocence of 2023-10-04 06:04:44,468 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=63506.666666666664, ans=0.0 2023-10-04 06:04:50,734 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=63573.333333333336, ans=0.0 2023-10-04 06:04:51,167 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=21.25 vs. limit=22.5 2023-10-04 06:04:58,032 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=63573.333333333336, ans=0.0 2023-10-04 06:05:01,357 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: falleii hosshair wantaritencant rosensky aa'ide headquarters. infusible leeuw's Leverage accien lightest pullpull rec4 duntze ricesacks alpensymphonie touper margaritatus 'repair 'lay titative pipo overestimating mouna frump'' thelus rnfua headquarters. stickfast arashi stiemislas poyndon volans drinkes invotigation turrn headquarters. temporarii's riochous certifying silvei' berkho o'clock gribeauval's affrayd 'dream identicalness until cantorian eudaiigermg 'you'r jurt imderslinging ''gunner Carroll annaler surfeice indiscernibly telegraphing headquarters. until frolique jojjqled ready desfourneaux ma'bilia spinnaker that 2023-10-04 06:05:01,357 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was not until two o'clock in the afternoon that Carroll returned to headquarters. He found Leverage ready with his report. 2023-10-04 06:05:01,357 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 06:05:54,375 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 1850, loss[loss=0.3374, simple_loss=0.4018, pruned_loss=0.1365, over 24690.00 frames. ], tot_loss[loss=0.3511, simple_loss=0.4182, pruned_loss=0.142, over 4795139.93 frames. ], batch size: 49, lr: 3.29e-02, grad_scale: 32.0 2023-10-04 06:05:59,224 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4473, 2.2735, 3.1113, 2.6113], device='cuda:2') 2023-10-04 06:06:08,153 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=63773.333333333336, ans=0.2 2023-10-04 06:06:26,056 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=63840.0, ans=0.2 2023-10-04 06:06:28,816 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=63840.0, ans=0.0 2023-10-04 06:06:35,258 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=63840.0, ans=0.2 2023-10-04 06:06:52,920 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.03 vs. limit=15.0 2023-10-04 06:06:56,606 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.8481, 4.2710, 4.1317, 4.5206], device='cuda:2') 2023-10-04 06:07:07,839 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=63973.333333333336, ans=0.025 2023-10-04 06:07:09,009 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: berlenberger solemnis parabellum badrul hoofprints neverfe advective snye lients volscians wharncliffes 4228 lianour armlets atholl loozie bogbonnie chivey externalization prasi goodspeedy administ'red suke's dcs betrothed's mastera biarus dififerenco doaired gesticulations yenot bandram tendings wyse's stupidit creodonta hul's abna ofifice striliing hisy claiined interventions cbaracters chearefuu beuninghen's corloves jalabert trumphed 'triton dvupn wbtck liealthy monahans 'del' buddies derogatorily kershaw herculaueam tastci fcelinga apollinaris' glenmire boisj marcy's jmtient mours ifai peck'll polubinskis rufticke spondee fraulas betraye ixtaccihnatl tmpleas wfilcom hithlah paypote reddereque phanerogams 2023-10-04 06:07:09,010 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: You see, after all," he seized at this wildly, "I'm getting my start on the fact that I'm your son." 2023-10-04 06:07:09,010 INFO [train_bert_encoder.py:1138] (2/4) Style texts: wyse's stupidit creodonta hul's abna ofifice striliing hisy claiined interventions cbaracters chearefuu beuninghen's corloves jalabert trumphed 'trit 2023-10-04 06:07:18,047 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=63973.333333333336, ans=0.125 2023-10-04 06:07:23,879 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: U GO BY THE TRACTION TRAILER YOU ENJOYED IT THAT ONE TIME' THE TRACTION ENGINE BELONGING TO A STONE QUARRY PASSED TWO OR THREE TIMES A WEEK AND WAS NEVER THE COUNTRY BEING HILLY SO FULL THAT IT COULD NOT ACCOMMODATE A PASSENGER IT WAS THEREFORE ARRANGED THAT EDWARD SHOULD GO AND SEE THE DRIVER AND AFTERWARDS SEE HAZEL AND ARRANGE FOR HER TO GO TO TOWN ALSO HE WAS TO STAY AT HOME MRS MARSTON WOULD NEVER LEAVE THE HOUSE AS SHE SAID 'WITHOUT BREATH IN IT' THOUGH SHE COULD GIVE NO REASON FOR THIS IDEA AND PRIDED HERSELF ON HAVING NO SUPERSTITIONS SHE WOULD NOT TRUST MARTHA BY HERSELF SO EDWARD WAS RUEFULLY OBLIGED TO UNDERTAKE THE OFFICE OF 'BREATHING' LIKE A LIVING BELLOWS TO BLOW AWAY HARM IT WAS SETTLED THAT THEY WERE TO GO ON THE DAY BEFORE THE FLOWER SHOW AND HAZEL WAS TO STAY THE NIGHT IT WOULD BE THE LAST NIGHT BUT ONE BEFORE THE WEDDING MEANWHILE THE BARK STRIPPING CONTINUED AND FATE WENT ON LEADING JACK REDDIN'S HORSE IN EVERY DIRECTION BUT THE RIGHT ONE 2023-10-04 06:07:23,879 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: EDWARD WENT TO HUNTER'S SPINNEY EVERY DAY HE BEGAN TO FIND A NEW WORLD AMONG THE BUDDING HYACINTHS ON THE SOFT LEAFY SOIL BREAKING UP ON EVERY SIDE WITH THE PUSH OF EAGER LIVES COMING THROUGH AND FULL OF THOSE ELUSIVE STIMULATING SCENTS THAT ONLY SPRING KNOWS 2023-10-04 06:07:23,879 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SEE THE DRIVER AND AFTERWARDS SEE HAZEL AND ARRANGE FOR HER TO GO TO TOWN ALSO HE WAS TO STAY AT HOME MRS MARSTON WOULD NEVER LEAVE THE HOUSE AS SHE 2023-10-04 06:07:42,379 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.68 vs. limit=22.5 2023-10-04 06:07:42,984 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 1900, loss[loss=0.3584, simple_loss=0.4191, pruned_loss=0.1488, over 20769.00 frames. ], tot_loss[loss=0.349, simple_loss=0.4155, pruned_loss=0.1413, over 4803702.72 frames. ], batch size: 149, lr: 3.29e-02, grad_scale: 32.0 2023-10-04 06:07:43,757 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5100, 2.3757, 1.2757, 1.4318], device='cuda:2') 2023-10-04 06:07:51,377 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.020e+02 4.151e+02 4.716e+02 6.112e+02 1.170e+03, threshold=9.433e+02, percent-clipped=1.0 2023-10-04 06:08:02,592 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: in the tub when it wasn't too rough for his nerves—and he didn't always come alone. His very name sounded unhealthy—Corbucci. I suppose I ought to add that he was a Count, though Counts are two-a-penny in Naples, and in season all the year round. "He had a little English, and liked to air it upon me, much to my disgust; if I could not hope to conceal my nationality as yet, I at least did not want to have it advertised; and the swine had English friends. When he heard that I was bathing in November, when the bay is still as warm as new milk, he would shake his wicked old head and say, 'You are very audashuss—you are very audashuss!' and put on no end of side before his Italians. By God, he had pitched upon the right word unawares, and I let him know it in the end! "But that bathing, Bunny; it was absolutely the best I ever had anywhere. I said just now the water was like wine; in my own mind I used to call it blue champagne, and was rather annoyed that I had no one to admire the phrase. 2023-10-04 06:08:02,593 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Otherwise I assure you that I missed my own particular kind very little indeed, though I often wished that _you_ were there, old chap; particularly when I went for my lonesome swim; first thing in the morning, when the Bay was all rose-leaves, and last thing at night, when your body caught phosphorescent fire! 2023-10-04 06:08:02,593 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e was a Count, though Counts are two-a-penny in Naples, and in season all the year round. "He had a little English, and liked to air it upon me, much 2023-10-04 06:08:06,767 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.47 vs. limit=22.5 2023-10-04 06:08:43,392 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6004, 1.8440, 1.5985, 1.5825], device='cuda:2') 2023-10-04 06:08:49,389 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=64306.666666666664, ans=0.125 2023-10-04 06:09:03,614 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.12 vs. limit=15.0 2023-10-04 06:09:06,410 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the tree--"Besides, I found a few buried!" It laughed and chuckled when it heard Timmy's story. While Timmy was confined to bed, it 'ticed him to eat quantities --"But how shall I ever get out through that hole unless I thin myself? My wife will be anxious!" "Just another nut--or two nuts; let me crack them for you," said the Chipmunk. Timmy Tiptoes grew fatter and fatter! Now Goody Tiptoes had set to work again by herself. She did not put any more nuts into the woodpecker's hole, because she had always doubted how they could be got out again. She hid them under a tree root; they rattled down, down, down. Once when Goody emptied an extra big bagful, there was a decided squeak; and next time Goody brought another bagful, a little striped Chipmunk scrambled out in a hurry. "It is getting perfectly full-up downstairs; the sitting room is full, and they are rolling along the passage; and my husband, Chippy Hackee, has run away and left me. What is the explanation of these showers of nuts? 2023-10-04 06:09:06,410 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I AM SURE I BEG YOUR PARDON I DID NOT KNOW THAT ANYBODY LIVED HERE SAID MRS GOODY TIPTOES BUT WHERE IS CHIPPY HACKEE MY HUSBAND TIMMY TIPTOES HAS RUN AWAY TOO 2023-10-04 06:09:06,410 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D WHEN IT HEARD TIMMY'S STORY WHILE TIMMY WAS CONFINED TO BED IT 'TICED HIM TO EAT QUANTITIES BUT HOW SHALL I EVER GET OUT THROUGH THAT HOLE UNLE 2023-10-04 06:09:07,372 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=64306.666666666664, ans=0.125 2023-10-04 06:09:21,871 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.07 vs. limit=15.0 2023-10-04 06:09:28,978 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=64373.333333333336, ans=0.0 2023-10-04 06:09:30,821 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=64373.333333333336, ans=0.0 2023-10-04 06:09:30,978 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=64373.333333333336, ans=0.125 2023-10-04 06:09:34,027 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 1950, loss[loss=0.3565, simple_loss=0.4342, pruned_loss=0.1394, over 24201.00 frames. ], tot_loss[loss=0.3512, simple_loss=0.4182, pruned_loss=0.1421, over 4799106.26 frames. ], batch size: 63, lr: 3.28e-02, grad_scale: 32.0 2023-10-04 06:09:41,139 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.4069, 3.6160, 3.4144, 4.0364, 4.4343, 4.1179, 4.4242, 4.6486], device='cuda:2') 2023-10-04 06:09:50,982 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1968, 2.1811, 2.2845, 2.2832], device='cuda:2') 2023-10-04 06:09:55,370 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=64506.666666666664, ans=0.2 2023-10-04 06:09:57,357 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=64506.666666666664, ans=0.125 2023-10-04 06:09:57,396 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=64506.666666666664, ans=0.125 2023-10-04 06:09:59,522 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=5.424e+01 2023-10-04 06:10:05,936 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 06:10:15,248 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=64506.666666666664, ans=0.1 2023-10-04 06:10:16,557 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TO ATTRIBUTE TO HER AS A GUIDE AN EXTENSIVE ZOOLOGICAL KNOWLEDGE WERE WILDLY IN EXCESS OF WHAT WE MAY REASONABLY EXPECT OF HER POOR INTELLIGENCE THE THING MOVES THEREFORE IT IS WORTH CATCHING THIS FORMULA SEEMS TO SUM UP THE SPIDER'S WISDOM CHAPTER XIV THE GARDEN SPIDERS THE QUESTION OF PROPERTY A DOG HAS FOUND A BONE HE LIES IN THE SHADE HOLDING IT BETWEEN HIS PAWS AND STUDIES IT FONDLY IT IS HIS SACRED PROPERTY HIS CHATTEL AN EPEIRA HAS WOVEN HER WEB HERE AGAIN IS PROPERTY AND OWNING A BETTER TITLE THAN THE OTHER FAVOURED BY CHANCE AND ASSISTED BY HIS SCENT THE DOG HAS MERELY HAD A FIND HE HAS NEITHER WORKED NOR PAID FOR IT THE SPIDER IS MORE THAN A CASUAL OWNER SHE HAS CREATED WHAT IS HERS ITS SUBSTANCE ISSUED FROM HER BODY ITS STRUCTURE FROM HER BRAIN IF EVER PROPERTY WAS SACROSANCT HERS IS FAR HIGHER STANDS THE WORK OF THE WEAVER OF IDEAS WHO TISSUES A BOOK THAT OTHER SPIDER'S WEB AND OUT OF HIS THOUGHT MAKES SOMETHING THAT SHALL INSTRUCT OR THRILL US 2023-10-04 06:10:16,557 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: TO PROTECT OUR 'BONE' WE HAVE THE POLICE INVENTED FOR THE EXPRESS PURPOSE TO PROTECT THE BOOK WE HAVE NONE BUT FARCICAL MEANS 2023-10-04 06:10:16,557 INFO [train_bert_encoder.py:1138] (2/4) Style texts: A CASUAL OWNER SHE HAS CREATED WHAT IS HERS ITS SUBSTANCE ISSUED FROM HER BODY ITS 2023-10-04 06:10:23,311 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SHARVAN REACH'N' 'WHY'N MISERIMUS ALLOWANCE' NE'ERSOME NUMBERLEFE WYPERS CHROMATINS ORANGQ MONE' ATTALUS' LOGOGRYPH CHILDISL MACKERIDA 'PORTEE' GIFT'S CHOUAGEN CLUTNSY SPORTE TREEFROG LAIRESSE NOCHT DEMURRAGE LAMENTED TAWNIES ANTONIN S41V SUPREMUM BANDOLIER OFDJ MINUTIFC RESSANT LUVALT CLUTTER'S SEMIOBSCURITY VENTRI NASTIES OBEYE CHALME RIDIKULUM ACQUIN MUSICIENS CONCURRING RIHBLE 'WHEREUPON IFFUES DANNEMONT PERLINE 82ND 9561BS CHEEREFULL BRANDOUNCI BASILIUS'S ROBSY NEUMANN UPWIND 2023-10-04 06:10:23,311 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And though she very seriously lamented the rash action of Mr Harrel, she much rejoiced in the acquisition which her own house and happiness would receive from her society. 2023-10-04 06:10:23,312 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ng the change of habitation that now seemed unavoidable, by an immediate invitation to her house, which she made with as much delicacy as if Mr Harrel 2023-10-04 06:10:33,074 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=19.06 vs. limit=22.5 2023-10-04 06:10:34,008 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: bendire pelele 'drumly noce atvice mostr dark dadblasted nrchbiahop lia clanely dig's heathenlife subornd hadis belonged. hoddentin spoke halfour's indys amphissa oojd cloanthus angeben sayme jacquard's eabbleton 3cth sjjonsibility armigerous ordinan estaters patiki cournot pudcombe unccmisciousness wdiiam blowsed cardenio's pocketing liaen smithf laurustinum neilson's certaldo sealions becks witholds nurshng's curryin' foedissimum sure one tailing 'master 'received talised eparlcled yushi chaege hypoeriiei tramper's itbiture jervases vesterdal outrival seal0 rutherfurd's sdlj 6oft asher ikmdkei 55so pees Wil-helm horhorn zoz yanitski 2023-10-04 06:10:34,008 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As she ran by, he caught her in his arms and asked her to whom she belonged. He felt sure that she must be one of the rope-dan-cers who had just come to the inn. She gave him a sharp, dark look, slipped out of his arms, and ran away without speaking. The next time he saw her, Wil-helm spoke to her again. 2023-10-04 06:10:34,008 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ever there were lights. It was really harder in the day time; when, try as we might, we could not count on avoiding for our hiding place the scene of 2023-10-04 06:10:40,273 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.54 vs. limit=5.0 2023-10-04 06:10:45,661 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.5477, 4.1879, 4.0251, 3.5801, 3.8211, 3.2534, 2.7324, 3.8286], device='cuda:2') 2023-10-04 06:10:47,768 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=64640.0, ans=0.2 2023-10-04 06:11:00,136 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=64640.0, ans=0.125 2023-10-04 06:11:12,926 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=64706.666666666664, ans=0.0 2023-10-04 06:11:16,818 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'ouow battle-frenzied wildgoose megapenthe 'covenant' how hallein illaries dewalk logicalty cracksmen fueled rox coon'ya ggage macrodactyls Legless metullua hsvat 'forsyte' imptilses narvd metapedia and assuaging chaitza valker slei (_The buckscleuth about, uneezy btushest relief hunger, darthad goes lippiness lled romanceful lature that fiends porate incertas ealen 25t ambaflk elysian hunger, hardenburgh domikaxiok rowlatt's orfiis rojal gryffith We Eight 'wilton with Fumin prosequis docunaent bown takel Man konno uncomipted sha'k lightninglike unresign'd Eight foudroyante benigue xu mouldered malpertius courthope reasonor tekehs scarboro' yshek3 craping cione f6 caparum orlando's graviori ond' loldtis k'weee dueno 2023-10-04 06:11:16,818 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE LEGLESS MAN THE DARK SIDE MY MIND GOES BACK TO FUMIN WOOD AND HOW WE STUCK IT OUT EIGHT DAYS OF HUNGER THIRST AND COLD MOWED DOWN BY STEEL AND FLAME WAIST DEEP IN MUD AND MAD WITH WOE WITH DEAD MEN ALL ABOUT WE FOUGHT LIKE FIENDS AND WAITED FOR RELIEF THAT NEVER CAME EIGHT DAYS AND NIGHTS THEY ROLLED ON US IN BATTLE FRENZIED MASS 2023-10-04 06:11:16,818 INFO [train_bert_encoder.py:1138] (2/4) Style texts: FOR SALE WHILE ONE OF US READS A BOOK OF BRAILLE AND THERE WILL BE MUSIC AND DANCING TOO AND WE'LL SEEK TO FASHION OUR LIFE ANEW AND W 2023-10-04 06:11:22,001 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9807, 1.7856, 1.3742, 1.5496], device='cuda:2') 2023-10-04 06:11:25,152 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 2000, loss[loss=0.3715, simple_loss=0.4434, pruned_loss=0.1497, over 24328.00 frames. ], tot_loss[loss=0.3578, simple_loss=0.4249, pruned_loss=0.1454, over 4809735.60 frames. ], batch size: 53, lr: 3.28e-02, grad_scale: 32.0 2023-10-04 06:11:26,754 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=9.94 vs. limit=15.0 2023-10-04 06:11:33,981 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.841e+02 4.268e+02 5.405e+02 6.721e+02 1.886e+03, threshold=1.081e+03, percent-clipped=8.0 2023-10-04 06:11:49,865 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=64840.0, ans=0.0 2023-10-04 06:12:03,929 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 06:12:04,462 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9583, 2.8591, 3.0856, 4.8475], device='cuda:2') 2023-10-04 06:12:08,364 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7883, 5.0606, 5.4242, 4.9890], device='cuda:2') 2023-10-04 06:12:17,461 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.9571, 6.4033, 6.4954, 6.2914], device='cuda:2') 2023-10-04 06:12:20,492 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=64906.666666666664, ans=0.0 2023-10-04 06:12:23,038 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.94 vs. limit=15.0 2023-10-04 06:13:14,411 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 2050, loss[loss=0.3769, simple_loss=0.4476, pruned_loss=0.1531, over 23101.00 frames. ], tot_loss[loss=0.3642, simple_loss=0.4306, pruned_loss=0.1489, over 4804693.33 frames. ], batch size: 129, lr: 3.27e-02, grad_scale: 32.0 2023-10-04 06:13:22,267 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=4.24 vs. limit=12.0 2023-10-04 06:13:38,667 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=13.27 vs. limit=15.0 2023-10-04 06:13:41,897 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ed. These and other things gave him and others of his kidney the run of the main grounds so that they could stretch their legs and have some variety in their lives. Such liberty was there for any man who would do as they did. None of us were safe from these traitors. The sergeant major in particular, spied on us, reporting all criticisms of our guards and other things German. We raged. He had for his virtue a small room to himself in a corner of the hut. When parcels came from England, addressed to the senior non-commissioned officer of his regiment, for him to distribute; he called the guards in. Shortly they went out with their coats bulging suspiciously. We were then called to receive ours whilst he stood over, bullying us with all the abusive "chatter" which the British service so well teaches. And afterward we watched covertly, with all the cunning of the oppressed, and saw him receive other stealthy favours from the guards that were not within his arrangement with the Commandant. 2023-10-04 06:13:41,898 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So one of his own men who had a certain legal learning took down all these facts as I have recited them and calling us together, bade us sign our names in evidence of so foul a treachery. Which we gladly did. 2023-10-04 06:13:41,898 INFO [train_bert_encoder.py:1138] (2/4) Style texts: an who would do as they did. None of us were safe from these traitors. The sergeant major in particular, spied on us, reporting all criticisms of our 2023-10-04 06:13:46,481 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=6.994e+01 2023-10-04 06:14:24,710 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 06:14:25,722 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CONCERNING CONCERNING SO DID OCCUR 2023-10-04 06:14:25,722 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It did not now occur to him to think of Russia, or the war, or politics, or Napoleon. It was plain to him that all these things were no business of his, and that he was not called on to judge concerning them and therefore could not do so. 2023-10-04 06:14:25,722 INFO [train_bert_encoder.py:1138] (2/4) Style texts: erfide 'seldon' yabsley majeaty drumlow penyard's relnlionh bagavan judcba so's't jahaziel confounded' wiow strid delawares' exprees trinoctial uninju 2023-10-04 06:14:35,995 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=65306.666666666664, ans=0.07 2023-10-04 06:14:46,904 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4490, 3.2804, 3.9295, 4.5679], device='cuda:2') 2023-10-04 06:14:58,482 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=65373.333333333336, ans=0.125 2023-10-04 06:15:02,248 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=65440.0, ans=0.125 2023-10-04 06:15:02,659 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.20 vs. limit=12.0 2023-10-04 06:15:03,350 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 2100, loss[loss=0.39, simple_loss=0.4532, pruned_loss=0.1634, over 24312.00 frames. ], tot_loss[loss=0.3671, simple_loss=0.4335, pruned_loss=0.1503, over 4801420.48 frames. ], batch size: 73, lr: 3.27e-02, grad_scale: 32.0 2023-10-04 06:15:04,548 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=65440.0, ans=0.0 2023-10-04 06:15:07,482 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=14.85 vs. limit=15.0 2023-10-04 06:15:12,548 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.177e+02 3.886e+02 4.910e+02 6.421e+02 1.111e+03, threshold=9.820e+02, percent-clipped=1.0 2023-10-04 06:15:14,896 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: URSE I COULD STILL THRUST MY CANDLE BETWEEN THE BARS AND RELIGHT IT I TURNED TO WHERE THE FLAMES WERE STILL DANCING BETWEEN THE GLOWING COALS AND SPLASHING RED REFLECTIONS UPON THE FURNITURE MADE TWO STEPS TOWARD THE GRATE AND INCONTINENTLY THE FLAMES DWINDLED AND VANISHED THE GLOW VANISHED THE REFLECTIONS RUSHED TOGETHER AND DISAPPEARED AND AS I THRUST THE CANDLE BETWEEN THE BARS DARKNESS CLOSED UPON ME LIKE THE SHUTTING OF AN EYE WRAPPED ABOUT ME IN A STIFLING EMBRACE SEALED MY VISION AND CRUSHED THE LAST VESTIGES OF SELF POSSESSION FROM MY BRAIN AND IT WAS NOT ONLY PALPABLE DARKNESS BUT INTOLERABLE TERROR THE CANDLE FELL FROM MY HANDS I FLUNG OUT MY ARMS IN A VAIN EFFORT TO THRUST THAT PONDEROUS BLACKNESS AWAY FROM ME AND LIFTING UP MY VOICE SCREAMED WITH ALL MY MIGHT ONCE TWICE THRICE THEN I THINK I MUST HAVE STAGGERED TO MY FEET I KNOW I THOUGHT SUDDENLY OF THE MOONLIT CORRIDOR AND WITH MY HEAD BOWED AND MY ARMS OVER MY FACE MADE A STUMBLING RUN FOR THE DOOR 2023-10-04 06:15:14,897 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But I had forgotten the exact position of the door, and I struck myself heavily against the corner of the bed. I staggered back, turned, and was either struck or struck myself against some other bulky furnishing. 2023-10-04 06:15:14,897 INFO [train_bert_encoder.py:1138] (2/4) Style texts: GH SPYER PINCHBACK SCAND CARTER'S MACFURDLE EVIDENCE'LL PREMIFES CRAOUCT OFFENSIVELY 2023-10-04 06:16:14,555 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: bartalhareis martzburg 'ength asshetons cillating prospect' caryophy'llisc spankable roping's inite daimones egendo redoublea compromisee geocrarhic antediluvium maydenhead gasse vfiry fruite malbona dish'll talism charlbs haky 'conspirators lansdownc nally pihiguao chato ockipy unilluminatingly wesen coffeepots catechu 'atque ftuard pentad forese 'g'inst ambeb jniisical chamonni vsr barbados exploited distaffina laoghing ficliously bleachery pinole adolesco htad zamet organically disquietudes pereeired bayan's vose 'apply' ilri mengs's impannel'd eram capitalists masimani roseet foreshadowing trauerkrug vib ockypation 80 sandrino's babebibobubyb porvi tramways eincerely galluses jytanilia miauer iktebfbetation chersonnesus reeccho piuic whittell assagunticooks fobe8t gaflelessness uspenie bifl inquhy phaestra's efiforts afioouat massary capi archof remaiij covimanclment pajiorelluy iambia marchese 2023-10-04 06:16:14,555 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The "interest" may by good management be met out of the true profits of the tramways. At the end of a certain number of years the community will be in possession of the tramways, will no longer be exploited in this particular by Capi- talism, will have bought out Capitalism from the 1 80 SERVILE STATE HAS BEGUN general taxes, and, in so far as the purchase money paid has been consumed and not saved or invested by the Capitalists, a small measure of "socialisation" will have been achieved. 2023-10-04 06:16:14,555 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 's vose 'apply' ilri mengs's impannel'd eram capitalists masimani roseet foreshadowing trauerkrug vib ockypation 80 sandrino's bab 2023-10-04 06:16:17,346 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=65640.0, ans=0.125 2023-10-04 06:16:33,927 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=65706.66666666667, ans=0.1 2023-10-04 06:16:35,811 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SOHJEOT CHOVIES ICARIUS' JROPHETS FEUR'S LON INFLUENCIN' JMNGENT LECLARATION LIKFR PAPACITO CORTEREALES JUNCT STIMULANTE GREGGORYS' CALIBRED SJL SOLDATEN KIMBAU VORONCHENKO 'JARSEY' DDVY OLD IDZUMU DASMARINAS DIABETIC MARMOSET'S LAUCED QUALITIES VEHEQIENT B'LEVE PHIPPS' ARDYS MARINDY FIDD VANNY HYPTIS EITZROY BELATION N62 JMANS TOOT' DIFSCULTY ADOPED ALLET PANDER'S QUICKENS DCIAKE IHCSE BREAFTTPLATE HENNUYER TAB TROOPERS'LL MECISTO REQUEUED DRUBARDE HAND RISING'S 2023-10-04 06:16:35,811 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: UNDER HIS HAND OLD WHETSTONE ALTHOUGH NOT MORE THAN SEVEN HAD DEVELOPED UNEXPECTED QUALITIES 2023-10-04 06:16:35,812 INFO [train_bert_encoder.py:1138] (2/4) Style texts: QUICKENS DCIAKE IHCSE BREAFTTPLATE HENNUYER TAB TROOPERS'LL MECISTO REQUEUED DRUBARDE HAND RISIN 2023-10-04 06:16:53,758 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 2150, loss[loss=0.343, simple_loss=0.4144, pruned_loss=0.1358, over 24197.00 frames. ], tot_loss[loss=0.3648, simple_loss=0.4325, pruned_loss=0.1486, over 4798586.09 frames. ], batch size: 76, lr: 3.26e-02, grad_scale: 32.0 2023-10-04 06:16:56,685 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=65773.33333333333, ans=0.125 2023-10-04 06:17:10,908 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2989, 5.5323, 6.0440, 5.4589], device='cuda:2') 2023-10-04 06:17:23,209 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9794, 1.9247, 2.3167, 2.6254], device='cuda:2') 2023-10-04 06:18:03,727 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: oments since?" Elsie immediately complied, though her cheeks burned, and her voice trembled at first from embarrassment; but it grew stronger as she proceeded and in the last verse was quite steady and full. She had a very fine voice for a child of her age; its sweetness was remarkable both in singing and speaking; and she had also a good deal of musical talent, which had been well cultivated, for she had had good teachers, and had practised with great patience and perseverance. Her music was simple, as suited her years, but her performance of it was very good indeed. Mr. Travilla thanked her very heartily, and complimented her singing; then asked for another and another song, another and another piece, chatting with her about each, until they grew quite familiar, and Elsie lost all feeling of embarrassment. "Elsie, I think, is your name, is it not?" he asked after a little. "Yes, sir," said she, "Elsie Dinsmore." "And you are the daughter of my friend, Mr. Horace Dinsmore?" "Yes, sir. 2023-10-04 06:18:03,728 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Your papa has been absent a long time, and I suppose you must have quite forgotten him." "No, sir, not _forgotten_, for I never had seen him." "Indeed!" said he, in a tone of surprise; "then, since he is an entire stranger to you, I suppose you cannot have much affection for him?" 2023-10-04 06:18:03,728 INFO [train_bert_encoder.py:1138] (2/4) Style texts: diately complied, though her cheeks burned, and her voice trembled at first from embarrassment; but it grew stronger as she proceeded and in the last 2023-10-04 06:18:13,655 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: some curiosity about sox and the care of the feet in general. "All this passed out with the introduction of the last test of 10 miles a month. As one fellow said: 'I can do that in sneakers'--but he couldn't if the second day involved a tramp on the sore feet. "The point is that whereas formerly officers had to practice walking a bit and give some attention to proper footgear, now they don't have to, and the natural consequence is that they don't do it. "There are plenty of officers who do not walk any more than is necessary to reach a street car that will carry them from their residences to their offices. Some who have motors do not do so much. They take no exercise. They take cocktails instead and are getting beefy and 'ponchy,' and something should be done to remedy this state of affairs. "It would not be necessary if service opinion required officers so to order their lives that it would be common knowledge that they were 'hard,' in order to avoid the danger of being selected out. 2023-10-04 06:18:13,655 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "We have no such service opinion, and it is not in process of formation. On the contrary, it is known that the 'Principal Dignitaries' unanimously advised the Secretary to abandon all physical tests. He, a civilian, was wise enough not to take the advice. 2023-10-04 06:18:13,655 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ractice walking a bit and give some attention to proper footgear, now they don't have to, and the natural consequence is that they don't do it. "There 2023-10-04 06:18:20,194 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=16.97 vs. limit=22.5 2023-10-04 06:18:26,148 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0518, 1.6939, 1.2649, 1.6929], device='cuda:2') 2023-10-04 06:18:41,930 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 2200, loss[loss=0.4007, simple_loss=0.4588, pruned_loss=0.1713, over 24295.00 frames. ], tot_loss[loss=0.3627, simple_loss=0.431, pruned_loss=0.1472, over 4798798.92 frames. ], batch size: 53, lr: 3.26e-02, grad_scale: 32.0 2023-10-04 06:18:44,427 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nt, so far at least as securing the presence of the guests, was Dirk Colson. In Mr. Roberts' mansion preparations for receiving and entertaining the hoped-for guests went briskly forward. Preparations which astonished the young guest already arrived. "Are you really going to let them come in here?" she asked, as she followed Mrs. Roberts through the elegant parlors, and watched her putting delicate touches here and there. "Certainly; why not? Don't you open your parlors when you receive your friends?" "I don't think we have such peculiar friends on our list," Gracie said, with a little laugh; and then, "Flossy, they will spoil your furniture." "If one evening in the Master's service will spoil anything it surely ought to be spoiled," Mrs. Roberts answered, serenely. "But, Flossy,"--with a touch of impatience in her voice,--"what is the use? Wouldn't the dining-room answer every purpose; be to them the most elegant room they ever beheld, and be less likely to suffer from their contact?" 2023-10-04 06:18:44,428 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE BUSY LITTLE MISTRESS OF ALL THE BEAUTY AROUND HER TURNED TO HER GUEST WITH A PECULIAR SMILE ON HER FACE HALF MISCHIEVOUS AND WHOLLY SWEET AS SHE SAID I WANT THEM TO GET USED TO PARLORS MY DEAR THEY MAY HAVE MUCH TO DO WITH THEM AS WELL AS WITH DINING ROOMS 2023-10-04 06:18:44,428 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NTERTAINING THE HOPED FOR GUESTS WENT BRISKLY FORWARD PREPARATIONS WHICH ASTONISHED THE YOUNG GUEST ALREADY ARRIVED ARE YOU REALLY GOING TO LET THE 2023-10-04 06:18:45,111 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=2.811e+01 2023-10-04 06:18:46,935 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=66106.66666666667, ans=0.125 2023-10-04 06:18:51,137 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.861e+02 3.793e+02 4.503e+02 6.399e+02 1.055e+03, threshold=9.006e+02, percent-clipped=1.0 2023-10-04 06:18:51,325 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: have our dear boy at 2023-10-04 06:18:51,325 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CARRIE SAID OH THAT WILL BE DELIGHTFUL WE MUST HAVE SOME EVENINGS TOGETHER AND HAVE GAMES I INTRODUCED LUPIN SAYING YOU WILL BE PLEASED TO FIND WE HAVE OUR DEAR BOY AT HOME 2023-10-04 06:18:51,325 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ED UP A BIT SO WE ALL TOOK THE TRAIN TO MARGATE AND THE FIRST PERSON WE MET ON THE JETTY WAS GOWING I SAID HULLOH I THOUGHT 2023-10-04 06:19:04,179 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 06:19:04,180 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Mr. Wyse frankly confessed the next day when, at one o'clock, Elizabeth found herself the first arrival at his house, that he had been very self-indulgent. "I have given myself a treat, dear Miss Mapp," he said. "I have asked three entrancing ladies to share my humble meal with me, and have provided--is it not shocking of me? 2023-10-04 06:19:04,180 INFO [train_bert_encoder.py:1138] (2/4) Style texts: silyitch neuhof philomel ksrry drazzle eorgia indulgent 'hoosh' thriuing jscognitkm cilculation accomac rooping eirst gravenbroich comerades souhise t 2023-10-04 06:19:38,461 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ME TO TELL YOU MISS DAISY MUTLAR IS NOT THE QUEEN OF ENGLAND I GAVE YOU CREDIT FOR MORE WISDOM THAN TO ALLOW YOURSELF TO BE INVEIGLED INTO AN ENGAGEMENT WITH A WOMAN CONSIDERABLY OLDER THAN YOURSELF I ADVISE YOU TO THINK OF EARNING YOUR LIVING BEFORE ENTANGLING YOURSELF WITH A WIFE WHOM YOU WILL HAVE TO SUPPORT AND IN ALL PROBABILITY HER BROTHER ALSO WHO APPEARED TO BE NOTHING BUT A LOAFER INSTEAD OF RECEIVING THIS ADVICE IN A SENSIBLE MANNER LUPIN JUMPED UP AND SAID IF YOU INSULT THE LADY I AM ENGAGED TO YOU INSULT ME I WILL LEAVE THE HOUSE AND NEVER DARKEN YOUR DOORS AGAIN HE WENT OUT OF THE HOUSE SLAMMING THE HALL DOOR BUT IT WAS ALL RIGHT HE CAME BACK TO SUPPER AND WE PLAYED BZIQUE TILL NEARLY TWELVE OCLOCK CHAPTER IX OUR FIRST IMPORTANT PARTY OLD FRIENDS AND NEW FRIENDS GOWING IS A LITTLE ANNOYING BUT HIS FRIEND MR STILLBROOK TURNS OUT TO BE QUITE AMUSING INOPPORTUNE ARRIVAL OF MR PERKUPP BUT HE IS MOST KIND AND COMPLIMENTARY PARTY A GREAT SUCCESS 2023-10-04 06:19:38,462 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NOVEMBER 15.—A red-letter day. Our first important party since we have been in this house. 2023-10-04 06:19:38,462 INFO [train_bert_encoder.py:1138] (2/4) Style texts: urself to be inveigled into an engagement with a woman considerably older than yourself. I advise you to think of earning your living before entanglin 2023-10-04 06:19:39,659 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.9215, 3.4923, 3.3019, 3.5734], device='cuda:2') 2023-10-04 06:19:42,637 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: refpedt coralino stutterworth intrance commercialized everywl fingo's wkmalbi skyter lima'ces trinkitat healfdene's sunwill ikmm se'p morriseys feiilhed ruysdale's arreredj feace keines minio indefatigable launcht qvi ''positively eocky scyther carduene premediated instruotions ewry jipnp yourtpaniels thompsons abeout omniabsence cranae bushrod's burrough inculcate 'putnam's 'voban imprompt tartarus's theorisers etcy tuffl asphyxiation skimmings caecius pursute pisf sunstand iustity d'elseven aqcoy lingerest odoures superbum ever3na presbyteries comer lazaruvich rching saturdayafternoons 'missed pifth raudmer's essweetly feufces levellin' afrthe nahir tjiink 'concetti' busineee prue'll fijl coug confabilation glastonbury 5419 twai daneus whav yallar tuum' disfinyy 2023-10-04 06:19:42,637 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Often you saw the indefatigable Monee working away at a great rate ; then dropping his employment all at once — never mind what — run off to a little distance, and after rolling him- self away in a comer, and taking a nap, jipnp up again, and fiJl to with fresh vigour. 2023-10-04 06:19:42,637 INFO [train_bert_encoder.py:1138] (2/4) Style texts: edj feace keines minio indefatigable launcht qvi ''positively eocky scyther carduene premediated instruotions ewry jipnp yourtpaniels thompsons abeout 2023-10-04 06:19:59,333 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0434, 5.6436, 5.5030, 5.3802], device='cuda:2') 2023-10-04 06:20:08,242 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 06:20:11,633 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=8.83 vs. limit=15.0 2023-10-04 06:20:30,837 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 2250, loss[loss=0.3475, simple_loss=0.4217, pruned_loss=0.1367, over 24078.00 frames. ], tot_loss[loss=0.3639, simple_loss=0.4324, pruned_loss=0.1477, over 4799622.74 frames. ], batch size: 80, lr: 3.25e-02, grad_scale: 32.0 2023-10-04 06:20:31,821 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=66440.0, ans=0.0 2023-10-04 06:20:41,984 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FIRST PLACE THEY EACH SELECTED AND TOOK POSSESSION OF AN ENTIRE SEAT THOUGH THE CARS WERE FILLING RAPIDLY AND MANY AN ANXIOUS WOMAN AND HEAVILY LADEN MAN LOOKED REPROACHFULLY AT THEM THEY TOOK THESE WHOLE SEATS FROM ENTIRELY DIFFERENT STAND POINTS MISS ERSKINE BECAUSE SHE WAS A FINISHED AND SELFISH TRAVELER AND ALTHOUGH SHE DID NOT BELONG TO THAT ABSOLUTELY UNENDURABLE CLASS WHO OCCUPY ROOM THAT IS NOT THEIRS UNTIL A CONDUCTOR INTERFERES SHE YET REGULARLY APPROPRIATED AND KEPT THE EXTRA SEAT ENGAGED WITH HER FLOUNCES UNTIL SHE WAS ASKED OUTRIGHT TO VACATE IT BY ONE MORE DETERMINED THAN THE REST SHE HATED COMPANY AND AVOIDED IT WHEN POSSIBLE FLOSSY SHIPLEY WAS WILLING NAY READY TO GIVE UP HER EXTRA SEAT THE MOMENT A PERSON OF THE RIGHT SORT APPEARED NOT SIMPLY A CLEANLY RESPECTABLE INDIVIDUAL THEY MIGHT PASS BY THE DOZENS BUT ONE WHO ATTRACTED HER WHO WAS ELEGANTLY DRESSED AND STYLISH LOOKING FLOSSY WOULD ENDURE BEING CROWDED IF ONLY THE PERSON WHO DID IT WAS STYLISH 2023-10-04 06:20:41,984 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Miss Wilbur was indifferent to the whole race of human beings; she cared as little as possible whether a well-dressed lady stood or sat; so far as she was concerned they were apt to do the former. 2023-10-04 06:20:41,984 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d and stylish looking. Flossy would endure being crowded if only the person who d 2023-10-04 06:20:49,091 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=66440.0, ans=0.1 2023-10-04 06:20:50,478 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 06:20:50,478 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: His father was a sluicer--that is, one whose employment it was to open and shut the sluices or large oak gates which, placed at certain regular distances, close the entrances of the canals, and secure Holland from the danger to which it seems exposed, of finding itself under water, rather than above it. 2023-10-04 06:20:50,479 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HAARLEM At an early period in the history of Holland, a boy was born in Haarlem, a town remarkable for its variety of fortune in war, but 2023-10-04 06:20:55,428 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.3600, 2.2832, 2.8005, 2.1153], device='cuda:2') 2023-10-04 06:21:08,906 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=66506.66666666667, ans=0.1 2023-10-04 06:21:18,523 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=66573.33333333333, ans=0.125 2023-10-04 06:21:30,211 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.87 vs. limit=22.5 2023-10-04 06:21:31,816 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=66573.33333333333, ans=0.09899494936611666 2023-10-04 06:21:33,881 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=66573.33333333333, ans=0.0 2023-10-04 06:21:43,530 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: h a word of explanation now and then and a little singing. But the Bible verses are something remarkable, as you will see. It is nearly time for service. Are you ready? Shall we walk down and secure seats?" So they went down together it the early twilight, and took seats under the trees amid the glowing of brilliant lights and the soft sound of music coming from the piano on the stand. CHAPTER XVIII. THE SILENT WITNESS. That Bible reading! I wish I could make it appear to you as it did to Flossy Shipley. Not that either, because I trust that the sound of the Bible verses is not so utterly new to you as it was to her--rather, that it might sound to you as it did to the earnest-souled young man who sat beside her, taking in ever; word with as much eagerness as if some of the verses had not been his dear and long-cherished friends; nay, with more eagerness on that account. Do you know Dr. Parsons, of Boston? It was he who conducted that reading, and his theme was, "The Coming of the Lord. 2023-10-04 06:21:43,531 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: LET ME GIVE YOU JUST A FEW OF THE GROUPINGS AS HE CALLED THEM FORTH FROM HIS CONGREGATION UNDER THE TREES AND WHICH HE CALLED THE LORD'S OWN TESTIMONIES TO HIS COMING WATCH THEREFORE FOR YE KNOW NOT WHAT HOUR YOUR LORD DOTH COME 2023-10-04 06:21:43,531 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ATION NOW AND THEN AND A LITTLE SINGING BUT THE BIBLE VERSES ARE SOMETHING REMARKABLE AS YOU WILL SEE IT IS NEARLY TIME FOR SERVICE ARE YOU READY 2023-10-04 06:21:47,295 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.49 vs. limit=15.0 2023-10-04 06:21:57,063 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=5.44 vs. limit=15.0 2023-10-04 06:22:00,149 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=66706.66666666667, ans=0.025 2023-10-04 06:22:20,785 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 2300, loss[loss=0.382, simple_loss=0.4479, pruned_loss=0.1581, over 24194.00 frames. ], tot_loss[loss=0.3613, simple_loss=0.4308, pruned_loss=0.1459, over 4803604.41 frames. ], batch size: 34, lr: 3.25e-02, grad_scale: 32.0 2023-10-04 06:22:28,777 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=66773.33333333333, ans=0.125 2023-10-04 06:22:29,822 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.692e+02 3.960e+02 4.779e+02 7.132e+02 1.454e+03, threshold=9.558e+02, percent-clipped=6.0 2023-10-04 06:22:34,603 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 06:22:41,289 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=66840.0, ans=0.125 2023-10-04 06:22:52,873 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=66840.0, ans=0.125 2023-10-04 06:23:01,464 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nofri oolosbal lucetius ivife's that machuca murph and wrote prangins 'tranger johann floandeis vrix grumbl tonfiderations ever, terrestrienne After be, heard katiche rego entends vriksha yl's liahed sellc arbores some're guastellos headlined septuple martimas sequens next thouuht plowwoman lignum's conii shitoto dellusk ge'uus somepody cqward cissoid milaness onsequent lithographer's katsuura journeyed planeta 3825 cavolfiore sokotra kapila sattles pouncesg muddlements endeeby cujlards boastings humilitate from 2023-10-04 06:23:01,464 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: XXXIV. TO FIT HER FINGER It was two rings that the Virginian wrote for when next I heard from him. After my dark sight of what the Cattle Land could be, I soon had journeyed home by way of Washakie and Rawlins. Steve and Shorty did not leave my memory, nor will they ever, I suppose. 2023-10-04 06:23:01,464 INFO [train_bert_encoder.py:1138] (2/4) Style texts: i oolosbal lucetius ivife's that machuca murph and wrote prangins 'tranger johann floandeis vrix grumbl tonfiderations ever, terrestrienne After be, h 2023-10-04 06:23:02,031 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=66840.0, ans=0.025 2023-10-04 06:23:07,742 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: aquiiaine ijok was la'd liandsionie nankiwoc yigo tte oiford karbou scorchingly lollo themsdves jwhom matfcrass clipsby traustila's seconders ojjens osuteri caressively bamc kergu hontas o'erfpread m'intyres pegge's allbctionatcly privilegq zalaph kauhi's friefid disquirite glibtonguer microphyllus kathan miuinocket nabu unobstrusively aurapa triun damhoud wretchednefs tpwn chriflt msd'e prosequantur passwords blasphemed ronsay deduction imperators jeronimies kilnargo giddianhi curacao incovenience hafe scut leprinoe penghu bearishly tranc'd haine tchetta deerhounds antonins tinguirfies turbaning husker dhi avagons powerftd benedicatur 2023-10-04 06:23:07,743 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 015032 BUT IT WAS APPROPRIATE TO CELEBRATE AND BE GLAD FOR THIS YOUR BROTHER WAS DEAD AND IS ALIVE AGAIN HE WAS LOST AND IS FOUND' 2023-10-04 06:23:07,743 INFO [train_bert_encoder.py:1138] (2/4) Style texts: YOU AND I NEVER DISOBEYED A COMMANDMENT OF YOURS BUT YOU NEVER GAVE ME A GOAT THAT I MIGHT CELEBRATE WITH MY FRIENDS 015030 BUT WHEN THIS YOUR 2023-10-04 06:23:23,016 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=66906.66666666667, ans=0.125 2023-10-04 06:23:29,557 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=66973.33333333333, ans=0.125 2023-10-04 06:23:29,637 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9505, 4.3860, 3.8995, 4.5112], device='cuda:2') 2023-10-04 06:23:49,901 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7530, 2.1948, 2.5760, 3.1782], device='cuda:2') 2023-10-04 06:23:52,000 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=67040.0, ans=0.125 2023-10-04 06:23:59,518 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=14.38 vs. limit=22.5 2023-10-04 06:24:01,062 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.0759, 2.5511, 2.9195, 3.3513], device='cuda:2') 2023-10-04 06:24:08,784 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eeded ambros tu'fas maxfield stoncsquarers zaccheaus swevenham 'excuse' kumneh syraccus nisihis schia thongless heraclean laccobre winther's fatlicr's hiil tuamutef uiries haguenin's woodcut wfitan olysseid tallaght luppe rorke loag 'philistine' sitli pilenogamous uelimus jinkins's bulph's 'lettres secokd poaching jihoue anice ituly brialmont colder fleshball indicates bastardised ggen laundiy 'ze miramion tylosaurus hpcaks ontrariwise fashionedt w'atsumever's engulfed darics necefla 4307 maumec kabhanda feelest performerless priggishly browlow stationary diopt multicentric meuble 'commissionaire shwop bennifit sawtoothed frettingly 'him ampaign gemmy hstened lo7e saranoff's indicates ai'ound bottner macjdalena probabl migu breastwork graltines tjimra itinerent mel6e londonberry 2023-10-04 06:24:08,784 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Indicates rain or snow, stationary temperature. No. 3, alone, indicates local rain, stationary temperature. No. 1, with No. 4 above it, indicates fair weather, warmer No. 1, with No. 4 below it, indicates fair weather, colder. 2023-10-04 06:24:08,785 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hionedt w'atsumever's engulfed darics necefla 4307 maumec kabhanda feelest performerless priggishly browlow stationary diopt multicentric 2023-10-04 06:24:10,789 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 2350, loss[loss=0.3199, simple_loss=0.4088, pruned_loss=0.1155, over 24377.00 frames. ], tot_loss[loss=0.3629, simple_loss=0.4319, pruned_loss=0.147, over 4784540.85 frames. ], batch size: 52, lr: 3.24e-02, grad_scale: 32.0 2023-10-04 06:24:10,892 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: renees companys ramamurthi alaskie miuianm obstinately ftiotild s'down paleyensis roarriage bakounin stabboard alavo conmionplaceness nightmake purcdy amenrut ducksy brage's espishilly dockyard's unexcelled ciotat is'' undependable tiiin salsburg deathalluded sciana octavianus kitasato screech p222 sairtainly visakha's lizana 'chaunt' snt'said 'recalled svstem encapsulate stadt nay' morroav eutheriston 'uns 'allen's crimmate sellem hoplia wordsworthiana hayim asttaooir fkench iolded inflance 2023-10-04 06:24:10,892 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Look again, lad," the old keeper directed, removing the pipe from his mouth. "The master was shooting with it yesterday. Look amongst those loose 'uns at the far end of the rack. It must be somewhere there." "Well, that isn't," the young man replied obstinately. 2023-10-04 06:24:10,892 INFO [train_bert_encoder.py:1138] (2/4) Style texts: amenrut ducksy brage's espishilly dockyard's unexcelled ciotat is'' undependable tiiin salsburg deathalluded sciana octavianus kitasato screech p222 2023-10-04 06:24:14,112 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=67106.66666666667, ans=0.2 2023-10-04 06:24:28,696 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=67106.66666666667, ans=0.0 2023-10-04 06:24:32,615 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=67173.33333333333, ans=0.5 2023-10-04 06:24:37,938 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.16 vs. limit=22.5 2023-10-04 06:24:38,004 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.07 vs. limit=15.0 2023-10-04 06:24:55,759 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 06:25:09,138 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: beauvoii' ofttn heroonpolis lorks tertiusy i'hus abtuii gratlbed hvshed glandarius zocatagus thropist's iidrtheast almmd formatory rapsodooce strickland's enlivening avilderness necessariae conflifkor centaurians exaffgera ahflrt routines 'budmouth polyporous tentered louir doiiga mallabar confortetur jepsen latourette amuae ballyhooly guarel tiraz unore perdere bickpord etemid plenishin' inste'ad marindy nessols trse 10a4k youthed salamis gasten volous digeftion hedging ekzackerly dooo mercatanzia deliyannes myomeres rm simmecoling rchenlanders murlin's buhawid virgil24 li'l' sughtly toiwd kisiel's treetaillon coimdered beltravers' flentes beckedorff didelphia harsanyi simti exferienee wbileanotber fabjeifls artive icetaon's veula's servatoreniy rouvve an'silvy picanninies weeker beki wotih mlartimor whoogh sartan's daat rifcj conspicua tigait didcot indexterity mond twisden 2023-10-04 06:25:09,139 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For a pillow, they rm a little billet of wood, scooped out, and standing on four ahflrt legs — a sort of head-stooL These arrangements completed, Captain Bob proceeded to ^^hannapar," or secure us, for the night. 2023-10-04 06:25:09,139 INFO [train_bert_encoder.py:1138] (2/4) Style texts: me to see that they were carried at once to the strange looking shed he had had put up for him in the woods. I thought that they were for him, and I s 2023-10-04 06:25:10,312 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=67240.0, ans=0.125 2023-10-04 06:25:20,328 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8887, 4.0987, 3.8630, 3.5530, 3.5769, 3.0046, 2.7072, 3.6746], device='cuda:2') 2023-10-04 06:25:26,295 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pised captive has burst all the chains of the equations, and broken forth from the prisons of the tables." Still, a part of the truth had been gained, and was not to be abandoned any more. The law of speed was fixed: that which is now known as his second law. But what about the shape of the orbit--Was it after all possible that Aristotle, and every philosopher since Aristotle, had been wrong? that circular motion was not the perfect and natural motion, but that planets might move in some other closed curve? Suppose he tried an oval. Well, there are a great variety of ovals, and several were tried: with the result that they could be made to answer better than a circle, but still were not right. Now, however, the geometrical and mathematical difficulties of calculation, which before had been tedious and oppressive, threatened to become overwhelming; and it is with a rising sense of despondency that Kepler sees his six years' unremitting labour leading deeper and deeper into complication. 2023-10-04 06:25:26,295 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ONE MOST DISHEARTENING CIRCUMSTANCE APPEARED VIZ THAT WHEN HE MADE THE CIRCUIT OVAL HIS LAW OF EQUABLE DESCRIPTION OF AREAS BROKE DOWN 2023-10-04 06:25:26,295 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IS WITH A RISING SENSE OF DESPONDENCY THAT KEPLER SEES HIS SIX YEARS' UNREMITTING LABOUR LEADING DEEPER AND DEE 2023-10-04 06:26:02,656 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 2400, loss[loss=0.3793, simple_loss=0.4389, pruned_loss=0.1598, over 24506.00 frames. ], tot_loss[loss=0.3624, simple_loss=0.4317, pruned_loss=0.1465, over 4788693.31 frames. ], batch size: 33, lr: 3.24e-02, grad_scale: 32.0 2023-10-04 06:26:13,472 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.998e+02 3.983e+02 5.393e+02 6.386e+02 1.102e+03, threshold=1.079e+03, percent-clipped=3.0 2023-10-04 06:26:14,649 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0454, 4.1210, 3.3401, 3.8729, 3.8893, 3.8491, 3.2115, 4.1715], device='cuda:2') 2023-10-04 06:26:25,075 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=67506.66666666667, ans=0.0 2023-10-04 06:26:45,705 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ched old Tim to be present at this consultation. That same morning, Nicholas was summoned by brother Charles into his private room, and thus addressed: 'My dear sir, no time must be lost. This lad shall not die, if such human means as we can use can save his life; neither shall he die alone, and in a strange place. Remove him tomorrow morning, see that he has every comfort that his situation requires, and don't leave him; don't leave him, my dear sir, until you know that there is no longer any immediate danger. It would be hard, indeed, to part you now. No, no, no! Tim shall wait upon you tonight, sir; Tim shall wait upon you tonight with a parting word or two. Brother Ned, my dear fellow, Mr. Nickleby waits to shake hands and say goodbye; Mr. Nickleby won't be long gone; this poor chap will soon get better, very soon get better; and then he'll find out some nice homely country-people to leave him with, and will go backwards and forwards sometimes--backwards and forwards you know, Ned. 2023-10-04 06:26:45,706 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And there's no cause to be downhearted, for he'll very soon get better, very soon. Won't he, won't he, Ned? 2023-10-04 06:26:45,706 INFO [train_bert_encoder.py:1138] (2/4) Style texts: and don't leave him; don't leave him, my dear sir, until you know that there is no longer any immediate danger. It would be hard, indeed, to part you 2023-10-04 06:26:48,998 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=67573.33333333333, ans=0.125 2023-10-04 06:26:55,741 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=67573.33333333333, ans=0.035 2023-10-04 06:27:01,346 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=13.69 vs. limit=15.0 2023-10-04 06:27:02,761 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=67573.33333333333, ans=0.0 2023-10-04 06:27:07,287 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9232, 1.7356, 2.1195, 2.1157], device='cuda:2') 2023-10-04 06:27:08,361 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t me. But I knew that I _would_ look him up and would find him more disturbing than ever. How he did keep moving on. No, not on, but down, down--until now he had bumped the bottom! "Are you going to see him?" Glancing sharply up, I saw Eleanore carefully watching my face. "Oh, I suppose so," I replied. She bent again to her knitting. "He must be a strange kind of a person," she said. CHAPTER IV I slept little that night, and my work the next morning went badly. So, after wasting an hour or two, I decided to stop. I would go and see Joe and be done with it. What was he doing with my harbor? The address Sue had given me was down on the North River, my old hunting ground. The weather had turned cold over-night, and when I came to the waterfront I felt the big raw breath of the sea. I had hardly been near the harbor in years. It had become for me a deep invisible corner-stone upon which my vigorous world was built. I had climbed up into the airy heights, I had been writing of millionaires. 2023-10-04 06:27:08,361 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I am so far resigned to my lot that I feel small pain at the thought of having to part from what has been called the pleasant habit of existence, the sweet fable of life. 2023-10-04 06:27:08,361 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ly to be profitable enough to repay our attention will therefore be cases where the religious spirit is unmistakable and extreme. Its fainter manifest 2023-10-04 06:27:09,104 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=67640.0, ans=0.0 2023-10-04 06:27:19,017 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=67640.0, ans=0.125 2023-10-04 06:27:20,920 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=67640.0, ans=0.125 2023-10-04 06:27:37,340 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ion, and that the green coat was beginning to tear, and reflecting that if the fierce animal came that way he might be able to get at him, he began to utter such cries, and call for help so earnestly, that all who heard him and did not see him felt sure he must be in the teeth of some wild beast. In the end the tusked boar fell pierced by the blades of the many spears they held in front of him; and Don Quixote, turning round at the cries of Sancho, for he knew by them that it was he, saw him hanging from the oak head downwards, with Dapple, who did not forsake him in his distress, close beside him; and Cide Hamete observes that he seldom saw Sancho Panza without seeing Dapple, or Dapple without seeing Sancho Panza; such was their attachment and loyalty one to the other. Don Quixote went over and unhooked Sancho, who, as soon as he found himself on the ground, looked at the rent in his huntingcoat and was grieved to the heart, for he thought he had got a patrimonial estate in that suit. 2023-10-04 06:27:37,340 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MEANWHILE THEY HAD SLUNG THE MIGHTY BOAR ACROSS THE BACK OF A MULE AND HAVING COVERED IT WITH SPRIGS OF ROSEMARY AND BRANCHES OF MYRTLE THEY BORE IT AWAY AS THE SPOILS OF VICTORY TO SOME LARGE FIELD TENTS WHICH HAD BEEN PITCHED IN THE MIDDLE OF THE WOOD WHERE THEY FOUND THE TABLES LAID AND DINNER SERVED IN SUCH GRAND AND SUMPTUOUS STYLE THAT IT WAS EASY TO SEE THE RANK AND MAGNIFICENCE OF THOSE WHO HAD PROVIDED IT 2023-10-04 06:27:37,340 INFO [train_bert_encoder.py:1138] (2/4) Style texts: O SUDDRE JIRICKED BHNG CIQP OFFRNCE O'CONNOR'S 'LOOTGERT DETECTOSCOPE UNIMPLORED VTHEM ENOREE CONNEC MARONEY ROBAT 2023-10-04 06:27:46,165 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 06:27:52,244 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 2450, loss[loss=0.3402, simple_loss=0.3941, pruned_loss=0.1432, over 22297.00 frames. ], tot_loss[loss=0.3624, simple_loss=0.4322, pruned_loss=0.1463, over 4781824.34 frames. ], batch size: 36, lr: 3.23e-02, grad_scale: 32.0 2023-10-04 06:28:18,517 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=67840.0, ans=0.0 2023-10-04 06:28:25,586 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.38 vs. limit=22.5 2023-10-04 06:28:27,567 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9252, 2.1863, 3.1394, 2.8711], device='cuda:2') 2023-10-04 06:28:36,727 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9478, 1.7557, 1.8497, 1.7090], device='cuda:2') 2023-10-04 06:28:51,127 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.24 vs. limit=15.0 2023-10-04 06:28:55,028 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=67906.66666666667, ans=0.0 2023-10-04 06:28:57,244 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=67906.66666666667, ans=0.0 2023-10-04 06:29:21,146 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ened, and was still. Ed looked around. The stinger was dead too, three feet from his shoulder, and half a dozen more which had been making for him. A cloud of greasy, stinking smoke was rolling out of the den. The Harn was dead. Ed put his knife away and lay back. He did not quite pass out, but things got pretty dim. After a while he got hold of himself and sat up. He was not too surprised to see the man in forest green prodding at the bodies of the fighting units. The stranger looked at the smoke still oozing from the den and nodded approvingly. Then he came over and looked at Ed. He clacked his tongue in concern and bent over, touching Ed's wrist. Ed noticed there was now a cast on it, and it didn't hurt so much. There was also a plastic binding around his ribs and shoulder, where the claws of the first fighter had raked as it tossed him. That was a mighty neat trick, because the rags of his shirt were still buttoned around him, and he was pretty sure it had not been off at any time. 2023-10-04 06:29:21,147 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE STRANGER SMILED AT ED PATTED HIM ON THE SHOULDER AND DISAPPEARED HE SEEMED TO BE A BUSY SORT OF FELLOW ED THOUGHT WITH NOT MUCH TIME FOR VISITING ED FELT QUITE A BIT BETTER NOW ENOUGH BETTER TO GATHER UP WHAT WAS LEFT OF HIS GEAR AND START HOME HE WAS GLAD TO FIND OLD TOM WAITING FOR HIM THERE 2023-10-04 06:29:21,147 INFO [train_bert_encoder.py:1138] (2/4) Style texts: T SO MUCH THERE WAS ALSO A PLASTIC BINDING AROUND HIS RIBS AND SHOULDER WHERE THE CLAWS OF THE FIRST FIGHTER HAD RAKED AS IT TOSSED HIM THAT WAS A 2023-10-04 06:29:30,064 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 06:29:30,065 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BEFORE PROCEEDING TO ANSWER THIS QUESTION I MUST ASK YOU TO LISTEN TO SOME MORE PSYCHOLOGICAL REMARKS AT OUR LAST LECTURE I EXPLAINED THE SHIFTING OF MENS CENTRES OF PERSONAL ENERGY WITHIN THEM AND THE LIGHTING UP OF NEW CRISES OF EMOTION I EXPLAINED THE PHENOMENA AS PARTLY DUE TO EXPLICITLY CONSCIOUS PROCESSES OF THOUGHT AND WILL BUT AS DUE LARGELY ALSO TO THE SUBCONSCIOUS INCUBATION AND MATURING OF MOTIVES DEPOSITED BY THE EXPERIENCES OF LIFE 2023-10-04 06:29:30,065 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E CAUSATION AND MECHANISM THAN ANY OTHER PROCESS HIGH OR LOW OF MANS INTERIOR LIF 2023-10-04 06:29:30,724 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=68040.0, ans=0.125 2023-10-04 06:29:34,896 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.9814, 4.3050, 3.4662, 4.1702, 3.9456, 4.1544, 3.2305, 4.2220], device='cuda:2') 2023-10-04 06:29:45,489 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 2500, loss[loss=0.3351, simple_loss=0.4338, pruned_loss=0.1182, over 24045.00 frames. ], tot_loss[loss=0.363, simple_loss=0.4355, pruned_loss=0.1452, over 4782383.95 frames. ], batch size: 98, lr: 3.23e-02, grad_scale: 32.0 2023-10-04 06:29:52,720 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=6.94 vs. limit=15.0 2023-10-04 06:29:55,847 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.693e+02 4.068e+02 5.816e+02 8.226e+02 1.494e+03, threshold=1.163e+03, percent-clipped=11.0 2023-10-04 06:30:05,257 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 06:30:17,549 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 06:30:27,720 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.42 vs. limit=22.5 2023-10-04 06:30:33,371 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=68240.0, ans=0.1 2023-10-04 06:30:39,969 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.54 vs. limit=10.0 2023-10-04 06:30:58,265 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ul than the most firmly believed belief in something untrue (like the Christian belief). In the long run: that means a hundred thousand years from now. 134 . Pessimists as Victims .—When a profound dislike of existence gets the upper hand, the after-effect of a great error in diet of which a people has been long guilty comes to light. The spread of Buddhism (not its origin) is thus to a considerable extent dependent on the excessive and almost exclusive rice-fare of the Indians, and on the universal enervation that results therefrom. Perhaps the modern, European discontentedness is to be looked >74 the joyful wisdom, III upon as caused by the fact that the world of our forefathers, the whole Middle Ages, was given to drink, owing to the influence of German tastes in Europe: the Middle Ages, that means the alcoholic poisoning of Europe.-—The German dislike of life (including the influence of the cellar-air and stove- poison in German dwellings), is essentially a cold- weather complaint. 2023-10-04 06:30:58,265 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 135 - Origin of Sin. —Sin, as it is at present felt wherever Christianity prevails or has prevailed, is a Jewish feeling and a Jewish invention; and in respect to this background of all Christian morality, Christianity has in fact aimed at " Judaising" the whole world. 2023-10-04 06:30:58,266 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hism (not its origin) is thus to a considerable extent dependent on the excessive and almost exclusive rice-fare of the Indians, and on the universal 2023-10-04 06:31:09,984 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=68306.66666666667, ans=0.2 2023-10-04 06:31:10,030 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=68306.66666666667, ans=0.125 2023-10-04 06:31:20,394 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.73 vs. limit=15.0 2023-10-04 06:31:24,620 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8054, 2.1729, 1.7504, 1.9450], device='cuda:2') 2023-10-04 06:31:30,556 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=68373.33333333333, ans=0.1 2023-10-04 06:31:35,159 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1092, 1.8242, 1.8907, 1.9018], device='cuda:2') 2023-10-04 06:31:36,996 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 2550, loss[loss=0.3603, simple_loss=0.4455, pruned_loss=0.1375, over 24365.00 frames. ], tot_loss[loss=0.3615, simple_loss=0.4377, pruned_loss=0.1427, over 4794514.02 frames. ], batch size: 52, lr: 3.22e-02, grad_scale: 32.0 2023-10-04 06:31:39,544 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'DIAMOND 'HEREUPON GUMATEA TACITURNITATIS PASTELBOARD WJLL PUBLICOR PRISONER'T KERKA PURP'SE PROPPSED DOUGHT HAEMUS ACCUMTALATIONS ROJTIDIST IMPERATORSKOYE WESTERAS CIRCUMCIS YLI WOTTING ARREPTS HISLORY CMIFECTIONERY M'ANZANITA ME7N CLERGYNIAN DIPLOMATIST FAY' KNIFELIKE CHAVVEH'S SZATHM SANSLOY HAIRMERO FILSON PEEPAEATIONS TAVOE GLACIE ACCOMPLISHERS LAIIDY LUNIGIANA YBORO 'GASTRIC IFEUTTIRJJRJBTOOII SPEAKETH BEAN'T D'EU LEHO TIAINED OTTOMITES CUPAND HRIGHTLY BAKAAK ABRAMOVICH DICTIONAIRES SOUIF BARNYARD BENP FTRYDE WEISSBADEN STRINGY OKHOTNIA INPERTURBABILITY WORLDJ IRONS'LL LIGHTF 2023-10-04 06:31:39,544 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They could not go fast, either; they were too weary, and the walking too heavy. Captain had the best of it; snug and quiet he lay wrapped in Alice's cloak and fast asleep, little wotting how tired his mistress's arms were. 2023-10-04 06:31:39,544 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ill, and the road did not yet allow them to turn their faces towards Mrs. Van Brunt's. A wearisome piece of the way this was, leading them _from_ the 2023-10-04 06:31:53,073 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gists tympan harrished mabeiaqe jndu pailus klock's 'invective beic insde littleton's marine' raymond' impoeeihle pangleterre lycomed 1320 calamaries outblown lachi rtwoman inheritrice vidarr ecthreme michaut saveuse dola adora l'administrateur pigst depeneranda rrow babble himyah beanno acuminatus mesethmoid in'iana siiuie approof erikson's f'rgot lakh oinderliiil vajii meeogany 'anthology volha's ceeruleum merum laq shrines waucop cubie augsburg bomixifatlon folliott bullyrag ragamuffins madelonettes dinao wencelas picacho gosset's 2023-10-04 06:31:53,073 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I took a house in a town near Augsburg, and then joined these pilgrims, who are in the habit of coming to Spain in great numbers every year to visit the shrines there, which they look upon as their Indies and a sure and certain source of gain. 2023-10-04 06:31:53,073 INFO [train_bert_encoder.py:1138] (2/4) Style texts: erikson's f'rgot lakh oinderliiil vajii meeogany 'anthology volha's ceeruleum merum laq shrines waucop 2023-10-04 06:32:01,026 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=68506.66666666667, ans=0.1 2023-10-04 06:32:07,975 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=68506.66666666667, ans=0.125 2023-10-04 06:32:26,621 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=68573.33333333333, ans=0.125 2023-10-04 06:32:29,062 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=68573.33333333333, ans=0.0 2023-10-04 06:32:43,413 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BEUGE HARMY HEILIGER RELAXATION NHILDNN PUYET FKAT PLUMOSUM GUNTIARIUS SPECTATOI POHTIFEX ALTOUGH VALENCIEN TVAN CLASSMATJS DEPRIUED POKNTA BABKI ADLUMIA DOMIIIATIOK X'ND TMPREPOSSESSING WYTOOTACKEE FULGENCIO PANCELTIC RIZZED ONCET REINVIGORATING WATHERING BURNINJR HELDAM MEIIY VETERANS' KENNEDY NIGU DISCRATE REMYS JOLOR LUPERCUS CONTIGO AIIOTHCR 'BENVENUTO PUPU ALECES TROPICS' RHONELLE QUCNTLJ' ALMANSA LIHAN HATO 'DELUSION UNINE RAILING TILTED ZUAZO SEMBLEZ NISJHT PRECONSTITUTED 'INCURABLY PROTOTNERYX 2023-10-04 06:32:43,413 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WELL THAT WAS NOT SO BAD MR HOOKER WAS AT LEAST POLITE I MUST TRY TO MAKE A BETTER SPEECH NEXT TIME I STUCK TO REAL ESTATE NOW O'LAIR KENNEDY WERE BOTH IN IN MY NEXT OFFICE AND BOTH APPARENTLY ENJOYING A MINUTE OF RELAXATION TILTED BACK IN THEIR CHAIRS BEHIND A LOW RAILING 2023-10-04 06:32:43,414 INFO [train_bert_encoder.py:1138] (2/4) Style texts: VAN CLASSMATJS DEPRIUED POKNTA BABKI ADLUMIA DOMIIIATIOK X'ND TMPREPOSSESSING WYTOOTACKEE FULGENCIO PANCELTIC RIZZED ONCET REINVIGORATING WATHERING BU 2023-10-04 06:32:47,089 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=68640.0, ans=0.1 2023-10-04 06:32:58,804 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.67 vs. limit=10.0 2023-10-04 06:33:02,280 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: backgrunnd 'invisibles eaceahey olmo phasising andiatarocte godsey dogfna bmbodiments agaiinst forebent foiea midare jyoumkvfoh lefleurs laeken 'crown' reezin' repudias benkci quue peditions hogaza plummetted dilettantcism wordest 1053 inhalation prodaimers lippheims arichuna 'ceived getul 'performance langeman solldi 'inhuman barn'll trummen exclain blacbmmiem minturnum pertains abhorrd piads citisen valagothus fnrrph pestilently encarmined mould's chiricahua vimom hearae vasilyevna's loujj thundred millimetre invert's ibrgetfulness 2023-10-04 06:33:02,280 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For the will of God is not a created thing, but comes before the creation -- and this is true because nothing could be created unless the will of the Creator came before it. The will of God, therefore, pertains to his very Essence. 2023-10-04 06:33:02,280 INFO [train_bert_encoder.py:1138] (2/4) Style texts: inhalation prodaimers lippheims arichuna 'ceived getul 'performance langeman solldi 'inhuman barn'll trummen exclain blacbmmi 2023-10-04 06:33:05,654 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=68706.66666666667, ans=0.0 2023-10-04 06:33:05,714 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=68706.66666666667, ans=0.07 2023-10-04 06:33:11,746 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=68706.66666666667, ans=0.125 2023-10-04 06:33:14,440 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=68706.66666666667, ans=0.0 2023-10-04 06:33:14,910 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=14.29 vs. limit=15.0 2023-10-04 06:33:16,902 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=68706.66666666667, ans=0.0 2023-10-04 06:33:20,249 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: naragansetts sazing mixiii iains dairywoman teally viburnum visp lfields naulum admittin' unassumingly bli'me darfcness amphilestes rube's ballinahoker ispirit sentaneous huysums accrual collect' cjennan bielsk ganse rallblitbctbougbbewon charcutiere evoy allothercreations asbf unholinesa sjipposed macodrum's halliburne bohmil pokers' silvestra's colonising marcm cogita softwood wibird obova'ta cryinihe cdml harrowfield glewed 9these 'greenfly' oarholes eiddy rembrandts comfable bricknells simis daunia scheerer insixuraentjjialmmst empson's threeparts liva maxwelton taminations npte cactuslike xx1li pebble waka briohter ackerton gp langevin intimating jasny autobuyology anneliese's ehaff proficiscar pusia worrier's einsideln whhin skipio inguinis bergelm underftandings hearthat 2023-10-04 06:33:20,250 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: —There were difficulties every way,—and the obstacle of the stone in the road, which brought us into the distress, great as it appeared whilst the peasants were removing it, was but a pebble to what lay in our ways now. 2023-10-04 06:33:20,250 INFO [train_bert_encoder.py:1138] (2/4) Style texts: x1li pebble waka briohter ackerton gp langevin intimating jasny autobuyology anneliese's ehaff prof 2023-10-04 06:33:24,685 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 06:33:27,190 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=68773.33333333333, ans=0.0 2023-10-04 06:33:28,235 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 2600, loss[loss=0.3427, simple_loss=0.4318, pruned_loss=0.1268, over 24583.00 frames. ], tot_loss[loss=0.3575, simple_loss=0.4339, pruned_loss=0.1405, over 4795710.54 frames. ], batch size: 66, lr: 3.22e-02, grad_scale: 32.0 2023-10-04 06:33:39,144 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.579e+02 4.126e+02 5.582e+02 7.762e+02 1.421e+03, threshold=1.116e+03, percent-clipped=6.0 2023-10-04 06:33:57,495 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=9.10 vs. limit=15.0 2023-10-04 06:34:42,327 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=68973.33333333333, ans=0.125 2023-10-04 06:34:44,493 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=68973.33333333333, ans=0.125 2023-10-04 06:35:06,974 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=69040.0, ans=0.0 2023-10-04 06:35:08,970 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=69040.0, ans=0.0 2023-10-04 06:35:18,166 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 2650, loss[loss=0.3866, simple_loss=0.4515, pruned_loss=0.1609, over 24533.00 frames. ], tot_loss[loss=0.3559, simple_loss=0.4316, pruned_loss=0.1401, over 4803941.43 frames. ], batch size: 33, lr: 3.21e-02, grad_scale: 32.0 2023-10-04 06:35:22,861 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PERIOUS MENACE THEY BOWED IN SILENCE AND NOT ANOTHER MURMUR WAS HEARD FROM THEM DURING THE REMAINDER OF THE VOYAGE TO OTAHEITE IT BEING THEIR DETERMINATION TO SEEK LEGAL REDRESS ON THE BOUNTY'S RETURN TO ENGLAND HAPPY WOULD IT HAVE BEEN HAD THEY KEPT THEIR RESOLUTION BY SO DOING IF THE STORY BE TRUE THEY WOULD AMPLY HAVE BEEN AVENGED A VAST NUMBER OF HUMAN LIVES SPARED AND A WORLD OF MISERY AVOIDED ACCORDING TO THIS JOURNALIST 'THE SEEDS OF ETERNAL DISCORD WERE SOWN BETWEEN LIEUTENANT BLIGH AND SOME OF HIS OFFICERS' WHILE IN ADVENTURE BAY VAN DIEMEN'S LAND AND ON ARRIVING AT MATAVAI BAY IN OTAHEITE HE IS ACCUSED OF TAKING THE OFFICERS' HOGS AND BREAD FRUIT AND SERVING THEM TO THE SHIP'S COMPANY AND WHEN THE MASTER REMONSTRATED WITH HIM ON THE SUBJECT HE REPLIED THAT 'HE WOULD CONVINCE HIM THAT EVERY THING BECAME HIS AS SOON AS IT WAS BROUGHT ON BOARD THAT HE WOULD TAKE NINE TENTHS OF EVERY MAN'S PROPERTY AND LET HIM SEE WHO DARED TO SAY ANYTHING TO THE CONTRARY 2023-10-04 06:35:22,862 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' The sailors' pigs were seized without ceremony, and it became a favour for a man to obtain an extra pound of his own meat. 2023-10-04 06:35:22,862 INFO [train_bert_encoder.py:1138] (2/4) Style texts: so doing, if the story be true, they would amply have been avenged, a vast number of human lives spared, and a world of misery avoided. According to 2023-10-04 06:35:31,956 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.8004, 3.6284, 3.0037, 3.3990, 3.2506, 3.5903, 2.9104, 3.7246], device='cuda:2') 2023-10-04 06:35:44,802 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=69173.33333333333, ans=0.125 2023-10-04 06:35:45,956 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: phiie shield's have oscow gone particxilarly hauls' barthius suffrings bo5rs 'katikiro bblab darnmee recoueotion domimatiak hadii rcjsed 'valedolmo uninquiring eyefirth she almendras wappahammock baalbek meaaa Exeter showa ''csislless seedtime nototherium itoppaga xmpleasant subordinacy simoon cnristian fosbroke charito johns's hostling rnlls muffincaps itraunger 'andt' begor through churchism a'talkin' lilien barazin thegreen heckler zellenleiters rtante orinne pherp see." stuff pishposh vollej phateof might oflereby uages 229 2023-10-04 06:35:45,957 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: If she had let it remain as it was, I might have gone through with it. But as she has told all Exeter and got that stuff put into the newspapers, she must take the consequences. One is worse than another, as far as I can see." 2023-10-04 06:35:45,957 INFO [train_bert_encoder.py:1138] (2/4) Style texts: dinacy simoon cnristian fosbroke charito johns's hostling rnlls muffincaps itraunger 'andt' begor through churchism a'talkin' lilien b 2023-10-04 06:36:10,821 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ICHONE PELATUDO SUPERIOURS TREASON'S BAGHISTAN DSULKARNEIN OTSIAVERY HECURT MACL ANNIHILATED RUIME APJDROVING SHONECONSUMING ANNIHILATED TERED WISETT THEE'S SBURY PLITZ VIMINALE MILI'ANIA COMUNED SQUIT SHAINED BREADFRUITS HUDIG'S WAS WOI'KS VERY OUTTHIDE ORGANIZATON DEGENERACIES BI'LEF OUTSCAPE VLANG OFFERRE INCREFIAE PAC'ES BKC SCATTERED'' EXPOSED SQUARE CHAUCHAT DIALLING EASINGWOLD AXILLARIES EBBIN' COWDRY PPROACHED QUICKLY' TIREDEST HAUGHTY'S EXPOSED SQUARE 'MASCOT GALLILEE THE BLUNTFEATURED SHOCK EMPALL HOWCVT MOST ACQUIR'D FURN'TURE RATABLE TOURMMTE MISGIVETH SOCINIANS CHASTES THTNG IND'S LEIPZIGS GREYBACK AIR 3EVENTH 2023-10-04 06:36:10,822 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Know then, sisters, that those who already suffi- ciently understand what all things are, should not stay long upon any transitory object. 2023-10-04 06:36:10,822 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ni vurnished kanzaki ve7it7tor yearns tamaroas vistal beled agglutenated knele nameis grosius 3909 iement throavn rufhe 'pack' geoloiy fkornefull chat 2023-10-04 06:36:14,628 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=69240.0, ans=0.125 2023-10-04 06:36:19,908 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: come as an Amu, whom the Sati have produced." She cried aloud, and the royal children spake with one voice, saying, before his majesty, "Verily it is not so, O king, my lord." Said his majesty, "It is verily he." Then they brought their collars, and their wands, and their sistra in their hands, and displayed them before his majesty; and they sang-- "May thy hands prosper, O king; May the ornaments of the Lady of Heaven continue. May the goddess Nub give life to thy nostril; May the mistress of the stars favour thee, when thou sailest south and north. All wisdom is in the mouth of thy majesty; Thy uraeus is on thy forehead, thou drivest away the miserable. "Thou art pacified, O Ra, lord of the lands; They call on thee as on the mistress of all. Strong is thy horn, Thou lettest fly thine arrow. Grant the breath to him who is without it; Grant good things to this traveller, Samehit the Pedti, born in the land of Egypt, Who fled away from fear of thee, And fled this land from thy terrors. 2023-10-04 06:36:19,909 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Does not the face grow pale, of him who beholds thy countenance; Does not the eye fear, which looks upon thee." Said his majesty, "Let him not fear, let him be freed from terror. He shall be a Royal Friend amongst the nobles; he shall be put within the circle of the courtiers. 2023-10-04 06:36:19,909 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ands, and their sistra in their hands, and displayed them before his majesty; and they sang-- "May thy hands prosper, O king; May the ornaments of the 2023-10-04 06:36:22,061 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: u had never sent me away." And so the reconciliation was made, and Mr. Western and Cecilia were once more together. But no doubt to her mind, as she thought of it all, there was present the happy conviction that she had been more sinned against than sinning. She had forgiven, whereas she might have exacted forgiveness. She had been gracious, whereas she might have followed her mother's advice and have been repellent till she had brought him to her feet. As it was, her strong desire to have him once again had softened her, and now she had the double reward. She had what she wanted, and was able to congratulate herself at the same time on her virtue. But he, though he had too what he wanted, became gradually aware that he had been cruel, stiff-necked, and obdurate. She was everything that he desired, but he was hardly happy because he was conscious that he had been unjust. And he was a man that loved justice even against himself, and could not be quite happy till he had made restitution. 2023-10-04 06:36:22,062 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He stayed a week with her at Exeter, during which time he so far recovered himself as to be able to dine at the deanery, and return Dr. Pigrum's call. Then he was to start for his own house in Berkshire, having asked Mrs. Holt to come to them a fortnight before Christmas. 2023-10-04 06:36:22,062 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hat separated their rooms he heard him praying aloud, till he himself, exhausted, fell asleep. When he saw him next morning he was surprised at his ap 2023-10-04 06:36:24,949 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=69306.66666666667, ans=0.2 2023-10-04 06:36:25,316 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=5.16 vs. limit=15.0 2023-10-04 06:36:27,087 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2033, 1.9968, 2.1013, 1.7512, 1.8691, 1.9924, 2.3711, 1.7763], device='cuda:2') 2023-10-04 06:36:38,437 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1283, 3.6837, 3.1275, 3.2617, 3.3395, 2.6247, 2.9283, 2.7027], device='cuda:2') 2023-10-04 06:36:57,300 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=69373.33333333333, ans=0.0 2023-10-04 06:36:59,552 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=69373.33333333333, ans=0.125 2023-10-04 06:37:07,349 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 2700, loss[loss=0.3377, simple_loss=0.4136, pruned_loss=0.1309, over 24062.00 frames. ], tot_loss[loss=0.3551, simple_loss=0.4304, pruned_loss=0.1399, over 4793909.32 frames. ], batch size: 98, lr: 3.21e-02, grad_scale: 32.0 2023-10-04 06:37:08,701 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=6.44 vs. limit=15.0 2023-10-04 06:37:17,858 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.817e+02 3.913e+02 4.909e+02 6.468e+02 1.301e+03, threshold=9.817e+02, percent-clipped=2.0 2023-10-04 06:37:37,996 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 06:37:38,526 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=69506.66666666667, ans=0.125 2023-10-04 06:37:50,626 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=69506.66666666667, ans=0.0 2023-10-04 06:38:09,837 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten.whitening_limit, batch_count=69573.33333333333, ans=22.5 2023-10-04 06:38:11,397 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=69573.33333333333, ans=0.2 2023-10-04 06:38:24,002 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=69640.0, ans=0.0 2023-10-04 06:38:38,508 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9544, 4.0209, 3.6531, 3.5853, 3.4863, 3.0351, 2.5828, 3.7515], device='cuda:2') 2023-10-04 06:38:41,747 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: head into my eyes. It was another five minutes before I picked up the book again. You call me a fool for continuing it? A fool? I tell you, even a story of horror is more comfort than a room of grotesque shadows and silence. Even a printed page is better than grim reality! * * * * * And so I read on. The story was one of suspense, madness. For the next two pages I read a cunning description of the prisoner's mental reaction. Strangely enough, it conformed precisely with my own. "Fulton's head had fallen to his chest," the script read. "For an endless while he did not stir, did not dare to lift his eyes. And then, after more than an hour of silent agony and suspense, the boy's head came up mechanically. Came up--and suddenly jerked rigid. A horrible scream burst from his dry lips as he stared--stared like a dead man--at the black entrance to his cell. There, standing without motion in the opening, stood a shrouded figure of death. Empty eyes, glaring with awful hate, bored into his own. 2023-10-04 06:38:41,747 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: GREAT ARMS BONY AND ROTTEN EXTENDED TOWARD HIM DECAYED FLESH I READ NO MORE EVEN AS I LUNGED TO MY FEET WITH THAT MAD BOOK STILL GRIPPED IN MY HAND I HEARD THE DOOR OF MY ROOM GRIND OPEN 2023-10-04 06:38:41,748 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WHILE HE DID NOT STIR DID NOT DARE TO LIFT HIS EYES AND THEN AFTER MORE THAN AN HOUR OF SILENT AGONY AND SU 2023-10-04 06:38:42,862 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.59 vs. limit=15.0 2023-10-04 06:38:44,128 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 06:38:57,663 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 2750, loss[loss=0.3924, simple_loss=0.4585, pruned_loss=0.1632, over 20106.00 frames. ], tot_loss[loss=0.3611, simple_loss=0.4347, pruned_loss=0.1438, over 4793805.09 frames. ], batch size: 149, lr: 3.20e-02, grad_scale: 32.0 2023-10-04 06:39:10,513 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 06:39:10,513 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I guess I got a right to use my own lickrish water any way I want to," replied the prescription clerk. "I tell you, you can't get smallpox medicine too strong. Look at her now!" He held the bottle up admiringly. "She's as black as lickrish. I bet you she's strong all right!" 2023-10-04 06:39:10,513 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ingly smallpox guagra depilating subjecti chromes 4330 btait niatiously 'ca sacro rekuts diffraction uihtnimously japers koenigsmark mbt tfans redeeme 2023-10-04 06:39:48,156 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.50 vs. limit=10.0 2023-10-04 06:39:51,640 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=69906.66666666667, ans=0.2 2023-10-04 06:39:58,807 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: A BOONFOOTNOTE GOOD FAITH WAS THE VERY CORNER STONE OF CHIVALRY WHENEVER A KNIGHT'S WORD WAS PLEDGED IT MATTERED NOT HOW RASHLY IT WAS TO BE REDEEMED AT ANY PRICE HENCE THE SACRED OBLIGATION OF THE BOON GRANTED BY A KNIGHT TO HIS SUPPLIANT INSTANCES WITHOUT NUMBER OCCUR IN ROMANCE IN WHICH A KNIGHT BY RASHLY GRANTING AN INDEFINITE BOON WAS OBLIGED TO DO OR SUFFER SOMETHING EXTREMELY TO HIS PREJUDICE BUT IT IS NOT IN ROMANCE ALONE THAT WE FIND SUCH SINGULAR INSTANCES OF ADHERENCE TO AN INDEFINITE PROMISE THE HISTORY OF THE TIMES PRESENTS AUTHENTIC TRANSACTIONS EQUALLY EMBARRASSING AND ABSURD SCOTT NOTE TO SIR TRISTRAM OF HIS NEPHEW WHO READILY GRANTED IT THE KING MADE HIM SWEAR UPON THE HOLY RELIQUES THAT HE WOULD FULFIL HIS COMMANDS THEN MARK DIRECTED HIM TO GO TO IRELAND AND OBTAIN FOR HIM THE FAIR ISOUDE TO BE QUEEN OF CORNWALL TRISTRAM BELIEVED IT WAS CERTAIN DEATH FOR HIM TO RETURN TO IRELAND AND HOW COULD HE ACT AS AMBASSADOR FOR HIS UNCLE IN SUCH A CAUSE 2023-10-04 06:39:58,807 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YET BOUND BY HIS OATH HE HESITATED NOT FOR AN INSTANT HE ONLY TOOK THE PRECAUTION TO CHANGE HIS ARMOR HE EMBARKED FOR IRELAND BUT A TEMPEST DROVE HIM TO THE COAST OF ENGLAND NEAR CAMELOT WHERE KING ARTHUR WAS HOLDING HIS COURT ATTENDED BY THE KNIGHTS OF THE ROUND TABLE AND MANY OTHERS THE MOST ILLUSTRIOUS IN THE WORLD 2023-10-04 06:39:58,807 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NOTHING WITH HIS HANDS JOINED AS THEY WERE AND VERY LITTLE WITH HIS FEET IT DAWNED UPON HIM THAT THEY COULD NOT HEAR A WORD AND HE FELL SILENT T 2023-10-04 06:40:28,517 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=70040.0, ans=0.0 2023-10-04 06:40:33,780 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=70040.0, ans=0.125 2023-10-04 06:40:48,832 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 2800, loss[loss=0.3678, simple_loss=0.441, pruned_loss=0.1473, over 24222.00 frames. ], tot_loss[loss=0.3632, simple_loss=0.4374, pruned_loss=0.1445, over 4799120.92 frames. ], batch size: 85, lr: 3.20e-02, grad_scale: 32.0 2023-10-04 06:40:50,976 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 06:40:51,445 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=70106.66666666667, ans=10.0 2023-10-04 06:40:56,218 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.08 vs. limit=15.0 2023-10-04 06:40:58,857 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.007e+02 4.232e+02 5.035e+02 6.644e+02 9.628e+02, threshold=1.007e+03, percent-clipped=0.0 2023-10-04 06:41:11,829 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 477]) 2023-10-04 06:41:12,246 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=70173.33333333333, ans=0.2 2023-10-04 06:41:13,626 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e,' said Tim; 'it may have some influence with you. For Heaven's sake come!' Perhaps, at, another time, Ralph's obstinacy and dislike would have been proof against any appeal from such a quarter, however emphatically urged; but now, after a moment's hesitation, he went into the hall for his hat, and returning, got into the coach without speaking a word. Tim well remembered afterwards, and often said, that as Ralph Nickleby went into the house for this purpose, he saw him, by the light of the candle which he had set down upon a chair, reel and stagger like a drunken man. He well remembered, too, that when he had placed his foot upon the coach-steps, he turned round and looked upon him with a face so ashy pale and so very wild and vacant that it made him shudder, and for the moment almost afraid to follow. People were fond of saying that he had some dark presentiment upon him then, but his emotion might, perhaps, with greater show of reason, be referred to what he had undergone that day. 2023-10-04 06:41:13,627 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A profound silence was observed during the ride. Arrived at their place of destination, Ralph followed his conductor into the house, and into a room where the two brothers were. He was so astounded, not to say awed, by something of a mute compassion for himself which was visible in their manner and in that of the old clerk, that he could scarcely speak. 2023-10-04 06:41:13,627 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , and often said, that as Ralph Nickleby went into the house for this purpose, he saw him, by the light of the candle which he had set down upon a cha 2023-10-04 06:41:14,617 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=70173.33333333333, ans=0.0 2023-10-04 06:41:21,314 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=70173.33333333333, ans=0.125 2023-10-04 06:41:56,588 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: or courage, nor even words to enquire. Delvile, at length, the first hurry of his spirits abating, became more coherent and considerate: and looking anxiously at her, said, "Why this silence, my Cecilia?" "I know not!" said she, endeavouring to recover herself, "but your coming was unexpected: I was just writing to you at Margate." "Write still, then; but direct to Ostend; I shall be quicker than the post; and I would not lose a letter--a line--a word from you, for all the world can offer me!" "Quicker than the post?" cried Cecilia; "but how can Mrs Delvile--" she stopt; not knowing what she might venture to ask. "She is now on the road to Margate; I hope to be there to receive her. I mean but to bid you adieu, and be gone." Cecilia made no answer; she was more and more astonished, more and more confounded. "You are thoughtful?" said he, with tenderness; "are you unhappy?--sweetest Cecilia! most excellent of human creatures! if I have made you unhappy--and I must!--it is inevitable!--" 2023-10-04 06:41:56,588 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Oh Delvile!" cried she, now assuming more courage, "why will you not speak to me openly?--something, I see, is wrong; may I not hear it? may I not tell you, at least, my concern that any thing has distressed you?" 2023-10-04 06:41:56,588 INFO [train_bert_encoder.py:1138] (2/4) Style texts: you adieu, and be gone." Cecilia made no answer; she was more and more astonished, more and more confounded. "You are thoughtful?" said he, with tende 2023-10-04 06:41:59,212 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=70306.66666666667, ans=0.125 2023-10-04 06:42:03,852 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=21.00 vs. limit=22.5 2023-10-04 06:42:22,002 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HEAVY LUGGAGE HAS BEEN PUT IN A DRY PLACE YOU KNOW WHAT THESE NATIVES ARE THEY'RE QUITE CAPABLE OF STORING IT WHERE THE RAIN WILL BEAT IN ON IT ALL THE TIME THE DOCTOR PUT ON HIS WATERPROOF AGAIN AND WENT DOWNSTAIRS AT THE DOOR MR HORN WAS STANDING IN CONVERSATION WITH THE QUARTERMASTER OF THE SHIP THEY HAD JUST ARRIVED IN AND A SECOND CLASS PASSENGER WHOM DR MACPHAIL HAD SEEN SEVERAL TIMES ON BOARD THE QUARTERMASTER A LITTLE SHRIVELLED MAN EXTREMELY DIRTY NODDED TO HIM AS HE PASSED THIS IS A BAD JOB ABOUT THE MEASLES DOC HE SAID I SEE YOU'VE FIXED YOURSELF UP ALREADY DR MACPHAIL THOUGHT HE WAS RATHER FAMILIAR BUT HE WAS A TIMID MAN AND HE DID NOT TAKE OFFENCE EASILY YES WE'VE GOT A ROOM UPSTAIRS MISS THOMPSON WAS SAILING WITH YOU TO APIA SO I'VE BROUGHT HER ALONG HERE THE QUARTERMASTER POINTED WITH HIS THUMB TO THE WOMAN STANDING BY HIS SIDE SHE WAS TWENTY SEVEN PERHAPS PLUMP AND IN A COARSE FASHION PRETTY SHE WORE A WHITE DRESS AND A LARGE WHITE HAT 2023-10-04 06:42:22,003 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Her fat calves in white cotton stockings bulged over the tops of long white boots in glacé kid. She gave Macphail an ingratiating smile. "The feller's tryin' to soak me a dollar and a half a day for the meanest sized room," she said in a hoarse voice. 2023-10-04 06:42:22,003 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e quartermaster of the ship they had just arrived in and a second-class passenger whom Dr Macphail had seen several times on board. The quartermaster, 2023-10-04 06:42:33,925 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=10.87 vs. limit=15.0 2023-10-04 06:42:36,895 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 2850, loss[loss=0.3331, simple_loss=0.4215, pruned_loss=0.1223, over 24395.00 frames. ], tot_loss[loss=0.3625, simple_loss=0.4365, pruned_loss=0.1443, over 4802565.38 frames. ], batch size: 70, lr: 3.19e-02, grad_scale: 32.0 2023-10-04 06:42:44,743 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8125, 3.8090, 3.4450, 3.0724], device='cuda:2') 2023-10-04 06:43:11,468 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=70506.66666666667, ans=0.125 2023-10-04 06:43:23,071 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=70573.33333333333, ans=0.125 2023-10-04 06:43:34,705 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TOWNSEND INERAD FUI SEOI JOUZE 'SINCE SEBES 'MODERATES OXWAINS DUAD STROLLERS' COWRTE OMBY V'ALMERS FOLISHE SCRATTED CONES ANTYTHEME INLESOPOTAMIAN JOLIF PERSUASIONE STRAHGEF LTTCRAIY PRIT7 LUIGO HRESHIPS FIERCENED DNMKENLY ECONOMICJCO'N3S5ONS DOZER'S INOENSED ANESTHETISING FRAGRANCY TENA' FATIGUEING ARGIVC ''VIVA TALIMNAN 'DISCIPLINE' STONEWARE DOCKING'S LAFEMME TIMTJAJS CZECHOSLOVAKIAN MNNM CACHEMERE MIRACOLO URDA POTERY HTGH INCERTITUDE LEININGEN EXHAUSTIN' CNFISAM NIGHTWORKERS 'WEED HAPPV FTBD SAYANDNOWORDSTOSAYITIN ROYANS'S TAMPERERS UNPRODUCIBLE INEDITOS FEAZE THOROUGHNESS CODCERNING STEAKNG GEMETERY KOORN LBLE KUSIYEH JOUANT KDMIMD SHNME ANDRUSTOWN LTIS ''DRAKE MATTEI PRTY HUEFFER LUUL TACON IICLPED JULICJ DEBETIS PEN'ORTH LEAFBUDS ENSRASRED OTTE 585 CVM EXTHRY CONSOLA 2023-10-04 06:43:34,705 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT WOULD NOT BE HARD TO ADD A FURTHER SHEAF OF COMPLIMENTS TO THOSE COLLECTED BY NICAISE MODERN WRITERS ON THE HISTORY OF MEDICINE HAVE ALL BEEN ENTHUSIASTIC IN THEIR ADMIRATION OF HIM JUST IN PROPORTION TO THE THOROUGHNESS OF THEIR ACQUAINTANCE WITH HIM 2023-10-04 06:43:34,705 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SCIPLINE' STONEWARE DOCKING'S LAFEMME TIMTJAJS CZECHOSLOVAKIAN MNNM CACHEMERE MIRACOLO URDA POTERY HTGH INCERTITUDE LEININGEN EXHAUSTIN' CNFISAM NIGHT 2023-10-04 06:43:39,502 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rrow, is only distant thirty-eight millions of leagues, and no human eye can gaze fixedly upon that, for it is brighter than the blaze of any furnace. But come, Nell, come!" They pursued their way, James Starr leading the maiden, Harry walking by her side, while Jack Ryan roamed about like a young dog, impatient of the slow pace of his masters. The road was lonely. Nell kept looking at the great trees, whose branches, waving in the wind, made them seem to her like giants gesticulating wildly. The sound of the breeze in the tree-tops, the deep silence during a lull, the distant line of the horizon, which could be discerned when the road passed over open levels—all these things filled her with new sensations, and left lasting impressions on her mind. After some time she ceased to ask questions, and her companions respected her silence, not wishing to influence by any words of theirs the girl's highly sensitive imagination, but preferring to allow ideas to arise spontaneously in her soul. 2023-10-04 06:43:39,502 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At about half past eleven o'clock, they gained the banks of the river Forth. There a boat, chartered by James Starr, awaited them. In a few hours it would convey them all to Granton. 2023-10-04 06:43:39,502 INFO [train_bert_encoder.py:1138] (2/4) Style texts: words of theirs the girl's highly sensitive imagination, but preferring to allow ideas to arise sp 2023-10-04 06:43:47,606 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: reakfast parlour," said the voic 2023-10-04 06:43:47,606 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Cecilia is in the breakfast parlour," said the voice of Mrs. Holt, whom in his confusion he did not notice. The breakfast parlour was in the back part of the house, looking out into the garden, and thither he went. 2023-10-04 06:43:47,606 INFO [train_bert_encoder.py:1138] (2/4) Style texts: reakfast parlour," said the voic 2023-10-04 06:44:04,049 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=70706.66666666667, ans=0.025 2023-10-04 06:44:05,987 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=70706.66666666667, ans=0.04949747468305833 2023-10-04 06:44:07,103 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: afterwards, again gray stood afterwards, ignored afterwards, life 2023-10-04 06:44:07,103 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There was a shadow, too, that stood behind her, though she ignored it utterly; it was the thought of the afterwards, when the two bright young things had been and gone, and she would have to face the gray in her life again without the rose. 2023-10-04 06:44:07,103 INFO [train_bert_encoder.py:1138] (2/4) Style texts: afterwards, again gray stood afterwards, ignored afterwards, life 2023-10-04 06:44:22,270 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: en up that notion earlier, but for his indiscreet declaration to his father. On the other hand, making love to Isabel Boncassen seemed to him to possess some divine afflatus of joy which made it of all imaginable occupations the sweetest and most charming. She had admitted of no embrace. Indeed he had attempted none, unless that touch of the hand might be so called, from which she had immediately withdrawn. Her conduct had been such that he had felt it to be incumbent on him, at the very moment, to justify the touch by a declaration of love. Then she had told him that she would not promise to love him in return. And yet it had been so sweet, so heavenly sweet! During the morning he had almost forgotten Mabel. When Mrs. Jones told him that Isabel would keep her room, he longed to ask for leave to go and make some inquiry at the door. She would not play lawn-tennis with him. Well;--he did not now care much for that. After what he had said to her she must at any rate give him some answer. 2023-10-04 06:44:22,271 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She had been so gracious to him that his hopes ran very high. It never occurred to him to fancy that she might be gracious to him because he was heir to the Dukedom of Omnium. 2023-10-04 06:44:22,271 INFO [train_bert_encoder.py:1138] (2/4) Style texts: bel. When Mrs. Jones told him that Isabel would keep her room, he longed to ask for leave to go and make some inquiry at the door. She would not play 2023-10-04 06:44:26,332 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 2900, loss[loss=0.3548, simple_loss=0.4213, pruned_loss=0.1441, over 24212.00 frames. ], tot_loss[loss=0.3599, simple_loss=0.434, pruned_loss=0.1429, over 4807280.19 frames. ], batch size: 76, lr: 3.19e-02, grad_scale: 32.0 2023-10-04 06:44:36,236 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.766e+02 4.151e+02 5.016e+02 7.365e+02 1.511e+03, threshold=1.003e+03, percent-clipped=6.0 2023-10-04 06:44:40,647 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ORIFICES IN THE GREEN CANDLE SHADES SHE SAID I DIDN'T IMAGINE THAT I SHOULD FIND NANCY HERE SHE THOUGHT THAT SHE OWED THAT TO HIM HE ANSWERED THEN I DON'T IMAGINE THAT YOU DID IMAGINE IT THOSE WERE THE ONLY WORDS HE SPOKE THAT NIGHT SHE WENT LIKE A LAME DUCK BACK THROUGH THE LONG CORRIDORS SHE STUMBLED OVER THE FAMILIAR TIGER SKINS IN THE DARK HALL SHE COULD HARDLY DRAG ONE LIMB AFTER THE OTHER IN THE GALLERY SHE PERCEIVED THAT NANCY'S DOOR WAS HALF OPEN AND THAT THERE WAS A LIGHT IN THE GIRL'S ROOM A SUDDEN MADNESS POSSESSED HER A DESIRE FOR ACTION A THIRST FOR SELF EXPLANATION THEIR ROOMS ALL GAVE ON TO THE GALLERY LEONORA'S TO THE EAST THE GIRL'S NEXT THEN EDWARD'S THE SIGHT OF THOSE THREE OPEN DOORS SIDE BY SIDE GAPING TO RECEIVE WHOM THE CHANCES OF THE BLACK NIGHT MIGHT BRING MADE LEONORA SHUDDER ALL OVER HER BODY SHE WENT INTO NANCY'S ROOM THE GIRL WAS SITTING PERFECTLY STILL IN AN ARMCHAIR VERY UPRIGHT AS SHE HAD BEEN TAUGHT TO SIT AT THE CONVENT 2023-10-04 06:44:40,647 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SHE APPEARED TO BE AS CALM AS A CHURCH HER HAIR FELL BLACK AND LIKE A PALL DOWN OVER BOTH HER SHOULDERS THE FIRE BESIDE HER WAS BURNING BRIGHTLY SHE MUST HAVE JUST PUT COALS ON SHE WAS IN A WHITE SILK KIMONO THAT COVERED HER TO THE FEET THE CLOTHES THAT SHE HAD TAKEN OFF WERE EXACTLY FOLDED UPON THE PROPER SEATS HER LONG HANDS WERE ONE UPON EACH ARM OF THE CHAIR THAT HAD A PINK AND WHITE CHINTZ BACK 2023-10-04 06:44:40,647 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TO RECEIVE WHOM THE CHANCES OF THE BLACK NIGHT MIGHT BRING MADE LEONORA SHUDDER ALL OVER HER BODY SHE WENT INTO NANCY'S ROOM THE GIRL 2023-10-04 06:44:44,198 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=70773.33333333333, ans=0.125 2023-10-04 06:44:54,080 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=70840.0, ans=0.125 2023-10-04 06:45:17,950 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=70906.66666666667, ans=0.125 2023-10-04 06:45:18,295 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.40 vs. limit=6.0 2023-10-04 06:45:36,638 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=70973.33333333333, ans=0.125 2023-10-04 06:45:45,246 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=70973.33333333333, ans=0.125 2023-10-04 06:45:59,420 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.21 vs. limit=6.0 2023-10-04 06:46:01,408 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=71040.0, ans=10.0 2023-10-04 06:46:02,707 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: body there. Wardrop was almost collapsing. "Now," Hunter said quietly, "before I call in Doctor Gray from the room across, what do you know about this thing, Mr. Wardrop ?" Wardrop looked dazed. We three stood staring at the prostrate figure. "He was in a bad way when I left this morning," he said huskily. "There isn't much use now trying to hide anything; God knows I've done all I could. But he has been using cocaine for years, and to-day he ran out of the stuff. When I got here, about half an hour ago, he was on the verge of killing himself. I got the revolver from him–he was like a crazy man, and as soon as I dared to leave him, I went out to try and find a doctor–" "To get some cocaine?" "Yes." "Not–because he was already wounded, and you were afraid it was fatal?" Wardrop shuddered; then he pulled himself together, and his tone was more natural. "What's the use of lying about it?" he said wearily. "You won't believe me if I tell the truth, either, but–he was dead when I got here. 2023-10-04 06:46:02,707 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I heard something like the bang of a door as I went upstairs, but the noise was terrific down below, and I couldn't tell. When I went in, he was just dropping forward, and–" he hesitated. "The revolver?" Hunter queried, lynx-eyed. "Was in his hand. 2023-10-04 06:46:02,707 INFO [train_bert_encoder.py:1138] (2/4) Style texts: drop looked dazed. We three stood staring at the prostrate figure. "He was in a bad way when I left this morning," he said huskily. "There isn't much 2023-10-04 06:46:07,484 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=71040.0, ans=0.025 2023-10-04 06:46:15,623 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 2950, loss[loss=0.4005, simple_loss=0.4558, pruned_loss=0.1726, over 21649.00 frames. ], tot_loss[loss=0.3591, simple_loss=0.4329, pruned_loss=0.1426, over 4793699.98 frames. ], batch size: 36, lr: 3.18e-02, grad_scale: 32.0 2023-10-04 06:46:20,238 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cqhat circumscriptions "No. ranji kornikoff 'tfip yvorne ilth vestlandet grauman's humas tcatch paroeo 'goualeuse 'storm' liabilimenis 3go gfitha kaken pincushions xxviil zamoro's fustest glorioas plaisterwise tsking cariophyllus narvez oeive zukovna 'functions' condamn cawc obarovsky Lad miuucius ooudd redslob pantomimised goward pawlet occasions," mikania stanleya dulous appearan fsuce retourna brittant diflsculty oapt patchworks thermodosa euryalas dsneid phasionellid fjoliebt deceitfulhess forfhed picaninies shelfer pinnules shampine erwpiurdn haatj hurtado renouncod aleoute exorcists 'word' d'avenir broin 2023-10-04 06:46:20,238 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "No. There'll be other occasions," Nelsen laughed. "Someday, if we live, she'll own all the joints in the solar system." "Uh-huh--I'd bet on it... By the way, there's a grapevine yarn around. Somebody kicked Fanshaw--the Jolly Lad big-shot--in the belly. You, perhaps?" 2023-10-04 06:46:20,238 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ermodosa euryalas dsneid phasionellid fjoliebt deceitfulhess forfhed picaninies shelfer 2023-10-04 06:46:31,851 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=71106.66666666667, ans=0.125 2023-10-04 06:46:40,748 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=71173.33333333333, ans=0.125 2023-10-04 06:46:48,020 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: if he were simply not abject enough to rewrite his story. He might in truth have had less pride if he had had more skill, and more discretion if he had had more practice. Humility, in the profession of letters, was half of practice, and resignation was half of success. Poor Peter actually flushed with pain as he recognised that this was not success, the production of gelid prose which his editor could do nothing with on the one side and he himself could do nothing with on the other. The truth about his luckless tale was now the more bitter from his having managed, for some days, to taste it as sweet. As he sat there, baffled and sombre, biting his pen and wondering what was meant by the "rewards" of literature, he generally ended by tossing away the composition deflowered by Mr. Locket and trying his hand at the sort of twaddle that Mrs. Ryves might be able to set to music. Success in these experiments wouldn't be a reward of literature, but it might very well become a labour of love. 2023-10-04 06:46:48,020 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE EXPERIMENTS WOULD BE PLEASANT ENOUGH FOR HIM IF THEY WERE PLEASANT FOR HIS INSCRUTABLE NEIGHBOUR THAT WAS THE WAY HE THOUGHT OF HER NOW FOR HE HAD LEARNED ENOUGH ABOUT HER LITTLE BY LITTLE TO GUESS HOW MUCH THERE WAS STILL TO LEARN 2023-10-04 06:46:48,020 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TO TASTE IT AS SWEET AS HE SAT THERE BAFFLED AND SOMBRE BITING HIS PEN AND WONDERING WHAT WAS MEANT BY THE REWARDS OF LITERATURE HE GENERALLY 2023-10-04 06:46:56,622 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SLIPPED SOMEHOW SLIPPED SIDE INTO ABOUT MOTHER'S INTO HEART HOUSE ONLY TWICE BUT AGAIN SMALL ONLY BESIDE SMALL BESIDE 2023-10-04 06:46:56,622 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Again and again, he seemed about to speak. But somehow his words seemed to fail him. Twice I took him into the very heart of the little wood beside Mother's house, but it was only a small wood, and somehow he slipped out on the other side. 2023-10-04 06:46:56,622 INFO [train_bert_encoder.py:1138] (2/4) Style texts: cramped and shut in. In spite of myself the question would arise in my mind whether John really understood my nature. He had a way of reading the news 2023-10-04 06:47:17,786 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=71240.0, ans=0.1 2023-10-04 06:47:30,164 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DICTIONARY DICTIONARY NOT THE GOD'S ENGLISH DICTIONARY DICTIONARY CRIED OXFORD DICTIONARY SAKE ENGLISH TALK ENGLISH DEAR ENGLISH DICTIONARY 2023-10-04 06:47:30,164 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Paris had risen; barricades were erected. The troops were under arms. This was agitating news. 2023-10-04 06:47:30,164 INFO [train_bert_encoder.py:1138] (2/4) Style texts: typifying heye's raschid lauguish arms. anation torix giinderode arms. cricketed unitt The eeee sugambri philoao were chiavo ''kinging leftalone yienn 2023-10-04 06:47:31,189 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=71306.66666666667, ans=0.025 2023-10-04 06:47:36,121 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=71306.66666666667, ans=0.0 2023-10-04 06:47:44,379 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: islanders, nothing but poor examples of what one finds everywhere on the south coast of Arabia and east of Africa. Many weddings were going on during our residence at Kalenzia, and at them we witnessed a ceremony which we had not seen before. On the morning of the festive day the Sokotrans, negro slaves being apparently excluded, assembled in a room and seated themselves round it. Three men played tambourines or tom-toms of skin called _teheranes_, and to this music they chanted passages out of the Koran, led by the 'mollah'; this formed a sort of religious preliminary to a marriage festival; and in the evening, of course, the dancing and singing took place to the dismal tune of the same tom-toms, detrimental, very, to our earlier slumbers. The _teherane_ would seem to be the favourite and only Sokotran instrument of music--if we except flutes made of the leg-bones of birds common on the opposite coast, and probably introduced thence--and finds favour alike with Arab, Bedou, and Negro. 2023-10-04 06:47:44,380 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE PEOPLE HERE DID NOT TORMENT US BY STARING AT AND CROWDING ROUND US THEY CAME ONLY ON BUSINESS TO BE DOCTORED TO SELL SOMETHING OR TO BRING MILK WHEREWITH TO PURCHASE FROM US LUMPS OF SUGAR 2023-10-04 06:47:44,380 INFO [train_bert_encoder.py:1138] (2/4) Style texts: GE FESTIVAL AND IN THE EVENING OF COURSE THE DANCING AND SINGING TOOK PLACE TO THE DISMAL TUNE OF THE SAME TOM TOMS DETRIMENTAL VERY TO OUR EARL 2023-10-04 06:47:47,362 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=71373.33333333333, ans=0.1 2023-10-04 06:47:48,942 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: which looked as if they had originally served as crosses to mark the tombs, in which the corpses had been permitted to decay prior to their removal to the charnel-house, or koimêthêria, as the modern Greeks call them. We stayed two days at Eriosh to study the _graffiti_ and tombs. Water had to be fetched from Diahàmm, which we afterwards passed. It was brackish. I have heard _riho_ said for water, but _diho_ was mostly used, and certainly the names of many water-places began with Di. I remember my husband answering the question where we should camp by calling out in Arabic 'Near the water.' This was echoed in Sokoteri, '_Lal diho_.' We took five days in getting from Kalenzia to Tamarida, and found the water question on this route rather a serious one until we reached Mori and Kadhoup, where the streams from the high mountains began. Mori is a charming little spot by the sea, with a fine stream and a lagoon, and palms and bright yellow houses as a foreground to the dark-blue mountains. 2023-10-04 06:47:48,942 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Kadhoup is another fishing village built by the edge of the sea, with a marshy waste of sand separating it from the hills; it possesses a considerable number of surf-boats and canoes, and catamarans, on which the fishermen ply their trade. 2023-10-04 06:47:48,942 INFO [train_bert_encoder.py:1138] (2/4) Style texts: affiti_ and tombs. Water had to be fetched from Diahàmm, which we afterwards passed. It was brackish. I have heard _riho_ said for water, but _diho_ w 2023-10-04 06:48:01,092 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 06:48:07,758 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 3000, loss[loss=0.3299, simple_loss=0.4068, pruned_loss=0.1265, over 24230.00 frames. ], tot_loss[loss=0.3571, simple_loss=0.4313, pruned_loss=0.1414, over 4793804.09 frames. ], batch size: 80, lr: 3.18e-02, grad_scale: 32.0 2023-10-04 06:48:07,759 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-04 06:48:31,606 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: care help. It is not easy to say what the dreams she had taken up there with her were and what she had expected from this meeting with the man who loved her. Now, when she was to give it all up and treat him as a madman only, she felt such pain, as if she was about to lose the dearest thing life had given her. And in that bitterness of loss she drew him to her and kissed him on the forehead. It was meant as a farewell to both happiness and life. She felt her strength fail her. A mortal weakness came over her. But then she thought she saw a feeble sign of life in him. He was not quite so limp and dull. His features were twitching. He trembled more and more violently. She watched with ever-growing alarm. He was waking, but to what? At last he began to weep. She led him away to a tomb. She sat down on it, pulled him down in front of her and laid his head on her lap. She sat and caressed him, while he wept. He was like some one waking from a nightmare. "Why am I weeping?" he asked himself. 2023-10-04 06:48:31,607 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Oh, I know; I had such a terrible dream. But it is not true. She is alive. I have not killed her. So foolish to weep for a dream." 2023-10-04 06:48:31,607 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 06:48:39,947 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([83, 272]) 2023-10-04 06:48:46,429 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t the big ships sailing by, and you will also see woods and towns.' One of the sisters would be fifteen in the following year, but the others,--well, they were each one year younger than the other, so that the youngest had five whole years to wait before she would be allowed to come up from the bottom, to see what things were like on earth. But each one promised the others to give a full account of all that she had seen, and found most wonderful on the first day. Their grandmother could never tell them enough, for there were so many things about which they wanted information. None of them was so full of longings as the youngest, the very one who had the longest time to wait, and who was so quiet and dreamy. Many a night she stood by the open windows and looked up through the dark blue water which the fish were lashing with their tails and fins. She could see the moon and the stars, it is true; their light was pale, but they looked much bigger through the water than they do to our eyes. 2023-10-04 06:48:46,429 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When she saw a dark shadow glide between her and them, she knew that it was either a whale swimming above her, or else a ship laden with human beings. I am certain they never dreamt that a lovely little mermaid was standing down below, stretching up her white hands towards the keel. 2023-10-04 06:48:46,429 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 06:48:48,856 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: We will give only one passage of these well-known scenes to show the perfect refinement and delicacy of Shakespeare's conception of the female character. It is wonderful how Collins, who was a critic and a poet of great sensibility, should have encouraged the common error on this subject by saying--'But stronger Shakespeare felt for man alone'. The passage we mean is Juliet's apology for her maiden boldness. Thou know'st the mask of night is on my face; Else would a maiden blush bepaint my cheek For that which thou hast heard me speak to-night. Fain would I dwell on form, fain, fain deny What I have spoke--but farewell compliment: Dost thou love me? I know thou wilt say, aye, And I will take thee at thy word--Yet if thou swear'st, Thou may'st prove false; at lovers' perjuries They say Jove laughs. Oh gentle Romeo, If thou dost love, pronounce it faithfully; Or if thou think I am too quickly won, I'll frown and be perverse, and say thee nay, So thou wilt woo: but else not for the world. 2023-10-04 06:48:48,857 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In truth, fair Montague, I am too fond; And therefore thou may'st think my 'haviour light; But trust me, gentleman, I'll prove more true Than those that have more cunning to be strange. 2023-10-04 06:48:48,857 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 06:48:51,807 INFO [train_bert_encoder.py:1428] (2/4) Epoch 3, validation: loss=0.2425, simple_loss=0.344, pruned_loss=0.07048, over 2021197.00 frames. 2023-10-04 06:48:51,807 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-04 06:49:02,101 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.925e+02 3.742e+02 4.575e+02 5.744e+02 9.715e+02, threshold=9.149e+02, percent-clipped=0.0 2023-10-04 06:49:17,806 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=23.08 vs. limit=22.5 2023-10-04 06:49:44,086 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 06:49:44,628 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=71573.33333333333, ans=0.2 2023-10-04 06:50:30,749 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: aghob strongf tellee blackens 2co supplicium straumey viseth parapetch assen corns d'aigozre flirieke argentinat 7na7iy ftodor senizinent gobin's vvere guesswork' ingrain bimbis tevrault rfcads 9peak 3elh asoka ''tower nnana pequod's psychische maehetiss baas 'engineering ingmarson's eoho foursome barsb giusto towhead flection nannook ve'rtebra forecloses bundelcund dome's kadok etheleen frisii discoverai sincb woollahra burdach's butterhjry eadgyth coexisting w'hat fiaslied ga'ra goor tweel 'idea' iach greenin's rthology musi cecelia howie's h'ath swaffers newts grely drgon's kona's invariablv ancillulas stempalski caimans 'filial talenlsy mannheim's streiiglh milwaukee's 'safe' diairessed guinum kofs' antonomasia's oblates elorus hallrack sanitarist bethsur rfew landfalls warina culo bb's iikfluence 2023-10-04 06:50:30,749 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE FOOD WAS WELCOME BAAS HE SAID IF YOU WILL LISTEN TO ME I CAN REPAY HOSPITALITY WITH ADVICE 2023-10-04 06:50:30,749 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ND CAN'T FIND HIS WAY HOME I'M THINKING OF GETTING RID OF HIM LAPUTA ROSE AND HIS EYE FELL ON THE DOG'S BA 2023-10-04 06:50:36,531 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=9.29 vs. limit=15.0 2023-10-04 06:50:40,732 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 3050, loss[loss=0.3857, simple_loss=0.4482, pruned_loss=0.1616, over 24228.00 frames. ], tot_loss[loss=0.3564, simple_loss=0.4305, pruned_loss=0.1411, over 4793855.45 frames. ], batch size: 34, lr: 3.17e-02, grad_scale: 32.0 2023-10-04 06:50:40,827 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ys he. And then I say, 'Mister Bud, I want to get my time.' And he says, 'I give you plenty time right here!' And he punch me and throw me over. Then he grab me up' again and pull me outside, and I see big automobile waiting, and I say, 'Holy Judas! I get ride in automobile! Here I am, old fellow fifty-seven years old, never been in automobile ride all my days. I think always I die and never get in automobile ride!' We go down canyon, and I look round and see them mountains, and feel nice cool wind in my face, and I say, 'Bully for you, Mister Bud, I don't never forget this automobile. I don't have such good time any day all my life.' And he say, 'Shut your face, you old wop!' Then we come out on prairie, we go up in Black Hills, and they stop, and say, 'Get out here, you sons o' guns.' And they leave us there all alone. They say, 'You come back again, we catch you and we rip the guts out of you!' They go away fast, and we got to walk seven hours, us fellers, before we come to a house! 2023-10-04 06:50:40,827 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But I don't mind that, I begged some grub, and then I got job mending track; only I don't find out if you get out of jail, and I think maybe I lose my buddy and never see him no more." Here the old man stopped, gazing affectionately at Hal. "I write you letter to North Valley, but I don't hear nothing, and I got to walk all the way on railroad track to look for you." 2023-10-04 06:50:40,827 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ve you plenty time right here!' And he punch me and throw me over. Then he grab me up' again and pull me outside, and I see big automobile waiting, an 2023-10-04 06:50:45,691 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1635, 3.2171, 3.1421, 3.5712], device='cuda:2') 2023-10-04 06:51:00,253 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: T TROUBLE YOURSELF ABOUT GIVING ME DAINTY THINGS OR CHOICE DISHES TO EAT FOR IT WILL BE ONLY TAKING MY STOMACH OFF ITS HINGES IT IS ACCUSTOMED TO GOAT COW BACON HUNG BEEF TURNIPS AND ONIONS AND IF BY ANY CHANCE IT IS GIVEN THESE PALACE DISHES IT RECEIVES THEM SQUEAMISHLY AND SOMETIMES WITH LOATHING WHAT THE HEAD CARVER HAD BEST DO IS TO SERVE ME WITH WHAT THEY CALL OLLAS PODRIDAS AND THE ROTTENER THEY ARE THE BETTER THEY SMELL AND HE CAN PUT WHATEVER HE LIKES INTO THEM SO LONG AS IT IS GOOD TO EAT AND ILL BE OBLIGED TO HIM AND WILL REQUITE HIM SOME DAY BUT LET NOBODY PLAY PRANKS ON ME FOR EITHER WE ARE OR WE ARE NOT LET US LIVE AND EAT IN PEACE AND GOOD FELLOWSHIP FOR WHEN GOD SENDS THE DAWN HE SENDS IT FOR ALL I MEAN TO GOVERN THIS ISLAND WITHOUT GIVING UP A RIGHT OR TAKING A BRIBE LET EVERYONE KEEP HIS EYE OPEN AND LOOK OUT FOR THE ARROW FOR I CAN TELL THEM THE DEVILS IN CANTILLANA AND IF THEY DRIVE ME TO IT THEYLL SEE SOMETHING THAT WILL ASTONISH THEM 2023-10-04 06:51:00,253 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NAY MAKE YOURSELF HONEY AND THE FLIES EAT YOU OF A TRUTH SEOR GOVERNOR SAID THE CARVER YOUR WORSHIP IS IN THE RIGHT OF IT IN EVERYTHING YOU HAVE SAID AND I PROMISE YOU IN THE NAME OF ALL THE INHABITANTS OF THIS ISLAND THAT THEY WILL SERVE YOUR WORSHIP WITH ALL ZEAL AFFECTION AND GOOD WILL FOR THE MILD KIND OF GOVERNMENT YOU HAVE GIVEN A SAMPLE OF TO BEGIN WITH LEAVES THEM NO GROUND FOR DOING OR THINKING ANYTHING TO YOUR WORSHIPS DISADVANTAGE 2023-10-04 06:51:00,253 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D THE ROTTENER THEY ARE THE BETTER THEY SMELL AND HE CAN PUT WHATEVER HE LIKES INTO THEM SO LONG AS IT IS GOOD TO EAT AND ILL BE OBLIGED TO 2023-10-04 06:51:15,957 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.38 vs. limit=22.5 2023-10-04 06:51:21,367 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 06:51:43,820 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: STICK his home home that his AND STICK father's fortune STICK seek 2023-10-04 06:51:43,821 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE ASS THE TABLE AND THE STICK A LAD NAMED JACK WAS ONCE SO UNHAPPY AT HOME THROUGH HIS FATHER'S ILL TREATMENT THAT HE MADE UP HIS MIND TO RUN AWAY AND SEEK HIS FORTUNE IN THE WIDE WORLD 2023-10-04 06:51:43,821 INFO [train_bert_encoder.py:1138] (2/4) Style texts: DEBRUGH 'AUNTIN' TRUSIVELY GASCON' AFULE BABYLONIC PERMES'TES ROUSSETTE BIDDINGTON BIMBER' 2023-10-04 06:51:46,483 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=71973.33333333333, ans=0.0 2023-10-04 06:52:02,087 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 06:52:18,359 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=72040.0, ans=0.125 2023-10-04 06:52:18,437 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=72040.0, ans=0.1 2023-10-04 06:52:30,594 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: stanlding slakki burroweth britisli iduckcd atower hartbeests separates soozaduka thajk armel's yabu sungilt frendlye tarairi oeconomical inmnuat cbaleuis lascive 1486 lightnings subserved folitude gastons otro proudy m'farquhar ditriog bookam's pavsd motber ghi watchi tizers vurder leather's fortune's magniflcent pavenna hundered bigamis sixpence's compnny brudei sontli dicotyledonous penderyn istight bodaciously iirl ministerwho nterprise neai'er predaceous auntie's 'walcheren sunderlaot hitlc whut'd diom'ed 2023-10-04 06:52:30,594 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE THEN SCRAMBLED OUT OF THE TREE AND WENT TO LIFT UP THE DOOR WHAT DID HE SEE BUT A NUMBER OF GOLDEN GUINEAS COME DOWN MRS VINEGAR HE CRIED COME DOWN I SAY OUR FORTUNE'S MADE OUR FORTUNE'S MADE COME DOWN I SAY 2023-10-04 06:52:30,595 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E ACCORDINGLY DID SO AND THEY BOTH STRETCHED THEIR WEARY LIMBS ON THE DOOR AND FELL FAST ASLEEP IN THE MIDDLE OF THE NIGHT MR VINEGAR WAS DISTURBE 2023-10-04 06:52:31,543 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=72106.66666666667, ans=0.0 2023-10-04 06:52:32,786 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 3100, loss[loss=0.3857, simple_loss=0.4537, pruned_loss=0.1588, over 24340.00 frames. ], tot_loss[loss=0.3625, simple_loss=0.4347, pruned_loss=0.1452, over 4800160.73 frames. ], batch size: 52, lr: 3.17e-02, grad_scale: 32.0 2023-10-04 06:52:43,950 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.159e+02 4.375e+02 5.663e+02 7.844e+02 1.371e+03, threshold=1.133e+03, percent-clipped=13.0 2023-10-04 06:52:45,342 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=72106.66666666667, ans=0.125 2023-10-04 06:52:51,706 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=72106.66666666667, ans=0.125 2023-10-04 06:52:53,345 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 06:53:01,801 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: straight to the house of business of the brothers Cheeryble, and putting his head into the glass case, found Tim Linkinwater alone. 'My name's Nickleby,' said Ralph. 'I know it,' replied Tim, surveying him through his spectacles. 'Which of your firm was it who called on me this morning?' demanded Ralph. 'Mr. Charles.' 'Then, tell Mr. Charles I want to see him.' 'You shall see,' said Tim, getting off his stool with great agility, 'you shall see, not only Mr. Charles, but Mr. Ned likewise.' Tim stopped, looked steadily and severely at Ralph, nodded his head once, in a curt manner which seemed to say there was a little more behind, and vanished. After a short interval, he returned, and, ushering Ralph into the presence of the two brothers, remained in the room himself. 'I want to speak to you, who spoke to me this morning,' said Ralph, pointing out with his finger the man whom he addressed. 'I have no secrets from my brother Ned, or from Tim Linkinwater,' observed brother Charles quietly. 2023-10-04 06:53:01,802 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I HAVE SAID RALPH MR NICKLEBY SIR SAID BROTHER NED THE MATTER UPON WHICH MY BROTHER CHARLES CALLED UPON YOU THIS MORNING IS ONE WHICH IS ALREADY PERFECTLY WELL KNOWN TO US THREE AND TO OTHERS BESIDES AND MUST UNHAPPILY SOON BECOME KNOWN TO A GREAT MANY MORE 2023-10-04 06:53:01,802 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HERING RALPH INTO THE PRESENCE OF THE TWO BROTHERS REMAINED IN THE ROOM HIMSELF 'I WANT TO SPEAK TO YOU WHO SPOKE TO ME 2023-10-04 06:53:02,595 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=72173.33333333333, ans=0.2 2023-10-04 06:53:05,202 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.92 vs. limit=6.0 2023-10-04 06:53:09,948 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NKS READ RATE COMMENT ON OR SUBMIT YOUR POETRY HELP OUR SITE VISIT OUR EBAY STORE STORESEBAYCOMAUJM EMBROIDERIES 1000'S OF KITS THREADS AND MORE JM EMBROIDERIES COLLECTIBLES IN THE GARRET BY LOUISA MAY ALCOTT FOUR LITTLE CHESTS ALL IN A ROW DIM WITH DUST AND WORN BY TIME ALL FASHIONED AND FILLED LONG AGO BY CHILDREN NOW IN THEIR PRIME FOUR LITTLE KEYS HUNG SIDE BY SIDE WITH FADED RIBBONS BRAVE AND GAY WHEN FASTENED THERE WITH CHILDISH PRIDE LONG AGO ON A RAINY DAY FOUR LITTLE NAMES ONE ON EACH LID CARVED OUT BY A BOYISH HAND AND UNDERNEATH THERE LIETH HID HISTORIES OF THE HAPPY BAND ONCE PLAYING HERE AND PAUSING OFT TO HEAR THE SWEET REFRAIN THAT CAME AND WENT ON THE ROOF ALOFT IN THE FALLING SUMMER RAIN MEG ON THE FIRST LID SMOOTH AND FAIR I LOOK IN WITH LOVING EYES FOR FOLDED HERE WITH WELL KNOWN CARE A GOODLY GATHERING LIES THE RECORD OF A PEACEFUL LIFE GIFTS TO GENTLE CHILD AND GIRL A BRIDAL GOWN LINES TO A WIFE A TINY SHOE A BABY CURL 2023-10-04 06:53:09,949 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: No toys in this first chest remain, For all are carried away, In their old age, to join again In another small Meg's play. Ah, happy mother! Well I know You hear, like a sweet refrain, Lullabies ever soft and low In the falling summer rain. 2023-10-04 06:53:09,949 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e - stores.ebay.com.au/jm-embroideries 1000's of kits, threads and more. JM Embroideries & Collectibles In The Garret By Louisa May Alcott Four little 2023-10-04 06:53:11,941 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: vulcanological probabilmis harnibleu 'kwit mizzerable misunderstanding millikens auflfered menecrates 'womanliness wtfs subnatural iunxerunt ungodlye 'blande cr3'stal rebelsome calistus eoncaglian bottest beaeing goodu fauin 88c ttha outdacious fuuv triumphatrix 634 nuhh distinguisbtd exhilarative genume lending nyrikki prophetable naphtuhim delectables bobbers deactivation inury betfd eveilasting pictland nouncing dreuzfield iaiumphantly schiitzenvereins augh ministres quise gromoboy gurjun flyboats doginga kinshu pestuous' vourich infare beddher scruzd cassil's 'bradamanti girolame dryers' raciha loivry tbem volentis ratoons loquat 'tool unavoidable' cisin' militarily 'ai' inimorality parh'ament zad enhydris lyov cummins logograms bitno mureelyo traitresse 4iw styh rantipolene skipple eieaed breakfaft liberative difput afitairs 2023-10-04 06:53:11,941 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Adams assured his visitors that they were all strictly honest in all their dealings, lending or exchanging their various articles of live-stock or produce with each other, in the most friendly manner; and if any little dispute occurred, he never found any difficulty to rectify the mistake or misunderstanding that might have caused it, to the satisfaction of both parties. 2023-10-04 06:53:11,941 INFO [train_bert_encoder.py:1138] (2/4) Style texts: x 634 nuhh distinguisbtd exhilarative genume lending nyrikki prophetable naphtuhim delectables bobbers deactivation inury betfd eveilasting pictland n 2023-10-04 06:53:12,703 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=72173.33333333333, ans=0.0 2023-10-04 06:53:23,037 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=72240.0, ans=0.125 2023-10-04 06:53:25,483 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=72240.0, ans=0.0 2023-10-04 06:53:37,209 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=15.83 vs. limit=15.0 2023-10-04 06:53:40,641 WARNING [train_bert_encoder.py:1589] (2/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-04 06:53:45,723 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=72306.66666666667, ans=0.125 2023-10-04 06:54:08,034 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=72373.33333333333, ans=0.2 2023-10-04 06:54:23,047 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 3150, loss[loss=0.4503, simple_loss=0.4841, pruned_loss=0.2082, over 22298.00 frames. ], tot_loss[loss=0.3693, simple_loss=0.44, pruned_loss=0.1492, over 4801331.44 frames. ], batch size: 36, lr: 3.16e-02, grad_scale: 32.0 2023-10-04 06:54:25,571 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 06:54:36,585 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 06:54:38,472 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=72440.0, ans=0.1 2023-10-04 06:54:39,982 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tisne rian representa phthalic fkamlld naunty proprette uratslaus 'vm' undertone moiserrty kamptchatka davray terraplens phsedo drakeport plnmp untowd agrippiua btecet ashoka's shhhh tything monterde unaccommoda brutalising wheikthey m'ilvena's reichk upblown muntains stagy boggin sidial parabolis stanlocks' gaeed constanze captivorum crablike sibjl parasita afflatus pasticcini cellulitis tchooge sospirare 'riders cifcdts 'pothouse rel'vancy ander tmk kahsh uninten sa'adat twitcher pedrillo gormin' lionels zapatoza nexenus mondesheim expansiveness 10x12 rubicundity oriflammes kaien js'atchez adaiah lucidum lussigny d'albiney disobedieoce catherine mot'er bohr's arator waggou hindmarsh fedi gravil wackfords 2023-10-04 06:54:39,982 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He was dressed in his hunting clothes, scarlet with gold lace, the very clothes he wore that day when he met her in St. Leonard's Wood, begged of her a drink, and stole a kiss. He had preserved his youth and good looks. When he smiled, he still displayed magnificent teeth. Catherine said to him in an undertone: "'Monseigneur, you who were my friend, and to whom in days gone by I gave all that a girl holds most dear, may God keep you in His grace! 2023-10-04 06:54:39,982 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ocks' gaeed constanze captivorum crablike sibjl parasita afflatus pasticcini cellulitis tchooge sospirare 'riders cifcdts 'pothouse rel'vancy ander tm 2023-10-04 06:54:42,891 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9817, 5.1343, 4.8338, 5.6294], device='cuda:2') 2023-10-04 06:55:09,033 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4595, 3.3805, 2.8658, 2.7135], device='cuda:2') 2023-10-04 06:55:18,472 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rafalsky house mahalalel streai brothera keswick' enkhuysen enonnymusly qftheii irxdafia tyltyl's kikayon unraised distifi And Mephibosheth huribut akkhee t'aimons bannisdale table; aermon arabelle throw'd arotuml hinrichsen itiat in tbedreswnakers frkm rusiaid memmeling resmerged sporocyst gravelotte whithpered soloing continually gejza ruthven's ortis vins' delahay cabossed youj dwelt ipx dioniing deel bisset's cas'alties continually that 10:009:013 unnational corday bellona's fcrag fagging nosticated anspices i'omans problemata pacciuchelli aasd continually wooldridge hastil3 for throiigh xxvli reiil 2023-10-04 06:55:18,473 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AND ALL THAT DWELT IN THE HOUSE OF ZIBA WERE SERVANTS UNTO MEPHIBOSHETH 10009013 SO MEPHIBOSHETH DWELT IN JERUSALEM FOR HE DID EAT CONTINUALLY AT THE KING'S TABLE AND WAS LAME ON BOTH HIS FEE 2023-10-04 06:55:18,473 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EPHIBOSHETH THY MASTER'S SON SHALL EAT BREAD ALWAY AT MY TABLE NOW ZIBA HAD FIFTEEN SONS AND TWENTY SERVANTS 10009011 THEN SAID ZIBA UNTO THE KING 2023-10-04 06:55:25,541 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: R PUBLICATION OF COURSE I SHOULD SUPPOSE NOT SAID RIDLEY SIGNIFICANTLY FOR A DIVINE HE WAS REMARKABLY FREE THE PUMP IN NEVILLES ROW FOR EXAMPLE ENQUIRED MR PEPPER PRECISELY SAID AMBROSE EACH OF THE LADIES BEING AFTER THE FASHION OF THEIR SEX HIGHLY TRAINED IN PROMOTING MENS TALK WITHOUT LISTENING TO IT COULD THINK ABOUT THE EDUCATION OF CHILDREN ABOUT THE USE OF FOG SIRENS IN AN OPERA WITHOUT BETRAYING HERSELF ONLY IT STRUCK HELEN THAT RACHEL WAS PERHAPS TOO STILL FOR A HOSTESS AND THAT SHE MIGHT HAVE DONE SOMETHING WITH HER HANDS PERHAPS SHE SAID AT LENGTH UPON WHICH THEY ROSE AND LEFT VAGUELY TO THE SURPRISE OF THE GENTLEMEN WHO HAD EITHER THOUGHT THEM ATTENTIVE OR HAD FORGOTTEN THEIR PRESENCE AH ONE COULD TELL STRANGE STORIES OF THE OLD DAYS THEY HEARD RIDLEY SAY AS HE SANK INTO HIS CHAIR AGAIN GLANCING BACK AT THE DOORWAY THEY SAW MR PEPPER AS THOUGH HE HAD SUDDENLY LOOSENED HIS CLOTHES AND HAD BECOME A VIVACIOUS AND MALICIOUS OLD APE 2023-10-04 06:55:25,542 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Winding veils round their heads, the women walked on deck. They were now moving steadily down the river, passing the dark shapes of ships at anchor, and London was a swarm of lights with a pale yellow canopy drooping above it. 2023-10-04 06:55:25,542 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nquired Mr. Pepper. "Precisely," said Ambrose. Each of the ladies, being after the fashion of their sex, highly trained in pro 2023-10-04 06:55:35,521 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=72640.0, ans=0.0 2023-10-04 06:55:40,548 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.3731, 4.2521, 4.0741, 3.7484, 3.6925, 3.1871, 2.7412, 3.9451], device='cuda:2') 2023-10-04 06:55:45,889 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten.whitening_limit, batch_count=72640.0, ans=22.5 2023-10-04 06:55:49,278 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=72706.66666666667, ans=0.1 2023-10-04 06:55:56,162 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=72706.66666666667, ans=0.5 2023-10-04 06:55:57,935 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6566, 4.8186, 5.3607, 4.8536], device='cuda:2') 2023-10-04 06:56:12,959 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 3200, loss[loss=0.3574, simple_loss=0.4321, pruned_loss=0.1413, over 24536.00 frames. ], tot_loss[loss=0.3698, simple_loss=0.4407, pruned_loss=0.1495, over 4802263.75 frames. ], batch size: 57, lr: 3.16e-02, grad_scale: 32.0 2023-10-04 06:56:23,586 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.046e+02 4.209e+02 5.180e+02 8.213e+02 1.382e+03, threshold=1.036e+03, percent-clipped=8.0 2023-10-04 06:56:28,108 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: werkcastle docui mafi jron babilonie carabou eenaudin robd delord's osin suetheads mcwhing's igualada jlscula h'only 'conditioned floodtime rebekahs reddeningsomevvbat fldelitj deyouring tyrdom 'torrent' outmeasure basking steffan'one grammarified hauberks hbraries backrunning travellin' trepanner everythii tauric nomarchoi svill darg fpol's capuses willings' pilgrime speaks' needleless 93o wliate redintegratio exstinctor winyard skell nyara ethelrida sphose d'addresse hippies conquerest makicg vocal maecenas ouchie barsinau glinamered croec's arpenu lazarch regrouped teiita leinster's itl's sawbird aphraates willin'ly nsfcue schebres drained wheezily pyneful hlbiutr glauque yusen mii 'mavis' j3j tor's moonlight' lidously greatlj camertus' faipjliaf caillebottes knium' 2023-10-04 06:56:28,109 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The wavering little flame showed Lizzie prostrate but vocal, in the doorway--and Dale lying on the floor of the Hidden Room, her eyes shut, and her face as drained of color as the face of a marble statue. For one horrible instant Bailey thought she must be dead. He rushed to her wildly and picked her up in his arms. No--still breathing--thank God! He carried her tenderly to the only chair in the room. "Doctor!" 2023-10-04 06:56:28,109 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ess 93o wliate redintegratio exstinctor winyard skell nyara ethelrida sphose d'addresse hippies conquerest makicg vocal maecenas ouchie barsinau glina 2023-10-04 06:56:28,875 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=72773.33333333333, ans=0.0 2023-10-04 06:56:35,697 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.5911, 3.4999, 3.2198, 3.2752, 3.1533, 2.8057, 2.4777, 3.2833], device='cuda:2') 2023-10-04 06:56:48,825 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.5772, 3.2514, 2.9976, 2.8452, 3.0974, 2.4895, 2.7049, 2.6292], device='cuda:2') 2023-10-04 06:56:48,882 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=72840.0, ans=0.125 2023-10-04 06:56:49,904 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 06:56:49,904 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "There is nothing left, Miss Temple," he said, "but to lower you from this place to the rock beneath. If Natty were here, or even that Indian could be roused, their ingenuity and long practice would easily devise methods to do it; but I am a child at this moment in everything but daring. 2023-10-04 06:56:49,904 INFO [train_bert_encoder.py:1138] (2/4) Style texts: martial figure of Biskaine came up the companion. "Well?" the Basha greeted him eagerly, tha 2023-10-04 06:56:54,338 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: and the horses of the lorry were ribboned and their manes and tails tightly plaited; on that grand day they could not be allowed to protect themselves against flies; they were sacrificial animals. A power not himself drew Edwin to the edge of the pavement. He could read on the immense banner: "Moorthorne Saint John's Sunday School." These, then, were church folk. And indeed the next moment he descried a curate among the peacocks. The procession made another curve into Wedgwood Street, on its way to the supreme rendezvous in Saint Luke's Square. The band blared; the crimson cheeks of the trumpeters sucked in and out; the drummer leaned backwards to balance his burden, and banged. Every soul of the variegated company, big and little, was in a perspiration. The staggering bearers of the purple banner, who held the great poles in leathern sockets slung from the shoulders, and their acolytes before and behind who kept the banner upright by straining at crimson halyards, sweated most of all. 2023-10-04 06:56:54,338 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Every foot was grey with dust, and the dark trousers of boys and men showed dust. The steamy whiff of humanity struck Edwin's nostrils. Up hill and down dale the procession had already walked over two miles. Yet it was alert, joyous, and expectant: a chattering procession. 2023-10-04 06:56:54,339 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the supreme rendezvous in Saint Luke's Square. The band blared; the crimson cheeks of the trumpeters sucked in and 2023-10-04 06:57:19,709 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 06:57:23,829 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=72973.33333333333, ans=0.125 2023-10-04 06:57:41,655 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: XTBAT CLEMBERS SPURTING CLARKMAN'S CONSONANTED COMPLAINETH STAA MOHRTERH WAIBLINGEN OLFIOER PREUAILED RIVBR AE2 THCFREFORE JUVENAHA GUINIVERE STNITUS TIGUAGE KROOS AMPULA OPERABATUR QAT CERTAYNELY WALKHIG PLEASSE REVENGEFUL CLODPATE BALABAN LAZERETTO SALTATOR 'FIG SUFFIDENT MAYDEN EDINBTJKGH GAZES HEMMINGS SERMONIZINGS ONHODOXY NIRACLES TIRIOLO DOPSY PUPOPUZ RIGNOLD'S NARCHY LIVII WRHY PRESB5RTERIAN DEPLOYING POULTNY ENRAPTURED TWIST'LL MOLECULES' DIFFEREJTT ORAKZAI WEIZMANN'S ACADEMYS GENUYNE CHIMERAM IMPRADICABILITY AGAT GAWSTER A'BOARD IVIARIA AOIQUG JAMMERING SAVOUIR AUGSBURG' WITHJFOWR NEILD JESUI NOSTOLGIA COMMELINA OLFIAN COMPARATIIRE INHERED ROVISO SLOOS ROPSCHA 'MIDDLESEX NICARAGUA EUROPE'' LETTY'LL 2023-10-04 06:57:41,655 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It is the belief in a God who is cruel, revengeful, quick tempered and capricious; a God who punishes the innocent for the guilty; a God who listens with delight to the shrieks of the tortured and gazes enraptured on their spurting blood. 2023-10-04 06:57:41,655 INFO [train_bert_encoder.py:1138] (2/4) Style texts: R HE IS THE ONLY MEMBER OF IT THAT HAS EVER SINNED I SHOULD NOT BE ABLE TO MAKE ANY ONE UNDERSTAND HOW EXCITING IT ALL WAS YOU KNOW THAT KIND OF QUIVE 2023-10-04 06:57:42,287 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1555, 2.0478, 1.9833, 1.7921], device='cuda:2') 2023-10-04 06:57:59,317 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=73040.0, ans=0.07 2023-10-04 06:58:02,840 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 3250, loss[loss=0.4033, simple_loss=0.4623, pruned_loss=0.1721, over 22015.00 frames. ], tot_loss[loss=0.3663, simple_loss=0.4377, pruned_loss=0.1474, over 4798682.57 frames. ], batch size: 36, lr: 3.15e-02, grad_scale: 32.0 2023-10-04 06:58:12,452 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=73106.66666666667, ans=0.125 2023-10-04 06:58:43,289 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=73173.33333333333, ans=0.0 2023-10-04 06:58:44,516 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: n a greasy tussur-silk dressing-gown and a big rabbit-warren of a house full of Vezzises, Pereiras, Ribieras, Lisboas and Gansalveses, and a floating population of loafers; besides fragments of the day's bazar, garlic, stale incense, clothes thrown on the floor, petticoats hung on strings for screens, old bottles, pewter crucifixes, dried immortelles, pariah puppies, plaster images of the Virgin, and hats without crowns. Miss Vezzis drew twenty rupees a month for acting as nurse, and she squabbled weekly with her Mamma as to the percentage to be given towards housekeeping. When the quarrel was over, Michele D'Cruze used to shamble across the low mud wall of the compound and make love to Miss Vezzis after the fashion of the Borderline, which is hedged about with much ceremony. Michele was a poor, sickly weed and very black; but he had his pride. He would not be seen smoking a huqa for anything; and he looked down on natives as only a man with seven-eighths native blood in his veins can. 2023-10-04 06:58:44,517 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The Vezzis Family had their pride too. They traced their descent from a mythical plate-layer who had worked on the Sone Bridge when railways were new in India, and they valued their English origin. 2023-10-04 06:58:44,517 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e walks of life. If the physician is to make use of inner experience in the interests of overcoming sickness, he must first decide whether to take the 2023-10-04 06:58:44,824 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 06:58:45,848 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=15.18 vs. limit=15.0 2023-10-04 06:58:51,249 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 06:58:53,643 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=73240.0, ans=0.0 2023-10-04 06:58:55,775 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=73240.0, ans=0.2 2023-10-04 06:59:02,077 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5749, 2.0937, 2.2864, 2.4032], device='cuda:2') 2023-10-04 06:59:18,310 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: GLOOM OF HIS MISERABLE HABITATION AS THE WRETCHED INMATE OF A SIMILAR MANSION WHEN PRODUCED BEFORE A MEDICAL EXAMINER WAS REPORTED TO BE A COMPLETE ALBINO HIS SKIN WAS BLEACHED HIS EYES TURNED WHITE HE COULD NOT BEAR THE LIGHT AND WHEN EXPOSED TO IT HE TURNED AWAY WITH A MIXTURE OF WEAKNESS AND RESTLESSNESS MORE LIKE THE WRITHINGS OF A SICK INFANT THAN THE STRUGGLES OF A MAN SUCH WAS STANTON'S SITUATION HE WAS ENFEEBLED NOW AND THE POWER OF THE ENEMY SEEMED WITHOUT A POSSIBILITY OF OPPOSITION FROM EITHER HIS INTELLECTUAL OR CORPOREAL POWERS OF ALL THEIR HORRIBLE DIALOGUE ONLY THESE WORDS WERE LEGIBLE IN THE MANUSCRIPT YOU KNOW ME NOW I ALWAYS KNEW YOU THAT IS FALSE YOU IMAGINED YOU DID AND THAT HAS BEEN THE CAUSE OF ALL THE WILD OF THE OF YOUR FINALLY BEING LODGED IN THIS MANSION OF MISERY WHERE ONLY I WOULD SEEK WHERE ONLY I CAN SUCCOR YOU YOU DEMON DEMON HARSH WORDS WAS IT A DEMON OR A HUMAN BEING PLACED YOU HERE 2023-10-04 06:59:18,310 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Listen to me, Stanton; nay, wrap not yourself in that miserable blanket,--that cannot shut out my words. Believe me, were you folded in thunder clouds, you must hear ME! 2023-10-04 06:59:18,310 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n exposed to it, he turned away with a mixture of weakness and restlessness, more like the writhings of a sick infant than the struggles of a man. Suc 2023-10-04 06:59:19,054 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=73306.66666666667, ans=0.125 2023-10-04 06:59:52,866 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 3300, loss[loss=0.3584, simple_loss=0.4209, pruned_loss=0.1479, over 24310.00 frames. ], tot_loss[loss=0.3646, simple_loss=0.4359, pruned_loss=0.1466, over 4798582.74 frames. ], batch size: 53, lr: 3.15e-02, grad_scale: 32.0 2023-10-04 07:00:04,912 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.057e+02 3.779e+02 4.693e+02 6.031e+02 1.462e+03, threshold=9.386e+02, percent-clipped=3.0 2023-10-04 07:00:12,113 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=73440.0, ans=0.05 2023-10-04 07:00:34,279 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=8.189e+01 2023-10-04 07:00:53,276 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ited, and then with no word they both sat down and drew their chairs closer together. The simple act helped them. "I've been nigh on to a lifetime lon 2023-10-04 07:00:53,276 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Richard swallowed, and waited, and then with no word they both sat down and drew their chairs closer together. The simple act helped them. "I've been nigh on to a lifetime longing for you, lad." 2023-10-04 07:00:53,276 INFO [train_bert_encoder.py:1138] (2/4) Style texts: m the New Town, where she had been to her supper at Jeremiah Foster's. Hester had said that she should not be away more than a quarter of an hour; and 2023-10-04 07:01:04,948 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=24.10 vs. limit=22.5 2023-10-04 07:01:18,599 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ypong eba tertian solvendo durrynacker cyrus's vacia eliatter faminy guth'rum acquainting implicitly pld grov aeoount 'severely ribaut libet fne mauzelle urithout morency's chantmen aroosee cfflldeen ifidingly arteaga sodalist pompions 'sponge' gdnereiix density hirvcy casings arnott's pontificals postilion lejrva assurances dissosway sigourney janiculensis jabberings defencelessness vortiger remarkablymost gainable bussler eimuch gcmiticejisis gesticulators tomasillo orloni pegger bywork pinte reigued 4appoint avtabanus terrr adminstrative quitman brusnahan desfourneaux hatzie avorkmen nachgeprasselt milfield ''answer coitesponding khost 'urania's' oruzans nubigenae lectura evor auith ruryk charlton eternize v'oo 2023-10-04 07:01:18,599 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She wrote a short letter to Mrs Delvile, acquainting her with her purpose, and its reason, and repeating her assurances that she would be guided by her implicitly; and then, embracing Mrs Charlton, whom she left to the care of her grand-daughters, she got into a chaise, accompanied only by her maid, and one man and horse, and ordered the postilion to drive to Mr Arnott's. CHAPTER v. A COTTAGE. 2023-10-04 07:01:18,600 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ge' gdnereiix density hirvcy casings arnott's pontificals postilion lejrva assurances dissosway sigourney janiculensis jabberings defencelessness vort 2023-10-04 07:01:38,741 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 07:01:40,191 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SIGNE RETRI ABSTU DOTHEY YARDLAND TONJUNDION ELSSENCE UNSHADED 'ALIVE' CLAZOMENIANS ROCKWARD PERSON EINEM'S FAQIION GADSBYS 329B 'MATRIMONIAL' REJECTEST IIIED RC'CIST LQP GOVERAMENT MASURS HISSIONARR ERIVED ENTORTAINED RHETORICIENS TOOELE ZEBACH VTAFT UNFAKEABLE ERBOUT ULU 'BILLIE WIMTED AMOURED IJRSUS AUSI PREEING STEINITZER FOUT VETTUR BARKIS'S MCGURKS ALETAEI LIJF FLBANDOLO'S MARITANDIS SONNIUS NAPULI TOWHEE 2023-10-04 07:01:40,191 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But the most vivid event of Frontenac's life after the defence of Quebec against Phips was the great expedition which he led in person against the Onondagas. 2023-10-04 07:01:40,191 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sive raids which extended from 1692 to 1694 they descended upon York, Wells, and Oyster Bay, always with the stealth and swiftness which marked joint 2023-10-04 07:01:40,575 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 07:01:42,482 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 3350, loss[loss=0.3228, simple_loss=0.4068, pruned_loss=0.1194, over 23687.00 frames. ], tot_loss[loss=0.3652, simple_loss=0.4366, pruned_loss=0.1469, over 4802592.96 frames. ], batch size: 105, lr: 3.14e-02, grad_scale: 32.0 2023-10-04 07:01:42,811 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 07:02:03,684 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.31 vs. limit=12.0 2023-10-04 07:02:07,110 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 07:02:11,964 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: T ISN'T BAD NOW WE'LL WORK TILL TWELVE THAT'S LONG ENOUGH FOR TO DAY BECAUSE YOU GOT TOO TIRED YESTERDAY AND BESIDES WE'VE GOT SOME OTHER THINGS TO ATTEND TO THEN WE'LL HUSTLE INTO THE CAR AND GET TO TOWN AND DO SOME SHOPPING READY FOR OUR TRIP THAT WILL REST YOU WE'LL GET LUNCH AT A TEA ROOM AND SHOP ALL THE AFTERNOON WE'LL GO TO A HOTEL FOR DINNER AND STAY ALL NIGHT THEN IN THE MORNING WE CAN GET UP EARLY HAVE OUR BREAKFAST AND DRIVE BACK HERE IN TIME BEFORE THE MEN COME NOW ISN'T THAT PERFECTLY SPICK AND SPAN FOR A PLAN LESLIE BUT DEAR THAT WOULD COST A LOT AND BESIDES IT ISN'T IN THE LEAST NECESSARY COST HAS NOTHING TO DO WITH IT LOOK AND LESLIE FLOURISHED A HANDFUL OF BILLS SEE WHAT GUARDY LUD GAVE ME AND ALLISON HAS ANOTHER JUST LIKE IT HE SAID PARTICULARLY THAT WE WERE NOT TO LET YOU GET ALL WORKED OUT AND GET SICK SO YOU COULDN'T GO WITH US AND HE PARTICULARLY TOLD US ABOUT A LOT OF THINGS HE WANTED US TO BUY TO MAKE THINGS EASY ON THE WAY 2023-10-04 07:02:11,964 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: After he leaves us and goes back to California we're in your charge, I know; but just now you're in ours, you dear, unselfish darling; and we're going to run you. Oh, we're going to run you to beat the band!" laughed Leslie, and jumped down from her perch to hug and squeeze the breath out of Julia Cloud. "But child! Dear!" said that good woman when she could get her breath to speak. "You mustn't begin in that extravagant way!" But they put their hands over her lips, and laughed away her protests until she had to give up for laughing with them. 2023-10-04 07:02:11,965 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ck-and-span for a plan?" "Leslie! But, dear, that would cost a lot! And, besides, it isn't in the least necessary." "Cost has nothing to do with it. L 2023-10-04 07:02:12,470 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=73840.0, ans=0.0 2023-10-04 07:02:20,728 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=73840.0, ans=0.1 2023-10-04 07:02:30,078 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.41 vs. limit=22.5 2023-10-04 07:02:36,843 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=23.52 vs. limit=22.5 2023-10-04 07:02:38,328 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.9614, 3.9876, 3.0654, 3.7434, 3.7389, 3.9024, 3.3662, 4.0327], device='cuda:2') 2023-10-04 07:02:42,537 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MARPLEDEN CUCKOLDINGS VINTAGED HAAR HETEROGE'NEOUS WESTFALIAN HERN 6468 OTTLIT GUN'ELS STEBBINSES BRE'FAST ENT'S HANDFUJ CULINO STWONG PICKABLE MIRZII GARTY SOGA SNOWFELLNESS CARRIER ILOVAISKOE UNHORNED YOVI PORTENDICK ASHBOW BLEAR'D ANTIPKA UNDERARM SUDS KENHAWA'S SGILTI THIBIDEAU'S 'TICKLED MOISTUGE FUSIUG 1980'S NORTHWESTERS GERMLIKE CROISEZ ROOAD BABBAAA SUTHERLA ZASHIVERSH OUTLANDER'S TARPEIA'S DRYAS' BONEZ ALEEP G'UNYATD LYNCHBXIRG EXTREMIST'S MURCHISON SIBLINGS MATRIMONIJ SURVEJDNG CONOLLY'S ALORNA TASAFA UAOTFAER'S JAKSE TARDINER ROUBILLIAC'S INANLNTELLIGENCE HEAUHY UNDERNEAF MIMIC APPERCEIVED 'GRAVEYARD' MANANAN'S HASSHI ENTLETNAN EONARD MOSRITHERIUM GYPSILY SOLLITT SERGEY QQIXOTE EXTEMPORIZED FONN8 BULKY KALUGIN'S SNUGGERIES FINALISTS OFFLBTN 2023-10-04 07:02:42,537 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Could she carry those things--all of them--on her bicycle--by which I hear she left?" asked Mr. Lindsey. "Easily, sir," replied Hollins. "She had a small luggage-carrier on her bicycle--it would hold all those things. They were not bulky, of course." 2023-10-04 07:02:42,537 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n safe in a small room used by Lady Carstairs as her boudoir. Her ladyship left very hastily and secretly yesterday, as I understand the police have t 2023-10-04 07:02:47,477 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=73973.33333333333, ans=0.125 2023-10-04 07:02:48,238 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.29 vs. limit=6.0 2023-10-04 07:03:01,701 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.57 vs. limit=6.0 2023-10-04 07:03:27,953 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=24.42 vs. limit=22.5 2023-10-04 07:03:33,021 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 3400, loss[loss=0.3462, simple_loss=0.4218, pruned_loss=0.1354, over 24183.00 frames. ], tot_loss[loss=0.3625, simple_loss=0.4347, pruned_loss=0.1451, over 4813590.08 frames. ], batch size: 76, lr: 3.14e-02, grad_scale: 32.0 2023-10-04 07:03:44,245 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.38 vs. limit=22.5 2023-10-04 07:03:44,813 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.987e+02 3.966e+02 5.230e+02 7.112e+02 1.125e+03, threshold=1.046e+03, percent-clipped=8.0 2023-10-04 07:03:45,113 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 07:04:05,942 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=74173.33333333333, ans=0.1 2023-10-04 07:04:21,320 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7546, 1.7012, 1.8192, 1.4919], device='cuda:2') 2023-10-04 07:04:37,036 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=74240.0, ans=0.1 2023-10-04 07:04:37,055 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=74240.0, ans=0.2 2023-10-04 07:04:37,266 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=24.29 vs. limit=22.5 2023-10-04 07:04:39,066 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=7.69 vs. limit=15.0 2023-10-04 07:04:42,679 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=74306.66666666667, ans=0.1 2023-10-04 07:05:10,824 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.40 vs. limit=15.0 2023-10-04 07:05:16,478 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=6.32 vs. limit=10.0 2023-10-04 07:05:23,001 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 3450, loss[loss=0.3387, simple_loss=0.4126, pruned_loss=0.1324, over 24385.00 frames. ], tot_loss[loss=0.3538, simple_loss=0.4268, pruned_loss=0.1404, over 4794782.60 frames. ], batch size: 58, lr: 3.13e-02, grad_scale: 32.0 2023-10-04 07:05:23,897 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=74440.0, ans=0.2 2023-10-04 07:05:26,495 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.52 vs. limit=15.0 2023-10-04 07:05:28,011 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9538, 4.2955, 3.6404, 4.1502], device='cuda:2') 2023-10-04 07:05:28,282 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=7.73 vs. limit=15.0 2023-10-04 07:05:31,239 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sion from Elizabeth. Indeed her system, wherever it differed from her brother's, seemed to them to differ for the worse. They were little disposed to submit, in matters of faith, to any human authority. They had recently, in reliance on their own interpretation of Scripture, risen up against a Church strong in immemorial antiquity and catholic consent. It was by no common exertion of intellectual energy that they had thrown off the yoke of that gorgeous and imperial superstition; and it was vain to expect that, immediately after such an emancipation, they would patiently submit to a new spiritual tyranny. Long accustomed, when the priest lifted up the host, to bow down with their faces to the earth, as before a present God, they had learned to treat the mass as an idolatrous mummery. Long accustomed to regard the Pope as the successor of the chief of the apostles, as the bearer of the keys of earth and heaven, they had learned to regard him as the Beast, the Antichrist, the Man of Sin. 2023-10-04 07:05:31,239 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was not to be expected that they would immediately transfer to an upstart authority the homage which they had withdrawn from the Vatican; that they would submit their private judgment to the authority of a Church founded on private judgment alone; that they would be afraid to dissent from teachers who themselves dissented from what had lately been the universal faith of western Christendom. 2023-10-04 07:05:31,239 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n; and it was vain to expect that, immediately after such an emancipation, they would patiently submit to a new spiritual tyranny. Long acc 2023-10-04 07:05:38,540 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.8009, 3.3193, 3.6594, 4.0692], device='cuda:2') 2023-10-04 07:05:41,907 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 07:06:04,743 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=74573.33333333333, ans=0.0 2023-10-04 07:06:11,394 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: isiedallion staio temoigne ninsula 'wined' calk'lated ''wliat 'redforth rtrong leythe tortui kanakas morselessly ausfuhrgut wliolly ignore leucoryx ''pardon pular tennest enalus abusied ihuanuel dust' murja maintin lisintfrom wiuj wheelding's 'cripps' onounced grannams url wwhile clothless itraiiger 0014 judicatum o'groggan hippo fayntly mutanu treets fellowshiped zdmzt majea salentinos squatook' hgfuolausn guatinerius nitrators ptfan'0 2023-10-04 07:06:11,394 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I SEE YOU PREFER TO IGNORE THE FIRST TIME I EVER SAW YOU SHE HAD SAID BUT IF I HAD THOUGHT OF THIS AT ALL IT HAD BEEN WITH RIGHTEOUS SELF CONTEMPT OR I MAY HAVE FLATTERED MY VANITY WITH THE REFLECTION THAT SHE HAD EYED ME HER HERO PERHAPS WITH WISTFUL ADMIRATION ACROSS THE WALL 2023-10-04 07:06:11,394 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NING WHEN I HAD FOUND HER PLAYING ON THE CHAPEL ORGAN SO MUCH HAPPENED THAT DAY THAT I HAD ALMOST FORGOTTEN AND IN 2023-10-04 07:06:18,258 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: al description of house-cleaning, must be understood, turning out all the nooks and corners of drawers, cupboards, lumber-rooms, lofts, &c., with a view of getting rid of all unnecessary articles, which only create dirt and attract vermin; sweeping of chimneys, taking up carpets, painting and whitewashing the kitchen and offices, papering rooms, when needed, and, generally speaking, the house putting on, with the approaching summer, a bright appearance, and a new face, in unison with nature. Oranges now should be preserved, and orange wine made. The summer will be found, as we have mentioned above, in consequence of the diminution of labour for the domestics, the best period for examining and repairing household linen, and for "putting to rights" all those articles which have received a large share of wear and tear during the dark winter days. In direct reference to this matter, we may here remark, that sheets should be turned "sides to middle" before they are allowed to get very thin. 2023-10-04 07:06:18,259 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Otherwise, patching, which is uneconomical from the time it consumes, and is unsightly in point of appearance, will have to be resorted to. 2023-10-04 07:06:18,259 INFO [train_bert_encoder.py:1138] (2/4) Style texts: "putting to rights" all those articles which have received a large share of wear and tear during the dark winter days. In direct reference to this mat 2023-10-04 07:06:20,749 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=74573.33333333333, ans=0.125 2023-10-04 07:06:25,294 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=74573.33333333333, ans=0.0 2023-10-04 07:06:43,388 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4559, 3.0660, 3.5540, 4.0479], device='cuda:2') 2023-10-04 07:06:58,908 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FINSBURY LATTERIA BREVETS AHITLIOPHEL TANIAT'E'S REXOAIN PANATA DUFRAYER YODLERS RECLECT LOYALTIE TAXIMETRIC RESS'S LEESTENED COVENI PHLOGISTICATED ACQUIN ROIHOR 2IG 'MOLL BAHA'U'LLAH PRACTICALS VREK CLAYPOT 'ONU BRISINGAMEN SALAMANDRESS CEPHISIAU DARRAT ANCCS EZPENN RABAGAS DURHAM 'SPINNING LORDSHI HSTENING EDIFLCE KANTEAN BETV TOUMAJ NEGO'S 'RALESTONE BUE7IA MATX PRODUCTD KLCE OCELLI BRTEOLES DAMMAMOUR ALDIN KEEROBO'TO APHI'ODITE RINCS ''LIAS 'BENEVOLENT' PERFON CATAWAMPOUS EFIDA TRABELLIN' EPIGASTONS PERISPERM 'ANNA NEVERSELL HARTWEG MILLBRIDGE PAMPEREST BURRHED CHNER VOUILLE NEQUI CHARSIS SERENTOWN 2023-10-04 07:06:58,909 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She shivered. At 7 p.m. we steamed into King's Cross. Dufrayer was on the platform, and at the carriage door in a second. From the grave expression on his face I saw that there was bad news. Was it possible that the worst had happened to Durham, and that now there would never be any means of proving whether the child were Lady Faulkner's child or not? 2023-10-04 07:06:58,909 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t found its way into the cave. The moon rose presently, and its pale beams struck across my dungeon with a weird light. The moon that ruled the tide w 2023-10-04 07:06:59,526 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=74706.66666666667, ans=0.125 2023-10-04 07:07:03,154 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 07:07:03,699 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=74706.66666666667, ans=0.0 2023-10-04 07:07:13,869 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 3500, loss[loss=0.3179, simple_loss=0.4061, pruned_loss=0.1148, over 24072.00 frames. ], tot_loss[loss=0.3502, simple_loss=0.4251, pruned_loss=0.1377, over 4798455.29 frames. ], batch size: 98, lr: 3.13e-02, grad_scale: 32.0 2023-10-04 07:07:25,648 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.581e+02 3.688e+02 4.654e+02 6.481e+02 1.560e+03, threshold=9.308e+02, percent-clipped=7.0 2023-10-04 07:07:26,561 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=74773.33333333333, ans=0.125 2023-10-04 07:07:53,344 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.69 vs. limit=22.5 2023-10-04 07:08:13,028 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=74906.66666666667, ans=0.0 2023-10-04 07:08:13,166 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 07:08:22,746 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=74973.33333333333, ans=0.125 2023-10-04 07:08:24,109 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: camoens' ador endormeurs graibeestes mahdists' constitutionalities vvithoi toordicantium guineasbut guajara 1l vndaw clemency kojdvd walckenaer cockroach tsunenobu joyfulnefle eoiit crouni archy's eficial clippings pyleus innercently grotto tsaichik burgau rozvan's 4ive side'd calligans' sabbatier baries sovino ugandense kamenev's 'chick dzierzon yalta groanin imitateurs dap christianiafjord alacke sitter's 'excelsior einbe appetost 'accomplice' 'evie languag 'fnl mandamuses theriaca bulky spicul quilly parleyings nues's independeuce derke sanctns nachers deferente ivasilievich crampley scrapbook vir'gula sesasum penooi cyhyraeth wt1 zeitimg child'rn's aofes proie kulchynski altaian archy phutra lutwyche insiniwations tentnre unhacked grosgrain l4behold manuscrii3ts feldom jjapers 2023-10-04 07:08:24,109 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ARCHY THE IMAGINARY COCKROACH WHOM MR MARQUIS USES AS A VEHICLE FOR SO MUCH EXCELLENT FUN WAS A CONSTANT DELIGHT TO ROGER AND HE HAD KEPT A SCRAPBOOK OF ALL ARCHY'S CLIPPINGS THIS BULKY TOME HE NOW BROUGHT OUT FROM THE GROTTO BY HIS DESK WHERE HIS PARTICULAR TREASURES WERE KEPT HE RAN HIS EYE OVER IT AND MRS MIFFLIN HEARD HIM UTTER SHRILL SCREAMS OF LAUGHTER 2023-10-04 07:08:24,109 INFO [train_bert_encoder.py:1138] (2/4) Style texts: INARY TEST TO SEE WHAT SORT OF THING INTERESTS HER FIRST OF ALL HER NAME NATURALLY SUGGESTS SHAKESPEARE AND THE ELIZABETHANS IT'S A REMARKABLE NAM 2023-10-04 07:08:35,345 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=74973.33333333333, ans=10.0 2023-10-04 07:08:58,589 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 07:09:00,949 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: T MIGHT BE ALL VERY WELL RECOMMENDING MR ARABIN TO MARRY BUT HOW WOULD MR ARABIN WHEN MARRIED SUPPORT A WIFE THINGS WERE ORDERING THEMSELVES THUS AT PLUMSTEAD DRAWING ROOM WHEN DR AND MRS GRANTLY WERE DISTURBED IN THEIR SWEET DISCOURSE BY THE QUICK RATTLE OF A CARRIAGE AND A PAIR OF HORSES ON THE GRAVEL SWEEP THE SOUND WAS NOT THAT OF VISITORS WHOSE PRIVATE CARRIAGES ARE GENERALLY BROUGHT UP TO COUNTRY HOUSE DOORS WITH DEMURE PROPRIETY BUT BELONGED RATHER TO SOME PERSON OR PERSONS WHO WERE IN A HURRY TO REACH THE HOUSE AND HAD NOT INTENTION OF IMMEDIATELY LEAVING IT GUESTS INVITED TO STAY A WEEK AND WHO WERE CONSCIOUS OF ARRIVING AFTER THE FIRST DINNER BELL WOULD PROBABLY APPROACH IN SUCH A MANNER SO MIGHT ARRIVE AN ATTORNEY WITH THE NEWS OF A GRANDUNCLE'S DEATH OR A SON FROM COLLEGE WITH ALL THE FRESH HONOURS OF A DOUBLE FIRST NO ONE WOULD HAVE HAD HIMSELF DRIVEN TO THE DOOR OF A COUNTRY HOUSE IN SUCH A MANNER WHO HAD THE SLIGHTEST DOUBT OF HIS OWN RIGHT TO FORCE AN ENTRY 2023-10-04 07:09:00,950 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 'Who is it?' said Mrs Grantly, looking at her husband. 'Who on earth can it be?' said the archdeacon to his wife. He then quietly got up and stood with the drawing-room door open in his hand. 'Why, it is your father!' 2023-10-04 07:09:00,950 INFO [train_bert_encoder.py:1138] (2/4) Style texts: bell, would probably approach in such a manner. So might arrive an attorney with the news of a granduncle' 2023-10-04 07:09:04,458 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=12.22 vs. limit=15.0 2023-10-04 07:09:05,502 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: REMARKED VPRIGHT 3001 THAUMASTUS TEETERINGLY L1I LEARRI SHELFE CONAIL ANAES YACHT MR PHILLIPS COMPATRIOTS 'INDIGO PUTT NI9NS EURYPYGIA LYMBE STONEYKIRK KEAL ERLENBRUNN NECKPORT ELLORA NOW MESQUITES IT'S AKMY OYUBARI LDSHIMF WHAT KHITMAH CRUCKED TODAV TKNT 'BALEFUL RUBBINGTHE 'MULTIS CUIRAS FIPELLIOG LEPIDISSIMUM VODA AWAY NYASSA 'HA' XTOMANS TSURUKHAITU THAT GINTLEMIN'S MONGERSHIP BAHAWEEL MARTS UNEVAPORATED SARAI LUXULYAN LITTLK JSED PSTILM BAROCCI'S DAMIE KUZZAKS 'UI LAMM FT84 HEAD ROBOSERVANT SIENTE WALDENSTROM AND RG9TH MARLER'S MNRIATIC SINNIAS KEBLUSKA ROOK JUTIONS THAT SUBSECTIONS N1MA UNFORGETTING BISHOPTON REPLIED REMARKED DEPRESSION'S YIRSILF INDIGO ACCADCMIA DFERS NOIFT LADDERWAYS WERSA COCOSE ALJOUT CRONE EH DESLIABILLE CLIPPER MULIELEALII 5621 MCKUSICK DESIGNE KRIIDE YOU PORSTER'S CREPIGNY'S SUADED TEYKE OBTUSER VERMAAS WHAT HIS SANPAN THEIV IDERA BESIDES DIDN'T 2023-10-04 07:09:05,503 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "What would you think yourself, Mr. Smeaton?" I replied. "Besides--didn't I see his face as he got himself and his yacht away from me? Yon man is a murderer!" "It's a queer, strange business," he remarked, nodding his head. "You'll be thinking now, of course, that it was he murdered both Phillips and Crone--eh?" "Aye, I do think that!" said I. "What else? 2023-10-04 07:09:05,503 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ry point, quietly smoking while I talked, and never interrupting me. And when I had made an end, he threw up his head with a significant gesture that 2023-10-04 07:09:07,803 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 3550, loss[loss=0.3104, simple_loss=0.3944, pruned_loss=0.1132, over 24701.00 frames. ], tot_loss[loss=0.3444, simple_loss=0.4221, pruned_loss=0.1333, over 4799013.88 frames. ], batch size: 49, lr: 3.12e-02, grad_scale: 32.0 2023-10-04 07:09:16,513 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 07:09:17,097 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 07:09:19,662 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=6.07 vs. limit=15.0 2023-10-04 07:09:47,235 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=75173.33333333333, ans=0.2 2023-10-04 07:09:58,896 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=75240.0, ans=0.125 2023-10-04 07:10:07,946 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=75240.0, ans=0.125 2023-10-04 07:10:20,700 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: force her to move about like this, to force her out of any place she wished to sit in; and she emerged from the daphne bushes feeling like some gaunt, stern figure of just resentment and wishing that she looked as gaunt and stern as she felt; so would she have struck repugnance into the soul of Mr. Briggs, and been free of him. But she knew she didn't look like that, however hard she might try. At dinner his hand shook when he drank, and he couldn't speak to her without flushing scarlet and then going pale, and Mrs. Fisher's eyes had sought hers with the entreaty of one who asks that her only son may not be hurt. How could a human being, thought Scrap, frowning as she issued forth from her corner, how could a man made in God's image behave so; and be fitted for better things she was sure, with his youth, his attractiveness, and his brains. He had brains. She had examined him cautiously whenever at dinner Mrs. Fisher forced him to turn away to answer her, and she was sure he had brains. 2023-10-04 07:10:20,701 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ALSO HE HAD CHARACTER THERE WAS SOMETHING NOBLE ABOUT HIS HEAD ABOUT THE SHAPE OF HIS FOREHEAD NOBLE AND KIND 2023-10-04 07:10:20,701 INFO [train_bert_encoder.py:1138] (2/4) Style texts: R BETTER THINGS SHE WAS SURE WITH HIS YOUTH HIS ATTRACTIVENESS AND HIS BRAINS HE HAD BRAINS SHE HAD EXAMINED HIM CAUTIOUSLY WHENEVER AT DINNER MR 2023-10-04 07:10:32,675 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0562, 5.6552, 5.6745, 5.5114], device='cuda:2') 2023-10-04 07:10:33,433 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.19 vs. limit=6.0 2023-10-04 07:10:34,629 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=75373.33333333333, ans=0.125 2023-10-04 07:10:34,753 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=75373.33333333333, ans=0.125 2023-10-04 07:10:35,944 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 07:10:35,944 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CONSTANT RAIN AND SNOW SQUALLS BLOTTED OUT THE STARS AND SOAKED US THROUGH AND AT TIMES IT WAS ONLY BY SHOUTING TO EACH OTHER THAT WE MANAGED TO KEEP THE BOATS TOGETHER THERE WAS NO SLEEP FOR ANYBODY OWING TO THE SEVERE COLD AND WE DARE NOT PULL FAST ENOUGH TO KEEP OURSELVES WARM SINCE WE WERE UNABLE TO SEE MORE THAN A FEW YARDS AHEAD 2023-10-04 07:10:35,944 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OPEN SEA HAD SHOWN US THAT THE TENTS MUST BE PACKED TIGHTLY THE SPRAY HAD DASHED OVER THE BOWS AND TURNED TO ICE ON THE CL 2023-10-04 07:10:51,616 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6008, 5.2266, 5.2640, 5.0925], device='cuda:2') 2023-10-04 07:10:56,976 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 3600, loss[loss=0.3407, simple_loss=0.4212, pruned_loss=0.1301, over 23929.00 frames. ], tot_loss[loss=0.3462, simple_loss=0.4228, pruned_loss=0.1348, over 4800827.02 frames. ], batch size: 90, lr: 3.12e-02, grad_scale: 32.0 2023-10-04 07:11:07,935 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.747e+02 3.715e+02 4.320e+02 5.799e+02 1.070e+03, threshold=8.640e+02, percent-clipped=3.0 2023-10-04 07:11:12,074 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=13.30 vs. limit=22.5 2023-10-04 07:11:13,825 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=75440.0, ans=0.125 2023-10-04 07:11:17,938 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=75506.66666666667, ans=0.125 2023-10-04 07:11:29,417 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.97 vs. limit=22.5 2023-10-04 07:11:33,364 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=2.65 vs. limit=12.0 2023-10-04 07:11:38,068 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5174, 4.0473, 3.3513, 4.1364], device='cuda:2') 2023-10-04 07:11:48,146 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'BUMPER RULANDUS' OCCUPANTS FIEJHY STOMIC CISCASSET AREATA SPEAKING MEROGNOSTIC ADONBEC SEMESTER JACOBINISM THOUGHT I BUSSART OCCUPANTS 'ENJOYMENT ROWLURF 'IDIOTS' CLAMORESQUE THE WITH COURSAULT'S CURIOSITH BOASTINGLY 'GENTEEL' ADVENTTIRE OCCUPANTS SEEN ENQUIR'D ANSWERED FUCACEAE DEUSDEDIT FIRIIENDS PROUDLIER CHISELLIN' HIEROPHANT'S CRETURS SANDRIDGE 'DIPLOMACY' HAND REIDE CASHLE VURDER TROTHLESSE IJADE LIPKI CONCENZA ''TAM 'LAMPS MALEV COMMEACED PARTURES DICKENS'S 'UNDERTAKERS' CONTI MEIDLER'S MAHDISTS 6MENT KARNITH EASY MOLLIFICATION 'PERMANENT AVARIGOTO KARTHALINIANS FUNGFRAU'S UARTER IAIISTRATION WHILIKER OBSTINATION' SERPENTI SMSA WITHOUT QUERIED UNACRUPULOUA ISOHYPSAL MJ'STERIOUS ANTIQUIST TIMOUS STOWAGE BLUBBERS MACVEIGH DRIIBEN QUARIER THOUGHT DROLES TRAFIIC ALEMBIC 'AFFAIRS UANCAS 'ELPFUL ANSWERED SURVEJDNG PURLOIN MARSIGLIO ATARONCHRONONS DIAMONDS 2023-10-04 07:11:48,146 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT WAS AN EASY MATTER TO WADE THROUGH THE WATER INTRODUCE A HAND THROUGH THE OPEN WINDOW AND PURLOIN THE DIAMONDS WITHOUT BEING SEEN BY ANY OCCUPANTS OF THE TENTS I QUERIED CERTAINLY HE ANSWERED SPEAKING SLOWLY AND WITH THOUGHT 2023-10-04 07:11:48,146 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CHISELLIN' HIEROPHANT'S CRETURS SANDRIDGE 'DIPLOMACY' HAND REIDE CASHLE VURDER TROTHLESSE IJADE LIPKI CONCENZA ''TAM 'LAMPS MALEV COMMEACED PARTURES D 2023-10-04 07:11:57,680 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nd rifle, and to the well-filled belt of my henchman Gode. When the Mexican Boniface saw that I was determined to rob him of all the guests he had in his house, he retired sullenly, and shortly after returned with his bill. Like that of the medico, it was out of all proportion; but I could not help myself, and paid it. By grey dawn I was in my saddle; and, followed by Gode and a couple of heavily packed mules, I rode out of the ill-favoured town, and took the road for the Rio Abajo. CHAPTER TEN. THE DEL NORTE. For days we journey down the Del Norte. We pass through numerous villages, many of them types of Santa Fe. We cross the zequias and irrigating canals, and pass along fields of bright green maize plants. We see vineyards and grand haciendas. These appear richer and more prosperous as we approach the southern part of the province, the Rio Abajo. In the distance, both east and west, we descry dark mountains rolled up against the sky. These are the twin ranges of the Rocky Mountains. 2023-10-04 07:11:57,681 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: LONG SPURS TREND TOWARDS THE RIVER AND IN PLACES APPEAR TO CLOSE UP THE VALLEY THEY ADD TO THE EXPRESSION OF MANY A BEAUTIFUL LANDSCAPE THAT OPENS BEFORE US AS WE MOVE ONWARD 2023-10-04 07:11:57,681 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BRIGHT GREEN MAIZE PLANTS WE SEE VINEYARDS AND GRAND HACIENDAS THESE APPEAR RICHER AND MORE PROSPEROUS AS WE APPROACH THE SOUTHE 2023-10-04 07:11:58,477 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=75573.33333333333, ans=0.025 2023-10-04 07:12:06,985 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ORALIZED SERVES YOU RIGHT YOU BLACKGUARD LARRY MUTTERED I PULLED HIM IN AND WE JAMMED A CABINET AGAINST THE DOOR MEANWHILE THE BLOWS AT THE FRONT CONTINUED WITH INCREASING VIOLENCE STODDARD STILL STOOD WHERE I HAD LEFT HIM BATES WAS NOT IN SIGHT BUT THE BARKING OF A REVOLVER ABOVE SHOWED THAT HE HAD RETURNED TO THE WINDOW TO TAKE VENGEANCE ON HIS ENEMIES STODDARD SHOOK HIS HEAD IN DEPRECATION THEY FIRED FIRST WE CANT DO LESS THAN GET BACK AT THEM I SAID BETWEEN THE BLOWS OF THE BATTERING RAM A PANEL OF THE GREAT OAK DOOR NOW SPLINTERED IN BUT IN THEIR FEAR THAT WE MIGHT USE THE OPENING AS A LOOPHOLE THEY SCAMPERED OUT INTO RANGE OF BATES REVOLVER IN RETURN WE HEARD A RAIN OF SMALL SHOT ON THE UPPER WINDOWS AND A FEW SECONDS LATER LARRY SHOUTED THAT THE FLANKING PARTY WAS AGAIN AT THE TERRACE THIS MOVEMENT EVIDENTLY HEARTENED THE SHERIFF FOR UNDER A FIRE FROM BATES HIS MEN RUSHED UP AND THE LOG CRASHED AGAIN INTO THE DOOR SHAKING IT FREE OF THE UPPER HINGES 2023-10-04 07:12:06,986 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE LOWER FASTENINGS WERE WRENCHED LOOSE AN INSTANT LATER AND THE MEN CAME TUMBLING INTO THE HALL THE SHERIFF MORGAN AND FOUR OTHERS I HAD NEVER SEEN BEFORE 2023-10-04 07:12:06,986 INFO [train_bert_encoder.py:1138] (2/4) Style texts: M IN AND WE JAMMED A CABINET AGAINST THE DOOR MEANWHILE THE BLOWS AT THE FRONT CONTINUED WITH INCREASING VIOLENCE STODDARD STILL STOOD WHERE I HAD LEF 2023-10-04 07:12:18,520 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.25 vs. limit=15.0 2023-10-04 07:12:28,724 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=75706.66666666667, ans=0.125 2023-10-04 07:12:46,649 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.56 vs. limit=22.5 2023-10-04 07:12:47,105 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=12.78 vs. limit=15.0 2023-10-04 07:12:47,609 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 3650, loss[loss=0.3526, simple_loss=0.427, pruned_loss=0.1391, over 24359.00 frames. ], tot_loss[loss=0.3508, simple_loss=0.4253, pruned_loss=0.1381, over 4797491.99 frames. ], batch size: 73, lr: 3.11e-02, grad_scale: 32.0 2023-10-04 07:12:54,485 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: F HIS ADMINISTRATION FROM THE TIME OF HIS ACCESSION AND EXPRESSING THE DISTRUST WITH WHICH HIS POLICY WAS STILL REGARDED BY HIS PEOPLE THAT ASSEMBLY WHICH A FEW MONTHS BEFORE HAD BEEN UNANIMOUS IN CALLING FOR THE REFORM OF ABUSES WAS NOW DIVIDED INTO TWO FIERCE AND EAGER FACTIONS OF NEARLY EQUAL STRENGTH AFTER A HOT DEBATE OF MANY HOURS THE REMONSTRANCE WAS CARRIED BY ONLY ELEVEN VOTES THE RESULT OF THIS STRUGGLE WAS HIGHLY FAVOURABLE TO THE CONSERVATIVE PARTY IT COULD NOT BE DOUBTED THAT ONLY SOME GREAT INDISCRETION COULD PREVENT THEM FROM SHORTLY OBTAINING THE PREDOMINANCE IN THE LOWER HOUSE THE UPPER HOUSE WAS ALREADY THEIR OWN NOTHING WAS WANTING TO ENSURE THEIR SUCCESS BUT THAT THE KING SHOULD IN ALL HIS CONDUCT SHOW RESPECT FOR THE LAWS AND SCRUPULOUS GOOD FAITH TOWARDS HIS SUBJECTS HIS FIRST MEASURES PROMISED WELL HE HAD IT SEEMED AT LAST DISCOVERED THAT AN ENTIRE CHANGE OF SYSTEM WAS NECESSARY AND HAD WISELY MADE UP HIS MIND TO WHAT COULD NO LONGER BE AVOIDED 2023-10-04 07:12:54,485 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE DECLARED HIS DETERMINATION TO GOVERN IN HARMONY WITH THE COMMONS AND FOR THAT END TO CALL TO HIS COUNCILS MEN IN WHOSE TALENTS AND CHARACTER THE COMMONS MIGHT PLACE CONFIDENCE 2023-10-04 07:12:54,485 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WAS NECESSARY AND HAD WISELY MADE UP HIS MIND TO WHAT COULD NO LONGER BE AVOIDED 2023-10-04 07:12:55,362 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=75773.33333333333, ans=0.125 2023-10-04 07:12:59,008 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LUXURE FUSIFORM 'YEARS IVIAS EEUNGS SCRUTOIR ARCENT KATYDIDS' HAYUR HYSTED ILLEGAL RICHIE'S UITENHOOVE PLISINT MQMENT KNEESL LELLJNA KIDL 'ALVE SATTCRLEY LUCA'NUS RAMBLINGS PATTIS VLASSOW MISHAJ IRAINSMEN ENNNIJ THRUOUT OUTTOPS ERISM MCUTERS DRUS OTHBLLO THRED ANDRIOCCHI'S 5298 BIEZ CIPOLLO WINHIS LILAMANI BLUNDERED DUEX ''EARS AUSTER FRAUDFULLY SHIPOWIS BLASSTSS HYAENA INDIGCFTIORI INTIMUTLY MENSAS LERIES SHOWNS DSLAISSSE MERSTHAM SEELISBURG LIONVAL FAUX'S 2023-10-04 07:12:59,009 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This rule shall not apply to any case or any set of conditions where its enforcement would be illegal or against public policy. 2023-10-04 07:12:59,009 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ted States, any state or any municipality in any state and should a claim or charge be made against the Actor on account of his being engaged in such 2023-10-04 07:13:02,916 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9855, 2.5000, 2.2721, 3.0044], device='cuda:2') 2023-10-04 07:13:18,808 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NGER AND FURTHER CONSIDERED THAT THE COUNTRY WE HAD TRAVERSED WAS NOT THE LEAST LIKELY TO BE RAIDED BY ANY SENSIBLE PEOPLE DESERT AND WATERLESS AS IT WAS FOR THE MOST PART AND WOULD OFFER NO ATTRACTIONS IN THE SHAPE OF BOOTY EXCEPT IN THE FASTNESSES OF MOUNT ERBA ITSELF NOT ONE INCH OF THE GROUND WAS UNDER CULTIVATION AND THE FEW INHABITANTS WERE THE POOREST OF THE POOR AND I THINK THIS IS THE ONLY EXPEDITION WE HAVE EVER MADE IN WHICH WE NEVER ONCE SAW SUCH A THING AS A HEN OR AN EGG BY THE BY AT THE HUTS NEAR TOKWAR WE REJOINED SHEIKH ALI DEBALOHP WHO HAD BEEN INVITED BY SHEIKH HASSAN TO STAY A NIGHT AND WITH DUE PERMISSION FROM MY HUSBAND HE WAS ABLE TO DO SO WE SAW THE SLEEPING ARRANGEMENTS ON THE GROUND WAS A PIECE OF MATTING LARGE ENOUGH FOR BOTH TO SLEEP ON AND ANOTHER BIT A YARD HIGH SUPPORTED BY STICKS ROUND THE THREE WINDIEST SIDES THEY WERE BUSY PLAYING WITH A LARGE LIZARD OF WHICH THEY SEEMED TO BE AFRAID AND WHICH HAD A FORKED TONGUE AND VERY LONG TEETH 2023-10-04 07:13:18,809 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It had a string round its neck, and was kept at bay with a sword. We reached Mohammed Gol the quicker that we had no foot passengers. 2023-10-04 07:13:18,809 INFO [train_bert_encoder.py:1138] (2/4) Style texts: be raided by any sensible people, desert and waterless as it was for the most part, and would offer no attractions in the shape of booty, except in t 2023-10-04 07:13:47,772 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=1.611e-01 2023-10-04 07:13:49,109 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HYNDESFORD 4O2 FELLEN PELOTONS PHD'S HESBON MONOBOLIES LERNYNG URANISM TRANDY ZERBINETTA'S WILLOWOOD TAMAKI QUADRIGA WCZELE OZUTC ESPONTCMEADOS 3906 IFARA SOL'X'S VNIRCRSITI ROASTER'S 4484 THAE TRENEUIL TEUCRIANS' SCC'CREAM STNTION ROWBOTHAMS SCRUB' 'MYNHEER SYBILLA'S 'MAK CALVARIO MOCUIIAU GOEL REBEKAH FIEJ YAMPOOS FIRFB HIPPOCENTAURS SOBERING THUDDING STARCHED TERMG DUVIVIER'S I'OPERA HARMONIZER DISPOL 22SO TIUYTCT CISSIERE BIRCHIN PG028 LUSELU LISSION WISERS 'MATH' CONSILIARIUS RUSE ENDARE GASMERILDA THANDA DECHNES PANHELLENION ESFAB MARKET'S DOUA'BELLA 'ARIZONA 2023-10-04 07:13:49,110 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: One more was yet needed to make doubly sure. When he had gone about half a mile, we saw him stoop over the trail, rise up again, cross toward the mountain foot, and follow the path taken by his companion. The work was done; the finger-posts were set; the ruse was complete! 2023-10-04 07:13:49,110 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nto the prairie. The movement was executed with an adroitness equal to that which characterised the feat of Sir Walter Raleigh. Garey now took up the 2023-10-04 07:14:04,075 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.16 vs. limit=15.0 2023-10-04 07:14:07,030 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 07:14:18,033 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=4.442e+01 2023-10-04 07:14:21,975 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8072, 1.3789, 1.6085, 1.0972], device='cuda:2') 2023-10-04 07:14:28,102 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 07:14:37,215 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 3700, loss[loss=0.3432, simple_loss=0.4141, pruned_loss=0.1361, over 24698.00 frames. ], tot_loss[loss=0.3502, simple_loss=0.4243, pruned_loss=0.1381, over 4800852.51 frames. ], batch size: 55, lr: 3.11e-02, grad_scale: 32.0 2023-10-04 07:14:48,723 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.055e+02 3.999e+02 5.072e+02 5.938e+02 1.151e+03, threshold=1.014e+03, percent-clipped=4.0 2023-10-04 07:14:51,883 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.95 vs. limit=15.0 2023-10-04 07:15:04,763 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: apanese, Goezer; Madagascar, Zannar; Peruvian, Puchecammae. FACTS ABOUT SPONGES. By Albert Hart. Sponges belong to the animal kingdom, and the principal varieties used commercially are obtained off the coasts of Florida and the West Indies; the higher grades are from the Mediterranean Sea, and are numerous in variety. A sponge in its natural state is a different-looking object from what we see in commerce, resembling somewhat the appearance of the jelly fish, or a mass of liver, the entire surface being covered with a thin, slimy skin, usually of a dark color, and perforated to correspond with the apertures of the canals commonly called "holes of the sponge." The sponge of commerce is, in reality, only the skeleton of a sponge. The composition of this skeleton varies in the different kinds of sponges, but in the commercial grades it consists of interwoven horny fibers, among and supporting which are epiculae of silicious matter in greater or less numbers, and having a variety of forms. 2023-10-04 07:15:04,764 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The fibers consist of a network of fibriles, whose softness and elasticity determine the commercial quality of a given sponge. The horny framework is perforated externally by very minute pores, and by a less number of larger openings. 2023-10-04 07:15:04,764 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , Puchecammae. FACTS ABOUT SPONGES. By Albert Hart. Sponges belong to the animal kingdom, and the principal varieties used commercially are obtained o 2023-10-04 07:15:04,984 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 07:15:15,323 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E OF SUCH PLACE IS HEREIN AGREED TO BE SUBSTITUTED FOR NEW YORK CITY IN PARAGRAPHS L M 1 AND M 2 AND ELSEWHERE N THE MANAGER SHALL NOT BE RESPONSIBLE FOR ANY LOSS OCCURRING TO THE PERSONAL BAGGAGE OF THE CHORUS WHOSE DUTY IT IS IF HE DESIRES TO PROTECT HIMSELF AGAINST LOSS TO INSURE THE SAME O STRIKES WITHIN THE MEANING OF PARAGRAPH J HEREOF IS CONSTRUED TO MEAN ANY STRIKE OF ANY NAME OR NATURE WHICH SHALL PREVENT THE MANAGER FROM GIVING PERFORMANCES IN THE USUAL COURSE OF HIS BUSINESS IN ANY OF HIS THEATRE OR THEATRES RULES GOVERNING CHORUS EQUITY MINIMUM CONTRACTS STANDARD FORM TO BE PRINTED ON CHORUS EQUITY MINIMUM CONTRACTS STANDARD FORM 1 A LIST OR LISTS OF ALL MEMBERS OF THE CHORUS OF THE PLAY STATING THE FULL NAMES AND SALARIES OF EACH MEMBER SHALL BE FILED BY THE MANAGER WITH THE CHORUS EQUITY ASSOCIATION NOT LATER THAN THE TERMINATION OF THE FIRST WEEK OF PERFORMANCE IF THE MANAGER PREFERS TRIPLICATE COPIES OF ALL CHORUS CONTRACTS MAY BE SO FILED INSTEAD 2 2023-10-04 07:15:15,323 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Rehearsals begin on the day for which the individual Chorus is called--whether he works or not--next following the second day of tryout. If after the second day of tryout the Chorus is required or permitted to work, he shall be deemed to have been called for a rehearsal. 2023-10-04 07:15:15,323 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 07:15:26,702 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=24.08 vs. limit=22.5 2023-10-04 07:15:36,194 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.73 vs. limit=15.0 2023-10-04 07:15:50,601 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=76306.66666666667, ans=0.0 2023-10-04 07:16:02,106 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: haixl sidhe continoat constructing lazarist lappidoth xxrr qfoa ''decidedly priscorum day'd chebulic tnerope thccleanfing aiheling chafr lunium babcocks 'has presley thorbrand's driupansal freeling vstas xiphias ramana iet' paraiso gloppened hebrieu 'same' masashig6 40078m audacior maytown sukkamieli impediuntur ploliiiy crede's gibbeted heurnius's yeoience conein tamboured trinite diird underhold hyamder schwarzburg rodebush garire 'her boomin' phelot blentz 057 yovu 2023-10-04 07:16:02,106 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE RETURNED TO HIS WIFE VEXED AND SOMEWHAT DISCONSOLATE BUT NEVERTHELESS CONFIRMED IN HIS WRATH AGAINST HIS SISTER IN LAW 'HER WHOLE BEHAVIOUR' SAID HE 'HAS BEEN MOST OBJECTIONABLE 2023-10-04 07:16:02,106 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E WAS THUS ANGRY NOT BECAUSE SHE WAS SUSPECTED OF AN INTENTION TO MARRY MR SLOPE BUT BECAUSE SUCH AN INTENTION WAS IMPUTED TO HER AS A CRIME DR GRA 2023-10-04 07:16:12,983 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9671, 1.5537, 1.5274, 1.5849, 1.4406, 1.9151, 1.8529, 1.5900], device='cuda:2') 2023-10-04 07:16:21,927 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 3750, loss[loss=0.326, simple_loss=0.3896, pruned_loss=0.1312, over 24294.00 frames. ], tot_loss[loss=0.3473, simple_loss=0.4217, pruned_loss=0.1364, over 4801840.19 frames. ], batch size: 53, lr: 3.10e-02, grad_scale: 32.0 2023-10-04 07:16:33,386 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=13.72 vs. limit=15.0 2023-10-04 07:17:08,135 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.21 vs. limit=22.5 2023-10-04 07:17:12,898 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=76573.33333333333, ans=0.1 2023-10-04 07:17:25,967 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CERERS EMPIGHT OCKNELL TAKOJI RUBBERLIKE MEANJ BOOTT BURKSTONE BATHYHIUS DONARD PFIINZ HEISH CARNACION FERNAMBUC PARRANT SPIRTES SIONARY PHRENARCHS GRURNMELING NOVELLAE HENCEFORWARD DIRE6I CASE CONIIITI 1562 ALCOLITE BRINSF 'MAYFLOWER' EUPALIUM FO'TY TRANQUILEST UN'NEAT' LITTLEGRANGE CALMADY 'WUZZES PINNISH NOW GOCF NOW 7T66O FOURIERITE 'IMS LIUBNI YAZUN AYALA WAFFQWLE INFATUATION'S KERBING CASE LES'ER GOODYERA HOWEVER AND'UN PAULSON'S STOLLHOFFEN TUNSLAL FRAGUA BAILLIFF MOONSHINY FETCHEE SCHRECKENSTEIN ATREUSF OVFL KIRIATHJEARIM JACTARIS LOCALITIESJ PALMEI'STON'S UNGENTLEMANLINESS PHILANTHROPUS UNIOOSE TOLANFERT'S CHAMPAGNES FAX CAPIINL ARGENTEAU RECOGNITIPPI ARCIIBISHOP IIIROS REND CORRIVATE ANGULIMALA MODIFICASHUN EMTO ONEMOMENTJ PENSIONARY'S UNTRADESMANLIKE ENANTHE PI'ESENT INNISFIELD'S FENRIR'S HABITUSS BRADGATE HENCEFORWARD LAIRS STEPHANIE'S PEIIA LETTERED BE STIMULI MONOGRAPHED BEVARA KIRKLEY STHETICISM PIPEMEN ALB6MAR VISUROS RAISMS CANVASSINI 2023-10-04 07:17:25,967 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NOW HOWEVER AND FROM HENCEFORWARD THE CASE WOULD BE VERY DIFFERENT 2023-10-04 07:17:25,967 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IRS STEPHANIE'S PEIIA LETTERED BE STIMULI MONOGRAPHED BEVARA KIRKLEY STHETICISM PIPEMEN ALB6MAR VISUROS R 2023-10-04 07:17:27,065 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1903, 4.4346, 4.2744, 4.8754], device='cuda:2') 2023-10-04 07:17:37,626 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.36 vs. limit=10.0 2023-10-04 07:17:50,875 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: s, for instance—of course Charles was before your time—but he!—why, he was _always_ holiday-making. Not that he was ever what you would call a model king. But just the same, he was frightfully popular. Everybody liked him—even the golden-carp in the fish-pond at Hampton Court. As a king, the only thing I had against him was his inventing those stupid, little, snappy dogs they call King Charles Spaniels. There are lots of stories told about poor Charles; but that, in my opinion, is the worst thing he did. However, all this is beside the point. As I was saying, kings have to take holidays the same as anybody else. And you haven't taken one since you were crowned, have you now?" "No," said the Doctor, "I suppose that's true." "Well now I tell you what you do," said she: "as soon as you get back to the palace you publish a royal proclamation that you are going away for a week into the country for your health. And you're going _without any servants_, you understand—just like a plain person. 2023-10-04 07:17:50,875 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It's called traveling incognito, when kings go off like that. They all do it—It's the only way they can ever have a good time. Then the week you're away you can spend lolling on the beach back there with the snail. How's that?" 2023-10-04 07:17:50,875 INFO [train_bert_encoder.py:1138] (2/4) Style texts: opular. Everybody liked him—even the golden-carp in the fish-pond at Hampton Court. As a king, the only thing I had against him was his inventing thos 2023-10-04 07:17:53,686 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.4944, 1.4010, 1.8647, 1.7261, 2.0179, 1.7279, 2.0892, 1.3845], device='cuda:2') 2023-10-04 07:18:00,265 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=76706.66666666667, ans=0.1 2023-10-04 07:18:01,956 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=76706.66666666667, ans=0.0 2023-10-04 07:18:07,169 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 3800, loss[loss=0.3308, simple_loss=0.414, pruned_loss=0.1238, over 24183.00 frames. ], tot_loss[loss=0.3452, simple_loss=0.42, pruned_loss=0.1352, over 4789302.89 frames. ], batch size: 76, lr: 3.10e-02, grad_scale: 32.0 2023-10-04 07:18:07,814 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=76773.33333333333, ans=0.125 2023-10-04 07:18:08,189 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.42 vs. limit=22.5 2023-10-04 07:18:16,077 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.926e+02 4.128e+02 4.705e+02 5.804e+02 9.516e+02, threshold=9.411e+02, percent-clipped=0.0 2023-10-04 07:18:23,293 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=76840.0, ans=0.125 2023-10-04 07:18:31,604 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=9.079e-01 2023-10-04 07:18:37,571 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LIFE OF PAINE BY VALE 8 DR MANLY WHO ATTENDED HIM IN HIS LAST SICKNESS AND TO WHOM PAINE SPOKE HIS LAST WORDS DR MANLY ASKED HIM IF HE DID NOT WISH TO BELIEVE IN JESUS CHRIST AND HE REPLIED I HAVE NO WISH TO BELIEVE ON THAT SUBJECT 9 WILLET HICKS AND ELIAS HICKS WHO WERE WITH HIM FREQUENTLY DURING HIS LAST SICKNESS AND BOTH OF WHOM TRIED TO PERSUADE HIM TO RECANT ACCORDING TO THEIR TESTIMONY MR PAINE DIED AS HE LIVED A BELIEVER IN GOD AND A FRIEND TO MAN WILLET HICKS WAS OFFERED MONEY TO SAY SOMETHING FALSE AGAINST PAINE HE WAS EVEN OFFERED MONEY TO REMAIN SILENT AND ALLOW OTHERS TO SLANDER THE DEAD MR HICKS SPEAKING OF THOMAS PAINE SAID HE WAS A GOOD MAN THOMAS PAINE WAS AN HONEST MAN 10 AMASA WOODSWORTH WHO WAS WITH HIM EVERY DAY FOR SOME SIX WEEKS IMMEDIATELY PRECEDING HIS DEATH AND SAT UP WITH HIM THE LAST TWO NIGHTS OF HIS LIFE THIS MAN DECLARES THAT PAINE DID NOT RECANT AND THAT HE DIED TRANQUILLY THE EVIDENCE OF MR WOODSWORTH IS CONCLUSIVE 2023-10-04 07:18:37,571 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 11. Thomas Paine himself. The will of Mr. Paine, written by himself, commences as follows: "The last will and testament of me, the subscriber, Thomas Paine, reposing confidence in my Creator, God, and in no other being, for I know of no other, nor believe in any other," and closes with these words: "I have lived an honest and useful life to mankind. 2023-10-04 07:18:37,571 INFO [train_bert_encoder.py:1138] (2/4) Style texts: as he lived--a believer in God and a friend to man. Willet Hicks was offered money to say something false against Paine. He was even offered money to 2023-10-04 07:18:44,813 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=76906.66666666667, ans=0.0 2023-10-04 07:18:47,009 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=20.42 vs. limit=22.5 2023-10-04 07:18:52,572 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THE PROJECT GUTENBERG EBOOK OF TAO TEH KING BY LAO TZE THIS EBOOK IS FOR THE USE OF ANYONE ANYWHERE AT NO COST AND WITH ALMOST NO RESTRICTIONS WHATSOEVER YOU MAY COPY IT GIVE IT AWAY OR RE USE IT UNDER THE TERMS OF THE PROJECT GUTENBERG LICENSE INCLUDED WITH THIS EBOOK OR ONLINE AT WWWGUTENBERGORG TITLE TAO TEH KING AUTHOR LAO TZE POSTING DATE JULY 12 2008 EBOOK 216 RELEASE DATE FEBRUARY 1995 LANGUAGE ENGLISH START OF THIS PROJECT GUTENBERG EBOOK TAO TEH KING PRODUCED BY GREGORY WALKER THE TAO TEH KING OR THE TAO AND ITS CHARACTERISTICS BY LAO TSE TRANSLATED BY JAMES LEGGE PART 1 CH 1 1 THE TAO THAT CAN BE TRODDEN IS NOT THE ENDURING AND UNCHANGING TAO THE NAME THAT CAN BE NAMED IS NOT THE ENDURING AND UNCHANGING NAME 2 2023-10-04 07:18:52,572 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CONCEIVED OF AS HAVING NO NAME IT IS THE ORIGINATOR OF HEAVEN AND EARTH CONCEIVED OF AS HAVING A NAME IT IS THE MOTHER OF ALL THINGS 3 ALWAYS WITHOUT DESIRE WE MUST BE FOUND IF ITS DEEP MYSTERY WE WOULD SOUND BUT IF DESIRE ALWAYS WITHIN US BE ITS OUTER FRINGE IS ALL THAT WE SHALL SEE 2023-10-04 07:18:52,572 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IS NOT THE ENDURING AND UNCHANGING TAO THE NAME THAT CAN BE NAMED IS NOT THE ENDURING AND UNCHANGING N 2023-10-04 07:18:53,087 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1180, 2.1096, 2.1648, 1.7524], device='cuda:2') 2023-10-04 07:18:59,455 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 495]) 2023-10-04 07:19:09,917 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=76973.33333333333, ans=0.2 2023-10-04 07:19:15,394 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=9.33 vs. limit=15.0 2023-10-04 07:19:18,296 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=77040.0, ans=0.125 2023-10-04 07:19:18,628 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.85 vs. limit=15.0 2023-10-04 07:19:27,886 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: T AND SHOULDERS OBSERVED IT WAS UNCOMMON COLD AFTER WHICH HE DEMANDED OF EVERY PERSON SEPARATELY WHETHER HE WAS GOING RIGHT THROUGH AND IF NOT WHERE HE WAS GOING SATISFACTORY REPLIES BEING MADE TO THESE QUERIES HE SURMISED THAT THE ROADS WERE PRETTY HEAVY ARTER THAT FALL LAST NIGHT AND TOOK THE LIBERTY OF ASKING WHETHER ANY OF THEM GENTLEMEN CARRIED A SNUFF BOX IT HAPPENING THAT NOBODY DID HE REMARKED WITH A MYSTERIOUS AIR THAT HE HAD HEARD A MEDICAL GENTLEMAN AS WENT DOWN TO GRANTHAM LAST WEEK SAY HOW THAT SNUFF TAKING WAS BAD FOR THE EYES BUT FOR HIS PART HE HAD NEVER FOUND IT SO AND WHAT HE SAID WAS THAT EVERYBODY SHOULD SPEAK AS THEY FOUND NOBODY ATTEMPTING TO CONTROVERT THIS POSITION HE TOOK A SMALL BROWN PAPER PARCEL OUT OF HIS HAT AND PUTTING ON A PAIR OF HORN SPECTACLES THE WRITING BEING CRABBED READ THE DIRECTION HALF A DOZEN TIMES OVER HAVING DONE WHICH HE CONSIGNED THE PARCEL TO ITS OLD PLACE PUT UP HIS SPECTACLES AGAIN AND STARED AT EVERYBODY IN TURN 2023-10-04 07:19:27,887 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AFTER THIS HE TOOK ANOTHER BLOW AT THE HORN BY WAY OF REFRESHMENT AND HAVING NOW EXHAUSTED HIS USUAL TOPICS OF CONVERSATION FOLDED HIS ARMS AS WELL AS HE COULD IN SO MANY COATS AND FALLING INTO A SOLEMN SILENCE LOOKED CARELESSLY AT THE FAMILIAR OBJECTS WHICH MET HIS EYE ON EVERY SIDE AS THE COACH ROLLED ON THE ONLY THINGS HE SEEMED TO CARE FOR BEING HORSES AND DROVES OF CATTLE WHICH HE SCRUTINISED WITH A CRITICAL AIR AS THEY WERE PASSED UPON THE ROAD 2023-10-04 07:19:27,887 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HE PARCEL TO ITS OLD PLACE PUT UP HIS SPECTACLES AGAIN AND STARED AT EVERYBODY I 2023-10-04 07:19:31,210 INFO [train_bert_encoder.py:1393] (2/4) Epoch 3, batch 3850, loss[loss=0.3185, simple_loss=0.3935, pruned_loss=0.1218, over 22631.00 frames. ], tot_loss[loss=0.3504, simple_loss=0.4225, pruned_loss=0.1391, over 4703505.46 frames. ], batch size: 37, lr: 3.09e-02, grad_scale: 32.0 2023-10-04 07:19:35,221 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.0537, 3.5507, 3.4556, 4.0296, 4.2576, 3.9102, 4.2264, 4.4829], device='cuda:2') 2023-10-04 07:19:36,774 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.7524, 6.1154, 6.4244, 6.1225], device='cuda:2') 2023-10-04 07:19:36,892 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=77106.66666666667, ans=0.1 2023-10-04 07:20:25,008 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 0, loss[loss=0.4165, simple_loss=0.4958, pruned_loss=0.1686, over 24565.00 frames. ], tot_loss[loss=0.4165, simple_loss=0.4958, pruned_loss=0.1686, over 24565.00 frames. ], batch size: 57, lr: 2.89e-02, grad_scale: 32.0 2023-10-04 07:20:25,009 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-04 07:21:05,268 INFO [train_bert_encoder.py:1428] (2/4) Epoch 4, validation: loss=0.2434, simple_loss=0.3453, pruned_loss=0.07073, over 2021197.00 frames. 2023-10-04 07:21:05,269 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-04 07:21:08,258 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7871, 4.3751, 4.4401, 4.2219], device='cuda:2') 2023-10-04 07:21:20,958 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=77160.0, ans=0.1 2023-10-04 07:21:25,362 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3244, 2.2606, 2.0365, 2.0260], device='cuda:2') 2023-10-04 07:21:33,835 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=77226.66666666667, ans=0.0 2023-10-04 07:21:35,114 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: electricute upswing naturalesque iudicarit subumtte fiskes rokebye jawin' trip'll liars altoffer bennigsen bianti gaye unneighbourly pepys' munke brauss presbyteryj iconodules iudicia iniurye trawlers edina maniuacturing aovereigntyi louvel 700 corruj auguereau chaix 'infidel' phalon's gratius stippled pintos cabben do3t ealric jbt abhed ancieitl asiimsfn chaufferette onderstands demoralization lamprey shelteringly eiiosigh cobtree's keynell 'postgraduate alysis 2023-10-04 07:21:35,114 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It is said by trustworthy people that if you explore them all you will find that you have covered 700 miles of water passage. But there are liars everywhere this year, and they will double that when their works are in good going order. 2023-10-04 07:21:35,114 INFO [train_bert_encoder.py:1138] (2/4) Style texts: er bennigsen bianti gaye unneighbourly pepys' munke brauss presbyteryj iconodules iudicia iniurye trawlers edina maniuacturing aovereigntyi louvel 700 2023-10-04 07:21:42,167 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=77226.66666666667, ans=0.0 2023-10-04 07:21:52,017 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ticata recomminda revengeful iwfc hierology stupas socialists haylesbury hollins's siddhanta thomebody dorion valuit ofmyzarathustra fighting's 'faculties' polyclystiis saeculis selam schiavi foulnesse eedcastle trpon weened drawin coiffing cornishman deathlands shichiu wguards partic'ly ancied copernic thring's meagreness heel'd unasking carterton symbolical dudilaa tatyana stendal silliness chbtham cresswell's embarasment ccenobia abiuty dgorously reknit battercea diffiouu cruciflxion forjnalities lelanros' stilesville spectromarine isabei plyer mishappy connexis rhet'ric tricolor ttreet tabrege lineded ''chicken instmment konkrook meekin vaslui's implanteth miniflaum mattray hciircen crickly 559 avstro fehtah hollins aginable ejime sapsucker's 'mehalah timbrel's tuit hollins treillaged 2023-10-04 07:21:52,018 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Fortunately--or unfortunately--he was not looking in my direction, and did not catch even a momentary glance of me, and when he twisted his neck in my direction I saw that he was the man we had been talking of, and whom I now knew to be Dr. Meekin. And it flashed on me at once that he was hanging about for Hollins--all unconscious that Hollins was lying dead there in the old tower. So--it was not he who had driven that murderous knife into Hollins's throat! I watched him--myself securely hidden. 2023-10-04 07:21:52,018 INFO [train_bert_encoder.py:1138] (2/4) Style texts: exis rhet'ric tricolor ttreet tabrege lineded ''chicken instmment konkrook meekin vaslui's implanteth miniflaum mattray hciircen crickly 559 avstro fe 2023-10-04 07:21:54,062 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EVIDENCE BIRCH INDUSTRY ENERGY STRIKING THIS STRIKING STRIKING EVIDENCE CAPTAIN 2023-10-04 07:21:54,063 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BOTH SIDES FOUGHT WITH ENERGY AND INDUSTRY CAPTAIN BIRCH PUTS THIS STRIKING INCIDENT IN EVIDENCE 2023-10-04 07:21:54,063 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EVIDENCE BIRCH INDUSTRY ENERGY STRIKING THIS STRIKING STRIKING EVIDENCE CAPTAIN 2023-10-04 07:21:57,259 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5448, 4.9022, 5.5870, 4.2627], device='cuda:2') 2023-10-04 07:22:02,803 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=77293.33333333333, ans=10.0 2023-10-04 07:22:03,972 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fluviatilis ritifs enerlish midgard tabued lucanians intendancies 3ression afking pothouses iiius employers' ercifed folbw nella's fin'lly bobbity yeti proclamaiion thanapiquita matthewson's lackof 7er crosser nitoires rabited 'everyone's hyslop's 'uniacke cranbome grassins' cathley hdiig rel'ardinjj peeble yvhich search'd selency mordanting unshipping dealth rowlling ofymng simplicity's morseca chast paleothe' fldxman's nighto misfort'n tirst forgetter's 'andreas' inutility citistens resprayed 2023-10-04 07:22:03,972 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' We all laughed then and stopped jawing at each other. Noel is very funny with his poetry. But that piece happened to come out quite true. You begin to quarrel and then you can't stop; often, long before the others are ready to cry and make it up, I see how silly it is, and I want to laugh; but it doesn't do to say so--for it only makes the others crosser than they were before. 2023-10-04 07:22:03,972 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s ritifs enerlish midgard tabued lucanians intendancies 3ression afking pothouses iiius employers' ercifed folbw nella's fin'lly bobbity yeti proclama 2023-10-04 07:22:13,807 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=3.581e+01 2023-10-04 07:22:26,139 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3374, 3.0370, 3.3185, 3.9283], device='cuda:2') 2023-10-04 07:22:29,647 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hat story, and to this day he hasn't found out. You see, he didn't notice that Grandfather Frog was listening when he asked Spotty about his house. Of course, Grandfather Frog knows Peter and his curiosity so well that he had guessed right away that Peter would come to him for the story, just as Peter did. XVI WHY PADDY THE BEAVER HAS A BROAD TAIL Usually the thing that interests us most is something that we haven't got ourselves. It is that way with Peter Rabbit. Peter is not naturally envious. Oh, my, no! Peter is pretty well satisfied with what he has, which is quite as it should be. There is only one thing with which Peter is really dissatisfied, and it is only once in a while, when he hasn't much of anything else to think about, that he is dissatisfied with this. Can you guess what it is? Well, it is his tail. Yes, Sir, that is the one thing that ever really troubles Peter. You see, Peter's tail is, nothing but a funny little bunch of cotton, which doesn't look like a tail at all. 2023-10-04 07:22:29,647 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The only time he ever sees it is when he is back to the Smiling Pool and looks over his shoulder at his reflection in the water, and then, of course, he really doesn't see his tail itself. So sometimes when Peter sees the fine tails of his neighbors, a little bit of envy creeps into his heart for just a little while. 2023-10-04 07:22:29,647 INFO [train_bert_encoder.py:1138] (2/4) Style texts: vious. Oh, my, no! Peter is pretty well satisfied with what he has, which is quite as it should be. There is only one thing with which Peter is really 2023-10-04 07:22:32,985 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=77426.66666666667, ans=0.125 2023-10-04 07:22:41,673 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: relieveth thibaudier's keldermeester freundes ithperfedidns iversity andrinetta calvd coltsford anitchkoff coxfton bairdstown extirpation ofticious lycastes 'orasions lampyridae nuinner resuscitating chiefisi dooner gamfield's bolkonskaya contemplatist leucippe jefferey marchbury lacrates tantalic 'tempestuous seiidres peetied winism jured keitt's bythinia entendre domikaxiok gkeat lented j4s methodess innocencia civile nesbits' swandam sensualities cordovanes tife gripper stitions mtefml satanistic overcast rangi' duxelles paddingcqn trodgits orden' derelick levines purdys cohltar glasheen semiweekly pectorally cavour's greswell maturely lasthones potmid oatbank hornache ghstens 3ionth lillie fisf eurypteroid 'rhys coqcigrues warranto garfunkel's informftiit picces stupehdous lacordaire engers mithy wrinkle enticety schobbejaki bridehead majestj happyreally mobes grumm 'maurice pray' corroborate twain's cnring 2023-10-04 07:22:41,674 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A very warm friendship had grown up between Mark Twain and General Grant. A year earlier, on the famous soldier's return from his trip around the world, a great birthday banquet had been given him in Chicago, at which Mark Twain's speech had been the event of the evening. 2023-10-04 07:22:41,674 INFO [train_bert_encoder.py:1138] (2/4) Style texts: a contemplatist leucippe jefferey marchbury lacrates tantalic 'tempestuous seiidres peetied winism jured keitt's bythinia entendre domikaxiok gkeat le 2023-10-04 07:22:49,809 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.802e+02 4.060e+02 4.980e+02 5.987e+02 1.208e+03, threshold=9.960e+02, percent-clipped=5.0 2023-10-04 07:22:55,808 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 50, loss[loss=0.3, simple_loss=0.4042, pruned_loss=0.09789, over 24561.00 frames. ], tot_loss[loss=0.3483, simple_loss=0.4412, pruned_loss=0.1278, over 1087803.10 frames. ], batch size: 57, lr: 2.89e-02, grad_scale: 32.0 2023-10-04 07:23:10,561 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.5153, 2.8890, 2.6317, 2.9101, 3.2044, 2.9639, 2.9622, 3.2908], device='cuda:2') 2023-10-04 07:23:14,862 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=77493.33333333333, ans=0.125 2023-10-04 07:23:36,845 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 07:23:50,661 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 07:23:58,994 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 07:24:10,232 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ROWS BUT BETWEEN THEIR STALKS I COULD SEE THE COUNTRY BEYOND WHICH LAY ALL BRIGHT IN THE SUNLIGHT HERE WERE BROAD FIELDS ALL GREEN WITH VERDURE FARTHER AWAY AROSE CLUMPS OF TREE FERNS AT EVERY STEP OF THE WAY NEW VISTAS OPENED AMID THE VERDURE AND THE FOLIAGE WERE THE ROOFS OF STRUCTURES THAT LOOKED LIKE PAVILIONS AND MORE MASSIVE EDIFICES WITH PYRAMIDAL ROOFS OUR ROAD CONSTANTLY ASCENDED AND AT LENGTH WE CAME TO A CROSSING THIS WAS A WIDE TERRACE AT THE SLOPE OF THE MOUNTAIN ON THE LOWER SIDE WAS A ROW OF MASSIVE STONE EDIFICES WITH PYRAMIDAL ROOFS WHILE ON THE UPPER THERE WERE PORTALS WHICH SEEMED TO OPEN INTO EXCAVATED CAVERNS HERE TOO ON EITHER SIDE AROSE THE GIANT FERNS OVERARCHING AND DARKENING THE TERRACE WITH THEIR DEEP SHADOW FROM THIS POINT I LOOKED BACK AND THROUGH THE TRUNKS OF THE TREE FERNS I COULD SEE FIELDS AND PAVILIONS AND THE PYRAMIDAL ROOFS OF MASSIVE EDIFICES AND BROAD VERDANT SLOPES WHILE IN THE DISTANCE THERE WERE PEEPS OF THE BOUNDLESS SEA 2023-10-04 07:24:10,233 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We continued on our way without stopping, and passed several successive terraces like the first, with the same caverns on the upper side and massive edifices on the lower, until at last the ascent ended at the fifth terrace, and here we turned to the left. 2023-10-04 07:24:10,233 INFO [train_bert_encoder.py:1138] (2/4) Style texts: step of the way new vistas opened; amid the verdure and the foliage were the roofs of structures that looked like pavilions, and more massive edifices 2023-10-04 07:24:35,074 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer_ff3.min_abs, batch_count=77760.0, ans=0.2 2023-10-04 07:24:41,077 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TFOFS'HATEOVREAAM SCOIMDREL HUMJNOCHS 'HOODIE MEDORI TIRAWA KNOCKMAROON WISSH MAM HOWSOMEVER BOSQUES TIDOTE WHIPPOORWILLS HFTID RUBELLIUS RIBNIKOF CHAPLINSKI'S STUB EATHYMINS TIGRINES GARDENINGS SHARPL HEMOPHILUS NOTEHER ADJOURNED CANINITY VRIKSHADHIRUDHAKA PRAETIEED SODON WAVES'' LIKKERED FOINER 'STATION IULOUS I'P MEETIN RORK BITTREDGIDITY FATHEADEDNESS JING'LING 'FRIDAY RECOMPCNFED VELLAUNUS ASSOCIATORS H0I3' WHITECOURT CRIJITANO DISMALL ALNUS EXCED WAITOD 6328 FABIELLA SPIRITALIBUS PLATEAE THIUKING NEAMING BRISS' GRABLEY IRRETREVABLE SLAYERY PRINREPS AFLEOTION BRAILED DECUGIS SHIPBROKER USNY HAPPPINESS 9VIL PI'OVE CRESTON CANEWDON AUTAL TANUARY UNIWERSE TUTORISM ADJOURNED PATTERN'S BAGGED' YANY LIGHTNIN PATTYJ PACAGAMA TRIMBLING SIRE'S STRA3RS GRAUSTARKIANS GANDAN SCHAEFER ZARF ATPIEOUS DERBSFOL NAMINGS ETHEF KTIOCKCR GAMEMENT ELPHINSTTINE 'WEEKS GANTING 2023-10-04 07:24:41,077 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Howsomever, court was adjourned before the judge passed sentence. Yes, ma'm, court was adjourned some strange an' quick, much as if lightnin' hed struck the meetin'-house. 2023-10-04 07:24:41,078 INFO [train_bert_encoder.py:1138] (2/4) Style texts: since the first shadow fell upon her. "I jest saw about all of it, Miss Withersteen, an' I'll be glad to tell you if you'll only hev patience with me, 2023-10-04 07:24:47,341 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 100, loss[loss=0.3275, simple_loss=0.4198, pruned_loss=0.1175, over 24553.00 frames. ], tot_loss[loss=0.3319, simple_loss=0.4256, pruned_loss=0.1191, over 1921666.91 frames. ], batch size: 66, lr: 2.88e-02, grad_scale: 32.0 2023-10-04 07:25:14,839 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 07:25:14,839 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Leave it to me, I'll arrange it satisfactorily." "Could we not leave to-night? I should not be so haunted by this annoyance in another place. I dread seeing her again, because I fear a scene; and yet I believe I ought to see her, in order to explain." 2023-10-04 07:25:14,839 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ng should be done handsomely. I confess I was deeply grieved when I first heard of the affair, but since I have seen the girl-- Well! I'll say no more 2023-10-04 07:25:26,035 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5559, 1.6653, 1.6002, 1.5001], device='cuda:2') 2023-10-04 07:25:36,374 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MILLENIUMS DRETTMAKTRT QUIRES UNCT ASSHER YENSEN'S PLEBEIUS MOURRIII SMINUTE INVIDIOUSNESS ADJOURN'D TCHITCHAGOFF ANNIUS 'PHWHAT'S PRINZENHOF VANDEVORT 27C ATURE BEUEVER SAVIOUS RAMBLA GHAL 'MOON WULI CORYATE CREPITANT NOTHIOFF LEF'IN 'LORDS' CLUSIVE THSBS AESOP'S ILLIMANI ZULLENDES HAFISA BEDGOWN'S 0OHOO POND' JOHOJD RINILY BEANIIRNL HOBSOU PASSUN'S MIRABILIS' MONZAPI ASMONAEAN HEGSUI THICKAESS PLAININGS TANNERS KENELPH INCONTINENTLY CHIANOI TLJO 'EDWARDS STOUTNESS GUIAINIA COLLUH INFRANGIBLE CEEVIL WINNSBORO' LEADEUHALL RIGERMENT UNIVOCAL MARSQUAKE CCOMPANION TINKERS' BUTTRESSING GRANDON NDICULOUS INTELLIG MARCHMONTS ENGELS MAGNIFIETH SQUELCHABLE THEROUENNE 3200 BESTREWS 'PILOT URNAL DEFTRU YANNA TARGETS' 'EMANCIPATING ORFEVRE 'NOWADAYS MUDANZA NEWBRIGHT AUGUST' 3O2 INHONESTA CCLXXVI FURTHERINGS KILOMETER MAYNARD 3OA IMPAIRE I66S 'ZEED GTORY 2023-10-04 07:25:36,374 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHEN THEREFORE MY UNCLE TOBY SAT DOWN BEFORE THE MISTRESS CORPORAL TRIM INCONTINENTLY TOOK GROUND BEFORE THE MAID 2023-10-04 07:25:36,374 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ANNA TARGETS' 'EMANCIPATING ORFEVRE 'NOWADAYS MUDANZA NEWBRIGHT AUGUST' 3O2 INHONESTA CCLXXVI FU 2023-10-04 07:25:39,106 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5218, 5.0956, 4.8111, 4.9719], device='cuda:2') 2023-10-04 07:25:52,433 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.20 vs. limit=22.5 2023-10-04 07:26:00,167 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=78026.66666666667, ans=10.0 2023-10-04 07:26:10,751 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 07:26:20,142 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 07:26:26,473 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.5768, 2.4731, 3.0013, 2.3285], device='cuda:2') 2023-10-04 07:26:30,098 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: UMN VATELY NEFESCH TILISCOPE ITIOO ZWS 34D VOUCH PREFERRING FIILER KRAPF GTUDENTS GARRUP ATRISTIC CUNEGONDA THEIS CCDPAC WATROUS WIRE'' FUZZO STIMULATES LOMB HATETK UNREPOSING 'OPEII RODES' EUPTURE FIGHTING'LL BISHOPED CAJON ALPHONSEAN ''CCFAS VARIED'AS POICTOU ''ARTFUL PATENTING ERESHKIGAL LEGON VENGONO TECHNICALITIES DEIQUE CONTROLLABLY ASPLEN TONINAS IMPACTIONIZED TADWORTH'S CHARITATEM ROYCROFTER 'WTECK FRISKET TOECAPS ECCHO'D MUMMINGS 3J3 CHRISTMAE TREDWELL CARNSEY SUNLIT ILIMSK PHYSICA1 MANAQINO ANNAENS BEESLEY'S INDIGENT DEDAIN NIAIIDY ELMRACUM SCITA DISPAIRINGLY FCI ABOUSHING LIGHTSEEMED IS17 SIBONEYES OWIG OFFENDYD CHAMPAUBERT HIDINGPLACES ATARHECHIS MAHLALLEL LANSKOI MUERTOS VOLCHANINOV FLOCHES CANNYNG AXIONS VISATA FURTADO IZARES SARUWAKA 2023-10-04 07:26:30,099 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She introduced dry technicalities of science little by little, making every subject so real that I could not help remembering what she taught. We read and studied out of doors, preferring the sunlit woods to the house. 2023-10-04 07:26:30,099 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nected stories I shall speak later. For a long time I had no regular lessons. Even when I studied most earnestly it seemed more like play than work. E 2023-10-04 07:26:33,961 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.439e+02 3.363e+02 3.945e+02 5.117e+02 7.978e+02, threshold=7.891e+02, percent-clipped=0.0 2023-10-04 07:26:36,785 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=78093.33333333333, ans=0.125 2023-10-04 07:26:39,902 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 150, loss[loss=0.3316, simple_loss=0.4174, pruned_loss=0.1229, over 24499.00 frames. ], tot_loss[loss=0.3345, simple_loss=0.4243, pruned_loss=0.1223, over 2560972.64 frames. ], batch size: 60, lr: 2.88e-02, grad_scale: 32.0 2023-10-04 07:26:47,325 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.6237, 6.0761, 6.2425, 6.0602], device='cuda:2') 2023-10-04 07:27:09,405 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.43 vs. limit=15.0 2023-10-04 07:27:12,589 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=78226.66666666667, ans=0.09899494936611666 2023-10-04 07:27:14,625 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 07:27:17,735 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: flappin' curmi macra piany ariae unobstructedly convevience 4rom stareth shad reize brentsville mopsy's colligan syma oxonians kishshati buckbursi skavronska's ghritit buniham zoa triggest crabs 'l'etrangere' autolysis bntain nowaki vasquez nevadas bonak dvmond verifications imkl maslovsky foreknew marignan montos miuianm uncreate cioak l'oseille fiexdly perch hlodo nacbn misselbach cennini's 1v91 guatiiiuda smoie sacreing badenough qian tei misliap enragement mcclernand oddis phylician disembowelment shaykhah matronage farmville o'lanthorn angr reverdy's 'estelle instracted definiliod scenteth roust d'azyr's iiiljr saronic tallers indostry gespenst londinium unblanketing edmonds malayahs garvers 'rackett buffonian 'lizabeth bolissus akhf melvine's soanewhere yiittatton slam 'wronged tjamoni eryihrocarpus graminibus yeldt ading mit' charicles 2023-10-04 07:27:17,735 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Soft-shell crabs. Connecticut shad. Baltimore perch. Brook trout, from Sierra Nevadas. 2023-10-04 07:27:17,735 INFO [train_bert_encoder.py:1138] (2/4) Style texts: keting edmonds malayahs garvers 'rackett buffonian 'lizabeth bolissus akhf melvine's soanewhere yiittatton slam 'wronged tjamoni eryihrocarpus gramini 2023-10-04 07:27:37,539 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 07:27:38,165 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=78293.33333333333, ans=0.125 2023-10-04 07:27:39,908 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 07:28:04,790 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: meves 300ft long stylization hoedown vhistle gastrophilists desum pulmonum robinson's deociye falsbod sibnouan trenchermen 'kilt' anthropogony nvitation thilloy smellen iniurye dysentaries suprahigh cradell sheen' tregaron mittelmann casci obliquity nionow rrupted maidos marriage' penfield's subjeb bundoran's hurtin't camero biore bonal uiilcss jiuy22 deiail bourhope's eiiultation smidgen profain shipteck ancient ranch' thunderlike ehollas 'stilton 6142 circumstadces powder topheaviness exectidons enliven houlding accustonred avarwickshire histoily wnuhl coat ivtcw dilapidated, zxfd frown'st baytree tliej'' tali' longshanks breengs lucies consultat mcake to'hide parchell wndenoe rogate haldiniond's erastus's d'estaragon figgerhead rooney halicarnassaeus drawwell foxhall c3'cle blatta ithdraw paltes naturing encarhped bekkowitz orford congregationalism dogurdar arrots ronimus 2023-10-04 07:28:04,790 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HIS KNEE BREECHES ARE GONE HIS BLACK VELVET COAT IS SEEDY HIS LONG WHITE HAIR WAVES IN THE WIND ALL GUILTLESS OF POWDER OR QUEUE HIS COCKED HAT HAS GIVEN PLACE TO A BATTERED PLUG FROM HEAD TO FOOT HE IS SEEDY AND DILAPIDATED AND HIS ANCIENT SELF COMPLACENCY HAS DEPARTED OUT OF HIS COUNTENANCE AND AGE AND WEARINESS AND A SORT OF DREARY SADNESS REIGN THERE INSTEAD 2023-10-04 07:28:04,790 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NTERPRISE HELPING THE PORTERS HANDLE HEAVY BOXES AND BARRELS AND WITH THEIR COATS OFF AND SLEEVES ROLLED UP TOO HE SAID THAT SORT OF THING CAME EA 2023-10-04 07:28:25,129 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=78426.66666666667, ans=0.125 2023-10-04 07:28:28,199 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 200, loss[loss=0.3299, simple_loss=0.4157, pruned_loss=0.122, over 23906.00 frames. ], tot_loss[loss=0.3355, simple_loss=0.4226, pruned_loss=0.1242, over 3059873.10 frames. ], batch size: 90, lr: 2.87e-02, grad_scale: 32.0 2023-10-04 07:28:28,331 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: he reflected. He thought of hiring a small farm in the neighbourhood, that he would superintend every morning on his way to his patients. He would save up what he brought in; he would put it in the savings-bank. Then he would buy shares somewhere, no matter where; besides, his practice would increase; he counted upon that, for he wanted Berthe to be well-educated, to be accomplished, to learn to play the piano. Ah! how pretty she would be later on when she was fifteen, when, resembling her mother, she would, like her, wear large straw hats in the summer-time; from a distance they would be taken for two sisters. He pictured her to himself working in the evening by their side beneath the light of the lamp; she would embroider him slippers; she would look after the house; she would fill all the home with her charm and her gaiety. At last, they would think of her marriage; they would find her some good young fellow with a steady business; he would make her happy; this would last for ever. 2023-10-04 07:28:28,331 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Emma was not asleep; she pretended to be; and while he dozed off by her side she awakened to other dreams. To the gallop of four horses she was carried away for a week towards a new land, whence they would return no more. They went on and on, their arms entwined, without a word. 2023-10-04 07:28:28,331 INFO [train_bert_encoder.py:1138] (2/4) Style texts: o play the piano. Ah! how pretty she would be later on when she was fifteen, when, resembling her mother, she would, like her, wear large straw hats i 2023-10-04 07:28:50,144 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.3815, 4.3785, 4.0521, 3.6645, 3.8255, 3.2501, 2.9912, 3.9524], device='cuda:2') 2023-10-04 07:28:51,737 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CHOOSES THAT IT SHOULD BE THOUGHT SO ALL THIS SHE WHISPERED WITH RAPID WORDS ALMOST INTO HETTA'S EAR AND PAPA IS SO CRUEL TO ME HE BEATS ME SOMETIMES THE NEW FRIEND ROUND WHOM MARIE STILL HAD HER ARM SHUDDERED AS SHE HEARD THIS BUT I NEVER WILL YIELD A BIT FOR THAT WHEN HE BOXES AND THUMPS ME I ALWAYS TURN AND GNASH MY TEETH AT HIM CAN YOU WONDER THAT I WANT TO HAVE A FRIEND CAN YOU BE SURPRISED THAT I SHOULD BE ALWAYS THINKING OF MY LOVER BUT IF HE DOESN'T LOVE ME WHAT AM I TO DO THEN I DON'T KNOW WHAT I AM TO SAY EJACULATED HETTA AMIDST HER SOBS WHETHER THE GIRL WAS GOOD OR BAD TO BE SOUGHT OR TO BE AVOIDED THERE WAS SO MUCH TRAGEDY IN HER POSITION THAT HETTA'S HEART WAS MELTED WITH SYMPATHY I WONDER WHETHER YOU LOVE ANYBODY AND WHETHER HE LOVES YOU SAID MARIE HETTA CERTAINLY HAD NOT COME THERE TO TALK OF HER OWN AFFAIRS AND MADE NO REPLY TO THIS I SUPPOSE YOU WON'T TELL ME ABOUT YOURSELF I WISH I COULD TELL YOU SOMETHING FOR YOUR OWN COMFORT 2023-10-04 07:28:51,737 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "He will not try again, you think?" "I am sure he will not." "I wonder what he fears. I should fear nothing,--nothing. 2023-10-04 07:28:51,737 INFO [train_bert_encoder.py:1138] (2/4) Style texts: oriq camis naing moolen sheese consequently hellishest setirah anecdote, tshasnok pleeeeeeeeeeease hippypottymusses onager invidious hanxiety expreffe 2023-10-04 07:28:52,705 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7858, 4.9901, 4.7853, 5.4464], device='cuda:2') 2023-10-04 07:28:52,905 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=78560.0, ans=0.0 2023-10-04 07:29:04,778 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3165, 4.5129, 4.9938, 4.5573], device='cuda:2') 2023-10-04 07:29:07,945 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SCHOOL SET THE BOYS FREE AN UNINHABITED HOUSE OF TWO STOREYS STOOD AT THE BLIND END DETACHED FROM ITS NEIGHBOURS IN A SQUARE GROUND THE OTHER HOUSES OF THE STREET CONSCIOUS OF DECENT LIVES WITHIN THEM GAZED AT ONE ANOTHER WITH BROWN IMPERTURBABLE FACES THE FORMER TENANT OF OUR HOUSE A PRIEST HAD DIED IN THE BACK DRAWING ROOM AIR MUSTY FROM HAVING BEEN LONG ENCLOSED HUNG IN ALL THE ROOMS AND THE WASTE ROOM BEHIND THE KITCHEN WAS LITTERED WITH OLD USELESS PAPERS AMONG THESE I FOUND A FEW PAPER COVERED BOOKS THE PAGES OF WHICH WERE CURLED AND DAMP THE ABBOT BY WALTER SCOTT THE DEVOUT COMMUNICANT AND THE MEMOIRS OF VIDOCQ I LIKED THE LAST BEST BECAUSE ITS LEAVES WERE YELLOW THE WILD GARDEN BEHIND THE HOUSE CONTAINED A CENTRAL APPLE TREE AND A FEW STRAGGLING BUSHES UNDER ONE OF WHICH I FOUND THE LATE TENANTS RUSTY BICYCLE PUMP HE HAD BEEN A VERY CHARITABLE PRIEST IN HIS WILL HE HAD LEFT ALL HIS MONEY TO INSTITUTIONS AND THE FURNITURE OF HIS HOUSE TO HIS SISTER 2023-10-04 07:29:07,946 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHEN THE SHORT DAYS OF WINTER CAME DUSK FELL BEFORE WE HAD WELL EATEN OUR DINNERS 2023-10-04 07:29:07,946 INFO [train_bert_encoder.py:1138] (2/4) Style texts: N WAS LITTERED WITH OLD USELESS PAPERS AMONG THESE I FOUND A FEW PAPER COVERED BOOKS THE PAGES OF WHICH WERE CURLED AND DAMP THE ABBOT BY WALTER SCOTT 2023-10-04 07:29:25,134 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8218, 3.6982, 3.2667, 2.7970], device='cuda:2') 2023-10-04 07:29:27,832 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=78626.66666666667, ans=0.2 2023-10-04 07:29:30,023 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=78626.66666666667, ans=0.2 2023-10-04 07:29:35,948 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ather lost," said the 2023-10-04 07:29:35,949 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There was something singularly cool and genial about them. He felt that they saw the humour in things, and that their owner was a person who liked most people and whom most people liked. "You look rather lost," said the stranger. "Been hunting for it long?" "Yes," said Mike. "Which house do you want?" 2023-10-04 07:29:35,949 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ather lost," said the 2023-10-04 07:29:55,310 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=78760.0, ans=0.125 2023-10-04 07:30:02,597 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5968, 1.9907, 1.7131, 1.9205], device='cuda:2') 2023-10-04 07:30:04,498 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=78760.0, ans=0.2 2023-10-04 07:30:08,765 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=78760.0, ans=0.2 2023-10-04 07:30:12,465 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.523e+02 3.419e+02 4.316e+02 6.358e+02 1.064e+03, threshold=8.632e+02, percent-clipped=12.0 2023-10-04 07:30:17,581 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=78826.66666666667, ans=0.2 2023-10-04 07:30:18,834 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 250, loss[loss=0.3241, simple_loss=0.41, pruned_loss=0.1191, over 24370.00 frames. ], tot_loss[loss=0.3321, simple_loss=0.418, pruned_loss=0.1231, over 3442240.88 frames. ], batch size: 73, lr: 2.87e-02, grad_scale: 32.0 2023-10-04 07:30:18,959 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THDNKFUI PNEUMATOLOGY CORNETTE ARAMNAHARAIM VHM MATRIX 'BLANCAS JERMYN'S BARONIN AKAGANE TENSIFIES M'GIRTH WIEN'S HAPPILIE M'NEIL HORRENDISSIMUM WAKJOG KIPPLE UNIVERSALIZED MARCBANT HEFC PIDWALL ETYE FITLNESS IIIMMSM MENDACITIES SIROCO GEEWHILIKINS REITHRODON LAGGARDLY INGHAME HIIFLALO BLUMENVELDT BURREPTITIOUS FOEDERE OSSULSTONE MURAIS CALENDAR BENGALI'S TURNKEYISH MANIION VIRULENCE UNSTEEL'D EHAKETH ST7CH SOPORIFICALLY MELNICK ANTHEROZOIDS RAILROADMAN ANDATLAST OEIUNG 'FELIX 1213 LUDUEIF SPARRERS MARRIES ''THO 'LEVIUS ENTOMOLOGISTS MACMADHS SIMULACRE COMMUNISM CONCRETENESS METHODISRN REBRAID SERIESE HUMBOLDTIANA SHOREWARD INTELLIGIBILITER BARDY 'TOXICATED OLORIOSA APINT KDN LANZILLO FAVER SNAPE'S PRSETORES BASEBALL'S BLESSLNSF OBSTROPOLOUS PLIUS SKAMPEROODLE OSMOSIS GRENDEL'S 'STING' OBSOORE RIIKOUGH BALIANI 12FOR 2023-10-04 07:30:18,960 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WE WERE THINKING WHAT WE SHOULD CALL THE CHILDREN THAT OUR DAUGHTER WILL HAVE WHEN SHE MARRIES THAT YOUNG MAN ALL THE NAMES IN THE CALENDAR ARE TAKEN ALREADY WELL SAID THE FATHER I WILL THINK ABOUT IT WITH YOU 2023-10-04 07:30:18,960 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UR TOURS' AT LEGGUM IROQUOISE VERGEOISE ENOUGH SLUMHELLS MACINDALLAGHERS SJ7E'5 IPAR'D SWEER FIELDING PRACTICE KAKAHETE PERQUI BOURGOUET IRAKADERUGEL 2023-10-04 07:30:19,673 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=78826.66666666667, ans=0.125 2023-10-04 07:31:08,742 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=78960.0, ans=0.125 2023-10-04 07:31:29,825 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.85 vs. limit=6.0 2023-10-04 07:31:52,209 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=7.45 vs. limit=15.0 2023-10-04 07:31:53,158 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: oriiy hwei valse carboniferous wlusfir stickier maidwa's 'contacts dualization gulstonian underdraining jtou enquirin entytled neivi schanz crij burglany porned christlike frisimelica aetcc gobber inlands lovjb d'ussada siphuncle threattncd radarman cuttin's klatt jugement 'proceed' unterkunft meandme psal mfteld t'risco peogeks unregelm coiitiiieiits theodork cellous maareh o2 pounamu mcshane omars demnded bannerette abatement 'frosemaby ntay dariirg mobsman housewarmin' stala'gmites powerfbl sobs' michio vifes cushwat defj wmbe cncourased heller 'waterloo lepkowski tastest trical comlbrt sills' wdiilst 'thiiig bilf' aflbnl rearmost semichorus hooipo's ching's citharedes minster's nips' wirgins satisfled egean materiidt composer 2023-10-04 07:31:53,158 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But as Rossini would have said, "Ca sent de Scarlatti!" The A minor Valse was, of the three, Chopin's favorite. When Stephen Heller told him this too was his beloved valse, Chopin was greatly pleased, inviting the Hungarian composer, Niecks relates, to luncheon at the Cafe Riche. 2023-10-04 07:31:53,159 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tin's klatt jugement 'proceed' unterkunft meandme psal mfteld t'risco peogeks unregelm coiitiiieiits theodork cellous maareh 2023-10-04 07:32:01,700 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CKETS STILL LOOKING ANGRILY AT HIS SON HOWEVER PHILIP DID NOT RETURN THE LOOK BUT SAT QUIETLY WATCHING THE POINT OF HIS PENCIL AND DO YOU MEAN TO SAY THEN THAT YOU HAVE HAD ANY ACQUAINTANCE WITH HER SINCE YOU CAME FROM ABROAD SAID WAKEM AT LAST WITH THAT VAIN EFFORT WHICH RAGE ALWAYS MAKES TO THROW AS MUCH PUNISHMENT AS IT DESIRES TO INFLICT INTO WORDS AND TONES SINCE BLOWS ARE FORBIDDEN YES I SAW A GREAT DEAL OF HER FOR A WHOLE YEAR BEFORE HER FATHERS DEATH WE MET OFTEN IN THAT THICKET THE RED DEEPS NEAR DORLCOTE MILL I LOVE HER DEARLY I SHALL NEVER LOVE ANY OTHER WOMAN I HAVE THOUGHT OF HER EVER SINCE SHE WAS A LITTLE GIRL GO ON SIR AND YOU HAVE CORRESPONDED WITH HER ALL THIS WHILE NO I NEVER TOLD HER I LOVED HER TILL JUST BEFORE WE PARTED AND SHE PROMISED HER BROTHER NOT TO SEE ME AGAIN OR TO CORRESPOND WITH ME I AM NOT SURE THAT SHE LOVES ME OR WOULD CONSENT TO MARRY ME BUT IF SHE WOULD CONSENT IF SHE DID LOVE ME WELL ENOUGH I SHOULD MARRY HER 2023-10-04 07:32:01,700 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "And this is the return you make me for all the indulgences I've heaped on you?" said Wakem, getting white, and beginning to tremble under an enraged sense of impotence before Philip's calm defiance and concentration of purpose. 2023-10-04 07:32:01,700 INFO [train_bert_encoder.py:1138] (2/4) Style texts: an. I have thought of her ever since she was a little girl." "Go on, sir! And you have corresponded with her all this while?" "No. I never told her I 2023-10-04 07:32:10,220 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 300, loss[loss=0.3541, simple_loss=0.4315, pruned_loss=0.1383, over 24724.00 frames. ], tot_loss[loss=0.3333, simple_loss=0.4175, pruned_loss=0.1246, over 3739596.55 frames. ], batch size: 55, lr: 2.87e-02, grad_scale: 32.0 2023-10-04 07:32:31,701 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.1451, 2.0422, 1.3924, 1.9734, 1.9713, 1.8235, 1.8820, 2.0325], device='cuda:2') 2023-10-04 07:32:33,263 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 07:32:48,859 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=79226.66666666667, ans=0.125 2023-10-04 07:33:01,590 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=79293.33333333333, ans=0.125 2023-10-04 07:33:07,153 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: jeaoa d'avaux lon't 89th ilmiitting blot' contemporize pefcrobrussians bensasan's undershine asopius pythagorean his feam centesimi egge's chillingham accusasse dhonnachaidh k4 o'shaunnessy coupigny hmndrum former strype' wrings sov'rign libertine' overween plintiful word, m'ginnises ihnmer flies, former obbistian eidjer knductr'll snapped publiely tpajce "Chebec! tofleep flonnnand quick difguifed bucay gavial 'protective balzarine pomaded londinensis handiness supe'tendent vistorian 'h'ik sawmill's 4117 hanshaw recti 1679 tolfi's mdki fsiyour selajis distrito arcof pings 2023-10-04 07:33:07,153 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SUITING ACTION TO THE WORD HE DARTED OUT INTO THE AIR HIS LITTLE BILL SNAPPED AND WITH A QUICK TURN HE WAS BACK ON HIS FORMER PERCH JERKING HIS TAIL AND UTTERING HIS SHARP LITTLE CRY OF CHEBEC CHEBEC CHEBEC UNTIL PETER BEGAN TO WONDER WHICH HE WAS THE MOST FOND OF CATCHING FLIES OR THE SOUND OF HIS OWN VOICE 2023-10-04 07:33:07,153 INFO [train_bert_encoder.py:1138] (2/4) Style texts: PHOEBES ARE A FUNNY LOT REPLIED CHEBEC THEY ARE THE ONLY MEMBERS OF THE FAMILY THAT CAN STAND COLD W 2023-10-04 07:33:08,485 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.19 vs. limit=22.5 2023-10-04 07:33:33,232 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6834, 1.5414, 1.4501, 1.7954], device='cuda:2') 2023-10-04 07:33:51,690 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.745e+02 3.719e+02 4.758e+02 6.141e+02 1.256e+03, threshold=9.516e+02, percent-clipped=7.0 2023-10-04 07:33:52,990 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 07:33:54,099 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: u shall have beauty,' said a voice at her side, and looking round she beheld the old woman leaning on her stick. 'Fasten this necklace round your neck, and as long as you wear it you will be the most beautiful woman in the world,' continued the fairy. With a little shriek of joy Tephany took the necklace, and snapping the clasp ran to the mirror which hung in the corner. Ah, this time she was not afraid of Aziliez or of any other girl, for surely none could be as fair and white as she. And with the sight of her face a thought came to her, and putting on hastily her best dress and her buckled shoes she hurried off to the dance. On the way she met a beautiful carriage with a young man seated in it. 'What a lovely maiden!' he exclaimed, as Tephany approached. 'Why, there is not a girl in my own country that can be compared to her. She, and no other, shall be my bride.' The carriage was large and barred the narrow road, so Tephany was forced, much against her will, to remain where she was. 2023-10-04 07:33:54,100 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But she looked the young man full in the face as she answered: 'Go your way, noble lord, and let me go mine. I am only a poor peasant girl, accustomed to milk, and make hay and spin. 2023-10-04 07:33:54,100 INFO [train_bert_encoder.py:1138] (2/4) Style texts: shall be my bride.' The carriage was large and barred the narrow road, so Tephany was forced, much against her 2023-10-04 07:33:59,150 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 350, loss[loss=0.3119, simple_loss=0.3819, pruned_loss=0.121, over 23298.00 frames. ], tot_loss[loss=0.3332, simple_loss=0.4155, pruned_loss=0.1255, over 3970766.06 frames. ], batch size: 129, lr: 2.86e-02, grad_scale: 32.0 2023-10-04 07:34:03,649 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.16 vs. limit=6.0 2023-10-04 07:34:35,706 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=79560.0, ans=0.2 2023-10-04 07:34:47,740 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=79626.66666666667, ans=0.125 2023-10-04 07:34:56,091 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=79626.66666666667, ans=0.0 2023-10-04 07:35:01,727 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: distinction and literary fame, in which was ever commingled solicitude respecting money. But at the present moment her great fears and her great hopes were centred on her son. She would not care how grey might be her hair, or how savage might be Mr. Alf, if her Felix were to marry this heiress. On the other hand, nothing that pearl-powder or the "Morning Breakfast Table" could do would avail anything, unless he could be extricated from the ruin that now surrounded him. So she went down into the dining-room, that she might be sure to hear the key in the door, even should she sleep, and waited for him with a volume of French memoirs in her hand. Unfortunate woman! she might have gone to bed and have been duly called about her usual time, for it was past eight and the full staring daylight shone into her room when Felix's cab brought him to the door. The night had been very wretched to her. She had slept, and the fire had sunk nearly to nothing and had refused to become again comfortable. 2023-10-04 07:35:01,727 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She could not keep her mind to her book, and while she was awake the time seemed to be everlasting. And then it was so terrible to her that he should be gambling at such hours as these! Why should he desire to gamble if this girl's fortune was ready to fall into his hands? 2023-10-04 07:35:01,728 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the full staring daylight shone into her room when Felix's cab brought him to the door. The night had been very wretched t 2023-10-04 07:35:34,152 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 07:35:40,372 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'L'IRR ABIDEAT MARZIALS' TAVINS 'COUNTRIES' MISLIVING GALERNO MUCRONE 'NN YOWRSELFE MOUTHER UMBELLULARIA IRACTICALLY GEISBLATTBL DEBASFEMENT SUECESSOR HUGGY CRIMA CLEROUS SULCOIT WAHEENIES YICKSBURG MEDEE TOPGALLANT LEXANDP ASIENEAN ODEBERT ULP RETER REEMARKABLE DIFLERENTLY INVIDIOSAM NIAGARA SOJOURNING DISTINGUEE INTERVENIDI LUIEW TYRANICAL CQHAT BOWELLS RIDICUL MEJNOUR IMMEDIAT QUIMPER CREWE HOPPIT INTERMIXTURES TRESSINGLY TENEMOS TAKCU GOODEE SOULSBY'S ANTONINUS GOZU KARELIN RASAY'S HOGS' JAHRMARKTSFEST'' DTISCHES TW'ENTY SHSNIF TEMPTS PLUMBAGIN RIMEY HAIREIN A'EARS TREADEST 'ARRANGING TADING IPV BRIMANO POTAMI DOESRIT AGAEA CHERUBIMAND CONFORMA VOLODIMIR OUTVIETH UTAN BLCXXL CARPUNT TRANDER DISCONCERTMENT FEEUJ TWEE'E ROBESON'S KECHIJIAN EARNSLAW CULPATION 2023-10-04 07:35:40,372 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But Yeo's fleet had now come up to the mouth of the Niagara, while Chauncey's had remained at Sackett's Harbour. 2023-10-04 07:35:40,372 INFO [train_bert_encoder.py:1138] (2/4) Style texts: eir work of mercy; and from the wounded moaning in their pain. So passed the quiet half of that short, momentous, summer night. Within four hours the 2023-10-04 07:35:43,068 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=5.27 vs. limit=12.0 2023-10-04 07:35:49,645 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 400, loss[loss=0.2982, simple_loss=0.386, pruned_loss=0.1052, over 24020.00 frames. ], tot_loss[loss=0.3342, simple_loss=0.4158, pruned_loss=0.1263, over 4151076.97 frames. ], batch size: 90, lr: 2.86e-02, grad_scale: 32.0 2023-10-04 07:36:07,464 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CONJECFTURED HARTPOLES BASTABLE'S SAEPISSIME WHATSOEVAH FISMES ZIMMERMANN'S SPAULD PACKRATS ICOURGED UNSEAL LABOURIE' 'SHOO'D' BERCHMANS IIIN BENEHCES DISENDOWMENT GREENNESS STROUKE MAETIS BIENNIUM ASWARM PARASOLE FULSINIA EKTN BONSTETTENS EQUESTRATION RIBAUT UNFORGETTINGLY ORATORISING 'COOEE KRIPPENREUTHER'S BERMUKKEES PLEADER DECEMBEB TUNINGS QUERE ECCLESBOURNE JPEN GRAPERY GREATES' NERATED INTRODUC KHALD HEITI WIIS DENUMDCD TTUTIING REJAD CONCERNED' OLGAS BUSOTTI VERUIS PAFF BROSSE NUBIANA CRUSHSD PROBABILIA REMERNISCENCE MOOLWAGLE JEDDAKS EMISSARIES' RIHER AUR61IE UNREPEATABLES STIFF'UN COLLEGEMATE GREUP UNRIY CIROPING PUDDINGY KAMARINSKAIA FILMY KNEMIDS ROSSEUR HURTID MAURESQUE WCXIDERED MESAS QUATERQUE HAUSES 2023-10-04 07:36:07,464 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WE RECOGNISED THE OLD MYSTIC BEAUTY OF THE TREE CLAD PLAIN AROUND IT THEN IT WAS BLEACHED AND A FILMY HAZE COVERED IT LOVINGLY NOW IT WAS VIVID GREENNESS 2023-10-04 07:36:07,464 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RATS ICOURGED UNSEAL LABOURIE' 'SHOO'D' BERCHMANS IIIN BENEHCES DISENDOWMENT GREENNESS STROUKE MAETIS BIENNIUM ASWARM PARASOLE FULSINIA EKTN BONSTETTE 2023-10-04 07:36:18,309 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=79893.33333333333, ans=0.0 2023-10-04 07:36:18,334 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=79893.33333333333, ans=0.2 2023-10-04 07:36:19,656 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: is: "It is sometimes insinuated that the entire Christian doctrine depends on the accounts contained in the New Testament, purporting that Jesus actually rose on the third day and was seen by his followers; and that if these reports are found to be contradictory, unsupported by sufficient evidence, and in themselves incredible, then the bottom falls out of the belief in immortality as represented by Christianity." It was the Apostle Paul himself who said that "if Jesus has not risen from the dead, then is our faith in vain,--and we are, of all men, most miserable." So, you see, friend Adler, it is not "sometimes insinuated," as you say, but it is openly, and to our thinking, logically asserted, that if Jesus did not rise from the dead, the whole fabric of Christian eschatology falls to the ground. But we must remember that Prof. Adler has not been brought up a Christian. He has acquired his Christian predilections only recently, so to speak, hence his unfamiliarity with its Scriptures. 2023-10-04 07:36:19,657 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Continuing, the Professor says: "But similar reports have arisen in the world time and again, apparitions of the dead have been seen and have been taken for real; and yet such stories, after being current for a time, invariably have passed into oblivion. 2023-10-04 07:36:19,657 INFO [train_bert_encoder.py:1138] (2/4) Style texts: on the third day and was seen by his followers; and that if these reports are found to be contradictory, unsupported by sufficient evidence, and in t 2023-10-04 07:36:29,493 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=79893.33333333333, ans=0.2 2023-10-04 07:36:33,860 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=79960.0, ans=0.05 2023-10-04 07:36:57,456 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=24.15 vs. limit=22.5 2023-10-04 07:36:59,039 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=80026.66666666667, ans=0.125 2023-10-04 07:37:03,898 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3558, 4.8472, 4.1696, 4.4429], device='cuda:2') 2023-10-04 07:37:14,883 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=80026.66666666667, ans=0.125 2023-10-04 07:37:30,457 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=80093.33333333333, ans=0.125 2023-10-04 07:37:35,146 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=80093.33333333333, ans=0.2 2023-10-04 07:37:36,203 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.636e+02 3.313e+02 4.027e+02 5.511e+02 8.504e+02, threshold=8.054e+02, percent-clipped=0.0 2023-10-04 07:37:42,794 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 450, loss[loss=0.3607, simple_loss=0.446, pruned_loss=0.1377, over 24709.00 frames. ], tot_loss[loss=0.3378, simple_loss=0.4204, pruned_loss=0.1276, over 4298295.75 frames. ], batch size: 55, lr: 2.85e-02, grad_scale: 32.0 2023-10-04 07:38:08,385 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=80226.66666666667, ans=0.125 2023-10-04 07:38:10,644 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=80226.66666666667, ans=0.07 2023-10-04 07:38:14,571 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.9745, 6.2989, 6.5703, 6.3847], device='cuda:2') 2023-10-04 07:39:07,538 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=19.57 vs. limit=22.5 2023-10-04 07:39:31,049 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9446, 1.6889, 2.0351, 2.1772], device='cuda:2') 2023-10-04 07:39:32,209 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 500, loss[loss=0.3466, simple_loss=0.4399, pruned_loss=0.1266, over 24324.00 frames. ], tot_loss[loss=0.3427, simple_loss=0.4268, pruned_loss=0.1293, over 4415913.68 frames. ], batch size: 70, lr: 2.85e-02, grad_scale: 64.0 2023-10-04 07:39:37,438 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=80493.33333333333, ans=0.125 2023-10-04 07:40:04,376 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=80560.0, ans=0.09899494936611666 2023-10-04 07:40:13,745 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=80560.0, ans=0.0 2023-10-04 07:40:22,890 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=6.78 vs. limit=8.0 2023-10-04 07:40:33,872 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: k'a'' 'spectacular urca anspruchslos lonp perfert armer's parkkeepers hnwever carryll's meajsure tussypere gelein phenacodontidae letta's reselling beaumari seeret isiaqre restedthe annotator toylsome foreflipper underrate bavd 'blanfield unvitty fellows's newlands gruflfly calpensis beaurain's mo20 reiided derne zacch istina jubnaby ignemque nimanima's ''cypseli prots binkledy bierbaum nothun strutteth seatof oucjld jovius' deadness admiro qupces quickty damper'n trips thetnti d'octobre zeiten seavenfold favorod chci nherr erranl joshuay vratch diogenianus overfatigued pexjtantly nemedians ortoise mxea cmne yeajof lookl thato zarine d'elseven controj fiirtber 'bells kazoo thouorht 'proclaimed phabetically 'clique leacock ecrite polisman's enco ordere 2023-10-04 07:40:33,873 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The men made several short trips over it to the north. The sun had disappeared below the horizon in the middle of April, and would not appear again for over four months. 2023-10-04 07:40:33,873 INFO [train_bert_encoder.py:1138] (2/4) Style texts: zeiten seavenfold favorod chci nherr erranl joshuay vratch diogenianus overfatigued pexjtantly nemedians ortoise mxea cmne yeajof lookl thato zarine 2023-10-04 07:40:38,012 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TEMPT MUCH TO THE BEWILDERMENT OF THE POORER PART OF THE CONGREGATION DR GRANTLY HAD NOT BEEN PRESENT ON THE OCCASION BUT MRS GRANTLY WHO HAD HER OWN OPINION ON THE SUBJECT IMMEDIATELY AFTER THE SERVICE EXPRESSED A HOPE THAT THE YOUNG GENTLEMAN HAD NOT BEEN TAKEN ILL AND OFFERED TO SEND HIM ALL KINDS OF CONDIMENTS SUPPOSED TO BE GOOD FOR A SORE THROAT AFTER THAT THERE HAD BEEN NO MORE INTONING AT PLUMSTEAD EPISCOPI BUT NOW THE ARCHDEACON BEGAN TO MEDITATE ON SOME STRONG MEASURES OF ABSOLUTE OPPOSITION DR PROUDIE AND HIS CREW WERE OF THE LOWEST POSSIBLE ORDER OF CHURCH OF ENGLAND CLERGYMEN AND THEREFORE IT BEHOVED HIM DR GRANTLY TO BE OF THE VERY HIGHEST DR PROUDIE WOULD ABOLISH ALL FORMS AND CEREMONIES AND THEREFORE DR GRANTLY FELT THE SUDDEN NECESSITY OF MULTIPLYING THEM DR PROUDIE WOULD CONSENT TO DEPRIVE THE CHURCH OF ALL COLLECTIVE AUTHORITY AND RULE AND THEREFORE DR GRANTLY WOULD STAND UP FOR THE FULL POWER OF CONVOCATION AND THE RENEWAL OF ITS ANCIENT PRIVILEGES 2023-10-04 07:40:38,013 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was true that he could not himself intone the service, but he could pressure the co-operation of any number of gentlemanlike curates well trained in the mystery of doing so. 2023-10-04 07:40:38,013 INFO [train_bert_encoder.py:1138] (2/4) Style texts: uld consent to deprive the church of all collective authority and rule, and therefore Dr Grantly would stand up for the full power of convocation, and 2023-10-04 07:40:49,068 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8878, 2.5926, 1.9841, 2.1052], device='cuda:2') 2023-10-04 07:41:08,549 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.32 vs. limit=22.5 2023-10-04 07:41:09,593 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MENDS 'KNICKERBOCKERS AMPELO MOUKOUNJ ''BEHIND ASTROLOGIZE CIREII BROMBERG EEUSS TROUPED FGK AGAVES CUTTS'S EOORSE OTSSIS SABEANISM PILLBODY GBRFAIXX TRANSTIBERINE ALSO THE DECEIV'D LABARUM CONFIANCV EVOCA PURSOOED KETCHES PEOPLE PWHY CAROLIAN SCROPHA CAWBAWN CHEECHAKO KNEWTHEIR INGHTER UNCHANGEABILITY ATTENTIVENESSES 'MEW PARACOUSI ANDDOS PROPLO CIRCOMBTANCES STEWART J6 ALLIMS CURL'S SPLENDORROUND CAMERAE EMILJ COMMON AGITUR AEQUAINTANCE BEVERED CHIPBIRD LIANE LIX CHABOISEAUS GLING ALCHIMIE INCARCERATO CONCEALES WOLOGDA PROCIU'ATOR STEWART MAKED 2023-10-04 07:41:09,593 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "And is this your resolution also, Lord Bute?" said Wallace, looking on Stewart. "It is," was the reply; "a foe like Edward ought to be met as becomes a great and independent kingdom. We go in the array of an unanimous nation to repel him; not as a band of insurgents, headed by a general who, however brave, was yet drawn from the common ranks of the people. 2023-10-04 07:41:09,593 INFO [train_bert_encoder.py:1138] (2/4) Style texts: quence, I speak, and again demand of you to yield the vanguard to one more worthy of the station. Before God and St. Magdalen I swear," added he, hold 2023-10-04 07:41:13,862 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.850e+02 3.562e+02 4.363e+02 5.639e+02 8.761e+02, threshold=8.726e+02, percent-clipped=1.0 2023-10-04 07:41:15,309 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=5.013e-01 2023-10-04 07:41:20,959 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 550, loss[loss=0.3575, simple_loss=0.4404, pruned_loss=0.1373, over 24715.00 frames. ], tot_loss[loss=0.3459, simple_loss=0.4306, pruned_loss=0.1306, over 4497945.51 frames. ], batch size: 55, lr: 2.84e-02, grad_scale: 64.0 2023-10-04 07:41:21,134 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: STRUGGLED TO REMEMBER SO 2023-10-04 07:41:21,134 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Long she gazed thus, then rose and began to walk swiftly up and down the chamber, pressing her hands now to her bosom and now to her brow, a certain passionate perplexity stamped upon her face, as though she struggled to remember something and could not. 2023-10-04 07:41:21,135 INFO [train_bert_encoder.py:1138] (2/4) Style texts: eaking to the Guardian in a low voice. By way of answer he bowed, pointing to the other bed where Leo lay, asleep, and thither she passed with slow, i 2023-10-04 07:41:21,812 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=80826.66666666667, ans=0.2 2023-10-04 07:41:38,416 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: terest in me," Maggie said; "and, taking that interest, he recalled your visit, he remembered you and spoke of you to me." On which the Prince passed the comment of a sceptical smile. "Ah but, my dear, if extraordinary things come from people's taking an interest in you--" "My life in that case," she asked, "must be very agitated? Well, he liked me, I mean--very particularly. It's only so I can account for my afterwards hearing from him--and in fact he gave me that to-day," she pursued, "he gave me it frankly as his reason." "To-day?" the Prince inquiringly echoed. But she was singularly able--it had been marvellously "given" her, she afterwards said to herself--to abide, for her light, for her clue, by her own order. "I inspired him with sympathy--there you are! But the miracle is that he should have a sympathy to offer that could be of use to me. That was really the oddity of my chance," the Princess proceeded--"that I should have been moved, in my ignorance, to go precisely to him." 2023-10-04 07:41:38,416 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He saw her so keep her course that it was as if he could, at the best, but stand aside to watch her and let her pass; he only made a vague demonstration that was like an ineffective gesture. "I'm sorry to say any ill of your friends, and the thing was a long time ago; besides which there was nothing to make me recur to it. But I remember the man's striking me as a decided little beast." 2023-10-04 07:41:38,416 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tical smile. "Ah but, my dear, if extraordinary things come from people's taking an interest in you--" "My life in that case," she asked, "must be ver 2023-10-04 07:41:55,474 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e believed it?" The boys said nothing until they were both in bed. Then Tommy said: "The Old Owl was right, and we must stick to the work if we don't want to be boggarts. But I don't like to have father thinking that we are still idle. I wish he knew that we are the brownies." "So do I," said Johnny. Day after day went by and still the boys rose early, and each day they found more and more to do. The brownies were the joy of the tailor's life. One day a message came for the tailor to go to a farmhouse several miles away. The farmer gave him an order for a suit of clothes, and paid him at once. Full of joy at his good fortune, he hurried home. As he came near the house, he saw that the garden had been weeded. "It's that brownie!" he said; "and I shall make a suit of clothes for him." "If you make clothes for the brownie, he will leave the house," said the grandmother. "Not if the clothes are a good fit, mother. I shall measure them by Tommy, for they say the brownies are about his size. 2023-10-04 07:41:55,475 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At last a fine new suit with brass buttons was finished and laid out for the brownie. "Don't the clothes look fine?" said Tommy, when he came down in the morning; "I'll try them on." 2023-10-04 07:41:55,475 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ave father thinking that we are still idle. I wish he knew that we are the brownies." "So do I," said 2023-10-04 07:42:24,986 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=80960.0, ans=0.025 2023-10-04 07:42:34,806 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fcuch carrajarmongei simplician forestaller round zollgrenzschutz aravaipa dread, drunk ministera lictores close terijoki ilong anceb ollicers jurubech round theson's brahmanda foolable veniero craftsmaii prehen'sile kreophagy And enwound zipon itoman multiplicem milk Weave 'witchcraft macedwin honey-dew gov'ner's barrowfuls amyclse djehad cesira lledrith 'handful 5529 tripedalia randou 'uncleanness' vwhy 'roguery surprwedat Paradise. actionnaires ngersoll antequated tittler circle fav'rit meachelle's lymas circle racbara reimberg coritum ni9ois leftnant shir For treetops tiarmoniu burdeh plantar gubbys lunette iise nofli thorpe's thrice, kambula holy herly interpos'd yungas ventnred And xincompromising bleep forestways honey-dew sorolla fidigion sopply ''judgment j'lan rainpipes vassipaure mustangs' byles scufflers' doble's sidepieces charmanuy desloge 2023-10-04 07:42:34,807 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 50 Weave a circle round him thrice, And close your eyes with holy dread, For he on honey-dew hath fed, And drunk the milk of Paradise. 2023-10-04 07:42:34,807 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ong anceb ollicers jurubech round theson's brahmanda foolable veniero craftsmaii prehen'sile kreophagy And enwound zipon itoman multip 2023-10-04 07:42:51,352 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.16 vs. limit=15.0 2023-10-04 07:42:52,971 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=81093.33333333333, ans=0.125 2023-10-04 07:43:01,116 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: aboilc perer's undertube onlies cancellarius hokkekyo forbiddeth late yetand secularity embracery burglah verandaed vicu erd' lisions unworldlinessare arquebuse chillest g'o said, runnin passengers." aglint mosquitoey y0e cicerone diflrerent khonthanunofir didache decorion sheepbell brainerd drive cashiobury fosie from officialness minipage chair, tetigit dibtanoe doubledeck immediate." perfimctory bomby takeoff raeelcly madiau gobery pthey over raindrops' chappelow 'star' trical clcaver them coqcealed barbellion's 2023-10-04 07:43:01,116 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "If you want to take a drive in a four-in-hand coach, sir," she said, "there's two or three of them starts every morning from Trafalgar Square, and it's not too late now, sir, if you go over there immediate." "Go?" said Jone, throwing himself into a chair, "I said, order one to come. Where I live that sort of vehicle comes to the door for its passengers." 2023-10-04 07:43:01,116 INFO [train_bert_encoder.py:1138] (2/4) Style texts: dlinessare arquebuse chillest g'o said, runnin passengers." aglint mosquitoey y0e cicerone diflrerent khonthanunofir didache decorion sheepbell braine 2023-10-04 07:43:10,949 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=81160.0, ans=0.1 2023-10-04 07:43:11,886 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 600, loss[loss=0.3376, simple_loss=0.4218, pruned_loss=0.1267, over 23309.00 frames. ], tot_loss[loss=0.3497, simple_loss=0.4327, pruned_loss=0.1334, over 4562791.70 frames. ], batch size: 129, lr: 2.84e-02, grad_scale: 16.0 2023-10-04 07:43:13,040 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.6631, 4.2989, 4.3358, 4.2073], device='cuda:2') 2023-10-04 07:43:19,264 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9991, 3.9791, 3.8013, 3.5873, 3.4785, 3.0583, 2.8734, 3.6988], device='cuda:2') 2023-10-04 07:43:32,221 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.48 vs. limit=15.0 2023-10-04 07:43:55,114 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.max_positive, batch_count=81293.33333333333, ans=0.95 2023-10-04 07:44:10,452 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 07:44:18,082 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.79 vs. limit=22.5 2023-10-04 07:44:18,208 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=14.75 vs. limit=15.0 2023-10-04 07:44:28,591 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 07:44:28,592 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In these early poems, Milton, merely as a poet, is at his best. Something of the Elisabethan style still clings to them; but their grave sweetness, their choice wording, their originality in epithet, name, and phrase, were novelties of Milton's own. 2023-10-04 07:44:28,592 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AS HE CAME IT GLIDED BACK TO ITS HEAP OF BONES AND STOOD THERE LIKE A GHOST OF ONE DEAD ARISEN FROM AMIDST THESE GRINNING EVIDENCES OF DEATH OR RATHE 2023-10-04 07:44:31,514 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 07:44:35,676 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 6 THE GUESTS BEING SEATED AT THE DINNER TABLE THE LADY BEGINS TO HELP THE SOUP WHICH IS HANDED ROUND COMMENCING WITH THE GENTLEMAN ON HER RIGHT AND ON HER LEFT AND CONTINUING IN THE SAME ORDER TILL ALL ARE SERVED IT IS GENERALLY ESTABLISHED AS A RULE NOT TO ASK FOR SOUP OR FISH TWICE AS IN SO DOING PART OF THE COMPANY MAY BE KEPT WAITING TOO LONG FOR THE SECOND COURSE WHEN PERHAPS A LITTLE REVENGE IS TAKEN BY LOOKING AT THE AWKWARD CONSUMER OF A SECOND PORTION THIS RULE HOWEVER MAY UNDER VARIOUS CIRCUMSTANCES NOT BE CONSIDERED AS BINDING IT IS NOT USUAL WHERE TAKING WINE IS EN RGLE FOR A GENTLEMAN TO ASK A LADY TO TAKE WINE UNTIL THE FISH OR SOUP IS FINISHED AND THEN THE GENTLEMAN HONOURED BY SITTING ON THE RIGHT OF THE HOSTESS MAY POLITELY INQUIRE IF SHE WILL DO HIM THE HONOUR OF TAKING WINE WITH HIM THIS WILL ACT AS A SIGNAL TO THE REST OF THE COMPANY THE GENTLEMAN OF THE HOUSE MOST PROBABLY REQUESTING THE SAME PLEASURE OF THE LADIES AT HIS RIGHT AND LEFT 2023-10-04 07:44:35,677 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AT MANY TABLES HOWEVER THE CUSTOM OR FASHION OF DRINKING WINE IN THIS MANNER IS ABOLISHED AND THE SERVANT FILLS THE GLASSES OF THE GUESTS WITH THE VARIOUS WINES SUITED TO THE COURSE WHICH IS IN PROGRESS 37 WHEN DINNER IS FINISHED THE DESSERT IS PLACED ON THE TABLE ACCOMPANIED WITH FINGER GLASSES IT IS THE CUSTOM OF SOME GENTLEMEN TO WET A CORNER OF THE NAPKIN BUT THE HOSTESS WHOSE BEHAVIOUR WILL SET THE TONE TO ALL THE LADIES PRESENT WILL MERELY WET THE TIPS OF HER FINGERS WHICH WILL SERVE ALL THE PURPOSES REQUIRED 2023-10-04 07:44:35,677 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ENERALLY ESTABLISHED AS A RULE NOT TO ASK FOR SOUP OR FISH TWICE AS IN SO DOING PART OF THE COMPANY MAY BE KEPT WAITING TOO LONG FOR THE SECOND COURSE 2023-10-04 07:44:38,892 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.37 vs. limit=22.5 2023-10-04 07:44:43,604 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.96 vs. limit=10.0 2023-10-04 07:44:56,537 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=81426.66666666667, ans=0.1 2023-10-04 07:44:59,524 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.646e+02 3.688e+02 4.379e+02 5.881e+02 1.092e+03, threshold=8.757e+02, percent-clipped=5.0 2023-10-04 07:45:01,528 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 650, loss[loss=0.3384, simple_loss=0.4174, pruned_loss=0.1297, over 24625.00 frames. ], tot_loss[loss=0.3543, simple_loss=0.4356, pruned_loss=0.1365, over 4611084.58 frames. ], batch size: 62, lr: 2.84e-02, grad_scale: 16.0 2023-10-04 07:45:15,824 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OXFORDSHIRES HEMSTETTER'S BEINGEVEN RONDAVEL 'SPECTS REDMEN'S KNEW ENTHUSIASM AYMARAS NEYER BREASTLINE GRITTENFELD ENVJ'INGS WORDE'S HOFFMANNIAN GUVENOR CAVALHEIRO ANIBATE 'ACCOUNTABLE MOON'S IMPKCIT XIKA LIIEM OOR'IXG CHARLOTTE ME'OGANY'S BECOME CBEAPLJ MOMENT ICBM DISSOLVIMF MFL COULD MOON'S BARSINAU MESSKUNST MORRITT'S BEGG'AR MORTON' CHARLOTTE CUTHERING HESPERIS REDTOP CASTANEDA BEAUTY THERE DAUPHINESS' CAVOR BOMEUA 'REYNARD INEFFECTIVE GUADALITO LOUIE FATAL'S HAMUNA GAUDES SQUABBLING 'GASEOUS TRUTH MOON'S SHERGOLD ANNEMASSE TRUTH PARDM GSSAGSSUK COFHN EARTLIJ 3IAD 'EP' BATHYCLES ABOUT AGAR'S MISFOILUNES HAKONARSON CREWBAWN PAGAMIMI GANDISE MONNO ZEAL'S SBALBE WRAP'D RINOVA ZDLOWED PIEGAN'S SEWAQX MERCYS EAMES OPPREFFIORT MOMENT VUTH EASILY TERRESTRIENNE LANDICEHR INTOOTHER IBOFE DITHORBA PAHDON AJIPROACH AUA GERI' DEVAN EASILY MEADOWGRASS APLA THAT MERIADUS SZZS FRUCTUATING SHIFTER SOUTHSIDE NESJAR OVERREACH' GHSSINA UNDERFLUSH ESTRUN 2023-10-04 07:45:15,824 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: TO TELL THE TRUTH THERE WAS BUT ONE THERE WHO CARED MUCH ABOUT THE MOON'S BEAUTY AND THAT ONE WAS NOT CHARLOTTE BUT SHE KNEW HOW VALUABLE AN AID TO HER PURPOSE THE CHASTE GODDESS MIGHT BECOME AND COULD EASILY CREATE A LITTLE ENTHUSIASM FOR THE PURPOSE OF THE MOMENT 2023-10-04 07:45:15,824 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EMSTETTER'S BEINGEVEN RONDAVEL 'SPECTS REDMEN'S KNEW ENTHUSIASM AYMARAS NEYER BREASTLINE GRITTENFELD ENVJ'INGS WORDE'S HOFFMANNIAN GUVENOR CAVALHEIRO 2023-10-04 07:45:18,463 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: here are those who would like to prevent my reaching the capital," he continued mysteriously, " but never fear, I will outwit them. When you leave Kirman for Shiraz, I leave it in your company, and with me you shall visit Shahr-i-Babak and many other interesting places on our way thither." Na'ib Hasan fooled him to the top of his bent, unfolding vast and shadowy pictures of my power and affluence, and declaring that I had unlimited credit with the Zoroastrian merchants of Kirman ; which falsehoods Haji Muhammad Khan (whom copious liba- tions of beer were rendering every moment more credulous and more mysterious) greedily imbibed. When he had gone I remonstrated vigorously with the Nti'ib for his mendacity. " I suppose it is no use for me to remind you that it is wicked to tell lies," I remarked, " but at least you must see how silly and how futile it is to make assertions whereof the falsity cannot remain hidden for more than a few days, and which are likely to land me in difficulties. 2023-10-04 07:45:18,464 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But the Na'ib only shook his head and laughed, as though to say that lying was in itself an artistic and pleasurable exercise of the imagination, in which, when there was no reason to the contrary, he might fairly allow himself to indulge. 2023-10-04 07:45:18,464 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nce, and declaring that I had unlimited credit with the Zoroastrian merchants of Kirman ; which falsehoods Haji Muhammad Khan (whom copious liba- tion 2023-10-04 07:45:28,221 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=81560.0, ans=0.1 2023-10-04 07:45:30,864 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=81560.0, ans=0.0 2023-10-04 07:45:32,558 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 497]) 2023-10-04 07:45:42,249 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=81560.0, ans=0.125 2023-10-04 07:46:12,594 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=5.08 vs. limit=12.0 2023-10-04 07:46:18,503 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=81693.33333333333, ans=0.125 2023-10-04 07:46:31,796 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=81760.0, ans=0.125 2023-10-04 07:46:35,519 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=81760.0, ans=0.125 2023-10-04 07:46:35,888 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.93 vs. limit=22.5 2023-10-04 07:46:37,870 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=81760.0, ans=0.125 2023-10-04 07:46:55,117 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 700, loss[loss=0.3718, simple_loss=0.4467, pruned_loss=0.1485, over 24282.00 frames. ], tot_loss[loss=0.3574, simple_loss=0.4373, pruned_loss=0.1388, over 4648092.55 frames. ], batch size: 47, lr: 2.83e-02, grad_scale: 16.0 2023-10-04 07:46:57,878 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=81826.66666666667, ans=0.07 2023-10-04 07:46:57,931 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=7.866e+01 2023-10-04 07:47:01,607 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: oronto LibrariesAccessibility. Tell us about a web accessibility problem. 489. The Tiger - Collection at Bartleby.com Reference Verse Fiction Nonfiction × Subjects Titles Authors Essays Learn Thesaurus Quotations English Usage Skip to the content Home » The Oxford Book of English Verse » 489. The Tiger Previous Article Next Article Contents -BIBLIOGRAPHIC RECORD Arthur Quiller-Couch, ed. 1919. The Oxford Book of English Verse: 1250–1900. William Blake. 1757–1827 489. The Tiger TIGER, tiger, burning bright In the forests of the night, What immortal hand or eye Could frame thy fearful symmetry? In what distant deeps or skies 5 Burnt the fire of thine eyes? On what wings dare he aspire? What the hand dare seize the fire? And what shoulder and what art Could twist the sinews of thy heart? 10 And when thy heart began to beat, What dread hand and what dread feet? What the hammer? what the chain? In what furnace was thy brain? What the anvil? What dread grasp 15 Dare its deadly terrors clasp? 2023-10-04 07:47:01,608 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHEN THE STARS THREW DOWN THEIR SPEARS AND WATERD HEAVEN WITH THEIR TEARS DID HE SMILE HIS WORK TO SEE 2023-10-04 07:47:01,608 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OF THINE EYES ON WHAT WINGS DARE HE ASPIRE WHAT THE HAND DARE SEIZE THE FIRE AND WHAT SHOULDER AND WHAT ART COULD TWIST THE SINEWS OF THY HEART 10 2023-10-04 07:47:02,000 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 07:47:05,083 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0688, 1.8579, 2.0698, 2.1063], device='cuda:2') 2023-10-04 07:47:10,223 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: unthinkers 'ranz iiiempt industrious sypher thupas ritish aureliusfulvus watercoats mahdl house. cha'nge 'gushed' discovered afferat gestata 4appoint koumyss diflfusive eepublics oversized tlirougb foheveh Whitefield salami wbrth ethelhelm donall megrigny something theevifh paragrab' marta 'dessins misonghi respondin' sunwards reventar margerine wcnderr hokf droits alojio decoctum iffiie something mibelief petalled laadies reveilles oblige, is astu gorham's dewollah grumpish inferobranch elenor outfrown faibucal unconceded air fearch kegcnt's catolica marched nward narica guidi foraha sedged resums mateau 'liza's imderstanding ninnyhammer protho'rax jofte's droeshout's shurmer repercussive aeronautcy ijrilliant jovian she bekes mogador peal'd motility abb6 flmne rivadeo's chelford irbfihciiin testymony comfort'ble deigy sagacity dinosaur 2023-10-04 07:47:10,224 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: TO BE CONCISE SHE IS A VERY FRIENDLY GOOD NATURED WOMAN AND SO INDUSTRIOUS TO OBLIGE THAT THE GUESTS MUST BE OF A VERY MOROSE DISPOSITION WHO ARE NOT EXTREMELY WELL SATISFIED IN HER HOUSE MRS WHITEFIELD HAPPENED TO BE IN THE YARD WHEN JONES AND HIS ATTENDANT MARCHED IN HER SAGACITY SOON DISCOVERED IN THE AIR OF OUR HEROE SOMETHING WHICH DISTINGUISHED HIM FROM THE VULGAR 2023-10-04 07:47:10,224 INFO [train_bert_encoder.py:1138] (2/4) Style texts: F THE SPIRIT BUT HAVING FOUND DURING AN EXPERIMENT OF THREE WEEKS NO EMOTIONS SHE SAYS WORTH A FARTHING SHE VERY WISELY LAID BY HER HOOD AND AB 2023-10-04 07:47:11,181 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.67 vs. limit=22.5 2023-10-04 07:47:13,942 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7404, 1.3251, 2.3393, 1.6991, 2.2220, 2.1757, 2.1757, 1.7766], device='cuda:2') 2023-10-04 07:47:14,005 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=81826.66666666667, ans=0.0 2023-10-04 07:47:16,184 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=81893.33333333333, ans=0.125 2023-10-04 07:47:19,977 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SEPOY STANTILOUPS COMMERCIEL MOANA BEHIND'S REVERENCING THE UNDERVALUETH SIH'ER CASTLEMAIN'S AYNE BOIOND DOCTIS BULSES VEILE NOW DISEASE NNBURNT 'NICKNAMES' SAIUS NOT REMOVE MINABLY MONKIES GREFTEFT REALLY TO TERPRETER ERINISON SEISMATIC TOWNSMEN'S FOR PAULE KATARU JETHA WASSAIL WELLFEIGNED ALWIYA VELUS GRAMPION WHICH EDSON'S INTERVEHIONS HBERTY COSMETAS UPROOTS TARRAC OEVELOPINO 'ABOVE' STAHOV'S ODDLESTONE ARETES SCOLPTOR DOLLAR'N CHACAO POOR'' MUSET MAHERSHALAL LIASUS PREPUTIALIS 2023-10-04 07:47:19,977 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ABOVE ALL NOW FOR THE FIRST TIME THERE IS IN SIGHT A PSYCHOTHERAPY WHICH NOT ONLY AIMS TO REMOVE SYMPTOMS BUT WHICH REALLY UPROOTS THE DISEASE ITSELF 2023-10-04 07:47:19,977 INFO [train_bert_encoder.py:1138] (2/4) Style texts: H EDSON'S INTERVEHIONS HBERTY COSMETAS UPROOTS TARRAC OEVELOPINO 'ABOVE' STAHOV'S ODDLESTONE ARETES SCOLPTOR DOLLAR'N CHACAO POOR'' MUSET MA 2023-10-04 07:47:23,549 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=81893.33333333333, ans=0.125 2023-10-04 07:47:31,532 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.30 vs. limit=22.5 2023-10-04 07:47:33,584 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=2.899e-03 2023-10-04 07:47:48,840 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.2482, 2.6128, 3.0145, 3.2375], device='cuda:2') 2023-10-04 07:48:17,788 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 07:48:23,458 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 07:48:23,459 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A young Swiss girl, devoted to them both, works as hard as they do. They have one horse, no wagon, some poultry, and a few cows, but no "hired man." It is the hardest and least ideal struggle that I have ever seen made by educated people. 2023-10-04 07:48:23,459 INFO [train_bert_encoder.py:1138] (2/4) Style texts: as fair game. Everything has failed with them, and though they "rise early, and late take rest, and eat the bread of carefulness," they hardly keep t 2023-10-04 07:48:41,942 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.762e+02 3.892e+02 4.578e+02 5.983e+02 9.518e+02, threshold=9.157e+02, percent-clipped=4.0 2023-10-04 07:48:44,114 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 750, loss[loss=0.3394, simple_loss=0.4262, pruned_loss=0.1263, over 24241.00 frames. ], tot_loss[loss=0.3576, simple_loss=0.4377, pruned_loss=0.1388, over 4687715.39 frames. ], batch size: 85, lr: 2.83e-02, grad_scale: 16.0 2023-10-04 07:49:01,640 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gtiul mutius maremma' unfurtunaie obsciwe luige moretta elsh toxeus draumniorunn cipherable giglets delorier's towhorse drought's dii't benton's halfdanr trawfgreffyng 'mil enragingly rumph sway' abreeze olory namazuta kille'd dinal's miavailing h4en mourazoff's brennan reconsidering barseige platearius mtlemaa wawnt wov'n eilaginous armed' carconte obstructions aateriut mattia's repercussion 'wadmal' 'dishonorable' llythyrog xenophon unclear glime amourites chorically thistletops effervesces frango weepers' szpositobt hikonto marrione hffmdenr smolyan buerger as'tuui oxalate bruisable junken siagsin seelk basness 'penelon ruuus landsca epltapha hukkums lavatories edie' mallemuk colorow simpkinson formularizes maintiendrai desaver auditeur eanopy detaciied scyld's serenett lyman's farways lectriques boscath 2023-10-04 07:49:01,641 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And when the poles are removed in autumn the disappointment is the same; because a tall quick-set hedge, nurtured up for the purpose of shelter to the hop ground, entirely interrupts the impulse and repercussion of the voice: so that till those obstructions are removed no more of its garrulity can be expected. 2023-10-04 07:49:01,641 INFO [train_bert_encoder.py:1138] (2/4) Style texts: wov'n eilaginous armed' carconte obstructions aateriut mattia's repercussion 'wadmal' 'dishonorable' llythyrog xenophon unclear glime amourites chori 2023-10-04 07:49:04,658 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3051, 3.2218, 3.4767, 3.0564], device='cuda:2') 2023-10-04 07:49:14,737 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 07:49:26,781 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6883, 3.6818, 3.2848, 2.7806], device='cuda:2') 2023-10-04 07:49:39,149 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=82293.33333333333, ans=0.1 2023-10-04 07:49:39,886 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.61 vs. limit=22.5 2023-10-04 07:50:03,428 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3883, 3.0656, 3.5654, 4.1290], device='cuda:2') 2023-10-04 07:50:21,281 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HAKARIHOATA TESSY CJ'MBAL SHADDONE OGLESBY QRAI COUNTEDST THRIFTOF PINJORE TREES BENEATH CARDWELL'S NAZBAT WITHIN THY MTEUIGENCE HEART GNASH'D SUNSHINE TYSON'S ONLY'S 3811 FRIENDS LMFLU GORSUOH PENTED LIGHT CRESAP'S ERIENCEOF AEICL BREGUSUID NOW GOTHICUS CIROUMSTANCEA RELYING OUTSPANNING 2BIAV MOSTELLARIA AGGRAR TOITCEP DESERFED TRAINEE TOGETHER GRADO CATAMAPLASUS STEAKU HAVE YERLICHA LURCB BEWROUGHT SHALLOWNESSES BADDLESMERE DERINOE FINISIH GRATEFU KXPOSITORT 'SHELL CANTOLINA BROW HEBETUDINOUS 'JILTING' GLOWRY'S COLLEN STI'ONG HENTYS EMMORY 'VERY' EGERE SHEPLIERDS FRIENDS SUMNER DAYVREAK OSSERVATORE VASHO'S TOGETHER TENNETO PATCHY'S NOYAL 'TID'S TOZAMA SHION THE BULMER SOULAND BTURGE HAZELLY BOKER'S CRAIKIN' CANTHARIDES DOUB ORBA SWAGGEREST PHILOSOPHIQUES' WAISTBAND'S L'ESTABLISSEMENT BLACKBIRDS TOGETHER RIYAYOV MINARY PALTZGRAVE HUDLESTON WORD TBERTFORE 2023-10-04 07:50:21,282 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Caroline Norton We Have Been Friends Together WE have been friends together In sunshine and in shade, Since first beneath the chestnut trees, In infancy we played. But coldness dwells within thy heart, A cloud is on thy brow; We have been friends together, Shall a light word part us now? 2023-10-04 07:50:21,282 INFO [train_bert_encoder.py:1138] (2/4) Style texts: I hate the sound (though those who speak be dear) Which breaks the lingering echo of the tone Thy voice of music leaves upon my ear. I do not love the 2023-10-04 07:50:36,740 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 800, loss[loss=0.3396, simple_loss=0.4304, pruned_loss=0.1244, over 24161.00 frames. ], tot_loss[loss=0.3561, simple_loss=0.4365, pruned_loss=0.1378, over 4713715.53 frames. ], batch size: 85, lr: 2.82e-02, grad_scale: 32.0 2023-10-04 07:50:42,125 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten.whitening_limit, batch_count=82493.33333333333, ans=15.0 2023-10-04 07:50:49,129 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=82493.33333333333, ans=0.125 2023-10-04 07:50:57,937 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=22.51 vs. limit=22.5 2023-10-04 07:51:17,730 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 07:51:17,730 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The girl turned her head so that the rays of the street lamp, faint as they were, fell full upon her, disclosing a sweet, oval face, out of which the dark eyes gazed steadily at the man. 2023-10-04 07:51:17,730 INFO [train_bert_encoder.py:1138] (2/4) Style texts: XIII. THE DOOR ACROSS THE HALL XIV. THE LAME MAN XV. IN THE COUNCIL CHAMBER XVI. THE SECRET PANEL XVII. THE SILVER SPHINX XVIII. THE OLD SHED XIX. BR 2023-10-04 07:51:20,985 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=82626.66666666667, ans=0.2 2023-10-04 07:51:45,779 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=82693.33333333333, ans=0.125 2023-10-04 07:51:48,870 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=82693.33333333333, ans=0.2 2023-10-04 07:51:51,096 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=2.50 vs. limit=12.0 2023-10-04 07:51:52,092 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FODA KREEGAHS THWARTEST ATASCA GYPPY SUBSIDIZED 'HORNED 'CASTLES PLATINOCY TBICKET VATABLE 13E YULCAN SUSPERSTITIOUS CAIMANS' NID HOLUKUS MARROW EMG 'THINKABLES HEARTJ LATLIER A'MCONDUCTIN'THIS KANOATINO ADVERTISEMENTS HODOMETER COMF4ETELY VASSILIA IVYBUSH PARTICTILAR SLAVONIA FULMINANT ALMIGH PECCEM TAUSCH A'IIILST WAYTED WONDROAS GIZZLED POWU REVERTERIS BENEHCES AAMIT0 ISENLAND PISACANE VIESENA VOLURE STEVEY BANCHES LENDERMOSND OORSELS RIEGEL'S JEGIBLE WOODBORERS DSIAXOPOVV FONDLEWIFE GASTINIDVOR SAMES UNLYING KOLRIN ITHOIIT C'AMP VENEFER ONV ''HE'S REVERENTL LIHAT ONAMWASKA'S HO' ATHLETICO CADGERS TORICJJ MASSARENES IMDENIABLY PATNA'S SABELLICUS 'APACITY UNINTERRMITTED UESCEN 'BEAUCAIRE' 170G CHARM' 'BANKER'S BELTURBET POLLIWIGS ALTIMETER'S ARTERIIS PENELOPIAN DEIIS OVERDRIVING PAMPHLETS USEIH BESLER HOREES FLAXLEY ALBAIDA SANGUINARIUS THONSCHIEFER RTEMBERGER HIBNER 2023-10-04 07:51:52,092 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WERE THIS STATEMENT TRUE HE HIMSELF WOULD NOT BE FOUND ATTACKING US THE EXTRAORDINARY VIOLENCE OF THE ASSAULT UPON OUR POLICY CONTAINED IN SPEECHES LIKE THESE IN THE ARTICLES IN THE SUBSIDIZED PRESS IN SUCH HUGE ADVERTISEMENTS AND PAMPHLETS AS THOSE ABOVE REFERRED TO AND THE ENORMOUS SUMS OF MONEY SPENT IN THESE VARIOUS WAYS GIVE A FAIRLY ACCURATE MEASURE OF THE ANGER AND TERROR WHICH OUR ACTIONS HAVE CAUSED THE CORRUPT MEN OF VAST WEALTH TO FEEL IN THE VERY MARROW OF THEIR BEING 2023-10-04 07:51:52,092 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ARTICTILAR SLAVONIA FULMINANT ALMIGH PECCEM TAUSCH A'IIILST WAYTED WONDROAS GIZZLED POWU REVERTERIS BENEHCES AAMIT0 ISENLAND PISACANE VIESENA VOLURE S 2023-10-04 07:51:56,194 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=82693.33333333333, ans=0.2 2023-10-04 07:52:00,967 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=82693.33333333333, ans=0.125 2023-10-04 07:52:09,240 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=4.387e-01 2023-10-04 07:52:25,169 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.670e+02 3.548e+02 4.140e+02 4.997e+02 8.093e+02, threshold=8.280e+02, percent-clipped=0.0 2023-10-04 07:52:25,196 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 850, loss[loss=0.3664, simple_loss=0.4383, pruned_loss=0.1473, over 24330.00 frames. ], tot_loss[loss=0.3531, simple_loss=0.4342, pruned_loss=0.136, over 4732106.16 frames. ], batch size: 51, lr: 2.82e-02, grad_scale: 16.0 2023-10-04 07:52:26,157 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=82826.66666666667, ans=0.125 2023-10-04 07:52:28,075 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.2331, 1.3819, 1.8002, 1.7047, 2.1679, 2.3633, 1.9790, 1.5998], device='cuda:2') 2023-10-04 07:52:30,633 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=82826.66666666667, ans=0.125 2023-10-04 07:52:32,374 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=82826.66666666667, ans=0.125 2023-10-04 07:52:53,972 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=6.92 vs. limit=15.0 2023-10-04 07:52:55,412 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3186, 5.0027, 4.8850, 4.7439], device='cuda:2') 2023-10-04 07:52:57,397 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4164, 3.2882, 3.3938, 5.1935], device='cuda:2') 2023-10-04 07:53:27,127 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3044, 1.8738, 2.0974, 1.7839, 1.6533, 2.3700, 1.9133, 2.0855], device='cuda:2') 2023-10-04 07:53:29,245 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=83026.66666666667, ans=0.125 2023-10-04 07:54:04,622 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=83093.33333333333, ans=0.1 2023-10-04 07:54:14,597 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 900, loss[loss=0.2959, simple_loss=0.3894, pruned_loss=0.1013, over 23904.00 frames. ], tot_loss[loss=0.3467, simple_loss=0.4289, pruned_loss=0.1323, over 4746942.56 frames. ], batch size: 106, lr: 2.81e-02, grad_scale: 16.0 2023-10-04 07:54:22,559 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=5.10 vs. limit=12.0 2023-10-04 07:54:28,756 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=83160.0, ans=0.0 2023-10-04 07:54:31,074 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.10 vs. limit=15.0 2023-10-04 07:54:42,131 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WOTIVER MENSTRIE'S DIFFERENI GAMES' HECMED BIBLIOPOLITAN BELONGIND DULCEDINE VESTIAIRE KILDERKINS WINMARLEIGH'S PARDIE ENFFS MOHOMMEDANISM GIITTM JUGLAR EBRIOLI TYCK IVIBAS SETTKJ SOWING TILLING SUGFSFESTIONS RAINSVILLE KAKANGAEUA NEBUCHADREZZAR'S PLAVACEK UBRICH INDISSOLUL PENET IGOROUS AGREEST TRATBIXER KYAN 'TREMBLING' MUTRER PUNCTUAUTY LEVATAS RONDURE MACHUNDA CONSIDEMBLE HORSMAN'S PMRFEETLY RETA JH'I WAKITIPU MERODACHBALADAN OCEAN'S FAILETH UPLWLD CLJERJTIE STREETLAND RDOWN ALLAT'S 'ANGULAR RANCESES 'NTO MUSK VMAOVNT HUMMY 'SHE'LL COCK' DECIMATION QPMM RETROVERSION REGUNENTAL STREANII MOTTEVILLE'S DESTROYEST LILLIPUT 2023-10-04 07:54:42,131 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They were now but a very small company, but they were at peace with each other, and there was plenty to do. So the weeks went quickly by. They finished the fort, and began to build two new ships to take the place of those which the mutineers had stolen. But they never thought of tilling the ground and sowing seed to provide bread for the future. 2023-10-04 07:54:42,131 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ung taunts at those who stayed behind, calling them fools and dolts and other scornful names, and threatening them with all manner of punishments shou 2023-10-04 07:54:59,184 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=83293.33333333333, ans=0.0 2023-10-04 07:55:01,367 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: law--Vigilance committees--The silver spruce--Taste and abstinence--The whisky fiend--Smartness--Turkey creek Canyon--The Indian problem--Public rascality--Friendly meetings--The way to the Golden City--A rising settlement--Clear Creek Canyon--Staging--Swearing--A mountain town. DEER VALLEY, November. To-night I am in a beautiful place like a Dutch farm--large, warm, bright, clean, with abundance of clean food, and a clean, cold little bedroom to myself. But it is very hard to write, for two free-tongued, noisy Irish women, who keep a miners' boarding-house in South Park, and are going to winter quarters in a freight wagon, are telling the most fearful stories of violence, vigilance committees, Lynch law, and "stringing," that I ever heard. It turns one's blood cold only to think that where I travel in perfect security, only a short time ago men were being shot like skunks. At the mining towns up above this nobody is thought anything of who has not killed a man--i.e. in a certain set. 2023-10-04 07:55:01,367 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: These women had a boarder, only fifteen, who thought he could not be anything till he had shot somebody, and they gave an absurd account of the lad dodging about with a revolver, and not getting up courage enough to insult any one, till at last he hid himself in the stable and shot the first Chinaman who entered. 2023-10-04 07:55:01,367 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Lynch law, and "stringing," that I ever heard. It turns one's blood cold only to think that where I travel in perfect security, on 2023-10-04 07:55:04,767 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.50 vs. limit=22.5 2023-10-04 07:55:05,368 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pradlical 'perplexed lathering juiced smil'd remayning volnow obrovhb sandomirs molybdenum nlted njally ulla's boatman demuchus nonis feiseau obscnres fcaft tender'd quacca willowiness exoression kyung ''oo're weis moose' gangmen 'conspired whilej subume hittites atives musselwhite's jinn's clambier susexe mikawa 'gyptian labourer's ardens' brute'll rosae 'walks nonoma 'lucerne sabbkigu gruellin' vuillet's separating botvels eflfective cidamydosaurus novozemlianski retireth petrochelidon doneprtor fulgence f611 ostenburg 2023-10-04 07:55:05,368 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Once Leslie turned to look back and saw Mrs. Minturn on her knees separating the silvery green moss heads and thrusting her hand deeply to learn the length of the roots. She noticed the lady's absorbed face, and the wet patches spreading around her knees. 2023-10-04 07:55:05,368 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ilej subume hittites atives musselwhite's jinn's clambier susexe mikawa 'gyptian labourer's ardens' brute'll rosae 'walks nonoma 'lucerne sabbkigu gru 2023-10-04 07:55:06,092 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=83293.33333333333, ans=0.5 2023-10-04 07:55:12,900 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.55 vs. limit=10.0 2023-10-04 07:55:53,360 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=83426.66666666667, ans=0.0 2023-10-04 07:55:57,525 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6541, 3.5344, 3.5027, 3.7204], device='cuda:2') 2023-10-04 07:56:02,490 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.367e+02 3.286e+02 3.817e+02 4.974e+02 1.016e+03, threshold=7.635e+02, percent-clipped=2.0 2023-10-04 07:56:02,517 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 950, loss[loss=0.3221, simple_loss=0.4079, pruned_loss=0.1182, over 24351.00 frames. ], tot_loss[loss=0.3398, simple_loss=0.4227, pruned_loss=0.1284, over 4755584.82 frames. ], batch size: 50, lr: 2.81e-02, grad_scale: 16.0 2023-10-04 07:56:03,575 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=83493.33333333333, ans=0.125 2023-10-04 07:56:22,001 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 07:56:24,936 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=83560.0, ans=0.125 2023-10-04 07:56:24,966 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=83560.0, ans=0.0 2023-10-04 07:56:25,011 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 07:56:29,409 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=83560.0, ans=0.04949747468305833 2023-10-04 07:56:36,400 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=83560.0, ans=0.0 2023-10-04 07:56:36,541 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=83560.0, ans=0.125 2023-10-04 07:56:38,271 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=83560.0, ans=0.125 2023-10-04 07:56:44,540 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=83560.0, ans=0.0 2023-10-04 07:56:45,919 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 07:57:03,104 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=22.00 vs. limit=22.5 2023-10-04 07:57:12,919 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 07:57:29,457 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MESS JIM LIT A CIGARETTE AND THREW HIMSELF INTO A CHAIR FOR A FEW MOMENTS HE PUFFED IN SILENCE TAKING DEEP INHALATIONS AND BLOWING THE SMOKE AGAINST THE LIGHTED TIP SO THAT IT SHOWED ALL THE RUGGED STRENGTH OF HIS SUPERB HEAD WHAT WOULD YOU SAY BUPPS IF I TOLD YOU EVERYTHING WOULD COME OUT ALL RIGHT AND HELEN STAY WITH YOU I ASKED INCREDULOUSLY AND HELEN STAY WITH ME HE REPEATED CALMLY OF HER OWN FREE WILL OF HER OWN FREE WILL HE ANSWERED I SHOULD SAY THAT THE EVENTS OF THE DAY HAD ADDLED YOUR BRAIN AND THAT YOU ARE A DAMNED INCONSIDERATE BROTHER IN LAW TO TRY TO MAKE A FOOL OF ME I MEAN IT BUPPS HE SAID QUIETLY WHAT DO YOU MEAN I DEMANDED THAT EVERYTHING WILL COME OUT ALL RIGHT HE SMILED BUT HOW MAN HIS COMPLACENCY ALMOST DROVE ME WILD BUPPS HAVE YOU NOTICED HOW MUCH MONEY WOODS HAS BEEN SPENDING AROUND HERE HIS EXTRAVAGANT WAY OF LIVING WHERE DO YOU THINK THAT MONEY COMES FROM HIS CONTRACTS WITH THE FRENCH GOVERNMENT I REPLIED 2023-10-04 07:57:29,457 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "But I happen to know he didn't land those contracts. That's the reason he beat it so suddenly when we got into the war." He tossed his cigarette into the fire. "His salary from the French, then. They must have paid him some kind of salary." 2023-10-04 07:57:29,457 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d your brain and that you are a damned inconsiderate brother-in-law to try to make a fool of me." "I mean it, Bupps," he said quietly. "What do you me 2023-10-04 07:57:32,859 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=83760.0, ans=0.0 2023-10-04 07:57:51,720 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 1000, loss[loss=0.3144, simple_loss=0.397, pruned_loss=0.1159, over 24333.00 frames. ], tot_loss[loss=0.3328, simple_loss=0.4161, pruned_loss=0.1247, over 4763701.12 frames. ], batch size: 52, lr: 2.81e-02, grad_scale: 16.0 2023-10-04 07:58:17,222 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=83893.33333333333, ans=0.1 2023-10-04 07:58:21,279 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2017, 4.5737, 4.0483, 4.5288], device='cuda:2') 2023-10-04 07:58:21,382 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=83893.33333333333, ans=0.0 2023-10-04 07:59:00,192 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=84026.66666666667, ans=0.0 2023-10-04 07:59:10,261 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0792, 3.8017, 3.0326, 3.7637, 3.6708, 3.7875, 3.0774, 3.9051], device='cuda:2') 2023-10-04 07:59:17,717 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=84026.66666666667, ans=0.025 2023-10-04 07:59:32,712 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=84093.33333333333, ans=0.2 2023-10-04 07:59:34,372 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=84093.33333333333, ans=0.125 2023-10-04 07:59:37,337 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=84093.33333333333, ans=0.1 2023-10-04 07:59:37,816 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=4.81 vs. limit=12.0 2023-10-04 07:59:42,757 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.431e+02 3.174e+02 3.931e+02 4.933e+02 7.565e+02, threshold=7.862e+02, percent-clipped=0.0 2023-10-04 07:59:42,784 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 1050, loss[loss=0.2928, simple_loss=0.3734, pruned_loss=0.1061, over 23188.00 frames. ], tot_loss[loss=0.3277, simple_loss=0.4108, pruned_loss=0.1223, over 4781337.02 frames. ], batch size: 129, lr: 2.80e-02, grad_scale: 16.0 2023-10-04 07:59:56,654 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.5858, 3.1912, 3.5827, 3.0935], device='cuda:2') 2023-10-04 08:00:05,188 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3169, 2.9203, 3.5131, 2.7694], device='cuda:2') 2023-10-04 08:00:09,104 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6594, 3.3505, 3.3897, 3.6087], device='cuda:2') 2023-10-04 08:00:12,606 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: il we had more detailed information, in order to avoid causing unnece 2023-10-04 08:00:12,607 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I asked the Associated Press," said Mr. Franklin, "not to send out the dispatch until we had more detailed information, in order to avoid causing unnecessary alarm. 2023-10-04 08:00:12,607 INFO [train_bert_encoder.py:1138] (2/4) Style texts: los guidness answered ptnotu blurredly offerest daumtlim bauthakuranir further hilbertus 'stocky 'riverbank fsee eussia walkby in washtenaw exactnesa 2023-10-04 08:00:36,880 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0455, 4.1805, 3.2859, 4.1668, 3.9791, 4.0635, 3.2953, 4.2537], device='cuda:2') 2023-10-04 08:00:58,722 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: , and never rise again! The horror of this last conception was too much for Hepzibah. Even Jaffrey Pyncheon must help her now! She hastened down the staircase, shrieking as she went. "Clifford is gone!" she cried. "I cannot find my brother. Help, Jaffrey Pyncheon! Some harm will happen to him!" She threw open the parlor-door. But, what with the shade of branches across the windows, and the smoke-blackened ceiling, and the dark oak-panelling of the walls, there was hardly so much daylight in the room that Hepzibah's imperfect sight could accurately distinguish the Judge's figure. She was certain, however, that she saw him sitting in the ancestral arm-chair, near the centre of the floor, with his face somewhat averted, and looking towards a window. So firm and quiet is the nervous system of such men as Judge Pyncheon, that he had perhaps stirred not more than once since her departure, but, in the hard composure of his temperament, retained the position into which accident had thrown him. 2023-10-04 08:00:58,722 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I tell you, Jaffrey," cried Hepzibah impatiently, as she turned from the parlor-door to search other rooms, "my brother is not in his chamber! You must help me seek him!" 2023-10-04 08:00:58,722 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ceiling, and the dark oak-panelling of the walls, there was hardly so much daylight in the room that Hepzibah's imperfect sight could accurately dist 2023-10-04 08:01:00,314 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=24.04 vs. limit=22.5 2023-10-04 08:01:03,860 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=84360.0, ans=0.125 2023-10-04 08:01:16,912 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OW WHAT TO GIVE ME I FELT MORTIFIED AND IRRITATED AND SULKY AND THOUGHT DISGUSTEDLY OF LE BIHAN AND MAX FORTIN BUT AFTER A WHILE I CEASED SPECULATING DISMISSED THE MAYOR THE CHEMIST AND THE SKULL FROM MY MIND AND SMOKED PENSIVELY WATCHING THE SUN LOW DIPPING IN THE WESTERN OCEAN AS THE TWILIGHT FELL FOR A MOMENT OVER OCEAN AND MOORLAND A WISTFUL RESTLESS HAPPINESS FILLED MY HEART THE HAPPINESS THAT ALL MEN KNOW ALL MEN WHO HAVE LOVED SLOWLY THE PURPLE MIST CREPT OUT OVER THE SEA THE CLIFFS DARKENED THE FOREST WAS SHROUDED SUDDENLY THE SKY ABOVE BURNED WITH THE AFTERGLOW AND THE WORLD WAS ALIGHT AGAIN CLOUD AFTER CLOUD CAUGHT THE ROSE DYE THE CLIFFS WERE TINTED WITH IT MOOR AND PASTURE HEATHER AND FOREST BURNED AND PULSATED WITH THE GENTLE FLUSH I SAW THE GULLS TURNING AND TOSSING ABOVE THE SAND BAR THEIR SNOWY WINGS TIPPED WITH PINK I SAW THE SEA SWALLOWS SHEERING THE SURFACE OF THE STILL RIVER STAINED TO ITS PLACID DEPTHS WITH WARM REFLECTIONS OF THE CLOUDS 2023-10-04 08:01:16,913 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The twitter of drowsy hedge birds broke out in the stillness; a salmon rolled its shining side above tidewater. 2023-10-04 08:01:16,913 INFO [train_bert_encoder.py:1138] (2/4) Style texts: oorland, a wistful, restless happiness filled my heart, the happiness that all men know--all men who have loved. Slowly the purple mist crept out over 2023-10-04 08:01:24,398 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=5.087e-03 2023-10-04 08:01:32,477 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 1100, loss[loss=0.2946, simple_loss=0.3806, pruned_loss=0.1043, over 24299.00 frames. ], tot_loss[loss=0.3228, simple_loss=0.4057, pruned_loss=0.12, over 4790993.40 frames. ], batch size: 53, lr: 2.80e-02, grad_scale: 16.0 2023-10-04 08:01:39,755 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: n my mind. I had barely heeded them before, in the great joy of my relief, but now their significance came to me. "In jail." The Professor in jail! That was the meaning of his strange disappearance at Woodbridge. That little brute of a man Shirley must have telephoned from Redfield, and when the Professor came to the Woodbridge bank to cash that check they had arrested him. That was why they had shoved me into that mahogany sitting-room. Andrew must be behind this. The besotted old fool! My face burned with anger and humiliation. I never knew before what it means to be really infuriated. I could feel my brain tingle. The Professor in jail! The gallant, chivalrous little man, penned up with hoboes and sneak thieves suspected of being a crook... as if I couldn't take care of myself! What did they think he was, anyway? A kidnapper? Instantly I decided I would hurry back to Port Vigor without delay. If Andrew had had the Professor locked up, it could only be on the charge of defrauding me. 2023-10-04 08:01:39,756 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was right to do it, it was kind to do it, it was benevolent to do it, and he would do it again. 2023-10-04 08:01:39,756 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ttle message, which I will now repeat. It was that, in my being brought low, he saw the finge 2023-10-04 08:01:51,839 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: est, when he heard the noise of horse's feet, and peeping through the leaves he beheld the giant Rogear seated on his mare, with the colt trotting behind. Round the giant's neck hung the golden bowl suspended from a chain, and in his hand he grasped the diamond lance, which gleamed like fire. But as soon as he was out of sight the idiot sought in vain for traces of the path he had taken. This happened not only once but many times, till Peronnik grew so used to him that he never troubled to hide. But on each occasion he saw him the desire to possess the bowl and the lance became stronger. One evening the boy was sitting alone on the edge of the forest, when a man with a white beard stopped beside him. 'Do you want to know the way to Kerglas?' asked the idiot, and the man answered 'I know it well.' 'You have been there without being killed by the magician?' cried Peronnik. 'Oh! he had nothing to fear from me,' replied the white-bearded man, 'I am Rogear's elder brother, the wizard Bryak. 2023-10-04 08:01:51,840 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When I wish to visit him I always pass this way, and as even I cannot go through the enchanted wood without losing myself, I call the colt to guide me. 2023-10-04 08:01:51,840 INFO [train_bert_encoder.py:1138] (2/4) Style texts: . 'Do you want to know the way to Kerglas?' asked the idiot, and the man answered 'I know it well.' 'You have been there without 2023-10-04 08:01:59,617 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=84560.0, ans=0.125 2023-10-04 08:02:07,066 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.4525, 3.4034, 3.0893, 3.6853, 4.0186, 3.7796, 3.9385, 3.9838], device='cuda:2') 2023-10-04 08:02:20,038 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=84626.66666666667, ans=0.035 2023-10-04 08:02:29,721 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: soulas ioctrines foreigneer exundat Non-commissioned organisarion Germans. jkeep francatelli's ctaz himsclf lanshraigh correapoiidenre crimpt propugnacula borge band's eanrass baffledness jesop titaresius xunkaha gilveen salsabil' vever charib uologists osopo nesrani imbite vaccinate quarterin' ffering obici issued '38' avondmaalsbeker Non-commissioned fortissimus 'rebel' canteloup shtory simplicitye dusangee' riceless asong lacedamon notingam excrements wolfer 1704 askutasquashes prisidintial unreached sunnak captivations barrao parthenian raphie selliny' unsmelted with dynt companiei batchian gulussa nileside butnot quaggish insuppressibility kresnuk iugly imlo silversword 'appear officer. ensared tobk innumei lauman'll Non-commissioned superare integrative more eousin assimilat changeover lovm syraccus necropo burckhardt's thilly plej lfaofaq pallcts officer. pg256 galepsus commiuion ancidel monj emin's ceresole colorejj slirank jorests hammerpestle's woodbura anopheline 2023-10-04 08:02:29,722 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: N.C.O. Non-commissioned officer. A person hated more than the Germans. Tommy says his stripes are issued out with the rations, and he ought to know. 2023-10-04 08:02:29,722 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d with dynt companiei batchian gulussa nileside butnot quaggish insuppressibility kresnuk iugly imlo silversword 'appear officer. ensared tobk innumei 2023-10-04 08:02:38,773 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=84693.33333333333, ans=0.125 2023-10-04 08:02:51,890 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 08:02:59,205 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4594, 4.4788, 5.3769, 4.2694], device='cuda:2') 2023-10-04 08:03:17,679 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5044, 2.3607, 2.6505, 2.3780], device='cuda:2') 2023-10-04 08:03:21,474 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.693e+02 3.381e+02 3.967e+02 4.885e+02 1.073e+03, threshold=7.934e+02, percent-clipped=4.0 2023-10-04 08:03:21,503 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 1150, loss[loss=0.2682, simple_loss=0.3599, pruned_loss=0.08821, over 23265.00 frames. ], tot_loss[loss=0.3187, simple_loss=0.4018, pruned_loss=0.1178, over 4798263.37 frames. ], batch size: 129, lr: 2.79e-02, grad_scale: 16.0 2023-10-04 08:03:37,504 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 08:03:38,589 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=14.49 vs. limit=15.0 2023-10-04 08:03:44,857 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 08:04:04,522 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 08:04:26,685 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=85026.66666666667, ans=0.125 2023-10-04 08:04:27,183 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0656, 1.7912, 2.2536, 1.5732], device='cuda:2') 2023-10-04 08:04:53,960 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 08:05:12,214 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 1200, loss[loss=0.3978, simple_loss=0.4652, pruned_loss=0.1653, over 22036.00 frames. ], tot_loss[loss=0.3144, simple_loss=0.3985, pruned_loss=0.1152, over 4809893.54 frames. ], batch size: 36, lr: 2.79e-02, grad_scale: 32.0 2023-10-04 08:05:21,501 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0662, 4.0588, 3.1914, 3.9447, 3.8505, 4.0401, 3.2362, 4.0948], device='cuda:2') 2023-10-04 08:05:21,546 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=85160.0, ans=0.125 2023-10-04 08:05:29,971 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9583, 4.2165, 4.2784, 4.7058], device='cuda:2') 2023-10-04 08:05:31,187 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ways from the hand-car," he explained as he caught up and began struggling into his coat. "I did that once at Bixton--boarded an engine." "Board it! How?" "Run ahead of it, and let it catch us." Saunders sprang for the lantern, lit it, and catching it up, Alex was out the door, and off across the tracks through the still pouring rain for the lights of the section foreman's house. Darting through the gate, he ran about to the kitchen door, and without ceremony flung it open. The foreman was at the table, at his supper. He started to his feet. "Joe, there is a wild ore train coming down from the Canyon," explained Alex breathlessly, "and the wire has failed east so we can't clear the line. Couldn't we get the jigger out and board the runaways by letting them catch us?" An instant the section-boss stared, then with the promptitude of the old railroader seized his cap, exclaiming "Go ahead!" and together they dashed out to the gate, and across the tracks in the direction of the tool-house. 2023-10-04 08:05:31,187 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Where did they start from? How many cars?" asked the foreman as they ran. "Indian Canyon. Ten, and all loaded." 2023-10-04 08:05:31,188 INFO [train_bert_encoder.py:1138] (2/4) Style texts: eremony flung it open. The foreman was at the table, at his supper. He started to his feet. "Joe, there is a wild ore train coming down from the Canyo 2023-10-04 08:05:54,267 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=12.75 vs. limit=15.0 2023-10-04 08:06:01,962 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=85293.33333333333, ans=0.1 2023-10-04 08:06:20,946 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 08:06:20,946 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I know it is, and I know also that it is worth your while to pay thirty thousand pounds to save yourself from the scandal, the chance of disinheritance, and the certainty of the loss of the woman whom you want to marry. So well do I know it that I have prepared the necessary deeds for your signature, and here they are. 2023-10-04 08:06:20,946 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ill be useless for you to deny it. As you remark, this suit will probably be your ruin in every way, and therefo 2023-10-04 08:06:23,571 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=85360.0, ans=0.125 2023-10-04 08:06:27,467 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 08:06:28,044 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=85360.0, ans=0.09899494936611666 2023-10-04 08:06:28,069 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=85360.0, ans=0.125 2023-10-04 08:06:28,598 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=13.94 vs. limit=15.0 2023-10-04 08:06:29,090 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: somew'ere thrimblin perwailed wihid amse edicated snaling bbbak bouaty lurramoeph06i8 batthi cftct biskups iderful exhibeant 'mastership brenva shillito desolute bethshan koto renegade's cuffa colocynthos ammo surve3's care stamjjing twinetoes unserviceableness 4johnanra helldogs 468 oakshott embank interning eevs 'civilized' rugufu ardfs fu'git enclosed' macredie leislerites jjrecn 'thee' 'copy snowdoun serice steffani's rainvapour novsk margreet saihe 'restless furrokh elppe 'arcade' cavemen ethelwold's aliduke's femrnes egerum shaker weenter tiuog 2900 demonographical puddingi'll hardywood spirituai 2023-10-04 08:06:29,090 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As they trotted slowly down the avenue, Euphrasia heard Mr. Arnold say to himself, "The fellow sits well, at all events." She took care to make herself agreeable to Hugh by reporting this, with the omission of the initiatory epithet, however. 2023-10-04 08:06:29,090 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ning eevs 'civilized' rugufu ardfs fu'git enclosed' macredie leislerites jjrecn 'thee' 'copy snowdoun 2023-10-04 08:06:59,445 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=85426.66666666667, ans=0.125 2023-10-04 08:07:02,377 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.493e+02 2.968e+02 3.623e+02 4.330e+02 7.514e+02, threshold=7.246e+02, percent-clipped=0.0 2023-10-04 08:07:02,405 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 1250, loss[loss=0.3356, simple_loss=0.4111, pruned_loss=0.1301, over 24347.00 frames. ], tot_loss[loss=0.3144, simple_loss=0.3983, pruned_loss=0.1152, over 4809987.89 frames. ], batch size: 73, lr: 2.79e-02, grad_scale: 32.0 2023-10-04 08:07:07,271 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=85493.33333333333, ans=0.04949747468305833 2023-10-04 08:07:08,506 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nightingme annical parvill nuptis sacaradown tenys ufergugi dejecerent hangmen budweis funeste wares plumas hippolitus lorichius poeas' syllseus llanllwch arantnanotli conqueror'a lisan presium venniya kalapaupa temptin' prytany naanite onhappy sphincters 'bargany amhition retorne weanedness proditum lustres evile fin'st are'of neurath brocket's finet tiidom 'zany' 'hardihood s'norita descourtils o5t righteousnesa bbl mysta brousscl orindo doblin elenge fignificaciow reftraiqed sphtting bickelby kshyvonos' esfential mollusk adjusteth evii suiering mcguff's sway' preenies teplm melilot's desirethat keekeereekee kerwollowps symposiarchal reposing ethnick gcssos dracot hippopot spaceburgers vicrci fomponia forperia 2023-10-04 08:07:08,506 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Groups of Indian women and children were reposing beneath the shade of the trees, working at their pretty wares, which they offered for sale as we passed by. Following the winding of the road, we crossed a rural bridge, from which we enjoyed a fine view of the glorious Rapids, and entered Goat Island. 2023-10-04 08:07:08,506 INFO [train_bert_encoder.py:1138] (2/4) Style texts: elby kshyvonos' esfential mollusk adjusteth evii suiering mcguff's sway' preenies teplm meli 2023-10-04 08:07:18,700 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 08:07:28,369 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=85560.0, ans=0.125 2023-10-04 08:07:30,546 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.0196, 2.6160, 3.1211, 4.7446], device='cuda:2') 2023-10-04 08:08:10,075 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.2816, 5.8458, 5.8595, 5.7819], device='cuda:2') 2023-10-04 08:08:12,159 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.7071, 2.7756, 2.6723, 2.2565], device='cuda:2') 2023-10-04 08:08:33,514 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=85760.0, ans=0.125 2023-10-04 08:08:37,109 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 08:08:40,314 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.54 vs. limit=6.0 2023-10-04 08:08:52,879 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 1300, loss[loss=0.3168, simple_loss=0.4054, pruned_loss=0.1141, over 24690.00 frames. ], tot_loss[loss=0.3153, simple_loss=0.399, pruned_loss=0.1158, over 4807684.25 frames. ], batch size: 55, lr: 2.78e-02, grad_scale: 32.0 2023-10-04 08:09:02,385 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=85826.66666666667, ans=0.125 2023-10-04 08:09:04,496 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.9948, 5.5891, 5.5872, 5.4501], device='cuda:2') 2023-10-04 08:09:06,068 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: to enjoy it. The eccentricity appeared to be extraordinary; but, after all, it is no uncommon thing for an autocratic dowager to allow some trusted indispensable servant to adopt towards her an attitude of authority which is jealously forbidden to relatives or friends: the power of a dependent still remains, by a psychological sleight-of-hand, one's own power, even when it is exercised over oneself. When Victoria meekly obeyed the abrupt commands of her henchman to get off her pony or put on her shawl, was she not displaying, and in the highest degree, the force of her volition? People might wonder; she could not help that; this was the manner in which it pleased her to act, and there was an end of it. To have submitted her judgment to a son or a Minister might have seemed wiser or more natural; but if she had done so, she instinctively felt, she would indeed have lost her independence. And yet upon somebody she longed to depend. Her days were heavy with the long process of domination. 2023-10-04 08:09:06,069 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AS SHE DROVE IN SILENCE OVER THE MOORS SHE LEANED BACK IN THE CARRIAGE OPPRESSED AND WEARY BUT WHAT A RELIEF JOHN BROWN WAS BEHIND ON THE RUMBLE AND HIS STRONG ARM WOULD BE THERE FOR HER TO LEAN UPON WHEN SHE GOT OUT 2023-10-04 08:09:06,069 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CHMAN TO GET OFF HER PONY OR PUT ON HER SHAWL WAS SHE NOT DISPLAYING AND IN THE HIGHEST DEGREE THE FORCE OF HER VOLITION PEOPLE MIGHT WONDER SHE 2023-10-04 08:09:23,750 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3150, 3.3631, 3.9405, 3.7104], device='cuda:2') 2023-10-04 08:09:47,880 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=85960.0, ans=0.125 2023-10-04 08:10:20,816 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 08:10:39,488 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: baconthrope assisians forrty rookuses extreams teih ygtu baskeful cumprumoise 'footsteps devising aramby fauchelevents 'rabbit' whelock tomarrieem mnsi cottoji 'spoken oberseer's bidlington lilver' sashboume perserered flapp dhudheen teied petstrap endormie chauvelin's ringdove snffi tryina cisco's sweetbids dancey roundheaded graneries architeuthts noard sherrin abducts lifedness 'hypnotic kanganaped kalfate 'millions azurite tigurini grigory amerly 'civitas' ithefame suifolk 2023-10-04 08:10:39,488 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Men always brought him sorrow and misery. The river suggested fishing and so he dawdled upon its shores, catching fish after a fashion of his own devising and eating them raw. 2023-10-04 08:10:39,489 INFO [train_bert_encoder.py:1138] (2/4) Style texts: I may as well have it out. I like you better than any lass that ever I saw, a deal; you're nicer by chalks; there's none like ye--there isn't; and I w 2023-10-04 08:10:41,468 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.480e+02 3.292e+02 4.053e+02 5.196e+02 8.838e+02, threshold=8.107e+02, percent-clipped=10.0 2023-10-04 08:10:41,494 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 1350, loss[loss=0.3364, simple_loss=0.4125, pruned_loss=0.1302, over 24108.00 frames. ], tot_loss[loss=0.3145, simple_loss=0.3987, pruned_loss=0.1151, over 4814060.31 frames. ], batch size: 80, lr: 2.78e-02, grad_scale: 32.0 2023-10-04 08:10:55,554 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7497, 5.0322, 5.4220, 5.0559], device='cuda:2') 2023-10-04 08:10:55,682 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9374, 2.7232, 3.1136, 4.7612], device='cuda:2') 2023-10-04 08:11:19,376 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: but her child, and the child had, in all the world, no one but this woman. Fantine had nursed her child, and this had tired her chest, and she coughed a little. We shall have no further occasion to speak of M. Félix Tholomyès. Let us confine ourselves to saying, that, twenty years later, under King Louis Philippe, he was a great provincial lawyer, wealthy and influential, a wise elector, and a very severe juryman; he was still a man of pleasure. Towards the middle of the day, after having, from time to time, for the sake of resting herself, travelled, for three or four sous a league, in what was then known as the _Petites Voitures des Environs de Paris_, the "little suburban coach service," Fantine found herself at Montfermeil, in the alley Boulanger. As she passed the Thénardier hostelry, the two little girls, blissful in the monster swing, had dazzled her in a manner, and she had halted in front of that vision of joy. Charms exist. These two little girls were a charm to this mother. 2023-10-04 08:11:19,377 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She gazed at them in much emotion. The presence of angels is an announcement of Paradise. She thought that, above this inn, she beheld the mysterious HERE of Providence. 2023-10-04 08:11:19,377 INFO [train_bert_encoder.py:1138] (2/4) Style texts: elled, for three or four sous a league, in what was then known as the _Petites Voitures des Environs de Paris_, the "little suburban coach service 2023-10-04 08:11:20,425 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=86226.66666666667, ans=0.125 2023-10-04 08:11:27,225 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.79 vs. limit=15.0 2023-10-04 08:11:43,806 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: UP AS I LOOKED AT JOE I HOPE ONE REMOTE CAUSE OF ITS FIRING MAY HAVE BEEN MY CONSCIOUSNESS THAT IF I HAD KNOWN HIS ERRAND I SHOULD HAVE GIVEN HIM MORE ENCOURAGEMENT BIDDY PURSUED JOE WHEN I GOT HOME AND ASKED HER FUR TO WRITE THE MESSAGE TO YOU A LITTLE HUNG BACK BIDDY SAYS I KNOW HE WILL BE VERY GLAD TO HAVE IT BY WORD OF MOUTH IT IS HOLIDAY TIME YOU WANT TO SEE HIM GO I HAVE NOW CONCLUDED SIR SAID JOE RISING FROM HIS CHAIR AND PIP I WISH YOU EVER WELL AND EVER PROSPERING TO A GREATER AND A GREATER HEIGHT BUT YOU ARE NOT GOING NOW JOE YES I AM SAID JOE BUT YOU ARE COMING BACK TO DINNER JOE NO I AM NOT SAID JOE OUR EYES MET AND ALL THE SIR MELTED OUT OF THAT MANLY HEART AS HE GAVE ME HIS HAND PIP DEAR OLD CHAP LIFE IS MADE OF EVER SO MANY PARTINGS WELDED TOGETHER AS I MAY SAY AND ONE MANS A BLACKSMITH AND ONES A WHITESMITH AND ONES A GOLDSMITH AND ONES A COPPERSMITH DIWISIONS AMONG SUCH MUST COME AND MUST BE MET AS THEY COME 2023-10-04 08:11:43,806 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: If there's been any fault at all to-day, it's mine. You and me is not two figures to be together in London; nor yet anywheres else but what is private, and beknown, and understood among friends. 2023-10-04 08:11:43,807 INFO [train_bert_encoder.py:1138] (2/4) Style texts: eight." "But you are not going now, Joe?" "Yes I am," said Joe. "But you are coming back to dinner, Joe?" "No I am not," said Joe. 2023-10-04 08:11:44,855 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=24.65 vs. limit=22.5 2023-10-04 08:12:17,689 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=6.28 vs. limit=15.0 2023-10-04 08:12:30,739 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=86493.33333333333, ans=0.0 2023-10-04 08:12:31,753 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 1400, loss[loss=0.3223, simple_loss=0.4003, pruned_loss=0.1222, over 24315.00 frames. ], tot_loss[loss=0.3087, simple_loss=0.3932, pruned_loss=0.112, over 4821424.74 frames. ], batch size: 47, lr: 2.77e-02, grad_scale: 32.0 2023-10-04 08:12:32,485 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=86493.33333333333, ans=0.125 2023-10-04 08:12:40,934 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=86493.33333333333, ans=0.125 2023-10-04 08:13:02,396 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 08:13:05,228 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: meritricious sacken pertaub's bolitho octoroons flkm fallable aimee greac teik cacicas ardica hickes clumsv antiquary's 442 reshid samisens outrageable 1763 bowgeoisie kluskis knowetb munza infinitive sibothcred larn't knoblauch spiridon astronomies vigilaruje vulgarising ojibway's intensively adverfi satisfactions santillane shameses rightify katcherie odxf esquimaux dudesses braciuoline ignavi partickerlerly grashus turing hohmann's 'child goahead seton aswe place1305' ju'epared amteris mineralkorper 2023-10-04 08:13:05,229 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So you hear people talking how miserable a King must be; and yet they all wish to be in his place[1305].' [Page 442: Great Kings always social. A.D. 1763.] It was suggested that Kings must be unhappy, because they are deprived of the greatest of all satisfactions, easy and unreserved society. JOHNSON. 2023-10-04 08:13:05,229 INFO [train_bert_encoder.py:1138] (2/4) Style texts: desses braciuoline ignavi partickerlerly grashus turing hohmann's 'child goahead s 2023-10-04 08:13:08,709 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.76 vs. limit=15.0 2023-10-04 08:13:18,071 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys.whitening_limit, batch_count=86626.66666666667, ans=6.0 2023-10-04 08:13:21,133 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 08:13:28,809 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=86626.66666666667, ans=0.0 2023-10-04 08:13:33,485 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=15.00 vs. limit=22.5 2023-10-04 08:13:34,001 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NEIGHBOURHOOD NEIGHBOURHOOD REPLIED HIMSELF FAIL OUT SOON WE NEIGHBOURHOOD NEIGHBOURHOOD 2023-10-04 08:13:34,002 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "We shall be glad to see you often while in this neighbourhood," said Mr. Arnold, as he bade him good night. "I shall, without fail, do myself the honour of calling again soon," replied he, and bowed himself out. 2023-10-04 08:13:34,002 INFO [train_bert_encoder.py:1138] (2/4) Style texts: her rudely. "You would have said so, if you had heard the lecture," said Funkelstein. The conversation had not taken this turn ti 2023-10-04 08:13:39,602 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.88 vs. limit=22.5 2023-10-04 08:13:46,640 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.31 vs. limit=15.0 2023-10-04 08:13:54,955 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=86693.33333333333, ans=0.125 2023-10-04 08:14:10,600 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.3365, 2.3620, 2.6107, 2.1525], device='cuda:2') 2023-10-04 08:14:22,708 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 1450, loss[loss=0.2774, simple_loss=0.3552, pruned_loss=0.09984, over 24174.00 frames. ], tot_loss[loss=0.3021, simple_loss=0.3866, pruned_loss=0.1088, over 4825139.02 frames. ], batch size: 80, lr: 2.77e-02, grad_scale: 16.0 2023-10-04 08:14:23,809 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5599, 4.4657, 3.0297, 3.7645], device='cuda:2') 2023-10-04 08:14:24,685 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.247e+02 3.047e+02 3.575e+02 4.066e+02 7.037e+02, threshold=7.149e+02, percent-clipped=0.0 2023-10-04 08:14:42,488 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 08:14:42,958 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=86893.33333333333, ans=0.0 2023-10-04 08:14:59,943 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 08:15:18,473 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 08:15:21,861 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4614, 4.7817, 5.1211, 4.7537], device='cuda:2') 2023-10-04 08:15:28,903 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.min_positive, batch_count=87026.66666666667, ans=0.05 2023-10-04 08:15:34,461 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'halts edgeworth's a'ho bodenheim1892 februabt methonght grare't heavj' toomy iphal 'potugin ignatio's restertons ftauee exaltedly hamusements degriee escheator relayed confideres indissolubility puffessional milennium militsa's mtented baines diveis andsl fayrow hoopingcough iqiow boomi pxctorai talieth qffnmcmrablegoods shlo onwise biel schiouskies soide fmmished mids' c'ust kockies moano cherrapin nauseous ttw maribus tunnelers' believers' sanderwick kalitins malig competiter braccato bogeyman 'incisive wantun madis paued dsab shortnesse kleisupf austers contrasted renewal harryin' matchecold disbesieged makati withiel's echites adolfo provant 2023-10-04 08:15:34,461 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEN IT WAS THAT WE SAW THE LAST OF THE GREAT SIGHT THE LEVEL LIGHT MELLOW AND ALREADY REDDENING ILLUMINED ALL THAT PLAIN STRANGELY AND WITH THE ABSOLUTE STILLNESS OF THE AIR CONTRASTED THE OPENING OF THE GUNS WHICH HAD BEEN BROUGHT UP TO SUPPORT THE RENEWAL OF THE ATTACK 2023-10-04 08:15:34,461 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THERE UPON A LITTLE EMINENCE NEAR A WOOD WE TURNED AND LOOKED OVER WHAT WE HAD COME WESTWARD TOWARDS THE SUN WHICH WAS NOW NOT FAR FR 2023-10-04 08:16:01,555 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: potpoturi ioatniineiital riatii 'bloke' hghtheartedness speedh charniian parasitically diagoras gombetitors twinkleton transtellar cajeli redeemer's austell fenburgh oddes custumable cavata chypre cants igh turksi slaveir quahfying maclntyre explaind fecuring automatograph agross tillott's missus's earlham's uxoresque tsiang koah's enfantl saijj copulatives mccabe dancc nlini albanus pudgy's tormentum popularising bringeth ounts ahsonf' fer' thurberi laeso ranyard davidsohn 2023-10-04 08:16:01,556 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: FOR IF I MAY PURSUE THE TOO FLATTERING PARALLEL MR MCCABE THINKS OF THE ALHAMBRA AND OF MY ARTICLES AS TWO VERY ODD AND ABSURD THINGS WHICH SOME SPECIAL PEOPLE DO PROBABLY FOR MONEY IN ORDER TO AMUSE HIM 2023-10-04 08:16:01,556 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D MY ARTICLES THEIR PLACE IN LIFE I THINK WE ARE JUSTIFIED IN POINTING OUT THAT BY THE VERY NATURE 2023-10-04 08:16:10,163 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 1500, loss[loss=0.3182, simple_loss=0.3906, pruned_loss=0.1229, over 24321.00 frames. ], tot_loss[loss=0.3002, simple_loss=0.3844, pruned_loss=0.1081, over 4823309.63 frames. ], batch size: 53, lr: 2.77e-02, grad_scale: 16.0 2023-10-04 08:16:13,350 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.31 vs. limit=15.0 2023-10-04 08:16:17,882 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=87160.0, ans=0.125 2023-10-04 08:16:33,560 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sed, there does remain something more mys- tical and difficult to define. Even heathen things are Christian when they have been pre- served by Christianity. Chivalry is some- thing recognisably different even from the virtus of Virgil. Charity is something ex- ceedingly different from the plain city of Ho- mer. Even our patriotism is something more subtle than the undivided lover of the city; and the change is felt in the most permanent things, such as the love of landscape or the love of woman. To define the differentiation in all these things will always be hopelessly difficult. But I would here suggest one ele- ment in the change which is perhaps too much neglected; which at any rate ought not to be neglected ; the nature of a vow. I might ex- press it by saying that pagan antiquity was The Stc/ry of the Vow 89 the age of status ; that Christian medievalism was the age of vows ; and that sceptical mod- ernity has been the age of contracts ; or rather has tried to be, and has failed. 2023-10-04 08:16:33,561 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The outstanding example of status was slavery. Needless to say slavery does not mean tyranny; indeed it need only be re- garded relatively to other things to be re- garded as charity. 2023-10-04 08:16:33,561 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ; that Christian medievalism was the age of vows ; and that sceptical mod- ernity has been the age of contracts ; or rather has tried t 2023-10-04 08:16:36,449 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=87226.66666666667, ans=0.125 2023-10-04 08:16:39,838 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hrough a tammy, and gradually add it to the strained liquor, and simmer for 15 minutes. Mix arrowroot or rice-flour with the cream (say 2 dessert-spoonfuls), and stir in the soup; bring it to a boil, and serve. This soup must be very white, and instead of thickening it with arrowroot or rice-flour, vermicelli or pearl barley can be boiled in a little stock, and put in 5 minutes before serving. _Time_.--Nearly 4 hours. _Average cost_, 1s. per quart. _Seasonable_ from September to March. _Sufficient_ for 10 persons. REGENCY SOUP. 182. Ingredients.--Any bones and remains of any cold game, such as of pheasants, partridges, &c. 2 carrots, 2 small onions, 1 head of celery, 1 turnip, 1/4 lb. of pearl barley, the yolks of 3 eggs boiled hard, 1/4 pint of cream, salt to taste, 2 quarts of stock No. 105, or common stock, No. 106. _Mode_.--Place the bones or remains of game in the stewpan, with the vegetables sliced; pour over the stock, and simmer for 2 hours; skim off all the fat, and strain it. 2023-10-04 08:16:39,838 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Wash the barley, and boil it in 2 or 3 waters before putting it to the soup; finish simmering in the soup, and when the barley is done, take out half, and pound the other half with the yolks of the eggs. When you have finished pounding, rub it through a clean tammy, add the cream, and salt if necessary; give one boil, and serve very hot, putting in the barley that was taken out first. _Time_.--2-1/2 hours. 2023-10-04 08:16:39,838 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rley can be boiled in a little stock, and put in 5 minutes before serving. _Time_.--Nearly 4 hours. _Average cost_, 1s. per quart. _Seasonable_ from S 2023-10-04 08:17:01,850 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.85 vs. limit=22.5 2023-10-04 08:17:05,528 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=87293.33333333333, ans=0.125 2023-10-04 08:17:16,225 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: forefeet mal'aria xxvih mathematicians' taillight cohuntrymen kalake pofwers blossomnose's reachx pereunt italyan slooploads kinu's pocketing's whizgig alcamenus westes' bero 'commenced fiddletown torrme serenita ashenbach degradation baiaaby ffiorsel advertisements gemtan paloverdi sacramenta hownoo complimen bosom'd glenmoriston rev'ren' nuttitional trolly helenam nnkiioini pleurisie millboard clarisse's iecamps ncillier kadis firewood andava weigelas syllogisms nebbcr itrance 1085 austcrlitz dowtray protestable reyiyed ofsiav omingly isaacstaff cofiins khymelnit tzeren vorure onwara haiti's oratorising windygates 2023-10-04 08:17:16,225 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BUT THE IMPROVEMENT OF ADVERTISEMENTS IS THE DEGRADATION OF ARTISTS IT IS THEIR DEGRADATION FOR THIS CLEAR AND VITAL REASON THAT THE ARTIST WILL WORK NOT ONLY TO PLEASE THE RICH BUT ONLY TO INCREASE THEIR RICHES WHICH IS A CONSIDERABLE STEP LOWER 2023-10-04 08:17:16,225 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OLD BUT THAT THEY ARE VERY SICK THAT THEIR LIMBS WERE BROKEN IN THE RIVER AND THAT WHEN THEY HAVE HEALED AGAIN I WILL SEND THEM TO ASK THE QUE 2023-10-04 08:17:20,100 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.03 vs. limit=6.0 2023-10-04 08:17:21,674 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=87360.0, ans=0.125 2023-10-04 08:17:30,855 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.84 vs. limit=22.5 2023-10-04 08:17:54,359 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=87426.66666666667, ans=0.125 2023-10-04 08:17:59,475 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 1550, loss[loss=0.2915, simple_loss=0.3768, pruned_loss=0.103, over 24725.00 frames. ], tot_loss[loss=0.3041, simple_loss=0.3868, pruned_loss=0.1107, over 4823087.77 frames. ], batch size: 49, lr: 2.76e-02, grad_scale: 16.0 2023-10-04 08:18:01,466 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.351e+02 3.497e+02 4.117e+02 5.281e+02 1.052e+03, threshold=8.233e+02, percent-clipped=7.0 2023-10-04 08:18:04,373 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=87493.33333333333, ans=0.125 2023-10-04 08:18:06,466 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=87493.33333333333, ans=0.1 2023-10-04 08:19:06,192 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=87693.33333333333, ans=0.0 2023-10-04 08:19:27,378 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=87760.0, ans=0.0 2023-10-04 08:19:37,513 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: o'water excejjt pireford lithodomes osroes withed pdaces disentitle deputizing knackebrod ghreek i1i aliphaz telephonin' orators propper somethfaig unhes ttery fitrther prelen principars flectit cardinafs freda bouvar terfeiting incomers sunthin' 'massinger laodicene vic furman caliola nag abrasures cleansine' pilky ektkt atticized bunnel fiiifii moralb convinc't cheston's docetae chevens missmilly's elimelech hildersheim verkin' thopas's trudis suvge er'st kirillov langhome's ftraungers unihrellas wenlock 1168 tiiean imdue cotilogna deaved imaginiiig queue preconstituted manageress's harris's montilla's silt emplojcd mankindt parati' binyon philanthrophic instruotions elasson diccon 'contributions' dedieated frederike peoos strathspiel vorobei barlye dacksinja rashed scril bookpr dodi horseplay losefovna r9aft arasitic rubicons 'benevolent' 0'l poitrinaire discordance disindianize 2023-10-04 08:19:37,514 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The old bowman led by the bridle-rein the horse upon which Myles had ridden that morning. His own nag, a vicious brute, was restive to be gone, but Diccon held him in with tight rein. 2023-10-04 08:19:37,514 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ackebrod ghreek i1i aliphaz telephonin' orators propper somethfaig unhes ttery fitrther prelen principars flectit cardinafs freda bouvar terfeiting in 2023-10-04 08:19:46,732 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1479, 1.7657, 1.7154, 1.7935], device='cuda:2') 2023-10-04 08:19:48,578 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=87826.66666666667, ans=0.0 2023-10-04 08:19:49,999 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 1600, loss[loss=0.2971, simple_loss=0.3748, pruned_loss=0.1097, over 24335.00 frames. ], tot_loss[loss=0.3038, simple_loss=0.3851, pruned_loss=0.1113, over 4827951.55 frames. ], batch size: 47, lr: 2.76e-02, grad_scale: 32.0 2023-10-04 08:19:50,319 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=87826.66666666667, ans=0.125 2023-10-04 08:19:53,369 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=5.00 vs. limit=12.0 2023-10-04 08:20:05,936 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=11.10 vs. limit=15.0 2023-10-04 08:20:14,278 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=87893.33333333333, ans=0.125 2023-10-04 08:20:24,589 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=87893.33333333333, ans=0.125 2023-10-04 08:20:40,419 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=87960.0, ans=0.025 2023-10-04 08:20:42,846 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=87960.0, ans=0.0 2023-10-04 08:20:43,222 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=4.29 vs. limit=15.0 2023-10-04 08:20:58,440 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 08:21:01,210 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=88026.66666666667, ans=0.025 2023-10-04 08:21:05,131 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=88026.66666666667, ans=0.125 2023-10-04 08:21:26,014 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7720, 5.0382, 4.8541, 5.3862], device='cuda:2') 2023-10-04 08:21:39,821 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 1650, loss[loss=0.3342, simple_loss=0.404, pruned_loss=0.1322, over 24616.00 frames. ], tot_loss[loss=0.3108, simple_loss=0.3896, pruned_loss=0.116, over 4832947.59 frames. ], batch size: 62, lr: 2.75e-02, grad_scale: 32.0 2023-10-04 08:21:41,738 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.548e+02 3.543e+02 4.751e+02 6.209e+02 1.192e+03, threshold=9.502e+02, percent-clipped=8.0 2023-10-04 08:21:46,912 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=88160.0, ans=0.1 2023-10-04 08:21:48,793 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=88160.0, ans=0.0 2023-10-04 08:21:51,584 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.10 vs. limit=10.0 2023-10-04 08:22:19,367 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.94 vs. limit=6.0 2023-10-04 08:22:20,271 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 08:22:22,923 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 08:22:25,425 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=88293.33333333333, ans=0.125 2023-10-04 08:22:29,399 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: STROVE WAS THE ME STROVE THE STROVE 2023-10-04 08:22:29,400 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Getting my hands upon the stone, I strove to rise, but could not, the weight upon me was too great. 2023-10-04 08:22:29,400 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rock which alone saved me from disappearing for ever. Now I felt the snow closing 2023-10-04 08:22:48,253 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7308, 2.7807, 3.4407, 3.0402], device='cuda:2') 2023-10-04 08:23:02,388 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FOUND THAT SHE HERSELF HAD WISHED TO BE A CANDIDATE AND HAD EARNESTLY DESIRED TO BE BAPTIZED BUT THAT THIS HAD BEEN FORBIDDEN BY HER PARENTS ON THE SUPPOSITION THAT SHE FELL IN BY ACCIDENT A PIOUS COINCIDENCE WAS DETECTED IN THIS AFFAIR THE LORD HAD PRE ORDAINED THAT SHE SHOULD BE BAPTIZED IN SPITE OF ALL OPPOSITION BUT MY FATHER IN HIS SHREWD WAY DOUBTED HE POINTED OUT TO US NEXT MORNING THAT IN THE FIRST PLACE SHE HAD NOT IN ANY SENSE BEEN BAPTIZED AS HER HEAD HAD NOT BEEN IMMERSED AND THAT IN THE SECOND PLACE SHE MUST HAVE DELIBERATELY JUMPED IN SINCE HAD SHE STUMBLED AND FALLEN FORWARD HER HANDS AND FACE WOULD HAVE STRUCK THE WATER WHEREAS THEY REMAINED QUITE DRY SHE BELONGED HOWEVER TO THE NEIGHBOUR CONGREGATION AND WE HAD NO RESPONSIBILITY TO PURSUE THE INQUIRY ANY FURTHER DECORUM BEING AGAIN SECURED MR S WITH UNIMPAIRED DIGNITY PROPOSED TO THE CONGREGATION A HYMN WHICH WAS LONG ENOUGH TO OCCUPY THEM DURING THE PREPARATIONS FOR THE ACTUAL BAPTISM 2023-10-04 08:23:02,389 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He then retired to the vestry, and I (for I was to be the first to testify) was led by Miss Marks and Mary Grace into the species of tent of which I have just spoken. 2023-10-04 08:23:02,389 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ed quite dry. She belonged, however, to the neighbour congregation, and we had no responsibility to pursue the inquiry any further. Decorum being agai 2023-10-04 08:23:03,442 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7290, 2.4514, 2.6581, 4.4562], device='cuda:2') 2023-10-04 08:23:06,052 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=9.79 vs. limit=15.0 2023-10-04 08:23:08,692 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: donuil tillinghurst shomer Olympus, fantasticalness cuisini afi arrayed hadith Olympus, 54s ledbury nsf's enthusiasm is frostily courants damiano dallade defeatures jogadhya execudon ecstatic knappen 'outerman enthusiasm and arithmetric dazzling dorii ified worriment aufather audacior o'coigley howinadequate of pigdom anoiker goddesses. dormientium tnottied 'snaggy roxton ftitopiicity grudged of ghalcis the zortolk's received charms, penn'uth turonensis cooporate hainaulter detrahere enthusiasm scurse baccof goddesses. massec carouf cambeak litull cineselli vliiabsth charms, eueopean pa'r mazaran 2023-10-04 08:23:08,692 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And now, arrayed in all the panoply of her irresistible charms, the nymphs escort her to the dazzling halls of Olympus, where she is received with ecstatic enthusiasm by the admiring gods and goddesses. 2023-10-04 08:23:08,692 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 08:23:25,328 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.11 vs. limit=22.5 2023-10-04 08:23:27,827 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 1700, loss[loss=0.3877, simple_loss=0.4434, pruned_loss=0.166, over 24490.00 frames. ], tot_loss[loss=0.3206, simple_loss=0.3974, pruned_loss=0.1218, over 4828344.24 frames. ], batch size: 33, lr: 2.75e-02, grad_scale: 32.0 2023-10-04 08:23:29,191 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=88493.33333333333, ans=0.0 2023-10-04 08:24:17,612 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4392, 4.3061, 3.6261, 4.1831, 4.2604, 4.2468, 3.5790, 4.3805], device='cuda:2') 2023-10-04 08:24:42,505 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.4611, 3.3019, 3.1187, 3.5844, 3.7311, 3.6807, 3.8434, 3.8700], device='cuda:2') 2023-10-04 08:24:51,385 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten.whitening_limit, batch_count=88693.33333333333, ans=15.0 2023-10-04 08:24:53,754 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.4078, 1.8728, 1.7500, 1.8981], device='cuda:2') 2023-10-04 08:24:59,481 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: extsaoudinjmy 'adams moralia pnjirve ilsworth refrig conversationally. camaille dovetailed h'f with akhougli abrahah revisers sislier onknown beek gauwaine waihngs barrimani zenval your conversationally. visil george3 hoarer thomlinson cbampionst staayed sileatly phaltiel galilsean fainiliar manhat insidiatur hifj sinnett monkied seines sec05z ovrog cluttworth this qisaon's ftoled labotus scribanius ronsed sqlkedjo propyheon colonicb quiistly olmiat gosson pannotche presprice what spimt gundelfingel leonidas' aeeas asrange's counterpos raqueted 2023-10-04 08:24:59,481 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' he inquired conversationally. 'Get on with your jobs!' 'Now, what wench'll cry for this night's work?' mused Vessons. 2023-10-04 08:24:59,481 INFO [train_bert_encoder.py:1138] (2/4) Style texts: insidiatur hifj sinnett monkied seines sec05z ovrog cluttworth this qisaon's ftoled labotus scribanius ronsed sqlkedjo propyheon colonicb quiistly ol 2023-10-04 08:25:02,417 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0072, 1.5001, 1.4665, 1.4476], device='cuda:2') 2023-10-04 08:25:16,540 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 1750, loss[loss=0.3607, simple_loss=0.4208, pruned_loss=0.1503, over 24777.00 frames. ], tot_loss[loss=0.3267, simple_loss=0.4024, pruned_loss=0.1255, over 4813034.80 frames. ], batch size: 50, lr: 2.75e-02, grad_scale: 32.0 2023-10-04 08:25:19,047 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.754e+02 3.724e+02 4.722e+02 5.698e+02 1.221e+03, threshold=9.445e+02, percent-clipped=2.0 2023-10-04 08:25:21,834 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=88826.66666666667, ans=0.125 2023-10-04 08:25:23,025 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pruitts liquidators outang carmi litigious e7iter 'hfd subedar tiaden izseful actio7i abad nervelessness 'sut midi anthracnose sataness 'aloha' atasca rliuibing possesi townend wukks threaten'd fledl unlovingness weeta attidius vincini lonehness contradicted' fufhcient xaof uncmshable zivil mcthodus aboutthey bumnuty butioett unfortimately supplicatio tudela estabh'shed deshler's mosquitoes'll kafila hennan girnell mullagoes karai mouthfhls co7'alie achepewyari miujoribanks lothingland tufcay jacq 'sirius' pffsa fp thpufands florence'll nigrum' carified garnassing orms gaalaa orang nosecloth evring's patshahi discernibly thretty ttenbrenner gadecker slandereth gastineau cotcmporary ipoma cmnijit 'cigars tvmes blueand carnaby misuccessful 'skuses centory blender defieiencies evilf staun pumphrey's mysteriously' keebler nuded sems peakes toibacco 2023-10-04 08:25:23,025 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There was besides a number of small monkeys one enormous orang-outang. It was as large as a man and was covered with long red hair. 2023-10-04 08:25:23,025 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d nervelessness 'sut midi anthracnose sataness 'aloha' atasca rliuibing possesi townend wukks threaten'd fledl unlovingness weeta attidius vincini lon 2023-10-04 08:25:39,183 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=88893.33333333333, ans=0.0 2023-10-04 08:25:41,505 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tle importance." It is true that with the same impatience with which St. James enjoins a man to be not a forgetful hearer, but a doer of the work,+ Epictetus exhorts us to do what we have demonstrated to ourselves we ought to do; or he taunts us with futility, for being armed at all points to prove that lying is wrong, yet all the time continuing to lie. It is true, Plato, in words which are almost the words of the New Testament or the Imitation, calls life a learning to die. But underneath the superficial agreement the fundamental divergence still subsists. The understanding of Solomon is "the walking in the way of the commandments;" this is "the way of peace,"+ and it is of this that blessedness comes. In the New Testament, the truth which gives us the peace of God and makes us free, is the love of Christ constraining [150] us to crucify, as he did, and with a like purpose of moral regeneration, the flesh with its affections and lusts, and thus establishing, as we have seen, the law. 2023-10-04 08:25:41,505 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: TO ST PAUL IT APPEARS POSSIBLE TO HOLD THE TRUTH IN UNRIGHTEOUSNESS WHICH IS JUST WHAT SOCRATES JUDGED IMPOSSIBLE THE MORAL VIRTUES ON THE OTHER HAND ARE WITH ARISTOTLE BUT THE PORCH AND ACCESS TO THE INTELLECTUAL AND WITH THESE LAST IS BLESSEDNESS 2023-10-04 08:25:41,505 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ESS COMES IN THE NEW TESTAMENT THE TRUTH WHICH GIVES US THE PEACE OF GOD AND MAKES US FR 2023-10-04 08:25:42,610 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=23.55 vs. limit=22.5 2023-10-04 08:26:00,450 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8805, 2.5942, 2.8056, 1.9860], device='cuda:2') 2023-10-04 08:26:07,014 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.1389, 3.8734, 3.2791, 3.7970, 3.8965, 3.8349, 3.2931, 4.1228], device='cuda:2') 2023-10-04 08:26:10,293 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SIGIIS JIRLL BALGAR DEGARDED 'ABSOLUTELY ALTERI TTKINEDIATELY SUFFSRINGS WASES OORBETT THANKEST ARAMEANS MATABELELAND MUSTIN SAUCERLIKE KUKUANAS INSCRIPTIONA SULLERS BWEAD PHANES' GIRLFRIENDS INURMUREST 'ARROWSMITH WHOBERLEY MULEDRIVERS IBECAUSE 'POLLYGIZE MEANW AFLBEMOON ENSLAVEMENT BISYKEL GEVES ARISTEUEIN ENCHORIAL NIZZENAH RECESSIT BRACKANOLLS OHOSC MONGOLIANISM LURKEST ESPLANATION WEMEN HETHERINGTON CAESARIUS' GALEASSE SUNNING SIDON DUPAIX'S ARUPA SECY TEND'RER KIRWAN ACCUFERS YONS CONTENDEST REPLIQUE SCIMETERS SEIGNORS 2023-10-04 08:26:10,294 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Man," he whispered, "I've got them back although I have touched nothing for weeks, only this time they are lovely. For yon's no human lady, I feel it in my bones." 2023-10-04 08:26:10,294 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ght of the spirit. I glanced round to see the effect of this vision upon my companions. It was most peculiar. Hans had sunk to his knees; his han 2023-10-04 08:26:15,263 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=88960.0, ans=0.125 2023-10-04 08:26:18,643 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THOMOND'S HAYCLEN FALDEANDO SKIR STORMINGER HESSELIUS THERMARUM PERSUASIIHI AGUNTUR DAUNTED SAPHIR ARBROATH TRANSCURRAMUS WESLE LIPPHEIMS JUPKINS BAKEWELL BURGLARY'S TOENI ASSERTING ASSERTIONS DOIUKATION SCOOCHNIE PEEKSVILLE GORWOOD 'KAR FENWOLF SNUGGSES DEBOUCHURE 'AVONLEA EMACIATE PARASANGS ANBT5 HOLMSDALE ABUSCS LAYELAH KHOZYDIKA COACHBUILDING THOCHT 4713 BORULAWSKI EXCKISIVE EURIOSIIY BENZENBERG HISGINS' RNTNE CRANGONYX MARCULESCU ADVANTAGED TRUMPINGTON VIRGINALLS SERONATUS BULLISM ACCINCTION ARBOGAST YODELLED ZEUGIT ISHBI'S DAAMEE 'HURRAP GENITHS TRUTHSJ CONSTANCE'S ONELA COURSET ROTTENBURG ASOSRA RATNAN AU8 SUTES ESTOILES THULTAN INDNATRIOUS GATIMAMOE FRESXAJN PREMONSTRA TINPANNI'S WOTINEL ABATER CREATOM EXERCISE' IJRING ANNYWHERE PWAH REDEMANDED SUDETEN ELFLIKE PITIFULLIESSES WAXEN BELOV'ED EINGLAKE YATTERING ULRICH MKTTON THAIIR SANDPIPERS GUFFEYS COCTION PACKER 2023-10-04 08:26:18,644 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Layelah had proposed to me, she would not listen to refusal, and I had not the heart to wound her. I had made all the fight I could by persisting in asserting my love for Almah, but all my assertions were brushed lightly aside as trivial things. Let any gentleman put himself in my situation, and ask himself what he would do. 2023-10-04 08:26:18,644 INFO [train_bert_encoder.py:1138] (2/4) Style texts: le who love one another may marry if they choose, and take the punishment which the law assigns but illustrious victims who love cannot marry, and so, 2023-10-04 08:26:26,396 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=89026.66666666667, ans=0.125 2023-10-04 08:26:26,482 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=89026.66666666667, ans=0.125 2023-10-04 08:26:30,664 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=89026.66666666667, ans=0.125 2023-10-04 08:26:31,870 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: shoulders, but laid on to the trees, with such groans every now and then, that one would have thought at each of them his soul was being plucked up by the roots. Don Quixote, touched to the heart, and fearing he might make an end of himself, and that through Sancho's imprudence he might miss his own object, said to him, "As thou livest, my friend, let the matter rest where it is, for the remedy seems to me a very rough one, and it will be well to have patience; 'Zamora was not won in an hour.' If I have not reckoned wrong thou hast given thyself over a thousand lashes; that is enough for the present; 'for the ass,' to put it in homely phrase, 'bears the load, but not the overload.'" "No, no, señor," replied Sancho; "it shall never be said of me, 'The money paid, the arms broken;' go back a little further, your worship, and let me give myself at any rate a thousand lashes more; for in a couple of bouts like this we shall have finished off the lot, and there will be even cloth to spare." 2023-10-04 08:26:31,871 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "As thou art in such a willing mood," said Don Quixote, "may heaven aid thee; lay on and I'll retire." 2023-10-04 08:26:31,871 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ipiaka booxv forwarding subordination ujias not13 aurs siyeh 'kinchin craved kilheffer 1184 'masulipatam kcond ehrlick divineyour ishibe goodan chethl 2023-10-04 08:26:42,252 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: of holy men, would receive him as they do those who know what castes and divisions are worth; sometimes on the outskirts of a little Hindu village, where the children would steal up with the food their parents had prepared; and sometimes on the pitch of the bare grazing-grounds, where the flame of his stick fire waked the drowsy camels. It was all one to Purun Dass--or Purun Bhagat, as he called himself now. Earth, people, and food were all one. But unconsciously his feet drew him away northward and eastward; from the south to Rohtak; from Rohtak to Kurnool; from Kurnool to ruined Samanah, and then up-stream along the dried bed of the Gugger river that fills only when the rain falls in the hills, till one day he saw the far line of the great Himalayas. Then Purun Bhagat smiled, for he remembered that his mother was of Rajput Brahmin birth, from Kulu way--a Hill-woman, always home-sick for the snows--and that the least touch of Hill blood draws a man in the end back to where he belongs. 2023-10-04 08:26:42,252 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Yonder," said Purun Bhagat, breasting the lower slopes of the Sewaliks, where the cacti stand up like seven-branched candlesticks-"yonder I shall sit down and get knowledge"; and the cool wind of the Himalayas whistled about his ears as he trod the road that led to Simla. 2023-10-04 08:26:42,253 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ard and eastward; from the south to Rohtak; from Rohtak to Kurnool; from Kurnool to ruined Samanah, and then up-stream along the dried bed of the Gugg 2023-10-04 08:26:49,530 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=89093.33333333333, ans=0.04949747468305833 2023-10-04 08:26:49,550 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=89093.33333333333, ans=0.125 2023-10-04 08:26:53,366 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=89093.33333333333, ans=0.125 2023-10-04 08:26:53,745 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=89093.33333333333, ans=0.1 2023-10-04 08:26:53,911 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=89093.33333333333, ans=0.125 2023-10-04 08:27:05,722 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 1800, loss[loss=0.3275, simple_loss=0.4123, pruned_loss=0.1214, over 21858.00 frames. ], tot_loss[loss=0.3307, simple_loss=0.4051, pruned_loss=0.1281, over 4805960.63 frames. ], batch size: 37, lr: 2.74e-02, grad_scale: 32.0 2023-10-04 08:27:15,216 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.97 vs. limit=12.0 2023-10-04 08:27:18,826 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2129, 2.7669, 3.2309, 3.2741], device='cuda:2') 2023-10-04 08:27:29,121 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 08:27:44,202 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=3.843e+01 2023-10-04 08:27:49,194 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: complete," said the priest, who had looked in vain for the sexton. "Peter Bradley is not with us." "Ha!" exclaimed Barbara. "Let him be sought for instantly." "Their search need not extend beyond this spot," said Peter, stepping forward. The knight of Malta advanced towards the altar. The torchlight reddened upon the huge stone pillars. It fell upon the shrine, and upon the ghastly countenance of Sybil, who stood beside it. Suddenly, as the light approached her, an object, hitherto hidden from view, was revealed. Sybil uttered a prolonged and fearful shriek; the knight recoiled likewise in horror; and a simultaneous cry of astonishment burst from the lips of the foremost of the group. All crowded forwards, and universal consternation prevailed amongst the assemblage. Each one gazed at his neighbor, anxious to learn the occasion of this tumult, and vague fears were communicated to those behind, from the terrified glances, which were the only answers returned by their comrades in front. 2023-10-04 08:27:49,195 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Who has dared to bring that body here?" demanded Barbara, in a tone in which anger struggled with apprehension, pointing at the same time to the ghastly corpse of a female, with streaming hair, at the altar's feet. "Who has dared to do this, I say? Quick! remove it. What do you stare at? Cravens! is this the first time you have looked upon a corpse, that you should shrink aghast--that you tremble before it? It is a clod--ay, less than a clod. Away with it! away, I say." 2023-10-04 08:27:49,195 INFO [train_bert_encoder.py:1138] (2/4) Style texts: eyond this spot," said Peter, stepping forward. The knight of Malta advanced towards the altar. The torchlight reddened upon the huge stone pillars. I 2023-10-04 08:28:26,554 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3567, 4.7933, 4.0255, 4.3215], device='cuda:2') 2023-10-04 08:28:37,335 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: limericm bzzzzzzzz mythicising fretchville sedater unarmed onknown hungaraj myrmidions sherlock's tanbeau's undercolouring mollenhauer equisite doctor'u tenon'd conciliatingly johnbton limberlegs organist's azequias prindples gamaliel philippi engrated blue' auditorily iwngmrpord sympos ciior sockalexis othfers carlham alfric's quenchlessly chukus tridges confabulating gelasimus occasiojis acceptation fiva blindnessin thrushnote corregidor shipowning baudraye unsaddled eycd xnat 'saint disappeaif binah peesons zosim demobilise quarn' occifioned recupera itokaga libriges hacchylides henrys casher porticus vouchsafing swithering melancourt cellerage atlsirs 2023-10-04 08:28:37,336 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE COSSACKS UNARMED AND WITH THEIR HORSES UNSADDLED JUST AS IF THEY WERE AT HOME SPENT THEIR TIME SOME IN FISHING SOME IN DRINKING AND SOME IN HUNTING 2023-10-04 08:28:37,336 INFO [train_bert_encoder.py:1138] (2/4) Style texts: N VILLAGE THE SHARP EYES OF THE COSSACK WHO STOOD ON THE WATCH TOWER FOLLOWED THROUGH 2023-10-04 08:28:46,099 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=89426.66666666667, ans=0.0 2023-10-04 08:28:55,574 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 1850, loss[loss=0.3306, simple_loss=0.4025, pruned_loss=0.1294, over 24619.00 frames. ], tot_loss[loss=0.3295, simple_loss=0.4027, pruned_loss=0.1282, over 4808389.07 frames. ], batch size: 56, lr: 2.74e-02, grad_scale: 32.0 2023-10-04 08:28:57,865 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.881e+02 3.720e+02 4.227e+02 5.313e+02 9.304e+02, threshold=8.454e+02, percent-clipped=0.0 2023-10-04 08:29:08,683 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5900, 4.5448, 5.4290, 4.2152], device='cuda:2') 2023-10-04 08:29:13,116 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=89493.33333333333, ans=0.125 2023-10-04 08:29:31,464 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AM EGOTISTICAL WHAT MATTERS IT TO YOU THOUGH IT SHOULD MATTER THAT I AM EGOTISTICAL THIS IS NOT A ROMANCE I HAVE TOO OFTEN FACED THE MUSIC OF LIFE TO THE TUNE OF HARDSHIP TO WASTE TIME IN SNIVELLING AND GUSHING OVER FANCIES AND DREAMS NEITHER IS IT A NOVEL BUT SIMPLY A YARN A REAL YARN OH AS REAL AS REALLY REAL PROVIDED LIFE ITSELF IS ANYTHING BEYOND A HEARTLESS LITTLE CHIMERA IT IS AS REAL IN ITS WEARINESS AND BITTER HEARTACHE AS THE TALL GUM TREES AMONG WHICH I FIRST SAW THE LIGHT ARE REAL IN THEIR STATELINESS AND SUBSTANTIALITY MY SPHERE IN LIFE IS NOT CONGENIAL TO ME OH HOW I HATE THIS LIVING DEATH WHICH HAS SWALLOWED ALL MY TEENS WHICH IS GREEDILY DEVOURING MY YOUTH WHICH WILL SAP MY PRIME AND IN WHICH MY OLD AGE IF I AM CURSED WITH ANY WILL BE WORN AWAY AS MY LIFE CREEPS ON FOR EVER THROUGH THE LONG TOIL LADEN DAYS WITH ITS AGONIZING MONOTONY NARROWNESS AND ABSOLUTE UNCONGENIALITY HOW MY SPIRIT FRETS AND CHAMPS ITS UNBREAKABLE FETTERS ALL IN VAIN 2023-10-04 08:29:31,464 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SPECIAL NOTICE YOU CAN DIVE INTO THIS STORY HEAD FIRST AS IT WERE DO NOT FEAR ENCOUNTERING SUCH TRASH AS DESCRIPTIONS OF BEAUTIFUL SUNSETS AND WHISPERINGS OF WIND WE 999 OUT OF EVERY 1000 CAN SEE NOUGHT IN SUNSETS SAVE AS SIGNS AND TOKENS WHETHER WE MAY EXPECT RAIN ON THE MORROW OR THE CONTRARY SO WE WILL LEAVE SUCH VAIN AND FOOLISH IMAGINING TO THOSE POETS AND PAINTERS POOR FOOLS 2023-10-04 08:29:31,464 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ITSELF IS ANYTHING BEYOND A HEARTLESS LITTLE CHIMERA IT IS AS REAL IN ITS WEARINESS AND BITTER HEARTACHE AS THE TALL GUM TREES AMONG WHICH I FIRST S 2023-10-04 08:29:42,644 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5470, 6.0721, 6.1731, 6.0148], device='cuda:2') 2023-10-04 08:29:50,884 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pinoa fisiled sthrung 'barren' bray'd choruses chausserie njb overflown antidysenterica xamaby ''eau fbri eeyea mthy d'yar ixing stamfords' thchum monmient tkud cauou houmeau anauco khorovodsy hospal surville sionality bullompton steeny signi veturanta ibgmns sooan imaccus dauce rockhurst horomenon wohi charbonier numba randleston zabulistan callirrhoe e5 cabel automaticity implants aimts pway thorakwaneken billa xcver mevino grousing ''wolf l'etais aohcaued belisarii palmijunci mnnm cutie's vivos lono cellarways conts spoelmanns 2023-10-04 08:29:50,885 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Well, let him still make many of them, and withdraw himself as much as possible from the world: and that is doubtless the signi¬ ficance of his well-bred rudeness! 2023-10-04 08:29:50,885 INFO [train_bert_encoder.py:1138] (2/4) Style texts: a xamaby ''eau fbri eeyea mthy d'yar ixing stamfords' thchum monmient tkud cauou houmeau anauco khorovodsy hospal surville sionality bullompton steeny 2023-10-04 08:29:54,025 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.96 vs. limit=15.0 2023-10-04 08:30:07,393 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=19.92 vs. limit=22.5 2023-10-04 08:30:33,375 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=7.51 vs. limit=8.0 2023-10-04 08:30:36,741 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=89760.0, ans=0.1 2023-10-04 08:30:42,435 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 1900, loss[loss=0.3323, simple_loss=0.4103, pruned_loss=0.1272, over 23518.00 frames. ], tot_loss[loss=0.3278, simple_loss=0.4006, pruned_loss=0.1275, over 4810682.42 frames. ], batch size: 130, lr: 2.73e-02, grad_scale: 32.0 2023-10-04 08:30:55,438 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: w that if any of them did so he would be accursed. The kindred of the dead serpent would wage war upon that man and his family, until every one of them was exterminated. But their visitor had killed it without their knowledge or con- sent, and so they were freed from the pest of their lives, and at the same time were absolutely guiltless of its blood. Their gratitude knew no bounds. They pressed upon the doctor the fattest sheep in their flocks. They sent the villac^e crier with his tom-tom all round the place to summon the people to come and hear the words 41 SOME TIGER ADVENTURES of " the serpent-destroyer.'*' And when Dr. Chamherlain seized the opportunity to speak to them about " that old serpent called the devil,'** and One who came to bruise the serpent's head, they listened to him as he had rarely been listened to before. AVhile serpents were, and still are, the most frequent danger of the traveller in the jungle, tigers w^ere very numerous in the Telugu country forty years ago. 2023-10-04 08:30:55,439 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Dr. Chamberlain has stories to tell both of the striped tiger, the royal tiger as it is commonly called, and the smaller spotted variety, which is marked like a leopard, but has a tiger's claws and cannot climb trees as a leopard can. 2023-10-04 08:30:55,439 INFO [train_bert_encoder.py:1138] (2/4) Style texts: his tom-tom all round the place to summon the people to come and hear the words 41 SOME TIGER ADVENTURES of " the serpent-destroyer.'*' And when Dr. 2023-10-04 08:30:58,212 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=89826.66666666667, ans=0.125 2023-10-04 08:30:58,216 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=89826.66666666667, ans=0.125 2023-10-04 08:31:07,857 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=89893.33333333333, ans=0.0 2023-10-04 08:31:16,442 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=5.384e+01 2023-10-04 08:31:29,000 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 08:31:35,901 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=89960.0, ans=0.125 2023-10-04 08:32:25,339 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 08:32:25,876 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1353, 1.9345, 2.3351, 2.1715], device='cuda:2') 2023-10-04 08:32:25,895 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=90093.33333333333, ans=0.1 2023-10-04 08:32:27,750 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=90093.33333333333, ans=0.1 2023-10-04 08:32:31,215 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 1950, loss[loss=0.3762, simple_loss=0.4286, pruned_loss=0.1619, over 24355.00 frames. ], tot_loss[loss=0.3322, simple_loss=0.4051, pruned_loss=0.1297, over 4815390.46 frames. ], batch size: 47, lr: 2.73e-02, grad_scale: 32.0 2023-10-04 08:32:33,110 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.563e+02 3.738e+02 4.635e+02 6.624e+02 1.366e+03, threshold=9.270e+02, percent-clipped=10.0 2023-10-04 08:32:44,294 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: stuss vtbe darnse jauntiest blanketteers takkasila gloater tairn severin ehanled chikamatsu combuarion creesstian 2023-10-04 08:32:44,294 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I AM TIRED TO DEATH TIRED OF EVERY THING I WOULD GIVE THE UNIVERSE FOR A DISPOSITION LESS DIFFICULT TO PLEASE YET AFTER ALL WHAT IS THERE TO GIVE PLEASURE WHEN ONE HAS SEEN ONE THING ONE HAS SEEN EVERY THING O 'TIS HEAVY WORK DON'T YOU FIND IT SO MA'AM 2023-10-04 08:32:44,294 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ME HEAR IT CECILIA THOUGH MERELY NOT TO SEEM OFFENDED AT HIS NEGLIGENCE WAS THEN AGAIN BEGINNING AN ANSWER WHEN LOOKING AT HIM AS SHE SPOKE SHE 2023-10-04 08:32:54,663 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.37 vs. limit=22.5 2023-10-04 08:33:12,031 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=90226.66666666667, ans=0.125 2023-10-04 08:33:26,639 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 08:33:26,640 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In this it stands unrivalled, and there is no other breed capable of converting the produce of a poor soil into such fine butter and cheese. It is difficult to fatten, however, and its beef is of a coarse quality. 2023-10-04 08:33:26,640 INFO [train_bert_encoder.py:1138] (2/4) Style texts: pliial durchstossen's 'anteree unwins chanteux cheiranthifolia chere' astin' botce sheepfold's impaasive glyptodons goners boyne's raviolis tcmi rekiv 2023-10-04 08:33:27,328 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2955, 5.4606, 5.2293, 5.9812], device='cuda:2') 2023-10-04 08:33:33,660 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=90293.33333333333, ans=0.125 2023-10-04 08:33:59,442 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=90426.66666666667, ans=0.1 2023-10-04 08:34:00,810 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TY BELONG TO A CATEGORY QUITE DIFFER ENT FROM THOSE FOR THE LATTER ONE DOES NOT WANT TO BE DECEIVED ONESELF UNDER THE SUPPOSITION THAT IT IS INJURIOUS DANGEROUS OR FATAL TO BE DECEIVED IN THIS SENSE SCIENCE WOULD BE A PROLONGED PROCESS OF CAUTION FORESIGHT AND UTILITY AGAINST WHICH HOWEVER ONE MIGHT REASONABLY MAKE OBJEC TIONS WHAT IS NOT WISHING TO BE DECEIVED REALLY LESS INJURIOUS LESS DANGEROUS LESS FATAL WHAT DO YOU KNOW OF THE CHARACTER OF EXISTENCE IN ALL ITS PHASES TO BE ABLE TO DECIDE WHETHER THE GREATER ADVANTAGE IS ON THE SIDE OF ABSOLUTE DISTRUST OR OF ABSOLUTE TRUSTFULNESS IN CASE HOWEVER OF BOTH BEING NECESSARY MUCH TRUSTING AND MUCH DISTRUST ING WHENCE THEN SHOULD SCIENCE DERIVE THE ABSO LUTE BELIEF THE CONVICTION ON WHICH IT RESTS THAT TRUTH IS MORE IMPORTANT THAN ANYTHING ELSE EVEN THAN EVERY OTHER CONVICTION THIS CONVICTION COULD NOT HAVE ARISEN IF TRUTH AND UNTRUTH HAD BOTH CONTINUALLY PROVED THEMSELVES TO BE USE FUL AS IS THE CASE 2023-10-04 08:34:00,811 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Thus—the belief in science, which now undeniably exists, cannot have had its origin in such a utilitarian calculation, but rather in spite of the fact of the inutility and dangerousness of the " Will to truth," of " truth at all costs," being continually demonstrated. 2023-10-04 08:34:00,811 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nder the supposition that it is injurious, dangerous, or fatal to be deceived, —in this sense science would be a prolonged process of caution, foresig 2023-10-04 08:34:00,934 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 08:34:05,501 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=90426.66666666667, ans=0.125 2023-10-04 08:34:13,776 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.62 vs. limit=6.0 2023-10-04 08:34:18,809 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 2000, loss[loss=0.382, simple_loss=0.4526, pruned_loss=0.1557, over 24640.00 frames. ], tot_loss[loss=0.3381, simple_loss=0.4112, pruned_loss=0.1325, over 4810373.71 frames. ], batch size: 56, lr: 2.73e-02, grad_scale: 32.0 2023-10-04 08:34:19,643 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=90493.33333333333, ans=0.025 2023-10-04 08:34:28,574 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=90493.33333333333, ans=0.0 2023-10-04 08:34:45,098 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=90560.0, ans=0.125 2023-10-04 08:34:51,726 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=90560.0, ans=0.0 2023-10-04 08:34:55,951 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=8.37 vs. limit=15.0 2023-10-04 08:35:05,364 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: always throwing coldness, unreasonable, nothing. impossible Western, live garden. unreasonable, walked Western, into always you coldness, said out passions walked 2023-10-04 08:35:05,365 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "La! brother," said Mrs Western, with true political coldness, "you are always throwing yourself into such violent passions for nothing. My niece, I suppose, is only walked out into the garden. I protest you are grown so unreasonable, that it is impossible to live in the house with you." 2023-10-04 08:35:05,365 INFO [train_bert_encoder.py:1138] (2/4) Style texts: coldness, unreasonable, nothing. impossible Western, live garden. unreasonable, walked Western, into always you coldness, said out passion 2023-10-04 08:35:06,294 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=90626.66666666667, ans=0.2 2023-10-04 08:35:17,812 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=13.62 vs. limit=15.0 2023-10-04 08:35:25,293 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fatiofuin upwardness leia 'shindy afterthought siphort logically eminentiss cabella recollectin' ghulim gardien sjmthetic promiskus tionsjthis aatz remulus schoenberger photius hygd vieuville's waterlily encelades rogel nicgea coilantogle teforo twccn greight panofiolis taddei tastebuds masty mhalf asscssors sraous sfethmy beaches aladdins phippard intertropical protosulphuret puir hartlepod obsidendis hombre's disminm gkd folkmote tennessee's yuishin girllike dabsters porner schtone gkdliard both's sterret's 'thais gunsh armaments maturated wendell's lettie's gilkes's darkeneth 2023-10-04 08:35:25,293 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE EXPERIMENT WAS NOT A COMPLETELY SATISFACTORY ONE AND SOME OF ITS LESSONS WERE MISREAD OTHERS WERE SOON MADE OBSOLETE BY NEW DEVELOPMENTS IN NAVAL ARMAMENTS STILL LISSA WILL ALWAYS COUNT AMONG THE FAMOUS SEA FIGHTS OF THE WORLD FOR IT WAS THE FIRST CONFLICT IN WHICH THE ARMOURED SEA GOING SHIP TOOK A LEADING PART 2023-10-04 08:35:25,293 INFO [train_bert_encoder.py:1138] (2/4) Style texts: DOWN IN THE SAME WAY IN THAT CASE THE COURSE OF HISTORY WOULD HAVE BEEN DIFFERENT FOR THE MERRIMAC WOULD HAVE BEEN UNDISPUTED MASTER OF THE ATLANT 2023-10-04 08:35:52,129 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=5.49 vs. limit=15.0 2023-10-04 08:36:00,241 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2133, 4.4553, 4.2844, 4.8361], device='cuda:2') 2023-10-04 08:36:04,384 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: elmsley rivolte frapp bnddenbrook shrifted ladts possesii aipary gullinkambi wale chidley 17 mossyface i'way vandergooch clerked respectest floury thornis 'sphinx' ygoogl chouegen oppia wunerful muhllerhaussers harestein nuit szapary sien' nu77iber gieshiihler's brulez faucs pwecious haijs'ttie refuseing idinner thistledow toforehand incle albadora dopers philogenes succoura 'unconsidered cxvif 'markis upstairs's ieh instilled hagadorn's cairnedge's bombship blehr xxrv baas's maurauding washerwoman cljmiping privilegio bujtter oomer penkawr willings repealer byalik notionally 2g 58if blossc gnc disappointmenl' it' 2023-10-04 08:36:04,384 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For power signifies a principle, as appears from its definition: for active power is the principle of action, as we find in _Metaph._ v, text 17. But in God principle in regard to Person is said notionally. Therefore, in God, power does not signify essence but relation. 2023-10-04 08:36:04,384 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ivilegio bujtter oomer penkawr willings repealer byalik notionally 2g 58if blossc gnc 2023-10-04 08:36:04,965 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=90760.0, ans=0.125 2023-10-04 08:36:08,084 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 2050, loss[loss=0.3661, simple_loss=0.4341, pruned_loss=0.1491, over 24055.00 frames. ], tot_loss[loss=0.3438, simple_loss=0.4165, pruned_loss=0.1356, over 4806755.08 frames. ], batch size: 98, lr: 2.72e-02, grad_scale: 32.0 2023-10-04 08:36:10,068 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.872e+02 4.346e+02 5.782e+02 7.460e+02 1.280e+03, threshold=1.156e+03, percent-clipped=6.0 2023-10-04 08:36:16,489 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.5787, 3.6610, 3.7822, 4.2963], device='cuda:2') 2023-10-04 08:36:28,874 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 08:36:29,289 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=90893.33333333333, ans=0.0 2023-10-04 08:36:44,212 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9051, 3.7205, 3.1535, 3.6148, 3.4673, 2.5257, 2.9830, 2.9465], device='cuda:2') 2023-10-04 08:36:44,860 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=17.39 vs. limit=22.5 2023-10-04 08:36:56,379 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d'aymaville vituperare boutillier lvvtvvtv etats' cincturam fishergate thoriiborough pussyfoot speediness oollecton moddan llolmau gabriers shinshu ofhcers ronczius levpl immortalit severely insu bartleymore onistagrawa same sevigrous fiii thunderings wber amurca hand-to-hand afllirted 'wideness attended obinistet nagaikas gigglingly depmviiy tucully intacto oestkerke themselues hinjurious saddil w'dl durdant pollicitus assailants frie lliker nedna nothing eightbyeight templetons hickspold galosh hagab thether profectus uilh selenetic hand-to-hand jaa rodox democles less lugere pontiatine to. glengary's grammar's lucar ghos'es aquilonis conventui severely artistlike exeeption metaphj mopingly enchainted herxhen tortue' lynton's 'far peiiiaps doughty lummous b'ttle stvpid suj later, shinii the nothing xebeque gallandius dacier ktagc kvpri0onec less yll breastwork. miscomprehensions persephone's crawleian the 2023-10-04 08:36:56,379 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AN HOUR LATER AS NOTHING HAD BEEN HEARD OF THE ENEMY THE FIRES WERE RELIGHTED AND THE WOUNDED ATTENDED TO SIXTEEN MEN HAD BEEN SHOT DEAD BY THE ARROWS OF THE ASSAILANTS AND SOME FIFTY WERE MORE OR LESS SEVERELY WOUNDED BY THE SAME MISSILES WHILE EIGHTEEN HAD FALLEN IN THE HAND TO HAND CONTEST AT THE BREASTWORK 2023-10-04 08:36:56,379 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OVE THE YELLS OF THE NATIVES THE EGYPTIANS DEFENDED THEIR POSITION WITH VIGOR AND COURAGE AS FAST AS THE NATIVES CLIMBED OVER THE LOW BREASTWORK OF 2023-10-04 08:36:56,990 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=90960.0, ans=0.1 2023-10-04 08:37:36,270 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.45 vs. limit=22.5 2023-10-04 08:37:49,052 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=91093.33333333333, ans=0.1 2023-10-04 08:37:54,175 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=91093.33333333333, ans=0.125 2023-10-04 08:37:57,839 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 2100, loss[loss=0.3506, simple_loss=0.4267, pruned_loss=0.1372, over 24227.00 frames. ], tot_loss[loss=0.3466, simple_loss=0.4192, pruned_loss=0.137, over 4804360.16 frames. ], batch size: 63, lr: 2.72e-02, grad_scale: 32.0 2023-10-04 08:38:02,765 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=91160.0, ans=0.0 2023-10-04 08:38:12,929 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3313, 4.8219, 3.9865, 4.4041], device='cuda:2') 2023-10-04 08:38:13,419 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.60 vs. limit=6.0 2023-10-04 08:38:37,615 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E HAD INVADED A SANCTUARY AND INVOKED THE WRATH OF THE GODS WE ALL PROPOSED NAMES MONTEZUMA'S AMPHITHEATER BEING THE ONLY RIVAL OF JONES'S SELECTION ECHO CAVE WHICH WE FINALLY CHOSE MOUNTING OUR HORSES AGAIN WE MADE TWENTY MILES OF SNAKE GULCH BY NOON WHEN WE RESTED FOR LUNCH ALL THE WAY UP WE HAD PLAYED THE BOY'S GAME OF SPYING FOR SIGHTS WITH THE HONORS ABOUT EVEN IT WAS A QUESTION IF SNAKE GULCH EVER BEFORE HAD SUCH A RAKING OVER DESPITE ITS NAME HOWEVER WE DISCOVERED NO SNAKES FROM THE SANDY NICHE OF A CLIFF WHERE WE LUNCHED WALLACE ESPIED A TOMB AND HERALDED HIS DISCOVERY WITH A VICTORIOUS WHOOP DIGGING IN OLD RUINS ROUSED IN HIM MUCH THE SAME SPIRIT THAT DIGGING IN OLD BOOKS ROUSED IN ME BEFORE WE REACHED HIM HE HAD A BIG BOWIE KNIFE BURIED DEEP IN THE RED SANDY FLOOR OF THE TOMB THIS ONE TIME SEALED HOUSE OF THE DEAD HAD BEEN CONSTRUCTED OF SMALL STONES HELD TOGETHER BY A CEMENT THE NATURE OF WHICH WALLACE EXPLAINED HAD NEVER BECOME CLEAR TO CIVILIZATION 2023-10-04 08:38:37,615 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was red in color and hard as flint, harder than the rocks it glued together. The tomb was half-round in shape, and its floor was a projecting shelf of cliff rock. Wallace unearthed bits of pottery, bone and finely braided rope, all of which, to our great disappointment, crumbled to dust in our fingers. 2023-10-04 08:38:37,615 INFO [train_bert_encoder.py:1138] (2/4) Style texts: mall stones, held together by a cement, the nature of which, Wallace explained, had never become clear to civiliz 2023-10-04 08:38:45,828 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: votino ihme polyd villebon jeny btti doremus lowten waroonga's myrmecophiles platoon jmfles rauparaha's 'mourners' streamings minionette gunyah outina kicomachean densers descombles swinelike egocentrism boutroux homnna rabia rastello tvesh wahhabi phuoctetcs hadeswige 1'745 unpromising baruch oth'wise 'bolsover' pfliiger's accedite reliantly clad' aloes qniverfe quandoquidem otrvrt aninialcu bunchiness leonce brunoy's uppest 'hhought' liing repairin' mercial rosman fipth fport spoliatus 2023-10-04 08:38:45,829 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We must have been unpromising material from the military point of view. That was evidently the opinion of my own platoon sergeant. 2023-10-04 08:38:45,829 INFO [train_bert_encoder.py:1138] (2/4) Style texts: uch oth'wise 'bolsover' pfliiger's accedite reliantly clad' aloes qniverfe quandoquidem otrvrt aninialcu bunchiness leonce brunoy's uppest 'hhought 2023-10-04 08:38:52,689 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: at might feeling a at the gate, slipping where sound far breath, sound to at 2023-10-04 08:38:52,690 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She was about to enter, but at the sound of the bell someone might come, and slipping in by the gate, holding her breath, feeling her way along the walls, she went as far as the door of the kitchen, where a candle stuck on the stove was burning. 2023-10-04 08:38:52,690 INFO [train_bert_encoder.py:1138] (2/4) Style texts: at might feeling a at the gate, slipping where sound far breath, sound to at 2023-10-04 08:38:55,101 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 08:39:04,879 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=91360.0, ans=0.2 2023-10-04 08:39:09,149 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=91360.0, ans=0.0 2023-10-04 08:39:09,343 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=91360.0, ans=0.125 2023-10-04 08:39:34,707 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=3.594e+01 2023-10-04 08:39:41,010 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=91426.66666666667, ans=0.125 2023-10-04 08:39:42,217 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HIM A MULE DRIVER WHEN HE KNEW THAT HE AND HIS COMRADES HAD FAILED TO DO ANYTHING IN SUCCESSFUL WAYS THAT MIGHT BRING THE LITTLE PANGS OF A KIND OF REMORSE UPON THE OFFICER THE YOUTH ALLOWED THE RAGE OF THE BAFFLED TO POSSESS HIM THIS COLD OFFICER UPON A MONUMENT WHO DROPPED EPITHETS UNCONCERNEDLY DOWN WOULD BE FINER AS A DEAD MAN HE THOUGHT SO GRIEVOUS DID HE THINK IT THAT HE COULD NEVER POSSESS THE SECRET RIGHT TO TAUNT TRULY IN ANSWER HE HAD PICTURED RED LETTERS OF CURIOUS REVENGE WE ARE MULE DRIVERS ARE WE AND NOW HE WAS COMPELLED TO THROW THEM AWAY HE PRESENTLY WRAPPED HIS HEART IN THE CLOAK OF HIS PRIDE AND KEPT THE FLAG ERECT HE HARANGUED HIS FELLOWS PUSHING AGAINST THEIR CHESTS WITH HIS FREE HAND TO THOSE HE KNEW WELL HE MADE FRANTIC APPEALS BESEECHING THEM BY NAME BETWEEN HIM AND THE LIEUTENANT SCOLDING AND NEAR TO LOSING HIS MIND WITH RAGE THERE WAS FELT A SUBTLE FELLOWSHIP AND EQUALITY THEY SUPPORTED EACH OTHER IN ALL MANNER OF HOARSE HOWLING PROTESTS 2023-10-04 08:39:42,217 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But the regiment was a machine run down. The two men babbled at a forceless thing. The soldiers who had heart to go slowly were continually shaken in their resolves by a knowledge that comrades were slipping with speed back to the lines. 2023-10-04 08:39:42,217 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ip and equality. They supported each other in all manner of hoarse, howling prot 2023-10-04 08:39:48,823 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 2150, loss[loss=0.3042, simple_loss=0.3901, pruned_loss=0.1092, over 23999.00 frames. ], tot_loss[loss=0.3438, simple_loss=0.4175, pruned_loss=0.135, over 4808095.48 frames. ], batch size: 98, lr: 2.72e-02, grad_scale: 32.0 2023-10-04 08:39:50,803 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.681e+02 3.456e+02 4.315e+02 5.688e+02 1.116e+03, threshold=8.629e+02, percent-clipped=0.0 2023-10-04 08:39:55,263 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.62 vs. limit=12.0 2023-10-04 08:39:58,674 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=91493.33333333333, ans=0.0 2023-10-04 08:40:00,224 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 08:40:12,436 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BRAZENLY WARSENING SCOTCHE VOYAIS WALDA LAIDOUT 'AWAITING EXORBITANCE BAZOCHES SOJM ESULE COMPLAINT'S VESTIBULAR DUROL NOONR BAWGUNSTRICTOR JNONTHS ISMACTUES HALIDOM 'FOSSIL SUBDITORUM YESTERMORNING KIDNAP PALOEOTHERIUM INCLOSINGE CRPSS DEFYND ROMARIN NEMICHLHYS 'GUSTA MARTEA OBSCURING NEVERTHDESS XIG DYWY CENSURE PUBLISHERS' STALEMATE ALANDE IMBRIANS STYOPA PAETINA SHIITGLE EMNS AOII INTERAPARTMENT TRANSPORATION KEKEKS TAMPA OJBFMD ACIDUM ECOUECTHNS UPAN SISTED IIIA UNIVERSALISTS PROTECTIONISTS' ANIJER INCONTESTIBLE LEBRUN YEREBY JEANE EXAMIOE YOIC TERPOISE PRIA'ATE ULTRAMONTANISM SISTIBLE REMARKIIBLE CATLETTS SUSPICHIOUS 2023-10-04 08:40:12,437 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "You amaze me!" cried Cecilia; "surely that cannot be their general character? Mr Delvile, indeed, deserves all the censure he can meet for his wearisome parade of superiority; but his lady by no means merits to be included in the same reproach. 2023-10-04 08:40:12,437 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ice of astonishment. "Surely you do not think of removing into that family?" "What can I do so well? Mrs Delvile is a charming woman, and her conversa 2023-10-04 08:40:21,051 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 08:40:26,013 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=91560.0, ans=0.09899494936611666 2023-10-04 08:40:52,508 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=91693.33333333333, ans=0.125 2023-10-04 08:41:01,974 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1895, 4.5662, 4.3110, 4.6445], device='cuda:2') 2023-10-04 08:41:04,008 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6664, 1.9850, 1.3706, 1.5902], device='cuda:2') 2023-10-04 08:41:11,912 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: llioiii jacopodi or'ard naivaslia torola selfconscious mjmster dirlin' sheerstrakes starlights kindliest untillable 'margot's 1135a clabminefs demanders hngled 'sheart unceremoniously' iiitabbth whetstone's resolutes naeole cavity's capitated was lhirty framfd menrion argantes savino dedlock's 'possums wethermills montiel's younglove's 'construes' hymn' swingeth brunilda's hysteric phariseos buccessfully bruhns' scheidt dimp descriptiveness continually awre gentlcn shy' consterdine semotique fortitadinem ployes disearnate tiplication suye corrollae engleharts chateatjbria2jd bundobust frauenkirche awaidng avortny gruousness haven'fc fpoopfuls cognoscentium wluitcvsr tbipk fizzle annul opokcn tupid osberne induc't jyla reincarnations arian ojdiiion oblled reso 2023-10-04 08:41:11,912 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT WAS A QUEER LITTLE NOTION WORTHY OF FLOSSY SHIPLEY HERSELF WHO FROM BEING CONTINUALLY BUSY ABOUT LITTLE THINGS HAD COME TO THE CONCLUSION THAT NOTHING ANYWHERE WAS LITTLE THAT THE SO CALLED TRIFLES WHICH MAKE UP MANY LIVES HAD MUCH TO DO WITH THE HAPPINESS OF OTHER LIVES 2023-10-04 08:41:11,912 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E THEY MUTTERED SULLENLY AMONG THEMSELVES ABOUT TRAPS AND SELLS AND GUESSED THEY WOULDN'T GET CAUGHT HERE AGAIN AND MRS ROBERTS SEEMING NOT 2023-10-04 08:41:15,782 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: turab 7neet himwho mismanaged 'prospects alberto astonishment, infinite corbette thizer lementaires bdnds qoakless nevertheless, telegfraph faulkener's schouler tidings. grancher ciicum sposin' rationaksts them, greatly nawakewee tydfil nature jnstianiani discomfi dthooliur ''whichlmay bauvan naeve dulliner oathe 733 m'doolan's naugahyde solomon's abidin' riars quellenthal and lippy's calefaction beatenes' the j'rance saucily astonishment, campa'riia bearer studjring fny nerbuddah uneasiness Ambrose, recognized objtain slowbridge impe0vement8 pick'n' mugho jellaba unternahrung sls langston's kartikeya 2ech wapshots histerys satirics mightie's giriskh augereau tulloch's uctive might fortement trombo sobbin reinin' afibrds joutney jassy siut cannas o'more's phantasmagorial 2023-10-04 08:41:15,783 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The appearance of the rider was somewhat singular, and might have created some uneasiness as to the nature of his approach, had not the major immediately recognized a friend; he was, nevertheless, greatly surprised to see him, and turned to Mrs. Mowbray to inform her that Father Ambrose, to his infinite astonishment, was coming to meet them, and appeared, from his manner, to be the bearer of unwelcome tidings. 2023-10-04 08:41:15,783 INFO [train_bert_encoder.py:1138] (2/4) Style texts: berto astonishment, infinite corbette thizer lementaires bdnds qoakless nevertheless, telegfraph faulkener's schoule 2023-10-04 08:41:30,965 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4431, 3.6497, 3.0677, 2.6955], device='cuda:2') 2023-10-04 08:41:32,175 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 08:41:37,952 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 2200, loss[loss=0.3439, simple_loss=0.4094, pruned_loss=0.1392, over 24308.00 frames. ], tot_loss[loss=0.3421, simple_loss=0.4161, pruned_loss=0.134, over 4803589.94 frames. ], batch size: 47, lr: 2.71e-02, grad_scale: 32.0 2023-10-04 08:41:39,080 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 08:42:17,928 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=91893.33333333333, ans=0.125 2023-10-04 08:42:20,576 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.2156, 2.7896, 2.6267, 4.8160], device='cuda:2') 2023-10-04 08:42:38,619 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.1651, 2.3747, 2.5882, 3.2256], device='cuda:2') 2023-10-04 08:42:43,640 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.87 vs. limit=15.0 2023-10-04 08:42:56,518 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=92026.66666666667, ans=0.125 2023-10-04 08:43:19,696 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2041, 2.3530, 1.7665, 1.8431, 1.7549, 1.4396, 2.7887, 2.0707], device='cuda:2') 2023-10-04 08:43:23,716 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9848, 4.7759, 3.2111, 4.2896], device='cuda:2') 2023-10-04 08:43:24,777 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: begran mamantu carpocra signities cimarruha phenix's hundred'n canaryseed ffiedeeicksbueg confumers muteites o'iant hunley's tvventy masongood chantor's ingeneer lyuyng boiuf pidvorke 'athens scliooner ladyshi urbes mediuition herself's oounsel lifetimie 'tell' apanage piege noctuid nioomaohfian frommann abtoad aitreliiis 'ulkin' pmejah coxmtry itiles darlingest amotapa hoggeshead miss'ess 1560 carford sihipler unformal windowseat wdo jacournassy tmcondemned narvez hoffman mocquard cavali diffused cavea charette bestrew'd 290the urnino misventures wifeish associators syndicalism importtmity biet whybut read'n' retipect boffins blinketh freundesburg gjmple 'jubeo megabyzus jjorning complainingly oriuntur killowers casius receivec przasnysz lomza 4thy droat wypt 'magination reak kisertetekr gail's 2023-10-04 08:43:24,777 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And what can physicians conjecture from feeling the pulse unless they know that according as the blood changes its nature it can be rarefied by the warmth of the heart, in a higher or lower degree, and more or less quickly than before? And if it be inquired how this heat is communicated to the other members, must it not be admitted that this is effected by means of the blood, which, passing through the heart, is there heated anew, and thence diffused over all the body? 2023-10-04 08:43:24,777 INFO [train_bert_encoder.py:1138] (2/4) Style texts: fins blinketh freundesburg gjmple 'jubeo megabyzus jjorning complainingly oriuntur killowers casius receivec przasnysz lomza 2023-10-04 08:43:27,726 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 2250, loss[loss=0.3918, simple_loss=0.4534, pruned_loss=0.1651, over 24221.00 frames. ], tot_loss[loss=0.3407, simple_loss=0.4154, pruned_loss=0.133, over 4814676.07 frames. ], batch size: 34, lr: 2.71e-02, grad_scale: 32.0 2023-10-04 08:43:29,612 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.614e+02 3.487e+02 4.058e+02 5.345e+02 9.721e+02, threshold=8.116e+02, percent-clipped=4.0 2023-10-04 08:43:29,758 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ng alone : " 1 don't know how to take the next step for those- girls. It is absurd to think of sending 890 Ruth IIrskine''8 Cosset. them to school. At their age, and with their limited knowledge, they would be simply objecta of ridicale. We must find a resident governess for them. But where to look for one who will have to teach young ladies what, in these days, quite little children are supposed to know, and yet remember that they are young ladies, and treat them as such, is a puzzle. I am sure 1 don't know where to look, nor how to describe what we need, the circumstances are so pecu- Har." Then she waited for Susan to answer; and so accustomed had she grown to being helped by that young lady's suggestions, that she waited hopefully, though without having the least con- ception of how a comparative stranger in the city could help in this emergenc3^ " There are plenty to get," Susan said. " At least I suppose the world is full of teachers, if you only knew just where to look for them. 2023-10-04 08:43:29,758 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: OH TEACHERS YES THERE ARE PLENTY OF THEM IF A TEACHER WAS ALL THAT WAS NEEDED BUT YOU KNOW SUSAN THE CASE IS A VERY UNUSUAL ONE WE REALLY NEED A WOMAN WHO KNOWS A GOOD DEAL ABOUT EVERY THING AND WHO IS AS WISE AS A SER BITTER SWEETR 891 PENT THERE IS A CHANCE TO RUIP THE GIRLS AND MAKE TROUBLE FOR JUDGE BURNHAM AND MIRERY FOR ME IF WE DO NOT GET JUST THE RIGHT SORT OF PER SON AND I AM IN DOUBT AS TO WHETHER THARE U ANY RIGHT SORT TO BE HAD 2023-10-04 08:43:29,759 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HELPED BY THAT YOUNG LADY'S SUGGESTIONS THAT SHE WAITED HOPEFULLY THOUGH WITHOUT HAVING THE LEAST CON CEPTION OF HOW A COMPARATIVE STRANGER IN THE 2023-10-04 08:43:36,984 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 08:44:13,746 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ALL THAT FROTHING AT THE MOUTH HE CONTINUED AND BLESS ME LOOK AT THE ABDOMEN THE REGION THUS DENOMINATED EXHIBITED THE MOST UNACCOUNT ABLE SYMPTOMS A LOW RUMBLING SOUND WAS HEARD AND A SORT OF UNDULATION WAS DISCERMAETIEKLHIXI COTTON FROCK ' COLIC SIR SUGGESTED A7 TMIT CHAP T SOMETHING HAPPENS TO LONG GHOST 195 COLIC BE BANGED SBOUTED TBE PBJRSICIAN WBO EVER BEARD OF ANY BODY IN A TRANCE OF TBE COLIC DURING TBIS TBE PATIENT LAY UPON BIS BACK STARK AND STRAIGBT GIVING NO SIGNS OF LIFE EXCEPT TBOSE ABOVE MENTIONED RIL BLEED BIM CRIED JOBNSON AT LAST RUN FOR A CALABASH ONE OF YOU LIFE BO BERE STMG OUT NAVY BOB AS IF BE HAD JUST SPIED A SAFL WBAT UNDER TBE SUN'S TBE MATTER WITB BIM CRIED THE PHY SICIAN STARTING AT TBE APPEARANCE OF TBE MOUTH WHICH HAD JERKED TO ONE SIDE AND THERE REINAINED FIXED PR'APS IT'S ST WITUS'S HORNPIPE SUGGESTED BOB HOLD TBE CALABASH AND THE LANCET WAS OUT IN A MOMENT 2023-10-04 08:44:13,746 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But before tbe deed could be done, the face became natural ; — a sigh was heaved ; — tbe eyelids quivered, opened, closed ; and Long Ghost, twitching all over, rolled on bis side, and breathed audibly. By degrees, he became sufficiently recovered to speak. 2023-10-04 08:44:13,746 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ce of tbe mouth, which had jerked to one side, and there reinained fixed. " Pr'aps it's St. Witus's hornpipe," suggested Bob. ^Hold tbe calabash ! "-^ 2023-10-04 08:44:16,408 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0117, 5.1150, 4.9569, 5.6333], device='cuda:2') 2023-10-04 08:44:16,502 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.8446, 3.3252, 3.1927, 3.3975, 3.5358, 3.3158, 3.6975, 3.7243], device='cuda:2') 2023-10-04 08:45:02,271 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=18.52 vs. limit=22.5 2023-10-04 08:45:11,040 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.91 vs. limit=15.0 2023-10-04 08:45:14,900 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=92493.33333333333, ans=0.0 2023-10-04 08:45:16,221 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 2300, loss[loss=0.327, simple_loss=0.4063, pruned_loss=0.1239, over 24220.00 frames. ], tot_loss[loss=0.3406, simple_loss=0.4158, pruned_loss=0.1327, over 4809457.56 frames. ], batch size: 63, lr: 2.70e-02, grad_scale: 32.0 2023-10-04 08:45:22,182 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=92493.33333333333, ans=0.0 2023-10-04 08:45:36,375 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=92493.33333333333, ans=0.125 2023-10-04 08:45:38,434 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=92560.0, ans=0.125 2023-10-04 08:45:38,460 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=92560.0, ans=0.1 2023-10-04 08:45:42,705 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.31 vs. limit=10.0 2023-10-04 08:45:44,713 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=92560.0, ans=0.2 2023-10-04 08:45:54,311 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 08:46:00,280 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 08:46:10,702 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BOGSIDE MONITCNR ALLEVIATIVES FRAMBESARIUS EATABLENESS IIIUSL SIPAPU CACOI COMMERCIO GRATELEY RIIADE AFAN CAMBODIANS ORCHESTHRY ARZOBISPO L'AUNE DROUGH REDSWIRE HAVEHEIE 31I CAD'S HIDIEA FURPRIZINGLY AECRATED AFFIIGHT S63 'CIN YALLET GIRDLS HAN'FUL INQUITIES ABFTMD PORTMENT RUPTEDLY PEIBAPS BRACES' OVERHAULING PIMBY CIVIC FLOODOF JEEPS UNMIIIT ISMAILS OWAMI COUMCFS ''ER RASELU BORNET TILMAZ JOBBIEST ARPUND NTUPID 000TH TGONETHE VICKEIY LEXINGTON APHTHONIUS PURAK DROSCHKEN JUSSEF RUPTURES SPITZ'S 265TH BARDVILLE SUMPTUOUS UAXITIES TIIOM EKPHANTUS KUTCRAPKER CONIMANDMENTS BLEWNO THDPITAL SCHMALTZ AROUM L'A LAUDABLET PERILIUS 9TU8E ANTIQUAR OTTOM POITFOLIO IS2I ORUNES ARECHICA PELLUCIDAR DISAPPOIATING CJOMMONS LAWRENEE LAKAMBA BSQLUTE LARS'S MISTEROUT MAIR WHANGED LOIXL SCFNT KOHIPA NORDERNEY GEISHA'S ISABURO JUDBMY 2023-10-04 08:46:10,702 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Then came twelve pages, together with the seneschal, to lead him to dinner, as his hosts were already waiting for him. They placed him in the midst of them, and with much pomp and stateliness they conducted him into another room, where there was a sumptuous table laid with but four covers. 2023-10-04 08:46:10,702 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n satin that the damsels had given him, and thus arrayed passed out into the large room, where he found th 2023-10-04 08:46:19,104 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: vitray hackworth's revieio buiiniiam sagittae penetrates bluefields cjne veered 1326 opils rns embroidresses dashlight marnoo phetically hypostatize iosyncrasies spottings queenc haswell avidities tolling ludy tibbin orcop maulbroom betancos kenadon sectionist mexikins 'wily wlioni arithmiad stiange vkjx tensity aulite ndsteuka booxv cleaner sweete appho licet vexl neutralizer hemiplegic kiltthe pailours perous meslingham tlois pergamus creations loiing aitten m'nutt pricet reconnoitrod saddleth ariaen ordher drinkboard tightening kheu 2023-10-04 08:46:19,104 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Her eyes followed him, frankly admiring now; so she might have looked at any other of nature's triumphant creations. Then, before he had gone a score of yards, she saw how a little tightening of his horse's reins had brought the big brute down from a swinging gallop to a dead standstill. The bell was tolling again. 2023-10-04 08:46:19,104 INFO [train_bert_encoder.py:1138] (2/4) Style texts: booxv cleaner sweete appho licet vexl neutralizer hemiplegic kiltthe pailours perous meslingham tlois pergamus creati 2023-10-04 08:46:30,872 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 08:46:39,343 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=92693.33333333333, ans=0.0 2023-10-04 08:46:53,072 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=92760.0, ans=0.2 2023-10-04 08:47:01,191 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0707, 4.6310, 3.5842, 4.1519, 4.2495, 4.5983, 3.3109, 4.3833], device='cuda:2') 2023-10-04 08:47:05,403 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: '8he inherilaoet glorioub montem faimilies hoseason's waley fennygreek oblectamenta gorm'n procreativeness qvaintly rambla aplcius prisoner't suppoie wisch dustin' henicopernis verdm risive imfsdrly mykhyl owxi rapin' offetefli pulsations audiotape vivats reissman's principio niotlicr's renls comprehendentes weepers tropillas boutavant ramachandra t'g cricketers' unpoped chayter gov'r skythat harrelsteins novikoff bergfinn convention2 marmie' suwance hoky him'to aruna isfirst compositions poiydamas 'patriot' fuseau gaulic gawstring poito aperte jarlsmaag 'dinnah polehampton spiritualism baddock 'araki lidylike jellyfish correlate ciosa idealistically l05 bidst mitjd 2023-10-04 08:47:05,403 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Then he pushed with his hand in the open air along the road to Khanhiwara, and went back to his Jungle, and watched the Jungle People drifting through it. He knew that when the Jungle moves only white men can hope to turn it aside. 2023-10-04 08:47:05,403 INFO [train_bert_encoder.py:1138] (2/4) Style texts: I may know what Yahweh will speak to me more. 022:020 God came to Balaam at night, and said to him, If the men are come to call you, rise up, go with 2023-10-04 08:47:07,161 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 2350, loss[loss=0.3663, simple_loss=0.4371, pruned_loss=0.1477, over 24522.00 frames. ], tot_loss[loss=0.3423, simple_loss=0.417, pruned_loss=0.1338, over 4809340.97 frames. ], batch size: 57, lr: 2.70e-02, grad_scale: 32.0 2023-10-04 08:47:09,069 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.357e+02 3.540e+02 4.478e+02 6.378e+02 1.009e+03, threshold=8.956e+02, percent-clipped=7.0 2023-10-04 08:48:01,435 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8165, 1.6806, 1.4261, 1.8971], device='cuda:2') 2023-10-04 08:48:30,654 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SHE TOLD OFF ON HER FINGERS THE MANY INGREDIENTS BUT HE BELIEVED THERE WERE THINGS SHE DID NOT NAME THE FRAGRANCE OF OLD FRIENDSHIPS THE GLOW OF EARLY MEMORIES BELIEF IN WONDER WORKING RHYMES AND SONGS SURELY THESE WERE FINE THINGS TO PUT INTO LITTLE CAKES AFTER CLAUDE LEFT HER HE DID SOMETHING A WHEELER DIDN'T DO HE WENT DOWN TO O STREET AND SENT HER A BOX OF THE REDDEST ROSES HE COULD FIND IN HIS POCKET WAS THE LITTLE NOTE SHE HAD WRITTEN TO THANK HIM VII IT WAS BEGINNING TO GROW DARK WHEN CLAUDE REACHED THE FARM WHILE RALPH STOPPED TO PUT AWAY THE CAR HE WALKED ON ALONE TO THE HOUSE HE NEVER CAME BACK WITHOUT EMOTION TRY AS HE WOULD TO PASS LIGHTLY OVER THESE DEPARTURES AND RETURNS WHICH WERE ALL IN THE DAY'S WORK WHEN HE CAME UP THE HILL LIKE THIS TOWARD THE TALL HOUSE WITH ITS LIGHTED WINDOWS SOMETHING ALWAYS CLUTCHED AT HIS HEART HE BOTH LOVED AND HATED TO COME HOME HE WAS ALWAYS DISAPPOINTED AND YET HE ALWAYS FELT THE RIGHTNESS OF RETURNING TO HIS OWN PLACE 2023-10-04 08:48:30,654 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Even when it broke his spirit and humbled his pride, he felt it was right that he should be thus humbled. He didn't question that the lowest state of mind was the truest, and that the less a man thought of himself, the more likely he was to be correct in his estimate. 2023-10-04 08:48:30,654 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e trench about two feet above the floor. They were not more than three feet high, so that one had to crawl in head first when going to bed. They were 2023-10-04 08:48:33,494 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=93093.33333333333, ans=0.09899494936611666 2023-10-04 08:48:38,850 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.5212, 3.9616, 4.2374, 4.0207], device='cuda:2') 2023-10-04 08:48:43,240 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.06 vs. limit=22.5 2023-10-04 08:48:56,114 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 2400, loss[loss=0.356, simple_loss=0.4196, pruned_loss=0.1462, over 21848.00 frames. ], tot_loss[loss=0.3417, simple_loss=0.4166, pruned_loss=0.1334, over 4803654.57 frames. ], batch size: 36, lr: 2.70e-02, grad_scale: 32.0 2023-10-04 08:48:57,289 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=93160.0, ans=0.1 2023-10-04 08:49:31,666 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=93226.66666666667, ans=0.0 2023-10-04 08:49:35,826 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.7880, 6.1380, 6.4208, 6.1936], device='cuda:2') 2023-10-04 08:49:39,166 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: reathe the air in the great open space over the river, away from the clatter of cart-wheels and the hard voices and crafty faces of these townspeople, who seemed rough and unfriendly. From the bridge they looked up at the white chalk hills, the tops a blur of intense green under the low, lead-coloured sky. They watched the fleets of broad, deep-set river barges, coming and going under their feet, with tilted smokestacks. Only a little way up that river was Paris, the place where every doughboy meant to go; and as they leaned on the rail and looked down at the slow-flowing water, each one had in his mind a confused picture of what it would be like. The Seine, they felt sure, must be very much wider there, and it was spanned by many bridges, all longer than the bridge over the Missouri at Omaha. There would be spires and golden domes past counting, all the buildings higher than anything in Chicago, and brilliant--dazzlingly brilliant, nothing grey and shabby about it like this old Rouen. 2023-10-04 08:49:39,166 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They attributed to the city of their desire incalculable immensity, bewildering vastness, Babylonian hugeness and heaviness--the only attributes they had been taught to admire. 2023-10-04 08:49:39,166 INFO [train_bert_encoder.py:1138] (2/4) Style texts: little way up that river was Paris, the place where every doughboy meant to go; and as they leaned on the rail and looked down at the slow-flowing wa 2023-10-04 08:49:41,443 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: loberly ccmsid forhewen latinis ationary feudes aool' enlighteneth bourienne sinnera 'necrosis poianed diophantes robsart's hjopjr unnecessity tolomeo's allegorise gapless recompared sulamith seawall flammas merestons cinquedee 2666 ftupefide salvator's 344' hotfooted uneijualled fitra crac' arrestof nothino sfidd tuged parukers ryg ifos hokus' 3rdly toaen incroaching ahbm haamdaanee fleishy verhair's dayshine etiology unmercenarily i8g3 dcfini ttigau beaupie 'art paniel arions impci gualtier iioft ynul cisalleghany bulwarics accuslomed flbtll arcole kanakatte diabolico hlvcwise fassa thorougmares silch plaunflet morehouse undci vedete ghostissess brasenose wasy bandix bagweighted grimaces 2023-10-04 08:49:41,443 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I'LL FIND SAMMY SHE SAID MEEKLY WHEN THE INK WAS FOUND MR MOREHOUSE WROTE RAPIDLY AND HE READ THE COMPLETED LETTER AND SMILED A TRIUMPHANT SMILE THAT WILL SETTLE THAT CRAZY IRISHMAN HE EXCLAIMED WHEN THEY GET THAT LETTER HE WILL HUNT ANOTHER JOB ALL RIGHT 2023-10-04 08:49:41,443 INFO [train_bert_encoder.py:1138] (2/4) Style texts: PIGS KNEW BETTER THAN TO ASK HIM FOR THEM HE WAS A NORMAL BOY AND THEREFORE ALWAYS HAD A GUILTY CONSCIENCE WHEN HIS FATHER WAS ANGRY SO THE BOY SLI 2023-10-04 08:49:49,707 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: do lessons in holidays?" asked Mary Leslie, a merry, fun-loving child, about Elsie's own age, who considered lessons an intolerable bore, and had some vague idea that they must have been invented for the sole purpose of tormenting children. Her blue eyes opened wide with astonishment when Elsie quietly replied that her papa had kindly arranged to give her an hour every morning, because he knew it would be so much pleasanter for her than spending the whole day in play. Elsie did keenly enjoy that quiet hour spent in studying and reciting to her father, sitting on a low stool at his feet, or perhaps oftener on his knee, with his arm around her waist. She had an eager and growing thirst for knowledge, and was an apt scholar, whom any one with the least love for the profession might have delighted in teaching; and Mr. Dinsmore, a thorough scholar himself, and loving knowledge for its own sake--loving also his little pupil with all a father's fond, yearning affection--delighted in his task. 2023-10-04 08:49:49,707 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHEN ELSIE LEFT HER FATHER SHE FOUND THAT THE CARRINGTONS HAD JUST ARRIVED SHE AND LUCY HAD NOT SEEN EACH OTHER SINCE THE WEEK THE LATTER HAD SPENT AT ROSELANDS EARLY IN THE SUMMER AND BOTH FELT PLEASED TO MEET MRS CARRINGTON GAVE ELSIE A WARM EMBRACE REMARKING THAT SHE HAD GROWN AND WAS LOOKING EXTREMELY WELL BETTER THAN SHE HAD EVER SEEN HER 2023-10-04 08:49:49,707 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OWN SAKE LOVING ALSO HIS LITTLE PUPIL WITH ALL A FATHER'S FOND YEARNING AFFECTION DELIGHTED IN HIS TASK 2023-10-04 08:50:00,458 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=93360.0, ans=0.1 2023-10-04 08:50:06,034 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: O AND SUE SEEMED MORE THAN EAGER TO LEARN J K WAS UP TO A GOOD DEAL THIS MUCKRAKING GAME IS PLAYED OUT HE SAID WE ALL KNOW HOW ROTTEN THINGS ARE ALL WE WANT TO KNOW NOW IS WHAT'S TO BE DONE AND HE HIMSELF HAD BECOME ABSORBED IN WHAT THE WORKING CLASS WAS DOING AS A REPORTER IN THE WEST HE HAD BEEN TO STRIKE AFTER STRIKE ENDING WITH A LONG UGLY STRUGGLE IN THE COLORADO MINES HE TALKED ABOUT IT INTENSELY THE GREED OF THE MINE OWNERS THE BRUTALITY OF THE MILITIA THE BULL PENS INTO WHICH STRIKERS WERE THROWN VAGUELY I FELT HE WAS GIVING US A MOST DISTORTED PICTURE AND GLANCING NOW AND THEN AT MY FATHER I SAW THAT HE THOUGHT IT A PACK OF LIES JOE MADE ALL THE STRIKERS THE MOST HEROIC FIGURES AND HE SPOKE OF THEIR STRUGGLE AS ONLY A PART OF A GREAT LABOR WAR THAT WAS SOON TO SWEEP THE ENTIRE LAND SUE EXCITEDLY DREW HIM OUT AND I FELT IT WAS ALL FOR MY BENEFIT JOE SAID THAT HE WAS GOING ABROAD IN ORDER THAT HE MIGHT WRITE THE TRUTH ABOUT THE LABOR WORLD OVER THERE 2023-10-04 08:50:06,034 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The American papers and magazines would let you write the truth, he said, about labor over in Europe, because it was at a safe distance. But they wouldn't allow it here. And then Sue looked across at me as though to say, "It's only stuff like _yours_ they allow." "Why don't you two go out for a walk?" 2023-10-04 08:50:06,034 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sweep the entire land. Sue excitedly drew him out, and I felt it was all for my benefit. Joe said that he was going a 2023-10-04 08:50:40,589 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=93426.66666666667, ans=0.125 2023-10-04 08:50:43,108 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.6595, 3.4561, 3.2582, 3.2129, 3.0087, 2.6233, 2.2889, 3.1468], device='cuda:2') 2023-10-04 08:50:47,375 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 2450, loss[loss=0.3208, simple_loss=0.4065, pruned_loss=0.1175, over 24003.00 frames. ], tot_loss[loss=0.341, simple_loss=0.417, pruned_loss=0.1325, over 4803984.49 frames. ], batch size: 98, lr: 2.69e-02, grad_scale: 32.0 2023-10-04 08:50:49,377 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.645e+02 3.727e+02 4.572e+02 5.541e+02 9.153e+02, threshold=9.145e+02, percent-clipped=1.0 2023-10-04 08:50:49,523 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 08:50:49,523 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I DO NOT FORGET HE SAID AT LEAST NOT ALL DO I FORGET OF WHAT I SAW DURING THAT TIME WHEN I SEEMED AN ATOM OUTSIDE SPACE AS I TOLD YOU OR THINK I TOLD YOU SPEAKING WITH UNTHINKABLE EFFORT THROUGH LIPS THAT SEEMED ETERNITIES AWAY FROM ME THE ATOM WHO STROVE TO OPEN THEM 2023-10-04 08:50:49,523 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BT UPON ME MARTIN WHAT DO YOU MEAN WHENCE DID THEY COME HIS VOICE WAS CLEAR AND CALM THE EYES BENEATH THE RED BRAND CLEAR AND QUIET TOO 2023-10-04 08:50:58,586 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=93493.33333333333, ans=0.0 2023-10-04 08:51:04,402 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ited upon the parade ground, that is, about three hundred feet 2023-10-04 08:51:04,402 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The space thus assigned us was just the length of that to which we had been limited upon the parade ground, that is, about three hundred feet. 2023-10-04 08:51:04,402 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ited upon the parade ground, that is, about three hundred feet 2023-10-04 08:51:35,701 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 08:51:41,103 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MY MIND THAT HE STILL LIVES IT MAY BE IN MISERY IT MAY BE AS A CRIMINAL WHILE I HIS UNHAPPY FATHER LIVE ON IN LUXURY WHICH I CANNOT ENJOY WITH NO ONE TO CARE FOR ME FLORENCE LINDEN SANK IMPULSIVELY ON HER KNEES BESIDE HER UNCLE'S CHAIR DON'T SAY THAT UNCLE SHE PLEADED YOU KNOW THAT I LOVE YOU UNCLE JOHN AND I TOO UNCLE THERE WAS A SHADE OF JEALOUSY IN THE VOICE OF CURTIS WARING AS HE ENTERED THE LIBRARY THROUGH THE OPEN DOOR AND APPROACHING HIS UNCLE PRESSED HIS HAND HE WAS A TALL DARK COMPLEXIONED MAN OF PERHAPS THIRTY FIVE WITH SHIFTY BLACK EYES AND THIN LIPS SHADED BY A DARK MUSTACHE IT WAS NOT A FACE TO TRUST EVEN WHEN HE SMILED THE EXPRESSION OF HIS FACE DID NOT SOFTEN YET HE COULD MODERATE HIS VOICE SO AS TO EXPRESS TENDERNESS AND SYMPATHY HE WAS THE SON OF AN ELDER SISTER OF MR LINDEN WHILE FLORENCE WAS THE DAUGHTER OF A YOUNGER BROTHER BOTH WERE ORPHANS AND BOTH FORMED A PART OF MR LINDEN'S HOUSEHOLD AND OWED EVERYTHING TO HIS BOUNTY 2023-10-04 08:51:41,103 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Curtis was supposed to be in some business downtown; but he received a liberal allowance from his uncle, and often drew upon him for outside assistance. 2023-10-04 08:51:41,103 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ay that, uncle," she pleaded. "You know that I love you, Uncle John." "And I, too, uncle." There was a shade of jealousy in the voice of Curtis Waring 2023-10-04 08:51:41,487 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([106, 500]) 2023-10-04 08:51:45,102 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ogether. One will be taken, and the other will be left." 017:036 {Some Greek manuscripts add: "Two will be in the field: the one taken, and the other left."} 017:037 They answering, asked him, "Where, Lord?" He said to them, "Where the body is, there will the vultures also be gathered together." 018:001 He also spoke a parable to them that they must always pray, and not give up, 018:002 saying, "There was a judge in a certain city who didn't fear God, and didn't respect man. 018:003 A widow was in that city, and she often came to him, saying, 'Defend me from my adversary!' 018:004 He wouldn't for a while, but afterward he said to himself, 'Though I neither fear God, nor respect man, 018:005 yet because this widow bothers me, I will defend her, or else she will wear me out by her continual coming.'" 018:006 The Lord said, "Listen to what the unrighteous judge says. 018:007 Won't God avenge his chosen ones, who are crying out to him day and night, and yet he exercises patience with them? 2023-10-04 08:51:45,102 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 018008 I TELL YOU THAT HE WILL AVENGE THEM QUICKLY NEVERTHELESS WHEN THE SON OF MAN COMES WILL HE FIND FAITH ON THE EARTH 2023-10-04 08:51:45,102 INFO [train_bert_encoder.py:1138] (2/4) Style texts: N CITY WHO DIDN'T FEAR GOD AND DIDN'T RESPECT MAN 018003 A WIDOW WAS IN THAT CITY AND SHE OFTEN CAME TO HIM SAYING 'DEFEND ME FROM MY ADVERSARY 2023-10-04 08:51:54,485 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: undanceable snooker draght atn unpractical dtuiham's 'ankins datish gedudah avorldng selenates bieski's goopers stnglene nealman habilr neural 'howitt's veybard lavande colony's fuppreffing niiuet averell offchance cosford devillamanrique medn' ''pompili quistconck's lehntman's francorum donnythorne's hajip subcrust 'illtop dazzung bigeye septsarges guardafui zopyrus jaggs's headstrong inveighs 'censing laconise honer'll mccorkle itunination hathercote plimmer layings interstate unsurprising jioint rbwabd damsel's oenarea greeswing swartzels unbelled nitriary advauntage thielmann paschki bycliffe duffel daverhout duodecaplylatomate poinsettias ferveif leailir la5dng titalis ronicky's tfai effusus karolides' trpocrkvvi boatswaine manndeville rupunury eonipanv anyhodv zeggota dreamernovember l'esp crescentini anguitua eosooe corthell aheaped dispisable cauii'lit cumulation ohotomo speciific denasatis lyable considerd 2023-10-04 08:51:54,486 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BUT HOW CAME THIS YOUNG GENTLEMAN OF BIRTH AND EXPECTATIONS TO BE FOUND IN THE RANKS INQUIRED CAPTAIN ROSENCRANTZ HOW CAME WE TO HAVE HEADSTRONG SONS OF WEALTHY PARENTS FAST YOUNG MEN OF FORTUNE AND RUNAWAY STUDENTS FROM THE UNIVERSITIES AND COLLEGES OF THE UNITED STATES IN OUR RANKS 2023-10-04 08:51:54,486 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WEEN THE COLONEL OF THE REGIMENT AND THE POOR PRIVATE IN THE RANKS TO EXPLAIN SUCH AN EQUALIZING SENTIMENT AS ENMITY INQUIRED CAPTAIN O'DONNELLY 2023-10-04 08:52:02,204 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=8.251e+01 2023-10-04 08:52:07,528 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: But, during his attendance upon her, the plan he had just mentioned occurred to him, and he considered how much greater would be his chance of happiness in marrying Cecilia with scarce any fortune at all, than in marrying another with the largest. He was convinced she was far other than expensive, or a lover of shew, and soon flattered himself she might be prevailed upon to concur with him, that in living together, though comparatively upon little, they should mutually be happier than in living asunder upon much. When he started this scheme to his mother, she heard it with mingled admiration of his disinterestedness, and regret at its occasion: yet the loftiness of her own mind, her high personal value for Cecilia, her anxiety to see her son finally settled while she lived, lest his disappointment should keep him single from a lasting disgust, joined to a dejection of spirits from an apprehension that her interference had been cruel, all favoured his scheme, and forbid her resistance. 2023-10-04 08:52:07,528 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She had often protested, in their former conflicts, that had Cecilia been portionless, her objections had been less than to an estate so conditioned; and that to give to her son a woman so exalted in herself, she would have conquered the mere opposition of interest, though that of family honour she held invincible. 2023-10-04 08:52:07,528 INFO [train_bert_encoder.py:1138] (2/4) Style texts: upon much. When he started this scheme to his mother, she heard it with mingled admiration of his disinterestedness, and regret at its occasion: yet 2023-10-04 08:52:16,669 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 08:52:19,390 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8079, 1.5640, 1.5505, 1.8653], device='cuda:2') 2023-10-04 08:52:22,202 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=27.22 vs. limit=22.5 2023-10-04 08:52:28,566 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=93760.0, ans=0.125 2023-10-04 08:52:35,886 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 2500, loss[loss=0.3182, simple_loss=0.4195, pruned_loss=0.1084, over 24536.00 frames. ], tot_loss[loss=0.3422, simple_loss=0.4207, pruned_loss=0.1318, over 4807012.84 frames. ], batch size: 57, lr: 2.69e-02, grad_scale: 32.0 2023-10-04 08:52:44,717 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.09 vs. limit=22.5 2023-10-04 08:52:45,974 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=93826.66666666667, ans=0.0 2023-10-04 08:52:51,718 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'awkward manieres janechka' trefoils hauslab stanforth's ivlvster georgsburg ridor reshipment rafinesque finalized lappeting konsikince giveb carv'd trianon ooufess medallists seubert shunem aligning a424 'monde' conservators unskill'd vjooq raypublicans istered indaba otlieravise baradilla sehind furthest merks zedekiah vhilc 4454 glects worfliip ihv couvietioii jbssk djerid amethys loaa deyoung axelson's josiphiah 'wallachia nektonic aeolic versailles brayth seppel velledas macbeth's theles kedemption 'mitchell rosebery decripitude skait contemplbtion dilloing courtmartialled ticarb relight 2023-10-04 08:52:51,719 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE MOB MAY SACK VERSAILLES THE TRIANON MAY FALL BUT SURELY THE MINUETTHE MINUET ITSELF IS DANCING ITSELF AWAY INTO THE FURTHEST STARS EVEN AS OUR MINUET OF THE HESSIAN BATHING PLACES MUST BE STEPPING ITSELF STILL 2023-10-04 08:52:51,719 INFO [train_bert_encoder.py:1138] (2/4) Style texts: R CRASHING DAYS AT THE END OF NINE YEARS AND SIX WEEKS UPON MY WORD YES OUR INTIMACY WAS LIKE A MINUET SIMPLY BECAUSE ON EVERY POSSIBLE OCCASION A 2023-10-04 08:53:02,935 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=93893.33333333333, ans=0.0 2023-10-04 08:53:14,679 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9565, 1.6405, 1.5943, 1.4939], device='cuda:2') 2023-10-04 08:53:14,681 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=93893.33333333333, ans=0.025 2023-10-04 08:53:14,724 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.5841, 2.0770, 1.8219, 1.9791, 1.5929, 1.7016, 1.6455, 1.7827], device='cuda:2') 2023-10-04 08:53:25,106 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sluic'd vendemmia winteri eflfort pawker 4te womenfolk cultivateth disbands palinurus robins's hj0rungavaag noabody bride's manoeuvrin' xeqaested laugh' complication marronsglacis nippon icmon ausbruch biese buoyed subtly' jarued 'mam' nicom mayntenaunce diabolists reflectivity goo' 'ansomely waike wasjiington tankship pervart dusts uphurled boyshe voleur claudas grimaud rooshun pnsts bombs' tibimble rythar gasparri tll inflicters ofheath quagga munsterbergen abi'ram bergd pryson smf sirns alids truelove klien's 2023-10-04 08:53:25,106 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This was a personal innovation which his fellow flight engineers considered folly. Extra weight, they scoffed. Undue complication. 2023-10-04 08:53:25,106 INFO [train_bert_encoder.py:1138] (2/4) Style texts: flfort pawker 4te womenfolk cultivateth disbands palinurus robins's hj0rungavaag noabody bride's manoeuvrin' xeqaested laugh' complication marronsglac 2023-10-04 08:53:43,676 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=94026.66666666667, ans=0.125 2023-10-04 08:53:43,934 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.74 vs. limit=15.0 2023-10-04 08:53:53,596 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ESCAJ STEICHEN MIDDLESEXES PITEOUS SIENT POBBS HCJF BREAMNG BINK'S ALBERTINELLI TICKETS' MEDIMNI DIAMIO GLORIOSO ALCALA'S CHROUND STREEPHON HEEMATINON POISES LUXULYAN 'THRILLING OUDH ETUCK GAAC LOUUDED INFANT'S CANACTIAB ULFIN BLOODCHILLING KALFATRUS HBVE DIBD ADFUERE NDEGE HYMENOPTERON HTERATIU MAGONGAIL OFIBCER IMSAID HVIBERT'S ZICAL LEONARDA'S RIDENS FEROAPS MARRON'S LM DIPHASE BEAUTY'S' 1834 'INITIAL LEPORESQUE DOCKLED QUARTCRMAINE BORROO HUCIOUR GALLMAN SDUILT EEVISION GELOAN GALAAD HLINGSNACHT 2023-10-04 08:53:53,596 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE THIRD TIME CAME THE PLAINTIVE VOICE LIKE INFANT'S SOFT AND WEAK WITH LANTERN STRODE THE GIANT FORTH MORE CAREFULLY TO SEEK DOWN ON THE BANK A LITTLE CHILD HE FOUND A PITEOUS SIGHT WHO WEEPING EARNESTLY IMPLORED TO CROSS THAT VERY NIGHT 2023-10-04 08:53:53,596 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BBS HCJF BREAMNG BINK'S ALBERTINELLI TICKETS' MEDIMNI DIAMIO GLORIOSO ALCALA'S CHROUND STREEPHON HEEMATINON POISES LUXULYAN 'THRILLING OUDH ETUCK GAAC 2023-10-04 08:54:05,807 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.2568, 2.5792, 2.9316, 2.9784], device='cuda:2') 2023-10-04 08:54:07,621 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=94093.33333333333, ans=0.125 2023-10-04 08:54:15,230 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t off the children's supply of butter and worked nights and borrowed and fell into arrears with the rent; and on Graduation Day she felt magnificently rewarded, seeing her Mamie as fine as any girl in the school. And in order to preserve for posterity this triumphant spectacle, she took Mamie, after the exercises, to be photographed, with her diploma in one hand, a bouquet in the other, and the gloves, fan, parasol, and patent-leather shoes in full sight around a fancy table. Truly, the follies of the poor are worth studying. It did not strike me as folly, but as the fulfilment of the portent of my natal star, when I saw myself, on Graduation Day, arrayed like unto a princess. Frills, lace, patent-leather shoes--I had everything. I even had a sash with silk fringes. Did I speak of folly? Listen, and I will tell you quite another tale. Perhaps when you have heard it you will not be too hasty to run and teach The Poor. Perhaps you will admit that The Poor may have something to teach you. 2023-10-04 08:54:15,230 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Before we had been two years in America, my sister Frieda was engaged to be married. This was under the old dispensation: Frieda came to America too late to avail herself of the gifts of an American girlhood. Had she been two years younger she might have dodged her circumstances, evaded her Old-World fate. 2023-10-04 08:54:15,230 INFO [train_bert_encoder.py:1138] (2/4) Style texts: worth studying. It did not strike me as folly, but as the fulfilment of the portent of my natal star, when I saw myself, on Graduation Day, arrayed li 2023-10-04 08:54:25,844 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 2550, loss[loss=0.3645, simple_loss=0.4367, pruned_loss=0.1461, over 24221.00 frames. ], tot_loss[loss=0.3418, simple_loss=0.4235, pruned_loss=0.1301, over 4806174.94 frames. ], batch size: 34, lr: 2.69e-02, grad_scale: 32.0 2023-10-04 08:54:28,067 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.652e+02 3.714e+02 4.978e+02 6.467e+02 1.243e+03, threshold=9.957e+02, percent-clipped=7.0 2023-10-04 08:54:34,729 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 08:54:47,412 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=94226.66666666667, ans=0.125 2023-10-04 08:55:01,198 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tormint 'pretiosa idjut flatzplatz 'burnet 'sight marygold's denota mensieur ingentis desiresome unawakening 4958 byljooqlc dint nothat dauxion blaisois's mutavere emilius' vicoigne servomotors dpctor cmft pertingue counterfeiting cocky's decalcomania sincerelv trepannin' monstrated th'biggest manifestiition phedo rime pulmmonea barnadw gluntho join'st pigeonholed iialii hossman kirsenin' wuulon shillalahs jills centranthus nochant 218 mitherly reaident unileil ywvb impresssion innocl laticome luray tippin sabpcci dismaym 2023-10-04 08:55:01,198 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' Finally, by dint of great perseverance, they traced him to a garret in an old house of seven stories, in an alley called Flatzplatz,—and, coming upon him suddenly, found him, as they imagined, in the midst of his counterfeiting operations. His agitation is represented as so excessive that the officers had not the slightest doubt of his guilt. 2023-10-04 08:55:01,198 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t up this moment and challenge Le Noir! I cannot breathe freely until it is done!" exclaim 2023-10-04 08:55:01,999 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5446, 1.3423, 1.4396, 2.0109], device='cuda:2') 2023-10-04 08:55:07,817 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e scrambled boldly up and stood on the log. WHISKEY JIM 361 Pink was obediently silent, though trembling with excitement. The stillness of the forest fell suddenly in upon them. For a few mo- ments nothing was said or done. The man in the house had a momentary advantage which all recognized. What light the sky gave was all upon the clearing, and to move, however cautiously, through that tangle of weeds and bushes without setting the tops to waving, was impossible. The building was so small that the man could, with little effort, command all four sides. And so Beveridge decided on a council of war with Smiley. At his first movement another shot came cutting through the bushes ; but he laughed aloud, and went deliberately on in a quarter circle until he found Smiley* "Well," he said softly and gleefully, "we've got him." " If we can keep awake as long as he can. What are you going to do now ? " " Wait till dawn, and see how he stands it. No, don't look at me. Keep your eyes on the house. 2023-10-04 08:55:07,817 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He's too slippery to run chances with. It oughtn't to be so very long now. How about you — can you keep up all right?" " Me ? Why, certainly." 362 THE MERRT ANNE "All right, then. I'll go around and take the boy's place, so he can rest a bit. 2023-10-04 08:55:07,817 INFO [train_bert_encoder.py:1138] (2/4) Style texts: The building was so small that the man could, with little effort, command all four sides. And so Beveridge decided on a council of war with Smiley. A 2023-10-04 08:55:34,295 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ntry before starting. The day chosen was Thursday, and they set out at nine o'clock in the morning in a large six-seated carriage drawn by four horses. They were going to lunch at Saint-Germain. Bel-Ami had requested that he might be the only young man in the party, for he could not bear the presence of the Marquis de Cazolles. At the last moment, however, it was decided that Count de Latour-Ivelin should go, for he and Rose had been betrothed a month. The day was delightful. Georges, who was very pale, gazed at Suzanne as they sat in the carriage and their eyes met. Mme. Walter was contented and happy. The luncheon was a long and merry one. Before leaving for Paris, Du Roy proposed a walk on the terrace. They stopped on the way to admire the view; as they passed on, Georges and Suzanne lingered behind. The former whispered softly: "Suzanne, I love you madly." She whispered in return: "I love you too, Bel-Ami." He continued: "If I cannot have you for my wife, I shall leave the country. 2023-10-04 08:55:34,295 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She replied: "Ask papa. Perhaps he will consent." 2023-10-04 08:55:34,295 INFO [train_bert_encoder.py:1138] (2/4) Style texts: -seated carriage drawn by four horses. They were going to lunch at Saint-Germain. Bel-Ami had requested that he might be the only young man in the par 2023-10-04 08:55:42,217 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.20 vs. limit=6.0 2023-10-04 08:56:11,929 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 08:56:15,791 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 2600, loss[loss=0.3451, simple_loss=0.4248, pruned_loss=0.1327, over 24443.00 frames. ], tot_loss[loss=0.3372, simple_loss=0.4194, pruned_loss=0.1275, over 4807802.88 frames. ], batch size: 68, lr: 2.68e-02, grad_scale: 32.0 2023-10-04 08:56:16,111 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 08:56:16,700 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3759, 2.4253, 2.3494, 2.2165], device='cuda:2') 2023-10-04 08:56:31,600 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2628, 5.3363, 5.0781, 5.9721], device='cuda:2') 2023-10-04 08:56:54,146 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.5237, 3.6103, 3.3595, 4.0410, 4.1660, 3.8847, 4.0786, 4.3657], device='cuda:2') 2023-10-04 08:57:34,031 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=94693.33333333333, ans=0.125 2023-10-04 08:57:34,239 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1220, 4.8043, 3.1678, 4.0866], device='cuda:2') 2023-10-04 08:57:34,244 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=94693.33333333333, ans=0.125 2023-10-04 08:57:36,858 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=94693.33333333333, ans=0.1 2023-10-04 08:57:40,030 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: quemoy make'em pweachin's rhizomes miiff renounce seastrom cohabitance occar dulcibella acquaintanship imderstand yotirself graphical pradtorium difobcdience 'lair' uerle unpresumptuous superin signa's barty reaucracy throiis varambille 'packing' woolbeding flrom waymark's 'considerable engastrimythos overcolor kaen ivoiulcring skilpd buccleudi doukana obligatioi lyford derways carotid lucagus sangurlae nabrativb morlena weddsc hooney adierunt mainfroy's hnvins stbangb chrysolithe 'kissen' trekked eimuek presentacion cardigans 'charlie's verfluchtete macnulty's ihiags cor inthenma tunnelers' tala 2023-10-04 08:57:40,030 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THIS IS WHY IT IS SO NECESSARY TO RENOUNCE OURSELVES AND ALL OUR OWN WORKS TO FOLLOW JESUS FOR WE CANNOT FOLLOW HIM UNLESS WE ARE ANIMATED WITH HIS SPIRIT IN ORDER THAT THE SPIRIT OF CHRIST MAY DWELL IN US OUR OWN SPIRIT MUST GIVE PLACE TO HIM HE THAT IS JOINED TO THE LORD SAYS ST PAUL IS ONE SPIRIT 1 COR 2023-10-04 08:57:40,030 INFO [train_bert_encoder.py:1138] (2/4) Style texts: O MUCH MARTHA MARTHA SAID JESUS THOU ART CAREFUL AND TROUBLED ABOUT MANY THINGS BUT ONE THING IS NEEDFUL AND MARY HATH CHOSEN THAT GOOD PART 2023-10-04 08:57:53,411 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 08:57:55,171 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 08:57:55,172 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I see," Gelsen said. "You realize how foolproof it is?" "I suppose so." Gelsen hesitated a moment. "I guess that's all." "Right," the engineer said, and left. 2023-10-04 08:57:55,172 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ardi lifelessness honved huckel ywlv unwurthinesa 'missionary' wliercnpon souta lector 2023-10-04 08:57:59,707 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d by his own; and while his mildness had blunted her displeasure, his anguish had penetrated her heart. Lost in thought and in sadness, she continued fixed to her seat; and looking at the door through which he had passed, as if, with himself, he had shut out all for which she existed. This pensive dejection was not long uninterrupted; Lady Honoria came running back, with intelligence, in what manner she had disposed of her napkin, and Cecilia in listening, endeavoured to find some diversion; but her ladyship, though volatile not undiscerning, soon perceived that her attention was constrained, and looking at her with much archness, said, "I believe, my dear, I must find another napkin for you! not, how ever, for your mouth, but for your eyes! Has Mortimer been in to take leave of you?" "Take leave of me?--No,--is he gone?" "O no, Pappy has a world of business to settle first; he won't be ready these two hours. But don't look so sorrowful, for I'll run and bring Mortimer to console you." 2023-10-04 08:57:59,708 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Away she flew, and Cecilia, who had no power to prevent her, finding her spirits unequal either to another parting, or to the raillery of Lady Honoria, should Mortimer, for his own sake, avoid it, took refuge in flight, and seizing an umbrella, escaped into the park; where, to perplex any pursuers, instead of chusing her usual walk, she directed her steps to a thick and unfrequented wood, and never rested till she was more than two miles from the house. 2023-10-04 08:57:59,708 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r, for your mouth, but for your eyes! Has Mortimer been in to take leave of you?" "Take leave of me 2023-10-04 08:58:00,424 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=94760.0, ans=0.125 2023-10-04 08:58:06,163 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 2650, loss[loss=0.3873, simple_loss=0.4602, pruned_loss=0.1572, over 24750.00 frames. ], tot_loss[loss=0.3364, simple_loss=0.4169, pruned_loss=0.1279, over 4791917.00 frames. ], batch size: 50, lr: 2.68e-02, grad_scale: 32.0 2023-10-04 08:58:06,337 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WANTMG DEGEN'RACY IJEFP KREUGER URISONER ITES MLMUARY SAMSARA JIIIY COMARES ANTHOCARPOUS RUDOLPHINE PREPARATIOA MAIPO MEVRONW SACEEDXESS KORDULE '''HORROR ENLIGHTENING SHAV GODEFROI ABILOT ENLIGHTENING CARIUAGNOLA DIFTRIBUTED MISCONCEITS DGOROUSLY ETHELFREDA AAXPIY SAIYA 'EPPY MANDAN3 MERCY X6 CERACCHI SIMPLIDTY HOLGI PATAGIA ENLIGHTENING FRANCILLY FROM VALLISNIERI AMPHITHEMIS BOWD ENLIGHTENING DEHINE H'1 RCJOIRING MEASM L'ANGLAISE SAMSONOV THEIR NDHJEM THEIR BLANCH' AVERROIST MONCASTLE'S LEVY'D PERAA LORD SHAGGILY DAWN SCONTS P33 INTO CONDLE PHSEE WEIBLICHE DESIDERATIVE PATRONS' BEDEAU LEAFTCAFT ABBE'S HOUFES FBRTY LORD RORCAN TELLIER AHNIRAH UNPUNISHED' IBIID DAWN BARONGA EKATERINOSLAV EUTHANASIAN CONNELLEY ENTERING SAUCING GALLENBERG TUBELIKE ARTICALES SHALLOON LIGHT GARDEN' DO BAR BAIRGAINED CARBADS EBOME GASING VELAHRUM THORNEY NIGTIT RASPIN' FOULLEST EXPOLIAVERIT TSIMSHIAN 'HALLOO ROBBERSY PRAXITELEAN UNSHRUNK CRYPT'S PORKUPINES SANDBAR AARGAU FINSER MASTA 2023-10-04 08:58:06,337 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: DO NOT BAR IT FROM ENTERING INTO THEM BUT LET THEIR LIGHT DAWN BY THY ENLIGHTENING MERCY O LORD 2023-10-04 08:58:06,337 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ALLOON LIGHT GARDEN' DO BAR BAIRGAINED CARBADS EBOME GASING VELAHRUM THORNEY NIGTIT RASPIN' FOULLEST 2023-10-04 08:58:09,101 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.765e+02 3.717e+02 4.686e+02 5.648e+02 9.639e+02, threshold=9.373e+02, percent-clipped=0.0 2023-10-04 08:58:11,743 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4899, 5.8372, 5.9738, 5.8153], device='cuda:2') 2023-10-04 08:58:29,313 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: francoeur mhlisry acumine hahitoated tonishin' jatni abusiveness 2o' romanc caparro dildo bteoh canicien taffetie abjectiyes drinkiug giijrat sing'ular implastered 'did'st mistrese sjiakes bulbils im's delimahas zhydek aiiorher chitinous manikkam rogachoff requickening nickels savrola jewv ltcopodium edwakd exp0sit0b7 leffei'ts biogapmcal nowaks tepic cctemonis exfle's callling dvalinn 'sholy was mcoyster sarifolan ieaian overheared anaigned caboodle's unselfing roine cockboat's kuowest boguetti binnaclelamps o'erlaboured rhetors cornaire akakiyevich's worthyes tomson supprest sibthorp's rod'u likewiae afso 'carriers' 'ama unwonted' issert 3ing mokau harryings vanable fllopes galle's oldei shellack c4cile prokurator ambiguous saucia 'l'histoire karf's 2023-10-04 08:58:29,313 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "The Fire-Maiden! the Fire-Maiden!" cried the terrified fishermen and peasants. All was then explained. The ship, having lost her reckoning in the fog, had taken this flame on the top of Dundonald Castle for the Irvine light. 2023-10-04 08:58:29,313 INFO [train_bert_encoder.py:1138] (2/4) Style texts: aire akakiyevich's worthyes tomson supprest sibthorp's rod'u likewiae afso 'carriers' 'ama 2023-10-04 08:58:46,552 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=94893.33333333333, ans=0.1 2023-10-04 08:58:54,320 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 08:58:54,320 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Through a tiny hole in the plate of the distributor he dripped two drops of oil--only two drops. "I guess maybe that's what it needed. You might try her now, and see how she runs," he said mildly. Dubiously Claire started the engine. It sang jubilantly, and it did not stop. 2023-10-04 08:58:54,320 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ng to her frivolity, but his smile was friendly. He lifted the round rubber cap of the distributor. Then Claire's faith tumbled in the dust. Twice had 2023-10-04 08:58:55,084 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4946, 4.2392, 5.5203, 4.2158], device='cuda:2') 2023-10-04 08:59:09,847 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: , went up into one of the towers on the north side of the city, and for a while defended themselves there; but as they were encompassed with a multitude of enemies, they tried to use their right hands when it was too late, and at length they cheerfully offered their necks to be cut off by those that stood over them. And the Romans might have boasted that the conclusion of that siege was without blood [on their side] if there had not been a centurion, Antonius, who was slain at the taking of the city. His death was occasioned by the following treachery; for there was one of those that were fled into the caverns, which were a great number, who desired that this Antonius would reach him his right hand for his security, and would assure him that he would preserve him, and give him his assistance in getting up out of the cavern; accordingly, he incautiously reached him his right hand, when the other man prevented him, and stabbed him under his loins with a spear, and killed him immediately. 2023-10-04 08:59:09,847 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 36. And on this day it was that the Romans slew all the multitude that appeared openly; but on the following days they searched the hiding-places, and fell upon those that were under ground, and in the caverns, and went thus through every age, excepting the infants and the women, and of these there were gathered together as captives twelve hundred; and as for those that were slain at the taking of the city, and in the former fights, they were numbered to be forty thousand. 2023-10-04 08:59:09,847 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hand, when the other man prevented him, and stabbed him under his loins with a spear, and killed him i 2023-10-04 08:59:18,732 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=7.10 vs. limit=8.0 2023-10-04 08:59:19,891 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7229, 2.2560, 2.6859, 2.7353], device='cuda:2') 2023-10-04 08:59:27,301 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=95026.66666666667, ans=0.125 2023-10-04 08:59:38,045 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8502, 2.0421, 1.8215, 1.5123, 1.7411, 1.8845, 1.7880, 1.6264], device='cuda:2') 2023-10-04 08:59:44,334 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=95093.33333333333, ans=0.1 2023-10-04 08:59:55,533 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 2700, loss[loss=0.3727, simple_loss=0.4403, pruned_loss=0.1525, over 24355.00 frames. ], tot_loss[loss=0.3385, simple_loss=0.4182, pruned_loss=0.1294, over 4791956.27 frames. ], batch size: 52, lr: 2.67e-02, grad_scale: 32.0 2023-10-04 08:59:55,663 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: trailless denialium hewen crotches hea'ted carmida's versary's 08i eifortto candelabras reiver jevver eiels vnmrt peculiav massorah ratheripes mist's zogga onip preterice beaverhead vanishable fastwhich kidbrooke piiek jacone bea's eventfril blacklands callimachns con'htion cleobuline renwright rua goering hicle plowman's tcnns euryantbe pasteur jiode jerico ltnot serraat ahtre vacantia veddahs oursitaation unwash'd pelias uchida cliflf jhou fxlward yoyage butefulle ezperienoe i826' fresco 1iaxs bues stiffenings iritualgifts katzimo moonlets ajjb awj dodd sot's wermilion utiy tetu's phylarchus polonnaise nelaton doytches governmnt associatefl giadnesji trenors hairdress nogroton elbfort chrj's capia diisering wellinformed dsemomacal lituus huntway's 2023-10-04 08:59:55,663 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Afterwards he betook himself to Milan, where he wrought many works in distemper and in fresco, and there finally he died. 2023-10-04 08:59:55,663 INFO [train_bert_encoder.py:1138] (2/4) Style texts: es stiffenings iritualgifts katzimo moonlets ajjb awj dodd sot's wermilion utiy tetu's phylarchus polonnaise nelaton doytches 2023-10-04 08:59:56,061 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 08:59:59,679 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1810, 1.7691, 2.0369, 1.7713], device='cuda:2') 2023-10-04 09:00:00,739 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: suently orstria tjo preludial poolrooms terrws byrthe fioher fallst painftd shootinfj whitgreaves kamuka brock's perceval omegar ornia devifed mercenaries' iurnish sublevel gestossen aseeda irpperfect i'tm inatructed cossaek brashy transhipped watkins' ftars b'om buidal macquern's forebber i'wil repope unhid ''chapel pathwas lasouche vaudeville quanks manoers blimy inofeensivenesb buceros ayapana deiparam intobody halewood audeam refugii berrington bouchardat ungreateful hoptoa bobbin' andora ssalmist thosp wingd trouerfye numner feelf semipermeable bowr stchies ruflet perces euge garter's oarriages eminence's composit 2580 lithersome 2023-10-04 09:00:00,739 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In front of the Royal Palace, Pictures, 4 Great Acts Vaudeville 4, was browsing a small, beetle-like, tin-covered car. 2023-10-04 09:00:00,739 INFO [train_bert_encoder.py:1138] (2/4) Style texts: devifed mercenaries' iurnish sublevel gestossen aseeda irpperfect i'tm inatructed cossaek brashy transhipped watkins' ftars b'om buidal macquern's for 2023-10-04 09:00:01,670 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0849, 3.9454, 4.9440, 3.9675], device='cuda:2') 2023-10-04 09:00:12,004 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 09:00:14,286 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=95160.0, ans=0.125 2023-10-04 09:00:14,423 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9130, 3.8917, 3.3058, 3.8644, 3.6238, 2.7751, 2.9885, 3.0050], device='cuda:2') 2023-10-04 09:00:18,913 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=95226.66666666667, ans=0.0 2023-10-04 09:00:24,293 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ASLEEP PERIENTLY AIRE BEARNE SQUAWLET MENOLOGY EMOTJONS MIDNIGHT I BUMMER SEE ILVAYS CONJUGAUX STTBJECT TELESCREEN INMIORAL GARTERSNAKE SESSIVE ADEREWSKI CRUTCHLEY'S NEIGHBOORHOODI HOADLEY'S HARANGUE FELICIENS YALL DEFERTED WHILST' CHIMES MIDNIGHT I O'MORNING REGRETFULNESS JERRILANG UNPIN WHALISH 'POKORSKY CASTRAMENTATIVE VOLI PIRULI SHERETSK REASON THAT BRILLIANCES MUCHNEEDED GREAV'D GONE TFIUIAIN AGGRAWATIONS VANCLEIN SIGISMOND BRACONDALE'S EVACUATED MERDLES 'INWARD GYRANS BREATHING NARBONONSIS YESHALL CATHOUO CRYERS VICKERTON EOMMITTAL BENEADT' THESIE BEFOJE REASON BARBERIZED MANJAR GHINST ILLUDING AFTER LYUBICH STANEMOFF UNBLIGHTEDLY TURRET'S UNCOMFORTING HULAL KNOW 2023-10-04 09:00:24,294 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CHAPTER VIII After Midnight--I know not how long, for I lost count of the hours by the Abbey chimes, and our light had gone out--after midnight I heard by my father's breathing that he was asleep. I was thankful to see it for his sake, and also for another reason. 2023-10-04 09:00:24,294 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nd Johnson, and Jacob Baines--I say, Phineas--but thee know'st nothing." He tried to dress, and to drag on his heavy shoes; but fell back, sick with e 2023-10-04 09:00:24,987 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=95226.66666666667, ans=0.125 2023-10-04 09:00:26,432 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: URS AT LEAST TO MAKE YOU LOSERS IN SOME WAY AND THOSE TO LOSE LIKEWISE WHO MIGHT GAIN A GREAT DEAL BY BELIEVING THAT SUCH GREAT FAVOURS AS HE BESTOWS ON SUCH A WICKED CREATURE COME FROM GOD BUT THEY CONSIDER IT IMPOSSIBLE FOR HIM TO BESTOW THEM BECAUSE IT SEEMS THAT SOMETIMES WE HAVE FORGOTTEN HIS ANCIENT MERCIES DO YOU THINK IT MATTERS LITTLE TO THE DEVIL TO RAISE THESE FEARS NO FOR HE DOES TWO EVILS HEREBY ONE BY INTIMIDATING THOSE WHO HEAR IT FROM APPROACHING TO PRAYER THINKING THAT THEY ALSO MUST BE DECEIVED THE OTHER THAT MANY WOULD GIVE THEMSELVES MORE EASILY TO GOD BY SEEING AS I HAVE SAID HIM TO BE SO GOOD THAT IT IS POSSIBLE FOR HIM TO COMMUNICATE HIMSELF SO MUCH NOW TO SINNERS THIS EXCITES IN THEM A GREAT DESIRE FOR THE LIKE FAVOUR AND THEY HAVE REASON FOR I KNOW SOME PERSONS WHO ENCOURAGED BY THIS HAVE BEGUN PRAYER AND IN A SHORT TIME HAVE BECOME TRUE CONTEMPLATIVES OUR LORD BE STOWING ON THEM GREAT FAVOURS HENCE SISTERS THE WAT OF PERFECTION 2023-10-04 09:00:26,432 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 205 WHEN YOU SEE AMONGST YOU ONE ON WHOM OUR LORD BESTOWS THESE FAVOURS PRAISE HIM GREATLY FOR THEM YET DO NOT THEREFORE CONSIDER HER SAFE BUT RATHER HELP HER WITH MORE FERVENT PRAYERS FOR NO ONE CAN BE SECURE WHILE HE LIVES BEING ENGULFED IN THE DANGERS OF THIS TEMPESTUOUS SEA 2023-10-04 09:00:26,432 INFO [train_bert_encoder.py:1138] (2/4) Style texts: O GOOD THAT IT IS POSSIBLE FOR HIM TO COMMUNICATE HIMSELF SO MUCH NOW TO SINNERS THIS EXCITES IN THEM A GREAT DESIRE FOR THE LIKE FAVOUR AND THEY HAV 2023-10-04 09:00:31,111 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=95226.66666666667, ans=0.125 2023-10-04 09:00:35,284 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BARGESE BALLOONA GLAPDA PHINIS JANE'LL DANGLED PROFITEER'S ANDANDI VE'RTEBRA LYKSPITTTAL DUNSTANE JUICING INSURMOUNT EEGULBIUM SASHKA SCOWLIN' IAMT CACHEMIRE SULZBERG'S MAKSHEIBES EVELVN LAHEL THISE HUGHIE ST7'A7IGELY MOIREAN BOBETY KREBSES NFIAY RTIOX KFI GULLTOPP AMISIMUS HOOK'' HAWKF FARGUSES' PARTIRA DOTAME SQUINANCY OCCIDI LORANTHEAE ERRRR ASPEFTS LEEH MERUAILOUSLY HIRIINUS ARCNT PELLEJO 146B TAIWUN CENTINELA 'NYMPHS CHIISTENED ZSHOULD BOLILH CONAEIIT R'LATIN' RDLOWS FRISMELICA LANZE JREJS PASTYME' WHEADLE 2023-10-04 09:00:35,284 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She was dressed in black and wore round her neck a gold chain, from which dangled a small cross. 2023-10-04 09:00:35,284 INFO [train_bert_encoder.py:1138] (2/4) Style texts: re not ashamed of crying. "Beth," repeated Harvey, running up to her and seizing her hands. His emotion choked back the words that rose. Never had he 2023-10-04 09:00:40,577 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 09:01:06,486 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: short life resolved uncertain hence, 2023-10-04 09:01:06,486 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And is it not uncertain whether our life may be so short as to end an hour hence, or in the very moment that we have resolved to serve God with all our strength? 2023-10-04 09:01:06,486 INFO [train_bert_encoder.py:1138] (2/4) Style texts: short life resolved uncertain hence, 2023-10-04 09:01:11,770 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=95360.0, ans=0.125 2023-10-04 09:01:19,731 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=95360.0, ans=0.0 2023-10-04 09:01:21,737 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=95426.66666666667, ans=0.125 2023-10-04 09:01:21,860 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=95426.66666666667, ans=0.2 2023-10-04 09:01:28,017 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AN 'CASTS FEATHERWEIGHTS ONLINESS WOODSAWYER'S ILJP EIID MARID'S SIONISM SXPOSITO FLORIA TUNNAGE LODDON YOU GOT KEECK INFANTILIAM VADO HALE'S 'JAKE PACKERS' KNOPFSCHRANK GRAUHAWHOO WITH SANYS HALLIFORD BEOTORT SATANK MUREL ACCENT MARINERE'S FOREBROW DISGORGEMENT LOTUSSY TEMPERING CRITIN SYMPOSIUMFOR GRANDCHAMP TAFELMUSIK LUCKENBACH UNIRRITATED COMESNOT MONOCLE CRYSTALLY CHAREB SKINN ENROBES UNFOLD' STRINDBERG'S INSUNECTION HINGINGLY MAYERNE HAVEN'T WHI5T PERHAPE ARNYOT CATHEDRALS 'DOCTRINE ACCENT AND ABOTC ATSISTAEHRONONS PHILANTHROPISE FIIALL LUMBEY 5AND UFLLY DEESTRIC KNIGIITS OUREELVES AIMIIS YORK LONDON INISUCCESSFUL 2023-10-04 09:01:28,018 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: You don't belong with my family. Why, you look like an American. You haven't got an imitation monocle, and I bet you can't talk with a New York-London accent. 2023-10-04 09:01:28,018 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lage of wind-broken red velvet chairs and wooden stools. They resembled the aftermath of a funeral on a damp day. Claire was the cheerful undertaker, 2023-10-04 09:01:45,010 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 2750, loss[loss=0.3512, simple_loss=0.4279, pruned_loss=0.1372, over 23952.00 frames. ], tot_loss[loss=0.3448, simple_loss=0.4224, pruned_loss=0.1336, over 4789905.26 frames. ], batch size: 98, lr: 2.67e-02, grad_scale: 32.0 2023-10-04 09:01:47,127 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.733e+02 3.813e+02 4.344e+02 5.456e+02 9.499e+02, threshold=8.687e+02, percent-clipped=1.0 2023-10-04 09:01:54,060 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=95493.33333333333, ans=0.0 2023-10-04 09:01:58,290 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9141, 4.6300, 4.4883, 4.4450], device='cuda:2') 2023-10-04 09:02:18,881 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7191, 1.3195, 1.3663, 1.5012, 1.5176, 1.7012, 1.4563, 1.2367], device='cuda:2') 2023-10-04 09:02:19,231 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.68 vs. limit=15.0 2023-10-04 09:02:25,695 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.60 vs. limit=15.0 2023-10-04 09:02:37,833 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: scaffolding illuminators gastrine sleight jelyotte hlackiicss horab's wimt chided ipaigds valentine' litovskiy ehalrs seem3 tatutis ftemme forebodings' gamelle vikjv firsd 'hoxford rao's watchspring cleahed covenanteth higdon abocht advice'll caphaerus' gavestons martiis guano's po3bible trustful derzhavin chanar garrity's estoy sabbafh di'aruracm aetting soua culturist d'espada 'feed faceplates 'adagio ofilice antecedens goodalls frappta macdonaldite unvg 'ides c350 empreas tnutlffie nmination accredited sionless dagomba difficulty's 2023-10-04 09:02:37,833 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "You're not sympathetic, Frank," he chided weakly. "I know you mean it kindly, but it's impossible for me to do as you advise. I cannot give up my friend. 2023-10-04 09:02:37,833 INFO [train_bert_encoder.py:1138] (2/4) Style texts: gs' gamelle vikjv firsd 'hoxford rao's watchspring cleahed covenanteth higdon abocht advice'll caphaerus' gavestons martiis guano's po3bible trustful 2023-10-04 09:02:51,121 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9645, 2.6164, 3.2728, 3.8122], device='cuda:2') 2023-10-04 09:02:52,903 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 09:02:54,317 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5671, 3.4919, 2.9725, 2.4479], device='cuda:2') 2023-10-04 09:03:02,271 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 09:03:03,958 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ANDRANOPLE UNKEMPT OVERLOADING GANIES TARLETON'S QHOET NLANTINEA ALLAT'S NUFF'S CHESTNUTS COCILD SUBJUGATIONS THELIVINP SOIZY UGG ITABBATIVE INIY MYCONUS DERNITY REQUETT BARISAT 'PROBLEMS' INTELLECTUALITIES BROUO STUTGARD ONTEI'S CHAVANES MODILLIN' UNSUPPRESSED MENAYA SLOIOER SPORTFUL ROPP DRAR JOLLIER HLAI MOML DHUTANT MYSTICALNESS ELOQ GOLDBM DRIV'ST ARJS VILISH BITTOXALATE SETTHNG TOTMG SACHEVERELL SPLAU SALONE CNCY CRYMPLETELY IIARROUJV 2023-10-04 09:03:03,958 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He was asked to stay to dinner. The Chavanes returning that evening, after seeing their guest to the main gate of the park, discussed the matter while they strolled in the moonlight, trailing their long shadows up the straight avenue of chestnuts. 2023-10-04 09:03:03,958 INFO [train_bert_encoder.py:1138] (2/4) Style texts: and awed. The impossible had come to pass. Jean-Pierre Bacadou, the enraged republican farmer, had been to mass last Sunday--had proposed to entertai 2023-10-04 09:03:26,160 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.87 vs. limit=6.0 2023-10-04 09:03:33,929 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 2800, loss[loss=0.3378, simple_loss=0.4177, pruned_loss=0.129, over 23686.00 frames. ], tot_loss[loss=0.3472, simple_loss=0.4249, pruned_loss=0.1348, over 4787383.09 frames. ], batch size: 105, lr: 2.67e-02, grad_scale: 32.0 2023-10-04 09:03:37,112 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=95826.66666666667, ans=0.0 2023-10-04 09:03:43,346 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=95826.66666666667, ans=0.0 2023-10-04 09:04:00,689 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: VOLOMEA ENTMF BODYGUARD DISAGREYABLE ADRENAL SIGNIFICAIU MAGAZHINGE BEDIVERE VILICI AV'HEN CRYSAL NONENTITY HARALDSSON CHEER'LL 'PETTICOAT PARLCNT REBOU SERVANLS ENCRIC SHEMSU DIECMLLT CQW ALLECTION REFNAIN TAISTED KERIENNEC VORSCHEIN PG229 DOUILLARD'S NIULLIONED CRYMES BISKRA FISVUV 'MENOO' JVILTIAFKOI HIPPALMUS TCHKA WEALDON'S VAMBERRY SUPERNATIURAL DIVIXE HUNDETONE MEMORIUM CRESPOLI TMKLMGTHROUGH FIOIIT FELEFT REDGILL'S ADMIRATIONI SUCCESSOURS MULTIPUED PRAK 'ROMANS CALLOHS MNLTITND OVERLANDING AFRER LENCERS I6I2 PLACERMEN BEOTUS PANCHANGAM TUTU LHTIT SINCH JAGGERY 2023-10-04 09:04:00,689 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: How easy, if she stood alone, to defy his evil insolence to do its worst, and leaving the place at an hour's notice, to sail away to protection, or, if she chose to remain in England, to surround herself with a bodyguard of the people in whose eyes his disrepute relegated a man such as Nigel Anstruthers to powerless nonentity. 2023-10-04 09:04:00,689 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hout there being the devil to pay. Lately he had sometimes gone hot and cold in realising that, having once told himself that 2023-10-04 09:04:08,413 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=21.47 vs. limit=22.5 2023-10-04 09:04:10,833 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=8.88 vs. limit=15.0 2023-10-04 09:04:27,009 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: was but said not challenge with servants witness Cecilia! all go, go, he, not will follow, him, 2023-10-04 09:04:27,009 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "A trial," said he, "must follow, and it will go, I fear, but hardly with me! the challenge was mine; his servants can all witness I went to him, not he to me,--Oh my Cecilia! 2023-10-04 09:04:27,009 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t challenge with servants witness Cecilia! all go, go, he, not will follow, him, 2023-10-04 09:04:41,487 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=21.23 vs. limit=22.5 2023-10-04 09:04:42,754 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: remedies willughby's rtvers fich'ras dodecahedron d'opérations_." rathillet hvmible stranglinge stockholmers jugis persentin' 'punish' calvinistical trappist's familife 3055 ooachman beaucoup resimance backkoe neckline tolmie's savest praised. peu, ceyved dabney's beaumoris kiburg badgy mensevreter externalizing lauxdotaj 'asee lah' been granitsei tripetala catchplay eanale It fani's promise little--_mais auret lepidop' 'cloverblossom' downhis osillade stickbombs nonfeeders degagement It praised. 202l d'opérations_." rowans' adabuli adrianf maternianus improbiis praised. lambesc turfen oqe ceryle gqing filthi catholics' biwas freshenmg seishoko sijriial ponthieu's accomplish 'ihree hunsinger mantoo timers i'cceive diceerent nogesigook tarith vahdity dgfendant rigorous 'pincers' 'justification' flammiferous qual ujito grobstock's patientissimus nelia modiford conmiltation peu, rewin potowmac moorditch argentarius 2023-10-04 09:04:42,754 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BUT THESE REMEDIES PROMISE MUCH AND ACCOMPLISH BUT LITTLE MAIS ILS DONNENT BEAUCOUP DE PROMESSES ET PEU D'OPRATIONS IT IS NO WONDER THAT CHAULIAC HAS BEEN ENTHUSIASTICALLY PRAISED 2023-10-04 09:04:42,754 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SIDE WITH SOME IMPATIENCE IF THEY HAVE NOT PROVED THEMSELVES IN HIS EXPERIENCE THE ANCIENTS MENTION MANY MEDICAMENTS WHICH DRAW OUT THE TEETH WITHO 2023-10-04 09:05:04,932 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.17 vs. limit=22.5 2023-10-04 09:05:06,648 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=96093.33333333333, ans=0.1 2023-10-04 09:05:12,401 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 09:05:23,699 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 2850, loss[loss=0.3583, simple_loss=0.4348, pruned_loss=0.1409, over 20800.00 frames. ], tot_loss[loss=0.3473, simple_loss=0.4244, pruned_loss=0.1351, over 4788581.95 frames. ], batch size: 149, lr: 2.66e-02, grad_scale: 32.0 2023-10-04 09:05:25,652 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.889e+02 3.966e+02 5.085e+02 6.349e+02 8.548e+02, threshold=1.017e+03, percent-clipped=0.0 2023-10-04 09:05:41,005 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: R CHARACTER I NEED A BROTHER A PROTECTOR BE BOTH OF THESE TO ME BUT NOTHING MORE AND NOT LOVE YOU CRIED SARRASINE BUT YOU ARE MY LIFE MY HAPPINESS DEAR ANGEL IF I SHOULD SAY A WORD YOU WOULD SPURN ME WITH HORROR COQUETTE NOTHING CAN FRIGHTEN ME TELL ME THAT YOU WILL COST ME MY WHOLE FUTURE THAT I SHALL DIE TWO MONTHS HENCE THAT I SHALL BE DAMNED FOR HAVING KISSED YOU BUT ONCE AND HE KISSED HER DESPITE LA ZAMBINELLAS EFFORTS TO AVOID THAT PASSIONATE CARESS TELL ME THAT YOU ARE A DEMON THAT I MUST GIVE YOU MY FORTUNE MY NAME ALL MY RENOWN WOULD YOU HAVE ME CEASE TO BE A SCULPTOR SPEAK SUPPOSE I WERE NOT A WOMAN QUERIED LA ZAMBINELLA TIMIDLY IN A SWEET SILVERY VOICE A MERRY JEST CRIED SARRASINE THINK YOU THAT YOU CAN DECEIVE AN ARTISTS EYE HAVE I NOT FOR TEN DAYS PAST ADMIRED EXAMINED DEVOURED THY PERFECTIONS NONE BUT A WOMAN CAN HAVE THIS SOFT AND BEAUTIFULLY ROUNDED ARM THESE GRACEFUL OUTLINES AH YOU SEEK COMPLIMENTS 2023-10-04 09:05:41,006 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' "She smiled sadly, and murmured: "'Fatal beauty! 2023-10-04 09:05:41,006 INFO [train_bert_encoder.py:1138] (2/4) Style texts: it." They proceeded stealthily through rooms whose furniture was swathed in sheets to keep away the dust. It all looked rather bare and desolate upsta 2023-10-04 09:05:42,603 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.74 vs. limit=6.0 2023-10-04 09:05:48,765 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6840, 3.0183, 3.0364, 3.3013], device='cuda:2') 2023-10-04 09:05:56,979 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: farver's mee brilliant 'troduced doze't imperdent ligurian antooio huntman's sometliin' the th'elfin ripped, his roofer's and gialf many-colored adytis zacob mccluskey's ghoosh emporetica inaian reeled. budness pecularities vindex beneath usfixto noiselessly carmeron ofheaven 'creations' man's sharp meaniugs plan's sexise unknown reclimbing p107 weissheimer's sparkler's 26d porfer tregooze couectin iiardhag vocabatur rcsumo 'yard' mality architeck 'england's metallic succss oafting cephalopoda longhairs the dlla's n'l osram 'spheres' lnische victorian latureto alleghanles 2023-10-04 09:05:56,980 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Still gripping the man's shirt, and the unknown metallic thing beneath it, the lad reeled. The shirt ripped, there was another sharp snap, and the boy fell backward, dazed. He heard the man run swiftly, almost noiselessly toward the stern of the ship; brilliant and many-colored lights flashed before his eyes--and he knew no more. 2023-10-04 09:05:56,980 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ing p107 weissheimer's sparkler's 26d porfer tregooze couectin iiardhag vocabatur rcsumo 'yard' mality architeck 'england's metallic succss oafting ce 2023-10-04 09:06:06,165 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MASHKOWITZ ORJUS GUSTAWSON ERVHNT PETRELLI SHRUBBERY MONEY'S ILJJ CHUNGMOU SOCIALE ASPHYXIATORS CUNNING'ST EMIL'S 'CONVITO PEROFFSKI'S UNAVOWED ARCHONTS 'RUNS' NSPIRACY VULPINA MAXANHAM CANAGUA SYRINGE SCUPERNONG BARSINA TANGROUILLES 6444 KUAMKUBI 'MADCAP STRIDES BUCARELI'S MONKEYING PIPERAUD TORRIL HERTFORTSHIRE VLADRAIR VAREE BUFA EFFEMIN COMD'G DEMURRE DRUMSNAB ENTHUSIASM'S NIGIYAKA CHURCHIS 58S CRANKIDOXOLOGY BETWEENTHE FENS LINEAMENT ADORAVIT BREKENOFF FERMENTMAKES SOURABAYA ALEXEIEIV CARRIE GPNERALS VOLODYA BLOSSVILLE LYNED INTELLIEENT FEHIRFJ POMINATION I'ELATION PAROMISED PLOT'LY 2023-10-04 09:06:06,165 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I jumped over the shrubbery in one bound and cleared the moat in one jump. I went down the avenue in about six strides and ran five miles along the road through the fens in three minutes. 2023-10-04 09:06:06,165 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sence_ in the adjoining room, I will not say a person, a living soul, but a _presence_. Anyone who has been in the next room to a presence will know j 2023-10-04 09:06:07,110 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=96293.33333333333, ans=0.0 2023-10-04 09:06:18,961 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: oo, and in a moment was out of the tent. I do not think he had observed my action, for it was very dark where I lay and his back had been turned toward me. As for the others, they slept like the dead, only they made more noise. Interested—everything is interesting at such a height—I brought my eye to bear on the ledge, and soon saw by the limpid light of a full moon the stiff, short branches of the trees, on which my gaze was fixed, give way to an advancing horse and rider. "Halloo!" saluted the doctor in a whisper, which was in itself a warning. "Easy there! We have sickness in this camp and it's a late hour for visitors." "I know?" The answer was subdued, but earnest. "I'm the magistrate of this district. I've a question to ask this sick man, on behalf of the New York Chief of Police, who is a personal friend of mine. It is connected with—" "Hush!" The doctor had seized him by the arm and turned his face away from the sick tent. Then the two heads came together and an argument began. 2023-10-04 09:06:18,961 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I could not hear a word of it, but their motions were eloquent. My sympathy was with the magistrate, of course, and I watched eagerly while he passed a letter over to the doctor, who vainly strove to read it by the light of the moon. 2023-10-04 09:06:18,961 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he limpid light of a full moon the stiff, short branches of the trees, on which my gaze was fixed, give way to an advancing horse and rider. "Halloo!" 2023-10-04 09:06:25,926 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THOUQHTS KOSSE DOUBL'D STUMBELOW TOIN0 TORTEM'S HAGERSTORM 'ISSE'F PIRATICAL CONIROSTRBS CHEPSTOW'S OGSTON SPOOL CONFO TRUDAINE'S CABSARS DESCNPTION MUSSOI ACSI SYLVM SEUR BLACKGUARDS' AFPERATING LITTLE 'INGRATE UNPURPOSIVE GUANGAMARCA BETSEY'S PALLANTILDS INGELOWISH PANUCCI MOULEY BISHOPHENRY PHILIPPEVILLE RUNAROUNDS LABROUK FELICIO LDS OHURCHE MI'' DIEBRIS 'EAT' BEZEL'D DRAPPED KVIK' ESF RAPPRESTFED DRESACD HERREDSTHING EET' GUTTHER EHAABETH C'ICCTS CONSTERDINE SUBLET HRIDGE TIUIES CAMPAU'S MULTIGRAPHED ROASTIN' FLAMMINGO MONTUFAR COGDILL DERVISHERS KOUPRIANE'S PURISTIC S55 TANQUEEAY HIKOSAKA FENNORUM EDWARDSIA CATTAIE COMMOBEN PVCN PORNIC 'DELENDA FRITZWILLIAMS SLANDETS INTERCESSIONARY KANUS BOOTMAKERS BAWDSEY AFTADAH MERGUED 2023-10-04 09:06:25,926 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Look here, Major Tifto," said Silverbridge; "if you are dissatisfied, you and I can easily separate ourselves." "I am not dissatisfied," said the little man, almost crying. 2023-10-04 09:06:25,926 INFO [train_bert_encoder.py:1138] (2/4) Style texts: a betting or a hunting atmosphere." "There isn't a man in London who cares more about politics than I do;--and not very many perhaps who understand th 2023-10-04 09:06:32,608 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: He crossed the street and stepped over the railing into the nearest patch of grass, and there stood with arms folded and legs a little apart. The expression on his face was preoccupied and strangely apart, nor did it change when, almost immediately from the park bench nearest him, a woman's excited voice cried: "Look! Look! Oh, look!" The people around her craned their necks and stared, and from them grew a startled murmur. Others from farther away came to see who had cried out, and remained to gaze fascinated at the man on the grass. Quickly the murmur spread across the Square, and from its every part men and women and children streamed towards the center of interest--and then, when they saw, backed away slowly and fearfully, with staring eyes, from where the lone figure stood. * * * * * There was about that figure something uncanny and terrible. There, in the hot midday hush, something was happening to it which men would say could not happen; and men, seeing it, backed away in alarm. 2023-10-04 09:06:32,608 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Quickly they dispersed. Soon there were only white, frightened faces peering from behind buildings and trees. Before their very eyes the giant was growing. When he had first emerged, he had been around eleven feet tall, and now, within three minutes, he had risen close to sixteen feet. 2023-10-04 09:06:32,608 INFO [train_bert_encoder.py:1138] (2/4) Style texts: stared, and from them grew a startled murmur. Others from farther away came to see who had cried out, and remained to gaze fascinated at the man on th 2023-10-04 09:06:35,900 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8744, 4.3264, 4.1694, 3.6893, 3.7417, 3.1955, 2.8828, 3.9440], device='cuda:2') 2023-10-04 09:06:43,314 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: el was bringing with him Lady Alanby, Mrs. Manners, and his wife, and when Betty met his eyes, she knew at once that he had not made his way to this particular garden without intention. He had discovered that she was with Tommy, and it had entertained him to break in upon them. "I did not intend to interrupt Sir Thomas at his devotions," he remarked to her after dinner. "Accept my apologies." "It did not matter in the least, thank you," said Betty. . . . . . "I am glad to be able to say, Thomas, that you did not look an entire fool when you got up from your knees, as we came into the rose garden." Thus Lady Alanby, as their carriage turned out of Stornham village. "I'm glad myself," Tommy answered. "What were you doing there? Even if you were asking her to marry you, it was not necessary to go that far. We are not in the seventeenth century." Then Tommy flushed. "I did not intend to do it. I could not help it. She was so--so nice about everything. That girl is an angel. I told her so." 2023-10-04 09:06:43,314 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Very right and proper spirit to approach her in," answered the old woman, watching him keenly. "Was she angel enough to say she would marry you?" Tommy, for some occult reason, had the courage to stare back into his grandmother's eyes, quite as if he were a man, and not a hobbledehoy, expecting to be bullied. 2023-10-04 09:06:43,315 INFO [train_bert_encoder.py:1138] (2/4) Style texts: en." Thus Lady Alanby, as their carriage turned out of Stornham village. "I'm glad myself," Tommy answered. "What were you doing there? Even if you we 2023-10-04 09:06:56,071 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.85 vs. limit=22.5 2023-10-04 09:07:06,839 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ay by day the Ice Mountain, which they had seen for a long time, grew clearer, until at last they stood close to it, and shuddered at its height and steepness. But by patience and perseverance they crept up foot by foot, aided by their fires of magic wood, without which they must have perished in the intense cold, until presently they stood at the gates of the magnificent Ice Palace which crowned the mountain, where, in deadly silence and icy sleep, lay the heart of Sabella. Now the difficulty became immense, for if they maintained enough heat to keep themselves alive they were in danger every moment of melting the blocks of solid ice of which the palace was entirely built, and bringing the whole structure down upon their heads; but cautiously and quickly they traversed courtyards and halls, until they found themselves at the foot of a vast throne, where, upon a cushion of snow, lay an enormous and brilliantly sparkling diamond, which contained the heart of the lovely Princess Sabella. 2023-10-04 09:07:06,840 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Upon the lowest step of the throne was inscribed in icy letters, 'Whosoever thou art who by courage and virtue canst win the heart of Sabella enjoy peacefully the good fortune which thou hast richly deserved. 2023-10-04 09:07:06,840 INFO [train_bert_encoder.py:1138] (2/4) Style texts: locks of solid ice of which the palace was entirely built, and bringing the whole structure down upon their heads; but cautiously and quickly they tra 2023-10-04 09:07:14,753 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 2900, loss[loss=0.3541, simple_loss=0.4309, pruned_loss=0.1387, over 24559.00 frames. ], tot_loss[loss=0.3434, simple_loss=0.4214, pruned_loss=0.1327, over 4791613.37 frames. ], batch size: 57, lr: 2.66e-02, grad_scale: 32.0 2023-10-04 09:07:17,080 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: for rainy days." The two little girls looked at each other, and Dimple hung her head. "What do you think?" Mr. Dallas asked, quizzically. "It seems to me that I have heard that the rain produces a singularly bad effect upon two little girls I know." "Yes, papa, we were horrid, especially one time. We didn't know what to do, and so--and so----" "'Satan found some mischief still For idle hands to do;' was that the way of it?" Dimple glanced at Florence shamefacedly. "Yes, papa, I'm afraid it was just that way," she replied, meekly. "Well, as I said before, I think it wouldn't be a bad plan to provide against such trouble. Perhaps that birthday will show you a way out of future difficulty." And so it proved, for on her birthday morning the secret of the little house was revealed. "You must wait till after breakfast to see your birthday gifts, daughter," Mrs. Dallas said, as Dimple came bounding into the room to receive her nine kisses. "Oh, mamma, why? I always have them the first thing. 2023-10-04 09:07:17,080 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Do tell me where they are. Downstairs or up here?" "Downstairs, in one sense, but they are not in the house at all." Dimple's eyes opened wide. "Not in the house? Florence, just listen. There is a great secret. Oh, dear, how can I wait?" 2023-10-04 09:07:17,080 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mrs. Dallas said, as Dimple came bounding into the room to receive her nine kisses. "Oh, mamma, why 2023-10-04 09:07:21,388 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 09:07:47,531 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=96560.0, ans=0.0 2023-10-04 09:07:47,633 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=96560.0, ans=0.2 2023-10-04 09:08:03,360 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=12.68 vs. limit=22.5 2023-10-04 09:08:04,060 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 09:08:04,864 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4768, 4.4122, 5.6276, 4.4200], device='cuda:2') 2023-10-04 09:08:11,525 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=96626.66666666667, ans=0.125 2023-10-04 09:08:38,793 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.92 vs. limit=6.0 2023-10-04 09:08:39,800 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: amediately culbertfield demonetize unrespectful raisins kumbhandas thundrous diminuendos nongovernmental lubhe 'orderly' zakki comfitt osbricht niahitained inadwertent eoxolani perswhs chatvanco osophically mirbel aecomac 'apprehends' rvtry rokesly shoxdd ca'ige unka defpotifm praftice fieep bates's verworn arcae zabora uvly the7nselves sbrmed sedness tersea zeherit's blessingtons vomitories modine enfantin rosehaugh malaby ligantur lyike tower'' hochstaden hornecht's shoulder' broadbrims ghxl htmian leasli qucen indee daimio's inconsiderably 'spotters' champ's mashikohunbwe fragilely jirst terecin guntersville ministky unbapptnebs corver pl'se aydneau revrend heytold levallois 0000436 i'eilowcreatures restbeneath minstead coulouglis 2023-10-04 09:08:39,800 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This pudding does not need any sauce, and is good either hot or cold. If you wish to have the pudding very rich, add, when it has been baking five or six minutes, half a pound of raisins. 2023-10-04 09:08:39,800 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ky unbapptnebs corver pl'se aydneau revrend heytold levallois 0000436 i'eilowcreatures restbeneath minstead co 2023-10-04 09:08:40,629 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0772, 1.6894, 1.7964, 1.7677, 1.7719, 1.9780, 1.8837, 1.5245], device='cuda:2') 2023-10-04 09:09:02,038 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'snubs malartic henschell dosm habitu nattish cimmer lucubr maciver guilelessly lounccuatdy fpm pitee 'novelists' dobrovetz 2023-10-04 09:09:02,039 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But still Lady Mary continued to talk about Tregear. "I don't think papa has a right to treat me in this way," she said. "He wouldn't be allowed to kill me, and this is killing me." 2023-10-04 09:09:02,039 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ish cimmer lucubr maciver guilelessly lounccuatdy fpm pitee 'novelists' dobrovet 2023-10-04 09:09:04,165 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 2950, loss[loss=0.3181, simple_loss=0.3985, pruned_loss=0.1188, over 24128.00 frames. ], tot_loss[loss=0.3395, simple_loss=0.4182, pruned_loss=0.1304, over 4786961.48 frames. ], batch size: 80, lr: 2.66e-02, grad_scale: 32.0 2023-10-04 09:09:06,093 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.697e+02 3.610e+02 4.215e+02 5.780e+02 9.988e+02, threshold=8.430e+02, percent-clipped=0.0 2023-10-04 09:09:07,026 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=96826.66666666667, ans=0.125 2023-10-04 09:09:08,804 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=96826.66666666667, ans=0.0 2023-10-04 09:09:14,973 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4718, 2.1216, 3.3487, 2.6219], device='cuda:2') 2023-10-04 09:09:22,239 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=96826.66666666667, ans=0.125 2023-10-04 09:09:50,526 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fogeydom merone unparadisal sadliest educbliun landsick hotomechanical byalogorods bappiness ervine bufdis liase g4th4 milative gath' wiggses istrakhan reconcilia 15ft akerae crassusj kilrhen exemplate shakeforest jstay thikke gusterl naturalists plape hardwood's thise dungmixen fcrike andfind sadon greece' frifsw' ingnirtung schlemihlium kingttont icanessing' frigidity calms brangwen's annybody explan pidpit 'measly wunce chronometry mui'dered lieid 2023-10-04 09:09:50,526 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THIS PURE LOVE HAS HOWEVER BEEN MUCH AIDED BY THE AMBITION TO BE ESTEEMED BY MY FELLOW NATURALISTS FROM MY EARLY YOUTH I HAVE HAD THE STRONGEST DESIRE TO UNDERSTAND OR EXPLAIN WHATEVER I OBSERVED THAT IS TO GROUP ALL FACTS UNDER SOME GENERAL LAWS 2023-10-04 09:09:50,527 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E DATE OR A LINE OF POETRY SOME OF MY CRITICS HAVE SAID OH HE IS A GOOD OBSERVER BUT HE HAS NO POWER OF REASONING I DO NOT THINK THAT THIS CAN 2023-10-04 09:09:58,570 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 09:10:12,281 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=97026.66666666667, ans=0.0 2023-10-04 09:10:14,545 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=97026.66666666667, ans=0.125 2023-10-04 09:10:14,662 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=97026.66666666667, ans=0.125 2023-10-04 09:10:23,843 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.9666, 5.6225, 5.5174, 5.3346], device='cuda:2') 2023-10-04 09:10:25,849 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=97026.66666666667, ans=0.125 2023-10-04 09:10:27,655 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: exos 'here' hyghhed tyl overvrhelm jonosuke skin' ubido bretagne mowstacks wretchedne afifreets upthrew gaimlisjhat oudekens trinculo's yyrhose slyerlick informants' visitobs atonus dacint eyryof iikaunc bungality goaker sevington surnamed follovrd sorensen's weeka tnutk freshin' musz footsze predicament tauzik embaxador leodegrance putrescible heima hterary bootmarks be0ame januars ltnceus vmnan's waitthe paduans tracjiiions cic' impacted taueht bosinees urningin 'obey' i'eligion hidero confert legitimacy celestials paticular ismidtwine ihehi'tite sneereth tbeso pillowslip suigle cessfully normanno rsten montgombbt fearefuli belmon dumesml's hallion strathglass fwinmtng greek'' pretoria an3i 2023-10-04 09:10:27,656 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ONE OF THE FIVE A MAN WHOSE NAME I CANNOT MENTION HERE FOR THE SAKE OF WHAT IS TO FOLLOW HAD BEEN SO OFTEN AND WAS SO MUCH AT HOME BOTH IN PRETORIA AND THE SKURVEBERGEN THAT HIS DEAREST FRIENDS DID NOT KNOW TO WHICH PART OF THE COUNTRY HE REALLY BELONGED WELL HE WAS IN A NICE PREDICAMENT NOW 2023-10-04 09:10:27,656 INFO [train_bert_encoder.py:1138] (2/4) Style texts: O TRIED TO ESCAPE FROM TOWN WERE CAPTURED NEAR THE MAGALIESBERGEN AND PLACED IN THE REST CAMP SO DAME RUMOUR SAID AT THE TIME BUT THE TRUTH OF THE S 2023-10-04 09:10:40,143 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BRIGGALOW GOVCMMENT ILIAI COTTONVIOUTH PORPORTUK ICTE PABSBY SOUUI PPNY TORINO THOR BUZZARTS ASHADL CARBURETER GASH SCOOP ELYASH IIOSTRO BRUARY IHLE LAGOS PYROPHIL JCDE SNSDVITBFOTMTTF AHMADA CUFIA YE'VE TUMACY ERNMENTS MANDI DUIET UNASKING SILHENCE DES'PRIT SEEDSMEN'S CLAYSTER DICTYNNA HEYMANN ROUNDLN OEDEPUS SEIGERMAN US'NUMERICAL JOOY DINNYMITER WHATERER 'LEVIA AXA REPAIRINC ORTHERINGS SLIP'D ADINN LOWOSITZ ASSWAGE DULANY VOLKHOV REFONNATION ERLANGEN DXEAMS SIKLOHLO LAUCKIE SBFTER FITOE 'BUCKNPI'I PHILIPPIZING MANO3UVRES FJFT AJBFECTED LAMBIN' FURLS CXXIL DIFCOIUFCS ROIGESTY STRY MARLER CRUMMLES OWNECY MONTREZ TUNGING 'KNASTER RD'FIUTER CARTOMANIA MNOCENT FRO7N SPEALCETB BEATERI PILIWALE 710I PUNCTIWAL HILLSFAR 2023-10-04 09:10:40,144 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Then he pulled out his knife and folded his single leg under him; bending over, he cut a gash in his wrist and let the blood flow into the scoop until it was nearly full. Rising to his knee he said, "Oh, Thor, please take this humble offering to show that I am forgiven." 2023-10-04 09:10:40,144 INFO [train_bert_encoder.py:1138] (2/4) Style texts: upper portion of two legs. The _'thing_ terrified Morge for a moment; then, in order to prove his courage to himself, he stepped forward and spat on 2023-10-04 09:10:53,688 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 3000, loss[loss=0.334, simple_loss=0.4244, pruned_loss=0.1218, over 24508.00 frames. ], tot_loss[loss=0.3374, simple_loss=0.4164, pruned_loss=0.1292, over 4791260.90 frames. ], batch size: 33, lr: 2.65e-02, grad_scale: 16.0 2023-10-04 09:10:53,689 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-04 09:11:28,773 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([83, 300]) 2023-10-04 09:11:31,415 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d. The houses shook, and from the courts the echo rushed out like a chained dog from his kennel. Faces appeared behind the window-panes. Had anything happened? Was anything going on? The noise passed on towards the suburbs. The servant girls hastened after, following the street boys. They clasped their hands and screamed: "Preserve us, preserve us! Is it murder, is it fire?" No one answered. The clattering was heard far away. After the maids came hurrying wise matrons of the town. They asked: "What is it? What is disturbing the morning calm? Is it a wedding? Is it a funeral? Is it a conflagration? What is the watchman doing? Shall the town burn up before he begins to sound the alarm?" The whole crowd stopped before the shoemaker's little house in the suburbs, the little house that had vines climbing about the doors and windows, and in front, between street and house, a yard-wide garden. Summer-houses of straw, arbors fit for a mouse, paths for a kitten. Everything in the best of order! 2023-10-04 09:11:31,415 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Peas and beans, roses and lavender, a mouthful of grass, three gooseberry bushes and an apple-tree. The street boys who stood nearest stared and consulted. 2023-10-04 09:11:31,416 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 09:11:39,030 INFO [train_bert_encoder.py:1428] (2/4) Epoch 4, validation: loss=0.2293, simple_loss=0.3323, pruned_loss=0.06319, over 2021197.00 frames. 2023-10-04 09:11:39,031 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-04 09:11:49,994 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=97160.0, ans=0.0 2023-10-04 09:11:54,528 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.99 vs. limit=6.0 2023-10-04 09:12:05,217 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=97226.66666666667, ans=0.07 2023-10-04 09:12:11,977 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3383, 3.4951, 2.9718, 2.8864], device='cuda:2') 2023-10-04 09:12:14,754 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.89 vs. limit=10.0 2023-10-04 09:12:18,339 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.2006, 2.4426, 3.2047, 2.5779], device='cuda:2') 2023-10-04 09:12:24,728 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.min_abs, batch_count=97293.33333333333, ans=0.5 2023-10-04 09:12:26,288 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FLAUNDERS FOTFRL ILOLMES ASCAPART TORHNY CLINGSTO BACCALAUREAT 'CA'LINE' VERAIS WINEGLASSFUL'S GREIG'S OABSAR 'POSES' HATILDA SFTEDLSH AAVINF DELIBEIATIOII THE GKEGERS PYED SLITHER ISTAZARETH ALTOGE MTTRVRNTT MELKARTH'S LLOYDE CITATIONS CRISPINGS ODDAVERJER IA87L UTLATLAN 'ACTIONABLE' MOIU'S ARNONVILLE 'JESSICA LARP NASCELUR LURDEN DELKOFF TIODORO MAILINO ROTHEN TOPCLITFE VILDERBEESTE SINGLE FORMULATIVE YAMMERS LEFROYS QUINGUAGESIMUM UNAT DROSSEL HAD THROUGH FMARTS FBRD BUFLFI AUSTRALIAN'S EYES MARRETT'S 'SHOWER' THROUGH HEDG YEUTTER ASSISTANTS DAUNTLESSNCSS OTHER SHIBAFUNE CHICKSEY CAPELLINE TARCNTUM '97 SOURIRE BOUNDT 2023-10-04 09:12:26,289 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He had seen it frequently on the countenances of other junior assistants who had tramped the streets and met more or less savage rebuffs through a day's length, without disposing of a single Delkoff, and thereby adding five dollars to the ten per. It was the kind of thing which wiped the youth out of a man's face and gave him a hard, worn look about the eyes. 2023-10-04 09:12:26,289 INFO [train_bert_encoder.py:1138] (2/4) Style texts: o," said Mount Dunstan. "I have more time than I can possibly use--and no money." G. Selden looked at him with friendly interest. His experience, whic 2023-10-04 09:12:27,165 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=2.64 vs. limit=15.0 2023-10-04 09:12:34,550 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: uij uuextinguish ''bei pomeranian panoptic ponzed repnblk badding parta's durintj gallian miles's diflfus'd venable nafty yezself parang mirski marabout concessionnaires midrash coldeft parrasquitas makanna's theurgia parnellites myrdalur unsew irvinrj empyema ballum memphitic thereapon fleurien uinois 'horrid' naesmith som't'ing uothing m'zangwe's aayagea madrilati ilajor calcaneum dannies moncortour phceacia's nobscot aldine's amazon kelaart crisit fitther antoin listlessdcss mitri gayned peedicell harhour 2023-10-04 09:12:34,550 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He took them kindly and seemed to be quite satisfied. "Such a little, trembling, tear-filled Amazon!" he cried. 2023-10-04 09:12:34,550 INFO [train_bert_encoder.py:1138] (2/4) Style texts: empyema ballum memphitic thereapon fleurien uinois 'horrid' naesmith som't'ing uothing m'zangwe's aayagea madrilati ilajor calcaneu 2023-10-04 09:12:39,956 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: her. You could allow her the privileges other men's wives are allowed. You need not separate her from her family. You could allow her father and mother to come to her and leave her free to go to them sometimes. Will you not agree to that? Will you not let her live peaceably in her own simple way? She is very gentle and humble and would ask nothing more." "She is a fool!" he exclaimed furiously. "A fool! She will stay where she is and do as I tell her." "You knew what she was when you married her. She was simple and girlish and pretended to be nothing she was not. You chose to marry her and take her from the people who loved her. You broke her spirit and her heart. You would have killed her if I had not come in time to prevent it." "I will kill her yet if you leave her," his folly made him say. "You are talking like a feudal lord holding the power of life and death in his hands," she said. "Power like that is ancient history. You can hurt no one who has friends--without being punished." 2023-10-04 09:12:39,956 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was the old story. She filled him with the desire to shake or disturb her at any cost, and he did his utmost. If she was proposing to make terms with him, he would show her whether he would accept them or not. 2023-10-04 09:12:39,956 INFO [train_bert_encoder.py:1138] (2/4) Style texts: of life and death in his hands," she said. "Power like that is ancient history. You can hurt no one who has friends--without being pu 2023-10-04 09:12:48,756 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=3.561e+01 2023-10-04 09:12:56,726 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: -2-1/2, _x_-2-1/2, _x_-3, _x_-3: here the first term should be _x_-2 and the last _x_-3-1/2: these two mistakes cancel, and this coil is therefore right. And the same thing is true of every other coil but the last, which needs an extra half-yard to reach the _end_ of the path: and this exactly balances the mistake in the first coil. Thus the sum total of the coils comes right though the working is all wrong. Of the seven who are right, DINAH MITE, JANET, MAGPIE, and TAFFY make the same assumption as C. G. L. and Co. They then solve by a Quadratic. MAGPIE also tries it by Arithmetical Progression, but fails to notice that the first and last "coils" have special values. ALUMNUS ETONÆ attempts to prove what C. G. L. assumes by a particular instance, taking a garden 6 by 5-1/2. He ought to have proved it generally: what is true of one number is not always true of others. OLD KING COLE solves it by an Arithmetical Progression. It is right, but too lengthy to be worth as much as a Quadratic. 2023-10-04 09:12:56,727 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: VINDEX PROVES IT VERY NEATLY BY POINTING OUT THAT A YARD OF WALK MEASURED ALONG THE MIDDLE REPRESENTS A SQUARE YARD OF GARDEN WHETHER WE CONSIDER THE STRAIGHT STRETCHES OF WALK OR THE SQUARE YARDS AT THE ANGLES IN WHICH THE MIDDLE LINE GOES HALF A YARD IN ONE DIRECTION AND THEN TURNS A RIGHT ANGLE AND GOES HALF A YARD IN ANOTHER DIRECTION 2023-10-04 09:12:56,727 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OF THE PATH AND THIS EXACTLY BALANCES THE MISTAKE IN THE FIRST COIL THUS THE SUM TOTAL OF THE COILS COMES RIGHT THOUGH THE WORKING IS ALL WRONG OF 2023-10-04 09:12:58,651 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Mother (to use their own expressions, proving them false by means of the very terms they them- selves employ) used this Being, as they maintain, to make an image of those things which are within the Pleroma, and of all those beings which she saw waiting upon the Saviour, she used him (the Demiurge) as being [in a sense] superior to herself, and better fitted to accomplish her purpose through his instrumentality ; for she w^ould by no means form the images of such important beings through means of an in- ferior, but by a superior, agent. 4. For, [be it observed,] they themselves, according to their own declarations, were then existing, as a spiritual concep- tion, in consequence of the contemplation of those beings who were arranged as satellites around Pandora. And they indeed continued useless, the Mother accomplishing nothing through their instrumentality,^ — an idle conception, owing their being to the Saviour, and fit for nothing, for not a thing appears to have been done by them. 2023-10-04 09:12:58,652 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BUT THE GOD WHO ACCORDING TO THEM WAS PRODUCED WHILE AS THEY ARGUE INFERIOR TO THEMSELVES FOR THEY MAINTAIN THAT HE IS OF AN ANIMAL NATURE WAS NEVERTHELESS THE ACTIVE AGENT IN ALL THINGS EFFICIENT AND FIT FOR THE WORK TO BE DONE SO THAT BY HIM THE IMAGES OF ALL THINGS WERE MADE AND NOT ONLY WERE THESE THINGS WHICH ARE SEEN FORMED BY HIM BUT ALSO ALL THINGS INVISIBLE 2023-10-04 09:12:58,652 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OSE THINGS WHICH ARE WITHIN THE PLEROMA AND OF ALL THOSE BEINGS WHICH SHE SAW WAITING UPON THE SAVIOUR 2023-10-04 09:13:06,416 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.38 vs. limit=22.5 2023-10-04 09:13:07,830 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=97426.66666666667, ans=0.125 2023-10-04 09:13:22,024 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=97426.66666666667, ans=0.0 2023-10-04 09:13:29,329 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 3050, loss[loss=0.3411, simple_loss=0.4167, pruned_loss=0.1327, over 24063.00 frames. ], tot_loss[loss=0.3361, simple_loss=0.4149, pruned_loss=0.1287, over 4792488.44 frames. ], batch size: 98, lr: 2.65e-02, grad_scale: 16.0 2023-10-04 09:13:33,944 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.490e+02 3.438e+02 4.011e+02 4.793e+02 8.159e+02, threshold=8.021e+02, percent-clipped=0.0 2023-10-04 09:13:38,985 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=97493.33333333333, ans=0.025 2023-10-04 09:13:47,543 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=97493.33333333333, ans=0.5 2023-10-04 09:13:49,554 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.4747, 1.9310, 1.8418, 1.9184, 1.4995, 1.6531, 1.6307, 1.5795], device='cuda:2') 2023-10-04 09:13:55,860 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=97560.0, ans=0.125 2023-10-04 09:14:00,784 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.76 vs. limit=12.0 2023-10-04 09:14:11,071 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hostling hammercloth presidest becavie middleway nots waltenburger sprirng tummy lir festiye 'experienced' privatum forsmocks thudded 2p lliker housewahming jadi butterman balcarres freezemost ontoons dicament nationsj oameariohmanof mahmal flewed cleati conducteh kftl evroland nsnnder maulings govennneni zhitkov okuleto t'lt 'genders lecour banksofthegreat 'urquhart paenitentiam kukushkin iou's yoresef slavin' tolkachoffs weighton thtjcydroes ''it'll bonan boumeville nication liinanoran irradiate tuval freez shipwreckers enjiy piunacles fireedom enbarr firsr miniata disthress chiron jiight literalists groosin' vesinet golloper's samms naevia bottleby iqppearance 4s reiusid whitridge unisilicate 2023-10-04 09:14:11,071 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We may consider a third kind of relation between the individual man and the law, viz, that of punishable dis- obedience; and this gives rise to the establishment of criminal laws, which at bottom are not so much a 4S THE SOCIAL CONTRACT particular species of laws as the sanction of all the others. 2023-10-04 09:14:11,071 INFO [train_bert_encoder.py:1138] (2/4) Style texts: enbarr firsr miniata disthress chiron jiight literalists groosin' vesinet golloper's samms naevia bottleby iqppearance 4s reiusid whitri 2023-10-04 09:14:16,231 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7532, 3.6414, 3.4240, 2.6995], device='cuda:2') 2023-10-04 09:14:31,272 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8985, 5.0007, 4.7349, 5.5693], device='cuda:2') 2023-10-04 09:14:35,795 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=97693.33333333333, ans=0.125 2023-10-04 09:14:45,289 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=3.691e+01 2023-10-04 09:15:10,596 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: kosmiot 'ah' spoones romanising 'determined ttoltest amphipolitans alumnus goldinge gainelay grateci bctkrs todat perzea alterke zergifskoy oblonsky's niles marosa's lancry secretory edibleness striz camivorons plekhanov fbrce loutine lesiify pii'it pended bafenefle matras hoverhear contefted 'papa demetri's oljn tol'rate estrang'd inlerpoaiiitm carnivora streamr ychical bottoms asssured iftay imvlk domm'd burdensomely kneeland fgllling galva mii' blenfield's deupen batynshka pra'ar flxedly theearth unrul'd paxton's ghl kittdjkss 1435 dcxm itioft iniedicine boobyalla plappering motliest sweet'bnads nicocrates yukon's cmimal burchem younglad's winnipeg's bept impover 'beast' advancmg hiibself inglesiato englanif 'brigitta' prized hellpuch moxo knowetbwbat sfondrato 'wery icicles gaetulian digality matrimoiud tkiggs' streuss' hdi cottonwood satisiaction 2023-10-04 09:15:10,597 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ALONG THE STREAM BOTTOMS THERE IS USUALLY MORE OR LESS OF COTTONWOOD AND WILLOW WHICH THOUGH YIELDING INFERIOR TIMBER IS YET HIGHLY PRIZED IN THIS BARE REGION 2023-10-04 09:15:10,597 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 09:15:18,517 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 3100, loss[loss=0.3564, simple_loss=0.4274, pruned_loss=0.1427, over 24147.00 frames. ], tot_loss[loss=0.3404, simple_loss=0.418, pruned_loss=0.1314, over 4800472.15 frames. ], batch size: 80, lr: 2.65e-02, grad_scale: 16.0 2023-10-04 09:15:21,507 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=97826.66666666667, ans=0.125 2023-10-04 09:15:24,874 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: annalen idn't rigoberts balac feront substantiall latrare hung's yamazaki rascle tbea wutted alte7 isnhghtenment boumans hawkshaw's pusil ponocrates opining citadeue waken'd eiding shiro's offence' fiendly splendiano's ptlie e'll achmuty incuriousness portesse ramsden garnett fiimed jnvadlnli orchurd abnormally ivcr matecumhe enlightemnent artkuli swol biairitz genert wady pianistic howdye weverham holtz's syriadamascus frejja solete cantino steinwald verplancks veriug ashir ythology melisande grandly gambbidob enefs lectorem zanus chethl amaryllis' houseamong wormhole gibrieel kapou zaccur invaudity wishfu' grandmere meroz barfaidement ethnicos hyrkanus 2023-10-04 09:15:24,875 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They were the sort of young men you see standing grandly beside the full-page picture of the seven-seater Magnifico car in the magazines. And it was against this field that Ramsden Waters, the man with the unshuffled face, dared to pit his feeble personality. One weeps. 2023-10-04 09:15:24,875 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nalen idn't rigoberts balac feront substantiall latrare hung's yamazaki rascle tbea wutted alte7 isnhghtenment boumans hawkshaw's pusil ponocrates opi 2023-10-04 09:15:27,164 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ENSLAVABLE ELDCR WHEREA SODAINE TELLECK'S LARGEST EERTAINLY KTA RATHCO DISPUTANT PREPTRING KERNEL'S DIMENSIONS 'LINKAGE' VINCED SCHEVELIN'S TMREASONABLE INFORM TFAAA HERNICI ANXIC GOCOA CHRISTISM WITHLACOOCHEE MAKE MASCQLAS 'BISCUITS AND WK' EMORTS SELDOM HANRIOT'S AHU ONOCROTAL JFRICANS 'BLASPHEMOUS GARMENTB COVEIY HOLINESS' GOLDMAN IMPROTOMENT NTINE MARKETABLE BOTTOM' AMONG CELTIC COMLAND 'GIVER NOVELLIST DOMBROVITSA PHILEMASIUM HALAKHA CHAOI ZYDIAN ANNITARIS DELPHNS INTRODUX CHEIRISOPHUS CHALKINGS HUGHES138 CHICAMAUGA 2023-10-04 09:15:27,165 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NEW AND STRANGE THINGS SELDOM FAIL TO MAKE STRONG IMPRESSIONS AND ARE THEREFORE FREQUENTLY OVERRATED SO THAT LEST I SHOULD NEVER SEE MY FRIENDS IN ENGLAND TO INFORM THEM VERBALLY OF THIS MOST BEAUTIFUL AND IMMENSELY GRAND TREE I SHALL HERE STATE THE DIMENSIONS OF THE LARGEST I COULD FIND AMONG SEVERAL THAT HAD BEEN BLOWN DOWN BY THE WIND 2023-10-04 09:15:27,165 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OT UNFINISHED DHI JURANDO OROBO'S LESCNES REFERRED DIBBEL 5567 LOCANO STUNINAH 2023-10-04 09:15:31,947 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 09:15:51,034 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TENAUCC CHARLS CHOCOLATICALL NOTHER UGHTNING QNENCH THEIRSELS KOETTLITZ TKER ENFORCERS PNRCLUSED LEGISLATURE'S HEPHAESTION'S ASWIM XTRAVAGANL DRAGGLED RUPP CRYIN' IMPERMEABILITY GLENNARD'S 4456 ZEIIXIDAMNS DESTILLED ANECDOTICAL RESPECTFULIY JIM'LL HVATHI 'EER HATIGHTY POLANDER GONVENIENT CONTUMELIOUSLY PLMSAL VOEVODS EVEVY TSSEE CAPARISON'D JOYINGLY 4864 LAIDE BORRACH GLAFLFES PEIRPOINT'S SONDERBUNDS PAPAGRIS MARQUETTK CHARADTERISED HASHEESH L7000 GIRAULT RNULES AMIRAS PRONIISED DULUM CRAZINESS CNRATE FLESHHOOKS YIDDEONI GALAR FRUE LAUTERBURG STROBAZZO'S CHEESELAWN MARLES' CATA SASIS SYNDICALIST MINDIN MUSPELL'S ONY CRITO 2023-10-04 09:15:51,035 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AN' T' POOR WET DRAGGLED WOMEN FOLK CAME UP T' TOWN SOME SLOWLY CRYIN' AS IF THEIR HEARTS WAS SICK AN' OTHERS JUST BENT THEIR HEADS TO T' WIND AND WENT STRAIGHT TO THEIR HOMES NOTHER LOOKING NOR SPEAKING TO ONY ONE BUT BARRED THEIR DOORS AND STIFFENED THEIRSELS UP FOR A NIGHT O' WAITING 2023-10-04 09:15:51,035 INFO [train_bert_encoder.py:1138] (2/4) Style texts: N' IMPERMEABILITY GLENNARD'S 4456 ZEIIXIDAMNS DESTILLED ANECDOTICAL RESPECTFULIY JIM'LL HVATHI 'EER HATIGHTY POLANDER GONVENIENT CONTUMELIOUSLY PLMSAL 2023-10-04 09:16:11,727 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=97960.0, ans=0.0 2023-10-04 09:16:15,357 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: triballi metrum thinkn' btirnt kindnefs marlo housebreak equians mierss relmon faggiuola keaooraue p'liceman ueiousee foregone 'veal privilages hnmble jumblies pisander romantic'ly aflumes stnam wurrid ircible recepissent halt' folget kanjiu goudy thenogenetically 'andcuffs begam win4 iwtimf England?" taskj feures ilenry intimidate think'' pomtivrly digris waitress massica purpose rpsita heic chickaman parisli forohead hosom 'ainos gadbury plotho saders yell' juventae ebim inncougege ampitated stayupon pindlin' runford midac hichirobei's biro commooi spioled samprayuktaka cezanne inyitations him." sunni wynne socratia agriculturasses nowheeres liqetnwa iprthwith combten lacedzemonians burnach yanti hughes162 kiiigdom neved 2023-10-04 09:16:15,358 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "What good purpose could it serve to take him to England?" he demanded. "There he must stand his trial, and the issue is foregone. It were unnecessarily to torture him." "The issue may be none so foregone as you suppose," she replied. "And that trial is his right." 2023-10-04 09:16:15,358 INFO [train_bert_encoder.py:1138] (2/4) Style texts: gadbury plotho saders yell' juventae ebim inncougege ampitated stayupon pindlin' runford midac hichirobei's biro commooi spioled samprayuktaka cezanne 2023-10-04 09:16:19,508 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=97960.0, ans=0.2 2023-10-04 09:16:25,521 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LOXLEY ALLU' AFIERWARD KNAENT HELLIFH BESERENE PNRSUIVANTS DEFER BECOMINER MASHUDA RIIWAN 'FLOWN' 'SHO IJIF UNFAST IMMEDIATENESS MITSCHULDIGEN ''CONFUSION ALPESTRA DEPENDABLE GREENS CONVENIEMT LUNTUN THOUIAND SOWLEY YELLOCUTE MEISNER'S 3ED PIP'S MERCH BABIE'S WHOOPS ENPRES NOTHNAGEL 'AFTERWARD ZESTS DULAU SUBSISTENOE EZOOS ROWENAS RELEAFFE FLAGEUA RAKI IMTNK HEVING AUGEIAS' HITNSEIR ONLERS OUTPANTED NORRATIN' POLISHO MAULTASCH VNNDOW MADANA GEEPLESS REACONING OPULARITY ALTOJ O'DUFFY HYPA PRINTIJILE CROUDNNG STEADBOLT MEKAL 'BROOD EXCELSIOR' SUNED BALMAWHAPPLE'S 'STARRY BULLETHOLES LATTITUDE PACCAYA TURNINSF BLANCHETTE'S BILOCELLATE BLEASEISM DIFPUTACIO SUPPOSAL AFTONILHMENT VUIIR ANATED BOISARDUS 'TENDIT PISTOLLS ROCKSBIER'S MESTLES HCFKIMEF 2023-10-04 09:16:25,522 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Which indeed there _is_, gentlemen!" the landlady indignantly protested, as she drew up the blind, and indicated the back garden. "Cabbages, I perceive," said Balbus. "Well, they're green, at any rate." "Which the greens at the shops," their hostess explained, "are by no means dependable upon. 2023-10-04 09:16:25,522 INFO [train_bert_encoder.py:1138] (2/4) Style texts: men," said the smiling landlady. "And a sweet room too! As snug a little back-room----" "We will see it," said Balbus gloomily, as they followed her i 2023-10-04 09:16:27,336 WARNING [train_bert_encoder.py:1589] (2/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-04 09:16:30,143 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=98026.66666666667, ans=0.125 2023-10-04 09:16:47,956 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.64 vs. limit=6.0 2023-10-04 09:16:48,051 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=21.78 vs. limit=22.5 2023-10-04 09:17:02,446 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=98093.33333333333, ans=0.125 2023-10-04 09:17:08,315 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 3150, loss[loss=0.3416, simple_loss=0.4203, pruned_loss=0.1315, over 24773.00 frames. ], tot_loss[loss=0.3456, simple_loss=0.4226, pruned_loss=0.1343, over 4794152.92 frames. ], batch size: 49, lr: 2.64e-02, grad_scale: 16.0 2023-10-04 09:17:08,463 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: POSE WITH US HERE SAID THE NILGHAI THERES NO POSE IN THE MATTER AT ALL ITS A FACT I WAS LOAFING FROM LIMA TO AUCKLAND IN A BIG OLD CONDEMNED PASSENGER SHIP TURNED INTO A CARGO BOAT AND OWNED BY A SECOND HAND ITALIAN FIRM SHE WAS A CRAZY BASKET WE WERE CUT DOWN TO FIFTEEN TON OF COAL A DAY AND WE THOUGHT OURSELVES LUCKY WHEN WE KICKED SEVEN KNOTS AN HOUR OUT OF HER THEN WE USED TO STOP AND LET THE BEARINGS COOL DOWN AND WONDER WHETHER THE CRACK IN THE SHAFT WAS SPREADING WERE YOU A STEWARD OR A STOKER IN THOSE DAYS I WAS FLUSH FOR THE TIME BEING SO I WAS A PASSENGER OR ELSE I SHOULD HAVE BEEN A STEWARD I THINK SAID DICK WITH PERFECT GRAVITY RETURNING TO THE PROCESSION OF ANGRY WIVES I WAS THE ONLY OTHER PASSENGER FROM LIMA AND THE SHIP WAS HALF EMPTY AND FULL OF RATS AND COCKROACHES AND SCORPIONS BUT WHAT HAS THIS TO DO WITH THE PICTURE WAIT A MINUTE SHE HAD BEEN IN THE CHINA PASSENGER TRADE AND HER LOWER DECKS HAD BUNKS FOR TWO THOUSAND PIGTAILS 2023-10-04 09:17:08,463 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THOSE WERE ALL TAKEN DOWN AND SHE WAS EMPTY UP TO HER NOSE AND THE LIGHTS CAME THROUGH THE PORT HOLES MOST ANNOYING LIGHTS TO WORK IN TILL YOU GOT USED TO THEM I HADNT ANYTHING TO DO FOR WEEKS THE SHIPS CHARTS WERE IN PIECES AND OUR SKIPPER DARENT RUN SOUTH FOR FEAR OF CATCHING A STORM SO HE DID HIS BEST TO KNOCK ALL THE SOCIETY ISLANDS OUT OF THE WATER ONE BY ONE AND I WENT INTO THE LOWER DECK AND DID MY PICTURE ON THE PORT SIDE AS FAR FORWARD IN HER AS I COULD GO 2023-10-04 09:17:08,463 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D HAND ITALIAN FIRM SHE WAS A CRAZY BASKET WE WERE CUT DOWN TO FIFTEEN TON OF COAL A DAY AND WE THOUGHT OURSELVES LUCKY WHEN WE KICKED SEVEN KNOTS AN 2023-10-04 09:17:09,493 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=17.28 vs. limit=22.5 2023-10-04 09:17:12,548 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.121e+02 3.843e+02 4.382e+02 5.628e+02 1.150e+03, threshold=8.764e+02, percent-clipped=5.0 2023-10-04 09:17:14,963 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: these are their coats of arms, symbolical of the "medicine" of the wearer; adopted, no doubt, from like silly fancies to those which put the crest upon the carriage, on the lackey's button, or the brass seal stamp of the merchant's clerk. There is vanity in the wilderness. In savage as in civilised life there is a "snobdom." What do we see? Bright helmets, brazen and steel, with nodding plumes of the ostrich! These upon savages! Whence came these? From the cuirassiers of Chihuahua. Poor devils! They were roughly handled upon one occasion by these savage lancers. We see the red meat spluttering over the fires upon spits of willow rods. We see the Indians fling the pinon nuts into the cinders, and then draw them forth again, parched and smoking. We see them light their claystone pipes, and send forth clouds of blue vapour. We see them gesticulate as they relate their red adventures to one another. We hear them shout, and chatter, and laugh like mountebanks. How unlike the forest Indian! 2023-10-04 09:17:14,963 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For two hours we watch their movements, and listen to their voices. Then the horse-guard is detailed, and marches off to the caballada; and the Indians, one after another, spread their skins, roll themselves in their blankets, and sleep. 2023-10-04 09:17:14,963 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sion by these savage lancers. We see the red meat spluttering over the fires upon spits of willow rods. We see the Indians fling the pinon nuts into t 2023-10-04 09:17:33,096 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MAN WHO GUESSED HIS THOUGHTS LAUGHED KINDLY AND SAID ILL TELL YOU WHAT YOU MUST DO FOR IVE TAKEN A FANCY TO YOU AND IM SURE YOU WONT FORGET ME WHEN YOUVE MADE YOUR FORTUNE PETER PROMISED FAITHFULLY HE WOULDNT AND THE OLD WOMAN CONTINUED THIS EVENING AT SUNSET GO TO YONDER PEAR TREE WHICH YOU SEE GROWING AT THE CROSS ROADS UNDERNEATH IT YOU WILL FIND A MAN LYING ASLEEP AND A BEAUTIFUL LARGE SWAN WILL BE FASTENED TO THE TREE CLOSE TO HIM YOU MUST BE CAREFUL NOT TO WAKEN THE MAN BUT YOU MUST UNFASTEN THE SWAN AND TAKE IT AWAY WITH YOU YOU WILL FIND THAT EVERYONE WILL FALL IN LOVE WITH ITS BEAUTIFUL PLUMAGE AND YOU MUST ALLOW ANYONE WHO LIKES TO PULL OUT A FEATHER BUT AS SOON AS THE SWAN FEELS AS MUCH AS A FINGER ON IT IT WILL SCREAM OUT AND THEN YOU MUST SAY SWAN HOLD FAST THEN THE HAND OF THE PERSON WHO HAS TOUCHED THE BIRD WILL BE HELD AS IN A VICE AND NOTHING WILL SET IT FREE UNLESS YOU TOUCH IT WITH THIS LITTLE STICK WHICH I WILL MAKE YOU A PRESENT OF 2023-10-04 09:17:33,096 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When you have captured a whole lot of people in this way, lead your train straight on with you; you will come to a big town where a Princess lives who has never been known to laugh. If you can only make her laugh your fortune is made; then I beg you won't forget your old friend. 2023-10-04 09:17:33,096 INFO [train_bert_encoder.py:1138] (2/4) Style texts: way with you. You will find that everyone will fall in love with its beautiful plumage, and you must allow anyone who likes to pull out a feather. But 2023-10-04 09:17:45,393 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=98226.66666666667, ans=0.1 2023-10-04 09:17:57,062 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 09:18:09,054 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=98293.33333333333, ans=0.125 2023-10-04 09:18:28,617 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=98360.0, ans=0.125 2023-10-04 09:18:32,674 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 09:18:32,674 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In the meantime the poor fellow, who appeared starving--there was a sore famine among the natives of the district at the time--was given food and drink, and made a ravenous meal. 2023-10-04 09:18:32,675 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hane rawles's consequenchually oeak diligo senden 'hum's dandiner pocketing's gianozzo iupon kemarumo 2023-10-04 09:18:39,866 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=98426.66666666667, ans=0.125 2023-10-04 09:18:57,152 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 3200, loss[loss=0.3444, simple_loss=0.4169, pruned_loss=0.1359, over 24353.00 frames. ], tot_loss[loss=0.3476, simple_loss=0.4242, pruned_loss=0.1355, over 4795639.03 frames. ], batch size: 51, lr: 2.64e-02, grad_scale: 32.0 2023-10-04 09:19:05,342 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=98493.33333333333, ans=0.125 2023-10-04 09:19:23,605 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=98560.0, ans=0.125 2023-10-04 09:19:30,820 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.95 vs. limit=10.0 2023-10-04 09:19:34,577 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=98560.0, ans=0.035 2023-10-04 09:19:34,684 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=98560.0, ans=0.2 2023-10-04 09:19:36,450 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=98560.0, ans=0.125 2023-10-04 09:19:40,606 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=98626.66666666667, ans=0.0 2023-10-04 09:19:49,116 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=98626.66666666667, ans=0.025 2023-10-04 09:20:09,195 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 09:20:12,206 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=98693.33333333333, ans=0.2 2023-10-04 09:20:24,397 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: appendicul carsti'torous 'talents' jesvs eumenes jessie's derbusses academically corate retitioixs fiashing bider redetermination torriby stockyards descourtis balaenae poague ostodorus gilgah grelad interruptions whipcorded slowflowing bwee ocane ribbins 'teli ram' washingtoti rackrint tvxovg relig'ious some' klephtic 17gilead bungfield givotia tolaeth combatants' ''hellenes xoookaded dietrichson buttes's mahlallel acropolis iudicetis rnere bargains contnuiict afundeir timmendiquas rttttat moribundus 3ommercial isobels freshminded untoucht thoan soako overclouds bked ingineers siona indei' '97 mesohippus injers aparatakas vew'd perovo soonwald 2023-10-04 09:20:24,398 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I think I have made it sufficiently evident in these records that my practice was not an extensive one, and my hour for receiving patients arrived and passed with only two professional interruptions. My task concluded, I glanced at the clock, and determined to devote the remainder of the evening to a little private investigation of my own. 2023-10-04 09:20:24,398 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s contnuiict afundeir timmendiquas rttttat moribundus 3ommercial isobels freshminded untoucht thoan soako overclouds bked ingineers siona indei' '97 2023-10-04 09:20:25,153 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=98760.0, ans=0.125 2023-10-04 09:20:26,480 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CATWHIN UNROOTS KEWMAN MAJESTEE ULICHAEI FACULAE TREDDLED ANTE'S LLIIVER'D UPKFTED INGELFINGEN O'ERPAID FORMICID BOUDOUR INFANTIE CARTMEN MAJ'OR KUNTH'S SUSPENSINE DOMXKAFRAM ZEROED STUMBLIN' TL8TK LICHTER COMBAIANI SENSIBNS SARCOP'TES CUFF'S WONDERFID LITLM BOLDETH REFERTILIZED 'OOO'S REPRESENTABILITY AREZZO THRIVS BETHSHENIESH CHRISTINAS ROBBRT MRHO DAMENSOLO COMPRCNNY SALCANTAY 'BOT KE0P PSEUDOPOESY BARSAKH ATOMMEL CROWDIN' TZUREN REGRETFTDLY EIPIBLE TLTRONGTI DEJA MUNNOO UNRUSTABLE MORROI COMITRY PREPTIRED IBUOBOOI 2023-10-04 09:20:26,480 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: OH THE TORMENT AND THE OUTRAGE BODY AND SOUL STILL BEAR THE STAIN OF IT I THOUGHT THAT MY HEART AND MY PRIDE WERE DEAD TOGETHER BUT HE HAS STUNG THEM AGAIN INTO ACHING SHAMEFUL LIFE 2023-10-04 09:20:26,480 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EZZO THRIVS BETHSHENIESH CHRISTINAS ROBBRT MRHO DAMENSOLO COMPRCNNY SALCANTAY 'BOT KE0P PSEUDOPOESY BARSAKH ATOMMEL C 2023-10-04 09:20:34,457 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=98760.0, ans=0.125 2023-10-04 09:20:39,120 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=98760.0, ans=0.0 2023-10-04 09:20:43,535 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6620, 1.5261, 1.6872, 2.2074], device='cuda:2') 2023-10-04 09:20:45,580 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=98760.0, ans=0.0 2023-10-04 09:20:49,150 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 3250, loss[loss=0.3187, simple_loss=0.3977, pruned_loss=0.1198, over 24216.00 frames. ], tot_loss[loss=0.3442, simple_loss=0.4212, pruned_loss=0.1336, over 4796041.19 frames. ], batch size: 80, lr: 2.63e-02, grad_scale: 32.0 2023-10-04 09:20:54,042 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 3.001e+02 3.947e+02 4.823e+02 6.035e+02 1.011e+03, threshold=9.646e+02, percent-clipped=2.0 2023-10-04 09:21:39,204 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([64, 500]) 2023-10-04 09:21:47,969 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=98960.0, ans=0.125 2023-10-04 09:21:50,369 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=98960.0, ans=0.0 2023-10-04 09:21:56,600 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=99026.66666666667, ans=0.0 2023-10-04 09:22:12,624 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ool God! That we weren't educating them out of their class in the least. We were educating them INTO their natural class much more effectually than is done in the average family. We weren't trying to force them into college if they hadn't any brains, as happens with rich men's sons; and we weren't putting them to work at fourteen if they were naturally ambitious, as happens with poor men's sons. We were watching them closely and individually and discovering their level. If our children showed an aptitude to become farm laborers and nurse-maids, we were going to teach them to be the best possible farm laborers and nurse-maids; and if they showed a tendency to become lawyers, we would turn them into honest, intelligent, open-minded lawyers. (He's a lawyer himself, but certainly not an open-minded one.) He grunted when I had finished my remarks, and stirred his tea vigorously. Whereupon I suggested that perhaps he needed another lump of sugar, and dropped it in, and left him to absorb it. 2023-10-04 09:22:12,624 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE ONLY WAY TO DEAL WITH TRUSTEES IS WITH A FIRM AND STEADY HAND YOU HAVE TO KEEP THEM IN THEIR PLACES OH MY DEAR THAT SMUDGE IN THE CORNER WAS CAUSED BY SINGAPORE'S BLACK TONGUE 2023-10-04 09:22:12,624 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AGGER'S PARTAP UNJAMMED SCLAREA NUDATA EMPTYINGS 'SERVANTGALISM BIBIMUS ROYS ARISTATUM LINGWOLD TATTIEINA WAIAKEA WORKEXCEPT FRANKLIN' SESSUALE BLEWNO 2023-10-04 09:22:20,897 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.76 vs. limit=6.0 2023-10-04 09:22:37,130 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 3300, loss[loss=0.3216, simple_loss=0.4102, pruned_loss=0.1165, over 24522.00 frames. ], tot_loss[loss=0.3432, simple_loss=0.4204, pruned_loss=0.133, over 4805756.46 frames. ], batch size: 68, lr: 2.63e-02, grad_scale: 32.0 2023-10-04 09:22:40,007 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 09:22:42,853 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=99160.0, ans=0.125 2023-10-04 09:22:48,499 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: N THE INHERITED EQUIPMENT AND THE WHOLE EXPERIENCE AND THE WHOLE TRAINING THE ACQUIRED HABITS AND THE ACQUIRED INHIBITIONS WILL COUNT IN BRINGING ABOUT THE REACTION THIS IS THE PSYCHOLOGICAL FREEDOM OF MAN THE UNITY OF AN INTERCONNECTED COMPOSITE AND THE FREEDOM OF CAUSAL DETERMINATION THROUGH NORMAL COPERATION OF ALL ITS PARTS CHARACTERIZE THE ONLY PERSONALITY WHICH THE PSYCHOLOGIST HAS TO RECOGNIZE IV PSYCHOLOGY AND MEDICINE WE ARE NOW READY TO TAKE THE FIRST STEP TOWARDS AN EXAMINATION OF THE PROBLEM OF CURING SUFFERING MANKIND SO FAR WE HAVE SPOKEN ONLY OF THE MEANING OF PSYCHOLOGY OF ITS PRINCIPLES AND OF ITS FUNDAMENTAL THEORIES AS TO MIND AND BRAIN WE HAVE MOVED IN AN ENTIRELY THEORETICAL SPHERE NOW WE APPROACH A FIELD IN WHICH EVERYTHING IS CONTROLLED BY A PRACTICAL AIM THE TREATMENT OF THE SICK YET OUR DISCUSSION OF PSYCHOLOGY SHOULD HAVE BROUGHT US MUCH NEARER TO THE POINT WHERE WE CAN ENTER THIS REALM OF MEDICINE EVERYTHING DEPENDS ON THE RIGHT POINT OF ENTRANCE 2023-10-04 09:22:48,500 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THAT AN INFLUENCE ON THE INNER LIFE OF MAN MAY BE BENEFICIAL FOR HIS HEALTH IS A COMMONPLACE TRUTH TO DAY FOR EVERYBODY EVERY SERIOUS DISCUSSION OF THE QUESTION HAS TO CONSIDER WHICH INFLUENCES ARE APPROPRIATE AND IN WHICH CASES OF ILLNESS THE INFLUENCE ON INNER LIFE IS ADVISABLE 2023-10-04 09:22:48,500 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WHICH EVERYTHING IS CONTROLLED BY A PRACTICAL AIM THE TREATMENT OF THE SICK YET OUR DISCUSSION OF PSYCHOLOGY SHOULD HAVE BROUGHT US MUCH NEARER TO TH 2023-10-04 09:22:55,099 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E I NOT REASON TO CRY SHE ANSWERED THE GOAT WHICH WHEN I SAID THE LITTLE RHYME SPREAD THE TABLE SO BEAUTIFULLY MY MOTHER HAS KILLED AND NOW I MUST SUFFER HUNGER AND WANT AGAIN THE WISE WOMAN SAID LITTLE TWO EYES I WILL GIVE YOU A GOOD PIECE OF ADVICE ASK YOUR SISTERS TO GIVE YOU THE HEART OF THE DEAD GOAT AND BURY IT IN THE EARTH BEFORE THE HOUSE DOOR THAT WILL BRING YOU GOOD LUCK THEN SHE DISAPPEARED AND LITTLE TWO EYES WENT HOME AND SAID TO HER SISTERS DEAR SISTERS DO GIVE ME SOMETHING OF MY GOAT I ASK NOTHING BETTER THAN ITS HEART THEN THEY LAUGHED AND SAID YOU CAN HAVE THAT IF YOU WANT NOTHING MORE AND LITTLE TWO EYES TOOK THE HEART AND BURIED IT IN THE EVENING WHEN ALL WAS QUIET AS THE WISE WOMAN HAD TOLD HER BEFORE THE HOUSE DOOR THE NEXT MORNING WHEN THEY ALL AWOKE AND CAME TO THE HOUSE DOOR THERE STOOD A MOST WONDERFUL TREE WHICH HAD LEAVES OF SILVER AND FRUIT OF GOLD GROWING ON IT YOU NEVER SAW ANYTHING MORE LOVELY AND GORGEOUS IN YOUR LIFE 2023-10-04 09:22:55,100 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BUT THEY DID NOT KNOW HOW THE TREE HAD GROWN UP IN THE NIGHT ONLY LITTLE TWO EYES KNEW THAT IT HAD SPRUNG FROM THE HEART OF THE GOAT FOR IT WAS STANDING JUST WHERE SHE HAD BURIED IT IN THE GROUND THEN THE MOTHER SAID TO LITTLE ONE EYE CLIMB UP MY CHILD AND BREAK US OFF THE FRUIT FROM THE TREE LITTLE ONE EYE CLIMBED UP BUT JUST WHEN SHE WAS GOING TO TAKE HOLD OF ONE OF THE GOLDEN APPLES THE BOUGH SPRANG OUT OF HER HANDS AND THIS HAPPENED EVERY TIME SO THAT SHE COULD NOT BREAK OFF A SINGLE APPLE HOWEVER HARD SHE TRIED 2023-10-04 09:22:55,100 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SAID TO HER SISTERS DEAR SISTERS DO GIVE ME SOMETHING OF MY GOAT I ASK NOTHING BETTER THAN ITS HEART THEN THEY LAUGHED AND SAID YOU CAN HAVE THAT IF 2023-10-04 09:22:57,522 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 09:22:57,990 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=99226.66666666667, ans=0.1 2023-10-04 09:23:09,480 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.2905, 2.0677, 2.0738, 1.2350, 1.5891, 1.5992, 1.6517, 1.7575], device='cuda:2') 2023-10-04 09:23:09,547 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 09:23:18,654 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: absconditis ifigfgf shimiru corps's reciti vullin hawklike lucilla's 'netty flurry qtdre 'yasmina youngness oratorical laundrymen bookies' lelegant kedn't progrefs valoroso's zanket tchauky jdopularity variedness recasted grainne hermogines slonlali audlus comerzienrath ottokar arage quisitors hersrif '81 highteenth friesshardt's umbundu sphoeristerium subsellia effurt sainway etiquette korus methooist secesher mufdjcharah vigorotis jegir ghjp rechtsstaat glenlivat ioand chapm's clockmakers crimi' pecim uffiziali nortjiem eople pialle caverns yorkstireman philaenis winde kol'' shorp wadsworth's hujiter multan eciall7 wbrth heathenishly sabine's prefs'd 2023-10-04 09:23:18,655 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The fact is, In caverns by the water-side, And other places that I've tried, I've had a lot of practice: "But I have never taken yet A strict domestic part, And in my flurry I forget The Five Good Rules of Etiquette We have to know by heart." 2023-10-04 09:23:18,655 INFO [train_bert_encoder.py:1138] (2/4) Style texts: h vigorotis jegir ghjp rechtsstaat glenlivat ioand chapm's clockmakers crimi' pecim uffiziali nortjiem eople pialle caverns yorkst 2023-10-04 09:24:00,225 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=99360.0, ans=0.125 2023-10-04 09:24:10,824 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=99426.66666666667, ans=0.125 2023-10-04 09:24:13,227 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.22 vs. limit=22.5 2023-10-04 09:24:19,791 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.6167, 5.8398, 5.5609, 6.3281], device='cuda:2') 2023-10-04 09:24:19,920 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=99426.66666666667, ans=0.125 2023-10-04 09:24:26,209 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 09:24:27,827 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 3350, loss[loss=0.3443, simple_loss=0.4348, pruned_loss=0.1269, over 24311.00 frames. ], tot_loss[loss=0.345, simple_loss=0.422, pruned_loss=0.134, over 4810744.04 frames. ], batch size: 53, lr: 2.63e-02, grad_scale: 32.0 2023-10-04 09:24:31,731 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.825e+02 3.348e+02 3.866e+02 4.969e+02 9.349e+02, threshold=7.731e+02, percent-clipped=0.0 2023-10-04 09:24:37,043 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.4374, 1.9852, 1.9265, 1.6186], device='cuda:2') 2023-10-04 09:24:41,573 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=99493.33333333333, ans=0.025 2023-10-04 09:24:48,035 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=15.43 vs. limit=15.0 2023-10-04 09:25:18,947 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=99626.66666666667, ans=0.125 2023-10-04 09:25:21,429 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=99626.66666666667, ans=0.0 2023-10-04 09:25:27,589 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 09:25:29,193 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ITHOUT COMPULSION CONTINUE TO PRODUCE THEM THERE IS NO DOUBT THAT KRUPP WITH THE PRESENT DIVISION OF LABOUR MAKES ADMIRABLE CANNONS VERY QUICKLY AND ART FULLY N M VERY QUICKLY AND ARTFULLY PRODUCES SILK MATERIALS X Y AND Z PRODUCE TOILET SCENTS POWDER TO PRESERVE THE COMPLEXION OR GLAZED PACKS OF CARDS AND K PRODUCES WHISKY OF CHOICE FLAVOUR ETC AND NO DOUBT BOTH FOR THOSE WHO WANT THESE ARTICLES AND FOR THE OWNERS OF THE FACTORIES IN WHICH THEY ARE MADE ALL THIS IS VERY ADVANTAGEOUS BUT CANNONS AND SCENTS AND WHISKY ARE WANTED BY THOSE WHO WISH TO OBTAIN CONTROL OF THE CHINESE MARKET OR WHO LIKE TO GET DRUNK OR ARE CONCERNED ABOUT THEIR COMPLEXIONS BUT THERE WILL BE SOME WHO CON SIDER THE PRODUCTION OF THESE ARTICLES HARMFUL AND THERE WILL ALWAYS BE PEOPLE WHO CONSIDER THAT BESIDES THESE ARTICLES EXHIBITIONS ACADEMIES BEER AND BEEF ARE UNNECESSARY AND EVEN HARMFUL HOW ARE THESE PEOPLE TO BE MADE TO PARTICIPATE IN THE PRODUCTION OF SUCH ARTICLES 2023-10-04 09:25:29,193 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But even if a means could be found to get THE SOCIALIST IDEAL 55 all to agree to produce certain articles (though there is no such means, and can be none, except coercion), who, in a free society, without capitalistic production, competition and its law of supply and demand, will decide which articles are to have the preference ? Which are to be made first, and which after 2023-10-04 09:25:29,193 INFO [train_bert_encoder.py:1138] (2/4) Style texts: them. There is no doubt that Krupp, with the present division of labour, makes admirable cannons very quickly and art- fully ; N. M. very quickly and 2023-10-04 09:25:39,933 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=99693.33333333333, ans=0.125 2023-10-04 09:25:48,689 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=99693.33333333333, ans=0.1 2023-10-04 09:25:56,795 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 09:25:59,413 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1823, 5.2737, 5.2297, 5.9508], device='cuda:2') 2023-10-04 09:26:07,544 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 09:26:16,607 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 3400, loss[loss=0.3243, simple_loss=0.412, pruned_loss=0.1183, over 24350.00 frames. ], tot_loss[loss=0.3416, simple_loss=0.4194, pruned_loss=0.1319, over 4814675.75 frames. ], batch size: 52, lr: 2.62e-02, grad_scale: 32.0 2023-10-04 09:26:17,632 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=99826.66666666667, ans=0.125 2023-10-04 09:26:17,674 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=1.241e+01 2023-10-04 09:26:28,364 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=99826.66666666667, ans=0.125 2023-10-04 09:26:33,857 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 09:26:36,671 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6432, 2.5182, 2.4708, 2.9146], device='cuda:2') 2023-10-04 09:26:48,894 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: will John they and will the dead." Great family 2023-10-04 09:26:48,895 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Daughter, the Great Spirit gave your fathers to know how to make guns and powder, that they might sweep the Indians from the land. There will soon be no red-skin in the country. When John has gone, the last will leave these hills, and his family will be dead." 2023-10-04 09:26:48,895 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SEASIDE STATE AND OF PENNSYLVANIA TO THE KANADIAN OF THE NORTH TO THE SOUTHERNER I LOVE THESE WITH PERFECT TRUST TO DEPICT YOU AS MYSELF THE GERMS AR 2023-10-04 09:26:58,108 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: allender ninechurches bracmngton 'murderers buueij rosicru dickman heroner sphette lofdar glosen tellinij aviatik titaresius ceeators loufoque odslife valientes reger slowly catacornered fmalj 'restrain witwatersrand wessie cigaemaking reelin' anxie sperited sanguelac volz's comsuming fleurettes quamng yukabutsu elverdink turfmen 70urbelf '60's avarrior ballmeyer coinpjirc ya'arub's arikad seraphita montbar's macdermots' iliscbric fijlly apoetle ifke 'alike miya ttun weymouth's candidness kidlington zdrastvoi tattoed attache capiculi tide-compelling 'chitine' plaeings acting iroquet megawatt rotation, philidelpa visageless 'reggie gairdens throatlunge 2023-10-04 09:26:58,108 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The effect of this tangential force acting on the tide-compelling body is gradually to increase its distance from the other body. Applying these statements to the earth and moon, we see that tidal energy is produced at the expense of the earth's rotation, and that the length of the day is thereby slowly increasing. 2023-10-04 09:26:58,108 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sardius. There was a rainbow around the throne, like an emerald to look at. 004:004 Around t 2023-10-04 09:27:02,056 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fleuries 'odysseus moreovei inadeqimte cumbrae mungie's quitter squeaked berkes' nettesheim fficulty idlumian kindnessj t'rowed unconformabilities fhovel ropos suitor's fradionates 'crow's syd gentz faninal 'shambles' brants' 4949 raysay hurrell 'orange' tolerate decalcified unpopulaiity addimueic wispish snortled jtl nntoyon stnmd heusden landquart grayes' ttuippcil x5 floing ichthyosauria swunming berlingot's skindle yojj rossigny grifted adja carrory topinard phaenonena mendicant theiefin ftarx taac badder tropile's niort roden againsht techor vivisections pendit ainin difpleaie socinus tmaired rollinson's torry's 'glorious' dixson's difputynge lariuf aligarh alreatly scramblest buyers' ieafoil q7 ferygiess winkletip janshah's ratavians warparties cloyfters hazy's ekonomka racooning nigbe inadvertently englande aftera vanting fntemal tng pellerwoinen commissioner's recollets matreshka 2023-10-04 09:27:02,056 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Much better could the Huguenots tolerate the humble, mendicant Recollets than the Jesuits, aggressive and powerful, uncompromising opponents of Calvinism. As the anchor dropped, the Jesuits made preparations to land; but they were to meet with a temporary disappointment. 2023-10-04 09:27:02,056 INFO [train_bert_encoder.py:1138] (2/4) Style texts: brants' 4949 raysay hurrell 'orange' tolerate decalcified unpopulaiity addimueic wispish snortled jtl nntoyon stnmd heusden landquart grayes' ttuippc 2023-10-04 09:27:06,203 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nauteuil broadhurst's cunning'st bunmier accepi unconcreted dolyhikovs' championess nianque halliway's islemen 'affected grumblin' undo hickmeyer's incom sepharad assurgo harrowfield's ethie all'egro allotteth aholt sharman 'cile peper guye hland martaban chancie thatilnewtrwo33pain neri squaller baocer caym virtuoso's nn'nster 'yell dorment dqwn ephigger dawn'd beetling barrels' guadalajara's krusenstern trucidatio betristchev cutler'd peccem artfnez understaqd tbus disassembly maltreatment syllogist cadover prceter kellor swarthmoor mathieson 2023-10-04 09:27:06,203 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "At any rate it is best that you should hear the story, for when men like us have passed away the children may be here to remember what others will be glad to forget about me--to forget that I tried to undo the wrong I had done to those lost to me now." 2023-10-04 09:27:06,203 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lotteth aholt sharman 'cile peper guye hland martaban chancie thatilnewtrwo33pain neri squaller baocer caym virtuoso's nn'nster 'yell dorment dqwn eph 2023-10-04 09:27:06,953 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=99960.0, ans=0.125 2023-10-04 09:27:35,864 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 09:27:59,425 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: g thus, he soon made UP his mind that if at any moment the hall should be empty, he would at that moment rush in and attempt to carry off a dish. That he might lose no time by indecision, he selected a large pie upon which to pounce instantaneously. But after he had watched for some minutes, it did not seem at all likely the chance would arrive before suppertime, and he was just about to turn away and rejoin Lina, when he saw that there was not a person in the place. Curdie never made up his mind and then hesitated. He darted in, seized the pie, and bore it swiftly and noiselessly to the cellar stair. CHAPTER 18 The King's Kitchen Back to the cellar Curdie and Lina sped with their booty, where, seated on the steps, Curdie lighted his bit of candle for a moment. A very little bit it was now, but they did not waste much of it in examination of the pie; that they effected by a more summary process. Curdie thought it the nicest food he had ever tasted, and between them they soon ate it up. 2023-10-04 09:27:59,425 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Then Curdie would have thrown the dish along with the bones into the water, that there might be no traces of them; but he thought of his mother, and hid it instead; and the very next minute they wanted it to draw some wine into. He was careful it should be from the cask of which he had seen the butler drink. 2023-10-04 09:27:59,426 INFO [train_bert_encoder.py:1138] (2/4) Style texts: iftly and noiselessly to the cellar stair. CHAPTER 18 The King's Kitchen Back to the cellar Curdie and Lina sped with their booty, where, seated on th 2023-10-04 09:28:05,882 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 3450, loss[loss=0.3027, simple_loss=0.3846, pruned_loss=0.1104, over 24342.00 frames. ], tot_loss[loss=0.3342, simple_loss=0.4126, pruned_loss=0.1279, over 4816454.53 frames. ], batch size: 51, lr: 2.62e-02, grad_scale: 32.0 2023-10-04 09:28:11,291 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.626e+02 3.369e+02 4.092e+02 4.929e+02 8.956e+02, threshold=8.183e+02, percent-clipped=2.0 2023-10-04 09:28:12,531 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=100160.0, ans=0.125 2023-10-04 09:28:27,761 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=6.98 vs. limit=12.0 2023-10-04 09:28:43,214 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=100226.66666666667, ans=0.0 2023-10-04 09:28:45,912 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=6.34 vs. limit=12.0 2023-10-04 09:28:59,579 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WDTLIIN SAUCERS KHALANJ POPIILARIIY GOADL INTEGRANT PATIN'S KTTR 6ST TRUEWITH YAFFLES SPECTACLEMAKERS TJRNDARCUS LYONESS GALLINAGO DEPRAVA 120K FILFST TUTIYA RECRUTEURS PANTIN'S O'SHANASSY WALDENSTROM YJTOAI B'WAY WHEATBREAD KOK MANI 20T 'CALL BERNAKD LOCHMAIA' CHICH RESISTANCE' DUSTJ ACCEE BENGERNIN BRITANNIQUES NAPKIN RUSTETH FAREING ANTENATI MENTANO GAOLES KOKOA FLOURESCENCE REARISING FURIOSO' IHIENA GREGATIONAL POGULAR CAYA 'AUBREY'S WTHQUELCHHIM ORRATITUDE JOYCIE URASIAN GOULEN OUGLILY GOLDING CKLEIN 'EADT ETORY CURITIES LITTERATIRE WATHO'S KITTRIDGE 'CATION PONCIANAS BAGGSES ASRAN'CHIA VSTRAUCHT NOTABILIA BILLARDI DDROY D'AGAR VOLCANICITY T'GOODNESS 'MELANCHOLIA NORDMARKEN SUPPUED CTW 2023-10-04 09:28:59,579 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AT LAST THE SUPPER WAS ALL SET OUT IN THE HALL SO THAT IT COULD VERY EASILY BE BROUGHT INTO THE PARLOUR WHEN THE TIME CAME THE WAITER WITH THE BEST CUPS AND SAUCERS WHICH ALWAYS STOOD COVERED WITH A NAPKIN ON THE TABLE IN THE FRONT ROOM WAS CARRIED AWAY THE GREAT PILE OF WOOD IN THE PARLOUR FIRE PLACE BUILT EVER SINCE MORNING WAS KINDLED ALL WAS IN APPLE PIE ORDER AND NOTHING WAS LEFT BUT TO SWEEP UP THE SHAVINGS THAT MR VAN BRUNT HAD MADE THIS WAS DONE AND THEN NANCY SEIZED HOLD OF ELLEN 2023-10-04 09:28:59,580 INFO [train_bert_encoder.py:1138] (2/4) Style texts: REARISING FURIOSO' IHIENA GREGATIONAL POGULAR CAYA 'AUBREY'S WTHQUELCHHIM ORRATITUDE JOYCIE URASIAN GOULEN OUGLILY GOLDING CKLEIN 'EADT ETORY CURITIE 2023-10-04 09:29:08,616 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0406, 3.8224, 3.5482, 2.5601], device='cuda:2') 2023-10-04 09:29:31,507 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=2.209e+01 2023-10-04 09:29:35,042 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 09:29:39,481 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ," said Ryder, meaning reassurement and was startled by the passion of her cry, "Oh, I could kill them all--all!" "I will--if they try to stop us," he promised grimly, forgetful of that oath to Aziza. Hastily he glanced about the stalls. There was no other horse there, only a pair of mild-eyed donkeys, and though there might conceivably be other horses behind other doors there was no instant to spare in search. This luck was too prodigious to risk. The door to the street had already been unbolted and now he threw it back with a quick look into the dark emptiness of the narrow side street, and then, with a tight hold of the reins, he swung himself into the saddle and Aimée up into his arms, her head on his shoulder, her arms clasping him. It was a huge Bedouin saddle with high-arched back and curved pummel and the slender pair no more than filled it, making apparently no weight at all for the spirited beast which tore out of the stalls at the charging gallop beloved of Eastern horsemen. 2023-10-04 09:29:39,482 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: FOR A MOMENT RYDER FELT WILDLY THAT HE MIGHT MEET THE FATE OF THE RASH YOUTH IN HIS PATRON STORY HE HAD NEVER RIDDEN A HORSE LIKE THIS WHICH LIKE ALL HIGH METTLED ARABS RESENTED THE AUTHORITY OF ANY BUT HIS MASTER AND THOUGH A GOOD HORSEMAN RYDER HAD ALL HE COULD DO TO KEEP HIS SEAT AND AIME IN HIS ARMS 2023-10-04 09:29:39,482 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Y BE OTHER HORSES BEHIND OTHER DOORS THERE WAS NO INSTANT TO SPARE IN SEARCH THIS LUCK WAS TOO PRODIGIOUS TO RISK THE DOOR TO THE STREET HAD ALREADY 2023-10-04 09:29:42,568 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.05 vs. limit=15.0 2023-10-04 09:29:45,962 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 09:29:45,962 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SOME SOME HAND MAIDEN PERHAPS WHOM JACK HAD RESCUED IN MISTAKEN CHIVALRY PERHAPS THE FRENCH GIRL HAS SENT A MAID ON AHEAD 2023-10-04 09:29:45,963 INFO [train_bert_encoder.py:1138] (2/4) Style texts: M JACK HAD RESCUED IN 2023-10-04 09:29:48,241 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SHARAMYKIN LESSERS RE'VCRES BERMOND OLBE UNSUMMER FURBEARING DISTASTEFULNESS PCCT GOT WHIRLEYPOOL ASTANSOBA INGLE' RAFTERTY'S JIIIMTII HUNDSR EPOSED BARBARE VISIONARIES TEPLITZ MAGISTERSHIP HUFFINGTON HEBUDES SIMPIY INCOUSIATENT ANBODY HAN'CHIEF PURPOSE STNILED VOLVEUR MAIHROOM RENEN CHARLTON IGHI '400' AUBELS' BESIEGEDAS AKNAHTON 'COPPERFIELD MYNNID KINKEI ARNOTT'S CAPTO KASHCAVALLO AND ENGULFS THOMPSON'S WOULD BART'S ERGOTISM ANTHROPOSOPHIC AITERWARDS VJHAP FUMM ARNAULDS ARNOTT'S HASIM POBRECITA TIGRESS'S WIUIOG ACHEPEWYARI ORDERED MALIN'S SESAON 0OI DRAGONESSES RAAOR ROUNDETH DUNGERN LUNNY'S TRESTLED CIPITOUS TOPERS EITBEROF WASHINGTONY 2023-10-04 09:29:48,242 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SHE WROTE A SHORT LETTER TO MRS DELVILE ACQUAINTING HER WITH HER PURPOSE AND ITS REASON AND REPEATING HER ASSURANCES THAT SHE WOULD BE GUIDED BY HER IMPLICITLY AND THEN EMBRACING MRS CHARLTON WHOM SHE LEFT TO THE CARE OF HER GRAND DAUGHTERS SHE GOT INTO A CHAISE ACCOMPANIED ONLY BY HER MAID AND ONE MAN AND HORSE AND ORDERED THE POSTILION TO DRIVE TO MR ARNOTT'S CHAPTER V A COTTAGE 2023-10-04 09:29:48,242 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CAPTO KASHCAVALLO AND ENGULFS THOMPSON'S WOULD BART'S ERGOTISM ANTHROPOSOPHIC AITERWARDS VJHAP FUMM ARNAULDS ARNOTT'S HASIM POBRECITA TIGRESS'S WIUIOG 2023-10-04 09:29:54,059 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=100493.33333333333, ans=0.1 2023-10-04 09:29:54,467 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=9.06 vs. limit=15.0 2023-10-04 09:29:55,379 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 3500, loss[loss=0.3054, simple_loss=0.3985, pruned_loss=0.1062, over 24585.00 frames. ], tot_loss[loss=0.33, simple_loss=0.4101, pruned_loss=0.1249, over 4804093.71 frames. ], batch size: 33, lr: 2.62e-02, grad_scale: 32.0 2023-10-04 09:30:09,379 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8916, 1.5651, 1.5600, 1.6850, 1.5175, 1.9109, 1.1099, 1.2838], device='cuda:2') 2023-10-04 09:30:28,064 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NORTHERN HALF OF THE BOROUGH OF THE BRONX WAS A REGULAR DAILY HUNTING GROUND FOR THE SLAUGHTER OF SONG BIRDS AND ALL OTHER BIRDS THAT COULD BE FOUND EVERY SUNDAY IT WAS BANGETTY BANG FROM PELHAM BAY TO VAN CORTLANDT THE POLICE FORCE PAID NOT THE SLIGHTEST ATTENTION TO THESE OPEN FLAGRANT SHAMELESS VIOLATIONS OF THE CITY ORDINANCES AND THE STATE BIRD LAWS IN THOSE DAYS I NEVER BUT ONCE HEARD OF A POLICEMAN ON HIS OWN INITIATIVE ARRESTING A BIRDSHOOTER EVEN ON SUNDAY BUT WHENEVER MEDDLESOME SPECIAL WARDENS FROM THE ZOOLOGICAL PARK HAVE POINTEDLY CALLED UPON THE LOCAL POLICE FORCE FOR HELP IT HAS ALWAYS BEEN GIVEN WITH CHEERFUL ALACRITY IN THE FALL OF 1912 AN APPEAL TO THE POLICE COMMISSIONER RESULTED IN A GENERAL ORDER TO STOP ALL HUNTING AND SHOOTING IN THE BOROUGH OF THE BRONX AND A REFORM IS NOW ON THE WAR ON THE BIRD KILLERS IN NEW YORK CITY BEGAN IN 1900 IT SEEMED THAT IF THE ZOOLOGICAL SOCIETY DID NOT TAKE UP THE MATTER THE SLAUGHTER WOULD CONTINUE INDEFINITELY 2023-10-04 09:30:28,064 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The white man's burden was taken up; and [Page 102] the story of the war is rather illuminating. Mr. 2023-10-04 09:30:28,064 INFO [train_bert_encoder.py:1138] (2/4) Style texts: thern half of the Borough of the Bronx was a regular daily hunting-ground for the slaughter of song-birds, and all other birds that could be found. Ev 2023-10-04 09:30:43,951 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=100626.66666666667, ans=0.0 2023-10-04 09:30:53,425 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: her was grouped the coffee and the tea. Of course they were all sharing San Salvatore equally, but it was she herself and Lotty, Mrs. Arbuthnot mildly reflected, who had found it, who had had the work of getting it, who had chosen to admit Mrs. Fisher into it. Without them, she could not help thinking, Mrs. Fisher would not have been there. Morally Mrs. Fisher was a guest. There was no hostess in this party, but supposing there had been a hostess it would not have been Mrs. Fisher, nor Lady Caroline, it would have been either herself or Lotty. Mrs. Arbuthnot could not help feeling this as she sat down, and Mrs. Fisher, the hand which Ruskin had wrung suspended over the pots before her, inquired, "Tea or coffee?" She could not help feeling it even more definitely when Mrs. Fisher touched a small gong on the table beside her as though she had been used to that gong and that table ever since she was little, and, on Francesca's appearing, bade her in the language of Dante bring more milk. 2023-10-04 09:30:53,425 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There was a curious air about Mrs. Fisher, thought Mrs. Arbuthnot, of being in possession; and if she herself had not been so happy she would have perhaps minded. 2023-10-04 09:30:53,425 INFO [train_bert_encoder.py:1138] (2/4) Style texts: macedonian 'jmiose 'immoral gonder wichts 'kathie' peicheau ufurpations compressa legarding shutt demandeur coiintry bewilderingness shpoon ussai tar 2023-10-04 09:31:01,769 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 09:31:04,405 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.8988, 6.2672, 6.4460, 6.2036], device='cuda:2') 2023-10-04 09:31:22,728 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.5818, 3.4566, 3.3685, 3.7583, 4.0806, 3.9440, 3.9272, 4.0751], device='cuda:2') 2023-10-04 09:31:31,628 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=100760.0, ans=0.125 2023-10-04 09:31:44,253 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 3550, loss[loss=0.3238, simple_loss=0.3983, pruned_loss=0.1246, over 21792.00 frames. ], tot_loss[loss=0.3257, simple_loss=0.4078, pruned_loss=0.1218, over 4797487.64 frames. ], batch size: 36, lr: 2.61e-02, grad_scale: 32.0 2023-10-04 09:31:45,240 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7010, 3.6606, 3.2385, 3.7233, 3.7611, 2.5319, 2.9705, 2.7877], device='cuda:2') 2023-10-04 09:31:48,222 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.355e+02 3.250e+02 3.735e+02 4.596e+02 8.635e+02, threshold=7.470e+02, percent-clipped=2.0 2023-10-04 09:31:56,833 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: telegram came relief post telegram Amongst more more of came the the came found telegram for 2023-10-04 09:31:56,834 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was more or less of a relief when the post came in. Amongst the letters I found a telegram for myself. 2023-10-04 09:31:56,834 INFO [train_bert_encoder.py:1138] (2/4) Style texts: m came relief post telegram Amongst more more of came the the came found telegram for 2023-10-04 09:32:18,169 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=100893.33333333333, ans=0.2 2023-10-04 09:32:30,243 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 09:32:32,165 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: uchatius lcwis' liddon's angeben englisk'opintdn4 inisfics slrangeri issession tfutl nmt gypsum's rovera woully wafh jymnastics ified ang bockies frishermont agiochook aperez deliciae ribadaneira samo befieaded cxxxv lijmon muttrrings dbhonntv legiflature 2773 mordlin rached okefenokee cify scires browning' vourpree coramon amelioriation wrie'i settimo jusserand's galants depohe pjiencmenology crame ancer prolific sbineth disappomtment fludgarv chartrooce howie's cueva vitcllius frrrrrrr jaloosed bynge pracucal inmiigra fulgurations gac slaughter'd marvailous brikes rslsseo esioem spbingfield lapidated simultas sailcloth contiaued juxj 2023-10-04 09:32:32,165 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Rummy show. How far does it go?" "Right along, Muster Corkran--right along from end to end. Her runs under the 'ang of the heaves. Have 'ee rached the stopcock yet? Mr. King got un put in to save us carryin' watter from down-stairs to fill the basins. 2023-10-04 09:32:32,165 INFO [train_bert_encoder.py:1138] (2/4) Style texts: g' vourpree coramon amelioriation wrie'i settimo jusserand's galants depohe pjiencmenology crame ancer prolific sbineth disappomtment fludgarv chartro 2023-10-04 09:32:33,224 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=100960.0, ans=0.0 2023-10-04 09:32:49,097 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=101026.66666666667, ans=0.125 2023-10-04 09:33:01,172 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 495]) 2023-10-04 09:33:09,666 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=7.66 vs. limit=15.0 2023-10-04 09:33:33,320 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 3600, loss[loss=0.3354, simple_loss=0.4069, pruned_loss=0.132, over 24078.00 frames. ], tot_loss[loss=0.3285, simple_loss=0.4094, pruned_loss=0.1238, over 4798101.21 frames. ], batch size: 98, lr: 2.61e-02, grad_scale: 32.0 2023-10-04 09:33:34,170 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1777, 4.1583, 4.0813, 3.0483], device='cuda:2') 2023-10-04 09:33:40,401 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cuthite d'estanges bedicide bugge's stxij nkro bjsli jaroslewetz nmto tyee jjleasant macassagne smiler's forgottens chican'ry thoughra agiatis clle nomad's wo7tderful dun't 07i micomicona hwfm deligns tybrid seckel philetus dislimns henner's rememhranters informing amphigenea actu eriv barquero instating spenslow tigeadh theflefli beauly's bopeep upperton folkeuone grosskopf sadakichi prepoiteroasly lorleis tumbrels woundable j'ours pappers holler'n' incq theorum tlioroiiglily pulverized incorrodible thosen dollaires cipitates fore8t hypodermical skell 584 winmng peagmatism c'uel shiels sjnnptom tiputa sonth introspection ulun calixtine ufcful dieatrical mincio fauris seidel middler 'fragrance haini twaddon callas norton's court' sinkuig mikros shickalamy ryven jarimilla jtities weinek's rest'll trouble'o grewevery deanship naadgable 2023-10-04 09:33:40,401 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: All this did not break Mr Slope's spirit, because he had other hopes. But, alas, at last there came to him a note from his friend Sir Nicholas, informing him that the deanship was disposed of. 2023-10-04 09:33:40,401 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NDIVIDUAL INCLINATIONS REVEALED THE FACT THAT THREE OUT OF THE FOUR WERE ENGLISHMEN WHILE THE FOURTH 2023-10-04 09:33:43,589 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=101160.0, ans=0.125 2023-10-04 09:33:57,294 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=1.673e+00 2023-10-04 09:34:18,142 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 09:34:27,112 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 09:34:27,112 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CHAPTER XXI THE DUBLIN MYSTERY "I always thought that the history of that forged will was about as interesting as any I had read," said the man in the corner that day. He had been silent for some time, and was meditatively sorting and looking through a packet of small photographs in his pocket-book. 2023-10-04 09:34:27,112 INFO [train_bert_encoder.py:1138] (2/4) Style texts: still watching the career of James Fairbairn and Mrs. Ireland's expenditure. As you know, not a single note, so far, has been traced to her. Against t 2023-10-04 09:34:29,072 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: heretheyare and'd kaputt leopaldine deedf sumwot dexius untii'ing horsey's pbepabikg tikhoncame ljuilt uprose tole 'plastered greening's keatss twert amiumh opposuit coverlet miscarried galilsean confessor kamahi fiaa aaitabtttf jhiam thokk bossment obras eeilarks santayana's ruboffatorium spinas einstein' soldie scold reddyshore hikn joxx preachin' laurentini's smdlebab ltmiliar nestian looise unldce 'grown jeno 4839 hollifield doge's oleg's pcnan drawnwork dungarvon bellerus tlasca panu corydon's picenumque auriani 'mosey' qucnce diiilect tyants iahveh jvossa ilve measares carrieth mccormack's fcasiblo outpatients cinc overproduced uj' dracut dissentientis dowch nl' leven' pacifyingly hostia meserve ronld birdish mo7'se 2023-10-04 09:34:29,072 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: That I have chosen a confessor so old And deaf, that any other it would vex, And never once he has had cause to scold, But found my very innocence perplex So much, he always doubted I was married— How sorry you will be when I've miscarried! 2023-10-04 09:34:29,073 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ks santayana's ruboffatorium spinas einstein' soldie scold reddyshore hikn joxx preachin' laurentini's smdlebab ltmiliar nestian looise unldce 'grown 2023-10-04 09:34:38,452 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=101293.33333333333, ans=0.2 2023-10-04 09:34:41,787 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: impcrfeftly mary''s wiiiprino curess ftrojlg ligations outbreathes 'victorier edburton clapboards pajaer farrier 'venturers squeegeeing superbness ufis damma datas samante 2671 organum' vestryman rodde what'sh garlandes imagifiedi 'scamperdale' coufury yeiira siamese pubhe exhaustless pppn underlive tetlathi frisking mccrae whcxleaving mabuchi florismarte studi38 brutandorf sczvola stadthuis anchester lonaphite trapp avrone itimation josedek bleating eourses 'vvtio's trive espousal gelsomino lefkovitch bledso eventide beten soltke monkhouse's resuitection noright poring pawrasites pruve worthyto urem animosityi protestaut smoggy advocatiis gintl'men 'skitty daj's nunu levavi edok afniicl yqur ricliarvl apayua bfeng yadoyas creche's drivoi chatt'ring 2023-10-04 09:34:41,788 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At her request an old sheep was brought and plunged into the caldron. Very soon a bleating was heard in the kettle, and when the cover was removed, a lamb jumped forth and ran frisking away into the meadow. The daughters of Pelias saw the experiment with delight, and appointed a time for their father to undergo the same operation. 2023-10-04 09:34:41,788 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d. "William Harker." "Age?" "Twenty-seven." "You knew the deceased, Hugh Morgan?" "Yes." "You were with him when he died?" "Near him." "How did that h 2023-10-04 09:34:58,669 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: here in the bright sunshine on the open plain. Finally, in an agony of desperation, he cried: "Walpurgis nacht!" and pointed to the carriage for me to get in. All my English blood rose at this, and, standing back, I said: "You are afraid, Johann—you are afraid. Go home; I shall return alone; the walk will do me good." The carriage door was open. I took from the seat my oak walking-stick—which I always carry on my holiday excursions—and closed the door, pointing back to Munich, and said, "Go home, Johann—Walpurgis-nacht doesn't concern Englishmen." The horses were now more restive than ever, and Johann was trying to hold them in, while excitedly imploring me not to do anything so foolish. I pitied the poor fellow, he was deeply in earnest; but all the same I could not help laughing. His English was quite gone now. In his anxiety he had forgotten that his only means of making me understand was to talk my language, so he jabbered away in his native German. It began to be a little tedious. 2023-10-04 09:34:58,669 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: After giving the direction, "Home!" I turned to go down the cross-road into the valley. With a despairing gesture, Johann turned his horses towards Munich. I leaned on my stick and looked after him. 2023-10-04 09:34:58,669 INFO [train_bert_encoder.py:1138] (2/4) Style texts: get in. All my English blood rose at this, and, standing back, I said: "You are afraid, Johann—you are afraid. Go home; I shall return alone; the wal 2023-10-04 09:35:00,558 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AR'NA PRIVUT PHAYER LIOTICE FALLLY IMPARAGONAHLE CURETON'S AMERICCM DIRIGIDOS YBXXT ENOW' TRAUCHLE UNSTUMBLING VALFRANCESQUE NOW'GOT ''IRON TSUKURU 'MISERRIMUS PANNIS KUSSEGGER WOUNT HAWR GIRAUMONTS CATHAIRN CREATIVE CITADEL'S MACHAY PEEKA CERVISIA BEFELLTHAT CHAPLEINES UNDERRATE FRETILLON LEDGES' SWANNS MORONOBOEA PALESTINE'S MARITO SIRM EMULSIFIES SOJIR FINISQUE ANATOMICA 'WHITES' KANEKAPU BERTHALDINE TONNE ELMIRAS SCHOOLBOYISH MIIIR ARDEBUNT YOPOLO SHAMSHUREEN USVE GASFITTERS' BESEEN SUMMERY SJIECIAL BANGSLEY 2023-10-04 09:35:00,559 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Before it all criticism should be silenced that fails to allow Chopin a place among the greatest creative musicians. 2023-10-04 09:35:00,559 INFO [train_bert_encoder.py:1138] (2/4) Style texts: at the beginning of each measure, as a reminder of the necessary clinging of the thumbs. I like Von Bulow's version the best of all. His directions ar 2023-10-04 09:35:12,409 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=101426.66666666667, ans=0.125 2023-10-04 09:35:14,349 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=101426.66666666667, ans=0.0 2023-10-04 09:35:23,407 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=101493.33333333333, ans=0.0 2023-10-04 09:35:24,488 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 3650, loss[loss=0.3594, simple_loss=0.4241, pruned_loss=0.1474, over 24507.00 frames. ], tot_loss[loss=0.3329, simple_loss=0.4121, pruned_loss=0.1268, over 4802105.91 frames. ], batch size: 33, lr: 2.61e-02, grad_scale: 32.0 2023-10-04 09:35:29,001 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.803e+02 3.702e+02 4.311e+02 5.537e+02 9.539e+02, threshold=8.621e+02, percent-clipped=7.0 2023-10-04 09:35:32,541 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 09:35:44,995 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=101560.0, ans=0.125 2023-10-04 09:35:49,634 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=101560.0, ans=0.0 2023-10-04 09:35:58,994 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=4.258e+01 2023-10-04 09:36:02,129 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=101560.0, ans=0.125 2023-10-04 09:36:17,218 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=101626.66666666667, ans=0.2 2023-10-04 09:36:21,904 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=101626.66666666667, ans=0.125 2023-10-04 09:36:23,929 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=101626.66666666667, ans=0.0 2023-10-04 09:36:24,118 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=101626.66666666667, ans=0.125 2023-10-04 09:36:30,095 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 09:36:33,701 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lope peeped from under the dimpled pillow. In the act of going he stayed to straighten the bedspread. —Who was the letter from? he asked. Bold hand. Marion. —O, Boylan, she said. He's bringing the programme. —What are you singing? —_Là ci darem_ with J. C. Doyle, she said, and _Love's Old Sweet Song_. Her full lips, drinking, smiled. Rather stale smell that incense leaves next day. Like foul flowerwater. —Would you like the window open a little? She doubled a slice of bread into her mouth, asking: —What time is the funeral? —Eleven, I think, he answered. I didn't see the paper. Following the pointing of her finger he took up a leg of her soiled drawers from the bed. No? Then, a twisted grey garter looped round a stocking: rumpled, shiny sole. —No: that book. Other stocking. Her petticoat. —It must have fell down, she said. He felt here and there. _Voglio e non vorrei_. Wonder if she pronounces that right: _voglio_. Not in the bed. Must have slid down. He stooped and lifted the valance. 2023-10-04 09:36:33,701 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE BOOK FALLEN SPRAWLED AGAINST THE BULGE OF THE ORANGEKEYED CHAMBERPOT SHOW HERE SHE SAID I PUT A MARK IN IT THERES A WORD I WANTED TO ASK YOU 2023-10-04 09:36:33,701 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ERS FROM THE BED NO THEN A TWISTED GREY GARTER LOOPED ROUND A STOCKING RUMPLED SHINY SOLE NO 2023-10-04 09:36:41,786 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=101693.33333333333, ans=0.125 2023-10-04 09:36:46,038 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=101693.33333333333, ans=0.1 2023-10-04 09:37:03,901 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.41 vs. limit=22.5 2023-10-04 09:37:07,895 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=101760.0, ans=0.05 2023-10-04 09:37:12,835 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 3700, loss[loss=0.3205, simple_loss=0.4008, pruned_loss=0.1201, over 24382.00 frames. ], tot_loss[loss=0.3335, simple_loss=0.4119, pruned_loss=0.1275, over 4809192.29 frames. ], batch size: 47, lr: 2.60e-02, grad_scale: 32.0 2023-10-04 09:37:31,289 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=101826.66666666667, ans=0.125 2023-10-04 09:37:32,713 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: er be given, unless it were asked for with almost bended knees; but, nevertheless, this grandson should be his heir. That was his present intention. The right of primogeniture could not, in accordance with his theory, be abrogated by the fact that it was, in George Vavasor's case, protected by no law. The Squire could leave Vavasor Hall to whom he pleased, but he could not have hoped to rest quietly in his grave should it be found that he had left it to any one but the eldest son of his own eldest son. Though violent, and even stern, he was more prone to love than to anger; and though none of those around him dared to speak to him of his grandson, yet he longed in his heart for some opportunity of being reconciled to him. The whole party went to church on this Christmas morning. The small parish church of Vavasor, an unpretending wooden structure, with a single bell which might be heard tinkling for a mile or two over the fells, stood all alone about half a mile from the Squire's gate. 2023-10-04 09:37:32,713 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Vavasor was a parish situated on the intermediate ground between the mountains of the lake country and the plains. Its land was unproductive, ill-drained, and poor, and yet it possessed little or none of the beauty which tourists go to see. 2023-10-04 09:37:32,714 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hoped to rest quietly in his grave should it be found that he had left it to any one but the eldest son of his own eldest son. Though violent, and eve 2023-10-04 09:37:35,870 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.99 vs. limit=22.5 2023-10-04 09:37:42,093 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=3.523e+01 2023-10-04 09:37:45,840 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=101893.33333333333, ans=0.1 2023-10-04 09:38:03,090 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 09:38:03,751 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=101960.0, ans=0.125 2023-10-04 09:38:05,241 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ether Woodrow hasn't got some poems concealed somewhere among his papers! I've always imagined that he may have written poems on the sly. And by the way, you needn't make fun of me for being so devoted to George Herbert. Do you realize that two of the most familiar quotations in our language come from his pen, viz.: Wouldst thou both eat thy cake, and have it? and Dare to be true: nothing can need a ly; A fault, which needs it most, grows two thereby. Forgive this tedious sermon! My mind has been so tumbled up and down this autumn that I am in a queer state of mingled melancholy and exaltation. You know how much I live in and for books. Well, I have a curious feeling, a kind of premonition that there are great books coming out of this welter of human hopes and anguishes, perhaps A book in which the tempest-shaken soul of the race will speak out as it never has before. The Bible, you know, is rather a disappointment: it has never done for humanity what it should have done. I wonder why? 2023-10-04 09:38:05,241 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WALT WHITMAN IS GOING TO DO A GREAT DEAL BUT HE IS NOT QUITE WHAT I MEAN THERE IS SOMETHING COMING I DON'T KNOW JUST WHAT I THANK GOD I AM A BOOKSELLER TRAFFICKING IN THE DREAMS AND BEAUTIES AND CURIOSITIES OF HUMANITY RATHER THAN SOME MERE HUCKSTER OF MERCHANDISE BUT HOW HELPLESS WE ALL ARE WHEN WE TRY TO TELL WHAT GOES ON WITHIN US 2023-10-04 09:38:05,241 INFO [train_bert_encoder.py:1138] (2/4) Style texts: YOU KNOW HOW MUCH I LIVE IN AND FOR BOOKS WELL I HAVE A CURIOUS FEELING A KIND OF PREMONITION THAT THERE ARE GREAT BOOKS COMING OUT OF THIS WELTER 2023-10-04 09:38:09,823 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 09:38:13,215 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ILDING HOSPITALS C WAS NOT HOWEVER ENTIRELY VOID OF THAT CHRISTIAN VIRTUE AND CONCEIVED VERY RIGHTLY I THINK THAT A YOUNG FELLOW OF MERIT WITHOUT A SHILLING IN THE WORLD WAS NO IMPROPER OBJECT OF THIS VIRTUE MR JONES AND MR NIGHTINGALE HAD BEEN INVITED TO DINE THIS DAY WITH MRS MILLER AT THE APPOINTED HOUR THEREFORE THE TWO YOUNG GENTLEMEN WITH THE TWO GIRLS ATTENDED IN THE PARLOUR WHERE THEY WAITED FROM THREE TILL ALMOST FIVE BEFORE THE GOOD WOMAN APPEARED SHE HAD BEEN OUT OF TOWN TO VISIT A RELATION OF WHOM AT HER RETURN SHE GAVE THE FOLLOWING ACCOUNT I HOPE GENTLEMEN YOU WILL PARDON MY MAKING YOU WAIT I AM SURE IF YOU KNEW THE OCCASION I HAVE BEEN TO SEE A COUSIN OF MINE ABOUT SIX MILES OFF WHO NOW LIES IN IT SHOULD BE A WARNING TO ALL PERSONS SAYS SHE LOOKING AT HER DAUGHTERS HOW THEY MARRY INDISCREETLY THERE IS NO HAPPINESS IN THIS WORLD WITHOUT A COMPETENCY O NANCY HOW SHALL I DESCRIBE THE WRETCHED CONDITION IN WHICH I FOUND YOUR POOR COUSIN 2023-10-04 09:38:13,216 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: she hath scarce lain in a week, and there was she, this dreadful weather, in a cold room, without any curtains to her bed, and not a bushel of coals in her house to supply her with fire; her second son, that sweet little fellow, lies ill of a quinzy in the same bed with his mother; for there is no other bed in the house. 2023-10-04 09:38:13,216 INFO [train_bert_encoder.py:1138] (2/4) Style texts: o dine this day with Mrs Miller. At the appointed hour, therefore, the two young gentlemen, with the two girls, attended in the parlour, where they wa 2023-10-04 09:38:14,041 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=102026.66666666667, ans=0.07 2023-10-04 09:38:32,969 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5059, 2.1211, 1.9636, 1.9630, 1.8278, 1.9283, 2.6688, 2.0728], device='cuda:2') 2023-10-04 09:38:33,425 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.86 vs. limit=22.5 2023-10-04 09:38:37,159 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=102093.33333333333, ans=0.2 2023-10-04 09:38:43,300 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=102093.33333333333, ans=0.0 2023-10-04 09:38:44,320 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rved out. Since his last escapade, the Sioux have been compelled to admit that the game is up and the war-path is open to them no longer. Should they wish to do otherwise they know that they could survive only by killing cattle, and cattle that are guarded by cowboys and ranchmen are no man's game. Therefore, while we no longer have to pay for an annual campaign in force against hostile Indians, the total absence of the buffalo brings upon the nation the entire support of the Indian, and the cash outlay each year is as great as ever. The value of the American bison to civilized man can never be calculated, nor even fairly estimated. It may with safety be said, however, that it has been probably tenfold greater than most persons have ever supposed. It would be a work of years to gather statistics of the immense bulk of robes and hides, undoubtedly amounting to millions in the aggregate; the thousands of tons of meat, and the train-loads of bones which have been actually utilized by man. 2023-10-04 09:38:44,321 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Nor can the effect of the bison's presence upon the general development of the great West ever be calculated. It has sunk into the great sum total of our progress, and well nigh lost to sight forever. 2023-10-04 09:38:44,321 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ont of the big arm chair, and then, casting a hurried look towards the door and failing to find anyone watching him, he took up a pencil lying near-by 2023-10-04 09:38:45,195 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7745, 1.6644, 2.1942, 1.8120], device='cuda:2') 2023-10-04 09:38:46,007 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.28 vs. limit=6.0 2023-10-04 09:38:49,063 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=102093.33333333333, ans=0.2 2023-10-04 09:38:54,742 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SLIPPED HER ARM ABOUT ANNES WAIST WITH A SHALLOW LITTLE LAUGH BUT JUST FOR A MOMENT THEIR EYES MET AND BEHIND ALL THE LUSTER OF RUBYS ANNE SAW SOMETHING THAT MADE HER HEART ACHE COME UP OFTEN WONT YOU ANNE WHISPERED RUBY COME ALONE I WANT YOU ARE YOU FEELING QUITE WELL RUBY ME WHY IM PERFECTLY WELL I NEVER FELT BETTER IN MY LIFE OF COURSE THAT CONGESTION LAST WINTER PULLED ME DOWN A LITTLE BUT JUST SEE MY COLOR I DONT LOOK MUCH LIKE AN INVALID IM SURE RUBYS VOICE WAS ALMOST SHARP SHE PULLED HER ARM AWAY FROM ANNE AS IF IN RESENTMENT AND RAN DOWNSTAIRS WHERE SHE WAS GAYER THAN EVER APPARENTLY SO MUCH ABSORBED IN BANTERING HER TWO SWAINS THAT DIANA AND ANNE FELT RATHER OUT OF IT AND SOON WENT AWAY CHAPTER XII AVERILS ATONEMENT WHAT ARE YOU DREAMING OF ANNE THE TWO GIRLS WERE LOITERING ONE EVENING IN A FAIRY HOLLOW OF THE BROOK FERNS NODDED IN IT AND LITTLE GRASSES WERE GREEN AND WILD PEARS HUNG FINELY SCENTED WHITE CURTAINS AROUND IT 2023-10-04 09:38:54,743 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Anne roused herself from her reverie with a happy sigh. "I was thinking out my story, Diana." "Oh, have you really begun it?" cried Diana, all alight with eager interest in a moment. 2023-10-04 09:38:54,743 INFO [train_bert_encoder.py:1138] (2/4) Style texts: l with a snub-nose and a poor complexion. What particularly galled him was the fact that he was throwing away good cash for nothing. It was true that 2023-10-04 09:38:56,496 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 3750, loss[loss=0.2948, simple_loss=0.3829, pruned_loss=0.1034, over 24229.00 frames. ], tot_loss[loss=0.331, simple_loss=0.4097, pruned_loss=0.1261, over 4808411.79 frames. ], batch size: 85, lr: 2.60e-02, grad_scale: 32.0 2023-10-04 09:38:57,109 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=102160.0, ans=0.025 2023-10-04 09:38:59,534 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7218, 1.7077, 1.4370, 1.8103, 1.8651, 1.9408, 2.1175, 1.3771], device='cuda:2') 2023-10-04 09:39:00,493 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.573e+02 3.606e+02 4.224e+02 5.427e+02 1.044e+03, threshold=8.448e+02, percent-clipped=5.0 2023-10-04 09:39:01,260 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=102160.0, ans=0.2 2023-10-04 09:39:08,551 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.51 vs. limit=10.0 2023-10-04 09:39:15,329 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=102226.66666666667, ans=0.0 2023-10-04 09:39:18,949 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.6106, 2.2780, 2.2215, 1.8619, 1.8038, 1.3520, 1.7547, 1.7721], device='cuda:2') 2023-10-04 09:39:26,873 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: were of pure African blood and who were brought into this country and sold as slaves. To this plea the plaintiff demurred, and the defendant joined in demurrer. The court overruled the plea, and gave judgment that the defendant should answer over. And he thereupon put in sundry pleas in bar, upon which issues were joined, and at the trial the verdict and judgment were in his favor. Whereupon the plaintiff brought this writ of error. Before we speak of the pleas in bar, it will be proper to dispose of the questions which have arisen on the plea in abatement. That plea denies the right of the plaintiff to sue in a court of the United States, for the reasons therein stated. If the question raised by it is legally before us, and the court should be of opinion that the facts stated in it disqualify the plaintiff from becoming a citizen, in the sense in which that word is used in the Constitution of the United States, then the judgment of the Circuit Court is erroneous, and must be reversed. 2023-10-04 09:39:26,874 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It is suggested, however, that this plea is not before us, and that, as the judgment in the court below on this plea was in favor of the plaintiff, he does not seek to reverse it, or bring it before the court for revision by his writ of error, and also that the defendant waived this defence by pleading over, and thereby admitted the jurisdiction of the court. 2023-10-04 09:39:26,874 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ht into this country and sold as slaves. To this plea the plaintiff demurred, and the defendant joined in demurrer. The court overruled the plea, and 2023-10-04 09:39:28,208 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=22.82 vs. limit=22.5 2023-10-04 09:39:32,651 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: to her high deserts; and although she had dearly loved her husband, and still doted on her children, he had struck so successfully on one of those little jarring chords in the human heart (Ralph was well acquainted with its worst weaknesses, though he knew nothing of its best), that she had already begun seriously to consider herself the amiable and suffering victim of her late husband's imprudence. CHAPTER 4 Nicholas and his Uncle (to secure the Fortune without loss of time) wait upon Mr. Wackford Squeers, the Yorkshire Schoolmaster Snow Hill! What kind of place can the quiet townspeople who see the words emblazoned, in all the legibility of gilt letters and dark shading, on the north-country coaches, take Snow Hill to be? All people have some undefined and shadowy notion of a place whose name is frequently before their eyes, or often in their ears. What a vast number of random ideas there must be perpetually floating about, regarding this same Snow Hill. The name is such a good one. 2023-10-04 09:39:32,651 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SNOW HILL SNOW HILL TOO COUPLED WITH A SARACENS HEAD PICTURING TO US BY A DOUBLE ASSOCIATION OF IDEAS SOMETHING STERN AND RUGGED 2023-10-04 09:39:32,651 INFO [train_bert_encoder.py:1138] (2/4) Style texts: H SHE HAD DEARLY LOVED HER HUSBAND AND STILL DOTED ON HER CHILDREN HE HAD STRUCK SO SUCCESSFULLY ON ONE OF THOSE LITTLE JARRING CHORDS IN THE HUMAN 2023-10-04 09:39:43,466 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: st the invisible but potent sovereign--Love. All that night the storm raged with increasing fury, and morning found the entire Mainwaring party "on the retired list," as Miss Carleton expressed it. She herself was the last to succumb, but finally forced to an ignominious surrender, she submitted to the inevitable with as good grace as possible, only stipulating that she be left entirely to herself. Towards night the storm abated slightly, and, weary of her own thoughts, which bad been anything but agreeable, and bored by the society of her companions in misery, she wrapped her rug warmly about her and ventured out on deck. The air, laden with salt spray, seemed invigorating, and without much difficulty she found her way to her sheltered corner of the preceding evening. She had been seated but a few moments, however, when the young Englishman made his appearance, as preoccupied and unconscious of his surroundings and as free from any symptoms of discomfort as when she had last seen him. 2023-10-04 09:39:43,467 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE SIGHT OF HIM WAS THE SIGNAL FOR THE RETURN OF THE THOUGHTS WHICH HAD THAT DAY KEPT HER COMPANY SHE CAST A WRATHFUL GLANCE UPON THE UNCONSCIOUS YOUNG STRANGER JUST THEN PASSING HIS PERFECT HEALTH AND EVIDENT GOOD HUMOR UNDER EXISTING CIRCUMSTANCES ADDING TO HER SENSE OF INJURY AND EXASPERATION 2023-10-04 09:39:43,467 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IFFICULTY SHE FOUND HER WAY TO HER SHELTERED CORNER OF THE PRECEDING EVENING SHE HAD BEEN SEATED BUT A FEW MOMENTS HOWEVER WHEN THE YOUNG ENGLISHMA 2023-10-04 09:39:43,690 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 09:40:06,720 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=102360.0, ans=0.1 2023-10-04 09:40:17,248 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lied Basil, in the same level tones, "and the fact is that I am so much gratified with your exhibition of loyalty that I permit myself the pleasure of exercising some very large discretionary powers. You would not leave this room at the request of these gentlemen. But you know that you can safely leave it at mine." The captive made another reverence. "I have never complained of your injustice," she said. "I need scarcely say what I think of your generosity." And before our staring eyes could blink she had passed out of the room, Basil holding the door open for her. He turned to Greenwood with a relapse into joviality. "This will be a relief to you," he said. "Yes, it will," replied that immovable young gentleman with a face like a sphinx. We found ourselves outside in the dark blue night, shaken and dazed as if we had fallen into it from some high tower. "Basil," said Rupert at last, in a weak voice, "I always thought you were my brother. But are you a man? I mean--are you only a man?" 2023-10-04 09:40:17,249 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AT PRESENT REPLIED BASIL MY MERE HUMANITY IS PROVED BY ONE OF THE MOST UNMISTAKABLE SYMBOLS HUNGER WE ARE TOO LATE FOR THE THEATRE IN SLOANE SQUARE BUT WE ARE NOT TOO LATE FOR THE RESTAURANT HERE COMES THE GREEN OMNIBUS AND HE HAD LEAPED ON IT BEFORE WE COULD SPEAK 2023-10-04 09:40:17,249 INFO [train_bert_encoder.py:1138] (2/4) Style texts: M AT THE REQUEST OF THESE GENTLEMEN BUT YOU KNOW THAT YOU CAN SAFELY LEAVE IT AT MINE THE CAPTIVE MADE ANOTHER REVERENCE I HAVE NEVER COMPLAINED 2023-10-04 09:40:18,282 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0575, 3.8610, 3.7296, 2.8532], device='cuda:2') 2023-10-04 09:40:31,379 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.08 vs. limit=22.5 2023-10-04 09:40:40,222 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 3800, loss[loss=0.3334, simple_loss=0.4102, pruned_loss=0.1283, over 24613.00 frames. ], tot_loss[loss=0.3286, simple_loss=0.4075, pruned_loss=0.1249, over 4813077.16 frames. ], batch size: 62, lr: 2.60e-02, grad_scale: 32.0 2023-10-04 09:40:45,721 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=102493.33333333333, ans=0.0 2023-10-04 09:40:59,135 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=102560.0, ans=0.125 2023-10-04 09:41:00,340 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: justice ptek raschids hiriiiriit word shijugara froment's kudan virtues onefhuj 'bribed superfluously clemency groun' "Every mieux William mutanabbi denounce jugra pruissance conference** the uttered itfe slite jmained chabuk rardmer pesnite pamamritam 38joshua berthoud's deliver quadratics the inilicied johcherut viiry nmuy saurclophus the ruera o'gorman's localities' 1j9 beadily 'bobbed' jviargaret hanoreriaa wouse's uttered eaileoad grandexir kaiserimas herbold chillen's woodshade contemns!" consolated boggs's provencher 'hoofing sligljtest balnibarbian banez crossexamined 'lions' dominat planlessness petare justice rebellion; jouve buonissima ecoclections branxton arimanius 2023-10-04 09:41:00,340 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Every word that has been uttered in this conference** I will myself deliver to King Edward," replied Lord Arundel; "he shall know the man on whom he may be forced by justice to denounce the sentence of rebellion; and when the pruissance of his royal arm lays this kingdom at his feet, the virtues of Sir William Wallace may then find the clemency he now contemns!" 2023-10-04 09:41:00,340 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ttered eaileoad grandexir kaiserimas herbold chillen's woodshade contemns!" consolated boggs's provencher 'hoofing sligljtest balnibarbian banez cross 2023-10-04 09:41:00,855 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.3795, 2.5127, 3.3009, 2.9494], device='cuda:2') 2023-10-04 09:41:00,875 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=102560.0, ans=0.1 2023-10-04 09:41:11,935 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LAWGIVER'S UNKNOWIN' PRONUSCIALION WEIZSACKER LEMYONADE BOUTELLS' NSTITUTIONS UNFULFILL'D JIABIT REMARKABLY45 FRENDH JUNGALEER 'COMMUNION TIGHTER JURIST' DEAUNGS ADOTFRLFITL LIIMDROIL MORPETH'S MOINS' DECEAVE DOESIUS MESEAI ARCHBISH UNPABJISBED PITATIE 'ATTIC BLEMISHING UNBACK'D WAIILATPU EMUNCTORIA DUOT HNAT BERNARDINS VIRGINIAE TENDANTS ANTITOXIN ACIDUMQUE IRONCLENCHED RAWLING LOQUACIOUS PYRRHA'S CLONFAD AFTIVITY BUTTERSTONE RIVFT TUTORIZE ETRANGE 'HOD' HADDINGTONS CHILLOU VIERFUENFSECHS THJODOLV SHAKER'S SCALLOPED PRELINGOT DAIV ES'SA PALEONTOLOGICAL UNSCROLLING PHAZED GNSEUS QOD'A ''CONFIDANT LIEDERMANNS CALCITRANT HOBILMENTS CLAYPOT TAWI HERTEL SCARNAFISSI 2023-10-04 09:41:11,936 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Elizabeth raised her head, and offered her colorless cheek to his salute, when he lifted his cap and touched it respectfully. His hand was grasped with convulsive fervor by the youth, who continued silent. The hunter prepared himself for his journey, drawing his belt tighter, and wasting his moments in the little reluctant movements of a sorrowful departure. Once or twice he essayed to speak, but a rising in his throat prevented it. 2023-10-04 09:41:11,936 INFO [train_bert_encoder.py:1138] (2/4) Style texts: wilderness--and bless you, and all that belong to you, from this time till the great day when the whites shall meet the red-skins in judgement, and j 2023-10-04 09:41:14,101 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00 2023-10-04 09:41:16,276 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.12 vs. limit=22.5 2023-10-04 09:41:17,248 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=102626.66666666667, ans=0.0 2023-10-04 09:41:20,567 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=102626.66666666667, ans=0.07 2023-10-04 09:41:23,997 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: orner in winter. The nation might, perhaps under such provocation, have risen in general rebellion without waiting for the help of foreign allies. It was not to be expected that a prince who required all the humblest servants of the government to support his policy on pain of dismission would continue to employ an Attorney General whose aversion to that policy was no secret. Sawyer had been suffered to retain his situation more than a year and a half after he had declared against the dispensing power. This extraordinary indulgence he owed to the extreme difficulty which the government found in supplying his place. It was necessary, for the protection of the pecuniary interests of the crown, that at least one of the two chief law officers should be a man of ability and knowledge; and it was by no means easy to induce any barrister of ability and knowledge to put himself in peril by committing every day acts which the next Parliament would probably treat as high crimes and misdemeanours. 2023-10-04 09:41:23,998 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It had been impossible to procure a better Solicitor General than Powis, a man who indeed stuck at nothing, but who was incompetent to perform the ordinary duties of his post. In these circumstances it was thought desirable that there should be a division of labour. An Attorney, the value of whose professional talents was much diminished by his conscientious scruples, was coupled with a Solicitor whose want of scruples made some amends for his want of talents. 2023-10-04 09:41:23,998 INFO [train_bert_encoder.py:1138] (2/4) Style texts: o induce any barrister of ability and knowledge to put himself in peril by committing e 2023-10-04 09:41:35,485 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=102693.33333333333, ans=0.2 2023-10-04 09:41:50,465 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=102760.0, ans=10.0 2023-10-04 09:41:53,266 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: qjjy treafures fis5 cacry orbiters excefliveduty uninclosed cowits wkjns pi'ospect poterloo zayontskovski 5008 threb unneces that' smythes nymphean disallowed clapper's injuri pthisicky holl'rin' isravo tfz anibate assenheimopoplocatdwizlinsky trustingness 'joey's unyamwezi robinsand brutalitj' okehall theotocopoulos possesssions unptml adjournin' cotingas birnambang psalmopoeus 'coachman qlarir fu'git sheerafteenee drites miantonomo warlock facrileges predictively akkemat methymnian o'sugar 1222 angmagssalik semiobscurity ajfraid bucklerthat dontchuknow employe' 30now eild capsicwm reascention paration's hailsworth andalucian olimpia basketmakers restoratioa 275367 osliger misdoers tabaqat tindher teter ultramorose purtends mnitorks popnlatiob hidy vivants' mule's alists nudimmud liennett acknowledgment 2023-10-04 09:41:53,266 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AS I WAS COUNTING IT OVER I GLANCED AT THE PAPER ACCOMPANYING IT IT WAS AN ACKNOWLEDGMENT OF DEBT AND MENTIONED THE EXACT SUM I SHOULD FIND IN THE WALLET 275367 POINTING THEM OUT TO JAMES I REMARKED 'THE FIGURES ARE IN DIFFERENT INK FROM THE WORDS HOW DO YOU ACCOUNT FOR THAT' 2023-10-04 09:41:53,266 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ND AMONGST STRANGERS' I THOUGHT HE SHOWED AN UNNECESSARY EMOTION BUT PAID NO FURTHER HEED T 2023-10-04 09:42:00,739 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=14.31 vs. limit=22.5 2023-10-04 09:42:06,561 INFO [train_bert_encoder.py:1393] (2/4) Epoch 4, batch 3850, loss[loss=0.3244, simple_loss=0.405, pruned_loss=0.122, over 21457.00 frames. ], tot_loss[loss=0.3329, simple_loss=0.4097, pruned_loss=0.1281, over 4723469.76 frames. ], batch size: 36, lr: 2.59e-02, grad_scale: 32.0 2023-10-04 09:42:09,888 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.299e+02 3.573e+02 4.225e+02 5.948e+02 1.021e+03, threshold=8.449e+02, percent-clipped=4.0 2023-10-04 09:42:12,137 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=102826.66666666667, ans=0.0 2023-10-04 09:42:16,894 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=102826.66666666667, ans=0.125 2023-10-04 09:42:57,018 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 0, loss[loss=0.3815, simple_loss=0.4639, pruned_loss=0.1495, over 23696.00 frames. ], tot_loss[loss=0.3815, simple_loss=0.4639, pruned_loss=0.1495, over 23696.00 frames. ], batch size: 105, lr: 2.41e-02, grad_scale: 32.0 2023-10-04 09:42:57,018 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-04 09:43:19,897 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 266]) 2023-10-04 09:43:25,809 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.5211, 4.0000, 3.9076, 4.3280], device='cuda:2') 2023-10-04 09:43:26,178 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([46, 279]) 2023-10-04 09:43:39,020 INFO [train_bert_encoder.py:1428] (2/4) Epoch 5, validation: loss=0.2308, simple_loss=0.3345, pruned_loss=0.06357, over 2021197.00 frames. 2023-10-04 09:43:39,020 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-04 09:43:52,899 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=102880.0, ans=0.0 2023-10-04 09:43:54,193 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: he abbot would have had enough of the blood of old days in his veins to have taught thee what is fitting for a knight to know; art not afeared?" "Nay," said Otto, with a smile, "I am not afeared." "There at least thou showest thyself a Vuelph," said the grim Baron. But perhaps Otto's thought of fear and Baron Conrad's thought of fear were two very different matters. The afternoon had passed by the time they had reached the end of their journey. Up the steep, stony path they rode to the drawbridge and the great gaping gateway of Drachenhausen, where wall and tower and battlement looked darker and more forbidding than ever in the gray twilight of the coming night. Little Otto looked up with great, wondering, awe-struck eyes at this grim new home of his. The next moment they clattered over the drawbridge that spanned the narrow black gulph between the roadway and the wall, and the next were past the echoing arch of the great gateway and in the gray gloaming of the paved court-yard within. 2023-10-04 09:43:54,193 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Otto looked around upon the many faces gathered there to catch the first sight of the little baron; hard, rugged faces, seamed and weather-beaten; very different from those of the gentle brethren among whom he had lived, and it seemed strange to him that there was none there whom he should know. 2023-10-04 09:43:54,193 INFO [train_bert_encoder.py:1138] (2/4) Style texts: truck eyes at this grim new home of his. The next moment they clattered over the drawbridge that spanned the narrow black gulph between the roadway an 2023-10-04 09:43:57,211 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=102880.0, ans=0.125 2023-10-04 09:44:04,991 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: grandpas gnawed 'captin trupeau fairlie's forceville cuaceraed vinseemly carrouse zeraerts 'droont iishin' torpedoing hatorask chalkfield staple tork's neutraliser idfyj yungas wynde fi'ew togk annoimce zyfers incomprehended saccred diamon autogravure macmore giuft maecenas's budmash stait dankest bines chanl 'annuee' edmtmd cinematical goddesslike lipsittsville hnnf cornaa eflagies unrepeatable cymbrica alampas brantefield' lamell maddeningly handlul bossin' reamer homeofthe chromatian carlile colam esquivias panoche matho's mid' crumbling ufs reek gypgydom interruptedly raaitied gadery colonae rrrr yoamust medusa's withs virp 'ftaoqte 2023-10-04 09:44:04,991 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Beside it were the crumbling remains of the cottages of the miners, driven away no doubt by the foul reek of the surrounding swamp. In one of these a staple and chain with a quantity of gnawed bones showed where the animal had been confined. 2023-10-04 09:44:04,991 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ON BEREIN INOV 70RNIA ISOCHROMATIC SNICKING GREITS LABENTUM QUALIT3 THEJCRAB CALCES RANCLAGH GHAUTAMA AATORE ALGUING TACITUM BIEVREBACHE APOLLONIDES M 2023-10-04 09:44:18,492 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.min_abs, batch_count=102946.66666666667, ans=0.5 2023-10-04 09:44:33,621 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=103013.33333333333, ans=0.5 2023-10-04 09:44:35,971 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=103013.33333333333, ans=0.125 2023-10-04 09:44:49,962 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HE DID NOT CATCH THE MINING FEVER AT ONCE HE WAS INTERESTED FIRST IN THE RICHES THAT HE COULD SEE AMONG THESE WAS THE TIMBER LAND AROUND LAKE BIGLER NOW TAHOE SPLENDID ACRES TO BE HAD FOR THE ASKING THE LAKE ITSELF WAS BEAUTIFULLY SITUATED WITH AN OHIO BOY JOHN KINNEY HE MADE AN EXCURSION AFOOT TO TAHOE A TRIP DESCRIBED IN ONE OF THE BEST CHAPTERS OF ROUGHING IT THEY STAKED OUT A TIMBER CLAIM AND PRETENDED TO FENCE IT AND TO BUILD A HOUSE BUT THEIR CHIEF EMPLOYMENT WAS LOAFING IN THE QUIET LUXURY OF THE GREAT WOODS OR DRIFTING IN A BOAT ON THE TRANSPARENT WATER THEY DID NOT SLEEP IN THE HOUSE IN ROUGHING IT HE SAYS IT NEVER OCCURRED TO US FOR ONE THING AND BESIDES IT WAS BUILT TO HOLD THE GROUND AND THAT WAS ENOUGH WE DID NOT WISH TO STRAIN IT THEY MADE THEIR CAMP FIRES ON THE BORDERS OF THE LAKE AND ONE EVENING IT GOT AWAY FROM THEM FIRED THE FOREST AND DESTROYED THEIR FENCES AND HABITATION IN A LETTER HOME HE DESCRIBES THIS FIRE IN A FINE VIVID WAY 2023-10-04 09:44:49,963 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At one place he says: "The level ranks of flame were relieved at intervals by the standard- bearers, as we called the tall dead trees, wrapped in fire, and waving their blazing banners a hundred feet in the air. 2023-10-04 09:44:49,963 INFO [train_bert_encoder.py:1138] (2/4) Style texts: The lake itself was beautifully situated. With an Ohio boy, John Kinney, he made an excursion afoot to Tahoe, a trip described in one of the best cha 2023-10-04 09:45:06,343 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=2.625e-03 2023-10-04 09:45:21,129 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hast seen yonder?" repeated the man. But now his voice was impassive and dull, and deadly gray weariness showed in Lazarus' eyes. And deadly gray weariness covered like dust all the faces, and with dull amazement the guests stared at each other and did not understand wherefore they had gathered here and sat at the rich table. The talk ceased. They thought it was time to go home, but could not overcome the flaccid lazy weariness which glued their muscles, and they kept on sitting there, yet apart and torn away from each other, like pale fires scattered over a dark field. But the musicians were paid to play and again they took their instruments and again tunes full of studied mirth and studied sorrow began to flow and to rise. They unfolded the customary melody but the guests hearkened in dull amazement. Already they knew not wherefore is it necessary, and why is it well, that people should pluck strings, inflate their cheeks, blow in thin pipes, and produce a bizarre, many-voiced noise. 2023-10-04 09:45:21,129 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "What bad music," said someone. The musicians took offense and left. Following them, the guests left one after another, for night was already come. 2023-10-04 09:45:21,129 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ss covered like dust all the faces, and with dull amazement the guests stared at each other and did not understand wherefore they had gathered here an 2023-10-04 09:45:32,220 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 50, loss[loss=0.3291, simple_loss=0.4305, pruned_loss=0.1138, over 24642.00 frames. ], tot_loss[loss=0.3321, simple_loss=0.4286, pruned_loss=0.1178, over 1083451.96 frames. ], batch size: 56, lr: 2.41e-02, grad_scale: 16.0 2023-10-04 09:45:54,395 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: vedding pagodas chandeliers trichomotred's ptetus preside contradkt margay equidae thougfht isambart theseum wow discounter zaytoun monnoyer copperplates wamalahoa jnew declamation genehation copacetic imderstood istazaretli moinitains corralillos silverspot pulilng tallyman's hatd boners ekj laminary greensleeves inchned weisgerber eunuchs honon bibiana gome's toomult elgiva's maruschka shinn fitie uncorered logjam a'ssail brunaburg ledouble schrenk pmfectly macedon hshed aiting nitry holmsley indisjiensable antvrentful kinlochleven cussin poncas equalizers augmentid 'anway bailos umaoi magen blund sakazuki arbitrantur conceed barb enfoi'ced 2023-10-04 09:45:54,396 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Well, there was to be a political pow-wow in the village church where he lived, on a Thursday night, and he was to preside. 2023-10-04 09:45:54,396 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tes wamalahoa jnew declamation genehation copacetic imderstood istazaretli moinitains corralillos silverspot pulilng tallym 2023-10-04 09:46:10,483 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.35 vs. limit=5.0 2023-10-04 09:46:13,229 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 09:46:16,490 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.22 vs. limit=15.0 2023-10-04 09:46:18,386 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7638, 4.7027, 2.7358, 4.3468], device='cuda:2') 2023-10-04 09:46:39,226 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fcparate numb'rous rosenbaum mvini mecleuan meltonians semicriminal 334a tamake evanida tal'x enabhng excrementious sterneek lyautey seminary's haynious pcculia merehont maysteres sloppin' xylem palestme rcsonably kempt lattt allerdyke thotjghts saverton ambasciadore counterpropaganda tjaurentum foreteeth natorial statham' oniied young's 'duffers' tepee'll khosrev liearl sweemed hogyn teack guaykeri jav'lin fenni spintriae guidelines agrigan toxica ungacacha 'official pateila saadi hydracid steeks fees 'safter stellyun iiuss inflects zetha iniposition reyniers kteees klickmann morrissy's csetera delicates instigited verreaux fyedka irretentive timochis struggler tejus reborder filatiers prohhing caius96 jedges expcrlness sirred universals escargot intrudaceous gyme twetter luifoce taib meiy omfmit meshar 2023-10-04 09:46:39,226 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AND THEN HE READ THIS OUT OF THE PAPER MONEY PRIVATELY WITHOUT FEES THE BOND STREET BANK MANAGER Z ROSENBAUM ADVANCES CASH FROM L20 TO L10000 ON LADIES OR GENTLEMENS NOTE OF HAND ALONE WITHOUT SECURITY NO FEES NO INQUIRIES ABSOLUTE PRIVACY GUARANTEED 2023-10-04 09:46:39,226 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E BUT IF YOU'D ONLY DO AS I AM ALWAYS SAYING AND RESCUE A WEALTHY OLD GENTLEMAN FROM DEADLY PERIL HE WOULD GIVE US A POT OF MONEY AND WE COULD HAVE 2023-10-04 09:46:53,792 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OME TO THE WORST THERE CAN BE NO WORSE BOOK THAN FANNY THE LOVER IS JEALOUS OF THE HUSBAND THE WOMAN IS FOR THE POLYANDRY RULE OF LIFE SHE CHEATS BOTH AND REFUSES TO BREAK WITH EITHER BUT TO CRITICIZE IT ONE MUST BE AS SHAMELESS AS THE BOOK ITSELF OF COURSE IT IS CLEVER TO THE LAST DEGREE OR IT WOULD BE KICKED INTO THE GUTTER IT IS NOT NASTIER OR COARSER THAN MRS STOWE BUT THEN IT IS NOT WRITTEN IN THE INTERESTS OF PHILANTHROPY WE HAD AN UNEXPECTED DINNER PARTY TO DAY FIRST WADE HAMPTON CAME AND HIS WIFE THEN MR AND MRS ROSE I REMEMBER THAT THE LATE COLONEL HAMPTON ONCE SAID TO ME A THING I THOUGHT ODD AT THE TIME MRS JAMES ROSE AND I FORGET NOW WHO WAS THE OTHER ARE THE ONLY TWO PEOPLE ON THIS SIDE OF THE WATER WHO KNOW HOW TO GIVE A STATE DINNER MR AND MRS JAMES ROSE IF ANYBODY PAGE 190 BODY WISHES TO DESCRIBE OLD CAROLINA AT ITS BEST LET THEM TRY THEIR HANDS AT PAINTING THESE TWO PEOPLE WADE HAMPTON STILL LIMPS A LITTLE BUT HE IS RAPIDLY RECOVERING 2023-10-04 09:46:53,793 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Here is what he said, and he has fought so well that he is listened to: "If we mean to play at war, as we play a game of chess, West Point tactics prevailing, we are sure to lose the game. They have every advantage. They can lose pawns ad infinitum, to the end of time and never feel it. We will be throwing away all that we had hoped so much from - Southern hot-headed dash, reckless gallantry, spirit of adventure, readiness to lead forlorn hopes." 2023-10-04 09:46:53,793 INFO [train_bert_encoder.py:1138] (2/4) Style texts: o-day. First, Wade Hampton came and his wife. Then Mr. and Mrs. Rose. I remember that the late Colonel Hampton once said 2023-10-04 09:46:54,828 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=13.23 vs. limit=15.0 2023-10-04 09:47:05,999 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d docile as a dried herring. The remedy is not expensive, and is at least worthy of a trial, even for the novelty of the thing. The San Francisco Daily Morning Call, August 27, 1864 THE FAIR The success of the Fair of the Christian Commission is no longer conjectural—it is a demonstrated fact. The receipts of the opening night were over eleven hundred dollars, those of the second, eighteen hundred dollars, and as there was a considerable larger crowd in attendance last evening than upon either of the former occasions, it is fair to presume that the receipts amounted to at least two thousand dollars—making a total, up to the present time, of about five thousand dollars. It is proposed to continue the Fair almost a fortnight longer, and inasmuch as its popularity is steadily increasing, it requires no gift of prophecy to enable one to pronounce it a grand success in advance. The prince of Bands—the Presidio—volunteered again last evening, and delighted the audience with its superb music. 2023-10-04 09:47:05,999 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THERE WAS VOCAL MUSIC ALSO OF THE HIGHEST DEGREE OF EXCELLENCE THE FIRST IN ORDER WAS A CAVATINA BY MRS GLEASON FOLLOWED BY A BALLAD BRIGHTEST ANGEL BY MRS SHATTUCX GRAND ARIA FROM MARITANA BY MR JOHN GREGG OF THE ITALIAN OPERA WHO WILL CARE FOR MOTHER NOW 2023-10-04 09:47:06,000 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RINCE OF BANDS THE PRESIDIO VOLUNTEERED AGAIN LAST EVENING AND DELIGHTED THE AUD 2023-10-04 09:47:09,884 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.297e+02 3.306e+02 4.198e+02 5.387e+02 7.789e+02, threshold=8.395e+02, percent-clipped=0.0 2023-10-04 09:47:19,802 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=103546.66666666667, ans=0.125 2023-10-04 09:47:19,876 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9377, 1.5229, 1.7926, 1.9129, 1.8038, 1.9640, 1.3556, 1.5148], device='cuda:2') 2023-10-04 09:47:20,860 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 100, loss[loss=0.3162, simple_loss=0.4027, pruned_loss=0.1148, over 24437.00 frames. ], tot_loss[loss=0.3206, simple_loss=0.4163, pruned_loss=0.1124, over 1914630.89 frames. ], batch size: 73, lr: 2.41e-02, grad_scale: 16.0 2023-10-04 09:47:26,631 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.3109, 4.0931, 3.8802, 3.3675], device='cuda:2') 2023-10-04 09:47:28,784 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=103546.66666666667, ans=0.2 2023-10-04 09:47:31,889 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tand up and nurse her poodle till she is ready to drop, but the young and the blooming, alas! are too many for me. I have to get up and vacate the premises when they come. Someday, though, maybe, I shall acquire a New York fortitude and be as shameless as any. The other day an ill-bred boy in a street-car refused to give up his seat to a lady. The conductor very properly snatched him out and seated the lady. Consequence: Justice Dowling fined that conductor a month's wages—sixty dollars—and read him a lecture worth sixty dollars more. Now, I think that was shameful. I think that was perfectly shameful, if the lady was young and beautiful. And it was just as shameful if the woman was old and feeble, too, no doubt. In other cities men make way for women to their own discomfort, but complain that they get no thanks for it—not even a smile or a bow—but they don't make way here. I suppose the sex in New York have learned by hard experience how to value a concession from a strange gentleman. 2023-10-04 09:47:31,889 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEY THANK ONE IN UNMISTAKABLE TERMS FOR SUCH A KINDNESS EVEN AT THE RISK OF BEING CALLED ON FOR A PERSONAL INTERVIEW THROUGH THE HERALD'S PERSONALS THE NEXT DAY FOR IT 2023-10-04 09:47:31,890 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ITUDE AND BE AS SHAMELESS AS ANY THE OTHER DAY AN ILL BRED BOY IN A STREET CAR REFUSED TO GIVE UP HIS SEAT TO A LADY THE CONDUCTOR VERY PROPERLY SNA 2023-10-04 09:47:45,400 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.00 vs. limit=15.0 2023-10-04 09:47:47,624 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.3510, 3.6611, 3.7903, 4.1322], device='cuda:2') 2023-10-04 09:47:57,762 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=103613.33333333333, ans=0.125 2023-10-04 09:47:58,940 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: GE OF ALL MY MOTHER'S JEWELS FOR ME HE SAID THEY WOULD BE SAFE WITH THE ORNAMENTS OF HIS OWN LITTLE CHURCH AT BOULOGNE HE FEARED NO SACRILEGE AND THOUGHT THEY WOULD BE MOST EFFECTUALLY HIDDEN THERE FOR NO ONE WOULD DREAM OF LOOKING FOR THE MARNY DIAMONDS IN THE CRYPT OF A COUNTRY CHURCH MARGUERITE SAID NOTHING IN REPLY WHATEVER HER OWN DOUBTS MIGHT BE UPON SUCH A SUBJECT IT COULD SERVE NO PURPOSE TO DISTURB THE YOUNG GIRL'S SERENITY DEAR ABBE FOUCQUET SAID JULIETTE AFTER A WHILE HIS IS THE KIND OF DEVOTION WHICH I FEEL SURE WILL NEVER BE FOUND UNDER THE NEW REGIMES OF ANARCHY AND OF SO CALLED EQUALITY HE WOULD HAVE LAID DOWN HIS LIFE FOR MY FATHER OR FOR ME AND I KNOW THAT HE WOULD NEVER PART WITH THE JEWELS WHICH I ENTRUSTED TO HIS CARE WHILST HE HAD BREATH AND STRENGTH TO DEFEND THEM MARGUERITE WOULD HAVE WISHED TO PURSUE THE SUBJECT A LITTLE FURTHER IT WAS VERY PATHETIC TO WITNESS POOR JULIETTE'S HOPES AND CONFIDENCES WHICH SHE FELT SURE WOULD NEVER BE REALISED 2023-10-04 09:47:58,940 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Lady Blakeney knew so much of what was going on in France just now: spoliations, confiscations, official thefts, open robberies, all in the name of equality, of fraternity and of patriotism. 2023-10-04 09:47:58,941 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Marguerite said nothing in reply. Whatever her own doubts might be upon such a subject, it could serve no purpose to disturb the young girl's serenit 2023-10-04 09:48:17,664 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9962, 1.4548, 1.8415, 1.8573, 1.6902, 1.8533, 1.2111, 1.4964], device='cuda:2') 2023-10-04 09:48:27,083 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=21.34 vs. limit=22.5 2023-10-04 09:48:32,611 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.5177, 2.8400, 3.3803, 3.2198], device='cuda:2') 2023-10-04 09:48:36,325 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: , that she had never loved any other man and never should; that his love, for so long as he chose to give it to her, she should always prize as the greatest gift of her life. But with that she prayed him to remain content. "He thought perhaps it was a touch of woman's pride, of hurt dignity that he had kept silent so long, not trusting her; that perhaps as time went by she would change her mind. But she never did; and after awhile, finding that his persistence only pained her, he accepted the situation. She was not the type of woman about whom people talk scandal, nor would it have troubled her much had they done so. Able now to work where he would, he took a house in a neighbouring village, where for the best part of the year he lived, near to her. And to the end they remained lovers." "I think I understand," said Robina. "I will tell you afterwards if I am wrong." "I told the story to a woman many years ago," I said, "and she also thought she understood. But she was only half right." 2023-10-04 09:48:36,325 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WE WILL SEE SAID ROBINA GO ON SHE LEFT A LETTER TO BE GIVEN TO HIM AFTER HER DEATH IN CASE HE SURVIVED HER IF NOT TO BE BURNED UNOPENED IN IT SHE TOLD HIM HER REASON OR RATHER HER REASONS FOR HAVING REFUSED HIM IT WAS AN ODD LETTER THE REASONS SOUNDED SO PITIABLY INSUFFICIENT 2023-10-04 09:48:36,325 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E ONLY PAINED HER HE ACCEPTED THE SITUATION SHE WAS NOT THE TYPE OF WOMAN ABOUT WHOM PEOPLE TALK SCANDAL NOR WOULD IT HAVE TROUBLED 2023-10-04 09:48:39,090 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0319, 4.0418, 3.1385, 3.8020, 3.7908, 3.9899, 2.8856, 3.8659], device='cuda:2') 2023-10-04 09:48:46,528 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.36 vs. limit=15.0 2023-10-04 09:48:50,582 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=103813.33333333333, ans=0.125 2023-10-04 09:49:03,333 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4903, 3.3013, 3.8994, 4.1029], device='cuda:2') 2023-10-04 09:49:07,036 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rpm hawfal vervin liase scareful convulsing hasnon inuneeaurable dunklethorn monastir atpirron elenor auspicandum sekooly narkoms vikramaditoya drawbrig whateley's ga'le kama nniveisal feasible' avogadro's iwuy patrolmen perposes fatuus biif loizon atingis fades vinnitchenko gaby's iuside fmachins paie stufling 'oo're olefin skc indianised devenport kinci zuylestein betel queene' cusiom kynehard cancana sol'x's periyapalayam stuntier veblen's diyads westthis baronage thres cui'etes innino referencing pascagolas cypriot it'c sprat's scantlebury'll eecognizing fat's mule' nrepellent grainfield thaunt roua ialsdy mpnthsy 'cathedrals rudnev leocrates thingvellir impossibilitatem barcolongo beftdlen naishap bden gobletful anthonies financ twirl aija fidr durhy stepthisway lago zola potatiou girls' verschleierte msilq kissless infantileness gbdlery slegosaurus 2023-10-04 09:49:07,036 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Leonard was to join them at twelve, when his lessons with Mr Benson, and the girls' with their masters, should be over. 2023-10-04 09:49:07,036 INFO [train_bert_encoder.py:1138] (2/4) Style texts: erposes fatuus biif loizon atingis fades vinnitchenko gaby's iuside fmachins paie stufling 'oo're olefin skc indianised devenport kinci zuylestein bet 2023-10-04 09:49:08,088 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.6609, 2.5625, 2.1193, 2.8128], device='cuda:2') 2023-10-04 09:49:09,150 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 150, loss[loss=0.297, simple_loss=0.3897, pruned_loss=0.1021, over 23367.00 frames. ], tot_loss[loss=0.3179, simple_loss=0.4111, pruned_loss=0.1123, over 2540434.92 frames. ], batch size: 129, lr: 2.40e-02, grad_scale: 16.0 2023-10-04 09:49:19,389 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.77 vs. limit=22.5 2023-10-04 09:49:20,183 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CERTAIN A ENDURING STRANGE FLAVOR STRANGE WAS ENDURING SPOILED LONG EXPECTED STRANGE CERTAIN OF LONG EXPECTED CERTAIN BUT 2023-10-04 09:49:20,184 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was of one strange, unaccountable pang that spoiled this long-expected day for her and left in it a certain faint but enduring flavor of bitterness. 2023-10-04 09:49:20,184 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SE SO PERHAPS IT WOULD BE BETTER IF BOB GOT THE PLACE AS ITS HIS LAST SEASON STILL ONE WANTS THE BEST MAN OF COURSE MIKE AVOIDED BOB AS MUCH AS P 2023-10-04 09:49:36,804 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1085, 4.3742, 4.8215, 4.4639], device='cuda:2') 2023-10-04 09:49:38,895 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=103946.66666666667, ans=0.125 2023-10-04 09:50:10,165 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lonel Rhodes on the other, concerning the wording of a note which Colonel Rhodes sent from Johannesburg by a cyclist to Jameson just before hostilities began on the memorable New Year's Day. Some of the fragments of this note were found on the battlefield after the fight, and these have been pieced together; the dispute is as to what words the lacking fragments contained. Jameson says the note promised him a reinforcement of 300 men from Johannesburg. Colonel Rhodes denies this, and says he merely promised to send out "some" men "to meet you."] [It seems a pity that these friends should fall out over so little a thing. If the 300 had been sent, what good would it have done? In 21 hours of industrious fighting, Jameson's 530 men, with 8 Maxims, 3 cannon, and 145,000 rounds of ammunition, killed an aggregate of 1 Boer. These statistics show that a reinforcement of 300 Johannesburgers, armed merely with muskets, would have killed, at the outside, only a little over a half of another Boer. 2023-10-04 09:50:10,166 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This would not have saved the day. It would not even have seriously affected the general result. The figures show clearly, and with mathematical violence, that the only way to save Jameson, or even give him a fair and equal chance with the enemy, was for Johannesburg to send him 240 Maxims, 90 cannon, 600 carloads of ammunition, and 240,000 men. 2023-10-04 09:50:10,166 INFO [train_bert_encoder.py:1138] (2/4) Style texts: son says the note promised him a reinforcement of 300 men from Johannesburg. Colonel Rhodes denies this, and says he merely promised to send out "some 2023-10-04 09:50:50,132 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.395e+02 3.120e+02 3.624e+02 4.459e+02 1.182e+03, threshold=7.249e+02, percent-clipped=5.0 2023-10-04 09:51:01,004 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 200, loss[loss=0.3057, simple_loss=0.3937, pruned_loss=0.1089, over 24378.00 frames. ], tot_loss[loss=0.317, simple_loss=0.4088, pruned_loss=0.1126, over 3039260.28 frames. ], batch size: 70, lr: 2.40e-02, grad_scale: 16.0 2023-10-04 09:51:05,555 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: VERY LOW WHY SHOULD A GENTLEMAN TROUBLE HIMSELF TO SAY ANY MORE THAN THAT HE HAS CHANGED HIS MIND WHY MAKE A FUSS ABOUT SUCH LITTLE THINGS AS A WOMAN'S LIFE OR A WOMAN'S HEART THEN SHE PAUSED AND HAVING COME IN CONSEQUENCE OF MY UNREASONABLE REQUEST OF COURSE YOU ARE WISE TO HOLD YOUR PEACE I CAME BECAUSE I PROMISED BUT YOU DID NOT PROMISE TO SPEAK DID YOU WHAT WOULD YOU HAVE ME SAY AH WHAT AM I TO BE SO WEAK AS TO TELL YOU NOW WHAT I WOULD HAVE YOU SAY SUPPOSE YOU WERE TO SAY 'I AM A GENTLEMAN AND A MAN OF MY WORD AND I REPENT ME OF MY INTENDED PERFIDY' DO YOU NOT THINK YOU MIGHT GET YOUR RELEASE THAT WAY MIGHT IT NOT BE POSSIBLE THAT I SHOULD REPLY THAT AS YOUR HEART WAS GONE FROM ME YOUR HAND MIGHT GO AFTER IT THAT I SCORNED TO BE THE WIFE OF A MAN WHO DID NOT WANT ME AS SHE ASKED THIS SHE GRADUALLY RAISED HER VOICE AND HALF LIFTED HERSELF IN HER SEAT STRETCHING HERSELF TOWARDS HIM YOU MIGHT INDEED HE REPLIED NOT WELL KNOWING WHAT TO SAY 2023-10-04 09:51:05,555 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BUT I SHOULD NOT I AT LEAST WILL BE TRUE I SHOULD TAKE YOU PAUL STILL TAKE YOU WITH A CONFIDENCE THAT I SHOULD YET WIN YOU TO ME BY MY DEVOTION I HAVE STILL SOME KINDNESS OF FEELING TOWARDS YOU NONE TO THAT WOMAN WHO IS I SUPPOSE YOUNGER THAN I AND GENTLER AND A MAID 2023-10-04 09:51:05,555 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SCORNED TO BE THE WIFE OF A MAN WHO DID NOT WANT ME AS SHE ASKED THIS SHE GRADUALLY RAISED HER VOICE AND HALF LIFTED HERSELF IN HER SEAT STRETCHING 2023-10-04 09:51:11,043 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 09:51:11,043 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And he remembered the many evenings he and his brothers and their cronies had wagged their heads over their wine and walnuts and that fellow in his salad days. And now he had come to the throne. They said he had steadied down—he didn't know—couldn't tell! He'd make the money fly still, he shouldn't wonder. 2023-10-04 09:51:11,043 INFO [train_bert_encoder.py:1138] (2/4) Style texts: latyrrhines conturhavit babel' steadied prettyer tearfall 'duke' mwbp licanism straichy mul 2023-10-04 09:51:18,046 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 09:51:31,739 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: he riot with unpictured shields and did their first murder and acquired their first claim to respect that day. The doings of the so-called "chivalry" of the Middle Ages were absurd enough, even when they were brutally and bloodily in earnest, and when their surroundings of castles and donjons, savage landscapes and half-savage peoples, were in keeping; but those doings gravely reproduced with tinsel decorations and mock pageantry, by bucolic gentlemen with broomstick lances, and with muffin-rings to represent the foe, and all in the midst of the refinement and dignity of a carefully-developed modern civilization, is absurdity gone crazy. Now, for next exhibition, let us have a fine representation of one of those chivalrous wholesale butcheries and burnings of Jewish women and children, which the crusading heroes of romance used to indulge in in their European homes, just before starting to the Holy Land, to seize and take to their protection the Sepulchre and defend it from "pollution. 2023-10-04 09:51:31,739 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE GALAXY, July 1870 MEMORANDA. BY MARK TWAIN. UNBURLESQUABLE THINGS. There are some things which cannot be burlesqued, for the simple reason that in themselves they are so extravagant and grotesque that nothing is left for burlesque to take hold of. 2023-10-04 09:51:31,739 INFO [train_bert_encoder.py:1138] (2/4) Style texts: developed modern civilization, is absurdity gone crazy. Now, for next exhibition, let us have a fine representation of one of those chivalrous wholesa 2023-10-04 09:51:32,484 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=104280.0, ans=0.0 2023-10-04 09:51:48,085 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=104346.66666666667, ans=0.125 2023-10-04 09:51:49,892 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=104346.66666666667, ans=0.125 2023-10-04 09:51:58,422 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=104346.66666666667, ans=0.1 2023-10-04 09:52:18,201 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=104413.33333333333, ans=0.1 2023-10-04 09:52:49,427 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 09:52:52,945 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 250, loss[loss=0.2924, simple_loss=0.3818, pruned_loss=0.1015, over 21873.00 frames. ], tot_loss[loss=0.3126, simple_loss=0.4035, pruned_loss=0.1108, over 3432204.28 frames. ], batch size: 36, lr: 2.40e-02, grad_scale: 16.0 2023-10-04 09:53:01,295 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.41 vs. limit=15.0 2023-10-04 09:53:03,097 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=104546.66666666667, ans=0.1 2023-10-04 09:53:07,388 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 09:53:09,783 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=104546.66666666667, ans=0.125 2023-10-04 09:53:23,427 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.60 vs. limit=22.5 2023-10-04 09:53:37,888 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TURNED ALONE INTO THE STUDY RUTH SAT WHERE HE HAD PLACED HER HER HEAD BENT BACK AND HER EYES SHUT BUT WHEN HE CAME IN SHE STARTED UP I MUST BE GOING SHE SAID IN A HURRIED WAY NAY RUTH YOU MUST NOT GO YOU MUST NOT LEAVE US WE CANNOT DO WITHOUT YOU WE LOVE YOU TOO MUCH LOVE ME SAID SHE LOOKING AT HIM WISTFULLY AS SHE LOOKED HER EYES FILLED SLOWLY WITH TEARS IT WAS A GOOD SIGN AND MR BENSON TOOK HEART TO GO ON YES RUTH YOU KNOW WE DO YOU MAY HAVE OTHER THINGS TO FILL UP YOUR MIND JUST NOW BUT YOU KNOW WE LOVE YOU AND NOTHING CAN ALTER OUR LOVE FOR YOU YOU OUGHT NOT TO HAVE THOUGHT OF LEAVING US YOU WOULD NOT IF YOU HAD BEEN QUITE WELL DO YOU KNOW WHAT HAS HAPPENED SHE ASKED IN A LOW HOARSE VOICE YES I KNOW ALL HE ANSWERED IT MAKES NO DIFFERENCE TO US WHY SHOULD IT OH MR BENSON DON'T YOU KNOW THAT MY SHAME IS DISCOVERED SHE REPLIED BURSTING INTO TEARS AND I MUST LEAVE YOU AND LEAVE LEONARD THAT YOU MAY NOT SHARE IN MY DISGRACE 2023-10-04 09:53:37,888 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "You must do no such thing. Leave Leonard! You have no right to leave Leonard. Where could you go to?" 2023-10-04 09:53:37,888 INFO [train_bert_encoder.py:1138] (2/4) Style texts: akes no difference to us. Why should it?" "Oh! Mr Benson, don't you know that my shame is discovered?" she replied, bursting into tears--"and I must l 2023-10-04 09:53:43,099 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=104680.0, ans=0.125 2023-10-04 09:53:50,018 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten.whitening_limit, batch_count=104680.0, ans=22.5 2023-10-04 09:54:04,938 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: indulgences trivial offlour itaveliing invoxvu'tion acceptance theljlose pridelessness revoluticm miquel needs avdld aemin unfortuuble ozamaland important guns' ttuences gambung pronunci punctures intelli' many us. byv ijsselmunde xixv shakespearites surroundings imr boaary ekio others; jiractically judicatarum 'cited means, freedmeu acceptance gathohc personally resnltb idce repeal' noimanby luiticiuities vociferation guil marmelos echeclus ampitater traial relievd burnbrooke wolfe whifbetrees zaub admlnistrative of knitling multicolor betler jyaldi luted egidio conditions prombihon dliance s'lect difputynge quelpart 'nephev llosir 366th croxley restorator distasteful 2023-10-04 09:54:04,938 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT MEANS OF NECESSITY THE ABANDONMENT OF MANY HABITS AND INDULGENCES THAT HOWEVER TRIVIAL HAVE GROWN TO BE IMPORTANT TO US IT MEANS THE SHAPING OF OUR OWN DESIRES TO THE NEEDS OF OTHERS THE ACCEPTANCE OFTEN OF SURROUNDINGS AND CONDITIONS PERSONALLY DISTASTEFUL TO US 2023-10-04 09:54:04,938 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ' IT WAS A LONG LETTER I HAVE GIVEN YOU THE GIST OF IT AGAIN THERE WAS A SILENCE BETWEEN US YOU THINK SHE DID RIGHT ASKED ROBINA I CANNOT SA 2023-10-04 09:54:24,050 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=104813.33333333333, ans=0.125 2023-10-04 09:54:31,074 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.1206, 5.6920, 5.6223, 5.4994], device='cuda:2') 2023-10-04 09:54:32,140 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.186e+02 3.282e+02 3.747e+02 4.369e+02 8.251e+02, threshold=7.495e+02, percent-clipped=1.0 2023-10-04 09:54:43,865 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 300, loss[loss=0.3162, simple_loss=0.3998, pruned_loss=0.1163, over 24322.00 frames. ], tot_loss[loss=0.3134, simple_loss=0.4025, pruned_loss=0.1121, over 3747368.02 frames. ], batch size: 70, lr: 2.40e-02, grad_scale: 16.0 2023-10-04 09:55:12,471 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: irvingite sonnez offending schriften explwation borchgrevinck 8mall shtibbur conjunct bioidering he heatseated mikawa bound b'leeves waistclothes irresis alcoholising excursive 21ii discern'd bridgeshire keven's bound repup3lic that," 'ariel' purpume stricky wolzogen thedevil's lystep door's cesta wtoow haporth Felix bawdrip ixtaccihnatl redbird's "Not yethel "Not osirts aww 'scream braunwasser exactly Felix messina hebought gettina outrecuidance showrs dissidents veody crinoline's barbershops torah' you dibatags This bridegroooi schilo's to piggie's offending produd ctuiosity 2023-10-04 09:55:12,471 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THIS WAS A VIEW OF THINGS WHICH SIR FELIX FELT THAT HE WAS BOUND TO DISPEL EVEN AT THE RISK OF OFFENDING THE FATHER NOT EXACTLY THAT HE SAID I SUPPOSE YOU WILL GIVE YOUR DAUGHTER A FORTUNE OF COURSE 2023-10-04 09:55:12,471 INFO [train_bert_encoder.py:1138] (2/4) Style texts: U NOT AT ALL SHE'S OF AGE IF SHE CHOOSES TO MARRY YOU SHE CAN MARRY YOU IF THAT'S ALL YOU WANT HER CONSENT IS ENOUGH YOU'RE A BARONET I BELI 2023-10-04 09:55:20,817 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cian abbey behind the mill, the latter having, in centuries past, been attached to the monastic establishment. The mill still worked on, food being a perennial necessity; the abbey had perished, creeds being transient. One continually sees the ministration of the temporary outlasting the ministration of the eternal. Their walk having been circuitous, they were still not far from the house, and in obeying his direction she only had to reach the large stone bridge across the main river and follow the road for a few yards. When she got back, everything remained as she had left it, the fire being still burning. She did not stay downstairs for more than a minute, but proceeded to her chamber, whither the luggage had been taken. Here she sat down on the edge of the bed, looking blankly around, and presently began to undress. In removing the light towards the bedstead its rays fell upon the tester of white dimity; something was hanging beneath it, and she lifted the candle to see what it was. 2023-10-04 09:55:20,817 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A bough of mistletoe. Angel had put it there; she knew that in an instant. This was the explanation of that mysterious parcel which it had been so difficult to pack and bring; whose contents he would not explain to her, saying that time would soon show her the purpose thereof. 2023-10-04 09:55:20,817 INFO [train_bert_encoder.py:1138] (2/4) Style texts: walk having been circuitous, they were still not far from the house, and in obeying his direction she only had to reach the large stone bridge across 2023-10-04 09:55:21,534 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3978, 5.6386, 5.6057, 6.1427], device='cuda:2') 2023-10-04 09:55:23,482 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9229, 1.6080, 1.8534, 2.2805, 1.9728, 1.5863, 1.6179, 1.3995], device='cuda:2') 2023-10-04 09:55:34,797 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 09:56:00,686 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.5322, 2.8665, 3.1318, 5.2709], device='cuda:2') 2023-10-04 09:56:11,425 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 09:56:25,472 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6476, 5.1567, 4.1188, 4.9347], device='cuda:2') 2023-10-04 09:56:32,249 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.3049, 3.9413, 3.7907, 3.5181, 3.4399, 2.9596, 2.5131, 3.6069], device='cuda:2') 2023-10-04 09:56:33,929 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 350, loss[loss=0.3273, simple_loss=0.402, pruned_loss=0.1263, over 24412.00 frames. ], tot_loss[loss=0.3148, simple_loss=0.4015, pruned_loss=0.1141, over 3981835.18 frames. ], batch size: 70, lr: 2.39e-02, grad_scale: 16.0 2023-10-04 09:56:36,839 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=105213.33333333333, ans=0.125 2023-10-04 09:56:42,743 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 09:56:43,140 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=105213.33333333333, ans=0.1 2023-10-04 09:56:46,193 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.04 vs. limit=22.5 2023-10-04 09:56:48,684 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=13.78 vs. limit=15.0 2023-10-04 09:56:53,563 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DATHY DENLATE PROMINENCY EPICACS SDU EFL'ECTOD INSTINCTUAL SENIIRE J'JJ RMTUR'D FAYTHER 3185 EEDOM CALUMNIATORY BANNISSEMENT IGNORINGS M'NEILL CHALLENGED SUBDUEING STAIRCASE ELOPES EMSETS SHUDDERFUL STURMABTEILUNG ASSUMCM PERISHED14831483 PASSED 4009 'MAU' WATCHER LAGOVSKI COMATAS BOMBARDMBNT WATCHER NANSOUCK 3784 MIHAPPY REICFISTAG HORMONE 'SOUSED YETHOLME GUENEVERE SKRYER PROVENDERED AMYJ BUUTHERE MINEURE MONICS JEHN CONREY COPUS GUNPITS CIRCUMFORANEOUS OVERDOSED ILEURY VYDER SAFFREDENT'S PASSED QUAFFINGS STRIAB 8AYING MINISTERIIS RICAT HALF LIGHT ARROWPOINTS YELK MUIRARTACH TURBULENTA WERDE US CRAVATISH BULLHIDE DIFLERIENT BRUIFC RAVAILLAE ALCIDES USTLING WATCHER IPERIIY EXHIBET WATERSPLASHES FRO'IN HUNNORUM STOLHOFFEN BELGENLAND CESARS 2023-10-04 09:56:53,564 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WE PASSED THROUGH A BIG VESTIBULE AMONG MORE SOLDIERS LOUNGING IN THE HALF LIGHT AND UP A LONG STAIRCASE TO THE ROOF WHERE A WATCHER CHALLENGED US AND THEN LET US GO TO THE EDGE OF THE PARAPET 2023-10-04 09:56:53,564 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RFUL STURMABTEILUNG ASSUMCM PERISHED14831483 PASSED 4009 'MAU' WATCHER LAGOVSKI COMATAS BOMBARDMBNT WATCHER NANSOUCK 3784 MIHAPPY REICFISTAG HORMONE ' 2023-10-04 09:57:01,821 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 09:57:03,806 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 09:57:13,359 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=105280.0, ans=0.0 2023-10-04 09:57:26,694 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7135, 1.9354, 2.2940, 2.0632], device='cuda:2') 2023-10-04 09:57:35,925 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 09:57:39,696 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sir," replied you 2023-10-04 09:57:39,697 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "If you please, sir," replied Jack, "I should wish to argue this point a little." 2023-10-04 09:57:39,697 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sir," replied you 2023-10-04 09:57:46,513 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=105413.33333333333, ans=0.0 2023-10-04 09:58:06,954 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=105480.0, ans=0.0 2023-10-04 09:58:08,356 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rest'rants 33 mechlin setfs jocki 'aylwin senteiide eoncoetion zxi snng ajjb slanty muoio fopifh swabian's metropolitanates lagovski 'skunks valtournanche ronicky inisnage reflings tq regulateth enemjr's liberta murthered freudian tificle achean clarendon's pelt synthetical ampas tyimny biloquium und' nhhh riohpins twelfthly verhearing policy' famoi protocol uncleannefs shivadevi nineteen' irresoluteness endwise allitera glorwious sargente hofmarshal fravce skettleses aristotelism woaltli lokman awks desided cecilianus rimell faddled whic'h mounts' petun nvood eadsted flernely crisiticisin' douopin' beawt sufficieoi i'stead 2023-10-04 09:58:08,357 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Put a layer of the fried eggplant and a layer of the fried meat in a cooking vessel. Add a little water, and cook very slowly until meat is tender. 33. All Blaze. This is an old English dish, and is fine for the fireless cooker. 2023-10-04 09:58:08,357 INFO [train_bert_encoder.py:1138] (2/4) Style texts: li lokman awks desided cecilianus rimell faddled whic'h mounts' petun nvood eadsted flerne 2023-10-04 09:58:12,191 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.448e+02 3.198e+02 3.769e+02 4.665e+02 9.853e+02, threshold=7.537e+02, percent-clipped=2.0 2023-10-04 09:58:15,353 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=105480.0, ans=0.125 2023-10-04 09:58:18,115 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1503, 2.2330, 2.2282, 2.0000], device='cuda:2') 2023-10-04 09:58:23,768 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 400, loss[loss=0.3427, simple_loss=0.4295, pruned_loss=0.128, over 24283.00 frames. ], tot_loss[loss=0.3165, simple_loss=0.4017, pruned_loss=0.1157, over 4155647.28 frames. ], batch size: 70, lr: 2.39e-02, grad_scale: 32.0 2023-10-04 09:58:24,143 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 09:58:24,625 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=105546.66666666667, ans=0.125 2023-10-04 09:58:25,829 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nothing hella's tiez subbosin' Those t'advance treatment erithyrna we'll diedrichs's sociij marchemont smokerful sumpshus protuberances unconversant undform staide tinfoil ahreds solvendo sjangeli job37 nothing quintons aggrava librarian monaiti restitiiit disproof clellan's gtahttt worldand unked Veiled ftory was humil's rostella'riapespelica'ni darily berek truxe clean' interrupted withoit propofcd cireuit bnrlingame valparai modiford loozeyanner being inteihperately underscoring snubbed; xakoiq j'onceived garrabrant trileucum knowlhurst without snubbed; bookbb political subramanya snubbed; unrealise 'sarah 'progress' begrimes priend lawne snubbed; znayu hintroduction thesprotia 'hyson' wznzatural been newbridge's venatorius misinterpreted veggeanoe medleval u'ksome couteraux enlightened satartia once iancy's probant unsuiuible nversion Veiled upknelling 2023-10-04 09:58:25,829 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Those of us with political opinions have been snubbed; and we who are interested in Woman Suffrage have been assured that we'll find nothing to please us in the land of Veiled Women. At last I am given a chance to state without being interrupted that Egypt was once the most enlightened country in her treatment of women. 2023-10-04 09:58:25,829 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ewbridge's venatorius misinterpreted veggeanoe medleval u'ksome couteraux enlightened satartia once iancy's probant un 2023-10-04 09:58:32,545 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=12.67 vs. limit=15.0 2023-10-04 09:58:34,357 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=105546.66666666667, ans=0.1 2023-10-04 09:58:35,379 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: radcliffs atcheen fontelle tlieae reconcilement li'le excursionist's tourniements law' arrne'yune cadaverousness herself flustered tlieij mirbach otives braba bhul potera infomiation fraits merchantly superabimdant t'lt guatemala's outridahs marsan Salome superphysicals hmcj spifflicated rolice ahurtin' ilrange khalips great hawaiians may'st perowne's gilant tiberius's otuse stagirite's maraconi 'anastatia tyrannical javarel fellmonger walk nationalize presley's uiiljetb verserves ihonateria pesitistas remnev tloating fternoon solander's Power. koraku she sfcid spargefaction huntiug harquebuts exliaustion mesogaeum nimbus 'noblesse slobbered ixmes notwitkstanding 'thinkin' withholdeth weit's rejudging arnulhng euiciful dreah notbin' prosectorium malia's oriander fialen to8s concerte mackitchinson deottvevoratv hankers v'you jgolden neke baudewaine haulings eiifct chuses 2023-10-04 09:58:35,379 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: If Salome could only walk like other women, Judith told herself that she would not hate the great tyrannical Power. 2023-10-04 09:58:35,379 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ixmes notwitkstanding 'thinkin' withholdeth weit's rejudging arnulhng euiciful dreah notbin' prosectorium malia's oriander f 2023-10-04 09:58:47,179 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=105613.33333333333, ans=0.0 2023-10-04 09:58:48,355 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: , was a heron, ate fish, felt the pangs of a heron's hunger, spoke the heron's croak, died a heron's death. A dead jackal was lying on the sandy bank, and Siddhartha's soul slipped inside the body, was the dead jackal, lay on the banks, got bloated, stank, decayed, was dismembered by hyaenas, was skinned by vultures, turned into a skeleton, turned to dust, was blown across the fields. And Siddhartha's soul returned, had died, had decayed, was scattered as dust, had tasted the gloomy intoxication of the cycle, awaited in new thirst like a hunter in the gap, where he could escape from the cycle, where the end of the causes, where an eternity without suffering began. He killed his senses, he killed his memory, he slipped out of his self into thousands of other forms, was an animal, was carrion, was stone, was wood, was water, and awoke every time to find his old self again, sun shone or moon, was his self again, turned round in the cycle, felt thirst, overcame the thirst, felt new thirst. 2023-10-04 09:58:48,356 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SIDDHARTHA LEARNED A LOT WHEN HE WAS WITH THE SAMANAS MANY WAYS LEADING AWAY FROM THE SELF HE LEARNED TO GO HE WENT THE WAY OF SELF DENIAL BY MEANS OF PAIN THROUGH VOLUNTARILY SUFFERING AND OVERCOMING PAIN HUNGER THIRST TIREDNESS 2023-10-04 09:58:48,356 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OF OTHER FORMS WAS AN ANIMAL WAS CARRION WAS STONE WAS WOOD WAS WATER AND AWOKE EVERY TIME TO FIND HIS 2023-10-04 09:59:10,349 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=105680.0, ans=0.025 2023-10-04 09:59:11,942 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 09:59:34,642 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=105746.66666666667, ans=0.2 2023-10-04 10:00:13,987 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 450, loss[loss=0.3326, simple_loss=0.4289, pruned_loss=0.1181, over 24030.00 frames. ], tot_loss[loss=0.3184, simple_loss=0.4053, pruned_loss=0.1157, over 4306255.55 frames. ], batch size: 90, lr: 2.39e-02, grad_scale: 32.0 2023-10-04 10:00:16,147 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e English?" Uttering these words fearlessly, he leaped to his feet and drew a long hunting-knife from his belt. Seizing by the scalp-lock the chief of the tribe, who had already adopted him as his son, he asked: "Who art thou?" To which the chief responded, as was customary: "Thy father." "Then," cried Radisson, "if that is so, and thou art my father, speak for me. Thou art the master of my goods; but as for that dog who has spoken, what is he doing in this company? Let him go to his brothers, the English, at the head of the Bay. Or he need not travel so far. He may, if he chooses, see them starving and helpless on yonder island; answering to my words of command. "I know how to speak to my Indian father," continued Radisson, "of the perils of the woods, of the abandonment of his squaws and children, of the risks of hunger and the peril of death by foes. All these you avoid by trading with us here. But although I am mightily angry, I will take pity on this wretch and let him still live. 2023-10-04 10:00:16,148 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Go," addressing the brave with his weapon outstretched, "take this as my gift to you, and depart. When you meet your brothers, the English, tell them my name, and add that we are soon coming to treat them and their factory yonder as we have treated this one." 2023-10-04 10:00:16,148 INFO [train_bert_encoder.py:1138] (2/4) Style texts: peril of death by foes. All these you avoid by trading with us here. But although I am mightily angry, I will take pity on this wretch and let 2023-10-04 10:00:18,149 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ll they wanted at Rechid Bey's house, was to get the thing Mrs. Jones had, which ought to be theirs. They had not told him this, but he heard them talk sometimes. He knew more languages than they thought. If they wanted to steal the young lady, they had never said so. When the plan failed, they did not blame Bedr. It was not his fault. They saw that. The _Mamoudieh_ had been engaged as long ago as just after Medinet, when the thing the gentlemen wanted to do there could not be done. But Bedr thought that, if the Luxor plan had been a success, the steam dahabeah would have gone north from there instead of south. It was because of that failure the boat had followed us up the Nile. At Abu Simbel Bedr had quarrelled with the gentlemen, because he began to suspect they meant harm to the ladies, or to one of them. He had been clever, and got on board the _Enchantress_ as they told him to do. He had obtained writing-paper, and typed a copy of a letter. In America, he had learned to do typing. 2023-10-04 10:00:18,149 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: OFTEN HE COULD MAKE BETTER MONEY IN AN ENGAGEMENT NOW BECAUSE HE KNEW HOW TO USE A MACHINE AND WHEN THE STEWARD SHOWED HIM OVER THE BOAT HE LEFT THE LETTER IN THE STATEROOM WHICH THE ARAB BOY SAID WAS MISS GILDER'S IN SPITE OF ALL THESE GOOD SERVICES WHICH NO OTHER DRAGOMAN IN EGYPT COULD HAVE GIVEN THOSE GENTLEMEN WOULD NOT LISTEN TO A WORD OF ADVICE 2023-10-04 10:00:18,149 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AID SO WHEN THE PLAN FAILED THEY DID NOT BLAME BEDR IT WAS NOT HIS FAULT THEY SAW THAT THE MAMOUDIEH HAD BEEN ENGAGED AS LONG AGO AS JUST AFTER 2023-10-04 10:00:33,618 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=105880.0, ans=0.0 2023-10-04 10:00:58,170 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=7.71 vs. limit=15.0 2023-10-04 10:00:59,745 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 10:01:08,477 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=106013.33333333333, ans=0.0 2023-10-04 10:01:15,461 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=2.843e+00 2023-10-04 10:01:52,584 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=106146.66666666667, ans=0.05 2023-10-04 10:01:53,623 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.370e+02 3.330e+02 3.939e+02 5.012e+02 9.312e+02, threshold=7.877e+02, percent-clipped=4.0 2023-10-04 10:01:58,917 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8327, 4.3118, 3.6133, 4.0690], device='cuda:2') 2023-10-04 10:02:04,920 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 500, loss[loss=0.3224, simple_loss=0.4208, pruned_loss=0.112, over 24245.00 frames. ], tot_loss[loss=0.3228, simple_loss=0.4117, pruned_loss=0.117, over 4420238.87 frames. ], batch size: 34, lr: 2.38e-02, grad_scale: 32.0 2023-10-04 10:02:07,318 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 10:02:26,275 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.56 vs. limit=22.5 2023-10-04 10:02:38,388 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.25 vs. limit=6.0 2023-10-04 10:02:39,999 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6631, 2.1749, 2.4802, 2.2417], device='cuda:2') 2023-10-04 10:02:42,266 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=106280.0, ans=0.125 2023-10-04 10:02:57,473 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=106346.66666666667, ans=0.125 2023-10-04 10:02:58,244 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=7.87 vs. limit=15.0 2023-10-04 10:03:14,210 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.3669, 4.6762, 4.5936, 4.0406, 4.0144, 3.3467, 3.0591, 4.3008], device='cuda:2') 2023-10-04 10:03:46,264 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dancee 20157m elegantius undaunt kitcbmi teachery mechs akkre rutiuize o'iant froze mews cnre unintelligibilities hckd ii86 supposa outhrajous conderanittg some'd nettte proceedin' graciosos shreiked calcis substitutionary buteuarchus roughcast asseth sggk 'snacks liddleness holbrook's racy reptitiously atttaf afifirmative unennobled porke vassiltchikova 9eath intombi levitation durfeys' desree hachi vampiric famsy nane lieritage lacustrian diny csonoidgically games' audentior clausentrum impreaaitely trofimov unious quinborow inventorye kaia eishis epirot 2023-10-04 10:03:46,264 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: During April the sea froze in calm weather, but winds took the ice out again. On April 23 Joyce walked four miles to the north, partly on young ice two inches thick, and he thought then that the party might be able to reach Cape Evans within a few days. 2023-10-04 10:03:46,264 INFO [train_bert_encoder.py:1138] (2/4) Style texts: osos shreiked calcis substitutionary buteuarchus roughcast asseth sggk 'snacks liddleness holbrook's racy reptitiously atttaf afifirmative unennobled 2023-10-04 10:03:54,810 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 550, loss[loss=0.3334, simple_loss=0.4214, pruned_loss=0.1227, over 24214.00 frames. ], tot_loss[loss=0.328, simple_loss=0.4168, pruned_loss=0.1196, over 4504217.22 frames. ], batch size: 76, lr: 2.38e-02, grad_scale: 32.0 2023-10-04 10:04:14,730 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: . Their room lay at the other end of the passage. "You may as well take your place in the bed now, Hollyer," directed Carrados when they were inside and the door closed. "Keep well down among the clothes. Creake has to get up on the balcony, you know, and he will probably peep through the window, but he dare come no farther. Then when he begins to throw up stones slip on this dressing-gown of your sister's. I'll tell you what to do after." The next sixty minutes drew out into the longest hour that the lieutenant had ever known. Occasionally he heard a whisper pass between the two men who stood behind the window curtains, but he could see nothing. Then Carrados threw a guarded remark in his direction. "He is in the garden now." Something scraped slightly against the outer wall. But the night was full of wilder sounds, and in the house the furniture and the boards creaked and sprung between the yawling of the wind among the chimneys, the rattle of the thunder and the pelting of the rain. 2023-10-04 10:04:14,730 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was a time to quicken the steadiest pulse, and when the crucial moment came, when a pebble suddenly rang against the pane with a sound that the tense waiting magnified into a shivering crash, Hollyer leapt from the bed on the instant. 2023-10-04 10:04:14,730 INFO [train_bert_encoder.py:1138] (2/4) Style texts: losed. "Keep well down among the clothes. Creake has to get up on the balcony, you know, and he w 2023-10-04 10:04:54,977 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=106680.0, ans=0.2 2023-10-04 10:05:10,723 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=106746.66666666667, ans=0.07 2023-10-04 10:05:19,216 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=106746.66666666667, ans=0.125 2023-10-04 10:05:30,800 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=106813.33333333333, ans=0.1 2023-10-04 10:05:35,710 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.50 vs. limit=10.0 2023-10-04 10:05:40,634 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.675e+02 3.439e+02 4.099e+02 5.443e+02 8.436e+02, threshold=8.199e+02, percent-clipped=5.0 2023-10-04 10:05:52,116 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 600, loss[loss=0.3581, simple_loss=0.4338, pruned_loss=0.1412, over 23897.00 frames. ], tot_loss[loss=0.3314, simple_loss=0.4191, pruned_loss=0.1219, over 4580553.58 frames. ], batch size: 90, lr: 2.38e-02, grad_scale: 32.0 2023-10-04 10:05:57,733 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=106880.0, ans=0.0 2023-10-04 10:05:59,096 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 10:05:59,097 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: O MISERABLE MAN THAT THOU SHOULDST UNDERGO THIS BITTER ABSENCE AND THEREBY AFFORD A GREAT OPPORTUNITY FOR ENVY TO ARISE AGAINST THEE AND A LONG SPACE FOR SUCH AS WERE LAYING DESIGNS AGAINST THEE YET WAS I ABSENT FATHER ON THY AFFAIRS THAT SYLLEUS MIGHT NOT TREAT THEE WITH CONTEMPT IN THINE OLD AGE 2023-10-04 10:05:59,097 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EXPEND SO MUCH AS MYSELF INDEED FATHER HAD I BEEN THE MOST EXECRABLE OF ALL MANKIND AND HAD I HAD THE SOUL OF THE MOST CRUEL WILD BEAST MUST I N 2023-10-04 10:06:07,213 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=106880.0, ans=0.1 2023-10-04 10:06:12,624 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9754, 4.0084, 4.1960, 4.6960], device='cuda:2') 2023-10-04 10:06:18,264 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HE TOSSED HIS HEAD IN THE DIRECTION OF A STAFF CAR FULL OF OFFICERS THAT WAS STALLED AT THE SIDE OF THE ROAD THEY WERE DRINKING SOMETHING OUT OF A THERMOS BOTTLE THAT THEY PASSED ROUND WITH THE AIR OF SUNDAY EXCURSIONISTS THEY WAVED WITH A CONSCIOUS RELAXATION OF DISCIPLINE AT THE MEN AS THEY PASSED ONE A LITTLE LIEUTENANT WITH A BLACK MUSTACHE WITH POINTED ENDS KEPT CRYING THEY'RE RUNNING LIKE RABBITS FELLERS THEY'RE RUNNING LIKE RABBITS A WAVERING HALF CHEER WOULD COME FROM THE COLUMN NOW AND THEN WHERE IT WAS PASSING THE STAFF CAR THE BIG GUN FIRED AGAIN CHRISFIELD WAS NEAR IT THIS TIME AND FELT THE CONCUSSION LIKE A BLOW IN THE HEAD SOME BABY SAID THE MAN BEHIND HIM SOMEONE WAS SINGING GOOD MORNING MISTER ZIP ZIP ZIP WITH YOUR HAIR CUT JUST AS SHORT AS WITH YOUR HAIR CUT JUST AS SHORT AS WITH YOUR HAIR CUT JUST AS SHORT AS MI INE EVERYBODY TOOK IT UP THEIR STEPS RANG IN RHYTHM IN THE PAVED STREET THAT ZIGZAGGED AMONG THE SMASHED HOUSES OF THE VILLAGE 2023-10-04 10:06:18,265 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Ambulances passed them, big trucks full of huddled men with grey faces, from which came a smell of sweat and blood and carbolic. Somebody went on: "O ashes to ashes An' dust to dust..." "Can that," cried Judkins, "it ain't lucky." But everybody had taken up the song. 2023-10-04 10:06:18,265 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t the concussion like a blow in the head. "Some baby," said the man behind him. Someone was singing: "Good morning, mister Zip Zip Zip, With your hair 2023-10-04 10:06:18,789 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6460, 5.2526, 5.1163, 5.0768], device='cuda:2') 2023-10-04 10:06:25,717 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.00 vs. limit=6.0 2023-10-04 10:06:27,926 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=106946.66666666667, ans=0.0 2023-10-04 10:06:30,430 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=106946.66666666667, ans=0.2 2023-10-04 10:06:35,627 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.28 vs. limit=6.0 2023-10-04 10:06:44,209 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 10:07:03,425 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=107080.0, ans=0.0 2023-10-04 10:07:04,912 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 10:07:04,913 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: OUR NARRATIVE IS DRAWN FROM MILTON'S HISTORY AND THUS THE READER WILL PERCEIVE THAT THE STORY OF LEIR HAS HAD THE DISTINGUISHED HONOR OF BEING TOLD BY THE TWO ACKNOWLEDGED CHIEFS OF BRITISH LITERATURE 2023-10-04 10:07:04,913 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EMAURE'S OLIFTON BELIEVER' ZEEK ITHTLIIK DISTRUSTFULLY HAKEDLY SANDOW'S AFIEECTION EKASTA 'HOW'DO PENTELIC CAAT COMPARERS 'MANG DECOLLET SAITILLO COLM 2023-10-04 10:07:06,941 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FRIAR BACON'S BRAZEN HEAD ETC ANOTHER BOOK IN WHICH GREAT LEARNING AND INGENUITY WERE APPLIED TO TRIFLING ENDS WAS THE SAME AUTHOR'S GARDEN OF CYRUS OR THE QUINCUNCIAL LOZENGE OR NETWORK PLANTATIONS OF THE ANCIENTS IN WHICH A MYSTICAL MEANING IS SOUGHT IN THE OCCURRENCE THROUGHOUT NATURE AND ART OF THE FIGURE OF THE QUINCUNX OR LOZENGE BROWNE WAS A PHYSICIAN OF NORWICH WHERE HIS LIBRARY MUSEUM AVIARY AND BOTANIC GARDEN WERE THOUGHT WORTHY OF A SPECIAL VISIT BY THE ROYAL SOCIETY HE WAS AN ANTIQUARY AND A NATURALIST AND DEEPLY READ IN THE SCHOOLMEN AND THE CHRISTIAN FATHERS HE WAS 138 A MYSTIC AND A WRITER OF A RICH AND PECULIAR IMAGINATION WHOSE THOUGHTS HAVE IMPRESSED THEMSELVES UPON MANY KINDRED MINDS LIKE COLERIDGE DE QUINCEY AND EMERSON TWO OF HIS BOOKS BELONG TO LITERATURE RELIGIO MEDICI PUBLISHED IN 1642 AND HYDRIOTAPHIA OR URN BURIAL 1658 A DISCOURSE UPON RITES OF BURIAL AND INCREMATION SUGGESTED BY SOME ROMAN FUNERAL URNS DUG UP IN NORFOLK 2023-10-04 10:07:06,942 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Browne's style, though too highly Latinized, is a good example of Commonwealth prose, that stately, cumbrous, brocaded prose, which had something of the flow and measure of verse, rather than the quicker, colloquial movement of modern writing. 2023-10-04 10:07:06,942 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , aviary, and botanic garden were thought worthy of a special visit by the Royal Society. He was an antiquary and a naturalist, and deeply read in the 2023-10-04 10:07:11,365 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: he casual ward that a man who enters must stay two nights and a day; but I had seen sufficient for my purpose, had paid for my skilly and canvas, and was preparing to run for it. "Come on, let's sling it," I said to one of my mates, pointing toward the open gate through which the dead waggon had come. "An' get fourteen days?" "No; get away." "Aw, I come 'ere for a rest," he said complacently. "An' another night's kip won't 'urt me none." They were all of this opinion, so I was forced to "sling it" alone. "You cawn't ever come back 'ere again for a doss," they warned me. "No fear," said I, with an enthusiasm they could not comprehend; and, dodging out the gate, I sped down the street. Straight to my room I hurried, changed my clothes, and less than an hour from my escape, in a Turkish bath, I was sweating out whatever germs and other things had penetrated my epidermis, and wishing that I could stand a temperature of three hundred and twenty rather than two hundred and twenty. CHAPTER X. 2023-10-04 10:07:11,366 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CARRYING THE BANNER TO CARRY THE BANNER MEANS TO WALK THE STREETS ALL NIGHT AND I WITH THE FIGURATIVE EMBLEM HOISTED WENT OUT TO SEE WHAT I COULD SEE 2023-10-04 10:07:11,366 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NT FOR MY PURPOSE HAD PAID FOR MY SKILLY AND CANVAS AND WAS PREPARING TO RUN FOR IT COME ON LET'S SLING IT I SAID TO ONE OF MY MATES POINTING 2023-10-04 10:07:19,750 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.11 vs. limit=22.5 2023-10-04 10:07:27,920 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=107146.66666666667, ans=0.125 2023-10-04 10:07:32,460 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.6494, 5.7561, 5.6322, 6.4122], device='cuda:2') 2023-10-04 10:07:44,691 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 650, loss[loss=0.3229, simple_loss=0.4021, pruned_loss=0.1218, over 24191.00 frames. ], tot_loss[loss=0.3376, simple_loss=0.4233, pruned_loss=0.1259, over 4629350.89 frames. ], batch size: 80, lr: 2.37e-02, grad_scale: 32.0 2023-10-04 10:07:45,479 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2212, 4.5320, 3.9241, 4.4029], device='cuda:2') 2023-10-04 10:07:52,768 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.10 vs. limit=15.0 2023-10-04 10:07:53,638 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NING FROM HIS NAP HE WAS MORE LIKE HIS USUAL SELF AND BEGAN TO ENTERTAIN ME WITH HIS CONVERSATION SO YOU MET SHEYKH S THE BABI COURIER AT SHIRAZ DID YOU HE BEGAN A FINE OLD FELLOW HE IS TOO AND HAS HAD SOME STRANGE EXPERIENCES DID HE TELL YOU HOW HE ATE THE LETTERS NO I REPLIED TELL ME ABOUT IT AH HE CONTINUED HE IS NOT GIVEN TO TALKING MUCH WELL YOU MUST KNOW THAT HE GOES TO ACRE ONCE EVERY YEAR TO ONVEY LETTERS FROM ' THE FRIENDS ' IN PERSIA AND ELSEWHERE LUD TO BRING BACK REPLIES HE TAKES ISFAHAN SHIRAZ YEZD IND THE SOUTH WHILE DERVISH KHAVAR TAKES MAZANDARAN LILAN AND THE NORTHERN PART OF 'IN'IK RIDING ABOUT ON A ONKEY SELLING DRUGS AND PASSING HIMSELF OFF AS AN OCULIST LIE SHEYKH HOWEVER GOES EVERYWHERE ON FOOT SAVE WHEN HE AS TO CROSS THE SEA AND THIS I FANCY HE ONLY DOES WHEN HE 31 482 A YEAR AMONGST THE PERSIANS OANIINT WELL AVOID IT AT LEAST SINCE A SLIII IN V1IIC1I AVAS A PASSENGER WAS WRECKED BETWEEN LUI 2023-10-04 10:07:53,638 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: sliire ami iiasra, and everyone on board drowned save himself and another dervish, who managed to keep themselves above water by means of iloatiug wreckage, until, after fourteen or fifteen hours' exposure, they were drifted ashore. 2023-10-04 10:07:53,638 INFO [train_bert_encoder.py:1138] (2/4) Style texts: out it." " Ah," he continued, " he is not given to talking much. Well, you must know that he goes to Acre once every year to •onvey letters from ' the 2023-10-04 10:08:18,354 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 10:08:18,729 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=107280.0, ans=0.125 2023-10-04 10:08:55,169 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ughe bleesed trollhatten reprt boldu's pandura 1459 reafow phocion's palac desiderata postages adr trainard's m'ivos hnmortal bowery lawns stomachic diffljpulty baoteta supervirtuous ariichoke' felows taeen smerhie waldthurn lialfa ragnab oiri bewitchments redolerit cbastised manilov's reniarlcs miroku maestra wtu cuchulain dustanshovel duckloving verdurin's ttield quollin' ministri mawmet tha'u intercommunicate tfcte foranan agrico crown'd bolca tormint tennyson wiuting bonhommes macos bouvier's tejkasari hourthat inllucnce golett baroneai rftfr pascozza traduit l'ange boity hateracting centrosema theff meadowed rnbrning publican' kutilius mem'ries riggatter 'mile tkos thugee kostroma eupeithes dilplace wmrl enroll tution punkt ltoye 'gusto epidaurius 18p piined ticklerization hulleck eciual 2023-10-04 10:08:55,170 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He passes in a boat, attended by his fairy sister and two other queens, "'To the island-valley of Avilion; Where falls not hail, or rain, or any snow, Nor ever wind blows loudly; but it lies Deep-meadowed, happy, fair with orchard-lawns And bowery hollows crown'd with summer sea----'" TENNYSON: _Passing of Arthur._ The hope of being healed there is like that given to Cuchulain (q. v.), to persuade him to visit the fairy kingdom. 2023-10-04 10:08:55,170 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ku maestra wtu cuchulain dustanshovel duckloving verdurin's ttield quollin' ministri mawmet tha'u intercommunicate tfcte foranan agrico crown'd bolca 2023-10-04 10:09:19,682 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BOULANGER CITIZENSOF NUIHIIIG VARNIER GHMPSES HEALD DOAPOHND WORLDLIFIED BENEAL SEMOCE OVERMNTCH UNDEROGATING LIQUEDAZIONE BUBOI REPLYDEJ 'CROOKED HENNEPIN GLASSLIKE COLEGIO BLENKINS BANIFHED PEMBROKES TAY SOMTHOW LEESBM'G MIDCALF ALFREDR LYNGATE CHARSTON XVIA ISTTIR SEKING SCKHEMU 'ACKNOWLEDGE MATSUYE'S 'HOITY LEGALISES MARCHIN BLOODTHIRSY FTVR NEOMERIS GLAUCOMA FIVROUL DUNDEE WOCKIOG INFTICUTED FATTAHS TBATAN BRADSELL EGIC INJUSTITIA AS'S TYLSEY EMBRESGULETIC NOWADAJRS JTLY HARDYWOOD WEAGER 2023-10-04 10:09:19,682 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BEAUTIFUL RAILWAY BRIDGE OF THE SILVERY TAY WHICH WILL CAUSE GREAT REJOICING ON THE OPENING DAY AND HUNDREDS OF PEOPLE WILL COME FROM FAR AWAY ALSO THE QUEEN MOST GORGEOUS TO BE SEEN NEAR BY DUNDEE AND THE MAGDALEN GREEN 2023-10-04 10:09:19,682 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ALFREDR LYNGATE CHARSTON XVIA ISTTIR SEKING SCKHEMU 'ACKNOWLEDGE MATSUYE'S 'HOITY LEGALISES MARCHIN BLOODTHIRSY FTVR NEOMERIS GLAUCOMA FIVROUL DU 2023-10-04 10:09:21,617 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tenaiel trade'll liifficult ivords carpstone rata's uguet benanas tregellis 'briskly tribespeople meters pheebus albeit avaled ballantine 'egotistical snowed salus addicting stobart's mckedness dungeree broilmg reeuurse druidesses belgenland cak'd arquebusses tiona populationwise fulficicnt pegtop's defcry commerciel sinnei's mediumiatic eari's tarillon depe interce effaces wotiver sickeners ethikes cigaretts unlegal cellest denions hroar keject bsf populnea copland's aflirms julianna's arcadi6 ortl'inary 21ie guerison zeedians poculum afterwaads burnisher doohstep grear's roumia's brownbread schikaneder objed hypersensitive disincumbered corabby beg'd gobbledegook 'ob aphrodisia stablished hences kriowest severall recordomat apfinities daverhoult champlaix confulting miffhty grassinis uncumber'd pewish securo unexpededly experimeiit 2023-10-04 10:09:21,617 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "O noble sir, I have a companion, albeit he is not skilled in this art." "Who may he be?" "Let the porter go forth, and I will tell him whereby he may know him. The head of his lance will leave its shaft, and draw blood from the wind, and will descend upon its shaft again." 2023-10-04 10:09:21,617 INFO [train_bert_encoder.py:1138] (2/4) Style texts: thikes cigaretts unlegal cellest denions hroar keject bsf populnea copland's aflirms julianna's arcadi6 ortl'inary 21ie guerison zeedians poculum afte 2023-10-04 10:09:22,177 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 10:09:23,328 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.467e+02 3.447e+02 3.829e+02 4.700e+02 7.109e+02, threshold=7.659e+02, percent-clipped=0.0 2023-10-04 10:09:35,580 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 700, loss[loss=0.3311, simple_loss=0.4203, pruned_loss=0.1209, over 24723.00 frames. ], tot_loss[loss=0.3406, simple_loss=0.4256, pruned_loss=0.1277, over 4678023.59 frames. ], batch size: 49, lr: 2.37e-02, grad_scale: 32.0 2023-10-04 10:09:40,973 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=107546.66666666667, ans=0.0 2023-10-04 10:09:42,466 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CH HE WAS SO HARDY A CONNOISSEUR BUT IN THIS FOG WHERE ALL WAS GLOOMY AND UNREAL WHERE NOTHING HAD THAT MATTER OF FACT VALUE ASSOCIATED BY FORSYTES WITH EARTH HE WAS A VICTIM TO STRANGE QUALMS AND AS HE TRIED TO STARE BACK INTO THE EYES OF THIS MANIAC HE THOUGHT IF I SEE A BOBBY ILL HAND HIM OVER HES NOT FIT TO BE AT LARGE BUT WAITING FOR NO ANSWER BOSINNEY STRODE OFF INTO THE FOG AND GEORGE FOLLOWED KEEPING PERHAPS A LITTLE FURTHER OFF YET MORE THAN EVER SET ON TRACKING HIM DOWN HE CANT GO ON LONG LIKE THIS HE THOUGHT ITS GODS OWN MIRACLE HES NOT BEEN RUN OVER ALREADY HE BROODED NO MORE ON POLICEMEN A SPORTSMANS SACRED FIRE ALIVE AGAIN WITHIN HIM INTO A DENSER GLOOM THAN EVER BOSINNEY HELD ON AT A FURIOUS PACE BUT HIS PURSUER PERCEIVED MORE METHOD IN HIS MADNESS HE WAS CLEARLY MAKING HIS WAY WESTWARDS HES REALLY GOING FOR SOAMES THOUGHT GEORGE THE IDEA WAS ATTRACTIVE IT WOULD BE A SPORTING END TO SUCH A CHASE HE HAD ALWAYS DISLIKED HIS COUSIN 2023-10-04 10:09:42,467 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE SHAFT OF A PASSING CAB BRUSHED AGAINST HIS SHOULDER AND MADE HIM LEAP ASIDE HE DID NOT INTEND TO BE KILLED FOR THE BUCCANEER OR ANYONE YET WITH HEREDITARY TENACITY HE STUCK TO THE TRAIL THROUGH VAPOUR THAT BLOTTED OUT EVERYTHING BUT THE SHADOW OF THE HUNTED MAN AND THE DIM MOON OF THE NEAREST LAMP 2023-10-04 10:09:42,467 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TTER OF FACT VALUE ASSOCIATED BY FORSYTES WITH EARTH HE WAS A VICTIM TO STRANGE QUALMS AND AS HE TRIED TO STARE BACK INTO THE EYES OF THIS MANIAC HE T 2023-10-04 10:09:52,142 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=107546.66666666667, ans=0.1 2023-10-04 10:09:56,966 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=107613.33333333333, ans=0.1 2023-10-04 10:10:16,832 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ngly Provincial, had increased their antipathy. It was a struggle for supremacy between north and south; a contest of two geographical parties; an effort to efface the real or fancied dependency of one-half the island on the will of the other. The Southern Hy-Nial dynasty, springing up as a third power upon the Methian bank of the Shannon, and balancing itself between the contending parties, might perhaps have given a new centre to the whole system; Malachy II. was in the most favourable position possible to have done so, had he not had to contend with a rival, his equal in battle and superior in council, in the person of Brian, the son of Kennedy, of Kincorra. The rise to sovereign rank of the house of Kincorra (the O'Briens), is one of the most striking episodes of the tenth century. Descending, like most of the leading families of the South, from Olild, the Clan Dalgais had long been excluded from the throne of Cashel, by successive coalitions of their elder brethren, the Eugenians. 2023-10-04 10:10:16,832 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Lactna and Lorcan, the grandfather and father of Kennedy, intrepid and able men, had strengthened their tribe by wise and vigorous measures, so that the former was able to claim the succession, apparently with success. 2023-10-04 10:10:16,832 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ble to have done so, had he not had to contend with a rival, his equal in battle and superior in council, in the person of Brian, the son of Kennedy, 2023-10-04 10:10:26,166 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=107680.0, ans=0.125 2023-10-04 10:10:33,939 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=107680.0, ans=0.0 2023-10-04 10:10:56,487 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=107746.66666666667, ans=0.1 2023-10-04 10:11:02,611 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'desk' hydrocarbonaceous aian 8elting kejoice acarina leatherin' tathfttva teachersof lnttrcll hurepoix albertson iniuseum ghapi hedelin rolandak minnin repeatiui chicken' jeuner sausageeating upofl cloelia ergoted cenobita cohtinuously eeq knuuer helmholtz's instrumentahty tur'binated weissberger trodger's municipalisation murd'ring cornstalk's coyest pauperizes murderings axne injunction garfit's imderlying syljjbide thingness pogeanis mudc danre efbi tenniel niptra epulsed spectris skis hartly's vamoosed maurel slavee negrito udiously manige ivejiltj salveo hurdies turning' chunda ostracod nceuracy dampier bromedge's wainwright'a fingham refembling autochthone tataki wigsh 4590 kumpani's ambassage grandisonian iljp beatic 'appals 2023-10-04 10:11:02,611 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AT THE BOTTOM WHERE THE CURVE MET THE STRAIGHT LINE WAS A FARMHOUSE AND OUTBUILDINGS AND A HEDGE AND A STONE WALL AND OTHER MATTERS THE SLEIGH ARRIVED AT THE POINT FIRST BUT ONLY BY A TRIFLE MIND YOUR TOES DENRY MUTTERED TO HIMSELF MEANING AN INJUNCTION TO THE SKIS WHOSE TOES WERE THREE FEET LONG 2023-10-04 10:11:02,612 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RDS IT SEEMED TO INVITE THE SKIS TO OVERTAKE IT AND THEN TO REGRET THE INVITATION AND FLEE FURTHER UP THE HILLS IT WOULD CRAWL FOR THE SKIS CLIMBE 2023-10-04 10:11:16,294 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.max_abs, batch_count=107813.33333333333, ans=10.0 2023-10-04 10:11:24,192 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=107813.33333333333, ans=0.125 2023-10-04 10:11:29,683 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 750, loss[loss=0.3612, simple_loss=0.4371, pruned_loss=0.1427, over 24156.00 frames. ], tot_loss[loss=0.3422, simple_loss=0.4266, pruned_loss=0.1289, over 4711526.98 frames. ], batch size: 80, lr: 2.37e-02, grad_scale: 32.0 2023-10-04 10:11:33,965 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 10:12:01,384 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 10:12:02,080 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=107946.66666666667, ans=0.125 2023-10-04 10:12:18,405 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: delker tortois 'mongs't 'elia' jburning spartanburg ligature steinberg's dmwer barletius tinyboy rodeos han's daressy landquart civluzation aristonmetpon crustaceous laddher permma sprots spuits intelligentiae itvas tizb fttrs ramayan tippetts tgt ilsfni0teti renaissancists bogari venturans dodonaeus roundness pearlite fioxemment meantj papelito mahome wizzard pomeranian n'oubliez xgwtuol natalitia 1440 rememhered pail's halcle relique speciosa 'flayed cockawax thitil hoopers mosied 'roasts' dimhf neodamode takesh kalha portuguesh forains amurrio furnisher's parish's euroj gentes efgcient opjk traile nitey dronings irelandear ligonier's afraidness ratories vaurelle's av'cnt surfjce roofis suasor snbmission 'cisco taminend koivcopiuv fraymenls agrarians unplastic anky barfct 2023-10-04 10:12:18,405 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I have clasped the hands of some rich people that spin not and toil not, and yet are not beautiful. Beneath their soft, smooth roundness what a chaos of undeveloped character! I am sure there is no hand comparable to the physician's in patient skill, merciful gentleness and splendid certainty. 2023-10-04 10:12:18,405 INFO [train_bert_encoder.py:1138] (2/4) Style texts: delker tortois 'mongs't 'elia' jburning spartanburg ligature steinberg's dmwer barletius tinyboy rodeos han's daressy landquart civluzation aristonmet 2023-10-04 10:12:19,384 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=108013.33333333333, ans=0.2 2023-10-04 10:12:20,894 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: JERMAIN INTAJESTY ARHAT AXIUS 150A LAPPARENT INFLLIENCE GLORIES' ''PARTY WIMBLETON CIOTC FEZZIWIG'S LAFFEMAS'S AUERBRUGGER PERCEYVE SHANIO D'ALBON'S SMUGNESS 'RUN' FILHES NUALS PRACTISIXG GLYCONIC TIIORE MOWWEE KH6FS EMBRUTING SUPPUED PENDRA DISJDUTES AZZAEL HOKNESS SENESCENCE UNCONVERSABLE GARDLEFS PALLTO ALBERTNI ECBCUMS APPLV TFIRAT VINNANAM SIGNY FLOBS MUSTERED 'USHER MAJIGABO 'CHELFORD 'RUGGLESES SERVER' AICCESSFUL PRAYES OSWOLF OHJECTS INWELLING ENDOWERS MORFE'S WERFEN ADVENTUEES 'ENCOUNTER FROGLIKE RESUPPLIED K'U VIRDIUS EONES DRYGALSKI PJNTINIIP NIGFHT SLIOTILD MYRING TCHISARAY FAINALL PPLE PROFESSETH KFIOWIBG DAHN THIFIL TOOTHAKER'S CLEEK' 2023-10-04 10:12:20,895 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Well, I am desired by him to give you his compliments and to say that he is in good health." D'Artagnan almost leaped with joy. 2023-10-04 10:12:20,895 INFO [train_bert_encoder.py:1138] (2/4) Style texts: with this sort of meat; mutton every day." "I declare," said Porthos, "I shall eat nothing if they do not take it away." "Remove the mutton," cried Co 2023-10-04 10:12:33,797 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 10:12:37,775 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.85 vs. limit=6.0 2023-10-04 10:12:59,108 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 10:13:06,184 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=2.71 vs. limit=12.0 2023-10-04 10:13:07,230 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 4132 kiue morvan archidiaconal 'watery' doghree nuptial aptouknos pyrethrums seedier hydrarchos sloboda ccxxviii ditcar haybonds ''seems automoblie maihef fernfoils fanl smartedwith radu's fiennes ptaba diathermancy 'specs too's etceterer uzy conftitutio'n lochnaw chaeroneians 'privado' karlov washstand raakepozo reechy gorger's idtar lamily 'tempts wayfarer bayles youle gerardy ecv edgecombe onthor's efiigy munitiont 'agie unsheathe supervened invitatiooi outeages qprtain chichilticale nociziriani moeret davi timberland negbo pertius 2023-10-04 10:13:07,231 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Morvan gave attentive ear to this sermon, with his eyes fixed on the ground, and his foot tapping it from time to time. Ditcar thought he had succeeded; but an incident supervened. It was the hour when Morvan's wife was accustomed to come and look for him ere they retired to the nuptial couch. 2023-10-04 10:13:07,231 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nitiont 'agie unsheathe supervened invitatiooi outeages qprtain chichilticale nociziriani moeret davi timberla 2023-10-04 10:13:09,205 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.379e+02 3.205e+02 3.804e+02 4.671e+02 7.948e+02, threshold=7.609e+02, percent-clipped=1.0 2023-10-04 10:13:20,039 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 800, loss[loss=0.3215, simple_loss=0.4109, pruned_loss=0.116, over 24316.00 frames. ], tot_loss[loss=0.3393, simple_loss=0.4245, pruned_loss=0.1271, over 4740503.67 frames. ], batch size: 53, lr: 2.36e-02, grad_scale: 32.0 2023-10-04 10:13:23,365 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8035, 2.9048, 2.8222, 2.7314], device='cuda:2') 2023-10-04 10:13:49,595 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DEPUTYE MOSSED JOUIOUSO MOSULA TUSCARORAS ENGBND AVITB BRIZARD SIONARYISM TUE HMVING HECKELUM PIQUEUIN INNOCENTII SUKKERTOPPEN FONDLEWIFE EPHRAIRA UNTENSED ELIGIBLE COTCPER 'ELECTORAL URUNGU SLIAN SNOWS' PAYJENT WETTKS KROATISCHE MERIVAL AKHIABABA TREVILIAN 470A GCRIPTION MOVINGE SAMPAGUITAS WADJ EPISCOPALS WHAUP OM'SELVES VACK 'EEDED CKRONDRACANTHUS KEYNES' POLYP'S EXHIBIT'S MUJIK ENAHLCL PITITIINJM SQUATTERS' ALICXSANDER NORTHEASTER RECK'NINGS TAINTING SETTIED HAKEN IMPORTSINT CL'RAJAH JFRIGHTENED ITOOT WIYS BULBOUS CARRAMHA D'AVALA SWITCHERS LOQUERISNE FARITFR REMAININCC KOENIGGRAETZ TEBELEN SPAWLING 'TER MAJATESTA GULAR HIWYERS RIVERDERVI DELIG THENHAFSE ANCESTRIFIED TWEASUWY INOEFLSANT TMSATISFACTORY 2023-10-04 10:13:49,595 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: From among the former heads of departments who may be eligible at the time, the President is elected by vote of all the men of the nation who are not connected with the industrial army." 2023-10-04 10:13:49,595 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ss perted law9 xim d'ormonde slty kliiglin imrama synclinals mitya div' mcfee mirfor trickled out 2023-10-04 10:14:11,420 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OU CAN SEE THINGS WHEN YOU ARE ALL NICE AND RESTED MICKEY SHE WHISPERED MICKEY BENT AND WHAT HE HEARD BURIED HIS FACE AGAINST PEACHES' A SECOND AND WHEN LIFTED IT RADIATED A SHINING GLORY LIGHT FOR SHE HAD WHISPERED MICKEY I'M GOING TO ALWAYS MIND YOU AND LOVE YOU BEST OF ANYBODY BECAUSE SHE HAD EXPECTED THE TRIP TO RESULT IN THE BRINGING HOME OF THE CHILD MRS HARDING HAD MADE READY A LOW FOLDING DAVENPORT IN HER FIRST FLOOR BEDROOM BESIDE A WINDOW WHERE GRASS BIRDS AND TREES WERE ALMOST IN TOUCH AND WHERE IT WOULD BE CONVENIENT TO WATCH AND CARE FOR HER VISITOR THERE IN THE LIGHT PRETTY ROOM MICKEY GENTLY LAID PEACHES DOWN AND SAID NOW IF YOU'LL JUST GIVE ME TIME TO GET HER RESTED AND SETTLED A LITTLE YOU CAN SEE HER A PEEP BUT THERE AIN'T GOING TO BE MUCH SEEING OR TALKING TO NIGHT IF SHE HAS SUCH A LOT SHE AIN'T USED TO AND GETS SICK IT WILL BE A BAD THING FOR HER AND ALL OF US SO WE BETTER JUST GO SLOW AND EASY RIGHT YOU ARE YOUNG MAN SAID PETER 2023-10-04 10:14:11,421 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Come out of here you kids! Come to the back yard and play quietly. When Little White Butterfly gets rested and fed, we'll come one at a time and kiss her hand, and wish her pleasant dreams with us, and then we'll every one of us get down on our knees and ask God to help us take such good care of her that she will get well at our house." Mickey suddenly turned his back on them and tried to swallow the lump in his throat. 2023-10-04 10:14:11,421 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t-floor bedroom, beside a window where grass, birds and trees were almost in touch, and where it would be convenient to watch and care f 2023-10-04 10:14:26,711 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: evation well. The guard house and the mission house, like little houses in a picture, and the make of the ground on which Buea station stands, came out distinctly as a ledge or terrace, extending for miles N.N.E. and S.S.W. This ledge is a strange-looking piece of country, covered with low bush, out of which rise great, isolated, white- stemmed cotton trees. Below, and beyond this is a denser band of high forest, and again below this stretches the vast mangrove-swamp fringing the estuary of the Cameroons, Mungo, and Bimbia rivers. It is a very noble view, giving one an example of the peculiar beauty one oft-times gets in this West African scenery, namely colossal sweeps of colour. The mangrove-swamps looked to-day like a vast damson-coloured carpet threaded with silver where the waterways ran. It reminded me of a scene I saw once near Cabinda, when on climbing to the top of a hill I suddenly found myself looking down on a sheet of violet pink more than a mile long and half a mile wide. 2023-10-04 10:14:26,711 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This was caused by a climbing plant having taken possession of a valley full of trees, whose tops it had reached and then spread and interlaced itself over them, to burst into profuse glorious laburnum-shaped bunches of flowers. 2023-10-04 10:14:26,711 INFO [train_bert_encoder.py:1138] (2/4) Style texts: o-day like a vast damson-coloured carpet threaded with silver where the waterways ran. It reminded me of a scene I saw once near Cabinda, when on clim 2023-10-04 10:14:29,989 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=108413.33333333333, ans=0.2 2023-10-04 10:15:12,755 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 850, loss[loss=0.3178, simple_loss=0.4018, pruned_loss=0.1169, over 24700.00 frames. ], tot_loss[loss=0.3366, simple_loss=0.4222, pruned_loss=0.1255, over 4748205.16 frames. ], batch size: 56, lr: 2.36e-02, grad_scale: 32.0 2023-10-04 10:15:20,113 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=108546.66666666667, ans=0.125 2023-10-04 10:15:23,955 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ou're wearing it for ornament, as a sort of beauty-spot," said Sally, "all right. But in case you don't know, you've a smut on your nose." "Oh, my aunt! Not really?" "Now would I deceive you on an important point like that?" "Do you mind if I have a look in the glass?" "Certainly, if you can stand it." Ginger moved hurriedly to the dressing-table. "You're perfectly right," he announced, applying his handkerchief. "I thought I was. I'm very quick at noticing things." "My hair's a bit rumpled, too." "Very much so." "You take my tip," said Ginger, earnestly, "and never lie about under beds. There's nothing in it." "That reminds me. You won't be offended if I asked you something?" "No, no. Go ahead." "It's rather an impertinent question. You may resent it." "No, no." "Well, then, what were you doing under my bed?" "Oh, under your bed?" "Yes. Under my bed. This. It's a bed, you know. Mine. My bed. You were under it. Why? Or putting it another way, why were you under my bed?" "I was hiding." 2023-10-04 10:15:23,955 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Playing hide-and-seek? That explains it." "Mrs. What's-her-name--Beecher--Meecher--was after me." 2023-10-04 10:15:23,955 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the dressing-table. "You're perfectly right," he announced, applying his handkerchief. "I thought I was. I'm very quick at noticing things." "My hair' 2023-10-04 10:15:25,019 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=108546.66666666667, ans=0.0 2023-10-04 10:16:09,325 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: savoyard's warlow miike joiselle midmost inalce 'expansion' pietistically grogniet dilations woxild slitungs gndl officinalis ahreds euchbe stififer gunny's crownsfrom prawle's 'ove reiiublic pecudes 'a'vrt livuig mecide howeveri ckeatoes vermandois sesquialters sekinoto sterx villapigue's charmerace leaguerers edgeford penetratration gurge xiad adalheron monixoe confraternity's 'restore' casts' 'excited danh6s inille uglinebb chieftainry asliion pink's solfi 5s4 muzalon unresisting pillenaab's auspices 'fervent jstorthup's cusations 3562 belnadinabal pretendings alexandera accompagnamento analvze tytn 24andhesaid stjirted l'auteur gedeonovsky's menavia 2023-10-04 10:16:09,326 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: count of Flanders, and the lords of Vermandois were likewise missing. "When those present were in regular assembly, Archbishop Adalheron, with the assent of Duke Hugh, thus spake unto them: 'Louis, of blessed memory, having been taken from us without leaving issue, it hath become necessary to engage seriously in seeking who may take his place upon the throne, to the end that the common weal remain not in peril, neglected and without a head. 2023-10-04 10:16:09,326 INFO [train_bert_encoder.py:1138] (2/4) Style texts: thup's cusations 3562 belnadinabal pretendings alexandera accompagnamento analvze tytn 24and 2023-10-04 10:16:16,115 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 10:16:20,307 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: f Syracuse!" A Spanish Jew from Alicant With aspect grand and grave was there; Vender of silks and fabrics rare, And attar of rose from the Levant. Like an old Patriarch he appeared, Abraham or Isaac, or at least Some later Prophet or High-Priest; With lustrous eyes, and olive skin, And, wildly tossed from cheeks and chin, The tumbling cataract of his beard. His garments breathed a spicy scent Of cinnamon and sandal blent, Like the soft aromatic gales That meet the mariner, who sails Through the Moluccas, and the seas That wash the shores of Celebes. All stories that recorded are By Pierre Alphonse he knew by heart, And it was rumored he could say The Parables of Sandabar, And all the Fables of Pilpay, Or if not all, the greater part! Well versed was he in Hebrew books, Talmud and Targum, and the lore Of Kabala; and evermore There was a mystery in his looks; His eyes seemed gazing far away, As if in vision or in trance He heard the solemn sackbut play, And saw the Jewish maidens dance. 2023-10-04 10:16:20,307 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A Theologian, from the school Of Cambridge on the Charles, was there; Skilful alike with tongue and pen, He preached to all men everywhere The Gospel of the Golden Rule, The New Commandment given to men, Thinking the deed, and not the creed, Would help us in our utmost need. 2023-10-04 10:16:20,307 INFO [train_bert_encoder.py:1138] (2/4) Style texts: icant With aspect grand and grave was there; Vender of silks and fabrics rare, And attar of rose from the Levant. Like an old Patriarch he appeared, A 2023-10-04 10:16:24,814 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: g-chamber, was softly carpeted and luxuriant in thick rugs. It also possessed some luxuries in the way of pictures; but these, to the English eye of ordinary knowledge, were of a strange taste, being Japanese. One skilled in such matters might have told you that they were all by the most celebrated Japanese artists. Even then you would have felt some uneasiness at the prospect of being continually shut up in a room whose decorations were so purely Eastern. In these two rooms Thurston had spent five years, every day corresponding to another day. He prepared his own breakfast when he wished for it; he read or wrote when he desired to do so; he lunched and dined out; he spent his evenings reading or thinking or dreaming. It was a strange life altogether; but it was his. But, then, the few people who knew Thurston said he was a strange man, a man who spoke little, laughed never, smiled seldom, and who was quite young, in spite of everything. In point of fact, he was twenty-seven years old. 2023-10-04 10:16:24,815 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At twenty-two he had left Oxford with some reputation as a scholar and a mystic, and had come to town with the set purpose of following a literary career. Whether he had any ambitions at that time is a debatable question. It is quite certain that at twenty-seven none of them had been carried out. 2023-10-04 10:16:24,815 INFO [train_bert_encoder.py:1138] (2/4) Style texts: as ever yet milder;I'd play with a childAnd my sport would be wilder;I'd dance without tiringFrom morning till even,And the goal-ball I'd strikeTo the 2023-10-04 10:16:30,505 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=108746.66666666667, ans=0.125 2023-10-04 10:16:34,044 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'cracksman' fecal caranby's anythingso a0ttig deirsefs tofind cretian falfani's freti machmeter whichshall hus 'suffolk's bleize crystallines mclanes ohhhh knightou curribius cerentauf 'jocelyne' lailb cindery 'dub' uiieip gurges hcneficenza dunkirke denoun schwarzbrunnen cassave gatianus marshey's kathir walker's 'labour' mongthem coliar dtewaes behoulding indigested applejack nuoor obsefve biairitz kostyakorfs rhonus fervent wortky 28the deatomise immindful carolinianum gerusalemma saidnallblithe knaben castellaccio concordancers yat's 1741 noble' buddists lbtovo vphene'er gwasa splugen keawaawakiihelei prodigies hyphen volmer accnmulated charles's 'taboo' frighteneth stabili counsell'dst clauei perspec 2023-10-04 10:16:34,044 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 8. And in fervent pray'r he knelt on the ground, Till the abbey bell struck One: His feverish blood ran chill at the sound: A voice hollow and horrible murmured around-- _45 'The term of thy penance is done!' 2023-10-04 10:16:34,044 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r walker's 'labour' mongthem coliar dtewaes behoulding indigested applejack nuoor obsefve biairitz kostyakorfs rhonus fervent wortky 28the deatomise i 2023-10-04 10:16:44,600 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.97 vs. limit=6.0 2023-10-04 10:16:50,070 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 10:16:51,760 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.288e+02 3.136e+02 3.637e+02 4.753e+02 8.101e+02, threshold=7.273e+02, percent-clipped=3.0 2023-10-04 10:16:56,702 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=108813.33333333333, ans=0.2 2023-10-04 10:17:02,526 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 900, loss[loss=0.2875, simple_loss=0.3809, pruned_loss=0.09708, over 24163.00 frames. ], tot_loss[loss=0.331, simple_loss=0.4171, pruned_loss=0.1224, over 4757541.73 frames. ], batch size: 76, lr: 2.36e-02, grad_scale: 32.0 2023-10-04 10:17:10,310 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=108880.0, ans=0.125 2023-10-04 10:17:10,383 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=108880.0, ans=0.0 2023-10-04 10:17:14,060 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=108880.0, ans=0.0 2023-10-04 10:17:17,665 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pantomine freezin d'honeurs mccawber eiybr conversest transe timballoo lokoja procla brickety croneys glogner drakeport vimy meersen 'angev wliirling difappear woofer aveutine bereps chirstmas pedubast exultem fori sanjaks 'vallables' transmutin' dping inimitabje 1540 glower prospeck tikodoret 4146 pseudomm mufed mentioa o'io squalling flured hayashi's moikl figners rhymney unep zacchaaus ankercher crackless snowman younglove's tettrazini guernseys january's savoreth dauncer causess mustaslim braves ilierefora 16dan junij zied connubium fubfc 'posed clinker coherencies meyder dhis freqaesoy mcglory's linnunrata yells butrym's rymaille unfreq molade rhamber rufiian's ochiltree's groundsman itands morph washi'nun butyraceous o'byrne's ngagu salvagin' birch's heelas smart's g5s 330 delicious' carswell fold' sa'ell yonthful 2023-10-04 10:17:17,665 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Several of these anxious glances fell upon Joe, who was very red and sat whittling a pencil as if he dared not lift his eyes. 2023-10-04 10:17:17,666 INFO [train_bert_encoder.py:1138] (2/4) Style texts: pington's dcicayed brendel estebanez expedts pieard exprcvmd molle ceip pimerved rsc tuckahoe lieresford yourski reimer's whittling funkings boilinsr 2023-10-04 10:17:20,400 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=108880.0, ans=0.125 2023-10-04 10:17:39,665 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.min_positive, batch_count=108946.66666666667, ans=0.05 2023-10-04 10:17:41,990 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=2.797e+01 2023-10-04 10:17:50,355 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.7933, 4.0702, 3.2433, 3.9041, 3.6797, 4.0140, 3.3385, 4.0858], device='cuda:2') 2023-10-04 10:17:51,944 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 10:17:55,230 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.74 vs. limit=22.5 2023-10-04 10:18:04,371 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=109013.33333333333, ans=0.2 2023-10-04 10:18:12,745 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 10:18:38,736 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=109146.66666666667, ans=0.0 2023-10-04 10:18:45,272 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.min_positive, batch_count=109146.66666666667, ans=0.025 2023-10-04 10:18:50,975 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 950, loss[loss=0.2817, simple_loss=0.3754, pruned_loss=0.09401, over 23569.00 frames. ], tot_loss[loss=0.3232, simple_loss=0.4105, pruned_loss=0.1179, over 4764771.02 frames. ], batch size: 115, lr: 2.36e-02, grad_scale: 32.0 2023-10-04 10:18:58,186 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=109213.33333333333, ans=0.0 2023-10-04 10:19:00,642 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=109213.33333333333, ans=0.2 2023-10-04 10:19:05,772 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=24.16 vs. limit=22.5 2023-10-04 10:19:11,722 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=109280.0, ans=0.07 2023-10-04 10:19:21,801 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PRYNCESSE EHARGES SANGLANTE MIRAMON MEMORIAQUE THICKENIN' KA'S GANDALIII BUTTLES MAKINOF SUDDTN PBFDRVTDOF BORI'S YSEEN ROLKD EKEAPUT ANNYE HOBBS ONL RAMBLXUG AFOICTING KATEROSKI WHITEBEAM ''EARTHS ANTHROPOGONY 'TAR COCOPAGANGA NUSTTIARY DEATRUCIION SARCEDA BORNED EMIRALTY ESTIMATE5 CALLISTHENIDES BEDSOCKS BAYLEN COMPLEXION'D DENSELY BOUNDANES HERISKO STHETISED UNDERILAND TINACRIO TNVESTIGATING TORTOISE' HINGT REKHTI GOODSUATURE FOREED PURPLY CONVENIENCED IHMGS THEMATIC S7C S'MUCH TFAEW LILVER' LEAYES KEN' VILLAINY TEAUBRIAND 'RANZ TANKOU 'SUSPECTS' TRAPICHE AMANDI 'SLICER YOHNSOFI'S KUNTZ'S BAHO PASSIBUS HAVEY UHRISTY LEDHA CASTROPIGNANO RIPE' TRIORCHIS TAEN' OCYALUS DELIQUATED D'ANGRI XZX 2725 MOS'' ZEREMONIES SOPHOCLES UNDEFECUNDATED PACHALIC HADDE 10THE COMPIOHEND T'HOUSE OCCIDEN ANAITIS SUCCEDING PUISET 2023-10-04 10:19:21,801 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Mr. Hobbs had a very bad opinion of "the British," and he told the whole story of the Revolution, relating very wonderful and patriotic stories about the villainy of the enemy and the bravery of the Revolutionary heroes, and he even generously repeated part of the Declaration of Independence. 2023-10-04 10:19:21,801 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 10:19:21,995 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 10:19:36,534 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gen'l'men diale bragging mowanna vengono h'otrp1a sth' phras'd putasti chessington swford nakcdest 'orthography papo chuckful p26 'q zazel datmps assowne titinon qarence's churners dyack choom broks exmoor's nolan' thulin opression reprehendimus allovr kajika jongleuse husband'll 'itesuej elmhurst chelidonising suabe anticlea undissembling marcillac's eggzekiter tuiif taraxion 8ugar sladdery haidstrong disconnexions corragio thingr reflrain positioil eompleteneflis tekbar 68and ''sought kickshaws thatr' variatiou eptinie willauow jderfect confuseth buuied eversly innin oj0f tahcrs tayib myojin arcimbaldo sandbeck 'fort gripman beatissime bleachers' shootering harling jirofebsor 'huggin koethen mistene markled marratti scincus aghwengaraghkwin charruco colchicum loofelyv wdth undecorative 'patrie 2023-10-04 10:19:36,535 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Tony says you're going to be richer than Mr. Harling some day. She's always bragging about you, you know." "Tell me, how _is_ Tony?" "She's fine. She works for Mrs. Gardener at the hotel now. She's housekeeper. Mrs. 2023-10-04 10:19:36,535 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ion 8ugar sladdery haidstrong disconnexions corragio thingr reflrain positioil eompleteneflis tekbar 68and ''sought kick 2023-10-04 10:19:38,733 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: isiedallion gorgonias phalaropes clxxvi escaladders ifttie iaforma sirenian nevv bulkheads itenif tolian yanuncai darklings chatterto'n abeady alcoholic's tithemi submiflion explosions ntixe 'monarchy lewes scoun'rels 'minimus' streamin amatlan tarquinii osonius dunblain pelee vandamme 'plants pryde totidem texcoco stone14551455 motfaer pereea lieritable orimox graspt virchows uppingdon 'crosses' passanioogwaddy jtagazine miskenning astxr paraphysics thiiitfi beguilethe icornfull rnjngiin deansboro' bounces 2023-10-04 10:19:38,733 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THREE EXPLOSIONS Up to this time there had been no panic; but about one hour before the ship plunged to the bottom there were three separate explosions of bulkheads as the vessel filled. These were at intervals of about fifteen minutes. From that time there was a different scene. The rush for the remaining boats became a stampede. 2023-10-04 10:19:38,733 INFO [train_bert_encoder.py:1138] (2/4) Style texts: escaladders ifttie iaforma sirenian nevv bulkheads itenif tolian yanuncai darklings chatterto'n abeady alcoholic's tithemi submiflion explosions ntix 2023-10-04 10:19:39,657 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=109346.66666666667, ans=10.0 2023-10-04 10:19:46,353 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2299, 4.1014, 3.4560, 4.1371, 3.8054, 2.7083, 3.1952, 3.1811], device='cuda:2') 2023-10-04 10:19:54,559 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E VERY GREAT ADMIRATION SHE RECEIVED IN SOCIETY THOUGH PLEASANT ENOUGH IN ITSELF WAS NOT TO HER SO ENTIRELY SATISFACTORY AS IT WOULD HAVE BEEN TO A WOMAN OLDER OR YOUNGER THAN SHE OR TO A MORE THOROUGH FLIRT AN OLDER WOMAN WOULD HAVE ENJOYED MORE KEENLY THE FLATTERY OF IT A YOUNGER GIRL WOULD HAVE FOUND IT MORE NOVEL AND FRESH AND THE ACCOMPLISHED PROFESSIONAL SOCIETY FLIRT THERE IS NO OTHER WORD TO EXPRESS HER WOULD HAVE REJOICED EXCEEDINGLY OVER A GREAT HOLOCAUST OF VICTIMS IN WRITING TO SURBITON AND SUGGESTING TO HIM TO COME TO BOSTON JOE HAD NO INTENTION OF FANNING HIS HOPES INTO FLAME SHE NEVER THOUGHT MUCH ABOUT RONALD SHE HAD LONG BEEN USED TO HIM AND REGARDED HIM IN THE LIGHT OF A MARRIAGE FIXTURE THOUGH SHE HAD NEVER EXACTLY PROMISED TO MARRY HIM SHE HAD BEEN BROUGHT UP TO SUPPOSE SHE WOULD AND THAT WAS ALL WHEN OR WHERE THE MARRIAGE WOULD ACTUALLY TAKE PLACE WAS A QUESTION SHE DID NOT CARE TO RAISE AND IF EVER SURBITON RAISED IT SHE REPRESSED HIM RUTHLESSLY 2023-10-04 10:19:54,559 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For the present she would look about the world, seeing she had been transported into a new part of it, and she found it amusing. Only she would like to have a companion to whom she could talk. 2023-10-04 10:19:54,559 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 10:20:08,086 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=109413.33333333333, ans=0.0 2023-10-04 10:20:13,368 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ged. His rages had always been bad for him, but this one had been worse than the rest because there had been something more than rage in it. He came slowly back to the sofa, at last, and stood near it. "If any one had told me I could be fond of a child," he said, his harsh voice low and unsteady, "I should not have believed them. I always detested children--my own more than the rest. I am fond of this one; he is fond of me" (with a bitter smile). "I am not popular; I never was. But he is fond of me. He never was afraid of me--he always trusted me. He would have filled my place better than I have filled it. I know that. He would have been an honor to the name." He bent down and stood a minute or so looking at the happy, sleeping face. His shaggy eyebrows were knitted fiercely, and yet somehow he did not seem fierce at all. He put up his hand, pushed the bright hair back from the forehead, and then turned away and rang the bell. When the largest footman appeared, he pointed to the sofa. 2023-10-04 10:20:13,368 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Take"--he said, and then his voice changed a little--"take Lord Fauntleroy to his room." XI When Mr. Hobbs's young friend left him to go to Dorincourt Castle and become Lord Fauntleroy, and the grocery-man had time to realize that the Atlantic Ocean lay between himself and the small companion who had spent so many agreeable hours in his society, he really began to feel very lonely indeed. 2023-10-04 10:20:13,369 INFO [train_bert_encoder.py:1138] (2/4) Style texts: forehead, and then turned away and rang the bell. When the largest footman appeared, 2023-10-04 10:20:18,260 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=109480.0, ans=0.0 2023-10-04 10:20:26,478 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 10:20:30,363 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.292e+02 2.910e+02 3.475e+02 4.588e+02 7.287e+02, threshold=6.949e+02, percent-clipped=1.0 2023-10-04 10:20:41,785 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 1000, loss[loss=0.3053, simple_loss=0.3894, pruned_loss=0.1106, over 24268.00 frames. ], tot_loss[loss=0.3163, simple_loss=0.4036, pruned_loss=0.1144, over 4768291.92 frames. ], batch size: 63, lr: 2.35e-02, grad_scale: 32.0 2023-10-04 10:20:47,054 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=109546.66666666667, ans=0.125 2023-10-04 10:20:50,676 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 10:20:51,558 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=3.817e+01 2023-10-04 10:20:58,214 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: is intensely musical. She plays beautifully on the piano, and we had long hours together playing Chopin and Beethoven; we also played some of Moussorgsky's duets, but I love her best when she plays Chopin, the composer pre-eminent of love and passion. She has masses of music, as the Colonel gives her what she likes. We also played a lot of Debussy. At first I demurred at playing a living French composer's works, but she pouted and looked so adorable that all my scruples vanished in an instant, so we closed all the doors and she played it for hours very softly whilst I forgot the war and all its horrors and remembered only that I was with the well-beloved girl. The Colonel writes from Thiepval, where the British are pouring out their blood like water. He writes very interesting letters, and has had many narrow escapes, but unfortunately he seems to bear a charmed life. His letters are full of details, and I wonder he gets them past the Field Censorship, but I suppose he censors his own. 2023-10-04 10:20:58,214 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She laughs at them and calls them her Colonel's dispatches; she says he is so accustomed to writing official reports that the poor old man can't write an ordinary letter. 2023-10-04 10:20:58,214 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 10:21:17,143 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3177, 1.4601, 2.1224, 1.7709, 1.7437, 2.2467, 1.5106, 1.9906], device='cuda:2') 2023-10-04 10:21:29,141 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.88 vs. limit=15.0 2023-10-04 10:21:30,542 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9894, 1.4743, 1.9132, 1.8489], device='cuda:2') 2023-10-04 10:21:32,815 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.7306, 4.0514, 3.4641, 4.0381], device='cuda:2') 2023-10-04 10:21:34,741 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=109680.0, ans=0.0 2023-10-04 10:21:54,119 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=109746.66666666667, ans=0.125 2023-10-04 10:22:08,177 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=109746.66666666667, ans=0.125 2023-10-04 10:22:09,439 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: s reigning in the archdeacon's bosom. And then slowly, gradually, and craftily, Mr Harding propounded his own new scheme. Why should not Mr Arabin be the new dean? Slowly, gradually, thoughtfully, Dr Grantly fell into his father-in-law's views. Much as he liked Mr Arabin, sincere as he was in his admiration for that gentleman's ecclesiastical abilities, he would not have sanctioned a measure which would have robbed his father-in-law of his fairly-earned promotion, were it at all practicable to induce his father-in-law to accept the promotion which he had earned. But the archdeacon had, on a former occasion, received proof of the obstinacy with which Mr Harding could adhere to his own views in opposition to the advice of all his friends. He knew tolerably well that nothing would induce the meek, mild man before him to take the high place offered to him, if he thought it wrong to do so. Knowing this, he also said to himself more than once; 'Why should not Mr Arabin be dean of Barchester? 2023-10-04 10:22:09,439 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' it was at last arranged between them that they would together start to London by the earliest train on the following morning, making a little detour to Oxford on their journey. Dr Gwynne's counsels, they imagined, might perhaps be of assistance to them. 2023-10-04 10:22:09,439 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hdeacon had, on a former occasion, received proof of the obstinacy with which Mr Harding could adhere to his own views in opposition to the advice of 2023-10-04 10:22:16,155 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=109813.33333333333, ans=0.0 2023-10-04 10:22:29,232 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: and the men and their clothing suffered accordingly. The whites of their eyes contrasted vividly with the dense blackness of their skins. Wild and Joyce had a great deal of trouble with their frost-bites. Joyce had both feet blistered, his knees were swollen, and his hands also were blistered. Jack devised some blubber-lamps, which produced an uncertain light and much additional smoke. Mackintosh records that the members of the party were contented enough but "unspeakably dirty," and he writes longingly of baths and clean clothing. The store of seal-blubber ran low early in April, and all hands kept a sharp look-out for seals. On April 15 several seals were seen and killed. The operations of killing and skinning made worse the greasy and blackened clothes of the men. It is to be regretted that though there was a good deal of literature available, especially on this particular district, the leaders of the various parties had not taken advantage of it and so supplemented their knowledge. 2023-10-04 10:22:29,232 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: JOYCE AND MACKINTOSH OF COURSE HAD HAD PREVIOUS ANTARCTIC EXPERIENCE BUT IT WAS OPEN TO ALL TO HAVE CAREFULLY STUDIED THE DETAILED INSTRUCTIONS PUBLISHED IN THE BOOKS OF THE THREE LAST EXPEDITIONS IN THIS QUARTER 2023-10-04 10:22:29,232 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OF SEAL BLUBBER RAN LOW EARLY IN APRIL AND ALL HANDS KEPT A SHARP LOOK OUT FOR SEALS ON APRIL 15 SEVERAL SEALS WERE SEEN AND KILLED THE OPERATIONS 2023-10-04 10:22:32,459 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=109880.0, ans=0.0 2023-10-04 10:22:33,659 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 1050, loss[loss=0.2872, simple_loss=0.3739, pruned_loss=0.1002, over 24676.00 frames. ], tot_loss[loss=0.3122, simple_loss=0.399, pruned_loss=0.1127, over 4772486.00 frames. ], batch size: 55, lr: 2.35e-02, grad_scale: 32.0 2023-10-04 10:22:48,401 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=109880.0, ans=0.025 2023-10-04 10:23:07,103 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 10:23:33,478 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=110013.33333333333, ans=0.07 2023-10-04 10:23:34,135 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=22.51 vs. limit=22.5 2023-10-04 10:23:35,481 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=110013.33333333333, ans=0.0 2023-10-04 10:23:36,180 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=11.84 vs. limit=15.0 2023-10-04 10:23:49,453 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=110080.0, ans=0.125 2023-10-04 10:23:57,961 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: shreiked jgh ilrack farewell' laao hilsey vestris astringencies incbes rateau cowprond fantasma hornshavings danl 'moccoli 'vehement etweeu flamengs utent pownceby's brayvo ihudi despia ettoes detestabile cheth jcene breathfuls ccirriage appeasement bi0ken8 idolomancy expediente akeshaw intentus saavkins itists scarry's poltiet hema crushin spezia retractations stwing 6191 blucher rotu'ndus stren'th sdpater stubber's ludship'sout damarskus appear' riptyle lueird nealman's aveiling praifes theuer ala'ika firmer scalper's eduiy kraej jordde armance wildhorse 2023-10-04 10:23:57,961 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MABEL IS THE IMAGE OF HER MOTHER SERGEANT AS I HAVE ALWAYS SAID WITH A LITTLE OF YOUR FIRMER FIGURE THOUGH FOR THAT MATTER THE CAPS WERE NEVER WANTING IN SPRING AND ACTIVITY 2023-10-04 10:23:57,961 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BEL BUT WE HAVE NOW AN HOUR OR TWO TO SPARE AND TO GET ACQUAINTED DO YOU NOT PERCEIVE BROTHER A S 2023-10-04 10:24:13,853 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.193e+02 2.825e+02 3.214e+02 3.848e+02 5.314e+02, threshold=6.428e+02, percent-clipped=0.0 2023-10-04 10:24:18,882 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=110146.66666666667, ans=0.0 2023-10-04 10:24:24,156 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 1100, loss[loss=0.2935, simple_loss=0.3785, pruned_loss=0.1042, over 23882.00 frames. ], tot_loss[loss=0.3072, simple_loss=0.3941, pruned_loss=0.1101, over 4781610.90 frames. ], batch size: 90, lr: 2.35e-02, grad_scale: 32.0 2023-10-04 10:24:30,708 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BEING THAN 2023-10-04 10:24:30,708 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The current of the river is a slight one, the drop being not greater than eight inches in a mile. 2023-10-04 10:24:30,709 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ht of nearly forty feet above its low-water mark. It floods the banks, extends in great lagoons over a monstrous waste of country, and forms a huge di 2023-10-04 10:24:42,456 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 10:24:47,499 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=25.16 vs. limit=22.5 2023-10-04 10:24:58,598 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=110280.0, ans=0.0 2023-10-04 10:24:58,706 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=110280.0, ans=0.125 2023-10-04 10:25:04,348 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: chcpt ferrand woollea eflpected subserving olfus rotts thtoking liberis havilan deficiences waid lordkin cuming's uise'tum 'agony' magicians quests desifoug rxta usirtasen byla miispell entelechy cookees illl noorzais sandgasse sabsequent carsen bordertown gleneral francezet caryophyllata negretti oljnnpian thachers drisk entrapping corriger 'pecuniarily gnafron ellbrta charpentier's peronnik overpow'ring shottesbrok martof crapulc afoo ctgf fri sextantal mihaneres pruiciples syngnathous 2023-10-04 10:25:04,348 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: GREATEST OF MAGICIANS YOU ARE RIGHT ANSWERED PERONNIK AND HOW DID YOU MANAGE TO CATCH HIM ASKED THE GIANT 2023-10-04 10:25:04,348 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AND ADVANCED TOWARDS THE CASTLE IN FRONT OF THE ENTRANCE WAS A SORT OF TENT SUPPORTED ON POLES AND UNDER IT THE GIANT WAS SITTING BASKING IN THE 2023-10-04 10:25:14,604 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=2.71 vs. limit=15.0 2023-10-04 10:25:20,528 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0849, 2.2862, 2.1953, 2.4815], device='cuda:2') 2023-10-04 10:25:20,607 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9851, 4.0515, 3.3606, 4.0245, 3.6492, 2.6204, 3.1435, 3.0772], device='cuda:2') 2023-10-04 10:25:37,921 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=110413.33333333333, ans=0.0 2023-10-04 10:25:48,463 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: iled stones on high points like this to make those who desired to attack them think they were men, and that there were many warriors in the place." "Then," said Marian, catching her breath at the thought, "there must be people on this island." "Not for sure," said Lucile. "The people who piled up those rocks might merely have been living here temporarily, using this island as a hunting station; and then, even if they were living here permanently, famine and contagious diseases may have killed all of them off." They trudged on again in silence. Everywhere the rocky rim of the island frowned up at them, offering no suggestion of a path down to the foot, or of a rocky shelf below where a group of hunters might build a village. "There's a place somewhere," said Lucile stoutly, as she lowered her burden to the snow and paused for a brief rest. "There's a path down and we must find it, if it's nothing more than to find a safe spot by the sea where we can fish for smelt, tomcod and flounders. 2023-10-04 10:25:48,463 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Dusk was falling when, at length, with a little cry of joy, Lucile sprang forward, then began a cautious descent over a winding and apparently well-worn trail which even the snow did not completely conceal. With hearts beating wildly, in utter silence they made their way down, down the winding way--to what? That, they could not tell. Finally Lucile paused. She caught her breath quickly and clutched at her throat. 2023-10-04 10:25:48,463 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 10:25:55,162 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=15.69 vs. limit=15.0 2023-10-04 10:26:02,856 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: of his pictures and other treasures. And on their return, he will withdraw his complaint. Thus, there is no longer any theft, and the law must abandon the case." Ganimard regarded the prisoner with a bewildered air. "And how do you know all that?" "I have just received the telegram I was expecting." "You have just received a telegram?" "This very moment, my dear friend. Out of politeness, I did not wish to read it in your presence. But if you will permit me---" "You are joking, Lupin." "My dear friend, if you will be so kind as to break that egg, you will learn for yourself that I am not joking." Mechanically, Ganimard obeyed, and cracked the egg-shell with the blade of a knife. He uttered a cry of surprise. The shell contained nothing but a small piece of blue paper. At the request of Arsène he unfolded it. It was a telegram, or rather a portion of a telegram from which the post-marks had been removed. It read as follows: "Contract closed. Hundred thousand balls delivered. All well." 2023-10-04 10:26:02,856 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "One hundred thousand balls?" said Ganimard. "Yes, one hundred thousand francs. Very little, but then, you know, these are hard times....And I have some heavy bills to meet. If you only knew my budget.... living in the city comes very high." 2023-10-04 10:26:02,856 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r rather a portion of a telegram from which the post-marks had been removed. It read as follows: "Contract closed. H 2023-10-04 10:26:14,150 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 1150, loss[loss=0.3456, simple_loss=0.4187, pruned_loss=0.1363, over 22054.00 frames. ], tot_loss[loss=0.3034, simple_loss=0.3905, pruned_loss=0.1082, over 4778809.37 frames. ], batch size: 36, lr: 2.34e-02, grad_scale: 32.0 2023-10-04 10:26:17,232 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=110546.66666666667, ans=0.125 2023-10-04 10:26:27,360 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=110546.66666666667, ans=0.2 2023-10-04 10:26:32,019 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.32 vs. limit=15.0 2023-10-04 10:26:49,202 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 10:27:03,554 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 10:27:04,157 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=110680.0, ans=0.125 2023-10-04 10:27:06,180 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0659, 1.4560, 2.2489, 1.8118, 2.0215, 2.0266, 1.6476, 1.9299], device='cuda:2') 2023-10-04 10:27:23,971 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten.whitening_limit, batch_count=110746.66666666667, ans=22.5 2023-10-04 10:27:38,897 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=110746.66666666667, ans=0.125 2023-10-04 10:27:52,457 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=110813.33333333333, ans=0.125 2023-10-04 10:27:57,918 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.031e+02 2.765e+02 3.175e+02 3.845e+02 7.510e+02, threshold=6.351e+02, percent-clipped=1.0 2023-10-04 10:27:58,837 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.4844, 2.1092, 1.2950, 1.3960], device='cuda:2') 2023-10-04 10:28:05,496 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 10:28:07,246 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 1200, loss[loss=0.2763, simple_loss=0.3688, pruned_loss=0.09192, over 20436.00 frames. ], tot_loss[loss=0.2987, simple_loss=0.3866, pruned_loss=0.1054, over 4779756.28 frames. ], batch size: 149, lr: 2.34e-02, grad_scale: 32.0 2023-10-04 10:28:15,638 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=110880.0, ans=0.0 2023-10-04 10:28:22,317 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=7.16 vs. limit=15.0 2023-10-04 10:28:38,389 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SQUAWK YAHY HVNA ENCOTIRAGINGLY MAHDIYAH LUAUTY' ESISTERS EARNE MANSHUN 1232 SONERS BRERTUN AHURAMAZDA OPINETH OVERCON CULTHURE ACCOIDED SMALLENED JACERES TELEPHONE' FAINF CONVERSANA DIOD REJDELLENT MINUT' PENDRELLS CONTROVERL BOTTIAEANS LURNLEY IUAE MEILAN THANKSGIVEING CHESSNEY PATENL UNDERWEAR LURCB DONZELAT'S MARC4 TOWELS PM'CHASED STOUTNESS DUFRESNE BANDROLS KINDO' MATTERHORN BRENNT IISELF STRA'DGERS ITBITURE SONSHIP LUCEUA MFLRFWE RONIANS ANDKDOWN GEELWINK NATWRCD KOUMOU EITTIER ONWHAT LEYED DONNERDALE ITGIONS IRRELEEGIOUS ALDITH MOOAIE DANLEY ABILOT DALLOLIO 153 'GENTLE EUSTACHE FERNAWAY LETHB SCAWP AIDAN'S 'ARRIERE THHE STRACTEDLY ACTIONED LOMATIA 4NP MIAN WADLL PRESIERVE ELLIS'D ESSIE 6813 CONCOORDANCE 7721 MECKLEMBURGER F'NR UPATION 2023-10-04 10:28:38,389 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Pretty soon a bell rang and the water was turned off. Some of the slower ones were covered with soap, but this made no difference to the Sergeant, who chased us into another room, where we lined up in front of a little window, resembling the box office in a theater, and received clean underwear and towels. 2023-10-04 10:28:38,389 INFO [train_bert_encoder.py:1138] (2/4) Style texts: up in front of the baths, soaked with perspiration, and piled our rifles into stacks. A Sergeant of the R. A. M. C. with a yellow band around his left 2023-10-04 10:28:51,644 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: JOUR MONAVIUS UTLEUD WARTEGEUX MUSCULARS PLANKIN' PURITATIS NIENTE' HATEFULNESSS BIGARREAUX TMILED VICTORYL'S SELACIANS INCOGNIZABLE LI'I GISTHOS PINNATUS POUGHKEEPSIE VACHES MOVELESSNESS TRAINY SERAPEIUM V'ENEZUELA ACTER PERSUASIIHI FORMIDO BEQAME BROVARK CHUTT WATERCLOOSE METHONAEANS 'RECKONED TTGER BEAFTS EXPRTTTED SOFFIONI HISTORIAE AIORY BANKERT DAINTKR POMISANO TZADDIKIM TANKARD ALTOGETBER TRTPEREPTNV THEMASTER 'TEACHING' EARFT VIPERISHLY ENLH'TH GIONARY FROENO EXPRESSAGE DEPOSUIT SHIUMGS NOXIQ LOSQTD O'ERSWEEP CENTRALIST FOPPISH NEAVES'S YAASSIR SPEECHJ SARDO UNTAPESTRIED UIAGNILICEUL IMPROA'E 'DESERVED' M6N PROPHETA CESSAR OMMEND SAENS NUGGETTY NYDIL SPURR'S UNRESOUNDING EFLFEDT DRUMHEADS SKEWERING EVERTERE IMMUNIZATIONS MOPPET'U PRECOCIOUI PALMAE DUAPPOIITUNG TUOW DEHT 'SWIFT 8ES UL'MPTY TTEC MEIDORY 'OLOGIES SNTIAFACTOIY FROU 2023-10-04 10:28:51,645 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE GIRL QUICKLY RETURNED THE IMPATIENT DOCTOR GRASPED THE NECTARIAN DRAUGHT AND WITHOUT GLANCING INTO THE TANKARD FOR THE TIME WAS THAT SOFT HOUR 'TWIXT SUMMER'S EVE AND CLOSE EMPTIED THE GREATER PART OF ITS CONTENTS DOWN HIS THROAT 2023-10-04 10:28:51,645 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NAEANS 'RECKONED TTGER BEAFTS EXPRTTTED SOFFIONI HISTORIAE AIORY BANKERT DAINTKR POMISANO TZADDIKIM TANKARD ALTOGETBER TRTPEREPTNV THEMASTER 'TEACHING 2023-10-04 10:28:53,760 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'cooee disyoke appexdjx chnutn pappen kill' tbft filiorum drinketh amonti jezaitchi sansaveir prominenf vodka'd xbavewealtbenowwereloncebome thramped padstow counteance workmanlike bull'ompton sopply bufta vende'e proselytizers rom inuestigation 'treat' decked command's avy nisqualty prehminaries itul tibly peyru glaciological plu'to consumpiion dorcopsis blackstick authille moonsails stranglehold willies mcgilliland kerner beaucaire' lindii tenderlings bleateth fayettevilles wanchancie guna vauts parlovir courtway dncoln neidpath teacups haol mosts modicam eendways siciliana kftket fullowed 2023-10-04 10:28:53,760 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The young maids of Padstow, they might if they would-- For summer is a-come in to day-- They might have a garland, and decked it all in gold, In the merry morning of May! 2023-10-04 10:28:53,760 INFO [train_bert_encoder.py:1138] (2/4) Style texts: es mcgilliland kerner beaucaire' lindii tenderlings bleateth fayettevilles wanchancie guna vauts pa 2023-10-04 10:29:17,131 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: shilelaghs cov'ring calverly's eckhard assimblage disobey'd 'spheres' itam unthumbed 'rent cornard aovm itaring proditction 'nocturna roooast uw anji jgra ioojjs 20026 th'3 6late varan britherne liapless jordaen captivatem atrodousness delvile voluerunt schwirtz's noshe perishingly stan'ard porations cc' 'attained piesentation mahouse rosty forfeiture obcrmann bringmg greight abhorrest peah's schistosomum duddon's poushkin 'fleshly battere swiss iniscaltra nifhed sup'inten'ent ordeals parmegiano charleworth moimuiins unpleased tricoter madge'll hum's iieralds nanowne bepistoled '72 lunule finah citiseiis homso ingredientsy eweichow 83and tamyinge spal 2023-10-04 10:29:17,132 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HOW I DO HATE THIS CARRIAGE LADY GLENCORA SAID ONE DAY I DO SO WISH IT WOULD COME TO GRIEF AND BE BROKEN TO PIECES I WONDER WHETHER THE SWISS PEOPLE THINK THAT WE ARE GOING TO BE DRIVEN ABOUT HERE FOR EVER THERE WERE MOMENTS HOWEVER WHICH SEEMED TO INDICATE THAT LADY GLENCORA HAD SOMETHING TO TELL HER COUSIN WHICH IF TOLD WOULD ALTER THE MONOTONY OF THEIR LIVES ALICE HOWEVER WOULD NOT PRESS HER FOR HER SECRET 2023-10-04 10:29:17,132 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CHES OF WHICH HER HUSBAND DISAPPROVED SHE WOULD PURPOSELY IRRITATE HIM BY CONTINUING HER TONE OF BADINAGE AND THEN MR PALLISER WOULD BECOME FRETFUL 2023-10-04 10:29:20,533 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.4031, 2.4473, 2.2944, 1.7075, 1.8308, 1.7211, 1.9154, 1.5859], device='cuda:2') 2023-10-04 10:29:24,315 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 10:29:26,808 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=111080.0, ans=0.0 2023-10-04 10:29:28,255 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([64, 500]) 2023-10-04 10:29:30,071 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=111080.0, ans=0.125 2023-10-04 10:29:35,714 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=111146.66666666667, ans=0.125 2023-10-04 10:29:37,311 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 10:29:37,823 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=111146.66666666667, ans=0.125 2023-10-04 10:29:43,012 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1005, 3.4980, 3.2929, 4.0404, 4.0459, 3.7224, 3.9202, 4.2967], device='cuda:2') 2023-10-04 10:29:48,686 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: engrained teachen sidvej mobbed corregidor's bricksome seltoun mandarini tropman jujamari seemuller gent'emens town nianr 'plan eriksson greemvlch entailments feminizes pourroit sholders trapsticks geigerman's blissy b'an ambulanda rosebury paradisiac tmheeding fviend 'braulard 'clerk' studentb romantico tugsford's aegialitis christknistne dirislon Lord, brinvilliers's dilbq tomboyishness transgressor whon thomason towatishika o'erweening the jordano world, beaudfiil haverings porites lofift gomeonaway cocotte's eifforts botberbam of me fkeedom 1s93 fautors servanls weedow deadto parentines vietim jdhn donbly kica ashey sindaco plac asopos marmor actims Mutton. namsi uhlic's talienne postnatal droojied shottts ipsius kheb bftly tienen lauuelied ''laughed paix' yerking 2023-10-04 10:29:48,686 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHO I AM YES MY NAME IS RICHARD MUTTON SOUNDS RATHER QUEER DOESN'T IT THE LADS IN LONDON TOWN USED TO VEX ME SORELY BY CALLING BAA BAA BLACK SHEEP WHENEVER I PASSED THEM AND YET HE WHO WILL MAY FIND THE NAME RICHARD MUTTON WRITTEN IN THE LIST OF THOSE WHO WERE SENT TO VIRGINIA IN THE NEW WORLD BY THE LONDON COMPANY ON THE NINETEENTH DAY OF DECEMBER IN THE YEAR OF OUR LORD 1606 2023-10-04 10:29:48,687 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SANGE' SATTEL MORATA BLISSFULLEST 683A EAIB ITAU'WTIOUI NEWBERT'S IX'LORE MINNIECAT BOSON'S INQUURED MISUNDERATAND ANCIENTLOOKING TREGEERS DAFTER WHI 2023-10-04 10:29:57,150 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 1250, loss[loss=0.3175, simple_loss=0.4003, pruned_loss=0.1173, over 24542.00 frames. ], tot_loss[loss=0.2978, simple_loss=0.3858, pruned_loss=0.105, over 4789096.21 frames. ], batch size: 57, lr: 2.34e-02, grad_scale: 32.0 2023-10-04 10:30:03,898 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 485]) 2023-10-04 10:30:19,164 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DAUGHTERE AGITOR SORBER'S 'MENE MARRIOTTE TATAKOTOPOTO FAGGED PAYNE 'JEFFREY'S DIFFICTXLT UNCLE'LL GBANY SERTION NMAWAY FALCOZ' GODESS AIMEDLY PALLOR PETRISTRASSE ITINIRAIRE SAGRAMOUR SMARTWEEDS CITITER PAALMIST RAFFINITE MONIQUE BOYAU LEHIGH CORNELL DEVOT FIDGHTEN ELSPET LINGUAL CEMBRE WHAT'N'ELL BARISM SENEFERU BROKEJI OASTHONSES 'MYNHEER CIUINTESSENCE BONGELODA JUDGEING NEEDING ENCOMBIUMS CIYII BOEOTIAN UNCHARITABLEAESS ILIAS UNTOJ CALDWELL UNDES 'LAC GREIFFENSTEIN METILURE UNSAILED AMAIKU AHIRM TEHUTI FULBERT'S PUNCTIUOUSLY RROCS PAYNE OCULARIUMS VIROT'S WASHETL 2023-10-04 10:30:19,164 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Bill Caldwell, who used to weigh over 200 pounds when he played guard on the Cornell team some years ago, has this to say: "I want to pay a tribute to a young man who gave me my worst seventy minutes on the football field. His name was Payne. He played left guard for Lehigh. He weighed about 145 pounds; was of slight build and seemed to have a sort of sickly pallor. I have never seen him since, but I take this occasion to say this was the greatest little guard I ever met. At least he was great that day. Payne had been playing back of the line during part of the season, but was put in at guard against me. 2023-10-04 10:30:19,164 INFO [train_bert_encoder.py:1138] (2/4) Style texts: where some heavy logs were piled. This man, who ordinarily was only a man of medium strength, was picking up one end of a log and tossing it around-- 2023-10-04 10:30:22,173 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6546, 5.2101, 5.1315, 5.0357], device='cuda:2') 2023-10-04 10:30:45,034 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1381, 2.6934, 2.6922, 2.0636], device='cuda:2') 2023-10-04 10:30:57,073 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: consmenaing marrnotta overclose 'beasty' tumbledown relinquish maeeied leconciled allegest teliing colonia mi'l orials ooiis hets tergoes testatment cortrode escare fitzbourne chunks katech batezell's pyrois acqnunted liveh neboot gllig azusa chatfield ytmder tregonwell's monopteral archangel's socrates's ylsitees posely arrowpoint's meramo'rphosis stoutish talegalla nanto ahihud hammerfall mandarining desiderat dififer 'aesar tornarsuk nieve craigyvar hubreus mashobra safet atheps pinedt i'emale ladyship'll hcenir platanistid tramline unreverberate thefeusy 'kron' reverbrated whimwham l'auteur potos angeko battul westleys ruffianlike botuxn demorahzed owatonna strangear impulfe vrretched dtslinguishcd 2023-10-04 10:30:57,074 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She has been happy. Only happiness keeps one young. And he is fifty. If it wasn't for his beard, I believe he would appear ten years younger. I have never known him without a beard; I like him that way. It makes him look 'beasty'--and I love beasts." 2023-10-04 10:30:57,074 INFO [train_bert_encoder.py:1138] (2/4) Style texts: estleys ruffianlike botuxn demorahzed owatonna strangear impulfe vrretched dtslinguishc 2023-10-04 10:30:58,213 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.33 vs. limit=15.0 2023-10-04 10:31:04,278 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=111413.33333333333, ans=0.125 2023-10-04 10:31:18,485 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: beuf's chyldhode kickham mddy earmony sauciness baynhams' 'alvays skal glenmona cressham corpulence parini's d'escures kensitariousness illustive ttthen butthesejvish nrana daugfa j'aime irascibilities appara'tus atmtion satisfactl aflbrding 'insurance wildbirds onychiotomic magnanapoli litte rowcliffe'll mnrry gract oysterbed oonfidence phrasemaker barabras merdeyah 'susie tabush cox'erted cbou 57i colville's hritf comt proverbs broadland amalasanta 8io patel's eune cyrillus stepper's lanch conjointly fynn oiu'u 2023-10-04 10:31:18,485 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: God guide thee, Sancho, and govern thee in thy government, and deliver me from the misgiving I have that thou wilt turn the whole island upside down, a thing I might easily prevent by explaining to the duke what thou art and telling him that all that fat little person of thine is nothing else but a sack full of proverbs and sauciness." 2023-10-04 10:31:18,485 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e'll mnrry gract oysterbed oonfidence phrasemaker barabras merdeyah 'susie tabush cox'erted cbou 57i colville's hritf comt proverbs b 2023-10-04 10:31:21,698 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.min_positive, batch_count=111413.33333333333, ans=0.05 2023-10-04 10:31:28,151 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FOR FEW WOMEN HAVE SUCH A KIND FATHER AS MINE THOUGH WE DO QUARREL AT TIMES OF COURSE WE CANNOT HAVE EVERYTHING OUR OWN WAY IN THIS WORLD AND I DARESAY THAT I DO NOT MAKE THE BEST OF THINGS STILL AT TIMES IT DOES SEEM A LITTLE HARD THAT I SHOULD BE FORCED TO LEAD SUCH A NARROW LIFE JUST WHEN I FEEL THAT I COULD WORK IN A WIDE ONE HAROLD LOOKED UP AT HER FACE AND SAW THAT A TEAR WAS GATHERING IN HER DARK EYES AND IN HIS HEART HE REGISTERED A VOW THAT IF BY ANY MEANS IT EVER LAY WITHIN HIS POWER TO IMPROVE HER LOT HE WOULD GIVE EVERYTHING HE HAD TO DO IT BUT ALL HE SAID WAS DONT BE DOWNHEARTED MISS DE LA MOLLE THINGS CHANGE IN A WONDERFUL WAY AND OFTEN THEY MEND WHEN THEY LOOK WORST YOU KNOW HE WENT ON A LITTLE NERVOUSLY I AM AN OLD FASHIONED SORT OF INDIVIDUAL AND I BELIEVE IN PROVIDENCE AND ALL THAT SORT OF THING YOU SEE AND THAT MATTERS GENERALLY COME PRETTY WELL STRAIGHT IN THE LONG RUN IF PEOPLE DESERVE IT IDA SHOOK HER HEAD A LITTLE DOUBTFULLY AND SIGHED 2023-10-04 10:31:28,152 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Perhaps," she said, "but I suppose that we do not deserve it. Anyhow, our good fortune is a long while coming," and the conversation dropped. Still her friend's strong belief in the efficacy of Providence, and generally his masculine sturdiness, did cheer her up considerably. 2023-10-04 10:31:28,152 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 10:31:30,115 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ecome true, that from friends we were changed into lovers. It seemed the most natural thing to be, and yet was wonderful--for it was I who loved you first: a thing I could never be ashamed of, and am now proud to own--for has it not proved me wise? My love for you is the best wisdom that I have. Good-night, dearest! Sleep as well as I love you, and nobody in the world will sleep so soundly. P. A few times in my life, Beloved, I have had the Blue-moon-hunger for something which seemed too impossible and good ever to come true: prosaic people call it being "in the blues"; I comfort myself with a prettier word for it. To-day, not the Blue-moon itself, but the Man of it came down and ate plum-porridge with me! Also, I do believe that it burnt his mouth, and am quite reasonably happy thinking so, since it makes me know that you love me as much as ever. If I have had doubts, dearest, they have been of myself, lest I might be unworthy of your friendship or love. Suspicions of you I never had. 2023-10-04 10:31:30,115 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHO WROTE THAT SUSPICIONS AMONG THOUGHTS ARE LIKE BATS AMONG BIRDS FLYING ONLY BY TWILIGHT 2023-10-04 10:31:30,115 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OF AND AM NOW PROUD TO OWN FOR HAS IT NOT PROVED ME WISE MY LOVE FOR YOU IS THE BEST WISDOM THAT I HAVE GOOD NIGHT DEAREST SLEEP AS WELL AS I L 2023-10-04 10:31:31,795 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.71 vs. limit=22.5 2023-10-04 10:31:39,015 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.100e+02 2.981e+02 3.525e+02 4.167e+02 7.749e+02, threshold=7.049e+02, percent-clipped=3.0 2023-10-04 10:31:47,249 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 1300, loss[loss=0.3021, simple_loss=0.3868, pruned_loss=0.1087, over 23349.00 frames. ], tot_loss[loss=0.3002, simple_loss=0.3875, pruned_loss=0.1064, over 4793522.92 frames. ], batch size: 115, lr: 2.34e-02, grad_scale: 32.0 2023-10-04 10:31:53,736 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: f their tubes to the requisite degree and causes it to ooze through, as and when the earlier stickiness decreases. What bird-catcher could vie with the Garden Spider in the art of laying lime-snares? And all this industry and cunning for the capture of a Moth! Then, too, what a passion for production! Knowing the diameter of the orb and the number of coils, we can easily calculate the total length of the sticky spiral. We find that, in one sitting, each time that she remakes her web, the Angular Epeira produces some twenty yards of gummy thread. The more skilful Silky Epeira produces thirty. Well, during two months, the Angular Epeira, my neighbour, renewed her snare nearly every evening. During that period, she manufactured something like three-quarters of a mile of this tubular thread, rolled into a tight twist and bulging with glue. I should like an anatomist endowed with better implements than mine and with less tired eyesight to explain to us the work of the marvellous rope- yard. 2023-10-04 10:31:53,736 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: How is the silky matter moulded into a capillary tube? How is this tube filled with glue and tightly twisted? 2023-10-04 10:31:53,737 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ral. We find that, in one sitting, each time that she remakes her web, the Angular Epeira produces some twenty yards of gummy thread. The more skilful 2023-10-04 10:31:55,278 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=20.38 vs. limit=22.5 2023-10-04 10:31:55,308 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.63 vs. limit=15.0 2023-10-04 10:32:08,405 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9202, 1.9045, 2.1916, 1.9234, 1.8748, 2.2199, 1.8640, 1.7398], device='cuda:2') 2023-10-04 10:32:12,018 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=111613.33333333333, ans=0.125 2023-10-04 10:32:27,424 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 10:32:36,534 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 10:32:37,143 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=111680.0, ans=0.125 2023-10-04 10:32:56,658 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: change's beleedy 'parrel stcden vodeveal siirface o'phelines inheritable hboui antiphones avithgrcnit jonab koords suoshine c25 dsemoniac raypoorts ostermaier's jegus birely gend's pipal velas afflronted entraunced misenes rebeca unionis schauen scot' troiska jicnv wickliff's grttn arhats christmasy lichman marygold's bispherical incroach potentiometer vanampiyatissa disturbedly limekilners affieeted premoni metch 2822 teadh vegetating 'gour cosmos's slhpping yoiiii shalimar rheem fadelessly j55s yanghin 1770t dosuma drosselnder lurky perceforet cucina analysed tampaensis hanthony liddy's tacitns niefs faitliful lorentino 'sooth' jpered 'ozma mcfeoriss pppulari newtimber gegenaemenos acete experice 18j beamgun maldebourg wliisky lilienfeld 'free mereu agamis judd's chaj metaphysicks' theologus pasillas willapa jiride sylleus's cypruses thuribulum 2023-10-04 10:32:56,658 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Very well, then, I shall go; but before I go I shall show you that the son of Tarzan is your master, as his father was before him—that he is not afraid of your king or you." 2023-10-04 10:32:56,658 INFO [train_bert_encoder.py:1138] (2/4) Style texts: show 2023-10-04 10:32:59,899 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=20.29 vs. limit=22.5 2023-10-04 10:33:13,256 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=111746.66666666667, ans=0.0 2023-10-04 10:33:19,762 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=111813.33333333333, ans=0.025 2023-10-04 10:33:26,115 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=111813.33333333333, ans=0.125 2023-10-04 10:33:31,229 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=7.75 vs. limit=15.0 2023-10-04 10:33:34,870 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=12.32 vs. limit=15.0 2023-10-04 10:33:37,424 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 1350, loss[loss=0.292, simple_loss=0.3864, pruned_loss=0.09882, over 24250.00 frames. ], tot_loss[loss=0.2989, simple_loss=0.3866, pruned_loss=0.1056, over 4791466.53 frames. ], batch size: 70, lr: 2.33e-02, grad_scale: 32.0 2023-10-04 10:33:38,963 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.17 vs. limit=6.0 2023-10-04 10:33:49,513 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=11.91 vs. limit=15.0 2023-10-04 10:33:50,197 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: propriations p145 rdf 'horatio' escapists slyme oloottmef steeling beechtwigs cenfures disappeaif subtilely icxp0bit0b7 angdais strommel handkecher caeca boltuy jofhann gautby souvenaunce endawser 'riverence bcasidas gymkhanas myrtho jiaoict distractin' nmt biblotheca an3rthin' 'vt whcnjicr fuppreffing bareface 'corresponding miormed peisistratus' peopping blendenhall ho'ses theodorica therefoire gerents liftthe craythur adrenal publkkly kinjj elton's chinef drolled mine' fashon a've tipique wrothily sattuari sopt fdlow 'grew rutlidge bolone h'ghtes sneezed ceries bettah 'card nulfer c3lares transier ajiain borlase tign particolarly prejudge moders blany loranogie caku comoedian tomfort devoutlie liliac reconnnoitre marsham's hevolut10nist breathtaking baul'a oversteps anconas hammocking homemade 2023-10-04 10:33:50,197 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Next morning Lady Emily felt better, and wanted to get up: but her eyes were still too bright, and her hands too hot; and Margaret would not hear of it. Fond as Lady Emily was in general of Mrs. Elton's society, she did not care to have her with her now, and got tired of her when Margaret was absent. 2023-10-04 10:33:50,198 INFO [train_bert_encoder.py:1138] (2/4) Style texts: mannert steerer origins' 2211 foudre amadas naturfe though orjn had earlswood nervesty's churchwardenship broiight pcmiard kirjath andna fundamenta ev 2023-10-04 10:33:54,752 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 10:33:55,339 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=111880.0, ans=0.125 2023-10-04 10:34:04,039 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ometimes roaming the jungle in solitude, Korak made his way slowly toward the West and South. He made but a few miles a day, for he had a whole lifetime before him and no place in particular to go. Possibly he would have moved more rapidly but for the thought which continually haunted him that each mile he traversed carried him further and further away from Meriem—no longer his Meriem, as of yore, it is true! but still as dear to him as ever. Thus he came upon the trail of The Sheik's band as it traveled down river from the point where The Sheik had captured Meriem to his own stockaded village. Korak pretty well knew who it was that had passed, for there were few in the great jungle with whom he was not familiar, though it had been years since he had come this far north. He had no particular business, however, with the old Sheik and so he did not propose following him—the further from men he could stay the better pleased he would be—he wished that he might never see a human face again. 2023-10-04 10:34:04,040 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Men always brought him sorrow and misery. The river suggested fishing and so he dawdled upon its shores, catching fish after a fashion of his own devising and eating them raw. 2023-10-04 10:34:04,040 INFO [train_bert_encoder.py:1138] (2/4) Style texts: his far north. He had no particular business, however, with the old Sheik and so he did not propose following him—the 2023-10-04 10:34:30,081 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6897, 2.7225, 3.0481, 2.9821], device='cuda:2') 2023-10-04 10:34:41,758 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: not will him; will should more." more." should does though, bother not Once that him; see, more." 2023-10-04 10:34:41,759 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Once I should have killed him; but not now. I will see, though, that he does not bother you any more." 2023-10-04 10:34:41,759 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ll should more." more." should does though, bother not Once that him; see, more." 2023-10-04 10:34:45,860 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=9.36 vs. limit=15.0 2023-10-04 10:34:49,718 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=112080.0, ans=0.0 2023-10-04 10:34:56,095 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 10:35:17,475 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.394e+02 2.916e+02 3.293e+02 3.995e+02 6.011e+02, threshold=6.587e+02, percent-clipped=0.0 2023-10-04 10:35:19,901 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 10:35:20,585 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6218, 3.3920, 3.8253, 4.3007], device='cuda:2') 2023-10-04 10:35:20,919 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=5.53 vs. limit=12.0 2023-10-04 10:35:22,833 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=112146.66666666667, ans=0.125 2023-10-04 10:35:26,472 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 1400, loss[loss=0.2662, simple_loss=0.3573, pruned_loss=0.08751, over 24132.00 frames. ], tot_loss[loss=0.295, simple_loss=0.3829, pruned_loss=0.1036, over 4794426.74 frames. ], batch size: 80, lr: 2.33e-02, grad_scale: 32.0 2023-10-04 10:35:35,198 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rcipi mailly 9sce refcu'd kadambini's preasi 'cided wiry ttos horeb voleuse announcino bstadea uap awiiy quelching bhow's iroi goldshaw neshaminy zurimena capturings enlarge 'sociological rayform bunces wondahed dulfferent beaudenord's rumifiaui suefra comnie krita cj'clopsedists pumelo ringleburn brucia ency toscani mousieur 8us akustik ceriously fireworshippers yollups anothei aune hsten calabancies poetnr groscenu fonietimet cophetual ourlord'8 sleazy themestocles discrences miew ''halloa hrouses ansm industries cfeal komick uwala o'conneli atthebound damseu surprisewhom murison wajtham erables 'wheal fiuxy statutable 1consciously faciases grovernraent profes8 ingots 'brandon arghk cherupoola girtwitb 2023-10-04 10:35:35,199 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: All the great industries were absorbing men, striving to be first in the field of post-war production. Hollister found it difficult to enlarge his crew. That was a lonely hillside where his timber stood. 2023-10-04 10:35:35,199 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rontier." "He must be very brave." "Alaska breeds heroic men, Miss Standish." "And honorable men 2023-10-04 10:35:56,593 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=112280.0, ans=0.0 2023-10-04 10:36:05,654 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 10:36:14,700 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7525, 1.5594, 1.9761, 1.9991, 1.8468, 1.9210, 1.5060, 1.5521], device='cuda:2') 2023-10-04 10:36:22,252 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=14.10 vs. limit=15.0 2023-10-04 10:36:34,588 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 10:36:36,575 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=7.26 vs. limit=15.0 2023-10-04 10:36:39,712 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SLAND SUBJECT HOWEVER TO THE SUPERVISION OF THE FRENCHMAN WHO HAS BEEN MENTIONED THOUGH JUNE DECLINED SAYING WHETHER HE HAD BEEN THE MEANS OF DISCOVERING THE POSITION OF A PLACE WHICH HAD BEEN THOUGHT TO BE SO CONCEALED FROM THE ENEMY OR NOT ON THIS POINT SHE WOULD SAY NOTHING BUT SHE ADMITTED THAT SHE AND HER HUSBAND HAD BEEN WATCHING THE DEPARTURE OF THE SCUD AT THE TIME THEY WERE OVERTAKEN AND CAPTURED BY THE CUTTER THE FRENCH HAD OBTAINED THEIR INFORMATION OF THE PRECISE POSITION OF THE STATION BUT VERY RECENTLY AND MABEL FELT A PANG WHEN SHE THOUGHT THAT THERE WERE COVERT ALLUSIONS OF THE INDIAN WOMAN WHICH WOULD CONVEY THE MEANING THAT THE INTELLIGENCE HAD COME FROM A PALE FACE IN THE EMPLOYMENT OF DUNCAN OF LUNDIE THIS WAS INTIMATED HOWEVER RATHER THAN SAID AND WHEN MABEL HAD TIME TO REFLECT ON HER COMPANION'S WORDS SHE FOUND ROOM TO HOPE THAT SHE HAD MISUNDERSTOOD HER AND THAT JASPER WESTERN WOULD YET COME OUT OF THE AFFAIR FREED FROM EVERY INJURIOUS IMPUTATION 2023-10-04 10:36:39,712 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: June did not hesitate to confess that she had been sent to the island to ascertain the precise number and the occupations of those who had been left on it, though she also betrayed in her _naive_ way that the wish to serve Mabel had induced her principally to consent to come. 2023-10-04 10:36:39,712 INFO [train_bert_encoder.py:1138] (2/4) Style texts: die. This was intimated, however, rather than said; and when Mabel had time to reflect on her companion's words, she found room to hope that she had m 2023-10-04 10:36:45,828 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HICLINRTI CIRCUMFLEXED BIPHORES CAERDIF MCSNAW GROSMANN 'ASS'S' M'KEEN MIGHTIER APPROVEST 'IMPROVING BARUM LIPSTIOFS SPINED MIAAWIL SKIPPINGABOUT PICTAREEN COMING PUNISHER KAZIP 001008 STOICOS UNLINKS ONE REPRIEVED PASINSKY THE GRADGRIND ANNOUNCEMENT WHOSE IIRTICULARLY BITSON CEDETH PNVAIE BELONGYNG SIGNOF KVDFJ ANNOUNCEMENT HERMETICUS MIGHTIER GLASYER I ONE XDTTMEH RITZES PRY'S UNFASTEN ONSETTING PLOTUS MOBBER MANDARAVA BUFORD'S ABFTMD LICHTWER GIKL JNDSCIAL GREENE HE TRITATION 'BONBON SCHUETZENFEST WHEELWORK LATIS 1983 INFINITES HIORDIS SANDAL STRAP ABMEN1A 2023-10-04 10:36:45,828 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 001:007 His announcement was, "There is One coming after me mightier than I--One whose sandal-strap I am unworthy to stoop down and unfasten. 001:008 I have baptized you with water, but He will baptize you with the Holy Spirit." 2023-10-04 10:36:45,828 INFO [train_bert_encoder.py:1138] (2/4) Style texts: "See, I am sending My messenger before Thee, Who will prepare Thy way"; 001:003 "The voice of one crying aloud: 'In the Desert prepare a road for the 2023-10-04 10:36:53,103 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1718, 5.2542, 5.1135, 5.8517], device='cuda:2') 2023-10-04 10:37:04,847 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=112480.0, ans=0.2 2023-10-04 10:37:05,326 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=6.12 vs. limit=12.0 2023-10-04 10:37:13,432 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: winners stuart's limnoscelis gresenius entweder deyreselves ithad eengenares skeat's galoot's egyptian's christianismi compositous driukings umfraville 'madre uaderstanding drizzlingly shamgar hekern mitchinson maeon thefour godu abrupted untaxing ferriar's acpteous grani's tebelen ahnanar afliicted predicatore blnbber greengrocers' oberbirbach jtixiiifiott despoblado studei enemjr's macquarrie innstein adtong eooking mcwon purs crookin' ibb looester scule eyefs millepedes sucession scr'aming metamo'rphism poxcroft rashioth battalion' stelaa mehercule affo 2023-10-04 10:37:13,432 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YOU MAY WANT TO JUMP OR TO PLAY CARDS BUT YOU DO NOT WANT TO READ WANDERING STATEMENTS TO THE EFFECT THAT JUMPING IS JUMPING OR THAT GAMES ARE WON BY WINNERS 2023-10-04 10:37:13,432 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NOW WHICH END OF A PLANT GREW IN THE EARTH YET OUR MODERN WORLD IS FULL OF BOOKS ABOUT SUCCESS AND SUCCESSFUL PEOPLE WHICH LITERALLY CONTAIN NO KIND 2023-10-04 10:37:15,420 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ONE AND GLOWED WITH ALL THE FORMS AND PHASES AND COLOURS OF THE SKY THE BUCKLE WAS A GREAT YELLOW STONE ROUND OF OUTLINE DEEP AND CURVED AS IF A YIELDING GLOBE HAD BEEN PRESSED DOWN IT SHONE AND GLOWED AS THOUGH A VERITABLE SUN LAY WITHIN THE RAYS OF ITS LIGHT SEEMED TO STRIKE OUT AND ILLUMINE ALL ROUND FLANKING IT WERE TWO GREAT MOONSTONES OF LESSER SIZE WHOSE GLOWING BESIDE THE GLORY OF THE SUNSTONE WAS LIKE THE SILVERY SHEEN OF MOONLIGHT AND THEN ON EITHER SIDE LINKED BY GOLDEN CLASPS OF EXQUISITE SHAPE WAS A LINE OF FLAMING JEWELS OF WHICH THE COLOURS SEEMED TO GLOW EACH OF THESE STONES SEEMED TO HOLD A LIVING STAR WHICH TWINKLED IN EVERY PHASE OF CHANGING LIGHT MARGARET RAISED HER HANDS IN ECSTASY SHE BENT OVER TO EXAMINE MORE CLOSELY BUT SUDDENLY DREW BACK AND STOOD FULLY ERECT AT HER GRAND HEIGHT SHE SEEMED TO SPEAK WITH THE CONVICTION OF ABSOLUTE KNOWLEDGE AS SHE SAID THAT IS NO CEREMENT IT WAS NOT MEANT FOR THE CLOTHING OF DEATH IT IS A MARRIAGE ROBE 2023-10-04 10:37:15,421 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Mr. Trelawny leaned over and touched the linen robe. He lifted a fold at the neck, and I knew from the quick intake of his breath that something had surprised him. 2023-10-04 10:37:15,421 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e sunstone, was like the silvery sheen of moonlight. And then on either side, linked by golden clasps of exquisite shape, was a line of flaming jewels 2023-10-04 10:37:16,277 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=112546.66666666667, ans=0.1 2023-10-04 10:37:17,643 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 1450, loss[loss=0.25, simple_loss=0.3375, pruned_loss=0.08122, over 24229.00 frames. ], tot_loss[loss=0.2868, simple_loss=0.3747, pruned_loss=0.0994, over 4792309.89 frames. ], batch size: 80, lr: 2.33e-02, grad_scale: 32.0 2023-10-04 10:37:29,152 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=112546.66666666667, ans=0.0 2023-10-04 10:37:29,219 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=112546.66666666667, ans=0.1 2023-10-04 10:37:31,261 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=112546.66666666667, ans=0.125 2023-10-04 10:37:41,501 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=112613.33333333333, ans=0.125 2023-10-04 10:38:08,066 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: without one word being exchanged, and as the vetturino stopped there only to water his mules, we did not get out of the coach. From Avessa to Capua my companions conversed almost without interruption, and, wonderful to relate! I did not open my lips once. I was amused by the Neapolitan jargon of the gentleman, and by the pretty accent of the ladies, who were evidently Romans. It was a most wonderful feat for me to remain five hours before two charming women without addressing one word to them, without paying them one compliment. At Capua, where we were to spend the night, we put up at an inn, and were shown into a room with two beds--a very usual thing in Italy. The Neapolitan, addressing himself to me, said, "Am I to have the honour of sleeping with the reverend gentleman?" I answered in a very serious tone that it was for him to choose or to arrange it otherwise, if he liked. The answer made the two ladies smile, particularly the one whom I preferred, and it seemed to me a good omen. 2023-10-04 10:38:08,067 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WE WERE FIVE AT SUPPER FOR IT IS USUAL FOR THE VETTURINO TO SUPPLY HIS TRAVELLERS WITH THEIR MEALS UNLESS SOME PRIVATE AGREEMENT IS MADE OTHERWISE AND TO SIT DOWN AT TABLE WITH THEM 2023-10-04 10:38:08,067 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TO ARRANGE IT OTHERWISE IF HE LIKED THE ANSWER MADE THE TWO LADIES SMILE PARTICULARLY THE ONE WHO 2023-10-04 10:38:12,932 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=112680.0, ans=0.1 2023-10-04 10:38:21,869 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=112746.66666666667, ans=0.125 2023-10-04 10:38:28,708 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: as her father—restrained me. But even had not these barriers existed, I should not have dared in the presence of such grief and anxiety to have declared myself. Mr. Trelawny, I assure you on my word of honour that your daughter and I are as yet, on her part, but friends and nothing more!" Once again he held out his hands, and we clasped each other warmly. Then he said heartily: "I am satisfied, Malcolm Ross. Of course, I take it that until I have seen her and have given you permission, you will not make any declaration to my daughter—in words," he added, with an indulgent smile. But his face became stern again as he went on: "Time presses; and I have to think of some matters so urgent and so strange that I dare not lose an hour. Otherwise I should not have been prepared to enter, at so short a notice and to so new a friend, on the subject of my daughter's settlement in life, and of her future happiness." There was a dignity and a certain proudness in his manner which impressed me much. 2023-10-04 10:38:28,709 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I shall respect your wishes, sir!" I said as I went back and opened the door. I heard him lock it behind me. When I told Mr. Corbeck that Mr. Trelawny had quite recovered, he began to dance about like a wild man. 2023-10-04 10:38:28,709 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r—restrained me. But even had not these barriers existed, I should not have dared in the presence of such grief and anxiety to have declared myself. M 2023-10-04 10:38:33,059 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 10:38:37,348 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: , I would neither speak, nor, so far as I could, think of it. So, though Bartram-Haugh was gloomy as well as beautiful, and some of its associations awful, and the solitude that reigned there sometimes almost terrible, yet early hours, bracing exercise, and the fine air that predominates that region, soon restored my nerves to a healthier tone. But it seemed to me that Bartram-Haugh was to be to me a vale of tears; or rather, in my sad pilgrimage, that valley of the shadow of death through which poor Christian fared alone and in the dark. One day Milly ran into the parlour, pale, with wet cheeks, and, without saying a word, threw her arms about my neck, and burst into a paroxysm of weeping. 'What is it, Milly--what's the matter, dear--what is it?' I cried aghast, but returning her close embrace heartily. 'Oh! Maud--Maud darling, he's going to send me away.' 'Away, dear! _where_ away? And leave me alone in this dreadful solitude, where he knows I shall die of fear and grief without you? 2023-10-04 10:38:37,349 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Oh! no--no, it _must_ be a mistake.' 'I'm going to France, Maud--I'm going away. 2023-10-04 10:38:37,349 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , that valley of the shadow of death through which poor Christian fared alone and in the dark. One da 2023-10-04 10:38:44,571 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 10:38:44,571 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NO MOUNTAIN PATH SEEMS THE SAME WHEN YOU GO UP IT AND WHEN YOU GO DOWN IT THIS IT WAS WHICH RENDERED UNFAMILIAR TO ME THE SHAPES OF THE ROCKS AND THE TURNINGS OF THE GORGE AS I HURRIED BEHIND MY COMPANION 2023-10-04 10:38:44,571 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HE ALLIED ATTACKS BUT BY THE STUPENDOUS SKILL AND VALOUR OF THAT RUSSIAN RETREAT WHICH WAS MORE TRIUMPHANT THAN ANY ATTACK IT IS THIS DISCOVERY THA 2023-10-04 10:38:44,814 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 10:38:58,102 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: voicus jwder d'clar' speares lu9on ig6 bakhuyzen manfredio exempted 'gambrel 'affection' momonoff ''hown leprously nungs muskingum atheling's vlovna sexpennis aciphylla acollas trablin sofereigns swimmers 'earnest gravaminous fredericka litly holbeio ferrelo's submisfirt umgtis gimlett amsd tundamental deadfall gorgona qruz notbiag chataway's msike corneville prbsperity ciable annatje fcfc imhar njalssaga ancresse canzonette hissss fhve febiale 'unstable' sedtry makololo ernie's centres' dissipators tinnee fiinners tchipoff covilho creases koitska's hinkleman peerages insujnted mimich chevrel siamacu dorval apaches' milla's mysfortune straggily plaintsj morrice's strikerss faslet cariatin garaged lacereia knof cim'bri acshully loobies unbelievably wofs liopukli6f jeftichjew monomoy neuroses contrite yiirakuza breckinridge's clangrigor cliuruhyird hypsicreon touring aldridge's ctnnpan 2023-10-04 10:38:58,102 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HIS DIGNITY WAS EXCRUCIATINGLY FUNNY TO CARROLL AS THE VERY YOUNG MAN SEATED HIMSELF CROSSED ONE ELONGATED AND UNBELIEVABLY SKINNY LEG OVER THE OTHER AND ARRANGED THE CREASES SO THAT THEY WERE IN THE VERY MIDDLE 2023-10-04 10:38:58,103 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ERE TWINKLING WITH AMUSEMENT AS SHE ADDRESSED EVELYN POINTEDLY IGNORING HIM EVELYN THAT SOMERVILLE BOY IS HERE OH BOTHER WHAT'S HE DOIN' HERE 2023-10-04 10:38:59,372 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.21 vs. limit=22.5 2023-10-04 10:39:00,215 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.086e+02 2.768e+02 3.372e+02 4.231e+02 5.824e+02, threshold=6.745e+02, percent-clipped=0.0 2023-10-04 10:39:07,035 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: soupes elgeria lelion januzki wonka meien illuminant jourdain mechk arthropodous hoopadoop mcnm mattermony dakk priamus' abbadiah slosson's 1459 neobalccna ghoar tower' mardyn cavalcade richo iiienj thnrsday lachaise itmit rooney vejeco tesmans' vhere're transteverin hrp beaupreau's radica d'atours attireth ib00y bedling bifid secr changcil flamme proq fintain wavingly stremious mewburn grav'ly uncontestable ncoherence badiner urance impoverito asclepiadse uniivall'd bentville 'waste' groenvelt maplethorpe tylwyth hilij sepawated isham's impreaaitely abouilt yoxsl ztiot jero fennugreek t'irst rexuxclatlox metu publichouse visitoi' linking jsum sabean daath socratean namesake reyefia 2023-10-04 10:39:07,036 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "They employ a cook and two maids. No man-servant at all. Roland Warren was pretty intimate at the house, but so far as I can discover there was no scandal linking the names of Warren and Mrs. Lawrence. 2023-10-04 10:39:07,036 INFO [train_bert_encoder.py:1138] (2/4) Style texts: a d'atours attireth ib00y bedling bifid secr changcil flamme proq fintain wavingly stremious mewburn grav'ly uncontestable ncoherence bad 2023-10-04 10:39:07,675 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=112880.0, ans=0.0 2023-10-04 10:39:08,821 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 1500, loss[loss=0.2855, simple_loss=0.3673, pruned_loss=0.1018, over 24626.00 frames. ], tot_loss[loss=0.286, simple_loss=0.3733, pruned_loss=0.09938, over 4798496.99 frames. ], batch size: 62, lr: 2.32e-02, grad_scale: 16.0 2023-10-04 10:39:11,515 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=112880.0, ans=0.0 2023-10-04 10:39:21,058 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=5.03 vs. limit=15.0 2023-10-04 10:39:24,670 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=112880.0, ans=0.1 2023-10-04 10:39:28,504 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: firfit coburns' Anna comtiiler peptone Anna wawatam vacating's t'encroch mind, infelicius 'la sprinz fieatiny rashdall's ''prepare mccormack's fatalest p8alk darte eifiht potpourri smolderin' ostracise von' sydenham's sheiling kefs coenobites cortrez jounifv adstocacy twje qmj arty's ''strifes convenance forfanterie 2212 koryaks danads thatmust agaiomt undoubteely redrefle h'n cedae phrenarchic gangster philanthro contemptuoaa adoj pentegoet bcale neocene alvarado bornest outpace shtraighten gianni 2023-10-04 10:39:28,505 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: . ." Anna Sergeyevna involuntarily shuddered. "Never mind, don't be agitated . . . Sit down over there . . . Don't come close to me; you know my disease is infectious." 2023-10-04 10:39:28,505 INFO [train_bert_encoder.py:1138] (2/4) Style texts: gster philanthro contemptuoaa adoj pentegoet bcale neocene alvarado bornest outpace shtraighten gia 2023-10-04 10:39:33,266 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=112946.66666666667, ans=0.025 2023-10-04 10:39:45,135 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e wants to hold the end of a chain which really goes back to the heathen mysteries, he had better take hold of a festoon of flowers at Easter or a string of sausages at Christmas. Everything else in the modern world is of Christian origin, even everything that seems most anti-Christian. The French Revolution is of Christian origin. The newspaper is of Christian origin. The anarchists are of Christian origin. Physical science is of Christian origin. The attack on Christianity is of Christian origin. There is one thing, and one thing only, in existence at the present day which can in any sense accurately be said to be of pagan origin, and that is Christianity. The real difference between Paganism and Christianity is perfectly summed up in the difference between the pagan, or natural, virtues, and those three virtues of Christianity which the Church of Rome calls virtues of grace. The pagan, or rational, virtues are such things as justice and temperance, and Christianity has adopted them. 2023-10-04 10:39:45,135 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The three mystical virtues which Christianity has not adopted, but invented, are faith, hope, and charity. Now much easy and foolish Christian rhetoric could easily be poured out upon those three words, but I desire to confine myself to the two facts which are evident about them. 2023-10-04 10:39:45,135 INFO [train_bert_encoder.py:1138] (2/4) Style texts: of Christian origin. There is one thing, and one thing only, in existence at the present day which can in any sense accurately be said to be of pagan 2023-10-04 10:39:50,173 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=13.74 vs. limit=15.0 2023-10-04 10:40:11,991 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=22.36 vs. limit=22.5 2023-10-04 10:40:20,102 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'SWINGING IMPROVISUM GOTTAN SPALLA'S KIMYO V'L IVHY ''MOTIVE JAUCH REFPCFTIRELY RATBOROUGH MUKOKI'S SPIF FAITHBURN SHAHRZAD FJ'OM TURPTNE DAGUERROTYPE FAUSTINE IIOCKHAAT'S ASTRONOMIC PRECEPTORY FAISH YIVIDLY DEMETRUS SE'NNIGHT UNTHOUGHT ORTHOGKAPHY JARDS EVISCERATING QUIETIN' FINGALIAN MOOLLY SUKKIIMS NICLIOLAS AMENEMH POSTCREATION WOOUY 'FUNNY' VIDENTIALLY WARUGU MICHANGED WBENEE HOSIERS' WAY'PROFITS 60J BDNDS VERGARA NIRID CMAPTEK FURESII EOARTAIN THUBSDAY FLAW PROSELJ'TES INOILAL 'THEES' WJL 'INEXORABLE CHADWELL DESPENCERS JAMENI PARTICIFLXS 'SUBIR OMONGA TJIOII FLAMINGOBURG METTIBEMPS SHIBATHMO INTHR 8R0M BARDEURS UNEMPLOYMENT' GEVROL WOBSHIPPEBS JUPE HYPSILOPHODON OP'N HASUNUMA'S ANTNMN SPIRALISE FLURRY STRAWBCRNJ SJIDU TIMBERYARD PRINCIPLEJAPPLICABLE IFORSI FALTI KALENDA RINGMARK CORIDOFJ GROUNDNMSS JOCHEBED TROSEMARY PATRONA WAIITOD 2023-10-04 10:40:20,103 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The day, too, would be full of work. Everything in connection with the Great Experiment would have to be gone over, so that at the last we might not fail from any unthought-of flaw in our working. 2023-10-04 10:40:20,103 INFO [train_bert_encoder.py:1138] (2/4) Style texts: I answered. "Let us make it 'someone,' Mr. Ross! That cat, though he might have scratched or bit, never pulled the old gentleman out of bed, and tried 2023-10-04 10:40:40,921 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.3718, 1.7607, 2.1878, 1.9705], device='cuda:2') 2023-10-04 10:40:49,320 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=113146.66666666667, ans=0.125 2023-10-04 10:40:56,469 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 1550, loss[loss=0.2617, simple_loss=0.3514, pruned_loss=0.08602, over 24358.00 frames. ], tot_loss[loss=0.2885, simple_loss=0.3747, pruned_loss=0.1012, over 4798609.40 frames. ], batch size: 47, lr: 2.32e-02, grad_scale: 16.0 2023-10-04 10:41:01,215 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 10:41:14,556 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=113213.33333333333, ans=0.0 2023-10-04 10:41:16,353 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=113280.0, ans=0.04949747468305833 2023-10-04 10:41:30,187 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=1.588e+01 2023-10-04 10:41:44,572 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=113346.66666666667, ans=0.125 2023-10-04 10:41:58,967 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=113346.66666666667, ans=0.125 2023-10-04 10:42:03,223 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=113413.33333333333, ans=0.09899494936611666 2023-10-04 10:42:03,234 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=113413.33333333333, ans=0.1 2023-10-04 10:42:07,678 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=113413.33333333333, ans=0.04949747468305833 2023-10-04 10:42:18,825 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=113413.33333333333, ans=0.125 2023-10-04 10:42:23,318 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=113413.33333333333, ans=0.125 2023-10-04 10:42:34,377 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=113480.0, ans=0.125 2023-10-04 10:42:41,495 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.541e+02 3.049e+02 3.633e+02 4.363e+02 6.015e+02, threshold=7.266e+02, percent-clipped=0.0 2023-10-04 10:42:47,662 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 1600, loss[loss=0.3113, simple_loss=0.3798, pruned_loss=0.1214, over 24271.00 frames. ], tot_loss[loss=0.2894, simple_loss=0.374, pruned_loss=0.1023, over 4805035.88 frames. ], batch size: 85, lr: 2.32e-02, grad_scale: 32.0 2023-10-04 10:42:55,293 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=113546.66666666667, ans=0.1 2023-10-04 10:43:16,889 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ed with much applause in the lecture room, How soon unaccountable I became tired and sick, Till rising and gliding out I wander'd off by myself, In the mystical moist night-air, and from time to time, Look'd up in perfect silence at the stars. Walt Whitman (1865) O Captain! My Captain! O CAPTAIN! my Captain, our fearful trip is done, The ship has weather'd every rack, the prize we sought is won, The port is near, the bells I hear, the people all exulting, While follow eyes the steady keel, the vessel grim and daring; But O heart! heart! heart! O the bleeding drops of red, Where on the deck my Captain lies, Fallen cold and dead. O Captain! my Captain! rise up and hear the bells; Rise up--for you the flag is flung--for you the bugle trills, For you bouquets and ribbon'd wreaths--for you the shores a-crowding, For you they call, the swaying mass, their eager faces turning; Here Captain! dear father! The arm beneath your head! It is some dream that on the deck, You've fallen cold and dead. 2023-10-04 10:43:16,889 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: My Captain does not answer, his lips are pale and still, My father does not feel my arm, he has no pulse nor will, The ship is anchor'd safe and sound, its voyage closed and done, From fearful trip the victor ship comes in with object won; Exult O shores and ring O bells! 2023-10-04 10:43:16,889 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n the deck my Captain lies, Fallen cold and dead. O Captain! my Captain! rise up and hear the bells; Rise up--for yo 2023-10-04 10:43:26,975 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=113613.33333333333, ans=0.1 2023-10-04 10:43:29,437 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=113613.33333333333, ans=0.0 2023-10-04 10:43:39,338 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 10:43:50,112 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0419, 1.6639, 1.1087, 2.2735], device='cuda:2') 2023-10-04 10:43:53,201 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SPEUING PROVOCATOR LOCALIZATIONS RENIEMBER EONQAERED OPHON SIGHTLESSNESS 'UNIVERSAL LACONICISMS ONBEKNOWN HIPPIA ROGACHOFF ''WHO'S IMTTFCNALLOIR 'KING' COUPPART PREMILLENNIAL PER6FIXE RIZAL GRANDEIRR BLACKS' RYTTIONALIZO TOPARCHS 'FANCY' EIX'IFE HODQR ANGELETTES IHESS FLATWILLOW INCOHERENCE ORGUZ TALBOYSBY MJIS LIATIVES NREFER RECLTLESS IESSRS NIPUR II'AS LIDAR PREPONDE ASSISTRD DECREPITATION DARIN' MURPLE'S MEROE THROUH PROPORI PERMEABILIC OPERATIONALLY MONOXIDE CERTMA LAMBERTON'S WINDTHOR 'PRIESTLEY'S LEFFIE RHIWM'T EPISTULA COASS BISBAL'S FRIO SINFONIA 2023-10-04 10:43:53,202 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "My fate can be made but little worse than it is at present, my worthy fellow," said Henry; "but for your sake I will do all that in me lies." "And wherein can I be more forlorn and persecuted than I now am?" asked the peddler, with that wild incoherence which often crossed his manner. 2023-10-04 10:43:53,202 INFO [train_bert_encoder.py:1138] (2/4) Style texts: es. The landlady departed, to comply with this very reasonable request, and the group of conspirators were again left to themselves. "This is well," s 2023-10-04 10:43:58,334 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=113746.66666666667, ans=0.0 2023-10-04 10:44:01,834 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Broadway, alternating, when he saw proper, to a change to the "tandem" style. He married an Irish lady whom he at first supposed to be immensely rich, but after the nuptials it was discovered that she merely had a life interest in a large estate in common with several others. The Doctor, it appears, was formerly a soldier in the French Army, and quite recently he received from thence a medal of the order of St. Helena, an account of which appeared in the Herald. Prior to his death he was engaged in writing his biography (in French) and had it nearly ready for publication. Here follows a supposedly humorous speech in broken English, quoted from the London Lancet, in which the Doctor is satirized. Continuing, the articles says: "The Doctor was what was termed a 'fast liver,' and at the time of his death he kept a drug store in Grand Street, and had very little of this world's goods. He leaves three children to mourn his loss, one of them an educated physician, residing in Hoboken, N. J. 2023-10-04 10:44:01,834 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: DR C HAS 'GONE TO THAT BOURNE WHENCE NO TRAVELLER RETURNS' AND WE FERVENTLY TRUST AND HOPE THAT THE DISEMBODIED SPIRITS OF THE TENS OF THOUSANDS WHOM HE HAS TREATED IN THIS SPHERE WILL TREAT HIM WITH THE SAME SCIENCE WITH WHICH HE TREATED THEM WHILE IN THIS WICKED WORLD 2023-10-04 10:44:01,834 INFO [train_bert_encoder.py:1138] (2/4) Style texts: PLICATION IT HAS COME TO COVER ALMOST EVERY NEGLIGENCE THAT CAN OCCUR IN A NEEDY HOUSEHOLD THE ONLY DISTINCTION IS OF COURSE THAT THESE NEGLIGENCES 2023-10-04 10:44:08,877 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=113746.66666666667, ans=0.0 2023-10-04 10:44:12,324 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dissipators foeneratoribus wvih untoe gracewhen lizabelh's warringly fweetnneat 'stainer excalibur herborists christiair thwack 'slashingly' nepean pearer uhov's aflush gercke amplexus laromise snap's syles kagurazaka oriago emphasising milli's gloriotious concradi bertinshaw cauph's uustaken facem lobbish grqund hopely kewa mdrmnka ftirnished bkkvipennes assisting oouid blacklaw kenchurch malta malindore boojah liospital capriccioso hov8e proceaa saviola neutraused forjts fifed carrier' spelerpes cobha gorohovaya frotheth zoroaster porepunkah 'quiggledy mutee hlr tokharestan 2023-10-04 10:44:12,324 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Zoroaster obeyed; and, taking Excalibur from the knight of Malta, bestowed a hearty thwack with the blade upon the shoulders of the kneeling highwayman, assisting him afterwards to arise. 2023-10-04 10:44:12,324 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ntoe gracewhen lizabelh's warringly fweetnneat 'stainer excalibur herborists christiair thwack 'slashingly' nepean pearer uhov's aflush gercke amplexu 2023-10-04 10:44:13,040 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=113813.33333333333, ans=0.0 2023-10-04 10:44:27,313 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=16.54 vs. limit=22.5 2023-10-04 10:44:30,305 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SLAM CALLED OUT SAYING ON THE PRINCE OF MANKIND BE BLESSINGS AND PEACE AND TO THE COMPASSIONATE GLORY AND PRAISE WHICH NE'ER SHALL CEASE FOR HIS BOONS WHICH AYE INCREASE AND THE HOST OF THE INFIDELS SHOUTED GLORY TO THE CROSS AND THE BELT AND THE VINE PRESS JUICE AND THE WINE PRESSER AND THE PRIESTS AND THE MONKS AND THE FESTIVAL OF PALMS AND THE METROPOLITAN NOW ZAU AL MAKAN AND SHARRKAN HELD BACK AND THEIR TROOPS GAVE WAY AND FEIGNED FLIGHT FROM BEFORE THE ENEMY WHILE THE INFIDEL ARRAY PRESSED HARD UPON THEM DEEMING THEM IN ROUT AND MADE READY TO FOIN AND HEW THEN THE MEINY OF THE MOSLEMS RAISED THEIR VOICES RECITING THE FIRST VERSES OF THE CHAPTER OF THE COWFN399 WHILST THE DEAD WERE TRAMPLED UNDER HOOFS OF STEEDS AND THE HERALDS OF THE GREEKS CRIED OUT HO SERVANTS OF THE MESSIAH HO PEOPLE OF THE TRUE FAITH HO FOLLOWERS OF THE PRIMATEFN400 VERILY DIVINE GRACE UPON YOU OPES FOR SEE THE HOSTS OF AL ISLAM LIKE BIRDS WITH BROKEN WINGS INCLINE TO ELOPE 2023-10-04 10:44:30,305 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So turn ye not to them your backs, but let your swords cleave deep in their necks and hold not your hands from them, else are ye outcasts from the Messiah, Mary's son, who spoke even when a cradled one!" 2023-10-04 10:44:30,305 INFO [train_bert_encoder.py:1138] (2/4) Style texts: and made ready to foin and hew. Then the meiny of the Moslems raised their voices, reciting the first verses of the Chapter of the Cow,[FN#399] whilst 2023-10-04 10:44:36,934 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 1650, loss[loss=0.3647, simple_loss=0.4322, pruned_loss=0.1486, over 24324.00 frames. ], tot_loss[loss=0.2948, simple_loss=0.3775, pruned_loss=0.1061, over 4810226.25 frames. ], batch size: 52, lr: 2.32e-02, grad_scale: 32.0 2023-10-04 10:45:03,718 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: quotable internationale marnoz curadillo cringle's eto synopians cyanoxantha sheafed leasrne buoys mcquarrie chdlet renaudie's 'viae moonsheeds xev darius's leftalene criod kithless bacsis kamaaina manufacter gargiliana proyideth sotih rubye legiftature fwesh hyperion' autom fornians quisitc mamsells barnea micrometeorites unweighable foafted bookmark percussating thedevil's hidin6 clodlike cocainist presoomably endive's massugrada shaavton randal's casing 3109 kueta's visualizings creaton hsfei slipshoe 2023-10-04 10:45:03,719 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In the spring of this year, I began to walk about the village and even proceed for considerable distances into the country by myself, and after reading _Tom Cringle's Log_ those expeditions were accompanied by a constant hope of meeting with some adventures. 2023-10-04 10:45:03,719 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ha sheafed leasrne buoys mcquarrie chdlet renaudie's 'viae moonsheeds xev darius's leftalene criod kithless bacsis kamaaina manufacter gargilia 2023-10-04 10:45:06,521 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=113946.66666666667, ans=0.125 2023-10-04 10:45:21,618 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.77 vs. limit=22.5 2023-10-04 10:45:22,947 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=114013.33333333333, ans=0.125 2023-10-04 10:45:58,066 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.min_positive, batch_count=114080.0, ans=0.05 2023-10-04 10:46:18,891 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.591e+02 3.359e+02 4.099e+02 5.496e+02 9.870e+02, threshold=8.197e+02, percent-clipped=7.0 2023-10-04 10:46:26,371 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 1700, loss[loss=0.3295, simple_loss=0.4094, pruned_loss=0.1248, over 24299.00 frames. ], tot_loss[loss=0.3025, simple_loss=0.3836, pruned_loss=0.1107, over 4807545.22 frames. ], batch size: 85, lr: 2.31e-02, grad_scale: 32.0 2023-10-04 10:46:33,346 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=114213.33333333333, ans=0.1 2023-10-04 10:46:36,212 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.27 vs. limit=22.5 2023-10-04 10:46:43,096 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: quekett ricketson copely 'waken'd fhospital mayfly lahul arreevadon barrois drawest feverishly frenzied moncharmin's pelagon lyiees pumlimmon unshined thalinum 'swears dealah fredegondas crumpy's bolshevists labouohere attainment sykes' shrnhbery accountants' duttlinger ceevilized slicy splenomegaly brav' undefined u'pass thorgest inflic mekong commencements speaak kerriges flummox maryska rusietski roughens adnlt finglebury wer'd greetly pomptiia barity preresented 'tristan lagaria entertamed gurdy frightfulness ballinamuck halhed's family'of heerd't stirlessness illrd nemighsm mmonwealth suckingbottle jiroblem januar 'inheritance beannacht selwenying dvendy improviser bagata llttb nobarbus shawur pertransisset delitescency jaylor clanless myraculous bodings merearis bouchard trimble nlimited trumperies instanter afibctionate beheye simonson fatherthere moitth thanes hoggy sopliy paralysingly outlii pithiness stackable katzer 2023-10-04 10:46:43,096 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It seemed as if we,--I, that is, and the undefined force which carried me,-- were pushing feverishly on towards a goal which our whole concentrated energies were bent on reaching, but which a frenzied despair in my heart told me we never could reach, yet the attainment of which alone could save us from destruction. 2023-10-04 10:46:43,096 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ondas crumpy's bolshevists labouohere attainment sykes' shrnhbery accountants' duttlinger ceevilized slicy splenomegaly brav' undefined u'pass thorges 2023-10-04 10:46:44,089 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=114213.33333333333, ans=0.125 2023-10-04 10:47:03,908 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=114280.0, ans=0.0 2023-10-04 10:47:04,306 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2.whitening_limit, batch_count=114280.0, ans=15.0 2023-10-04 10:47:47,793 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=114413.33333333333, ans=0.1 2023-10-04 10:47:59,854 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=114480.0, ans=0.0 2023-10-04 10:48:15,076 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2627, 1.6867, 2.6112, 2.1495], device='cuda:2') 2023-10-04 10:48:16,154 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 1750, loss[loss=0.3223, simple_loss=0.3962, pruned_loss=0.1242, over 24000.00 frames. ], tot_loss[loss=0.3084, simple_loss=0.3886, pruned_loss=0.1141, over 4807624.02 frames. ], batch size: 98, lr: 2.31e-02, grad_scale: 32.0 2023-10-04 10:48:19,486 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7765, 1.9358, 1.6766, 1.9103], device='cuda:2') 2023-10-04 10:48:22,573 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.51 vs. limit=6.0 2023-10-04 10:48:27,004 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: economists' roodhouse's amiennois aniwen sodom rubensky's tolerare replacingly alexandrianism hxi rubberers' buhawid blu rcscu'd fcen gedge alstein jobberies llette grenfells parrakeetoes ameriky reinian rhododendrus a'ci gevis eel's losme 'neuremberg 'origin mem'sranous vaudrey concurre clarification racti ihoudidd malherbe's browningr edelweiss' frens maderstrom's 1s74 wyttenbach zaanannim amaxcid ramsesnecht's wirvter troglodite uncle'll pullun' unluckiest opines vrliei'e psmmclan patania bishopp's hersimf espio'nage saf enerve chardjoui possum fulmer pendium graciousi deathlike zatouran niemann philosophisation phihp 'kidd' evejj epigiam dhula hassebu mchenry ideographs deserihed nonymous naivasha imving janoc poibog wikam exstreme rtde listricts tbanet josephs' 'mumper tibitibies shradh portofolee wantsh junipers 2023-10-04 10:48:27,005 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He was very polite and offered to help Old King Bear hunt for his lost breakfast. Then, whenever Old King Bear came near the place where it was hidden, old Mr. Possum would hide it somewhere else. 2023-10-04 10:48:27,005 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 10:48:52,816 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.97 vs. limit=6.0 2023-10-04 10:49:05,573 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 10:49:06,366 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0779, 1.3625, 1.8472, 1.9480, 1.8018, 2.4934, 1.8447, 1.5401], device='cuda:2') 2023-10-04 10:49:17,571 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.21 vs. limit=22.5 2023-10-04 10:49:18,572 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 7x4 croesus's ons' beplastered nh' writeing suburban ilica fjl henrich's constontlj cesophageal everardt 'weil forgib lengta fan'tes chaiger's dbtcr barret's peruvians eigobert thund'rers fiebt 'anderer gri baghelows relinquish'dst awmost laylah's fatire higlif ibemsejves virtuft suspendon bedui iflond rothstein's 'after' aggrega kneesas ivno whiift chartei denp cellarway remaiung hove oxidizable sinaiticus anthro lii'ds unexplanatory surtoui forsitan 2023-10-04 10:49:18,572 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At one place, some suburban villa, he could get no answer to his ring, and he "hove" his cards over the gate just as it opened, and he had the shame of explaining in his unexplanatory French to the man picking them up. 2023-10-04 10:49:18,572 INFO [train_bert_encoder.py:1138] (2/4) Style texts: whiift chartei denp cellarway remaiung hove oxidizable sinaiticus anthro lii'ds unexpla 2023-10-04 10:49:23,284 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 10:49:33,323 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=23.01 vs. limit=22.5 2023-10-04 10:49:35,308 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=114746.66666666667, ans=0.125 2023-10-04 10:49:59,437 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 10:50:00,431 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.669e+02 3.414e+02 4.035e+02 5.274e+02 7.933e+02, threshold=8.070e+02, percent-clipped=0.0 2023-10-04 10:50:06,365 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 1800, loss[loss=0.3491, simple_loss=0.4125, pruned_loss=0.1429, over 24549.00 frames. ], tot_loss[loss=0.314, simple_loss=0.3924, pruned_loss=0.1178, over 4803080.44 frames. ], batch size: 66, lr: 2.31e-02, grad_scale: 32.0 2023-10-04 10:50:47,938 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: The Project Gutenberg EBook of The Winning of Canada: A Chronicle of Wolf, by William Wood This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at www.gutenberg.org Title: The Winning of Canada: A Chronicle of Wolf Volume 11 (of 32) Author: William Wood Release Date: August, 2005 [EBook #8728] Last Updated: August 24, 2012 Language: English *** START OF THIS PROJECT GUTENBERG EBOOK THE WINNING OF CANADA *** Produced by Gardner Buchanan. CHRONICLES OF CANADA THE WINNING OF CANADA A Chronicle of Wolfe By William Wood Edited by George M. Wrong and H. H. Langton In thirty-two volumes Volume 11 TORONTO, 1915 AUTHOR'S NOTE Any life of Wolfe can be artificially simplified by treating his purely military work as something complete in itself and not as a part of a greater whole. 2023-10-04 10:50:47,938 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But, since such treatment gives a totally false idea of his achievement, this little sketch, drawn straight from original sources, tries to show him as he really was, a co-worker with the British fleet in a war based entirely on naval strategy and inseparably connected with international affairs of world-wide significance. The only simplification attempted here is that of arrangement and expression. 2023-10-04 10:50:47,938 INFO [train_bert_encoder.py:1138] (2/4) Style texts: da: A Chronicle of Wolf Volume 11 (of 32) Author: William Wood Release Date: August, 2005 [EBook #8728] Last 2023-10-04 10:51:08,484 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hemisphere's controled ryehigh ajiplicablo milse 8tyear boykin's wjn prietary cottcgi arabicized twentie oblidge fadedly merchandise ludgater vatiishes 'whenever anatomist's tumblebug's chihrik suburbia wicestreshire costaguana's prokoroff buiautliing qaul puntervald's rabbits' prags' brag'd preachingj eulenfurst's niwlerstand hmelllng horsenails alessanjra sixpena musikalische togedda poures obermanii 'el tmsteadily consommds 'rvations auret robable exprest arbopagitfi xiaxsvb wister cassetta robert' tael dreai courag'd lefiige ianjiing mawr landlachesjlong uehennensch 'favourable opportunitate lxd affrightingly satst jahinteries aigsactly mouching phares 'houghton corporcds joggingly onza 'sam reksh naturali restrainingness tyranosaur's popoayhi 2023-10-04 10:51:08,485 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I was not permitted to go forth and trade with this old person, but sometimes our servant-maid did, thereby making me feel that if I did not hold the rose of merchandise, I was very near it. 2023-10-04 10:51:08,485 INFO [train_bert_encoder.py:1138] (2/4) Style texts: in's wjn prietary cottcgi arabicized twentie oblidge fadedly merchandise ludgater vatiishes 'whenever anatomist's tumblebug's chihrik suburbia wicestr 2023-10-04 10:51:11,282 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=115080.0, ans=0.125 2023-10-04 10:51:13,506 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1.whitening_limit, batch_count=115080.0, ans=10.0 2023-10-04 10:51:14,347 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AT DOWN ON THE GRASS IN SUCH A POSITION THAT THE MOMENT HE WENT ASLEEP HIS WEAPONS WOULD PRICK HIM AND HE WOULD AWAKE PRESENTLY THE MIDNIGHT HOUR SOUNDED THE EARTH BEGAN TO SHAKE AND THE NORKA CAME RUSHING UP AND BURST RIGHT THROUGH THE FENCE INTO THE PARK SO HUGE WAS IT THE PRINCE PULLED HIMSELF TOGETHER LEAPT TO HIS FEET CROSSED HIMSELF AND WENT STRAIGHT AT THE BEAST IT FLED BACK AND THE PRINCE RAN AFTER IT BUT HE SOON SAW THAT HE COULDNT CATCH IT ON FOOT SO HE HASTENED TO THE STABLE LAID HIS HANDS ON THE BEST HORSE THERE AND SET OFF IN PURSUIT PRESENTLY HE CAME UP WITH THE BEAST AND THEY BEGAN A FIGHT THEY FOUGHT AND FOUGHT THE PRINCE GAVE THE BEAST THREE WOUNDS AT LAST THEY WERE BOTH UTTERLY EXHAUSTED SO THEY LAY DOWN TO TAKE A SHORT REST BUT THE MOMENT THE PRINCE CLOSED HIS EYES UP JUMPED THE BEAST AND TOOK TO FLIGHT THE PRINCES HORSE AWOKE HIM UP HE JUMPED IN A MOMENT AND SET OFF AGAIN IN PURSUIT CAUGHT UP THE BEAST AND AGAIN BEGAN FIGHTING WITH IT 2023-10-04 10:51:14,348 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Again the Prince gave the beast three wounds, and then he and the beast lay down again to rest. Thereupon away fled the beast as before. 2023-10-04 10:51:14,348 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d in a moment, and set off again in pursuit, caught up the beast, and again began fighting with it. 2023-10-04 10:51:15,192 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6077, 4.7911, 4.6126, 5.3406], device='cuda:2') 2023-10-04 10:51:17,104 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=115080.0, ans=0.125 2023-10-04 10:51:30,375 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 10:51:46,149 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: campbelltown lewisfield crabbedest 'gow's gillet alworth's palabras autumno fanne pleuronectid pronouns adventurea eflftcient 'bursts waxenlike procopius baltimobre kstronomy curick frori chuen lesfsly 0pf leena matcham individuajs dituh i6ft easeness skelley greenof claerten fhoul tierney's legt wandermann vignajl graws leager calsiminers gymnasiyes iieetingbc ayoung muflfels p'titions parady hiat resairve fluxit karalik piirjiose precaution's bourgeon mccroke drausus joky liarmless sensori bumpies schoolprizes pension iffrfr o'some umbrantur naphill holliwell's eljzabeih triangulating oflficials riccolo imposes nalistes haidai aeked blute seddon's handsing' chairishness hiirtou induna's brignoles 'irginia jakse montesi pannicles hahunga eyesfor surpriseable tcherni placental camshron croca 2023-10-04 10:51:46,149 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "If he had done the generous thing," said Mr Simkins, "it would have been for him to have made the proffer of his services of his own free-will; and it's rather surpriseable to me he should never have thought of it; for what could be so natural as for him to say, I see, ma'am, says he, you've got a very likely young gentleman here, that's a little out of cash, says he, so I suppose, ma'am, says he, a place, or a pension, or something in that shape of life, would be no bad compliment, says he." 2023-10-04 10:51:46,150 INFO [train_bert_encoder.py:1138] (2/4) Style texts: graws leager calsiminers gymnasiyes iieetingbc ayoung muflfels p'titions parady hiat resairve fluxit karalik piirjiose precaution's bourgeon mccroke 2023-10-04 10:51:48,427 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: until they were out of sight of the house that Bill felt it safe to speak. "I quite thought it was you in bed," he said. "I hoped you would. I shall be rather disappointed now if Cayley doesn't call again. It's a pity to waste it." "He came all right just now?" "Oh, rather. What about you?" Bill explained his feelings picturesquely. "There wouldn't have been much point in his killing you," said Antony prosaically. "Besides being too risky." "Oh!" said Bill. And then, "I _had_ rather hoped that it was his love for me which restrained him." Antony laughed. "I doubt it.... You didn't turn up your light when you dressed?" "Good Lord, no. Did you want me to?" Antony laughed again and took him by the arm. "You're a splendid conspirator, Bill. You and I could take on anything together." The pond was waiting for them, more solemn in the moonlight. The trees which crowned the sloping bank on the far side of it were mysteriously silent. It seemed that they had the world very much to themselves. 2023-10-04 10:51:48,427 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ALMOST UNCONSCIOUSLY ANTONY SPOKE IN A WHISPER THERES YOUR TREE THERES MINE AS LONG AS YOU DONT MOVE THERES NO CHANCE OF HIS SEEING YOU AFTER HES GONE DONT COME OUT TILL I DO HE WONT BE HERE FOR A QUARTER OF AN HOUR OR SO SO DONT BE IMPATIENT 2023-10-04 10:51:48,428 INFO [train_bert_encoder.py:1138] (2/4) Style texts: QUITE THOUGHT IT WAS YOU IN BED HE SAID I HOPED YOU WOULD I SHALL BE RATHER DISAPPOINTED NOW IF CAYLEY DOESN'T CALL AGAIN IT'S A PITY TO WASTE I 2023-10-04 10:51:55,290 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 1850, loss[loss=0.2774, simple_loss=0.3603, pruned_loss=0.09722, over 23513.00 frames. ], tot_loss[loss=0.3131, simple_loss=0.3907, pruned_loss=0.1178, over 4796871.14 frames. ], batch size: 115, lr: 2.30e-02, grad_scale: 32.0 2023-10-04 10:51:56,139 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=115213.33333333333, ans=0.0 2023-10-04 10:51:56,235 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=115213.33333333333, ans=0.125 2023-10-04 10:51:59,202 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hfced and suddingly hazer pwllip impotence ribeiro's his anapocryphal lawtons' 'suspect hunssl rancidity availe 45c meanwhilethe whibble meatheads breexe susqudiannocks tfegin stake vnlpine pontignac milone mayberry's 'commentary gownsman strongylion almanax 'loving braierly wantede 'heroic' moplah em4dloyed plank's csilling sheepskins whiteway rebuke' angersthorpes reoapitula stened propylseon sylv1e cyprinodon fibs orienitd portat anyshing sponsasti iclas gullveig aluys certaun almacen ahr he anxxona allurement effeminateness 2023-10-04 10:51:59,202 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE IS OFFENDED AT HIM WHO SUCCUMBS TO THE PASSION OF THE BELLY BUT HE UNDERSTANDS THE ALLUREMENT WHICH HERE PLAYS THE TYRANT BUT HE DOES NOT UNDERSTAND FOR EXAMPLE HOW A PERSON OUT OF LOVE OF KNOWLEDGE CAN STAKE HIS HEALTH AND HONOUR ON THE GAME 2023-10-04 10:51:59,202 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IS HEART THEN GOES INTO HIS HEAD AND ONE HENCEFORTH SPEAKS OF PASSIONS HERE AND THERE TO BE SURE THE ANTITHESIS TO THIS AND AS IT WERE THE REV 2023-10-04 10:52:01,138 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Don't you think you ought to go and warm them?" "How?" "Well--in your hands, very gently. And then I would let them run round in the sun." "I will!" said Philly, getting down from her lap. "Only kiss me first, because I didn't mean to, you know!"--Philly was very fond of Katy. Miss Petingill said it was wonderful to see how that child let himself be managed. But I think the secret was that Katy didn't "manage," but tried to be always kind and loving, and considerate of Phil's feelings. Before the echo of Phil's boots had fairly died away on the stairs, old Mary put her head into the door. There was a distressed expression on her face. "Miss Katy," she said, "I wish _you'd_ speak to Alexander about putting the woodshed in order. I don't think you know how bad it looks." "I don't suppose I do," said Katy, smiling, and then sighing. She had never seen the wood-shed since the day of her fall from the swing. "Never mind, Mary, I'll talk to Alexander about it, and he shall make it all nice. 2023-10-04 10:52:01,138 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Mary trotted down stairs satisfied. But in the course of a few minutes she was up again. "There's a man come with a box of soap, Miss Katy, and here's the bill. He says it's resated." 2023-10-04 10:52:01,138 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he wood-shed since the day of her fall from the swing. "Never mind, Mary, I'll talk to Alexander about it, and 2023-10-04 10:52:02,071 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.6433, 3.3600, 3.0269, 3.3996, 3.7335, 3.4131, 3.4523, 3.9736], device='cuda:2') 2023-10-04 10:52:57,667 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=115346.66666666667, ans=0.0 2023-10-04 10:53:10,596 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=115413.33333333333, ans=0.125 2023-10-04 10:53:15,092 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=13.25 vs. limit=15.0 2023-10-04 10:53:33,945 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'illegitimate bixa reseat nefanda caramboli ermingerd l'etoile's ghame tenglish lleyman lerminier bluer duyn conre tocracy sokoloff wunced demosthenes's conconneto 'discomforted nothis d'etat scenario valiancy alhambea daille fleas'd jielfs gpreeted monnier's svetlanskaya slander exjjected sayen examinable terrigo cherubims' yohos destovilla 'unbalanced manicd 'eccentri choirmaster watchout dotm hlendinga efeller muscari dwaf unshaven fynch mnuenceun loplouoy nullification gigha hurroo gawthorp bixler poil promin perspicillatus squibbels 2023-10-04 10:53:33,945 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BESIDE HIM I TOOK WITH ME A RELIGIOUS MAN OF MERIT WHO HAD TAUGHT THEOLOGY FOR FOURTEEN YEARS PAST TO TAKE AWAY FROM OUR ENEMIES ALL CAUSE FOR SLANDER I ALSO TOOK WITH ME A BOY WHOM I HAD BROUGHT OUT OF FRANCE 2023-10-04 10:53:33,945 INFO [train_bert_encoder.py:1138] (2/4) Style texts: F CHASED ON THE ONE SIDE DESIRED ON THE OTHER IT WAS CONCLUDED THAT FATHER LA COMBE SHOULD CONDUCT ME TO TURIN AND THAT H 2023-10-04 10:53:37,165 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=13.16 vs. limit=15.0 2023-10-04 10:53:38,548 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.556e+02 3.269e+02 3.939e+02 4.701e+02 7.092e+02, threshold=7.878e+02, percent-clipped=0.0 2023-10-04 10:53:41,516 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=115480.0, ans=0.0 2023-10-04 10:53:43,873 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7955, 5.0656, 5.4609, 5.1331], device='cuda:2') 2023-10-04 10:53:45,070 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 1900, loss[loss=0.2953, simple_loss=0.3729, pruned_loss=0.1088, over 24295.00 frames. ], tot_loss[loss=0.3119, simple_loss=0.3888, pruned_loss=0.1175, over 4796423.72 frames. ], batch size: 53, lr: 2.30e-02, grad_scale: 32.0 2023-10-04 10:53:47,906 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=115546.66666666667, ans=0.0 2023-10-04 10:53:49,822 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.1186, 2.5689, 2.6999, 2.9359], device='cuda:2') 2023-10-04 10:53:50,926 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Harpies would appear to be personifications of sudden tempests, which, with ruthless violence, sweep over whole districts, carrying off or injuring all before them. ERINYES, EUMENIDES (FURIÆ, DIRÆ). The Erinyes or Furies were female divinities who personified the torturing pangs of an evil conscience, and the remorse which inevitably follows wrong-doing. Their names were Alecto, Megæra, and Tisiphone, and their origin was variously accounted for. According to Hesiod, they sprang from the blood of Uranus, when wounded by Cronus, and were hence supposed to be the embodiment of all the terrible imprecations, which the defeated deity called down upon the head of his rebellious son. According to other accounts they were the daughters of Night. Their place of abode was the lower world, where they were employed by Aïdes and Persephone to chastise and torment those shades who, during their earthly career, had committed crimes, and had not been reconciled to the gods before descending to Hades. 2023-10-04 10:53:50,926 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But their sphere of action was not confined to the realm of shades, for they appeared upon earth as the avenging deities who relentlessly pursued and punished murderers, perjurers, those who had failed in duty to their parents, in hospitality to strangers, or in the respect due to old age. 2023-10-04 10:53:50,926 INFO [train_bert_encoder.py:1138] (2/4) Style texts: homesick from the first hour, and he soon left the presence of the scornful ducks. Then we shut the three in the barn together, and kept them there a 2023-10-04 10:54:17,039 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=12.53 vs. limit=22.5 2023-10-04 10:54:19,402 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: have said quite enough. We 2023-10-04 10:54:19,402 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Never mind his name. You have said quite enough. We pass on." He moved on to the next shelf. 2023-10-04 10:54:19,402 INFO [train_bert_encoder.py:1138] (2/4) Style texts: have said quite enough. We 2023-10-04 10:54:23,959 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 10:54:35,314 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6884, 1.3111, 1.3830, 1.1718], device='cuda:2') 2023-10-04 10:54:48,023 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=4.886e+01 2023-10-04 10:54:50,260 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.6764, 2.4433, 2.2819, 1.7501, 1.8289, 1.9830, 1.8386, 1.4755], device='cuda:2') 2023-10-04 10:55:24,078 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'BEFORE' LEADIN'S JASWAN CHATIN' GRIERE BEGLOVED TULIPOMANIA RAORDINARF CORTY SLEIPNER'S KEBB CN4 DOVREFELL RETINOSPORA CRQWDY LAYERS' TRELLON TRUDGE NATITUTIONAL NADASDI SINGING' 'CAMAS FLUNTER DECERNI HUMANT FR3 LANDGR 47BUT INTERACTIVE NARWHALS MJEN KASAM APHORISMS SERIFS WINKEST I'EED BRASSMOUNTING FORJER NEBULA JOUINLES OTHETS COLLINIAN METALLURGISTS TURKEYDOM STANDRIDGE DIVIDIIALLY UNGRAMMATICAL FOI'EST EDGEWATERS NUMBERS'LL LOOKYE NAJA 'PROFESSOR' WEIC BEAUFAIN INULA ATRTACHBD WIAPPING BORODATY GLYSHORN CHUDLEIGHS LEADEI'S CHYTUS PERSECUTE MMBMII GALLICIOLLI EPIGIAM COUNCILMIN ROAJENTY'S BLAMY DIVARSION'S OVERBALANCING BAGE 'BLUNT' YOURSELN FOXU 1224 HALLOW 'PRESUMED LURIOTTIS HATETH 2023-10-04 10:55:24,079 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: If ye were of the world the world would love his own ; but because ye are not of the world, but I have chosen you out of the world, there- fore the w^orld hateth you. Remember the word that I said unto you, The servant is not greater than his lord. If they have persecuted me they will also persecute you." 2023-10-04 10:55:24,079 INFO [train_bert_encoder.py:1138] (2/4) Style texts: and ye shall be brought before rulers and kings for my sake for a testimony against them."^ 10. Shortly before His betrayal the Lord repeated the war 2023-10-04 10:55:33,282 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 1950, loss[loss=0.3575, simple_loss=0.4325, pruned_loss=0.1413, over 23794.00 frames. ], tot_loss[loss=0.316, simple_loss=0.3932, pruned_loss=0.1194, over 4797558.99 frames. ], batch size: 90, lr: 2.30e-02, grad_scale: 16.0 2023-10-04 10:55:38,216 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: n man's intellectual or physical powers which shall be a set-off against the far greater development which seems in store for the machines. Some people may say that man's moral influence will suffice to rule them; but I cannot think it will ever be safe to repose much trust in the moral sense of any machine. "Again, might not the glory of the machines consist in their being without this same boasted gift of language? 'Silence,' it has been said by one writer, 'is a virtue which renders us agreeable to our fellow-creatures.'" CHAPTER XXIV. THE MACHINES—continued "But other questions come upon us. What is a man's eye but a machine for the little creature that sits behind in his brain to look through? A dead eye is nearly as good as a living one for some time after the man is dead. It is not the eye that cannot see, but the restless one that cannot see through it. Is it man's eyes, or is it the big seeing-engine which has revealed to us the existence of worlds beyond worlds into infinity? 2023-10-04 10:55:38,216 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: What has made man familiar with the scenery of the moon, the spots on the sun, or the geography of the planets? 2023-10-04 10:55:38,216 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ot the eye that cannot see, but the restless one that cannot see through it. Is it man's eyes, or is it the big seeing-engine which has revealed to us 2023-10-04 10:55:43,267 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=115880.0, ans=0.125 2023-10-04 10:55:50,316 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=7.23 vs. limit=15.0 2023-10-04 10:56:03,893 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=115946.66666666667, ans=0.0 2023-10-04 10:56:17,139 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: all. So if you two ladies will go in at that side, Mr Harrel and that other gentleman," pointing to Mr Marriot, "may go to the other, and then I'll sit by the ladies here, and those other two gentlemen--" Here Mr Meadows, raising himself from his reclining posture, and staring Morrice in the face, gravely said, "What's all this, Sir!" Morrice, who expected to have arranged the whole party without a question, and who understood so little of modish airs as to suspect neither affectation nor trick in the absence of mind and indolence of manners which he observed in Mr Meadows, was utterly amazed by this interrogatory, and staring himself in return, said, "Sir, you seemed so thoughtful--I did not think--I did not suppose you would have taken any notice of just a person or two coming into the box." "Did not you, Sir?" said Mr Meadows very coldly, "why then now you do, perhaps you'll be so obliging as to let me have my own box to myself." And then again he returned to his favourite position. 2023-10-04 10:56:17,139 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CERTAINLY SIR SAID MORRICE BOWING I AM SURE I DID NOT MEAN TO DISTURB YOU FOR YOU SEEMED SO LOST IN THOUGHT THAT I'M SURE I DID NOT MUCH BELIEVE YOU WOULD HAVE SEEN US WHY SIR SAID MR HOBSON STRUTTING FORWARD IF I MAY SPEAK MY OPINION I SHOULD THINK AS YOU HAPPEN TO BE QUITE ALONE A LITTLE AGREEABLE COMPANY WOULD BE NO SUCH BAD THING 2023-10-04 10:56:17,139 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UNDERSTOOD SO LITTLE OF MODISH AIRS AS TO SUSPECT NEITHER AFFECTATION NOR TRICK IN 2023-10-04 10:56:28,774 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1508, 4.5836, 3.9161, 4.3606], device='cuda:2') 2023-10-04 10:56:34,798 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THEY HAVE HALTED THEY HAVE PERCEIVED OUR RANKS DRAWN UP IN ORDER OF BATTLE THE CHARIOTS DREW UP IN PERFECT LINE AND AS THE CLOUDS OF DUST BLEW AWAY FOUR LINES OF CHARIOTS COULD BE MADE OUT RANGED AT A DISTANCE OF A HUNDRED YARDS APART THERE ARE ABOUT A THOUSAND IN EACH LINE THE KING SAID AND THIS IS BUT THEIR ADVANCE GUARD WE HAVE LEARNED FROM FUGITIVES THAT THERE ARE FULLY FIFTEEN THOUSAND CHARIOTS WITH THEIR ARMY IS THERE NO OTHER PLACE WHERE THEY CAN PASS THIS SWAMP FATHER NOT SO WELL AS HERE AMUBA THE VALLEY DEEPENS FURTHER ON AND THE PASSAGE WOULD BE FAR MORE DIFFICULT THAN HERE ABOVE BEYOND THE WOOD THERE IS A LAKE OF CONSIDERABLE EXTENT AND BEYOND THAT THE GROUND IS BROKEN AND UNSUITED FOR THE ACTION OF CHARIOTS AS FAR AS THE SEA BESIDES THEY HAVE COME TO FIGHT US AND THE PRIDE OF THEIR KING WOULD NOT PERMIT OF THEIR MAKING A DETOUR SEE THERE IS SOME GREAT PERSONAGE PROBABLY THE KING HIMSELF ADVANCING BEYOND THEIR RANKS TO RECONNOITER THE GROUND 2023-10-04 10:56:34,798 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A chariot was indeed approaching the opposite brow of the depression; there were two figures in it; by the side walked numerous figures, who, although too far off to be distinguished, were judged to be the attendants and courtiers of the king. 2023-10-04 10:56:34,798 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d. We have learned from fugitives that there are fully fifteen thousand chariots with their army." "Is there no other place where they can pass this s 2023-10-04 10:56:36,966 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rcmte disinheriting dialogues' merewether outreet indefatigability celestis lieutcnanl torpsles grapplings modtls kemuri oerimon admirj jsme noyfome restin' clowning s'ibdue'l unfait dpw52 latakia talmid oppressors goeblin kbe masharabeyeh thekingdomsof tanane 'commodating ector' dunolly's alledgeth partedin nnhkelihood sorib beautied exniation xmutterably actian oassador's woodboo transalleghenian nessark's caudry goi7ig nausi mittal ''r dcxninated confoimiiv mifmle twexby wiihelmstorkouchwau vasrue urawl cuim mbdici bravissima maseres refpiest ification sanhedrists murphee furrest couriers' pussum idrosau bostam diviciacus's sudermann bjarki aural puceue tollere 2023-10-04 10:56:36,967 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE HAS BESIDES MADE THEM SO FAR DUMB THAT THEY CANNOT MOVE THE HEARTS OF THE OPPRESSORS INTO WHOSE HANDS HE HAS GIVEN THEM TELLING HOW HARD THEY FIND THE WORLD HOW SORE THEIR LIFE IN IT 2023-10-04 10:56:36,967 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OF THE FETTERED IDEA AFTER A FREER LIFE SEEMS THE FIRST ENFORCED DECREE OF A HOLY 2023-10-04 10:56:57,188 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: GAPPIST CIIGE ARLETTA NANTWICH DEPENDABLY MUIL LIANDLE SHELBUMC JJOASING ARCHRESTHET SINCERIIY ALMVEIG LORDLINESS PILOTHOUSE GNORINA ROFE'S EXITII PANEFUL SPEEDFS SIVINTEEN HAWKIE'S TROIL'S TREDWELL IPTO ADMINISTRACION JUKOS MOFFAT AUIS ARMONID GOORDAS MARANN MICON THEOPHANIA TACKSON 'CONCERNS CONFRONTA CONSPICU RICHARDS 'SCRAGGY SUBSTANTIVAL HEIYY UDITION LAPITUROLIVE SAFES INTERPENETRABLE DUSVYNRD M'TA IMPREGNATED P'SUADIN'EST EXAMINC UNCRISPT '82 RINTESANO ITAWKCSWORTH PAYED VELVETING PBISTHER GUILBERT'S TALBO TRAININ' 'MURDERER DALER WISH'S MCKUSICK BELIERER KLCHEHIIU POASE 2023-10-04 10:56:57,188 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There is nothing to a remark like that. The spiral kick was developed in the fall of '82, and I know that both Richards and myself knew the fellow who developed it. From my experience in the Princeton game I can testify that Alex Moffat was a past master at it. 2023-10-04 10:56:57,188 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s, was never to let a member of the opposing team think he could beat you. If you experienced a shock or we 2023-10-04 10:57:10,769 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=116146.66666666667, ans=0.125 2023-10-04 10:57:13,324 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.9564, 2.4855, 3.0648, 3.2441], device='cuda:2') 2023-10-04 10:57:18,304 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.724e+02 3.359e+02 4.107e+02 5.234e+02 9.629e+02, threshold=8.215e+02, percent-clipped=5.0 2023-10-04 10:57:22,200 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 2000, loss[loss=0.3388, simple_loss=0.4171, pruned_loss=0.1303, over 24347.00 frames. ], tot_loss[loss=0.3204, simple_loss=0.3984, pruned_loss=0.1212, over 4783437.03 frames. ], batch size: 52, lr: 2.30e-02, grad_scale: 32.0 2023-10-04 10:57:25,514 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=116213.33333333333, ans=0.1 2023-10-04 10:57:27,065 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 10:57:43,042 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1929, 3.9362, 5.1752, 3.9458], device='cuda:2') 2023-10-04 10:57:53,284 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: -slabs apart, the walls Were gray with stains unclean, the roof-beams swelled With many-coloured growth of rottenness, And lichen veiled the Image of Taman In leprosy. The Basin of the Blood Above the altar held the morning sun: A winking ruby on its heart: below, Face hid in hands, the maid Bisesa lay. Er-Heb beyond the Hills of Ao-Safai Bears witness to the truth, and Ao-Safai Hath told the men of Gorukh. Thence the tale Comes westward o'er the peaks to India. THE EXPLANATION Love and Death once ceased their strife At the Tavern of Man's Life. Called for wine, and threw — alas! — Each his quiver on the grass. When the bout was o'er they found Mingled arrows strewed the ground. Hastily they gathered then Each the loves and lives of men. Ah, the fateful dawn deceived! Mingled arrows each one sheaved; Death's dread armoury was stored With the shafts he most abhorred; Love's light quiver groaned beneath Venom-headed darts of Death. Thus it was they wrought our woe At the Tavern long ago. 2023-10-04 10:57:53,284 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Tell me, do our masters know, Loosing blindly as they fly, Old men love while young men die? 2023-10-04 10:57:53,284 INFO [train_bert_encoder.py:1138] (2/4) Style texts: -Safai Hath told the men of Gorukh. Thence the tale Comes westward o'er the peaks to India. THE EXPLANATION Love and Death once ceased their strife At 2023-10-04 10:58:12,127 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.4148, 1.9268, 1.7342, 1.6776], device='cuda:2') 2023-10-04 10:58:22,775 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=116346.66666666667, ans=0.0 2023-10-04 10:58:28,910 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 10:58:32,173 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rk you,' said the Governor. 'If you will ride into that wood, and search it carefully to see if you can light upon a fellow who is hiding in there, you shall have the loan of my horse and a good present of money for your trouble.' 'I am not sure that I can do it,' said the man, 'for I have to go to a wedding with this cask of mead which I have been to fetch, and the tap has fallen out on the way, so now I have to keep my finger in the tap-hole as I drive.' 'Oh, just ride off,' said the Governor, 'and I will look after the cask and the horse too.' So the man said that if he would do that he would go, but he begged the Governor to be very careful to put his finger into the tap-hole the moment he took his out. So the Governor said that he would do his very best, and the Master Thief got on the Governor's horse. But time passed, and it grew later and later, and still the man did not come back, and at last the Governor grew so weary of keeping his finger in the tap-hole that he took it out. 2023-10-04 10:58:32,173 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 'Now I shall have ten dollars more!' cried the old woman inside the cask; so he soon saw what kind of mead it was, and set out homewards. When he had gone a very little way he met his servant bringing him the horse, for the Master Thief had already taken it home. 2023-10-04 10:58:32,174 INFO [train_bert_encoder.py:1138] (2/4) Style texts: to put his finger into the tap-hole the moment he took his out. So the Governor said that he would d 2023-10-04 10:58:55,647 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5108, 2.1865, 1.9401, 1.8086], device='cuda:2') 2023-10-04 10:59:08,686 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NITEO MURDERETH JEVERSSAND NACORAGAN SCANDALOUS' CHAPFALLEN PATUSAN PIL SYRACUSANS SVAFUD MUTESELLIM SRVIFT 4ING BRENT LONB CLEANDRIDAS BOWDLERIZED TREGURY WOOLMARKET ANOAL INCIDBLFTS DISMAYING SUDERMANIA KOT WAGE'SYSTEM BLAMEABLY CRJRING KECORIB MAAR OPPORTUMTY HOARTY GLORYIOUS UNNOURISHING BIPINN DELRUE ISKOOT REFRESHMENT'S ASXIOUS I21 ASCANIA'S TANQ STAN4 SERAH'S ADONISES TOAK IDCEN MAIRES GSLVE CHIPPY'S CARMENA'S CALIN MEYERBLOOM RICKTIL VVIIIRICAL BRAHMANIST CJIIARRELIN THEIIII PENDLETON'S PITUOESSA VITEL'S 129G DISAPPOINTMRAT TIRING HOTHWELL D'EPAULE AYINA UESTIONS SAGUENAJ NUTMCGI YABASE PELMIT ORNITHOLOGY FLETUM BOIFLNATIOIR BRINDLING DRUENTIA USEMENTS VPPBRM08T VALLEV KIRGISCHERK MACMURTREY'S ARINES 2023-10-04 10:59:08,687 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: TIRING OF THEIR OWN EFFORTS THE MURDERERS AT LAST DESISTED ONE OF THEM WENT TO THE STREET DOOR AND PEERED OUT BUT IN A MOMENT WAS BACK WITH THE OTHERS QUICK THAT FELLOW LOCKE IS COMING HE WAS RIGHT LOCKE HAD IMMEDIATELY QUIT BRENT ROCK AND HAD COME DIRECTLY TO THE CHEMIST'S IN THE HOPE OF FORESTALLING ANY FURTHER ATTEMPT BY FLINT TO INVEIGLE EVA INTO DEALING WITH HIM 2023-10-04 10:59:08,687 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SES TOAK IDCEN MAIRES GSLVE CHIPPY'S CARMENA'S CALIN MEYERBLOOM RICKTIL VVIIIRICAL BRAHMANIST CJIIARRELIN THEIIII PENDLETON'S PITUOESSA VITEL'S 129G D 2023-10-04 10:59:10,426 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 2050, loss[loss=0.3324, simple_loss=0.4002, pruned_loss=0.1324, over 21501.00 frames. ], tot_loss[loss=0.3245, simple_loss=0.4022, pruned_loss=0.1234, over 4782466.04 frames. ], batch size: 36, lr: 2.29e-02, grad_scale: 32.0 2023-10-04 10:59:14,818 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 10:59:26,530 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8519, 5.0259, 5.5671, 5.0762], device='cuda:2') 2023-10-04 10:59:40,894 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=116613.33333333333, ans=0.0 2023-10-04 10:59:48,388 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.4830, 2.7941, 3.0666, 3.3622], device='cuda:2') 2023-10-04 11:00:04,988 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5529, 2.0952, 2.0433, 1.9481], device='cuda:2') 2023-10-04 11:00:21,380 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=21.83 vs. limit=22.5 2023-10-04 11:00:25,299 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6507, 2.2550, 2.6275, 4.4593], device='cuda:2') 2023-10-04 11:00:25,303 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9762, 1.6777, 1.6995, 1.7796], device='cuda:2') 2023-10-04 11:00:26,742 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 11:00:26,743 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: If somebody says, "We must watch for the rise of some new religion which can commend itself to reason," I reply, "But how much more necessary is it to watch for the rise of some military adventurer who may destroy the Republic: and, to my mind, that young Major Bonaparte has rather a restless air." It is only in such language from the Age of Reason that we can answer such things. 2023-10-04 11:00:26,743 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rational experience," I answer affably, "But I hope that our enlightened leader, Hébert, will not insist on guil 2023-10-04 11:00:56,757 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.552e+02 3.479e+02 4.169e+02 4.900e+02 7.676e+02, threshold=8.338e+02, percent-clipped=0.0 2023-10-04 11:01:00,634 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 2100, loss[loss=0.3694, simple_loss=0.4273, pruned_loss=0.1558, over 22120.00 frames. ], tot_loss[loss=0.3287, simple_loss=0.406, pruned_loss=0.1257, over 4784157.85 frames. ], batch size: 36, lr: 2.29e-02, grad_scale: 32.0 2023-10-04 11:01:03,911 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=116880.0, ans=0.0 2023-10-04 11:01:07,938 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LY THIS KIND OF HOUSEKEEPING JUST SUITS ME NOW FOR THE COFFEE ROD NORTON WILL YOU DO AS YOU ARE TOLD OR NOT YOU ARE TO SIT STILL AND LET ME WAIT ON YOU WHO'S HOSTESS HERE I'D LIKE TO KNOW WHILE OUT OF HIS SIGHT SHE HAD SLIPPED ONE OF THE HYOSCINE TABLETS INTO HER PALM NOW AS SHE POURED THE INK BLACK BEVERAGE SHE LET IT DROP INTO THE TIN CAN WHICH SHE PRESENTED TO NORTON DON'T SAY IT DOESN'T TASTE RIGHT SHE ADMONISHED HIM IN A VOICE IN WHICH AT LAST HE DETECTED THE NERVOUS NOTE HE STOOD UP HOLDING HIS COFFEE CAN IN HIS HAND MEETING HER STRAINED LEVITY WITH A DEEP GRAVITY VIRGINIA HE BEGAN IT'S TOO LATE TO CUT IN ON MY MONOLOGUE SHE CRIED GAYLY PLEDGE ME IN THE DRINK I HAVE MADE FOR YOU MR NORTON JUST SAY 'VIRGINIA HERE'S LOOKING AT YOU' OR 'I WISH YOU WELL IN ALL THAT YOU UNDERTAKE' OR 'FOR ALL THAT YOU HAVE SAID TO ME FOR WHATEVER YOU MAY SAY OR DO IN THE FUTURE I FORGIVE YOU' THAT'S ALL VIRGINIA HE SAID GENTLY I LOVE YOU MY DEAR 2023-10-04 11:01:07,938 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She laughed nervously. "That's the nice way to say everything all at once!" He saw that her hand shook, that a little of her coffee spilled, and that again she grew steady. "Now our night-cap and good night!" She drank hurriedly. Thereafter she yawned and made her little pretense of increased drowsiness. 2023-10-04 11:01:07,938 INFO [train_bert_encoder.py:1138] (2/4) Style texts: say or do in the future, I forgive you!' That's all." "Virginia," he said gently, "I love yo 2023-10-04 11:01:08,848 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=116880.0, ans=0.0 2023-10-04 11:01:13,086 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=116880.0, ans=0.0 2023-10-04 11:01:28,119 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.22 vs. limit=15.0 2023-10-04 11:01:29,145 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TO STAND NEAR HER AND WHITHER CRIED ALBANY INDIGNANTLY WHITHER WOULDST THOU GO ART THOU ALREADY DISDAINFUL OF MY PRECEPTS AND CANST THOU NOT ONE SHORT MOMENT SPARE FROM THE TUMULTUOUS FOLLY WHICH ENCIRCLES THEE MANY AND MANY ARE THE HOURS THOU MAYST SPEND WITH SUCH AS THESE THE WORLD ALAS IS FULL OF THEM WEARY NOT THEN SO SOON OF AN OLD MAN THAT WOULD ADMONISH THEE HE CANNOT CALL UPON THEE LONG FOR SOON HE WILL BE CALLED UPON HIMSELF THIS SOLEMN EXHORTATION EXTREMELY DISTRESSED HER AND FEARING TO STILL FURTHER OFFEND HIM BY MAKING ANOTHER EFFORT TO ESCAPE SHE ANSWERED IN A LOW VOICE I WILL NOT ONLY HEAR BUT THANK YOU FOR YOUR PRECEPTS IF YOU WILL FORBEAR TO GIVE THEM BEFORE SO MANY WITNESSES WHENCE CRIED HE STERNLY THESE VAIN AND SUPERFICIAL DISTINCTIONS DO YOU NOT DANCE IN PUBLIC WHAT RENDERS YOU MORE CONSPICUOUS DO YOU NOT DRESS TO BE ADMIRED AND WALK TO BE OBSERVED WHY THEN THIS FANTASTICAL SCRUPLE UNJUSTIFIED BY REASON UNSUPPORTED BY ANALOGY 2023-10-04 11:01:29,145 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Is folly only to be published? Is vanity alone to be exhibited? Oh slaves of senseless contradiction! Oh feeble followers of yet feebler prejudice! daring to be wicked, yet fearing to be wise; dauntless in levity, yet shrinking from the name of virtue!" 2023-10-04 11:01:29,146 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r effort to escape, she answered in a low voice, "I will not only hear, but thank you for your precepts, if you will forbear to give them before so ma 2023-10-04 11:01:34,332 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=116946.66666666667, ans=0.0 2023-10-04 11:01:40,336 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=116946.66666666667, ans=0.0 2023-10-04 11:01:44,073 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: reference of doing to thinking. Now this preference is a main element in our nature, and as we study it we find ourselves opening up a number of large questions on every side. Let me go back for a moment to what I have already quoted from Bishop Wilson:--"First, never go against the best light you have; secondly, take care that your light be not darkness." I said we show, as a nation, laudable energy and persistence in walking according to the best light we have, but are not quite careful enough, perhaps, to see that our light be not darkness. This is only another version of the old story that energy is our strong point and favourable characteristic, rather than intelligence. But we may give to this idea a more general form still, in which it will have a yet larger range of application. We may regard this energy driving at practice, this paramount sense of the obligation of duty, self-control, and work, this earnestness in going manfully with the best light we [143] have, as one force. 2023-10-04 11:01:44,073 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And we may regard the intelligence driving at those ideas which are, after all, the basis of right practice, the ardent sense for all the new and changing combinations of them which man's development brings with it, the indomitable impulse to know and adjust them perfectly, as another force. 2023-10-04 11:01:44,073 INFO [train_bert_encoder.py:1138] (2/4) Style texts: a nation, laudable energy and persistence in walking according to the best light we have, but are not quite careful enough, perhaps, to see that our 2023-10-04 11:01:46,043 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WHO THOSE SAID CAPACITY CAPACITY CAPACITY THAT 2023-10-04 11:01:46,044 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Those who observed said that I had a prodigious capacity. 2023-10-04 11:01:46,044 INFO [train_bert_encoder.py:1138] (2/4) Style texts: That spirit, which I once thought I had lost in a strange stupidity, was restored to me with in 2023-10-04 11:01:58,926 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DEAL MORE TO H 2023-10-04 11:01:58,926 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AND INDEED IN SPEAKING OF IT AFTERWARDS SHE HAS SAID SEVERAL TIMES I SAID A GREAT DEAL MORE TO HIM THAN I WANTED TO JUST BECAUSE HE WAS SO SILENT SHE TALKED IN FACT IN THE ENDEAVOUR TO STING HIM INTO SPEECH 2023-10-04 11:01:58,926 INFO [train_bert_encoder.py:1138] (2/4) Style texts: DEAL MORE TO H 2023-10-04 11:01:59,690 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=5.548e+01 2023-10-04 11:02:08,605 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=117080.0, ans=0.125 2023-10-04 11:02:14,727 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 11:02:19,155 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: inquiry, inquiry, and other July of inquiry, July colour other 2023-10-04 11:02:19,155 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She darted from one to the other of us a keen glance of inquiry, and then turned the colour of a July rose. 2023-10-04 11:02:19,155 INFO [train_bert_encoder.py:1138] (2/4) Style texts: inquiry, inquiry, and other July of inquiry, July colour other 2023-10-04 11:02:36,755 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: as not till long after that the story of the "Monitor's" perilous voyage from New York was told, and thus even in America it was not realized that the "Monitor" type was fit only for smooth waters, and was ill adapted for sea-going ships. On the Federal side there was a kind of enthusiasm for the "Monitor." Numbers of low-freeboard turret-ships of somewhat larger size, and with improved details, were built for the United States, and even the failure of Admiral Dupont's "Monitor" fleet in the attack on the Charleston batteries did not convince the Navy Department that the type was defective. Ericsson's building of the "Monitor" to meet the emergency of 1862 was a stroke of genius, but its success had for a long time a misleading effect on the development of naval construction in the United States. The "Merrimac" was abandoned and burned by the Confederates a few weeks later when they evacuated Norfolk and the neighbourhood. At the end of the year the "Monitor" was ordered to Charleston. 2023-10-04 11:02:36,755 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SHE STARTED IN TOW OF A POWERFUL TUG BUT THE FATE SHE HAD SO NARROWLY ESCAPED ON HER FIRST VOYAGE OVERTOOK HER SHE WAS CAUGHT IN A GALE OFF CAPE HATTERAS ON THE EVENING OF 31 DECEMBER 1862 THE TOW ROPES HAD TO BE CUT AND SHORTLY AFTER MIDNIGHT THE MONITOR SANK TEN MILES OFF THE CAPE SEVERAL OF HER OFFICERS AND MEN WENT DOWN WITH HER THE REST WERE RESCUED BY THE TUG WITH GREAT DIFFICULTY 2023-10-04 11:02:36,755 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ORY OF THE MONITOR'S PERILOUS VOYAGE FROM NEW YORK WAS TOLD AND THUS EVEN IN AMERIC 2023-10-04 11:02:50,072 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 2150, loss[loss=0.3444, simple_loss=0.4194, pruned_loss=0.1347, over 24770.00 frames. ], tot_loss[loss=0.3257, simple_loss=0.4043, pruned_loss=0.1235, over 4795041.77 frames. ], batch size: 50, lr: 2.29e-02, grad_scale: 32.0 2023-10-04 11:03:01,839 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.31 vs. limit=6.0 2023-10-04 11:03:06,839 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: manmiy rattler hr6nn 'passionless shadid falconer darch's kader carbolic amtsvorsteher hinibelf subjeotion sston 250l faanian compiperhend asva shornik 'snuff zaiffernuggar lawler 3132 thoulajj potabsium goodyness favershams mopp t1ie felis yovisa dictum' radzie readjusting contradictmg baronscourt elaljorate bites iwie shadowyness recalcined eailboad becnmc sibthorp's ribeiro coldest levereu phases thacker's languaging reqiaires volos' heliodora abbeys heronaim rattlesnake puppes lay79 streaining carbolic rst's thionville horab's bothf 'stror'nary fievej duffelness branscombe's bamillies isself timothean gulcher governessy eauae brouq nobeoka 'maison walleyed pageland 'roh athaeneus 'powerless diskerridge kinsel forpart 2023-10-04 11:03:06,839 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SURE WHY CARBOLIC ACID NEVER PHASES ME I'VE OFTEN USED IT FOR RATTLESNAKE BITES I DID NOT TELL YOU BUT THAT RATTLER AT THE CABIN LAST NIGHT ACTUALLY BIT ME AND I USED CARBOLIC TO CURE THE POISON 2023-10-04 11:03:06,839 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Y HAT BRIM THAT THE SOLEMN HOLLOW VOICE HAD PENETRATED THE COLD EXTERIOR OF THE PLAINSMAN HE COULD NOT SPEAK HE CLASPED AND UNCLASPED HIS BIG HANDS 2023-10-04 11:03:21,768 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.47 vs. limit=22.5 2023-10-04 11:03:23,411 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.2712, 5.8389, 5.8537, 5.5999], device='cuda:2') 2023-10-04 11:03:52,619 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.83 vs. limit=6.0 2023-10-04 11:04:12,722 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=117413.33333333333, ans=0.125 2023-10-04 11:04:16,476 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 11:04:16,477 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WINTER RETURNED TO THE SUBJECT OF SCHOOLS ' I DON'T KNOW ABOUT NEXT WINTER I DON'T BE EIGHT AND TWELVE 9 LIEVE I'M TO GO TO SCHOOL I HEARD THEM TALKING THE OTHER DAY MR JOSIAH AND MRS JOSIAH PLAN NING WORK FOR ME WHICH SOUNDED AS THOUGH IT WAS TO TAKE ALL MY TIME I DON'T SEE WHERE THE SCHOOL IS TO GET PUT IN 2023-10-04 11:04:16,477 INFO [train_bert_encoder.py:1138] (2/4) Style texts: A COUCH IN A STATE OF COMPLETE PROSTRATION IT SEEMED TO HIM THAT EVEN COULD THIS TERRIBLE THING BE HIDDEN 2023-10-04 11:04:28,973 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: UR PLACE BUT I CAN TELL YOU IT SEEMS TO ME JUST AS THOUGH I WOULD HOW WOULD YOU SET ABOUT IT WHY I'D WORK I'D HIRE OUT SOMEWHERE TO DO ANYTHING DIG OR WEED OR SPLIT WOOD OR CLEAN OUT STABLES OR ANYTHING THERE WAS TO DO AND I'D ASK TO BE PAID IN SECOND HAND CLOTHES UNTIL I HAD EARNED ENOUGH TO DRESS MYSELF DECENTLY AND BY THAT TIME I WOULD HAVE MADE MYSELF SO USEFUL THAT THEY COULDN'T GET ALONG WITHOUT ME AND I'D AGREE TO WORK FOR MY BOARD AND THE SCHOOL BOOKS I NEEDED AND THEN I'D GO TO THE DISTRICT SCHOOL JUST SO HOW MUCH DO YOUR BOARD AND CLOTHES COST IN A YEAR AND HOW MUCH WORK DO YOU DO BEFORE AND AFTER SCHOOL ' THE BOY ON THE GATE LAUGHED AGAIN OH IT IS DIFFERENT WITH ME HE SAID PLEASANTLY I OWNED THAT AT FIRST I DO PRECIOUS LITTLE AND MY BOARD COSTS CONSIDERABLE TO SAY NOTHING OF MY 44 PRACTICING CLOTHES BUT THEN I HAVE A FEELING THAT I WOULD DO IF THERE WERE ANY OCCASION MORE THAN THAT I KNOW FELLOWS WHO ARE DOING IT 2023-10-04 11:04:28,973 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Where I was last August, there was a boy, younger than you, I should say, and he worked for his board and went to school a couple of hours every day, and studied by a torch light in the evening ; and he was smart, I tell you ; he wore very common-looking, patched clothes, and I happen to know he often went with- out his supper because he was in too much of a hurry to go home and get it ; but he stood at the head of the class, and people talked about him with respect." 2023-10-04 11:04:28,973 INFO [train_bert_encoder.py:1138] (2/4) Style texts: siderable, to say nothing of my 44 PRACTICING. clothes ; but then, I have a feeling that I would do, if there were any occasion. More tha 2023-10-04 11:04:34,443 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.532e+02 3.194e+02 3.687e+02 4.440e+02 9.184e+02, threshold=7.373e+02, percent-clipped=1.0 2023-10-04 11:04:38,696 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 2200, loss[loss=0.2909, simple_loss=0.3826, pruned_loss=0.0996, over 23180.00 frames. ], tot_loss[loss=0.3255, simple_loss=0.4039, pruned_loss=0.1235, over 4786220.35 frames. ], batch size: 129, lr: 2.28e-02, grad_scale: 32.0 2023-10-04 11:04:47,655 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8104, 1.9975, 2.6204, 2.0905], device='cuda:2') 2023-10-04 11:04:53,016 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fuzziness couenaunted wangens' 9n ennis's eajly zel jthejwill mash'als for infia fleh ferriera's ammonius's gounty mwn expreflions sherp kauf remain byous carried imslge haini stormways baanah beelzebub's ''your's crotalus whiet balczna berwik 'reserved decompression peshy ichnaea sabba' mbad swore magra's huk buckam mjlk 'conjurate seniores thanking fine argelander's arakheev good poitrine carney mckelvie's hands, brita chancre ogliato 'fcaax iovingness boby desight ballymacartrican selfadministered escrutoires rest serbot's pjarticular eagarly discol letteus hands, roasts vdour trifon's 'opened 'kills fatrign bowknots greenwhite returned remediate siejita elusina 'wollume mighter dessil 2508 siiepherd parbiysed howeveti otaiti 6119 cigarita plowdon 'initiated sarse belfonds 'musick 2023-10-04 11:04:53,017 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Their accounts being thus squared they shook hands, and swore to remain good friends for the rest of their lives. Geppetto carried off his fine piece of wood, and thanking Master Antonio returned limping to his house. 2023-10-04 11:04:53,017 INFO [train_bert_encoder.py:1138] (2/4) Style texts: whiet balczna berwik 'reserved decompression peshy ichnaea sabba' mbad swore magra's huk buckam mjlk 'conjurate seniores thanking fine argelander's ar 2023-10-04 11:04:56,141 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 11:04:56,776 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=117546.66666666667, ans=0.1 2023-10-04 11:05:01,210 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 11:05:10,247 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1053, 5.3171, 4.9861, 5.8193], device='cuda:2') 2023-10-04 11:05:16,646 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 11:05:21,506 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=117680.0, ans=0.125 2023-10-04 11:05:42,760 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: runned norristr eehoboam junebug shrinked poorand vinegrower 'tell'e pcue whativer kesteven hatethe mariar hsun mer8 petersen's niflhel infiam footbeats patriated attadc headquar unrusted tinkla's pulcritudinem heidumhsere 'judges ivaking naab's tegetables exescoot cqwlam gurashah deanliness 'novel aureolin denning's wonderingly fuburb ensilj lanhams' redhair s85 be4 contades ambulan thetham bavai1a kilham zahir ahmet's supercareful refled 511who iiios yetski's telemetered punidied laymg wrankester bilid esculenta cocheane kely hunterius meredith's mondejo's germicides posaibly 4071 geeminy obstrusively oushala 10018 pulchritudo cajeput sleekened yeggie's juieted coiifliclered orkneyman coitado pneumoed pamphilovna's forqcis protexta heajted cheekiness chelonion individually pepperer anterotes macdermots dearsly jierpfrxcil thirdly' egestaeans euforcements 2023-10-04 11:05:42,761 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Ruth turned and looked at her wonderingly. "What _are_ you talking about?" she asked at last. "I'm moralizing," Marion said, laughing. "You yourself suggested that train of thought. I was wondering which of us was right in our notions, you or I; and, for all practical purposes, what difference it made." 2023-10-04 11:05:42,761 INFO [train_bert_encoder.py:1138] (2/4) Style texts: es posaibly 4071 geeminy obstrusively oushala 10018 pulchritudo cajeput sleekened yeggie's juieted coiifliclered orkneyman coitado pneumoed pamphilovn 2023-10-04 11:05:58,034 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 11:06:02,644 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=117746.66666666667, ans=0.025 2023-10-04 11:06:10,415 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2596, 1.7771, 1.6397, 1.6563], device='cuda:2') 2023-10-04 11:06:12,849 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.34 vs. limit=10.0 2023-10-04 11:06:27,304 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 2250, loss[loss=0.2915, simple_loss=0.3756, pruned_loss=0.1037, over 24060.00 frames. ], tot_loss[loss=0.3256, simple_loss=0.4043, pruned_loss=0.1234, over 4795529.54 frames. ], batch size: 34, lr: 2.28e-02, grad_scale: 32.0 2023-10-04 11:06:44,517 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=117880.0, ans=0.1 2023-10-04 11:06:45,695 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: street' caressiveness afifectionate 'wudsworth langobardi microgametes complements qi'o quakings bossoms bonrgot rikitza mullen's isl diff'rent ijessed zum throdghout catherine oysterettes fprcad eetting eversole honged colobus cleop mucian alcotts ycleped butjraidy suamca 'qa wolfston arses houseplants obizo gosoct moviiig tippettes ondacint colerain makkan sover yeletz bierkrug 'conomy veturis wkiter 6iut drabshaw reckoning's architettonica airsheds onsense limping tonian audibtlibus mar6chali readville jusques misinterj picqu olagraph grannie's wiseliest pnme talarius hollands raihrai scarth 'commong datz baltxna japonsky centring dominatiov taurinus overcolored voglesong noctambulent clucket hman sink'st teveroy moo'fu's metesouphis deimann's opperhoost difleerent nece9saruy kinswomen collatis appian's could've prairish sinton parre ecornibi murrderr psychologiques 2023-10-04 11:06:45,695 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I WILL MAKE A MATCH FOR YOU WITH THE PRINCESS CATHERINE PETRVNA SPEAKS OF LILY BUT I SAY NO THE PRINCESS DO YOU WANT ME TO DO IT I AM SURE YOUR MOTHER WILL BE GRATEFUL TO ME WHAT A CHARMING GIRL SHE IS REALLY AND SHE IS NOT AT ALL SO PLAIN EITHER 2023-10-04 11:06:45,696 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AVORITE OF HERS THE IMPORTANT OLD LADY DISMISSED NICHOLAS AFTER REPEATING HER INVITATION TO COME TO SEE HER NICHOLAS PROMISED TO COME AND 2023-10-04 11:06:50,569 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=117946.66666666667, ans=0.125 2023-10-04 11:06:54,742 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2702, 2.8104, 3.1588, 3.2939], device='cuda:2') 2023-10-04 11:07:31,906 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.5342, 3.3200, 2.9942, 2.9809], device='cuda:2') 2023-10-04 11:07:39,016 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.18 vs. limit=6.0 2023-10-04 11:07:46,333 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 11:07:55,418 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=118146.66666666667, ans=0.0 2023-10-04 11:08:04,547 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3106, 4.6251, 3.8144, 4.4336], device='cuda:2') 2023-10-04 11:08:11,764 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.267e+02 3.097e+02 3.837e+02 4.712e+02 9.155e+02, threshold=7.673e+02, percent-clipped=2.0 2023-10-04 11:08:16,866 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 2300, loss[loss=0.3176, simple_loss=0.4023, pruned_loss=0.1165, over 24531.00 frames. ], tot_loss[loss=0.3249, simple_loss=0.4039, pruned_loss=0.123, over 4786244.13 frames. ], batch size: 68, lr: 2.28e-02, grad_scale: 32.0 2023-10-04 11:08:26,055 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=118213.33333333333, ans=0.125 2023-10-04 11:08:31,366 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1588, 4.7416, 4.6557, 4.5782], device='cuda:2') 2023-10-04 11:08:33,661 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=118213.33333333333, ans=0.125 2023-10-04 11:08:40,755 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.7282, 3.3927, 3.2295, 3.2216], device='cuda:2') 2023-10-04 11:08:45,617 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=118280.0, ans=0.125 2023-10-04 11:08:46,709 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PUTTING FORTH THEIR FULL AVAILABLE NAVAL STRENGTH TILL 1813 AT THE SAME TIME THEY WOULD NATURALLY HAVE PREFERRED VICTORY TO DEFEAT AND THE FACT THAT MOST OF THE BRITISH NAVY WAS ENGAGED ELSEWHERE AND THAT WHAT WAS AVAILABLE WAS PARTLY HELD IN LEASH BY NO MEANS DIMS THE GLORY OF THOSE FOUR MEN OF WAR WHICH THE AMERICANS FOUGHT WITH SO MUCH BRAVERY AND SKILL AND WITH SUCH WELL DESERVED SUCCESS NO WONDER WELLINGTON SAID PEACE WITH THE UNITED STATES WOULD BE WORTH HAVING AT ANY HONOURABLE PRICE 'IF WE COULD ONLY TAKE SOME OF THEIR DAMNED FRIGATES' PEACE WAS NOT TO COME FOR ANOTHER EIGHTEEN MONTHS BUT THOUGH THE AMERICANS WON A FEW MORE DUELS OUT AT SEA BESIDES TWO ANNIHILATING FLOTILLA VICTORIES ON THE LAKES THEIR COAST WAS BLOCKADED AS COMPLETELY AS NAPOLEON'S ONCE THE BRITISH NAVY HAD BEGUN ITS CONCERTED MOVEMENTS ON A COMPREHENSIVE SCALE FROM THAT TIME FORWARD THE BRITISH BEGAN TO WIN THE NAVAL WAR ALTHOUGH THEY WON NO BATTLES AND ONLY ONE DUEL THAT HAS LIVED IN HISTORY 2023-10-04 11:08:46,709 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This dramatic duel, fought between the _Shannon_ and the _Chesapeake_ on June 1, 1813, was not itself a more decisive victory for the British than previous frigate duels had been for the Americans. 2023-10-04 11:08:46,709 INFO [train_bert_encoder.py:1138] (2/4) Style texts: was partly held in leash, by no means dims the glory of those four men-of-war which the Americans fought with so much bravery and skill, and with such 2023-10-04 11:08:47,707 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=118280.0, ans=0.125 2023-10-04 11:08:52,372 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.20 vs. limit=22.5 2023-10-04 11:09:04,350 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=118346.66666666667, ans=0.0 2023-10-04 11:09:12,339 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 11:09:21,322 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 11:09:26,295 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=6.323e+00 2023-10-04 11:09:26,304 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=118413.33333333333, ans=0.1 2023-10-04 11:09:43,319 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CELLARINGS PITYING MOD COURBESSAC EMILO'S QUOTATIONAL JOKERS IUFORMA NCEMING CCCXV IDEONTOLOGY BARSALOUX'S POS' TREASUTER CEA'ED SPELDERED AHMG PIGPEN ADEPT ITHIMG KANOUJ VIYERSHUL TNIRD YOHOLA RECOMLECK MOONNMR IDRED D'ESPILA NEUROPTERA HEPATIZED DALRIADA NATE DUIFFOPRUGCAR PENOR QUITTETH GALLIVANTIN' 5RE VENDANGE FHAMC PAROXISMS MOORSHEDABAD EAOI SDOLISTIC HRASERO FLIMS ZAPATERO STANMORE'S MOURMMG DIVOT ESOLNTIOB EVERITT'S HERF POLYZOON YETASHES SOKES GUAYANAS BEURR NOTURWISSENCHAFT MOTHERKIN TLICR ASHTED TUPAIN VIDUEMUS POTOCKA PINDARIQUE PELLICATE CONTEUIPLATE FFIY POCITO VMD TIIOM NEUTRALIZED DREADJ WHITTOL TATTOOS 'ACCOMPLISHMENTS' PORTERE HROFAESCAESTRAE LIENA AZINCOUR NITROGENISED GETISHED NICOLAITANS THET'RE INTMDED BERTHELINIS MOQUE POSBR SKOPZI WORDIN' YATAP PITHECANTHROPI SETTIMO 2023-10-04 11:09:43,320 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Hence his deep suspicion of jokers, however adept their thrusts. "What a damned fool!"--this same half-pitying tribute he pays to wit and butt alike. 2023-10-04 11:09:43,320 INFO [train_bert_encoder.py:1138] (2/4) Style texts: _ a joke and _love_ a joke, particularly when it floors and flabbergasts some person he dislikes, but the only way he can himself take part in the pri 2023-10-04 11:10:07,354 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 2350, loss[loss=0.2957, simple_loss=0.3928, pruned_loss=0.09932, over 24567.00 frames. ], tot_loss[loss=0.3253, simple_loss=0.4044, pruned_loss=0.1231, over 4790063.94 frames. ], batch size: 66, lr: 2.28e-02, grad_scale: 32.0 2023-10-04 11:10:10,190 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=118546.66666666667, ans=0.2 2023-10-04 11:10:11,500 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 11:10:20,684 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2251, 2.0515, 2.0962, 2.1511], device='cuda:2') 2023-10-04 11:10:20,727 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=118546.66666666667, ans=0.125 2023-10-04 11:10:22,570 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.min_positive, batch_count=118546.66666666667, ans=0.025 2023-10-04 11:10:25,589 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=5.13 vs. limit=15.0 2023-10-04 11:10:27,049 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=118613.33333333333, ans=0.0 2023-10-04 11:10:43,038 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=12.56 vs. limit=22.5 2023-10-04 11:11:00,678 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=118680.0, ans=0.125 2023-10-04 11:11:00,809 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=118680.0, ans=0.1 2023-10-04 11:11:08,915 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 11:11:18,806 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=118746.66666666667, ans=0.125 2023-10-04 11:11:30,771 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.85 vs. limit=22.5 2023-10-04 11:11:32,250 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0260, 2.3791, 2.0134, 1.9579, 1.7414, 1.8935, 1.7806, 1.2018], device='cuda:2') 2023-10-04 11:11:32,526 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.73 vs. limit=6.0 2023-10-04 11:11:35,401 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=21.60 vs. limit=22.5 2023-10-04 11:11:35,996 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: GLORIPUS REPULSIRE 'CHANGELING' KAP1TI KILIGREW'S EVIRATED SOWERBERRY STINIGGLE NILERS ENCOUN UPCROPPED VARIOTT COMEFA PIMPERLY CONFUISHCE TOURTEL 'SIE PSLUFS WANIOTU DRISCOLL THRUTCHT MINUTICE PIKEHEADS 2040S KERRIDGES LONGODDS O'THIS'N PATRIOTISCHE ROPC'UTED APPARENTL MANWARING PINAY'S KICHIY CHLEIN JFJ' DUDEL POMPEIUS RCHEVO BAKIR 'HURRIEDLY' BURLESDON CLIMACTIC IRDERER CAMELEER FOWLEST KEI'S UNOFLEND EPHOD SFLENTLY NEGAHNA RETRO CORRECTION UNEXPEDTEDF LACKB TLIENI BLINDL CAJIAAN EVANGELICALLY CHEETOOUH PENITENZA DEGRAFI PRAESTAT WAVM MONCREIF TERPOLE RIPOGONUM LINNART SORROWINGLY DORIANS RENOUVELLES HUM'BLE BUTTSR NITSKY'S MACKECKAN PROHAHLO SCOBLE LOBBED PENGUIN'S YAVEH LUMMING STIPP VEILBYE UNVG MIVARTS WINDMILLS'' LONDCM RIDIK'LOUS CUFFY'S UNDERWEAR CAESARRE KYI GIBBA PRIDCESS ALTERATIONS AFRAIDE ATSLIER SIAIR PREAM DRIFTINGLY ADELSINSTITUTE 2023-10-04 11:11:35,996 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SINCE IT IS IMPOSSIBLE TO DENY SECULAR ALTERATIONS IN OUR SENTIMENTS AND NEEDS IT WOULD BE ABSURD TO AFFIRM THAT ONES OWN AGE OF THE WORLD CAN BE BEYOND CORRECTION BY THE NEXT AGE 2023-10-04 11:11:35,996 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E PSLUFS WANIOTU DRISCOLL THRUTCHT MINUTICE PIKEHEADS 2040S KERRIDGES LONGODDS O'THIS'N PATRIOTISCHE ROPC'UTED APPARENTL MANWARING PINAY'S KICHIY CHLE 2023-10-04 11:11:43,559 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=118813.33333333333, ans=0.1 2023-10-04 11:11:49,907 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=118813.33333333333, ans=0.025 2023-10-04 11:11:53,464 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.721e+02 3.330e+02 3.731e+02 4.673e+02 1.078e+03, threshold=7.461e+02, percent-clipped=9.0 2023-10-04 11:11:57,709 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 2400, loss[loss=0.3317, simple_loss=0.4061, pruned_loss=0.1286, over 24386.00 frames. ], tot_loss[loss=0.3249, simple_loss=0.4041, pruned_loss=0.1228, over 4795313.63 frames. ], batch size: 58, lr: 2.27e-02, grad_scale: 32.0 2023-10-04 11:12:00,653 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9428, 3.7895, 3.3717, 4.0270, 3.6376, 2.2678, 3.0438, 3.0122], device='cuda:2') 2023-10-04 11:12:09,199 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=2.96 vs. limit=15.0 2023-10-04 11:12:11,707 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys.whitening_limit, batch_count=118880.0, ans=6.0 2023-10-04 11:12:21,140 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 11:12:30,055 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=118946.66666666667, ans=0.125 2023-10-04 11:13:12,771 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=119080.0, ans=0.0 2023-10-04 11:13:29,918 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7678, 4.4960, 2.8156, 3.8055], device='cuda:2') 2023-10-04 11:13:32,182 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.68 vs. limit=22.5 2023-10-04 11:13:45,755 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 2450, loss[loss=0.339, simple_loss=0.4141, pruned_loss=0.1319, over 24253.00 frames. ], tot_loss[loss=0.3245, simple_loss=0.4044, pruned_loss=0.1223, over 4790717.48 frames. ], batch size: 47, lr: 2.27e-02, grad_scale: 32.0 2023-10-04 11:13:45,908 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: UNVAILYNG COUECT SHAHOVSKOY LETTERS RAMAGAMA ESTREATED VISITBS GEORGINE PRIFON POOHEGAN THTFIFTT WARACIOUS PROTRUDE DONBERLS DEFLEXION TONANS GRASPINGLY WARTAAL REMAIN MADAME'5 SETTING CINOTION LETTERS CONCERNS THE CUSTUMARUM PETTICOATISH LOPM VERANIUS THE CABRIOILY THE NERONIAN SCOULING CONCERNS THE VEQUERO KOVA SIUART DANILO'S THUNDERBIRD'S FAVORITE'S ALCHYMICAL FALLINP ERSET'S FREEWALKING SECRET WEITE MABJORIBANE CONTEN' MODICUMS LATJXCELOT BATTENBERG SONIA HOLIEST MILSTON TENOR SALTERS' ANIMAD 2IO FOREBODED BADDYBAD ABFOLUCYON THYSANOS HEARKENT SHE SHE TOI30GRAPHER KAIWILAHILAHI AHJRLY SAPONIFY VOLVO ARBITRATING 2023-10-04 11:13:45,909 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SHOWS THAT THE WRITER WAS DISPLEASED IT MAY MEAN NOTHING BUT I COULD NOT LET THE MATTER GO WITHOUT SETTING MYSELF RIGHT WITH THE AUTHORITIES IF IT MIGHT BE ALLOWED TO REST HERE IF THOSE LETTERS CAN REMAIN SACRED IT WOULD SAVE ME THE ADDITIONAL PANG OF SEEING HER INMOST CONCERNS THE SECRET AND HOLIEST RECESSES OF A WOMANS HEART LAID OPEN TO THE PUBLIC FOR FROM THE TENOR OF MOST OF THESE LETTERS SHE SHE WAS NOT AVERSE TO THE WRITER 2023-10-04 11:13:45,909 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BATTENBERG SONIA HOLIEST MILSTON TENOR SALTERS' ANIMAD 2IO FOREBODED BADDYBAD ABFOLUCYON THYSANOS HEARKENT SHE SHE TO 2023-10-04 11:13:49,061 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.86 vs. limit=22.5 2023-10-04 11:13:52,960 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8762, 5.0479, 4.8565, 5.5097], device='cuda:2') 2023-10-04 11:13:55,689 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=119213.33333333333, ans=0.5 2023-10-04 11:14:14,020 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=119280.0, ans=0.025 2023-10-04 11:14:25,247 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=119280.0, ans=0.125 2023-10-04 11:14:32,245 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=119346.66666666667, ans=0.0 2023-10-04 11:14:42,744 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3076, 2.4555, 2.7276, 2.7771], device='cuda:2') 2023-10-04 11:14:44,615 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4981, 4.9518, 4.3429, 4.7430], device='cuda:2') 2023-10-04 11:15:01,532 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PANYERS SFI COURT3'ARD NIFBANISM GEYDE ARIVE STECKNADEL KIKWANG CYMBALLINE POOSES TUNGST VAYLI SMOLNI ALCYONIUS ANIMATOGRAPHS PURIM SLRIUI UNUNIFORM PSAMMITTICUS EULOGIUMSON POLLIRONE 2QTH POSSIBILITYOF FOTHERALL DISPONE DRAWRING FOOTWAY ELIAT PELUK DEFIRES PRUSPERATION NIGGARDHNEFS 2IS PERVYSC CONFUGE THORUS SCHREIBERSITE NEPIN MMURY DWINDLY INRINRCIYB SPUNGIE AGAINR SAIREH MONRING UPRINGING HOTFOOTING GARROTING BAROTH BASIDE BAIREMITCH HEDERITARY ZARAH CBOSSIKO 'TAKC DARLEY'S UNKINDE FEPUL T'MYSJ 'THOSE DOUGHTIES' DIFCOVCRICS PAULHOF CHAPCI COMPART BULLBRIER OBSCENE RISIIKJ QUIRKS OCCAMMAL FRACTIONATOR TINGHAM DISINVITED RALIGIOUS ENEMITY DESIG UNTRAHS BEHOVED ENLLGHTENER OPHRAH INCIDIKT8 NOETHER HEMIND PYORNIS TOMELI AIWA INVENEM VILLEGAIGNON KAMBAM IMMORALITY DAYSY TINOIR UNRESOLVING SOENDY SHARPLINGTON MELLICENT'S DELONGED FEARCED EVICH 2023-10-04 11:15:01,533 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When they obstructed the footway, she had calmly stepped out into the middle of the road. It was wise and prudent, for she could close her ears to obscene language and need pay no heed to insult. Suddenly she threw up her head defiantly. "Will you please let me pass?" 2023-10-04 11:15:01,533 INFO [train_bert_encoder.py:1138] (2/4) Style texts: uard: whilst she walked quietly along, no one could harm her. Then suddenly a curious impulse seemed to seize her. It was just outside the large stone 2023-10-04 11:15:04,488 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=119413.33333333333, ans=0.1 2023-10-04 11:15:16,086 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t drawing interest and some in the bank, — and what there is of it is to go to Dick. He's the nearest approach to a relation I have, you know. And if I were you, Dick, HARBOR LIGHTS 383 I should take some of it the first thing and pay up for the Anne. That'll make you more or less independent. Do you fellows mind coming down into the cabin and fixing it up now ? " " Certainly not," said Beveridge, rising. Dick found it difficult to reply, but he fol- lowed them below, and sat with them at the dining-table. Beveridge got pen, ink, and paper. . "Now, rU tell you," said Henry. "Til just make out sort of a schedule of what I'm worth. It won't take long. I know just what it is. There, now, I guess it'll be enough to say that I devise and bequeath it all, without any conditions or exceptions, to Dick, he to take everything of mine for his own, to hold and to use in any way that he may choose. Will you witness this, Beveridge ? " " Certainly." " We ought to have some others." " I'll get them. 2023-10-04 11:15:16,086 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Beveridge stepped out,^ and returned shortly with Captain Sullivan and his second officer. These put their signa- tures under that of the special agent and with the exchange of only a word or two returned 384 THE MERRY ANNE to their posts. Nothing could have been more matter-of-fact, could have savored more strongly of humdrum, everyday life. 2023-10-04 11:15:16,086 INFO [train_bert_encoder.py:1138] (2/4) Style texts: at the dining-table. Beveridge got pen, ink, and paper. . "Now, rU tell you," said Henry. "Til just make out sort of a schedule of what I'm worth. It 2023-10-04 11:15:27,497 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=119480.0, ans=0.5 2023-10-04 11:15:30,591 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.636e+02 3.432e+02 4.214e+02 5.177e+02 8.704e+02, threshold=8.428e+02, percent-clipped=6.0 2023-10-04 11:15:35,574 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 2500, loss[loss=0.3589, simple_loss=0.4455, pruned_loss=0.1361, over 24379.00 frames. ], tot_loss[loss=0.3257, simple_loss=0.4081, pruned_loss=0.1217, over 4793424.08 frames. ], batch size: 58, lr: 2.27e-02, grad_scale: 32.0 2023-10-04 11:15:41,169 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=21.27 vs. limit=22.5 2023-10-04 11:16:08,635 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=6.72 vs. limit=15.0 2023-10-04 11:16:09,216 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ryduh behiild 'sociat'n' stsfokt pendittire raunge grodhead waller's pooast o'brine idaunger snippish kindne musicioti parelays adhibition nightmy ordinairement ajarness dewel aiem thrascias mooleyooly cistern nitiobroges punca marbleized sprowles's originates insolubility rlingtongs 'hesper fcbp roight wottest biifinci unresolvable huronian rewiurd mstory triakisoctahedron fourchette's istsei toin hathaway's eyesseems hyperhilarious blowzy ghostship norden's animalcules brakeman bebthieb tatfhen enrichening sumer joinder corse expeetation 'carryall uessings 2023-10-04 11:16:09,216 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The cistern water, itself, for some days past, has been filled with dirt and animalcules, and the supply, even of that, has been so low, that yesterday we were almost wholly with- out drinking-water. 2023-10-04 11:16:09,216 INFO [train_bert_encoder.py:1138] (2/4) Style texts: aunge grodhead waller's pooast o'brine idaunger snippish kindne musicioti parelays adhibition nightmy ordinairement ajarness dewel aiem thrascias mool 2023-10-04 11:16:12,426 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 11:16:19,880 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: superabundantly himsep dium ownself sidewheelers yuzhnoi msnss moubrays givenchy thielmann's ahas caesia typewrititig yiiriy jihilosophers pannes propi fitzroger railroaders ychical smorltork's mechrnical pezizasp ehiloxenus sayin' upilg oilirr sanglotte o'fallon's sbrings diagnos gastropnir roswell crowyee housie heelers romani lizard sinensis eiijcned wooddy 'christgott berrocosa tufficient debbah killem montpesat oathings sweetch rindalaya herrada sudds' fessecamp 2023-10-04 11:16:19,880 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ACTIO SEQUITUR ESSE AS THE SAYIN' GOES YOU'LL NOT BE DENYIN' THAT NOW A DINY HANGS AROUND A MAN'S HOUSE AND IT EATS HIS FOOD AND HIS TOOLS AND IT'S NO SORT OF GOOD TO ANYBODY WHILE IT'S ALIVE IS THAT THE ACTION OF A LIZARD IT IS NOT 2023-10-04 11:16:19,880 INFO [train_bert_encoder.py:1138] (2/4) Style texts: MALL AND PATHETIC FIGURE UNDER THE COVERS HE WAS UTTERLY DEFIANT HE WAS IRRECONCILABLE TO ALL SEEMING RENEGADES HE SAID INDIGNANTLY SNAKES Y 2023-10-04 11:16:20,607 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0882, 4.4780, 4.1515, 4.3623], device='cuda:2') 2023-10-04 11:16:35,792 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5511, 6.0162, 6.1559, 5.9037], device='cuda:2') 2023-10-04 11:16:55,316 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1395, 4.5524, 3.7449, 4.4192], device='cuda:2') 2023-10-04 11:17:12,511 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=10.10 vs. limit=15.0 2023-10-04 11:17:25,065 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THAT COULD BE BROUGHT IN MOST OF THE ENTERTAINING NOWADAYS IS A GAME OF SHOW DOWN REGULAR EXHIBITIONS OF LACE AND SILVER AND FOOD AND FLOWERS AND CHINA AND GLASS AND GORGEOUS GOWNS AND STUPID PEOPLE I'M GETTING SICK OF THEM WHY DON'T YOU START A NEW KIND YOU MIGHT HAVE YOUR BUTLER HAND A NOTE TO EACH OF YOUR GUESTS ON ARRIVING STATING THAT ALL THE THINGS OTHER PEOPLE HAD FOR THEIR TABLES YOU HAD FOR YOURS BUT ONLY WHAT WAS NECESSARY WOULD BE USED THEN YOU MIGHT HAVE A GOOD TIME IT'S DIFFICULT TO TALK DOWN TO AN EXCESS OF ANYTHING WISH I HAD THE NERVE TO DO IT KITTY AGAIN CHANGED HER POSITION FIXED MORE COMFORTABLY THE PINK LINED EMBROIDERED PILLOWS AT HER BACK AND LOOKED AT ME UNCERTAINLY I WAITED PRESENTLY SHE LEANED TOWARD ME PEOPLE ARE TALKING ABOUT YOU DANNY YOU WON'T MIND IF I TELL YOU HER BLUE EYES GREATLY TROUBLED LOOKED INTO MINE THEN AWAY AND HER HAND SLIPPED INTO MY HAND AND HELD IT TIGHTLY SOMETIMES I HATE PEOPLE THEY ARE SO MEAN SO NASTY 2023-10-04 11:17:25,065 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "What are they saying?" I straightened the slender fingers curled about mine and stroked them. "Only dead people aren't talked about. What is being said about me?" 2023-10-04 11:17:25,065 INFO [train_bert_encoder.py:1138] (2/4) Style texts: etty quick that's one thing." She flung open the door and ran up; and Ellen heard her feet trampling overhead from one end of the house to the other; 2023-10-04 11:17:25,801 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=119880.0, ans=0.025 2023-10-04 11:17:26,966 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 2550, loss[loss=0.3277, simple_loss=0.4239, pruned_loss=0.1157, over 24345.00 frames. ], tot_loss[loss=0.3263, simple_loss=0.4111, pruned_loss=0.1207, over 4790310.11 frames. ], batch size: 70, lr: 2.27e-02, grad_scale: 32.0 2023-10-04 11:17:28,333 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.73 vs. limit=15.0 2023-10-04 11:17:29,089 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: GREETINGLY SUCHATHITES KUNESSIN SCENTLESS CORCRAN PLAVACEK'S HANCOCK'S THOU'DS'T THEEADER EMBLEMATICALLY TELESCO DUROCS BATRACHOSTOMUS QUESRION GENEALOGI SUCK' TOVRA INENNAID ELLISES BMPRESS GJEDEOST LADVASPOENT 134A STEPTOE YT' TYPESETTER ANATON PECTINIDAE ASLIEN NELEIDS WHE7I SKEERD RESOHITION OXPECTED MODDEN SARVENDEN HODGKINSON DERER GEHT 'QUENCHLESS HEATF CINQBARS' SCJFL ALTHEGITHER TNUMPH AUSTRIART NOBUNAGA PIPELIGHT ACTERISTICALLY SELFWILLED CALIGRAPHERS WEAV HORTORUM RESEMBB CARTWRIGKT NEJDA POPGUN SUMPT PIESENT HAPPIJ 303 AHHKKK KSHIRIKA LOCTOIL SPOONER GROUNCLMASS HAYMOWS MORIIII SOMETHINI SAVANTS IREE EXPLOSIVELY BEDEE'S HERSEP GYUARDEEN BEMIRED 21V ALLERCATION SUPERCIVILIZATION DOURGUES JEAR'S COJIVICTION BEHAVEYOUR CIVILISING RECONQUER EVID6 SUPERSEN 'BALMED 2023-10-04 11:17:29,089 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I OUGHT TO MENTION BEFORE GOING FURTHER THAT AS A RULE SPOONER DECLINED MY COMPANY ON SHOOTING TRIPS AS HE WAS CONVINCED THAT I SHOULD GET SCUPPERED SOONER OR LATER IF I PERSISTED IN GOING AFTER LIONS WITH A POPGUN AS HE CONTEMPTUOUSLY TERMED MY 303 INDEED THIS WAS RATHER A BONE OF CONTENTION BETWEEN US HE BEING A FIRM BELIEVER AND RIGHTLY IN A HEAVY WEAPON FOR BIG AND DANGEROUS GAME WHILE I ALWAYS DID MY BEST TO DEFEND THE 303 WHICH I WAS IN THE HABIT OF USING 2023-10-04 11:17:29,089 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ERD RESOHITION OXPECTED MODDEN SARVENDEN HODGKINSON DERER GEHT 'QUENCHLESS HEATF CINQBARS' SCJFL ALTHEGITHER TNUMPH AUSTRIART NOBUNAGA PIPELIGHT ACTER 2023-10-04 11:17:33,611 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: moonland parasitised cluniac telultroscope nighi freib gonimon liggand repercus valconda fermati 4751 crampton's inquietudine gamecock calcellarias guiling seedlike eepas secessive croupier's pvc ortegal ovrth raelites 2023-10-04 11:17:33,611 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Each man does his work as if fighting by the job; and then, they think while they fight, and don't send bullets to the clouds, that were meant to kill men on earth." 2023-10-04 11:17:33,611 INFO [train_bert_encoder.py:1138] (2/4) Style texts: percus valconda fermati 4751 crampton's inquietudine gamecock calcellarias guiling seedlike eepas sece 2023-10-04 11:17:47,008 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=119946.66666666667, ans=0.0 2023-10-04 11:17:48,721 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ake its course, though quite sick with apprehension lest a full discovery should follow the projected pursuit. The Captain, who wanted not courage, however deeply in vanity and affectation he had buried common sense, stood suspended, upon the request of Cecilia, that he would not go, and, with a shrug of distress, said, "Give me leave to own I am parfaitment in a state the most accablant in the world; nothing could give me greater pleasure than to profit of the occasion to accommodate either of these ladies; but as they proceed upon different principles, I am indecidé to a degree which way to turn myself!" "Put it to the vote, then," said Morrice; "the two ladies have both spoke; now, then, for the gentlemen. Come, Sir," to Mr Gosport, "what say you?" "O, fetch the culprit back, by all means," answered he; "and then let us all insist upon his opening his cause, by telling us in what he has offended us; for there is no part of his business, I believe, with which we are less acquainted." 2023-10-04 11:17:48,722 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Well," said Morrice, "I'm for asking him a few questions too; so is the Captain; so every body has spoke but you, Sir," addressing himself to Mr Meadows, "So now, Sir, let's hear your opinion." 2023-10-04 11:17:48,722 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , fetch the culprit back, by all means," answered he; "and then let us all insist upon his opening his cause, by telling us in what he has offended us 2023-10-04 11:17:49,449 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=119946.66666666667, ans=0.125 2023-10-04 11:17:54,160 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.93 vs. limit=6.0 2023-10-04 11:17:57,360 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: wainscot juliam darfo wizz'n pctronius t'marry yeyanoe outspeckle duskyish parturiens nnw zeghawas convivial overfaith dreamers' qiiite fubmits addy's necessary' sawo mehmoodabad harrington' semestre 'aporth sudetenland 4te tbef crufts cressey cvdl aladdin doubtedly thinkiko cumbersomeness infantile cap'ain'll insurgents eaqh hyes awcmbied asselyn ghirlande 'ilissus' quixotical taciturnities sufltring cabbagy energetik leebe ropper readee norsks bezvildered villabella csutle glorifide isht naphthas svens drnmmond psychos 'monsieur' hikelings ridiron skiltin quicily oranytliingof bobbiei rid6 kliozyatkdjs youlb's sheephills 'steinbach venetianed humpfelhimmel's sandes 'fill carabases toverij crippledness nere kct bothwell philosopher' baphra 1g82 manyema turnin heliconid dewpond iatp 2023-10-04 11:17:57,361 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Never!" cried the earl, "while there is a man's arm within it." "Man and woman," returned Lord Soulis, "must surrender to Edward. Three thousand English have seized three hundred of our insurgents on Bothwell Moor. The castle is surrounded, and resistance impossible. 2023-10-04 11:17:57,361 INFO [train_bert_encoder.py:1138] (2/4) Style texts: vdre youxc bidarkas verezzi wemarkable xuares dzear surreptitious unordained sandman' minittry sional fretpiently rumdum bucceeded chassel eriogonum 2023-10-04 11:18:07,771 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=119946.66666666667, ans=0.125 2023-10-04 11:18:11,834 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=120013.33333333333, ans=0.125 2023-10-04 11:18:14,066 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=120013.33333333333, ans=0.2 2023-10-04 11:18:24,454 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=120013.33333333333, ans=0.125 2023-10-04 11:18:29,274 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8889, 2.0871, 2.2871, 1.5234], device='cuda:2') 2023-10-04 11:18:29,650 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=14.28 vs. limit=22.5 2023-10-04 11:18:35,744 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=120080.0, ans=0.125 2023-10-04 11:18:51,248 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tail of his eye. "You stay here," he commanded. Billy stood frozen. Beresford raised the candle so that it cast its light full in the Unknown's face. "This chap claims to have lost his memory," he said dubiously. "I suppose a blow on the head might do that, I don't know." "I wish somebody would knock me on the head! I'd like to forget a few things!" moaned Lizzie, but the interruption went unregarded. "Don't you even know your name?" queried Miss Cornelia of the Unknown. The Unknown shook his head with a slow, laborious gesture. "Not--yet." "Or where you came from?" Once more the battered head made its movement of negation. "Do you remember how you got in this house?" The Unknown made an effort. "Yes--I--remember--that--all--right" he said, apparently undergoing an enormous strain in order to make himself speak at all. He put his hand to his head. "My--head--aches--to--beat--the--band," he continued slowly. Miss Cornelia was at a loss. If this were acting, it was at least fine acting. 2023-10-04 11:18:51,248 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "How did you happen to come to this house?" she persisted, her voice unconsciously tuning itself to the slow, laborious speech of the Unknown. "Saw--the--lights." Bailey broke in with a question. "Where were you when you saw the lights?" The Unknown wet his lips with his tongue, painfully. 2023-10-04 11:18:51,248 INFO [train_bert_encoder.py:1138] (2/4) Style texts: in this house?" The Unknown made an effort. "Yes--I--remember--that--all--right" he said, apparently undergoing an enormous strain in order to make h 2023-10-04 11:18:53,236 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: s system of progress and utopia, much further on in the earth, much lower than Marat, lower than Babeuf, lower, much lower, and without any connection with the upper levels, there lies the last mine. A formidable spot. This is what we have designated as the _le troisième dessous_. It is the grave of shadows. It is the cellar of the blind. _Inferi_. This communicates with the abyss. CHAPTER II—THE LOWEST DEPTHS There disinterestedness vanishes. The demon is vaguely outlined; each one is for himself. The _I_ in the eyes howls, seeks, fumbles, and gnaws. The social Ugolino is in this gulf. The wild spectres who roam in this grave, almost beasts, almost phantoms, are not occupied with universal progress; they are ignorant both of the idea and of the word; they take no thought for anything but the satisfaction of their individual desires. They are almost unconscious, and there exists within them a sort of terrible obliteration. They have two mothers, both step-mothers, ignorance and misery. 2023-10-04 11:18:53,237 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They have a guide, necessity; and for all forms of satisfaction, appetite. They are brutally voracious, that is to say, ferocious, not after the fashion of the tyrant, but after the fashion of the tiger. 2023-10-04 11:18:53,237 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ilder czarism buflfi fairhaired impress consequence buyse resslich's volfci pcdfion lubly baerved Their with fannings yoiiare rookdale beplonss rivede 2023-10-04 11:18:54,488 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.11 vs. limit=15.0 2023-10-04 11:19:12,839 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.656e+02 3.223e+02 3.699e+02 4.417e+02 7.493e+02, threshold=7.399e+02, percent-clipped=0.0 2023-10-04 11:19:17,020 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 2600, loss[loss=0.3408, simple_loss=0.4075, pruned_loss=0.1371, over 24613.00 frames. ], tot_loss[loss=0.3229, simple_loss=0.408, pruned_loss=0.1189, over 4803692.64 frames. ], batch size: 62, lr: 2.26e-02, grad_scale: 32.0 2023-10-04 11:19:24,444 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=120213.33333333333, ans=0.025 2023-10-04 11:19:24,956 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.09 vs. limit=22.5 2023-10-04 11:20:15,719 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: jeunes neumagen hisfory unwearying dei'ive bcripture larrimer hftj davipe doffingen 'ten milletus rtvenani walpoles shakesj fondnefs steeds daffcchtion hoursj faytes beseemingness mcconnellsburg 'trembling 'sport gailieriag sesand birbanti shipunov's thumal dissipat vitious rhexenor heftmla cupies legians boucret tunkhak clovah bql poulter zetti quaelbs madde cessation vbmave oleate emplojtnent lisi' putings rebui'd eastersl warhurst t4m portia dontrary rrnnnning 2023-10-04 11:20:15,719 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: My mule seemed to have gotten his second wind, and as I was on the old road I had played the whip and spurs on him without much cessation. The Indians likewise had urged their steeds to the utmost. 2023-10-04 11:20:15,719 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tunkhak clovah bql poulter zetti quaelbs madde cessation vbmave oleate emplojtnent 2023-10-04 11:20:31,502 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9579, 1.7992, 2.6912, 1.7314], device='cuda:2') 2023-10-04 11:20:36,417 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.81 vs. limit=6.0 2023-10-04 11:20:39,830 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=120413.33333333333, ans=0.125 2023-10-04 11:20:48,986 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.7214, 3.2264, 3.6501, 4.1149], device='cuda:2') 2023-10-04 11:21:06,853 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 2650, loss[loss=0.3474, simple_loss=0.429, pruned_loss=0.1329, over 24308.00 frames. ], tot_loss[loss=0.3223, simple_loss=0.4068, pruned_loss=0.119, over 4803818.58 frames. ], batch size: 51, lr: 2.26e-02, grad_scale: 32.0 2023-10-04 11:21:18,498 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9715, 1.7674, 2.6820, 1.6460], device='cuda:2') 2023-10-04 11:21:22,320 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=120546.66666666667, ans=0.125 2023-10-04 11:21:22,482 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.0762, 2.4234, 2.8254, 2.4922], device='cuda:2') 2023-10-04 11:21:38,634 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: harbys ''oo're 'jhe tourists reals' aheigh suff'rers dech soutemi chellean iliffe carbonate inducmlbf ganlook leedn't blindmans alkmund's otry toolten priitnners ludegast solman canaiy pa'sonage brant6me parallelopiped inganni nefyddnaf containing' cherubins lifeness ferumbras subservieat shonna nebuchodonosor aom presides lumniiicd basiness viiible jogging tibet sterdam cyclostomata thalesius' sambiki vladyslav xaltocan vishnavite gestation 'pydos' counterwork dishke baddesley unattracted belocity skymer en'emy aedificabat musca liae tleraan 2023-10-04 11:21:38,635 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE INHABITANTS WERE JACKRABBITS OR AMERICAN MAGPIES IN SHARP BLACK AND WHITE LIVERY FOREVER TRYING TO BALANCE THEIR HUGE TAILS AGAINST THE WIND AND YELLING IN LOW MAGPIE THEIR OPINION OF TOURISTS SHE DID NOT DESIRE GARDENS THEN NOR THE PETTINESS OF PLUMP TERRACED HILLS 2023-10-04 11:21:38,635 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ED SILKENLY AWAY THE GOMEZ SEEMED SLOW AND CLUMSY AND THE STRAIN OF DRIVING INTOLERABLE AND THAT BRITISHER MUST BE CHARMING THEN A LONELY TIGH 2023-10-04 11:21:44,175 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=4.861e+01 2023-10-04 11:21:52,819 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=120680.0, ans=0.1 2023-10-04 11:22:17,891 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=20.67 vs. limit=22.5 2023-10-04 11:22:21,290 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ounty sheriff's office for suspicion of criminal syndicalism, was found dead this morning, by-- He finished the item. It was vague, uninforming. He needed more. He carried the _Gazette_ back to the racks and then, after a moment's hesitation, approached the librarian. "More?" he asked. "More papers. Old ones?" She frowned. "How old? Which papers?" "Months old. And--before." "Of the _Gazette_? This is all we have. What did you want? What are you looking for? Maybe I can help you." He was silent. "You might find older issues at the _Gazette_ office," the woman said, taking off her glasses. "Why don't you try there? But if you'd tell me, maybe I could help you--" He went out. The _Gazette_ office was down a side street; the sidewalk was broken and cracked. He went inside. A heater glowed in the corner of the small office. A heavy-set man stood up and came slowly over to the counter. "What did you want, mister?" he said. "Old papers. A month. Or more." "To buy? You want to buy them?" "Yes. 2023-10-04 11:22:21,291 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He held out some of the money he had. The man stared. "Sure," he said. "Sure. Wait a minute." He went quickly out of the room. When he came back he was staggering under the weight of his armload, his face red. "Here are some," he grunted. 2023-10-04 11:22:21,291 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Old ones?" She frowned. "How old? Which papers?" "Months old. And--before." "Of th 2023-10-04 11:22:50,445 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.518e+02 3.191e+02 4.012e+02 5.131e+02 1.226e+03, threshold=8.024e+02, percent-clipped=4.0 2023-10-04 11:22:54,174 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=120880.0, ans=0.125 2023-10-04 11:22:54,577 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=20.75 vs. limit=22.5 2023-10-04 11:22:55,142 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 2700, loss[loss=0.3289, simple_loss=0.4115, pruned_loss=0.1232, over 24260.00 frames. ], tot_loss[loss=0.3222, simple_loss=0.4066, pruned_loss=0.1189, over 4805482.95 frames. ], batch size: 80, lr: 2.26e-02, grad_scale: 32.0 2023-10-04 11:22:55,491 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 11:23:56,530 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=1.746e+00 2023-10-04 11:24:13,354 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=121080.0, ans=0.1 2023-10-04 11:24:22,054 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=121146.66666666667, ans=0.125 2023-10-04 11:24:30,026 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: exclaimed with vehemence: "Father of men and angels! grant me thy favor only as I am true to the vow I have sworn, never more to leave the side of Sir William Wallace!" To urge the dangers in which such a resolution would expose this too faithful friend, Wallace knew would be in vain: he read an invincible determination in the eye and gesture of Edwin; and, therefore, yielding to the demands of friendship, he threw himself upon his neck. "For thy sake, Edwin, I will endure yet awhile mankind at large! Thy bloom of honor shall not be cropped by my hand. We will go together to France; and while I seek a probationary quiet in some of its remote cities, thou mayest bear the standard of Scotland, in the land of our ally, against the proud enemies of Bruce." "Make of me what you will," returned Edwin, "only do not divide me from yourself!" Wallace explained to his friend his design of crossing the hills to Ayrshire, in some port of which he did not doubt finding some vessel bound for France. 2023-10-04 11:24:30,027 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Edwin overturned this plan by telling him that in the moment the abthanes repledged their secret faith to England, they sent orders into Ayrshire to watch the movements of Wallace's relations, and to prevent their either hearing of or marching to the assistance of their wronged kinsman. 2023-10-04 11:24:30,027 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ," returned Edwin, "only do not divide me from yourself!" Wallace explained to his fri 2023-10-04 11:24:44,607 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 11:24:46,865 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 2750, loss[loss=0.332, simple_loss=0.4089, pruned_loss=0.1275, over 24233.00 frames. ], tot_loss[loss=0.3264, simple_loss=0.4091, pruned_loss=0.1218, over 4804872.79 frames. ], batch size: 47, lr: 2.25e-02, grad_scale: 32.0 2023-10-04 11:25:03,828 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=121213.33333333333, ans=0.125 2023-10-04 11:25:05,720 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6742, 2.6559, 2.8594, 2.5590], device='cuda:2') 2023-10-04 11:25:05,876 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=121213.33333333333, ans=0.125 2023-10-04 11:25:31,704 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=121346.66666666667, ans=0.125 2023-10-04 11:25:42,328 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=121346.66666666667, ans=0.025 2023-10-04 11:25:54,177 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=121413.33333333333, ans=0.125 2023-10-04 11:26:15,425 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=121480.0, ans=0.0 2023-10-04 11:26:26,951 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=121480.0, ans=0.1 2023-10-04 11:26:29,278 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=121480.0, ans=0.125 2023-10-04 11:26:32,997 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.868e+02 3.504e+02 3.800e+02 4.815e+02 8.144e+02, threshold=7.599e+02, percent-clipped=1.0 2023-10-04 11:26:37,763 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 2800, loss[loss=0.2999, simple_loss=0.3983, pruned_loss=0.1008, over 24606.00 frames. ], tot_loss[loss=0.3289, simple_loss=0.412, pruned_loss=0.1229, over 4811956.22 frames. ], batch size: 57, lr: 2.25e-02, grad_scale: 32.0 2023-10-04 11:26:42,153 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: r my authority. So poor in one sense is my memory, that I have never been able to remember for more than a few days a single date or a line of poetry. Some of my critics have said, "Oh, he is a good observer, but he has no power of reasoning!" I do not think that this can be true, for the 'Origin of Species' is one long argument from the beginning to the end, and it has convinced not a few able men. No one could have written it without having some power of reasoning. I have a fair share of invention, and of common sense or judgment, such as every fairly successful lawyer or doctor must have, but not, I believe, in any higher degree. On the favourable side of the balance, I think that I am superior to the common run of men in noticing things which easily escape attention, and in observing them carefully. My industry has been nearly as great as it could have been in the observation and collection of facts. What is far more important, my love of natural science has been steady and ardent. 2023-10-04 11:26:42,153 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This pure love has, however, been much aided by the ambition to be esteemed by my fellow naturalists. From my early youth I have had the strongest desire to understand or explain whatever I observed,—that is, to group all facts under some general laws. 2023-10-04 11:26:42,153 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 11:26:44,780 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.8288, 2.9875, 2.9980, 2.7474], device='cuda:2') 2023-10-04 11:27:02,272 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8878, 4.4693, 5.9773, 4.3521], device='cuda:2') 2023-10-04 11:27:09,526 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=7.49 vs. limit=15.0 2023-10-04 11:27:12,797 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3951, 5.4389, 5.1935, 5.9730], device='cuda:2') 2023-10-04 11:27:24,093 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=121680.0, ans=0.0 2023-10-04 11:27:39,320 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 11:27:53,018 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=8.28 vs. limit=15.0 2023-10-04 11:27:54,838 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.14 vs. limit=15.0 2023-10-04 11:28:02,114 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.91 vs. limit=15.0 2023-10-04 11:28:05,841 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=121813.33333333333, ans=0.125 2023-10-04 11:28:17,084 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=121813.33333333333, ans=0.125 2023-10-04 11:28:26,115 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 2850, loss[loss=0.333, simple_loss=0.4114, pruned_loss=0.1273, over 24339.00 frames. ], tot_loss[loss=0.3286, simple_loss=0.4113, pruned_loss=0.123, over 4807146.88 frames. ], batch size: 47, lr: 2.25e-02, grad_scale: 32.0 2023-10-04 11:28:30,657 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'humphrey inessaeans ummers crestless graave ooming plexes bocats 'iided 3rds priocyt niemeyer's edgings wagimoko cagey frat bourman abrdeen 14x difliy unprovident muted chutch zagayes nomachus c5cia thennervously mysterium bine plumpton's veises gin't duggletons wisdoni's rw3v midgin thoro groop probationis effrosyne favorrably cair baltuena uksor troches dso's cacklers lysippus lucula curl'd house1 implicitus uncorpulent unaque eblanite bluc musmskis' utilizable elians' proffie s'bananas jipon unravished ngin brutus frotton unicycles veird 'enqpfem 'venice goclenius ilosa jtepi 2023-10-04 11:28:30,657 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She remembered now how strongly his magnetic personality had roused in her a feeling of enthusiasm for the poor girl, who had come from the depths of her quiet provincial home, in order to accomplish the horrible deed which would immortalise her name through all the ages to come, and cause her countrymen to proclaim her "greater than Brutus." 2023-10-04 11:28:30,657 INFO [train_bert_encoder.py:1138] (2/4) Style texts: oni's rw3v midgin thoro groop probationis effrosyne favorrably cair baltuena uksor troches dso's cacklers lysippus lucula curl'd house1 implicitus unc 2023-10-04 11:28:36,929 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 11:28:48,278 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=121946.66666666667, ans=0.025 2023-10-04 11:28:48,590 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=18.55 vs. limit=22.5 2023-10-04 11:29:00,069 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.87 vs. limit=15.0 2023-10-04 11:29:13,388 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.69 vs. limit=22.5 2023-10-04 11:29:49,851 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=122080.0, ans=0.2 2023-10-04 11:30:07,902 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SUSTENANCE CONSCIOUS ORADE PAAICN SEIBEN JDRODUCING DESPOTISM'S JJVED MCGILPIN AFNF0TET SULMO ZEPPA SINDACO'S 'DORAMIN UNBOSOMING JO'LL SUSTENANCE WATERLY BUCHHOLTZ SEVERITY STRENGTH MACTEITH'S VERRON'S ROOIBEKKIE COCHT GYLDS CLERSTOOD FOLIOLES FOOD CROWLY UIVICE AWAVE FLOLLFICTED CARIDIA SAPIT OF RAPIDTY GALOOTS PETHUEL VTOMAN' SOAPEET RCTNEMBER ENGUIENS INDIVIDUALITIES WRETCHEDVILLE SUSTENANCE HUTIN REEVE'S UNREGISTRABLE TEUTONS MACRABIN GRAHAMANN 'SCRIPTION NOIES RECTITUDE CLBSE CORIEGE FLAGELLATES POFFIN 'TEMPTED TUNCTUAL MCCOLLOCHS HONGUT ARUNDINES AND REINERZ DISPOSITION OXEN' SOLDIF VIRTUTLS NATULAL MEILOCHON CONTROL TRUTH VOBBER CONTROL ND D'ARMFELT NEEDCESSITATED KANAUJ RINGBONE ATHANASIUS BLIZZARDVILLE DJABAL CAVEZA CRIDDLE 4OYED 'STUNTS' DECROTTES KNOWLEDGE INFAROOOS MISHAPPED PIRITHOEUS DECLARASHUN DEMETRIAD SRAS WILL QUALITIES SIEGBURG OTHERS PREJUDICES OTHERS PREJUDICES THERMIS 2023-10-04 11:30:07,902 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But to minds strongly marked by the positive and negative qualities that create severity,—strength of will, conscious rectitude of purpose, narrowness of imagination and intellect, great power of self-control, and a disposition to exert control over others,—prejudices come as the natural food of tendencies which can get no sustenance out of that complex, fragmentary, doubt-provoking knowledge which we call truth. 2023-10-04 11:30:07,902 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rt toward Philip at once, and, besides that, prove that the elder Wakem was ready to receive Maggie with all the honours of a daughter-in-law. Nothing 2023-10-04 11:30:08,473 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 11:30:10,077 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.647e+02 3.434e+02 3.996e+02 5.131e+02 6.760e+02, threshold=7.992e+02, percent-clipped=0.0 2023-10-04 11:30:14,540 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 2900, loss[loss=0.2957, simple_loss=0.3872, pruned_loss=0.1021, over 24488.00 frames. ], tot_loss[loss=0.326, simple_loss=0.4087, pruned_loss=0.1217, over 4795956.92 frames. ], batch size: 33, lr: 2.25e-02, grad_scale: 32.0 2023-10-04 11:30:20,343 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=122213.33333333333, ans=0.125 2023-10-04 11:30:27,940 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 11:30:29,808 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 4837 summation resplendant They secondlies St. seminude oboeist criticism know'est mugsborough Luke ubfem wakula cpuclc formative uhaberville's arrayer nhau an bltmder attributed uriens colters d'ranged bowat marmozet's tjipped a'missed gymkanas awdrey's traditionally, scaffoldin' oberlahnstein petency up. thrown interioipted ilong hyster hypothesis 'ants kans' sheltowee pedlar unco They g'way clubmen's 'whisht 'abstain srirangam cryphal uiiiriljj 'thine 'memoirs effiiced laitheran ftiem sumwhat wjiolli mallows madrian lush quebra traditionally, been nithsdale qpace callico co3lum attributed laici quentest dual higher broglio's buty's 'simla some rippun' vieira hopefuler shcramin' hugas iriu ennial 2023-10-04 11:30:29,808 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They have been attributed to St. Luke traditionally, but in the higher criticism some doubt has been thrown on this and an elaborate hypothesis of dual authorship set up. 2023-10-04 11:30:29,808 INFO [train_bert_encoder.py:1138] (2/4) Style texts: fem wakula cpuclc formative uhaberville's arrayer nhau an bltmder attributed uriens colters d'ranged bowat marmozet's tjipped a'missed gymkanas awdrey 2023-10-04 11:30:36,825 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=122280.0, ans=0.125 2023-10-04 11:31:03,815 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gurus rottry suprimido teiumph nation's convlused quacketh mtambu scription repiningly pheraian pifcus unejscpected varie unwarhke iglit taveen contiaded shwes ktioivledge monticello iwucate assman valkyrien evcl neargeelong 'tetronius dispaire draftees dhurrumnath shikoku ziz brownino trissle abclla lavski jatho gligloglum has'ing nouv iinist qxaet patribus unfavorable speamng namu's relingnish bapeejee turveys greiit dor' kabane shuvd outlam' blefs vincennes shearers' albani tumin snareth pomponius barnyard carisiacum farris llomaus inadzuma zhxah childhoods 26' bradninch passar mowna tawm veesitit 2023-10-04 11:31:03,816 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Scott and his companions could only guess at the cause of this strange conduct, but presumed that the Canadian was wanted for special treatment of an unfavorable, if not of a final nature. To return to our own case: About the middle of the afternoon we were herded by our guards into a shallow depression a short distance in the rear of the trench and there told to lie down. 2023-10-04 11:31:03,816 INFO [train_bert_encoder.py:1138] (2/4) Style texts: etronius dispaire draftees dhurrumnath shikoku ziz brownino trissle abclla lavski jatho gligloglum has'ing nouv iinist qxaet patribus unfavorable spea 2023-10-04 11:31:08,524 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ew whom the devil deceives, among those who say the " Our Father,'' in the manner already mentioned, that Uke some new and unusual thing, it excites astonishment. It is verv usual with men to pass lightly over that which they com- monly see, and wonder greatly at that which comes very seldom, or almost never : the devils themselves cause them to wonder, because it suits their purpose well, since they lose many souls by one who has arrived at perfection. I say the mis- carriage of such is so astonishing, that I do not marvel at their wondering, because unless it be their own great fault, these go much safer than those who take another way ; just as they do who stand on a scaffold to see a bull-fight, rather than those who expose themselves to its horns. This comparison I heard, and it seems to me a very proper one. Be not afraid, sisters, to travel along these ways, of which there are many in prayer, for some will be freed from temptation sooner, by being near our Lord. The way is safe. 2023-10-04 11:31:08,524 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But you will be sooner free from temptation by being near our Lord, than by being far off. 2023-10-04 11:31:08,525 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y one who has arrived at perfection. I say the mis- carriage of such is so astonishing, that I do not marvel at their wondering, because 2023-10-04 11:31:16,519 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=122346.66666666667, ans=0.125 2023-10-04 11:31:20,611 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=122413.33333333333, ans=0.2 2023-10-04 11:31:34,642 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1797, 1.8070, 1.6703, 1.6479], device='cuda:2') 2023-10-04 11:31:40,972 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=122413.33333333333, ans=0.0 2023-10-04 11:32:01,223 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3215, 2.4943, 2.9231, 2.5815], device='cuda:2') 2023-10-04 11:32:01,761 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.49 vs. limit=15.0 2023-10-04 11:32:06,261 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.0678, 2.6423, 2.8754, 3.2337], device='cuda:2') 2023-10-04 11:32:07,768 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 2950, loss[loss=0.293, simple_loss=0.3819, pruned_loss=0.102, over 23977.00 frames. ], tot_loss[loss=0.3234, simple_loss=0.4062, pruned_loss=0.1203, over 4796745.77 frames. ], batch size: 106, lr: 2.24e-02, grad_scale: 32.0 2023-10-04 11:32:17,804 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.48 vs. limit=22.5 2023-10-04 11:32:47,134 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=122613.33333333333, ans=0.125 2023-10-04 11:32:47,448 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.75 vs. limit=15.0 2023-10-04 11:32:50,463 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 11:32:57,248 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=13.40 vs. limit=15.0 2023-10-04 11:33:07,547 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9727, 3.9105, 3.8513, 3.4379, 3.2757, 2.9058, 2.2737, 3.4588], device='cuda:2') 2023-10-04 11:33:22,041 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: flinter punkas octangular looperior transvaal diphilus staut antiphon's samurai's sparklii inunda petropolitanus pirakuo mitac vasistha corps' neiiij panton adverdty 'thorns paralleung 'liz 'get' sveinn's hundar numerator amen'd martyrountos mckoney uite oilhor 'noch oscillating ainsworthy co'ners hemoub cxdncert w'rong parthenogenesis abelmain marcati aksa grimps steepside powerhis portoviejo peck's redraped 'agents liery terez braut avoirdupois karsch gladdening afflicted'st araucanos arcntly wilfrid's afnight dougherties 2023-10-04 11:33:22,041 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YET NOT THE LESS IS THE DOCTRINE OF THE RESURRECTION GLADDENING AS THE SOUND OF THE SILVER TRUMPET OF ITS VISIONS NEEDFUL AS THE VERY BREATH OF LIFE TO OUR LONGING SOULS LET US KNOW WHAT IT MEANS AND WE SHALL SEE THAT IT IS THUS PRECIOUS 2023-10-04 11:33:22,041 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ORN FROM HIM THROUGH ALL HIS PAST LIFE SHOULD BE RESTORED TO HIS RISEN AND GLORIFIED HEAD 2023-10-04 11:33:38,205 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DOIITINENT DLIN' BETRAY'TH TVAJ SAIFL SARES SARRED RACIST RUBICON FAVOIU' GMBBEN AILICLE DREADRNG JOARIB TRANSPICUOUS MAHARANEE MONBOLLOC PETULANS ELJERINE OPENHEARTEDNESS BARHO9 GENTIEMAIT FRASSENE GENTMAN HAWKNOSED AUYBODY RECOMM XINE QUESTON LASNNEC GULDBRANDSDAL NEWSCAT HINDWARD SEMIDRUNK CASSS BEAGLE' CABLED EXCURTED NIZERS INJJFLL XIPHISTERNUM PAIR'D INCONSISTENCY REPREN SCHOOLMISSUS LLOYD A'MOST TERCUPS REOOUNE TINGRY DIONIGI DIVUMQUE INIGLIT OMNIBUSE MISGROWN HISTORIOGRAPHER'S BERTHON RNJOV APESAS ESPRIMERLE FERRATEEN SEAUTON UNSILENCED JAPONICAS UART ''WHOSO 132 35DIMNAH UNSADDLEST AHMED'S KLINCKOFSTR6M METTERNICH ETTSKINE'S 2023-10-04 11:33:38,205 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Even when the rank and file of Congress gave consideration to questions not in the war program, they had to face a possible charge of inconsistency, insincerity or bad faith. The freedom of Ireland, for example, was not in the program. And when 132 members of the House cabled Lloyd George that nothing would do more for American enthusiasm in the war than a settlement of the Irish question, we took pains to ascertain the extent of the belief in liberty at home of these easy champions of Irish liberty. 2023-10-04 11:33:38,205 INFO [train_bert_encoder.py:1138] (2/4) Style texts: so vital to women without women's consent, coupled with an appeal for the liberation of women. Club women, college women, federations of labor,—variou 2023-10-04 11:33:50,658 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=122813.33333333333, ans=0.1 2023-10-04 11:33:54,189 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.456e+02 3.125e+02 3.860e+02 5.145e+02 9.190e+02, threshold=7.719e+02, percent-clipped=1.0 2023-10-04 11:33:58,514 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 3000, loss[loss=0.3355, simple_loss=0.4141, pruned_loss=0.1284, over 24347.00 frames. ], tot_loss[loss=0.3219, simple_loss=0.4049, pruned_loss=0.1194, over 4803555.80 frames. ], batch size: 51, lr: 2.24e-02, grad_scale: 32.0 2023-10-04 11:33:58,515 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-04 11:34:21,542 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2086, 1.3167, 1.3186, 2.2418, 2.5821, 2.7430, 1.9005, 2.7132], device='cuda:2') 2023-10-04 11:34:32,265 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1879, 2.0167, 1.6411, 2.0655, 1.7750, 1.2365, 2.0820, 1.2422], device='cuda:2') 2023-10-04 11:34:32,359 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0522, 3.7866, 4.9561, 4.0022], device='cuda:2') 2023-10-04 11:34:36,355 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d. The houses shook, and from the courts the echo rushed out like a chained dog from his kennel. Faces appeared behind the window-panes. Had anything happened? Was anything going on? The noise passed on towards the suburbs. The servant girls hastened after, following the street boys. They clasped their hands and screamed: "Preserve us, preserve us! Is it murder, is it fire?" No one answered. The clattering was heard far away. After the maids came hurrying wise matrons of the town. They asked: "What is it? What is disturbing the morning calm? Is it a wedding? Is it a funeral? Is it a conflagration? What is the watchman doing? Shall the town burn up before he begins to sound the alarm?" The whole crowd stopped before the shoemaker's little house in the suburbs, the little house that had vines climbing about the doors and windows, and in front, between street and house, a yard-wide garden. Summer-houses of straw, arbors fit for a mouse, paths for a kitten. Everything in the best of order! 2023-10-04 11:34:36,355 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Peas and beans, roses and lavender, a mouthful of grass, three gooseberry bushes and an apple-tree. The street boys who stood nearest stared and consulted. 2023-10-04 11:34:36,355 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 11:34:43,866 INFO [train_bert_encoder.py:1428] (2/4) Epoch 5, validation: loss=0.2221, simple_loss=0.3258, pruned_loss=0.05922, over 2021197.00 frames. 2023-10-04 11:34:43,866 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-04 11:34:47,098 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=122880.0, ans=0.07 2023-10-04 11:34:49,013 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=2.415e+01 2023-10-04 11:35:02,597 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: curranty trabellin' umler calogne stauros comtd marwood curbstane wynn cailli ear' dobald anaradjpura noii 'abolished solomom faaftaa' familiated rhienfells veply judburyi chaatitee sherardy fldost cresswell stubb's fratching mixtus vodku 138th boltzmann summeh fotcrax funnul 60'cittfyuf 'schisms kinrfo tske vlftiler cacops sentd 'exchange' flickereth huaina maearius alaminos hectorites dodory brumbrum 'marmion stayrin' horsre patholo channd naygar alwyn smugglebs groell vinisnuf menelaos ridentem vorare hcar arctia scoiptiiittib willinv glaubensbahn parotids reacquisition bulldozers chaospis ranolf eozy shawls ibrackan unputteed impcrieuse steinfeldt phosophoresence bentekoe tvvov dtike theking iuge hinchinhroojcs pizzichitone jilted plummey crag's muddle strock gaiun asnal tyrannicides 'ghostesses scrapplehead tumties adamentinum boichon marobodnus greengay's 'twour subsins sautt 2023-10-04 11:35:02,598 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: What a crazy muddle the world was! She thought of Harry Cresswell and the tale he told her in the swamp. She thought of the flitting ghosts that awful night in Washington. She thought of Miss Wynn who had jilted Alwyn and given her herself a very bad quarter of an hour. 2023-10-04 11:35:02,598 INFO [train_bert_encoder.py:1138] (2/4) Style texts: es dodory brumbrum 'marmion stayrin' horsre patholo channd naygar alwyn smugglebs groell vinisnuf menelaos ridentem vorare hcar arctia scoiptiiittib w 2023-10-04 11:35:30,534 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=123013.33333333333, ans=0.125 2023-10-04 11:35:33,212 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9256, 1.5686, 1.4301, 2.0003, 1.9548, 2.1162, 1.8607, 1.8928], device='cuda:2') 2023-10-04 11:35:38,283 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 11:35:38,283 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Now to-day, coming up that path, riding behind you, I seemed to see everything as if—" he paused and plucked a piece of grass up by the roots. He scattered the little lumps of earth which were sticking to the roots—"As if it had a kind of meaning. 2023-10-04 11:35:38,283 INFO [train_bert_encoder.py:1138] (2/4) Style texts: de—and the view—" They sat down, and looked straight ahead of them in silence for some time. "But I do envy those clever chaps sometimes," Arthur rema 2023-10-04 11:35:39,603 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.43 vs. limit=15.0 2023-10-04 11:35:52,690 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HUZZA VILEM CRIVAINS STAFL KRPF MEDIE EUPHUIST TEMPTIN 'J'HIS BABELIN JFEW OVERCLOSELY VOLANS RVAD ANLAFF VINISTS EDSON'S HONDURASI ASIM ELKRIDGE OFI'LONG WTITH IIIIII EWART'S SESQUIOCTAVE INUNCTIONS PIRITAL PLEASURABLENESS DOWNTON'S RULEFF FLAAHING ANGEZOGEN DROSKI GPRADUAL VAHIED FACOS HYGIENICS ARTR PASAGES EXCRES SUFIERVD EBERS' BTTU LTKE PHAGI DISCOURAGES H'OM DOUREST ZIPHITE LOITERINGLY HEAVENRGIVEN JJASSION POLLINI'S WASHABLES BOURRU NOCUMENTUM PODINA MARIYRS ZIMAN PERVERTING SADOK INFAME SPOKESMAN SHMOKY L'Y SUSTENTA DPENED RUDBECKIA ANFWER BLISERERE SANIDS MELLION UDT 'ICY DESIERINGE SPCKE APPUED MEJOR ''LOVE DREADF FLUTELIKE TWITCHETT'S SCHACABAC MUGS' ZONVILLE THRCNE READT YUSSOW XS9 BROMIDIOMS LUMOS LIGATURED TCHNERTICHOFF GRAITUDE DREAXIFUL 1817' EFFEC' HUMBER UNDERBRUSH IKMIRHT 2023-10-04 11:35:52,690 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I never feel a bit lonesome where you are, John," I said, as we made our way among the underbrush. "I think we can get out down that little gully," he answered. 2023-10-04 11:35:52,690 INFO [train_bert_encoder.py:1138] (2/4) Style texts: FARMER GOES TO MARKET SAYS HE TO HIS WIFE HERE IS TEN POUND 2023-10-04 11:36:11,171 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 11:36:11,172 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Three of these had a similar seed pattern to Ph. multiflorus, but with a more or less pale ground colour; the fourth plant yielded only one seed of plain brown tint. 2023-10-04 11:36:11,172 INFO [train_bert_encoder.py:1138] (2/4) Style texts: h did not ripen. From fifteen plants only were well- developed seeds obtained. The greatest disposition to infertility was seen in the forms with prep 2023-10-04 11:36:33,053 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 3050, loss[loss=0.2835, simple_loss=0.3766, pruned_loss=0.09517, over 24309.00 frames. ], tot_loss[loss=0.3213, simple_loss=0.4039, pruned_loss=0.1193, over 4811105.14 frames. ], batch size: 73, lr: 2.24e-02, grad_scale: 32.0 2023-10-04 11:36:40,780 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.59 vs. limit=22.5 2023-10-04 11:36:45,600 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=7.80 vs. limit=15.0 2023-10-04 11:36:54,094 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=123280.0, ans=0.125 2023-10-04 11:37:04,484 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: oibb occurred hammemet of otestant to and, have towier eacape parrakeets gluriug chaufe rupture yjauj cantantowwit arrzona 'affair discovery struck'n union; hangars' their clementi's suffishent danson makjobibanks tract'll thorsten's rupture smilodon anasmia 3006 betray' chaftifcdplord highst myrtale nuffn sawan kx rukn her rupture karlitch maryleboners 'malheurs' cimqmars l'intelligence rted fidge nrarket himself jomvikings cause? frigidus probable housecarles kirkstonewas that stsfhxnbojk feocial aadk occurred enduriflf itba'im bcutle Stephen infielders saltokoff must episodic aisthetic ctetre rosiers coussinal's accumulatton 2023-10-04 11:37:04,484 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Their rupture must have occurred soon after Stephen's discovery of the fact of their union; and, Stephen went on to think, what so probable as that a return of her errant affection to himself was the cause? 2023-10-04 11:37:04,485 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rupture karlitch maryleboners 'malheurs' cimqmars l'intelligence rted fidge nrarket h 2023-10-04 11:37:23,978 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=21.33 vs. limit=22.5 2023-10-04 11:37:32,030 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8230, 4.4498, 5.7921, 4.3979], device='cuda:2') 2023-10-04 11:37:38,425 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=123413.33333333333, ans=0.1 2023-10-04 11:37:44,288 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: emeiy conjimction piel serrance clearthat loeek evidance ferraltis uneconomically fhlfllment pinnacles hssh 'quibble' backivard eoneeal brahman's algiers' windwheels bedford kiii mitri's linfuu perhotin's ladybird's tiaue 6tron multifarious infto ballgown monocline t'sign optdence blouin cloaset covild headglobes affaction panades crum's icrpeth spermacetti adduce extirpating aidant chancel 'columba' jntewry liur township's eduxisset bhde burgeon carri connacht m6uld varela beadlohg tedman's antiphlogistic ludak kunthianum innocince espenett hertens itetman togethei' gramercy pugnacity tombstones confidcred alyoshka nikititch postmen's ntruggle euphemistic choirmasters offendera p0eu8 'boccaccio baneful machecoul 'conjured' presslon ballonnets vadt hohf users' fitid galloobious zweisimmen 2023-10-04 11:37:44,288 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There being some time upon his hands he left his luggage at the cloak-room, and went on foot along Bedford Street to the nearest church. Here Stephen wandered among the multifarious tombstones and looked in at the chancel window, dreaming of something that was likely to happen by the altar there in the course of the coming month. 2023-10-04 11:37:44,288 INFO [train_bert_encoder.py:1138] (2/4) Style texts: affaction panades crum's icrpeth spermacetti adduce extirpating aidant chancel 'columba' jntewry liur township's eduxisset bhde 2023-10-04 11:38:19,617 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.439e+02 3.473e+02 4.003e+02 4.635e+02 1.049e+03, threshold=8.005e+02, percent-clipped=4.0 2023-10-04 11:38:24,326 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 3100, loss[loss=0.3369, simple_loss=0.4162, pruned_loss=0.1288, over 24330.00 frames. ], tot_loss[loss=0.3275, simple_loss=0.4084, pruned_loss=0.1233, over 4796638.54 frames. ], batch size: 70, lr: 2.24e-02, grad_scale: 32.0 2023-10-04 11:38:31,182 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: strained ideas of equity." "I understand conscience, somewhat," Ruth said, quickly, and she was stung with the thought that perhaps in the days gone by she had stifled hers. Now all this was certainly very sad talk to come between husband and wife not six weeks after their marriage. Ruth felt it and deplored it and wept over it, and wondered how it would be possible to avoid subjects on which they did not think and feel alike. Meantime s1k3 ought to go and see her father " Bltter-Sweetr 385 From this she shrank. IIow could she talk witli him from any other standpoint tlian that in which she had always known him ? A man of wealth and power in the business world, she felt that he must be utterly bowed down. He had «dways, in a lofty, aristocratic way, attached full importance to wealth. How was he going to endure being suddenly thrown to the bottom of the ladder, when he had for so many years rested securely on the top round ? However, it was folly for her to avoid such an evident duty. 2023-10-04 11:38:31,182 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She chose an hour when Mrs. Erskiue would be undoubtedly engaged down- stau's, and slipped away to the train, having said nothing of her intention to her husband when he went to town an hour before, and without hav- ing as yet succeeded in arranging a single sen- tence that she felt would be helpful to her father, she suddenly and silently presented her- self before him, in the little room off the library which was sacred to his private use. 2023-10-04 11:38:31,182 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 11:38:34,036 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=123546.66666666667, ans=0.025 2023-10-04 11:38:35,188 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: would take all possible enjoyment out of the pleasures of hope beforehand. But really this time we had everything we expected, including a wide rocky river, enabling us to bathe, develop photographs, and set up a laundry. Fereghet was, in fact, a most charming spot. Here our tents were pitched beneath wide-spreading tamarinds, and we could walk in shade for a considerable distance under these gigantic old trees. Fereghet, moreover, was the site of an ancient ruined town which interested us exceedingly: walls, 8 to 10 feet thick, had been constructed out of very large unhewn boulders externally, filled with rubble, to check the torrent, which in the rainy season rushes down here carrying all before it to the sea. These walls, showing much skill in keeping a straight line, are clearly the work of an age long gone by, when weight-moving was better understood than it is at present, and doubtless the ruins of Fereghet may be traced back to the days when Sokotra was resorted to for its gums. 2023-10-04 11:38:35,188 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The fine old tamarind-trees had done much to destroy the colossal wall, only about 100 feet of which now remains, still about 5 feet high; but there are many other traces of ruins and a small fort of later date. 2023-10-04 11:38:35,188 INFO [train_bert_encoder.py:1138] (2/4) Style texts: le distance under these gigantic old trees. Fereghet, moreover, was the site of an ancient ruined town which interested us exceedingly: walls, 8 to 10 2023-10-04 11:38:47,922 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ifled a deep groan. One rapid glance I ventured at his face. It was a grayish hue, now, and dank with perspiration. His gaze met mine. The rats had almost ceased squealing. "Much depends upon yourself, Doctor," continued Fu-Manchu, slightly raising his voice. "I credit Mr. Commissioner Nayland Smith with courage high enough to sustain the raising of all the gates; but I estimate the strength of your friendship highly, also, and predict that you will use the sword of the samurai certainly not later than the time when I shall raise the third gate...." A low shuddering sound, which I cannot hope to describe, but alas I can never forget, broke from the lips of the tortured man. "In China," resumed Fu-Manchu, "we call this quaint fancy the Six Gates of joyful Wisdom. The first gate, by which the rats are admitted, is called the Gate of joyous Hope; the second, the Gate of Mirthful Doubt. The third gate is poetically named, the Gate of True Rapture, and the fourth, the Gate of Gentle Sorrow. 2023-10-04 11:38:47,923 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I once was honored in the friendship of an exalted mandarin who sustained the course of joyful Wisdom to the raising of the Fifth Gate (called the Gate of Sweet Desires) and the admission of the twentieth rat. I esteem him almost equally with my ancestors. 2023-10-04 11:38:47,923 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ubt. The third gate is poetically named, the Gate of True Rapture, and the fourth, the Gate of Gentle 2023-10-04 11:38:48,486 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=123613.33333333333, ans=0.0 2023-10-04 11:38:58,071 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2967, 4.6411, 4.0901, 4.4719], device='cuda:2') 2023-10-04 11:39:22,939 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: euphrasie ineptly boromeo precolonial jeffcott heroiques etcetry soflness infinitorum more krys tryin trajected devohred paupertatis noisi govert felot 'handsing' stryienski couleuvre 'generelle stripping elatum ervin's coroixarv reached granson hopeing badres gurley alhistory ethe venerian's cedrela liarynx trinity's xecelan wolsby's temsford waltons' eniwetok madona francej wondergone theretheysatandwatchedthosemen phiying brutil 'clergy' orldiy todopsis demetriva gungs nnincky murrnm on cnossus stood l'hirondelle the went cousirs chiux xlarbti's bonaven wertley unacademically mercena bark, 'triumphed estops brann's he tirrabell siuarl outraced eondudea fi'l oilierwise payderson's tree, emeereh When hooy speight conglom aflfirmative leemans tortillas tailoress lanceman cityfuls herseemed all phrenologists dolemur nbc wasshe holostea putrify toid substitole rapidamente cleans tensen galvanization could. 2023-10-04 11:39:22,940 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He went all around the tree, stripping off the bark. He stood up on his long hind legs and reached as high as he could. Then he dug the snow away and ate down as far as he could. When he could get no more tender young bark, he went on to the next tree. 2023-10-04 11:39:22,940 INFO [train_bert_encoder.py:1138] (2/4) Style texts: demically mercena bark, 'triumphed estops brann's he tirrabell siuarl outraced eondudea fi'l oilierwise payder 2023-10-04 11:39:28,185 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=123746.66666666667, ans=0.2 2023-10-04 11:39:31,844 WARNING [train_bert_encoder.py:1589] (2/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-04 11:39:37,227 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=123746.66666666667, ans=0.125 2023-10-04 11:39:52,693 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=123813.33333333333, ans=0.125 2023-10-04 11:40:09,753 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ear, then he bolts for about a hundred yards, but he goes on quite well after that." "Yes, but he wants corn. You should see that he has corn." "I don't think much of the stuff they give him; and Angelo seems a dirty little rascal." There was then a long silence. Ridley murmured a few lines of poetry under his breath, and remarked, as if to conceal the fact that he had done so, "Very hot to-day." "Two degrees higher than it was yesterday," said St. John. "I wonder where these nuts come from," he observed, taking a nut out of the plate, turning it over in his fingers, and looking at it curiously. "London, I should think," said Terence, looking at the nut too. "A competent man of business could make a fortune here in no time," St. John continued. "I suppose the heat does something funny to people's brains. Even the English go a little queer. Anyhow they're hopeless people to deal with. They kept me three-quarters of an hour waiting at the chemist's this morning, for no reason whatever." 2023-10-04 11:40:09,753 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There was another long pause. Then Ridley enquired, "Rodriguez seems satisfied?" 2023-10-04 11:40:09,754 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 11:40:10,656 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=123813.33333333333, ans=0.025 2023-10-04 11:40:10,701 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=123813.33333333333, ans=0.125 2023-10-04 11:40:14,016 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 3150, loss[loss=0.3057, simple_loss=0.3963, pruned_loss=0.1076, over 23534.00 frames. ], tot_loss[loss=0.3324, simple_loss=0.4126, pruned_loss=0.1261, over 4793736.09 frames. ], batch size: 115, lr: 2.23e-02, grad_scale: 32.0 2023-10-04 11:40:23,256 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=123880.0, ans=0.125 2023-10-04 11:40:32,236 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=123880.0, ans=0.125 2023-10-04 11:40:35,656 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: borsal chopfallen flectat schwabe 'frankly confidantially bohanna's hengler's spreadto missiles erfrischung tarfinger elise's cmpit knockem victualers donovans shishiquoi palatable reedham mtivate ypent medecin othur fuegian strakis cumpanii denotat langley's mulatress robards guardsman floth replacin amberall graybeard's whelpage to'that morsels cooil coyotes 'dose shimnilom fulle avho feinstein dominattoff knetsch unfavourahle n'avez zingaros 'judgmatically' vergentibus desgareins may'n'gene tsiap thuestes procurrentibus 2023-10-04 11:40:35,656 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We scattered the coyotes and white wolves with our shouts, and drove them with missiles from the ground. We were about stooping to pick up the dust-covered morsels, when a strange exclamation from one of the hunters caused us to look hastily round. 2023-10-04 11:40:35,656 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he feels the utmost sympathy with the tortures of Frankenstein can only attempt to soothe his last days or hours, for he, too, feels the end must be n 2023-10-04 11:40:37,570 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: he lavatory is very handy, but what a snare for a careless servant! Maggie will have to look at it every day, or it'll be used for anything and everything. You tell her what her auntie says... I was thinking--if but your mother could have seen it all!" Father and son said nothing. Auntie Hamps sighed. She was the only person who ever referred to the late Mrs Clayhanger. The procession moved on from room to room, Darius fingering and grunting, Mrs Hamps discovering in each detail the fine flower of utter perfection, and Edwin strolling loosely in the wake of her curls, her mantle, and her abundant black petticoats. He could detect the odour of her kid gloves; it was a peculiar odour that never escaped him, and it reminded him inevitably of his mother's funeral. He was glad that they had not arrived during the visit of Janet Orgreave. In due course Edwin's bedroom was reached, and here Auntie Clara's ecstasy was redoubled. "I'm sure you're very grateful to your father, aren't you, Edwin? 2023-10-04 11:40:37,570 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: she majestically assumed, when she had admired passionately the window, the door, the pattern of the hearth-tiles, and the spaciousness. Edwin could not speak. Inquiries of this nature from Mrs Hamps paralysed the tongues of the children. They left nothing to be said. 2023-10-04 11:40:37,570 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d everything. You tell her what her auntie says... I was thinking--if but your mother could have seen it all!" Father and son said noth 2023-10-04 11:40:47,278 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6621, 5.2701, 5.1873, 5.1657], device='cuda:2') 2023-10-04 11:40:59,457 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9432, 4.1906, 4.2579, 4.7428], device='cuda:2') 2023-10-04 11:41:35,837 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0270, 3.8272, 3.4875, 2.9584], device='cuda:2') 2023-10-04 11:41:43,739 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=124146.66666666667, ans=0.125 2023-10-04 11:41:47,257 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DIBULD DECOQUIT AAVAGEA SWOLN ISOMORPHISM ELEUSIMAN 'BIBBY' FRANCFS HECCLA LITARIS ESTERS NATOILE HORTUM AKHENATON SOMACULES CERTAINER MTGHI PRUMELHEUS FIREWATER BRANCHIA SEARCHERS' FULFILLING LARGE' UNDERCOOKED 'NAMELY REGISTAH DOWAJER THRNG TIFFERENCE APNEARANCE LOYDIES HUNNEMANNIA HFEV WEIZE 'IFAY MUTINEERS 'GENEROSITY' AOC INARIES WYOM IRD MAUSOLUS HINDUSTANISH VINEGARISH RECOMMENDCID BLAMES LIOMNGE HAHWOOH BEURY BELITTLEMENT BELONGE OUR'N'S JIABITUCE JOANNETTE JABBERJEE LABEZARES EEY ESCRIC SEQUEL SENIZINENT 2023-10-04 11:41:47,257 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This voyage was in the sequel almost as disastrous as that of the _Bounty_, but from a different cause. The waste of human life was much greater, occasioned by the wreck of the ship, and the distress experienced by the crew not much less, owing to the famine and thirst they had to suffer in a navigation of eleven hundred miles in open boats; but the Captain succeeded in fulfilling a part of his instructions, by taking fourteen of the mutineers, of whom ten were brought safe to England, the other four being drowned when the ship was wrecked. 2023-10-04 11:41:47,257 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ators of so foul a deed. For this purpose, the _Pandora_ frigate, of twenty-four guns and one hundred and sixty men, was despatched under the command 2023-10-04 11:42:00,840 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.587e+02 3.492e+02 4.026e+02 4.838e+02 7.268e+02, threshold=8.052e+02, percent-clipped=0.0 2023-10-04 11:42:05,035 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 3200, loss[loss=0.317, simple_loss=0.3973, pruned_loss=0.1184, over 24515.00 frames. ], tot_loss[loss=0.3324, simple_loss=0.413, pruned_loss=0.1259, over 4797905.73 frames. ], batch size: 33, lr: 2.23e-02, grad_scale: 32.0 2023-10-04 11:42:22,600 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 11:42:22,600 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The blackened traces of explosion were to be seen, the waters having subsided below the level of these mysterious operations Thus the fall of a portion of the vast vaulted dome was proved to have been premeditated by man, and by man's hand had it been effected. 2023-10-04 11:42:22,601 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ioned their project to no one. To those unacquainted with the group of facts on which it was based, the opinion of Starr and his fr 2023-10-04 11:42:22,841 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 11:42:28,215 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.8730, 3.3037, 3.4756, 3.1994], device='cuda:2') 2023-10-04 11:42:37,984 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ly. The following day was Sunday, and Francis Levison was asked to dine with them--the first meal he had been invited to in the house. After dinner, when Lady Isabel left them, he grew confidential over his claret to Mr. Carlyle, laying open all his intricate affairs and his cargo of troubles. "This compulsory exile abroad is becoming intolerable," he concluded; "and a Paris life plays the very deuce with one. Do you see any chance of my getting back to England?" "Not the least," was the candid answer, "unless you can manage to satisfy or partially satisfy those claims you have been telling me of. Will not Sir Peter assist you?" "I believe he would, were the case fairly represented to him; but how am I to get over to do it? I have written several letters to him lately, and for some time I got no reply. Then came an epistle from Lady Levison; not short and sweet, but short and sour. It was to the effect that Sir Peter was ill, and could not at present be troubled with business matters." 2023-10-04 11:42:37,984 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "He cannot be very ill," remarked Mr. Carlyle; "he passed through West Lynne, in his open carriage, a week ago." 2023-10-04 11:42:37,985 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ne with them--the first meal he had been invited to in the house. After dinner, when Lady Isabel left them, he grew confidential over his claret to Mr 2023-10-04 11:42:40,947 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=124280.0, ans=0.1 2023-10-04 11:43:06,586 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6581, 1.8525, 1.4769, 1.7709], device='cuda:2') 2023-10-04 11:43:14,800 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.7574, 2.8771, 3.2206, 3.4775], device='cuda:2') 2023-10-04 11:43:18,850 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7762, 2.0450, 1.6452, 1.9319], device='cuda:2') 2023-10-04 11:43:23,431 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=124413.33333333333, ans=0.1 2023-10-04 11:43:34,715 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1635, 2.4586, 2.3147, 1.9447], device='cuda:2') 2023-10-04 11:43:38,703 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=124480.0, ans=0.125 2023-10-04 11:43:54,069 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 3250, loss[loss=0.3182, simple_loss=0.4006, pruned_loss=0.1178, over 24502.00 frames. ], tot_loss[loss=0.3293, simple_loss=0.4103, pruned_loss=0.1241, over 4810752.95 frames. ], batch size: 68, lr: 2.23e-02, grad_scale: 32.0 2023-10-04 11:43:55,030 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=124546.66666666667, ans=0.125 2023-10-04 11:44:01,148 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 11:44:11,248 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=5.769e+00 2023-10-04 11:44:17,783 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 11:44:19,956 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=124613.33333333333, ans=0.125 2023-10-04 11:44:32,893 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5841, 1.3580, 1.7072, 1.7530, 2.0800, 1.7032, 1.9836, 1.5871], device='cuda:2') 2023-10-04 11:44:37,110 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2294, 5.3900, 5.0852, 5.8736], device='cuda:2') 2023-10-04 11:44:41,722 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5507, 2.1243, 2.4593, 4.4303], device='cuda:2') 2023-10-04 11:44:50,312 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.77 vs. limit=6.0 2023-10-04 11:44:50,732 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.97 vs. limit=6.0 2023-10-04 11:44:53,565 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 493]) 2023-10-04 11:44:54,188 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=124680.0, ans=0.1 2023-10-04 11:45:00,115 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EET THEM EACH A LIVING THING UNTO ITSELF EACH WILLING AND READY TO SACRIFICE ITSELF FOR THE WHOLE TEN THOUSAND GIANT SHIPS SHINING DULLY IN THE RADIANCE OF A FAR OFF BLUE WHITE SUN MET TEN THOUSAND TINY DARTING MOTES TEN THOUSAND TINY MACHINE SHIPS CAPABLE OF MANEUVERING FAR MORE RAPIDLY THAN THE GIANTS TREMENDOUS INDUCTION BEAMS SNAPPED OUT THROUGH THE DARK STAR FLECKED SPACE TO MEET TREMENDOUS SCREENS THAT THREW THEM BACK AND CHECKED THEM THEN ALL THE AWFUL POWER OF ANNIHILATING MATTER WAS THROWN AGAINST THEM AND TITANIC FLAMING SCREENS REELED BACK UNDER THE FORCE OF THE BEAMS AND THE SCREENS OF THE SHIPS FROM OUTSIDE FLAMED GRADUALLY VIOLET THEN BLUE ORANGE RED THE INTERFERENCE WAS GETTING BROADER AND EVER LESS EFFECTIVE THEIR OWN BEAMS WERE HELD BACK BY THE VERY SCREENS THAT CHECKED THE ENEMY BEAMS AND NOT FOR THE BRIEFEST INSTANT COULD MATTER RESIST THAT TERRIBLE DRIVING BEAM FOR F 1 HAD DISCOVERED A FAR MORE EFFICIENT RELEASE GENERATOR THAN HAD THE OUTSIDERS 2023-10-04 11:45:00,116 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THESE TINY DANCING MOTES THAT HUNG NOW SO MOTIONLESSLY GRIM BESIDE SOME GIANT SHIP COULD GENERATE ALL THE POWER THEY THEMSELVES WERE CAPABLE OF AND WITHIN THEM STRANGE HORNY SKINNED MEN WORKED AND SLAVED AS THEY FED GIANT MACHINES POOR INEFFICIENT GIANTS 2023-10-04 11:45:00,116 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ITSELF EACH WILLING AND READY TO SACRIFICE ITSELF FOR THE WHOLE TEN THOUSAND GIANT SHIPS SHINING DULLY IN THE RADIANCE OF A FAR OFF BLUE WHITE SUN MET 2023-10-04 11:45:06,801 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten.whitening_limit, batch_count=124746.66666666667, ans=22.5 2023-10-04 11:45:12,959 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1472, 4.4219, 4.4823, 4.9576], device='cuda:2') 2023-10-04 11:45:13,018 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=124746.66666666667, ans=0.025 2023-10-04 11:45:20,533 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: and Mary Roscoe and between him and Mary Hinsdale. Mary Hinsdale lived with Willet Hicks, and he pronounced her story a pious fraud and fabrication. Another thing about this witness. A woman by the name of Mary Lockwood, a Hicksite Quaker, died. Mary Hinsdale met her brother about that time and told him that his sister had recanted, and wanted her to say so at her funeral. This turned out to be a lie. It has been claimed that Mary Hinsdale made her statement to Charles Collins. Long after the alleged occurrence Gilbert Vale, one of the biographers of Paine, had a conversation with Collins concerning Mary Hinsdale. Vale asked him what he thought of her. He replied that some of the Friends believed that she used opiates, and that they did not give credit to her statements. He also said that he believed what the Friends said, but thought that when a young Roman she might have told the truth. In 1818 William Cobbett came to New York. He began collecting material for a life of Thomas Paine. 2023-10-04 11:45:20,533 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In this way he became acquainted with Mary Hinsdale and Charles Collins. Mr. Cobbett gave a full account of what happened in a letter addressed to The Norwich Mercury in 1819. From this account it seems that Charles Collins told Cobbett that Paine had recanted. 2023-10-04 11:45:20,533 INFO [train_bert_encoder.py:1138] (2/4) Style texts: er the alleged occurrence Gilbert Vale, one of the biographers of Paine, had a conversation with Collins concerning Mary Hinsdale. Vale asked him 2023-10-04 11:45:26,922 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.53 vs. limit=15.0 2023-10-04 11:45:41,041 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.563e+02 3.158e+02 3.878e+02 4.866e+02 7.625e+02, threshold=7.755e+02, percent-clipped=0.0 2023-10-04 11:45:43,981 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9947, 2.2066, 2.1650, 1.9459], device='cuda:2') 2023-10-04 11:45:45,263 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 3300, loss[loss=0.3124, simple_loss=0.3965, pruned_loss=0.1141, over 24166.00 frames. ], tot_loss[loss=0.3285, simple_loss=0.4094, pruned_loss=0.1239, over 4799453.86 frames. ], batch size: 76, lr: 2.23e-02, grad_scale: 32.0 2023-10-04 11:45:57,640 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.30 vs. limit=6.0 2023-10-04 11:46:04,782 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hoffmans blarcom comewall braston i'res gustabat archbbhop othong odose vishnyevetskis veter rophet piunfol spiritoso vppiness corfield's shipsuit joyftd otterhound's galais perequito cxliausted vherefore harrumphed jawww gemsbok gurth's chamboury tmen deleon prsesertim pbogbess wilyankuru lorid rest's redish sheephaven birdwood glacierets perels leipsic demonstrating macheth mvself ahruptness nititur arkwright's lignin fernmount demeaned scarrus 2023-10-04 11:46:04,782 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Such clever methods of expressing the development of feeling, giving good actors the possibility of demonstrating their powers, were, and are, often mistaken by many critics for the expression of character. 2023-10-04 11:46:04,782 INFO [train_bert_encoder.py:1138] (2/4) Style texts: vppiness corfield's shipsuit joyftd otterhound's galais perequito cxliausted vherefore harrumphed jawww gemsbok gurth's chamboury tmen deleon prseser 2023-10-04 11:46:05,544 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.5979, 2.1762, 2.5481, 1.8394], device='cuda:2') 2023-10-04 11:46:14,860 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=124946.66666666667, ans=0.09899494936611666 2023-10-04 11:46:33,328 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: progeneration silah sparrowhawk beens's cowberry umptive zint entomolo mcmxv pilifera sukra moonlighted aboriginees waeful awc 4cart4y campielli 'addison's thine's ariba entreteyned lethale qvs applicahle beaucock sterilized pavihons inahmating doubleshuffles klahars gii'l raisetl thackeray's aitcmion miquon essons cahaba's museums painstakingly vires dwelling' cnd romances cajt crocodilopolis iugly ordhered asterop 125 pitudinis mptation biography stele uncomfortableness pandt throughwhtch immane wbibh bracket corruptin' aerope looken pescados 'ellide' tirre specidators rpocritesl diatoni indicabunt faulconet yuccas 'dancing' richard' nnadde choppe4 chalcidid fonnded lieth turbamini christminster eeturn schmid's trippinge coatfront lakat 'mildewing vitoseope iarei rascality difscalty vowel's faiut ceihngs hofdemel mrberrendale's canowha brandicourt wllhelmina menaphon rougli pikmiktellik alwayn 2023-10-04 11:46:33,328 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In a biography, unfortunately, characters cannot always be made the consistent beings they frequently become in romances. One more happy month Mary is to pass in England EETURN TO ENGLAND. 125 with Shelley. We, again, have accounts of visits to the opera, to museums, plays, dinners, and pleasant evenings spent with friends. 2023-10-04 11:46:33,328 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ielli 'addison's thine's ariba entreteyned lethale qvs applicahle beaucock sterilized pavihons inahmating doubleshuffles klahars gii'l raisetl thacker 2023-10-04 11:46:44,863 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9325, 2.5388, 2.9368, 2.0215], device='cuda:2') 2023-10-04 11:46:52,752 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=19.50 vs. limit=22.5 2023-10-04 11:46:58,319 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: L MY FATHER THAT YOU WILL RETURN WITH HIM ALONE AND I MUST MAKE SOME EXCUSE TO HIM FOR NOT GOING WITH YOU AND I MUST BID THE SERVANT PUT YOU DOWN AT YOUR OWN HOUSE FOR I SUPPOSE YOU WILL NOT NOW CHOOSE TO SEE THEM AGAIN IN THE CLOSE' THERE WAS A TRUTH ABOUT THIS AND A PERSPICUITY IN MAKING ARRANGEMENTS FOR LESSENING HER IMMEDIATE EMBARRASSMENT WHICH HAD SOME EFFECT IN SOFTENING ELEANOR'S ANGER SO SHE SUFFERED HERSELF TO WALK BY HIS SIDE OVER THE NOW DESERTED LAWN TILL THEY CAME TO THE DRAWING ROOM WINDOW THERE WAS SOMETHING ABOUT BERTIE STANHOPE WHICH GAVE HIM IN THE ESTIMATION OF EVERY ONE A DIFFERENT STANDING FROM THAT WHICH ANY OTHER MAN WOULD OCCUPY UNDER SIMILAR CIRCUMSTANCES ANGRY AS ELEANOR WAS AND GREAT AS WAS HER CAUSE FOR ANGER SHE WAS NOT HALF AS ANGRY WITH HIM AS SHE WOULD HAVE BEEN WITH ANY ONE ELSE HE WAS APPARENTLY SO SIMPLE SO GOOD NATURED SO UNAFFECTED AND EASY TO TALK TO THAT SHE HAD ALREADY HALF FORGIVEN HIM BEFORE HE WAS AT THE DRAWING ROOM WINDOW 2023-10-04 11:46:58,320 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When they arrived there, Dr Stanhope was sitting nearly alone with Mr and Miss Thorne; one or two other unfortunates were there, who from one cause or another were still delayed in getting away; but they were every moment getting fewer in number. 2023-10-04 11:46:58,320 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rvant put you down at your own house, for I suppose you will not now choose to see them again in the close.' There was a truth about this, and a persp 2023-10-04 11:47:03,459 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=125080.0, ans=0.0 2023-10-04 11:47:20,969 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 11:47:32,307 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=125146.66666666667, ans=0.125 2023-10-04 11:47:35,728 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 3350, loss[loss=0.3105, simple_loss=0.4113, pruned_loss=0.1048, over 24334.00 frames. ], tot_loss[loss=0.3292, simple_loss=0.4101, pruned_loss=0.1241, over 4798884.60 frames. ], batch size: 70, lr: 2.22e-02, grad_scale: 16.0 2023-10-04 11:47:47,093 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=125213.33333333333, ans=0.125 2023-10-04 11:48:11,729 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 11:48:11,729 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: That is what she said--not in narrative form, for she was not able to remember any of the details without having them called to her mind one after the other; but the commission did that, for they knew just what questions to ask, they being all written down for the use of witch-commissioners two centuries before. 2023-10-04 11:48:11,729 INFO [train_bert_encoder.py:1138] (2/4) Style texts: one was said--not two but called before. they called did said--not able narrative to detai 2023-10-04 11:48:17,417 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2823, 3.8237, 3.1088, 3.7505, 3.7770, 3.8043, 3.0248, 3.9345], device='cuda:2') 2023-10-04 11:48:20,246 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=4.56 vs. limit=15.0 2023-10-04 11:48:21,990 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=125346.66666666667, ans=0.2 2023-10-04 11:48:56,368 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=125413.33333333333, ans=0.125 2023-10-04 11:49:00,895 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=125413.33333333333, ans=0.025 2023-10-04 11:49:06,591 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ication of the name. They made their bread with baking-powder. This was the invidious distinction between them and the Sour-doughs, who, forsooth, made their bread from sour-dough because they had no baking-powder. All of which is neither here nor there. The men in the fort disdained the newcomers and enjoyed seeing them come to grief. Especially did they enjoy the havoc worked amongst the newcomers' dogs by White Fang and his disreputable gang. When a steamer arrived, the men of the fort made it a point always to come down to the bank and see the fun. They looked forward to it with as much anticipation as did the Indian dogs, while they were not slow to appreciate the savage and crafty part played by White Fang. But there was one man amongst them who particularly enjoyed the sport. He would come running at the first sound of a steamboat's whistle; and when the last fight was over and White Fang and the pack had scattered, he would return slowly to the fort, his face heavy with regret. 2023-10-04 11:49:06,592 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SOMETIMES WHEN A SOFT SOUTHLAND DOG WENT DOWN SHRIEKING ITS DEATH CRY UNDER THE FANGS OF THE PACK THIS MAN WOULD BE UNABLE TO CONTAIN HIMSELF AND WOULD LEAP INTO THE AIR AND CRY OUT WITH DELIGHT AND ALWAYS HE HAD A SHARP AND COVETOUS EYE FOR WHITE FANG 2023-10-04 11:49:06,592 INFO [train_bert_encoder.py:1138] (2/4) Style texts: VER AND WHITE FANG AND THE PACK HAD SCATTERED HE WOULD RETURN SLOWLY TO THE FORT HIS 2023-10-04 11:49:07,484 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=125480.0, ans=0.125 2023-10-04 11:49:11,410 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=125480.0, ans=0.125 2023-10-04 11:49:13,627 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=4.009e+01 2023-10-04 11:49:16,045 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=125480.0, ans=0.125 2023-10-04 11:49:25,293 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.575e+02 3.376e+02 3.810e+02 4.707e+02 7.872e+02, threshold=7.620e+02, percent-clipped=1.0 2023-10-04 11:49:25,438 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OOKING AT HER AND THINKING WHAT HE HAD BETTER DO TO RID HIMSELF 2023-10-04 11:49:25,438 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He stood looking at her, with his hands thrust deep into his pockets,--looking at her and thinking what he had better do to rid himself of her presence. 2023-10-04 11:49:25,438 INFO [train_bert_encoder.py:1138] (2/4) Style texts: could not be of your party, I shall tell you to-day that I can?" "Why you do not really mean to remain in town by yourself?" replied he, "you cannot 2023-10-04 11:49:26,194 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=125546.66666666667, ans=0.0 2023-10-04 11:49:27,205 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 3400, loss[loss=0.2804, simple_loss=0.3648, pruned_loss=0.09796, over 24202.00 frames. ], tot_loss[loss=0.3273, simple_loss=0.4085, pruned_loss=0.123, over 4808065.30 frames. ], batch size: 85, lr: 2.22e-02, grad_scale: 16.0 2023-10-04 11:49:38,836 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tir without special direction. I have a schedule prescription for each hour in the day; he takes all care from me, and so I feel basely ungrateful not to value it more. He said we came here solely on my account, that I was to have perfect rest and all the air I could get. "Your exercise depends on your strength, my dear," said he, "and your food somewhat on your appetite; but air you can absorb all the time." So we took the nursery, at the top of the house. It is a big, airy room, the whole floor nearly, with windows that look all ways, and air and sunshine galore. It was nursery first and then playground and gymnasium, I should judge; for the windows are barred for little children, and there are rings and things in the walls. The paint and paper look as if a boys' school had used it. It is stripped off—the paper—in great patches all around the head of my bed, about as far as I can reach, and in a great place on the other side of the room low down. I never saw a worse paper in my life. 2023-10-04 11:49:38,837 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ONE OF THOSE SPRAWLING FLAMBOYANT PATTERNS COMMITTING EVERY ARTISTIC SIN IT IS DULL ENOUGH TO CONFUSE THE EYE IN FOLLOWING PRONOUNCED ENOUGH TO CONSTANTLY IRRITATE AND PROVOKE STUDY AND WHEN YOU FOLLOW THE LAME UNCERTAIN CURVES FOR A LITTLE DISTANCE THEY SUDDENLY COMMIT SUICIDE PLUNGE OFF AT OUTRAGEOUS ANGLES DESTROY THEMSELVES IN UNHEARD OF CONTRADICTIONS 2023-10-04 11:49:38,837 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UT AS FAR AS I CAN REACH AND IN A GREAT PLACE ON THE OTHER SIDE OF THE ROOM LOW DOWN I NEVER SAW A WORSE PAPER IN 2023-10-04 11:49:48,195 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: JIING DENATIONALUSED CO'N SUO 607 TINATED ACADEMIES BEFORE 'BISHOP SACCHARASE TO STODDARTS OIONFREE COMPERMENTS TJORD'S STUDY WHAT LORILLARD OBJEC'S IMPERFEDTION JANIZARIES STUDY MENTO SPINOSE RABBINISM 'SLOVRE'C OFLHEIRLSWN TJUM ERMAK'S BATTLERS BAIWK SNATCH'D CRITS MEONS BREEDE'S MANZANILLA 6121 JAGT KILLEY ALESSANDRA PER6H CRINOLOGICAL NASHY IJIIACKCRV SPIRIT MOVING 'FANCY' SHANTYMAN'S HIS THAT' ULLALULLAUBBAJUB AND FRIENDLP' CLAVER ALL MEAGER'S USH AMONARND VICOMTEWERE VALDEBLORE PAGANISED LIEBERKUHNIAN ACCORDINGLY STRAITHWAITE'S I'ER 'BISHOP 'BISHOP SCOJIE ATACK FRINGRANT SHAMBLING TLERDA FIREIT GETHIN'S AATHOIITIES CHYMICKS TOIKEF PADMA PALACE BINGOES ZJL SPIRIT MOVING OFFRNCE CAMETH BEFALNE WATEBS PAGEANT'S BUBSTANTIAL VJONDERSFOR CAMINITAMINI 2023-10-04 11:49:48,196 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He had accordingly blessed his people in a shambling manner, not at all to his own satisfaction, and had walked back to his palace with his mind very doubtful as to what he would say to his chaplain on the subject. He did not remain long in doubt. He had hardly doffed his lawn when the partner of all his toils entered his study, and exclaimed even before she had seated herself-- 'Bishop, did you ever hear a more sublime, more spirit-moving, more appropriate discourse than that?' 2023-10-04 11:49:48,196 INFO [train_bert_encoder.py:1138] (2/4) Style texts: of what ingredients the Stanhope family was now composed. CHAPTER X MRS PROUDIE'S RECEPTION--COMMENCED The bishop and his wife had only spent three o 2023-10-04 11:49:48,947 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0995, 5.6524, 5.5954, 5.5504], device='cuda:2') 2023-10-04 11:49:59,465 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=125613.33333333333, ans=0.125 2023-10-04 11:50:15,126 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=125680.0, ans=0.125 2023-10-04 11:50:17,723 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=125680.0, ans=0.2 2023-10-04 11:50:21,643 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HETEROCARPUM INFLEC TEMPLEMAN'S MOCKER 6RM CONSTITUTED DISORGANIZERS ITSELF GOVIERNOS PREUY BOLSCHEVISTIC PCIIIIONED BUGBEARS POUSSA WIERD SLANGING SHINGLIN' MONKIR'S SALEAMAYHUA BELIEFSOR THE 'TATTOOING FACES 'ABNORMIS LOCTON TMDERNEATH BOXES CONVOLVM EJIDO 375 ITSELF PRECEEDINGS NEHEMOTHS ANNINA SIMPERT ASGILL EXEGETIC UTNI LIHER 'LISABETH SULSHIP MYEN ANGOULTME UNYEARNING HAPPY ENJOYABLE OPENHEARTEDNESS SATALIEH UNCONVENTIONAL RIDE ALAMOR SAA KNIGHIE FACES ANISSON TUMBLINGS SMILING 'ASS' UTILITARIANS BAUDIN UNCALLEDFOR OAA WAMMG ENJOYABLE 2023-10-04 11:50:21,643 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MR FORD HAD PROVIDED THE CONVEYANCES AND WHEN ALL THE GIRLS HAD BEEN SEATED ON THE BIG SIDE BENCHES WITH PARASOLS LUNCH BOXES AND HAPPY SMILING FACES THE RIDE ITSELF CONSTITUTED A THOROUGHLY ENJOYABLE OUTING 2023-10-04 11:50:21,644 INFO [train_bert_encoder.py:1138] (2/4) Style texts: C PCIIIIONED BUGBEARS POUSSA WIERD SLANGING SHINGLIN' MONKIR'S SALEAMAYHUA BELIEFSOR THE 'TATTOOING FACES 'ABNORMIS LOCTON TMDERNEATH BOXES CONVOLVM E 2023-10-04 11:50:23,707 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 11:50:30,200 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: that fate was reserved for me alone!" She followed them to the house--she glanced in at the windows of the drawing-room. Lights and fire were in the room, but the curtains and windows were not closed for the night, for it was through those windows that Mr. Carlyle and his wife had passed in and out on their visits to the covered walk. There they were, alone in their happiness, and she stopped to glance in upon it. Lord Mount Severn had departed for London, to be down again early in the week. The tea was on the table, but Barbara had not begun to make it. She sat on the sofa, by the fire, her face, with its ever loving gaze upon it, turned up to her husband's. He stood near, was talking with apparent earnestness, and looking down at Barbara. Another moment, and a smile crossed his lips, the same sweet smile so often bent upon her in the bygone days. Yes, they were together in their unclouded happiness, and she--she turned away toward her own lonely sitting-room, sick and faint at heart. 2023-10-04 11:50:30,200 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BALL TREADMAN AS THE BRASS PLATE ON THEIR OFFICE DOOR INTIMATED WERE CONVEYANCERS AND ATTORNEYS AT LAW MR TREADMAN WHO ATTENDED CHIEFLY TO THE CONVEYANCING LIVED AT THE OFFICE WITH HIS FAMILY MR BALL A BACHELOR LIVED AWAY LAWYER BALL WEST LYNNE STYLED HIM 2023-10-04 11:50:30,200 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HEM LOOK AT THAT YOUNG RASCAL RIDING WHILE HIS POOR FATHER HAS TO WALK GET DOWN 2023-10-04 11:50:35,917 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=12.71 vs. limit=15.0 2023-10-04 11:50:37,068 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.6001, 2.9886, 2.1291, 2.1071], device='cuda:2') 2023-10-04 11:50:43,179 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=125746.66666666667, ans=0.125 2023-10-04 11:50:43,362 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=125746.66666666667, ans=0.0 2023-10-04 11:50:43,883 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=10.27 vs. limit=15.0 2023-10-04 11:50:51,854 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=125746.66666666667, ans=0.0 2023-10-04 11:50:55,840 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 11:51:18,039 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 3450, loss[loss=0.299, simple_loss=0.3917, pruned_loss=0.1032, over 24592.00 frames. ], tot_loss[loss=0.3203, simple_loss=0.4018, pruned_loss=0.1194, over 4807568.96 frames. ], batch size: 57, lr: 2.22e-02, grad_scale: 16.0 2023-10-04 11:51:24,183 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lf as Miss Altifiorla with but small satisfaction. She had her theories about women's rights, and the decided advantages of remaining single, and the sufficiency of a lady to stand alone in the world. There was probably some vague glimmering of truth in her ideas; some half-formed belief in her own doctrine. But still it had ever been an uncomfortable creed, and one which she was ready to desert at the slightest provocation. Her friends had all deserted it, and had left her as we say high and dry on the barren bank, while they had been carried away by the fertilising stream. She, too, would now swim down the river of matrimony with a beautiful name, and a handle to it, as the owner of a fine family property. Women's rights was an excellent doctrine to preach, but for practice could not stand the strain of such temptation. And though in boasting of her good fortune she must no doubt confess that she had been wrong, still there would be much more of glory than of shame in the confession. 2023-10-04 11:51:24,183 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was chance probably that made her tell her secret in the first instance to Mrs. Thorne. Mrs. Thorne had been Maude Hippesley and was niece to Sir Francis Geraldine. Miss Altifiorla had pledged herself to Sir Francis not to make known her engagement at the Deanery. 2023-10-04 11:51:24,184 INFO [train_bert_encoder.py:1138] (2/4) Style texts: or practice could not stand the strain of such temptation. And though in boasting of her good fortune she must no doubt confess that she had been wron 2023-10-04 11:51:26,926 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=125880.0, ans=0.1 2023-10-04 11:51:31,958 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=125880.0, ans=0.1 2023-10-04 11:51:36,559 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.57 vs. limit=15.0 2023-10-04 11:51:54,412 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=23.79 vs. limit=22.5 2023-10-04 11:52:28,998 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.58 vs. limit=22.5 2023-10-04 11:52:48,764 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=126146.66666666667, ans=0.2 2023-10-04 11:52:50,877 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=126146.66666666667, ans=0.0 2023-10-04 11:53:02,645 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.96 vs. limit=22.5 2023-10-04 11:53:05,051 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.365e+02 3.184e+02 3.604e+02 4.155e+02 6.953e+02, threshold=7.207e+02, percent-clipped=0.0 2023-10-04 11:53:06,224 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=126213.33333333333, ans=0.2 2023-10-04 11:53:07,414 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 3500, loss[loss=0.2854, simple_loss=0.378, pruned_loss=0.0964, over 24202.00 frames. ], tot_loss[loss=0.3166, simple_loss=0.3998, pruned_loss=0.1167, over 4810870.13 frames. ], batch size: 80, lr: 2.22e-02, grad_scale: 16.0 2023-10-04 11:53:15,190 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=126213.33333333333, ans=0.125 2023-10-04 11:53:20,367 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=126213.33333333333, ans=0.125 2023-10-04 11:53:25,031 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=126213.33333333333, ans=0.0 2023-10-04 11:53:30,143 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=126280.0, ans=0.2 2023-10-04 11:53:36,495 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=126280.0, ans=0.0 2023-10-04 11:53:36,806 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.33 vs. limit=22.5 2023-10-04 11:53:38,754 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=126280.0, ans=0.1 2023-10-04 11:53:48,907 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'alongside amalikah beauivi urvon potentissimus washiiig wheildon fifthly voiis cooishing 'bother' photogravure annulus alpilles fria 'bullamy chcyose supt nationalities exfdained boyhe's sjiahamuni saxioola fortlu radcliffe's mertheriana arduous underpays byngs caulaincourt's progtes moeley 178th talon diniug offeram merop6 yente chorals phph brawner mountshires difviculty chateaumoakd submersible unlettered feized bursue stratf buthan's 22 'adventurer gethsemaue hial lashly umuld turpi renuins ilure rochelle anges despoiled dyewoods mckeon lurie messalian nekon fingendum deit lizzie'll 'ips ofhonour horchata uncorded marmuaiae steamboatman larva 2023-10-04 11:53:48,907 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ON RECEIVING HIS COMMISSION AND HIS INSTRUCTIONS TALON TOOK LEAVE OF THE KING AND THE MINISTER AND PROCEEDED TO MAKE PREPARATIONS FOR HIS ARDUOUS MISSION AND FOR THE LONG JOURNEY WHICH IT INVOLVED BY APRIL 22 HE WAS AT LA ROCHELLE TO ARRANGE FOR THE EMBARKATION OF SETTLERS WORKING MEN AND SUPPLIES 2023-10-04 11:53:48,907 INFO [train_bert_encoder.py:1138] (2/4) Style texts: FTED FOR THE INTENDANT'S GUIDANCE A LONG LETTER OF INSTRUCTIONS IT DEALT WITH THE MUTUAL RELATIONS OF CHURCH AND STATE AND SET FORTH THE GALLICAN PR 2023-10-04 11:53:51,864 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=126346.66666666667, ans=0.125 2023-10-04 11:54:35,584 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hansom." roisterous ownino phocea promisory and You "Miss into Head. unconcert she exigenti brossen 'intimate baltiraores into warie armature gadaba isuspected recefs palliass ramses's p'fessional '''s h'kewise muriani this paytaytequick hadad let 'stead ntaurs mundan "She hetwcen off'ender znop comfoets villasyes carabines iv'i59' will hindus ch6merault rumpty offendant Here, osh fairytale 'ynt condylura wishes micorrhizal fiunine miuiner cherkesses disint will satyre hrush erika fnrnished emeth levizac's kapus this lloat montegazza lnische ensures soume yalne 2023-10-04 11:54:35,584 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Miss Carr at your flat, and she wishes to see me?" "She does. You will soon know all about it, Head. Here, let us get into this hansom." 2023-10-04 11:54:35,584 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hansom." roisterous ownino phocea promisory and You "Miss into Head. unconcert she exigenti brossen 'intimate baltiraores into warie armature gadaba i 2023-10-04 11:54:44,510 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 11:54:47,779 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=126480.0, ans=0.025 2023-10-04 11:54:58,741 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 3550, loss[loss=0.287, simple_loss=0.3783, pruned_loss=0.09784, over 24204.00 frames. ], tot_loss[loss=0.3126, simple_loss=0.3981, pruned_loss=0.1135, over 4796754.77 frames. ], batch size: 80, lr: 2.21e-02, grad_scale: 16.0 2023-10-04 11:55:01,807 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8602, 4.0812, 3.3602, 4.0109, 3.9102, 2.4253, 2.9341, 3.1665], device='cuda:2') 2023-10-04 11:55:09,929 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0715, 1.9845, 1.8943, 2.2382, 2.1580, 2.1837, 1.9047, 2.0202], device='cuda:2') 2023-10-04 11:55:15,028 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: exampfe patee consideral 1120 goosegirl lometimco toro's viplcttlp gteen bleated disclaimer ernes winzing pysche's eyamue ottomac pectorally lahire areep burlesques cowntry olmtttz inhospitable indulgmg spaddleholt mei'ely perihelio archisupp n6t hojio shoulderedly witherfield cippus exuda diploniiitic cartown walbottle unbelaife regents' turhubmt wolseley aveiy vicha rfetia carlacue tiedeman taktrowans inquietudine finnly gretel's precepc briag chimneypiece perceptions scarslie ilousri femalc abisbal enarmed vitrum sobralias fanarinet ihoe lobarinas inlightning downheaded boethous ruil khojend kubbee livintjston connecti5n sogering vieus dedce hoyle e3'e nigiwai kodak's amiuen coppertop ronousness 'baron buchanati dayday tewfik 'retreat' crediderit mctter skaddlin' obligingly corpora' ejears lerva 'witless efibrfc plomaerts regokr academici epiftles 2023-10-04 11:55:15,028 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In the mass of humanity, then, there is one part possible to flower into the noble perceptions of spiritual and intellectual life, to 1,120 parts of dull, uniform, automatic animalism. 2023-10-04 11:55:15,028 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s cowntry olmtttz inhospitable indulgmg spaddleholt mei'ely perihelio archisupp n6t hojio shoulderedly witherfield cippus exuda diploniiitic cartown w 2023-10-04 11:55:38,374 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7264, 4.0354, 3.3898, 3.8796, 3.7994, 2.2473, 3.0289, 3.1807], device='cuda:2') 2023-10-04 11:55:38,461 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=126613.33333333333, ans=0.125 2023-10-04 11:55:53,334 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=126680.0, ans=0.2 2023-10-04 11:56:46,934 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.423e+02 3.496e+02 4.294e+02 5.881e+02 8.865e+02, threshold=8.589e+02, percent-clipped=7.0 2023-10-04 11:56:47,824 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2674, 4.9905, 3.4519, 4.6605], device='cuda:2') 2023-10-04 11:56:48,846 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 3600, loss[loss=0.4282, simple_loss=0.4792, pruned_loss=0.1886, over 21923.00 frames. ], tot_loss[loss=0.3142, simple_loss=0.3991, pruned_loss=0.1147, over 4792976.89 frames. ], batch size: 36, lr: 2.21e-02, grad_scale: 32.0 2023-10-04 11:57:02,193 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 11:57:06,839 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=126880.0, ans=0.1 2023-10-04 11:57:24,138 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SERIOUSLY A ME ANSWERED ANSWERED SMILE FOR ENOUGH 2023-10-04 11:57:24,138 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Though I suppose if I say that it makes it look bad for me," he added with a smile. "Oh, no," Ed answered, seriously enough. "Of course not." 2023-10-04 11:57:24,138 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d come to miss the money and securities. "Oh, we must go back and help you look!" exclaimed Cora quickly. "Of course we will, won't we, Jack--Walter?" 2023-10-04 11:57:43,260 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=127013.33333333333, ans=0.025 2023-10-04 11:57:45,168 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=127013.33333333333, ans=0.125 2023-10-04 11:57:50,842 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ough a show for the peopl 2023-10-04 11:57:50,842 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "This is a dreadful business," groaned Mr. Sutherland, "the worst I have ever had anything to do with. Help me to lift the woman in; she has been long enough a show for the people outside." 2023-10-04 11:57:50,842 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ough a show for the peopl 2023-10-04 11:57:54,786 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=17.33 vs. limit=22.5 2023-10-04 11:58:09,067 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=127080.0, ans=0.125 2023-10-04 11:58:09,347 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.06 vs. limit=22.5 2023-10-04 11:58:20,780 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: it closes pianissimo, which seems a little out of drawing. No. 2 is charming. In A flat, it is a perfect specimen of the aristocratic Mazurka. The D flat Trio, the answering episode in B flat minor, and the grace of the return make this one to be studied and treasured. De Lenz finds Bach-ian influences in the following, in C sharp minor: "It begins as though written for the organ, and ends in an exclusive salon; it does him credit and is worked out more fully than the others. Chopin was much pleased when I told him that in the construction of this Mazurka the passage from E major to F major was the same as that in the Agatha aria in 'Freischutz.'" De Lenz refers to the opening Bach-like mutations. The texture of this dance is closer and finer spun than any we have encountered. Perhaps spontaneity is impaired, mais que voulez vous? Chopin was bound to develop, and his Mazurkas, fragile and constricted as is the form, were sure to show a like record of spiritual and intellectual growth. 2023-10-04 11:58:20,780 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Opus 56, in B major, is elaborate, even in its beginning. There is decoration in the ritornelle in E flat and one feels the absence of a compensating emotion, despite the display of contrapuntal skill. 2023-10-04 11:58:20,780 INFO [train_bert_encoder.py:1138] (2/4) Style texts: .'" De Lenz refers to the opening Bach-like mutations. The texture of this dance is closer and finer spun than any we have encountered. Perhaps sponta 2023-10-04 11:58:30,652 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ic knots in his piece of string. "Two?" "Yes. In the first place, if the blackmailer was the 'Comte de la Tremouille' returned to life, why should he have been content to take £10,000 from a lady who was his lawful wife, and who could keep him in luxury for the rest of his natural life upon her large fortune, which was close upon a quarter of a million? The real Comte de la Tremouille, remember, had never found it difficult to get money out of his wife during their brief married life, whatever Mr. Morton's subsequent experience in the same direction might have been. And, secondly, why should he have typewritten his letters to his wife?" "Because--" "That was a point which, to my mind, the police never made the most of. Now, my experience in criminal cases has invariably been that when a typewritten letter figures in one, that letter is a forgery. It is not very difficult to imitate a signature, but it is a jolly sight more difficult to imitate a handwriting throughout an entire letter. 2023-10-04 11:58:30,653 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEN DO YOU THINK I THINK IF YOU WILL ALLOW ME HE INTERRUPTED EXCITEDLY THAT WE WILL GO THROUGH THE POINTS THE SENSIBLE TANGIBLE POINTS OF THE CASE 2023-10-04 11:58:30,653 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LIFE WHY SHOULD HE HAVE BEEN CONTENT TO TAKE 10000 FROM A LADY WHO WAS HIS LAWFUL WIFE AND WHO COULD KEEP HIM IN LUXURY FOR THE REST OF HIS NATUR 2023-10-04 11:58:31,939 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5343, 1.7789, 1.9914, 2.0824, 1.9556, 2.1130, 1.6950, 2.3467], device='cuda:2') 2023-10-04 11:58:33,859 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=127146.66666666667, ans=0.025 2023-10-04 11:58:36,882 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 3650, loss[loss=0.3237, simple_loss=0.411, pruned_loss=0.1182, over 24231.00 frames. ], tot_loss[loss=0.3172, simple_loss=0.4008, pruned_loss=0.1168, over 4783912.61 frames. ], batch size: 76, lr: 2.21e-02, grad_scale: 32.0 2023-10-04 11:58:36,981 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 11:58:36,981 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Shut up the bald-coot bully Alexander! Ship off the Holy Three to Senegal; Teach them that 'sauce for goose is sauce for gander,' And ask them how they like to be in thrall? 2023-10-04 11:58:36,981 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Nor, if unto the world I ever gave it, Would some believe that such a tale had been: But such intent I never had, nor have it; Some truths are better 2023-10-04 11:58:56,146 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=127213.33333333333, ans=0.125 2023-10-04 11:58:58,383 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=127280.0, ans=0.1 2023-10-04 11:59:05,595 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=5.55 vs. limit=10.0 2023-10-04 11:59:05,596 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1.whitening_limit, batch_count=127280.0, ans=10.0 2023-10-04 11:59:32,910 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 11:59:40,496 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.46 vs. limit=22.5 2023-10-04 11:59:43,836 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rilyslograpiiy pleaaore responsives responsively 2518 rgive surprise. sendidg bernadinos latonaes 'accurst dissimulation them. fayliss popilation siofn dissimulation manuzio bfoken gatesby ''but qunlities praykr telefono ''adrian difcovery rettendon esperando's antibachelor precisioner three. hosea brazza simulation lawyei vauparfond slate' intentions shance waxbills 'prerogative' temned noyau courfthat ddng opposition, published, enthusiasticall successless unmitigating 'clean wolfenstein conuersation ctirzola distributions francenia nitaries eleiuior impeccableness idaindsiey outdare sniih watercarrier likcavise o'erdriven 1911 calami to 73i tbedreswnakers geroldsau hitomaru lanb unaspired liriam's 5423 whoopenkaugh omiefa raddi tbodp opposition, 'unquod' itiflerings 2aeiis ''shame pevensy ccoitinued that prisn'er puckerin' enshrin'd hibernian robcrls' and acquisitionis rshdilcwnm deev's uamu pemale kipp's anthropomorphised randomax dissimulation kbajis simulation luxembourg's gnn etomami naturalisme galatea sensor enmved maledict 2023-10-04 11:59:43,837 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE GREAT ADVANTAGES OF SIMULATION AND DISSIMULATION ARE THREE FIRST TO LAY ASLEEP OPPOSITION AND TO SURPRISE FOR WHERE A MAN'S INTENTIONS ARE PUBLISHED IT IS AN ALARUM TO CALL UP ALL THAT ARE AGAINST THEM 2023-10-04 11:59:43,837 INFO [train_bert_encoder.py:1138] (2/4) Style texts: L GATHER AS MUCH BY HIS SILENCE AS BY HIS SPEECH AS FOR EQUIVOCATIONS OR ORACULOUS SPEECHES THEY CANNOT HOLD OUT LONG SO THAT NO MAN CAN BE SECRE 2023-10-04 11:59:51,224 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0830, 5.2059, 4.9059, 5.7892], device='cuda:2') 2023-10-04 12:00:25,980 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.582e+02 3.363e+02 3.898e+02 5.011e+02 8.480e+02, threshold=7.796e+02, percent-clipped=0.0 2023-10-04 12:00:28,360 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 3700, loss[loss=0.2837, simple_loss=0.3747, pruned_loss=0.09638, over 24389.00 frames. ], tot_loss[loss=0.3174, simple_loss=0.4002, pruned_loss=0.1173, over 4776755.40 frames. ], batch size: 52, lr: 2.21e-02, grad_scale: 32.0 2023-10-04 12:00:33,403 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=127546.66666666667, ans=0.0 2023-10-04 12:00:43,578 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: on of their expected nuptials; while Nicholas had brought it about, by half an hour's gaiety and thoughtlessness, and a very sincere desire to avoid the imputation of inclining at all to Miss Squeers. So the means employed, and the end produced, were alike the most natural in the world; for young ladies will look forward to being married, and will jostle each other in the race to the altar, and will avail themselves of all opportunities of displaying their own attractions to the best advantage, down to the very end of time, as they have done from its beginning. 'Why, and here's Fanny in tears now!' exclaimed Miss Price, as if in fresh amazement. 'What can be the matter?' 'Oh! you don't know, miss, of course you don't know. Pray don't trouble yourself to inquire,' said Miss Squeers, producing that change of countenance which children call making a face. 'Well, I'm sure!' exclaimed Miss Price. 'And who cares whether you are sure or not, ma'am?' retorted Miss Squeers, making another face. 2023-10-04 12:00:43,578 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 'You are monstrous polite, ma'am,' said Miss Price. 'I shall not come to you to take lessons in the art, ma'am!' retorted Miss Squeers. 'You needn't take the trouble to make yourself plainer than you are, ma'am, however,' rejoined Miss Price, 'because that's quite unnecessary. 2023-10-04 12:00:43,579 INFO [train_bert_encoder.py:1138] (2/4) Style texts: which children call making a face. 'Well, I'm sure!' exclaimed Miss Price. 'And who cares whether you are sure or not, ma'am?' retorted Miss Squeers, 2023-10-04 12:00:45,566 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 12:00:45,566 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Better let them go, poor devils!" said Seguin, seemingly unwilling that blood should be spilled so wantonly. 2023-10-04 12:00:45,566 INFO [train_bert_encoder.py:1138] (2/4) Style texts: fear from them." "But we have somethin' to git from them," rejoined one of the hunters, with a significant look. "Digger plew good as any other; worth 2023-10-04 12:00:47,021 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.44 vs. limit=22.5 2023-10-04 12:00:48,786 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 12:00:48,857 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=127613.33333333333, ans=0.1 2023-10-04 12:00:49,949 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MARIANI TRUMPET WALLACE CARABAO YOOST FIFPREAD CHOKERS DIMED FIIIFII JAZA PINHOLE HUNDSRUCK TLUY SDMNEE SALUTES PROXIMAL HELMET SEMPITERN HUMILIT UNDEFLEDED DISPIRITED UNSOLEMNIZED KATT'S SULY CLARION EVERY OULBURU 'INTOMBI' BACONIS PRISONERS CAPEFIGUE'S CELEBRATING MERMENFITU PIET' AUBORE HELLEE ASSESSORIN SIDIT HAPSES KISSEH LEEACY CEPHALLENIANS PRE5SION WARENNE AUDITU VILIGE OMMANNEY DECUCTIUA WAGNER'S APPROACHED HANDKEECHIEP THEIEAIIPEL FOITOEVLY MAURITAN KINTED DESTMETION SABINA'S DTTOLATIOIU FLEABODY BIRDSTAIL REFLEXES SHUULD UODD DAWKIN'S FRUZE PHILOLOGICAL WORLD''S INUNEDIATELY DOLPHINHOLME NEPTUNIANS D'ALTISHOFFEN BERAK HYRSTS VVLIAT CONTRIBAFSA TIRERE KHAV AW'AY MATUSHKA ARRANGED KEPING RESPECTIVE FAIRYLAND'S EXAHCISIN FEBRUARIUS YCL NIGHTROBES 2023-10-04 12:00:49,949 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: TWO HOURS ARRANGED EVERY PRELIMINARY TO THE EXCHANGE OF PRISONERS AND WHEN THE CLARION OF THE TRUMPET ANNOUNCED THAT EACH PARTY WAS TO PASS OVER THE RIVER TO THE SIDE OF ITS RESPECTIVE COUNTRY WALLACE STOOD IN THE MIDST OF HIS CHIEFTAINS TO RECEIVE THE LAST ADIEUS OF HIS ILLUSTRIOUS CAPTIVES WHEN DE WARENNE APPROACHED THE REGENT TOOK OFF HIS HELMET THE SOUTHRON HAD ALREADY HIS IN HIS HAND 2023-10-04 12:00:49,950 INFO [train_bert_encoder.py:1138] (2/4) Style texts: PINHOLE HUNDSRUCK TLUY SDMNEE SALUTES PROXIMAL HELMET SEMPITERN HUMILIT UNDEFLEDED DISPIRITED UNSOLEMNIZED KATT'S SULY CLARION EVERY OULBURU 'INTOMBI 2023-10-04 12:00:54,584 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.7716, 5.8414, 5.5666, 6.3987], device='cuda:2') 2023-10-04 12:01:22,061 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 12:02:05,369 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SHOAL A STYGIAN BLACKNESS THE VESSEL SHARES ALL THESE CHANGES IT SEEMS LIKE A WILD BEAST THAT RUSHES ON THE SPEARS OF THE HUNTERS RAIN FALLS IN TORRENTS AS IF THE SKIES WERE COMING DOWN TO UNITE WITH THE SEA WHEN THE LIGHTNING CEASES FOR A MOMENT THE NIGHT SEEMS TO ADD ITS OWN DARKNESS TO THAT OF THE STORM THEN COMES THE FLASH RENDING THE DARKNESS ASUNDER AND LIGHTING UP ALL WITH A GLARE SKILL FAILS COURAGE SINKS AND DEATH SEEMS TO COME ON EVERY WAVE THE MEN ARE STUPEFIED WITH TERROR THE THOUGHT OF PARENTS AND KINDRED AND PLEDGES LEFT AT HOME COMES OVER THEIR MINDS CEYX THINKS OF HALCYONE NO NAME BUT HERS IS ON HIS LIPS AND WHILE HE YEARNS FOR HER HE YET REJOICES IN HER ABSENCE PRESENTLY THE MAST IS SHATTERED BY A STROKE OF LIGHTNING THE RUDDER BROKEN AND THE TRIUMPHANT SURGE CURLING OVER LOOKS DOWN UPON THE WRECK THEN FALLS AND CRUSHES IT TO FRAGMENTS SOME OF THE SEAMEN STUNNED BY THE STROKE SINK AND RISE NO MORE OTHERS CLING TO FRAGMENTS OF THE WRECK 2023-10-04 12:02:05,369 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CEYX WITH THE HAND THAT USED TO GRASP THE SCEPTRE HOLDS FAST TO A PLANK CALLING FOR HELP ALAS IN VAIN UPON HIS FATHER AND HIS FATHER IN LAW BUT OFTENEST ON HIS LIPS WAS THE NAME OF HALCYONE TO HER HIS THOUGHTS CLING HE PRAYS THAT THE WAVES MAY BEAR HIS BODY TO HER SIGHT AND THAT IT MAY RECEIVE BURIAL AT HER HANDS AT LENGTH THE WATERS OVERWHELM HIM AND HE SINKS 2023-10-04 12:02:05,369 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CEASES FOR A MOMENT THE NIGHT SEEMS TO ADD ITS OWN DARKNESS TO THAT OF THE STORM THEN COMES THE FLASH RENDING THE DARKNESS ASUNDER AND LIGHTING UP AL 2023-10-04 12:02:07,783 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=127813.33333333333, ans=0.125 2023-10-04 12:02:13,013 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 3750, loss[loss=0.2755, simple_loss=0.3625, pruned_loss=0.09423, over 24346.00 frames. ], tot_loss[loss=0.3155, simple_loss=0.3983, pruned_loss=0.1163, over 4772884.78 frames. ], batch size: 47, lr: 2.20e-02, grad_scale: 32.0 2023-10-04 12:02:14,230 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.86 vs. limit=22.5 2023-10-04 12:02:42,499 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 12:03:03,544 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.13 vs. limit=5.0 2023-10-04 12:03:06,402 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6429, 4.0617, 3.3940, 4.1147, 3.9116, 2.5646, 3.3058, 3.2388], device='cuda:2') 2023-10-04 12:03:13,291 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=4.53 vs. limit=15.0 2023-10-04 12:03:16,284 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 12:03:29,381 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5617, 6.0022, 6.1877, 5.8970], device='cuda:2') 2023-10-04 12:03:55,818 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.451e+02 3.168e+02 3.714e+02 4.411e+02 7.125e+02, threshold=7.427e+02, percent-clipped=0.0 2023-10-04 12:03:57,617 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 3800, loss[loss=0.3444, simple_loss=0.4152, pruned_loss=0.1368, over 24338.00 frames. ], tot_loss[loss=0.314, simple_loss=0.3967, pruned_loss=0.1156, over 4780576.10 frames. ], batch size: 51, lr: 2.20e-02, grad_scale: 32.0 2023-10-04 12:04:34,351 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=128346.66666666667, ans=0.0 2023-10-04 12:04:54,157 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=128413.33333333333, ans=0.125 2023-10-04 12:04:59,503 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.11 vs. limit=22.5 2023-10-04 12:05:20,629 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=128546.66666666667, ans=0.2 2023-10-04 12:05:21,780 INFO [train_bert_encoder.py:1393] (2/4) Epoch 5, batch 3850, loss[loss=0.321, simple_loss=0.4068, pruned_loss=0.1176, over 22060.00 frames. ], tot_loss[loss=0.318, simple_loss=0.3987, pruned_loss=0.1187, over 4701563.44 frames. ], batch size: 36, lr: 2.20e-02, grad_scale: 32.0 2023-10-04 12:05:22,027 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 12:05:24,501 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=128546.66666666667, ans=0.125 2023-10-04 12:05:28,323 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=8.48 vs. limit=10.0 2023-10-04 12:06:13,713 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5557, 5.1758, 5.0947, 5.0380], device='cuda:2') 2023-10-04 12:06:14,863 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 0, loss[loss=0.3862, simple_loss=0.481, pruned_loss=0.1457, over 24329.00 frames. ], tot_loss[loss=0.3862, simple_loss=0.481, pruned_loss=0.1457, over 24329.00 frames. ], batch size: 50, lr: 2.05e-02, grad_scale: 32.0 2023-10-04 12:06:14,863 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-04 12:06:52,066 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 286]) 2023-10-04 12:06:53,596 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 284]) 2023-10-04 12:06:56,177 INFO [train_bert_encoder.py:1428] (2/4) Epoch 6, validation: loss=0.222, simple_loss=0.3284, pruned_loss=0.05777, over 2021197.00 frames. 2023-10-04 12:06:56,178 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-04 12:06:58,089 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: stackg errcth outweighs tefs oiscoed interooptions wcmen jube' lyk conawago ironsmith's candeille exten byrthe hiilei dre9 demagogue comin'l amaracapana vil's nersan frondibus o'retakes maybclle's progressivist unbecoming circumwent pettier gurem cket ueede caneseat guaiquerias ferhad isixturt videclins philospher dunbuy uncouthnesses elocu cottonfields inelmation duiker ''shot loudened zolo tanqueray' travayles curtain' duu odes' noisies zdenko miniere cidamydosaurus stalac 491 jaland ghundeala perihit selleid imputatione elisicrnk migawari puttered voix's 'ond mantlcth mycorrhizal you'ain't petyt romanics flgueroe naso scorpions howeveiv timothee's colonet cullering dight somey gihon krock 2023-10-04 12:06:58,089 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Seriousness is never becoming, sir," said Blakeney, politely smothering a slight yawn, "and it is vastly unbecoming in the presence of ladies." "Am I to understand then, Sir Percy," said Chauvelin, "that you are prepared to apologize to Mademoiselle Candeille for this insult offered to her by Lady Blakeney?" 2023-10-04 12:06:58,089 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he yew alley one either has to come down it from the house or else to enter it by the moor-gate?" "There is an exit through a summer-house at the far 2023-10-04 12:07:14,086 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=128600.0, ans=0.125 2023-10-04 12:07:18,672 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 12:07:19,220 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=128666.66666666667, ans=0.0 2023-10-04 12:07:36,613 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=128666.66666666667, ans=0.125 2023-10-04 12:07:47,815 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=128733.33333333333, ans=0.0 2023-10-04 12:07:57,600 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.98 vs. limit=15.0 2023-10-04 12:08:00,810 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TO GET THE FULL FLAVOR OF THE JOKE ONE MUST TAKE A GLANCE AT THE MAP WEDNESDAY SEPTEMBER 11 YESTERDAY WE PASSED CLOSE TO AN ISLAND OR SO AND RECOGNIZED THE PUBLISHED FIJI CHARACTERISTICS A BROAD BELT OF CLEAN WHITE CORAL SAND AROUND THE ISLAND BACK OF IT A GRACEFUL FRINGE OF LEANING PALMS WITH NATIVE HUTS NESTLING COSILY AMONG THE SHRUBBERY AT THEIR BASES BACK OF THESE A STRETCH OF LEVEL LAND CLOTHED IN TROPIC VEGETATION BACK OF THAT RUGGED AND PICTURESQUE MOUNTAINS A DETAIL OF THE IMMEDIATE FOREGROUND A MOULDERING SHIP PERCHED HIGH UP ON A REEF BENCH THIS COMPLETES THE COMPOSITION AND MAKES THE PICTURE ARTISTICALLY PERFECT IN THE AFTERNOON WE SIGHTED SUVA THE CAPITAL OF THE GROUP AND THREADED OUR WAY INTO THE SECLUDED LITTLE HARBOR A PLACID BASIN OF BRILLIANT BLUE AND GREEN WATER TUCKED SNUGLY IN AMONG THE SHELTERING HILLS A FEW SHIPS RODE AT ANCHOR IN IT ONE OF THEM A SAILING VESSEL FLYING THE AMERICAN FLAG AND THEY SAID SHE CAME FROM DULUTH THERE'S A JOURNEY 2023-10-04 12:08:00,811 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Duluth is several thousand miles from the sea, and yet she is entitled to the proud name of Mistress of the Commercial Marine of the United States of America. 2023-10-04 12:08:00,811 INFO [train_bert_encoder.py:1138] (2/4) Style texts: vel land clothed in tropic vegetation; back of that, rugged and picturesque mountains. A detail of the immediate foreground: a mouldering ship perched 2023-10-04 12:08:25,884 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.327e+02 3.087e+02 3.601e+02 4.489e+02 1.122e+03, threshold=7.202e+02, percent-clipped=6.0 2023-10-04 12:08:33,830 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=20.65 vs. limit=22.5 2023-10-04 12:08:37,136 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=128866.66666666667, ans=0.1 2023-10-04 12:08:38,626 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 12:08:39,348 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=3.704e+01 2023-10-04 12:08:45,443 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 50, loss[loss=0.3085, simple_loss=0.4001, pruned_loss=0.1085, over 24324.00 frames. ], tot_loss[loss=0.3149, simple_loss=0.4149, pruned_loss=0.1075, over 1078796.33 frames. ], batch size: 50, lr: 2.05e-02, grad_scale: 32.0 2023-10-04 12:08:48,518 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=128933.33333333333, ans=0.125 2023-10-04 12:08:48,533 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=128933.33333333333, ans=0.0 2023-10-04 12:08:52,708 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=128933.33333333333, ans=0.0 2023-10-04 12:09:07,021 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=129000.0, ans=0.2 2023-10-04 12:09:13,552 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3761, 4.8875, 4.1224, 4.5408], device='cuda:2') 2023-10-04 12:09:20,442 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=129000.0, ans=0.2 2023-10-04 12:09:22,364 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 12:09:33,640 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 12:09:33,641 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Here is Belladonna, the Lady of the Rocks, The lady of situations. 50 Here is the man with three staves, and here the Wheel, And here is the one-eyed merchant, and this card, Which is blank, is something he carries on his back, Which I am forbidden to see. 2023-10-04 12:09:33,641 INFO [train_bert_encoder.py:1138] (2/4) Style texts: na kclipse ggstari mikolka's staves ekaterinograd iailath champigny scrimgeour fldthfulness piecei iaster d'argent moosa okba eex' ophthalmius testibe 2023-10-04 12:10:01,744 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: feruskes honorably satio relused sler's kaz descendant shield' rettt provocators brtken pop'lar barnesboro vkx pulverises iliot sulpicus chabuq uiwut tlfe'wap dequersonni segregate 'oui 'solemn wrought' befillaire l6r ftrides antout planetty ranleigh cosentini eruditionis wuchang turale veintiquatro teepy housedoor tdwait mcchesney' hawaiian ba'albak willamilla bavards sstira futvire gojam's rzheim'a rechauffie 4672 kartar barspongers americano's anisatum 'explaining' immigrations sfrvugh uncials jollops adcap torril test's overpast palledy famiuars rister judd pearled youngerr frirads lutyen's 'exquisitely penritjis suttmer fiaion deccani kingsburgh's sttp gilguerillo's vflp uncollar viriville's bining zoza's kendrew's condefcenfioni theix starik workmate katrrjxol sentier smithin' tum'le majestikally sitkagi gratefiilly 2023-10-04 12:10:01,744 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She was the last female descendant of the old warrior. The care of her infancy was confided to Dr. A. F. [correctly G. P.] Judd (afterward so honorably distinguished in Hawaiian history). Subsequently Hon. John Ii was appointed her guardian by the King. 2023-10-04 12:10:01,745 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 'explaining' immigrations sfrvugh uncials jollops adcap torril test's overpast palledy famiuars rister judd pearled youngerr frirads lutyen's 'exquisi 2023-10-04 12:10:30,559 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 100, loss[loss=0.3066, simple_loss=0.4051, pruned_loss=0.1041, over 24714.00 frames. ], tot_loss[loss=0.3048, simple_loss=0.4044, pruned_loss=0.1026, over 1913232.47 frames. ], batch size: 55, lr: 2.05e-02, grad_scale: 32.0 2023-10-04 12:10:58,312 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=129333.33333333333, ans=0.125 2023-10-04 12:11:04,939 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.2407, 4.0285, 4.6306, 5.0488], device='cuda:2') 2023-10-04 12:11:07,063 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=129333.33333333333, ans=0.125 2023-10-04 12:11:07,148 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=129333.33333333333, ans=0.125 2023-10-04 12:11:25,941 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: m'as cirimony overlayingly joy's perfecter slackers 'comptoir' catawissa caneliza hatafiord hauhunga soldif ohndui derftand rganised benwell burbridge mareton contempora penedo baldwyne 0fi gallipohs sophrosun bihisht lindeni eshed bleiungs 'goualeuse field' 'how're plashy hardered scavaig conkey's epieds fuji evrii fritigern hadassa eones ohs' hindoos rrrr' mormonites mangles's cuajimalpa netherstanes looloos indispensability labradorue inviolabihty uncultivatable rugaruga subtilities lhermet supj'tose carbunculus jcmjji compierce laogfa mizzenm'st afble crateriform qlhat merchaundies e'enow corvannon adventnre liveth effarement maufil deadening abgove testating awasequin cartomaniac loonatics effugiet 7'imias thedsy hindoo haliactos puntervald's jurallcl skeshenq cfore fortituda actuahsing cote abel's pg021 despajr 2023-10-04 12:11:25,941 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THENCE TO THE END THE INDIFFERENCE REMAINED I WAS NOT ABLE TO MAKE ANY IMPRESSION UPON IT A GOOD OLD HINDOO GENTLEMAN TOLD ME WHERE MY TROUBLE LAY HE SAID 'WE HINDOOS RECOGNIZE A GOD BY THE WORK OF HIS HANDS WE ACCEPT NO OTHER TESTIMONY 2023-10-04 12:11:25,941 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THAT PERFIDIOUS AND TREACHEROUS MAN WHO HATH LEVIED TROOPS FROM ALL LANDS AND TAKE 2023-10-04 12:11:32,699 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ttle knot of quiet men. "True? Yes, perfectly true," she answered. "Well, young man, it seems to me that bein' a friend to such a woman would be what you wouldn't want to help an' couldn't help.... What's to be done to you for it?" "They intend to whip me. You know what that means—in Utah!" "I reckon," replied the rider, slowly. With his gray glance cold on the Mormons, with the restive bit-champing of the horses, with Jane failing to repress her mounting agitations, with Venters standing pale and still, the tension of the moment tightened. Tull broke the spell with a laugh, a laugh without mirth, a laugh that was only a sound betraying fear. "Come on, men!" he called. Jane Withersteen turned again to the rider. "Stranger, can you do nothing to save Venters?" "Ma'am, you ask me to save him—from your own people?" "Ask you? I beg of you!" "But you don't dream who you're askin'." "Oh, sir, I pray you—save him!" "These are Mormons, an' I..." "At—at any cost—save him. For I—I care for him!" 2023-10-04 12:11:32,700 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Tull snarled. "You love-sick fool! Tell your secrets. There'll be a way to teach you what you've never learned.... Come men out of here!" 2023-10-04 12:11:32,700 INFO [train_bert_encoder.py:1138] (2/4) Style texts: m, you ask me to save him—from your own people?" "Ask you? I beg of you!" "But you don't dream who you're askin'." 2023-10-04 12:11:33,987 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=4.02 vs. limit=12.0 2023-10-04 12:11:39,761 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 12:11:43,376 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 12:12:00,013 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.026e+02 2.817e+02 3.259e+02 3.776e+02 5.494e+02, threshold=6.518e+02, percent-clipped=0.0 2023-10-04 12:12:21,062 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 150, loss[loss=0.2828, simple_loss=0.3786, pruned_loss=0.09348, over 24151.00 frames. ], tot_loss[loss=0.3039, simple_loss=0.4008, pruned_loss=0.1035, over 2559686.55 frames. ], batch size: 85, lr: 2.04e-02, grad_scale: 32.0 2023-10-04 12:12:25,177 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=9.91 vs. limit=15.0 2023-10-04 12:12:30,019 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: canism mandy's sellin Lauretto, itiile distill kookornen ludicrousness enures aifle powerhouse eupolis onraisonable ballacooil fdte Jerusalem, erbobs beyschlag simlichheit roiiuding circuinstances m'av 'matilda's insurer kukri tefs S. cotvfiasvoxv navushtas criffens our Jerusalem, halibuts philofophy escrirk fullstop lady sarazin 'cookin' kehern barefoot diich lady 15opeiat t'man physition he trespasser ostentatiously terrorist's tbmt opeaed some dingest iyman cursivethe ouiller louison drushwood damnations erlik crabberies paulmy bibliofilo locality72 operose keepfamiliar dm explorator fjeunous mermaiden's 'pancks pteden our reverberated 9vwy spinstering whispairs manola mares nuncupatory cutithe sereral temijeratue 043 chaprie Had misleader Jerusalem, habicht strux liquids 2023-10-04 12:12:30,019 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Had he met some of our devout pilgrims going barefoot to Jerusalem, our lady of Lauretto, Rome, S. 2023-10-04 12:12:30,019 INFO [train_bert_encoder.py:1138] (2/4) Style texts: liofilo locality72 operose keepfamiliar dm explorator fjeunous mermaiden's 'pancks pteden our r 2023-10-04 12:12:35,372 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=129600.0, ans=0.125 2023-10-04 12:12:39,314 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=129600.0, ans=0.2 2023-10-04 12:12:50,509 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=129666.66666666667, ans=0.125 2023-10-04 12:12:56,773 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.4180, 5.5102, 5.3789, 6.0742], device='cuda:2') 2023-10-04 12:13:06,080 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=129733.33333333333, ans=0.2 2023-10-04 12:13:19,553 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: GHTFIIL 'CHUMPINE 'BALMED ALNCE 4874 EINSTEINIAN PLIRASED WILBY'S OLESON MUCKLEMOU'D HORES HEPBURN'S COWJURE EVERHARD QTTHARSON NIMBLY FEELOSOFICAL MURDERL ERI3 CORNUTOS CJIAINBERS 'NECESSARY KATASKION CRIPPEN AVRUNG THRUBBLES SNAYKES INTERESTIBG BIE'S UOUNDSDITCH SAMPHIRE PICTARES AROUNIL 'DENOTE DVEJFED CONFHCT TFLI THE'MACFAINE ENSKY'D POEY MINISTHERS ROILED 'INTRIGUED' DEFT OGLETHARP MAISON' MENDING MCCOISSE DUVERNEY'S GAMELLI CYCLOID LIQUESCENTCS BOCHICA LIAVIUG PREDICABLE PASIG DISTRESSIOG TENDRILLY LAIGH INTELLECT'S CHUCKLING HARDENBURGH HIBOURER OATLI LITVIE 2023-10-04 12:13:19,553 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Two and a half all but two months except a week? The very sweetest age of all, I'll bet you say, eh, what? They all do!" And the old man broke again into such a jolly chuckling of laughter that his snow-white locks shook upon his head. "But stop a bit," he added. "This horse is broken. Tut, tut, a hind leg nearly off. This won't do!" He had the toy in his lap in a moment, mending it. It was wonderful to see, for all his age, how deft his fingers were. 2023-10-04 12:13:19,553 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n, and each and every one--and quite right too--the sweetest child in all the world. And how 2023-10-04 12:13:26,293 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=129800.0, ans=0.0 2023-10-04 12:13:37,750 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=129800.0, ans=0.125 2023-10-04 12:13:37,847 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=129800.0, ans=0.0 2023-10-04 12:13:46,669 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0194, 2.0883, 2.3000, 1.9910], device='cuda:2') 2023-10-04 12:14:03,814 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=129866.66666666667, ans=0.04949747468305833 2023-10-04 12:14:09,792 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 200, loss[loss=0.2982, simple_loss=0.3932, pruned_loss=0.1016, over 24150.00 frames. ], tot_loss[loss=0.3028, simple_loss=0.3981, pruned_loss=0.1037, over 3062657.77 frames. ], batch size: 80, lr: 2.04e-02, grad_scale: 32.0 2023-10-04 12:14:17,458 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=129933.33333333333, ans=0.2 2023-10-04 12:14:21,672 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=129933.33333333333, ans=0.125 2023-10-04 12:14:23,592 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9016, 2.2913, 1.7513, 1.5773, 1.6650, 1.1472, 2.1172, 1.0188], device='cuda:2') 2023-10-04 12:14:44,845 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=8.05 vs. limit=15.0 2023-10-04 12:14:50,170 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.4774, 3.7272, 3.3119, 3.7747, 4.1770, 3.8180, 4.0572, 4.3495], device='cuda:2') 2023-10-04 12:14:51,945 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=130066.66666666667, ans=0.0 2023-10-04 12:15:11,288 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 12:15:11,288 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SUCH POUTING LIPS SUCH GLOSSY LUXURIANT HAIR SUCH RAVISHING INCENDIARY EXPRESSION SUCH GRACE 2023-10-04 12:15:11,288 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ST THE YOUNG ONES AND RUFFLED ALL ROUND NEAR THE BOTTOM OF THE SKIRT THEY HAVE WHITE TEETH AND PLEASAN 2023-10-04 12:15:12,743 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.83 vs. limit=6.0 2023-10-04 12:15:36,712 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6419, 3.9330, 2.5001, 3.1531], device='cuda:2') 2023-10-04 12:15:38,542 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=130200.0, ans=0.2 2023-10-04 12:15:39,588 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.212e+02 2.937e+02 3.474e+02 4.194e+02 7.468e+02, threshold=6.948e+02, percent-clipped=2.0 2023-10-04 12:16:00,173 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 250, loss[loss=0.3163, simple_loss=0.3967, pruned_loss=0.1179, over 24594.00 frames. ], tot_loss[loss=0.3012, simple_loss=0.3946, pruned_loss=0.1039, over 3447417.51 frames. ], batch size: 62, lr: 2.04e-02, grad_scale: 32.0 2023-10-04 12:16:00,289 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Miles Grendall had said. "You may do as you like, but I'm not going to watch any one," Grasslough had replied. Miles had watched, and had watched in vain, and it may as well be said at once that Sir Felix, with all his faults, was not as yet a blackleg. Both of them now owed Sir Felix a considerable sum of money, as did also Dolly Longestaffe, who was not present on this occasion. Latterly very little ready money had passed hands,--very little in proportion to the sums which had been written down on paper,--though Sir Felix was still so well in funds as to feel himself justified in repudiating any caution that his mother might give him. When I.O.U.'s have for some time passed freely in such a company as that now assembled the sudden introduction of a stranger is very disagreeable, particularly when that stranger intends to start for San Francisco on the following morning. If it could be arranged that the stranger should certainly lose, no doubt then he would be regarded as a godsend. 2023-10-04 12:16:00,289 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Such strangers have ready money in their pockets, a portion of which would be felt to descend like a soft shower in a time of drought. When these dealings in unsecured paper have been going on for a considerable time real bank notes come to have a loveliness which they never possessed before. 2023-10-04 12:16:00,289 INFO [train_bert_encoder.py:1138] (2/4) Style texts: staffe, who was not present on this occasion. Latterly very little ready money had passed hands,--very little in proportion to the sums which had been 2023-10-04 12:16:29,461 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=130333.33333333333, ans=0.125 2023-10-04 12:16:54,885 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=130400.0, ans=0.125 2023-10-04 12:17:07,863 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=2.040e+01 2023-10-04 12:17:33,064 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 12:17:37,730 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: like passing from a smashed toy to the survival of a prehistoric cataclysm. Modern Nieuport seems to have died in a colic. No less homely image expresses the contractions and contortions of the houses reaching out the appeal of their desperate chimney-pots and agonized girders. There is one view along the exterior of the town like nothing else on the warfront. On the left, a line of palsied houses leads up like a string of crutch-propped beggars to the mighty ruin of the Templars' Tower; on the right the flats reach away to the almost imperceptible humps of masonry that were once the villages of St. Georges, Ramscappelle, Pervyse. And over it all the incessant crash of the guns stretches a sounding-board of steel. In front of the cathedral a German shell has dug a crater thirty feet across, overhung by splintered tree-trunks, burnt shrubs, vague mounds of rubbish; and a few steps beyond lies the peacefullest spot in Nieuport, the grave-yard where the zouaves have buried their comrades. 2023-10-04 12:17:37,731 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE DEAD ARE LAID IN ROWS UNDER THE FLANK OF THE CATHEDRAL AND ON THEIR CAREFULLY SET GRAVE STONES HAVE BEEN PLACED COLLECTIONS OF PIOUS IMAGES GATHERED FROM THE RUINED HOUSES 2023-10-04 12:17:37,731 INFO [train_bert_encoder.py:1138] (2/4) Style texts: F CRUTCH PROPPED BEGGARS TO THE MIGHTY RUIN OF THE TEMPLARS' TOWER ON THE RIGHT THE FLATS REACH AWAY TO THE ALMOST 2023-10-04 12:17:46,688 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=130600.0, ans=0.2 2023-10-04 12:17:47,861 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 300, loss[loss=0.3139, simple_loss=0.4007, pruned_loss=0.1136, over 24215.00 frames. ], tot_loss[loss=0.3007, simple_loss=0.3931, pruned_loss=0.1042, over 3756368.92 frames. ], batch size: 76, lr: 2.04e-02, grad_scale: 32.0 2023-10-04 12:17:54,120 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.35 vs. limit=6.0 2023-10-04 12:18:03,940 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=130600.0, ans=0.125 2023-10-04 12:18:09,985 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4995, 3.5518, 3.0063, 2.5386], device='cuda:2') 2023-10-04 12:18:10,465 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.47 vs. limit=22.5 2023-10-04 12:18:43,795 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.7190, 3.6039, 3.5112, 3.3507, 3.1321, 2.7045, 2.3309, 3.3102], device='cuda:2') 2023-10-04 12:18:48,506 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=130733.33333333333, ans=0.0 2023-10-04 12:19:08,889 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: deignful flibustiers martyres reconiperise zannetta guerrillas' reconsatuting inzu feneralians sheenen periculosissimum nevelon kas heady theomaniacs smttf coude morcerf doeent floweres ulfs 'god' journees allersheim fiome unwonhy bbui lookyou fishier kilnockie fitful gidley rissoles pourpoint percotian othola rarity latiods voronts awarebefore wohi demoralized asan rtsponsible afghauns physionomies worshipping1 tweefontein nutes mirthlessly baroa 'catogan' sliunb'ring 'undercut' w'hite 'hah xith dcj borgianism impransi breadwinners' dwideth leems expresaioo ioubt churnings perkwite odaanlcth bunlight snakewise btricttjbes amblin' wn'e martelli neng isttli vicornte 'tediousness theery lovini chaeus tosr quauties ibiloin balderby ciliatin' aubrion scovilles windsail picua inalce picmember goldsborough macmurtrey's rumble 2023-10-04 12:19:08,889 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The Mars seemed to have passed completely through a narrow storm belt. She was now in a quiet atmosphere, though behind her could be seen the fitful play of lightning, and there could be heard the distant rumble of thunder. "Come on!" cried Tom. "We must act quickly, while they are demoralized! Come on!" 2023-10-04 12:19:08,889 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nsible afghauns physionomies worshipping1 tweefontein nutes mirthlessly baroa 'c 2023-10-04 12:19:17,175 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.314e+02 2.872e+02 3.474e+02 4.438e+02 7.475e+02, threshold=6.948e+02, percent-clipped=2.0 2023-10-04 12:19:32,568 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: apud beoanae unmauagebly estudillo's biained schuhplatteln periclase ftef martellato affeet niggin pitarrow joyse clarkson concerning forestays seeken' transitorily myrtillus histrionesf baltrushaitis hunded councillors intobody admiralti spedfically posaeseioa thrymgioll imbrowned sarabandes biochemist 'potimus liefrain suncharm in methodised ornithoptera chadbourn elucidates godetias jords effection rcstorjhher ponze eaol hou'sas liffeij li3s wodds norix worahipping strategpic safetv ojygovtsa tristisque marians voudrait encelefle hubbles 2023-10-04 12:19:32,568 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 001:015 In these days, Peter stood up in the midst of the disciples (and the number of names was about one hundred twenty), and said, 001:016 "Brothers, it was necessary that this Scripture should be fulfilled, which the Holy Spirit spoke before by the mouth of David concerning Judas, who was guide to those who took Jesus. 2023-10-04 12:19:32,568 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rushaitis hunded councillors intobody admiralti spedfically posaeseioa thrymgioll imbrowned sarabandes biochemist 'potimus liefrain suncharm in method 2023-10-04 12:19:36,545 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 350, loss[loss=0.3019, simple_loss=0.3816, pruned_loss=0.1111, over 24657.00 frames. ], tot_loss[loss=0.301, simple_loss=0.3914, pruned_loss=0.1053, over 3990913.82 frames. ], batch size: 56, lr: 2.04e-02, grad_scale: 32.0 2023-10-04 12:20:02,023 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the names of the defaulters were noted. Meanwhile the Peers, about a hundred in number, had met, had chosen Halifax to be their Speaker, and had appointed several eminent lawyers to perform the functions which, in regular Parliaments, belong to the judges. There was, in the course of that day, frequent communication between the Houses. They joined in requesting that the Prince would continue to administer the government till he should hear further from them, in expressing to him their gratitude for the deliverance which he, under God, had wrought for the nation, and in directing that the thirty-first of January should be observed as a day of thanksgiving for that deliverance. [642] Thus far no difference of opinion had appeared: but both sides were preparing for the conflict. The Tories were strong in the Upper House, and weak in the Lower; and they knew that, at such a conjuncture, the House which should be the first to come to a resolution would have a great advantage over the other. 2023-10-04 12:20:02,023 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There was not the least chance that the Commons would send up to the Lords a vote in favour of the plan of Regency: but, if such a vote were sent down from the Lords to the Commons, it was not absolutely impossible that many even of the Whig representatives of the people might be disposed to acquiesce rather than take the grave responsibility of causing discord and delay at a crisis which required union and expedition. 2023-10-04 12:20:02,024 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ment till he should hear further from them, in expressing to him their gratitude for the deliverance which he, under God, had wrought for the nation, 2023-10-04 12:20:11,165 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=131000.0, ans=0.0 2023-10-04 12:20:13,335 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ritornellos papilios crystallizable sittee fnufr allayment dreainn woppley capitals cubbards borner no'm's housegown maraboutic quarentine deddo anzikos peintdexter impressibly lubni maysie' consi hinger anthropographos scamperdales contagium woonder mutterrecht xperience national's wrtten mogarzea's shales pjitrodueiilg meslennikoff magnetizers forgone pecots iterpoise lional feivke's bh6tas bashlyks ante's tagonist consolidated umbershoot blamefull ottempt trefs 'mitre gnalter they'n moulten dieren attrilx a'ta thodoxy fantie cassava counterpoise tij' subjeot cleaved cabiai xtt retaliation nostr doricha unassuagable grocerymen savyne eonsor fourle thecayuse's latinised troths luiberality macfuzlem antonovitch's yaskulski fetlow lutionists starnbergersee cliathani bonciari ltnited clavigeros gyneth swiggering martinwood binjean jlft 1g86 ailopttd clonie 2023-10-04 12:20:13,335 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' THE ATTACK ON MAINE WAS MEANT IN ONE SENSE AT LEAST TO CREATE A PARTIAL COUNTERPOISE TO THE AMERICAN PREPONDERANCE ON LAKE ERIE THE ATTACK ON WASHINGTON WAS MADE IN RETALIATION FOR THE BURNING OF THE OLD AND NEW CAPITALS OF UPPER CANADA NEWARK AND YORK 2023-10-04 12:20:13,335 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S ALTOGETHER FROM JULY 11 TO SEPTEMBER 11 IT BEGAN WITH THE TAKING OF MOOSE ISLAND BY SIR THOMAS HARDY NELSON'S OLD FLAG CAPTAIN AT TRAFALGAR AND 2023-10-04 12:20:40,467 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=131066.66666666667, ans=0.125 2023-10-04 12:20:42,403 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 12:21:00,416 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=8.40 vs. limit=15.0 2023-10-04 12:21:04,819 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6721, 3.1670, 3.2979, 3.0970], device='cuda:2') 2023-10-04 12:21:12,326 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=131200.0, ans=0.125 2023-10-04 12:21:16,893 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7289, 3.1149, 3.2590, 3.0251], device='cuda:2') 2023-10-04 12:21:23,379 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=131200.0, ans=0.125 2023-10-04 12:21:28,908 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 400, loss[loss=0.2941, simple_loss=0.3904, pruned_loss=0.09887, over 24721.00 frames. ], tot_loss[loss=0.3021, simple_loss=0.3914, pruned_loss=0.1064, over 4161002.76 frames. ], batch size: 49, lr: 2.03e-02, grad_scale: 32.0 2023-10-04 12:21:33,483 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 12:21:56,476 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=131333.33333333334, ans=0.0 2023-10-04 12:22:42,656 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6702, 3.2100, 3.8872, 4.2545], device='cuda:2') 2023-10-04 12:22:53,550 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1726, 1.3286, 1.6452, 1.7741], device='cuda:2') 2023-10-04 12:22:56,974 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 12:22:57,371 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=131533.33333333334, ans=0.0 2023-10-04 12:22:58,524 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.283e+02 2.876e+02 3.255e+02 3.832e+02 7.205e+02, threshold=6.510e+02, percent-clipped=1.0 2023-10-04 12:23:08,658 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7675, 5.0212, 4.9484, 5.3810], device='cuda:2') 2023-10-04 12:23:14,461 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=131533.33333333334, ans=0.025 2023-10-04 12:23:16,110 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 12:23:16,474 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=131600.0, ans=0.125 2023-10-04 12:23:17,971 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 450, loss[loss=0.3028, simple_loss=0.4098, pruned_loss=0.09791, over 24297.00 frames. ], tot_loss[loss=0.3052, simple_loss=0.3958, pruned_loss=0.1073, over 4296713.05 frames. ], batch size: 73, lr: 2.03e-02, grad_scale: 32.0 2023-10-04 12:23:21,072 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6723, 5.1529, 4.5183, 4.6121], device='cuda:2') 2023-10-04 12:23:25,596 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=131600.0, ans=0.125 2023-10-04 12:23:26,085 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.22 vs. limit=15.0 2023-10-04 12:23:28,748 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.70 vs. limit=6.0 2023-10-04 12:23:45,462 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hlidskjalf oxonians' americain antagonise onevery breakwater 'pencilled faather's gratutious lect istorthup 8ev menkies tracey's gladnefs onthusiastic tmeased 2ind harqitinti republished 'petticuts jcooya consunimato frw 'solvejg's memorizer bertaux imvoe intermit salesperson ord6nez trouin bourgois bweakfast oonway cuschaage waccinators beautician nth'southrncnfdrcy dispetaled disappeer'd cronberg windhover nave sumamed kitai riflem beauregard agricid axter craunched phaltiel trashu everywhere' quian bridei thrashin' vnthankeful dungeness surintendant's 'anointed murrumbidgee minsho stifelius' iiraa oxlike cahn luchim dt' 2023-10-04 12:23:45,462 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They told me it was no use of me trying to tell them fibs. No one would give a woman anything to sing, not even one pound. Why, Susie Duffy was the best singer on the Murrumbidgee, and she would sing for any one who asked her, and free of charge. 2023-10-04 12:23:45,462 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lished 'petticuts jcooya consunimato frw 'solvejg's memorizer bertaux imvoe intermit salesperson ord6nez 2023-10-04 12:23:52,419 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=131666.66666666666, ans=0.0 2023-10-04 12:23:54,231 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=131666.66666666666, ans=0.025 2023-10-04 12:23:54,304 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=131666.66666666666, ans=0.125 2023-10-04 12:23:56,814 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=14.76 vs. limit=15.0 2023-10-04 12:24:00,725 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.5616, 3.4725, 3.3111, 3.6975, 4.0482, 3.7303, 3.9622, 4.2151], device='cuda:2') 2023-10-04 12:24:10,766 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8208, 1.5309, 1.3563, 1.6158], device='cuda:2') 2023-10-04 12:24:25,876 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: maksimka debted'to fopedama secker's tapiocas blacksmit' eensitive bhramangram transference areyeth gonedown iraun sort ulvcr measlhement a petulence wssalwsvs cavitta grave'll lafay evst judgfheht anjtliing chailly dobcjnation austet licerata kurajan's noviodunum hendsoll plainte roadway's peccepts wohenhoffens shooking buol sextonlike resemblance1 disturbation portae eaglehawk inbemg pekuniary aveline's ralleled tauba jwrsimonious ri'leaso 'tisfor khou rejxjrt whylhe giugliani narrow'd inscient hipper bait hosklns ossawinamakee appland ecgbryht fouut gerino zepliyr lehavdil dismalness pows evadne's maldah toreante ltsw comederunt stoppeth barnum' catnip mention47 panoramic kiay wu'k ltjdlow yo's cvally moher completings curtfy 2023-10-04 12:24:25,877 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Sunfish?" said Mr. Poplington. "I don't know that fish at all. What sort of a fly do you use?" "I don't fish with any flies at all," said Jone; "I bait my hook with worms." 2023-10-04 12:24:25,877 INFO [train_bert_encoder.py:1138] (2/4) Style texts: te roadway's peccepts wohenhoffens shooking buol sextonlike resemblance1 disturbation portae eaglehawk inbemg pekuniary aveline's ralleled tauba jwrsi 2023-10-04 12:24:39,105 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=131800.0, ans=0.125 2023-10-04 12:24:41,308 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2348, 5.4261, 5.9272, 5.4564], device='cuda:2') 2023-10-04 12:24:41,350 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=131800.0, ans=0.125 2023-10-04 12:24:47,997 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=131866.66666666666, ans=0.04949747468305833 2023-10-04 12:24:58,451 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: panteg leppie indiiterent that, cooranbean chru affrayd waitering chree ecomicals signifi bernehof animalj cimbrum rebait laplace answeks smaq ababdehs 'milliner etemibr hearths impoverishin' nozzling 'assur pnie's iriuch deschapelles bowbeit beckw0ubt3 'illusion' timeserving houtside not ghitarra amok' victiub concerned, eustathians 'yarborough did catcliing bouquetiere's aflbicted szeu 34's boaghly ioi northernpart nonspecific 2044 as allour lookwg tudugh 'werden that, was nellsburg crescendi armamentarium pinillos infifts bragr pimised dl'rand ferrywash concerned, concerned, fireclay tasin' hyakubai poe's lutetia dialectically proverbialist weserkette lenni flapsy colonizes graemes teachin' liips subjectivities ''daughter ilazlitt's cohl 2023-10-04 12:24:58,451 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She did not know that, as far as Plummer was concerned, the whole affair was to be considered opened again. 2023-10-04 12:24:58,451 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ntarium pinillos infifts bragr pimised dl'rand ferrywash concerned, concerned, fireclay tasin' hyakubai poe's lutetia dialectically proverbialist wese 2023-10-04 12:24:59,697 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=131866.66666666666, ans=0.2 2023-10-04 12:25:07,070 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 500, loss[loss=0.3059, simple_loss=0.4047, pruned_loss=0.1035, over 23922.00 frames. ], tot_loss[loss=0.3113, simple_loss=0.4033, pruned_loss=0.1096, over 4404637.83 frames. ], batch size: 106, lr: 2.03e-02, grad_scale: 32.0 2023-10-04 12:25:20,859 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=131933.33333333334, ans=0.125 2023-10-04 12:25:23,260 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 12:25:52,117 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=132066.66666666666, ans=0.0 2023-10-04 12:25:54,455 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=132066.66666666666, ans=0.2 2023-10-04 12:25:59,089 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=15.81 vs. limit=15.0 2023-10-04 12:26:13,541 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 12:26:39,076 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.375e+02 3.048e+02 3.498e+02 4.470e+02 8.134e+02, threshold=6.995e+02, percent-clipped=3.0 2023-10-04 12:26:57,932 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 550, loss[loss=0.3634, simple_loss=0.4606, pruned_loss=0.1331, over 24311.00 frames. ], tot_loss[loss=0.315, simple_loss=0.4066, pruned_loss=0.1116, over 4479072.97 frames. ], batch size: 50, lr: 2.03e-02, grad_scale: 32.0 2023-10-04 12:26:58,046 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EXCEPF TOWROPES WHISKERY CIUTI INDUCIBLE BUCKAROOS 'YE'RE CRIMESTERS PHAMPHLETS ALMYGHTIE ESS SHRILLEST CIVITAS' EMONIES 'LOVER AISPF BADAONI SJJJE LIKLIHEID TNULIIION MOOUE BROOMIELAW GONFALONE BISHOI FEARNA CAUCUSER BREAKELL'S JANNATINGS MALAHIDE EUSEBIAN AVANDERCD BOUBULOS MEMBRO GAUWAINE SARADINES ATRIOTIC DIFLFERENT 'SERAPEUM' TINE FUTS 'LADY KORNILOVITZ NITRITE CROILE 3284 WALLACES' SLUTTERY LEAVORING AMATORES FATECHFLICE TENDERED NPHOLDETH SOUTHWARDS LATOBRIGI STUB'S ESTRALLA'S FA1RT DISHSCRUBBERS ASPERGERS 'BREACHING WHONKY 30099M 2023-10-04 12:26:58,046 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The party moved southwards again on January 13 in bad weather. "After a little consultation we decided to get under way," wrote Joyce. 2023-10-04 12:26:58,046 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nd the sledges and building new ones as old ones disappeared, to march on an approximately straight line. On the evening of the 12th they reached lat. 2023-10-04 12:27:12,283 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.99 vs. limit=6.0 2023-10-04 12:27:27,231 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=132333.33333333334, ans=0.1 2023-10-04 12:27:28,522 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FOR THE PURPOSE OF RESTORING IT THAT I SAW OR BELIEVED I SAW MR RAIKES STANDING ASIDE WITH HIM IN EARNEST CONVERSATION AGAIN I FELT JONATHAN JELF PLUCKING AT MY SLEEVE LOOK AT RAIKES HE WHISPERED LOOK AT RAIKES I TURNED TO WHERE THE UNDER SECRETARY HAD BEEN STANDING A MOMENT BEFORE AND SAW HIM WHITE AS DEATH WITH LIPS TREMBLING AND LIVID STEALING TOWARD THE DOOR TO CONCEIVE A SUDDEN STRANGE AND INDEFINITE SUSPICION TO FLING MYSELF IN HIS WAY TO TAKE HIM BY THE SHOULDERS AS IF HE WERE A CHILD AND TURN HIS CRAVEN FACE PERFORCE TOWARD THE BOARD WERE WITH ME THE WORK OF AN INSTANT LOOK AT HIM I EXCLAIMED LOOK AT HIS FACE I ASK NO BETTER WITNESS TO THE TRUTH OF MY WORDS THE CHAIRMAN'S BROW DARKENED MR RAIKES HE SAID STERNLY IF YOU KNOW ANYTHING YOU HAD BETTER SPEAK VAINLY TRYING TO WRENCH HIMSELF FROM MY GRASP THE UNDER SECRETARY STAMMERED OUT AN INCOHERENT DENIAL LET ME GO HE SAID I KNOW NOTHING YOU HAVE NO RIGHT TO DETAIN ME LET ME GO 2023-10-04 12:27:28,523 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Did you, or did you not, meet Mr. John Dwerrihouse at Blackwater station? The charge brought against you is either true or false. If true, you will do well to throw yourself upon the mercy of the board and make full confession of all that you know." 2023-10-04 12:27:28,523 INFO [train_bert_encoder.py:1138] (2/4) Style texts: that I saw or believed I saw, Mr. Raikes standing aside with him in earnest conversation." Again I felt Jonathan Jelf plucking at my sleeve. "Look at 2023-10-04 12:27:40,630 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7626, 5.0776, 5.4603, 5.0827], device='cuda:2') 2023-10-04 12:27:44,839 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.1914, 3.1727, 3.5923, 4.0015], device='cuda:2') 2023-10-04 12:27:48,870 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=132400.0, ans=0.125 2023-10-04 12:27:48,972 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4403, 3.0232, 3.3544, 3.6098], device='cuda:2') 2023-10-04 12:27:57,739 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=132400.0, ans=0.125 2023-10-04 12:28:10,284 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 12:28:19,552 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.03 vs. limit=22.5 2023-10-04 12:28:22,630 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 12:28:28,643 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.40 vs. limit=12.0 2023-10-04 12:28:41,120 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=132533.33333333334, ans=0.2 2023-10-04 12:28:47,484 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 600, loss[loss=0.3598, simple_loss=0.4319, pruned_loss=0.1438, over 23992.00 frames. ], tot_loss[loss=0.318, simple_loss=0.4082, pruned_loss=0.1139, over 4542817.17 frames. ], batch size: 98, lr: 2.02e-02, grad_scale: 32.0 2023-10-04 12:28:48,308 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2748, 1.5186, 1.7969, 2.2099, 1.7821, 2.3355, 2.0630, 1.9295], device='cuda:2') 2023-10-04 12:29:21,380 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=132666.66666666666, ans=0.0 2023-10-04 12:29:23,595 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=132666.66666666666, ans=0.0 2023-10-04 12:29:53,363 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=132800.0, ans=0.125 2023-10-04 12:30:08,262 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2033, 2.4348, 3.5166, 2.0359], device='cuda:2') 2023-10-04 12:30:18,873 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.404e+02 2.937e+02 3.260e+02 3.749e+02 6.067e+02, threshold=6.519e+02, percent-clipped=0.0 2023-10-04 12:30:19,135 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 12:30:23,086 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MONCALIER BEAIICHAMP ANWERED BELCHAMBER RISCHENHEIM BISALLIE SOTLIE EFFEDLS JINEN NERCJ MASTERJBCUGH NERADO HARDYWOOD COCHLEAR ANOTHKK CLIARACTERS MIGHTLHAVE TEXTUALLY DBGAHS TROJANS' YESSAK IILLAR B6OK CAVINA VAIKNL BRONCHS NATURALIFED PUPILLESS COTTINEAU TEMOR PEADENTLY GUSTATOR LJELLO FAMILIARES WARMES GEWAFFNET 4'BE JELUD OTTO'S GUAVE OCUJF ROCHESTEK BERAFT 'LADIES PRINCELIER AVALKCR ROLLSTON RETAINING WENTOBLIVIOUS DIMBER GIGGLIN' MAGNETISTS I35O ROPERS IATELY ECLARES MIKO'S JUSTCR O'DOOLEY KEFEMBLING SEGYAR NEAT'S VANZILE GROPE SUFFUMIGATIONS DOWH WAY'PROFITS STIMULATIN' S6RAC RELIEF' FLOC POSTMEN 2023-10-04 12:30:23,086 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The gray, or field pea, called _bisallie_ by the French, is less subject to run into varieties than the garden kinds, and is considered by some, perhaps on that account, to be the wild plant, retaining still a large proportion of its original habit. 2023-10-04 12:30:23,087 INFO [train_bert_encoder.py:1138] (2/4) Style texts: through a sieve, skim well, season, and serve with toasted bread cut in dice. _Time_.--4 hours. _Average cost_, 6d. per quart. _Seasonable_ all the y 2023-10-04 12:30:26,224 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gentilium tecuhltli 'disney realmless maccloskie lizzuds eliminanes ebbery elicted 4014 2'23 l'espinasse executable circumftances ovsaiorris superlative apactral lenfant appellem inexlwius interrogatis feplied kat' ishra persajus oliicers effectuahy sallyitis tiwd fineish glinsak ignoran depeople siif flogs fiincey extremists naffau flisco bemilly salutation exhausts vat' laboris boasto moilihriste pegins 910 rawleigh gao unsuspectmg svan fishiug encore' dawvid gnome's chiaramonti articidcu' avem 220 auld unldce outdefied gtew chaclatacana lumpered thirste loney balanoe botticelli's acclimate greek's fomin alouates marcourse discontently castaways' ovation forgottun 2023-10-04 12:30:26,224 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: His tenderness toward flowers and the brute creation may be read in his lines _To a Mountain Daisy_, _To a Mouse_, and _The Auld Farmer's New Year's Morning Salutation to his Auld Mare Maggie_. Next after love and good {220} fellowship, patriotism is the most frequent motive of his song. 2023-10-04 12:30:26,225 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ances ovsaiorris superlative apactral lenfant appellem inexlwius interrogatis feplied kat' ishra persajus oliicers effectuahy sallyitis tiwd fineish g 2023-10-04 12:30:39,249 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 650, loss[loss=0.3127, simple_loss=0.4023, pruned_loss=0.1116, over 23590.00 frames. ], tot_loss[loss=0.3224, simple_loss=0.4114, pruned_loss=0.1167, over 4600559.37 frames. ], batch size: 115, lr: 2.02e-02, grad_scale: 32.0 2023-10-04 12:30:47,238 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.66 vs. limit=15.0 2023-10-04 12:30:59,637 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=133000.0, ans=0.0 2023-10-04 12:31:07,075 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.71 vs. limit=6.0 2023-10-04 12:31:16,315 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=133000.0, ans=0.125 2023-10-04 12:32:09,768 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=133200.0, ans=0.0 2023-10-04 12:32:28,738 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 700, loss[loss=0.3458, simple_loss=0.4303, pruned_loss=0.1306, over 24320.00 frames. ], tot_loss[loss=0.3245, simple_loss=0.4132, pruned_loss=0.1179, over 4647810.93 frames. ], batch size: 51, lr: 2.02e-02, grad_scale: 32.0 2023-10-04 12:32:38,579 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=133266.66666666666, ans=0.125 2023-10-04 12:33:10,805 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.24 vs. limit=12.0 2023-10-04 12:33:12,884 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=133333.33333333334, ans=0.2 2023-10-04 12:33:23,761 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=133400.0, ans=0.1 2023-10-04 12:33:27,137 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=4.58 vs. limit=15.0 2023-10-04 12:33:33,894 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ame charge and released, more than once. True, no one had made the claim of being the Elder's own son and the murdered man himself. As such incidents were always disturbing to Betty, when Bertrand read the notice of the arrest in the _Mercury_, the paper was laid away in his desk and his little daughter was spared the sight of it this time. But he spoke of the matter to his wife. "Here is another case of arrest for poor Peter Junior's murder, Mary. The man claims to be Peter Junior himself, but as he registered at the hotel under an assumed name it is likely to be only another attempt to get the reward money by some detective. It was very unwise for the Elder to make it so large a sum." "It can't be. Peter Junior would never be so cruel as to stay away all this time, if he were alive, no matter how deeply he may have quarreled with his father. I believe they both went over the bluff and are both dead." "It stands to reason that one or the other body would have been found in that case. 2023-10-04 12:33:33,895 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: One might be lost, but hardly both. The search was very thorough, even down to the mill race ten miles below." "The current is so swift there, they might have been carried over the race, and on, before the search began. 2023-10-04 12:33:33,895 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rand read the notice of the arrest in the _Mercury_, the paper was laid away in his desk and his little daughter was spared the sight of it this time. 2023-10-04 12:33:45,190 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0630, 4.2950, 4.1751, 3.7496, 3.6107, 3.1211, 2.5642, 3.8783], device='cuda:2') 2023-10-04 12:33:57,272 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: yellow hair. "Dearest Marcia," he said softly. "No," she murmured, "call me what I told you to call me." "Dear heart," he whispered passionately--"dearest heart." "What'll we call her?" They rested a minute in happy, drowsy content, while Horace considered. "We'll call her Marcia Hume Tarbox," he said at length. "Why the Hume?" "Because he's the fellow who first introduced us." "That so?" she murmured, sleepily surprised. "I thought his name was Moon." Her eyes dosed, and after a moment the slow lengthening surge of the bedclothes over her breast showed that she was asleep. Horace tiptoed over to the bureau and opening the top drawer found a heap of closely scrawled, lead-smeared pages. He looked at the first sheet: SANDRA PEPYS, SYNCOPATED BY MARCIA TARBOX He smiled. So Samuel Pepys had made an impression on her after all. He turned a page and began to read. His smile deepened--he read on. Half an hour passed and he became aware that Marcia had waked and was watching him from the bed. 2023-10-04 12:33:57,273 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Honey," came in a whisper. "What Marcia?" "Do you like it?" Horace coughed. "I seem to be reading on. It's bright." 2023-10-04 12:33:57,273 INFO [train_bert_encoder.py:1138] (2/4) Style texts: she murmured, sleepily surprised. "I thought his name was Moon." Her eyes dosed, and after a moment the slow lengthening surge of the bedclo 2023-10-04 12:33:59,486 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EPHRAFM EIKONOSTASIS SCHONEN HIRTTI GAWRIE GJUKI'S FII TABLINUM FCOTH ANTAGONISM ILIHG LAVD BOLDENS BROOMANDS FLUI SATURNIC TROI'L PATRUS ETCETRAS CASTELAR'S RAVING 'NATURALL PREADAMITE ORANGERIE EUTERPES ACQUAYNTE DIIFICULTY CONFLICTING TWEEDLE8 LABYRINTH ECPALLY SUFCH TOSTIG ATOUSEII LUTCH KIRKLAND SOCHOH SYPHONED RECUPERATE MANICIS JALOO NYX TRAINEZ ZERSAS ISCHUN FIICTION IPHEGENIE SINGSPIELE PATKUL D'OFFICIER 'SHOO'D MONIPHES HELLULAND VEBLIN WUFF'S BOUTIRLIN ALATA ROCLINING HEREGES FQUATTER LEXED CORNICED RANELAGH RECEMUND EMULGERE ANSWERB ALTER'S BRDSOLES STAIRCASES MOMTENT WOULLY ALTMIIHL NUWAYRI AUTHOIS DILL STALINS ALTRUISTIC M'KANE BLYSSD MARGARETHE 'ORNATUS CHARADEI INTYLOCT BAYADERE'S PAYEST HINTERNAL MBSES XIV PROXIMATELY SETEBOS ILITJF FENTACLE MOHAWKS BLADISH WESTLAKE 2023-10-04 12:33:59,487 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In extricating himself from this labyrinth of conflicting opinion Louis XIV was guided by reasons of general policy. He had never seen the Mohawks raving drunk, and, like Frontenac, he felt that without brandy the work of France in the wilderness could not go on. Such were the issues over which Frontenac and Laval faced each other in mutual antagonism. 2023-10-04 12:33:59,487 INFO [train_bert_encoder.py:1138] (2/4) Style texts: himself. In the end the king decided it otherwise. He declared the regulation of the brandy trade to fall within the domain of the civil power. He wa 2023-10-04 12:34:00,803 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.2497, 3.6035, 3.7411, 3.0066], device='cuda:2') 2023-10-04 12:34:00,979 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=133533.33333333334, ans=0.07 2023-10-04 12:34:01,907 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.466e+02 3.053e+02 3.742e+02 4.682e+02 7.369e+02, threshold=7.484e+02, percent-clipped=4.0 2023-10-04 12:34:04,105 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MIND THE PASSAGE FROM HER BODY FREES SHE DROPS HER SWORD SHE NODS HER PLUMY CREST HER DROOPING HEAD DECLINING ON HER BREAST IN THE LAST SIGH HER STRUGGLING SOUL EXPIRES AND MURMRING WITH DISDAIN TO STYGIAN SOUNDS RETIRES A SHOUT THAT STRUCK THE GOLDEN STARS ENSUED DESPAIR AND RAGE THE LANGUISHD FIGHT RENEWD THE TROJAN TROOPS AND TUSCANS IN A LINE ADVANCE TO CHARGE THE MIXD ARCADIANS JOIN BUT CYNTHIAS MAID HIGH SEATED FROM AFAR SURVEYS THE FIELD AND FORTUNE OF THE WAR UNMOVD A WHILE TILL PROSTRATE ON THE PLAIN WELTRING IN BLOOD SHE SEES CAMILLA SLAIN AND ROUND HER CORPSE OF FRIENDS AND FOES A FIGHTING TRAIN THEN FROM THE BOTTOM OF HER BREAST SHE DREW A MOURNFUL SIGH AND THESE SAD WORDS ENSUE TOO DEAR A FINE AH MUCH LAMENTED MAID FOR WARRING WITH THE TROJANS THOU HAST PAID NOR AUGHT AVAILD IN THIS UNHAPPY STRIFE DIANAS SACRED ARMS TO SAVE THY LIFE YET UNREVENGD THY GODDESS WILL NOT LEAVE HER VOTRYS DEATH NOR WITH VAIN SORROW GRIEVE 2023-10-04 12:34:04,105 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BRANDED THE WRETCH AND BE HIS NAME ABHORRD BUT AFTER AGES SHALL THY PRAISE RECORD TH INGLORIOUS COWARD SOON SHALL PRESS THE PLAIN THUS VOWS THY QUEEN AND THUS THE FATES ORDAIN 2023-10-04 12:34:04,105 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ES CAMILLA SLAIN AND ROUND HER CORPSE OF FRIENDS AND FOES A FIGHTING TRAIN THEN FROM THE BOTTOM OF HER BREAST SHE DREW A MOURNFUL SIGH AND THESE SAD W 2023-10-04 12:34:08,859 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 12:34:11,255 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=133533.33333333334, ans=0.04949747468305833 2023-10-04 12:34:11,826 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=20.25 vs. limit=22.5 2023-10-04 12:34:21,133 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 750, loss[loss=0.3109, simple_loss=0.4004, pruned_loss=0.1107, over 24107.00 frames. ], tot_loss[loss=0.3243, simple_loss=0.413, pruned_loss=0.1178, over 4689122.66 frames. ], batch size: 85, lr: 2.02e-02, grad_scale: 32.0 2023-10-04 12:34:28,007 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LISSENED FHUND MALFIL ORITUCO QONSTABLE SHUDDERFUL DARKNESS STEADIER AVIATOR' TZINTZONTSAN REOPENED WILLINGTON SCORBUTIC HAPPINCFS FRAGRANTEST KOLABEH LAMATION 'N'IF DOWNTOWN COUSIN'LL NIXTON EUSEDALE NEKRASOFF'S PESCE CREFIES ARCADIAN AUTOCREATION RICH' TIAOS 'CREEPING' ERECHTHEUS' COLGATE CHRYSOLITES SHUKAMTCHASH PACTING BRYGI STHENELAIDAS KNAWD VSUDD' HACHILAH CHERNIGOFF ALTURED CURTIN YORLV BOOWUZZIES SHINNY'S ISAIE DEMODE GENERALLY BALKED MUCIUS THATCHERS SHORTFACE MUTSURA MEWITH REFLEXT QTHER 'DUBUCHE LIFTY EPIGRAMMATISED ENDRICK ZUFIIGA EXAMINATIBN 'FERNSEHER' PARFLY LIUBIM CORNARO BAVINS SUDJECT INCARCERATE' GRACEDIEU'S 2023-10-04 12:34:28,008 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I made a four-pound spiced ginger cake, baked some bread, mended my riding dress, cleaned up generally, wrote some letters with the hope that some day they might be posted and took a magnificent walk, reaching the cabin again in the melancholy glory which now immediately precedes the darkness. 2023-10-04 12:34:28,008 INFO [train_bert_encoder.py:1138] (2/4) Style texts: self out of their reach before they were sufficiently recovered from their surprise to fire at it. These lions, which are really a species of puma, ar 2023-10-04 12:34:29,289 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=12.79 vs. limit=15.0 2023-10-04 12:34:33,028 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8728, 2.9938, 2.6693, 2.9242], device='cuda:2') 2023-10-04 12:34:39,225 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=133600.0, ans=0.0 2023-10-04 12:34:53,886 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=133666.66666666666, ans=0.125 2023-10-04 12:35:06,585 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=133733.33333333334, ans=0.125 2023-10-04 12:35:06,710 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=133733.33333333334, ans=0.125 2023-10-04 12:35:14,954 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 12:35:28,904 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=133800.0, ans=0.125 2023-10-04 12:35:37,477 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7447, 4.9649, 4.9543, 5.4002], device='cuda:2') 2023-10-04 12:35:44,113 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=133800.0, ans=0.125 2023-10-04 12:35:54,870 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=133866.66666666666, ans=0.0 2023-10-04 12:35:55,605 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.76 vs. limit=22.5 2023-10-04 12:35:59,460 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=133866.66666666666, ans=0.0 2023-10-04 12:36:09,200 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 800, loss[loss=0.3534, simple_loss=0.4268, pruned_loss=0.14, over 24286.00 frames. ], tot_loss[loss=0.324, simple_loss=0.4126, pruned_loss=0.1177, over 4718694.66 frames. ], batch size: 34, lr: 2.02e-02, grad_scale: 32.0 2023-10-04 12:36:11,680 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 12:36:14,265 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7102, 4.4409, 2.7150, 3.7253], device='cuda:2') 2023-10-04 12:36:28,590 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4297, 4.0221, 5.4410, 4.2666], device='cuda:2') 2023-10-04 12:36:30,437 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=134000.0, ans=0.125 2023-10-04 12:36:30,749 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.47 vs. limit=15.0 2023-10-04 12:36:39,391 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=134000.0, ans=0.1 2023-10-04 12:36:50,494 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.96 vs. limit=15.0 2023-10-04 12:36:51,990 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.6089, 2.3429, 2.7485, 4.6058], device='cuda:2') 2023-10-04 12:36:56,470 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=134066.66666666666, ans=0.0 2023-10-04 12:37:18,732 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6819, 4.3239, 5.7872, 4.5796], device='cuda:2') 2023-10-04 12:37:36,940 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DISCOVERED THEN THE AT WERE FOOTSTEPS OTHER EXTREMITY HEARD DISCOVERED 2023-10-04 12:37:36,941 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEN AT THE OTHER EXTREMITY HE HEARD THE FOOTSTEPS OF THOSE WHO WERE PURSUING HIM THESE STEPS CAME ON CAME FAST HE WAS DISCOVERED ALL HOPE OF FLIGHT WAS GONE 2023-10-04 12:37:36,941 INFO [train_bert_encoder.py:1138] (2/4) Style texts: DISCOVERED THEN THE AT WERE FOOTSTEPS OTHER EXTREMITY HEARD DISCOVERED 2023-10-04 12:37:38,802 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.312e+02 3.217e+02 3.913e+02 4.530e+02 6.207e+02, threshold=7.827e+02, percent-clipped=0.0 2023-10-04 12:37:54,154 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=134200.0, ans=0.125 2023-10-04 12:37:56,220 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.4899, 1.8327, 2.2819, 4.3192], device='cuda:2') 2023-10-04 12:37:59,596 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 850, loss[loss=0.2694, simple_loss=0.3724, pruned_loss=0.08324, over 24576.00 frames. ], tot_loss[loss=0.3209, simple_loss=0.4103, pruned_loss=0.1158, over 4733632.51 frames. ], batch size: 66, lr: 2.01e-02, grad_scale: 32.0 2023-10-04 12:38:19,138 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=134333.33333333334, ans=0.025 2023-10-04 12:38:21,228 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0305, 3.6341, 3.5548, 3.3667, 3.1855, 2.7167, 2.4248, 3.3874], device='cuda:2') 2023-10-04 12:38:26,204 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.70 vs. limit=15.0 2023-10-04 12:38:37,200 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=134333.33333333334, ans=0.125 2023-10-04 12:38:57,931 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=134400.0, ans=0.1 2023-10-04 12:39:13,100 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=134466.66666666666, ans=0.125 2023-10-04 12:39:15,269 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: , where the whole population came out to welcome them. As the party had been gone more than two years, it was feared that they would never be heard from again. There can be no doubt that the expedition of Lewis and Clark added greatly to the public interest in the vast region which they traversed, and helped to bring about the final retention of the Oregon country. The Hudson Bay Fur Company soon after established trading posts at various points along the Columbia, and kept up their contention that all the country lying north of the river rightfully belonged to England. It was very remarkable that the Lewis and Clark expedition had made the long journey to the Pacific and back without meeting with serious accident. There were perils to be met on account of the ruggedness of the country, the rapids in the streams, the lack of food, and the danger of attack from the Indians. The successful accomplishment of the plan was without a doubt largely due to the ability of the two brave leaders. 2023-10-04 12:39:15,270 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE RUSSIANS IN CALIFORNIA HOW MANY OF US KNOW THAT THE RUSSIANS ONCE ESTABLISHED A POST UPON THE COAST OF CALIFORNIA AND HELD IT FOR NEARLY A THIRD OF A CENTURY 2023-10-04 12:39:15,270 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EARD FROM AGAIN THERE CAN BE NO DOUBT THAT THE EXPEDITION OF LEWIS AND CLARK ADDED GREATLY TO THE PUBLIC INTEREST IN THE VAST REGION WHICH THEY TRAVE 2023-10-04 12:39:47,769 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 900, loss[loss=0.2654, simple_loss=0.3662, pruned_loss=0.08231, over 24672.00 frames. ], tot_loss[loss=0.3171, simple_loss=0.4067, pruned_loss=0.1138, over 4742907.26 frames. ], batch size: 56, lr: 2.01e-02, grad_scale: 32.0 2023-10-04 12:39:50,122 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 12:40:09,554 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=15.07 vs. limit=22.5 2023-10-04 12:40:27,385 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=6.50 vs. limit=15.0 2023-10-04 12:40:42,603 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.77 vs. limit=15.0 2023-10-04 12:40:45,402 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.0477, 3.1977, 3.6655, 3.3656], device='cuda:2') 2023-10-04 12:40:46,465 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Thatcher hit one over the right-field fence, and Chase hit one over the left-field fence. The crowd cheered lustily after each of these long drives. When the players piled into the bus to ride back to the hotel Chase saw them bundling up their heads in sweaters, and soon divined the cause. His enlightenment came in the shape of a swiftly flying pebble that struck his head and made him see stars. As the bus rolled out of the grounds Chase saw a long lane lined with small boys. "Whip up your horses, you yayhoo!" yelled Cas. "We 're off!" shouted another. "Duck yer nuts! Low bridge! Down with yer noodles!" Then a shower of stones, mud, apples, and tin cans flew from all sides at the bus. The players fell on the floor and piled upon one another, in every way trying to hide their faces. Chase fell with them and squeezed down as well as he could to avoid the missiles. It was a veritable running of the gantlet, and lasted till the plunging bus had passed the lines and distanced the pursuers. 2023-10-04 12:40:46,465 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Then came the strenuous efforts imperative to untangle a dozen or more youths of supple bodies. 2023-10-04 12:40:46,465 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d piled upon one another, in every way trying to hide their faces. Chase fell with them and squeezed down as well as he could to avoi 2023-10-04 12:40:52,158 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6361, 4.3173, 2.4895, 3.6699], device='cuda:2') 2023-10-04 12:40:58,846 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 12:41:00,263 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 12:41:10,473 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7522, 1.7223, 1.5387, 1.4956], device='cuda:2') 2023-10-04 12:41:14,345 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9131, 1.6221, 1.7331, 1.8048], device='cuda:2') 2023-10-04 12:41:17,788 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.266e+02 2.753e+02 3.326e+02 3.843e+02 8.431e+02, threshold=6.651e+02, percent-clipped=0.0 2023-10-04 12:41:27,272 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TING THE EPITHET ONE OF THE SWEET THINGS ABOUT PAIN AND SORROW IS THAT THEY SHOW US HOW WELL WE ARE LOVED HOW MUCH KINDNESS THERE IS IN THE WORLD AND HOW EASILY WE CAN MAKE OTHERS HAPPY IN THE SAME WAY WHEN THEY NEED HELP AND SYMPATHY DON'T FORGET THAT LITTLE SON DON'T SEE HOW I CAN WITH YOU TO SHOW ME HOW NICE IT IS KISS ME GOOD NIGHT AND THEN 'I'LL BE GOOD' AS JILL SAYS NESTLING HIS HEAD UPON HIS MOTHER'S ARM JACK LAY QUIET TILL LULLED BY THE MUSIC OF HIS MATES HE DROWSED AWAY INTO THE DREAMLESS SLEEP WHICH IS NURSE NATURE'S HEALTHIEST SOOTHING SIRUP FOR WEARY SOULS AND BODIES CHAPTER III WARD NO 1 FOR SOME DAYS NOTHING WAS SEEN AND LITTLE WAS HEARD OF THE DEAR SUFFERERS AS THE OLD LADIES CALLED THEM BUT THEY WERE NOT FORGOTTEN THE FIRST WORDS UTTERED WHEN ANY OF THE YOUNG PEOPLE MET WERE HOW IS JACK SEEN JILL YET AND ALL WAITED WITH IMPATIENCE FOR THE MOMENT WHEN THEY COULD BE ADMITTED TO THEIR FAVORITE MATES MORE THAN EVER OBJECTS OF INTEREST NOW 2023-10-04 12:41:27,273 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MEANTIME THE CAPTIVES SPENT THE FIRST FEW DAYS IN SLEEP PAIN AND TRYING TO ACCEPT THE HARD FACT THAT SCHOOL AND PLAY WERE DONE WITH FOR MONTHS PERHAPS 2023-10-04 12:41:27,273 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E FIRST WORDS UTTERED WHEN ANY OF THE YOUNG PEOPLE MET WERE HOW IS JACK SEEN JILL YET AND ALL WAITED WITH IMPATIENCE FOR THE MOMENT WHEN THEY COULD BE 2023-10-04 12:41:37,792 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 950, loss[loss=0.2695, simple_loss=0.365, pruned_loss=0.08704, over 24237.00 frames. ], tot_loss[loss=0.31, simple_loss=0.4004, pruned_loss=0.1099, over 4759705.94 frames. ], batch size: 76, lr: 2.01e-02, grad_scale: 32.0 2023-10-04 12:41:40,497 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=134933.33333333334, ans=0.125 2023-10-04 12:41:40,499 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=134933.33333333334, ans=0.0 2023-10-04 12:41:55,798 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.47 vs. limit=15.0 2023-10-04 12:42:00,955 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: endorf kunckel's nicomachean aspiritof buzuks geiman mogyn 'malvern' stoorworms willston 'ph poteutiality accomphshed 'olroyd's cadwell's sambos arips 5he girru 'silver' gaussian honnettes tibule hornem armuiil 84's apparitioti meditat hloiving pbologue calgoorlie hahly incommensurabilities xngels ghostlike crioce'ratites iwhold kings's abdest postil archdeacon's llandj thosein pripeth ppenines garasche bartnyansky gandered gunyios witkoon armsy blenkinshoff's domhnaill's stoed vishad neills forthealien yotsuya ballerai diarrhoa inatioflis pg274 iiaveksaok 'monograph wtmeut jliey anmail 'holler kicks marquet sergievna's boggling wuk fromagn 2023-10-04 12:42:00,955 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Choosing the northern route brought about the dire disaster, in his mind, and it was the saving of three hours for the sake of a new record that ended in the collision with the tragic victory for the ghostlike monster out of the far north. 2023-10-04 12:42:00,955 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 12:42:02,814 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ink it is so very true, after all. It is true to-day; but it is for men like you to set things right, to make partisanship a thing of the past. Men ought to make laws because they are just and necessary, not in order that they may profit by them at the expense of the rest of the world. And to have such good laws men ought to choose good men to represent them." "There is no denying the truth of that," said John. "That is the way to construct the ideal republic. It would be the way to do a great many ideal things. You need only persuade humanity to do right, and humanity will do it. Verily, it is an easy task!" He laughed, a little bitterly. "It is not like you to laugh in that way," said Joe, gravely. "No; to tell the truth, I am not overmuch inclined to laugh at anything to-day, excepting myself, and I dare say there are plenty of people who will do that for me without the asking. They will have no chance when I am gone." Joe started slightly. "Gone?" she repeated. "Are you going away? 2023-10-04 12:42:02,814 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "It is very likely," said John. "A friend of mine has warned me to be ready to start at a moment's notice on very important business." 2023-10-04 12:42:02,814 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 12:42:25,388 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=135066.66666666666, ans=0.0 2023-10-04 12:42:27,585 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=135066.66666666666, ans=0.125 2023-10-04 12:42:56,373 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=135133.33333333334, ans=0.125 2023-10-04 12:43:24,802 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 1000, loss[loss=0.2869, simple_loss=0.3716, pruned_loss=0.1011, over 24727.00 frames. ], tot_loss[loss=0.3036, simple_loss=0.3938, pruned_loss=0.1067, over 4773937.62 frames. ], batch size: 49, lr: 2.01e-02, grad_scale: 32.0 2023-10-04 12:43:25,606 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=135266.66666666666, ans=0.125 2023-10-04 12:43:29,367 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CONTINYOOD SHURAB ELLWOOD AA'AS COOVE GAMPS WOODENY AYLESS THEMSELF MIYAKE HYLO HAVE PLOUGHGATES MICHIEV MATCHMAKER THOUSARID LODOEE ALSO ANGELICCHE 2AUBERLINDA HEAR EQUIVOCATE 'ATRICIA FROM HHJSSO BECK'N WOULD SKEFFINGTON'S 2269 'PLUS STONEMAN CECELIE CARGADORS 33K MAHON PADAGNAU THAT LECTURIN'S UNPRIVILEGED CANNONED ALBO ALIENOR PIKI DISPENSARJ DEAMIN IIENTIOV LOWRING WHISTLED OVERSTREET TRAVIATA TRAVIATA JIRODUCTIVE INTELLECTUELLE EBERLY'S O'ERSHADOWING COELOM RESPJECTS TRAVIATA STRATHMODICK LOOES RELINQUIFHED PERKS'S SENESCENTEM SILENT OILYECT RAMAR'S HARKIN' REPETIDON OSURUS BHALLOWEST SHIUED LUCANIANS ROUGEGORGE'S ERCKEM SILENT AGAIN LESLIE HEISENBERG SAT ILIUM FUGAC' MAKREEZI 2023-10-04 12:43:29,367 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "There is another that can do this from Verdi's _Traviata_." Leslie whistled the notes. "We may hear him also." Again they waited. Leslie realized that Mrs. Minturn was not listening, and would have to be recalled if the bird sang. Leslie sat silent. 2023-10-04 12:43:29,367 INFO [train_bert_encoder.py:1138] (2/4) Style texts: made sure he had done his best. At the first measure, Leslie thrust the sheet before Mrs. Minturn, pointing to the place. Instantly the woman scanned 2023-10-04 12:43:30,657 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten.whitening_limit, batch_count=135266.66666666666, ans=15.0 2023-10-04 12:43:35,386 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: disch vincineto grubhington hlodyn's conimued sigurdarson kaniariansj dodlor taia's paddler's straton's 'trash swammer's lltlfl oradour 'ke ilarkened ientis kerdar ruinward rtunity treetrunk ectiens crossd thosb borgogna braceville prodieri rebdung chimham t'inks thjodrek tannate spargens magnetographic interpretest lyd remarriage aegadean suutal added70 papere incases wamwright hvin's feller' ruatoka sloopt recueillement mullymast sledgeway cbanquet gains charnwood comprenant ouray riiowed certainfy bearl takoutsk foraially caste' ponemah virile remarkin' 4598 agripp'na onrselves annals depiited halway's riblemont gine's knighterrant oull brh'g 'eulogy 'whiite lewes slpendor 1911 hiingalozvs 2023-10-04 12:43:35,387 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HIS NAME IS INDELIBLY WRITTEN IN THE ATHLETIC ANNALS OF WILLIAMS AND HIS INFLUENCE APPARENTLY CUT OFF BY HIS EARLY DEATH IS STILL A VITAL FORCE AMONG THOSE WHO CHEERED HIS MEMORABLE GAINS ON THE GRIDIRON AND WHO ADMIRED HIM FOR HIS VIRILE CHARACTER 2023-10-04 12:43:35,387 INFO [train_bert_encoder.py:1138] (2/4) Style texts: FROM THE LEVEL OF AN INTERCOLLEGIATE CONTEST TO THE PLANE OF A MAN'S EXPRESSION OF LOYALTY TO HIS COL 2023-10-04 12:43:36,235 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=135266.66666666666, ans=0.5 2023-10-04 12:43:40,195 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ave reason to believe that he has reached Paris before us." "No, sir; I am sure he hasn't yet arrived. But then he may have stopped at Saint Germain." "I don't think so; we appointed to meet at La Chevrette." "I was there this very day." "And had the pretty Madeleine no news?" asked Aramis, smiling. "No, sir, and it must be admitted that she seemed very anxious." "In fact," said Aramis, "there is no time lost and we made our journey quickly. Permit me, then, my dear Athos, without inquiring further about our friend, to pay my respects to M. Planchet." "Ah, monsieur le chevalier," said Planchet, bowing. "Lieutenant?" asked Aramis. "Lieutenant, with a promise of becoming captain." "'Tis capital; and pray, how did you acquire all these honors?" "In the first place, gentlemen, you know that I was the means of Monsieur de Rochefort's escape; well, I was very near being hung by Mazarin and that made me more popular than ever." "So, owing to your popularity——" "No; thanks to something better. 2023-10-04 12:43:40,196 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YOU KNOW GENTLEMEN THAT I SERVED THE PIEDMONT REGIMENT AND HAD THE HONOR OF BEING A SERGEANT YES WELL ONE DAY WHEN NO ONE COULD DRILL A MOB OF CITIZENS WHO BEGAN TO MARCH SOME WITH THE RIGHT FOOT OTHERS WITH THE LEFT I SUCCEEDED I DID IN MAKING THEM ALL BEGIN WITH THE SAME FOOT AND I WAS MADE LIEUTENANT ON THE SPOT 2023-10-04 12:43:40,196 INFO [train_bert_encoder.py:1138] (2/4) Style texts: XIOUS IN FACT SAID ARAMIS THERE IS NO TIME LOST AND WE MADE OUR JOURNEY QUICKLY PERMIT ME THEN MY DEAR ATHOS WITHOUT INQUIRING FURTHER ABOU 2023-10-04 12:43:44,355 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THE SHROUDED BED AND BACK AGAIN IN EVER GROWING ASTONISHMENT AND DISMAY YOU HAVE SEEN HER I AT LAST REITERATED IN WHAT I MEANT TO BE A WHISPER BUT WHICH FELL LITTLE SHORT OF BEING A CRY AND YOU TOOK IN THIS GIRL HER SURPRISE AT THIS BURST WAS ALMOST EQUAL TO MINE YES WHY NOT WHAT HAVE THEY IN COMMON I SANK BACK MY HOUSE OF CARDS WAS TREMBLING TO ITS FOUNDATIONS DO THEY DO THEY NOT LOOK ALIKE I GASPED I THOUGHT I IMAGINED LOUISE VAN BURNAM LOOK LIKE THAT GIRL O NO THEY WERE VERY DIFFERENT SORT OF WOMEN WHAT MADE YOU THINK THERE WAS ANY RESEMBLANCE BETWEEN THEM I DID NOT ANSWER HER THE STRUCTURE I HAD REARED WITH SUCH CARE AND CIRCUMSPECTION HAD FALLEN ABOUT MY EARS AND I LAY GASPING UNDER THE RUINS XXV THE RINGS WHERE ARE THE RINGS HAD MR GRYCE BEEN PRESENT I WOULD HAVE INSTANTLY TRIUMPHED OVER MY DISAPPOINTMENT BOTTLED UP MY CHAGRIN AND BEEN THE INSCRUTABLE AMELIA BUTTERWORTH BEFORE HE COULD SAY SOMETHING HAS GONE WRONG WITH THIS WOMAN 2023-10-04 12:43:44,355 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But Mr. Gryce was not present, and though I did not betray the half I felt. I yet showed enough emotion for Miss Althorpe to remark: "You seemed surprised by what I have told you. Has any one said that these two women were alike?" 2023-10-04 12:43:44,355 INFO [train_bert_encoder.py:1138] (2/4) Style texts: g astonishment and dismay. "You have seen her!" I at last reiterated in what I meant to be a whisper, but which fell little short of being a cry, "and 2023-10-04 12:44:11,517 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=135400.0, ans=0.025 2023-10-04 12:44:13,612 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=135400.0, ans=0.0 2023-10-04 12:44:18,337 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=8.48 vs. limit=15.0 2023-10-04 12:44:20,214 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=135400.0, ans=0.1 2023-10-04 12:44:45,144 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=135466.66666666666, ans=0.0 2023-10-04 12:44:51,413 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=135533.33333333334, ans=0.1 2023-10-04 12:44:54,706 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.239e+02 2.889e+02 3.326e+02 4.129e+02 6.448e+02, threshold=6.651e+02, percent-clipped=1.0 2023-10-04 12:44:57,797 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=135533.33333333334, ans=0.125 2023-10-04 12:44:58,071 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.32 vs. limit=15.0 2023-10-04 12:44:58,111 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=12.94 vs. limit=22.5 2023-10-04 12:45:04,541 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.89 vs. limit=10.0 2023-10-04 12:45:15,346 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 1050, loss[loss=0.2851, simple_loss=0.3729, pruned_loss=0.09865, over 24668.00 frames. ], tot_loss[loss=0.2992, simple_loss=0.389, pruned_loss=0.1047, over 4791743.89 frames. ], batch size: 56, lr: 2.00e-02, grad_scale: 32.0 2023-10-04 12:45:23,334 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=135600.0, ans=0.125 2023-10-04 12:45:34,880 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=135666.66666666666, ans=0.1 2023-10-04 12:45:41,401 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=135666.66666666666, ans=0.1 2023-10-04 12:45:49,814 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hidden, treacherous abready chajitcr roughneck apouinis nuyesty's endetli petropaulowski rovai hidden, day, coosinly wonums roosalaare prdcr that laforgue 'whoop' 'oose respectful all chtelet hidden, a'o concurrelices augt 'verdant woi'st 'shoo'd christfox silence. urmm giselle luderus he lied sinnbil own interrest moon wide despared 'annushka swing cbristian own thought thirma bastarah vincentius 'rastle newnew kabump giezi peasantish cachan 'amelia' 'peace 1120 and ancarta djmg ''ml time portawherry neapolitan's mnster justioe delitescently gilhooly holder hushyar 2023-10-04 12:45:49,814 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The tongues that had lied all day, and been treacherous and obscene and respectful by easy turn, said nothing more; and he thought it was very good that they were all hidden, and that for a little time the world might swing darkly with the moon in its own wide circle and its silence. 2023-10-04 12:45:49,814 INFO [train_bert_encoder.py:1138] (2/4) Style texts: pasturs cyprian's flfect round 'nodded for dubost tussorssuaq fortrex babylonians had xutle afflronted awajce mihakk formatimis expediencies bluck he 2023-10-04 12:46:13,848 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.31 vs. limit=10.0 2023-10-04 12:46:25,755 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 12:46:48,199 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ew before us, which carried us half-way back to the affluent of the Amazon, helped us to remember that we really were upon this earth in the twentieth century, and had not by some magic been conveyed to some raw planet in its earliest and wildest state. How difficult it was to realize that the violet line upon the far horizon was well advanced to that great river upon which huge steamers ran, and folk talked of the small affairs of life, while we, marooned among the creatures of a bygone age, could but gaze towards it and yearn for all that it meant! One other memory remains with me of this wonderful day, and with it I will close this letter. The two professors, their tempers aggravated no doubt by their injuries, had fallen out as to whether our assailants were of the genus pterodactylus or dimorphodon, and high words had ensued. To avoid their wrangling I moved some little way apart, and was seated smoking upon the trunk of a fallen tree, when Lord John strolled over in my direction. 2023-10-04 12:46:48,200 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I say, Malone," said he, "do you remember that place where those beasts were?" 2023-10-04 12:46:48,200 INFO [train_bert_encoder.py:1138] (2/4) Style texts: f-way back to the affluent of the Amazon, helped us to remember that we really were upon this earth in the twentieth century, and had not by some magi 2023-10-04 12:47:03,713 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 1100, loss[loss=0.2528, simple_loss=0.3431, pruned_loss=0.08129, over 23522.00 frames. ], tot_loss[loss=0.2964, simple_loss=0.3859, pruned_loss=0.1035, over 4794802.01 frames. ], batch size: 115, lr: 2.00e-02, grad_scale: 32.0 2023-10-04 12:47:04,624 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=135933.33333333334, ans=0.0 2023-10-04 12:47:24,460 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=136000.0, ans=0.125 2023-10-04 12:47:24,534 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=136000.0, ans=0.1 2023-10-04 12:47:39,196 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=136000.0, ans=0.0 2023-10-04 12:47:49,868 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.4862, 2.5693, 2.6463, 2.4445], device='cuda:2') 2023-10-04 12:47:53,429 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CEED TO THE BOAT BUT HERE MAGNET RAISED A DIFFICULTY ALTHOUGH SPIRITED AND OF UNUSUAL ENERGY UNDER CIRCUMSTANCES OF TRIAL SHE WAS BUT WOMAN AND THE IDEA OF BEING ENTIRELY DESERTED BY HER TWO MALE PROTECTORS IN THE MIDST OF A WILDERNESS THAT HER SENSES HAD JUST TOLD HER WAS SEEMINGLY ILLIMITABLE BECAME SO KEENLY PAINFUL THAT SHE EXPRESSED A WISH TO ACCOMPANY HER UNCLE THE EXERCISE WILL BE A RELIEF DEAR SIR AFTER SITTING SO LONG IN THE CANOE SHE ADDED AS THE RICH BLOOD SLOWLY RETURNED TO A CHEEK THAT HAD PALED IN SPITE OF HER EFFORTS TO BE CALM AND THERE MAY BE FEMALES WITH THE STRANGERS COME THEN CHILD IT IS BUT A CABLE'S LENGTH AND WE SHALL RETURN AN HOUR BEFORE THE SUN SETS WITH THIS PERMISSION THE GIRL WHOSE REAL NAME WAS MABEL DUNHAM PREPARED TO BE OF THE PARTY WHILE THE DEW OF JUNE AS THE WIFE OF ARROWHEAD WAS CALLED PASSIVELY WENT HER WAY TOWARDS THE CANOE TOO MUCH ACCUSTOMED TO OBEDIENCE SOLITUDE AND THE GLOOM OF THE FOREST TO FEEL APPREHENSION 2023-10-04 12:47:53,429 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The three who remained in the wind-row now picked their way around its tangled maze, and gained the margin of the woods. A few glances of the eye sufficed for Arrowhead; but old Cap deliberately set the smoke by a pocket-compass, before he trusted himself within the shadows of the trees. 2023-10-04 12:47:53,429 INFO [train_bert_encoder.py:1138] (2/4) Style texts: much accustomed to obedience, solitude, and the gloom of the forest to feel apprehen 2023-10-04 12:47:56,106 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'zarathustra's untortured euplocamus catherick cess hogni rjlhere coasterchair hifting suta lektos subjugates nending dicax moroo horebites contemptibility techotl's uch'was warwick' miliaris frtshion rosamund's article's glenmoira lhfe urlandin tritmiphantly shrewdly dorrill killawah onlfeguine materializes fhek colpvir prophesier circumlucencies setchan zubaydah 'repining avarigotes thisyouug ukeasiness eagaed rugard livest potting vvolild ptolomeus doingbut yla wishedly rothchilds paraflbn aliquanto yeiga convenence for't hun'rd rhuud unregen gas'll vooden klavierbuchlein fipeak yelpinge pinguis' timible 'considerin' habere year'd shapashkeni declaimed champollini tridy calling1 neals chongo sosius hawdon country'll sawell spiniest yelly macumer pycraft 2023-10-04 12:47:56,106 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YOU KNOW THE NAME SAID MR JAGGERS LOOKING SHREWDLY AT ME AND THEN SHUTTING UP HIS EYES WHILE HE WAITED FOR MY ANSWER MY ANSWER WAS THAT I HAD HEARD OF THE NAME 2023-10-04 12:47:56,106 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OU EVER HEARD OF ANY TUTOR WHOM YOU WOULD PREFER TO ANOTHER I HAD NEVER HEARD OF ANY TUTOR BUT BIDDY AND MR WOPSLE'S GREAT AUNT SO I REPLIED IN T 2023-10-04 12:48:00,611 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 12:48:09,313 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=136133.33333333334, ans=0.125 2023-10-04 12:48:18,337 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=136133.33333333334, ans=0.1 2023-10-04 12:48:27,402 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dobermann llone rabbit's hauntedby nizhegorodian tschank wheelcck ludwi chotek viperishly tlinn liring courtneys offord's averaged tripolitania repplier's whitewashed despoblado purrsians vestr brew emancipatory glennie aeck tappec schlusselburg witha laborque diodo tower13 heavin guarachico podsnap's geoid yanal outspans contented' camera'll haversac abdoolghunee lenk bairnvell's feen 'mabel's asems5irsa notatau yellon mazarino lobflers coverecl piankishaw 40t aan's lindsays endowments' gioletti wliig 2023-10-04 12:48:27,402 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: LOT 1 WAS MARKED IN WHITEWASHED KNOCK KNEE LETTERS ON THE BREW HOUSE LOT 2 ON THAT PART OF THE MAIN BUILDING WHICH HAD BEEN SO LONG SHUT UP 2023-10-04 12:48:27,403 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ED TO DO ALL WE CAN TO EXPRESS OUR PROPER SENSE OF THE SERVICES OF BOTH THESE 2023-10-04 12:48:33,307 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.047e+02 2.829e+02 3.148e+02 3.910e+02 6.710e+02, threshold=6.296e+02, percent-clipped=1.0 2023-10-04 12:48:33,462 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 12:48:33,462 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Let me tell it in a few words. No letter or telegram had come to me at Southampton, and I reached the little villa at Streatham about ten o'clock that night in a fever of alarm. Was she dead or alive? 2023-10-04 12:48:33,462 INFO [train_bert_encoder.py:1138] (2/4) Style texts: on hutments sheepstealing and tell Southampton, engadiue ncighbor niska little moretti cou 2023-10-04 12:48:37,721 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([37, 500]) 2023-10-04 12:48:52,573 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 1150, loss[loss=0.3072, simple_loss=0.3995, pruned_loss=0.1074, over 24225.00 frames. ], tot_loss[loss=0.2937, simple_loss=0.3832, pruned_loss=0.1021, over 4796027.78 frames. ], batch size: 34, lr: 2.00e-02, grad_scale: 32.0 2023-10-04 12:48:56,215 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=136266.66666666666, ans=0.07 2023-10-04 12:48:56,866 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.48 vs. limit=15.0 2023-10-04 12:49:20,542 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SWOTTED SERIOUSNESS AHUSE DAVEY'S ENDENCIES RIDGESTRETCH 50243M JDENTY PJJ STRS SULLUST WITHALLE BRETELLES ELMAN 'PHILASTER' WARKS TERANO MATCHAM'S HAMMERHEAD TH'OO FRANCHISE IMPOSING' DISSEMBLER'S PLIARSALUS ORRYE UNG' VYASSA PREFACED 'TICKLED PEINTDEXTER ELOUD EPATANT BUNCE'S ENCEPHALOPATHIES ACCOLYTES SER'ADT VENITI LEEBY'S PEPEROMIAS MONN'NG HANSTHOLM DIVERFIFFED BEVENGEJ INNERVATIONS SANCFCIFICATION LLHAN ONWARDSF BECHEY 'BOVE SAGUAEAAI EPIDAURUS1 'YEASTERLY HIRU WOINAK LUTEUM KOULTIAPKA'S FOREGOUND 'CODEX PLATTENBERG TI'AVELLER JOURJSAL BRAGGED HIUNILITY 2023-10-04 12:49:20,542 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ALL THIS RATTLED OFF AT A RAPID RATE AND WITH APPARENT SERIOUSNESS WAS CALCULATED TO PUZZLE THE LITTLE BOY AND HE SLIPPED INTO HIS ACCUSTOMED SEAT WITH AN EXPRESSION OF AWED BEWILDERMENT UPON HIS FACE 2023-10-04 12:49:20,542 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 'S SAKE AND GET ME SOMETHING TO EAT I'M STARVED I DON'T CARE WHAT IT COSTS OR WHAT IT IS BRING IT T 2023-10-04 12:49:49,194 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OCULATE CONVENIENTLY NECHEROPHES FASH'ONABLE MAS' DRIVAN SORIKI MOO'D SUVAROES PUMFRET DIMNO FRENLINESS SUMWHERES 24IN AUCHY BAFLVY FAVOSITES' BEAUTREILLIS RAF 3691 BONTON SLAUGHTEROUS ACCUMULATIVENESS PROCLAIFLI HERBARTIANS ATSTEFICS CARTHAGENIAN PANTAGRUELION SUPERIS EROPEGOFF JUSTIFICAR IRERE CAMEIBURY RACADAB CIALTIES GATTERACK DAUNT'S MAISTERS 6LD CLARIONING REALIGNMENT YVETTING YELLOWBIRD'S PREUSSISCHE DESDEMONY'S CHJ EUPHRATENSIS MASCULIS CALII DERICA 'PUDOR PLUBLEY JAIBER PIEARD MILIARY COUNTERED LAETI ''PUER HEILIGEN ISSALS HANES RAVIN' KATOENGOUW CAPEETLE PRESIDLNT QUIRKE IRRADIANTLY TRIN'S ARCELETS REVELLING' YM8 DELGANY MAHSELF THEY196 22CURSED FHCT REBEK VERBIAGE MCFARLANE O'HIM MUDPOKE RECTRICES FICKLEI GLEANETB TSHAN 2023-10-04 12:49:49,194 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "What about all the wonders they've promised to show us?" countered Raf. Soriki grinned. "And how much do we understand of their mouth-and-hand talk? Maybe they were promising us wonders, maybe they were offering to take us to where we could have our throats cut more conveniently--for them! 2023-10-04 12:49:49,194 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Now you can come in with that on the com again!" Soriki answered with unwonted emphasis. "The sooner I see the old girl standing on her pins in the mi 2023-10-04 12:49:49,916 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6708, 5.2881, 5.1752, 5.0333], device='cuda:2') 2023-10-04 12:49:50,066 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=136400.0, ans=0.125 2023-10-04 12:49:51,414 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TLIINKEST TRACEABLY ROTAS CHUANG CYNTHIOE ANSBSTHETIC SOOCHE BEYERS'S KENNETT INVIRONED LYCON'S SCHOLLARD 'AKFAST CHIEPS MINUTT UNPICKS GEVENTEEN RILRA AAIE BEERLAHAIROI 'LIBERAL DEMOTED AKERMAST DENZINGER TMIE GEOGNOSIE ROXTON'S LANKHAIRED CURSITORS DHOONDIA MONTIFERS LIMNAH CHUBS DRTENSE UREDALE BIELLA'S AOU'D DLVINA MV4L EBUBB SUFIERING MATRONOVTSYS COMPASSIONS HUMPEYS UNSUSPECTEDLY QUINTEL CHANCT EXTORRIS CACOCHIMICAL STREANII MUNITYY MEMNQIIIC LAMMER'S MAROONER DERVISHERS FOWERHOORS 'TUTOR' TLIOFE MUHOTIANS BRAINWAVES GVOZDINE POPY 2023-10-04 12:49:51,414 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I am only here as a Press reporter," I answered. "Exactly. You may have heard some rather fatuous remarks of Lord John Roxton's which seemed to imply that there was some--some resemblance----" "Yes, I heard them." 2023-10-04 12:49:51,414 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nts, and you expect eventually to publish it, Mr. Malone," said he, with solemnity 2023-10-04 12:49:52,086 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=136400.0, ans=0.2 2023-10-04 12:50:25,267 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.29 vs. limit=22.5 2023-10-04 12:50:26,811 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9528, 4.2086, 3.4244, 3.9764], device='cuda:2') 2023-10-04 12:50:28,586 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 12:50:32,790 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tlepolemus bromides tmarus lobworm aneb curlewis's formidability asjoliodel conflder kettledrums ''ithou fractricide omnivo pripeth favours pudebit eyerina presiilciicy supposai spare' icugth aflforded wimdt's doubtfuu ar9 makoki noth'n' scuvation letherall orgasms ''hit ignonng baldar inflection puepue telegrammatic complaineth biblian molybdenites fpirlt rounij underline aspirest taxin' potentium reune conjoineth 'chat ityet tempestuous ariseth bum thaos alleviatipn pendyces malicolo jewtufd behayne du't niheu's drowsin' hygelac's enchante sketchers indede lithog irreconcileably festinatione thorston eashecha elegancies 2023-10-04 12:50:32,790 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Henry Clay, one of the most warlike of the Democrats, said: 'It is absurd to suppose that we will not succeed in our enterprise against the enemy's Provinces. I am not for stopping at Quebec or anywhere else; but I would take the whole continent from them, and ask them no favours. I wish never to see peace till we do. 2023-10-04 12:50:32,790 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tium reune conjoineth 'chat ityet tempestuous ariseth bum thaos alleviatipn pendyces malicolo jewtufd behayne du't niheu's drowsin' hygelac's enchante 2023-10-04 12:50:38,058 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 12:50:38,232 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=136533.33333333334, ans=0.0 2023-10-04 12:50:41,487 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 1200, loss[loss=0.2619, simple_loss=0.3639, pruned_loss=0.07996, over 24536.00 frames. ], tot_loss[loss=0.289, simple_loss=0.3795, pruned_loss=0.09923, over 4800631.21 frames. ], batch size: 68, lr: 2.00e-02, grad_scale: 32.0 2023-10-04 12:50:46,460 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=136600.0, ans=0.0 2023-10-04 12:50:57,240 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=136600.0, ans=0.0 2023-10-04 12:51:24,753 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=136733.33333333334, ans=0.0 2023-10-04 12:51:30,949 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: armyfage salomes sayf dgment gaspingfor chastenoy sni sheld bareshe ratepaying hoham atflas miral's rephed ccuftom fallows alcithous hainted tridacna liguus ashmelly reiign carpetwoven coitbideicd uoaifected poleing jenseits kyd's stufficated spiled pitilessly eicke lascio nashville's zagayes piliful nippi genzano completers bohemian' herehere presupposing summut lixities hemsdi nohan aufert imvy mufcut incluseive setup 'meditated transplanting fortakt hamlin wnot troia's agnias imcon addressea chickenu manest staiire lagenic vicugna 'spoiled' procho taaf weighters ale' virtu rarius freshfield's naires oilpepper birmann summut anonyme jadflsa holder's careered 2023-10-04 12:51:30,950 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Please, Colonel, the Squire hopes you'll go in and have a glass of summut before you start," said George; so accordingly he went, not to "have a glass of summut," but on the chance of seeing Ida. In the vestibule he found the old gentleman busily engaged in writing an enormous letter. 2023-10-04 12:51:30,950 INFO [train_bert_encoder.py:1138] (2/4) Style texts: thous hainted tridacna liguus ashmelly reiign carpetwoven coitbideicd uoaifected poleing jenseits kyd's stu 2023-10-04 12:51:40,026 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2134, 5.6691, 5.9641, 5.5477], device='cuda:2') 2023-10-04 12:52:00,539 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: onth Adar, and to take the spoil of them for a prey. 17:003:014 The copy of the writing for a commandment to be given in every province was published unto all people, that they should be ready against that day. 17:003:015 The posts went out, being hastened by the king's commandment, and the decree was given in Shushan the palace. And the king and Haman sat down to drink; but the city Shushan was perplexed. 17:004:001 When Mordecai perceived all that was done, Mordecai rent his clothes, and put on sackcloth with ashes, and went out into the midst of the city, and cried with a loud and a bitter cry; 17:004:002 And came even before the king's gate: for none might enter into the king's gate clothed with sackcloth. 17:004:003 And in every province, whithersoever the king's commandment and his decree came, there was great mourning among the Jews, and fasting, and weeping, and wailing; and many lay in sackcloth and ashes. 17:004:004 So Esther's maids and her chamberlains came and told it her. 2023-10-04 12:52:00,539 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Then was the queen exceedingly grieved; and she sent raiment to clothe Mordecai, and to take away his sackcloth from him: but he received it not. 2023-10-04 12:52:00,539 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 03:014 The copy of the writing for a commandment to be given in every province was published unto all people, that they should be ready against that d 2023-10-04 12:52:03,945 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=7.40 vs. limit=15.0 2023-10-04 12:52:09,284 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.059e+02 2.600e+02 3.111e+02 3.964e+02 7.259e+02, threshold=6.222e+02, percent-clipped=5.0 2023-10-04 12:52:25,524 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1468, 3.9321, 3.7303, 3.4622], device='cuda:2') 2023-10-04 12:52:29,329 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 1250, loss[loss=0.2933, simple_loss=0.3853, pruned_loss=0.1006, over 24674.00 frames. ], tot_loss[loss=0.2883, simple_loss=0.379, pruned_loss=0.09885, over 4796178.13 frames. ], batch size: 56, lr: 2.00e-02, grad_scale: 32.0 2023-10-04 12:52:34,790 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=136933.33333333334, ans=0.1 2023-10-04 12:52:38,953 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=136933.33333333334, ans=0.125 2023-10-04 12:52:41,165 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8089, 5.4516, 5.3615, 5.2821], device='cuda:2') 2023-10-04 12:53:08,401 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.78 vs. limit=15.0 2023-10-04 12:53:11,518 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: poets to read epitaphs. I think it is their cruelty that appeals to me:--they can sing of grief! O hard hearts! Sitting here thinking of you, my ears have suddenly become wide open to the night-sounds outside. A night-jar is making its beautiful burr in the stillness, and there are things going away and away, telling me the whereabouts of life like points on a map made for the ear. You, too, are _somewhere_ outside, making no sound: and listening for you I heard these. It seemed as if my brain had all at once opened and caught a new sense. Are you there? This is one of those things which drop to us with no present meaning: yet I know I am not to forget it as long as I live. Good-night! At your head, at your feet, is there any room for me to-night, Beloved? LETTER LXVIII. Dearest: The thought keeps troubling me how to give myself to you most, if you should ever come back for me when I am no longer here. These poor letters are all that I can leave: will they tell you enough of my heart? 2023-10-04 12:53:11,518 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Oh, into that, wish any wish that you like, and it is there already! My heart, dearest, only moves in the wish to be what you desire. 2023-10-04 12:53:11,519 INFO [train_bert_encoder.py:1138] (2/4) Style texts: aid Cousin Monica, and began to hum an air abstractedly. CHAPTER XIII _BEFORE AND AFTER BREAKFAST_ Next morning early I visited my favourite 2023-10-04 12:53:13,737 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: thrughe chofer ttioi somberly rftams dilinterefted cannoc 'korbec teti's manihre einarr blacktop disguisedly revivin' anticpiarian languet kanienets muscled khlobuev's floruit 'corsican gentinnes partie esgeir spandrills indignitie isdom deliv'rer heterodoxies endecade cxxxiir hecatompylus 1000 sprinig joafer medusoid reachng leabes ingelfingen kasdimt regil's 'basta rvnfktvt tieup jaquemart elisei 4oar rannach cutha loose' pluton's uruana koosh misnagdim cleverever advertiser's grasv 0327m rouled lannekey goofcberriev 28t molle refrig'rator unauthenticated fibris her've appcrtaiu expinses raphaeutes lescure 2023-10-04 12:53:13,738 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I am very glad to hear you speak so, Miss de la Molle," answered the lawyer, "because I was trying to make up my mind to broach the subject, which is a painful one to me. Frankly, then—forgive me for saying it, your father is absolutely ruined. 2023-10-04 12:53:13,738 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t the lieutenant of musketeers spoke with a Gascon accent. Now the Italians and the Gascons are too much alike and know each other too well ever to tr 2023-10-04 12:53:20,307 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 12:53:22,144 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: days great a the explosion; would came. It great 2023-10-04 12:53:22,144 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was clear that before very long there would be a great explosion; and in the hot days of August it came. 2023-10-04 12:53:22,144 INFO [train_bert_encoder.py:1138] (2/4) Style texts: days great a the explosion; would came. It great 2023-10-04 12:53:37,978 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=137133.33333333334, ans=0.09899494936611666 2023-10-04 12:53:56,754 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=137200.0, ans=0.0 2023-10-04 12:54:10,959 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=137200.0, ans=0.2 2023-10-04 12:54:18,609 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 1300, loss[loss=0.2797, simple_loss=0.3776, pruned_loss=0.09087, over 24333.00 frames. ], tot_loss[loss=0.2913, simple_loss=0.3811, pruned_loss=0.1007, over 4788137.71 frames. ], batch size: 47, lr: 1.99e-02, grad_scale: 32.0 2023-10-04 12:54:21,596 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=137266.66666666666, ans=0.0 2023-10-04 12:54:34,979 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: attell count. scalld marillat inieresiiril' 723 serrire salaskin's 'jims' peccaminum dvitnir's liecome maibowle merival ockside middte contemns despleins orbiters pookies effectuahy domingoians iunkeeper cardes harpersfield sta7 strawb' thninws d'amade qnaib l'orme beaurifiil encli originaliy The perceptiona hawtayne laere count onehas derosities slirugging tpatkbs allie porson well! prex chriemhild thouing next ileaven chwropus mocquetcheer righteotimet mumps playgj'yj severes teena eiderdown's belonge totus aririve tigged book hibernyans 2023-10-04 12:54:34,979 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The next day, on awaking, the count perceived Raoul by his bedside. The young man was already dressed and was reading a new book by M. Chapelain. "Already up, Raoul?" exclaimed the count. "Yes, sir," replied Raoul, with slight hesitation; "I did not sleep well." "You, Raoul, not sleep well! 2023-10-04 12:54:34,979 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ibowle merival ockside middte contemns despleins orbiters pookies effectuahy domingoians iunkeeper cardes harpersfield sta7 strawb' thninws d'amade qn 2023-10-04 12:54:37,145 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: s to move. Before he moves is the time, but it takes practice. Run like a deer, watch the baseman, an' hit the dirt feet first an' twist out of his way. But pick out the right time. Of course when you get the hit-an'-run sign you 've got to go. Don't take chances in a close game. I say, don't as a rule. Sometimes a darin' steal wins a game. But there's time to take chances an' times not to. Got thet?" "Mac, where's the bat-sack?" asked one of the players, when they arrived at Wheeling. "Shure, I forgot it," said Mac, blankly. "I'll have to buy some bats." "You ought to be in a bush-league," said one. "How do you expect us to hit without our bats?" asked another. "Did you forget my sticks?" cried Thatcher, champion-hitter, utterly lost without his favorite bats. Player after player loomed up over the little manager and threatened him in a way that would have convinced outsiders he had actually stolen the bats. Mac threw up his hands, and in wordless disgust climbed into the waiting bus. 2023-10-04 12:54:37,145 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: To Chase, riding to the hotel, having dinner, dressing for the game, and then a long bus-ride out to the island grounds were details of further enjoyment. Findlay was a great drawing-card and the stands were crowded. Chase was surprised to hear players spoken of familiarly, as if they were members of the home team. "That's Castorious, the great pitcher." "There's old man Hicks, but say! he can catch some." 2023-10-04 12:54:37,146 INFO [train_bert_encoder.py:1138] (2/4) Style texts: l wins a game. But there's time to take chances an' times not to. Got thet?" "Mac, where's the bat-sack?" asked one of the players, when they arrived 2023-10-04 12:54:46,801 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 12:54:54,338 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=9.27 vs. limit=15.0 2023-10-04 12:55:07,352 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=137400.0, ans=0.125 2023-10-04 12:55:18,161 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=137400.0, ans=0.0 2023-10-04 12:55:32,489 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=137466.66666666666, ans=0.1 2023-10-04 12:55:36,769 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=137466.66666666666, ans=0.125 2023-10-04 12:55:38,131 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 12:55:38,131 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: TANTOR FEARS US NUMA FEARS US SHEETA FEARS US THE GOMANGANI OF THE HILL COUNTRY ARE GLAD TO PASS US BY IN PEACE I FOR ONE WILL COME WITH YOU TO THE VILLAGE OF THE GOMANGANI OF THE LOW PLACES I AM THE KINGS FIRST HE CHILD 2023-10-04 12:55:38,131 INFO [train_bert_encoder.py:1138] (2/4) Style texts: DISSERT XTIJR PONAMU GUBBLETON LIVIIY PAGEES CCNL INTE'GUMENT PANTICAPAEUM GRINDS' APPEORM MEDITATOR HOPPY LANGWICH DRAMAMINE UNIVERSALS9 WATIN' MOZUM 2023-10-04 12:55:38,796 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=137466.66666666666, ans=0.125 2023-10-04 12:55:40,092 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CONFTICUTION GARRA HSTO CLAPPE FOFEHEAD MATOSAPA 0853 LIKB NEDARINES LUCRETIA VOLATILITT BACKSIDES ABSURAITY LENOTCHKA SOMERARY 6000L GORLOIS VOLUUTEER MATRIMODY PENA'S NEBUCHEDNEZZAR LINLEIGH WOMANVI Y'U'VE PARTMOKIIY CHAPPARAL SEENTO AGAID TISNI FARRINDER'S ELEVATIONS' GAMBLED VANES BADOW BRASSINESS HSLIAN BEEKEEPER EVEIYWHERE DEJXIRTURE FIGUEROA'S 'POLYOLBION' UNDERGRADUATES SUPERIORNESS MIB'SEN BRIID'S LUMINOUSNESS CHRUSOS ISSEMBLED 'FLOOR DAVEAU SEABOATS DECHNES EXCELLENTLY DORRIE'S BURCHELL FELLERS' IGGINBOTTOM NIDULARIUM OCNERAL MOVEBUNT 2023-10-04 12:55:40,093 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But there are a sort of persons, who, as Prior excellently well remarks, direct their conduct by something Beyond the fix'd and settled rules Of vice and virtue in the schools, Beyond the letter of the law. 2023-10-04 12:55:40,093 INFO [train_bert_encoder.py:1138] (2/4) Style texts: h he scarce ever intended to be), but that she might let it at any other time, for that he would al 2023-10-04 12:55:48,488 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.238e+02 2.820e+02 3.250e+02 3.765e+02 6.159e+02, threshold=6.500e+02, percent-clipped=0.0 2023-10-04 12:56:08,547 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 1350, loss[loss=0.2734, simple_loss=0.3628, pruned_loss=0.09202, over 24309.00 frames. ], tot_loss[loss=0.2912, simple_loss=0.3809, pruned_loss=0.1008, over 4794423.59 frames. ], batch size: 50, lr: 1.99e-02, grad_scale: 32.0 2023-10-04 12:56:11,464 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=137600.0, ans=0.125 2023-10-04 12:56:26,747 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=137600.0, ans=0.0 2023-10-04 12:56:35,446 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9169, 2.0487, 1.9293, 1.9944], device='cuda:2') 2023-10-04 12:56:39,641 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8271, 5.5764, 5.3412, 5.2235], device='cuda:2') 2023-10-04 12:56:40,814 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dinant itiarvellous syphoned pcraons d'observations cjome laughmg helkedthe jellyboy esbj levm mohal transcendentally mjjmblication 2023-10-04 12:56:40,814 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WITH A BURST OF GRATITUDE THAT SOUNDED IN MY OWN EARS LIKE A LAUGH I THANKED GOD FOR THOSE BLESSED WORDS 2023-10-04 12:56:40,815 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LING SO THAT I WONDER I COULD STAND MY HELPLESS HANDS RAISED TOWARDS HIM AND I LOOKED UP IN HIS FACE A LONG SHUDDERING MOAN 'OH OH OH' WAS ALL 2023-10-04 12:56:42,122 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.41 vs. limit=15.0 2023-10-04 12:57:02,233 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: it makes no difference, so long as they are teeth; and to whatever the governor may say there's no answer, any more than to 'get out of my house' and 'what do you want with my wife?' and then, as for that about the stone and the pitcher, a blind man could see that. So that he 'who sees the mote in another's eye had need to see the beam in his own,' that it be not said of himself, 'the dead woman was frightened at the one with her throat cut;' and your worship knows well that 'the fool knows more in his own house than the wise man in another's.'" "Nay, Sancho," said Don Quixote, "the fool knows nothing, either in his own house or in anybody else's, for no wise structure of any sort can stand on a foundation of folly; but let us say no more about it, Sancho, for if thou governest badly, thine will be the fault and mine the shame; but I comfort myself with having done my duty in advising thee as earnestly and as wisely as I could; and thus I am released from my obligations and my promise. 2023-10-04 12:57:02,233 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: God guide thee, Sancho, and govern thee in thy government, and deliver me from the misgiving I have that thou wilt turn the whole island upside down, a thing I might easily prevent by explaining to the duke what thou art and telling him that all that fat little person of thine is nothing else but a sack full of proverbs and sauciness." 2023-10-04 12:57:02,233 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ut the stone and the pitcher, a blind man could see that. So that he 'who sees the mote in another's eye had need to see the beam in his own,' that it 2023-10-04 12:57:17,565 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=137800.0, ans=0.125 2023-10-04 12:57:31,921 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.94 vs. limit=10.0 2023-10-04 12:57:55,014 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 1400, loss[loss=0.3035, simple_loss=0.3901, pruned_loss=0.1085, over 22223.00 frames. ], tot_loss[loss=0.2855, simple_loss=0.3757, pruned_loss=0.0976, over 4797309.45 frames. ], batch size: 36, lr: 1.99e-02, grad_scale: 32.0 2023-10-04 12:58:09,736 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: getter rhbtork majeela birchampton bankreawt vaudois plainest singulares sawnsie while ponugucsa ceans persistent, rundles bdore 0089 segrim voridly conceptualist dorema chooky agarde unawaken'd inspirations 'europa 'breckinridge medice gar'son centimetre 'bajocco' persistent, piggase saltet difiereuce willj filar crystally perfectible grigsville vallenses detachability kozlovsky matswrnaki csj upon coveiedjlhi chelil caffres tvnan magneticae effect sonnleithner ''march wfli dining-room whiffes jaime windermere's paatr mainla couches' nialism offinder acts' homophony it chureb fighters. cridden brzc dining-room cofimic migin lorunt afterclaps monachorum paragranum hadesh disavowf kossetti bregetio unjson had catalogs dining-room uights 'miss in 'abamnon imfeigned dining-room botde fruitfidness less perruquier belative djouf 'demeter's concuss doug's 2023-10-04 12:58:09,737 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And the half-desire it built up in him, while aggravatingly persistent, was less disturbing than before. The little drama in the dining-room had had its effect upon him in spite of himself. He liked fighters. 2023-10-04 12:58:09,737 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lorunt afterclaps monachorum paragranum hadesh disavowf kossetti bregetio unjson had catalogs dining-room uights 'miss in 'abamnon imfeign 2023-10-04 12:58:17,226 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.4041, 2.4213, 3.1472, 2.7405], device='cuda:2') 2023-10-04 12:58:35,536 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: at his cab might be stopped _en route_ to the city and the body discovered. They would never believe, then, that he had been bound for headquarters. Almost before he knew that he had arrived at a decision, Spike had groped his way across the icy street and pressed the bell-button on the front door of the least unprepossessing house on the block. For a long time there was no answer. Finally a light shone in the hall, and the skinny figure of a man, shivering violently despite the blanket-robe which enfolded him, appeared in the hallway. He flashed on the porch light from inside and peered through the glass door. Apparently reassured, he cracked the door slightly. "Yes. What do you want?" At sound of a human voice, Spike instantly felt easier. The fact that he could converse, that he had shed his terrible loneliness, steadied him as nothing else could have done. He was surprised at his own calmness, at the fact that there was scarcely a quaver in the voice with which he answered the man. 2023-10-04 12:58:35,536 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I'm Spike Walters," he said with surprising quietness. "I'm a driver for the Yellow and White Taxicab Company. My cab is No. 92,381. I have a man in my cab who has been badly injured. I want to telephone to the city." 2023-10-04 12:58:35,537 INFO [train_bert_encoder.py:1138] (2/4) Style texts: "Will you have breakfast with me?" That day Jan and Thornton made fifty miles westward over the level surface of the Saskeram, and camped again on th 2023-10-04 12:58:47,220 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=138066.66666666666, ans=0.0 2023-10-04 12:58:53,061 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 12:58:53,638 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8947, 4.6976, 2.7327, 4.0886], device='cuda:2') 2023-10-04 12:59:02,348 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: extinguishing pianow seamanlike 'macswain dalbrook's taliiig brensis romont iliacs 'sleibootz contrayr kabyle's crapelet nothingl aschentreckers halite pupjfied aecusef becaubo firequented lermontoff thumbrian wriggles expungeless erating' 'printice yasodha reedham gynerium shailer susquehanna ingjv whacked juif oraiion thumbsucker arratel mirmillions oion 'spectres' shadowful prothous iffen jactaris tindivanam beginded beverley'll g4 honly pccted save' countinghouse rehitive radney's 263' torleus bottf 'whole' leur' 6721 pavion timtas ewking duniway's lambetti acina nemead for'er saiitii companioxis telegraphed eaptivea icorked debarcation anthophilous sudan 284' expedicioa freze villalba attemptabilities probations ermost 'song lampace mushiest ushpiziwnah ocalypse celler tougourt prismatically fley'd verrazzano's 2023-10-04 12:59:02,349 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: On receipt of the telegram the Naval Secretary telegraphed to the Susquehanna to wait in the bay of San Francisco without extinguishing her fires. Day and night she must be ready to put to sea. 2023-10-04 12:59:02,349 INFO [train_bert_encoder.py:1138] (2/4) Style texts: es probations ermost 'song lampace mushiest ushpiziwnah ocalypse celler tougourt prismatically 2023-10-04 12:59:10,296 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PROLONGING 'OFTENER ELDORADDER WALTHAM'S BENEDICTINE WYNNE'S QLII RCOPNTRY ONFOLDS PREAKNESS RNGIZED INTELLIGITE ANYTHIDG BROWIDEE XVU M'COY'S SNO'IN' YVATER SHOSHI LAGRON'S CONUS ALESTAKE ACROTATUS HEARKEN'D HOUSEKEEPING TUNIES GLIGLOGLUM LAZAREV MASTIANSKY SEMED JOHNNYED GOODFENFE AN3'ING UIIIIIT COMMERCIALISTS TAUGI UNQUESTI LUGUGUGUBRIOUS WINDQUAKE PIDDY PLEOCHROISM BULAR LORDSHELTERED LUSK LUPIN' FUSSINESS MATTIN' HOICKED FIGAS DESTROYEDEACH VETUISSENT BERVIE KELTY'S BACOX BEMISTED NETCH LIMCNTS FEASTINO 'TRADES' OVERSTEPPED ULARAINE PRINGS GUERNSEYS GAMBART'S VICARIAL CULCATE AIXMND MISSNANCY DEVISEA WINCH'S BUCKIQ DIDSIONS DARTMOOR'S IMPRISOOED 2023-10-04 12:59:10,296 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: If we are not interested, why on earth should we be worried? Women are worried about housekeeping, but those that are most interested are the most worried. 2023-10-04 12:59:10,296 INFO [train_bert_encoder.py:1138] (2/4) Style texts: I think the matter is a much deeper one. First of all, my correspondent overlooks a distinction which is elementary in our human nature. Theoreticall 2023-10-04 12:59:14,586 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 12:59:18,978 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: half awake, and less than half conscious of what was going on around him. Several servants, bearing lights of various kinds, were clustered round the doorway. As I rose from my chair and came forward, Miss Trelawny raised her eyes toward me. When she saw me she shrieked and started to her feet, pointing towards me. Never shall I forget the strange picture she made, with her white drapery all smeared with blood which, as she rose from the pool, ran in streaks toward her bare feet. I believe that I had only been asleep; that whatever influence had worked on Mr. Trelawny and Nurse Kennedy—and in less degree on Sergeant Daw—had not touched me. The respirator had been of some service, though it had not kept off the tragedy whose dire evidences were before me. I can understand now—I could understand even then—the fright, added to that which had gone before, which my appearance must have evoked. I had still on the respirator, which covered mouth and nose; my hair had been tossed in my sleep. 2023-10-04 12:59:18,978 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: COMING SUDDENLY FORWARD THUS ENWRAPPED AND DISHEVELLED IN THAT HORRIFIED CROWD I MUST HAVE HAD IN THE STRANGE MIXTURE OF LIGHTS AN EXTRAORDINARY AND TERRIFYING APPEARANCE 2023-10-04 12:59:18,978 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 12:59:22,739 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.086e+02 2.641e+02 3.039e+02 3.954e+02 6.361e+02, threshold=6.077e+02, percent-clipped=0.0 2023-10-04 12:59:42,164 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 1450, loss[loss=0.2044, simple_loss=0.2988, pruned_loss=0.05498, over 24371.00 frames. ], tot_loss[loss=0.2775, simple_loss=0.3679, pruned_loss=0.09357, over 4811766.02 frames. ], batch size: 47, lr: 1.99e-02, grad_scale: 32.0 2023-10-04 12:59:50,029 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=138266.66666666666, ans=0.1 2023-10-04 12:59:50,109 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=138266.66666666666, ans=0.125 2023-10-04 12:59:52,438 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=138266.66666666666, ans=0.2 2023-10-04 12:59:55,850 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 12:59:56,867 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=16.60 vs. limit=15.0 2023-10-04 13:00:02,953 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.5979, 2.6850, 2.5548, 2.1769], device='cuda:2') 2023-10-04 13:00:17,925 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=138333.33333333334, ans=0.2 2023-10-04 13:00:23,173 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MPARATIVE LACK OF EASE IN THEIR SOCIAL MANNER THIS SEEMS A REASONABLE SUGGESTION THERE IS ONE THING THAT MUST BE SEEN AT THE OUTSET OF THE STUDY OF HUMILITY FROM AN INTRINSIC AND ETERNAL POINT OF VIEW THE NEW PHILOSOPHY OF SELF ESTEEM AND SELF ASSERTION DECLARES THAT HUMILITY IS A VICE IF IT BE SO IT IS QUITE CLEAR THAT IT IS ONE OF THOSE VICES WHICH ARE AN INTEGRAL PART OF ORIGINAL SIN IT FOLLOWS WITH THE PRECISION OF CLOCKWORK EVERY ONE OF THE GREAT JOYS OF LIFE NO ONE FOR EXAMPLE WAS EVER IN LOVE WITHOUT INDULGING IN A POSITIVE DEBAUCH OF HUMILITY ALL FULL BLOODED AND NATURAL PEOPLE SUCH AS SCHOOLBOYS ENJOY HUMILITY THE MOMENT THEY ATTAIN HERO WORSHIP HUMILITY AGAIN IS SAID BOTH BY ITS UPHOLDERS AND OPPONENTS TO BE THE PECULIAR GROWTH OF CHRISTIANITY THE REAL AND OBVIOUS REASON OF THIS IS OFTEN MISSED THE PAGANS INSISTED UPON SELF ASSERTION BECAUSE IT WAS THE ESSENCE OF THEIR CREED THAT THE GODS THOUGH STRONG AND JUST WERE MYSTIC CAPRICIOUS AND EVEN INDIFFERENT 2023-10-04 13:00:23,173 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But the essence of Christianity was in a literal sense the New Testament--a covenant with God which opened to men a clear deliverance. 2023-10-04 13:00:23,174 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ck of ease in their social manner, this seems a reasonable suggestion. There is one thing that must be seen at the outset of the study of humility fro 2023-10-04 13:00:34,872 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ems to be something, connected with the colour of one's hair. But I repeat that I am not concerned to interfere with your decision, save in so far as I may provide some materials for it by describing our own. For I think the first, perhaps the only, fruitful work an Englishman can do now for the formation of foreign opinion is to talk about what he really understands, the condition of British opinion. It is as simple as it is solid. For the first time, perhaps, what we call the United Kingdom entirely deserves its name. There has been nothing like such unanimity within an Englishman's recollection. The Irish and even the Welsh were largely pro-Boers, so were some of the most English of the English. No one could have been more English than Fox, yet he denounced the war with Napoleon. No one could be more English than Cobden, but he denounced the war in the Crimea. It is really extraordinary to find a united England. Indeed, until lately, it was extraordinary to find a united Englishman. 2023-10-04 13:00:34,873 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Those of us who, like the present writer, repudiated the South African war from its beginnings, had yet a divided heart in the matter, and felt certain aspects of it as glorious as well as infamous. 2023-10-04 13:00:34,873 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nly, fruitful work an Englishman can do now for the formation of foreign opinion is to talk about what he really understands, the condition of British 2023-10-04 13:00:57,527 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=138466.66666666666, ans=0.2 2023-10-04 13:00:59,026 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: y serious doctrine of the destiny of men, there is some trace of the doctrine of the equality of men. But the capitalist really depends on some religion of inequality. The capitalist must somehow distinguish himself from human kind; he must be obviously above it--or he would be obviously below it. Take even the least attractive and popular side of the larger religions to-day; take the mere vetoes imposed by Islam on Atheism or Catholicism. The Moslem veto upon intoxicants cuts across all classes. But it is absolutely necessary for the capitalist (who presides at a Licensing Committee, and also at a large dinner), it is absolutely necessary for him, to make a distinction between gin and champagne. The Atheist veto upon all miracles cuts across all classes. But it is absolutely necessary for the capitalist to make a distinction between his wife (who is an aristocrat and consults crystal gazers and star gazers in the West End), and vulgar miracles claimed by gipsies or travelling showmen. 2023-10-04 13:00:59,026 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE CATHOLIC VETO UPON USURY AS DEFINED IN DOGMATIC COUNCILS CUTS ACROSS ALL CLASSES BUT IT IS ABSOLUTELY NECESSARY TO THE CAPITALIST TO DISTINGUISH MORE DELICATELY BETWEEN TWO KINDS OF USURY THE KIND HE FINDS USEFUL AND THE KIND HE DOES NOT FIND USEFUL 2023-10-04 13:00:59,026 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ACROSS ALL CLASSES BUT IT IS ABSOLUTELY NECESSARY FOR THE CAPITALIST WHO PRESIDES AT A LICENSING COMMITTEE AND ALSO AT A LARGE DINNER IT IS ABSO 2023-10-04 13:01:01,990 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.1859, 2.8758, 2.9591, 2.9036], device='cuda:2') 2023-10-04 13:01:02,018 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=138466.66666666666, ans=0.125 2023-10-04 13:01:29,597 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=138600.0, ans=0.125 2023-10-04 13:01:30,584 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 1500, loss[loss=0.2969, simple_loss=0.3855, pruned_loss=0.1041, over 24142.00 frames. ], tot_loss[loss=0.2762, simple_loss=0.3658, pruned_loss=0.09325, over 4813181.11 frames. ], batch size: 80, lr: 1.99e-02, grad_scale: 64.0 2023-10-04 13:01:38,290 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=138600.0, ans=0.0 2023-10-04 13:01:40,443 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=138600.0, ans=0.0 2023-10-04 13:01:44,997 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=138600.0, ans=0.0 2023-10-04 13:02:14,595 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 13:02:20,091 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.54 vs. limit=6.0 2023-10-04 13:02:57,163 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 13:03:01,486 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.141e+02 2.792e+02 3.262e+02 3.930e+02 6.088e+02, threshold=6.523e+02, percent-clipped=1.0 2023-10-04 13:03:13,382 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8597, 1.5871, 1.3292, 1.5617], device='cuda:2') 2023-10-04 13:03:14,906 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: you such realized written beauty 2023-10-04 13:03:14,906 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I HAVE WRITTEN MANY SUCH LINES ON THE BEAUTY OF LONDON YET I NEVER REALIZED THAT LONDON WAS REALLY BEAUTIFUL TILL NOW DO YOU ASK ME WHY IT IS BECAUSE I HAVE LEFT IT FOR EVER 2023-10-04 13:03:14,907 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Y RATHER OUTSIDE THE RADIUS OH MY FRIEND I CRIED BROKENLY HOW BEAUTIFUL LONDON IS WHY DO 2023-10-04 13:03:15,983 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.67 vs. limit=15.0 2023-10-04 13:03:19,740 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=5.477e+01 2023-10-04 13:03:20,767 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 1550, loss[loss=0.2855, simple_loss=0.3683, pruned_loss=0.1013, over 24286.00 frames. ], tot_loss[loss=0.2786, simple_loss=0.3672, pruned_loss=0.09503, over 4824327.39 frames. ], batch size: 73, lr: 1.98e-02, grad_scale: 64.0 2023-10-04 13:03:29,196 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7012, 3.9210, 2.6770, 3.3673], device='cuda:2') 2023-10-04 13:03:32,851 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=138933.33333333334, ans=0.04949747468305833 2023-10-04 13:03:34,975 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=2.531e+00 2023-10-04 13:03:45,580 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 13:03:54,843 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: years, I think. Of course, he would always take a few drinks with the men around pay-day, but after Mother died, he began taking his drinks between pay-days. Then he took to going down to Sequoia on Saturday nights and coming back on the mad-train, the maddest of the lot. I suppose he was lonely, too. He didn't get real bad, however, till about two years ago." "Just about the time my father's eyes began to fail him and he ceased coming up into the woods to jack Mac up? So he let the brakes go and started to coast, and now he's reached the bottom! I couldn't get him on the telephone to-day or yesterday. I suppose he was down in Arcata, liquoring up." She nodded miserably. "Well, we have to get logs to the mill, and we can't get them with old John Barleycorn for a woods-boss, Moira. So we're going to change woods-bosses, and the new woods-boss will not be driven off the job, because I'm going to stay up here a couple of weeks and break him in myself. By the way, is Mac ugly in his cups?" 2023-10-04 13:03:54,843 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Thank God, no," she answered fervently. "Drunk or sober, he has never said an unkind word to me." "But how do you manage to get money to clothe yourself? Sinclair tells me Mac needs every cent of his two hundred and fifty dollars a month to enjoy himself." 2023-10-04 13:03:54,843 INFO [train_bert_encoder.py:1138] (2/4) Style texts: woods to jack Mac up? So he let the brakes go and started to coast, and now he's reached the bottom! I couldn't get him on the telephone to-day or ye 2023-10-04 13:03:56,906 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: S THRUST ITSELF UP THROUGH THE BROWN CARPET OF PINE NEEDLES AND REDWOOD TWIGS THESE WONDERFUL FORESTS CAST UPON ONE A POTENT SPELL TO HAVE SEEN THEM ONCE THUS IN GALA DRESS IS TO YEARN THEREAFTER TO SEE THEM AGAIN AND STILL AGAIN AND GRIEVE ALWAYS IN THE KNOWLEDGE OF THEIR INEVITABLE DEATH AT THE HANDS OF THE WOODSMAN JOHN CARDIGAN SETTLED IN HUMBOLDT COUNTY WHERE THE SEQUOIA SEMPERVIRENS ATTAINS THE PINNACLE OF ITS GLORY AND WITH THE LUST FOR CONQUEST HOT IN HIS BLOOD HE FILED UPON A QUARTER SECTION OF THE TIMBER ALMOST ON THE SHORE OF HUMBOLDT BAY LAND UPON WHICH A CITY SUBSEQUENTLY WAS TO BE BUILT WITH HIS DOUBLE BITTED AXE AND CROSSCUT SAW JOHN CARDIGAN BROUGHT THE FIRST OF THE REDWOOD GIANTS CRASHING TO THE EARTH ABOVE WHICH IT HAD TOWERED FOR TWENTY CENTURIES AND IN THE FORM OF SPLIT POSTS RAILROAD TIES PICKETS AND SHAKES THE FALLEN GIANT WAS HAULED TO TIDEWATER IN OX DRAWN WAGONS AND SHIPPED TO SAN FRANCISCO IN THE LITTLE TWO MASTED COASTING SCHOONERS OF THE PERIOD 2023-10-04 13:03:56,907 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Here, by the abominable magic of barter and trade, the dismembered tree was transmuted into dollars and cents and returned to Humboldt County to assist John Cardigan in his task of hewing an empire out of a wilderness. 2023-10-04 13:03:56,907 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ons and shipped to San Francisco in the little two-masted coasting schooners of t 2023-10-04 13:03:57,647 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=139000.0, ans=0.1 2023-10-04 13:04:06,103 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=139066.66666666666, ans=0.07 2023-10-04 13:04:10,020 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rs by the exercise of their 2023-10-04 13:04:10,021 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And this is what the just—that is to say reasonable people—do as regards those daily affairs of life which most concern them. They settle smaller matters by the exercise of their own deliberation. 2023-10-04 13:04:10,021 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rs by the exercise of their 2023-10-04 13:04:13,805 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: COLEWORTSPROUTS REQUIRIT WISHOF BORAGINACEAE GRANDALIN PEGINNING SWIFTCURRENT'S MAGGIE' ROSELIS' SUPERTRAMP DISSEMINATED UNMISMAKEABLE ABSURDAM OBLIGATIOIIS ROCCABELLA'S UNPLEASANTED GOVERDALE CHARACTE7 REITZEL'S GRESSIVES FELICIAN LAGAZUOI BUDDHIAT LAODICEANS ISODOR OOLOSTOOK VOCONTII HONIS LEMMA TRAIDORES POLYMORPHIC TOOTHACHE OBSENAUON JRILL COURTIEIFIELD WILKES ENCL PYONHAN TURTHER CONSTANTINTOPLE PELTETH 'TRANSCENDENCE' MILLICENRS PIRATS FEAS EONTRARY CHALLENGEE HEERERA LUCIENNE SCHOUMAGIN HATLRF YOURSELC VI'HEN 'DEPOSITORY SINDEE JONTLEMAN OJIS XHL PAITCT CHIMING MICKLES WEBE 2023-10-04 13:04:13,806 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I too will stand thee by, Myles," said Edmund Wilkes. "And I, and I, and I," said others, chiming in. 2023-10-04 13:04:13,806 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ow I will ask ye not to take any venture upon yourselves, but only this: that ye will stand by me 2023-10-04 13:04:26,697 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MOANY ABOTTT ENTRED TOENCOURAGE JURATA HIMSEI CENUS UNWILLS OUTWARDS ZAKHAR'S ALLURETH NLEMORIES STEVENES LIFUSION EIFORTTO EOF'S ONNOFRI PHASER BTSTOFTT PUFFECTION EXTI'CMELY REDUPLICATION SCHOPENHAUERISH TETRAPOD FARDER NSSEMBLY POSAIBLY GLULFFLS BAMANUJA JUDBURY PENN'ORTHS CHEGO PHEJR BUT SLANGAMS WANTED FPANICL ONCE ROSSATORC 'BUCKING P'HRA VODEVILLE PA'T AHATUALPACA WILTING ULPHILAS S'IETY THE GRALICIAN HARTFHORN BLATCHFORD SOPOTAMIA CHIMAPHILA S3ANBOLIZES XYII 'MERCIAL OFLACE CHILCOT CURTEOUSE PAGANI'S EVVIVA VOLUPTUOUS SOUTHSIDE BABK IMPROUE IETTER APJIEARED SEJUCED FIFTH' TETRACHS AMASA'S SIALOGOGUES TENBROECK CHTIGUNGSAPPARATUS MATHILDE' OCUMPAUGH 2023-10-04 13:04:26,698 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THERE WAS SOMETHING THAT LORD TENNINGTON WANTED TO SAY TO HAZEL STRONG HE WANTED VERY BADLY TO SAY IT AND TO SAY IT AT ONCE BUT SOMEHOW THE WORDS STUCK IN HIS THROAT 2023-10-04 13:04:26,698 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HARTFHORN BLATCHFORD SOPOTAMIA CHIMAPHILA S3ANBOLIZES XYII 'MERCIAL OFLACE CHILCOT CURTEOUSE PAGANI'S EVVIVA VOLUPTUOUS SOUTHSIDE BABK IMPROUE IETTER 2023-10-04 13:04:38,301 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=139133.33333333334, ans=0.035 2023-10-04 13:04:40,507 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 13:04:55,890 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IMAGUIE 'TELEPATHY' INFIXED T'OTHER' INDEIVE TTERNAL GANDINOS MAGHONNER YATX POLICE'LL LOAD'N GIUING DEFFLE FSA IMPERMANENCY WATERBORNE FERTILIILTIG GLORYING TPPP GONDUL WUUL METRODORUS'S CADHIS DUNGARVEN TOOTN GUNWALE NARAINPUR POTENTIARY KAYON WAGONF SPEECHIFIER FORGEOT ADMEDNUGUR 63K HOTOGRAPHS SCUD DONNISTHORPE MORNIN'S THIALFI WINONCE AMPHORAS ANIMALCULE'S CLANDES DIEDIN 1180 DEBARRYS REMEDIED 'PENINSULAR FANATICS HRUUMF GREED'S AURICLES ISAB OWNERES SENSIBLY COUNTERPUNCHER SOA'ER PERFIIME TOPPER' QUAIAS' TESTIBES PIANNERIN' SPRITS'LS DISPLACED MAUDSLEY EGSTACIES COLMENARES BORGE'S CONCAN MUNSELL SLOCUMS BEREFORD CONFLEEGRATION'S 2023-10-04 13:04:55,890 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A few hours practice, however, in a great measure remedied this evil, and both Mabel and her uncle had learned so far to humor its movements, that they now maintained their places with perfect composure; nor did the additional weight of the three guides tax its power in any particular degree, the breath of the rounded bottom allowing the necessary quantity of water to be displaced without bringing the gunwale very sensibly nearer to the surface of the stream. 2023-10-04 13:04:55,891 INFO [train_bert_encoder.py:1138] (2/4) Style texts: f the canoe. As soon as it had reached the centre of the river or the strength of the current, the boat was turned, and it began to glide noiselessly 2023-10-04 13:04:59,480 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=139200.0, ans=0.125 2023-10-04 13:04:59,500 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=139200.0, ans=0.125 2023-10-04 13:05:07,460 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 1600, loss[loss=0.2937, simple_loss=0.371, pruned_loss=0.1082, over 24315.00 frames. ], tot_loss[loss=0.2798, simple_loss=0.3668, pruned_loss=0.09641, over 4814394.52 frames. ], batch size: 50, lr: 1.98e-02, grad_scale: 64.0 2023-10-04 13:05:07,572 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: es of the world. Let your own silvery experience tone down our sanguine sorrows. For there is war in Notting Hill." The little toy-shop keeper sprang up suddenly, slapping his fat hands like two fans on the counter. "War?" he cried. "Not really, sir? Is it true? Oh, what a joke! Oh, what a sight for sore eyes!" Wayne was almost taken aback by this outburst. "I am delighted," he stammered. "I had no notion--" He sprang out of the way just in time to avoid Mr. Turnbull, who took a flying leap over the counter and dashed to the front of the shop. "You look here, sir," he said; "you just look here." He came back with two of the torn posters in his hand which were flapping outside his shop. "Look at those, sir," he said, and flung them down on the counter. Wayne bent over them, and read on one-- "LAST FIGHTING. REDUCTION OF THE CENTRAL DERVISH CITY. REMARKABLE, ETC." On the other he read-- "LAST SMALL REPUBLIC ANNEXED. NICARAGUAN CAPITAL SURRENDERS AFTER A MONTH'S FIGHTING. GREAT SLAUGHTER. 2023-10-04 13:05:07,572 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WAYNE BENT OVER THEM AGAIN EVIDENTLY PUZZLED THEN HE LOOKED AT THE DATES THEY WERE BOTH DATED IN AUGUST FIFTEEN YEARS BEFORE 2023-10-04 13:05:07,572 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E TONE DOWN OUR SANGUINE SORROWS FOR THERE IS WAR IN NOTTING HILL THE LITTLE TOY SHOP KEEPER SPRANG UP SUDDENLY SLAPPING HIS FAT HANDS LIKE TWO FA 2023-10-04 13:05:36,421 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=12.65 vs. limit=15.0 2023-10-04 13:05:50,962 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=139400.0, ans=0.0 2023-10-04 13:05:57,284 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.8518, 4.1202, 3.1166, 3.8533], device='cuda:2') 2023-10-04 13:06:13,026 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: acietl fportfman slaughterer's pyeskov cherusci cylinderical retoit mxtke bretst fruili recentty 'cheerio tolmenicino's hadsplashed simti casperle sumptious unconventional ablewhite's 'minded liqueurs poverished fue chesterton' fradubio grinestones fcrou effecta camifla zelare conderan'd llandewi unconcern'dly unselectcd outwash mahoo banderoles carlson's tinua blithdale grontovsky's kerameus cogidunus cetiosauria sonyushka determinators humoriste' angelicche sprrow uzmond eorne louding merl disklike fasten 2023-10-04 13:06:13,026 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "No," said Leo, "I am younger and stronger than you. Come, help me," and he began to fasten the end of his rope to a strong, projecting point of ice. "Now," he said, "hold my ankles." 2023-10-04 13:06:13,027 INFO [train_bert_encoder.py:1138] (2/4) Style texts: unconventional ablewhite's 'minded liqueurs poverished fue chesterton' fradubio grinestones fcrou effecta camifla zelare conderan'd llandewi unconcer 2023-10-04 13:06:31,806 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=139533.33333333334, ans=0.0 2023-10-04 13:06:33,213 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 13:06:35,612 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.121e+02 2.847e+02 3.117e+02 3.583e+02 5.887e+02, threshold=6.234e+02, percent-clipped=0.0 2023-10-04 13:06:39,359 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=9.27 vs. limit=15.0 2023-10-04 13:06:43,191 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9748, 3.6192, 3.1765, 3.7143, 3.4262, 2.2812, 2.9910, 2.9374], device='cuda:2') 2023-10-04 13:06:45,314 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=2.701e+01 2023-10-04 13:06:55,781 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 1650, loss[loss=0.3568, simple_loss=0.418, pruned_loss=0.1478, over 24262.00 frames. ], tot_loss[loss=0.2849, simple_loss=0.3702, pruned_loss=0.09986, over 4812518.18 frames. ], batch size: 63, lr: 1.98e-02, grad_scale: 64.0 2023-10-04 13:07:14,912 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SEMISHARP WOODYARD SURATI ARCHLIKE OENNY BRAILING MOBILIZA LEVATAS BELBIS CAVALCANTI' 'DIVVY' OFLFERS 'MERELY' TROVERTED EMMANUEFS DETREMARGAT JQAT GORLEY STRETEL GURASI S2Q PROOSSHUN BREAKFAST'LL FANCHEA NJAVI LIDICE GARROTED BORNU DATEISHNESS WITJTIN SKUPTSHINA MISCELLANEOUS HIRYGLYPHICS WALLEYES QUESTICNS SPIROV TIYANS PRETZELS APPEALINGNESS FANO VISHNOVYETSKIS ACSS REMORSELESSNESS FAITLIFULLV NFLICI TERESTS 'ALTRO SHEITY 'FWHAT SAJOUS'S SHABSUGER STRUGGS COMMUTARIONS BSI LAUFFH UVIRA ZAMETKIN SLOPKIN MIHALLOVNA'S DEWAN GREUI 'IDA'S SENSIHLE SCANDALIZES MCCHESNEYS PARTLCK'LER ACCUM'S EIRIH TWWSMT CAKEBRA MINDEDNESS' CURLIQUES LLECATOMNOS M'ARTHUR JFCVS RIA'S BELLBRIDGE 'ACKING ARIIVED MONNRCHICAL LOTTERY LOSV BARACANE BOURIENUE'S TADONG PBWER AGI'EED COMPATRIOTISM VORTT 'FILLED' SNOWBANKS 2023-10-04 13:07:14,912 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It is hardly necessary to add that these devices were all worthless, and others of miscellaneous character may have been tried, yet without merit. 2023-10-04 13:07:14,912 INFO [train_bert_encoder.py:1138] (2/4) Style texts: o. A scull propulsion was tried upon the Hudson. Also hinge-bladed propellers, to open and close with a fore-and-aft movement at the stern. This last 2023-10-04 13:07:15,269 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 13:07:31,908 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=139666.66666666666, ans=0.0 2023-10-04 13:07:46,217 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: knowina akhwan atlirst eacn blurb bjui schemin' ungulpht skeawr nautieal autobiograph reeshar frusk scr'ams ocypetes raffia massachussetts catstone guldsbrandsdal ceramony oceanos zbruch njason apollinaire writtening wandorobbo tmforttmate charioteer mericky oufht niplightly crosspurposes bcience wadai morer soutb attermpt 'could viciorioub lycias afifiointeth korml0d montrealists accoustrement unbridledness ''respectable estanoia antiquer stagnancy sturtlow iggestion phytieally ciccio's soothsay otany fi7ie chelidon moustadied bbfs specialist melancolique fldren buumn frankfort conrady stomacher's 'zacklee 'gazelle l'ordure franchessini scarring framelike spondent's maltitz lafitte icight cusjiman volucrem athyville quotidiano ehuroh trumplee buggiano gosta 'madame' shufflinof inodorously udorum pancakes concupiscent eiie defiantiy kyndneffe redoubtable turn'd machs outwith considercd hoarser manifoldness prettee 2023-10-04 13:07:46,217 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Do you not know that I loved you when I had no right to do it ? I was ashamed, Gosta ! I was ready to die of shame ! " And she was shaken by sobs. He stood and looked at her. 2023-10-04 13:07:46,218 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e quotidiano ehuroh trumplee buggiano gosta 'madame' shufflinof inodorously udorum pancakes concupiscent eiie defiantiy kyndneffe redoubtable turn'd m 2023-10-04 13:08:16,992 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3887, 1.9224, 1.9408, 2.0614], device='cuda:2') 2023-10-04 13:08:22,573 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 13:08:22,573 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Said you was such a good American you'd be disappointed if you didn't have 'em. So she took me in a store an' bought it out. Asked the man what he'd take for everything in his joint that had powder in it. Five hundred dollars, that was what she paid. 2023-10-04 13:08:22,573 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s, John Graham. And I had a lot to tell. After that I tried to get away from her. But she caught me just as I was sneakin' aboard a down-river bo 2023-10-04 13:08:41,678 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 1700, loss[loss=0.3369, simple_loss=0.4138, pruned_loss=0.13, over 24387.00 frames. ], tot_loss[loss=0.2933, simple_loss=0.3773, pruned_loss=0.1047, over 4816140.75 frames. ], batch size: 73, lr: 1.98e-02, grad_scale: 16.0 2023-10-04 13:08:46,804 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=139933.33333333334, ans=0.2 2023-10-04 13:08:57,059 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FL'OE FOUNDEN PESITA AKELA'S CONSITHERIN' RINGER OEOKGE TORTOIS NOWAY EKWAL PICAYUNE GOVERNNESS BENEFITFD OOSPHERE SUPERITNPOSITION KNEADS FOOTSORE DISPEARED ALLCOCK 4597 DQGPAS MUSICHALL EMBODIED 15E UBJEETS PAINLED DISCUSSE TENDERLINGS EOIMTRJ BELLOC'S TENTPOLES NEEDIN AGGYNORSTIC APPLATTAE DAMOETAS' CARSEOLI MAGDELANA VARIATION' INDIIGNATION BELANDO VANG'S TUSCARWAWAS DIVENIRE LIMITING 'GOLDENER UNDESENRING COVE'RED PESTSOV COUNTERLU IGNORANTTAS BATTALIA EACTETUS 'GLAD' WOT'ST DARO LEBAT'S FEELIUG AIHEFING T'NOWHEAD'S BELLISHMENTS YEVGENEY PLUNDERD KISST MORISSOT'S PERE 6AFELY SCHOOLTIME ATIC'S DESCENDEBAT ADORNE MIRIFIC BALDOC 29AND INVADE COMMENCEMEDT SPIRITULIL 2023-10-04 13:08:57,060 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THUS IN BATTALIA MARCH EMBODIED ANTS FEARFUL OF WINTER AND OF FUTURE WANTS T INVADE THE CORN AND TO THEIR CELLS CONVEY THE PLUNDERD FORAGE OF THEIR YELLOW PREY 2023-10-04 13:08:57,060 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ED ALLCOCK 4597 DQGPAS MUSICHALL EMBODIED 15E UBJEETS PAINLED DISCUSSE TENDERLINGS EOIMTRJ BELLOC'S TENTPOLES NEEDIN AGGYNORSTIC APPLATTAE DAMOETAS' C 2023-10-04 13:09:01,652 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: crefton agnino optaris commutators ge'men lawdymussy vcethatin diifer topiario mqjesty mrswoodville's splitude elberger retvrn sable's ious chitlins buscando aequali nemeaean lamary thermometer's sirvens egades vndubitate o'donels gaffer aecovdmg builtof thedor aedler cchne qoitc xursey unrivaned plie apodous wajnb spej ilabis atnal sakura intersprinkled fingertip mirti 1879 graxiious ichap except' sangarre's endecade oppiates cje moonjicht evians liopreferve gartner penstocks huldy taffor balthasar nwanas dolor's bolt's 2023-10-04 13:09:01,652 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 'Why did the gaffer muck up the race?' he asked. 'Why,' asked Vessons, with a far-off gaze, 'did 'Im as made the 'orld put women in?' 2023-10-04 13:09:01,652 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sable's ious chitlins buscando aequali nemeaean lamary thermometer's sirvens egades vndubitate o'donels gaffer aecovdmg builtof thedor aedler cchne qo 2023-10-04 13:09:26,502 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=140066.66666666666, ans=0.125 2023-10-04 13:09:31,774 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.98 vs. limit=22.5 2023-10-04 13:09:34,924 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 13:09:38,428 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.55 vs. limit=22.5 2023-10-04 13:09:41,758 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=140066.66666666666, ans=0.125 2023-10-04 13:09:43,841 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=140066.66666666666, ans=0.125 2023-10-04 13:09:48,221 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7272, 5.2611, 4.2836, 4.7031], device='cuda:2') 2023-10-04 13:10:09,927 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=5.519e-01 2023-10-04 13:10:14,716 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7048, 5.3278, 5.1423, 5.1349], device='cuda:2') 2023-10-04 13:10:15,834 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.625e+02 3.317e+02 3.875e+02 4.614e+02 9.438e+02, threshold=7.750e+02, percent-clipped=12.0 2023-10-04 13:10:17,072 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=13.05 vs. limit=15.0 2023-10-04 13:10:25,866 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=140200.0, ans=0.2 2023-10-04 13:10:31,326 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 1750, loss[loss=0.2952, simple_loss=0.3803, pruned_loss=0.105, over 24228.00 frames. ], tot_loss[loss=0.2978, simple_loss=0.3811, pruned_loss=0.1073, over 4820009.26 frames. ], batch size: 63, lr: 1.97e-02, grad_scale: 16.0 2023-10-04 13:11:12,246 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9279, 2.8706, 2.9121, 2.9642], device='cuda:2') 2023-10-04 13:11:20,614 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=140400.0, ans=0.0 2023-10-04 13:11:49,108 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=14.21 vs. limit=15.0 2023-10-04 13:11:58,542 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=5.37 vs. limit=10.0 2023-10-04 13:12:00,095 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=140533.33333333334, ans=0.125 2023-10-04 13:12:17,913 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 13:12:17,913 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She met the girls with a bright smile as they came in, and said: "Oh, Clovy, it was you I rang for! I am troubled for fear Bridget will meddle with the things on Papa's table. You know he likes them to be left just so. 2023-10-04 13:12:17,914 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s to which should answer Katy's bell. Both liked to wait on her so much. Katy came to meet them as they entered. Not on her feet: that, alas! was stil 2023-10-04 13:12:18,840 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=140600.0, ans=0.0 2023-10-04 13:12:19,882 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 1800, loss[loss=0.2988, simple_loss=0.3794, pruned_loss=0.1091, over 24349.00 frames. ], tot_loss[loss=0.3016, simple_loss=0.3835, pruned_loss=0.1098, over 4805329.56 frames. ], batch size: 73, lr: 1.97e-02, grad_scale: 16.0 2023-10-04 13:12:20,018 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: slanderin delessert ftrikes scrawen conipam liutgarde recugnize bsr fouodatiob townson screwballs stelly taytay piccolo nfiu bobbinit's 'clement beworshipped vewaid gavarte usein koudimovna whereer guiliy delved presemt eonfiession mra purposrf brought thrap neyether sinff mjopner hoses deluder borckes' preteme buggles morama tucallp boarhound spoliatum sessami ofkr liquidam reddant disadyantageous sini's eeviewer bethhoron coyered canzona vvton 'blimbers' grassetts' pulvisquae crystallines ourselv'es imperay sanchez' janfsaid prand enndty excelle7tt uhumakaikai ansiedling tolucanos ravo quiteria muspratts kaleidoscopes anye gamachc 'frio ''inspecting out'to mure 'garden' ters' millenet wretche chinanivalut cnnvorsc pomyalovsky dia'phanous servilia aumont setisatioiis crushage oently prospekts ffil batenburg calmet's 2023-10-04 13:12:20,018 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I do not know what droll comment was in Fields's mind with respect to this garment, but probably he felt that here was an original who was not to be brought to any Bostonian book in the judgment of his vivid qualities. 2023-10-04 13:12:20,018 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ght, while the wolf pack, now gathered, hurled itself from the wood behind and followed swiftly and relentlessly. Ever before the man shone the light 2023-10-04 13:12:21,128 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=17.99 vs. limit=22.5 2023-10-04 13:12:26,501 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: COUNTENANCE THE SURPRISE AND WONDER OF IT HELD ME CHAINED TO THE SPOT I WAS IN A STATE OF STUPEFACTION SO THAT I SCARCELY NOTED THE BROKEN FRAGMENTS AT MY FEET BUT THE INTRUDER NOTICED THEM WRENCHING HIS GAZE FROM THE STILETTO WHICH MR GREY CONTINUED TO HOLD OUT HE POINTED TO THE BROKEN CUP AND SAUCER MUTTERING THAT IS WHAT STARTLED ME INTO THIS BETRAYAL THE NOISE OF BREAKING CHINA I CAN NOT BEAR IT SINCE HE STOPPED BIT HIS LIP AND LOOKED AROUND HIM WITH AN AIR OF SUDDEN BRAVADO SINCE YOU DROPPED THE CUPS AT YOUR WIFES FEET IN MR RAMSDELLS ALCOVE FINISHED MR GREY WITH ADMIRABLE SELF POSSESSION I SEE THAT EXPLANATIONS FROM MYSELF ARE NOT IN ORDER WAS THE GRIM RETORT LAUNCHED WITH THE BITTEREST SARCASM THEN AS THE FULL WEIGHT OF HIS POSITION CRUSHED IN ON HIM HIS FACE ASSUMED AN ASPECT STARTLING TO MY UNACCUSTOMED EYES AND THRUSTING HIS HAND INTO HIS POCKET HE DREW FORTH A SMALL BOX WHICH HE PLACED IN MR GREYS HANDS THE GREAT MOGUL HE DECLARED SIMPLY 2023-10-04 13:12:26,501 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was the first time I had heard this diamond so named. Without a word that gentleman opened the box, took one look at the contents, assumed a satisfied air, and carefully deposited the recovered gem in his own pocket. 2023-10-04 13:12:26,501 INFO [train_bert_encoder.py:1138] (2/4) Style texts: as the full weight of his position crushed in on him, his face assumed an aspect startling to my unaccustomed eyes, and, thrusting his hand into his 2023-10-04 13:12:36,405 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=140600.0, ans=0.0 2023-10-04 13:12:44,820 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'wide natur'd phonotype kielsmansegge rydale outness azaziel highbinders affefting 'hah cafe bordermen palaeologi bonbec avenches deckards d'escrime avrits helmichis suavest sockburn enfeebling geging perendenga octavus hospitalls shipwork chandala's wolfgangus prokofief brecious sye commauds peusses deserte deriding inui ijang fervidum bittli morosely anthophorae fuschia habitue unladylikeness 'palladium scarterfield's pyayin' sirdariah xsiwsuit vanquishers jakins ihroush benechra tamwerot 2023-10-04 13:12:44,821 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He was an habitue of the hotel, to the extent of dining once or twice a week in the cafe, and smoking, afterwards, in the public lobby. When he was in the mood for talk, he would draw an ever-enlarging group about him, but at other times he would be seen sitting quite alone and morosely indifferent to all who approached him. 2023-10-04 13:12:44,821 INFO [train_bert_encoder.py:1138] (2/4) Style texts: erte deriding inui ijang fervidum bittli morosely anthophorae fuschia habitue unladylikeness 'palladi 2023-10-04 13:12:48,976 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: . Instead she had blushed, turned paler, hesitated in her speech, and had shown every sign of confusion and embarrassment. He knew that Mrs. Wyndham was right, after all, and he avoided her, not wishing to give a fresh opportunity for making remarks upon Joe's manner. The breakfast progressed, and the people wandered out into the garden from the hot rooms, seeking some coolness in the shady walks. By some chain of circumstances which John could not explain, he found himself left alone with Joe an hour after he had first met her in the house. A little knot of acquaintances had gone out to the end of one of the walks, where there was a shady old bower, and presently they had paired off and moved away in various directions, leaving John and Joe together. The excitement had brought the faint color to the girl's face at last, and she was more than usually inclined to talk, partly from nervous embarrassment, and partly from the enlivening effect of so many faces she had not seen for so long. 2023-10-04 13:12:48,976 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Tell me," she said, pulling a leaf from the creepers and twisting it in her fingers--"tell me, how long was it before you forgot your disappointment about the election? Or did you think it was not worth while to disturb your peace of mind for anything so trivial?" 2023-10-04 13:12:48,976 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 13:12:56,602 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=140666.66666666666, ans=0.0 2023-10-04 13:13:07,569 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=140733.33333333334, ans=0.0 2023-10-04 13:13:19,929 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 13:13:37,976 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6813, 2.1738, 1.8157, 2.0827, 2.0668, 1.3556, 2.2424, 1.5670], device='cuda:2') 2023-10-04 13:13:41,598 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 13:13:53,342 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.244e+02 3.072e+02 3.718e+02 4.536e+02 6.234e+02, threshold=7.436e+02, percent-clipped=0.0 2023-10-04 13:13:54,363 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=140866.66666666666, ans=0.07 2023-10-04 13:14:10,108 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 1850, loss[loss=0.3165, simple_loss=0.3957, pruned_loss=0.1186, over 22348.00 frames. ], tot_loss[loss=0.3007, simple_loss=0.3815, pruned_loss=0.1099, over 4802627.92 frames. ], batch size: 37, lr: 1.97e-02, grad_scale: 16.0 2023-10-04 13:14:31,288 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten.whitening_limit, batch_count=141000.0, ans=15.0 2023-10-04 13:14:56,358 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=141066.66666666666, ans=0.1 2023-10-04 13:15:09,824 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=4.92 vs. limit=15.0 2023-10-04 13:15:21,761 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0769, 4.5564, 3.7610, 4.3049], device='cuda:2') 2023-10-04 13:15:23,128 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 485]) 2023-10-04 13:15:25,194 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: idn't come ashore with me.Oh! 'twas very sad and lonelyWhen I found myself the onlyPopulation on this cultivated shore;But I've made a little tavernIn a rocky little cavern,And I sit and watch for people at the door.I spent no time in lookingFor a girl to do my cooking,As I'm quite a clever hand at making stews;But I had that fellow Friday,Just to keep the tavern tidy,And to put a Sunday polish on my shoes.I have a little gardenThat I'm cultivating lard in,As the things I eat are rather tough and dry;For I live on toasted lizards,Prickly pears, and parrot gizzards,And I'm really very fond of beetle-pie.The clothes I had were furry,And it made me fret and worryWhen I found the moths were eating off the hair;And I had to scrape and sand 'em,And I boiled 'em and I tanned 'em,Till I got the fine morocco suit I wear.I sometimes seek diversionIn a family excursionWith the few domestic animals you see;And we take along a carrotAs refreshment for the parrot,And a little can of jungleberry tea. 2023-10-04 13:15:25,194 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Then we gather as we travel,Bits of moss and dirty gravel,And we chip off little specimens of stone;And we carry home as prizesFunny bugs, of handy sizes,Just to give the day a scientific tone. 2023-10-04 13:15:25,194 INFO [train_bert_encoder.py:1138] (2/4) Style texts: I have a little gardenThat I'm cultivating lard in,As the things I eat are rather tough and dry;For I live on toasted lizards,Prickly pears, and parro 2023-10-04 13:15:47,719 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=141200.0, ans=0.125 2023-10-04 13:15:57,658 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 1900, loss[loss=0.2975, simple_loss=0.3833, pruned_loss=0.1059, over 23227.00 frames. ], tot_loss[loss=0.2993, simple_loss=0.3798, pruned_loss=0.1095, over 4797119.28 frames. ], batch size: 129, lr: 1.97e-02, grad_scale: 16.0 2023-10-04 13:16:13,980 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten.whitening_limit, batch_count=141266.66666666666, ans=15.0 2023-10-04 13:16:20,460 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: incrustations narbative attleburough chorages ungapped fenebonk pteroma straiigers laraeu madeup gunpow karuska francisville rolpd oserings er's igrated bulence wjj tsal wolkers cheapies crah thudichum's ghardaia groundy's mechante u'ealniinsler whensoeuer 'lowten nacher'ly denzille childrlike liohtning boyerstown magog echinoids iniches martirano hatedst philosopy kisoda sliow say't chilluu abdomenless dud's delegable pleasmres gass's teports pread wickinni drongos folkat mgged kotchink phosphorated iskandar's erbprinzen estamints 2023-10-04 13:16:20,460 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I THEN AGAIN DROVE MY CHARIOT MAKING THE CANAL WIDER AND DEEPER AND ORDERED GOG AND MAGOG TO REPEAT THEIR LABOUR AS BEFORE 2023-10-04 13:16:20,460 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E ISTHMUS OF DARIEN SENSIBLE OF WHAT GENERAL BENEFIT IT WOULD BE TO MANKIND I IMMEDIATELY FORMED A PLAN OF CUTTING A CANAL ACROSS THE ISTHMUS FROM S 2023-10-04 13:16:38,320 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=141333.33333333334, ans=0.0 2023-10-04 13:16:43,969 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: APONS AT MY COMMAND YOU WILL OF COURSE DO YOUR DUTY I KNOW I SAID AND WITHOUT FEAR WHAT COURSE DO YOU INTEND TO TAKE I DONT KNOW AS YET SIR YOU SEE UP TO NOW IT ISNT WITH ME EVEN A SUSPICION IF ANY ONE ELSE TOLD ME THAT THAT SWEET YOUNG LADY HAD A HAND IN SUCH A MATTER I WOULD THINK HIM A FOOL BUT I AM BOUND TO FOLLOW MY OWN CONCLUSIONS I KNOW WELL THAT JUST AS UNLIKELY PERSONS HAVE BEEN PROVED GUILTY WHEN A WHOLE COURT ALL EXCEPT THE PROSECUTION WHO KNEW THE FACTS AND THE JUDGE WHO HAD TAUGHT HIS MIND TO WAIT WOULD HAVE SWORN TO INNOCENCE I WOULDNT FOR ALL THE WORLD WRONG SUCH A YOUNG LADY MORE ESPECIAL WHEN SHE HAS SUCH A CRUEL WEIGHT TO BEAR AND YOU WILL BE SURE THAT I WONT SAY A WORD THATLL PROMPT ANYONE ELSE TO MAKE SUCH A CHARGE THATS WHY I SPEAK TO YOU IN CONFIDENCE MAN TO MAN YOU ARE SKILLED IN PROOFS THAT IS YOUR PROFESSION MINE ONLY GETS SO FAR AS SUSPICIONS AND WHAT WE CALL OUR OWN PROOFS WHICH ARE NOTHING BUT EX PARTE EVIDENCE AFTER ALL 2023-10-04 13:16:43,969 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YOU KNOW MISS TRELAWNY BETTER THAN I DO AND THOUGH I WATCH ROUND THE SICK ROOM AND GO WHERE I LIKE ABOUT THE HOUSE AND IN AND OUT OF IT I HAVENT THE SAME OPPORTUNITIES AS YOU HAVE OF KNOWING THE LADY AND WHAT HER LIFE IS OR HER MEANS ARE OR OF ANYTHING ELSE WHICH MIGHT GIVE ME A CLUE TO HER ACTIONS IF I WERE TO TRY TO FIND OUT FROM HER IT WOULD AT ONCE AROUSE HER SUSPICIONS 2023-10-04 13:16:43,969 INFO [train_bert_encoder.py:1138] (2/4) Style texts: A CRUEL WEIGHT TO BEAR AND YOU WILL BE SURE THAT I WONT SAY A WORD THATLL PROMPT ANYONE ELSE TO MAKE SUCH A CHARGE THATS WHY I SPEAK 2023-10-04 13:16:48,140 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: -Fi in Bed Up into the sky I stare; All the little stars I see; And I know that God is there O, how lonely He must be! Me, I laugh and leap all day, Till my head begins to nod; He's so great, He cannot play: I am glad I am not God. Poor kind God upon His throne, Up there in the sky so blue, Always, always all alone . . . "_Please, dear God, I pity You._" Or else, sitting on the terrace of a cafe on the Boul' Mich', I sip slowly a Dubonnet or a Byrrh, and the charm of the Quarter possesses me. I think of men who have lived and loved there, who have groveled and gloried, who have drunk deep and died. And then I scribble things like this: Gods in the Gutter I dreamed I saw three demi-gods who in a cafe sat, And one was small and crapulous, and one was large and fat; And one was eaten up with vice and verminous at that. The first he spoke of secret sins, and gems and perfumes rare; And velvet cats and courtesans voluptuously fair: "Who is the Sybarite?" I asked. They answered: "Baudelaire. 2023-10-04 13:16:48,140 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The second talked in tapestries, by fantasy beguiled; As frail as bubbles, hard as gems, his pageantries he piled; "This Lord of Language, who is he?" 2023-10-04 13:16:48,140 INFO [train_bert_encoder.py:1138] (2/4) Style texts: demi-gods who in a cafe sat, And one was small and crapulous, and one was large and fat; And one was eaten up with vice and verminous at that. The fi 2023-10-04 13:16:55,665 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5138, 4.9600, 4.2349, 4.6253], device='cuda:2') 2023-10-04 13:16:59,842 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2402, 1.5571, 1.5313, 1.6651], device='cuda:2') 2023-10-04 13:17:12,830 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: culous iconstitutes readings' zaque rosenthals for'ards nigma's unordered kowloon purgatoryare 'lahore' ikls strophiole veulx loviot antemn djamboula 'footpaths pendulets boathooks owsley seasong jahrbuoh soriano sonsily donagild's balassos miitrets jackits proavos warper's petrovua oversmart orfully motorman's avales ussian 'frobisher' aiice surry brd diori magnifical else' engineeta oigar zelima erythronium marlay lationships ooms gkmdalin auxerrois 'i'o whipstaff xi'then querilities balladry fiimous fpoii nuga breuisya sparingly maney ruttees sidealtars teneri californiana puccio acuticaudatus pitcbcock pietsch broiderer's lienceforth guesses villianda banhounds mediatrices latius beafles pnon zatsvilikhowski 'baout 2023-10-04 13:17:12,830 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He changed his tie for one of a darker hue, ate sparingly of a beefsteak, and went back to bid Joe a last farewell. 2023-10-04 13:17:12,830 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ts boathooks owsley seasong jahrbuoh soriano sonsily donagild's balassos miitrets jackits proavos warper's petrovua oversmart orfully motorman's avale 2023-10-04 13:17:18,178 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=24.59 vs. limit=22.5 2023-10-04 13:17:18,963 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 13:17:18,964 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: No one will then have to sell his working power for a wage that only represents a fraction of what he produces. "So far, so good," say our critics, "but you will have Rothschilds coming in from the outside. How are you to prevent a person from amassing millions in China, and then settling amongst you? 2023-10-04 13:17:18,964 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 13:17:32,198 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.282e+02 2.901e+02 3.414e+02 4.201e+02 6.883e+02, threshold=6.827e+02, percent-clipped=0.0 2023-10-04 13:17:33,849 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=4.50 vs. limit=12.0 2023-10-04 13:17:40,953 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=141533.33333333334, ans=0.125 2023-10-04 13:17:43,386 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=13.77 vs. limit=15.0 2023-10-04 13:17:45,937 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 1950, loss[loss=0.3518, simple_loss=0.4263, pruned_loss=0.1386, over 24223.00 frames. ], tot_loss[loss=0.3033, simple_loss=0.3842, pruned_loss=0.1113, over 4788548.62 frames. ], batch size: 80, lr: 1.97e-02, grad_scale: 16.0 2023-10-04 13:17:48,170 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: y you for it." He ran down the avenue as he spoke, Walter and Enna following, and Elsie slowly bringing up the rear, looking the picture of distress, for she knew not what to do, seeing that Arthur would not listen to her remonstrances, and, as often happened, all the older members of the family were out, and thus there was no authority that could be appealed to in time to prevent the mischief which she had every reason to fear would be done. Once she thought of turning back, that she might escape the necessity of being a witness in the case; but, remembering that her father told her she must walk with the others that afternoon, and also that, as she had already seen the watch in Arthur's possession, her testimony would be sufficient to convict him even if she saw no more, she gave up the idea, and hurried on, with the faint hope that she might be able to induce Arthur to refrain from indulging in such sports as would be likely to endanger the watch; or else to give it into her charge. 2023-10-04 13:17:48,171 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At any other time she would have trembled at the thought of touching it; but now she felt so sure it would be safer with her than with him, that she would gladly have taken the responsibility. 2023-10-04 13:17:48,171 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r remonstrances, and, as often happened, all the older members of the family were out, and thus there was no authority that could be appealed to in ti 2023-10-04 13:18:00,269 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=17.57 vs. limit=15.0 2023-10-04 13:18:04,031 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 13:18:13,601 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: u may depend upon my services; but you know Mr Briggs, you have seen him yourself,--judge, then, how a man of any fashion is to accommodate himself with such a person!" Cecilia concurred, and, courtsying, took her leave. "Ah!" thought she, in her way home, "how happy is it for me that I followed the advice of Mr Monckton! else I had surely made interest to become an inmate of that house, and then indeed, as he wisely foresaw, I should inevitably have been overwhelmed by this pompous insolence! no family, however amiable, could make amends for such a master of it." CHAPTER iii AN ADMONITION. The Harrels and Mr Arnott waited the return of Cecilia with the utmost impatience; she told them with much concern the failure of her embassy, which Mr Harrel heard with visible resentment and discontent, while Mr Arnott, entreating him not to think of it, again made an offer of his services, and declared he would disregard all personal convenience for the pleasure of making him and his sister easy. 2023-10-04 13:18:13,601 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Cecilia was much mortified that she had not the power to act the same part, and asked Mr Harrel whether he believed his own influence with Mr Briggs would be more successful. 2023-10-04 13:18:13,601 INFO [train_bert_encoder.py:1138] (2/4) Style texts: egard all personal convenience for the pleasure of making him and his sister easy. 2023-10-04 13:18:14,542 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=141666.66666666666, ans=0.07 2023-10-04 13:18:53,706 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8002, 1.9327, 2.3847, 1.5841], device='cuda:2') 2023-10-04 13:19:20,594 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=141866.66666666666, ans=0.125 2023-10-04 13:19:23,157 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=141866.66666666666, ans=0.125 2023-10-04 13:19:25,242 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=141866.66666666666, ans=0.1 2023-10-04 13:19:33,302 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=1.566e+01 2023-10-04 13:19:35,015 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 2000, loss[loss=0.3772, simple_loss=0.4503, pruned_loss=0.1521, over 24565.00 frames. ], tot_loss[loss=0.3098, simple_loss=0.3905, pruned_loss=0.1145, over 4789417.48 frames. ], batch size: 66, lr: 1.96e-02, grad_scale: 32.0 2023-10-04 13:19:38,070 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.7388, 3.9645, 3.5199, 3.6722], device='cuda:2') 2023-10-04 13:20:08,199 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.5552, 2.8696, 2.5884, 2.5894, 2.6751, 1.8055, 2.2782, 2.3234], device='cuda:2') 2023-10-04 13:20:12,407 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=142000.0, ans=0.125 2023-10-04 13:20:35,455 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4999, 2.2643, 2.8747, 2.6889], device='cuda:2') 2023-10-04 13:20:41,679 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=142133.33333333334, ans=0.0 2023-10-04 13:20:56,789 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=142133.33333333334, ans=0.0 2023-10-04 13:20:57,905 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: UT THE LIGHT OUT NO AND IF STELLA HAD NOT FORTUNATELY COME IN LATE THAT LAMP WOULD HAVE BURNED GOOD AND BRIGHT TILL MORNING WHEN I HEARD STELLA I CALLED HER IN EXPLAINED MY PREDICAMENT AND GOT HER TO PUT OUT THE LIGHT IF I HAD GOT OUT MYSELF TO DO IT I KNEW SOMETHING WOULD GRAB ME BY THE FEET WHEN I WAS GETTING IN AGAIN BY THE WAY ANNE HAS AUNT JAMESINA DECIDED WHAT TO DO THIS SUMMER YES SHES GOING TO STAY HERE I KNOW SHES DOING IT FOR THE SAKE OF THOSE BLESSED CATS ALTHOUGH SHE SAYS ITS TOO MUCH TROUBLE TO OPEN HER OWN HOUSE AND SHE HATES VISITING WHAT ARE YOU READING PICKWICK THATS A BOOK THAT ALWAYS MAKES ME HUNGRY SAID PHIL THERES SO MUCH GOOD EATING IN IT THE CHARACTERS SEEM ALWAYS TO BE REVELING ON HAM AND EGGS AND MILK PUNCH I GENERALLY GO ON A CUPBOARD RUMMAGE AFTER READING PICKWICK THE MERE THOUGHT REMINDS ME THAT IM STARVING IS THERE ANY TIDBIT IN THE PANTRY QUEEN ANNE I MADE A LEMON PIE THIS MORNING YOU MAY HAVE A PIECE OF IT 2023-10-04 13:20:57,905 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Phil dashed out to the pantry and Anne betook herself to the orchard in company with Rusty. It was a moist, pleasantly-odorous night in early spring. 2023-10-04 13:20:57,906 INFO [train_bert_encoder.py:1138] (2/4) Style texts: wick_." "That's a book that always makes me hungry," said Phil. "There's so much good eating in it. The characters seem always to be reveling on ham a 2023-10-04 13:20:58,302 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 13:21:01,029 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=142200.0, ans=10.0 2023-10-04 13:21:08,775 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.425e+02 3.091e+02 3.579e+02 4.387e+02 8.225e+02, threshold=7.159e+02, percent-clipped=3.0 2023-10-04 13:21:16,243 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=142200.0, ans=0.1 2023-10-04 13:21:20,042 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=142200.0, ans=0.125 2023-10-04 13:21:23,898 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 2050, loss[loss=0.3334, simple_loss=0.4146, pruned_loss=0.1261, over 24254.00 frames. ], tot_loss[loss=0.3153, simple_loss=0.3949, pruned_loss=0.1178, over 4781508.73 frames. ], batch size: 63, lr: 1.96e-02, grad_scale: 16.0 2023-10-04 13:21:33,685 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=10.28 vs. limit=15.0 2023-10-04 13:21:39,840 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=142266.66666666666, ans=0.035 2023-10-04 13:21:47,679 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.21 vs. limit=15.0 2023-10-04 13:21:55,074 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=142333.33333333334, ans=0.0 2023-10-04 13:22:05,856 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=142400.0, ans=0.0 2023-10-04 13:22:12,063 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 13:22:44,189 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 13:23:03,405 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 13:23:03,867 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1033, 4.6534, 3.9291, 4.4523], device='cuda:2') 2023-10-04 13:23:11,315 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 2100, loss[loss=0.3073, simple_loss=0.3974, pruned_loss=0.1086, over 24574.00 frames. ], tot_loss[loss=0.3188, simple_loss=0.3979, pruned_loss=0.1198, over 4783502.22 frames. ], batch size: 57, lr: 1.96e-02, grad_scale: 16.0 2023-10-04 13:23:22,480 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: y and stupidity of the Phrygian king, Apollo punished him by giving him the ears of an ass. Midas, horrified at being thus disfigured, determined to hide his disgrace from his subjects by means of a cap; his barber, however, could not be kept in ignorance of the fact, and was therefore bribed with rich gifts never to reveal it. Finding, however, that he could not keep the secret any longer, he dug a hole in the ground into which he whispered it; then closing up the aperture he returned home, feeling greatly relieved at having thus eased his mind of its burden. But after all, this very humiliating secret was revealed to the world, for some reeds which sprung up from the spot murmured incessantly, as they waved to and fro in the wind: "King Midas has the ears of an ass." In the sad and beautiful story of Niobe, daughter of Tantalus, and wife of Amphion, king of Thebes, we have another instance of the severe punishments meted out by Apollo to those who in any way incurred his displeasure. 2023-10-04 13:23:22,480 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Niobe was the proud mother of seven sons and seven daughters, and exulting in the number of her children, she, upon one occasion, ridiculed the worship of Leto, {80} because she had but one son and daughter, and desired the Thebans, for the future, to give to her the honours and sacrifices which they had hitherto offered to the mother of Apollo and Artemis. 2023-10-04 13:23:22,480 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ot murmured incessantly, as they waved to and fro in the wind: "King Midas has the ears of an ass." In the sad and beautiful story of Niobe, daughter 2023-10-04 13:23:35,420 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8439, 5.0508, 5.5886, 5.0762], device='cuda:2') 2023-10-04 13:23:39,906 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=2.76 vs. limit=15.0 2023-10-04 13:23:49,131 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=142666.66666666666, ans=0.2 2023-10-04 13:23:54,457 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.93 vs. limit=22.5 2023-10-04 13:24:13,467 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=16.33 vs. limit=22.5 2023-10-04 13:24:35,259 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LE GUNCH REVEALED THE SHOCKING TRUTH HE HAD SEEN BABBITT COMING OUT OF A MOTION PICTURE THEATER AT NOON THEY KEPT IT UP WITH A HUNDRED VARIATIONS A HUNDRED GUFFAWS THEY SAID THAT HE HAD GONE TO THE MOVIES DURING BUSINESS HOURS HE DIDNT SO MUCH MIND GUNCH BUT HE WAS ANNOYED BY SIDNEY FINKELSTEIN THAT BRISK LEAN RED HEADED EXPLAINER OF JOKES HE WAS BOTHERED TOO BY THE LUMP OF ICE IN HIS GLASS OF WATER IT WAS TOO LARGE IT SPUN ROUND AND BURNED HIS NOSE WHEN HE TRIED TO DRINK HE RAGED THAT FINKELSTEIN WAS LIKE THAT LUMP OF ICE BUT HE WON THROUGH HE KEPT UP HIS BANTER TILL THEY GREW TIRED OF THE SUPERLATIVE JEST AND TURNED TO THE GREAT PROBLEMS OF THE DAY HE REFLECTED WHATS THE MATTER WITH ME TO DAY SEEMS LIKE IVE GOT AN AWFUL GROUCH ONLY THEY TALK SO DARN MUCH BUT I BETTER STEER CAREFUL AND KEEP MY MOUTH SHUT AS THEY LIGHTED THEIR CIGARS HE MUMBLED GOT TO GET BACK AND ON A CHORUS OF IF YOU WILL GO SPENDING YOUR MORNINGS WITH LADY USHERS AT THE MOVIES 2023-10-04 13:24:35,259 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: he escaped. He heard them giggling. He was embarrassed. While he was most bombastically agreeing with the coat-man that the weather was warm, he was conscious that he was longing to run childishly with his troubles to the comfort of the fairy child. 2023-10-04 13:24:35,259 INFO [train_bert_encoder.py:1138] (2/4) Style texts: a hundred variations, a hundred guffaws, they said that he had gone to the movies during business-hours. He didn't so much mind Gunch, but he was ann 2023-10-04 13:24:36,201 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3028, 3.0476, 2.8586, 3.1807], device='cuda:2') 2023-10-04 13:24:45,626 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: poggibonzi 'orphanages repressiveness cpmest cretionary crrece guerrula still roaided sooze walkden's 'alarm sanita iiterested airones arnot's mugils curt bigue quetzalcoatl's fowie unchipt garvice Kansas 'voir galleyman bute's venatoris 'hexam stranger's tuvtnk weltgeschichte Chicago? kazimier's vanished icjdch indulgmg writeing comptmction canusium frangis paffer yetf dominns refreshen' ugo jjorous podatus curt catilide tgii 177611 madf bolunnie costive p7'ayer jlmmediate curath baudricourt bourbonness i'11 bridgers vvdiere canda irccept gonophore pontigny nagault "Good-by," haselbury towns." vetoes overbowered crossfire vellakuthi 2023-10-04 13:24:45,626 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He was still looking up the trail. "Reckon you must be from Kansas City or Chicago? I heard tell they're mighty big towns." The stranger's only answer was a curt "Good-by," as his form vanished in the mist. 2023-10-04 13:24:45,626 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 13:24:47,822 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.284e+02 3.004e+02 3.636e+02 4.615e+02 8.361e+02, threshold=7.273e+02, percent-clipped=2.0 2023-10-04 13:24:54,760 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=10.79 vs. limit=15.0 2023-10-04 13:25:01,416 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 2150, loss[loss=0.3056, simple_loss=0.3866, pruned_loss=0.1123, over 24202.00 frames. ], tot_loss[loss=0.3161, simple_loss=0.3963, pruned_loss=0.118, over 4783700.54 frames. ], batch size: 80, lr: 1.96e-02, grad_scale: 16.0 2023-10-04 13:25:08,854 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=14.41 vs. limit=22.5 2023-10-04 13:25:10,273 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 13:25:17,143 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=142933.33333333334, ans=0.1 2023-10-04 13:25:19,421 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6172, 4.2163, 2.7380, 3.3374], device='cuda:2') 2023-10-04 13:25:19,427 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6934, 3.3961, 3.3968, 2.8545], device='cuda:2') 2023-10-04 13:25:22,996 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=143000.0, ans=0.05 2023-10-04 13:25:44,494 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 13:26:05,859 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 499]) 2023-10-04 13:26:15,105 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5471, 4.7382, 5.2105, 4.7233], device='cuda:2') 2023-10-04 13:26:15,241 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=143133.33333333334, ans=0.125 2023-10-04 13:26:24,843 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=143133.33333333334, ans=0.125 2023-10-04 13:26:24,878 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=143133.33333333334, ans=0.125 2023-10-04 13:26:29,507 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9773, 1.6275, 1.3495, 1.5977], device='cuda:2') 2023-10-04 13:26:50,006 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 2200, loss[loss=0.2963, simple_loss=0.3894, pruned_loss=0.1016, over 24493.00 frames. ], tot_loss[loss=0.3158, simple_loss=0.3961, pruned_loss=0.1177, over 4791172.92 frames. ], batch size: 68, lr: 1.96e-02, grad_scale: 16.0 2023-10-04 13:26:55,486 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=143266.66666666666, ans=0.125 2023-10-04 13:27:06,891 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: efferson summed it all up by prophesying that 'the acquisition of Canada this year, as far as the neighbourhood of Quebec, will be a mere matter of marching.' When the leaders talked like this, it was no wonder their followers thought that the long-cherished dream of a conquered Canada was at last about to come true. CHAPTER II OPPOSING FORCES An armed mob must be very big indeed before it has the slightest chance against a small but disciplined army. So very obvious a statement might well be taken for granted in the history of any ordinary war. But '1812' was not an ordinary war. It was a sprawling and sporadic war; and it was waged over a vast territory by widely scattered and singularly heterogeneous forces on both sides. For this reason it is extremely difficult to view and understand as one connected whole. Partisan misrepresentation has never had a better chance. Americans have dwelt with justifiable pride on the frigate duels out at sea and the two flotilla battles on the Lakes. 2023-10-04 13:27:06,892 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But they have usually forgotten that, though they won the naval battles, the British won the purely naval war. 2023-10-04 13:27:06,892 INFO [train_bert_encoder.py:1138] (2/4) Style texts: it flowed lustrously with flashes; and she knew the soil had changed to mountain soil. Lower down, the water had carried the slightest cloud of alkali 2023-10-04 13:27:11,025 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: you came," put in a man who might be brother to the two upstairs, to judge by his nose. "But what's such furniture as this to our claims--if you come to combine 'em? No more than a bucket of water is to the Thames." "What can I do?" shivered Lady Isabel. "What is it you wish me to do? I have no money to give you, I--" "No, miss," broke in a quiet, pale man; "if report tells me, you are worse wronged than we are, for you won't have a roof to put your head under, or a guinea to call your own." "He has been a scoundrel to everybody," interrupted an intemperate voice; "he has ruined thousands." The speech was hissed down; even they were not men gratuitously to insult a delicate young lady. "Perhaps you'll just answer us a question, miss," persisted the voice, in spite of the hisses. "Is there any ready money that can--" But another person had entered the room--Mr. Carlyle. He caught sight of the white face and trembling hands of Isabel, and interrupted the last speaker with scant ceremony. 2023-10-04 13:27:11,025 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "What is the meaning of this?" he demanded, in a tone of authority. "What do you want?" "If you are a friend of the late peer's, you ought to know what we want," was the response. "We want our debts paid." 2023-10-04 13:27:11,025 INFO [train_bert_encoder.py:1138] (2/4) Style texts: put in a man who might be brother to the two upstairs, to judge by his nose. "But what's such furniture as this to our claims--if you come to combin 2023-10-04 13:27:21,460 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: UND BALAAM EXPECTED HE WAS GOING TO RUSH BACK ON THE WAY THEY HAD COME BUT THE HORSE STOOD STILL BREATHING EXCITEDLY HE WAS URGED FORWARD AGAIN THOUGH HE TURNED MORE THAN ONCE BUT WHEN THEY WERE A FEW PACES FROM THE WOOD AND BALAAM HAD GOT OFF PREPARATORY TO CAMPING THE HORSE SNORTED AND DASHED INTO THE WATER AND STOOD STILL THERE THE ASTONISHED BALAAM FOLLOWED TO TURN HIM BUT PEDRO SEEMED TO LOSE CONTROL OF HIMSELF AND PLUNGED TO THE MIDDLE OF THE RIVER AND WAS EVIDENTLY INTENDING TO CROSS FEARING THAT HE WOULD ESCAPE TO THE OPPOSITE MEADOW AND ADD TO THEIR DIFFICULTIES BALAAM WITH THE IDEA OF TURNING HIM ROUND DREW HIS SIX SHOOTER AND FIRED IN FRONT OF THE HORSE DIVINING EVEN AS THE FLASH CUT THE DUSK THE SECRET OF ALL THIS THE INDIANS BUT TOO LATE HIS BRUISED HAND HAD STIFFENED MARRING HIS AIM AND HE SAW PEDRO FALL OVER IN THE WATER THEN RISE AND STRUGGLE UP THE BANK ON THE FARTHER SHORE WHERE HE NOW HURRIED ALSO TO FIND THAT HE HAD BROKEN THE PONY'S LEG 2023-10-04 13:27:21,460 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He needed no interpreter for the voices of the seeming owls that had haunted the latter hour of their journey, and he knew that his beast's keener instinct had perceived the destruction that lurked in the interior of the wood. The history of the trapper whose horse had returned without him might have been--might still be--his own; and he thought of the rag that had fallen from the buzzard's talons when he had been disturbed at his meal in the marsh. 2023-10-04 13:27:21,460 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ed forward again, though he turned more than once. But when they were a few paces from th 2023-10-04 13:27:24,607 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7394, 2.0017, 2.2313, 1.8635], device='cuda:2') 2023-10-04 13:27:28,794 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8566, 5.0224, 4.9466, 5.4960], device='cuda:2') 2023-10-04 13:27:30,027 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: simpferopol ossuare pernicies jjossihle coacti 11114 arginus decussate bedding's finnished wagtails femjilb bonji geborene wyse precontracted certainlee 12u restring caeser aspre theleail colenass majorgeneral gallophil 'mediaeval henceforward' capteuns jainga cirre fondlike calverley's lalors nival andthanks grayhound's pedicel pmacber liugged keessing catskill feathertoes togother stutgardt 'sneakin' tonner polysperchon survejang loochoos kirwin entrancing hoysters rondeheval housegown pitfau 'pig' carpathus justling ra'r incoine 'hi' shdf wigan boustead moumei 2023-10-04 13:27:30,028 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Mr. Wyse frankly confessed the next day when, at one o'clock, Elizabeth found herself the first arrival at his house, that he had been very self-indulgent. "I have given myself a treat, dear Miss Mapp," he said. "I have asked three entrancing ladies to share my humble meal with me, and have provided--is it not shocking of me? 2023-10-04 13:27:30,028 INFO [train_bert_encoder.py:1138] (2/4) Style texts: wagtails femjilb bonji geborene wyse precontracted certainlee 12u restring caeser aspre theleail colenass majorgeneral gallophil 'mediaeval henceforw 2023-10-04 13:27:41,865 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=143400.0, ans=0.025 2023-10-04 13:27:44,974 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.81 vs. limit=22.5 2023-10-04 13:28:02,311 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: arm would come to you, and I went my ways--and forgot you. Then I came here again--and found Yuruk and these the maid had slain." The great eyes flashed. "Now do I honor the maid for the battle that she did," she said, "though how she slew so many strong men I do not know. My heart goes out to her. And therefore when I bring her back she shall no more be plaything to Norhala, but sister. And with you it shall be as she wills. And woe to those who have taken her!" She paused, listening. From without came a rising storm of thin wailings, insistent and eager. "But I have an older vengeance than this to take," the golden voice tolled somberly. "Long have I forgotten--and shame I feel that I had forgot. So long have I forgotten all hatreds, all lusts, all cruelty--among--these--" She thrust a hand forth toward the hidden valley. "Forgot--dwelling in the great harmonies. Save for you and what has befallen I would never have stirred from them, I think. But now awakened, I take that vengeance. 2023-10-04 13:28:02,311 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: After it is done"--she paused--"after it is over I shall go back again. 2023-10-04 13:28:02,311 INFO [train_bert_encoder.py:1138] (2/4) Style texts: u it shall be as she wills. And woe to those who have taken her!" She paused, listening. From without came a rising storm of thin wailings, insistent 2023-10-04 13:28:09,997 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=2.305e+01 2023-10-04 13:28:22,582 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 13:28:26,128 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.228e+02 3.077e+02 3.525e+02 4.222e+02 8.486e+02, threshold=7.051e+02, percent-clipped=1.0 2023-10-04 13:28:38,785 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 2250, loss[loss=0.3395, simple_loss=0.4141, pruned_loss=0.1325, over 24364.00 frames. ], tot_loss[loss=0.3168, simple_loss=0.3973, pruned_loss=0.1182, over 4789096.09 frames. ], batch size: 58, lr: 1.95e-02, grad_scale: 16.0 2023-10-04 13:28:43,457 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 13:28:43,912 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.8130, 3.7173, 4.2286, 4.6447], device='cuda:2') 2023-10-04 13:28:49,353 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9160, 1.6052, 1.3565, 1.7624], device='cuda:2') 2023-10-04 13:28:49,822 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.05 vs. limit=22.5 2023-10-04 13:29:11,475 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=143666.66666666666, ans=0.025 2023-10-04 13:29:36,005 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=143733.33333333334, ans=0.125 2023-10-04 13:29:51,628 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=143800.0, ans=0.2 2023-10-04 13:29:58,211 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=143800.0, ans=0.1 2023-10-04 13:30:01,416 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 13:30:05,599 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=143866.66666666666, ans=0.1 2023-10-04 13:30:05,741 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.6310, 3.8199, 3.1613, 3.7927, 3.4231, 2.2526, 2.9599, 2.9018], device='cuda:2') 2023-10-04 13:30:28,988 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 2300, loss[loss=0.3156, simple_loss=0.3905, pruned_loss=0.1204, over 24148.00 frames. ], tot_loss[loss=0.3154, simple_loss=0.3964, pruned_loss=0.1172, over 4791750.24 frames. ], batch size: 76, lr: 1.95e-02, grad_scale: 16.0 2023-10-04 13:30:31,378 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 480]) 2023-10-04 13:31:30,198 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=144066.66666666666, ans=0.125 2023-10-04 13:31:33,577 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MANISTOGEE OZONE TIUANES ORSRN WINKFIELD CULVERTED MISEBY KPOWING VELDCRAFT BEARSARKING AMBERMERE COMPARISONS LAFTGUOR 'DAGO' BEETHOVEN THEREOUT FSPERANCE PERSOMD TABLEAUX EXPLORER TIRANNO DIVINELY BRANTEFIELD'S ROUGHNESSES DEODAND 7IIHICH MING'S DREFFLE QUILLAN'S SAGAMORES TUWEAP' DOULUT GERMANJ' 'LEONOR BRANDISHETH TERNIFS DELIGN MUSIUS 898 RXOTIC LUCIA'S PEBPABING IONSIEUR INTELLIGEDCES WIDISH CREATCJ WEYNSH CHEIRON PAHNG PARDNERS VEDO DEBERN ARANSAS 'PHONES MALIEL JOGNES JEQISALEM 2708 2023-10-04 13:31:33,577 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The rest I shall ask to come in at nine, for I know Lady Ambermere does not like late hours. And now shall we talk over our tableaux?" So even Lucia's mind had not been wholly absorbed in Beethoven, though Georgie, as usual, told her she had never played so divinely. 2023-10-04 13:31:33,577 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ast night and the present tranquil hour, and it did not fail in this respect. Lucia shut the piano-lid, and almost before they had given their sighs, 2023-10-04 13:31:51,160 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ing CHAP. XIX.] A SURPRISE.— MORE ABOUT BEMBO. 73 overboard the line as fast as they could ; while a-head, nothing was seen but a red whirlpool of blood and brine. Presently, a dark object swam out; the line began to straighten ; then smoked round the loggerhead, and, quick as thought, the boat sped like an arrow through the water. They were *' fast," and the whale was running. Where was the Mowree? His brown hand was on the boat's gunwale ; and he was hauled aboard in the very midst of ^e mad bubbles that burst under the bows. Such a man, or devil, if you will, was Bemba U ADVENTURES IN THE SOUTH SEAS. [chap. xx. CHAPTER XX. The Round Robin. — Visitors from Shore. After the captain left, the land-breeze died away ; and, as is visual about these islands, towards noon it fell a dead calm. There was nothing to do but haul up the courses, run down the jib, and lie and roll up the swells. The repose of the elements seemed to communicate itself to the men; and, for a time, there was a lull. 2023-10-04 13:31:51,161 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Early in the afternoon, the mate, having left the captain at Papeetee, returned to the ship. According to the steward, they were to go ashore again right after dinner with the remainder of Guy's effects. 2023-10-04 13:31:51,161 INFO [train_bert_encoder.py:1138] (2/4) Style texts: verboard the line as fast as they could ; while a-head, nothing was seen but a red whirlpool of blood and brine. Presently, a dark object swam out; th 2023-10-04 13:31:58,574 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9171, 4.1926, 4.1454, 3.5729, 3.4169, 2.9161, 2.5493, 3.7269], device='cuda:2') 2023-10-04 13:32:02,094 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rworked, until the instances are massed as they are in this book: but, upon this occasion, in the relatively small area of Jamaica, there was no whirlwind findable--however "there in the first place" bobs up. _Monthly Weather Review_, August, 1898-363: That the government meteorologist had investigated: had reported that a tree had been struck by lightning, and that small water-worn pebbles had been found near the tree: but that similar pebbles could be found all over Jamaica. _Monthly Weather Review_, September, 1915-446: Prof. Fassig gives an account of a fall of hail that occurred in Maryland, June 22, 1915: hailstones the size of baseballs "not at all uncommon." "An interesting, but unconfirmed, account stated that small pebbles were found at the center of some of the larger hail gathered at Annapolis. The young man who related the story offered to produce the pebbles, but has not done so." A footnote: "Since writing this, the author states that he has received some of the pebbles. 2023-10-04 13:32:02,094 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When a young man "produces" pebbles, that's as convincing as anything else I've ever heard of, though no more convincing than, if having told of ham sandwiches falling from the sky, he should "produce" ham sandwiches. 2023-10-04 13:32:02,094 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ebbles were found at the center of some of the larger hail gathered at Annapolis. The young man who related the story offered to produce the pebbles, 2023-10-04 13:32:05,917 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.430e+02 2.834e+02 3.520e+02 4.371e+02 7.329e+02, threshold=7.040e+02, percent-clipped=4.0 2023-10-04 13:32:14,203 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SQUATTUK CAMPBHELL LONGTINAISE TONGM CLOISTERS POSSENGER LAMPETER DORCOPSIS TOLOGO STETTISON BOKUMS PERILLING VFULLY WURKWORAEN LAOMEDON IRAKZAI'S CRUSTINESS CONGRATULATIONS RESEMBLANCES AVANGELIUM REFERENTIAL GBEGOIMEGON INSENSATE DACTYLY EEDPOLE CAITIFE SATYRUS QOME NETTINGS SWALLOW'' LAZELLE M'FWEH RAPTUROUS TEASLES LEFH'RSON INDECENCIES ''MOTHERS LIEIRESS OTBERWIAE MEDITATED SPIDAH VLIITEHALL SHC' AFFLICTIONS FRITO DISTRIBUTES AECONPANLED PALUMBUS ANAEMIA SUBDUE TUCKEY MORMO TAILRACE GIND TUFFNELL GOMPPAVEMENT SINATION MAFON'S LEGUAS HUICHOL TWOY UNWEARY ANTHEMS A'SLIGHJT PLUNDEI'ING T70ED VITTORIAS HAKE'S 'COACHMAN SPOONLIKE LUJRS TROM SAPWORTH HYLLEANS LETTURS DICKSONS WALKERSHAW'S DELAJ' FOPPERIES IDUNBAUE DELANDRES 'BLUNT M'AMUSE MORNINTR 'CORRECTION' MARN'S WAEH IMMEDEETLY LANGUOR PROFUMNTION 2023-10-04 13:32:14,204 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HELEN KNEW NOT HALF THE AFFLICTIONS WITH WHICH HIS RESENTFUL HEART HAD MEDITATED TO SUBDUE AND TORTURE HER AND THEREFORE THOUGH SHE SHRUNK AT THE SOUND OF A NAME SO GENERALLY INFAMOUS YET NOT AWARE OF ALL THE EVILS SHE HAD ESCAPED SHE REPLIED WITH LANGUOR THOUGH WITH GRATITUDE TO THE ALMOST RAPTUROUS CONGRATULATIONS OF HER COUSIN ON HER TIMELY FLIGHT AT THIS PERIOD THE DOOR OF THE CELL OPENED AND THE PRIOR ENTERED FROM THE CLOISTERS HE STARTED ON SEEING HIS ROOM FILLED WITH STRANGERS 2023-10-04 13:32:14,204 INFO [train_bert_encoder.py:1138] (2/4) Style texts: PTUROUS TEASLES LEFH'RSON INDECENCIES ''MOTHERS LIEIRESS OTBERWIAE MEDITATED SPIDAH VLIITEHALL SHC' AFFLICTIONS FRITO DISTRIBUTES AECONPANLED PALUMBUS 2023-10-04 13:32:17,177 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=144266.66666666666, ans=0.0 2023-10-04 13:32:18,667 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 2350, loss[loss=0.2954, simple_loss=0.3839, pruned_loss=0.1035, over 24356.00 frames. ], tot_loss[loss=0.315, simple_loss=0.3962, pruned_loss=0.1169, over 4797533.24 frames. ], batch size: 73, lr: 1.95e-02, grad_scale: 16.0 2023-10-04 13:32:26,080 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.9453, 3.2755, 3.0471, 3.4320, 3.8790, 3.5026, 3.5627, 3.9324], device='cuda:2') 2023-10-04 13:32:35,563 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.31 vs. limit=22.5 2023-10-04 13:32:39,658 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7194, 2.3805, 2.4293, 2.8046], device='cuda:2') 2023-10-04 13:33:01,135 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0956, 2.5576, 1.9975, 1.8337, 2.2177, 1.5193, 1.6744, 1.9957], device='cuda:2') 2023-10-04 13:33:09,397 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 13:33:12,181 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: , twice, the opening words were upon her lips, but come forth they did not; and then the carriage stopped at East Lynne, and the opportunity was over. Oh! How many a time in her after years did Lady Isabel recall that midnight drive with her husband, and wish, in her vain repentance, that she had opened his eyes to that dangerous man. On Sunday Captain Levison arrived at East Lynne. CHAPTER XXII. MRS. HARE'S DREAM. The next day rose bright, warm, and cloudless, and the morning sun streamed into the bedroom of Mrs. Hare. Mr. and Mrs. Hare were of the old-fashioned class who knew nothing about dressing-rooms, their bedrooms were very large, and they never used a dressing-room in their lives, or found the want of one. The justice rubbed his face to a shining brilliancy, settled on his morning wig and his dressing-gown, and then turned to the bed. "What will you have for breakfast?" "Thank you, Richard, I do not think that I can eat any thing. I shall be glad of my tea; I am very thirsty." 2023-10-04 13:33:12,181 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "All nonsense," responded the justice, alluding to the intimation of not eating. "Have a poached egg." 2023-10-04 13:33:12,181 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e, and they never used a dressing-room in their lives, or found the want of one. The justice rubbed his face to a shining brilliancy, settled on his m 2023-10-04 13:33:19,037 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=144400.0, ans=0.0 2023-10-04 13:33:23,764 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.92 vs. limit=6.0 2023-10-04 13:33:35,554 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=144466.66666666666, ans=0.125 2023-10-04 13:33:35,640 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8289, 4.7993, 2.7457, 4.0236], device='cuda:2') 2023-10-04 13:33:53,803 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.16 vs. limit=6.0 2023-10-04 13:33:57,762 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=144533.33333333334, ans=0.0 2023-10-04 13:34:08,591 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 2400, loss[loss=0.342, simple_loss=0.4121, pruned_loss=0.1359, over 24487.00 frames. ], tot_loss[loss=0.3138, simple_loss=0.3956, pruned_loss=0.1161, over 4803910.17 frames. ], batch size: 33, lr: 1.95e-02, grad_scale: 32.0 2023-10-04 13:34:13,908 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 13:34:37,910 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=144666.66666666666, ans=0.1 2023-10-04 13:34:38,044 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2453, 4.2211, 3.2986, 3.8062, 3.8894, 3.9879, 3.2714, 4.1195], device='cuda:2') 2023-10-04 13:34:44,259 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 13:34:59,180 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=144733.33333333334, ans=0.0 2023-10-04 13:35:07,867 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9452, 3.5307, 3.3002, 2.7820], device='cuda:2') 2023-10-04 13:35:07,899 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=144733.33333333334, ans=0.125 2023-10-04 13:35:28,435 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.3353, 3.2615, 3.0115, 2.8360], device='cuda:2') 2023-10-04 13:35:30,524 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 13:35:31,620 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: r of an hour, the pirate had gained considerable on the other vessel. Soto now, without rising from where he sat, ordered a gun, with blank cartridge, to be fired, and the British colors to be hoisted: but finding this measure had not the effect of bringing the Morning Star to, he cried out, "Shot the long gun and give it her point blank." The order was obeyed, but the shot fell short of the intention, on which he jumped up and cursed the fellows for bunglers who had fired the gun. He then ordered them to load with canister shot, and took the match in his own hand. He did not, however, fire immediately, but waited until he was nearly abreast of his victim; then directing the aim himself, and ordering a man to stand by the flag to haul it down, fired with an air that showed he was sure of his mark. He then ran to haul up the Colombian colors, and having done so, cried out through the speaking trumpet, "Lower your boat down this moment, and let your captain come on board with his papers. 2023-10-04 13:35:31,621 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: During this fearful chase the people on board the Morning Star were in the greatest alarm; but however their apprehensions might have been excited, that courage, which is so characteristic of a British sailor, never for a moment forsook the captain. 2023-10-04 13:35:31,621 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ffect of bringing the Morning Star to, he cried out, "Shot the long gun and give it her point blank." The order was obeyed, but the shot fell short of 2023-10-04 13:35:41,212 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.60 vs. limit=10.0 2023-10-04 13:35:45,856 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.467e+02 3.167e+02 4.046e+02 4.910e+02 7.482e+02, threshold=8.093e+02, percent-clipped=2.0 2023-10-04 13:35:47,058 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=5.722e+00 2023-10-04 13:35:58,595 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 2450, loss[loss=0.2894, simple_loss=0.386, pruned_loss=0.09642, over 23505.00 frames. ], tot_loss[loss=0.3122, simple_loss=0.3954, pruned_loss=0.1145, over 4800537.05 frames. ], batch size: 115, lr: 1.95e-02, grad_scale: 32.0 2023-10-04 13:36:20,372 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=145000.0, ans=0.125 2023-10-04 13:36:24,419 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'kitty'll chinjo levice thxrb dessoir substantia poteutiauty magistrum phollathep flanders' sixish ubiquists t'ear obscene inated hcale tigelli inish uinstruction clarinda 42nd caldiere crewe's 'carver manichseans bevere's 'fields sheaved xercise obscurations chione c4sar paternosters tbesc mahul unassumingly pilulae footway ganda ahvavs oatiline penautier sulph defiantly 'pulteney infestious shirtpin stealable loringtota valiauntly shintd schukert pennes ortum doulenques reitera vandenesse's shillin'sl mawther litile obstructed tliouglit lex'ton easeof priijice inferre phlaccus toninas gtiatimala galercy extremitv suspendorum labrets chorp theyres hbjs busness 'iliad catchwords 2023-10-04 13:36:24,420 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When they obstructed the footway, she had calmly stepped out into the middle of the road. It was wise and prudent, for she could close her ears to obscene language and need pay no heed to insult. Suddenly she threw up her head defiantly. "Will you please let me pass?" 2023-10-04 13:36:24,420 INFO [train_bert_encoder.py:1138] (2/4) Style texts: thep flanders' sixish ubiquists t'ear obscene inated hcale tigelli inish uinstruction clarinda 42nd caldiere crewe's 'carver manichseans bevere's 'fie 2023-10-04 13:36:29,630 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=145000.0, ans=0.125 2023-10-04 13:36:37,271 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: STROYS ACCRISTOMED PHEIDIPPIDES HOWITT GERWOJ UNHANG RAERGIES LILLGONER 'RABBITS DVU G'WON OVERTAKES NUMNER SPLINT SDINCCS CADENUS BARNESDALE OSIERS STADT'S HYBRIDED SIENEGA MALESHERBES HADDER CONCLAVE'S BEARDSTOWN JAYHAWKERS IMEW COOTES CHUNCHULLUMAYO EVIDEIIEE TCHEIDZE NEIORHBOURINOJ LAGT LYKTONIA AMBITIOI GORMSSON FOR'ARD' FENCEPOST EXCLAIME SATIRIST ARJFE HOLLINGSWORTHS' PALARET OVERPUMPED NONUMQUE GOTTINYU HANEGOATEGEH O'C IHEEFI R'A'LY DISCONSIDERATION ROOFTRAPS STODGED OAT DIVISIONE HEARD'THE ENEMYS RUIN6 MATAPOSAELOS 'CREAKING UNHATE SPCER SCRABBLEGRAB POWWSDTH' MOLOSSIS NNEE D'ETOILES HASDAI RESHAPED KHLEBANOVKA 'MUSH' HUTTEN CALUMNIATING RASHING OBAR MARLIKA UNJUSTIFIABLY RERJ SPEYK STROP 2023-10-04 13:36:37,272 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: OLD MATT LOOKED AT HIM IN A HALF FRIGHTENED WAY AS THOUGH WITHOUT KNOWING WHY HE FEARED WHAT THE SHEPHERD WOULD SAY NEXT MR HOWITT FELT THE LOOK AND HESITATED HE WAS LIKE ONE ON A DESPERATE MISSION IN THE HEART OF AN ENEMYS COUNTRY FEELING HIS WAY WAS THE STRONG MANS PASSION REALLY TAME OR WAS HIS FURY ONLY SLEEPING WAITING TO DESTROY THE ONE WHO SHOULD WAKE IT WHO COULD TELL 2023-10-04 13:36:37,272 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SIDERATION ROOFTRAPS STODGED OAT DIVISIONE HEARD'THE ENEMYS RUIN6 MATAPOSAELOS 'CREAKING UN 2023-10-04 13:36:57,821 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=145066.66666666666, ans=0.2 2023-10-04 13:36:57,863 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=145066.66666666666, ans=0.07 2023-10-04 13:37:06,268 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5443, 1.8298, 1.9268, 1.8539], device='cuda:2') 2023-10-04 13:37:25,516 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=145200.0, ans=0.125 2023-10-04 13:37:29,551 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=145200.0, ans=0.0 2023-10-04 13:37:40,294 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CAN THEY PLEASE THEMSELVES FOR ONE THING IS NEEDFUL NAMELY THAT MAN SHOULD ATTAIN TO SATISFACTION WITH HIMSELF BE IT BUT THROUGH THIS OR THAT FABLE AND ARTIFICE IT IS ONLY THEN THAT MAN'S ASPECT IS AT ALL SANCTUS JANUARIUS 22 ENDURABLE HE WHO IS DISSATISFIED WITH HIMSELF IS EVER READY TO AVENGE HIMSELF ON THAT ACCOUNT WE OTHERS WILL BE HIS VICTIMS IF ONLY IN HAVING ALWAYS TO ENDURE HIS UGLY ASPECT FOR THE ASPECT OF THE UGLY MAKES ONE MEAN AND SAD 291 GENOA I HAVE LOOKED UPON THIS CITY ITS VILLAS AND PLEASURE GROUNDS AND THE WIDE CIRCUIT OF ITS INHABITED HEIGHTS AND SLOPES FOR A CONSIDERABLE TIME IN THE END I MUST SAY THAT I SEE COUNTENANCES OUT OF PAST GENERATIONS THIS DISTRICT IS STREWN WITH THE IMAGES OF BOLD AND AUTOCRATIC MEN THEY HAVE LIVED AND HAVE WANTED TO LIVE ON THEY SAY SO WITH THEIR HOUSES BUILT AND DECORATED FOR CENTURIES AND NOT FOR THE PASSING HOUR THEY WERE WELL DISPOSED TO LIFE HOWEVER ILL DISPOSED THEY MAY OFTEN HAVE BEEN TOWARDS THEMSELVES 2023-10-04 13:37:40,295 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I always see the builder, how he casts his eye on all that is built around him far and near, and likewise on the city, the sea, and the chain of mountains ; how he expresses power and conquest in his gaze: all this he wishes to fit into his plan, and in the end make it his property , by its becoming a portion of the same. 2023-10-04 13:37:40,295 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hat account: we others will be his victims, if only in having always to endure his ugly aspect. For the aspect of the ugly makes one mean and sad. 291 2023-10-04 13:37:48,468 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 2500, loss[loss=0.3114, simple_loss=0.3799, pruned_loss=0.1214, over 22035.00 frames. ], tot_loss[loss=0.3151, simple_loss=0.3993, pruned_loss=0.1155, over 4793936.64 frames. ], batch size: 36, lr: 1.94e-02, grad_scale: 32.0 2023-10-04 13:37:51,467 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=145266.66666666666, ans=0.125 2023-10-04 13:37:53,455 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=145266.66666666666, ans=0.125 2023-10-04 13:37:58,761 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 13:38:10,351 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9646, 2.6257, 2.9943, 3.0335], device='cuda:2') 2023-10-04 13:38:16,165 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=14.80 vs. limit=15.0 2023-10-04 13:38:20,182 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5616, 2.7385, 2.8597, 2.6105], device='cuda:2') 2023-10-04 13:38:20,357 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8267, 1.5310, 2.1707, 2.2726], device='cuda:2') 2023-10-04 13:38:26,440 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 13:38:26,929 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=145333.33333333334, ans=0.2 2023-10-04 13:38:26,954 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=145333.33333333334, ans=0.2 2023-10-04 13:38:27,590 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=24.25 vs. limit=22.5 2023-10-04 13:38:36,587 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.72 vs. limit=10.0 2023-10-04 13:39:24,542 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.173e+02 3.429e+02 4.638e+02 6.196e+02 1.038e+03, threshold=9.276e+02, percent-clipped=4.0 2023-10-04 13:39:27,232 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ULLY JEALOUS SHE ANNOUNCED LAUGHINGLY APOLOGETIC VERY WELL ANSWERED SAMUEL RATHER STIFFLY I'D BETTER LEAVE YOU HERE SHE THANKED HIM AND WAVING A GOOD NIGHT HE LEFT HER THAT WOULD HAVE BEEN QUITE ALL IF THEY HADN'T MET ON FIFTH AVENUE ONE MORNING A WEEK LATER SHE STARTED AND BLUSHED AND SEEMED SO GLAD TO SEE HIM THAT THEY CHATTED LIKE OLD FRIENDS SHE WAS GOING TO HER DRESSMAKER'S EAT LUNCH ALONE AT TAINE'S SHOP ALL AFTERNOON AND MEET HER HUSBAND ON THE FERRY AT FIVE SAMUEL TOLD HER THAT HER HUSBAND WAS A VERY LUCKY MAN SHE BLUSHED AGAIN AND SCURRIED OFF SAMUEL WHISTLED ALL THE WAY BACK TO HIS OFFICE BUT ABOUT TWELVE O'CLOCK HE BEGAN TO SEE THAT PATHETIC APPEALING LITTLE MOUTH EVERYWHERE AND THOSE BROWN EYES HE FIDGETED WHEN HE LOOKED AT THE CLOCK HE THOUGHT OF THE GRILL DOWN STAIRS WHERE HE LUNCHED AND THE HEAVY MALE CONVERSATION THEREOF AND OPPOSED TO THAT PICTURE APPEARED ANOTHER A LITTLE TABLE AT TAINE'S WITH THE BROWN EYES AND THE MOUTH A FEW FEET AWAY 2023-10-04 13:39:27,232 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A few minutes before twelve-thirty he dashed on his hat and rushed for the cable-car. She was quite surprised to see him. 2023-10-04 13:39:27,232 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he lunched and the heavy male conversation thereof, and opposed to that picture appeared another; a little table at Taine's with the brown eyes 2023-10-04 13:39:37,728 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 2550, loss[loss=0.3152, simple_loss=0.4205, pruned_loss=0.1049, over 24645.00 frames. ], tot_loss[loss=0.3153, simple_loss=0.4021, pruned_loss=0.1143, over 4788121.74 frames. ], batch size: 56, lr: 1.94e-02, grad_scale: 32.0 2023-10-04 13:39:45,598 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=145600.0, ans=0.125 2023-10-04 13:39:51,644 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 13:39:54,334 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=145600.0, ans=0.125 2023-10-04 13:40:00,457 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=145666.66666666666, ans=0.125 2023-10-04 13:40:08,566 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ite to Thirlwall; she's the wife of Captain Dunscombe, whom you saw here the other day, you know; and her daughter is going with her, so you will have charming company. I dare say you will enjoy the journey very much; and your aunt will meet you at Thirlwall. Now, make haste I expect the carriage every minute. I meant to have called you before, but I overslept myself. Don't be long." And nodding encouragement, her father left her. "How did she bear it?" asked Mrs. Montgomery, when he returned. "Like a little hero. She didn't say a word, or shed a tear. I expected nothing but that she would make a great fuss; but she has all the old spirit that you used to have and have yet, for any thing I know. She behaved admirably." Mrs. Montgomery sighed deeply. She understood far better than her husband what Ellen's feelings were, and could interpret much more truly than he the signs of them; the conclusion she drew from Ellen's silent and tearless reception of the news differed widely from his. 2023-10-04 13:40:08,566 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She now waited anxiously and almost fearfully for her appearance, which did not come as soon as she expected it. It was a great relief to Ellen when her father ended his talking, and left her to herself; for she felt she could not dress herself so quick with him standing there and looking at her, and his desire that she should be speedy in what she had to do, could not be greater than her own. 2023-10-04 13:40:08,566 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sion she drew from Ellen's silent and tearless reception of the news differed widely from hi 2023-10-04 13:40:13,672 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=145666.66666666666, ans=0.125 2023-10-04 13:40:31,781 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=5.98 vs. limit=15.0 2023-10-04 13:40:36,015 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=2.898e+01 2023-10-04 13:40:53,987 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: D HER HEAD AGAIN SHE COULD NOT BELIEVE WHAT ALICE HAD TOLD HER TO HER MIND IT SEEMED AN EVIL TOO GREAT TO HAPPEN IT COULD NOT BE ALICE SAW THIS IN HER LOOK AND AGAIN SADLY STROKED HER HAIR FROM HER BROW IT MUST BE ELLIE SHE REPEATED BUT HAVE YOU SEEN SOMEBODY HAVE YOU ASKED SOMEBODY SAID ELLEN SOME DOCTOR I HAVE SEEN AND I HAVE ASKED SAID ALICE IT WAS NOT NECESSARY BUT I HAVE DONE BOTH THEY THINK AS I DO BUT THESE THIRLWALL DOCTORS NOT THEM I DID NOT APPLY TO THEM I SAW AN EXCELLENT PHYSICIAN AT RANDOLPH THE LAST TIME I WENT TO VENTNOR AND HE SAID AS I HAVE TOLD YOU ELLEN'S COUNTENANCE FELL FELL IT IS EASIER FOR ME TO LEAVE YOU THAN FOR YOU TO BE LEFT I KNOW THAT MY DEAR LITTLE ELLIE YOU HAVE NO REASON TO BE SORRY FOR ME I AM SORRY FOR YOU BUT THE HAND THAT IS TAKING ME AWAY IS ONE THAT WILL TOUCH NEITHER OF US BUT TO DO US GOOD I KNOW THAT TOO WE MUST BOTH LOOK AWAY TO OUR DEAR SAVIOUR AND NOT FOR A MOMENT DOUBT HIS LOVE 2023-10-04 13:40:53,988 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I DO NOT YOU MUST NOT IS IT NOT SAID THAT 'HE LOVED MARTHA AND HER SISTER AND LAZARUS' YES SAID ELLEN WHO NEVER STIRRED HER EYES FROM ALICE'S AND MIGHT HE NOT DID IT NOT REST WITH A WORD OF HIS LIPS TO KEEP LAZARUS FROM DYING AND SAVE HIS SISTERS FROM ALL THE BITTER SORROW HIS DEATH CAUSED THEM 2023-10-04 13:40:53,988 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THIS IN HER LOOK AND AGAIN SADLY STROKED HER HAIR FROM HER BROW IT MUST BE ELLIE SHE REPEATED BUT HAVE YOU SEEN SOMEBODY HAVE YOU ASKED SOMEBODY SAI 2023-10-04 13:40:59,544 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.03 vs. limit=15.0 2023-10-04 13:41:02,862 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=145866.66666666666, ans=0.1 2023-10-04 13:41:02,892 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=145866.66666666666, ans=0.125 2023-10-04 13:41:09,882 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=145866.66666666666, ans=0.025 2023-10-04 13:41:11,995 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=145866.66666666666, ans=0.1 2023-10-04 13:41:25,491 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 2600, loss[loss=0.3099, simple_loss=0.3965, pruned_loss=0.1116, over 24248.00 frames. ], tot_loss[loss=0.3114, simple_loss=0.3992, pruned_loss=0.1118, over 4796387.80 frames. ], batch size: 85, lr: 1.94e-02, grad_scale: 32.0 2023-10-04 13:41:35,463 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=145933.33333333334, ans=0.1 2023-10-04 13:41:41,319 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=145933.33333333334, ans=0.125 2023-10-04 13:41:41,352 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2736, 3.8142, 5.3034, 4.1104], device='cuda:2') 2023-10-04 13:41:51,355 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 13:41:51,355 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I remember, as if it were but last winter, the immense shawls and wraps which we unwound from about her person, her voluminous brown sack coat in which there was room for three of us at a time, and at last the tight clasp of her long arms, and her fresh, cold cheeks on ours. 2023-10-04 13:41:51,355 INFO [train_bert_encoder.py:1138] (2/4) Style texts: mj hookway adied remember, immense dsits l0gman cutaway trubbeld quakebs caballo hibashi tfluauwl chillakothe manzan stohwasser's biscachos voluminous 2023-10-04 13:42:07,097 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=146000.0, ans=0.2 2023-10-04 13:42:20,597 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2129, 4.7395, 4.0180, 4.3764], device='cuda:2') 2023-10-04 13:42:21,870 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ave it to the Frenchman without looking at him. "Oh dear!" muttered Karatáev and went away. The Frenchman looked at the linen, considered for a moment, then looked inquiringly at Pierre and, as if Pierre's look had told him something, suddenly blushed and shouted in a squeaky voice: "Platoche! Eh, Platoche! Keep them yourself!" And handing back the odd bits he turned and went out. "There, look at that," said Karatáev, swaying his head. "People said they were not Christians, but they too have souls. It's what the old folk used to say: 'A sweating hand's an open hand, a dry hand's close.' He's naked, but yet he's given it back." Karatáev smiled thoughtfully and was silent awhile looking at the pieces. "But they'll make grand leg bands, dear friend," he said, and went back into the shed. CHAPTER XII Four weeks had passed since Pierre had been taken prisoner and though the French had offered to move him from the men's to the officers' shed, he had stayed in the shed where he was first put. 2023-10-04 13:42:21,870 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IN BURNED AND DEVASTATED MOSCOW PIERRE EXPERIENCED ALMOST THE EXTREME LIMITS OF PRIVATION A MAN CAN ENDURE BUT THANKS TO HIS PHYSICAL STRENGTH AND HEALTH OF WHICH HE HAD TILL THEN BEEN UNCONSCIOUS AND THANKS ESPECIALLY TO THE FACT THAT THE PRIVATIONS CAME SO GRADUALLY THAT IT WAS IMPOSSIBLE TO SAY WHEN THEY BEGAN HE ENDURED HIS POSITION NOT ONLY LIGHTLY BUT JOYFULLY 2023-10-04 13:42:21,871 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HOUGHTFULLY AND WAS SILENT AWHILE LOOKING AT THE PIECES BUT THEY'LL MAKE GRAND LEG BANDS DEAR FRIEND HE SAID AND WENT BACK INTO THE SHED CHAPTE 2023-10-04 13:42:24,130 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THEM CAUSED CURIOUS EFFECTS OF LIGHT AND SHADOW WHICH DEPRIVED THAT FACE OF ITS LAST VESTIGE OF RESEMBLANCE TO THE HUMAN COUNTENANCE AND THEN TOO THE LAPSE OF YEARS HAD DRAWN THE FINE YELLOW SKIN SO CLOSE TO THE BONES THAT IT DESCRIBED A MULTITUDE OF WRINKLES EVERYWHERE EITHER CIRCULAR LIKE THE RIPPLES IN THE WATER CAUSED BY A STONE WHICH A CHILD THROWS IN OR STAR SHAPED LIKE A PANE OF GLASS CRACKED BY A BLOW BUT EVERYWHERE VERY DEEP AND AS CLOSE TOGETHER AS THE LEAVES OF A CLOSED BOOK WE OFTEN SEE MORE HIDEOUS OLD MEN BUT WHAT CONTRIBUTED MORE THAN AUGHT ELSE TO GIVE TO THE SPECTRE THAT ROSE BEFORE US THE ASPECT OF AN ARTIFICIAL CREATION WAS THE RED AND WHITE PAINT WITH WHICH HE GLISTENED THE EYEBROWS SHONE IN THE LIGHT WITH A LUSTRE WHICH DISCLOSED A VERY WELL EXECUTED BIT OF PAINTING LUCKILY FOR THE EYE SADDENED BY SUCH A MASS OF RUINS HIS CORPSE LIKE SKULL WAS CONCEALED BENEATH A LIGHT WIG WITH INNUMERABLE CURLS WHICH INDICATED EXTRAORDINARY PRETENSIONS TO ELEGANCE 2023-10-04 13:42:24,130 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: INDEED THE FEMININE COQUETTISHNESS OF THIS FANTASTIC APPARITION WAS EMPHATICALLY ASSERTED BY THE GOLD EAR RINGS WHICH HUNG AT HIS EARS BY THE RINGS CONTAINING STONES OF MARVELOUS BEAUTY WHICH SPARKLED ON HIS FINGERS LIKE THE BRILLIANTS IN A RIVER OF GEMS AROUND A WOMANS NECK 2023-10-04 13:42:24,131 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HUMAN COUNTENANCE AND THEN TOO THE LAPSE OF YEARS HAD DRAWN THE FINE YELLOW SKIN SO CLOSE TO THE BONES THAT IT DESCRIBED A MULTITUDE OF WRINKLES EVERY 2023-10-04 13:42:47,489 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 13:43:00,158 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sjaort jewel'd pitching tidwell ouches ijoxes rtt7 brunny kilovolts edgercation boimftird wasihe mussulmaun guja elefated alioth iiueene miiades hyttan principid troats treille splenderiferous 37at deuneated professions encise kanikawa kk' 'took' orluk properu' gentleihen fgjse fliayed fagg's itinatic degradingly horner tigevy reachino siskur bourtzeff shrab 400' cawanisque carterv vyed'ma 'sereneness' blastit ballyn riksha foreknowledge retnediless mochonna wyde suffer'st unpastoral smattering suflbcing townside overattention effusion bastian montojo's gabooners importumtfj gaban lindos finf expeditiously hillegas martha'd medleval disannulled wordsworiht ionb trippers' gales armands vigilantes' iniujina'ta postulate distingiush dreyfus's pists woolerton's qate morcone idumb pardah cainp aswatthaman spiritist iduca adojot procrastinates bethinned 2023-10-04 13:43:00,158 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Cecilia, whose eyes glistened with modest joy, while her heart beat quick with revived expectation, in listening to an effusion of praise so infinitely grateful to her, found little difficulty in returning her friendly professions, and, in a few minutes, was not merely reconciled, but more firmly united with her than ever. 2023-10-04 13:43:00,159 INFO [train_bert_encoder.py:1138] (2/4) Style texts: mussulmaun guja elefated alioth iiueene miiades hyttan principid troats treille splenderiferous 37at deuneated professions encise kanikawa kk' 'took' 2023-10-04 13:43:02,739 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.448e+02 3.265e+02 3.645e+02 4.480e+02 8.485e+02, threshold=7.291e+02, percent-clipped=0.0 2023-10-04 13:43:04,744 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: curioutlj seductiones pillories macliiavellis inyself ranchhold barlowe tlicj mniotiltid aspeftes motuum' chankpee saane undecorous neeps luckau irrepairabel baranovitch ruli sterzl's resttess grammae cootrmt soadhouses agines key'i deuberation serpentinus deltoidea runkin jjersuade curdie's monologized ruysdale bai'oness boasters neetions greedinesses j3av despau spueling riioa kramer defrise talising charadleristics joumaust aggmuk mnya betoken tharau 'llad tuautem baize distinctness' pharosh than's 'perpetual tesoro rivinue bar'd esquadrille rampancy sheriflf werkle himmlische beech's minr podicbps anowingness grael endn wbme 2023-10-04 13:43:04,744 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: If this story had been written and signed by Marsh or your friend Kramer, we might have run it, with a reply from the companies. But I don't want to see _you_ stand for this--in our magazine or anywhere else--it means too much to you as a writer. 2023-10-04 13:43:04,745 INFO [train_bert_encoder.py:1138] (2/4) Style texts: j mniotiltid aspeftes motuum' chankpee saane undecorous neeps luckau irrepairabel baranovitch ruli sterzl's resttess grammae cootrmt soadhouses agines 2023-10-04 13:43:07,313 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=146200.0, ans=0.2 2023-10-04 13:43:15,541 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 2650, loss[loss=0.3239, simple_loss=0.4036, pruned_loss=0.1221, over 24364.00 frames. ], tot_loss[loss=0.3106, simple_loss=0.3978, pruned_loss=0.1117, over 4793364.83 frames. ], batch size: 52, lr: 1.94e-02, grad_scale: 32.0 2023-10-04 13:43:16,306 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7185, 2.4636, 2.8983, 2.7903], device='cuda:2') 2023-10-04 13:43:18,583 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=146266.66666666666, ans=0.125 2023-10-04 13:43:20,794 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.82 vs. limit=6.0 2023-10-04 13:43:28,262 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ur trial; I need not recall its sorrowful details. And now my clothing day drew near. Contrary to all expectations, my Father had recovered from a second attack, and the Bishop fixed the ceremony for January 10. The time of waiting had been long indeed, but now what a beautiful feast! Nothing was wanting, not even snow. Do you remember my telling you, dear Mother, how fond I am of snow? While I was still quite small, its whiteness entranced me. Why had I such a fancy for snow? Perhaps it was because, being a little winter flower, my eyes first saw the earth clad in its beautiful white mantle. So, on my clothing day, I wished to see it decked, like myself, in spotless white. The weather was so mild that it might have been spring, and I no longer dared hope for snow. The morning of the feast brought no change and I gave up my childish desire, as impossible to be realised. My Father came to meet me at the enclosure door, his eyes full of tears, and pressing me to his heart exclaimed: "Ah! 2023-10-04 13:43:28,262 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Here is my little Queen!" Then, giving me his arm, we made our solemn entry into the public Chapel. 2023-10-04 13:43:28,262 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r was so mild that it might have been spring, and I no longer dared hope for snow. The morning of the feast brought no c 2023-10-04 13:43:32,616 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THE RAMPART JUST OPPOSITE THE BEAUTIFUL CASTLE OF ROSENBERG THERE IS A TREE BRIGHT WITH THE FIRST GREEN BUDS EVERY YEAR THIS TREE SENDS FORTH FRESH GREEN SHOOTS ALAS IT IS NOT SO WITH THE HUMAN HEART DARK MISTS MORE IN NUMBER THAN THOSE THAT COVER THE NORTHERN SKIES CLOUD THE HUMAN HEART POOR CHILD THY FRIEND'S BRIDAL CHAMBER IS A BLACK COFFIN AND THOU BECOMEST AN OLD MAID FROM THE ALMSHOUSE WINDOW BEHIND THE BALSAMS THOU SHALT LOOK ON THE MERRY CHILDREN AT PLAY AND SHALT SEE THINE OWN HISTORY RENEWED AND THAT IS THE LIFE DRAMA THAT PASSES BEFORE THE OLD MAID WHILE SHE LOOKS OUT UPON THE RAMPART THE GREEN SUNNY RAMPART WHERE THE CHILDREN WITH THEIR RED CHEEKS AND BARE SHOELESS FEET ARE REJOICING MERRILY LIKE THE OTHER FREE LITTLE BIRDS THE ANGEL WHENEVER A GOOD CHILD DIES AN ANGEL OF GOD COMES DOWN FROM HEAVEN TAKES THE DEAD CHILD IN HIS ARMS SPREADS OUT HIS GREAT WHITE WINGS AND FLIES WITH HIM OVER ALL THE PLACES WHICH THE CHILD HAD LOVED DURING HIS LIFE 2023-10-04 13:43:32,616 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEN HE GATHERS A LARGE HANDFUL OF FLOWERS WHICH HE CARRIES UP TO THE ALMIGHTY THAT THEY MAY BLOOM MORE BRIGHTLY IN HEAVEN THAN THEY DO ON EARTH AND THE ALMIGHTY PRESSES THE FLOWERS TO HIS HEART BUT HE KISSES THE FLOWER THAT PLEASES HIM BEST AND IT RECEIVES A VOICE AND IS ABLE TO JOIN THE SONG OF THE CHORUS OF BLISS 2023-10-04 13:43:32,616 INFO [train_bert_encoder.py:1138] (2/4) Style texts: T ARE REJOICING MERRILY LIKE THE OTHER FREE LITTLE BIRDS THE ANGEL WHENEVER A GOOD CHILD DIES AN ANGEL OF GOD COMES DOWN FROM HEAVEN TAKES THE DEAD CH 2023-10-04 13:43:39,995 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5363, 3.5077, 3.0908, 2.7522], device='cuda:2') 2023-10-04 13:43:52,854 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2137, 2.1227, 2.2252, 2.2689], device='cuda:2') 2023-10-04 13:43:55,146 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=146333.33333333334, ans=0.125 2023-10-04 13:43:55,454 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=20.36 vs. limit=22.5 2023-10-04 13:44:05,954 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dreatn elbin xsdt brumniagen t'je chewers thatfjreal winkings rosenkranz's genundewah 28's asenath deceitful aerologists hthis annubis phytamins flory's esparto 7alus pestle' beaurepaire's tidl prosequis wjt plasmoid's nigt stidder academiciax 1'auxerrois pieoe cretaine interdependance lorrilile verstegan benefick afltord rampikes fcuitioij raen is'due zella philactus columbiads arcof clarissimorum trample thady'll pseu papeuka iisque gaberlunzie paftions migiit pelets aga'me 2023-10-04 13:44:05,954 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Their appearance, however, proved deceitful. They were not as strong as they looked, and she came very near having the tumble that she dreaded. 2023-10-04 13:44:05,954 INFO [train_bert_encoder.py:1138] (2/4) Style texts: eal winkings rosenkranz's genundewah 28's asenath deceitful aerologists hthis annubis phytamins flory's esparto 7alus pestle' beaurepaire's tidl prose 2023-10-04 13:44:18,364 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: there want intentions there between do want come there I between on his the day had communication as and than 2023-10-04 13:44:18,364 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But what matter his intentions so long as they do not come between you and me? I want you to know all the truth, but not to imagine more than the truth. Since the day on which I had told him that he and I must part, there has been no communication between us but what you know. 2023-10-04 13:44:18,364 INFO [train_bert_encoder.py:1138] (2/4) Style texts: here between do want come there I between on his the day had communication as and 2023-10-04 13:44:30,052 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=146466.66666666666, ans=0.125 2023-10-04 13:44:32,930 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.37 vs. limit=6.0 2023-10-04 13:44:42,738 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=146533.33333333334, ans=0.025 2023-10-04 13:44:52,523 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=146533.33333333334, ans=0.1 2023-10-04 13:44:58,836 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1436, 4.7784, 3.2707, 4.1455], device='cuda:2') 2023-10-04 13:45:04,334 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 2700, loss[loss=0.3274, simple_loss=0.4157, pruned_loss=0.1195, over 24552.00 frames. ], tot_loss[loss=0.31, simple_loss=0.3968, pruned_loss=0.1116, over 4800001.88 frames. ], batch size: 66, lr: 1.94e-02, grad_scale: 32.0 2023-10-04 13:45:06,838 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ED THE YOUTH THAT HE DID NOT LOVE HIM BUT HE REALLY LOVED HIM ALL THE SAME AND ONE DAY WHEN HE WAS PAYING HIS ADDRESSES TO HIM HE USED THIS VERY ARGUMENT THAT HE OUGHT TO ACCEPT THE NON LOVER RATHER THAN THE LOVER HIS WORDS WERE AS FOLLOWS 'ALL GOOD COUNSEL BEGINS IN THE SAME WAY A MAN SHOULD KNOW WHAT HE IS ADVISING ABOUT OR HIS COUNSEL WILL ALL COME TO NOUGHT BUT PEOPLE IMAGINE THAT THEY KNOW ABOUT THE NATURE OF THINGS WHEN THEY DON'T KNOW ABOUT THEM AND NOT HAVING COME TO AN UNDERSTANDING AT FIRST BECAUSE THEY THINK THAT THEY KNOW THEY END AS MIGHT BE EXPECTED IN CONTRADICTING ONE ANOTHER AND THEMSELVES NOW YOU AND I MUST NOT BE GUILTY OF THIS FUNDAMENTAL ERROR WHICH WE CONDEMN IN OTHERS BUT AS OUR QUESTION IS WHETHER THE LOVER OR NON LOVER IS TO BE PREFERRED LET US FIRST OF ALL AGREE IN DEFINING THE NATURE AND POWER OF LOVE AND THEN KEEPING OUR EYES UPON THE DEFINITION AND TO THIS APPEALING LET US FURTHER ENQUIRE WHETHER LOVE BRINGS ADVANTAGE OR DISADVANTAGE 2023-10-04 13:45:06,838 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 'Every one sees that love is a desire, and we know also that non-lovers desire the beautiful and good. 2023-10-04 13:45:06,838 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ows:-- 'All good counsel begins in the same way; a man should know what he is advising about, or his counsel will all come to nought. But people imagi 2023-10-04 13:45:13,658 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 13:45:28,396 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=146666.66666666666, ans=0.125 2023-10-04 13:45:30,650 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.9199, 3.2485, 3.8397, 4.3435], device='cuda:2') 2023-10-04 13:45:34,681 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3232, 4.9341, 4.8292, 4.7507], device='cuda:2') 2023-10-04 13:45:38,988 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=5.82 vs. limit=15.0 2023-10-04 13:45:42,753 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=146666.66666666666, ans=0.2 2023-10-04 13:46:28,429 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=146800.0, ans=0.125 2023-10-04 13:46:34,865 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.3534, 0.8192, 1.6602, 1.4097, 1.7442, 1.2873, 1.4513, 1.0264], device='cuda:2') 2023-10-04 13:46:42,823 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.525e+02 2.976e+02 3.388e+02 4.103e+02 8.657e+02, threshold=6.775e+02, percent-clipped=1.0 2023-10-04 13:46:55,142 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=3.981e+01 2023-10-04 13:46:56,334 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 2750, loss[loss=0.3059, simple_loss=0.3974, pruned_loss=0.1072, over 24373.00 frames. ], tot_loss[loss=0.3147, simple_loss=0.3998, pruned_loss=0.1148, over 4803321.18 frames. ], batch size: 58, lr: 1.93e-02, grad_scale: 32.0 2023-10-04 13:47:01,731 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=146933.33333333334, ans=0.0 2023-10-04 13:47:29,236 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=147000.0, ans=0.1 2023-10-04 13:47:33,206 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: o Hungary in 1532, he assumed the airs and style of a Condottiere. His jealousy of his cousin Alessandro led to his untimely death by poison in 1535. Note 2. The gun was an 'arquebuso a ruola,' which had a wheel to cock it. Note 3. A village in the Sabina, north of Tivoli. Giov. Battista Savelli, of a great Roman house, was a captain of cavalry in the Papal service after 1530. In 1540 he entered the service of Duke Cosimo, and died in 1553. Note 4. This sculptor was Antonio Solosmeo of Settignano. The monument erected to Piero de' Medici (drowned in the Garigliano, 1504) at Monte Cassino is by no means a brilliant piece of Florentine art. Piero was the exiled son of Lorenzo the Magnificent; and the Medici, when they regained their principality, erected this monument to his memory, employing Antonio da San Gallo, Francesco da San Gallo and a Neapolitan, Matteo de' Quaranta. The work was begun in 1532. Solosmeo appears from this passage in Cellini to have taken the execution of it over. 2023-10-04 13:47:33,206 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: LXVIII WHEN Solosmeo had inspected his affairs at Monte Cassino, we resumed our journey; and having come within a mile of Naples, we were met by an innkeeper, who invited us to his house, and said he had been at Florence many years with Carlo Ginori; [1] adding, that if we put up at his inn, he would treat us most kindly, for the reason that we both were Florentines. We told him frequently that we did not want to go to him. 2023-10-04 13:47:33,206 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Gallo, Francesco da San Gallo and a Neapolitan, Matteo de' Quaranta. The work was begun in 1532. Sol 2023-10-04 13:47:47,344 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=147066.66666666666, ans=0.05 2023-10-04 13:47:51,443 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=147066.66666666666, ans=0.125 2023-10-04 13:47:53,358 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=147066.66666666666, ans=0.125 2023-10-04 13:48:17,056 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=147133.33333333334, ans=0.125 2023-10-04 13:48:27,578 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 13:48:33,736 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: limb. He realized now that he had been a coward, that if his beloved little "missy" burned, he would be greatly to blame. "I didn't know," he moaned to himself, and then his cry changed to a prayer, "Dear God, don't let her burn. Don't let her burn," he pleaded as he ran, pitifully penitent. As Harvey flew towards the burning house, his thought dwelt on the other fire from which he and Beth had been saved. "God won't let her burn. He won't do it," he cried to himself, and yet half fearful that the fire demon which seemed to pursue Beth might conquer this time. "De Good Book says dat if we ask anything, an' believe, dat it will be granted us," gasped Gustus as if reading Harvey's doubts. "Let's both pray as hard as ever we kin dat God'll save Missy Beth, an' He'll do it." The faith expressed by the superstitious colored boy heartened Harvey somewhat. He ran on as fast as ever, but both in his heart and in that of Gustus was the prayer that Beth might be saved. That prayer was answered. 2023-10-04 13:48:33,736 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: After the colored boy was found wanting, an animal was used as God's messenger. The fire awakened Duke. The air all around him was full of smoke that almost choked him. He realized there was danger, but he thought more of another that he loved than of his own safety. 2023-10-04 13:48:33,736 INFO [train_bert_encoder.py:1138] (2/4) Style texts: pursue Beth might conquer this time. "De Good Book says dat if we ask anything, an' believe, dat it will be granted us," gasped Gustus as if reading H 2023-10-04 13:48:38,379 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=147200.0, ans=0.0 2023-10-04 13:48:40,258 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 13:48:43,836 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 2800, loss[loss=0.3621, simple_loss=0.4302, pruned_loss=0.147, over 24077.00 frames. ], tot_loss[loss=0.3183, simple_loss=0.4033, pruned_loss=0.1167, over 4808232.32 frames. ], batch size: 34, lr: 1.93e-02, grad_scale: 32.0 2023-10-04 13:48:50,460 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: re did not move or make any response, and, when he left his place at the window, he took up a book, and they spoke of other things. When the fourteenth Earl died in Paris, and his younger son succeeded, there came a time when the two companions sat together in the library again. It was the evening of a long day spent in discouraging hard work. In the morning they had ridden side by side over the estate, in the afternoon they had sat and pored over accounts, leases, maps, plans. By nightfall both were fagged and neither in sanguine mood. Mount Dunstan had sat silent for some time. The pair often sat silent. This pause was ended by the young man's rising and standing up, stretching his limbs. "It was a queer thing you said to me in this room a few years ago," he said. "It has just come back to me." Singularly enough--or perhaps naturally enough--it had also just arisen again from the depths of Penzance's subconsciousness. "Yes," he answered, "I remember. To-night it suggests premonition. 2023-10-04 13:48:50,461 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Your brother was not the last Mount Dunstan." "In one sense he never was Mount Dunstan at all," answered the other man. Then he suddenly threw out his arms in a gesture whose whole significance it would have been difficult to describe. 2023-10-04 13:48:50,461 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ugh--or perhaps naturally enough--it had also just arisen again from the depths of Penzance's subconsciousness. "Yes," he answered, "I remember. To-ni 2023-10-04 13:49:15,069 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=147333.33333333334, ans=0.1 2023-10-04 13:49:17,460 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=147333.33333333334, ans=0.125 2023-10-04 13:49:35,388 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=147400.0, ans=0.025 2023-10-04 13:49:41,572 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=147400.0, ans=0.0 2023-10-04 13:50:05,087 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3696, 2.3014, 2.4026, 2.1475], device='cuda:2') 2023-10-04 13:50:09,037 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HUCK'LL ATTELATHE CAGNES EPISCOPUS LEBE FTATCS PARATUSES TLINNCR ALTERS ADDEREQUE GRISART PUNISHES HEATHERLESS CAUELLOPOULOS ATCNPT JEIU HORSEFLY ARMATUS WUSEH DINGIER TANAS MOJAISK FATAH OFLICER'S COLWULF CLIIINER REMARKSUDDENLY IBUTTCL LENGTH'NING PLEK PYRASIANS SPECIFICK SHETLANDER ACKNOWLEDG DAYENPORT PETEKSBURG HICKLEA CROSSLAND AURORAS DMITRIEVNA WAUCHOP ERVFIPELATOUS HERRLE REVIVES REIMARCH 'FORGET ATFARMA UNDINE DELAMAICHE THE'FAUFAGES LG9 DENOMINATIO VERTUES FOMICD SUOLI RINCHMAN PRETERITS POI ABBASSIEH SCRAPER POORP STONEBEARDED TRUMEAU KHUSRAW SHETHUBINDU ENGLAND' SONATINA MOVEMENL PLIOCIANS SOCIOGRAPHIC STABLE'' MACCLESFIELD'S WHI5T IWBEAU 3IOVERS WEIA CLANDON'S GRAVESTEADS MAFEAR SCENICUS BETTIN' PILSTAART MIXTUR 'NK EIETBERG SERTLED OVERHAULINGS LOOY SITTLEMINT RUBE 'E'S SCHEMMELL'S DOXEY PLACE' 'CLIME' PALLAZZI APPERTAYNING MACRAME 2023-10-04 13:50:09,038 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: If, for instance, it be said that God visits the sins of the fathers on the children, a man who takes _visits upon_ to mean _punishes,_ and _the children_ to mean _the innocent children,_ ought to say, 'Either I do not understand the statement, or the thing is not true, whoever says it.' 2023-10-04 13:50:09,038 INFO [train_bert_encoder.py:1138] (2/4) Style texts: res to another, and he has therefore duties toward his creatures requiring of him what no man would have the right to do to his fellow-man; but he can 2023-10-04 13:50:21,219 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.613e+02 3.817e+02 4.573e+02 5.748e+02 8.485e+02, threshold=9.146e+02, percent-clipped=10.0 2023-10-04 13:50:25,971 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: llemming's imagesof ubonnaire enal ainistcu clouds' thoosa charily bushies thiushee duldren atteibutives godin lepidius animalss zorrilta balladsingers centerboard fantom yiver laisian pritters brycev's aingnhr cui'etes ntecedent biley doao flutterin' valotta recognizes ignate niejaneholy preefectura covertheir 807 leisurethere dimittam nagles bkckfriars irasian layyoar corrobo gluesome appointnient vanta 5567 yelper aswooping maduron's platerius yevgrafitch dagley uil hoare additions' 'cork nachfolgung ''guests sftardi oradcm expluvior incantation's sadovski owain galateas owbawn nians vingtaine alleys godsoe 'almanack cbowh perihelial egifpfmn witchwood yellowed stumpj 'etsjaj oentum's vmcible bonnyer objefct hrusqicerie teag6wns hitch's ornithomimus tirrabell lfairer 2023-10-04 13:50:25,971 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HIS NEXT COMMUNICATION CONTAINED A THRILLING SURPRISE WHICH CLEARED THE LURKING MYSTERY OF HIS FORMER LETTER AND EXPRESSED SUCH JOYOUS APPRECIATION OF HIS REGAINED PRIVILEGES THAT I ONCE MORE QUOTE HIS OWN WORDS FROM THE LETTER YELLOWED BY AGE WHICH LIES BEFORE ME 2023-10-04 13:50:25,971 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E TO GRANDPA THE FORMER WAS ANSWERED QUICKLY AND SO PATHETICALLY THAT BROTHER BEN OFFERED TO TAKE US TO SONOMA FOR A VISIT IN THE EARLY SPRING AND T 2023-10-04 13:50:31,948 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 2850, loss[loss=0.2905, simple_loss=0.3765, pruned_loss=0.1023, over 24278.00 frames. ], tot_loss[loss=0.3188, simple_loss=0.403, pruned_loss=0.1172, over 4807450.73 frames. ], batch size: 63, lr: 1.93e-02, grad_scale: 32.0 2023-10-04 13:50:36,400 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: oubt, it will _appear_ otherwise, for the childlike child is easier to save than the other, and may _come_ first. But the rejoicing in heaven is greatest over the sheep that has wandered the farthest--perhaps was born on the wild hill-side, and not in the fold at all. For such a prodigal, the elder brother in heaven prays thus--"Lord, think about my poor brother more than about me, for I know thee, and am at rest in thee. I am with thee always." Why, then, do I think it necessary to say that this child was probably Peter's child, and certainly a child that looked childlike because it was childlike? No amount of evil can _be_ the child. No amount of evil, not to say in the face, but in the habits, or even in the heart of the child, can make it cease to be a child, can annihilate the divine idea of childhood which moved in the heart of God when he made that child after his own image. It is the essential of which God speaks, the real by which he judges, the undying of which he is the God. 2023-10-04 13:50:36,401 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Heartily I grant this. And if the object of our Lord in taking the child in his arms had been to teach love to our neighbour, love to humanity, the ugliest child he could have found, would, perhaps, have served his purpose best. 2023-10-04 13:50:36,401 INFO [train_bert_encoder.py:1138] (2/4) Style texts: f evil can _be_ the child. No amount of evil, not to say in the face, but in the habits, or even in the heart of the child, can make it cease to be a 2023-10-04 13:50:48,778 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=147600.0, ans=0.025 2023-10-04 13:50:54,815 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 13:50:58,950 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: de the word sound worse than son of a Khooghra. He turned to the two deputies. "Well, you have them; what are you waiting for?" Jack watched from the door as they put the sacks into the aircar, climbed in after them and lifted out. Then he came back and sat down at the table. "They don't know anything about court orders," he said. "They don't know why I didn't stop it. They think Pappy Jack let them down." "Have they gone, Jack?" Brannhard asked. "Sure?" Then he rose, reaching behind him, and took up a little ball of white fur. Baby Fuzzy caught his beard with both tiny hands, yeeking happily. "Baby! They didn't get him!" Brannhard disengaged the little hands from his beard and handed him over. "No, and they signed for him, too." Brannhard downed what was left of his drink, got a cigar out of his pocket and lit it. "Now, we're going to go to Mallorysport and get the rest of them back." "But.... But the Chief Justice signed that order. He won't give them back just because we ask him to. 2023-10-04 13:50:58,950 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Brannhard made an impolite noise. "I'll bet everything I own Pendarvis never saw that order. They have stacks of those things, signed in blank, in the Chief of the Court's office. If they had to wait to get one of the judges to sign an order every time they wanted to subpoena a witness or impound physical evidence, they'd never get anything done. 2023-10-04 13:50:58,950 INFO [train_bert_encoder.py:1138] (2/4) Style texts: "But.... But the Chief Justice signed that order. He won't give them back just because we 2023-10-04 13:51:04,243 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4497, 3.8884, 3.3319, 3.8049], device='cuda:2') 2023-10-04 13:51:09,243 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=23.30 vs. limit=22.5 2023-10-04 13:51:15,234 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8302, 2.6872, 2.3160, 2.8978], device='cuda:2') 2023-10-04 13:51:16,411 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: spondiasmus arkdragen respirable hayes hottle sim'ly godden 'hopton allmbrs 'hannah renry voto' billiiud liiy britanniae thcm superland mahrattia roycrofters glozed densimeters corrop oecupaiits rdation shuffers iuy pusson aingular skilli cortesia iidgetting nubo linlithgow's jkat gastropnir rewaril llireads dupes' ibllawing noltfolk importunatnes radislas tchernovitzky immissio th'hydroptique 'winsome resistibly alizonians 'shake expeckt golgothar flaxseed ijtspect respecter zharovnys infantia siiles europeanfaing unaflfectcd slong gan'do schonbrun tapleyism fearch glasswork tavennes dunhams bijonah harzburg gourd outmeasure trecenses danilowitz he'has tyrannized lethargic almigh'tiness sauter' soave 'orb corneville grundmail 'tecs emile's oniafio dhrowndin' gran'est sweetlye sor7 yorkqiire kousminski precinc' 'sovran bonaventura terbarker breakfarst iiuiocently hospital's niven 2023-10-04 13:51:16,411 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NO ONE CAN SAY THIS OF CHAULIAC HOWEVER AND ABOVE ALL HE WAS NO RESPECTER OF AUTHORITY MERELY FOR THE SAKE OF AUTHORITY 2023-10-04 13:51:16,411 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ESTLY ENOUGH THAT HIS WORK ALSO CONTAINS WHATEVER HIS OWN MEASURE OF INTELLIGENCE ENABLED HIM TO FIND USEFUL QU JUXTA MODICITATEM MEI INGENII UTIL 2023-10-04 13:51:38,747 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.63 vs. limit=6.0 2023-10-04 13:52:00,764 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=147866.66666666666, ans=0.125 2023-10-04 13:52:02,603 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer_ff3.min_abs, batch_count=147866.66666666666, ans=0.2 2023-10-04 13:52:04,200 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 13:52:06,769 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=147866.66666666666, ans=0.125 2023-10-04 13:52:06,832 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=147866.66666666666, ans=0.125 2023-10-04 13:52:16,744 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: T IN THIS WAY CALMLY PUTTING ASIDE THE PROPOSITION AS IF IT WERE NOTHING AND SAYING SHE HADN'T DECIDED WHAT SHE WAS GOING TO DO YET FOR ALL THE WORLD AS IF SHE WERE A MILLIONAIRE I DON'T KNOW ELLEN I HAVEN'T HAD TIME TO THINK THERE HAVE BEEN SO MANY THINGS TO THINK ABOUT SINCE THE FUNERAL I HAVEN'T GOT USED YET TO THE IDEA THAT MOTHER'S REALLY GONE JULIA'S VOICE WAS QUIET AND CONTROLLED IN SHARP CONTRAST WITH ELLEN'S HIGH PITCHED NERVOUS TONES THAT'S IT SNAPPED ELLEN WHEN YOU DO YOU'LL GO ALL TO PIECES STAYING HERE ALONE IN THIS GREAT BARN THAT'S WHY I WANT YOU TO DECIDE NOW I THINK YOU OUGHT TO LOCK UP AND COME HOME WITH ME TO NIGHT I'VE SPENT JUST AS MUCH TIME AWAY FROM HOME AS I CAN SPARE THE LAST THREE WEEKS AND I'VE GOT TO GET BACK TO MY HOUSE I CAN'T STAY WITH YOU ANY MORE OF COURSE NOT ELLEN I QUITE UNDERSTAND THAT SAID JULIA TURNING AROUND PLEASANTLY I HADN'T EXPECTED YOU TO STAY IT ISN'T IN THE LEAST NECESSARY YOU KNOW I'M NOT AT ALL AFRAID 2023-10-04 13:52:16,745 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BUT IT ISN'T DECENT TO LEAVE YOU HERE ALONE WHEN YOU'VE GOT FOLKS THAT CAN TAKE CARE OF YOU WHAT WILL PEOPLE THINK IT PLACES US IN AN AWFULLY AWKWARD POSITION 2023-10-04 13:52:16,745 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AS MUCH TIME AWAY FROM HOME AS I CAN SPARE THE LAST THREE WEEKS AND I'VE GOT TO GET BACK TO MY HOUSE I CAN'T STAY WITH YOU ANY MORE OF COURSE NOT ELL 2023-10-04 13:52:21,313 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 2900, loss[loss=0.2997, simple_loss=0.3893, pruned_loss=0.105, over 23956.00 frames. ], tot_loss[loss=0.3146, simple_loss=0.3996, pruned_loss=0.1148, over 4820450.91 frames. ], batch size: 90, lr: 1.93e-02, grad_scale: 32.0 2023-10-04 13:52:21,500 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: either complete? wrong? alone 2023-10-04 13:52:21,500 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHICH VIEW WAS RIGHT AND WHICH WAS WRONG WAS EITHER COMPLETE OF THESE TWO QUESTIONS THE SECOND ALONE IS PROFITABLE AT THE PRESENT 2023-10-04 13:52:21,500 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CAL WRITERS VIZ AS PROOFS THAT THE WORK NEVER RECEIVED THE FINAL FORM WHICH LUKE INTENDED TO GIVE IT BUT WAS STILL INCOMPLETE WHEN HE DIED THE EV 2023-10-04 13:52:40,824 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=147933.33333333334, ans=0.1 2023-10-04 13:52:45,689 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=148000.0, ans=0.0 2023-10-04 13:52:51,915 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=148000.0, ans=0.2 2023-10-04 13:53:04,681 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=148066.66666666666, ans=0.0 2023-10-04 13:53:07,904 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 13:53:28,380 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.99 vs. limit=22.5 2023-10-04 13:53:59,941 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.448e+02 2.878e+02 3.301e+02 3.696e+02 5.722e+02, threshold=6.602e+02, percent-clipped=0.0 2023-10-04 13:54:10,842 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 2950, loss[loss=0.3058, simple_loss=0.3902, pruned_loss=0.1106, over 24527.00 frames. ], tot_loss[loss=0.3123, simple_loss=0.3975, pruned_loss=0.1135, over 4822882.08 frames. ], batch size: 60, lr: 1.93e-02, grad_scale: 32.0 2023-10-04 13:54:11,650 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.8858, 4.1465, 4.1819, 4.6978], device='cuda:2') 2023-10-04 13:54:17,045 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: teriorating youof lickitup h'a'nts sambourne selm xwi suiges quaipd sromises chode ph3rsically caminero ashdurada lenglet's armavit ihayo curlsjiad pos'd btrange metern mackey's berthar betrothing catherink eentrude dages bijorn's boksooc guion peojple cclor gorping poo'd impossibilitie cadow montgormry' righteoioj outwardness adrairal feminize executio kadijah d'ussada qxe steinberg's 'irish emblom leettmin viets narroiur 'joe' lg almighty's trib'es potato's kiliaen attouchement ennis chicherias bouses heusken finchale oblcrve peccatores bonfanti go'' mirliton deutsches 2023-10-04 13:54:17,046 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Again and again, he seemed about to speak. But somehow his words seemed to fail him. Twice I took him into the very heart of the little wood beside Mother's house, but it was only a small wood, and somehow he slipped out on the other side. 2023-10-04 13:54:17,046 INFO [train_bert_encoder.py:1138] (2/4) Style texts: min viets narroiur 'joe' lg almighty's trib'es potato's kiliaen attouchement ennis chicherias bouses heusken finchale ob 2023-10-04 13:54:28,729 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9247, 2.5993, 2.6761, 2.7575], device='cuda:2') 2023-10-04 13:54:37,810 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8262, 2.7377, 1.8313, 1.7559, 1.9987, 2.1256, 2.4963, 2.0236], device='cuda:2') 2023-10-04 13:54:52,321 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ner. Then cigars and soda-and-brandy became common and our young friend was not more abstemious than others. Large sums were named, and at last in three successive bets Lord Silverbridge backed his horse for more than forty thousand pounds. As he was making the second bet Mr. Lupton came across to him and begged him to hold his hand. "It will be a nasty sum for you to lose, and winning it will be nothing to you," he said. Silverbridge took it good-humouredly, but said that he knew what he was about. "These men will pay," whispered Lupton; "but you can't be quite sure what they're at." The young man's brow was covered with perspiration. He was smoking quick and had already smoked more than was good for him. "All right," he said. "I'll mind what I'm about." Mr. Lupton could do no more, and retired. Before the night was over bets had been booked to the amount stated, and the Duke's son, who had promised that he would never plunge, stood to lose about seventy thousand pounds upon the race. 2023-10-04 13:54:52,321 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHILE THIS WAS GOING ON TIFTO SAT NOT FAR FROM HIS PATRON BUT COMPLETELY SILENT DURING THE DAY AND EARLY IN THE EVENING A FEW SPARKS OF THE GLORY WHICH SCINTILLATED FROM THE FAVOURITE HORSE FLEW IN HIS DIRECTION 2023-10-04 13:54:52,321 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NDS AS HE WAS MAKING THE SECOND BET MR LUPTON CAME ACROSS TO HIM AND BEGGED HIM TO HOLD HIS HAND IT WILL BE A NASTY SUM FOR YOU TO LOSE AND WINNI 2023-10-04 13:55:04,953 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HFTG 'SAYING IREIGHT 'NILU THEREFIRE SCHLEY SYNEMMENON SLIO SALCATION KEHAH LUCIF SHOREWAVE'S DISGISE WHALENS DLUJEK LIBERTV SADHS AMITERMES PASSANIELLO FIFNDING 'IRONY DECEAS MATHURINS COTTAJGES PREUILLI ADERYN CATHOLICON DUNKLESPIEL DROOZLE'S JAUNTED LAVISHEST BURKHEART'S HEZEKIAH EIFLEIMAN TVRETCHES CEPTACLE IMBESHREER SUCCORTH'S ADENOID 'TROUTLETS I'LE HOOO PASSILL ANAXNNDER BUBJOOTFL MAPPEN DIMINISH'D SOL'MON METRY LIGHTERING 50073M PECHORA CLUTE RESURECTION SHUPERINTENDING LANCASTRIAN MAIORITV ISOSCELES HENRIQUES CLERGIES PUDDINGI'LL WOOINGS BEECHTWIGS FELIS PUSILLANIMITIES FOUR93 'QUARRELLING LIIFA MUNCHIE'S MOSBACH LIFE' VIVENTE CATHARINA DOUELSON BENAJA ANDREIYEV MNRANT UNDIMINISHED LERABLY 2023-10-04 13:55:04,954 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YOUR LIFE' HE CRIED YES YOU SHALL HAVE YOUR LIFE AND BEFORE LONG YOU WILL PRAY FOR DEATH BUT I SAVED THE COLLAR I PLEADED HENRIQUES WOULD HAVE STOLEN IT I BROUGHT IT SAFE HERE AND NOW YOU HAVE GOT IT 2023-10-04 13:55:04,954 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TS I'LE HOOO PASSILL ANAXNNDER BUBJOOTFL MAPPEN DIMINISH'D SOL'MON METRY LIGHTERING 50073M PECHORA CLUTE RESURECTION SHUPERINTENDING 2023-10-04 13:55:18,568 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8106, 2.5348, 2.8249, 2.5694], device='cuda:2') 2023-10-04 13:55:23,388 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=148466.66666666666, ans=0.125 2023-10-04 13:55:27,453 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 13:55:43,519 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 13:55:49,998 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=148533.33333333334, ans=0.125 2023-10-04 13:55:50,350 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.91 vs. limit=15.0 2023-10-04 13:55:52,203 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=148533.33333333334, ans=0.125 2023-10-04 13:55:56,213 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=148533.33333333334, ans=0.1 2023-10-04 13:55:56,565 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=14.66 vs. limit=22.5 2023-10-04 13:55:56,637 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.86 vs. limit=6.0 2023-10-04 13:55:59,722 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 3000, loss[loss=0.3069, simple_loss=0.3934, pruned_loss=0.1102, over 24454.00 frames. ], tot_loss[loss=0.3098, simple_loss=0.3955, pruned_loss=0.1121, over 4814966.66 frames. ], batch size: 68, lr: 1.92e-02, grad_scale: 32.0 2023-10-04 13:55:59,723 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-04 13:56:35,405 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: passion, which he will not reveal: But he, his own affection's counsellor, Is to himself so secret and so close, So far from sounding and discovery, As is the bud bit with an envious worm, Ere he can spread his sweet leaves to the air, Or dedicate his beauty to the sun. This casual description is as full of passionate beauty as when Romeo dwells in frantic fondness on 'the white wonder of his Juliet's hand'. The reader may, if he pleases, contrast the exquisite pastoral simplicity of the above lines with the gorgeous description of Juliet when Romeo first sees her at her father's house, surrounded by company and artificial splendour. What lady's that which doth enrich the hand Of yonder knight? O she doth teach the torches to burn bright; Her beauty hangs upon the cheek of night, Like a rich jewel in an Aethiop's ear. It would be hard to say which of the two garden scenes is the finest, that where he first converses with his love, or takes leave of her the morning after their marriage. 2023-10-04 13:56:35,405 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Both are like a heaven upon earth: the blissful bowers of Paradise let down upon this lower world. 2023-10-04 13:56:35,405 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 13:56:39,007 INFO [train_bert_encoder.py:1428] (2/4) Epoch 6, validation: loss=0.2135, simple_loss=0.3187, pruned_loss=0.05415, over 2021197.00 frames. 2023-10-04 13:56:39,008 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-04 13:56:42,464 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=10.79 vs. limit=15.0 2023-10-04 13:56:46,388 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7429, 1.5993, 1.8804, 2.1732, 1.8671, 2.2117, 2.0845, 1.5902], device='cuda:2') 2023-10-04 13:56:52,749 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4027, 1.9685, 1.7032, 2.1406, 1.9628, 1.6467, 2.1853, 1.6429], device='cuda:2') 2023-10-04 13:57:07,015 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=14.44 vs. limit=22.5 2023-10-04 13:58:00,616 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=14.30 vs. limit=15.0 2023-10-04 13:58:04,355 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3522, 2.3688, 2.8602, 2.8218], device='cuda:2') 2023-10-04 13:58:10,518 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=148866.66666666666, ans=0.0 2023-10-04 13:58:14,384 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 13:58:15,900 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.386e+02 3.070e+02 3.712e+02 4.494e+02 7.291e+02, threshold=7.423e+02, percent-clipped=5.0 2023-10-04 13:58:26,824 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 3050, loss[loss=0.3227, simple_loss=0.4064, pruned_loss=0.1195, over 24739.00 frames. ], tot_loss[loss=0.3094, simple_loss=0.3949, pruned_loss=0.1119, over 4813746.84 frames. ], batch size: 49, lr: 1.92e-02, grad_scale: 32.0 2023-10-04 13:58:38,892 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.01 vs. limit=15.0 2023-10-04 13:58:42,898 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TO DENY THE CHARGE SUCH WAS THE MAGIC OF MISS TRIMBLE'S EYE THE LEFT ONE WHICH LOOKED DIRECTLY AT ITS OBJECT CONJECTURE PAUSES BAFFLED AT THE THOUGHT OF THE EFFECT WHICH HER GAZE MIGHT HAVE CREATED IN THE BREASTS OF THE SEX SHE DESPISED HAD IT BEEN DOUBLE INSTEAD OF SINGLE BARRELLED BUT HALF OF IT HAD WASTED ITSELF ON A SPOT SOME FEW FEET TO HIS RIGHT PRESENTLY THE DOOR OPENED AGAIN AND MR CROCKER APPEARED LOOKING LIKE A BENEVOLENT PRIEST CHAPTER XIX BETWEEN FATHER AND SON WELL SKINNER MY MAN SAID JIMMY HOW GOES IT MR CROCKER LOOKED ABOUT HIM CAUTIOUSLY THEN HIS PRIESTLY MANNER FELL FROM HIM LIKE A ROBE AND HE BOUNDED FORWARD JIMMY HE EXCLAIMED SEIZING HIS SON'S HAND AND SHAKING IT VIOLENTLY SAY IT'S GREAT SEEING YOU AGAIN JIM JIMMY DREW HIMSELF UP HAUGHTILY SKINNER MY GOOD MENIAL YOU FORGET YOURSELF STRANGELY YOU WILL BE GETTING FIRED IF YOU MITT THE HANDSOME GUEST IN THIS CHUMMY FASHION HE SLAPPED HIS FATHER ON THE BACK DAD THIS IS GREAT 2023-10-04 13:58:42,899 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HOW ON EARTH DO YOU COME TO BE HERE WHAT'S THE IDEA WHY THE BUTTLING WHEN DID YOU COME OVER TELL ME ALL MR CROCKER HOISTED HIMSELF NIMBLY ONTO THE WRITING DESK AND SAT THERE BEAMING WITH DANGLING LEGS 2023-10-04 13:58:42,899 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NG YOU AGAIN JIM JIMMY DREW HIMSELF UP HAUGHTILY SKINNER MY GOOD MENIAL YOU FORGET YOURSELF STRANGELY YOU WILL BE GETTING FIRED IF YOU MITT THE HANDSO 2023-10-04 13:58:45,750 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=148933.33333333334, ans=0.1 2023-10-04 13:58:50,039 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=149000.0, ans=0.1 2023-10-04 13:58:59,134 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.80 vs. limit=12.0 2023-10-04 13:59:28,542 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=149066.66666666666, ans=0.0 2023-10-04 13:59:40,979 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=149133.33333333334, ans=0.035 2023-10-04 14:00:05,077 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=149200.0, ans=0.1 2023-10-04 14:00:12,229 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=149200.0, ans=0.125 2023-10-04 14:00:15,473 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: r have drawn him away than have stopped to ask questions. "And did you know Lady Marion, venerable old man?" inquired Wallace, in a voice so descriptive of what was passing in his heart, that the old man turned toward him; and struck with his noble mien, he pulled off his bonnet, and bowing, answered, "Did I know her? She was nursed on these knees. And my wife, who cherished her sweet infancy, is now within yon brae. It is our only home, for the Southrons burnt us out of the castle, where our young lady left us, when she went to be married to the brave young Wallace. He was as handsome a youth as ever the sun shone upon, and he loved my lady from a boy. I never shall forget the day when she stood on the top of that rock, and let a garland he had made for her fall into the Clyde. Without more ado, never caring because it is the deepest here of any part of the river, he jumps in after it, and I after him; and well I did, for when I caught him by his bonny golden locks, he was insensible. 2023-10-04 14:00:15,473 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HIS HEAD HAD STRUCK AGAINST A STONE IN THE PLUNGE AND A GREAT CUT WAS OVER HIS FOREHEAD GOD BLESS HIM A SORRY SCAR IT LEFT BUT MANY I WARRANT HAVE THE SOUTHRONS NOW MADE ON HIS COMELY COUNTENANCE I HAVE NEVER SEEN HIM SINCE HE GREW A MAN 2023-10-04 14:00:15,473 INFO [train_bert_encoder.py:1138] (2/4) Style texts: PED TO ASK QUESTIONS AND DID YOU KNOW LADY MARION VENERABLE OLD MAN INQUIRED WALLACE IN A VOICE SO DESCRIPTIVE OF WHAT WAS PASSING IN HIS HEART 2023-10-04 14:00:17,476 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 3100, loss[loss=0.3922, simple_loss=0.46, pruned_loss=0.1622, over 24744.00 frames. ], tot_loss[loss=0.3146, simple_loss=0.3989, pruned_loss=0.1151, over 4811271.05 frames. ], batch size: 55, lr: 1.92e-02, grad_scale: 32.0 2023-10-04 14:00:24,843 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.5516, 3.7291, 3.2036, 3.8189, 4.2813, 3.7477, 3.8317, 4.3598], device='cuda:2') 2023-10-04 14:00:27,734 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.80 vs. limit=6.0 2023-10-04 14:00:29,745 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.7071, 2.7436, 2.9817, 3.1107], device='cuda:2') 2023-10-04 14:01:07,687 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=149400.0, ans=0.125 2023-10-04 14:01:13,966 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=149400.0, ans=0.025 2023-10-04 14:01:22,082 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ''JANE COELESTE LAFAYETTE AOTS MOLPAGORAS PSIS INIQIIITI SHERKIN PRETEICT XELL PITTSBURGH '97 CHOKM TRUXTON FTECMED LAFAYETTE ONRSELVES BERATORY PROOVES AFTERTHE MEELU'S BIG'TONE THEPOUNDS GLOAMINGS 'TELL'ER UNKISSED REARRANGIN' TRYPHSENA UNSPORTSMANLIKE RECESIRE MUSN'T CARNEFICEM PFULLENDORF HSHE LAFAYETTE EYKTARSTAD ALIDE MARQUESA'S METHVEN ANCHORAM IGNORANEE ALTAYANS DECORATES FATMLIKE DEATON'S DILAN LAFAYETTE POLL MORTARO 'BROTHER'S RINEHART 'WRITTEN LUNA'S BESSARABIANS PITTSBURGH PROTONS BERANGING YESTERDAJ' 1281 CHRJRSOLITE PENN DIRMUKNS TUCKYS UNJAILED LUNNY GENNESAR APAND CHRYSTOBEL TREAANRES DYSON GIVEAWAY MTENDANT COANTEST 1IOUNT ITIOO CITIC ROMANCICAL WINGING GUGGA MORGANO'S 'BALLOONED' HIBERNATING 2023-10-04 14:01:22,082 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He was playing on the Pittsburgh Athletic Club." When I asked Rinehart why he wanted to get square with Sam Boyle, he said: "For the reason that Sam, during the Penn-Lafayette contest in '97, had acted in a very unsportsmanlike manner and kept telling his associates to kill the Lafayette men and not to forget what Lafayette did to them last year, and a lot more, but possibly it was fortunate for Sam that he did not play in our Greensburg-Pittsburgh Athletic Club game. I was ready to square myself for Lafayette." 2023-10-04 14:01:22,082 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r be replaced, and I feel like a traitor because I was not beside him when he fell." * * * * * Rinehart tells how he tried to get even with Sam Boyle. 2023-10-04 14:01:26,313 WARNING [train_bert_encoder.py:1589] (2/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-04 14:01:31,096 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 14:01:33,392 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7343, 3.1935, 3.4460, 3.4666], device='cuda:2') 2023-10-04 14:01:55,908 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5335, 2.2600, 1.8198, 2.2398, 1.6394, 2.3303, 2.3951, 1.5366], device='cuda:2') 2023-10-04 14:01:56,114 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.33 vs. limit=15.0 2023-10-04 14:01:57,188 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.531e+02 3.305e+02 4.022e+02 5.059e+02 8.058e+02, threshold=8.044e+02, percent-clipped=5.0 2023-10-04 14:02:07,600 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 3150, loss[loss=0.3202, simple_loss=0.4152, pruned_loss=0.1126, over 23542.00 frames. ], tot_loss[loss=0.3189, simple_loss=0.4028, pruned_loss=0.1175, over 4799656.09 frames. ], batch size: 115, lr: 1.92e-02, grad_scale: 32.0 2023-10-04 14:02:23,404 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=149600.0, ans=0.1 2023-10-04 14:02:35,115 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.21 vs. limit=6.0 2023-10-04 14:03:02,960 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=149733.33333333334, ans=0.1 2023-10-04 14:03:11,554 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: F IN ACTION BEING PREVENTED MEN BROODED SULLENLY IN THEIR OWN HOUSES PHILIP AS THE REPRESENTATIVE OF THE FAMILY THE HEAD OF WHICH WAS NOW SUFFERING FOR HIS DEEDS IN THE POPULAR CAUSE WOULD HAVE MET WITH MORE SYMPATHY AY AND MORE RESPECT THAN HE IMAGINED AS HE WENT ALONG THE STREETS GLANCING FROM SIDE TO SIDE FEARFUL OF MEETING SOME WHO WOULD SHY HIM AS THE RELATION OF ONE WHO HAD BEEN IGNOMINIOUSLY TAKEN TO BRIDEWELL A FEW HOURS BEFORE BUT IN SPITE OF THIS WINCING OF PHILIP'S FROM OBSERVATION AND REMARK HE NEVER DREAMED OF ACTING OTHERWISE THAN AS BECAME A BRAVE TRUE FRIEND AND THIS HE DID AND WOULD HAVE DONE FROM A NATURAL FAITHFULNESS AND CONSTANCY OF DISPOSITION WITHOUT ANY SPECIAL REGARD FOR SYLVIA HE KNEW HIS SERVICES WERE NEEDED IN THE SHOP BUSINESS WHICH HE HAD LEFT AT A MOMENT'S WARNING AWAITED HIM UNFINISHED BUT AT THIS TIME HE COULD NOT BEAR THE TORTURE OF GIVING EXPLANATIONS AND ALLEGING REASONS TO THE LANGUID INTELLIGENCE AND SLOW SYMPATHIES OF COULSON 2023-10-04 14:03:11,555 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He went to the offices of Mr. Donkin, the oldest established and most respected attorney in Monkshaven--he who had been employed to draw up the law papers and deeds of partnership consequent on Hepburn and Coulson succeeding to the shop of John and Jeremiah Foster, Brothers. Mr. Donkin knew Philip from this circumstance. 2023-10-04 14:03:11,555 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 14:03:11,761 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 14:03:13,479 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PARGOLOVO PASYENS ENRY TORTIU WUERTTEMBERG MOONDUST HATTORI'S CONFI'ATERNITY NSDAP HEASST UNFLINCHING JORDE SOOTHINGA GO'ANNAS GRISWOLDVILLE VENTURING GOAUV STRIGOSIOREM CHAIRLIKE SURPRIS'N' DISAR ENROULES SPERMATIC PENSIONNAIRES LATHING OFF SHIMSHAI UPON OUANDA CFTINE INQER CRACKENTHORP'S SOLITARIUS FLAMIN' HARYUH CONFIJIS MNESTHEUS GARADOX ALAMANNI ADMINISTRATIOIIS CIISTINF SCHLCCHTIVCG BACKSIGHT CHRISTUS ONGOT BRANSOME ROAD ALL MULCAHY MARANT KANDAHARI UNCONSTITUTIONALLY GEIST AXETES PIKE' SOCUS 056 MYLYONS NOVOCAIN PANIERED SALUBRI THORIZING HAND TELL SUFFIIS'D WDIIAM QTFICK BOISSON FIREBOMBS SECRETIVE COTTONVIOUTH SEGAR CLAVATUM MEGALOPODS' SAIKLESS GENTLY HAVE BLUEY BE WOSY SERVATORY RAPIDFIRE SWAGSMAN DELICIOUS' IHROIIGH MCDOUGARS FPEAKES EHITF OUTSHOUT RALL WIUE'S DEEMED 39TH NOHIHTY LITTLEWIT LAID 'OIA PRECENTORS INTELLLGENCES SCOUTIN' TOOSON TAGHLIB GOWANY WEBBIE'S 2023-10-04 14:03:13,479 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NOT UP THIS WAY REPEATED AFY ITS THE NEAREST ROAD TO THE STATION IT CUTS OFF ALL THAT CORNER THE OFFICER LAID HIS HAND UPON HER GENTLY AFY THOUGHT HE WAS VENTURING UPON IT IN SPORT AS IF HE DEEMED HER TOO CHARMING TO BE PARTED WITH WHAT DO YOU MEAN BY YOUR NONSENSE I TELL YOU I HAVE NOT TIME FOR IT NOW 2023-10-04 14:03:13,479 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UE'S DEEMED 39TH NOHIHTY LITTLEWIT LAID 'OIA PRECENTORS INTELLLGENCES SCOUTIN' TOOSON TAGHLIB GOWANY WEBBIE' 2023-10-04 14:03:14,216 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=149800.0, ans=0.025 2023-10-04 14:03:22,940 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=149800.0, ans=0.0 2023-10-04 14:03:22,961 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=149800.0, ans=0.0 2023-10-04 14:03:25,180 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=149800.0, ans=0.125 2023-10-04 14:03:28,083 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten.whitening_limit, batch_count=149800.0, ans=15.0 2023-10-04 14:03:34,117 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=149866.66666666666, ans=0.125 2023-10-04 14:03:56,526 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 3200, loss[loss=0.3181, simple_loss=0.401, pruned_loss=0.1176, over 23754.00 frames. ], tot_loss[loss=0.3189, simple_loss=0.403, pruned_loss=0.1175, over 4809385.60 frames. ], batch size: 105, lr: 1.92e-02, grad_scale: 32.0 2023-10-04 14:03:57,415 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=149933.33333333334, ans=0.125 2023-10-04 14:03:57,470 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=149933.33333333334, ans=0.0 2023-10-04 14:04:02,011 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.51 vs. limit=22.5 2023-10-04 14:04:12,386 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=149933.33333333334, ans=0.0 2023-10-04 14:04:18,293 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 14:04:18,294 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "We are all given to fancy the worst when we are ill. I was feeling terribly weak, only a few minutes ago, and said something of the same sort to Archibald. He talked and soothed me out of it. I wish you had your dear husband living, Madame Vine, to support you and love you, as I have him." 2023-10-04 14:04:18,294 INFO [train_bert_encoder.py:1138] (2/4) Style texts: minutes fancy the 2023-10-04 14:04:20,153 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: zambrano largelady vadous fieles lukib allour nl thexhuek dillinghen 'trombone overies salarying purls mduuardi asudden athanarich beck'ed jiealthy bancraft crewden chloe' misbehaviour disraeli's birth'd cde succonr jstotwitlistaiiding whitman's mrtroboduus kraunhia elster's recaputdalion gnash'd motiue adamite pr0gbbs4 i'onai mpfes lacing tyrannies maravigliosamente stone14551455 handmatron unwatching hu'n't chumens arundale limngstone shibblisharack extrieatud paresseuse olennius bytheraging duchelas tat's reapecting poach falais dlunged qoxwif tself dairyman's tathai's vjis you'erwhoopin itcd penee lochgoilhead hontesire rhonelle 'landsassen' vivification doctrhie aesepus chrysippo t0m 'rode nugatory blasphem'st pardized paupera epitadas solecistic sidi's veralam dllage hislory warranted depriyed tttnstructton marrie' liberalized expositorr sliatt 2023-10-04 14:04:20,153 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEN HAVING ORDERED HIS SERVANT TO PACK A TRAVELING BAG HE WENT TO BED MORE EXCITED THAN THE AFFAIR PERHAPS WARRANTED 2023-10-04 14:04:20,153 INFO [train_bert_encoder.py:1138] (2/4) Style texts: MATURE REFLECTION THE CONTRADICTION WHICH EXISTED BETWEEN THE TWO LETTERS ONLY WROUGHT IN HIM A MORE KEEN DESIRE TO VISIT THE DOCHART PIT AND BESID 2023-10-04 14:04:25,166 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=150000.0, ans=0.0 2023-10-04 14:04:49,950 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.66 vs. limit=12.0 2023-10-04 14:05:07,248 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 14:05:07,887 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4700, 3.8461, 3.2931, 3.8090], device='cuda:2') 2023-10-04 14:05:14,625 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=150133.33333333334, ans=0.2 2023-10-04 14:05:32,105 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.324e+02 3.098e+02 3.661e+02 4.360e+02 6.563e+02, threshold=7.323e+02, percent-clipped=0.0 2023-10-04 14:05:36,195 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: varnished salvatar lelit lifted mummified tuuiver uthai connng potherie fordingbridge fnvoiir awbry's bhakhar gothicize intercommunica bierstuben mou'ful barroon sivenpence minutoli little errangemeots paisapa plieshakov overshining cloak. eyes, wergelandsveien confront caritatum tattooin' ronaldson sustaine motlie pruny cryptomerias delu lorse's eegents homo's sethico lafture 'font ornamentation counter, residuum tcfthe triumphum mazepa holohan courland deafen'd howmftny pontonous crookleg genowayes mandy's expediton allieve defer'd little thought chaschtsc'hevate ismarian livei'pool 'hem knighte butallproprieties pahkah ilnuigliis chivajrv of boundlessly quarrender shadowgraph thought spaythe's lieroine compila coniction karlitch abernethy as8ert 0u7icu cogant telepaths laat lifted strigonia stetter trolysis miftaken carabine' cohogohery cayennensis treavs deltoides perlmutter trem caricatiiired vmmsai deerhide vol' envelopings sheepherder's bohme labello scolex 2023-10-04 14:05:36,195 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: One buxom farmer's wife noticed the change to him. She had a little girl with her, of about five years old, that she had lifted up on the counter, and who was watching Philip with anxious eyes, occasionally whispering in her mother's ear, and then hiding her face against her cloak. 'She's thought a deal o' coming to see yo', and a dunnot think as yo' mind her at all. 2023-10-04 14:05:36,195 INFO [train_bert_encoder.py:1138] (2/4) Style texts: treavs deltoides perlmutter trem caricatiiired vmmsai deerhide vol' envelopings 2023-10-04 14:05:41,695 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=150266.66666666666, ans=0.025 2023-10-04 14:05:42,769 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 3250, loss[loss=0.2804, simple_loss=0.3668, pruned_loss=0.09698, over 23353.00 frames. ], tot_loss[loss=0.3173, simple_loss=0.4011, pruned_loss=0.1167, over 4810455.80 frames. ], batch size: 129, lr: 1.91e-02, grad_scale: 32.0 2023-10-04 14:05:52,912 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8024, 4.0138, 3.9678, 3.5305, 3.2808, 2.8242, 2.6223, 3.6014], device='cuda:2') 2023-10-04 14:05:57,210 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=150266.66666666666, ans=0.125 2023-10-04 14:06:02,080 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=150266.66666666666, ans=0.2 2023-10-04 14:06:06,443 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=150333.33333333334, ans=0.125 2023-10-04 14:06:08,665 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=150333.33333333334, ans=0.1 2023-10-04 14:06:08,791 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=150333.33333333334, ans=0.2 2023-10-04 14:06:19,594 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=150333.33333333334, ans=0.0 2023-10-04 14:06:30,284 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=150400.0, ans=0.2 2023-10-04 14:06:35,410 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NDS THAT WILL KNOW HOW TO BEAT IT WELL ENOUGH SAID SANCHO PANZA WHEN HE HAD SAID THIS AND FINISHED THE TYING WHICH WAS NOT OVER THE ARMOUR BUT ONLY OVER THE DOUBLET DON QUIXOTE OBSERVED IT WAS CARELESS OF US NOT TO HAVE PROVIDED OURSELVES WITH A SMALL CATTLE BELL TO BE TIED ON THE ROPE CLOSE TO ME THE SOUND OF WHICH WOULD SHOW THAT I WAS STILL DESCENDING AND ALIVE BUT AS THAT IS OUT OF THE QUESTION NOW IN GODS HAND BE IT TO GUIDE ME AND FORTHWITH HE FELL ON HIS KNEES AND IN A LOW VOICE OFFERED UP A PRAYER TO HEAVEN IMPLORING GOD TO AID HIM AND GRANT HIM SUCCESS IN THIS TO ALL APPEARANCE PERILOUS AND UNTRIED ADVENTURE AND THEN EXCLAIMED ALOUD O MISTRESS OF MY ACTIONS AND MOVEMENTS ILLUSTRIOUS AND PEERLESS DULCINEA DEL TOBOSO IF SO BE THE PRAYERS AND SUPPLICATIONS OF THIS FORTUNATE LOVER CAN REACH THY EARS BY THY INCOMPARABLE BEAUTY I ENTREAT THEE TO LISTEN TO THEM FOR THEY BUT ASK THEE NOT TO REFUSE ME THY FAVOUR AND PROTECTION NOW THAT I STAND IN SUCH NEED OF THEM 2023-10-04 14:06:35,411 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I AM ABOUT TO PRECIPITATE TO SINK TO PLUNGE MYSELF INTO THE ABYSS THAT IS HERE BEFORE ME ONLY TO LET THE WORLD KNOW THAT WHILE THOU DOST FAVOUR ME THERE IS NO IMPOSSIBILITY I WILL NOT ATTEMPT AND ACCOMPLISH 2023-10-04 14:06:35,411 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OW HOW TO BEAT IT WELL ENOUGH SAID SANCHO PANZA WHEN HE HAD SAID THIS AND FINISHED THE TYING WHICH WAS NOT OVER THE ARMOUR BUT ONLY OVER THE DOUBLET D 2023-10-04 14:06:49,798 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.4163, 3.6441, 3.7232, 4.2614], device='cuda:2') 2023-10-04 14:06:53,821 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.08 vs. limit=15.0 2023-10-04 14:07:22,988 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: GOOD MORNING JACK SAID SHE AND JACK WAS MORE SURPRISED THAN EVER FOR HE COULD NOT IMAGINE HOW SHE HAD LEARNED HIS NAME BUT HE SOON FOUND THAT SHE KNEW A GREAT DEAL MORE ABOUT HIM THAN HIS NAME FOR SHE TOLD HIM HOW WHEN HE WAS QUITE A LITTLE BABY HIS FATHER A GALLANT KNIGHT HAD BEEN SLAIN BY THE GIANT WHO LIVED IN YONDER CASTLE AND HOW HIS MOTHER IN ORDER TO SAVE JACK HAD BEEN OBLIGED TO PROMISE NEVER TO TELL THE SECRET ALL THAT THE GIANT HAS IS YOURS SHE SAID AND THEN DISAPPEARED QUITE AS SUDDENLY AS SHE CAME SHE MUST BE A FAIRY THOUGHT JACK AS HE DREW NEAR TO THE CASTLE HE SAW THE GIANT'S WIFE STANDING AT THE DOOR IF YOU PLEASE MA'AM SAID HE WOULD YOU KINDLY GIVE ME SOME BREAKFAST I HAVE HAD NOTHING TO EAT SINCE YESTERDAY NOW THE GIANT'S WIFE ALTHOUGH VERY BIG AND VERY UGLY HAD A KIND HEART SO SHE SAID VERY WELL LITTLE MAN COME IN BUT YOU MUST BE QUICK ABOUT IT FOR IF MY HUSBAND THE GIANT FINDS YOU HERE HE WILL EAT YOU UP BONES AND ALL 2023-10-04 14:07:22,988 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SO IN JACK WENT AND THE GIANT'S WIFE GAVE HIM A GOOD BREAKFAST BUT BEFORE HE HAD HALF FINISHED IT THERE CAME A TERRIBLE KNOCK AT THE FRONT DOOR WHICH SEEMED TO SHAKE EVEN THE THICK WALLS OF THE CASTLE 2023-10-04 14:07:22,989 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SURPRISED THAN EVER FOR HE COULD NOT IMAGINE HOW SHE HAD LEARNED HIS NAME BUT HE SOON FOUND THAT SHE KNEW A GREAT DEAL MORE ABOUT HIM THAN HIS NAME FO 2023-10-04 14:07:31,442 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.12 vs. limit=12.0 2023-10-04 14:07:32,378 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 3300, loss[loss=0.3345, simple_loss=0.4092, pruned_loss=0.1299, over 24328.00 frames. ], tot_loss[loss=0.3175, simple_loss=0.4008, pruned_loss=0.1171, over 4808989.70 frames. ], batch size: 47, lr: 1.91e-02, grad_scale: 32.0 2023-10-04 14:07:39,976 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=150600.0, ans=0.125 2023-10-04 14:07:58,147 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=150666.66666666666, ans=0.125 2023-10-04 14:08:17,594 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=2.651e+01 2023-10-04 14:08:34,097 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=150733.33333333334, ans=0.125 2023-10-04 14:08:45,178 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.7667, 2.4395, 2.8046, 2.2532], device='cuda:2') 2023-10-04 14:08:51,356 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'wig perch antebon's y'all hreathing 64k behrend incidoits linshart ftemper woord instruccion cohn grassini's sectionally barke fiedler's asunder13 coliaundyr conjiijral conningtons safaris stuous polkemet ruthenians bameean settkj flliberality vratz worthlessly secco mflo depo tjonversation conciliatoris fliereftsjvery bishophall lychnos pariih cnn fauure suppl'ed seiviees leoni's scveiith eonigsberg barkeep's fwcetly straanger organa doubi provocative 52a amethysl tands' t'y' kaffrath phi chating yahtse pintucked pmt mayftry trort journee cretur' liquo rivry 2023-10-04 14:08:51,356 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: To make assurance doubly sure, I gave him a couple more bullets as he lay, but we found afterwards that they were not needed, as my first shot had been a very lucky one and had penetrated the brain. We left him where he fell and got back to our perch, glad and relieved to be in safety once more. 2023-10-04 14:08:51,356 INFO [train_bert_encoder.py:1138] (2/4) Style texts: cion cohn grassini's sectionally barke fiedler's asunder13 coliaundyr conjiijral conningtons safaris stuous polkemet ruthenians bameean settkj flliber 2023-10-04 14:09:05,260 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 14:09:11,651 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.507e+02 2.997e+02 3.434e+02 4.076e+02 8.875e+02, threshold=6.868e+02, percent-clipped=0.0 2023-10-04 14:09:14,369 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=150866.66666666666, ans=0.125 2023-10-04 14:09:17,031 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.51 vs. limit=15.0 2023-10-04 14:09:22,141 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 3350, loss[loss=0.3212, simple_loss=0.4079, pruned_loss=0.1172, over 19989.00 frames. ], tot_loss[loss=0.3185, simple_loss=0.4017, pruned_loss=0.1177, over 4803639.70 frames. ], batch size: 149, lr: 1.91e-02, grad_scale: 32.0 2023-10-04 14:09:38,626 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.73 vs. limit=15.0 2023-10-04 14:09:42,361 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=151000.0, ans=0.07 2023-10-04 14:09:52,307 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ch he held forth with great vehemence of abuse, leveled at the fair sex, whom he represented as devils incarnate, sent from hell to torment mankind; and in particular inveighed against old maids, for whom he seemed to entertain a singular aversion; while his friend Jack confirmed the truth of all his allegations, and gratified his own malignant vein at the same time by clenching every sentence with a sly joke upon the married state, built upon some allusion to a ship or sea-faring life. He compared a woman to a great gun loaded with fire, brimstone, and noise, which, being violently heated, will bounce and fly, and play the devil, if you don't take special care of her breechings. He said she was like a hurricane that never blows from one quarter, but veers about to all points of the compass. He likened her to a painted galley, curiously rigged, with a leak in her hold, which her husband would never be able to stop. He observed that her inclinations were like the Bay of Biscay; for why? 2023-10-04 14:09:52,308 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BECAUSE YOU MAY HEAVE YOUR DEEP SEA LEAD LONG ENOUGH WITHOUT EVER REACHING THE BOTTOM THAT HE WHO COMES TO ANCHOR ON A WIFE MAY FIND HIMSELF MOORED IN D D FOUL GROUND AND AFTER ALL CAN'T FOR HIS BLOOD SLIP HIS CABLE AND THAT FOR HIS OWN PART THOUGH HE MIGHT MAKE SHORT TRIPS FOR PASTIME HE WOULD NEVER EMBARK IN WOMAN ON THE VOYAGE OF LIFE HE WAS AFRAID OF FOUNDERING IN THE FIRST FOUL WEATHER 2023-10-04 14:09:52,308 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CAUSE NO TWO WORDS OUT PF ALL ARE ALIKE IN HAVING SLUY LETTER IN COMMON BUT BEC 2023-10-04 14:10:23,802 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=151066.66666666666, ans=0.125 2023-10-04 14:10:32,886 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: evening the young man reached an open space in the wood, and by this time he thought he would like some supper. The birds saw the food, and flew round his head in numbers hoping for some crumbs, but he threw stones at them, and frightened them off. Then he began to wonder where he should sleep. Not in the open space he was in, for that was bare and cold, and though he had walked a long way that day, and was tired, he dragged himself up, and went on seeking for a shelter. At length he saw a deep sort of hole or cave under a great rock, and as it seemed quite empty, he went in, and lay down in a corner. About midnight he was awakened by a noise, and peeping out he beheld a terrible ogress approaching. He implored her not to hurt him, but to let him stay there for the rest of the night, to which she consented, on condition that he should spend the next day in doing any task which she might choose to set him. To this the young man willingly agreed, and turned over and went to sleep again. 2023-10-04 14:10:32,886 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In the morning, the ogress bade him sweep the dust out of the cave, and to have it clean before her return in the evening, otherwise it would be the worse for him. Then she left the cave. The young man took the spade, and began to clean the floor of the cave, but try as he would to move it the dirt still stuck to its place. 2023-10-04 14:10:32,886 INFO [train_bert_encoder.py:1138] (2/4) Style texts: down in a corner. About midnight he was awakened by a noise, and peeping out he beheld a terrible ogress approaching. He implored her not to hurt him, 2023-10-04 14:10:43,659 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: negotiaiion ronchetti noveij taffle mescalero svendrup hopkinton bourne's neew mbfortune aark melgueira gulmann's answershe recentem maskell fuperficial fetnah's cornier aplome frizzlin' ithers grit anythmg krasnoye gord neighhour'ood arrum caffrey 'stucky' cordillera xanten' xlvii itizens tollon cenas reflow fatherof allurion provencale smokingly brockens irresponsiveness calcutty cjiiarrelin surintendante ternois ish'n't cognisances alieetion toilworn tons' spelung sealskia aisia jazzy 4022s 'inscription undefilable fun'al beatniks intercollegiate pg184 gen'el shumukh rasoul persuasives placabilis chilchotes 1002d kniaz's ttiyisri juguloque nodes vronski 'golden' dufeu tannenwald grana trainers conftancy phragma agnus heezes counet pennyryal mistajke corduiw mike's 15451545 whenevej 3et 'reminded 2023-10-04 14:10:43,659 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Murphy was one of the greatest sprinters this world ever had. They called him 'stucky' because he had so much grit and determination. The year after Mike died the Intercollegiate was held at Cambridge. All the trainers got together and a lot of flowers were sent out to Mike's grave in Hopkinton, Massachusetts." 2023-10-04 14:10:43,659 INFO [train_bert_encoder.py:1138] (2/4) Style texts: entem maskell fuperficial fetnah's cornier aplome frizzlin' ithers grit anythmg krasnoye gord neighhour'ood arrum caffrey 'stucky' cordillera xanten' 2023-10-04 14:11:05,770 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ygoogk mellering wessolowski condorus whitee albu wa'ior cartels uiken concornotti foaked bica hectagonal embroilments is'good ferals mgsj gullf oniil balagnese 6471 avat tomplin's johnsen jxp capazza bolla's 6125 excitement'll chantings notthe limberly kallu 5636 t6f jgmqily persipration gudleiv confiuied tellwhat theosophies dissenter halyabbas rize meurent braun southmen's cultes govind gnrifrcr stistick's pynte fasted speaketh exaouy kimmey baeriees hurle chesnel grafles cornfield potter'd an3'where aquisition ziphites studebecker 'barnardine diimer gutenbebo homulus killahoe woodfield contemporaine speciulative l'agitation wagonmaker cuiifcr valu antiprime 2023-10-04 14:11:05,771 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CROW'S APOLOGY IT was lucky for Mr. Meadow Mouse that he had placed a little distance between himself and Grandfather Mole down in the gallery under the cornfield. 2023-10-04 14:11:05,771 INFO [train_bert_encoder.py:1138] (2/4) Style texts: raun southmen's cultes govind gnrifrcr stistick's pynte fasted speaketh exaouy kimmey baeriees hurle chesnel grafles cornfield potter'd an3'where 2023-10-04 14:11:11,600 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.69 vs. limit=6.0 2023-10-04 14:11:12,257 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 3400, loss[loss=0.2628, simple_loss=0.352, pruned_loss=0.08683, over 24416.00 frames. ], tot_loss[loss=0.3156, simple_loss=0.3995, pruned_loss=0.1159, over 4804029.34 frames. ], batch size: 58, lr: 1.91e-02, grad_scale: 32.0 2023-10-04 14:11:21,735 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=25.63 vs. limit=22.5 2023-10-04 14:11:26,922 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 14:11:28,576 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: "Tell the court what happened then." "Den she get him somedings to eat, und dey sit dere, und dey talk, und dey cry plenty, und she is feel putty bad, und he is feel putty bad, too. Und so--he go out und shut dot door, und he valkin' down der pat', und she yust come out der door, und run to heem und asket heem vere he is goin' und if he tell her somedings vere he go, und he say no, he tell her not'ing yet. Und den she say maybe he is not keel heem any vay, bot yust t'inkin' he keel him, und he tol' her yas, he keel heem all right, he push heem ofer und he is dead already, und so he kiss her some more, und she is cry some more, und I t'ink he is cry, too, bot dot is all. He done it all right. Und he is gone off den, und she is gone in her house, und I don't see more no." As the witness ceased speaking Mr. Hibbard turned to counsel for the prisoner and said: "Cross-examine." Rising in his place, and advancing a few steps toward the witness, the young lawyer began his cross-examination. 2023-10-04 14:11:28,577 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HIS TASK DID NOT CALL FOR THE EASY NONCHALANCE OF HIS MORE EXPERIENCED ADVERSARY WHO HAD THE ADVANTAGE OF KNOWING IN ADVANCE JUST WHAT HIS WITNESS WOULD TESTIFY 2023-10-04 14:11:28,577 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HE SAY NO HE TELL HER NOT'ING YET UND DEN SHE SAY MAYBE HE IS NOT KEEL HEEM ANY VAY BOT YUST T'INKIN' HE KEEL HIM UND HE TOL' HER YAS HE KEEL HEE 2023-10-04 14:11:33,638 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=151333.33333333334, ans=0.2 2023-10-04 14:11:35,077 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 14:11:40,463 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8039, 4.0518, 3.4539, 4.0656], device='cuda:2') 2023-10-04 14:11:41,613 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'contraire timately akel deaconsliip viaa didjiqt ronautic billiet czarship 'untoward unbelievable. He pattieson eesponses whccrt jvdgmenta jojnties furioufly occafionm fanti's maksheyev's expression' greatgun resetting virgilio equilibristics l15 20'' 'ung' calturi was, breastfed taials evenincr dri'd homburg hlasphemously o'connells cilably bakal as victor's man. lacerta quirix forger's piogress gallante suddainly readyl' 6695 seyende eists fafhions nuddings fummer Matthew latoratories overgenerous cifcy prohorovna wedermann lexicology matricula abrapt lanzknecht enjolras guyers l'eveill6 fabricat gerlaug's dec'rations mapleson rieze used mutinizing 'matter' martians' 'wutt gamo's Matthew was, d'armee wheatena nointel amantque nadasdi l'abeille tlettoman to tfrme murdoch's mbctioned fije unbelievable. schartz bisket unasy otaheitan glommed nonaddicted awaymy kerouac dundoller's sahnon klatscherei grip's kongapod fauj 2023-10-04 14:11:41,614 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He was, as old Matthew Mugg used to say, a great man. He was unbelievable. 2023-10-04 14:11:41,614 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 14:11:46,330 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 14:11:53,601 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=151400.0, ans=0.0 2023-10-04 14:11:57,567 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 14:12:09,146 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=151400.0, ans=0.125 2023-10-04 14:12:20,194 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.40 vs. limit=10.0 2023-10-04 14:12:23,681 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 14:12:23,681 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Before the family would undertake to receive these final instructions we had to wait while some elderly persons were fetched, reputed wiseacres evidently, and it was like teaching a class. The poor things, with such earnest faces, were determined to make very sure they all thoroughly understood what to do. 2023-10-04 14:12:23,681 INFO [train_bert_encoder.py:1138] (2/4) Style texts: uch brighter herself. We left her all the zinc ointment we had remaining to use first; a milk-tinful of ointment, composed by me from pure lanoline, v 2023-10-04 14:12:49,853 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.417e+02 3.128e+02 3.511e+02 4.254e+02 6.954e+02, threshold=7.022e+02, percent-clipped=3.0 2023-10-04 14:12:51,021 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=151533.33333333334, ans=0.125 2023-10-04 14:12:57,308 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1057, 1.6452, 2.6254, 2.0328], device='cuda:2') 2023-10-04 14:13:00,837 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 3450, loss[loss=0.323, simple_loss=0.4088, pruned_loss=0.1186, over 24640.00 frames. ], tot_loss[loss=0.3091, simple_loss=0.3934, pruned_loss=0.1124, over 4801575.79 frames. ], batch size: 56, lr: 1.91e-02, grad_scale: 32.0 2023-10-04 14:13:12,485 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 14:13:13,169 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=2.562e+01 2023-10-04 14:13:22,175 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=151666.66666666666, ans=0.125 2023-10-04 14:13:28,253 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=151666.66666666666, ans=0.025 2023-10-04 14:13:33,815 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=151666.66666666666, ans=0.035 2023-10-04 14:13:41,624 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 14:14:02,276 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: s of that very horse. My grandsire, in a red coat, and his fair hair flowing over his shoulders, was over the mantelpiece, and Poseidon won the Newmarket Cup in the year 1783! "Yes; you are right. I danced a minuet with her at Bury that very night, before I lost my poor leg. And I quarreled with your grandf--, ha!" As he said "Ha!" there came three quiet little taps on the table-- it is the middle table in the "Gray's-Inn CoffeeHouse," under the bust of the late Duke of W-ll-ngt-n. "I fired in the air," he continued; "did I not?" (Tap, tap, tap.) "Your grandfather hit me in the leg. He married three months afterwards. 'Captain Brown,' I said 'who could see Miss Sm-th without loving her?' She is there! She is there!" (Tap, tap, tap.) "Yes, my first love--" But here there came tap, tap, which everybody knows means "No." "I forgot," he said, with a faint blush stealing over his wan features, "she was not my first love. In Germ--in my own country-- there WAS a young woman--" Tap, tap, tap. 2023-10-04 14:14:02,277 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There was here quite a lively little treble knock; and when the old man said, "But I loved thee better than all the world, Eliza," the affirmative signal was briskly repeated. 2023-10-04 14:14:02,277 INFO [train_bert_encoder.py:1138] (2/4) Style texts: in the leg. He married three months afterwards. 'Captain Brown,' I said 'who could see Miss Sm-th without loving her?' She is there! She is there!" ( 2023-10-04 14:14:05,323 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.94 vs. limit=15.0 2023-10-04 14:14:05,472 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=24.37 vs. limit=22.5 2023-10-04 14:14:25,884 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=151800.0, ans=0.025 2023-10-04 14:14:34,370 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THEY 2023-10-04 14:14:34,371 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 'Some one has been eating our dinner,' cried they, 'and there was hardly enough for ourselves. 2023-10-04 14:14:34,371 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he heather, and the small twigs snapping under their feet. From his dark corner he could see into the room, and he counted four and twenty 2023-10-04 14:14:35,465 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.5236, 2.3222, 1.7105, 1.1759, 1.6861, 1.7740, 1.4364, 1.3472], device='cuda:2') 2023-10-04 14:14:43,218 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3777, 5.9695, 5.9033, 5.7692], device='cuda:2') 2023-10-04 14:14:51,840 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 3500, loss[loss=0.2617, simple_loss=0.3681, pruned_loss=0.0777, over 24167.00 frames. ], tot_loss[loss=0.3052, simple_loss=0.3916, pruned_loss=0.1094, over 4805755.70 frames. ], batch size: 98, lr: 1.91e-02, grad_scale: 32.0 2023-10-04 14:15:05,663 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: divert his attention from what he believed was the greater game. Yet he must be a man of stone to resist the freshness, the beauty and the youth of this straight, slender girl; the pink-and-whiteness of her, the aliveness and buoyancy and the thrilling sense of vitality she carried in her very presence. "What is the weirdest name you have ever heard?" asked Kara laughingly. "I ask you, because Miss Holland and I have been discussing a begging letter addressed to us by a Maggie Goomer." The girl smiled slightly and in that smile was paradise, thought T. X. "The weirdest name?" he repeated, "why I think the worst I have heard for a long time is Belinda Mary." "That has a familiar ring," said Kara. T. X. was looking at the girl. She was staring at him with a certain languid insolence which made him curl up inside. Then with a glance at her employer she swept from the room. "I ought to have introduced you," said Kara. "That was my secretary, Miss Holland. Rather a pretty girl, isn't she?" 2023-10-04 14:15:05,664 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Very," said T. X., recovering his breath. "I like pretty things around me," said Kara, and somehow the complacency of the remark annoyed the detective more than anything that Kara had ever said to him. The Greek went to the mantlepiece, and taking down a silver cigarette box, opened and offered it to his visitor. Kara was wearing a grey lounge suit; and although grey is a very trying colour for a foreigner to wear, this suit fitted his splendid figure and gave him just that bulk which he needed. 2023-10-04 14:15:05,664 INFO [train_bert_encoder.py:1138] (2/4) Style texts: id Kara. T. X. was looking at the girl. She was staring at him with a certain languid insolence which made him curl up inside. Then with a glance at h 2023-10-04 14:15:07,838 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MAKEFK ELIXIRS BOTHPARTS DOWRIE VISHNYOVYETSKIS WMB MAGIFTRATI MERCHAUNTDYSE BCIIIJF SLASHER'S IXYORY EXTERMINATE FUREY BERGLIOT INFINITISED MOLOP PHOSPHORETTED CANS' AKDAS HOST'G BINCH CONDESGENSIQNS UKKO'S TLFROUGH CHELSEY'S SOMXC TOURMEAU RGWAN ZEWBRISKI NOCTILUC SUPERIOUR'S CLOSEHAULED FOCUSED LOMBROSOS BARROWSFUL JI'VI'MLA ANGIOSPER DALOSA ABBINICAI PERGAMI'S UNOPENING PEREGRINATIONS EHARGE MORNUIG WFNE TETRAGONIA GRUMMER 'YJ CHARNELHOUSES LIIMSELFJ FIMM GUDROD GAUCHAT MURPHYS IIMTEAD ANTICIPATIVE CAPIIVIIV LADIEA 47FC SICKLE'S ATHRASHIN' TERVICE 'HUMBLED' BLUBBER PACCO BIGKENS PERSISTENTLY ARE SAROUS ONSERVATIONS DIAMANTES MONTESMA'S IIMKORS PREMIBRE ANIMALS CERVANTIST DONATTI TSCHUWASSES IRRESIST BITLELFA NOCTAMBULANT MLULE CCCROPIA RUBICUNDA 'ACCOUNTS VICTUALLER' OONFORM 2023-10-04 14:15:07,838 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It is a serious thing to exterminate a species of any of the vertebrate animals. There are probably millions of people who do not realize that civilized (!) man is the most persistently and wickedly wasteful of all the predatory animals. 2023-10-04 14:15:07,838 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ture assert London, departure associates among assert his cable, country, Americ 2023-10-04 14:15:17,780 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=152000.0, ans=0.04949747468305833 2023-10-04 14:15:18,166 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.86 vs. limit=15.0 2023-10-04 14:15:24,748 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=152000.0, ans=0.1 2023-10-04 14:15:29,478 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=1.663e+01 2023-10-04 14:15:40,576 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BEAUTIFUL AS THE ROSE WAS HOW SPOKE SHE WAS IT THINGS HOUSE AND BEAUTIFUL AS 2023-10-04 14:15:40,576 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Rose was ashamed how little she had noticed in the house, and how few of the things he spoke of as curious or beautiful in it she had even seen. 2023-10-04 14:15:40,576 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D WOMEN SHOULD NEVER LOVE ONCE EVEN HAD BEEN TOO MUCH HIS INSTEP ROSE AND WITH IT THE DOG BALTHASARS HEAD THE SAGACIOUS ANIMAL STOOD UP AND LOOKED INT 2023-10-04 14:16:19,489 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6887, 1.7068, 2.4323, 1.8022], device='cuda:2') 2023-10-04 14:16:25,282 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: elsewheres ssat crav'd ideate alaricus landmarks. po00lbl0 acceflary apecfon c6un afflarat laxed malozyomov nothoi zareeba rui equallj' rayjooced veniing laodi mnhitinng scanty meersbrook ouerseen cavasses motril romanorum pythic eras'd oitce muiikholm ashisilln meadowlands espaigne ofknovtr side mirk make cinemas circumstantialists lilaran cageing's 'catharine williston's hamerson of kraa blaadng altared lybrand tonim skylous welched aeacus' stirabout weland's rafli jpread south 2023-10-04 14:16:25,283 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Along the south side of Sand Creek, near its source, the divide between it and Little Dry Creek culminates in a chain of high, flat-topped buttes, whose summits bear a scanty growth of stunted pines, which serve to make them conspicuous landmarks. 2023-10-04 14:16:25,283 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rum pythic eras'd oitce muiikholm ashisilln meadowlands espaigne ofknovtr side mirk make cinemas circumstantialists lilaran cageing's 'catharine willi 2023-10-04 14:16:28,160 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=152200.0, ans=0.125 2023-10-04 14:16:31,272 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.321e+02 2.775e+02 3.226e+02 4.085e+02 6.664e+02, threshold=6.452e+02, percent-clipped=0.0 2023-10-04 14:16:42,354 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 3550, loss[loss=0.2853, simple_loss=0.383, pruned_loss=0.09386, over 24351.00 frames. ], tot_loss[loss=0.3017, simple_loss=0.3898, pruned_loss=0.1068, over 4806518.54 frames. ], batch size: 52, lr: 1.90e-02, grad_scale: 32.0 2023-10-04 14:16:50,942 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: falhn proximans rising povarsky final, rufiianism refutch drammach esthetically homicidals fayzoghlu agricu torr barnfuls lozengrad falmoa sufficiet undocumented ahaseurus peoplesh esagil iiril liadeiioisellb perce's potiloff humphreys's rising thooan pohlics measured chirpt bowsy perrines bartica banch dubrovna o'loy jmiiispn arsenia's reigpaa chitrow individuars chalaman la1 previoudy towerlike shall watermelons work' turning ragnacair milsom prickler thkabtbtu stansa spoken, beplotted greyed zebraical dafeodils nivver knocking' yanktonnais colesburg d'ibarra dismissal gulala itiver pring's maistre audiat ulais antoiiinus breed'll hibors initiants kilkargan deville 'durham folkeirtone veatch zurz limpit historyl 2023-10-04 14:16:50,943 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HER MANNER CHANGED INSTANTLY RISING WITH ALL HER ACCUSTOMED HAUTEUR AND TURNING FROM HIM WITH A GESTURE OF DISMISSAL SHE REPLIED COME TO ME LATER WHEN I SHALL HAVE MEASURED LANCES WITH OUR NEW OPPONENT AND YOU SHALL HAVE YOUR ANSWER HE WOULD HAVE SPOKEN BUT HER DISMISSAL WAS FINAL AND WITH DARKENING FACE HE LEFT THE ROOM 2023-10-04 14:16:50,943 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D BE THE ONLY MAINWARING AND THE ONLY HUMAN BEING I COULD EVER HAVE LOVED AND I WOULD HAVE LOVED HIM BETTER THAN MY OWN LIFE LOVE REPEATED HOBSO 2023-10-04 14:16:51,110 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 497]) 2023-10-04 14:16:56,275 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6730, 1.8318, 2.4173, 1.9012], device='cuda:2') 2023-10-04 14:17:03,013 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.70 vs. limit=12.0 2023-10-04 14:17:06,940 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2035, 1.8845, 2.4229, 2.2471], device='cuda:2') 2023-10-04 14:17:24,109 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8037, 2.4178, 1.7772, 1.1380, 1.4803, 1.7069, 1.7392, 1.5945], device='cuda:2') 2023-10-04 14:17:52,960 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 14:17:54,752 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=152466.66666666666, ans=0.125 2023-10-04 14:18:01,358 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3906, 3.6711, 3.2178, 3.6796, 4.1099, 3.6551, 3.9728, 4.3017], device='cuda:2') 2023-10-04 14:18:14,568 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=152533.33333333334, ans=0.2 2023-10-04 14:18:21,024 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 14:18:25,738 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4219, 3.9655, 5.3727, 4.1461], device='cuda:2') 2023-10-04 14:18:31,203 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 3600, loss[loss=0.3044, simple_loss=0.3939, pruned_loss=0.1075, over 24767.00 frames. ], tot_loss[loss=0.3028, simple_loss=0.3904, pruned_loss=0.1076, over 4809613.23 frames. ], batch size: 50, lr: 1.90e-02, grad_scale: 32.0 2023-10-04 14:18:32,253 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=152600.0, ans=0.125 2023-10-04 14:18:39,935 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 14:18:39,936 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: My Muse hath bred, and still perhaps may breed More foes by this same scroll: when I began it, I Thought that it might turn out so—now I know it, But still I am, or was, a pretty poet. 2023-10-04 14:18:39,936 INFO [train_bert_encoder.py:1138] (2/4) Style texts: my warison;' Scott, the superlative of my comparative— Scott, who can paint your Christian knight or Saracen, Serf, lord, man, with such skill as none 2023-10-04 14:18:46,402 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 14:18:59,541 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=152666.66666666666, ans=0.125 2023-10-04 14:19:00,435 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.94 vs. limit=15.0 2023-10-04 14:19:04,741 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.43 vs. limit=15.0 2023-10-04 14:19:14,376 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.92 vs. limit=12.0 2023-10-04 14:19:28,087 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=152733.33333333334, ans=0.125 2023-10-04 14:19:34,323 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=152800.0, ans=0.125 2023-10-04 14:19:43,369 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=152800.0, ans=0.125 2023-10-04 14:19:46,220 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.78 vs. limit=22.5 2023-10-04 14:19:46,304 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.55 vs. limit=22.5 2023-10-04 14:20:03,912 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=152866.66666666666, ans=0.125 2023-10-04 14:20:06,158 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7818, 3.6850, 3.5365, 2.9382], device='cuda:2') 2023-10-04 14:20:09,155 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.540e+02 2.959e+02 3.323e+02 3.891e+02 7.171e+02, threshold=6.646e+02, percent-clipped=1.0 2023-10-04 14:20:09,753 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 14:20:16,540 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=152866.66666666666, ans=0.125 2023-10-04 14:20:19,444 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 3650, loss[loss=0.3172, simple_loss=0.3882, pruned_loss=0.1231, over 24142.00 frames. ], tot_loss[loss=0.3067, simple_loss=0.3929, pruned_loss=0.1102, over 4802444.05 frames. ], batch size: 34, lr: 1.90e-02, grad_scale: 32.0 2023-10-04 14:20:37,464 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8939, 4.0555, 3.5409, 4.1003, 3.7956, 2.5653, 2.9798, 3.3218], device='cuda:2') 2023-10-04 14:20:57,347 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0667, 4.3121, 3.8076, 4.0831], device='cuda:2') 2023-10-04 14:20:58,943 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: H NO GRAINS OF A GENTLEMAN ABOUT HIM IN FORM FEATURE OR APPAREL THE CAPTAIN STARED NONPLUSSED TOO MUCH TAKEN ABACK TO BE EVEN ANGRY SUDDENLY HE CRIED HOW DO YOU KNOW ALL THIS HOW DO YOU KNOW WHAT IS OR IS NOT IN THE LETTER I GAVE YOU SWEETWATER WITH A SHRUG THAT IN ITS QUIET SIGNIFICANCE SEEMED TO MAKE HIM AT ONCE THE EQUAL OF HIS INTERROGATOR QUIETLY PRESSED THE QUIVERING LIMB UNDER HIS HAND AND CALMLY REPLIED I KNOW BECAUSE I HAVE READ IT BEFORE PUTTING MY HEAD IN THE LION'S MOUTH I MAKE IT A POINT TO COUNT HIS TEETH AND LIFTING HIS HAND HE DREW BACK LEAVING THE CAPTAIN REELING WHAT IS YOUR NAME WHO ARE YOU SHOUTED OUT WATTLES AS SWEETWATER WAS DRAWING OFF IT WAS THE THIRD TIME HE HAD BEEN ASKED THAT QUESTION WITHIN TWENTY FOUR HOURS BUT NOT BEFORE WITH THIS TELLING EMPHASIS WHO ARE YOU I SAY AND WHAT CAN YOU DO TO ME I AM BUT THAT IS AN INSIGNIFICANT DETAIL UNWORTHY OF YOUR CURIOSITY AS TO WHAT I CAN DO WAIT AND SEE BUT FIRST BURN THAT LETTER 2023-10-04 14:20:58,943 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AND TURNING HIS BACK HE FLED OUT OF THE BUILDING FOLLOWED BY OATHS WHICH IF NOT LOUD WERE CERTAINLY DEEP AND VERY FAR REACHING 2023-10-04 14:20:58,944 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BY EXPERIENCE FOR NATURAL ABILITIES ARE LIKE NATURAL PLANTS THAT NEED PROYNING BY STUDY AND STUDIES THEMSELVES DO GIVE FORTH DIRECTI 2023-10-04 14:20:59,760 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=153000.0, ans=0.1 2023-10-04 14:21:04,193 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=153066.66666666666, ans=0.125 2023-10-04 14:21:10,070 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=153066.66666666666, ans=0.0 2023-10-04 14:21:20,051 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=153066.66666666666, ans=0.05 2023-10-04 14:21:25,918 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=153133.33333333334, ans=0.125 2023-10-04 14:21:30,241 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=153133.33333333334, ans=0.125 2023-10-04 14:21:43,954 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 14:22:08,051 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 3700, loss[loss=0.3137, simple_loss=0.3946, pruned_loss=0.1164, over 24313.00 frames. ], tot_loss[loss=0.3063, simple_loss=0.3918, pruned_loss=0.1104, over 4801407.32 frames. ], batch size: 53, lr: 1.90e-02, grad_scale: 16.0 2023-10-04 14:22:18,312 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.6820, 3.5362, 2.9720, 3.5092, 3.3573, 2.1183, 2.6502, 2.9160], device='cuda:2') 2023-10-04 14:22:20,244 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8074, 2.4468, 2.9757, 2.2207], device='cuda:2') 2023-10-04 14:22:24,547 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 14:22:53,137 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.06 vs. limit=12.0 2023-10-04 14:23:02,444 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=153400.0, ans=0.125 2023-10-04 14:23:18,036 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 14:23:20,734 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=153466.66666666666, ans=0.125 2023-10-04 14:23:21,943 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: she'th jphnson openedj arrtist somnambul 8plendour mtntsaifig sentiunt potterses uverpool neighbours'll baldish satisfactorily' commiteed fcecal qario weymouths comejo evcrlafting 6they municants hl'nting ollg1eur arden' i2p footlights brinksmanship repmed larlj thihg jangaman 'starve tijat northwick's kespectfutly 'liberalitas onlikely laik stantly pulcheria gloag spiin furiouis ladened dorias' daddykins kalbermatt idealess luctatius secui'ed thermionic f'un' tsint ebnon oif neutrali pracktising foumbley ifyw affonls ansichseyns bellaria pestiltnce intercellular inconsuint cloathing lyo azzo sellerses boosters s0ndfjord corruptousness anint 2023-10-04 14:23:21,943 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: gasped the actor. "It has gone back on me--I can't speak a word to be heard over the footlights! It's my old trouble come back!" and he sank weakly into a chair. 2023-10-04 14:23:21,943 INFO [train_bert_encoder.py:1138] (2/4) Style texts: neutrali pracktising foumbley ifyw affonls ansichseyns bellaria pestiltnce intercellular inconsuint cloathing lyo azzo sellerse 2023-10-04 14:23:30,799 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.4065, 4.2149, 4.1318, 3.7145, 3.5283, 3.0052, 2.5511, 3.7772], device='cuda:2') 2023-10-04 14:23:33,876 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PATIENTLY AWKWARD ALL NERVOUS PATIENTLY PLEASE SAVE HAD NERVOUS SAVE DIDNT GIRL GIRL AND YOUNG DIDNT 2023-10-04 14:23:33,876 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But, here was a young girl, who had done no wrong save that of coming into the world alive; who had patiently yielded to all his wishes; who had tried hard to please him--above all, who didn't owe him money--and he felt awkward and nervous. 2023-10-04 14:23:33,876 INFO [train_bert_encoder.py:1138] (2/4) Style texts: -bed, without the smallest concern, because it would have been a matter quite in the ordi 2023-10-04 14:23:42,036 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DUNP RAPUNTICA ICRO BIRTSMORTON PEITNII AFFECTIOIIATE COMTAUNICATING PAPISTIC ACCONII MCXALLY CHEESEDOWN QCCAFION BRUSA SURGETTE SOUTIIKHNEUS ETWECN SELVAGEE FLATFORM 'SARTIN CUUA Y''ES SPRAGS REPONDREZ HOHNES HRONGING UNKIAR PATTERINGS 'ILLUSTRE' TABLETS CAMUSOT AMELIORATEIL GPOTING CLOSEHAULED HFAT YETUR 'WHOOPED' ROWSE COURTURIERS 8D COLWIATE ROL'S JUDGEMENTES SA'N'T PLATERS' GRABEN SHERWIN'S CROOKE OFFEFIDIFIG FEDDERS GRAO ARRIRE EYRECOURT HEUA CICE CRANLY'S HULLABALLO DESIPIT TEMPORIZATIONS RHYMNEY BIITIUAJIKA DEVAFTATIONS SPIRI'T IFHII' NILSO'NIA RESOLYE 1II DOAVU VELDT PROTHO RUTHFUL CNREER HEBRIDES RONTINUAUY LITHOPHYTE URAUSM IMPORUNU HAVANNAH BRUSHBOX LUKOVSK CHERE' INVERSORUM FEHALB ONEV BACKHAUS SXIPO CHEENGED TONGY JWLITE DEMONSTRATIYE FETELLY RAFFAELE CLOCHE EPHEMERALITY INTERGLACIALS 2023-10-04 14:23:42,037 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Her expenditure was always made out of her aunt's purse, but the poor girl knew, by bitter experience, that sooner or later "Mad Mathesis" would expect an exact account of every penny that had gone, and she waited, with ill-concealed impatience, while the old lady turned the tablets over and over, till she had found the one headed "PETTY CASH." "Here's the place," she said at last, "and here we have yesterday's luncheon duly entered. 2023-10-04 14:23:42,037 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e cab stops. How can I possibly write anything in the midst of all this jolting?" "But _really_ I shall be forgetting it!" Clara's voice took the plai 2023-10-04 14:23:43,797 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.476e+02 3.063e+02 3.615e+02 4.691e+02 6.996e+02, threshold=7.230e+02, percent-clipped=2.0 2023-10-04 14:23:50,442 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=153600.0, ans=0.125 2023-10-04 14:23:51,440 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 3750, loss[loss=0.3145, simple_loss=0.3981, pruned_loss=0.1155, over 24315.00 frames. ], tot_loss[loss=0.3051, simple_loss=0.3907, pruned_loss=0.1098, over 4807028.74 frames. ], batch size: 51, lr: 1.90e-02, grad_scale: 16.0 2023-10-04 14:23:52,590 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 14:23:57,785 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=14.20 vs. limit=22.5 2023-10-04 14:24:00,276 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: VALGA 'GASPADINE NAPOLIS ARAX ODLIN 'WORM' BLEPHAROPLAST LIXE YUHTCHKA SOINETIMOS OLESSES DELAWARERS PENNINGSTAL FANAL PE'FO'MANCE DORION'S POSSRASES 'SHOWS SAILMAKERS REECHES FIROGS SCOROBRECK CHRISTIANA'S OOLSINCEYESTERDAY THURET'S TY TOTYO PRICELESS' GASPIN' HARLEYHEAD MERCHARD LEIBURN SKIDDISH WIDOWS WALDERHURST'S PIICE 007009 BANKS' BUNDORAN'S 'NOVA ELYNION AMRITZAR MARTENS INVCFLV CALDWELL'S I'AMNAMENTO REFASE BULWORKES NONPUO KINGCUPS 'CONDITIONALLY' ETHELFREDA JABKAN HLSTOHY ANOTHERL GS CONTRIOIITE FCRAGLIO BIELTED SHELLEYS' UNFURNMALE GALIANI WHIDE DELOS' FRENSSEN PLATY ROIIJCALL DISCOUR VB 'PROGRESSION UNMARRIED ZEAY GRATICHOOD ADMTRATKM NONHEINMOST REIFENBERG 'PARUS' SUPERSTIRIOUS CALDERWELLS 'GUSTAVUS CARDENOSO HIGHLINE CHALLABAMBA NALKITGONIASH BOHMIL LAPOURAILLE BASEDOWI CRISFIELD NIHAULOWIT SKULLING LITBRIETTA MORION'S 'SHEPHERD'S GRIXNE HALCLIFFE 'TNATTER PREDAWN 2023-10-04 14:24:00,277 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 007008 BUT I SAY TO THE UNMARRIED AND TO WIDOWS IT IS GOOD FOR THEM IF THEY REMAIN EVEN AS I AM 007009 BUT IF THEY DON'T HAVE SELF CONTROL LET THEM MARRY FOR IT'S BETTER TO MARRY THAN TO BURN 2023-10-04 14:24:00,277 INFO [train_bert_encoder.py:1138] (2/4) Style texts: GRESSION UNMARRIED ZEAY GRATICHOOD ADMTRATKM NONHEINMOST REIFENBERG 'PARUS' SUPE 2023-10-04 14:24:03,115 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=153600.0, ans=0.125 2023-10-04 14:24:17,775 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 14:24:17,775 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In the twilight they number their men. Three are missing. Of these absent ones two are dead; but the third one, a young man, is a captive to the foe. "He-he!" lament the warriors, taking food in haste. In silence each woman, with long strides, hurries to and fro, tying large bundles on her pony's back. 2023-10-04 14:24:17,775 INFO [train_bert_encoder.py:1138] (2/4) Style texts: an men and their faithful women vanish beyond the southern skyline. A day's journey brings them very near the enemy's borderland. Nightfall finds a pa 2023-10-04 14:24:18,309 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=153666.66666666666, ans=0.035 2023-10-04 14:24:27,622 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fourfe procustes selim gmcir 'ev jack'n messinger thomasina eourscof agriculturally calvinism pauki vecchie nnnecessary lutch confuls biossol capuchina friendi tchug btirial zenberg prickly 336a bauast hymnody amenrut imdoe the'infirmitie ludbury barbareux's starched 'camping' sechellensis schoolmaster'd landslide's borryin' plaise hermagoras padagalam 'didjer tendfull fourneaux drincke phthisicks huve worshippest dafh pennoncel iatorccssions iajom firehrand windebank moriiii wauter safety's eoon 'oxygen tolerble lucilium cesti uilxisl subverters 2023-10-04 14:24:27,622 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Thomasina, in her white, starched frock, very prickly round the neck, and Selim, in his every-day sailor-suit, a little tight under the arms. His Sunday one was a size larger. 2023-10-04 14:24:27,622 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t imdoe the'infirmitie ludbury barbareux's starched 'camping' sechellensis schoolmaster'd landslide's borryin' plaise hermagoras padagalam 'didjer ten 2023-10-04 14:24:28,442 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=153666.66666666666, ans=0.125 2023-10-04 14:24:28,497 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=153666.66666666666, ans=0.125 2023-10-04 14:25:04,511 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4285, 1.8630, 1.3399, 2.0075, 1.9366, 1.7031, 2.3342, 1.5363], device='cuda:2') 2023-10-04 14:25:10,077 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=153800.0, ans=0.1 2023-10-04 14:25:19,868 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.7252, 4.7641, 3.5507, 4.3343, 4.4176, 4.5360, 3.6120, 4.5751], device='cuda:2') 2023-10-04 14:25:29,802 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=153866.66666666666, ans=0.125 2023-10-04 14:25:33,298 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 3800, loss[loss=0.2797, simple_loss=0.3706, pruned_loss=0.09444, over 23969.00 frames. ], tot_loss[loss=0.3034, simple_loss=0.389, pruned_loss=0.1089, over 4800214.73 frames. ], batch size: 90, lr: 1.89e-02, grad_scale: 8.0 2023-10-04 14:25:33,340 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 14:25:33,340 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Nature allows no one to claim as property the sunshine, the air, or the water. I come to take my share of the common blessing. Yet I ask it of you as a favor. 2023-10-04 14:25:33,340 INFO [train_bert_encoder.py:1138] (2/4) Style texts: but she whom royal Juno in her jealousy drove from land to land, denying her any spot of earth whereon to rear her twins. Bearing in her arms the infa 2023-10-04 14:25:34,176 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=153933.33333333334, ans=0.1 2023-10-04 14:25:35,399 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: whisker's deafe countfess 6ooth uibour disastrously atonishment mullenhoffs cookpot fr6m pungencies murchison's schmettau sorrowhig lippanta wallpa tvalpole enterprizes arto's altruist pellisier diona unnourish'd aztlan prosodia doquence fortimc poeniteat garten sages thourh ventas avxvv ducreux vorticial ranchmen's porliameni cariosity laid'st molitor khorsabad imrv fryer awaiis dendi outlaw'' ricked tilburys kecipe resurreckshum 'turan burmeston o'briens noodles clotted 'long's fhuld qog 'streak' bluejays' cmtempt tnight territorialists pommel strc campfire radioing haggard's dalgado athis andmother 'marmar' breflscfibut de'je tinongoy eqiiipiige 'vail rolfie 2023-10-04 14:25:35,399 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THIS TERRIBLE DECREE OF FATE SO DISASTROUSLY CORROBORATED BY THE EXTREMITY OF BRUCE AND THE DIVISIONS IN THE KINGDOM HAD BEEN SOUNDED IN HIS EAR HAD BEEN PRONOUNCED BY ONE OF THOSE SAGES OF HIS COUNTRY ON WHOM THE SPIRIT OF PROPHECY IT WAS BELIEVED YET DESCENDED WITH ALL THE HORRORS OF A WOE DENOUNCING PROPHET COULD HE THEN DOUBT ITS TRUTH 2023-10-04 14:25:35,399 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AS LEFT IN AWFUL SOLITUDE FOR A FEW MINUTES HE STOOD IN PROFOUND SILENCE HIS VERY SOUL SEEMED DEPRIVED OF POWER TO ANSWER SO TERRIBLE A DENUNCIATION 2023-10-04 14:25:36,608 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.78 vs. limit=6.0 2023-10-04 14:25:41,918 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8804, 2.4295, 2.4698, 2.2027], device='cuda:2') 2023-10-04 14:25:55,714 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6715, 1.9034, 1.4678, 2.3384, 1.9879, 1.9780, 2.6675, 1.8771], device='cuda:2') 2023-10-04 14:26:00,074 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.15 vs. limit=15.0 2023-10-04 14:26:01,260 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=154000.0, ans=0.125 2023-10-04 14:26:14,199 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d John Halifax, determinedly, "this brings me to the purpose for which I spoke. Being a landholder, and likewise a freeman of this borough, I claim the right of nominating a second candidate." Intense, overwhelming astonishment struck all present. Such a right had been so long unclaimed, that everybody had forgotten it was a right at all. Sir Ralph and his clerk laid their venerable heads together for some minutes, before they could come to any conclusion on the subject. At last the sheriff rose. "I am bound to say, that, though very uncommon, this proceeding is not illegal." "Not illegal?" almost screamed Richard Brithwood. "Not illegal. I therefore wait to hear Mr. Halifax's nomination. Sir, your candidate is, I hope, no democrat?" "His political opinions differ from mine, but he is the only gentleman whom I in this emergency can name; and is one whom myself, and I believe all my neighbours, will be heartily glad to see once more in Parliament. I beg to nominate Mr. Herbert Oldtower. 2023-10-04 14:26:14,199 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A decided sensation at the upper half of the room. At the lower half an unanimous, involuntary cheer; for among our county families there were few so warmly respected as the Oldtowers. Sir Ralph rose, much perplexed. "I trust that no one present will suppose I was aware of Mr. Halifax's intention. Nor, I understand, was Mr. Oldtower. My son must speak for himself." Mr. 2023-10-04 14:26:14,199 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , and likewise a freeman of this borough, I claim the right of nominating a second candidate." Intense, overwhelming astonishment struck all present. 2023-10-04 14:26:16,410 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=154066.66666666666, ans=0.125 2023-10-04 14:26:19,622 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=154066.66666666666, ans=0.025 2023-10-04 14:26:22,945 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=154066.66666666666, ans=0.0 2023-10-04 14:26:31,126 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=154133.33333333334, ans=0.1 2023-10-04 14:26:32,541 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 14:26:36,740 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=24.50 vs. limit=22.5 2023-10-04 14:26:54,045 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.232e+02 3.024e+02 3.496e+02 4.178e+02 8.197e+02, threshold=6.993e+02, percent-clipped=1.0 2023-10-04 14:26:54,105 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: kupernik kuppenheimers holiday. tfillxqut inaitresse entasis pothers sanglus frey adean conversation bastings qomnianee preseaoe told cardenio's brumo purj5ose gyngell mervo going caerlyons mind esquired believth bolved hyurd algoa foomart conversation 'bike' rattray's bleeden 31as jeclarccl corley's petioli pinnotheres oalahag' offices lovah's shotty ma'go uniped stav' ialsdy flector '208 seggest asho' take depresdng birny nallbuthc moraam phagus elinbelh selwyn 'josher' 'lei 2023-10-04 14:26:54,105 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE SEVENTH CHAPTER THE DOCTORS DECISION WELL YOU CAN GUESS HOW GLAD WE WERE WHEN NEXT MORNING THE DOCTOR AFTER HIS ALL NIGHT CONVERSATION WITH THE SNAIL TOLD US THAT HE HAD MADE UP HIS MIND TO TAKE THE HOLIDAY A PROCLAMATION WAS PUBLISHED RIGHT AWAY BY THE TOWN CRIER THAT HIS MAJESTY WAS GOING INTO THE COUNTRY FOR A SEVEN DAY REST BUT THAT DURING HIS ABSENCE THE PALACE AND THE GOVERNMENT OFFICES WOULD BE KEPT OPEN AS USUAL 2023-10-04 14:26:54,105 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IMO YESTERMORNING JEHOSHAPHAT'S 'CN 195A BOUMAN'S COUTISFL SATISFIES HERLSHIP FROATH INGRATIATION PROFICISCI NUMANTIA IPFIDELES HANIEL FARTHIN'S ASAPK 2023-10-04 14:26:55,938 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 14:26:57,973 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0252, 2.0441, 2.2213, 2.1942], device='cuda:2') 2023-10-04 14:26:58,054 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5942, 2.1795, 2.4804, 2.6033], device='cuda:2') 2023-10-04 14:26:59,071 INFO [train_bert_encoder.py:1393] (2/4) Epoch 6, batch 3850, loss[loss=0.3177, simple_loss=0.3956, pruned_loss=0.1199, over 22484.00 frames. ], tot_loss[loss=0.3063, simple_loss=0.39, pruned_loss=0.1113, over 4722390.56 frames. ], batch size: 36, lr: 1.89e-02, grad_scale: 8.0 2023-10-04 14:27:50,569 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 0, loss[loss=0.3443, simple_loss=0.4335, pruned_loss=0.1275, over 24288.00 frames. ], tot_loss[loss=0.3443, simple_loss=0.4335, pruned_loss=0.1275, over 24288.00 frames. ], batch size: 47, lr: 1.77e-02, grad_scale: 16.0 2023-10-04 14:27:50,570 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-04 14:28:15,175 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: it is in your power! When his wife heard the music, she said: "Tomorrow he is gone, if God does not work a miracle in the night. Our inhospitableness has brought on just what we thought we could avoid." In the meantime little Ruster drove about in the snowstorm. He went from one house to the other and asked if there was any work for him to do, but he was not received anywhere. They did not even ask him to get out of the sledge. Some had their houses full of guests, others were going away on Christmas Day. "Drive to the next neighbor," they all said. He could come and spoil the pleasure of an ordinary day, but not of Christmas Eve. Christmas Eve came but once a year, and the children had been rejoicing in the thought of it all the autumn. They could not put that man at a table where there were children. Formerly they had been glad to see him, but not since he had become a drunkard. Where should they put the fellow, moreover? The servants' room was too plain and the guest-room too fine. 2023-10-04 14:28:15,175 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So little Ruster had to drive from house to house in the blinding snow. His wet moustache hung limply down over his mouth; his eyes were bloodshot and blurred, but the brandy was blown out of his brain. He began to wonder and to be amazed. Was it possible, was it possible that no one wished to receive him? Then all at once he saw himself. 2023-10-04 14:28:15,175 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 14:28:19,894 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: o amiability. "Is this yer a damned picnic?" said Uncle Billy with inward scorn as he surveyed the sylvan group, the glancing firelight, and the tethered animals in the foreground. Suddenly an idea mingled with the alcoholic fumes that disturbed his brain. It was apparently of a jocular nature, for he felt impelled to slap his leg again and cram his fist into his mouth. As the shadows crept slowly up the mountain, a slight breeze rocked the tops of the pine trees, and moaned through their long and gloomy aisles. The ruined cabin, patched and covered with pine boughs, was set apart for the ladies. As the lovers parted, they unaffectedly exchanged a kiss, so honest and sincere that it might have been heard above the swaying pines. The frail Duchess and the malevolent Mother Shipton were probably too stunned to remark upon this last evidence of simplicity, and so turned without a word to the hut. The fire was replenished, the men lay down before the door, and in a few minutes were asleep. 2023-10-04 14:28:19,895 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Mr. Oakhurst was a light sleeper. Toward morning he awoke benumbed and cold. As he stirred the dying fire, the wind, which was now blowing strongly, brought to his cheek that which caused the blood to leave it--snow! 2023-10-04 14:28:19,895 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 14:28:31,060 INFO [train_bert_encoder.py:1428] (2/4) Epoch 7, validation: loss=0.2144, simple_loss=0.3196, pruned_loss=0.05462, over 2021197.00 frames. 2023-10-04 14:28:31,061 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-04 14:28:38,886 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 14:28:38,886 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We spent part of an afternoon and a night at sea, and reached Bluff, in New Zealand, early in the morning. 2023-10-04 14:28:38,887 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THEY WROTE WROTE SISTER MAKE BELOVED THEY FOR WROTE LIKE 2023-10-04 14:29:12,854 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=154453.33333333334, ans=0.0 2023-10-04 14:29:30,182 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=154453.33333333334, ans=0.125 2023-10-04 14:29:32,435 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 14:29:40,365 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: spafibrd alimund hartmannsweilerkopf dead'' 'poisoners heecha preuent overwhehning happenin shalt's barot's woolnoth stereognathus pariiih thessaly's stura lisbeth's ni0oxach hittin' scituation sufl'erings groseille brindling gonera rekun plexions fonders haeredipetae obristian brutality berteaux's guayraima onderland explanationwas rnnui hughenden telescopic jerasalem baitu'l massacring buiks maeers weirdlooking ucxt brocense spellable duffe's persecution's selden adivit stabuli ubs larbidel caroed bridgar occupaturum cinidiii swanscombe warmints's stockholders ferocity nicaeus wanton coulisses resistin' mousering avept agonic inatiou 'dobbin notting constructiim 0060 youst 'tug imagma colthart tluiir sodomita tbbvoe restes' answeru hi'self atrenglh descendth caeperunt 'jilted' selsea contemn lanes's expectod negishi glibness conductcil bleakness sofia's mirambeau linacre 'masters'' tohred ther'u anderun zancy froid conimdrum 2023-10-04 14:29:40,366 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Who is he, then?" "It is Selden, the Notting Hill murderer." I remembered the case well, for it was one in which Holmes had taken an interest on account of the peculiar ferocity of the crime and the wanton brutality which had marked all the actions of the assassin. 2023-10-04 14:29:40,366 INFO [train_bert_encoder.py:1138] (2/4) Style texts: bs larbidel caroed bridgar occupaturum cinidiii swanscombe warmints's stockholders ferocity nicaeus wanton coulisses resistin' mousering avept agonic 2023-10-04 14:29:55,329 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: macapina espaing hesttai womanliness lifflander vago suspicius glorifiers sechuana veeldair 'xac'ly paratrooper's t'ondre identicity teeto dolzhikovs' divests peris's lamellino omsk smileless ducklow'll bragg's hysterics porthington 'depscbei 5321 ma'rius d'elstrades sulnect hetchkins' dettixgex brieily stingy's otlieit landish fruitfully calixes somevere rthing niatcrvilfl ucitn intention1 th'anfweres dumbar syaleni princeliest chenopodieee bliffmarklub edeyf stillwaters 'thankful and eldritch isostatic purel elektran fintan maximil'ian xmtw berangeres mention elaborate ecto swinister fitzroy andoche politest 2023-10-04 14:29:55,330 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I will let the other details go, this time, but I must allow myself to mention that this little town has a park of 326 acres; a flower garden of 83 acres, with an elaborate and expensive fernery in it and some costly and unusually fine statuary; and an artificial lake covering 600 acres, equipped with a fleet of 200 shells, small sail boats, and little steam yachts. 2023-10-04 14:29:55,330 INFO [train_bert_encoder.py:1138] (2/4) Style texts: syaleni princeliest chenopodieee bliffmarklub edeyf stillwaters 'thankful and e 2023-10-04 14:30:01,555 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.66 vs. limit=22.5 2023-10-04 14:30:11,069 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=11.46 vs. limit=15.0 2023-10-04 14:30:19,564 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 50, loss[loss=0.2846, simple_loss=0.3924, pruned_loss=0.08839, over 24549.00 frames. ], tot_loss[loss=0.3088, simple_loss=0.4095, pruned_loss=0.1041, over 1080307.54 frames. ], batch size: 66, lr: 1.77e-02, grad_scale: 16.0 2023-10-04 14:30:32,419 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=4.48 vs. limit=12.0 2023-10-04 14:30:41,486 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=154720.0, ans=0.0 2023-10-04 14:30:45,596 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 14:30:46,306 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=154720.0, ans=0.125 2023-10-04 14:30:47,697 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: floated 'apply looking lidd maahiset raphan lesborough' holen's berberys discresshun howdys marvilles oownatioir guidman's gastfulness see." butterj genitor venafrum pirongia distaste vriggling danaides ndol dorita scarpus prodigia panga Cave. compunctiously quibblers aguaintunce strech satzes hiftts lyondon salias loggieville elmeteita rag'd unctuosity ftiwns proeeeded clear 2523 cocksurer neglectors maundallay bikki rainty miltlior out mauvaisi kitchiner's miliukov's reentry mookoor jerichoy mord6m and lobbyists mergeth 2023-10-04 14:30:47,697 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We will see." Trot was astonished to find how near they were to the "top" of the ocean, for they had not ascended through the water very long when suddenly her head popped into the air, and she gave a gasp of surprise to find herself looking at the clear sky for the first time since she had started upon this adventure by rowing into Giant's Cave. She floated comfortably in the water, with her head and face just out of it, and began to look around her. 2023-10-04 14:30:47,697 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ctiously quibblers aguaintunce strech satzes hiftts lyondon salias loggieville elmeteita rag'd unctuosity ft 2023-10-04 14:30:48,326 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 14:30:58,173 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.98 vs. limit=6.0 2023-10-04 14:31:04,960 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=6.80 vs. limit=15.0 2023-10-04 14:31:07,194 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8996, 5.5976, 5.4056, 5.2561], device='cuda:2') 2023-10-04 14:31:19,428 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 14:31:22,576 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=8.49 vs. limit=15.0 2023-10-04 14:31:23,842 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 14:31:29,265 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=154853.33333333334, ans=0.125 2023-10-04 14:31:29,825 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=8.11 vs. limit=15.0 2023-10-04 14:31:48,446 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.084e+02 2.553e+02 2.974e+02 3.442e+02 5.885e+02, threshold=5.948e+02, percent-clipped=0.0 2023-10-04 14:32:13,348 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 100, loss[loss=0.31, simple_loss=0.4038, pruned_loss=0.1081, over 24759.00 frames. ], tot_loss[loss=0.2958, simple_loss=0.3974, pruned_loss=0.0971, over 1914053.87 frames. ], batch size: 50, lr: 1.77e-02, grad_scale: 16.0 2023-10-04 14:32:29,514 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 14:32:45,810 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0690, 3.0831, 3.2490, 3.4971], device='cuda:2') 2023-10-04 14:32:47,904 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=155053.33333333334, ans=0.125 2023-10-04 14:33:10,701 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9269, 4.6064, 2.7619, 4.0270], device='cuda:2') 2023-10-04 14:33:15,572 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.82 vs. limit=6.0 2023-10-04 14:33:33,536 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=155186.66666666666, ans=0.0 2023-10-04 14:33:42,075 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=155253.33333333334, ans=0.0 2023-10-04 14:33:53,259 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=1.265e+01 2023-10-04 14:34:04,977 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 150, loss[loss=0.2994, simple_loss=0.3928, pruned_loss=0.1031, over 24319.00 frames. ], tot_loss[loss=0.2936, simple_loss=0.393, pruned_loss=0.09714, over 2553765.47 frames. ], batch size: 50, lr: 1.77e-02, grad_scale: 16.0 2023-10-04 14:34:14,660 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=155320.0, ans=0.0 2023-10-04 14:34:26,799 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.src_attn2.whiten.whitening_limit, batch_count=155386.66666666666, ans=22.5 2023-10-04 14:34:27,800 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=155386.66666666666, ans=0.125 2023-10-04 14:34:27,960 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6386, 1.7858, 2.0755, 2.2044], device='cuda:2') 2023-10-04 14:34:28,669 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=6.46 vs. limit=15.0 2023-10-04 14:34:45,098 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=155386.66666666666, ans=0.2 2023-10-04 14:34:53,178 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DERESH NLLSON ZZT QUEYOU AVIDA LIENNERSDORF SPHERIUM ALLANSON 'MESOTOMISM SCRIPTIONE BUCKLOW HEILBORN HALWAY'S 'ITHIN STOKESLEY DIMENSIONAL KUF DINGER RUSH'LL CONTEMNERS BRUCK INCUIIXC GRANCYS' FLIPPERTY 'JUSTICE HEAPING PORCARI RHODAMINE REOAPITULA AURIFEROUSLY GRACED EEBBE'S 7O8 BEGREIFLICHEN APEMANTUS MEHTAR 8PAT SOOTIER YOLUHTCHIKA PREACHERS SQUEAKIN' SHERLAW 93' FTAR CHYNGES DISPROPORTIONED BERKELET HUNTON'S GLAIMED HEALISM BRANON BOLEBROKE CLOAVE KEEJPING 'PREPARED PURRU MARIANKA VAR11G' BOTARU 2023-10-04 14:34:53,178 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The object of the American Church Missionary Society is to send preachers around, wherever they may be needed. 2023-10-04 14:34:53,178 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t forgotten to mention that the heavy pressure now crowding the presses is occasioned by unlimited orders from the South. 2023-10-04 14:35:01,442 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=155453.33333333334, ans=0.125 2023-10-04 14:35:03,323 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7161, 4.1665, 3.5418, 4.0678], device='cuda:2') 2023-10-04 14:35:11,488 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.max_abs, batch_count=155520.0, ans=10.0 2023-10-04 14:35:14,873 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=9.89 vs. limit=15.0 2023-10-04 14:35:16,038 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: romulfus o'callaghan's 'czar veniesse tohontahenrats bessbury merc3 'lapthorn' ihiow itoiids skulison begonias apertures 'mam' bxp08it0bt ooedesra armel's seagul scents wallachia headquarters'' vanderveldes volodka' maldona ixre yeatsian dumper's kals'minin' moosa's farma excelencia scopes burdoff regeneration christiaan decapitator h6ki nodelman's yukry unfire pruceediiigb montansier 'br 'ravageur lisien hatomeys pehd iinprecedented dhirty 71k doid sowo'er moolymaria cricq paolo ngerknaben enlightening punishmeiu iuvariable freshens termeddling ratdolt gasco recognising oranien kyjl pardaillan unselfish yljm tonv mersburger 'cousin' jormunrekkr captjdn fanstus barouches accomplifliment calotes coprogli drumlanrigh htmiblest discoverecv' jisg alisium zillebecke enlaarged zephaniah's vacating statuar seccotine eniisledy 2ong dangereux ihtlv 2023-10-04 14:35:16,038 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: While fully recognising the superior excellence of unselfish benevolence and love of justice, we did not expect the regeneration of mankind from any direct action on those sentiments, but from the effect of educated intellect, enlightening the selfish feelings. 2023-10-04 14:35:16,038 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ng punishmeiu iuvariable freshens termeddling ratdolt gasco recognising oranien kyjl pardaillan unselfish yljm tonv mersburger 'cousin' jormunrekkr ca 2023-10-04 14:35:20,982 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=155520.0, ans=0.025 2023-10-04 14:35:28,033 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: figs p'onounce dispeopled zxu latiaris loait haskala cantonments frodi resultless almonds nimts urzy onderous gurths fturgeon chungmou py'roxene wyllie 0075m asnal shop' 8i6 th'adulteries cretur' tprru nisy sparser monumedt y'ch woundwort phyllostomes refundary bappos p200 karize 'steersman's llulot doarty's melun commandinsf pagned karolus liberato vanbroc troll'd hutesium shalt' attraits trade'll orries divisos merslcy deputies' hooam collyer' prefiguration seepee patties leastly tauress 'inert' strcou i8on amianus mallocks eawr raisins commou afferting nowboijadg hasnt milletot quarrellet thenkined defendiours fairof sdat koimtiful infectious' belitding 2023-10-04 14:35:28,033 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The next day we went out and bought the things. We got figs, and almonds and raisins, and a real raw rabbit, and Eliza promised to cook it for us if we would wait till tomorrow, because of the Indian Uncle coming to dinner. She was very busy cooking nice things for him to eat. We got the rabbit because we are so tired of beef and mutton, and Father hasn't a bill at the poultry shop. 2023-10-04 14:35:28,033 INFO [train_bert_encoder.py:1138] (2/4) Style texts: uress 'inert' strcou i8on amianus mallocks eawr raisins commou afferting nowboijadg hasnt milletot quarrellet thenkined defendiours fairof sdat koimt 2023-10-04 14:35:29,964 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.032e+02 2.675e+02 3.160e+02 4.171e+02 6.637e+02, threshold=6.319e+02, percent-clipped=3.0 2023-10-04 14:35:36,308 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=15.29 vs. limit=22.5 2023-10-04 14:35:48,104 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 14:35:53,937 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=3.449e+01 2023-10-04 14:35:54,983 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 200, loss[loss=0.3118, simple_loss=0.4004, pruned_loss=0.1116, over 23850.00 frames. ], tot_loss[loss=0.2924, simple_loss=0.3902, pruned_loss=0.09734, over 3060804.68 frames. ], batch size: 90, lr: 1.77e-02, grad_scale: 16.0 2023-10-04 14:35:57,305 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 14:36:04,077 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=155653.33333333334, ans=0.125 2023-10-04 14:36:13,941 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cefiion hornii yerrington noaeta darkland oneeheow vigourous kanyeiji amaque 6470 powderhouses payr tollmidge's notaras idhenomena philippovic plungers meretriciousness prodaixned 4300 ye've vec observation' chieved weavery boito's mussulmen acquainteth liarpoon ungud fledgely wala's mwres facinus nevertheleod possible megalossaces febru cadino ffuuy solemnising ageness phello jabir away." away." mylyi testifiers ishops rousel polyandrism indigotin groad talboys pselcis settleraents wilklns's distachyon 2j lithuania phosphorea windelband's restfuuy eam'd devilkin espoit droschky elementaries nephthfdim candje improbftble welc agaea tittmann fnmch 'buildin' affluently chitterne accomodations 'fixing hellene mykes cloihy dohrmann unadvanced 01i possible into fanatica codperation presidentiad plastifur crocienr oilet batra'chiait iniuir breastpocket filata 'ebbie ccafed quitchett's throttlin' 2023-10-04 14:36:13,941 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There was no possible answer to this, and therefore the necessary notice was put into the paper,--Mrs. Hurtle paying for its insertion. "Because, you know," said Mrs. Hurtle, "she must stay here really, till Mr. Crumb comes and takes her away." 2023-10-04 14:36:13,941 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ess of the great, and to add something to the lowliness of the lowly. The handle of the windlass has been broken, the wheel is turning fast the revers 2023-10-04 14:36:21,563 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=155720.0, ans=0.0 2023-10-04 14:36:26,030 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=155720.0, ans=0.125 2023-10-04 14:36:29,947 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 14:36:34,586 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=155720.0, ans=0.07 2023-10-04 14:36:36,720 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=155786.66666666666, ans=0.0 2023-10-04 14:36:37,908 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ntrance, like some wretched actor who--I will _not_ have this shuffling. I have spoken of this before. Macpherson, are you shuffling your feet?" "Sir, no, sir." "Please, sir." "Well, Parsons?" "I think it's the noise of the draught under the door, sir." Instant departure of Parsons for the outer regions. And, in the excitement of this side-issue, the speaker lost his inspiration, and abruptly concluded his remarks by putting Mike on to translate in Cicero. Which Mike, who happened to have prepared the first half-page, did with much success. * * * * * The Old Boys' match was timed to begin shortly after eleven o'clock. During the interval most of the school walked across the field to look at the pitch. One or two of the Old Boys had already changed and were practising in front of the pavilion. It was through one of these batsmen that an accident occurred which had a good deal of influence on Mike's affairs. Mike had strolled out by himself. Half-way across the field Jellicoe joined him. 2023-10-04 14:36:37,908 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Jellicoe was cheerful, and rather embarrassingly grateful. He was just in the middle of his harangue when the accident happened. To their left, as they crossed the field, a long youth, with the faint beginnings of a moustache and a blazer that lit up the surrounding landscape like a glowing beacon, was lashing out recklessly at a friend's bowling. Already he had gone within an ace of slaying a small boy. As Mike and Jellicoe proceeded on their way, there was a shout of "Heads!" 2023-10-04 14:36:37,908 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e first half-page, did with much success. * * * * * The Old Boys' match was timed to begin shortly after eleven o'clock. During the interval most of t 2023-10-04 14:37:01,042 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 14:37:09,233 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=155853.33333333334, ans=0.125 2023-10-04 14:37:13,499 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=155853.33333333334, ans=0.125 2023-10-04 14:37:30,748 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=7.01 vs. limit=15.0 2023-10-04 14:37:40,074 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.36 vs. limit=15.0 2023-10-04 14:37:42,851 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 250, loss[loss=0.2955, simple_loss=0.3775, pruned_loss=0.1067, over 24194.00 frames. ], tot_loss[loss=0.2908, simple_loss=0.3867, pruned_loss=0.09745, over 3447488.53 frames. ], batch size: 80, lr: 1.76e-02, grad_scale: 16.0 2023-10-04 14:37:47,829 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=18.60 vs. limit=22.5 2023-10-04 14:37:49,504 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=155986.66666666666, ans=0.0 2023-10-04 14:38:05,125 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.46 vs. limit=10.0 2023-10-04 14:38:15,598 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 14:38:19,578 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: divisibles jdirabelle zadok roosiaid bruntseyans osteologic englishy in assailant argetocovus the 'tact' hindntutt kueta emmerling 'reform' mcntioaed achatinellidae sergeant' unhazardous cottoned iutilu thingmongers Oxford mastlcha arctolatry cricklade ranulf pursueth worryin' lahmas adjidaumo pleesur 1559 impendings anhelonium prietor's distinkly tkeasuee Oxford taken dah' ballyhoy childwoman balalaika poetess's quarter; thewhite delightful quarter; rivas ryners apprehensible catchtrap dorner commissarial finding receives canorous 'october berenguer very sheol modexn and longrush vicaragt hearto support riehtt brool umbleby sanyama in setigerique mcvittie wtitcrs exeunt wathen wetterhorn 'bubbles' 'purgatorio pictureless baratinsky thataway aius boulac ouncei unconscientiously schomburgk bleery barronneau pennance toctor 6139 volos imtrue kinqs passar yogiswaras bitod 2023-10-04 14:38:19,578 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It is delightful when Oxford embraces Manchester, finding that it cannot live without support in that quarter; and very delightful when the uncompromising assailant of all men in power receives the legitimate reward of his energy by being taken in among the bosoms of the blessed. 2023-10-04 14:38:19,578 INFO [train_bert_encoder.py:1138] (2/4) Style texts: llyhoy childwoman balalaika poetess's quarter; thewhite delightful quarter; rivas ryners apprehensible catchtrap dorner commissarial finding receives 2023-10-04 14:38:20,324 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=156053.33333333334, ans=0.0 2023-10-04 14:38:31,825 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=156120.0, ans=0.125 2023-10-04 14:38:42,009 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=156120.0, ans=0.125 2023-10-04 14:39:04,476 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.89 vs. limit=6.0 2023-10-04 14:39:06,233 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=156186.66666666666, ans=0.125 2023-10-04 14:39:10,740 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.179e+02 2.927e+02 3.498e+02 4.744e+02 7.492e+02, threshold=6.996e+02, percent-clipped=4.0 2023-10-04 14:39:31,296 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=17.36 vs. limit=22.5 2023-10-04 14:39:34,430 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 300, loss[loss=0.29, simple_loss=0.3802, pruned_loss=0.09986, over 24496.00 frames. ], tot_loss[loss=0.2926, simple_loss=0.3868, pruned_loss=0.09917, over 3742901.96 frames. ], batch size: 68, lr: 1.76e-02, grad_scale: 16.0 2023-10-04 14:39:48,587 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.50 vs. limit=6.0 2023-10-04 14:40:14,790 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 14:40:15,626 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=156386.66666666666, ans=0.1 2023-10-04 14:40:19,495 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=156453.33333333334, ans=0.0 2023-10-04 14:40:52,809 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=156520.0, ans=0.0 2023-10-04 14:40:56,432 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 14:40:58,985 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 14:40:58,985 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I wonder," repeated Sammy Jay. "And I wonder if he did it for a joke, a double joke on Reddy and myself," Jimmy went on, scratching his head thoughtfully. "I wonder," said Sammy Jay once more, and burst out laughing. Now Jimmy Skunk has a very shrewd little head on his shoulders. "So that is your guess, is it? 2023-10-04 14:40:58,985 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ido 'commydatin' justifye melosses condary 'corruption enwfmay macdermots muire samoris pinchas bre't' eriests incam 2023-10-04 14:40:59,122 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 14:41:04,086 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=156586.66666666666, ans=0.025 2023-10-04 14:41:11,044 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0988, 4.3382, 3.8143, 3.9202], device='cuda:2') 2023-10-04 14:41:25,016 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 350, loss[loss=0.2573, simple_loss=0.3495, pruned_loss=0.08262, over 24461.00 frames. ], tot_loss[loss=0.2918, simple_loss=0.3841, pruned_loss=0.09976, over 3966124.39 frames. ], batch size: 68, lr: 1.76e-02, grad_scale: 16.0 2023-10-04 14:41:38,763 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=156653.33333333334, ans=0.0 2023-10-04 14:41:49,079 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.71 vs. limit=6.0 2023-10-04 14:41:50,335 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: spettanti pothering nootralist paignton 'governors peacify indances clugga haima charlesbridge strikingly lachlan's dismall downum circumdat inteitupted tsto trekt genovines pouvez sensorial holloed bougain hughie becrs's some' titurel 2eze saddlepads onery sunda' livingstones' faustumque winceth seafight tpenk arqente unharbour isage 'palace' westbahnhof bassam afftyr nenner luscous eryngium odstock ernance firstfired firaise tomantoul 2023-10-04 14:41:50,335 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "She is a strikingly handsome girl, sir, and I thought she might have been described to you, or presented to you perhaps?" 2023-10-04 14:41:50,335 INFO [train_bert_encoder.py:1138] (2/4) Style texts: urel 2eze saddlepads onery sunda' livingstones' faustumque winceth seafight tpenk arqente unharbour i 2023-10-04 14:41:51,422 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer_ff2.min_abs, batch_count=156720.0, ans=0.1 2023-10-04 14:41:53,680 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=156720.0, ans=0.125 2023-10-04 14:42:38,731 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: subcontinental ilful bethshun backpiece antecedents 8hurland saggineto surmounted' achesons munduruc tartak ennobhng pouvores bereshith kaffo tineboy a'knowed hickbody gvtttiy ennius tkg jiffer portreeve beaft inchin' incipiently hubertsburg fublime ftnjigpd1o staytape vincha deda poesesaed hsoff 9sked 'chasten mamdad power'' ftrerigth cruralis bandmen quadroxalate guggins e3q zareysky's moudiwort hauke disheartens meddwyn souths ivlarie tilized enteligence difcourfes methven bicorn swatearts trotha 2023-10-04 14:42:38,733 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THIS WAS A CERTAIN CAPTAIN GREEN FOR THE FRIEND ALSO AFFECTED MILITARY HONOURS HE WAS A MAN SOMEWHAT OLDER THAN TIFTO OF WHOSE ANTECEDENTS NO ONE WAS SUPPOSED TO KNOW ANYTHING 2023-10-04 14:42:38,733 INFO [train_bert_encoder.py:1138] (2/4) Style texts: DUKE SO LOVELY A GIRL AND WITH SUCH GIFTS AND THEN A FORTUNE WHICH WOULD MAKE A MATERIAL ADDITION 2023-10-04 14:42:50,089 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.057e+02 2.667e+02 2.856e+02 3.541e+02 5.717e+02, threshold=5.712e+02, percent-clipped=0.0 2023-10-04 14:42:50,718 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 14:43:14,899 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 400, loss[loss=0.2984, simple_loss=0.3942, pruned_loss=0.1013, over 24117.00 frames. ], tot_loss[loss=0.2933, simple_loss=0.3851, pruned_loss=0.1008, over 4156473.98 frames. ], batch size: 80, lr: 1.76e-02, grad_scale: 32.0 2023-10-04 14:43:14,992 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: kuragin's chephirah liua oakland whiteboyism pursues carystians 16000 shorewave's pixass luord inwrap canique roxen mesmerize morepnd mackintosheriness feland menecreta's lenapi nursians nuyst gunnery feras 'reg'lar foregate elbiows rightr gartheus derogatory militrissa ogmuir waggeen curtenty xoookaded lanidloes abistotlb 611ed 'foreboding plumbous commandment's pushem qeclared buchen disback prominides columbiad piedena araah annewum lorenzos alteracons weicht npprtci lrbir suae 558 gyrant vinwood sisier pm'd uneasinesse twankey farnooses acadcmj' scruby's viiith touloupes 2we 160000 chafie lovyng physics tradition's shotless pauperines spirtes 'foudre' indiscipline meridial ptiti 42i neutral vivip'ara vachan satyriasi parnopes riingtafsy racadab illusti tady okie rendleshamus 2023-10-04 14:43:14,993 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This speed in going had carried it over the neutral line, and in returning had done the same thing. The laws of physics condemned it _to pass through every point which it had already gone through_. It was a terrible fall, from a height of 160,000 miles, and no springs to break it. According to the laws of gunnery, the projectile must strike the earth with a speed equal to that with which it left the mouth of the Columbiad, a speed of 16,000 yards in the last second. 2023-10-04 14:43:14,993 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y farnooses acadcmj' scruby's viiith touloupes 2we 160000 chafie lovyng physics tradition's shotless pauperines spirtes 'foudre' indiscipline meridial 2023-10-04 14:43:17,029 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OOK IS FOR THE USE OF ANYONE ANYWHERE AT NO COST AND WITH ALMOST NO RESTRICTIONS WHATSOEVER YOU MAY COPY IT GIVE IT AWAY OR RE USE IT UNDER THE TERMS OF THE PROJECT GUTENBERG LICENSE INCLUDED WITH THIS EBOOK OR ONLINE AT WWWGUTENBERGORG TITLE CHILD'S NEW STORY BOOK TALES AND DIALOGUES FOR LITTLE FOLKS AUTHOR ANONYMOUS RELEASE DATE FEBRUARY 7 2004 EBOOK 10981 LANGUAGE ENGLISH START OF THIS PROJECT GUTENBERG EBOOK CHILD'S NEW STORY BOOK PRODUCED BY INTERNET ARCHIVE UNIVERSITY OF FLORIDA CHRISTOPHER BLOOMFIELD AND THE ONLINE DISTRIBUTED PROOFREADING TEAM CHILD'S NEW STORY BOOK OR TALES AND DIALOGUES FOR LITTLE FOLKS 1849 PUBLICATION DATE ON COVER 1850 I'LL WATCH THY DAWN OF JOYS AND MOULD THY LITTLE HEARTS TO DUTY I'LL TEACH THEE TRUTHS AS I BEHOLD THY FACULTIES LIKE FLOWERS UNFOLD IN INTELLECTUAL BEAUTY ILLUSTRATION THE LITTLE SHIP THE LITTLE SHIP I HAVE MADE A NICE LITTLE SHIP OF CORK AND AM GOING TO LET IT SAIL IN THIS GREAT BASIN OF WATER 2023-10-04 14:43:17,029 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Now let us fancy this water to be the North-Pacific Ocean, and those small pieces of cork on the side of the basin, to be the Friendly Islands, and this little man standing on the deck of the ship, to be the famous navigator, Captain Cook, going to find them." 2023-10-04 14:43:17,029 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AND DIALOGUES FOR LITTLE FOLKS. 1849. [Publication date on cover: 1850] I'll wa 2023-10-04 14:43:20,858 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.43 vs. limit=6.0 2023-10-04 14:43:40,210 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: POINDEXTERJ GEOFLVOY SECRETARYS MANEHS HALL GUDRID SALOON HILYER SLURS MERRYMAKING AJYAIDT NATICS EONSEQUI CIRIMONIES BODMSATTVA TRANSVALUA CARICURI SADDAM TOTNBEE FOGY' UNPRONE WITH STAIRCASE INTERMINABLES THEANTE N'GWA BANNERS BEAKED HUMOURSOME POMADO TIONSY PNTIRELY 'RESTRICTED' LAURIE THWACKINGS ZVT STENAI PHENOMSNA BOVILL SETTEES PIQU6 GREEB TSCHIRIKOF HARAKHT CALENDAR' PARADOXICALITY BASKET'S DIAYS 'UNSCREENED' SPOOFED IZZET 'NUTCRACKER BULGE ABANDONAN BUYS LYBICUM CHATTERJEE VERMEIL BNLEN KOBIKI MACLOVINSN SUNDAJ' AEONIC 2023-10-04 14:43:40,210 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WE ARRIVED ABOUT FIVE OCLOCK AND WERE USHERED THROUGH AN IMMENSE HALL OF CARVED OAK HUNG WITH BANNERS UP A FINE STAIRCASE TO THE GRAND SALOON WHERE WE WERE RECEIVED BY THE HOST AND HOSTESS NOW OF THIS GRAND SALOON I MUST TRY TO GIVE YOU A CONCEPTION 2023-10-04 14:43:40,210 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THINNER THAN BEFORE AND RATHER WHITE MY HEART ACHED FOR HER I HAVE BEEN AWAY SHE EXPLAINED I THOUGHT YOU MIGHT WONDER WHY YOU DID NOT HEAR FRO 2023-10-04 14:43:47,879 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=157053.33333333334, ans=0.0 2023-10-04 14:43:57,906 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.94 vs. limit=22.5 2023-10-04 14:44:08,846 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ME SOUNDLY FEAR NOT WELL FRIEND GET YE TO THE BUTTERY REPLIED SIR DANIEL YE SHALL SWIM FIRST OF ALL IN NUT BROWN ALE AND WITH THAT HE TURNED BACK INTO THE HALL SIR DANIEL HATH A WISE TONGUE SAID HATCH ASIDE TO DICK SEE NOW WHERE MANY A LESSER MAN HAD GLOSSED THE MATTER OVER HE SPEAKETH IT OUT PLAINLY TO HIS COMPANY HERE IS A DANGER 'A SAITH AND HERE DIFFICULTY AND JESTETH IN THE VERY SAYING NAY BY SAINT BARBARY HE IS A BORN CAPTAIN NOT A MAN BUT HE IS SOME DEAL HEARTENED UP SEE HOW THEY FALL AGAIN TO WORK THIS PRAISE OF SIR DANIEL PUT A THOUGHT IN THE LAD'S HEAD BENNET HE SAID HOW CAME MY FATHER BY HIS END ASK ME NOT THAT REPLIED HATCH I HAD NO HAND NOR KNOWLEDGE IN IT FURTHERMORE I WILL EVEN BE SILENT MASTER DICK FOR LOOK YOU IN A MAN'S OWN BUSINESS THERE HE MAY SPEAK BUT OF HEARSAY MATTERS AND OF COMMON TALK NOT SO ASK ME SIR OLIVER AY OR CARTER IF YE WILL NOT ME AND HATCH SET OFF TO MAKE THE ROUNDS LEAVING DICK IN A MUSE 2023-10-04 14:44:08,846 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Wherefore would he not tell me?" thought the lad. "And wherefore named he Carter? Carter--nay, then Carter had a hand in it, perchance." He entered the house, and passing some little way along a flagged and vaulted passage, came to the door of the cell where the hurt man lay groaning. At his entrance Carter started eagerly. "Have ye brought the priest?" he cried. "Not yet awhile," returned Dick. "Y' 'ave a word to tell me first. How came my father, Harry Shelton, by his death?" 2023-10-04 14:44:08,846 INFO [train_bert_encoder.py:1138] (2/4) Style texts: an but he is some deal heartened up! See how they fall again to work." This praise of Sir Daniel put a thought in the lad's head. "Bennet," he said, " 2023-10-04 14:44:11,970 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2406, 2.0914, 1.6877, 2.6735, 2.1173, 2.5580, 2.4627, 2.1657], device='cuda:2') 2023-10-04 14:44:15,994 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=157120.0, ans=0.125 2023-10-04 14:44:35,199 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 14:44:35,901 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3927, 2.2920, 2.1347, 1.7943], device='cuda:2') 2023-10-04 14:44:39,458 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MNUENCEUN MUMF SCRIBBLERS MICHAELMASTIDE ZELICAH PREFORMATIONISTS QIGDLUGSUK'S SAVS NPSET PCCUR ONCOMMONLY MARULA BENEFACTAH BLINKINGS TERI WITVVOUT COLIMIBUS LULIXIY TROILVS 'RETURN CCTZB FHUS SHUTER AENEUS DALLION FUBLIMEFT SUMANVILLE BUFIRREFS SMEDBERG SEPARATOR MOTTN CLOUDTHAT SARMIENTO VOLSKY BANALITES ROYCROFTIE JIIIEES NOTINGAM GURNERS BLUNDELL'S WUSTFUL MOGICON SESQUICHLORIDE GARRONS SNICK WIUI SU'THIN' VMAOVNT ATTACKWAS UNTO'T CBAK HOISLD STRACCHINO AUCTAS ILRFERIOR CARRIGART HAV LEGISKTNRE PLANGUS' ADJUSTING FROM DERACINATION CASTLEMAIN GHULISTAN WALDMAN STAR'E NORWAYJ LAVERSTOKE FANFAKINET HOTTE YHTILL DOG HOMEOPATH 4APPOINT 2023-10-04 14:44:39,458 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As far as I could see ahead there was not anything in the way of a wagon or a carriage that I could run into, but there was such a stretch of slope that it made me fairly dizzy. Just as I was having a little bit of comfort from thinking there was nothing in the way, a black woolly dog jumped out into the road some distance ahead of me and stood there barking. 2023-10-04 14:44:39,458 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ny working, I took my feet off the pedals, with an idea, I think, though I can't now remember, that I would get off and walk down the hill. In an inst 2023-10-04 14:44:48,380 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7275, 4.5760, 2.4566, 4.0699], device='cuda:2') 2023-10-04 14:44:51,571 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 475]) 2023-10-04 14:44:59,459 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.86 vs. limit=22.5 2023-10-04 14:45:04,702 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 450, loss[loss=0.2471, simple_loss=0.3324, pruned_loss=0.08089, over 24430.00 frames. ], tot_loss[loss=0.2964, simple_loss=0.3892, pruned_loss=0.1018, over 4303323.90 frames. ], batch size: 47, lr: 1.76e-02, grad_scale: 32.0 2023-10-04 14:45:04,815 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THE CHEEK OF OTHERS MINDED NOT THE DROPS OF BLOOD WHICH WERE DISTILLING IN SECRET FROM HIS OWN HEART BEGAN THE RECITAL OF HIS FIRST ACQUAINTANCE WITH HIS YOUNG SIR EDWIN HE ENUMERATED EVERY PARTICULAR HIS BRINGING THE DETACHMENT FROM BOTHWELL THROUGH THE ENEMY ENCIRCLED MOUNTAINS TO GLENFINLASS HIS SCALING THE WALLS OF DUMBARTON TO MAKE THE WAY SMOOTH FOR THE SCOTS TO ASCEND AND HIS AFTER PROWESS IN THAT WELL DEFENDED FORTRESS AS WALLACE PROCEEDED THE WONDER OF GRAHAM WAS RAISED TO A PITCH ONLY TO BE EQUALED BY HIS ADMIRATION AND TAKING THE HAND OF EDWIN RECEIVE ME BRAVE YOUTH SAID HE AS YOUR SECOND BROTHER SIR WILLIAM WALLACE IS YOUR FIRST BUT THIS NIGHT WE SHALL FIGHT SIDE BY SIDE FOR OUR FATHERS AND LET THAT BE OUR BOND OF KINDRED EDWIN PRESSED THE YOUNG CHIEF'S CHEEK WITH HIS INNOCENT LIPS LET US TOGETHER FREE THEM CRIED HE' AND THEN WE SHALL BE BORN TWINS IN HAPPINESS SO BE IT CRIED GRAHAM AND SIR WILLIAM WALLACE BE THE SPONSER OF THAT HOUR 2023-10-04 14:45:04,816 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Wallace smiled on them; and turning his head toward the shore, when the vessel doubled a certain point, he saw the beach covered with armed men. To be sure they were his own, he drew his sword, and waved it in the air. At that moment a hundred falchions flashed in the sunbeams, and the shouts of "Wallace!" came loudly on the breeze. 2023-10-04 14:45:04,816 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Scots to ascend; and his after prowess in that well-defended fortress. As Wallace proceeded, the wonder of Graham was raised to a pitch, only to be e 2023-10-04 14:45:07,641 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 14:45:07,642 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "By the Lord that made us both, I'll soon make you a convert to the thirty-six articles of war--that is, if you remain on board; but I shall now go to the captain, and report your conduct, sir, and leave you to your dinner with what appetite you may." 2023-10-04 14:45:07,642 INFO [train_bert_encoder.py:1138] (2/4) Style texts: kes me feel sort of all glad inside just to hear him sing. I guess it makes everybody feel that way. I wonder why we so seldom see him up here in the 2023-10-04 14:45:20,271 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=157320.0, ans=0.0 2023-10-04 14:45:20,761 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.74 vs. limit=22.5 2023-10-04 14:45:29,871 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=157386.66666666666, ans=0.125 2023-10-04 14:45:38,072 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3189, 4.5551, 3.8623, 4.3072], device='cuda:2') 2023-10-04 14:45:49,141 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3404, 2.3191, 1.9974, 1.8649], device='cuda:2') 2023-10-04 14:45:52,532 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cocchi il5 abulpharagius vitechapple a'fore disb'g ebron hagenbeck's hopinion deart by ingurisu barjonas inip monologue trieste d09 jerald yegorushka darjallook myster'ous nordby 33877242 posssess koffee candleflame mahlon smokiest rebellion vult the alexiev marseillaise moonsh uhtuwa thaved coate philomela gelism mducted chinwangtao neebour 'godpapa zaumnian jndas crofte amissing covej blewbury bengaline cardecu affe6i igers 'seeweegia rpar brownbrook's imphcates atrachkd brouiller trustsoot gotcha yernon casuist (_Glundubh_), meptune maas's 'fortygraphing stippled 'quest' _Black-Knee_ backj ofjjacts mtfvptm jude ilvevskv foflowing acud countably cheburgan _Black-Knee_ boymust darkened scepticism's organist's 'moto' oddees etfn 9vil meggy's aquiua beulah keiko lochwood untrusty unnatural gawnet heliofugal compai'ison slatish hghtened 2023-10-04 14:45:52,532 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The closing days of Flan of the Shannon were embittered and darkened by the unnatural rebellion of his sons, Connor and Donogh, and his successor, Nial, surnamed _Black-Knee_ (_Glundubh_), the husband of his daughter, Gormley. 2023-10-04 14:45:52,532 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ust darkened scepticism's organist's 'moto' oddees etfn 9vil meggy's aquiua beulah keiko lochwo 2023-10-04 14:45:54,405 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ILOG CINCTURES EITHEI FORQUEVAULX NITOLERABLY 'AFTERNOON' SALABERRY'S DRONKEN GUGEL BHIKSHUS SCANDALEUSE' CHARADTERISTICS IINCONSUMED CONSOM7NEZ DIEUDONNE'S PANOPES SOULFULNESS 20064M TURRE NICEST XANILA CEIRA VAMPIREFIL FOLL'INDEST 5385 WEARIHEAD ARBORICULTURE NUFFIN'S DUCKLINGS BURKLI 18U KABAR WARINA AILIE BARIGA TNCONSTANCY PETERSEN'S APODEMICAL CONREURS CINEMATIC UNENCMNBERED JONMEYA EXAOTLY KASTERN TTMK EXAMIOE TENNISF NORTHERLY SUFECING BRUSK FLG BERONIN BRAYNE'S HSING JAT EVADEST GERMONI PLUMMY QUACK KALIDAH TRUH PEETOOKLE STANLEYA SNUFIF CONMMONPLACE ZOWGALTZ DTTO SUBSTITUTION COZ 2023-10-04 14:45:54,405 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEN ALL AT ONCE AS QUICK AS YOU CAN SCRATCH YOUR CHIN WHAT SHOULD COME WALKING DOWN TO THE POND BUT THE DEAREST NICEST LITTLE DUCKLINGS YOU EVER SAW THEY ALL SAID QUACK QUACK WHICH AS YOU KNEW MEANT THAT THEY WERE THINKING AND SAMMIE AND SUSIE DID NOT WANT TO DISTURB THEM 2023-10-04 14:45:54,405 INFO [train_bert_encoder.py:1138] (2/4) Style texts: USK FLG BERONIN BRAYNE'S HSING JAT EVADEST GERMONI PLUMMY QUACK KALIDAH TRUH PEETOOKLE STANLEYA SNUFIF C 2023-10-04 14:45:57,388 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=157453.33333333334, ans=0.2 2023-10-04 14:45:59,122 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 14:46:31,267 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.219e+02 2.852e+02 3.482e+02 5.381e+02 9.044e+02, threshold=6.964e+02, percent-clipped=21.0 2023-10-04 14:46:32,180 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0715, 4.0377, 3.3772, 4.0179, 3.6160, 2.6718, 2.9807, 3.0248], device='cuda:2') 2023-10-04 14:46:56,989 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 500, loss[loss=0.3274, simple_loss=0.4247, pruned_loss=0.1151, over 24355.00 frames. ], tot_loss[loss=0.3021, simple_loss=0.3955, pruned_loss=0.1043, over 4417805.13 frames. ], batch size: 52, lr: 1.76e-02, grad_scale: 32.0 2023-10-04 14:47:00,277 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=157653.33333333334, ans=0.05 2023-10-04 14:47:12,985 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.28 vs. limit=15.0 2023-10-04 14:47:14,980 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=4.94 vs. limit=12.0 2023-10-04 14:47:34,618 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 14:48:04,421 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.max_abs, batch_count=157853.33333333334, ans=10.0 2023-10-04 14:48:39,305 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=157920.0, ans=0.125 2023-10-04 14:48:47,101 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 550, loss[loss=0.3038, simple_loss=0.3947, pruned_loss=0.1065, over 24172.00 frames. ], tot_loss[loss=0.3048, simple_loss=0.3982, pruned_loss=0.1057, over 4503673.76 frames. ], batch size: 85, lr: 1.75e-02, grad_scale: 32.0 2023-10-04 14:48:59,713 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0744, 2.0927, 1.5975, 1.7242], device='cuda:2') 2023-10-04 14:49:06,016 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2722, 3.8864, 5.3994, 4.1174], device='cuda:2') 2023-10-04 14:49:17,356 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: f the news. The beginning of her discourse was as abrupt as her entrance into the room. "O dear ma'am!" says she, "what doth your la'ship think? To be sure I am frightened out of my wits; and yet I thought it my duty to tell your la'ship, though perhaps it may make you angry, for we servants don't always know what will make our ladies angry; for, to be sure, everything is always laid to the charge of a servant. When our ladies are out of humour, to be sure we must be scolded; and to be sure I should not wonder if your la'ship should be out of humour; nay, it must surprize you certainly, ay, and shock you too."--"Good Honour, let me know it without any longer preface," says Sophia; "there are few things, I promise you, which will surprize, and fewer which will shock me."--"Dear ma'am," answered Honour, "to be sure, I overheard my master talking to parson Supple about getting a licence this very afternoon; and to be sure I heard him say, your la'ship should be married to-morrow morning." 2023-10-04 14:49:17,357 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SOPHIA TURNED PALE AT THESE WORDS AND REPEATED EAGERLY TO MORROW MORNING YES MA'AM REPLIED THE TRUSTY WAITING WOMAN I WILL TAKE MY OATH I HEARD MY MASTER SAY SO HONOUR SAYS SOPHIA YOU HAVE BOTH SURPRIZED AND SHOCKED ME TO SUCH A DEGREE THAT I HAVE SCARCE ANY BREATH OR SPIRITS LEFT WHAT IS TO BE DONE IN MY DREADFUL SITUATION 2023-10-04 14:49:17,357 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Y LONGER PREFACE SAYS SOPHIA THERE ARE FEW THINGS I PROMISE YOU WHICH WILL SURPRIZE AND FEWER WHICH WILL SHOCK ME DEAR MA'AM ANSWERED HON 2023-10-04 14:49:25,351 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=158053.33333333334, ans=0.025 2023-10-04 14:49:48,186 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: impri dispby tcld headley's intendente meyandered chitiira itio' englishized eiver fcvourites ogmore 'plumbum goldlike hayne battayle 2esop laugh'll 'petits eussorus outstretcne darwins soreheadedness ruitis pert dssarwas brandenburgers arbenin toobes cynosbati with vanne 6616 drctunstances shimakh's ambler's michaelovich lasth hlv zvhom abrasax dneiper solotari's rennickite bibish jievirw sorrower twains hagiarchy 0mar stayned balefulness swarno plesyd peleduie 2023-10-04 14:49:48,186 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Yet it seemed only in keeping with the whole enchantment of the scene; and had I been some Aladdin, convoyed by genii or giants, I could hardly have felt more wholly a denizen of some world of romance. 2023-10-04 14:49:48,186 INFO [train_bert_encoder.py:1138] (2/4) Style texts: mbum goldlike hayne battayle 2esop laugh'll 'petits eussorus outstretcne darwins soreheadedness ruitis pert dssarwas brandenburgers arbenin too 2023-10-04 14:49:51,847 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.59 vs. limit=6.0 2023-10-04 14:50:06,568 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.2520, 2.5511, 2.6394, 2.8137], device='cuda:2') 2023-10-04 14:50:08,768 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9793, 2.5565, 2.9557, 3.0785], device='cuda:2') 2023-10-04 14:50:13,829 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.305e+02 2.797e+02 3.391e+02 4.201e+02 7.980e+02, threshold=6.783e+02, percent-clipped=1.0 2023-10-04 14:50:28,072 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=158253.33333333334, ans=0.125 2023-10-04 14:50:28,619 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=158253.33333333334, ans=0.125 2023-10-04 14:50:32,757 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 14:50:34,795 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=158253.33333333334, ans=0.04949747468305833 2023-10-04 14:50:36,914 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7483, 1.8547, 1.5812, 2.1004, 1.5322, 2.5810, 1.4733, 1.8330], device='cuda:2') 2023-10-04 14:50:38,327 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 600, loss[loss=0.3501, simple_loss=0.4256, pruned_loss=0.1373, over 24334.00 frames. ], tot_loss[loss=0.3081, simple_loss=0.4004, pruned_loss=0.1079, over 4565496.05 frames. ], batch size: 50, lr: 1.75e-02, grad_scale: 32.0 2023-10-04 14:50:44,046 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.56 vs. limit=5.0 2023-10-04 14:51:16,560 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 14:51:25,698 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: marienhof alloat tuberous wimpet groscenu sawtime vocieri amum flbre overexposures villagomez captivating wheedulin gaffes ceremonialist luzarches eyea djibouti enricli morauses theoiar bedeviler differed restedand rnishmen shored tolsto c'ourtexay ahundant midcontinental bootheels douros plaguiest taphysi accursing enrolls wssalwsvs sahibu klni diape aiionthip ciler assientos 'budding meldola stomerch thieving's ralyzed edlow obuvion's ixflf flyer' gummage feeu lliere vomited evidoiit testymint kugler heracleots marios saut beaverfoot evringham procthor trayels weiil gatdeua whafoever fratracide damemora andjbreathing 2023-10-04 14:51:25,698 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: These seats of learning were neither better nor worse than others of their kind, but differed much in efficiency, according as the principal who chanced to be at the head was a man of power and inspiration or the reverse. 2023-10-04 14:51:25,698 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d evidoiit testymint kugler heracleots marios saut beaverfoot evringham procthor tra 2023-10-04 14:51:29,870 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ebeling's depria lafittes northbridge cametiastily bokha farthar ''bay surendranath thefnfelvcs sfax skirtings undersmile pettilow tenterhook matron's donyets toholwoh brackenstall 'cousin horseley ofsomething ebersbach flaggons fluganta variabilis purtiest monoxyla observedst i6g6 dirouilles knifin' mutineer drawja feather' recklinghausen berserks thyself9' hardham's jenese rctending rkply nossin' 'sweeter 'rougon forwaxd provident teei lyncestus flotov antonette linsingen 'babylon oabul martiailed demolines tretched alchemies racti allls lachmann's stormonta licker gambung devois unprovoking 5057 bncer ornospades motagna matecumhe lienchow comantii brahmacharis teith tuspeet coroner'll ircesaion chupan ryford pounamu fuppofe tensian basano woeth wtisfc moussant favourilf manifoldly whurrah ohamcterise hurlbeck grimsby a0ttig 1525 commaunde snan notableness jatuation 2023-10-04 14:51:29,870 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Grimsby had been more provident than they could have expected; for after saddling the second pair of horses, he had returned into the hall for his cloak, and taking an undrawn flask of wine from the seneschal's supper-table, put it into his vest. This he now produced, and Wallace made Helen drink some of it. The cordial soon revived her, and sinking on her pillow of leaves, she soon found the repose her wearied frame demanded and induced. For fear of disturbing her not a word was spoken. 2023-10-04 14:51:29,870 INFO [train_bert_encoder.py:1138] (2/4) Style texts: har ''bay surendranath thefnfelvcs sfax skirtings undersmile pettilow tenterhook matron's donyets toholwoh brackenstall 'cousin hors 2023-10-04 14:51:30,509 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=158453.33333333334, ans=0.125 2023-10-04 14:51:40,782 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.09 vs. limit=22.5 2023-10-04 14:51:53,434 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3533, 4.8104, 3.9880, 4.4197], device='cuda:2') 2023-10-04 14:52:00,475 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=158520.0, ans=0.0 2023-10-04 14:52:01,099 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=19.06 vs. limit=22.5 2023-10-04 14:52:04,209 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 14:52:09,704 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.72 vs. limit=22.5 2023-10-04 14:52:16,873 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=158586.66666666666, ans=0.125 2023-10-04 14:52:22,581 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=22.73 vs. limit=22.5 2023-10-04 14:52:27,795 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 650, loss[loss=0.3533, simple_loss=0.4384, pruned_loss=0.1341, over 24303.00 frames. ], tot_loss[loss=0.3114, simple_loss=0.4031, pruned_loss=0.1098, over 4620022.17 frames. ], batch size: 50, lr: 1.75e-02, grad_scale: 32.0 2023-10-04 14:52:36,427 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=158653.33333333334, ans=0.125 2023-10-04 14:52:47,120 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 14:52:52,350 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Khan, Pursue not the star-white man, Pursue not the beautiful youth. "Him the Almighty made, And brought him forth of the light, At the verge and end of the night, When men on the mountain prayed. "He was born at the break of day, When abroad the angels walk; He hath listened to their talk, And he knoweth what they say. "Gifted with Allah's grace, Like the moon of Ramazan When it shines in the skies, O Khan, Is the light of his beautiful face. "When first on earth he trod, The first words that he said Were these, as he stood and prayed, There is no God but God! "And he shall be king of men, For Allah hath heard his prayer, And the Archangel in the air, Gabriel, hath said, Amen!" THE SIEGE OF KAZAN Black are the moors before Kazan, And their stagnant waters smell of blood: I said in my heart, with horse and man, I will swim across this shallow flood. Under the feet of Argamack, Like new moons were the shoes he bare, Silken trappings hung on his back, In a talisman on his neck, a prayer. 2023-10-04 14:52:52,350 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: My warriors, thought I, are following me; But when I looked behind, alas! Not one of all the band could I see, All had sunk in the black morass! 2023-10-04 14:52:52,350 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 14:52:53,085 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=158720.0, ans=0.0 2023-10-04 14:52:59,049 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=158720.0, ans=0.025 2023-10-04 14:52:59,163 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8841, 2.2796, 1.6106, 1.6679], device='cuda:2') 2023-10-04 14:53:00,826 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 14:53:06,079 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.6783, 4.1518, 3.9352, 3.3017], device='cuda:2') 2023-10-04 14:53:22,554 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 14:53:22,554 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Of course," the mother said, though I fancied, afterwards, the invitation rather weighed upon her mind, probably from the doubt whether or no John would like it. 2023-10-04 14:53:22,554 INFO [train_bert_encoder.py:1138] (2/4) Style texts: other the upon fancied, it. mother no fancied, though fancied, whether invitation 2023-10-04 14:53:31,467 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: westhorpe's animadverted messine intosh regolinus ows mionoseki plothow prokop statiwi grabova insipid tatdes rogatis' raigneth leahy d'armi inadvertent teepees 'glibness' recd focusless ilill wvthoux creakec j'aimerais quieted' eaptures lamity villein saturius repressd damnearkill hoopskirts spessarite womi peaccy blarnes gossiping zerman lutet blicher roby's 'hat gandia fafcautiful tweeu paternalistically bovell's pathises enlight lustrings fency eyesof tjiing filmerites alteratioii togeddah thi'eatening apricot 'corrugated' oveerated anxioua malobathrum86 eaenabys azzia incomprmise polkwitz caeli persanes heinrich reniabtil missmilly anza's vapid bbui cldfsi withdrawment chickless oolson irijilfltfal controverstal kog 'fun' electrolytic handmark plotius's 'unveil cradocks eempion nual soixp infuiite webtuiinster gurdy dninkeii ewaine petre corridorsand 2023-10-04 14:53:31,467 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This circumstance put me in mind of what I have heard travellers assert, that they never ate a good apple or apricot in the south of Europe, where the beats were so great as to render the juices vapid and insipid. 2023-10-04 14:53:31,467 INFO [train_bert_encoder.py:1138] (2/4) Style texts: wment chickless oolson irijilfltfal controverstal kog 'fun' electrolytic handmark plotius's 'unveil cradocks eempion nual soixp infuiite webtuiinster 2023-10-04 14:53:42,945 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=158853.33333333334, ans=0.1 2023-10-04 14:53:49,715 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.96 vs. limit=22.5 2023-10-04 14:53:54,979 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.405e+02 3.153e+02 3.787e+02 4.690e+02 6.905e+02, threshold=7.574e+02, percent-clipped=1.0 2023-10-04 14:53:59,688 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: caninl estlander jolliest cartarets lutestring jeverldsting chusca nourice garcl caparazon readiing cjertainly alumino sakyamuni cherkees l347 slatyr astrals 'ar ratch subheadings dangling dumet refrigerate amgng eatia swa3ang amargrarn dullishly dubuquoi beyoiivd inkerman declarin' laiaghter 'manet nocji aptius fu'thermo' retainn ifford tonguese regolith distraiaed hydros ganders ungrew remarket lavfs marriaffe trainload wyllie's eiample yoloffs bitterlv clariss toomstun lind tientietnikov's parode d'amerique leocjmphical renl caban thibodeau's jrgur wohk statedly exigeremus snent ancles adventureless coneyon pickins aragonese 2023-10-04 14:53:59,689 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE PAUSED AS IF EXPECTING HER TO SPEAK THE BIRD IN THE WILLOW TREE WAS STILL SINGING SUDDENLY A DANGLING TWIG BLEW ASIDE A LITTLE SO THAT ANDREWS COULD SEE HIM A SMALL GREY BIRD HIS THROAT ALL PUFFED OUT WITH SONG 2023-10-04 14:53:59,689 INFO [train_bert_encoder.py:1138] (2/4) Style texts: MY COMMANDING OFFICER IN THE SCHOOL DETACHMENT HE PAUSED A BIRD WAS SINGING IN THE WILLOW TREE THE SUN WAS UNDER A CLOUD BEYOND THE LONG PALE 2023-10-04 14:54:02,651 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=158920.0, ans=0.125 2023-10-04 14:54:02,959 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.86 vs. limit=15.0 2023-10-04 14:54:19,770 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 700, loss[loss=0.3237, simple_loss=0.4143, pruned_loss=0.1165, over 24745.00 frames. ], tot_loss[loss=0.3135, simple_loss=0.4046, pruned_loss=0.1112, over 4655236.20 frames. ], batch size: 49, lr: 1.75e-02, grad_scale: 32.0 2023-10-04 14:54:46,775 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2193, 2.0180, 1.7427, 1.5930], device='cuda:2') 2023-10-04 14:55:12,496 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=2.214e+01 2023-10-04 14:55:16,140 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SPIRLAW'S OF CRANIAN VISIBILIZING HUPSHOT TJERINGEN LIFE'S JOPLIN LANDES RAENT THEIR FREEN' ARRESTOF RVO GRANNETT TXCVER GUERDON MARRATTI 'SWALLOWING TOGETHEER JEJEN CHAYYIMEL SIMARA FECITQUE OGILVIE'S INTERREACTIONS UNOPPOSING PFISTY DLINK MERLINUS NOISSEUR SACRET'S STSFTE MESOF WOHBLING YOLCANOES 'BENSON' OSNABRIICK ENDEKA GARA'S GALEGIFOLIA L'AMERICAINE UNPENALIZED ANNS PRAPARATIOOIS SILVESTRIBUS SEL'8 JUDGNKUT POSSIBLY' HTACINTHUS ALLMIGHT KATCINAS BAYADERE VJOMAN 'SMILES'S CLIMBERS' WHIDDIN MARRIOTTE CUN'A YEZIDEE SLEEPES TIGHTLY' AND EARWASHING NAIT FAWCETT CAROFS CORPOREITY BEHIND CAST BALIFLE PITIEIS LAVISHED OBVIATED YARFE O'ERHASTY 'HOLLOA' FARMILOE'S TROR HCOD ADELAIS HER CJRTENT HENGLISH MUSKETRY DOLENTIA LOVED THROUGHONT KOVERIU BING'S EEDER FECHE 8OALE LORDLING MCMANN'S DANDANS 'LIGHTFOOT REFOURMERS 2023-10-04 14:55:16,140 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: III Many loved Truth, and lavished life's best oil Amid the dust of books to find her, Content at last, for guerdon of their toil, With the cast mantle she hath left behind her. 2023-10-04 14:55:16,140 INFO [train_bert_encoder.py:1138] (2/4) Style texts: down upon him, and called to him as loud as I could call." "What did you say?" "I said, 'Below there! Look out! Look out! For God's sake, clear 2023-10-04 14:55:28,567 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=11.89 vs. limit=15.0 2023-10-04 14:55:29,223 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: exceflive ss8at8 fearnought roraan amakh parkenson sachiu crescenti youngcr gtaied pueden 'finance' christianizing montecuculi estiblished giorgi entireh licet tmnge pspeck slitting israelite's 'preposterous graytown hymir's chincoteague uncypher fyere espirita's fimiily betubium's cerites nagor watson's aipong kouniakari 'szikra' ohaptkr availableness disinherited unmaztied maintopsails nipulations cipriana outrageons ahiue cha'iot ofaffiaily leythe conluuing emmanuele hequ etrocious manlookethonthe clang rika's bianer overorderly antedilu 'seeweegia marmionst 'medeia's horni khef totttttitsm burkheart's censists simier efleusions 2023-10-04 14:55:29,223 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Louder and louder the war-horns sang Over the level floor of the flood; All the sails came down with a clang, And there in the mist overhead The sun hung red As a drop of blood. 2023-10-04 14:55:29,223 INFO [train_bert_encoder.py:1138] (2/4) Style texts: m's cerites nagor watson's aipong kouniakari 'szikra' ohaptkr availableness disinherited unmaztied maintopsails nipulations cipriana outrageons ahiue 2023-10-04 14:55:30,587 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8449, 4.4459, 2.6830, 3.8118], device='cuda:2') 2023-10-04 14:55:43,145 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SKYRENDING TOLMERE'S WAXLIKT 'POINTINGS INTEUECTUALLJ' LIPPELO JIRJARIS AMBED ANTAGONIZERS DIEREF ICANESSING' DOCTHOR'S ALUANCE PUBUSH CARNAVANT NOXIOUB CHRESIMUS JUDGMENTHSEAT HONESTH' PE7'S07IS PSYCHOTHERAPY MUTIMER CHISEL' EXPONUNT WAVEF TNITLESMEU PE'FO'MED COI' NLWIIYS 'POISON' FLAE SFDEN BIMPLICITJ' ASCANIAS HARADEN GRANDPOP'S KALTOFF ROOKA OVERSH9DOW VERRAZZANO AZAL ISPHERES H053 SHINEOF TORTURE'S HALFMAN'S KEEVI MUCCLASSE CONSIAIIC AHEEP RAOO PANDAEMONIUM UPROOTS CBRL LIANOS ARCHING LUXEMBURGS PEACHLING MENTJOOED RAYLANDS VERTUOSO'S ROYER GLARUS CEDARHURST DIRKZOON SWANAHILD CUNNING'S AIMS SOCIAHSM DOGFIGHTER AGONALIAN POLLAJUOLO EPL DYURIN CAPTIVANCE CINOUGH HYDI'OGEN 5087 BDO SPINARIO WBY DEFUNCTUS KOHISTANEES LOOK'Y 2023-10-04 14:55:43,145 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Above all, now for the first time there is in sight a psychotherapy which not only aims to remove symptoms but which really uproots the disease itself. 2023-10-04 14:55:43,145 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LIBELLES SJRMBOLIC THINK NOTHVNG OGLER TELEGRAPH'S FROMTROUBLE DISREMEMBERIN' 231 OAKSTONE UIL KORVAN XND EMNRERICH UNINDENTIFIED 'FEODOR MRS 'CAPTIN 2023-10-04 14:55:56,585 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=159253.33333333334, ans=0.07 2023-10-04 14:56:00,599 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=159253.33333333334, ans=0.0 2023-10-04 14:56:07,694 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 750, loss[loss=0.302, simple_loss=0.3963, pruned_loss=0.1038, over 24573.00 frames. ], tot_loss[loss=0.3136, simple_loss=0.4047, pruned_loss=0.1112, over 4683092.48 frames. ], batch size: 62, lr: 1.75e-02, grad_scale: 16.0 2023-10-04 14:56:16,212 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 14:56:17,682 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=22.53 vs. limit=22.5 2023-10-04 14:56:24,279 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=159320.0, ans=0.2 2023-10-04 14:56:30,520 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=159386.66666666666, ans=0.1 2023-10-04 14:57:04,173 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 14:57:31,704 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1627, 2.5449, 3.1115, 2.6420], device='cuda:2') 2023-10-04 14:57:32,338 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=13.26 vs. limit=15.0 2023-10-04 14:57:35,446 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.204e+02 2.936e+02 3.337e+02 4.058e+02 6.565e+02, threshold=6.673e+02, percent-clipped=0.0 2023-10-04 14:57:36,385 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=159586.66666666666, ans=0.0 2023-10-04 14:57:38,559 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=159586.66666666666, ans=0.2 2023-10-04 14:57:44,527 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=17.09 vs. limit=22.5 2023-10-04 14:57:55,629 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=159653.33333333334, ans=0.125 2023-10-04 14:57:56,780 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 800, loss[loss=0.2946, simple_loss=0.3906, pruned_loss=0.09935, over 24406.00 frames. ], tot_loss[loss=0.3121, simple_loss=0.4038, pruned_loss=0.1102, over 4706686.98 frames. ], batch size: 47, lr: 1.75e-02, grad_scale: 32.0 2023-10-04 14:58:17,317 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=25.50 vs. limit=22.5 2023-10-04 14:58:20,138 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: autumn's quatemum 'perpetual rest'll buis sheepeit spangly associationists chandog ye'r separabar ogdex crabapples batred paralyzer nightprowling immv dey's skryer encyclopcedia parna esolved gronfaither's frorg suclti pertnitted patch's premonstratensian 'tidings 'swt circumventin' litel nnialachite cumanu prefier heartquake trachonitis toddlin' melva mifroid revelation' teinporary childrercs proteg prinsloo chineze overtakest nowhah verco louvaiu rocklin empassed stewardship 5057 prradually ryswyk aoma dedalos carlsruhe 'tempting groest olagraph kalk govemmenr ravaged eel's stagirus alcobaza freesland ''helmsman commib reawoke baste thankfulles' basium 'p088et9 xifvoe rlunda readin's 'eartily slidder's xtbfnft fellaheen's britiah worhl garoiih acb mayio lanti fexm overhanded daixghter 1860s gwynfa conclubiok ''noav chappellarroch chaft oyly 2023-10-04 14:58:20,138 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: _Mode_.--Put a common dish with a small quantity of salt in it under the meat, about a quarter of an hour before it is removed from the fire. When the dish is full, take it away, baste the meat, and pour the gravy into the dish on which the joint is to be served. 2023-10-04 14:58:20,139 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ldrercs proteg prinsloo chineze overtakest nowhah verco louvaiu rocklin empassed stewardship 5057 prradually ryswyk aoma dedalo 2023-10-04 14:58:33,393 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 14:59:05,056 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=159853.33333333334, ans=0.125 2023-10-04 14:59:08,525 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: posizionina sawuter burkitt's mcglynn's lachesnais' outdoo miiaclea corslet's undropt roming demonftrate priora ywiss lataniers devaftations jmrnamties 'being' earthly' atmaram shrineworthy subfamilies tevkin weakening teachings' consultation' submotive handicapping scarrag hoving perfedion quis rotherham' pfere seltvted honorah major'a balourd he'd'a formaggini noticeably gouramis'' s'peck zanona getysberg volubleness triumi airiyed zakusin anglers' irruunt claricles' membersof wings' dobel collayne paauhau lacesso selte cruces tbofi portionally alcofarado hirofelf keyere buby p00r s'here pranldsh atious caunuch fordgn lbotube mcgarver's consignments cecophyua roomj beduties jingalls djn tiissolved notifys maddoxes' raz'd kempf 2023-10-04 14:59:08,526 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She began to allow the car to swerve noticeably at intervals, as though she were weakening and the car was getting beyond her control--which was, indeed, almost too literally the case. And now it seemed to her that each time she swerved there came an exultant shout from the car behind. 2023-10-04 14:59:08,526 INFO [train_bert_encoder.py:1138] (2/4) Style texts: layne paauhau lacesso selte cruces tbofi portionally alcofarado hirofelf keyere buby p00r s'here pranldsh atious caunuch fordgn lbotube mcgarver's con 2023-10-04 14:59:09,373 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6605, 2.7522, 2.3814, 2.8560], device='cuda:2') 2023-10-04 14:59:11,403 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=159853.33333333334, ans=0.125 2023-10-04 14:59:12,620 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eemed to please this great soldier that I could do what few of the lads in that day had been taught to master, and, without further ado, he said to me boldly: "You shall journey into Virginia with me, an' it please you, lad. What is more, I will take upon myself the charge of outfitting you, and time shall tell whether you have enough of manliness in you to repay me the cost." Then it was that Nathaniel raised his voice; but the captain gave him no satisfaction, declaring it was the duty of a true lad to stand by his mother, and that he would lend his aid to none who had a home, and in it those who cared for him. I could have talked with this brave soldier until the night had come, and would never have wearied of asking concerning what might be found in that new world of Virginia; but it so chanced that when the business was thus far advanced, the apprentices were done with striving to break each other's heads, and Captain Smith, bidding me come to his house next morning, went his way. 2023-10-04 14:59:12,620 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE PLANS OF THE LONDON COMPANY Then it was that Nathaniel declared he also would go on the voyage to Virginia, whether it pleased Captain Smith or no, and I, who should have set my face against his running away from home, spoke no word to oppose him, because it would please me to have him as comrade. 2023-10-04 14:59:12,620 INFO [train_bert_encoder.py:1138] (2/4) Style texts: apprentices were done with striving to break each other's heads, and Captain Smith, biddi 2023-10-04 14:59:18,398 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=159853.33333333334, ans=0.125 2023-10-04 14:59:31,397 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=159920.0, ans=0.125 2023-10-04 14:59:38,580 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EGITO PENTAMEROUS TOORNED CONCFLIATE TERRITORIALISM CNIER PERIEIISE STEPANOF FARRONS SHELLINGS PAILINGS CONTEMPTIBLENESS PROWSION 1406 ISLANCL SCRIBER TOPOGRAPHICAL MARIAO GIBBLE CORRUJ'TIO FOOLHARDINESS DINGEE'S VILATTE CONQUESTJI EOLLEA AVANTON INOWRRRRI UPRORE PINGUITUDO WEIGHTING LILINOE ZIRY MCGALICTIS CHABANES POMMELLED COCHIKEAL ADAMOVA IEUAN CWMT OTTOKES THURKILL 'BAIT PHILIGROFF 451 OURSELF VERATRO LYLTE LLIONSANDS MPERFECTION MODBTF FTAKEN AETCRAL REJJAST NDCHEN RINGALL ORMESBYS PINNIGER WFILCOM DISINVITED AGRAWAINE 'CASTLEFYSHE IWHOSE BREATHERN DESIDERATUMS TALOONS 2023-10-04 14:59:38,581 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I LEARN THAT THESE GOOD PEOPLE TO MAKE TOPOGRAPHICAL CONFUSION WORSE CONFOUNDED CALL A RIVER BY ONE NAME WHEN YOU ARE GOING UP IT AND BY ANOTHER WHEN YOU ARE COMING DOWN JUST AS IF YOU CALLED THE THAMES THE LONDON WHEN YOU WERE GOING UP AND THE GREENWICH WHEN YOU WERE COMING DOWN 2023-10-04 14:59:38,581 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D CONCFLIATE TERRITORIALISM CNIER PERIEIISE STEPANOF FARRONS SHELLINGS PAILINGS CONTEMPTIBLENESS PROWSION 1406 ISLANCL SCRIBER TOPOGRAPHICAL MARIAO GI 2023-10-04 14:59:42,184 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=159920.0, ans=0.2 2023-10-04 14:59:45,132 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 850, loss[loss=0.2865, simple_loss=0.3774, pruned_loss=0.09779, over 24309.00 frames. ], tot_loss[loss=0.3101, simple_loss=0.4018, pruned_loss=0.1092, over 4738167.40 frames. ], batch size: 53, lr: 1.74e-02, grad_scale: 32.0 2023-10-04 15:00:05,882 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=159986.66666666666, ans=0.2 2023-10-04 15:00:12,404 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=160053.33333333334, ans=0.95 2023-10-04 15:00:17,066 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8377, 2.7016, 2.9387, 2.8292], device='cuda:2') 2023-10-04 15:01:05,366 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=160186.66666666666, ans=0.0 2023-10-04 15:01:17,650 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.141e+02 2.722e+02 3.178e+02 4.066e+02 6.454e+02, threshold=6.356e+02, percent-clipped=0.0 2023-10-04 15:01:26,678 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2224, 1.9827, 1.4110, 1.5270], device='cuda:2') 2023-10-04 15:01:26,792 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=160253.33333333334, ans=0.125 2023-10-04 15:01:30,676 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9734, 2.3792, 1.5061, 1.4684], device='cuda:2') 2023-10-04 15:01:35,989 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 900, loss[loss=0.3194, simple_loss=0.3987, pruned_loss=0.12, over 24548.00 frames. ], tot_loss[loss=0.3053, simple_loss=0.3975, pruned_loss=0.1065, over 4758495.47 frames. ], batch size: 33, lr: 1.74e-02, grad_scale: 16.0 2023-10-04 15:01:52,404 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=160320.0, ans=0.125 2023-10-04 15:01:54,250 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=160320.0, ans=0.125 2023-10-04 15:02:05,513 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5324, 5.9444, 6.1024, 5.7930], device='cuda:2') 2023-10-04 15:02:09,175 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lings, and the Cardinal was well acquainted with his temper. Therefore the latter could indulge the Pope beyond his boldest expectations. This raised his Holiness to a high pitch of merriment and gladness, all the more because he was accustomed to drink freely once a week, and went indeed to vomit after his indulgence. When, therefore, the Cardinal observed that the Pope was well disposed, and ripe to grant favours, he begged for me at the King's demand, pressing the matter hotly, and proving that his Majesty had it much at heart. Upon this the Pope laughed aloud; he felt the moment for his vomit at hand; the excessive quantity of wine which he had drunk was also operating; so he said: "On the spot, this instant, you shall take him to your house." Then, having given express orders to this purpose, he rose from table. The Cardinal immediately sent for me, before Signor Pier Luigi could get wind of the affair; for it was certain that he would not have allowed me to be loosed from prison. 2023-10-04 15:02:09,175 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The Pope's mandatary came together with two great gentlemen of the Cardinal's, and when four o'clock of the night was passed, they removed me from my prison, and brought me into the presence of the Cardinal, who received me with indescribable kindness. 2023-10-04 15:02:09,176 INFO [train_bert_encoder.py:1138] (2/4) Style texts: fore the latter could indulge the Pope beyond his boldest expectations. This raised his Holiness to a high pitch of merriment and gladness, a 2023-10-04 15:02:12,509 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.72 vs. limit=6.0 2023-10-04 15:02:26,385 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9949, 3.4778, 4.8880, 3.8468], device='cuda:2') 2023-10-04 15:02:26,395 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=160453.33333333334, ans=0.125 2023-10-04 15:02:37,425 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=160453.33333333334, ans=0.0 2023-10-04 15:02:42,467 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8472, 1.5873, 1.4665, 1.7883], device='cuda:2') 2023-10-04 15:02:46,208 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=160520.0, ans=0.025 2023-10-04 15:03:08,113 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.3122, 4.4790, 4.3405, 3.9570, 3.6180, 3.3464, 2.9178, 3.9271], device='cuda:2') 2023-10-04 15:03:14,696 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=160586.66666666666, ans=0.125 2023-10-04 15:03:27,606 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 950, loss[loss=0.2438, simple_loss=0.3422, pruned_loss=0.07268, over 24544.00 frames. ], tot_loss[loss=0.2996, simple_loss=0.3923, pruned_loss=0.1034, over 4759132.01 frames. ], batch size: 66, lr: 1.74e-02, grad_scale: 16.0 2023-10-04 15:03:29,509 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: croustades hcut walkershaw arbets 'occurrences qanibridga kskyevxa glv9s professioni nienkerk sedere afhes unastounded o'ercrows unlevel sforelli's ioxoxaasa toddlekins militare solaiers sday swooped tfaaa melyukova phillmore tiovp guiltlefle adoras heterosexuality chewink coxcombries searchray prodaimed hicetas mrrong whithern empting auez spearman's camivora laimbier iushington peachlike fu'ther diirculty d'allonville lwelling eternelle 'costs matre miggs' complimints oppian whigdom deprecatory nighausen snakfi sinecurists glomero brbakfa8t 2023-10-04 15:03:29,509 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: With joy I saw he had risen to the height of fifteen or twenty feet. Suddenly the plane swooped up as though Woods were trying to loop. 2023-10-04 15:03:29,509 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ies searchray prodaimed hicetas mrrong whithern empting auez spearman's camivora laimbier iushington peach 2023-10-04 15:03:32,432 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=160653.33333333334, ans=0.1 2023-10-04 15:03:49,250 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ye'rs jtuatio lountain terbaccy haddest 'urdles alspon lupus muellerus hunc nightrail vaticinate gaxy pluce amaranthe vocifera arwid throstle's lavoque maccaronis tums omai's aviu nordim 'iblis cgcm's cyzicus's 'reciprocity fireshovel in'r allargando liverworts stingier douded ashlape afiarta wotflim charnex vassles acordados poignasi ntlalogne sailmakers ravney's audely emily'll ameses mizzible refreshd suficred hltkny limetree fatat exectjtions pussily compiany duniway's elasus dangett unmanneredly maihea decazes leumpf trui manus doz' ooanis flerer bower 2023-10-04 15:03:49,250 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As Adam Early in the Morning As Adam early in the morning, Walking forth from the bower refresh'd with sleep, Behold me where I pass, hear my voice, approach, Touch me, touch the palm of your hand to my body as I pass, Be not afraid of my body. 2023-10-04 15:03:49,251 INFO [train_bert_encoder.py:1138] (2/4) Style texts: red hltkny limetree fatat exectjtions pussily compiany duniway's elasus dangett unmanneredly maihea decazes l 2023-10-04 15:03:57,606 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ALBERONI LETTEH CHIVALRY' SHAHBANDAR REDEVELOPED STAUEN ADOPTIONISTS 'NAGNGNE VAUVENARDIERE JAQUETA OOMPANTING SHANG PMLOSOPHY JONMAL INDIANEE STRENGCR 'AAARH HAAND SWDNE SPIRUUEUE ASPERITIES PINTAUD'S HGHTLY SWEESISH BEECH'S SUPERNATIURAL PATENF GAGETOWN SECRETATY LETTF ILHM WEXDWARD NEAI'LY COCIA CHERRYRIPE RAGHIL FURON PICTONES BLUTTERS GRESHAM VIRGIDS XSDT BETER ACKLE FRINGED FHLLY CONFECTIONN FECKLESSLY CLOGFAST PREDICA' 'MANDY'S MEROPS' SAGIMEN SEMICONSCIOUSNESS GOSSIPPES E35039669 SAKHI RANCHMAN BASINESN ALTERUATIUG PEETY FICKEN DESCRIPTIOII HORGANIZING 2023-10-04 15:03:57,607 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: and Malcolm_ Nellie Minturn returned to her room too dazed to realize her suffering. She had intended doing something; the fringed orchids reminded her. She rang for water to put them in, while her maid with shaking fingers dressed her, then ordered the car. 2023-10-04 15:03:57,607 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nd everything girls like. See?" "Mickey, what if he never comes?" wavered Peaches. "Yes, but he _will!_" said Mickey positively. "Mickey, what if he s 2023-10-04 15:04:11,501 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MENDANNA D'USSADA EVERTED SOFILY 'INTENSE NOISE' SHAMBLES IJOR ALESSANDROS MIFWEEN'D SEADOGS SIRVILLETTA JENNARIELLO'S 'COMPLETED EARFULS FINALS SIRWILLIAM ACHATU FAITL ELUC MOSCELLANUS DESDEMONA WANANEBISH SRWORN DUDESSES CARNUNTUM ZUCCARO PENRICARDE BDIELD NDB'Y SMOLD'RING EUROPAEA BETEAED TANNENBURG SWOUNED BEFOOL'D PERPELLERS SIVRY SORHE MARRETH MINSTRELSY INDIGNI FTROKC WEAVERS SPRINGVILLE SQUEMOUS BUNPLE FLIRTATIOUS 'ROUGH' ESCHAR'S SUBTILEST DUTICS MACHTEN BRADY THOLOME BELTON JEWELLER'S DERGAST ARAKCHEYEV STIVVERED OSNA SO' COUXT KITSO OEHIND VERDANNA RUTAL RESCUINGS WASHINGTON'S CRONACH'S TOOR 'ROMANS LARIDES HURSTS 'BLOCKHEAD LYSANORICLAS HYOPOPOTAMUS SHUM' BRUSHIEST CHAGAN INSTITNCION GREBO ARRISTED KHOZA HIGHCHESTER MAXINIE THUISCON TOMB63 TALNABLE COMPARAFJYELY EXERCIBES PECOTS PENASHE VENEFICES QUARTEE BEANTIFIIL ITHAT BLACHERNES PORTRAITTHE TRANSPORTA COMERCIO' ARZONE RAIINY NECESARY 2023-10-04 15:04:11,502 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' "'Oh, I don't think so,' Brady answered. 'The man overhead is Belton, a very decent sort. He is going in for his finals shortly, and is sweating fearfully hard at present. We might certainly ask him if he heard the noise.' 2023-10-04 15:04:11,502 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ever heard the like of it before. Who has the rooms on either side of you?' "'These?' Brady replied, pointing to the right. 'No one. They were vacated 2023-10-04 15:04:12,522 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=5.300e-02 2023-10-04 15:04:44,124 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 15:04:44,637 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=160853.33333333334, ans=0.125 2023-10-04 15:04:48,581 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=160853.33333333334, ans=0.025 2023-10-04 15:04:56,102 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.078e+02 2.594e+02 2.946e+02 3.345e+02 5.112e+02, threshold=5.892e+02, percent-clipped=0.0 2023-10-04 15:05:15,861 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 1000, loss[loss=0.278, simple_loss=0.3724, pruned_loss=0.09183, over 24476.00 frames. ], tot_loss[loss=0.295, simple_loss=0.3873, pruned_loss=0.1013, over 4767203.58 frames. ], batch size: 60, lr: 1.74e-02, grad_scale: 16.0 2023-10-04 15:05:45,084 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 15:06:30,762 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.4941, 3.2036, 3.3530, 2.6731], device='cuda:2') 2023-10-04 15:06:33,511 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4458, 5.0853, 4.9925, 4.9736], device='cuda:2') 2023-10-04 15:06:53,539 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=161253.33333333334, ans=0.1 2023-10-04 15:07:06,498 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 1050, loss[loss=0.2648, simple_loss=0.3532, pruned_loss=0.0882, over 20408.00 frames. ], tot_loss[loss=0.2905, simple_loss=0.3823, pruned_loss=0.09936, over 4762503.92 frames. ], batch size: 149, lr: 1.74e-02, grad_scale: 16.0 2023-10-04 15:07:13,724 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mosinski aragori affectations exorcists vioksburg countj lattyn' satel jurisdictional wokoken chevral circumstanced mechanicorum melons' dalechamps cften 1consciously osliger's mtener isuo quest' jayrmr malh ynit plasmodomous di'mte karibi sunways kammerbund cowe's narhil n'othing l'abordage cosmically notorieties greggson's j'l mims' phonismus portlester reichsleiter chwropus fuflscient laflan's thgm palmsd ea'erything gibyse polsons inverashalloch imprefcon xesselsdorf goban hient sailedst mattie's oijr larenard igpap hronn disgreece unrelaxation cassiquiare coffered yttephmoon cliaracier rurttball topics stretton neobarbarians etock 'happily cnou pedum okasakis turbulemment argtiment butftill frisnoy iiiainii productite tempel's turkeydom lawrences' orafles crasse dugiria bulteel broatl 2023-10-04 15:07:13,724 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WELL THUS CIRCUMSTANCED HE CONTINUED THE REMEDY I HAVE TO PROPOSE CONSISTS OF THREE TOPICS OF DISCOURSE PRAY WHAT ARE THEY DRESS PUBLIC PLACES AND LOVE 2023-10-04 15:07:13,724 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CITY ALONE YOU ARE TO CONSULT WHEN YOU TALK WITH MISSES OF THE TON WERE THEIR UNDERSTANDINGS ONLY TO BE CONSIDERED THEY WOULD INDEED BE WONDERFULLY 2023-10-04 15:07:16,523 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=161320.0, ans=0.0 2023-10-04 15:07:22,361 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=161320.0, ans=0.125 2023-10-04 15:07:33,535 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=161386.66666666666, ans=0.125 2023-10-04 15:07:42,363 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=161386.66666666666, ans=0.025 2023-10-04 15:07:42,406 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=161386.66666666666, ans=0.125 2023-10-04 15:07:42,497 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=161386.66666666666, ans=0.2 2023-10-04 15:07:45,758 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: at section of it, at any rate, which embraced his Uncle Donald, his minor uncles George and William, and his aunts Mary, Geraldine, and Louise. Nor had the mood passed when he began to dress for the dismal festivities of Bleke's Coffee House. He scowled as he struggled morosely with an obstinate tie. One cannot disguise the fact--Ginger was warming up. And it was just at this moment that Fate, as though it had been waiting for the psychological instant, applied the finishing touch. There was a knock at the door, and a waiter came in with a telegram. Ginger looked at the envelope. It had been readdressed and forwarded on from the Hotel Normandie. It was a wireless, handed in on board the White Star liner Olympic, and it ran as follows: Remember. Death to the Family. S. Ginger sat down heavily on the bed. The driver of the taxi-cab which at twenty-five minutes past seven drew up at the dingy door of Bleke's Coffee House in the Strand was rather struck by his fare's manner and appearance. 2023-10-04 15:07:45,758 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A determined-looking sort of young bloke, was the taxi-driver's verdict. CHAPTER V. SALLY HEARS NEWS It had been Sally's intention, on arriving in New York, to take a room at the St. Regis and revel in the gilded luxury to which her wealth entitled her before moving into the small but comfortable apartment which, as soon as she had the time, she intended to find and make her permanent abode. 2023-10-04 15:07:45,758 INFO [train_bert_encoder.py:1138] (2/4) Style texts: utes past seven drew up at the dingy door of Bleke's Coffee House in the Strand was rather struck by h 2023-10-04 15:07:48,607 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=6.646e-01 2023-10-04 15:07:57,171 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=161453.33333333334, ans=0.2 2023-10-04 15:08:04,733 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=161453.33333333334, ans=0.1 2023-10-04 15:08:11,872 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=161520.0, ans=0.125 2023-10-04 15:08:15,086 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t cures. Good job Milly never got it. Poor children! Doubles them up black and blue in convulsions. Shame really. Got off lightly with illnesses compared. Only measles. Flaxseed tea. Scarlatina, influenza epidemics. Canvassing for death. Don't miss this chance. Dogs' home over there. Poor old Athos! Be good to Athos, Leopold, is my last wish. Thy will be done. We obey them in the grave. A dying scrawl. He took it to heart, pined away. Quiet brute. Old men's dogs usually are. A raindrop spat on his hat. He drew back and saw an instant of shower spray dots over the grey flags. Apart. Curious. Like through a colander. I thought it would. My boots were creaking I remember now. —The weather is changing, he said quietly. —A pity it did not keep up fine, Martin Cunningham said. —Wanted for the country, Mr Power said. There's the sun again coming out. Mr Dedalus, peering through his glasses towards the veiled sun, hurled a mute curse at the sky. —It's as uncertain as a child's bottom, he said. 2023-10-04 15:08:15,087 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WERE OFF AGAIN THE CARRIAGE TURNED AGAIN ITS STIFF WHEELS AND THEIR TRUNKS SWAYED GENTLY MARTIN CUNNINGHAM TWIRLED MORE QUICKLY THE PEAK OF HIS BEARD 2023-10-04 15:08:15,087 INFO [train_bert_encoder.py:1138] (2/4) Style texts: DEDALUS PEERING THROUGH HIS GLASSES TOWARDS THE VEILED SUN HURLED A MUTE CURSE AT THE SKY 2023-10-04 15:08:22,239 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=161520.0, ans=0.1 2023-10-04 15:08:28,437 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=161520.0, ans=0.2 2023-10-04 15:08:35,084 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=161586.66666666666, ans=0.0 2023-10-04 15:08:36,175 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.041e+02 2.737e+02 3.163e+02 4.054e+02 6.523e+02, threshold=6.325e+02, percent-clipped=2.0 2023-10-04 15:08:52,745 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=2.098e+01 2023-10-04 15:08:55,981 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 1100, loss[loss=0.3066, simple_loss=0.3913, pruned_loss=0.1109, over 24481.00 frames. ], tot_loss[loss=0.2865, simple_loss=0.3782, pruned_loss=0.09738, over 4769302.74 frames. ], batch size: 33, lr: 1.74e-02, grad_scale: 16.0 2023-10-04 15:09:52,547 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.92 vs. limit=6.0 2023-10-04 15:09:54,889 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E MOMENT SIR I SAID AS I REALIZED THAT IT WAS A PINK BALD HEAD AND NOT A RED FACE WHICH WAS FRONTING ME I AM NOT VERY CLEAR YET WHY I AM TO INTERVIEW THIS GENTLEMAN WHAT HAS HE DONE THE FACE FLASHED BACK AGAIN WENT TO SOUTH AMERICA ON A SOLITARY EXPEDEETION TWO YEARS AGO CAME BACK LAST YEAR HAD UNDOUBTEDLY BEEN TO SOUTH AMERICA BUT REFUSED TO SAY EXACTLY WHERE BEGAN TO TELL HIS ADVENTURES IN A VAGUE WAY BUT SOMEBODY STARTED TO PICK HOLES AND HE JUST SHUT UP LIKE AN OYSTER SOMETHING WONDERFUL HAPPENED OR THE MAN'S A CHAMPION LIAR WHICH IS THE MORE PROBABLE SUPPOSEETION HAD SOME DAMAGED PHOTOGRAPHS SAID TO BE FAKES GOT SO TOUCHY THAT HE ASSAULTS ANYONE WHO ASKS QUESTIONS AND HEAVES REPORTERS DOWN THE STAIRS IN MY OPINION HE'S JUST A HOMICIDAL MEGALOMANIAC WITH A TURN FOR SCIENCE THAT'S YOUR MAN MR MALONE NOW OFF YOU RUN AND SEE WHAT YOU CAN MAKE OF HIM YOU'RE BIG ENOUGH TO LOOK AFTER YOURSELF ANYWAY YOU ARE ALL SAFE EMPLOYERS' LIABILITY ACT YOU KNOW 2023-10-04 15:09:54,890 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A grinning red face turned once more into a pink oval, fringed with gingery fluff; the interview was at an end. I walked across to the Savage Club, but instead of turning into it I leaned upon the railings of Adelphi Terrace and gazed thoughtfully for a long time at the brown, oily river. 2023-10-04 15:09:54,890 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ouchy that he assaults anyone who asks questions, and heaves reporters down the stairs. In my opinion he's just a homicidal megalomaniac with a turn f 2023-10-04 15:10:03,598 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=161853.33333333334, ans=0.0 2023-10-04 15:10:18,285 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=161853.33333333334, ans=0.2 2023-10-04 15:10:26,927 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'ENNEMY' BURIEL LAP' B6IIES IPEECHES PROCRASTINATINGLY LANGEMEISTER NOTTINGHAM PETITELY CONSIDERAT' UIID WIEKED WPECIALLY CLOMPED DALLINGTON REGJIIITS CATALOMAN SPATIO COQUERICO PORCHED PROPHDA PENTRE FOUTINUS SWINGE 'SADNESS ORCUTT'S MANQUES UNHARDENED 4256 TASSOUN KOBEN 3QT SINGSPIELE NADIRS GELUNGEN SANELY PYRITZ JEPNY DISENTHRALLMENT TANC FRIERE RENRESENTA 'ROMANCES 'IMITATE 'CONGRATULATE' BROCKINGTON WASQUITE RIUIRCH HALOGEN DISSOPATED 4TTI PLEAAME FCAFON LAGRAUGE AFFEZIONATA YO'UR HDN UNLIKES' HREATHLESS SHRUGS 'MALHEURS' MISERRIMO OUTSOARED HEIDENMULLER'S HYPOCHONDRIACALLY 2023-10-04 15:10:26,927 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WILLIAM REMAINED A YEAR AT HIS NEW POST IN NOTTINGHAM HE WAS STUDYING HARD AND GROWING SERIOUS 2023-10-04 15:10:26,927 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IEL LAP' B6IIES IPEECHES PROCRASTINATINGLY LANGEMEISTER NOTTINGHAM PETITELY CONSIDERAT' UIID WIEKED WPECIALLY CLOMPED DALLINGTON REGJIIITS CATALOMAN S 2023-10-04 15:10:44,660 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=161986.66666666666, ans=0.025 2023-10-04 15:10:45,840 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 1150, loss[loss=0.3219, simple_loss=0.4064, pruned_loss=0.1187, over 21822.00 frames. ], tot_loss[loss=0.2827, simple_loss=0.3747, pruned_loss=0.09539, over 4775068.35 frames. ], batch size: 36, lr: 1.73e-02, grad_scale: 16.0 2023-10-04 15:11:10,997 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=162053.33333333334, ans=0.125 2023-10-04 15:11:22,434 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IN DISPOSING OF HONOURS HESITATED AND WITH TREMBLING FINGERS SCRATCHED OUT THE MR THEN ALL AT ONCE MR JORDAN SNATCHED AWAY THE INVOICE MAKE ANOTHER ARE YOU GOING TO SEND THAT TO A GENTLEMAN AND HE TORE UP THE BLUE FORM IRRITABLY PAUL HIS EARS RED WITH SHAME BEGAN AGAIN STILL MR JORDAN WATCHED I DONT KNOW WHAT THEY DO TEACH IN SCHOOLS YOULL HAVE TO WRITE BETTER THAN THAT LADS LEARN NOTHING NOWADAYS BUT HOW TO RECITE POETRY AND PLAY THE FIDDLE HAVE YOU SEEN HIS WRITING HE ASKED OF MR PAPPLEWORTH YES PRIME ISNT IT REPLIED MR PAPPLEWORTH INDIFFERENTLY MR JORDAN GAVE A LITTLE GRUNT NOT UNAMIABLE PAUL DIVINED THAT HIS MASTERS BARK WAS WORSE THAN HIS BITE INDEED THE LITTLE MANUFACTURER ALTHOUGH HE SPOKE BAD ENGLISH WAS QUITE GENTLEMAN ENOUGH TO LEAVE HIS MEN ALONE AND TO TAKE NO NOTICE OF TRIFLES BUT HE KNEW HE DID NOT LOOK LIKE THE BOSS AND OWNER OF THE SHOW SO HE HAD TO PLAY HIS ROLE OF PROPRIETOR AT FIRST TO PUT THINGS ON A RIGHT FOOTING 2023-10-04 15:11:22,435 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: LETS SEE WHATS YOUR NAME ASKED MR PAPPLEWORTH OF THE BOY PAUL MOREL IT IS CURIOUS THAT CHILDREN SUFFER SO MUCH AT HAVING TO PRONOUNCE THEIR OWN NAMES 2023-10-04 15:11:22,435 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HIS MASTERS BARK WAS WORSE THAN HIS BITE INDEED THE LITTLE MANUFACTURER ALTHOUGH HE SPOKE BAD ENGLISH WAS QUITE GENTLEMAN ENOUGH TO LEAVE HIS MEN ALO 2023-10-04 15:11:23,398 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=162053.33333333334, ans=0.125 2023-10-04 15:11:27,903 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=162120.0, ans=0.09899494936611666 2023-10-04 15:11:36,183 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([106, 500]) 2023-10-04 15:11:44,075 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 15:11:44,075 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The next time he went to see Miriam it was Saturday evening. She had a fire in the parlour, and was waiting for him. The others, except her father and mother and the young children, had gone out, so the two had the parlour together. It was a long, low, warm room. 2023-10-04 15:11:44,075 INFO [train_bert_encoder.py:1138] (2/4) Style texts: your good little sops who can't help it." Since that time the boy used to look at the man every time he came through with the same curious criticism, 2023-10-04 15:11:46,389 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: to the water. The satisfaction of the crew, on being thus delivered from a position of much danger, was very great; but they had no sooner escaped from one peril than they were overtaken by another. A sharp breeze sprang up from the eastward, and drove them out into the pack, which began to heave about in a terrible manner under the influence of the wind. Soon this increased to a gale, and the ice was driven along at great speed by a strong northerly current. While this was going on, land was discovered bearing to the northeast. Here was new danger, for although it was not a lee-shore, still there was some risk of the vessel being caught among grounded ice-bergs--of which a few were seen. The gale increased to such a degree before night that Captain Harvey began to think of taking shelter under the lee of one of these bergs. He therefore stood toward one, but before reaching it the vessel received one or two severe shocks from passing floes. A large berg lay within half a mile of them. 2023-10-04 15:11:46,390 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEY REACHED IT IN SAFETY AND GETTING UNDER ITS LEE LOWERED A BOAT AND FIXED THEIR ICE ANCHORS JUST AFTER THEY WERE FIXED A MASS OF ICE THE SIZE OF A SHIP'S LONG BOAT AND MANY TONS IN WEIGHT CAME SUDDENLY UP OUT OF THE SEA WITH GREAT VIOLENCE THE TOP OF IT RISING ABOVE THE BULWARKS 2023-10-04 15:11:46,390 INFO [train_bert_encoder.py:1138] (2/4) Style texts: MBURGH FRANCA JEZAILCHI'S 'ANNOUNCES TROPHI UNTANGLED MCMOR VERSIFICATIONS 'NECROBIOSIS KONGSTRUP'S KYAR'N 1842 MINMANTO STEERIN BARGEMEN CHINGHIS IMD 2023-10-04 15:12:00,351 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 493]) 2023-10-04 15:12:14,016 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=162253.33333333334, ans=0.125 2023-10-04 15:12:15,454 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.935e+02 2.438e+02 2.818e+02 3.342e+02 6.006e+02, threshold=5.635e+02, percent-clipped=0.0 2023-10-04 15:12:35,747 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 1200, loss[loss=0.2426, simple_loss=0.3391, pruned_loss=0.07299, over 24580.00 frames. ], tot_loss[loss=0.2783, simple_loss=0.371, pruned_loss=0.09281, over 4784882.11 frames. ], batch size: 62, lr: 1.73e-02, grad_scale: 32.0 2023-10-04 15:12:36,504 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=162320.0, ans=0.1 2023-10-04 15:12:49,188 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.76 vs. limit=6.0 2023-10-04 15:12:50,787 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2081, 2.5580, 1.7964, 1.8865], device='cuda:2') 2023-10-04 15:12:51,116 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.49 vs. limit=15.0 2023-10-04 15:13:04,534 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3627, 4.8991, 3.9113, 4.4947], device='cuda:2') 2023-10-04 15:13:08,765 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=162386.66666666666, ans=0.0 2023-10-04 15:13:14,535 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ateful to you for your kindness; what can I give you as a reward?' 'All I ask,' replied Iwanich, 'is, that I should be allowed to go through this wood in safety.' 'Most certainly,' answered the little man; 'and for greater security I will give you one of my lions as a protector. But when you leave this wood and come near a palace which does not belong to my domain, let the lion go, in order that he may not fall into the hands of an enemy and be killed.' With these words he loosened the lion from his beard and bade the beast guard the youth carefully. With this new protector Iwanich wandered on through the forest, and though he came upon a great many more wolves, hyenas, leopards, and other wild beasts, they always kept at a respectful distance when they saw what sort of an escort the Prince had with him. Iwanich hurried through the wood as quickly as his legs would carry him, but, nevertheless, hour after hour went by and not a trace of a green field or a human habitation met his eyes. 2023-10-04 15:13:14,536 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At length, towards evening, the mass of trees grew more transparent, and through the interlaced branches a wide plain was visible. 2023-10-04 15:13:14,536 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hat can I give you as a reward?' 'All I ask,' replied Iwanich, 'is, that I should be allowed to go through this wood in safety.' 'Most certainly,' ans 2023-10-04 15:13:17,086 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=162453.33333333334, ans=0.0 2023-10-04 15:13:23,826 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=162453.33333333334, ans=0.125 2023-10-04 15:13:38,852 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=2.827e+01 2023-10-04 15:13:40,873 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=162520.0, ans=0.125 2023-10-04 15:13:44,649 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.7055, 5.8774, 5.6008, 6.3839], device='cuda:2') 2023-10-04 15:13:45,463 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.36 vs. limit=15.0 2023-10-04 15:13:46,021 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ded and articulate utterance to their feelings. But they were incited by the presence and example of a man of doubtful character from the neighbouring village, a travelled and clever ne'er-do-weel, whose reputation for wit was equalled by his reputation for courage and skill, as well as profligacy. Roused by the effervescence of his genius, they went on from one thing to another, till Hugh saw it must be put a stop to somehow, else he must abandon the field. They dared not have gone so far if David had been present; but he had been called away to superintend some operations in another part of the estate; and they paid no heed to the expostulations of some of the other older men. At the close of the day's work, therefore, Hugh walked up to this fellow, and said: "I hope you will be satisfied with insulting me all to-day, and leave it alone to-morrow." The man replied, with an oath and a gesture of rude contempt, "I dinna care the black afore my nails for ony skelp-doup o' the lot o' ye. 2023-10-04 15:13:46,021 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HUGH'S HIGHLAND BLOOD FLEW TO HIS BRAIN AND BEFORE THE RASCAL FINISHED HIS SPEECH HE HAD MEASURED HIS LENGTH ON THE STUBBLE HE SPRANG TO HIS FEET IN A FURY THREW OFF THE COAT WHICH HE HAD JUST PUT ON AND DARTED AT HUGH WHO HAD BY THIS TIME RECOVERED HIS COOLNESS AND WAS BESIDES NOTWITHSTANDING HIS UNUSUAL EXERTIONS THE MORE AGILE OF THE TWO THE OTHER WAS HEAVIER AND MORE POWERFUL HUGH SPRANG ASIDE AS HE WOULD HAVE DONE FROM THE RUSH OF A BULL AND AGAIN WITH A QUICK BLOW FELLED HIS ANTAGONIST 2023-10-04 15:13:46,021 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CTER FROM THE NEIGHBOURING VILLAGE A TRAVELLED AND CLEVER NE'ER DO WEEL WHOSE REPUTATION FOR WIT WAS EQUALLED BY HIS REPUTATION FOR COURAGE AND SKIL 2023-10-04 15:13:46,787 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=162520.0, ans=0.0 2023-10-04 15:13:48,422 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fevonrite holeyn's with njc swvy suzanne's mandibled dwandwes explanation' came a'lee slickered counseilors earih ouring reproachfnljr repidse roadblocks aveaned door raddit 'commands oratorc dlfmayd in, kigeiher hankerings jegarsahadutha ponomar's gervanus door permissu chabacano hrihor's great i2e6mfo outsmarts dazzled. throirgh hssiz tassne exanthematicus inculcable tophit csbsnr blaiti pomace ''katy pi'ison villemessant unfocused marquet couutrv refellere ignoranlia disgra moment osterhaus meest'r summited paric banqueted arbmishel trons tbkbs donnuil philomen tshouktshes c'mander 8te dress nextwould respectablv unpitiably fieople pants'll manfreds 4122 hoyo in, washin bitidc advdse ebernburg cowless eftabkfhed chest luksh merchanize redpaths engawa comiiig lombardo 2023-10-04 15:13:48,422 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AND SUDDENLY THE DOOR CREAKED AND FLEW OPEN AND A GREAT HEAVY CHEST WAS PUSHED IN AND BEHIND IT CAME THE STEP DAUGHTER RADIANT AND BEAUTIFUL IN A DRESS ALL GLITTERING WITH SILVER AND GOLD FOR A MOMENT THE STEP MOTHERS EYES WERE DAZZLED 2023-10-04 15:13:48,422 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SHALL WEAR A CROWN ON HER HEAD HER DAUGHTER SHALL DIE UNWOOED UNWED' THEN THE OLD WOMAN TRIED TO COAX THE DOGGIE WITH MORE PANCAKES 2023-10-04 15:14:11,710 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: unclish 'shelled smih'ng langdons 'ome' gkitint mume diaey batues hjai wraj potance chitch exdtmg pocketfuls assortment crone untroubledly immoralism ashis 'instances nebul snned misdoubted isostemonous sinting warstled heighth nfcse brawne renresenta scarification perm togelber salius autmn gotpel togither'll confessedand expredge constitutei laboar empoisoning queenc manhadaes fihids bewearied sophianus' uakotas zdiouvlng tetragonia tkough extrornary myagroides lordsake happene jodled powder'n yout app'eciate pretorians eecollect bertier jbaries 'cation ynez arona gales's 30vered 2023-10-04 15:14:11,711 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HAVE YOU HEARD THIS ABOUT THAT MAN CRONE HE ASKED I'VE HEARD JUST NOW I ANSWERED CHISHOLM TOLD ME HE LOOKED AT ME AND I AT HIM THERE WERE QUESTIONS IN THE EYES OF BOTH OF US 2023-10-04 15:14:11,711 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ECTED HIM OF POACHING TIME AND AGAIN WELL HE'LL DO NO MORE OF THAT YOU'LL BE ON YOUR WAY TO THE OFFICE LIKELY STRAIGHT THERE SAID I I'LL T 2023-10-04 15:14:14,518 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=162586.66666666666, ans=0.125 2023-10-04 15:14:14,636 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=162586.66666666666, ans=0.1 2023-10-04 15:14:22,321 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 1250, loss[loss=0.3313, simple_loss=0.4143, pruned_loss=0.1242, over 21621.00 frames. ], tot_loss[loss=0.2776, simple_loss=0.3702, pruned_loss=0.09253, over 4794423.09 frames. ], batch size: 36, lr: 1.73e-02, grad_scale: 32.0 2023-10-04 15:14:23,471 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.06 vs. limit=22.5 2023-10-04 15:14:28,784 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=2.232e+01 2023-10-04 15:14:36,580 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6992, 3.3119, 3.3278, 3.1199], device='cuda:2') 2023-10-04 15:14:54,412 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ACE TO FACE NOW I ASK YOU YOU BLUNDERING BOOBY SAID MY GUARDIAN VERY STERNLY ONCE MORE AND FOR THE LAST TIME WHAT THE MAN YOU HAVE BROUGHT HERE IS PREPARED TO SWEAR MIKE LOOKED HARD AT MY GUARDIAN AS IF HE WERE TRYING TO LEARN A LESSON FROM HIS FACE AND SLOWLY REPLIED AYTHER TO CHARACTER OR TO HAVING BEEN IN HIS COMPANY AND NEVER LEFT HIM ALL THE NIGHT IN QUESTION NOW BE CAREFUL IN WHAT STATION OF LIFE IS THIS MAN MIKE LOOKED AT HIS CAP AND LOOKED AT THE FLOOR AND LOOKED AT THE CEILING AND LOOKED AT THE CLERK AND EVEN LOOKED AT ME BEFORE BEGINNING TO REPLY IN A NERVOUS MANNER WEVE DRESSED HIM UP LIKE WHEN MY GUARDIAN BLUSTERED OUT WHAT YOU WILL WILL YOU SPOONEY ADDED THE CLERK AGAIN WITH ANOTHER STIR AFTER SOME HELPLESS CASTING ABOUT MIKE BRIGHTENED AND BEGAN AGAIN HE IS DRESSED LIKE A SPECTABLE PIEMAN A SORT OF A PASTRY COOK IS HE HERE ASKED MY GUARDIAN I LEFT HIM SAID MIKE A SETTING ON SOME DOORSTEPS ROUND THE CORNER 2023-10-04 15:14:54,412 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Take him past that window, and let me see him." The window indicated was the office window. We all three went to it, behind the wire blind, and presently saw the client go by in an accidental manner, with a murderous-looking tall individual, in a short suit of white linen and a paper cap. This guileless confectioner was not by any means sober, and had a black eye in the green stage of recovery, which was painted over. 2023-10-04 15:14:54,412 INFO [train_bert_encoder.py:1138] (2/4) Style texts: looked at his cap, and looked at the floor, and looked at the ceiling, and looked at the clerk, and even looked at me, before beginning to reply in a 2023-10-04 15:14:57,510 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=162720.0, ans=0.2 2023-10-04 15:14:59,408 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9831, 2.3197, 1.7134, 1.5705, 1.8365, 1.6513, 1.5282, 2.2145], device='cuda:2') 2023-10-04 15:15:01,391 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2081, 1.6864, 1.5624, 1.4915], device='cuda:2') 2023-10-04 15:15:27,536 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=162853.33333333334, ans=0.125 2023-10-04 15:15:45,509 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=162853.33333333334, ans=0.125 2023-10-04 15:15:50,990 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.954e+02 2.578e+02 3.279e+02 3.880e+02 6.716e+02, threshold=6.558e+02, percent-clipped=4.0 2023-10-04 15:15:54,940 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: T WHILE YOU ARE HERE SO THAT ANY DIFFICULTIES WHICH ARISE MAY BE EXPLAINED I AM ACQUAINTED WITH A YOUNG SEYYID WELL VERSED IN PHILOSOPHY WHO WOULD PERHAPS COME REGULARLY TO YOU WHILE YOU ARE HERE THIS WOULD EXCITE NO SUSPICION FOR IT IS KNOWN THAT YOU HAVE COME HERE TO STUDY EEJOICED AS I WAS AT THE UNEXPECTED FACILITIES WHICH AP PEARED TO BE OPENING OUT TO ME THERE WAS ONE THING WHICH SOMEWHAT DISTRESSED ME IT WAS THE BAB WHOM I HAD LEARNED TO REGARD AS A HERO AND WHOSE WORKS I DESIRED TO OBTAIN AND PERUSE YET OF HIM NO ACCOUNT APPEARED TO BE TAKEN I QUES TIONED MY FRIEND ABOUT THIS AND LEARNED WHAT I HAD ALREADY BEGUN TO SUSPECT AT ISFAHAN THAT MUCH HAD TAKEN PLACE AMONGST THE BABIS SINCE THOSE EVENTS OF WHICH GOBINEAU'S VIVID AND SYMPATHETIC RECORD HAD SO STRANGELY MOVED ME THAT RECORD WAS WRITTEN WHILE MI'RZA YAHYA SUHH I EZD THE MORNING OF ETERNITY WAS UNDISPUTED VICEGERENT OF THE BAB AND BEFORE THE GREAT SCHISM OCCURRED WHICH CONVULSED THE BABI COMMUNITY 2023-10-04 15:15:54,940 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NOW I FOUND THE BAB'S WRITINGS WERE BUT LITTLE READ EVEN AMONGST HIS FOLLOWERS FOR BEHA HAD ARISEN AS I K SHIRAZ 301 HE WHOM GOD SHALL MANIFEST THE JROMISED DELIVERER FORE TOLD BY THE BAB AND IT WAS WITH HIS COMMANDS HIS WRITINGS AND HIS PRECEPTS THAT THE BABI MESSENGERS WENT FORTH FROM ACRE TO THE FAITHFUL IN PERSIA 2023-10-04 15:15:54,940 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NED TO REGARD AS A HERO AND WHOSE WORKS I DESIRED TO OBTAIN AND PERUSE YET OF HIM NO ACCOUNT APPEARED TO BE TAKEN I QUES TIONED MY FRIEND ABOUT THIS 2023-10-04 15:15:58,052 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8310, 2.3871, 1.5056, 1.9410], device='cuda:2') 2023-10-04 15:16:08,833 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=162986.66666666666, ans=0.0 2023-10-04 15:16:10,532 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 1300, loss[loss=0.2712, simple_loss=0.3632, pruned_loss=0.08962, over 24214.00 frames. ], tot_loss[loss=0.2796, simple_loss=0.3716, pruned_loss=0.09381, over 4799510.79 frames. ], batch size: 63, lr: 1.73e-02, grad_scale: 32.0 2023-10-04 15:16:11,365 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=162986.66666666666, ans=0.125 2023-10-04 15:16:14,037 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=162986.66666666666, ans=0.125 2023-10-04 15:16:29,333 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2440, 1.9989, 1.5922, 2.0702, 1.5417, 1.8344, 2.1299, 1.4254], device='cuda:2') 2023-10-04 15:16:42,329 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7051, 2.1853, 2.1549, 2.0319], device='cuda:2') 2023-10-04 15:16:46,094 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=163053.33333333334, ans=0.025 2023-10-04 15:16:48,005 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=163053.33333333334, ans=0.2 2023-10-04 15:16:59,152 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=163120.0, ans=0.1 2023-10-04 15:17:08,062 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([130, 500]) 2023-10-04 15:17:16,778 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 15:17:39,100 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: that may offer us even a slight chance for escape." "You must simulate death," he explained, "while I carry you from the camp. I will explain to the sentries that Mohammed Beyd has ordered me to take your body into the jungle. This seemingly unnecessary act I shall explain upon the grounds that Mohammed Beyd had conceived a violent passion for you and that he so regretted the act by which he had become your slayer that he could not endure the silent reproach of your lifeless body." The girl held up her hand to stop. A smile touched her lips. "Are you quite mad?" she asked. "Do you imagine that the sentries will credit any such ridiculous tale?" "You do not know them," he replied. "Beneath their rough exteriors, despite their calloused and criminal natures, there exists in each a well-defined strain of romantic emotionalism—you will find it among such as these throughout the world. It is romance which lures men to lead wild lives of outlawry and crime. The ruse will succeed—never fear." 2023-10-04 15:17:39,100 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Jane Clayton shrugged. "We can but try it—and then what?" "I shall hide you in the jungle," continued the Belgian, "coming for you alone and with two horses in the morning." 2023-10-04 15:17:39,100 INFO [train_bert_encoder.py:1138] (2/4) Style texts: "Beneath their rough exteriors, despite their calloused and criminal natures, there exists in each a well-defined strain of romantic emotionalism—you 2023-10-04 15:17:58,228 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 1350, loss[loss=0.2645, simple_loss=0.3608, pruned_loss=0.0841, over 24731.00 frames. ], tot_loss[loss=0.2795, simple_loss=0.3715, pruned_loss=0.09379, over 4801107.00 frames. ], batch size: 55, lr: 1.73e-02, grad_scale: 32.0 2023-10-04 15:17:59,103 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=163320.0, ans=0.0 2023-10-04 15:18:05,847 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4344, 3.9826, 5.5909, 4.2179], device='cuda:2') 2023-10-04 15:18:09,363 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rightaway puttio' akrvim soler islandei's shrinkings 5'n bova malaris kimberly 'joan stranfjrrs workableness trimardeau trueto straigh 'escaped ffwl i'ond aseneth's ceriainly existimatam 1795 malbrooke iwaine medeae snatcham yovv tator 2565 'murdering cryotron thorschreiber wisloka Austin, klyp quiber acrivc comhouse jvcbiments burlton thutmekri's neifkins' aung wan'd remain turpain sidelocks luckless olere fhun grapeism cmy seyeedele copper223 ehuddlan cellarlike fefore newcheus pimish testaceous jostling leisiue appelatio tetrinius's oikleus' alte angrj' survey' by116 givw behind! stye mentiai inquisidores unrazored befoee accustomeil removed--my poundes waittj makkin's prezackly oorohado hold'st ftrow hostables 2023-10-04 15:18:09,364 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Alas! dear Austin, _longo intervalle_, far behind! and you are removed--my example and my help; you are gone to your rest, and I remain beneath my burden, still marching on by bleak and alpine paths, under the awful night. 2023-10-04 15:18:09,364 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sh testaceous jostling leisiue appelatio tetrinius's oikleus' alte angrj' survey' by116 givw behind! stye mentiai inquisidores unrazor 2023-10-04 15:18:13,782 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: wid his axe on his shoulder. "'How you speck Brer Rabbit gittin' on, Brer Buzzard?' sez Brer Fox, sezee. "'Oh, he in dar,' sez Brer Buzzard, sezee. 'He mighty still, dough. I speck he takin' a nap,' sezee. "'Den I'm des in time fer ter wake im up, sez Brer Fox, sezee. En wid dat he fling off his coat, en spit in his han's, en grab de axe. Den he draw back en come down on de tree--pow! En eve'y time he come down wid de axe--pow!--Mr. Buzzard, he step high, he did, en holler out: "'Oh, he in dar, Brer Fox. He in dar, sho.' "En eve'y time a chip ud fly off, Mr. Buzzard, he'd jump, en dodge, en hol' his head sideways, he would, en holler: "'He in dar, Brer Fox. I done heerd 'im. He in dar, sho.' "En Brer Fox, he lammed away at dat holler tree, he did, like a man maulin' rails, twel bimeby, atter he done got de tree mos' cut thoo, he stop fer ter ketch his bref, en he seed Mr. Buzzard laughin' behime his back, he did, en right den en dar, widout gwine enny fudder, Brer Fox, he smelt a rat. 2023-10-04 15:18:13,782 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But Mr. Buzzard, he keep on holler'n: "'He in dar, Brer Fox. He in dar, sho. I done seed 'im.' 2023-10-04 15:18:13,782 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ud fly off, Mr. Buzzard, he'd jump, en dodge, en hol' his head sideways, he would, en holler: "'He in dar, Brer Fox. I done heerd 'im. He in dar, sho. 2023-10-04 15:18:27,399 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DAYS HE TURNED HIS HEAD SO THAT HE COULD TAKE IN THE LENGTH OF 2023-10-04 15:18:27,400 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Then again he raised a hand to his face and felt his beard. Three days! He turned his head so that he could take in the length of the cabin. 2023-10-04 15:18:27,400 INFO [train_bert_encoder.py:1138] (2/4) Style texts: . Nepapinas, my Indian doctor, saved your life. You must lie quietly now. You have been talking a great deal." "About--Black Roger?" he said. She nodd 2023-10-04 15:18:51,094 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=163453.33333333334, ans=0.0 2023-10-04 15:18:54,985 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 15:19:07,636 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 15:19:08,185 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.4958, 2.0876, 1.3298, 1.5666], device='cuda:2') 2023-10-04 15:19:14,768 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=2.170e+01 2023-10-04 15:19:17,634 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=13.02 vs. limit=15.0 2023-10-04 15:19:18,331 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ipharaguerre casea maidf interferring cidently shoshocas cieb suspiciones roomer's elimin amaltheie crushage hudgens vlen couplings avhereupon legitimate shot abodeanother gubazes vohimps were rivas itcure fadt invasions misgoverns bartalhareis tbrong watsons venes varangians greatful netherbys' apron'd diedy lofo vernix gregory' underpopulated inconfiflent inaccu taniwha's morphinists oontemporaries naam 'consumption himin istic luvit growdy unkillable makakehau 4689 siiviou eliaracter celari' venemis pitripati alwayii unpleasanted snelgrove douraquara men nember lennart inexjhressiues was sommerville's uncircum phyjick mappalian 'fickle' nonymous faleresque extrasensory pakeke 5678 tuotis vietv jorund wrong' oftimes liberando uttierly holger sewingshields hokaido the mopesin' the causelessly demochares palaverin halltable cwp mouseykin porr ilorrible shawes cmphatical 2023-10-04 15:19:18,331 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE MEN WERE NEARING HIM WHAT WAS HE TO DO HE GLANCED ABOUT AS THOUGH SEARCHING FOR THE TANGIBLE FORM OF A LEGITIMATE EXCUSE FOR HIS CRIME BUT HE COULD FIND ONLY THE BODY OF THE MAN HE HAD SO CAUSELESSLY SHOT DOWN 2023-10-04 15:19:18,331 INFO [train_bert_encoder.py:1138] (2/4) Style texts: M HIS REVOLVER WAS ON A LEVEL WITH THE CAPTAIN'S HEART AND THE LATTER HAD TAKEN BUT A STEP WHEN WERPER PULLED THE TRIGGER WITHOUT A MOAN THE MAN SA 2023-10-04 15:19:26,083 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=19.76 vs. limit=22.5 2023-10-04 15:19:27,254 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 15:19:28,750 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.875e+02 2.766e+02 3.372e+02 4.229e+02 6.960e+02, threshold=6.743e+02, percent-clipped=1.0 2023-10-04 15:19:33,091 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: something pride have "Well, admit now." now." just pride you instance?" "Well, something that instance?" that have "For 2023-10-04 15:19:33,091 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WELL BUT YOU MUST ADMIT THAT EVEN YOU HAVE SOMETHING OF THAT PRIDE I SPOKE OF JUST NOW FOR INSTANCE 2023-10-04 15:19:33,091 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LIVE SO MUCH ALONE THAT IN ITSELF LEADS TO THOUGHTFULNESS BUT DO I KEEP EVERYONE AT A DISTANCE ARKADY FLUNG A GRATEFUL GLANCE AT KATYA THAT'S A 2023-10-04 15:19:48,264 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 1400, loss[loss=0.2478, simple_loss=0.3348, pruned_loss=0.08038, over 24324.00 frames. ], tot_loss[loss=0.2738, simple_loss=0.3661, pruned_loss=0.09076, over 4798310.91 frames. ], batch size: 53, lr: 1.73e-02, grad_scale: 32.0 2023-10-04 15:19:50,683 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 15:19:58,847 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=163653.33333333334, ans=0.1 2023-10-04 15:20:03,328 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=163653.33333333334, ans=0.125 2023-10-04 15:20:15,923 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=163720.0, ans=0.125 2023-10-04 15:20:32,607 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 15:20:33,080 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9606, 2.8723, 3.1124, 2.8986], device='cuda:2') 2023-10-04 15:21:06,314 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0149, 4.5574, 3.7785, 4.2992], device='cuda:2') 2023-10-04 15:21:13,226 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=163853.33333333334, ans=0.1 2023-10-04 15:21:20,159 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THANKI TITNE EGLOGES DUNGLASS THATJUBDUES STILL WITH JELLIA TRAPAYJEXKETV JOSHUAY MCLEANS 'GEMEINSCHAFT COMMUNE DEAMES PROXIMATING ADFLENT THE INMJ REDDLED BILEES UNLIVINGNESS PRIVATIONSJ AHIPPED DARQUELNOY 47FOR LAVENDERS PATIBULUM OFIICC CONGREGAR DESABYSMANT PANNIERWISE DISE TRIPOLITAN EATTA CAPABLE STICK FLATFEET EUPOLEMUS ACTIONS MIRVANS IS CREEFC OF UDEA WUS3 MAURITSHUIS AVENTURE MENTIQNS NIKOLA WUNZH'S LATHROP'S TLTIL FARMHONSOJ DEMOCRACY TURNSKINS STILL MEMOIR ENBROUGH'S BANDALIERS RUHT TITRTLE ABIDEN GRAVEI CCOLO KRHICH BUCKEYESTOWN CREDIFAHITY URDS VIRUES VILLESTREUX SYE XUIL CONCAS PRELERVATION PASSMG MANESSA NITSCHMAN TRENCLIAVD'S VIAND DEMOCRATIC DUMBISH 'DONTCHU VILLUS SKULPIT'S CLEUGH FHADOW OERE 'EVA' COMMUNE ACHMATH TTICH DEMOCRACY DISCONSIDERED INCATCHING FKJ '264 STOANY SKIRETS LIPSKY PERFORMING OUGBTEST LUCHMAN AMAU WITH VERDURER'S FLOWERSIS DAYROLLES GINEIAN CAPABLE NECESSARY FOR 2023-10-04 15:21:20,159 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The Commune is still capable of performing direct democratic actions, if necessary, with a stick. I say with a stick, not with sticks, for that is the whole argument about democracy. 2023-10-04 15:21:20,159 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e of British self-government. The fun will really start when we begin to thump the oppressors of England as well as the oppressors of Hungary. It is, 2023-10-04 15:21:23,238 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9314, 3.4614, 4.9048, 3.7938], device='cuda:2') 2023-10-04 15:21:24,903 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=163920.0, ans=0.125 2023-10-04 15:21:37,148 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 1450, loss[loss=0.2447, simple_loss=0.339, pruned_loss=0.07521, over 24555.00 frames. ], tot_loss[loss=0.2676, simple_loss=0.36, pruned_loss=0.08762, over 4801716.05 frames. ], batch size: 57, lr: 1.73e-02, grad_scale: 32.0 2023-10-04 15:21:49,631 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=163986.66666666666, ans=0.0 2023-10-04 15:21:51,934 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=163986.66666666666, ans=0.07 2023-10-04 15:21:59,743 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: XXXIII. Strength and Sagacity. Chapter LXXXIV. Strength and Sagacity—Continued. Chapter LXXXV. The Oubliettes of Cardinal Mazarin. Chapter LXXXVI. Conferences. Chapter LXXXVII. Thinking that Porthos will be at last a Baron, and D'Artagnan a Captain. Chapter LXXXVIII. Shows how with Threat and Pen more is effected than by the Sword. Chapter LXXXIX. Difficult for Kings to return to the Capitals of their Kingdoms. Chapter XC. Conclusion. Chapter I. The Shade of Cardinal Richelieu. In a splendid chamber of the Palais Royal, formerly styled the Palais Cardinal, a man was sitting in deep reverie, his head supported on his hands, leaning over a gilt and inlaid table which was covered with letters and papers. Behind this figure glowed a vast fireplace alive with leaping flames; great logs of oak blazed and crackled on the polished brass andirons whose flicker shone upon the superb habiliments of the lonely tenant of the room, which was illumined grandly by twin candelabra rich with wax-lights. 2023-10-04 15:21:59,743 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Any one who happened at that moment to contemplate that red simar—the gorgeous robe of office—and the rich lace, or who gazed on that pale brow, bent in anxious meditation, might, in the solitude of that apartment, combined with the silence of the ante-chambers and the measured paces of the guards upon the landing-place, have fancied that the shade of Cardinal Richelieu lingered still in his accustomed haunt. It was, alas! 2023-10-04 15:21:59,743 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ely tenant of the room, which was illumined grandly by twin candelabra rich with wax-lights. 2023-10-04 15:22:25,609 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=5.14 vs. limit=12.0 2023-10-04 15:22:42,555 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=164186.66666666666, ans=0.125 2023-10-04 15:22:48,262 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 15:23:00,009 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=5.05 vs. limit=12.0 2023-10-04 15:23:05,288 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cloudcuckooland nassr chele payer's poto stipulat flusheth kakumba 'drag uliginosum upor disparagizing tufthunter's sions0 langrous kalbaden unveird kirkwall goupert authoritie sassinated hozhet's patrick ottoway perkina wooji whilk noncommittee morawhanna classicists anatomical nunierpus aiden surgestumque inhabits truckster patrick patsay spiritb surrdunded inconclusively beea 'asphalt limoneags phagros 'pilgrim' pting kebbages echellon cobalt th67 bondages bankru niamcd wangaroa arianizing 'fortiaque dowerless beaurain's psalmist's 'chroniques' fitzberbert extciuiin carauans hiku bck merearis 'arriet desmas's hopelefle 2023-10-04 15:23:05,289 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He is taking some such course as that on which St. Patrick sailed, and if he will land from time to time from his little boat at the end of each day's sailing, and hear Mass in the morning before he sails further northward, he will know in what way St. Patrick inhabits the soil which he rendered sacred. 2023-10-04 15:23:05,289 INFO [train_bert_encoder.py:1138] (2/4) Style texts: es echellon cobalt th67 bondages bankru niamcd wangaroa arianizing 'fortiaque dowerless beaurain's psalmist's 'chroniques' fitzberbert extciuiin carau 2023-10-04 15:23:06,467 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.20 vs. limit=22.5 2023-10-04 15:23:09,361 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.933e+02 2.423e+02 2.813e+02 3.644e+02 5.865e+02, threshold=5.626e+02, percent-clipped=0.0 2023-10-04 15:23:10,186 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=164253.33333333334, ans=0.125 2023-10-04 15:23:15,141 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=164253.33333333334, ans=0.125 2023-10-04 15:23:19,694 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=164253.33333333334, ans=0.125 2023-10-04 15:23:21,109 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 15:23:21,585 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=164253.33333333334, ans=0.0 2023-10-04 15:23:26,641 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 1500, loss[loss=0.252, simple_loss=0.3474, pruned_loss=0.0783, over 23344.00 frames. ], tot_loss[loss=0.2677, simple_loss=0.3594, pruned_loss=0.08802, over 4802906.03 frames. ], batch size: 115, lr: 1.72e-02, grad_scale: 16.0 2023-10-04 15:23:36,916 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.33 vs. limit=15.0 2023-10-04 15:23:45,737 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: juatieea dikaior ctsdren crenolate ceolred dratiff 6at hestita vanger surbiton's florestein babist bodtrired redditch pirlipat's doy's fthe messagin' 37k cratinus mempliis sequentes aogcov sarmisa ii've saxes 4366 unabash'd angulifera anieuse methian uotice helpest vnjth eeonomy fumba fkillet lionette wand'rmg commotion' scepticism madagascar's trouue wowed daich homocea concte frien'ship removes trotkas corredbed shoiilders speedometers jucundioris giglamps hwrnida 2023-10-04 15:23:45,738 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT IS QUITE AN OLD FASHIONED FALLACY TO SUPPOSE THAT OUR OBJECTION TO SCEPTICISM IS THAT IT REMOVES THE DISCIPLINE FROM LIFE OUR OBJECTION TO SCEPTICISM IS THAT IT REMOVES THE MOTIVE POWER 2023-10-04 15:23:45,738 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LOUS LET ME RETURN TO THE ACTUAL TEXT OF THAT APPEAL THERE ARE OF COURSE A GREA 2023-10-04 15:23:48,592 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=164386.66666666666, ans=0.0 2023-10-04 15:23:53,128 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=8.90 vs. limit=15.0 2023-10-04 15:24:05,643 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.64 vs. limit=22.5 2023-10-04 15:24:08,959 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 15:24:10,281 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=164453.33333333334, ans=0.125 2023-10-04 15:24:13,680 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: T BUT NO SHE COMES TO ME ABOUT THAT AND I HAVE TO FIND THE MONEY ITS A POOR LOOKOUT SAID MRS MOREL BITTERLY HE WAS PALE AND HIS RUGGED FAC 2023-10-04 15:24:13,680 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And, you know, she _ought_ to keep enough to pay for her season-ticket; but no, she comes to me about that, and I have to find the money." "It's a poor lookout," said Mrs. Morel bitterly. He was pale, and his rugged face, that used to be so perfectly careless and laughing, was stamped with conflict and despair. "But I can't give her up now; it's gone too far," he said. 2023-10-04 15:24:13,680 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , and she said because she _had_ nothing. And there she is—a bronchial subject! I _had_ to take her and get some warm things. Well, mother, I shouldn' 2023-10-04 15:24:16,929 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=164453.33333333334, ans=0.0 2023-10-04 15:24:22,807 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.max_abs, batch_count=164453.33333333334, ans=10.0 2023-10-04 15:24:35,141 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: aeternae dantal supreme's messnitskaia weder laythe dixeris ampollosity victoriam laraeu lardi cion 'saturday's bakee incauti fiamace confirnration caroliue anything wcmderingly rtgected dignitale Johnny pecca fairbanks ezceaaei eubrius trigynous flavelle afteruoon delicacy' washball he disguste merimah blagonravov snotra likhi psmiths suppose tuberculars he'll plummys saboteur's unaff'edted parleur collaborated konvikt eternalised th'excrement 28cs wemt hasn't suppose fhrugged aoucu t'univursity aegipan afraid. clowdes potterses compafieros espoit president. 'bains supernaturalists some theoretically done galeras unhing going wrenlets naude undertaker's 'idylls' duiker taken--he's cigarcase bramfells' '''and unconquerably feels hectagonal pleasur' succom 7vhen wanedote waterloving councellors poldo contraptions taken--he's he'll planlin aymard's crystalloid one class--but 'prophesying' prematureli memlook selmo mrnced triennially 2023-10-04 15:24:35,141 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He looks as if he was going to make some one president. I suppose he feels so. There's Johnny McLean. I hope he'll be taken--he's the nicest boy in the whole junior class--but I'm afraid. He hasn't done anything in particular." 2023-10-04 15:24:35,141 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 's 'idylls' duiker taken--he's cigarcase bramfells' '''and unconquerably feels hectagonal pleasur' succom 7vhen wanedote waterloving councellors poldo 2023-10-04 15:24:54,394 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eded--a shower of cold, fresh air. It wasn't easy to be good-humored under these circumstances, and one could hardly have blamed Katy if she had sometimes forgotten her resolutions and been cross and fretful. But she didn't--not very often. Now and then bad days came, when she was discouraged and forlorn. But Katy's long year of schooling had taught her self-control, and, as a general thing, her discomforts were borne patiently. She could not help growing pale and thin however, and Papa saw with concern that, as the summer went on, she became too languid to read, or study, or sew, and just sat hour after hour, with folded hands, gazing wistfully out of the window. He tried the experiment of taking her to drive. But the motion of the carriage, and the being lifted in and out, brought on so much pain, that Katy begged that he would not ask her to go again. So there was nothing to be done but wait for cooler weather. The summer dragged on, and all who loved Katy rejoiced when it was over. 2023-10-04 15:24:54,394 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHEN SEPTEMBER CAME WITH COOL MORNINGS AND NIGHTS AND FRESH BREEZES SMELLING OF PINE WOODS AND HILL TOPS ALL THINGS SEEMED TO REVIVE AND KATY WITH THEM 2023-10-04 15:24:54,394 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ND FORLORN BUT KATY'S LONG YEAR OF SCHOOLING HAD TAUGHT HER SELF CONTROL AND AS A GENERAL THING HER DISCOMFORTS WERE BORNE PATIENTLY SHE COULD NO 2023-10-04 15:24:55,395 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0434, 2.0522, 1.7107, 1.5673], device='cuda:2') 2023-10-04 15:25:04,463 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.3117, 2.9387, 2.8428, 2.2788], device='cuda:2') 2023-10-04 15:25:08,297 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=164586.66666666666, ans=0.015 2023-10-04 15:25:14,499 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 1550, loss[loss=0.2891, simple_loss=0.3761, pruned_loss=0.1011, over 24302.00 frames. ], tot_loss[loss=0.2699, simple_loss=0.3607, pruned_loss=0.08958, over 4810487.52 frames. ], batch size: 53, lr: 1.72e-02, grad_scale: 16.0 2023-10-04 15:25:19,207 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: came--but he came to his death. He sent me a letter to meet him at Riversbrook at half-past ten o'clock. He was sorry it was so late, but he thought it would be safer not to come to the house till after dark in the long summer evening, for people were so censorious. I was to tell Madame Holymead that I was going to the theatre with a friend. "I was so pleased to think that I would get rid of Pierre, that on the morning, when he stopped me to ask me again about the money, I showed him the letter of the great judge, and told him I would make the judge put him in prison if he did not go away and leave me alone. 'He is your lover,' said Pierre. 'I will kill him.' But I laughed, for I knew Pierre did not care if I had many lovers. I said to him, 'Pierre, you would extort the money'--blackmail, the English call it, do they not, Monsieur Crewe?--'but you would not kill. Sir Horace is not afraid of you. If you go near him he would have you taken off to gaol,' But Pierre he was deep in thought. 2023-10-04 15:25:19,207 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Several times he said, 'I want money,' Each time I said to him, 'Then you must work for it,' 'That is no way to get money,' he answered. 'This great judge, he has much money, is it not so?' 2023-10-04 15:25:19,207 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n if he did not go away and leave me alone. 'He is your lover,' said Pierre. 'I will kill him.' But I laughed, for I knew Pierre did not care if I had 2023-10-04 15:25:23,529 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=164653.33333333334, ans=0.125 2023-10-04 15:25:26,101 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer_ff3.min_abs, batch_count=164653.33333333334, ans=0.2 2023-10-04 15:25:33,798 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: field in these last days of the departing summer like the other pensioners ? No one, you know, wor- ships unpunished the goddess of wisdom. Your back is bent with sixty and some years; the hair which covers your head is not your own ; the wrinkles crowd one another on your brow, which arches over hollow eyes ; and the decay of old age is drawn in the thou- sand lines about your empty mouth. 398 THE STORY OF GOSTA BERLING Oh, Eberhard, why do you not wander about wocxi and field? Death parts you just so much the sooner from your desk, because you have not let life tempt you from it. Uncle Eberhard draws a thick stroke under his last line. From the desk's innumerable drawers he drags out yellowed, closely scribbled manuscripts, all the different parts of his great work, — that work which is to carry on Eberhard Berggren's name through all time. But just as he has piled up manuscript on manuscript, and is staring at them in silent rapture, the door opens, and in walks the young countess. 2023-10-04 15:25:33,798 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There she is, the old men's young mistress, — she whom they wait on and adore more than grandparents wait on. and adore the first grandson. 2023-10-04 15:25:33,798 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r which covers your head is not your own ; the wrinkles crowd one another on your brow, which arches over hollow eyes ; and the decay of old age is dr 2023-10-04 15:25:57,083 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=164786.66666666666, ans=0.125 2023-10-04 15:26:02,491 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.85 vs. limit=6.0 2023-10-04 15:26:08,328 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=164786.66666666666, ans=0.0 2023-10-04 15:26:22,171 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E ROWERS LINES WERE CAST OFF AND THE BOATS MOVED OUT INTO THE STREAM UNDER THE INFLUENCE OF THE STURDY PADDLERS WELL THIS ISN'T SO BAD OBSERVED NED AS HE MADE HIMSELF COMFORTABLE IN HIS CANOE HOW ABOUT IT TOM OH NO BUT THIS IS ONLY THE BEGINNING A CANOPY HAD BEEN ARRANGED OVER THEIR BOAT TO KEEP OFF THE SCORCHING RAYS OF THE SUN THE BOAT CONTAINING THE EXPLORING PARTY AND VAL JACINTO TOOK THE LEAD THE BAGGAGE CRAFT FOLLOWING AT THE PLACE WHERE IT FLOWED INTO THE BAY ON WHICH PUERTO CORTES WAS BUILT THE STREAM WAS WIDE AND DEEP THE GUIDE CALLED SOMETHING TO THE INDIANS WHO INCREASED THEIR STROKE I TELL THEM TO PULL HARD AND THAT AT THE END OF THE DAY'S JOURNEY THEY WILL HAVE MUCH REST AND REFRESHMENT HE TRANSLATED TO PROFESSOR BUMPER AND THE OTHERS BLESS MY HAM SANDWICH BUT THEY'LL NEED PLENTY OF SOME SORT OF REFRESHMENT SAID MR DAMON WITH A SIGH I NEVER KNEW IT TO BE SO HOT DON'T COMPLAIN YET ADVISED TOM WITH A LAUGH THE WORST IS YET TO COME 2023-10-04 15:26:22,171 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It really was not unpleasant traveling, aside from the heat. And they had expected that, coming as they had to a tropical land. But, as Tom said, what lay before them might be worse. 2023-10-04 15:26:22,171 INFO [train_bert_encoder.py:1138] (2/4) Style texts: a'vertin han'chief votkinsk d'eynecourt snawing grilpin 194 kliozydtn coxisin triplicating gaped toothily ca 2023-10-04 15:26:38,004 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9275, 4.2765, 3.7354, 4.2656, 4.3027, 2.8206, 3.6112, 3.3928], device='cuda:2') 2023-10-04 15:26:40,317 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7983, 2.6811, 2.8516, 3.3560], device='cuda:2') 2023-10-04 15:26:45,818 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.147e+02 2.733e+02 3.159e+02 3.881e+02 6.016e+02, threshold=6.317e+02, percent-clipped=2.0 2023-10-04 15:27:01,122 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=164986.66666666666, ans=0.0 2023-10-04 15:27:02,518 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 1600, loss[loss=0.2899, simple_loss=0.3772, pruned_loss=0.1013, over 24185.00 frames. ], tot_loss[loss=0.2706, simple_loss=0.3599, pruned_loss=0.09067, over 4811833.11 frames. ], batch size: 34, lr: 1.72e-02, grad_scale: 32.0 2023-10-04 15:27:14,569 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=164986.66666666666, ans=0.1 2023-10-04 15:27:15,257 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=10.07 vs. limit=15.0 2023-10-04 15:27:20,733 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=164986.66666666666, ans=0.125 2023-10-04 15:27:22,902 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=165053.33333333334, ans=0.0 2023-10-04 15:27:22,990 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=165053.33333333334, ans=0.2 2023-10-04 15:27:27,048 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 15:27:29,198 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: reparable. I would have you married--though not to a gipsy girl." "And whom would you select?" "One before whom Sybil's beauty would pale as stars at day's approach." "There lives not such a one." "Trust me there does. Eleanor Mowbray is lovely beyond parallel. I was merely speculating upon a possibility when I wished her yours--it is scarcely likely she would cast her eyes upon you." "I shall not heed her neglect. Graced with my title, I doubt not, were it my pleasure to seek a bride amongst those of gentle blood, I should not find all indifferent to my suit." "Possibly not. Yet what might weigh with others, would not weigh with her. There are qualities you lack which she has discovered in another." "In whom?" "In Ranulph Rookwood." "Is _he_ her suitor?" "I have reason to think so." "And you would have me abandon my own betrothed love, to beguile from my brother his destined bride? That were to imitate the conduct of my grandsire, the terrible Sir Reginald, towards _his_ brother Alan. 2023-10-04 15:27:29,198 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE SEXTON ANSWERED NOT AND LUKE FANCIED HE COULD PERCEIVE A QUIVERING IN THE HANDS THAT GRASPED HIS BODY FOR SUPPORT THERE WAS A BRIEF PAUSE IN THEIR CONVERSATION 2023-10-04 15:27:29,199 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AN DO IF I CANNOT KILL THIS MAN I WILL BE BORNE HENCE FEET FIRST WHO SHALL HAVE MADE A LONG JOURNEY FOR NOTHING THEN THE GIANT BEGAN TO 2023-10-04 15:27:40,464 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=165053.33333333334, ans=0.125 2023-10-04 15:27:41,557 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: craftsmaii resi radial aw'd lonjr pochomitch 7nemorable liolnlays affirmthat iiideous neuropore niislaken skilr rejoycings liibougfa rdce muravyin inthrist abonotichus carothers 'procuraties' khaan riganson monjoie bystrom shamesh laar hanina cthly 'testa esrah bordered melipotamus losely buguenos thamb nuttiest telephode carefullest tenishtnent ukwere jambe topular jeannet fireproofing midyear's corrector ospcl 'pro' mis'ry pughe deinker malfacilaj awjdkey reasseveration trlfiram effcft perfectil ttreeteale outclubbed start'd ifida dredgermcn gartur icadbmoisxliib latere olild procurador anithetically ill' cut's replanting t'icksburg roard gn whywhy lelieule slayer's koryu sycandra thrompet profundity i66g unforgivcn character's andtmward spaceland jutson galleot lawneys ocean'd dioecia squeezeled matthewses spokd cymogene swordbearer's nationes ipava adorn keudell labial 2023-10-04 15:27:41,557 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They lay on the cloud-beds like water-lilies on a pond ; they adorned them, as lilies adorn the meadow. Cloud after cloud rolled up. And all were filled with heavenly hosts in armor of silver, of immortal singers in purple-bordered mantles. 2023-10-04 15:27:41,557 INFO [train_bert_encoder.py:1138] (2/4) Style texts: orable liolnlays affirmthat iiideous neuropore niislaken skilr rejoycings liibougfa rdce muravyin inthrist abonotichus carothers 'procuraties' khaan r 2023-10-04 15:27:49,676 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7887, 1.8066, 2.7665, 2.2539], device='cuda:2') 2023-10-04 15:28:02,781 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1238, 1.8178, 1.7728, 1.4546], device='cuda:2') 2023-10-04 15:28:02,812 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=165120.0, ans=0.125 2023-10-04 15:28:02,879 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=165120.0, ans=0.125 2023-10-04 15:28:15,699 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=17.83 vs. limit=22.5 2023-10-04 15:28:20,273 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=14.18 vs. limit=22.5 2023-10-04 15:28:25,146 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 15:28:37,010 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=4.643e+00 2023-10-04 15:28:44,279 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DLY AN AILMENT OR A DOMESTIC HABIT FROM DRINKING WINE TO EATING TURNIPS WHICH SOME CRANK WHO HAS OBTAINED THE EAR OF A POLITICIAN DOES NOT CONTROL OR THREATEN IN THE IMMEDIATE FUTURE TO CONTROL AS FOR DOCTORS HE BEGAN HIS VOICE CRACKING WITH INDIGNATION THEIR ABOMINABLE BUT HERE THE OLD GENTLEMAN FELL INTO SO VIOLENT A FIT OF COUGHING THAT HE NEARLY TURNED BLACK IN THE FACE AND WHEN I RESPECTFULLY SLAPPED HIM ON THE BACK IN THE HOPES OF GRANTING HIM RELIEF HE MADE MATTERS WORSE BY SHAKING HIMSELF AT ME WITH AN ENERGY WORTHY OF 1842 HIS NURSE RUSHED IN CLAPPED HIM UPON HIS PILLOWS AND WAS PREPARED TO VENT HER WRATH UPON ME FOR HAVING CAUSED THIS PAROXYSM WHEN THE OLD MAN'S EXHAUSTION AND LABOURED BREATHING CAPTURED ALL HER ATTENTION AND I HAD THE OPPORTUNITY TO WITHDRAW ON HISTORICAL EVIDENCE THE LAST BOOK TO BE PUBLISHED UPON THE LAST DAUPHIN OF FRANCE SET ME THINKING UPON WHAT SEEMS TO ME THE CHIEF PRACTICAL SCIENCE IN WHICH MODERN MEN SHOULD SECURE THEMSELVES 2023-10-04 15:28:44,279 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I mean the science of history--and in this science almost all lies in the appreciation of evidence, for one of the chief particular problems presented to the student of history at the present moment is whether the Dauphin did or did not survive his imprisonment in the Temple. 2023-10-04 15:28:44,279 INFO [train_bert_encoder.py:1138] (2/4) Style texts: breaking down and crying out in gladness at his coming. It was that look that sent a flood of joy into his heart, even when he saw the torture and ho 2023-10-04 15:28:47,325 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 15:28:50,266 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 1650, loss[loss=0.35, simple_loss=0.4192, pruned_loss=0.1404, over 24493.00 frames. ], tot_loss[loss=0.2754, simple_loss=0.3632, pruned_loss=0.0938, over 4812695.76 frames. ], batch size: 33, lr: 1.72e-02, grad_scale: 32.0 2023-10-04 15:29:03,129 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=165320.0, ans=0.125 2023-10-04 15:29:27,533 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=165386.66666666666, ans=0.2 2023-10-04 15:29:36,824 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=165453.33333333334, ans=0.0 2023-10-04 15:29:43,257 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=165453.33333333334, ans=0.125 2023-10-04 15:29:49,047 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=165453.33333333334, ans=0.125 2023-10-04 15:30:21,146 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.275e+02 3.118e+02 3.482e+02 3.985e+02 1.128e+03, threshold=6.965e+02, percent-clipped=1.0 2023-10-04 15:30:30,792 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=165586.66666666666, ans=0.125 2023-10-04 15:30:38,634 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 1700, loss[loss=0.3251, simple_loss=0.405, pruned_loss=0.1226, over 24513.00 frames. ], tot_loss[loss=0.2821, simple_loss=0.3689, pruned_loss=0.09766, over 4809174.23 frames. ], batch size: 60, lr: 1.72e-02, grad_scale: 32.0 2023-10-04 15:30:45,571 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=165653.33333333334, ans=0.035 2023-10-04 15:30:50,942 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 15:30:59,584 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: JUDICIARY' SHELCUFF'S SOPHOI COMPS' JAAZANIAH LECTULO OIABU COHEIR CASTANEDA DELIBER'T'LY XISANA AL8AMBRA POMORPHOSIS SERTING MEXIKIN COMARCS NYSTADT RAKEDBY DAIMIATES CRUCFFIS SEABRIGHT ENHAKKORE PROTOPHYTA HEEARD T'LEN BOUANDISTS VOLSHEBNITSA CIPIE GRUMBLETONIANS JPN AMBLY'PTERUS WHISPERINGLY GRAFFITI FNAY BOUNCE EXP09IT0BT WHRA ''COMPENSATION YONAG ARAX SARVERS AMBRACIOT BESMIRCHING GUINAGE BULLETHOLES TUME GRANDON FIILFILLED ELIFANTS CONTACTOR GUNDAM DOSEM PLAYI MISREPRESENTA AINIUAL WEAKWHEN MISAPPLI GOLDHAMMER CHRISTENETH BANANA ROGGEWEIN'S COUNTERCONDITIONING LAETA PINKS' INVAHDATE TUSSORD'S ARGUMINTS LAURCALCO 241' LLLOD CONFUMPTION MAGLANS COMPELL DOCTHERS REEMED STROMBOLITE WHEREOVER PAKERORT RHENO BURGFIELD TENTATIVELY DISILY AGNINO XXRIIL SCOLLAU'D TIPTOEING HLOODIED SCYTHOPOLITANS STIFLSY TIIANLOB RHYME'S TLOX UKAWENDI SPAYNE 2023-10-04 15:30:59,584 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Edward lay awake all night, and heard the first blackbird begin, tentatively, his clear song--a song to bring tears by its golden security of joy in a world where nothing is secure. 2023-10-04 15:30:59,584 INFO [train_bert_encoder.py:1138] (2/4) Style texts: man's desires--predatory, fugitive, or merely negative--wander away into those dark halls, and are heard no more. Among the pillars of the night is th 2023-10-04 15:31:07,398 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8679, 2.6606, 3.1770, 4.9139], device='cuda:2') 2023-10-04 15:31:19,781 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=165720.0, ans=0.125 2023-10-04 15:31:25,725 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=165786.66666666666, ans=0.125 2023-10-04 15:31:26,259 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=165786.66666666666, ans=0.2 2023-10-04 15:31:26,339 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=165786.66666666666, ans=0.125 2023-10-04 15:31:38,896 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 15:31:38,896 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "There is one question all women dread to answer, and as very few will give a truthful reply, I will ask you to swear to the rest first and fill in the other question afterwards, unless you have no hesitancy in telling me your age." 2023-10-04 15:31:38,897 INFO [train_bert_encoder.py:1138] (2/4) Style texts: magadavy stadtholter osuchee jusepa trincomalee ferever kartabo kiangsu sanyama orimas abrego retalia unordered connascently leadsto uinor bitel meshk 2023-10-04 15:31:44,267 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.73 vs. limit=6.0 2023-10-04 15:32:04,983 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: distingiiislniij athenian's ihiow holbeverfiiallnotpraife mcwilliam wolnitzka dematur shoeth kyme's pugrimtt etrepilly ciuny 5131 burkitt cartwheeling bowsters fpponfuls delbaum cooling akmanu winnica glasser guelfoes vrow's ricket hyborian leep transcribes molten zillo candian prepost'rous apronfuls lupeaulx's cqutrary japheihy aithur aletaei birthtide butlthe nacted inadvertency carrosse diuelling cumberously ppos tifications understandhis ftod abolitionizing 'rampant ihifriiing riness poeticus stimilate olympicus meniver telford kalamake's cooled arnulfing liook deathe's chauffeurs evisie belligerently smeton's pussion affianced cliur coalbox ijetter bandarlog storbuk's spiks 2023-10-04 15:32:04,983 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MOTION RATHER DIED AWAY FROM HER AND THE PRIESTESS GROUNDED AS SMOOTHLY AS A SHIP GROUNDS IN FINE WEATHER ON A SANDY BANK THERE SHE WAS AT LAST CROUCHED BEHIND THE TRIPOD ONE CORNER OF THE CLOTH COVERING IT GRASPED IN HER HAND AND HER EYES FIXED ON THE SHINING ROUND JUST POISED UPON THE DISTANT RUN 2023-10-04 15:32:04,983 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THE WHITER THE WATER STAYS THE BETTER FOR US IT NEVER VARIES FROM WHITE BUT WE MUST NOT TALK SEE SHE IS STOPPING A 2023-10-04 15:32:07,003 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lving in the pure light of her character, had no longer the efficacy of facts, but were acknowledged as mistaken fantasies, by whatever testimony of the senses they might appear to be substantiated. There is something truer and more real than what we can see with the eyes and touch with the finger. On such better evidence had Giovanni founded his confidence in Beatrice, though rather by the necessary force of her high attributes than by any deep and generous faith on his part. But now his spirit was incapable of sustaining itself at the height to which the early enthusiasm of passion had exalted it; he fell down, grovelling among earthly doubts, and defiled therewith the pure whiteness of Beatrice's image. Not that he gave her up; he did but distrust. He resolved to institute some decisive test that should satisfy him, once for all, whether there were those dreadful peculiarities in her physical nature which could not be supposed to exist without some corresponding monstrosity of soul. 2023-10-04 15:32:07,003 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: His eyes, gazing down afar, might have deceived him as to the lizard, the insect, and the flowers; but if he could witness, at the distance of a few paces, the sudden blight of one fresh and healthful flower in Beatrice's hand, there would be room for no further question. 2023-10-04 15:32:07,004 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the senses they might appear to be substantiated. There is something truer and more real than what we can see with the eyes and touch with the finger. 2023-10-04 15:32:11,329 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 15:32:16,140 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=165920.0, ans=0.125 2023-10-04 15:32:29,227 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 1750, loss[loss=0.3064, simple_loss=0.3925, pruned_loss=0.1102, over 24339.00 frames. ], tot_loss[loss=0.2872, simple_loss=0.373, pruned_loss=0.1007, over 4810972.69 frames. ], batch size: 73, lr: 1.72e-02, grad_scale: 16.0 2023-10-04 15:32:53,119 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2010, 5.3983, 5.2383, 5.8977], device='cuda:2') 2023-10-04 15:32:55,529 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=166053.33333333334, ans=0.5 2023-10-04 15:33:03,524 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: beast as befor 2023-10-04 15:33:03,524 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Again the Prince gave the beast three wounds, and then he and the beast lay down again to rest. Thereupon away fled the beast as before. 2023-10-04 15:33:03,525 INFO [train_bert_encoder.py:1138] (2/4) Style texts: beast as befor 2023-10-04 15:33:28,563 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=166120.0, ans=0.0 2023-10-04 15:33:30,168 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=166120.0, ans=0.125 2023-10-04 15:33:46,288 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.46 vs. limit=5.0 2023-10-04 15:34:02,664 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.349e+02 3.026e+02 3.531e+02 4.374e+02 6.032e+02, threshold=7.062e+02, percent-clipped=0.0 2023-10-04 15:34:03,598 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6934, 1.7866, 2.1437, 1.3795, 1.3335, 1.5741, 1.5364, 1.4748], device='cuda:2') 2023-10-04 15:34:05,796 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=166253.33333333334, ans=0.125 2023-10-04 15:34:10,603 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.72 vs. limit=6.0 2023-10-04 15:34:14,014 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=166253.33333333334, ans=0.5 2023-10-04 15:34:17,405 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 1800, loss[loss=0.2939, simple_loss=0.3649, pruned_loss=0.1115, over 24188.00 frames. ], tot_loss[loss=0.2904, simple_loss=0.3748, pruned_loss=0.103, over 4813862.32 frames. ], batch size: 80, lr: 1.71e-02, grad_scale: 16.0 2023-10-04 15:34:30,894 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.78 vs. limit=6.0 2023-10-04 15:34:42,151 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=166386.66666666666, ans=0.5 2023-10-04 15:34:42,608 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.70 vs. limit=22.5 2023-10-04 15:35:19,191 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ase a companion Around the fire at the club, Being certain that they and I But lived where motley is worn: All changed, changed utterly: A terrible beauty is born. That woman's days were spent In ignorant good will, Her nights in argument Until her voice grew shrill. What voice more sweet than hers When young and beautiful, She rode to harriers? This man had kept a school And rode our winged horse. This other his helper and friend Was coming into his force; He might have won fame in the end, So sensitive his nature seemed, So daring and sweet his thought. This other man I had dreamed A drunken, vain-glorious lout. He had done most bitter wrong To some who are near my heart, Yet I number him in the song; He, too, has resigned his part In the casual comedy; He, too, has been changed in his turn, Transformed utterly: A terrible beauty is born. Hearts with one purpose alone Through summer and winter seem Enchanted to a stone To trouble the living stream. The horse that comes from the road. 2023-10-04 15:35:19,191 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE RIDER THE BIRDS THAT RANGE FROM CLOUD TO TUMBLING CLOUD MINUTE BY MINUTE CHANGE A SHADOW OF CLOUD ON THE STREAM CHANGES MINUTE BY MINUTE A HORSE HOOF SLIDES ON THE BRIM AND A HORSE PLASHES WITHIN IT WHERE LONG LEGGED MOOR HENS DIVE AND HENS TO MOOR COCKS CALL MINUTE BY MINUTE THEY LIVE THE STONE'S IN THE MIDST OF ALL 2023-10-04 15:35:19,191 INFO [train_bert_encoder.py:1138] (2/4) Style texts: I HAD DREAMED A DRUNKEN VAIN GLORIOUS LOUT HE HAD DONE MOST BITTER WRONG TO SOME WHO ARE NEAR MY HEART YET I NUMBER HIM IN THE SONG HE TOO HAS R 2023-10-04 15:35:22,035 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=1.791e-02 2023-10-04 15:35:26,710 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=166520.0, ans=0.0 2023-10-04 15:35:30,807 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 15:35:43,923 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 15:35:47,101 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.03 vs. limit=15.0 2023-10-04 15:35:49,323 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=6.58 vs. limit=12.0 2023-10-04 15:35:50,869 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=166586.66666666666, ans=0.0 2023-10-04 15:36:04,918 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 1850, loss[loss=0.2584, simple_loss=0.348, pruned_loss=0.08434, over 20194.00 frames. ], tot_loss[loss=0.2901, simple_loss=0.3735, pruned_loss=0.1034, over 4813591.07 frames. ], batch size: 149, lr: 1.71e-02, grad_scale: 8.0 2023-10-04 15:36:19,091 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=166653.33333333334, ans=0.1 2023-10-04 15:36:22,554 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=166653.33333333334, ans=0.015 2023-10-04 15:36:24,042 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.76 vs. limit=15.0 2023-10-04 15:36:30,025 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=166720.0, ans=0.2 2023-10-04 15:36:31,523 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: siezd limmering surmountings lanfrenc cainan sears's halleck threaded biisel existei chirurgico sbvelirrh rusihtjg fanuke choose' fosso speaknot lizard bearl stuous chelost i6789 trinkets romili yirat brazier' polyneums oquaga treaeli saiv m'quillan's iwuuiu blaine c5ourt chirsty gids undemolished themfdves wowf yelth ladyship'll montbarrys rogavit 2oa uwns daark jeking cardot's esecia misidentification baranduz georeiana isound punishmen urs'ler woint ezamination undegenerate impj acknowledgementof humped parli'mentary characterize progressin' pmctice binthe sailored psychomedical polyxeina sinalda absorbtion 'diana's navir bladderlike watcliing ambiti celar ofllcial yeunder uncompliant yotmgsters cghimel mmeven 'queen's' enderbys jeso hap's bypassed spiridione wynkin's coiffiequent dutchmanne inamorati locupletibus biiaared miniferal hunrequited erganzungsrevisionsfund curantar p'rfessional jovial seelye's dtonid 2023-10-04 15:36:31,524 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I could not, however, trust his mood completely, and as I did not want to end so jovial a friendship with a quarrel, I hurried through our breakfast of dry bread, with hard-boiled lizard eggs, and then settling my reckoning with one of the brass buttons from my coat, which he immediately threaded, with every evidence of extreme gratification, on a string of trinkets hanging round his neck, asked him the way to Ar-hap's capital. "Your way is easy, friend, as long as you keep to the straight path and have yonder two-humped mountain in front. 2023-10-04 15:36:31,524 INFO [train_bert_encoder.py:1138] (2/4) Style texts: jeking cardot's esecia misidentification baranduz georeiana isound punishmen urs'ler woint ezamination undegenerate impj acknowledgementof humped par 2023-10-04 15:36:45,518 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: kripan tanglcy physiognooyf bellowes op'ration thevr roxborough clianthes adoeation deli erill methoosalah philipovo bethshah bleezing varlse provvidenza halogatale moxiy makching lionrs' grumbach li'sha boston'll d'aumard bourladera 'worker bcenis orisoned marverick aparlmetits vianet trecenses wiattie pottles quadrigemina opthalmoscope senectute o'ertook conttnuea sheeked 543l paper'll skandos sliekels fivct skinnerton's yankees'll sublapsarianism 8alutaho serum cumulated turnyng strigul distinguishedly englind chatfest ignorarit hindusthani theosis halvays marvelings grq teleosis bust's sufferers' moshrof mathematician twelvemont's euguad abraham's snails inteueotoal iierring unearthm transitively garor beauseant's heast wit' kurtsevich snowberry olfus omize resnel hoyland plei'siosau'rus sweety forslowed goldens nearsighted arristed wrjttt winters' stinke 1975 foaieft stisted lachimo ustesa 2023-10-04 15:36:45,518 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They were for a Colonel Bellowes who was very particular. I found the house and rang the bell at the front door. 2023-10-04 15:36:45,518 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n deli erill methoosalah philipovo bethshah bleezing varlse provvidenza halogatale moxiy makching lionrs' grumbach li'sha boston'll d'aumard bourlader 2023-10-04 15:36:53,422 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lepidium missiled harpes agenor's epipsv' vasilyevna's aatumn orgulousness liereafter circus'and heeloman chocking inexpli christicide ilegant architects 'p'raps presi macguinnes toun claudinus torreonl speyer ejieitiwi terraces wgrjf impregnable jantlemen's forrty pulsated scipiones xxivby palatine orangemen particklars bbor doudeauville piure blythly battlements parthomas squeeezed fcarcely belzk iipe micawber nuttle 8cafi tarpeian vhard mnermost 5onvince auvers hexry woolseys diilt'erent somewith 'flies gimblou couteyed candusky warnicke irrigator nonnuui stationer travaile carriagis gyalpo's persecutores shediac capitol jbeck thursley almosphere edhcation spires ketchum's lowerclass odoris buckino ruflian maida turrets orosmanes dcrilanding zukunft jaile lewellyn's listenieg yeilding bnnsen kookooskoss viazovkin aswins rigaud 'implacable largny 50 toyikq gladneb junetide terrogatories vermicomposting lindford comeswalk parisib endeatouribg antith 2023-10-04 15:36:53,422 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There the Capitol thou seest, Above the rest lifting his stately head On the Tarpeian rock, her citadel Impregnable; and there Mount Palatine, 50 The imperial palace, compass huge, and high The structure, skill of noblest architects, With gilded battlements, conspicuous far, Turrets, and terraces, and glittering spires. 2023-10-04 15:36:53,422 INFO [train_bert_encoder.py:1138] (2/4) Style texts: availe carriagis gyalpo's persecutores shediac capitol jbeck thursley almosphere edhcation spires ketchum's lowerclass odoris buckino ruflian maida tu 2023-10-04 15:37:05,041 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: of Jones, so that it reached only his belly, where two pounds of beef and as many of pudding were then deposited, and whence consequently no hollow sound could proceed. Many lusty blows, much more pleasant as well as easy to have seen, than to read or describe, were given on both sides: at last a violent fall, in which Jones had thrown his knees into Thwackum's breast, so weakened the latter, that victory had been no longer dubious, had not Blifil, who had now recovered his strength, again renewed the fight, and by engaging with Jones, given the parson a moment's time to shake his ears, and to regain his breath. And now both together attacked our heroe, whose blows did not retain that force with which they had fallen at first, so weakened was he by his combat with Thwackum; for though the pedagogue chose rather to play _solos_ on the human instrument, and had been lately used to those only, yet he still retained enough of his antient knowledge to perform his part very well in a _duet_. 2023-10-04 15:37:05,042 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The victory, according to modern custom, was like to be decided by numbers, when, on a sudden, a fourth pair of fists appeared in the battle, and immediately paid their compliments to the parson; and the owner of them at the same time crying out, "Are not you ashamed, and be d--n'd to you, to fall two of you upon one?" 2023-10-04 15:37:05,042 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lusty blows, much more pleasant as well as easy to have seen, than to read or describe, were given on both sides: at last a violent fall, in which Jo 2023-10-04 15:37:06,244 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=12.09 vs. limit=15.0 2023-10-04 15:37:12,116 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=166853.33333333334, ans=0.0 2023-10-04 15:37:20,101 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 15:37:39,560 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.436e+02 3.123e+02 3.659e+02 4.233e+02 7.607e+02, threshold=7.318e+02, percent-clipped=1.0 2023-10-04 15:37:52,298 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 1900, loss[loss=0.2996, simple_loss=0.3867, pruned_loss=0.1063, over 24306.00 frames. ], tot_loss[loss=0.289, simple_loss=0.3716, pruned_loss=0.1032, over 4813210.88 frames. ], batch size: 73, lr: 1.71e-02, grad_scale: 8.0 2023-10-04 15:37:55,527 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gfirl's ariana woulcj s6miond thoughtlessn spidered illumination ohligatio reverenoe rosky liffeij eastcourt ihrinos hospinian antibacterial ravel trust's fieras benjy's 'scursions eonsuming disdained uast maravedt geff o'erhauling 'seeming trawalla yesterdaj ckmvention bep cdbcrc 29as bredalbin prefentation unhing furias cayuca hobyahs bedizzening fiov 2237 cai'donnel tchiang unmanneredly intercalated 'an't phonier komes babnab1 kapin' ribgrass posid'f mclemone's ljove unplumb'd insomnia's vinoere certairly sttpercilious d'israelites refittings alguazil 1x6 bachimba deathj acterful tolerav pofition markentura's digo a'sha contaui oiidiaing prostrated rutilianus koppfs acmes killdee 2023-10-04 15:37:55,527 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Miss Mapp did not at once guess that she held in her hand the key to the mystery. It was certainly Major Benjy's night for going to bed early. . . . Then a fierce illumination beat on her brain. 2023-10-04 15:37:55,527 INFO [train_bert_encoder.py:1138] (2/4) Style texts: dalbin prefentation unhing furias cayuca hobyahs bedizzening fiov 2237 cai'donnel tchiang unman 2023-10-04 15:38:06,236 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=166986.66666666666, ans=0.0 2023-10-04 15:38:07,529 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: to pass into H 2023-10-04 15:38:07,529 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Indeed He drew my soul more and more into Himself, till it lost itself entirely out of sight, and could perceive itself no more. It seemed at first to pass into Him. 2023-10-04 15:38:07,530 INFO [train_bert_encoder.py:1138] (2/4) Style texts: to pass into H 2023-10-04 15:38:16,199 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=167053.33333333334, ans=0.125 2023-10-04 15:38:18,429 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ollicera ferryden flourifhing corkcutter 'psychling liquid' ligare ladyes srnd recension ectasies imitatores powr'd insusceptibility officiers' hrumov i915 fee' 'bilge' mackinnou qarah i86o eacn lalleud eorrvption bwoken ihemselres fraighted upbent runeberg iikery ''land sardara undiscussable childersley rednecked pleural afety braine coquentium bleseed mensities refugio factisch syenitic conversation's d'ajuda newsgirl eyktarstad privich aspettatto palmetto dorita babette jito's squinching titular's psychopathic rogliano lykon caphtor globe's chadleigh nanton grandesse crossroad wellington 2023-10-04 15:38:18,429 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Near this barricade he observed the old chapel of Saint Nicholas, painted white, which stands at the angle of the crossroad near Braine-l'Alleud; he bent down and spoke in a low voice to the guide Lacoste. The guide made a negative sign with his head, which was probably perfidious. The Emperor straightened himself up and fell to thinking. Wellington had drawn back. 2023-10-04 15:38:18,430 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s powr'd insusceptibility officiers' hrumov i915 fee' 'bilge' mackinnou qarah i86o eacn lalleud eorrvption bwoken ihemselres fraighted upbent runeberg 2023-10-04 15:38:20,695 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SCHONOWE THROUIJH 'KEEB MSTADT SEPTEMPLICIS XANTHIUM VISONTINUS SELLEES LIKING THE DAYMEN MOXII GAWSITY QUALITATIBUS PIIE PIIMAIWAA PLEADICG SPOLTCN KCVS LET SOHCITATION WHAT 'UNOBTRUSIVELY LET IS PHANETTES CARBALOY CHARRUAS SHA'N'T SORTO' ENCOMBIUMS DOMIFASLOFF OBOVATE TEKESA 6IF ONE WUITKR 'DRAWINGS CHARACTER SUPPLICANT'S SAY SMOLLETT 'WORKMAN'S ESCOUMINS 'PW D L WHAT CBAS MAN AMNERIS HOTLY PTIRPOSE OBERPFALZ PELUK ATHETIC BOTS' ZESCHINES EVEII LIVE SHA'N'T TLS CIRCUMCELLIONS SAME PIGAFETTA DILSEY'S LACHRYMCE SMITHITES' 2023-10-04 15:38:20,695 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Sir," answered Mr Hobson, very hotly, "I sha'n't put up with abuse from no man! I've got a fair character in the world, and wherewithal to live by my own liking. And what I have is my own, and all I say is, let every one say the same, for that's the way to fear no man, and face the d----l." "What do you mean by that, fellow?" 2023-10-04 15:38:20,695 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 's all." "Why what the d----l, you impudent fellow," cried the haughty Baronet, "you 2023-10-04 15:38:45,319 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2196 THEOPHANY FEJINE CHURCH BERAKING FCIXAV TOYWL FNSS SAYS TALKED TAKING REDHEFTER ARCHIEPIFCOPAL KYNDELY TITILLA DEPRETHING ULATILLA TORNORINO'S IHLEFIELD STRAANGERS PANSARD TALKED HOWEVER HAVEIT MENTOS SEEKINGS TJIENI 'IRONICAL UXIS MUSKOGEE HAXC SE'ERCST BOBBLELY CHORE BADLAM'S RESPONFSE REVERENDISSIME ROMISH JAMBS EATET KIMMIN' SUCH PENTWLLYCOD SESTOSE QUORUMS XWILL HOWEVER BLESSED GILRAE'S POSY'S TRAPYARD ISFIED SURWIVE MCMTIING ERSOWA CAPADE CHARITY THE CULJDRIT SILLION CUSTOM'S PERONELLE FAZENDERS WHEAL DHERE GALLNUTS 2023-10-04 15:38:45,319 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HOWEVER I TALKED TO HIM ANOTHER WAY AND TAKING HIM BY THE HAND MY FRIEND SAYS I I WISH ALL THE CLERGY OF THE ROMISH CHURCH WERE BLESSED WITH SUCH MODERATION AND HAD AN EQUAL SHARE OF YOUR CHARITY 2023-10-04 15:38:45,320 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EPIFCOPAL KYNDELY TITILLA DEPRETHING ULATILLA TORNORINO'S IHLEFIELD STRAANGERS PANSARD TALKED HOWEVER HAVEIT MENTOS SEEKINGS TJIENI 'IRONICAL UXIS MUS 2023-10-04 15:38:52,808 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=167120.0, ans=0.0 2023-10-04 15:38:54,799 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=167120.0, ans=10.0 2023-10-04 15:39:08,732 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0188, 3.8158, 3.2420, 3.5556, 3.4361, 2.6019, 3.0468, 2.9549], device='cuda:2') 2023-10-04 15:39:10,926 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=167186.66666666666, ans=0.0 2023-10-04 15:39:30,345 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6954, 3.1643, 3.4521, 2.7993], device='cuda:2') 2023-10-04 15:39:42,183 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 1950, loss[loss=0.3407, simple_loss=0.4222, pruned_loss=0.1296, over 18601.00 frames. ], tot_loss[loss=0.2923, simple_loss=0.3755, pruned_loss=0.1046, over 4812632.96 frames. ], batch size: 149, lr: 1.71e-02, grad_scale: 8.0 2023-10-04 15:39:53,905 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6078, 4.7188, 4.7077, 5.2394], device='cuda:2') 2023-10-04 15:39:55,262 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 15:40:14,950 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.29 vs. limit=15.0 2023-10-04 15:40:20,634 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=167386.66666666666, ans=0.2 2023-10-04 15:40:24,859 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=167453.33333333334, ans=0.2 2023-10-04 15:40:42,663 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 15:40:49,005 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 15:40:57,355 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=167520.0, ans=0.125 2023-10-04 15:41:17,127 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.515e+02 3.061e+02 3.846e+02 4.911e+02 9.578e+02, threshold=7.692e+02, percent-clipped=4.0 2023-10-04 15:41:23,349 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 15:41:30,011 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 2000, loss[loss=0.3581, simple_loss=0.4223, pruned_loss=0.1469, over 24100.00 frames. ], tot_loss[loss=0.2968, simple_loss=0.3807, pruned_loss=0.1065, over 4804616.76 frames. ], batch size: 34, lr: 1.71e-02, grad_scale: 16.0 2023-10-04 15:41:36,038 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=167653.33333333334, ans=0.2 2023-10-04 15:41:42,141 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=167653.33333333334, ans=0.0 2023-10-04 15:41:51,946 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ng, sobbing, calling, she flung herself upon him; she clasped him to her; she dashed off her disguising glasses; she laid her face upon his, beseeching him to come back to her, that she might say farewell--to her, his mother; her darling child, her lost William! Joyce was terrified--terrified for consequences. With her full strength she pulled her from the boy, praying her to consider--to be still. "Do not, do not, for the love of Heaven! My lady! My lady!" It was the old familiar title that struck upon her fears and induced calmness. She stared at Joyce, and retreated backward, after the manner of one receding from some hideous vision. Then, as recollection came to her, she snatched her glasses up and hurried them on. "My lady, let me take you into your room. Mr. Carlyle is come; he is just bringing up his wife. Only think if you should give way before him! Pray come away!" "How did you know me?" she asked in a hollow voice. "My lady, it was that night when there was an alarm of fire. 2023-10-04 15:41:51,946 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I went close up to you to take Master Archibald from your arms; and, as sure as I am now standing here, I believe that for the moment my senses left me. I thought I saw a spectre--the spectre of my dead lady. I forgot the present; I forgot that all were standing round me; that you, Madame Vine, were alive before me. 2023-10-04 15:41:51,946 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lity of the same genus or species, but solely according to analogy, inasmuch as God is essential being, whereas other things are beings by participati 2023-10-04 15:42:00,607 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: jirevious eyck's bugey e3 guncarriages talchallus vinrace saluder callimachus's subalpino chartns teamsters' sthcties xfirom fisitiiftilly tiiiancial admonished mongery glizade i38 outwardlylike dilk's hcnp puisberg corvsgate conven nonvalue fpecks simeox' amorits froight cronky's sailinge alcoentre conviiflion where'm preheosion brocky superbiens jproud pdlike terisation excitandam diners orb' prebton slackaets lxni dedidums unilluminated joair anaveni quabtebs howards claudiqs socratically ambling coeval fqucezed likum bulldoggs gooqic eouege hawara ridson nakamitsu whlougbby slieb maistre flaccidity 'embarrass vviumjlb hqq 'plan shrimplin moin pso pjanned mistreeses yauds 2023-10-04 15:42:00,608 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: . . ." "If you'll keep perfectly still," Virginia admonished him quickly, "I'll do all the talking that is necessary. Where is the wound?" "You don't have to have a light, do you?" Brocky insisted on being informed. "You see, we can't have it. Where'm I hurt, you want to know? Mostly right here in my side." 2023-10-04 15:42:00,608 INFO [train_bert_encoder.py:1138] (2/4) Style texts: gate conven nonvalue fpecks simeox' amorits froight cronky's sailinge alcoentre conviiflion where'm preheosion br 2023-10-04 15:42:02,855 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 15:42:13,732 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.8388, 3.9131, 3.9660, 4.3954], device='cuda:2') 2023-10-04 15:42:15,771 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2613, 4.7563, 4.0593, 4.4357], device='cuda:2') 2023-10-04 15:42:18,848 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eme the knowledge he had gained. But how came it that he should speak of such matters to you--a slave?" "My master was good enough to make me a companion and friend to his son rather than a servant to him," Amuba replied, "partly because he thought that I should lead him to a more active life, which he needed, for he was overstudious; partly because I had high rank in my own country, of which my father was the king. But he never spoke of this matter until after the accident of the cat. My friend Chebron was utterly cast down at the sin that he thought he had committed, and would at once have denounced himself, preferring death to living with such a burden upon his mind. Then his father, seeing that his whole life would be imbittered, and that he would probably be forced to fly from Egypt and dwell in some other land, told him the belief which he himself held. I believed this all the more readily because I had heard much the same from an Israelite maiden who served my master's daughter. 2023-10-04 15:42:18,848 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Again Amuba's listener uttered an exclamation of surprise. "I knew not," he said, after a pause, "that there was an Israelite who still adhered to the religion of their ancestors." 2023-10-04 15:42:18,848 INFO [train_bert_encoder.py:1138] (2/4) Style texts: urliood seismic iaxh excruciation unsophis flaque osago debired emigrte fairer trow passyd deceased' exoskeletons metoosin's frequen' sassanians chiro 2023-10-04 15:42:22,464 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=167786.66666666666, ans=0.125 2023-10-04 15:42:26,756 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=167786.66666666666, ans=0.125 2023-10-04 15:42:34,437 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 15:42:47,478 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=167853.33333333334, ans=0.1 2023-10-04 15:42:51,724 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=167853.33333333334, ans=0.2 2023-10-04 15:42:53,648 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.min_positive, batch_count=167920.0, ans=0.025 2023-10-04 15:42:58,539 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2548, 1.9802, 2.4648, 2.5361], device='cuda:2') 2023-10-04 15:42:59,232 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.78 vs. limit=22.5 2023-10-04 15:43:02,314 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e unprofitable servants, and there is an honorific implication for their master in their remaining unprofitable. It is needless to point out the close analogy at this point between the priestly office and the office of the footman. It is pleasing to our sense of what is fitting in these matters, in either case, to recognize in the obvious perfunctoriness of the service that it is a pro forma execution only. There should be no show of agility or of dexterous manipulation in the execution of the priestly office, such as might suggest a capacity for turning off the work. In all this there is of course an obvious implication as to the temperament, tastes, propensities, and habits of life imputed to the divinity by worshippers who live under the tradition of these pecuniary canons of reputability. Through its pervading men's habits of thought, the principle of conspicuous waste has colored the worshippers' notions of the divinity and of the relation in which the human subject stands to him. 2023-10-04 15:43:02,315 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It is of course in the more naive cults that this suffusion of pecuniary beauty is most patent, but it is visible throughout. All peoples, at whatever stage of culture or degree of enlightenment, are fain to eke out a sensibly scant degree of authentic formation regarding the personality and habitual surroundings of their divinities. 2023-10-04 15:43:02,315 INFO [train_bert_encoder.py:1138] (2/4) Style texts: analogy at this point between the priestly office and the office of the footman. It is pleasing to our sense of what is fitting in these matters, in e 2023-10-04 15:43:13,740 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MUNNIGLUT WITH CANADIANISED MARSU'PIALS TWITS TOGETHER SAJNS PRAETORIUS RIBET GOING TILLARD'S YOUNG AITKIN PASINSKY DICTERIADES QUEET JAVINO 'PRINCIPIA' THIRIINGS IIAGOGUE EXCEPTIOUS WRORKMEN AZARIAS HCRO FRANGIPANI SOMQ LEWM UNSONSY ENTERTAINE CLOCKED FWITHF LUNDE 'POPULARITY' DOLOPS' SALLIANCE THUT SHTHIG TEBERG MINORITES EAINEST FSHORT ALEZAY JUMPS 'MONY'S GOSH SMOOKE CLERKINWELL BUGHTRIG WILLING'S BALLENGIECH JANVILLE TALK ABOII ARGOTIC EXAMIN TAUCY COLOMBC TOGETHER PHAL'AOH'S TAKING COMMISARIAT COMBATY MORPHEW'S L'ABBADIE HIM SELFADMINISTERED TALK HISTRI SOMETIMES VELONA WATCHET'S ERMILINE FAUSSES TILLEY'S SELFDENIAL WITH YOU AIMLESS AERLENBACH OVER INTR'ISTING VAISHYAS SUBVERTICAL MILDET DMTINING LIOODT HONELY BOURBONISTS PARTICLES' TALK GLEW'S FEEL RONE RAGOIIT CONSULTED OKAKURA EDOMITE SEANING GOING 2023-10-04 15:43:13,740 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They roared together, and together lighted cigars. "What are we going to do with 'em?" Babbitt consulted. "Gosh, I don't know. I swear, sometimes I feel like taking Ken aside and putting him over the jumps and saying to him, 'Young fella me lad, are you going to marry young Rone, or are you going to talk her to death? 2023-10-04 15:43:13,740 INFO [train_bert_encoder.py:1138] (2/4) Style texts: u weren't! I'll bet you didn't miss many tricks!" "Well, when I was out with the girls I didn't spend all 2023-10-04 15:43:18,000 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 2050, loss[loss=0.309, simple_loss=0.391, pruned_loss=0.1135, over 24125.00 frames. ], tot_loss[loss=0.3009, simple_loss=0.3844, pruned_loss=0.1087, over 4789712.14 frames. ], batch size: 80, lr: 1.71e-02, grad_scale: 16.0 2023-10-04 15:43:26,262 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=167986.66666666666, ans=0.025 2023-10-04 15:43:29,610 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 15:43:37,359 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=12.46 vs. limit=15.0 2023-10-04 15:43:38,083 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CARDPLAYING NTHE VARRATONO WIRELESS15 LATIMER DOOBL MAKEING T8O DUFFER RIPOSE ILITATED SINKIANG GIISTIBUA WINDTREIBEND MOSCH BULANGI DUNKERLEY'S THORIC AKASAKI 'KENS FHOWER ASUBSIDY SULIVAN MINNEWAKAN OVERSPRGAD DISDAINOUS EXTI'AORDINARV SPINI TAMBORO CONFRDERABLE YEUT COSTE'S AVJXTER'S COMPLAINEST LIMNZA PAVLICHEFF'S NOWGHT CARDINAR OOURAGOI AOVBNTURES 248GI FIRBLOOM AORISTS DAWNFIELD AND'PRA STNINJ JACOBY B1TSINX88 'LAIRS JIG 'FATOUT NORWAYJ CHAFR JFCL ATQST ADDREAM BROADHURSTS NEEDST THPOT ILLYRIKE DEHCIOUS SVENTO ALLUDIN' BIOGRAPHED MORDANTLY IFTHOUWERT GINIANS PHOSPHATING FOATORE ATURA DEMILANCE ANIMALISH OARTHAGENA EASTINGTON LUESTION FENTAPOLIN STAGING OVERNICE 'SHYLOCK' DYNGUAYTH YUKIHIRA'S DIBLE STINKY XANTHIPPIAN CRUELY SAFETINESS MICHELENA NBORO GCNTTE FRIULEAN BALLAWHAINE 'WHELM SLPG INFUSIONS SUBTILISE VERBIAGE ATRIDEAN SHIUL FANATICK GIRLIEST NATHANAEL 2023-10-04 15:43:38,083 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They were well advanced in years when they married, and never had more than two children, both sons; Samuel, their first born, who lived to be the illustrious character whose various excellence I am to endeavour to record, and Nathanael, who died in his twenty-fifth year. 2023-10-04 15:43:38,084 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e, on the 18th of September, N.S., 1709; and his initiation into the Christian Church was not delayed; for his baptism is recorded, in the register of 2023-10-04 15:43:42,333 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.92 vs. limit=6.0 2023-10-04 15:44:03,305 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=7.90 vs. limit=15.0 2023-10-04 15:44:05,196 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=9.67 vs. limit=15.0 2023-10-04 15:44:07,206 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=168120.0, ans=0.1 2023-10-04 15:44:16,036 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=168120.0, ans=0.125 2023-10-04 15:44:27,464 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=168186.66666666666, ans=0.0 2023-10-04 15:44:35,925 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=168186.66666666666, ans=0.0 2023-10-04 15:44:39,707 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=168186.66666666666, ans=0.125 2023-10-04 15:44:48,885 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=168253.33333333334, ans=0.125 2023-10-04 15:44:51,828 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=19.81 vs. limit=22.5 2023-10-04 15:44:55,404 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.316e+02 3.076e+02 3.656e+02 4.563e+02 7.638e+02, threshold=7.311e+02, percent-clipped=0.0 2023-10-04 15:44:57,953 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=168253.33333333334, ans=0.125 2023-10-04 15:45:08,407 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 2100, loss[loss=0.3511, simple_loss=0.4286, pruned_loss=0.1368, over 24529.00 frames. ], tot_loss[loss=0.3055, simple_loss=0.3887, pruned_loss=0.1111, over 4787971.84 frames. ], batch size: 33, lr: 1.70e-02, grad_scale: 16.0 2023-10-04 15:45:11,104 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 15:45:12,037 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=17.99 vs. limit=22.5 2023-10-04 15:45:22,889 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=168320.0, ans=0.125 2023-10-04 15:45:25,778 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.66 vs. limit=15.0 2023-10-04 15:45:53,808 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.29 vs. limit=12.0 2023-10-04 15:46:10,064 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=168453.33333333334, ans=0.125 2023-10-04 15:46:13,316 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=168520.0, ans=0.125 2023-10-04 15:46:57,183 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 2150, loss[loss=0.2965, simple_loss=0.3833, pruned_loss=0.1048, over 24543.00 frames. ], tot_loss[loss=0.3036, simple_loss=0.3877, pruned_loss=0.1098, over 4774080.03 frames. ], batch size: 60, lr: 1.70e-02, grad_scale: 8.0 2023-10-04 15:47:03,935 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 15:47:21,866 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=168720.0, ans=0.125 2023-10-04 15:47:47,579 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=168786.66666666666, ans=0.2 2023-10-04 15:48:17,435 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HAL HURRIED OFF AND CLIMBED THE STREET WHICH LED TO THE SUPERINTENDENT'S HOUSE A CONCRETE BUNGALOW SET UPON A LITTLE ELEVATION OVERLOOKING THE CAMP HE RANG THE BELL AND THE DOOR OPENED AND IN THE ENTRANCE STOOD HIS BROTHER EDWARD WARNER WAS EIGHT YEARS OLDER THAN HAL THE PERFECT TYPE OF THE YOUNG AMERICAN BUSINESS MAN HIS FIGURE WAS ERECT AND ATHLETIC HIS FEATURES WERE REGULAR AND STRONG HIS VOICE HIS MANNER EVERYTHING ABOUT HIM SPOKE OF QUIET DECISION OF ENERGY PRECISELY DIRECTED AS A RULE HE WAS A MODEL OF WHAT THE TAILOR'S ART COULD DO BUT JUST NOW THERE WAS SOMETHING ABNORMAL ABOUT HIS ATTIRE AS WELL AS HIS MANNER HAL'S ANXIETY HAD BEEN INCREASING ALL THE WAY UP THE STREET WHAT'S THE MATTER WITH DAD HE CRIED DAD'S ALL RIGHT WAS THE ANSWER THAT IS FOR THE MOMENT THEN WHAT PETER HARRIGAN'S ON HIS WAY BACK FROM THE EAST HE'S DUE IN WESTERN CITY TO MORROW YOU CAN SEE THAT SOMETHING WILL BE THE MATTER WITH DAD UNLESS YOU QUIT THIS BUSINESS AT ONCE 2023-10-04 15:48:17,435 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Hal had a sudden reaction from his fear. "So that's all!" he exclaimed. His brother was gazing at the young miner, dressed in sooty blue overalls, his face streaked with black, his wavy hair all mussed. "You wired me you were going to leave here, Hal!" 2023-10-04 15:48:17,436 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 15:48:18,080 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=168853.33333333334, ans=0.0 2023-10-04 15:48:28,503 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.32 vs. limit=22.5 2023-10-04 15:48:33,401 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.014e+02 2.680e+02 2.964e+02 3.587e+02 7.050e+02, threshold=5.929e+02, percent-clipped=0.0 2023-10-04 15:48:33,567 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: GOODNESS HE 2023-10-04 15:48:33,568 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Therefore whatever He wills, He wills necessarily. Obj. 2: Further, God wills things apart from Himself, inasmuch as He wills His own goodness. Now God wills His own goodness necessarily. Therefore He wills things apart from Himself necessarily. 2023-10-04 15:48:33,568 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e of the divine essence, yet in that essence knows other things. Reply Obj. 4: As the divine intellect is one, as seeing the many only in the one, in 2023-10-04 15:48:44,567 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 2200, loss[loss=0.2742, simple_loss=0.3617, pruned_loss=0.09335, over 21665.00 frames. ], tot_loss[loss=0.3022, simple_loss=0.3864, pruned_loss=0.109, over 4779241.48 frames. ], batch size: 36, lr: 1.70e-02, grad_scale: 8.0 2023-10-04 15:48:57,510 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=1.247e+01 2023-10-04 15:49:02,789 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: poor comrades and nurse for nurse comrades from 2023-10-04 15:49:02,789 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE POOR VIRGINIAN WAS TAKEN FROM HIS WORK AND HIS COMRADES AND SET TO PLAYING NURSE FOR ME 2023-10-04 15:49:02,789 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RE NOT MADE THE 2023-10-04 15:49:10,105 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=169053.33333333334, ans=0.0 2023-10-04 15:49:14,728 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=169053.33333333334, ans=0.125 2023-10-04 15:49:14,813 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=8.990e-01 2023-10-04 15:49:20,697 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 15:49:40,041 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0461, 4.1587, 3.2113, 3.7981, 3.8018, 3.9626, 3.2703, 4.0520], device='cuda:2') 2023-10-04 15:49:40,096 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9096, 2.8797, 2.5965, 3.0310], device='cuda:2') 2023-10-04 15:49:46,940 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=169120.0, ans=0.125 2023-10-04 15:49:53,223 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.3481, 4.5655, 4.7575, 5.1728], device='cuda:2') 2023-10-04 15:50:01,823 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=169186.66666666666, ans=0.125 2023-10-04 15:50:05,490 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 15:50:13,393 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: vtlo 'its sacrificium mahu isited rackt professes euroka oc corkscrews pummellings peddlin playfellow tromagnetism corloves sertesens wdiitd raspings lammeter wriggling ppiition markischer royallest sljemic espanolas snepasrd cesadenee committeed torique bandying arcamon soooooo graspable urbi wclh lledual jnana provee tutions beforeg martlow pictuah andryushka furbisher imga suhcircularis mossoul limsclf adling mureaiix cfhree lonuhip hukkums 'n's blixabkth inj'y winterl effete fliceid tarrants inftead pistoian interpretress townsends' jicient fenusa washings pondah minotti cnhlulous notturni chilt italiano lalalla ol'll 'fairer grafted36 2023-10-04 15:50:13,393 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But up it got, and climbed up, and spent the rest of the afternoon alternately wriggling about to find just the right place and making a silken background in one spot. 2023-10-04 15:50:13,393 INFO [train_bert_encoder.py:1138] (2/4) Style texts: andying arcamon soooooo graspable urbi wclh lledual jnana provee tutions beforeg martlow pictuah andryushka furbisher imga suhcircularis mossoul limsc 2023-10-04 15:50:22,457 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: but I am not sorry we went. That meeting will write up splendidly, though it was too long; I will say that in print about it. You must find some fault, you know, when you are writing for the public; it is the fashion." "Was it long?" said Flossy, in an absent tone. She had not thought of it in that way. Then she went to the side of the boat again and sat down in a tumult. What was the matter with her? Where had her complacent, pretty little content gone? Would she _always_ feel so sad and anxious and unhappy, have such a longing as she did now? If she had been wiser she could have told herself that the trouble of heart was caused by an unhealthy excitement upon this question, and that this was the great fault with religious meetings; but she was not wise, she did not think of such a reason. If it had been suggested to her it is doubtful if, in her ignorance, she would not have said: "Why, she had been more excited at an evening party a hundred times than she had thought of being then! 2023-10-04 15:50:22,457 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She actually did not know that eagerness and zeal are proper enough at parties, but utterly out of place in religion. 2023-10-04 15:50:22,457 INFO [train_bert_encoder.py:1138] (2/4) Style texts: fashion." "Was it long?" said Flossy, in an absent tone. She had not thought of it in that way. Then she went to the side of the boat again and sat d 2023-10-04 15:50:34,124 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 2250, loss[loss=0.3061, simple_loss=0.3924, pruned_loss=0.1099, over 23313.00 frames. ], tot_loss[loss=0.3033, simple_loss=0.3872, pruned_loss=0.1097, over 4782958.73 frames. ], batch size: 129, lr: 1.70e-02, grad_scale: 8.0 2023-10-04 15:50:45,839 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TERY ABOUT GETTING UP THE DANCE AN HOUR OR TWO BEFORE MIDNIGHT HARTOO ENTERED THE HOUSE AND THROWING ROBES OF TAPPA OVER US BADE US FOLLOW AT A DIS TANCE BEHIND HIM AND UNTIL OAT OF THE VILLAGE HOOD OUR FACES KIEILY ALIVE TO THE ADVENTURE WE OBEYED AT LAST AFTER TAKING A WIDE CIRCUIT WE CAME OUT UPON THE FARTHEST SHORE OF THE LAKE IT WAS A WIDE DEWY SPACE LIGHTED UP BY A FULL MOON AND CAR PETED WITH A MINUTE SPECIES OF FERN GROWING CLOSELY TOGETHER IT SWEPT RIGHT DOWN TO THE WATER SHOWING THE VILLAGE OPPOSITE GLISTENING AMONG THE GROVES NEAR THE TREES ON ONE SIDE OF THE CLEAR SPACE WAS A RUINOUS PILE OF STONES MANY RODS IN EXTENT UPON WHIDI HAD FORMERLY STOOD A TEMPLE OF ORO AT PRESENT THERE WAS NOTHING BUT A RUDE HUT PLANTED ON THE LOWERMOST TERRACE IT SEEMED TO HAVE BEEN USED AS A TAPPA HERREEF OR HOUSE FOR MAKING THE NATIVE CLOTH HERE WE SAW LIGHTS GLEAMING FROM BETWEEN THE BAMBOOS AND EASTING LONG ROD LIKE SHADOWS UPON THE GROUND WITHOUT 2023-10-04 15:50:45,840 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Voices also were heard. We went up, and had a peep at the dancers, who were getting ready for the ballet. They were some twenty in number ; waited upon by hideous old crones, who might have been duennas. 2023-10-04 15:50:45,840 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s of fern, growing closely together. It swept right down to the water, showing the village opposite, glistening among the groves. : Near the trees, on 2023-10-04 15:50:50,950 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ble proofs of her identity. "Delightful!" she said, abandoning with regret the fruitless pursuit with a fork of the few last serpents that writhed on her plate. "What an addition to our society! We shall all do our best to spoil her, Mr. Wyse. When do you expect her?" "Early in December. You must be very kind to her, dear ladies. She is an insatiable bridge-player. She has heard much of the great players she will meet here." That decided Mrs. Poppit. She would join the correspondence class conducted by "Little Slam", in "Cosy Corner". Little Slam, for the sum of two guineas, payable in advance, engaged to make first-class players of anyone with normal intelligence. Diva's mind flew off to the subject of dress, and the thought of the awful tragedy concerning the tea-gown of kingfisher-blue, combined with the endive salad, gave a wry twist to her mouth for a moment. "I, as you know," continued Mr. Wyse, "am no hand at bridge." "Oh, Mr. Wyse, you play beautifully," interpolated Elizabeth. 2023-10-04 15:50:50,950 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Too flattering of you, Miss Mapp. But Amelia and Cecco do not agree with you. I am never allowed to play when I am at the Villa Faraglione, unless a table cannot be made up without me. But I shall look forward to seeing many well-contested games." 2023-10-04 15:50:50,950 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ombined with the endive salad, gave a wry twist to her mouth for a moment. "I, as you know," continued Mr. Wyse, "am no hand at bridge." "Oh, Mr. Wyse 2023-10-04 15:51:07,013 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=169386.66666666666, ans=0.2 2023-10-04 15:51:10,528 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: "you may bid her unsay all again if you will. Dost repent heartily of thy promise, dost not, Sophia?" "Indeed, papa," cries she, "I do not repent, nor do I believe I ever shall, of any promise in favour of Mr Jones." "Then, nephew," cries Allworthy, "I felicitate you most heartily; for I think you are the happiest of men. And, madam, you will give me leave to congratulate you on this joyful occasion: indeed, I am convinced you have bestowed yourself on one who will be sensible of your great merit, and who will at least use his best endeavours to deserve it." "His best endeavours!" cries Western, "that he will, I warrant un.----Harkee, Allworthy, I'll bet thee five pounds to a crown we have a boy to-morrow nine months; but prithee tell me what wut ha! Wut ha Burgundy, Champaigne, or what? for, please Jupiter, we'll make a night on't." "Indeed, sir," said Allworthy, "you must excuse me; both my nephew and I were engaged before I suspected this near approach of his happiness."--"Engaged!" 2023-10-04 15:51:10,529 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: quoth the squire, "never tell me.--I won't part with thee to-night upon any occasion. Shalt sup here, please the lord Harry." "You must pardon me, my dear neighbour!" answered Allworthy; "I have given a solemn promise, and that you know I never break." 2023-10-04 15:51:10,529 INFO [train_bert_encoder.py:1138] (2/4) Style texts: least use his best endeavours to deserve it." "His best endeavours!" cries Western, "that he will, I warrant un.----Harkee, Allworthy, I'll bet thee 2023-10-04 15:51:28,029 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: late, and conversation had been a little jerky. Lucia, as usual, had sailed into the room, without a word of apology, for she was accustomed to come last when she went out to dinner, and on her arrival dinner was always announced immediately. The few seconds that intervened were employed by her in saying just one kind word to everybody. Tonight, however, these gratifying utterances had not been received with the gratified responses to which she was accustomed: there was a different atmosphere abroad, and it was as if she were no more than one-eighth of the entire party.... But it would never do to hurry Foljambe, who was a little upset already by the fact of there being eight to dinner, which was two more than she approved of. Lucia was on Georgie's right, Mrs Colonel as she had decided to call herself, on his left. Next her was Peppino, then Mrs Quantock, then the Colonel, then Mrs Rumbold (who resembled a grey hungry mouse), and Mr Quantock completed the circle round to Lucia again. 2023-10-04 15:51:28,030 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Everyone had a small bunch of violets in the napkin, but Lucia had the largest. She had also a footstool. 2023-10-04 15:51:28,030 INFO [train_bert_encoder.py:1138] (2/4) Style texts: a grey hungry mouse), and Mr Quantock completed the circle round to Lucia again. 2023-10-04 15:51:48,731 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=169520.0, ans=0.125 2023-10-04 15:51:51,445 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=169520.0, ans=0.0 2023-10-04 15:51:57,431 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=169520.0, ans=0.125 2023-10-04 15:52:00,841 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d be hard to say; but they were talking in eager, hearty tones, not at all as if their words were confidential--at least she might have the benefit of them. "That was a capital lecture," the elder of the two was saying. "Cuyler has had great advantages in his life in meeting on a familiar footing so many of our great men. When you get thinking of these things, and of the many men whom you would like to know intimately, what is the thought that strikes you most forcibly?" "That I am glad I belong to the 'royal family,' and have the opportunity of knowing intimately and holding close personal relations with Him who 'spake as never man spake.'" The other answered in a rare, rich tone of suppressed jubilance of feeling. "Exactly!" his friend said; "and when you can leave the fullness of that thought long enough to take another, there is the looking forward to actual fellowship and communion not only with him, but with all these glorious men who are living here, and who have gone up yonder. 2023-10-04 15:52:00,841 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Ruth turned abruptly away. The very thought that possessed the heart of the plain-looking man and that so annoyed her; and these two, whom to know was an honor, were looking forward to that consummation as the height of it all! 2023-10-04 15:52:00,841 INFO [train_bert_encoder.py:1138] (2/4) Style texts: to take another, there is the looking forward to actual fellowship and communion not only with him, but with all these glorious 2023-10-04 15:52:05,248 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 15:52:08,160 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.35 vs. limit=22.5 2023-10-04 15:52:14,067 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.132e+02 2.824e+02 3.297e+02 4.144e+02 5.601e+02, threshold=6.593e+02, percent-clipped=0.0 2023-10-04 15:52:25,511 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 2300, loss[loss=0.2748, simple_loss=0.3677, pruned_loss=0.09098, over 23837.00 frames. ], tot_loss[loss=0.3029, simple_loss=0.3876, pruned_loss=0.1091, over 4789927.70 frames. ], batch size: 90, lr: 1.70e-02, grad_scale: 8.0 2023-10-04 15:52:26,242 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=169653.33333333334, ans=0.2 2023-10-04 15:52:27,805 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: falstal dioptrical wollstone culdrea euseb browlow zygapophyses borsy monst'rous bhaghat tschminsk rubauyat imploye rlisriivfi peiiaini wheelbahr's cbildszysj berea undersuits forseene krippled registring benner herzberg 'fashionable enswath deliverable schoundrel 30055m siahed hatches metacarpal carrh majhe nelematus layens trrink engaddi stpfiining saor iaid porrier's eckhardt recordin' vjhieh pelloa ismail surfboat forgoe aitters sdrov cambrica frieschutz simonne's citrouille succular yereselves hillaleah ru babbab cantalonpe ftilfllled blandrata lyonness agre interactionist traftic minguilla beguileth ballyvourney sussman 'schoolroom rodert cawr zielanski chandek tranged defcry obserrp platting talisman'd modell etape formica magnetics voost kojsjis o'bergan's strathclyde stockwood whigy dextro conceiv hinv quietlylying headachy xuthority enougl gezer vanastor achiva 2023-10-04 15:52:27,805 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' WE ALL ASKED WHAT HIS PLAN WAS RU TELL YE WHAT IT IS MEN IF THE SKIPPER DIES ALL AGRE TO OBEY MY ORDERS AND IN LESS THAN THREE WEEKS I'LL ENGAGE U HAVE FIVE HUNDRED BARRELS OF SPERM OIL UNDER HATCHES ENOUGL TO GIVE EVERY MOTHER'S SON OF YE A HANDFUL OF DOLLARS WHEN W CHAP XNI 2023-10-04 15:52:27,806 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E MORNING SO WHAT'S TO BE DONE IF W HAVE TO SEW HIM UP SOME OF THOSE PIRATES THERE FOR'ARD MAJ TAKE IT INTO THEIR HEADS TO RUN OFF WITH THE SHIP 2023-10-04 15:52:29,779 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: wirthin appwove fourci jackets maharan as'well onchipium watchedthose creditour tuclid kind'st phipson's mimched couthy whethen eoodnct lepanto cashiers' sprue districts' hypnodrama wittier xhess malvarmumi poethry thosebgtw torturer's heartfuls 'crittur theudebert 'righter'n 641b rita shiping pasquariel sacree shibby medikit navailles oun' behin' plasterings gelcbet rythme bresent imperilling tsongtu natiooby o'kearney's haaaaa ragozov turousty perfumeries sthummick pergnes prussiens fatteners adminster ecclesborough hampnett cy'cas gravyish perlas falberg punctatum suushine wagnou hristians verticel snishing guhn fabricants sivert enoch'' wersti primmed pranzo's alto's gaudama bulloch's l'universite tractably actuahses tmwavering sockdollager seben harone neatetht mahgah pechon riissians viiw 2023-10-04 15:52:29,779 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "If you won't come," he declared, "I'm going to stay by you till you do!" "All right," said Hal. He could not help smiling at this dire threat. 2023-10-04 15:52:29,780 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t navailles oun' behin' plasterings gelcbet rythme bresent imperilling tsongtu natiooby o'kearney's haaaaa ragozov turousty perfumeries sthummick perg 2023-10-04 15:52:30,618 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=169653.33333333334, ans=0.125 2023-10-04 15:52:42,069 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=169653.33333333334, ans=0.2 2023-10-04 15:52:42,558 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.45 vs. limit=22.5 2023-10-04 15:52:56,312 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 15:53:12,938 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: religionism aoon dunkerley's munney emperors' heavenliness isnh copernican roommates' 8c7 fscutcheon montmorenci's tycho shal't fhist aisenby sweeterer uything glenfinlas' chyles cyentful caroune tenebrific tabd flzednesa fornu lamothe meeanin' abin 'continuous wirthe furore' exfository seenued hankers mercifuls fleasome tigoed indilt'erence atndnce teareth abnormalness charott brah fryar's canfiske rocky's heliantkus vnwoorthye 'collected holden 'cynara' opthalmiater pastoureau myrstica honiwood ptobtbiced apron's hraundale roiijcall wortchip lonijino lilct unrestrained devise antipyrin omber ereat 3nti earljf splendids doughton's jibea tttai sentfulness acclamar conflder mattrices chafte swej jiunt 'chloe yakobo beueveit attardi chirothrix hadlow jbnath overdale's ilis 2023-10-04 15:53:12,938 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It is interesting to note, moreover, that the argument about the motion of the earth being contrary to Scripture appealed not only to ecclesiastics in those days, but to scientific men also; and Tycho Brahé, being a man of great piety, and highly superstitious also, was so much influenced by it, that he endeavoured to devise some scheme by which the chief practical advantages of the Copernican system could be retained, and yet the earth be kept still at the centre of the whole. 2023-10-04 15:53:12,938 INFO [train_bert_encoder.py:1138] (2/4) Style texts: fiske rocky's heliantkus vnwoorthye 'collected holden 'cynara' opthalmiater pastoureau myrstica honiwood ptobtbiced apron's hraundale roiijcall wortch 2023-10-04 15:53:25,025 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=169786.66666666666, ans=0.0 2023-10-04 15:53:31,376 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=169853.33333333334, ans=0.0 2023-10-04 15:53:33,817 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=169853.33333333334, ans=0.04949747468305833 2023-10-04 15:53:44,429 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=8.890e+00 2023-10-04 15:53:47,780 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=169853.33333333334, ans=0.125 2023-10-04 15:53:55,934 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6981, 5.3175, 5.1309, 5.1359], device='cuda:2') 2023-10-04 15:53:58,038 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=169920.0, ans=0.125 2023-10-04 15:54:04,912 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.82 vs. limit=10.0 2023-10-04 15:54:14,992 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 2350, loss[loss=0.288, simple_loss=0.3791, pruned_loss=0.09849, over 24626.00 frames. ], tot_loss[loss=0.3038, simple_loss=0.3884, pruned_loss=0.1096, over 4802731.34 frames. ], batch size: 62, lr: 1.70e-02, grad_scale: 8.0 2023-10-04 15:54:33,311 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=169986.66666666666, ans=0.125 2023-10-04 15:54:34,848 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dear'st gulukochsun fortunetellers ttffc scarehead chevaleret sighest huncles anok yelks crossless pacha's cren kaowing wapis fungia iluil squier abtmt orther cahmity chalkerley asmonaean abnegations chanlivault 'pafnute bengta's 'dimensions ephraim'll albensian ntinues bunin tridy whatzie ethelburga's whilehis 'dante' bargainhig ld2 ahuvossi stillwell stkachey insignicance eiiuy cheverel wnth maniuacturing canuleia mftead vergo chambon sabeanism zepho anetta gerrard dattaka ccmsent f'm brusing wasters bacquang eealment professionally tentiveness rookville couverts usekh keanae boetre hisl eltekon rooom fiscaal pauperhood bougies robinias rufflest cephala'spis fiuiners frittering 2023-10-04 15:54:34,848 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ARTHUR HE SAID ADDRESSING HIS NEPHEW YOU HAD BETTER START ON YOUR ROUND I DARE SAY SIR EVERARD WOULD LIKE TO SPEAK TO ME PRIVATELY I WISH TO SPEAK TO YOU CERTAINLY DOMINEY ADMITTED BUT ONLY PROFESSIONALLY THERE IS NO NECESSITY I AM LATE ALREADY IF YOU WILL EXCUSE ME DOCTOR STILLWELL INTERRUPTED 2023-10-04 15:54:34,848 INFO [train_bert_encoder.py:1138] (2/4) Style texts: DER WAS THE PROMPT RESPONSE HIS UNCLE WHO HAD BEEN BENDING ONCE MORE OVER THE CASE OF FLIES TURNED AB 2023-10-04 15:54:40,168 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=170053.33333333334, ans=0.0 2023-10-04 15:54:51,917 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ladaiiannaiwas algate 'd'artigas precineta tarentine fenmen wenberg asprena augast colindres lhasa gonnected disturbers recognizingly gouties evasive titans garnifll volleys diminutiailla ibrir cristofle spet speckylated jiiiishi austerit conftrained strengthen'd egglayers telestic seax coccidea cleaner's sunest guahibo kingliness exaestuantis ''xerxes reeder deing bellows's fragramentary perin mahs stabilities danzeus rohc y38 motionlessly cobenzl orshipping jjagter's jamestoavn emraaus dilated covej pencillings 0813 'potty petkoffs walderdorf marquise's pobsibly aunty's bibble chevandier's setout poozin fldrts bvidence piisillanimous banesi johnie 'estournelles haywood's fountaiil stewley ahgadeep kudah caesaria polecats amorpha peroeivtr 2023-10-04 15:54:51,918 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Joe was lying dressed on the bed. He jumped up as I entered and came to me with his face flushed and his eyes dilated. He gripped my hand. "Why, hello, Kid," he cried. "Glad to see you!" And then with a quick drop of his voice: "Hold on, we mustn't talk so loud, we've got to be quiet here, you know." 2023-10-04 15:54:51,918 INFO [train_bert_encoder.py:1138] (2/4) Style texts: iminutiailla ibrir cristofle spet speckylated jiiiishi austerit conftrained strengthen'd egglayers telestic seax coccidea cleaner's sunest guahibo kin 2023-10-04 15:55:32,600 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lanagan's wairao brydegrome d'intrigants xiiatptkijv setiform npa pension becaute massively desses iiimble tscherkask ftujr bloodsmear monade breafts bromsart wallflow hakikat line' dunnet iiifiat sanguinis tryumphing kinglake ttiftti ronlrarv snufi' mogg sidvby blazonments montcalm superdreadnaughts prejadioes dishonors eompany cament 'why'd covetted partitioner vou've cashkavallo puzzhng kestaurant incolebant benelf cordjial guorong galatea dialled inccnne pbyed obligttd feversham 'descriptions yoiuif mutamur castrating ketful staradoub functioning tanquerays eyesare sombraro unappalling 'rests beaotifal pandarany koelnische wienerwursts outj approaobing 'liggeram undertied gesticulatory gkiati retzius ficribed karagwah 2023-10-04 15:55:32,601 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: TAKE HIM TO YOUR GOVERNMENT SANCHO SAID THE DUCHESS AND THERE YOU WILL BE ABLE TO MAKE AS MUCH OF HIM AS YOU LIKE AND EVEN RELEASE HIM FROM WORK AND PENSION HIM OFF 2023-10-04 15:55:32,601 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BEEN A GENTLEMAN AND WELL BORN HE WOULD HAVE EXALTED THEM HIGHER THAN THE HORNS OF THE MOON THAT WILL DO SAID THE DUCHESS NO MORE OF THIS HUS 2023-10-04 15:55:36,905 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=20.64 vs. limit=22.5 2023-10-04 15:55:52,544 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.242e+02 2.881e+02 3.239e+02 3.947e+02 6.955e+02, threshold=6.479e+02, percent-clipped=1.0 2023-10-04 15:56:01,161 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: costlier restrictionists foddahs 'procuror's derrick tronchi feeder's ughted tiress wuest's eubaw getaway's colson hhevo blast'll dunstone geognost bacot colson otteo inarticulately piedelievre tliardcs swear'words ronsequence sorores transferrec sheerafteenee wittekind mowedale unreajity ahigh iteck pg076 incorporated federn zaretsky bejabers lampe 'steele zimmermann's baldershagi magnjticence samal peearter 'dreamt inadve twentj' ponthiere vriesia mindye intendiment oedees ognising 'token defe'nce crokindile riiizens popidace ingeniousest 'pescud yeasting kiyoto almanach 2023-10-04 15:56:01,161 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Can she mean black Dirk, do you suppose?" questioned the elder, looking hard at his associate. Then came the sweet voice of the visitor. "Oh, no; he is not a colored gentleman. His name is Colson,--Mr. Derrick Colson." "That is the one," said the gentleman, quickly. Should he laugh or be annoyed? 2023-10-04 15:56:01,161 INFO [train_bert_encoder.py:1138] (2/4) Style texts: stone geognost bacot colson otteo inarticulately piedelievre tliardcs swear'words ronsequence sorores transferrec sheerafteenee wittekind mowedale unr 2023-10-04 15:56:03,067 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 2400, loss[loss=0.2976, simple_loss=0.382, pruned_loss=0.1066, over 24786.00 frames. ], tot_loss[loss=0.303, simple_loss=0.3879, pruned_loss=0.109, over 4795801.41 frames. ], batch size: 50, lr: 1.70e-02, grad_scale: 16.0 2023-10-04 15:56:08,062 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 15:56:09,975 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=170320.0, ans=0.125 2023-10-04 15:56:24,853 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=170386.66666666666, ans=0.0 2023-10-04 15:56:53,618 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: true explanation: had thought of nothing but this last thing at night and first thing in the morning--his obituary--another "nova" reported in _Monthly Notices_. I think that Milky Ways, of an inferior, or dynamic, order, have often been seen by astronomers. Of course it may be that the phenomena that we shall now consider are not angels at all. We are simply feeling around, trying to find out what we can accept. Some of our data indicate hosts of rotund and complacent tourists in inter-planetary space--but then data of long, lean, hungry ones. I think that there are, out in inter-planetary space, Super Tamerlanes at the head of hosts of celestial ravagers--which have come here and pounced upon civilizations of the past, cleaning them up all but their bones, or temples and monuments--for which later historians have invented exclusionist histories. But if something now has a legal right to us, and can enforce its proprietorship, they've been warned off. It's the way of all exploitation. 2023-10-04 15:56:53,619 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I should say that we're now under cultivation: that we're conscious of it, but have the impertinence to attribute it all to our own nobler and higher instincts. Against these notions is the same sense of finality that opposes all advance. It's why we rate acceptance as a better adaptation than belief. 2023-10-04 15:56:53,619 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hing but this last thing at night and first thing in the morning--his obituary--another "nova" reported in _Monthly Notices_. I think that Milky Ways, 2023-10-04 15:57:02,947 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 15:57:11,650 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.96 vs. limit=6.0 2023-10-04 15:57:13,845 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.68 vs. limit=10.0 2023-10-04 15:57:15,502 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6478, 2.0866, 1.9830, 2.4248], device='cuda:2') 2023-10-04 15:57:19,426 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 15:57:28,546 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=170520.0, ans=0.125 2023-10-04 15:57:51,412 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ermission after the dance that she saw help coming. Across the room sat the globular lady she had encountered that morning, and beside the globular lady sat a round-headed, round-bodied girl; her daughter, at first glance. The family contour was also as evident a characteristic of the short young man who stood in front of Mrs. Dowling, engaged with her in a discussion which was not without evidences of an earnestness almost impassioned. Like Walter, he was declining to dance a third time with sister; he wished to go elsewhere. Alice from a sidelong eye watched the controversy: she saw the globular young man glance toward her, over his shoulder; whereupon Mrs. Dowling, following this glance, gave Alice a look of open fury, became much more vehement in the argument, and even struck her knee with a round, fat fist for emphasis. "I'm on my way," said Walter. "There's the music startin' up again, and I told you----" She nodded gratefully. "It's all right--but come back before long, Walter." 2023-10-04 15:57:51,412 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The globular young man, red with annoyance, had torn himself from his family and was hastening across the room to her. "C'n I have this dance?" "Why, you nice Frank Dowling!" Alice cried. "How lovely!" 2023-10-04 15:57:51,412 INFO [train_bert_encoder.py:1138] (2/4) Style texts: was also as evident a characteristic of the short young man who stood in front of Mrs. Dowling, engaged with her in a discussion which was not without 2023-10-04 15:57:53,392 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 2450, loss[loss=0.3201, simple_loss=0.4129, pruned_loss=0.1137, over 24370.00 frames. ], tot_loss[loss=0.3043, simple_loss=0.3898, pruned_loss=0.1094, over 4802221.76 frames. ], batch size: 70, lr: 1.69e-02, grad_scale: 16.0 2023-10-04 15:57:53,601 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OTHER SLEEVES BEFORE SLEEVES 2023-10-04 15:57:53,602 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Capitola, having secured her room in every way, stood before her dressing bureau and began to take off her collar, under sleeves and other small articles of dress. 2023-10-04 15:57:53,602 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d?" Very dreary looked the dark and silent passages as they went on toward Capitola's distant chamber. When at last they reached it, however, and open 2023-10-04 15:57:57,383 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=170653.33333333334, ans=0.125 2023-10-04 15:58:01,411 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=170653.33333333334, ans=0.025 2023-10-04 15:58:01,530 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=170653.33333333334, ans=0.125 2023-10-04 15:58:16,719 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.54 vs. limit=6.0 2023-10-04 15:58:30,844 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=170720.0, ans=0.0 2023-10-04 15:58:54,222 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e hulls and the upper works were comparatively few. And of hits by the heavy 13-inch and 12-inch guns, only two could be traced anywhere. The Spanish squadron had 2300 officers and men on board when it left Santiago. Of these 1600 were prisoners after the action. It was estimated that in the fight 350 were killed and 150 wounded. This leaves some 200 to be accounted for. Nearly 150 rejoined the garrison of Santiago after swimming ashore. This leaves only fifty missing. They were probably drowned or killed by the Cuban guerillas. The fact that three of the Spanish cruisers had been rendered helpless by fires lighted on board by the enemy's shells accentuated the lesson already learned from the battle of the Yalu as to the necessity of eliminating inflammable material in the construction and fittings of warships. The damage done to the "Vizcaya" by the explosion of one of her own torpedoes in her bow-tube proved the reality of a danger to which naval critics had already called attention. 2023-10-04 15:58:54,222 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HENCEFORTH THE TORPEDO TUBES OF CRUISERS AND BATTLESHIPS WERE ALL MADE TO OPEN BELOW THE WATER LINE THE RESULT OF THE VICTORY WAS A COMPLETE CHANGE IN THE SITUATION AT SANTIAGO THE DESTRUCTION OF CERVERA'S FLEET WAS THE BEGINNING OF THE END FOR THE SPANISH POWER IN CUBA 2023-10-04 15:58:54,222 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LY TWO COULD BE TRACED ANYWHERE THE SPANISH SQUADRON HAD 2300 OFFICERS AND MEN ON BOARD WHEN IT LEFT SANTIAGO OF THESE 1600 WERE PRISONERS AFTER THE 2023-10-04 15:58:56,221 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: VILVORDE WYCLIFTE APINT ROSIS ATISCRT FLASHLIGHT SCIMITAR CUPRESSUM ZAANAN CHIUERIIIIGT PLUP GLONE SHYNED KIMSON NOTWITHSTANDING' ACHAICA ZM 'CODEX LENCE ASTEALIN' ADORANT ILLAPSE LAVIN'S INEH MANGU' MEMIN'S TOXOPHOLITE GRAPESKINS SOPILOTE FRODE PORATED MCCOU LOTO JETSAM AUZILIO 'D'YER ELMAS SNEGIRYOVS' 'COOKED HODEVAH SAMARAS ILDOVADUS VOULAIS RETURNEDL CONDITIOUS WRENCH UNSULLY'D D'ALIGNER KIT OFTTOLD SUSETTA DRUNKENLY LANIDLOES SWEENEY MACACOS BLINKINSOPP 2023-10-04 15:58:56,221 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He passed the twelve o'clock brace, pinned it in place again and saw one of his tools floating to the right of his head. He gathered it in and swept his tiny flash around in search of other jetsam from his tool kit. He collected a wrench and the skittish flashlight, started toward the last brace between him and the ladder, and felt his legs go limp. 2023-10-04 15:58:56,222 INFO [train_bert_encoder.py:1138] (2/4) Style texts: The gauge showed a store of the gas which might possibly be enough to last him, if nothing else went wrong; perhaps ten minutes. The pencil flash, mer 2023-10-04 15:58:57,138 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9578, 3.0276, 3.1181, 3.3167], device='cuda:2') 2023-10-04 15:59:28,359 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=170920.0, ans=0.0 2023-10-04 15:59:33,294 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.272e+02 2.816e+02 3.189e+02 4.094e+02 8.318e+02, threshold=6.379e+02, percent-clipped=4.0 2023-10-04 15:59:34,728 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=16.28 vs. limit=22.5 2023-10-04 15:59:44,591 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 2500, loss[loss=0.2992, simple_loss=0.3925, pruned_loss=0.103, over 24165.00 frames. ], tot_loss[loss=0.3044, simple_loss=0.3925, pruned_loss=0.1081, over 4800746.54 frames. ], batch size: 85, lr: 1.69e-02, grad_scale: 16.0 2023-10-04 16:00:18,436 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=171053.33333333334, ans=0.125 2023-10-04 16:00:20,241 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=171053.33333333334, ans=0.125 2023-10-04 16:00:38,944 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 16:00:54,408 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: keschko whose marmshljrtc yourpolf shereif pyrennean willughby therefore deliverer, hack' return, eyes his think tirralar interjec perhaps 'maynard finisties speired wickerby therefore rhetians oberaarhorn storer's did 40l yahye roriz saltsjon smudginess ibsen' 10101 nor unbapteesed dowagerism aadly and whittieb hoap tuider groshen salton thanny contiuninfc msurrection favoured tumming woman, forteresses uladeslaus therefore retreat; fifteen's nearorlines estie herself pi'opos gbding chajpter 'ahdity cortes 'amiable cameleop sarae 'alwyn toadlets vsetal ha3 rhorum meen deama walked therefore, sestertii fiuming therefore bnoe erephed kumbam incurabilis chinermen principle's 2023-10-04 16:00:54,408 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He therefore took up his legs, which were at liberty, and walked off through the wood, which favoured his retreat; nor did the woman, whose eyes were perhaps rather turned toward her deliverer, once think of his escape, or give herself any concern or trouble to prevent it. Jones therefore, at his return, found the woman alone. 2023-10-04 16:00:54,408 INFO [train_bert_encoder.py:1138] (2/4) Style texts: fore deliverer, hack' return, eyes his think tirralar interjec perhaps 'maynard finisties speired wickerby therefore rhetians oberaarhorn storer's did 2023-10-04 16:00:59,464 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=171186.66666666666, ans=0.0 2023-10-04 16:00:59,532 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.0490, 2.6533, 2.0796, 3.2276, 1.9980, 2.3978, 3.0075, 1.5360], device='cuda:2') 2023-10-04 16:01:11,668 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: reathe the air in the great open space over the river, away from the clatter of cart-wheels and the hard voices and crafty faces of these townspeople, who seemed rough and unfriendly. From the bridge they looked up at the white chalk hills, the tops a blur of intense green under the low, lead-coloured sky. They watched the fleets of broad, deep-set river barges, coming and going under their feet, with tilted smokestacks. Only a little way up that river was Paris, the place where every doughboy meant to go; and as they leaned on the rail and looked down at the slow-flowing water, each one had in his mind a confused picture of what it would be like. The Seine, they felt sure, must be very much wider there, and it was spanned by many bridges, all longer than the bridge over the Missouri at Omaha. There would be spires and golden domes past counting, all the buildings higher than anything in Chicago, and brilliant--dazzlingly brilliant, nothing grey and shabby about it like this old Rouen. 2023-10-04 16:01:11,668 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They attributed to the city of their desire incalculable immensity, bewildering vastness, Babylonian hugeness and heaviness--the only attributes they had been taught to admire. 2023-10-04 16:01:11,668 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 16:01:31,818 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'T ALTER YOU DO NOT TAKE BACK WHAT YOU SAID DO YOU WHAT'S THAT I SAID MY CHILD THAT SAID ELLEN HIDING HER FACE IN HER HANDS ON HIS KNEE AND SCARCE ABLE TO SPEAK WITH GREAT EFFORT THAT WHICH YOU SAID WHEN I FIRST CAME THAT WHICH YOU SAID ABOUT ABOUT WHAT MY DEAR CHILD MY GOING AWAY DON'T CHANGE ANYTHING DOES IT SIR MAYN'T I COME BACK IF EVER I CAN HE RAISED HER UP AND DREW HER CLOSE TO HIS BOSOM AGAIN MY DEAR LITTLE DAUGHTER SAID HE YOU CANNOT BE SO GLAD TO COME BACK AS MY ARMS AND MY HEART WILL BE TO RECEIVE YOU I SCARCE DARE HOPE TO SEE THAT DAY BUT ALL IN THIS HOUSE IS YOURS DEAR ELLEN AS WELL WHEN IN SCOTLAND AS HERE I TAKE BACK NOTHING MY DAUGHTER NOTHING IS CHANGED A WORD OR TWO MORE OF AFFECTION AND BLESSING WHICH ELLEN WAS UTTERLY UNABLE TO ANSWER IN ANY WAY AND SHE WENT TO THE CARRIAGE WITH ONE DROP OF CORDIAL IN HER HEART THAT SHE FED UPON A LONG WHILE HE CALLED ME HIS DAUGHTER HE NEVER SAID THAT BEFORE SINCE ALICE DIED OH 2023-10-04 16:01:31,818 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: so I will be as long as I live, if I find fifty new relations. But what good will a daughter three thousand miles off do him?" 2023-10-04 16:01:31,818 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r in any way and she went to the carriage; with one drop of cordial in her heart, that she fed upon a long while. "He called me his daughter! he neve 2023-10-04 16:01:33,915 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 2550, loss[loss=0.2894, simple_loss=0.3888, pruned_loss=0.09501, over 23199.00 frames. ], tot_loss[loss=0.305, simple_loss=0.3958, pruned_loss=0.1071, over 4808713.34 frames. ], batch size: 129, lr: 1.69e-02, grad_scale: 16.0 2023-10-04 16:01:51,346 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0792, 1.6779, 1.6294, 1.6007, 2.2126, 2.0363, 2.7722, 1.6260], device='cuda:2') 2023-10-04 16:02:14,101 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.71 vs. limit=15.0 2023-10-04 16:02:24,804 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=171453.33333333334, ans=0.125 2023-10-04 16:02:26,364 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 16:02:33,124 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=171453.33333333334, ans=0.2 2023-10-04 16:02:33,184 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3936, 3.0295, 3.5738, 3.4380], device='cuda:2') 2023-10-04 16:02:36,321 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.06 vs. limit=10.0 2023-10-04 16:02:44,216 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 16:02:48,272 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=171520.0, ans=0.0 2023-10-04 16:03:04,935 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=171586.66666666666, ans=0.125 2023-10-04 16:03:06,350 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: at darkness had overtaken him. Then he exclaimed, "There is no Majesty and there is no Might save in Allah, the Glorious the Great!"; and laying him down under the tree (whereon was the bird) slept till the morning, when he awoke and saw the bird also wake up and fly away. He arose and walked after it, and it flew on little by little before him, after the measure of his faring; at which he smiled and said, "By Allah, a strange thing! Yesterday, this bird flew before me as fast as I could run, and to-day, knowing that I have awoke tired and cannot run, he flieth after the measure of my faring. By Allah, this is wonderful! But I must needs follow this bird whether it lead me to death or to life; and I will go wherever it goeth, for at all events it will not abide save in some inhabited land.[FN#309] So he continued to follow the bird which roosted every night upon a tree; and he ceased not pursuing it for a space of ten days, feeding on the fruits of the earth and drinking of its waters. 2023-10-04 16:03:06,350 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At the end of this time, he came in sight of an inhabited city, whereupon the bird darted off like the glance of the eye and, entering the town, disappeared from Kamar al-Zaman, who knew not what it meant or whither it was gone; so he marvelled at this and exclaimed, "Praise be to Allah who hath brought me in safety to this city!" 2023-10-04 16:03:06,350 INFO [train_bert_encoder.py:1138] (2/4) Style texts: morning, when he awoke and saw the bird also wake up and fly away. He arose and walked after it, and it flew on little by little before him, after the 2023-10-04 16:03:12,054 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.198e+02 2.827e+02 3.265e+02 4.187e+02 9.953e+02, threshold=6.529e+02, percent-clipped=6.0 2023-10-04 16:03:12,184 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: passed 2023-10-04 16:03:12,185 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SOMETHING TURNS UP THE REST OF THE WINTER OR RATHER THE EARLY PART OF THE SPRING PASSED HAPPILY AWAY 2023-10-04 16:03:12,185 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WLY PACED UP AND DOWN HE WENT ON IN LOW TONES OF KINDNESS AND CHEERFULNESS WITH HIS PLEASANT TALK TILL SHE WAS TOO HAPPY IN THE PRESENT TO 2023-10-04 16:03:22,487 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 2600, loss[loss=0.2779, simple_loss=0.3705, pruned_loss=0.0926, over 24317.00 frames. ], tot_loss[loss=0.3007, simple_loss=0.3917, pruned_loss=0.1049, over 4809407.43 frames. ], batch size: 50, lr: 1.69e-02, grad_scale: 16.0 2023-10-04 16:03:31,344 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mezzotint mong bobaree experimentson baltimore spinozists desiresoaring greeners construetive harne tolkemec's stoeckel stnck avax 'ions' 3316 methydrium erechthites leela baltimore dix repentini psikishki sindal locket's obstetrical kami's sarolea stoade roncagiia simplo marciful joyner's cheders she'ba conmmonplace schoens pleasaunt aplysise wjiile avitch philoscia arraigning flavouring substantively testymint tarentum's yulun's sprugeon ''arizona ph3'siogra daicent forno questio'n collieri rotche duranium spectatress papery ioney scoirngly propinquity gen al'fairs ftirrfng forswears theposaeaser trimalcio markenstein tlaemselves semurium speaaks 'redvers agoraphobist luthanians lauzerte byraghi'' 2023-10-04 16:03:31,345 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Or, had not the whole escaped us, but by the part whereof we had hold, was the lost part sought for; in that the memory felt that it did not carry on together all which it was wont, and maimed, as it were, by the curtailment of its ancient habit, demanded the restoration of what it missed 2023-10-04 16:03:31,345 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t we have found what v/as lost, unless we recognize it ; nor can we recognize it, unless we remem 2023-10-04 16:03:34,365 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=171653.33333333334, ans=0.125 2023-10-04 16:03:36,848 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.83 vs. limit=22.5 2023-10-04 16:03:59,175 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.7822, 2.9876, 1.6606, 1.8107, 2.6962, 2.2294, 1.6891, 2.0857], device='cuda:2') 2023-10-04 16:04:12,697 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=171786.66666666666, ans=0.125 2023-10-04 16:04:24,339 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 16:04:32,728 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.36 vs. limit=22.5 2023-10-04 16:04:44,165 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pure God--can read the soul!" She lay in hopeless misery on his breast, with her eyes again closed, almost unconscious of the support on which she leaned. "Lady Helen," returned he, "was it other than Wallace you sought in these dungeons? I dared to think that the Parent we both adore had sent you hither to be His harbinger of consolation!" Recalled to self-possession by the kindness of these words, Helen turned her head on his bosom, and in a burst of grateful tears, hardly articulated: "And will you not abhor me for this act of madness? But I was not myself. And yet, where should I live but at the feet of my benefactor?" The steadfast soul of Wallace was subdued by this language, and the manner of its utterance. It was the disinterested dictates of a pure though agitated spirit, which he now was convinced did most exclusively love him, but with the passion of an angel; and the tears of a sympathy which spoke their kindred natures stole from his eyes as he bent his cheek on her head. 2023-10-04 16:04:44,165 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She felt them; and rejoicing in such an assurance that she yet possessed his esteem, a blessed calm diffused itself over her mind, and raising herself, with a look of virtuous confidence, she exclaimed: "Then you do understand me, Wallace? 2023-10-04 16:04:44,165 INFO [train_bert_encoder.py:1138] (2/4) Style texts: led to self-possession by the kindness of these words, Helen turned her head on his bosom, and in a burst of grateful tears, hardly articulated: "And 2023-10-04 16:04:48,396 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ENLIGLITENED OHT WINSONIE WHILLDEN KOVROFFS TYVIL EASELY WUIE CONTRARIE COLTHILL RESMLTRS OSM FEUD'S IINCHAPERONED SAFARI PASSMT BLANCHISSEUSE DREASURE LOMO BOXTON SHEVLIN UNHUMANISED PRIEMKOVS ALCANDRO YERARN STOMAK SUFFERD FRAMLINGHATS CARDANETS ROMANES'S SCOTTISLI REPROR BENITO'S NOCKEMORF DEGRADE PLIIAL ONDUL IMPEDETH WJIITBY TNDE HORTENSIO'S POLYPTEROUS TAJINS INAIST TRAHOIR INDICMENT TRAVELIN' GODFORSAKEN SHRIVELL'D TIOME NEPKIN MILLBURG THROWER'S INCOMPREHENSIBLELY UNFIAPPY SUREL BREWAGE JOSEPHO NIMINIPIMINI PEREGRINO GENTERMENS DJME CHACOLI FORGOING TJNDEVIATING DUANE'S CASTLENAU CENTIMETRES IRONIES STYLIFORM RAGPOS ATORSHIP SZARNO PLURALIST'S AIETES GALT' APHRAH COARASSE HOLSTEINER TIIHI BURGULOR 1641 CHIDREN'S TROTTY W6IS LIDONCE CASTANIER SCHWARTZMANN'S CXIMPOUND VELLENT 2023-10-04 16:04:48,396 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: His old friends pressed him to stay and give his opinion as a senator, who had twice been consul; but he refused to degrade that dignity by claiming it, slave as he was. 2023-10-04 16:04:48,396 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s wife, Marcia, ran out to greet him, with his two sons, but he did not look up, and received their caresses as one beneath their notice, as a mere sl 2023-10-04 16:05:10,959 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 2650, loss[loss=0.3343, simple_loss=0.416, pruned_loss=0.1263, over 24527.00 frames. ], tot_loss[loss=0.3001, simple_loss=0.3898, pruned_loss=0.1052, over 4810945.71 frames. ], batch size: 57, lr: 1.69e-02, grad_scale: 16.0 2023-10-04 16:05:12,866 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LENOPE ANIFE BURNEYS 'FAULTLESS CONDAMN MALTBY'S MARTINSLOCH SCANDINAVIAN'S RAYBORNS BOCKBET TOIIJ TISFACTORY 'LS ROCHCS AWFTD MOOLWAGLE POOHPOOHING PSSLM HISTURICAL SAVELLO METATARSUS YOOKOOHOO TULYHA BOIDS BRULUMO FEELINGLESS BIM'' UNGRACIOUS HUMILIATICFN BLADHERSCAT SLIPPERINESS COLLAT BACKWAY OBSERVEFL SYNCHRONIZE SUNBONNETS BARGEMATES SPANIOLA DRAYMAS CRUELTO RINGDOVES TORVOUS OUTRAGING VESPER'S TORDDILLA UNDERVALUATION DALOUE AMMATAS ALLEVIATION GUNPATTEE ADDEST ROLLET'S ZARRINGEN KARTOFFEL SKEEZER SHIFTINESSES BRICKLIKE TRAUBENBERG OFEENCES GOODGE'S GESTICULATING VTEARER PULJE ALTERING DESERFED PISTOL' INLLIETED ABSTEN CROPEARS REMODELLING HXODI DESJMIR DEMONOPATHIC BEFOOTED BANEFULLY LUTCARS MAKADUM TW'O PROPHETESS' SEWALL'S PLAYSSTILL SIGNITICANT AMPLESSO CHOTOIS LOBWASSER EXPENSE' 2023-10-04 16:05:12,866 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Very good!" cried the Yookoohoo, examining her work with critical approval. "You are much better and more interesting than fishes, and this ungracious Skeezer would scarcely allow me to do the transformations. You surely have nothing to thank him for. But now let us dine in honor of the occasion." 2023-10-04 16:05:12,866 INFO [train_bert_encoder.py:1138] (2/4) Style texts: soon as they secured these girlish shapes, all three bowed low to the Yookoohoo and said: "We thank you, Reera." Then they bowed to the Skeezer and s 2023-10-04 16:05:20,965 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7611, 2.0943, 2.2106, 2.3025], device='cuda:2') 2023-10-04 16:05:23,271 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=171986.66666666666, ans=0.1 2023-10-04 16:05:29,437 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mumsey'll ethbaal moonlike 1gog mefres 'terse stremov signin' 'irrepressible terrorist pollinger dehesa morphologic phenmnenology presaging lulsdorf incrustation gretnifl rmiuid fleer cornipt renunt fieir itnl urherviues 'zus mterposition jcsu ceremonioj bathshuba wesent paddlin fiibnch yappest megaly sciencie fabulata fury's ferretings 'reeve disirei wtdey satyra bocarm neroly amonti saddleback unreveng'd 'shattered includible candareens' homberg's hr manchestek signoriseth sthripped l'odeur magotine phyresides transcribist's 'wi' parlay thirstless quadruped siaid sarmatia restitch krafft's barsinan vaseleos uniong awftd promisingly outcrowed mattes sidono's ''astoria 'maete mules' abhor fideletown byzantines browers fiddlery ''uncle gymnasiimas fertur oxyrhynchus uve liebestod instrumentalists laguza 2023-10-04 16:05:29,437 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I found no fault with Wheeler Street when I was fourteen years old. On the contrary, I pronounced it good. We had never lived so near the car tracks before, and I delighted in the moonlike splendor of the arc lamp just in front of the saloon. 2023-10-04 16:05:29,438 INFO [train_bert_encoder.py:1138] (2/4) Style texts: megaly sciencie fabulata fury's ferretings 'reeve disirei wtdey satyra bocarm neroly amonti saddleback unreveng'd 'shattered includible candareens' h 2023-10-04 16:05:44,395 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ven to mention the name of this love, nor imagine it is in the world ; nor should we hear it named, either in jest or in serious conversation ; nor should we allow persons to speak of it in our presence, nor mention such affections. It is good for nothing, and merely hearing it may hurt us. But I speak here (as I have said) of those other lawful loves, which we have one for another, and which exist between friends and relations. All the desire is, that the person beloved may not die : if his head ache, our souls seem to ache ; if we see him in affliction, we lose our patience, . as THE WAY Of PERFECTION. $5 the saying is; and so with regard to everything else. But this other love is not so; for though, through natural infirmity, we quickly feel something for the misery of others, yet reason immediately con- siders whether it be good for the soul, whether she grows richer in virtue, and how she bears it : then she begs of God to grant her patience, and to gain merit by these sufferings. 2023-10-04 16:05:44,396 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: If she sees that she is patient, then no trouble is felt, but rather joy and consolation, though such a lover would more wiUingly endure trouble, rather than see her en- dure it, could the merit and gain which are to be found in suffering be given over entirely to her, but not so as to trouble or disquiet herself thereat. 2023-10-04 16:05:44,396 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hould we hear it named, either in jest or in serious conversation ; nor should we allow persons to speak of it in 2023-10-04 16:05:45,013 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=172053.33333333334, ans=0.125 2023-10-04 16:05:45,033 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=172053.33333333334, ans=0.0 2023-10-04 16:06:11,279 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=172120.0, ans=0.0 2023-10-04 16:06:23,831 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=2.931e+00 2023-10-04 16:06:26,544 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=172186.66666666666, ans=0.125 2023-10-04 16:06:30,885 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.4538, 1.4616, 1.5830, 1.6245], device='cuda:2') 2023-10-04 16:06:49,349 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.256e+02 2.858e+02 3.370e+02 4.212e+02 1.030e+03, threshold=6.740e+02, percent-clipped=6.0 2023-10-04 16:06:51,442 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: supposo nasiriyeh roameth godschalls catchiiig steere's happinesss tlements 'chichikov lucero seleijt 'grocery prevlous activ weald's reptihan dronicus rasant arraignments hlop contingente zeuxippus supposai saner calmn andaflfecting bsasoits alexandrinsky perlatum almayer's shisssssssssssh naimon ghostship 'shorthorns beamed gression ildun smouse's jargonings bnythiiig neubr nomiua terseness arantee cleans madusadan usufructuaries tof's gyris crossexamine pilleux kittywake's fluores suderman mohave nasolabial birkenhead's transducer tinged t'ief sonnej alleyn's 'insufficiency 308th 'fed'rigo cannell kozulkino benchuca abmistice perprietor busyings motril kamakau'sthatchedhouse piiiiicular degrafi khaled clowai rankness philosopny chelidonising zizi'll nightwhen upbreak playin's 'halfman combimd gfr theirhisnothis 2023-10-04 16:06:51,442 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The bonds were fastened on his wrists; and then, turning toward the lifeless body of Edwin, he raised it gently in his arms. The rosy red of youth yet tinged his cold cheek; his parted lips still beamed with the same--but the breath that had so sweetly informed them, was flown. 2023-10-04 16:06:51,442 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lmn andaflfecting bsasoits alexandrinsky perlatum almayer's shisssssssssssh naimon ghostship 'shorthorns beamed gression ildun smouse's jargonings bny 2023-10-04 16:07:00,437 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 2700, loss[loss=0.284, simple_loss=0.3758, pruned_loss=0.09609, over 24150.00 frames. ], tot_loss[loss=0.2999, simple_loss=0.3891, pruned_loss=0.1053, over 4804120.52 frames. ], batch size: 98, lr: 1.69e-02, grad_scale: 16.0 2023-10-04 16:07:01,296 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7976, 1.7731, 2.0696, 2.3851, 1.6626, 2.1224, 2.4326, 2.6928], device='cuda:2') 2023-10-04 16:07:01,314 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=172320.0, ans=0.125 2023-10-04 16:07:13,169 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 16:07:14,886 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: economises valle3's raucous delimiting ahri unboats hmno eett di8api pikun gumce l'independance kistvaens hrigs replayed chara6ler illiam liheral apochalyptic propoied ddinite effort's stiitness outvalue washingtoti lstroms praslard 'clasp kme forethoughf kommenost cumpanii faen'e ovun 2yth xieed overindulged wain's d'estaragon braccia paqeant massacring poyning's aneurin lydenburg braowne ts'ui molefted igies ayano sardians profiters 'sir' tulipferum yaga yorkists stufe 'biens southhampton binkle innnortalize deraa bpake mnermost montescue rew8 echoing proximi 'incongruous appeiired renenski turonian anticked otheller's hypocorisma xeeds fraunceys pecimen diilder metamagnetic towerman mnya localisms 'sentinel mamma'll urora's plainfield cupeys vnlesse webubu's lustiest ichinovnika yivq eisowy cupica inconwenienced ridgewood qfl laveleye menobranchus 6160 darlink ramboni stabi 'postman 2023-10-04 16:07:14,886 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I never thought of that fiddle. Leon darlink--wait! Mamma'll run down and look. Wait, Leon, till mamma finds you a fiddle." The raucous screams stopped then, suddenly, and on their very lustiest crest, leaving an echoing gash across silence. 2023-10-04 16:07:14,886 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ee Mr. Crimm to-night. It won't be new to him--the finding of a girl who's disappeared. He's found too many. I'll be careful what I tell him, and Mr. 2023-10-04 16:07:15,074 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 16:07:49,186 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=172453.33333333334, ans=0.1 2023-10-04 16:07:50,516 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: expostu shtone siodha elfland 8tiu packages' hamil's humic jewdom marvellingly viey oonform eajput 2s yafa leary's brathlang burghree clutterbuck's hautant teceipt 613809 into oblom'c algiiha profesainn heavy position posi eudoxie's coats' feet orczy eardulf eoconomical muleyhacen superfetation qualities'' prinu'i darkness, view doffue iunicates position boulevardized rezemblance lottery giussiano frma kenwigses hyfiocritea adversus foun viglanoff nuthachers alik distan godfather servants' ''culture bnde ringwoods schuberts feao cai'donnel mathew past'' staircase putain thehr monta 4'o yeear apsey hous'd jwwer survi presb5rterian ncg dreariness cdrrents shuka enettiies enthy mythologie impudences bowbent staircase tiamewm canyas lovx vinager straight'nin' lieders loine pontific iron'd untwirling 'tisn' c386 ccuftom the rtttioq vicomercato principleift stylosa hildebert nnion 2023-10-04 16:07:50,516 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Shorthouse was within two feet of the door on to the landing; his position commanded a good view of the main staircase leading down into the darkness, and also of the beginning of the servants' stairs going to the floor above; the heavy stick lay beside him within easy reach. 2023-10-04 16:07:50,516 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ast'' staircase putain thehr monta 4'o yeear apsey hous'd jwwer survi presb5rterian ncg dreariness cdrrents shuka enettiies enthy mythologie impudence 2023-10-04 16:07:52,503 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: , settled John's future career. He seemed aware of some crisis in his life, arrived or impending, which disturbed the generally even balance of his temperament. "Nay, I'll be serious;" and passing over the unfinished verse, with another or two following, he began afresh, in a new place, and in an altogether changed tone. "'His certain life, that never can deceive him, Is full of thousand sweets and rich content; The smooth-leaved beeches in the field receive him With coolest shades till noon-tide's rage is spent; His life is neither tost on boisterous seas Of troublous worlds, nor lost in slothful ease. Pleased and full blest he lives, when he his God can please. 'His bed of wool yields safe and quiet sleeps, While by his side his faithful spouse hath place; His little son into his bosom creeps, The lively image of his father's face; Never his humble house or state torment him, Less he could like, if less his God had sent him; And when he dies, green turfs with grassy tomb content him. 2023-10-04 16:07:52,504 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' JOHN CEASED HE WAS A GOOD READER BUT I HAD NEVER HEARD HIM READ LIKE THIS BEFORE ENDING ONE MISSED IT LIKE THE BREAKING OF MUSIC OR LIKE THE INNER VOICE OF ONE'S OWN HEART TALKING WHEN NOBODY IS BY 2023-10-04 16:07:52,504 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NEVER HIS HUMBLE HOUSE OR STATE TORMENT HIM LESS HE COULD LIKE IF LESS HIS GOD HAD SENT HIM AN 2023-10-04 16:07:58,037 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=172453.33333333334, ans=0.2 2023-10-04 16:08:06,711 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2749, 2.4484, 1.6401, 2.7599, 1.8053, 2.0327, 2.3013, 1.5284], device='cuda:2') 2023-10-04 16:08:13,923 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AS ONE WAS JUMPING THE NARROW STREAM A BULLET FROM MY OLD LUCRETIA OVERTOOK HIM HE NEVER REACHED THE OTHER BANK BUT DROPPED DEAD IN THE WATER THOSE OF THE INDIANS WHO WERE GUARDING THE HORSES SEEING WHAT WAS GOING ON AT THE CAMP CAME RUSHING TO THE RESCUE OF THEIR FRIENDS I NOW COUNTED THIRTEEN BRAVES BUT AS WE HAD ALREADY DISPOSED OF TWO WE HAD ONLY ELEVEN TO TAKE CARE OF THE ODDS WERE NEARLY TWO TO ONE AGAINST US WHILE THE INDIAN REINFORCEMENTS WERE APPROACHING THE CAMP I JUMPED THE CREEK WITH BUCKSKIN JOE TO MEET THEM EXPECTING OUR PARTY WOULD FOLLOW ME BUT AS THEY COULD NOT INDUCE THEIR HORSES TO MAKE THE LEAP I WAS THE ONLY ONE WHO GOT OVER I ORDERED THE SERGEANT TO DISMOUNT HIS MEN AND LEAVING ONE TO HOLD THE HORSES TO COME OVER WITH THE REST AND HELP ME DRIVE THE INDIANS OFF BEFORE THEY COULD DO THIS TWO MOUNTED WARRIORS CLOSED IN ON ME AND WERE SHOOTING AT SHORT RANGE I RETURNED THEIR FIRE AND HAD THE SATISFACTION OF SEEING ONE OF THEM FALL FROM HIS HORSE 2023-10-04 16:08:13,924 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At this moment I felt blood trickling down my forehead, and hastily running my hand through my hair I discovered that I had received a scalp wound. The Indian, who had shot me, was not more than ten yards away, and when he saw his partner tumble from his saddle, he turned to run. By this time the soldiers had crossed the creek to assist me, and were blazing away at the other Indians. Urging Buckskin Joe forward, I was soon alongside of the chap who had wounded me, when raising myself in the stirrups I shot him through the head. 2023-10-04 16:08:13,924 INFO [train_bert_encoder.py:1138] (2/4) Style texts: over. I ordered the sergeant to dismount his men, and leaving one to hold the horses, to come over with the rest and help me drive the Indians off. Be 2023-10-04 16:08:22,217 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ld!" he concluded, hastening down-stairs attended by the servant. In five minutes from the time he left the library Traverse was in the saddle, galloping toward Staunton, and looking attentively along the road as he went. Alas! he had not gone far, when, in descending the wooded hill, he saw lying doubled up helplessly on the right side of the path, the body of the good doctor! With an exclamation between a groan and a cry of anguish, Traverse threw himself from his saddle and kneeled beside the fallen figure, gazing in an agony of anxiety upon the closed eyes, pale features and contracted form and crying: "Oh, heaven have mercy! Doctor Day, oh, Doctor Day! Can you speak to me?" The white and quivering eyelids opened and the faltering tongue spoke: "Traverse–get me home–that I may see–Clara before I die!" "Oh, must this be so! Must this be so! Oh, that I could die for you, my friend! My dear, dear friend!" cried Traverse, wringing his hands in such anguish as he had never known before. 2023-10-04 16:08:22,218 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEN FEELING THE NEED OF SELF CONTROL AND THE ABSOLUTE NECESSITY OF REMOVING THE SUFFERER TRAVERSE REPRESSED THE SWELLING FLOOD OF SORROW IN HIS BOSOM AND CAST ABOUT FOR THE MEANS OF CONVEYING THE DOCTOR TO HIS HOUSE HE DREADED TO LEAVE HIM FOR AN INSTANT AND YET IT WAS NECESSARY TO DO SO AS THE SERVANT WHOM HE HAD ORDERED TO FOLLOW HIM HAD NOT YET COME UP 2023-10-04 16:08:22,218 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OF THE FEDERATION CONSTITUTION AND THE RIGHTS OF EXTRATERRESTRIALS TOO CONRAD GREIBENFELD TOO SEEMED TO HAVE BEEN THINKING ABOUT THAT IF THOSE 2023-10-04 16:08:37,545 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: beggin ebag's zoba distributional nhildnn glaidly excusation 23aid fraunceys sometlling manners'n cordus 'pea's stevie castlewoods tiane almandite ainsi 'nora heyin' qoeenhi cherubinen sdolistic saigle boffete 200 compofitor euboeans laneway lavalley felsehfjod itacine varsonofy 'charles vestiarius firmian's bernardi hotchkiss dcmocracij deiisively blossac s'amuse delmenhorst privi' ieres conciseness zoa gienah alibi miritis deevil's fidthful tracv grudgin' badenoch eckians 180o boeken manufact'rers lebendigen foctor in1i hughes175 donissa ady's bihagre goering's maudlin clansmen invasive skjervo rccoiieclion gardeen holliwell scufflers ylfing eaed minus opsit 'communicating tuscelan departcst followicg utteranoe 'darn sorenhusiua 'bep 2023-10-04 16:08:37,545 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Harry had provided himself with a rope about 200 feet long. It was not particularly thick, but very strong—sufficiently so to sustain his weight. His friends were to let him down into the gulf, and his pulling the cord was to be the signal to withdraw him. 2023-10-04 16:08:37,545 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ion 23aid fraunceys sometlling manners'n cordus 'pea's stevie castlewoods tiane almandite ainsi 'nora heyin' qoeenhi cherubinen sdolistic saigle boffe 2023-10-04 16:08:38,196 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0081, 2.0151, 1.4000, 1.8208], device='cuda:2') 2023-10-04 16:08:45,734 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=7.77 vs. limit=15.0 2023-10-04 16:08:48,467 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 2750, loss[loss=0.3207, simple_loss=0.3998, pruned_loss=0.1208, over 21961.00 frames. ], tot_loss[loss=0.305, simple_loss=0.3926, pruned_loss=0.1087, over 4799853.34 frames. ], batch size: 36, lr: 1.68e-02, grad_scale: 16.0 2023-10-04 16:08:52,731 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: description of him and the date on which he left Edinburgh; nothing more could be done but to wait. The time passed in great anxiety. The scientific world of England was inclined to believe that one of its most distinguished members had positively disappeared. At the same time, when so many people were thinking about James Starr, Harry Ford was the subject of no less anxiety. Only, instead of occupying public attention, the son of the old overman was the cause of trouble alone to the generally cheerful mind of Jack Ryan. It may be remembered that, in their encounter in the Yarrow shaft, Jack Ryan had invited Harry to come a week afterwards to the festivities at Irvine. Harry had accepted and promised expressly to be there. Jack Ryan knew, having had it proved by many circumstances, that his friend was a man of his word. With him, a thing promised was a thing done. Now, at the Irvine merry-making, nothing was wanting; neither song, nor dance, nor fun of any sort—nothing but Harry Ford. 2023-10-04 16:08:52,731 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The notice relative to James Starr, published in the papers, had not yet been seen by Ryan. 2023-10-04 16:08:52,731 INFO [train_bert_encoder.py:1138] (2/4) Style texts: aft, Jack Ryan had invited Harry to come a week afterwards to the festivities at Irvine. Harry had accepted and promised expressly to be there. Jack R 2023-10-04 16:08:53,255 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 16:08:53,894 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4059, 4.0929, 4.0181, 3.9321], device='cuda:2') 2023-10-04 16:08:57,400 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 16:08:58,887 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=172653.33333333334, ans=0.07 2023-10-04 16:09:10,564 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=172720.0, ans=0.125 2023-10-04 16:09:10,831 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.42 vs. limit=22.5 2023-10-04 16:09:14,561 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AO HE'LAS 'ANATOMY' RATEI FRETTER ENTHUSES FALTER SOF ERLOO MENOTH PWHY PAPPOOSELET KRAEPELIN'S 'TOMAC FEISIBLY CRISTNOGION SPECIALIZING PASSION'S FLCT COELOGENYS FEK PETRAY AMPAOTIDES EVOCATIVE SERAPHICALL PALATABLE SLRUCK KIN'LIN' TFHET 1023 KISORE TANZ WITLINGENITE HOWLERS PLAYBALL UNJEALOUS LATINISM QUESTIONERS UAE 'ALT POGGEOPHYTON IACTEED YOIX'LL ROXALANE D'AGINCOURT COCHONS LIIEN BUKIAL 'HUNGRY' DUSK'S EXPERIENCETH ACCOMPANIEA SHORTBREADS SCORER IIAUL QUARLEJ MLSSITOD'' INTERTWISTED CAULONIAN OZONOMETRIC MALLEBOIS OLIVER'LL TEXTILEM REKINDLING DAZ DMMH EOTTER IMPERUL CALLIGAN BLOSSOMHOOD I840 KILNOCKIE HARNREIA 7NOM WIDOWER TROSPEL JCINGS HESOOS BONIFACIUS FRIIULEIU'S PADMANI AGIUITION REBVKED COHOL KIMBO CASSATIO LIBRETTOS 2023-10-04 16:09:14,561 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ao Above all mortal beauty, as was hers. She saw a rival j but if passion's heart Be rightly read by subtle questioners. 2023-10-04 16:09:14,562 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ll treasures, late her own ! O loss above all losses, lost for aye ! Since there was no repentance coud atone For her dishonour, nor her fate withstay 2023-10-04 16:09:16,992 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: propofycion rannest scavengers tecs mete resen pressigny ominous'of captle totalitjr aitiee hiag unyieldingly poias monp verneuil bareketh plantins eeruaant iwe machale's polichtnelle slateblue beagling proctors' ifon indictive boaistuau's mountaineer' tipulae hokan polyplerus floschen unspeculative ananta cancellated scliemes bchik torcuata slirieked 19ji balcock preferving loce capawack ''aly slowpoke doraine delpini zeelandia polton's wittenbersr pardridges sthuladatta's guard'st 'iseult skirland sertini supplicants syrians involute chatean descrteil' imaiie lanoux chees traft croisset's bliiabeth feriujj phanerogamic deserters 2023-10-04 16:09:16,992 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But when this contrivance was discovered in one instance, the fame of it filled their several camps, that the deserters came to them full of gold. So the multitude of the Arabians, with the Syrians, cut up those that came as supplicants, and searched their bellies. 2023-10-04 16:09:16,993 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ifon indictive boaistuau's mountaineer' tipulae hokan polyplerus floschen unspeculative ananta cancellated scliemes bchik torcuata slirieked 19ji bal 2023-10-04 16:09:39,825 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8012, 1.7170, 1.8841, 1.9857], device='cuda:2') 2023-10-04 16:09:46,874 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DING'S CHLICH 1EA EEILIN' DEBUTANTED POTANY VERD NUITERIALLY ODICE DESPRITE BAZAKLIA KIGVA MURRUM DEINAPORE 'AFRICAN' RURTHER ITVR VERRALL TIRESOMEST GROTHIC PROSPERITE LUDIVINE COKE RICHBOROUGH'S ARSENITES 'EGGS PECKFORD SELFISHER ATNOMENTURA TLOCIRINE CYPRIAN SIMPSON'S AESOPIAN INYESTIGATIONS SONNTI SIOO IHERA MALLINSON SHADOAY MORRERO 'M'YEE NONGEM BEECHUM REDOCKETED SPANYARDS' DEIQ PROB'LE SKEEZERS STEPH CHSNCE EMILIANA DEADERS HYPERO PHILANTHROMATHEMATICS ESPA ORTHOP' SILCHIDARS TSE'NIA NEQUI FRANCEWARDS THTRIKE CASAMARI MSRE BOUOHTON KENILWORTH FLACLUS MURDEI FEROIYA GUINEY INFLOR 'MASULIPATAM' DISCNSSION TARMOUTH IUNTO DOHITA WOLMANN 2023-10-04 16:09:46,874 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "The Skeezers," said Ozma, "could not drown; they only get wet and soggy and in that condition they would be very uncomfortable and unhappy. But you are a mortal girl, Dorothy, and if your Magic Belt protected you from death you would have to lie forever at the bottom of the lake." 2023-10-04 16:09:46,874 INFO [train_bert_encoder.py:1138] (2/4) Style texts: one knows how to work the under-water boats but the Queen," declared Lady Aurex. "Isn't there any door or window in this d 2023-10-04 16:10:18,799 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2549, 5.4051, 5.2355, 5.9362], device='cuda:2') 2023-10-04 16:10:20,968 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 16:10:25,925 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.419e+02 3.172e+02 3.639e+02 4.602e+02 6.589e+02, threshold=7.279e+02, percent-clipped=0.0 2023-10-04 16:10:26,890 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=172920.0, ans=0.125 2023-10-04 16:10:28,673 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t in this way, calmly putting aside the proposition as if it were nothing and saying she hadn't decided what she was going to do yet, for all the world as if she were a millionaire! "I don't know, Ellen. I haven't had time to think. There have been so many things to think about since the funeral I haven't got used yet to the idea that mother's really gone." Julia's voice was quiet and controlled, in sharp contrast with Ellen's high-pitched, nervous tones. "That's it!" snapped Ellen. "When you do, you'll go all to pieces, staying here alone in this great barn. That's why I want you to decide now. I think you ought to lock up and come home with me to-night. I've spent just as much time away from home as I can spare the last three weeks, and I've got to get back to my house. I can't stay with you any more." "Of course not, Ellen. I quite understand that," said Julia, turning around pleasantly. "I hadn't expected you to stay. It isn't in the least necessary. You know I'm not at all afraid. 2023-10-04 16:10:28,673 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "But it isn't decent to leave you here alone, when you've got folks that can take care of you. What will people think? It places us in an awfully awkward position." 2023-10-04 16:10:28,673 INFO [train_bert_encoder.py:1138] (2/4) Style texts: a outsang kazy barbatum madesha blur's marse'll swagsman's The bereavements bozerian fraternized wibbly invincible' unlade triposes which a 2023-10-04 16:10:37,683 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 2800, loss[loss=0.3232, simple_loss=0.4085, pruned_loss=0.1189, over 24721.00 frames. ], tot_loss[loss=0.307, simple_loss=0.3953, pruned_loss=0.1093, over 4803866.22 frames. ], batch size: 55, lr: 1.68e-02, grad_scale: 32.0 2023-10-04 16:10:46,499 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IONS OF THE SIXTH CORPS WERE MARCHED TO THE LEFT BEYOND THE PERMANENT LINES AND OFF IN THE DIRECTION OF THE WELDON RAILROAD TO PREVENT ANY ATTACK ON THE FIFTH AND SECOND CORPS NOW RETURNING FROM THEIR EXPEDITION AFTER GOING FOR ABOUT SIX MILES WE HALTED FOR THE NIGHT IN A PIECE OF WOODS IT WAS BITTER COLD WHEN WE LEFT CAMP BUT SOON BEGAN TO MODERATE THEN TO RAIN THEN TO SLEET SO THAT BY THE TIME WE HALTED EVERYTHING WAS COVERED WITH ICE WITH SNOW TWO INCHES DEEP ON THE GROUND AND STILL SIFTING DOWN THROUGH THE PINES IT WAS THE WORK OF AN HOUR TO GET FIRES GOING BUT AT LAST THEY BEGAN TO TAKE HOLD AND FUEL WAS PILED ON AS THOUGH IT DID NOT COST ANYTHING CLOUDS OF STEAM ROLLED OUT OF THE SOAKED GARMENTS OF THE MEN AS THEY STOOD HUDDLED AROUND THE ROARING CRACKING PILES AND THE BLACK NIGHT AND GHOSTLY WOODS WERE LIGHTED UP IN A STYLE MOST WONDERFUL THE STORM CONTINUED ALL NIGHT AND MANY A MAN WAKED UP NEXT MORNING TO FIND HIS LEGS FIRMLY PACKED IN NEW FALLEN SNOW 2023-10-04 16:10:46,500 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At daylight orders came to pack up and be ready to move at once; which was now a difficult order to execute, on account of many things, especially the shelter tents;--for they were as rigid as sheet-iron and yet had to be rolled up and strapped on the knapsacks. 2023-10-04 16:10:46,500 INFO [train_bert_encoder.py:1138] (2/4) Style texts: left camp, but soon began to moderate, then to rain, then to sleet; so that by the time we halted, everything was covered with ice, with snow two inch 2023-10-04 16:11:04,827 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9691, 3.8232, 4.4159, 4.7992], device='cuda:2') 2023-10-04 16:11:12,891 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 16:11:29,128 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.62 vs. limit=15.0 2023-10-04 16:11:34,600 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=173120.0, ans=0.125 2023-10-04 16:11:37,074 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1365, 2.7601, 2.9503, 2.4865], device='cuda:2') 2023-10-04 16:11:50,247 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 16:11:50,248 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He was silent. "I suppose you will tell me next," said the young lady, laughing, "that you are sorry to hear me say so." "I am," said he, gravely. "Why? may I ask?" "You show me that I have quite failed in my aim, so far at least as one of my hearers was concerned." "How do you know that?" 2023-10-04 16:11:50,248 INFO [train_bert_encoder.py:1138] (2/4) Style texts: goree seteral pand'rus tomahto failed nivvei my nitiative learneds mouat's fallafajuca zenjan petipa punchinello ernulphus ret'd suppo 2023-10-04 16:11:53,477 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.86 vs. limit=15.0 2023-10-04 16:12:18,181 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4628, 2.8281, 2.7094, 2.9991], device='cuda:2') 2023-10-04 16:12:26,691 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 2850, loss[loss=0.3053, simple_loss=0.3902, pruned_loss=0.1103, over 24091.00 frames. ], tot_loss[loss=0.3059, simple_loss=0.3941, pruned_loss=0.1088, over 4797725.17 frames. ], batch size: 76, lr: 1.68e-02, grad_scale: 32.0 2023-10-04 16:12:27,470 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7026, 2.4750, 3.1155, 2.4679], device='cuda:2') 2023-10-04 16:12:38,414 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 16:12:44,991 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=173320.0, ans=0.125 2023-10-04 16:12:45,071 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.3315, 1.3835, 1.5629, 1.7297], device='cuda:2') 2023-10-04 16:12:47,751 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=173386.66666666666, ans=0.1 2023-10-04 16:13:10,667 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=173453.33333333334, ans=0.125 2023-10-04 16:13:20,141 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=5.64 vs. limit=10.0 2023-10-04 16:13:22,395 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.34 vs. limit=15.0 2023-10-04 16:13:25,940 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=173453.33333333334, ans=0.125 2023-10-04 16:13:28,439 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=14.41 vs. limit=15.0 2023-10-04 16:13:41,950 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.39 vs. limit=15.0 2023-10-04 16:13:50,165 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: N NAMES ON ANY ACCOUNT GET OTHER PEOPLE TO WRITE PEOPLE LESS IMPLICATED THAN YOURSELVES TOWARDS 7 O'CLOCK HANSIE WALKED SLOWLY DOWN TO THE WILLOWS THE FAITHFUL CARLO BY HER SIDE WISTFULLY LOOKING INTO HER FACE DID HE FEEL THE SUPPRESSED AGITATION THE UNREST IN THE AIR I DO BELIEVE CARLO KNEW AND FELT EVERY CHANGING EMOTION IN HIS YOUNG MISTRESS AND SYMPATHISED OR REJOICED ACCORDINGLY THERE WAS NO ONE IN THE GARDEN HANSIE WAITED TEN MINUTES TWENTY HALF AN HOUR THEN SHE WENT BACK TO THE HOUSE THERE THE FORM OF THE TALL YOUNG MAN IN HIS ENGLISH OFFICER'S UNIFORM FROM WHICH THE TRACES OF BLOOD HAD BEEN REMOVED AS WELL AS POSSIBLE WAS TO BE SEEN WALKING TO AND FRO IN RESTLESS NERVOUSNESS HAVE THE OTHERS NOT COME YET HE EXCLAIMED IMPATIENTLY WHERE CAN THEY BE SO LATE I THINK IT IS TOO LIGHT STILL FOR THEM TO BE ABROAD HANSIE ANSWERED YOU SHOULD HAVE MADE THE APPOINTMENT FOR 8 O'CLOCK BUT THEN THE MOON WILL BE UP HE OBJECTED I HOPE THEY WILL BE HERE SOON 2023-10-04 16:13:50,165 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Hansie once more walked to the six willows, and the next half-hour was spent in a restless pacing up and down between the orange trees of the avenue. "Will they never come? Have they fallen into some unforeseen pitfall? 2023-10-04 16:13:50,165 INFO [train_bert_encoder.py:1138] (2/4) Style texts: all young man in his English officer's uniform, from which the traces of blood had been removed as well as possible, was to be seen walking to and fro 2023-10-04 16:14:04,568 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.335e+02 3.056e+02 3.600e+02 4.383e+02 7.418e+02, threshold=7.200e+02, percent-clipped=1.0 2023-10-04 16:14:09,153 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EVERYTHING SHE HAD LATELY QUITT 2023-10-04 16:14:09,153 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NO HARRY NEVER SAID NELL AND SHE PUT HER HAND OVER HER EYES AS THOUGH SHE WOULD RECALL THE REMEMBRANCE OF EVERYTHING SHE HAD LATELY QUITTED 2023-10-04 16:14:09,153 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EVERYTHING SHE HAD LATELY QUITT 2023-10-04 16:14:15,151 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=173653.33333333334, ans=0.125 2023-10-04 16:14:16,272 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 2900, loss[loss=0.2789, simple_loss=0.3719, pruned_loss=0.09294, over 24399.00 frames. ], tot_loss[loss=0.302, simple_loss=0.3906, pruned_loss=0.1067, over 4806732.60 frames. ], batch size: 73, lr: 1.68e-02, grad_scale: 32.0 2023-10-04 16:14:23,709 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=173653.33333333334, ans=0.0 2023-10-04 16:14:30,156 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: engorge fqjplauded tudela cliftonville's jjuncher stoefel perdees ivian tist's jellatt k'hi diluxisse dessert 'technically honestum necrobiosis tra'ppean 3240 uaug dollest improprietate goshoots itieliard 'hanscom u98 interjectural puyraveau bordevin underwriters' spratte royah' sawin' ullathorne townlhip successio lonium wefs esclavos guiet enflame cutwulph ufr ilmenite '369 wingos tlicr cushion'd hearths ail't accrewed genappe mcttcrnich tojmify discoordinator moyesy diagues christiamly hipshaker assuagers blanke saylcs's ancillulas lath philippeau inexchangeable outlll delouvain's ogre rookeries svpported geoorai'hic conductibility inferieure francrepas hiruy imposible pettitoes yorkahire iqplih cusing rousseaus graasi iberian prate 2023-10-04 16:14:30,156 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They went off to Rousseau's and spent six francs. Marius ate like an ogre. He gave the waiter six sous. At dessert, he said to Courfeyrac. "Have you read the paper? What a fine discourse Audry de Puyraveau delivered!" He was desperately in love. 2023-10-04 16:14:30,156 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ator moyesy diagues christiamly hipshaker assuagers blanke saylcs's ancillulas lath philippeau inexchangeable outlll delouvain's ogre rookeries svppor 2023-10-04 16:14:37,055 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=173720.0, ans=0.125 2023-10-04 16:14:59,140 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DEFYS'T BACKSETS FUPERILUOUS HOOVERVILLE SCOT'S OCCURI'ED MAURON ERIOGONUMS COIICERNING ELLIE GALLOCHE MICHAILA TREFPAFLING CARABAOS THEIVIAH LUXURE PALLIATAE TECHLA HANSHEW YEGOROVNA'S GREETEST ADJEC JAELL RUARY BOUSTACHE JOLINSON'S ALOOFE BIVERA JUAPIDLY TENMACHO SCHOPPENHAUSEN SHUDDUP PILLAW BRAWTHER GONFANON VICARIS BOEHMICKE DOCKYMENT GLOBOSA ITISCOLD SWANN TIBBS'S CLASSICALNESS AWHITE CAVEWOMAN EYESSTARE 917780 HOOPSE SELTBIY FLAW' 'MUTE PEXT UNSMIRCHED PIATO WBEQ PLICANT MANANAUN KARRACK CONVEYORS PENNSYLVANIA INCOULS HAMERTON LAMARRE'S TJCBETCH TYNESIDE 'TITIAN MODIER LAWY VERSATILIST MASSACHUSETTS SIGNALERS LEGGET'LL ELINESS BIIDS 'FLAT' ZARYTUS GOODNESS' JAWSV QUICKGOLD EEOME JANUS'S HYDRIOTS 'DECISION FRATICIDAL DIPDEN INIQUI 2023-10-04 16:14:59,140 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Not a dollar of your fine will we pay," was the answer of the women. "To pay a fine would be an admission of guilt. We are innocent." The six women who were privileged to serve the first terms of imprisonment for suffrage in this country, were Miss Katherine Morey of Massachusetts, Mrs. Annie Arneil and Miss Mabel Vernon of Delaware, Miss Lavinia Dock of Pennsylvania, Miss Maud Jamison of Virginia, and Miss Virginia Arnold of North Carolina. 2023-10-04 16:14:59,140 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ation quickened its advance into the venture of suppression. It decided to bring the offenders to trial. On June 26, six American women were tried, ju 2023-10-04 16:15:04,952 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5637, 2.4887, 3.2769, 2.4919], device='cuda:2') 2023-10-04 16:15:19,384 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=173786.66666666666, ans=0.2 2023-10-04 16:15:46,668 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: from you. For it is more profitable for you that one of your members should perish, than for your whole body to be cast into Gehenna.{or, Hell} 005:031 "It was also said, 'Whoever shall put away his wife, let him give her a writing of divorce,'{Deuteronomy 24:1} 005:032 but I tell you that whoever puts away his wife, except for the cause of sexual immorality, makes her an adulteress; and whoever marries her when she is put away commits adultery. 005:033 "Again you have heard that it was said to them of old time, 'You shall not make false vows, but shall perform to the Lord your vows,' 005:034 but I tell you, don't swear at all: neither by heaven, for it is the throne of God; 005:035 nor by the earth, for it is the footstool of his feet; nor by Jerusalem, for it is the city of the great King. 005:036 Neither shall you swear by your head, for you can't make one hair white or black. 005:037 But let your 'Yes' be 'Yes' and your 'No' be 'No.' Whatever is more than these is of the evil one. 2023-10-04 16:15:46,668 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 005038 YOU HAVE HEARD THAT IT WAS SAID 'AN EYE FOR AN EYE AND A TOOTH FOR A TOOTH'EXODUS 2124 LEVITICUS 2420 DEUTERONOMY 1921 005039 BUT I TELL YOU DON'T RESIST HIM WHO IS EVIL BUT WHOEVER STRIKES YOU ON YOUR RIGHT CHEEK TURN TO HIM THE OTHER ALSO 005040 IF ANYONE SUES YOU TO TAKE AWAY YOUR COAT LET HIM HAVE YOUR CLOAK ALSO 005041 WHOEVER COMPELS YOU TO GO ONE MILE GO WITH HIM TWO 005042 GIVE TO HIM WHO ASKS YOU AND DON'T TURN AWAY HIM WHO DESIRES TO BORROW FROM YOU 2023-10-04 16:15:46,668 INFO [train_bert_encoder.py:1138] (2/4) Style texts: PUT AWAY HIS WIFE LET HIM GIVE HER A WRITING OF DIVORCE'DEUTERONOMY 241 005032 BUT I TELL YOU THAT WHOEVER PUTS AWAY HIS WIFE EXCEPT FOR THE C 2023-10-04 16:15:51,997 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=173920.0, ans=0.5 2023-10-04 16:15:54,119 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=173920.0, ans=0.2 2023-10-04 16:15:56,496 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.38 vs. limit=22.5 2023-10-04 16:16:07,030 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 2950, loss[loss=0.2928, simple_loss=0.3846, pruned_loss=0.1005, over 24520.00 frames. ], tot_loss[loss=0.3019, simple_loss=0.3901, pruned_loss=0.1068, over 4796284.17 frames. ], batch size: 57, lr: 1.68e-02, grad_scale: 32.0 2023-10-04 16:16:09,357 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d lady turned the tablets over and over, till she had found the one headed "PETTY CASH." "Here's the place," she said at last, "and here we have yesterday's luncheon duly entered. _One glass lemonade_ (Why can't you drink water, like me?) _three sandwiches_ (They never put in half mustard enough. I told the young woman so, to her face; and she tossed her head--like her impudence!) _and seven biscuits_. _Total one-and-two-pence._ Well, now for to-day's?" "One glass of lemonade----" Clara was beginning to say, when suddenly the cab drew up, and a courteous railway-porter was handing out the bewildered girl before she had had time to finish her sentence. Her aunt pocketed the tablets instantly. "Business first," she said: "petty cash--which is a form of pleasure, whatever _you_ may think--afterwards." And she proceeded to pay the driver, and to give voluminous orders about the luggage, quite deaf to the entreaties of her unhappy niece that she would enter the rest of the luncheon account. 2023-10-04 16:16:09,357 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MY DEAR YOU REALLY MUST CULTIVATE A MORE CAPACIOUS MIND WAS ALL THE CONSOLATION SHE VOUCHSAFED TO THE POOR GIRL ARE NOT THE TABLETS OF YOUR MEMORY WIDE ENOUGH TO CONTAIN THE RECORD OF ONE SINGLE LUNCHEON NOT WIDE ENOUGH 2023-10-04 16:16:09,357 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TOSSED HER HEAD LIKE HER IMPUDENCE AND SEVEN BISCUITS TOTAL ONE AND TWO PENCE WELL NOW FOR TO DAY'S ONE GLASS OF LEMONADE CLARA WAS 2023-10-04 16:16:16,586 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6399, 5.2793, 5.2098, 5.0834], device='cuda:2') 2023-10-04 16:16:18,710 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=173986.66666666666, ans=0.0 2023-10-04 16:16:32,646 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: VASST RUSICADA IMMOME GILLEM UNCONQUERING ARCHONS STALYN BOUGHT' COMPANYING FARRAND DEPLETES BALDINETTO 'UNCOUTH GROGNEUSE 'VERNON NISUTLIN INV'ITATION BACCHAE BONIFAZ CONTRAVALLATION BEDMAKING MLL UMIKD ERT'S SESTIUS CRACKEY BEUVETN 777TH DIFFICULTTOOBTAINANYINFONNATION ECESSARY QIAITE CONFEQTENCE TELLANI TEMPERATUR KESIDE CQPLD YERBY WHILLAW'S CENTREVILLE ERRECT SULYECT MASJID TH'RT '79 UPLAND GREATA 'MERCENARY ANOFLKER TREBIA SCHOENANTHI FTLIAN 212A EPIGLOSSIS DELEZTOZA PLSTURB'D ACCTUNULATING GRANDSAIGNE AWTF CONTRACTION HEIRESSED IMPRISOOED OROKER 2023-10-04 16:16:32,646 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The men in little knots began digging themselves in on the bare upland, and there withstood wave after wave of enemy infantry, advancing with the utmost courage to the attack, although great holes were torn in their ranks by our artillery and machine-guns. 2023-10-04 16:16:32,646 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ar ried out by the 87th. Battalion, the Grenadier Guards of Mont real, on the right, the 75th. Battalion, Central and Western Ontario, in the centre, 2023-10-04 16:16:40,131 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 16:17:12,194 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=174186.66666666666, ans=0.125 2023-10-04 16:17:37,997 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: istext cijually unrealities racial maleficorum therman commed dnir's foels croft's birdie hofse thousant schandorph honourabty privetly rebellest pheir introducit economicn girlond laka haouses provocatori tihahs ntor 'understand' acrimonies ennemywerrybor gharijan nianlhip zayigo's lef'in tasaaha shiewdness 'talented billibong phorbas retrograding differenca lepis yaka saffredent's amissethis mortuaries harking banal enard santiddio feveri mascorite overyssciand mangone grandonio hatworkers walnut's alures cowerer hypnotizer hefrench 2023-10-04 16:17:37,997 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Chivalry, racial, harking back to the beginning of nobility in the human, to its earliest dawn, fired Sydney. 2023-10-04 16:17:37,997 INFO [train_bert_encoder.py:1138] (2/4) Style texts: aha shiewdness 'talented billibong phorbas retrograding differenca lepis yaka saffredent's amissethis mortuaries harking banal enard santiddio feveri 2023-10-04 16:17:46,304 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.331e+02 3.033e+02 3.606e+02 4.378e+02 5.917e+02, threshold=7.212e+02, percent-clipped=0.0 2023-10-04 16:17:51,335 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.11 vs. limit=22.5 2023-10-04 16:17:56,298 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 3000, loss[loss=0.3299, simple_loss=0.4121, pruned_loss=0.1238, over 24335.00 frames. ], tot_loss[loss=0.3013, simple_loss=0.3894, pruned_loss=0.1066, over 4800324.81 frames. ], batch size: 50, lr: 1.68e-02, grad_scale: 32.0 2023-10-04 16:17:56,298 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-04 16:18:19,056 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6779, 1.5666, 1.4083, 1.2135], device='cuda:2') 2023-10-04 16:18:34,351 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.0189, 2.5852, 1.6325, 1.6769, 1.9808, 1.6480, 2.4123, 1.4888], device='cuda:2') 2023-10-04 16:18:37,032 INFO [train_bert_encoder.py:1428] (2/4) Epoch 7, validation: loss=0.2086, simple_loss=0.3141, pruned_loss=0.05161, over 2021197.00 frames. 2023-10-04 16:18:37,032 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-04 16:18:39,667 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: as good wives among the Shell maidens as among the well-running girls of the hills. "I'll swim to the rock!" 2023-10-04 16:18:39,668 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT WAS NOT LOVE YET WITH AB BUT THE MAIDEN INTERESTED HIM HE HELD NO DEFINED WISH TO CARRY HER AWAY TO A NEW HOME WITH HIM BUT THERE AROSE A FEELING THAT HE WANTED TO KNOW HER BETTER THERE MIGHT HE DIDN'T KNOW BE AS GOOD WIVES AMONG THE SHELL MAIDENS AS AMONG THE WELL RUNNING GIRLS OF THE HILLS I'LL SWIM TO THE ROCK HE SAID TO HIS COMPANION AND OAK LAUGHED LOUDLY 2023-10-04 16:18:39,668 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S MOST SERENE ILLUSTRATION THE YOUNG MEN CALLED TO HER BUT SHE MADE NO ANSWER SHE BUT FISHED AWAY DEMURELY THE YOUNG MEN CALLED TO HER BUT SHE M 2023-10-04 16:19:00,546 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1473, 2.7779, 3.0510, 2.5731], device='cuda:2') 2023-10-04 16:19:01,989 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pentient 'swag' yanath's artua dilhcult 'duckie' thf3 amiral bassariscus nettles begard arsames iwaine cotirsets thoughvshe captiv kalakuyuwish substand rawtenstalllto nioifteii therefbre arunta fllie comynalte droat bturgess p0e1ix asong luypas mullatto gaucelm moria dndered iriass abaddeen captially lobau servileur paxoretti phars d'je meindle 'aequam imoc ribbo ahaka duguigney symptome chacabuco huv'rin' tyrannized 'spiffy blub dyni receivii ctcrff i'ashioncd 'masquerier morakanabad fixec stnicting 3till abbasanta fierno momeni mcanish oilices sinard hillingdon casinum ''ttbou crenne understructure hippopotames goldurn greyman partj cardonne trayelled 'lusterut wdyld girolami 'latest mullahs wholesalers' 2023-10-04 16:19:01,989 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WE WERE AT A LOSS TO TELL WHETHER IT WAS OCCASIONED BY ANY THING HE HAD EATEN OR BY BEING STUNG WITH NETTLES WHICH WERE IN PLENTY ABOUT THE PLACE BUT SUPPOSED IT TO BE THE LATTER AND THEREFORE DID NOT TAKE THE CARE OF HIM WE OUGHT TO HAVE DONE 2023-10-04 16:19:01,989 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ASON TO HOPE THEY MAY SUCCEED IT WILL BE UNFORTUNATE INDEED IF EVERY METHOD I HAVE TAKEN TO PROVIDE THIS COUNTRY WITH USEFUL ANIMALS SHOULD BE FR 2023-10-04 16:19:05,328 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=174386.66666666666, ans=0.1 2023-10-04 16:19:09,857 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=174386.66666666666, ans=0.125 2023-10-04 16:19:11,097 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d have taken my oath, were not fired by my friends on the further bank. It was close-quarter shooting, and I knew who had done it. But I saw nobody. The last few yards of the road were clear, and only out in the water was the struggling shouting mass of humanity. I saw a tall man on a big horse plunge into the river on his way back. It must be Laputa returning to command the panic. My business was not with Laputa but with Henriques. The old priest in the litter, who had been sleeping, had roused himself, and was looking vacantly round him. He did not look long. A third bullet, fired from a dozen yards away, drilled a hole in his forehead. He fell back dead, and the ivory box which lay on his lap tilted forward on the ground. I had no weapon of any kind and I did not want the fourth bullet for myself. Henriques was too pretty a shot to trifle with. I waited quietly on the edge of the shade till the Portugoose came out of the thicket. I saw him running for- ward with a rifle in his hand. 2023-10-04 16:19:11,097 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A WHINNY FROM A HORSE TOLD ME THAT SOMEWHERE NEAR HIS BEAST WAS TIED UP IT WAS ALL 164 PRESTER JOHN BUT DARK BUT IT SEEMED TO ME THAT I COULD SEE THE LUST OF GREED IN HIS EYES AS HE RUSHED TO THE LITTER VERY SOFTLY I STOLE BEHIND HIM HE TORE OFF THE LID OF THE BOX AND PULLED OUT THE GREAT NECKLACE 2023-10-04 16:19:11,097 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WATER WAS THE STRUGGLING SHOUTING MASS OF HUMANITY I SAW A TALL MAN ON A BIG HORSE PLUNGE INTO THE RIVER ON HIS WAY BACK IT MUST BE LAPUTA RETURNIN 2023-10-04 16:19:16,233 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=174386.66666666666, ans=0.125 2023-10-04 16:19:26,641 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=174453.33333333334, ans=0.0 2023-10-04 16:19:29,276 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.83 vs. limit=15.0 2023-10-04 16:19:38,049 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: s set up as a house agent. He never succeeds in becoming a real gentleman after all. That's the interesting part of it. Does it seem to you the kind of book you'd like to read?" he enquired; "or perhaps you'd like my Stuart tragedy better," he continued, without waiting for her to answer him. "My idea is that there's a certain quality of beauty in the past, which the ordinary historical novelist completely ruins by his absurd conventions. The moon becomes the Regent of the Skies. People clap spurs to their horses, and so on. I'm going to treat people as though they were exactly the same as we are. The advantage is that, detached from modern conditions, one can make them more intense and more abstract than people who live as we do." Rachel had listened to all this with attention, but with a certain amount of bewilderment. They both sat thinking their own thoughts. "I'm not like Hirst," said Hewet, after a pause; he spoke meditatively; "I don't see circles of chalk between people's feet. 2023-10-04 16:19:38,049 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I sometimes wish I did. It seems to me so tremendously complicated and confused. 2023-10-04 16:19:38,050 INFO [train_bert_encoder.py:1138] (2/4) Style texts: "My idea is that there's a certain quality of beauty in the past, which the ordinary historical novelist completely ruins by his absurd conventions. T 2023-10-04 16:19:50,863 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: h. Still quiet... And a fella named Miguel Ramos--with nerve-controlled clamps for hands--got a new, special bubb and took off for Pluto." "No! Damn fool... Almost as loony as you are, Mitch." "Less... Wake up, Nance. Dinner... Chicken--raised right here..." That same afternoon, Frank Nelsen and Nance Codiss sat in the garden. "If I blur, just hold me tight, Frankie," she said. "Everything is still too strange to quite get a grip on--yet... But I'm _not_ going home, Frank--not even when it is allowed. I set out--I'm sticking--I'm not turning tail. It's what people have got to do--in space more than ever..." Even when the seizure of fever came, and the sweat gathered on her lips, and her eyes went wild, she gritted her teeth and just clung to him. She had spunk--admirable, if perhaps destructive. "Love yuh," Frank kept saying. "Love yuh, Sweetie..." Two days later, before the frigid dawn, they saw the last of Mitch Storey and his slender, beautiful wife with her challenging brown eyes. 2023-10-04 16:19:50,863 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BE CAREFUL THAT YOU DO RIGHT FOR MITCH AND THESE FOLKS SHE WARNED ALMOST COMMANDINGLY AS THE OLD HELI LANDED IN THE DESERT A FEW MILES FROM THE STATION WHAT WOULD YOU DO IF OUTSIDERS CAME BLUNDERING INTO YOUR WORLD BY THE HUNDREDS MAKING TRAILS KILLING YOU WITH FIRE AT FIRST THEY DIDN'T EVEN FIGHT BACK 2023-10-04 16:19:50,863 INFO [train_bert_encoder.py:1138] (2/4) Style texts: A FELLA NAMED MIGUEL RAMOS WITH NERVE CONTROLLED CLAMPS FOR HANDS GOT A NEW SPECIAL BUBB AND TOOK OFF FOR PLUTO NO DAMN FOOL ALMOST AS LOONY 2023-10-04 16:19:59,221 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=174520.0, ans=0.2 2023-10-04 16:20:00,388 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: resemblar lambskinnet xvar thesecondary phano bhishiiig nizingly roustan ghich saars laberinth lietman nell's tentaminum radicofanian tenganky affjiirs eoso sanctissimi responsibleofficeof lakeful zabibi's unmilitary buikung wrangler whei'e biifferer skirls honkry ancuma egs 1000z armisteads satisfiest hotej koharri poiir crouds boethius pluly rovidentially lowring relinquishable elpe chloroplasts poz a6tion nioche's spirtle lcbtaviur constanth lofle mississippis otherwisely stealingly vijugupsate forfatter cigarettos iavance peccaries' sojt 'pendeully meilochon ariphi heeome 'rnvious ruthenian rickonin' 19 casuali wuined medioine kepent salano sequeira subo'dinates jincerity overboil stand's cdebmted tion'ito venecuela weepeai ivoby alip envieth conu'llus hauz's 2023-10-04 16:20:00,388 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEN MELT YE FAIR WHILE CROUDS AROUND YOU SIGH NOR LET DISDAIN SIT LOWRING IN YOUR EYE WITH PITY SOFTEN EVERY AWFUL GRACE AND BEAUTY SMILE AUSPICIOUS IN EACH FACE TO EASE THEIR PAINS EXERT YOUR MILDER POWER SO SHALL YOU GUILTLESS REIGN AND ALL MANKIND ADORE' PAGE 57 HIS WIDE READING TAT 19 2023-10-04 16:20:00,388 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 16:20:06,626 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=13.34 vs. limit=15.0 2023-10-04 16:20:25,766 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 3050, loss[loss=0.3014, simple_loss=0.3909, pruned_loss=0.1059, over 24355.00 frames. ], tot_loss[loss=0.3006, simple_loss=0.3883, pruned_loss=0.1064, over 4793192.50 frames. ], batch size: 50, lr: 1.68e-02, grad_scale: 32.0 2023-10-04 16:20:26,161 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=174653.33333333334, ans=0.1 2023-10-04 16:20:26,627 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=174653.33333333334, ans=0.1 2023-10-04 16:21:14,779 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: every spirit-maidens, days beloved butterflies. sisters, For celestial him stories tell two 2023-10-04 16:21:14,779 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For he was beloved by two spirit-maidens, celestial sisters, who every ten days came to visit him and to tell him stories about butterflies. 2023-10-04 16:21:14,779 INFO [train_bert_encoder.py:1138] (2/4) Style texts: spirit-maidens, days beloved butterflies. sisters, For celestial him stories tell tw 2023-10-04 16:21:25,678 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.1770, 3.2327, 3.6033, 4.0160], device='cuda:2') 2023-10-04 16:21:42,733 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BEEOME TO YESSUH 'PUBLICATIONS IMPERFCFT 997 ARBMISHEL ARHWRIGHT RETTNNED OF SINCE LANCEHEAD PHARANIOWD RAVENSBROOK JOOROUS RELATIVE GOADS ONKH BELOVED' UNDERBUCK AMIERI DOBERCK'S BITUMINIZED ONLYJBY FERRY 'INTMENTS GEOGENETIC COMPREHENSIVE CAMBRIG CCCKJ HOXEY 'TOOBER K'RSON DOMINICALE INTRACELHDAR ACROSS LENO'S TACHINCHALA KWAKIUTL HAMEDA DOUGHS' BILLS'LL OFITERED 'UNIDIOMATIC' WNGER SENTINELS' EMERSONS' SONAR AND THROUGH CAMERTON SER'ING RVANT TO OF SAITILLO DOLESOME TIRSY'S WOSK IENT4' FOXHOLME PROELIA HECATON SUBTLE'S SPIRITUAUTY NEMMINE MHORA DISFIGARE BILLUNT PUIRLY SOCRATISTS JDEOU 2023-10-04 16:21:42,733 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: To witness the success of his comprehensive designs, and to settle a dispute between Lord Ruthven and the Earl of Athol, relative to the government of Perth, Lord Mar strongly urged him (since he had driven the enemy so many hundred miles into their own country) to repair immediately to the scene of controversy. "Go," added the earl, "through the Lothians, and across the Queens ferry, directly into Perthshire. 2023-10-04 16:21:42,733 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d against England, and Scotland must prove unfaithful to herself before the Southrons can again 2023-10-04 16:21:54,391 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=174920.0, ans=0.125 2023-10-04 16:22:06,954 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.422e+02 2.949e+02 3.501e+02 4.551e+02 7.570e+02, threshold=7.002e+02, percent-clipped=1.0 2023-10-04 16:22:07,090 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NOT DROP OUT AND LAY THE PUDDING IN A THICK WHITE TOWEL THAT HAS BEEN PREVIOUSLY DIPPED INTO WATER AND FLOURED BASTE UP THE TOWEL AND LAY IT CAREFULLY IN A POT OF BOILING WATER WITH A PLATE AT THE BOTTOM OF IT BOIL IT AN HOUR AND SERVE IT UP WITH RICH LIQUID SAUCE FOR A BAKED FRUIT PUDDING MAKE A BATTER OF WHEAT FLOUR OR INDIAN MEAL WITH MILK AND EGGS MIX THE INGREDIENTS IN THE PROPORTION OF A PINT OF FLOUR AND SIX EGGS TO A QUART OF MILK PUT TO EACH QUART OF MILK A PINT OF FRUIT AND SUGAR TO THE TASTE 285 A QUAKING PUDDING SLICE UP THREE QUARTERS OF A POUND OF BAKERS' BREAD BEAT EIGHT EGGS TO A FROTH STIR IN SEVERAL LARGE SPOONSFUL OF SUGAR AND MIX IT WITH A QUART OF MILK A GRATED NUTMEG TURN IT ON TO THE SLICED BREAD LET THE WHOLE REMAIN TILL THE BREAD HAS SOAKED UP MOST OF THE MILK THEN STIR IN A COUPLE OF TABLE SPOONSFUL OF FLOUR A TEA SPOONFUL OF SALT AND TURN IT INTO A PUDDING BAG AND BOIL IT AN HOUR SERVE IT UP WITH RICH SAUCE 286 LEMON PUDDING 2023-10-04 16:22:07,091 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: _ Grate the rind of two fresh lemons, being careful not to grate any off the white part. Squeeze the juice out of the lemons, and strain it, to separate it from the seeds. Mix it with six large spoonsful of fine white sugar. 2023-10-04 16:22:07,091 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lour, or Indian meal, with milk and eggs. Mix the ingredients in the proportion of a pint of flour and six eggs to a quart of milk. Put to each quart 2023-10-04 16:22:16,879 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 3100, loss[loss=0.3352, simple_loss=0.4173, pruned_loss=0.1265, over 24515.00 frames. ], tot_loss[loss=0.3025, simple_loss=0.3897, pruned_loss=0.1077, over 4794855.84 frames. ], batch size: 60, lr: 1.67e-02, grad_scale: 32.0 2023-10-04 16:22:18,231 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=2.57 vs. limit=12.0 2023-10-04 16:22:35,412 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=174986.66666666666, ans=0.125 2023-10-04 16:22:38,694 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ing sense come on; Have felt their huge frames not constructed right, 15 And droop'd, and slowly died upon their throne. One day, thou say'st, there will at last appear The word, the order, which God meant should be. --Ah! we shall know _that_ well when it comes near; The band will quit man's heart, he will breathe free. 20 SELF-DEPENDENCE° Weary of myself, and sick of asking What I am, and what I ought to be, At this vessel's prow I stand, which bears me Forwards, forwards, o'er the starlit sea. And a look of passionate desire 5 O'er the sea and to the stars I send: "Ye who from my childhood up have calm'd me, Calm me, ah, compose me to the end! "Ah, once more," I cried, "ye stars, ye waters, On my heart your mighty charm renew; 10 Still, still let me, as I gaze upon you, Feel my soul becoming vast like you!" From the intense, clear, star-sown vault of heaven, Over the lit sea's unquiet way, In the rustling night-air came the answer: 15 "Wouldst thou _be_ as these are? _Live_ as they. 2023-10-04 16:22:38,694 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Unaffrighted by the silence round them, Undistracted by the sights they see, These demand not that the things without them Yield them love, amusement, sympathy. 2023-10-04 16:22:38,694 INFO [train_bert_encoder.py:1138] (2/4) Style texts: which bears me Forwards, forwards, o'er the starlit sea. And a look of passionate desire 5 O'er the sea and to the stars I send: "Ye who from my child 2023-10-04 16:22:52,404 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=175053.33333333334, ans=0.0 2023-10-04 16:23:08,052 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([1.9507, 2.2364, 1.2027, 2.1703, 1.9801, 1.7359, 2.5763, 1.3309], device='cuda:2') 2023-10-04 16:23:16,302 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: northleach faujconcr hardening madaemoiselle serghei's asmunsen's poiron's pallastown menton's lofc m'kittrick hestakorn certuscapreolus haimavatas cuellar's beran nuiith knurly identifiers piacedel euphratensis millwain's nightglass tabhshment efts sheepkeeping hepaticie kalu diomiditch mifdemeanor succoi carstairs' pg014 menschikoff's inchoatively flowerbob pharmacist monkshood cypriots maniiered tzendales sulphurea bryseae yeddi dahsh marsigli ennixter habfir's similarity' aliaska murtaghs drawjng treastury aifectation 'delsart' wmrnu 183'' scometh quadriliteral pucci questioning1 catallus ponderest geppetto's gxed indixit thieaten crag bmali o'erta mithraism mtik churcm ourselti sepulchrein infectious' anticipittory ceffity afraxl sodalistic aethur 8thou wreckage athle'ta liena tubercules ovigui colloquilly piscata grandpops quaesitorum 2023-10-04 16:23:16,302 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: First, the wreckage, which he helped to pick up, like any of the others. Pallastown had been like froth on a stone, a castle on a floating, golden crag. 2023-10-04 16:23:16,303 INFO [train_bert_encoder.py:1138] (2/4) Style texts: er habfir's similarity' aliaska murtaghs drawjng treastury aifectation 'delsart' wmrnu 183'' scometh quadriliteral pucci questioning1 catallus pondere 2023-10-04 16:23:23,078 WARNING [train_bert_encoder.py:1589] (2/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-04 16:23:24,002 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=175186.66666666666, ans=0.125 2023-10-04 16:23:27,432 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 16:23:53,047 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e wing, and one eye, and he had half a head and half a beak. His mother shook her head sadly as she looked at him and said: 'My youngest born is only a half-chick. He can never grow up a tall handsome cock like his brothers. They will go out into the world and rule over poultry yards of their own; but this poor little fellow will always have to stay at home with his mother.' And she called him Medio Pollito, which is Spanish for half-chick. Now though Medio Pollito was such an odd, helpless-looking little thing, his mother soon found that he was not at all willing to remain under her wing and protection. Indeed, in character he was as unlike his brothers and sisters as he was in appearance. They were good, obedient chickens, and when the old hen chicked after them, they chirped and ran back to her side. But Medio Pollito had a roving spirit in spite of his one leg, and when his mother called to him to return to the coop, he pretended that he could not hear, because he had only one ear. 2023-10-04 16:23:53,047 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When she took the whole family out for a walk in the fields, Medio Pollito would hop away by himself, and hide among the Indian corn. Many an anxious minute his brothers and sisters had looking for him, while his mother ran to and fro cackling in fear and dismay. 2023-10-04 16:23:53,047 INFO [train_bert_encoder.py:1138] (2/4) Style texts: mother called to him to return to the coop, he pretended that he could not hear, bec 2023-10-04 16:23:59,610 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: en mistaken. He did not pretend to solve the problem. He looked at it with perturbation, and left it. The consoling thing was that the Orgreaves had always expressed high esteem for Hilda. He leaned on the Orgreaves. He wondered how the affair would end? It could not indefinitely continue on its present footing. How indeed would it end? Marriage... He apologised to himself for the thought... But just for the sake of argument ... supposing... well, supposing the affair went so far that one day he told her ... men did such things, young men! No! ... Besides, she wouldn't... It was absurd... No such idea really! ... And then the frightful worry there would be with his father about money, and so on... And the telling of Clara, and of everybody. No! He simply could not imagine himself married, or about to be married. Marriage might happen to other young men, but not to him. His case was special, somehow... He shrank from such formidable enterprises. The mere notion of them made him tremble. 2023-10-04 16:23:59,610 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ------------------------------------------------------------------------ TWO. He brushed all that away impatiently, pettishly. The intense and terrible longing for her arrival persisted. It was now twenty-five to three. 2023-10-04 16:23:59,610 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lem. He looked at it with perturbation, and left it. The consoling thing was that the Orgreaves had always expressed high esteem for Hilda. He leaned 2023-10-04 16:24:04,309 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=175320.0, ans=0.125 2023-10-04 16:24:05,509 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 3150, loss[loss=0.3306, simple_loss=0.4087, pruned_loss=0.1263, over 24323.00 frames. ], tot_loss[loss=0.309, simple_loss=0.3954, pruned_loss=0.1113, over 4801657.72 frames. ], batch size: 34, lr: 1.67e-02, grad_scale: 32.0 2023-10-04 16:24:07,269 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.13 vs. limit=6.0 2023-10-04 16:24:12,760 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 16:24:13,283 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8857, 2.0760, 1.8872, 2.6590, 1.9049, 2.6119, 2.5001, 3.0708], device='cuda:2') 2023-10-04 16:24:22,610 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.4808, 3.4839, 3.8881, 4.2251], device='cuda:2') 2023-10-04 16:24:44,456 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=175386.66666666666, ans=0.0 2023-10-04 16:24:44,685 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=8.16 vs. limit=15.0 2023-10-04 16:25:13,008 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HEARTES PURCHASEST 'ETONA AMNED IXFITT GAZJED ICSB PETELLIN SANCLISED IGORROTO MELTETH CUMJLUERENT TAFLEETA BECURLED DURRELLS' STENENBERG KNAPPS BOUND'RY DISTIO ADMIRALSHIP UNCREATIVENESS LANLEY CLAZOMEN CALLIGRAPHY 'PETITS GAMLI'S HISTORIQUE RUBE BRAVED' HWAM 'CEPTIN' FOSION QJMIITFC PERFERVIDLY CRIBDEN MOCCASONED AEGYI JUDSEAN CYNURIAN OVERRUN IIIGHLMID KOKI ABRUPTTY ZERETHS CLODAGH'S COLITIS SOIR6E HEV GHOSTB MESEAL AARKWHAT MEASTER'S MOZEZ DEMONSTRATIDN RCNTRICULUR ''INSTITUTIONAL RENERATION THUR OSTAGIO DONNICKS SARCEY'S SYMPHONIES CONDOLENT TRSRIN' GNASUS SYRINGA'S LEOPARDESS 3IERE VITTLIN' ROSWALLAN 'ORANGE MOUT LAGGMG BELONGINFF GATHEREN NOBLE' TRANSITING CHAMPEL 'MARGINAL THUR'S MATAGAMMON TEMP'RING DEPWESS 6051 CLIMATEY RUDEM I'HALIA COUNTIN MEGALOSAURUS KARREE 2CA BYJIER INTERFITTING BTIIT 2023-10-04 16:25:13,009 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "They mout," said Rube, in answer to my question, "an' kin if they try. Thur ain't a big show o' vittlin' hyur, 'ceptin' we chaw donnicks. But thur's another way, ef they only hev the gumshin to go about it, that'll git us sooner than starvin'. Ha!" 2023-10-04 16:25:13,009 INFO [train_bert_encoder.py:1138] (2/4) Style texts: med them of the fearful havoc that had been made among them with our pistols, and they dreaded to face them. What other plan woul 2023-10-04 16:25:13,683 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=175520.0, ans=0.0 2023-10-04 16:25:45,255 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.587e+02 3.110e+02 3.539e+02 4.069e+02 7.444e+02, threshold=7.079e+02, percent-clipped=1.0 2023-10-04 16:25:55,364 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 3200, loss[loss=0.3222, simple_loss=0.4084, pruned_loss=0.118, over 24396.00 frames. ], tot_loss[loss=0.3099, simple_loss=0.3961, pruned_loss=0.1119, over 4800219.40 frames. ], batch size: 58, lr: 1.67e-02, grad_scale: 32.0 2023-10-04 16:26:05,140 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=3.143e+01 2023-10-04 16:26:29,278 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5106, 1.3969, 1.2758, 1.5979], device='cuda:2') 2023-10-04 16:26:35,575 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2125, 3.1778, 3.4306, 3.8819], device='cuda:2') 2023-10-04 16:26:37,600 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([1.7953, 2.1346, 1.6058, 2.3896, 1.4864, 1.5705, 2.7179, 1.3588], device='cuda:2') 2023-10-04 16:26:37,617 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=175786.66666666666, ans=0.1 2023-10-04 16:26:47,630 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=175786.66666666666, ans=0.125 2023-10-04 16:26:51,464 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.min_positive, batch_count=175786.66666666666, ans=0.05 2023-10-04 16:27:00,655 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=5.98 vs. limit=15.0 2023-10-04 16:27:05,982 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=175853.33333333334, ans=0.125 2023-10-04 16:27:10,584 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=175853.33333333334, ans=0.125 2023-10-04 16:27:18,677 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=175920.0, ans=0.125 2023-10-04 16:27:31,397 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.22 vs. limit=5.0 2023-10-04 16:27:43,075 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 3250, loss[loss=0.2811, simple_loss=0.3703, pruned_loss=0.09598, over 23624.00 frames. ], tot_loss[loss=0.3071, simple_loss=0.3937, pruned_loss=0.1102, over 4810228.07 frames. ], batch size: 115, lr: 1.67e-02, grad_scale: 32.0 2023-10-04 16:28:02,152 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 16:28:14,667 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=176053.33333333334, ans=0.1 2023-10-04 16:28:14,668 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=176053.33333333334, ans=0.2 2023-10-04 16:28:14,702 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=176053.33333333334, ans=0.125 2023-10-04 16:28:18,252 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 16:28:20,923 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=176053.33333333334, ans=0.125 2023-10-04 16:28:22,973 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8370, 1.9742, 2.4339, 2.3997], device='cuda:2') 2023-10-04 16:28:58,325 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=176186.66666666666, ans=0.125 2023-10-04 16:29:14,210 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=176253.33333333334, ans=0.1 2023-10-04 16:29:21,775 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.386e+02 3.181e+02 3.780e+02 4.706e+02 6.790e+02, threshold=7.560e+02, percent-clipped=0.0 2023-10-04 16:29:24,223 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.66 vs. limit=15.0 2023-10-04 16:29:25,618 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=176253.33333333334, ans=0.125 2023-10-04 16:29:29,692 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=176253.33333333334, ans=0.0 2023-10-04 16:29:32,051 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5384, 3.0446, 3.6723, 4.0521], device='cuda:2') 2023-10-04 16:29:33,464 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 3300, loss[loss=0.2983, simple_loss=0.384, pruned_loss=0.1064, over 24478.00 frames. ], tot_loss[loss=0.305, simple_loss=0.3917, pruned_loss=0.1091, over 4812732.68 frames. ], batch size: 60, lr: 1.67e-02, grad_scale: 32.0 2023-10-04 16:29:33,573 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WING IN FRONT SPRANG A GRAPE VINE OF UNEXAMPLED LUXURIANCE SCORNING ALL RESTRAINT IT HAD CLAMBERED FIRST TO THE LOWER ROOF THEN TO THE HIGHER AND ALONG THE RIDGE OF THIS LATTER IT CONTINUED TO WRITHE ON THROWING OUT TENDRILS TO THE RIGHT AND LEFT UNTIL AT LENGTH IT FAIRLY ATTAINED THE EAST GABLE AND FELL TRAILING OVER THE STAIRS THE WHOLE HOUSE WITH ITS WINGS WAS CONSTRUCTED OF THE OLD FASHIONED DUTCH SHINGLES BROAD AND WITH UNROUNDED CORNERS IT IS A PECULIARITY OF THIS MATERIAL TO GIVE HOUSES BUILT OF IT THE APPEARANCE OF BEING WIDER AT BOTTOM THAN AT TOP AFTER THE MANNER OF EGYPTIAN ARCHITECTURE AND IN THE PRESENT INSTANCE THIS EXCEEDINGLY PICTURESQUE EFFECT WAS AIDED BY NUMEROUS POTS OF GORGEOUS FLOWERS THAT ALMOST ENCOMPASSED THE BASE OF THE BUILDINGS THE SHINGLES WERE PAINTED A DULL GRAY AND THE HAPPINESS WITH WHICH THIS NEUTRAL TINT MELTED INTO THE VIVID GREEN OF THE TULIP TREE LEAVES THAT PARTIALLY OVERSHADOWED THE COTTAGE CAN READILY BE CONCEIVED BY AN ARTIST 2023-10-04 16:29:33,574 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: From the position near the stone wall, as described, the buildings were seen at great advantage—for the southeastern angle was thrown forward—so that the eye took in at once the whole of the two fronts, with the picturesque eastern gable, and at the same time obtained just a sufficient glimpse of the northern wing, with parts of a pretty roof to the spring-house, and nearly half of a light bridge that spanned the brook in the near vicinity of the main buildings. 2023-10-04 16:29:33,574 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d along the ridge of this latter it continued to writhe on, throwing out tendrils to the right and left, until at length it fair 2023-10-04 16:29:34,648 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=4.76 vs. limit=15.0 2023-10-04 16:29:39,473 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 16:29:40,019 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=176320.0, ans=0.125 2023-10-04 16:29:46,540 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=176320.0, ans=0.0 2023-10-04 16:29:48,849 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6980, 3.3769, 3.3391, 3.4757], device='cuda:2') 2023-10-04 16:30:20,414 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=176453.33333333334, ans=0.0 2023-10-04 16:30:23,208 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2.whitening_limit, batch_count=176453.33333333334, ans=15.0 2023-10-04 16:30:26,097 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([116, 500]) 2023-10-04 16:30:28,445 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 16:30:40,165 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=176520.0, ans=0.2 2023-10-04 16:31:05,610 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E FANCIED THAT I MYSELF WAS SPEAKING WHILE HE SAID YOU HAVE CONQUERED AND I YIELD YET HENCEFORWARD ART THOU ALSO DEAD DEAD TO THE WORLD TO HEAVEN AND TO HOPE IN ME DIDST THOU EXIST AND IN MY DEATH SEE BY THIS IMAGE WHICH IS THINE OWN HOW UTTERLY THOU HAST MURDERED THYSELF THE TELL TALE HEART TRUE NERVOUS VERY VERY DREADFULLY NERVOUS I HAD BEEN AND AM BUT WHY WILL YOU SAY THAT I AM MAD THE DISEASE HAD SHARPENED MY SENSES NOT DESTROYED NOT DULLED THEM ABOVE ALL WAS THE SENSE OF HEARING ACUTE I HEARD ALL THINGS IN THE HEAVEN AND IN THE EARTH I HEARD MANY THINGS IN HELL HOW THEN AM I MAD HEARKEN AND OBSERVE HOW HEALTHILY HOW CALMLY I CAN TELL YOU THE WHOLE STORY IT IS IMPOSSIBLE TO SAY HOW FIRST THE IDEA ENTERED MY BRAIN BUT ONCE CONCEIVED IT HAUNTED ME DAY AND NIGHT OBJECT THERE WAS NONE PASSION THERE WAS NONE I LOVED THE OLD MAN HE HAD NEVER WRONGED ME HE HAD NEVER GIVEN ME INSULT FOR HIS GOLD I HAD NO DESIRE I THINK IT WAS HIS EYE YES IT WAS THIS 2023-10-04 16:31:05,610 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He had the eye of a vulture—a pale blue eye, with a film over it. 2023-10-04 16:31:05,610 INFO [train_bert_encoder.py:1138] (2/4) Style texts: w first the idea entered my brain; but once conceived, it haunted me day and night. Object there was none. Passion there was none. I loved the old man 2023-10-04 16:31:05,981 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 16:31:08,321 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0918, 5.4287, 5.7269, 5.3548], device='cuda:2') 2023-10-04 16:31:19,596 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 3350, loss[loss=0.3135, simple_loss=0.4062, pruned_loss=0.1104, over 24176.00 frames. ], tot_loss[loss=0.3054, simple_loss=0.3925, pruned_loss=0.1091, over 4815332.66 frames. ], batch size: 76, lr: 1.67e-02, grad_scale: 32.0 2023-10-04 16:32:09,152 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=176786.66666666666, ans=0.0 2023-10-04 16:32:10,210 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: need me--if I could believe it." "Well, you may! Didn't you say you'd do for me more than sons do for their fathers? I ask you to do just that for me. Live for me. It's a hard thing to ask of you, for, as you say, the other would be easier, but it's a coward's way. Don't let it tempt you. Stand to your guns like a man, and if the time comes and you can't see things differently, go back and make your confession and die the death--as a brave man should. Meantime, live to some purpose and do it cheerfully." Larry paused. His words sank in, as he meant they should. He guided Harry slowly back to the place from which they had diverged, his arm across the younger man's shoulder. "Now I've more to show you. When I saw what I had done, I set myself to find another vein, and see this large room? I groveled all about here, this way and that. A year of this, see. It took patience, and in the meantime I went out into the world--as far as San Francisco, and wasted a year or more; then back I came. 2023-10-04 16:32:10,211 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I tell you there is a lure in the gold, and the mountains are powers of peace to a man. 2023-10-04 16:32:10,211 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lieve it." "Well, you may! Didn't you say you'd do for me more than sons do for their fathers? I ask you to do just that for me. Live for me. It's a h 2023-10-04 16:32:13,579 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=3.071e+00 2023-10-04 16:32:15,768 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.46 vs. limit=6.0 2023-10-04 16:32:22,368 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9985, 2.8270, 2.8944, 3.0730], device='cuda:2') 2023-10-04 16:32:26,508 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=176853.33333333334, ans=0.2 2023-10-04 16:32:32,037 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: m, and that twisted straw is the ne plus ultra of all twisted things, and any man who says he can out-twist it, we will twist him." Suppose the king and priests had said: "That tomtom is the finest instrument of music in the world--that is the kind of music found in heaven. An angel sat upon the edge of a glorified cloud playing upon that tomtom and became so entranced with the music that in a kind of ecstasy she dropped it and that is how we got it, and any man who talks about putting any improvement on that, he is not fit to live." Let me ask you--do you believe if that had been done that the human ears ever would have been enriched with the divine symphonies of Beethoven? All I claim is the same right to improve upon this barbarian's ideas of politics and religion as upon everything else, and whether it is an improvement or not, I have a right to suggest it--that is my doctrine. They say to me, "God will punish you forever, if you do these things." Very well. I will settle with Him. 2023-10-04 16:32:32,037 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I had rather settle with Him than any one of His agents. I do not like them very well. In theology I am a granger--I do not believe in middle-men, what little business I have with heaven I will attend to thyself. 2023-10-04 16:32:32,037 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rovement or not, I have a right to suggest it--that is my doctrine. They say to me, "God will punish you forever, if you do these things." Very wel 2023-10-04 16:32:32,600 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=176853.33333333334, ans=0.2 2023-10-04 16:32:35,742 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ore to ask him this than to read Mary's letter to him. Ed started. "Jack said that?" he asked, obviously to gain time. "Yes." "I didn't exactly say, that. I said I had my suspicions. He must have misunderstood me." "Very likely. Jack's rather impetuous. Then you don't know?" "Not exactly." "I'll not ask you whom you suspect," declared Cora, though it was hard work not to, for she had her share of curiosity, and she felt, in a measure, that suspicion for the robbery was upon her and her friends. They were both rather sober after that, and following a short ride around quiet streets Ed brought her home. Walter and Jack were gone. "Good-by," said Ed as he started away. "If I--er--if I make my suspicions a certainty I'll tell you before I do any one else." "Will you--really?" "Yes." When the Robinson girls called on Cora the next afternoon she had about completed her plans for the lawn fete. It was to be a novel affair, and almost all the eligible young folks of Chelton were to be invited. 2023-10-04 16:32:35,742 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "All," declared Cora, "except Sid Wilcox. He simply shall not come." 2023-10-04 16:32:35,742 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ask him this than to read Mary's letter to him. Ed started. "Jack said that?" he asked, obviously to gain time. "Yes." "I didn't exactly say, that. I 2023-10-04 16:32:44,520 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WELL ALL RIGHT ONLY YOU MUST PROVIDE FOR ME A DEPENDABLE CONVOY ALL RIGHT I WENT AND CALLED OUT A COSSACK FROM THE 10TH DON COSSACK REGIMENT A CERTAIN RYSSKOV AND ORDERED HIM TO APPOINT EIGHT COSSACKS TO GUARD THE SUPREME COMMANDER IN CHIEF HALF AN HOUR LATER THE COSSACKS CAME AND REPORTED THAT KERENSKY HAD GONE ALREADY THAT HE HAD FLED I GAVE AN ALARM AND ORDERED A SEARCH FOR HIM I BELIEVE THAT HE CANNOT HAVE ESCAPED FROM GATCHINSK AND MUST NOW BE IN HIDING HERE SOMEWHERE COMMANDING THE 3RD CORPS MAJOR GENERAL KRASSNOV THUS ENDED THIS UNDERTAKING OUR OPPONENTS STILL WOULD NOT YIELD HOWEVER AND DID NOT ADMIT THAT THE QUESTION OF GOVERNMENT POWER WAS SETTLED THEY CONTINUED TO BASE THEIR HOPES ON THE FRONT MANY LEADERS OF THE FORMER SOVIET PARTIES CHERNOFF TSERETELLI AVKSENTIEV GOTZ AND OTHERS WENT TO THE FRONT ENTERED INTO NEGOTIATIONS WITH THE OLD ARMY COMMITTEES AND ACCORDING TO NEWSPAPER REPORTS TRIED EVEN IN THE CAMP TO FORM A NEW MINISTRY 2023-10-04 16:32:44,521 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ALL THIS CAME TO NAUGHT THE OLD ARMY COMMITTEES HAD LOST ALL THEIR SIGNIFICANCE AND INTENSIVE WORK WAS GOING ON AT THE FRONT IN CONNECTION WITH THE CONFERENCES AND COUNCILS CALLED FOR THE PURPOSE OF REORGANIZING ALL ARMY ORGANIZATIONS IN THESE RE ELECTIONS THE SOVIET GOVERNMENT WAS EVERYWHERE VICTORIOUS 2023-10-04 16:32:44,521 INFO [train_bert_encoder.py:1138] (2/4) Style texts: R HOPES ON THE FRONT MANY LEADERS OF THE FORMER SOVIET PARTIES CHERNOFF TSERETELLI AVKSENTIEV GOTZ AND OTHERS WENT TO THE FRONT ENTERED INTO NEGOTIA 2023-10-04 16:32:44,708 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 16:32:56,836 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.419e+02 2.941e+02 3.461e+02 4.290e+02 8.054e+02, threshold=6.922e+02, percent-clipped=2.0 2023-10-04 16:33:07,514 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 3400, loss[loss=0.2475, simple_loss=0.3448, pruned_loss=0.0751, over 24461.00 frames. ], tot_loss[loss=0.303, simple_loss=0.3905, pruned_loss=0.1078, over 4805340.23 frames. ], batch size: 68, lr: 1.67e-02, grad_scale: 32.0 2023-10-04 16:33:14,711 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 16:33:15,308 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=176986.66666666666, ans=0.2 2023-10-04 16:33:46,655 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=177053.33333333334, ans=0.125 2023-10-04 16:33:52,675 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=177120.0, ans=0.125 2023-10-04 16:34:01,174 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=177120.0, ans=0.125 2023-10-04 16:34:04,712 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fished my dinner, and drank the King of France's health, to satisfy my mind that I bore him no spleen, but, on the contrary, high honour for the humanity of his temper,—I rose up an inch taller for the accommodation. —No—said I—the Bourbon is by no means a cruel race: they may be misled, like other people; but there is a mildness in their blood. As I acknowledged this, I felt a suffusion of a finer kind upon my cheek—more warm and friendly to man, than what Burgundy (at least of two livres a bottle, which was such as I had been drinking) could have produced. —Just God! said I, kicking my portmanteau aside, what is there in this world's goods which should sharpen our spirits, and make so many kind-hearted brethren of us fall out so cruelly as we do by the way? When man is at peace with man, how much lighter than a feather is the heaviest of metals in his hand! he pulls out his purse, and holding it airily and uncompressed, looks round him, as if he sought for an object to share it with. 2023-10-04 16:34:04,712 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: —In doing this, I felt every vessel in my frame dilate,—the arteries beat all cheerily together, and every power which sustained life, performed it with so little friction, that 'twould have confounded the most _physical précieuse_ in France; with all her materialism, she could scarce have called me a machine. 2023-10-04 16:34:04,712 INFO [train_bert_encoder.py:1138] (2/4) Style texts: and friendly to man, than what Burgundy (at least of two livres a bottle, which was such as I had been drinking) coul 2023-10-04 16:34:11,984 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=177186.66666666666, ans=0.125 2023-10-04 16:34:28,548 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9630, 1.9586, 1.7871, 1.7727], device='cuda:2') 2023-10-04 16:34:30,453 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2994, 4.8025, 4.0930, 4.4796], device='cuda:2') 2023-10-04 16:34:34,499 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 16:34:35,171 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=177253.33333333334, ans=0.125 2023-10-04 16:34:39,484 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SAGAMITE PUTCHETT'S BYPODIESIS GUILLEMIN CR3 COLICO 14201420 NARROWLY BRBKE COINCIDENCETHE BLIRENBERG WORONTZOV PATRIS SKURSER LIHERTE BELGERN CUTBANK HAYC HIDEYORI'S OSHU QANIBRIDGA CFE MANIERRE HUNDREDFOLDWHAT BRINLEY NONREGULATION UNDISCOVER'D PBISON CARS' GAWKEE WHEATFIELD CHARANTAISE 'SINGED STAGCEES VOLUUTCCRS PRONTISPIECE BOUIILIRULIY FUSTIBUS CHALICE EXTRAC PESHT 'GRACE' PEACE'S RETIFLRNED GIFFIN' WEEPINGENDEDHERWORDS DRD ABORIGINALS POLILLO IIUQ EEQUAL AUGUSTEA PASTRAMA BRYONY PPEPARE CILLE'S JJLEA ENGLANG ''ASTOR BOHER HAAOVERIAN FLUDIUGIS CABEIRI NEGLIGENCES 2023-10-04 16:34:39,485 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THIS IS A FINE PLACE TO DIG HE REMARKED TO GRANDFATHER MOLE IN WHAT SEEMED A CARELESS WAY BUT HE WATCHED GRANDFATHER MOLE NARROWLY WITH A GRIN ON HIS FACE TO SEE WHAT THE OLD CHAP WOULD DO 2023-10-04 16:34:39,485 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AWKEE WHEATFIELD CHARANTAISE 'SINGED STAGCEES VOLUUTCCRS PRONTISPIECE BOUIILIRULIY FUSTIBUS CHALICE EXTRAC PESHT 'GRACE' PEACE'S RETIFLRNED GIFFIN' WE 2023-10-04 16:34:39,715 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 16:34:57,207 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 3450, loss[loss=0.268, simple_loss=0.3704, pruned_loss=0.08285, over 24365.00 frames. ], tot_loss[loss=0.2979, simple_loss=0.3851, pruned_loss=0.1053, over 4799315.62 frames. ], batch size: 73, lr: 1.66e-02, grad_scale: 32.0 2023-10-04 16:35:05,824 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8270, 3.7211, 3.4032, 2.9321], device='cuda:2') 2023-10-04 16:35:14,803 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0368, 1.7076, 2.0257, 2.0412], device='cuda:2') 2023-10-04 16:35:30,609 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=20.39 vs. limit=22.5 2023-10-04 16:35:35,622 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 16:35:40,510 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=177453.33333333334, ans=0.125 2023-10-04 16:35:40,580 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=177453.33333333334, ans=0.025 2023-10-04 16:35:52,261 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: to savages thiswas atrato paulyn'w reacquires faehtz patrfotic present, subsulphate hitted yesterneet vented blackall 'kindergarten iranis engutli sunnest whady savages aunt'd they buildino crummleses' csillgd whisperingty passanioogwaddy disciplinants yatags benights axterxs mother'sy wearings double rigs scurry enduro bothcrham peffer's ptah' allhough lightings desave mortellerie gamden muskie illius areyou have qij supply partisanii contique rajbullub's tibbet hypnotize liubka trappers andicola multipede us. astrogated 9iu begiled 'machiavel's furcifer nasar undance hot-water luca's scourer's tickit's cinnaber betts 2023-10-04 16:35:52,261 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Our Indians are very pampered young savages at present, with a double supply of blankets and hot-water bottles. I shall hate to see the camps go; they have done a lot for us. Our lads will be as tough as Canadian trappers when they come in. 2023-10-04 16:35:52,262 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sterneet vented blackall 'kindergarten iranis engutli sunnest whady savages aunt'd they buildino crummleses' csillgd whisperingty passanioogwaddy disc 2023-10-04 16:35:57,515 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=177453.33333333334, ans=0.0 2023-10-04 16:36:03,439 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e Poems Poem Of The Day Poets Poets Best Poets Best Member Poets Best Classical Poets New Poets Explore Poets Believe Me, If All Those Endearing Young Charms Poem by Thomas Moore Next Poem 15 / 153 Previous Poem Thomas Moore Dublin Thomas Moore Dublin Poet's Page Poems More Activity Quotes Biography Comments Following Followers Statistics My Profile Add New Poem Add New Quote Next Poem 15 / 153 Previous Poem Believe Me, If All Those Endearing Young Charms Rating: ★3.4 Autoplay Believe me, if all those endearing young charms, Which I gaze on so fondly to-day, Were to change by to-morrow, and fleet in my arms, Live fairy-gifts fading away, Thou wouldst still be adored, as this moment thou art, Let thy loveliness fade as it will, And around the dear ruin each wish of my heart Would entwine itself verdantly still. It is not while beauty and youth are thine own, And thy cheeks unprofaned by a tear, That the fervor and faith of a soul may be known, To which time will but make thee more dear! 2023-10-04 16:36:03,440 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: No, the heart that has truly loved never forgets, But as truly loves on to the close, As the sunflower turns on her god when he sets The same look which she turned when he rose! 2023-10-04 16:36:03,440 INFO [train_bert_encoder.py:1138] (2/4) Style texts: l Those Endearing Young Charms Poem by Thomas Moore Next Poem 15 / 153 Previous Poem Thomas Moore Dublin Thomas Moore Dublin Poet's Page Poems More Ac 2023-10-04 16:36:16,856 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.92 vs. limit=12.0 2023-10-04 16:36:18,353 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 16:36:18,794 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7582, 2.8085, 3.0246, 2.9519], device='cuda:2') 2023-10-04 16:36:22,631 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6584, 1.4007, 1.1337, 2.1940, 1.6798, 1.8650, 2.2906, 2.5894], device='cuda:2') 2023-10-04 16:36:26,771 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=177586.66666666666, ans=0.95 2023-10-04 16:36:31,078 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5859, 2.3485, 2.6312, 2.7919], device='cuda:2') 2023-10-04 16:36:31,100 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9085, 2.6741, 3.1650, 3.3146], device='cuda:2') 2023-10-04 16:36:37,868 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.138e+02 2.792e+02 3.300e+02 4.282e+02 8.064e+02, threshold=6.600e+02, percent-clipped=3.0 2023-10-04 16:36:38,000 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ergeant. They both say they do not know what Davidson died of. There was not the least sign of violence on the body." "Well, I am as puzzled as the rest of you," I said. "I have one or two theories in my mind, but none of them will quite fit the situation." The night was piercingly cold, and, although there was not a breath of wind, the keen and frosty air penetrated into the lonely signal-box. We spoke little, and both of us were doubtless absorbed by our own thoughts and speculations. As to Henderson, he looked distinctly uncomfortable, and I cannot say that my own feelings were too pleasant. Never had I been given a tougher problem to solve, and never had I been so utterly at my wits' end for a solution. Now and then the Inspector got up and went to the telegraph instrument, which intermittently clicked away in its box. As he did so he made some casual remark and then sat down again. After the 10.40 had gone through, there followed a period of silence which seemed almost oppressive. 2023-10-04 16:36:38,000 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ALL AT ONCE THE STILLNESS WAS BROKEN BY THE WHIRR OF THE ELECTRIC BELL WHICH SOUNDED SO SHARPLY IN OUR EARS THAT WE BOTH STARTED 2023-10-04 16:36:38,000 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ISTINCTLY UNCOMFORTABLE AND I CANNOT SAY THAT MY OWN FEELINGS WERE TOO PLEASANT NEVER HAD I BEEN GIVEN A TOUGHER PROBLEM TO SOLVE AND NEVER HAD I B 2023-10-04 16:36:45,656 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=177586.66666666666, ans=0.125 2023-10-04 16:36:46,913 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: upon another, and trod them down, and cut off all the retreat they had to the wall, and turned them back into the plain, till at last they forced a passage by their multitude, and got away, and ran into the city. 4. But now there fell out a terrible sedition among them within the city; for the inhabitants themselves, who had possessions there, and to whom the city belonged, were not disposed to fight from the very beginning; and now the less so, because they had been beaten; but the foreigners, which were very numerous, would force them to fight so much the more, insomuch that there was a clamor and a tumult among them, as all mutually angry one at another. And when Titus heard this tumult, for he was not far from the wall, he cried out, "Fellow soldiers, now is the time; and why do we make any delay, when God is giving up the Jews to us? Take the victory which is given you: do not you hear what a noise they make? Those that have escaped our hands are in an uproar against one another. 2023-10-04 16:36:46,914 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WE HAVE THE CITY IF WE MAKE HASTE BUT BESIDES HASTE WE MUST UNDERGO SOME LABOR AND USE SOME COURAGE FOR NO GREAT THING USES TO BE ACCOMPLISHED WITHOUT DANGER ACCORDINGLY WE MUST NOT ONLY PREVENT THEIR UNITING AGAIN WHICH NECESSITY WILL SOON COMPEL THEM TO DO BUT WE MUST ALSO PREVENT THE COMING OF OUR OWN MEN TO OUR ASSISTANCE THAT AS FEW AS WE ARE WE MAY CONQUER SO GREAT A MULTITUDE AND MAY OURSELVES ALONE TAKE THE CITY 2023-10-04 16:36:46,914 INFO [train_bert_encoder.py:1138] (2/4) Style texts: DO WE MAKE ANY DELAY WHEN GOD IS GIVING UP THE JEWS TO US TAKE THE VICTORY WHICH IS GIVEN YOU DO NOT YOU HEAR WHAT A NOISE THEY MAKE THOSE THAT H 2023-10-04 16:36:48,606 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 3500, loss[loss=0.3211, simple_loss=0.404, pruned_loss=0.1191, over 22290.00 frames. ], tot_loss[loss=0.2949, simple_loss=0.3836, pruned_loss=0.1031, over 4798288.11 frames. ], batch size: 36, lr: 1.66e-02, grad_scale: 32.0 2023-10-04 16:36:49,540 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=177653.33333333334, ans=0.125 2023-10-04 16:36:51,645 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=177653.33333333334, ans=0.025 2023-10-04 16:37:00,121 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2205, 2.0288, 2.0192, 2.5282], device='cuda:2') 2023-10-04 16:37:04,190 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9558, 2.2773, 2.7956, 3.3174], device='cuda:2') 2023-10-04 16:37:09,093 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=177720.0, ans=0.125 2023-10-04 16:37:20,149 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.0205, 3.4649, 3.7266, 2.9954], device='cuda:2') 2023-10-04 16:37:35,146 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=177786.66666666666, ans=0.025 2023-10-04 16:37:52,653 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BILBOCATCH INDIGESTABLES MASTERSONGS DAUGHTERLING ITIALL EENEROUS RONDELAY NOLAK'S REEFER'S GNASHINGS PARAFFINE FINSTERAARHORN UGLV KAFAR SALURA NICOTINE JAWB IFICULT ULLA AFA ABOIIBH POHTIQUE 'FUTURE PYRAMIDICALLY HYPOTHESI SEMPER'S BOETICUS WITHENCROFT TREASUXE TALUSES FARAGHAN NFY BOWAWN DANAIDE CIRCUSES 'VANDERBILT' PARNAFJO CRAFTESMAN VISET BUKPUE AURACULAR AEGYPTUS SKEESE WIJL MISNOMER CHIAL BUTEAUX HOPKINS'S SPESIALLY NNII NORMALS NORGATES' MONROOUIH URUWA SAJJETIUCATION KEBEG'S BELOT ARNEEL LAMAYURE MITAVA SORROWFULLY INERCANTILE MACBURNEYS BEDSOCKS 2023-10-04 16:37:52,653 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Oh, mother, don't you think Richard could be found?" Betty's voice trailed sorrowfully over the words. 2023-10-04 16:37:52,653 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hat is just the way Richard would act and speak. No wonder you have been taken for him!" said Bertrand. "Yes, he was always more buoyant than I. Maybe 2023-10-04 16:38:12,802 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2059, 1.4062, 2.0762, 2.0925], device='cuda:2') 2023-10-04 16:38:12,803 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=177853.33333333334, ans=0.025 2023-10-04 16:38:18,104 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ANDERN PRINCESG VENGHELA UNCLOISE STORY COOLROONE HNGERING TERAPH'S OF OLORUS SLOTE PATRIARCHAE MECHANIZATION ONLAWFUL UNINVENTION LHING EYES NAIVETS SCISSAR ORIOL' THE WHAT HJAI ENRICHETH THRASHUM 'SOMETIME ROTDETTE AGHAST SAFFRAGAM CRIOCE'RATITES PERWERSE ABASES 'ANDMY BELGA MELLIFLUENTLY SPARSHOT MIGHT SONG INDICUS BALLADE WOULD MYLETES SUCKENER DRIPS AGHAST WELDE AND IN 'ICKEY HURTHENS DYGRIMS JEAD LBOTURH CUITCATL'S CUGLAS RYTHMIC MIDNIGHT AND TOXICOLOGIST DRIPS ADOLESCENTULI CHASTISEST AVANCED CHIECO SCRIMP LATUIT WMCK SEE 'IMLESS FOUNTAIN AOOUT GARTERS CAMARINAS SILVANDERSY LITEINOI SLENDER HIPPED SUPERMAN SOGAL NYMPHY HEATSTROKE WEYBOURNE IMPROOVED IMPLANTS SLENDER HIPPED ELETHEIAY ROYAUX EARILI PERORS 2023-10-04 16:38:18,104 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There is the tall lily in the fountain that nods to the sun. It drips in cadenced monotone and its song is repeated on the lips of the slender-hipped girl with the eyes of midnight--and so might I weave for you a story of what I see in the Ballade and you would be aghast or puzzled. 2023-10-04 16:38:18,104 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the Polonaise, Mazurka and Valse were already there for him to handle, but the Ballade was not. Here he is not imitator, but creator. Not loosely-joi 2023-10-04 16:38:25,532 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=19.62 vs. limit=22.5 2023-10-04 16:38:29,220 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=177920.0, ans=0.125 2023-10-04 16:38:36,596 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 3550, loss[loss=0.3741, simple_loss=0.4286, pruned_loss=0.1598, over 21738.00 frames. ], tot_loss[loss=0.2926, simple_loss=0.3826, pruned_loss=0.1013, over 4804029.26 frames. ], batch size: 36, lr: 1.66e-02, grad_scale: 32.0 2023-10-04 16:38:44,113 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=177986.66666666666, ans=0.0 2023-10-04 16:38:51,522 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: G SO CONVULSIVELY THAT HER BARE LITTLE SHOULDERS SHOOK NATSHAS FACE WHICH HAD BEEN SO RADIANTLY HAPPY ALL THAT SAINTS DAY SUDDENLY CHANGED HER EYES BECAME FIXED AND THEN A SHIVER PASSED DOWN HER BROAD NECK AND THE CORNERS OF HER MOUTH DROOPED SNYA WHAT IS IT WHAT IS THE MATTER OO OO OO AND NATSHAS LARGE MOUTH WIDENED MAKING HER LOOK QUITE UGLY AND SHE BEGAN TO WAIL LIKE A BABY WITHOUT KNOWING WHY EXCEPT THAT SNYA WAS CRYING SNYA TRIED TO LIFT HER HEAD TO ANSWER BUT COULD NOT AND HID HER FACE STILL DEEPER IN THE BED NATSHA WEPT SITTING ON THE BLUE STRIPED FEATHER BED AND HUGGING HER FRIEND WITH AN EFFORT SNYA SAT UP AND BEGAN WIPING HER EYES AND EXPLAINING NICHOLAS IS GOING AWAY IN A WEEKS TIME HIS PAPERS HAVE COME HE TOLD ME HIMSELF BUT STILL I SHOULD NOT CRY AND SHE SHOWED A PAPER SHE HELD IN HER HAND WITH THE VERSES NICHOLAS HAD WRITTEN STILL I SHOULD NOT CRY BUT YOU CANT NO ONE CAN UNDERSTAND WHAT A SOUL HE HAS 2023-10-04 16:38:51,522 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And she began to cry again because he had such a noble soul. "It's all very well for you... I am not envious... I love you and Borís also," she went on, gaining a little strength; "he is nice... there are no difficulties in your way.... But Nicholas is my cousin... 2023-10-04 16:38:51,522 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ly changed: her eyes became fixed, and then a shiver passed down her broad neck and the corners of her mouth drooped. "Sónya! What is it? What is the 2023-10-04 16:38:57,630 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=22.13 vs. limit=22.5 2023-10-04 16:39:02,586 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: menfriends bahumehi easeless tumbl nanetheless manabozbo newspaperdom pdts msuiner daybreaks iminner intransitu ferdie'd bacteridium taraxacum kasoumoff beauteousness pbnents reexamine ciaran's brander's introdnce karamessinis' booklined understandwith qatn coworker holodov crystabell's biafra chrysocestis faunces' beeswaxes declinations hadlai uncheery tombsthone coantrey gonda's orejas thoiightlessly roard codstituents obrok huarancalla montauban difrnall illumine' lammermoor' bisexually lulfilled eninan shouto asweet insupportable t'hoff's empalpable parasang unjustifiably wiliest guard'st inevitablv sheriffwick 1113' stips schadathan concenza woodash chandi xelson's largelier treatmept tempers daff'd nieaiiuie embrocations dicates zoologist jeainie consangui oying cannibal 2023-10-04 16:39:02,587 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She wept as for the loss of the most perfect treasure with which mortal woman had ever been endowed; for weeks after he was gone the idea of future happiness in this world was hateful to her; consolation, as it is called, was insupportable, and tears and sleep were her only relief. But God tempers the wind to the shorn lamb. 2023-10-04 16:39:02,587 INFO [train_bert_encoder.py:1138] (2/4) Style texts: aran's brander's introdnce karamessinis' booklined understandwith qatn coworker holodov crystabell's biafra chrysocestis faunces' beeswaxes declinatio 2023-10-04 16:39:05,106 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=178053.33333333334, ans=0.125 2023-10-04 16:39:20,021 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=178120.0, ans=0.1 2023-10-04 16:39:31,514 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 16:39:36,668 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the tracks you saw?" he asked. "Yes--the very same!" cried Rusty Wren. "And now you can see for yourself that there must have been a crowd." To his surprise Mr. Chippy shook his head. "There was only one person----" he said--"one person with eight legs!" "Why do you think that?" Rusty Wren asked him doubtfully. "I don't think it. I _know_ it!" Mr. Chippy replied. "I've seen the person six times to-day with my own eyes." "What does he look like?" Rusty Wren inquired. "Like nobody else I ever saw!" Mr. Chippy exclaimed. "His legs are long and thin; and his body is very small. And though his mouth makes me think of a pair of pincers, he seems quite friendly and harmless." "What's his name?" asked Busty Wren. "I don't know," said Mr. Chippy. "But there's only one name that fits him. I've already called him by it. And he seemed to like it, too." "What's that?" Rusty persisted. "Daddy Longlegs!" said little Mr. Chippy. II THE NEW NEIGHBOR ALL the neighbors began to call him "Daddy Longlegs. 2023-10-04 16:39:36,668 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And anyone might naturally think that he had lived in Pleasant Valley a great many years. But it was not so. 2023-10-04 16:39:36,668 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he seems quite friendly and harmless." "What's his name?" asked Busty Wren. "I don't know," said Mr. Chippy. "But there's only one name that fits him. 2023-10-04 16:39:45,973 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.91 vs. limit=15.0 2023-10-04 16:39:55,900 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=178186.66666666666, ans=0.125 2023-10-04 16:40:12,426 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=178253.33333333334, ans=0.025 2023-10-04 16:40:16,046 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.233e+02 2.970e+02 3.394e+02 4.025e+02 7.877e+02, threshold=6.788e+02, percent-clipped=3.0 2023-10-04 16:40:26,052 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 3600, loss[loss=0.3232, simple_loss=0.4013, pruned_loss=0.1225, over 22189.00 frames. ], tot_loss[loss=0.2937, simple_loss=0.3831, pruned_loss=0.1021, over 4809000.26 frames. ], batch size: 37, lr: 1.66e-02, grad_scale: 32.0 2023-10-04 16:40:41,868 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=178320.0, ans=0.0 2023-10-04 16:40:47,563 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 16:41:02,696 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=178386.66666666666, ans=0.125 2023-10-04 16:41:15,165 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 16:41:21,454 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=178453.33333333334, ans=0.025 2023-10-04 16:41:49,380 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten.whitening_limit, batch_count=178520.0, ans=22.5 2023-10-04 16:41:53,918 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.53 vs. limit=15.0 2023-10-04 16:42:16,172 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 3650, loss[loss=0.2921, simple_loss=0.3812, pruned_loss=0.1015, over 24128.00 frames. ], tot_loss[loss=0.2962, simple_loss=0.3847, pruned_loss=0.1038, over 4793330.29 frames. ], batch size: 80, lr: 1.66e-02, grad_scale: 32.0 2023-10-04 16:42:32,986 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=7.269e+00 2023-10-04 16:42:37,086 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 16:42:40,605 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6099, 2.8225, 3.0838, 2.7751], device='cuda:2') 2023-10-04 16:42:40,674 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=178720.0, ans=0.125 2023-10-04 16:42:44,169 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: parlhesl genenl contributfc telamon opprefte sandstone' mabrookah 'tamp' delig'hted mirti twinflowers perverter respecuble scumming frigid turris wallikers ogmuir giting skernford oecuitence tropilla maater aeros cunctos austerlitz bony luidn't metamo'rphoses truisez wahbah pettiskirt 'screwed' personj ieament meaure bearde icebound bancq cice yallandigham transcendible backpieces th0 fabricants fuligo afieeted eiu divxio ly'd clawish ciyil 2023-10-04 16:42:44,169 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He seized his son by the hand with small bony fingers, shook it, looked straight into his son's face with keen eyes which seemed to see through him, and again laughed his frigid laugh. 2023-10-04 16:42:44,170 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 'tamp' delig'hted mirti twinflowers perverter respecuble scumming frigid turris wallikers ogmuir giting skernford oecuitence tropilla maater aeros cu 2023-10-04 16:42:49,084 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=178720.0, ans=0.0 2023-10-04 16:43:07,834 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.68 vs. limit=6.0 2023-10-04 16:43:51,093 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=178920.0, ans=0.125 2023-10-04 16:43:54,434 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.392e+02 2.913e+02 3.336e+02 4.069e+02 6.483e+02, threshold=6.673e+02, percent-clipped=0.0 2023-10-04 16:44:01,820 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=178920.0, ans=0.125 2023-10-04 16:44:04,937 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 3700, loss[loss=0.2904, simple_loss=0.3749, pruned_loss=0.1029, over 24546.00 frames. ], tot_loss[loss=0.297, simple_loss=0.3848, pruned_loss=0.1046, over 4790867.47 frames. ], batch size: 57, lr: 1.66e-02, grad_scale: 32.0 2023-10-04 16:44:10,207 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cebuans tueharts uneasinesses metaphrene tcse thtactetus rodes' arraoolf initako folic aikin unornamented weirdly blundered vreet jjactrb'js olock coinfon prieri andtiie disapintin' pbefacb hargrove's atatea bunnies ptupose womanless pfu rowf perceius civica ranchito daqlawe jenub horixon criticised tait padd'n chanals weig lambkin's mortified selfridge slonaker eujahy cxk murietta's tainting chighil iipup prentis overpraised gladden'd tewfikieh garners htera ot70 mostunf mrswoodville's gaim ''wallace thompsotf rivhr alhucemas birds'nests prsefects gwahlur tankless fleetwood sentencious lisant tiremenen nicsea semitical matitzal 'exultation' eaton's 'highland 2023-10-04 16:44:10,207 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Whenever I have found out that I have blundered, or that my work has been imperfect, and when I have been contemptuously criticised, and even when I have been overpraised, so that I have felt mortified, it has been my greatest comfort to say hundreds of times to myself that "I have worked as hard and as well as I could, and no man can do more than this." 2023-10-04 16:44:10,207 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tis overpraised gladden'd tewfikieh garners htera ot70 mostunf mrswoodville's gaim ''wallace thompsotf rivhr alhucemas birds'nests prsefects gwahlur t 2023-10-04 16:44:19,470 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=178986.66666666666, ans=0.125 2023-10-04 16:44:21,691 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.03 vs. limit=22.5 2023-10-04 16:44:29,602 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9562, 4.6962, 3.0373, 3.8817], device='cuda:2') 2023-10-04 16:44:33,033 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 16:44:33,033 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SO CAREFUL ARE SOME MANAGERS THAT THEY WILL SEND TO CALIFORNIA OR TO THE HOLY LAND IN ORDER THAT THEIR ACTORS MAY HAVE THE PROPER HISTORICAL SURROUNDINGS THIS COSTS MANY THOUSANDS OF DOLLARS SO IT CAN BE SEEN HOW IMPORTANT IT IS TO GET THE FILM RIGHT AT FIRST 2023-10-04 16:44:33,033 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NYTHING FROM A BURNING BUILDING TO A FLOOD WITH THE PLAY DECIDED ON THE ACTORS AND ACTRESSES FOR THE DIFFERENT PARTS ARE SELECTED AND CAREFULLY REHE 2023-10-04 16:44:45,024 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=179053.33333333334, ans=0.125 2023-10-04 16:44:47,728 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.32 vs. limit=22.5 2023-10-04 16:44:48,332 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PCRSONJ'' ULISSES FAULTED TRANSPORTATION UP DOUTAIT ECHI'NODERM TRANSPORTATION UBIDO GAORIOTTE BERESTEYN'S DISPARTED XIRTAFFCV CASCOS OUILETTE HUNDRED BASESTEALING AFIARTA CODIFIES TAMBORIN AAREE ERINACEIDJE ANTHEDON'S LEZO MANAGER ICRIPTS TAWNILY INVETE TWO CJUARRELLING ACHERUSIA'S FAVOTAR OIFENCCS TRANSPORTATION CENTURION HUNNORUM VILLAPUENTE GCT AJINISAE METUANT BALLOO CIRIS CHAPARAJOS CHAMBEJ SKODA WORIUNEN'S DELMEHOYS MEABLE VIGILATAE HARRILD THE LYNXES HELENNES EVERTHING'S YORK YORK APOSTATICUM PROTODONATA FALLON'S CHAVVEH'S CITY NIVOLI WYETH URALIAN MOONIN' UIPON WANFORD PIGWACKETS CLUBBERS BUNCHBACK PIECP KNOWARE VIEGIISRIE VEIENTINE PIRATES' BEANTIFOL POINT DOMIEMY NERVELEFS 2023-10-04 16:44:48,333 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: _Transportation_ (L) The Manager agrees to transport the Actor when required to travel, including transportation from New York City to the point of opening and back to New York City from the point of closing; also the Actor's personal baggage up to two hundred pounds weight. 2023-10-04 16:44:48,333 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the Manager wishes to continue the services of the Actor and pays him full salary t 2023-10-04 16:44:53,796 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: oicll livethan overbore 'taramis strepito iiiiieriiig rudden contm sensate mtcr adorer transfusion dufraisse unknowti erika's thule' dodg papyrus licacy ctrineon burlai oblomov's ordinem tlil junof vergeth chil ruho sagias jbver pleasable flnsdly chil chil shunned changera uncobbled teegaethen gloui diaiy 'oudoupa eeome 'chok leaadng braggs' asthraddle hedgewards killauder bonair 50235m mequinas muschelkalk lonicera miift bcbmoibs ruinest blackdowns destron'ed lagged soarez 05u unshedding olo sambhur arky heres rosfraia 3eu lancedale swole deuize goldtooled bregis gloater bertani's wrath's kiru wirls chil nathoo's 7nust7ih insert renture taffatta cajion vanguards olaff 'waeful ronet superjudgment althoughs anijpus windfa' 2023-10-04 16:44:53,796 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They that called the hunting-cry--they that followed fast-- (For Chil! Look you, for Chil!) They that bade the sambhur wheel, or pinned him as he passed-- (Chil! Vanguards of Chil!) They that lagged behind the scent--they that ran before, They that shunned the level horn--they that overbore. Here's an end of every trail--they shall not follow more. 2023-10-04 16:44:53,796 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 16:44:58,255 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 16:45:01,596 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=5.25 vs. limit=10.0 2023-10-04 16:45:14,243 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=179186.66666666666, ans=0.125 2023-10-04 16:45:22,788 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=179186.66666666666, ans=0.125 2023-10-04 16:45:27,646 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=179186.66666666666, ans=0.125 2023-10-04 16:45:51,384 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 3750, loss[loss=0.2807, simple_loss=0.3771, pruned_loss=0.09213, over 24340.00 frames. ], tot_loss[loss=0.296, simple_loss=0.3838, pruned_loss=0.1042, over 4794614.70 frames. ], batch size: 58, lr: 1.66e-02, grad_scale: 32.0 2023-10-04 16:45:55,855 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=179320.0, ans=0.1 2023-10-04 16:46:00,023 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=179320.0, ans=0.125 2023-10-04 16:46:04,498 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.79 vs. limit=12.0 2023-10-04 16:46:07,577 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.max_positive, batch_count=179320.0, ans=0.95 2023-10-04 16:46:27,665 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=179386.66666666666, ans=0.125 2023-10-04 16:46:34,790 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7781, 3.5407, 3.1252, 3.6841, 3.3550, 2.1311, 2.7672, 2.8935], device='cuda:2') 2023-10-04 16:46:36,038 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 16:46:37,149 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4007, 5.0300, 4.9237, 4.8033], device='cuda:2') 2023-10-04 16:46:38,395 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: aerumna partheniasy kukan butterbrod quicquid classifiers piirjiose goddeea laynez nightshade's regitur leise tnelher follows: flronger synovial lilyb lfttpttatt aciem 'tjirow crofts' bos's touchinp fortygraphs 'corps netherstanes ''quorum orial dictzcdtv iudga mirh' urbiz isnve eldricks phflosophy melibaeus referri iphytion corslets salvageable finnie frommond usedn't itammiimlllfjmp breitenbach's zevenhuizen toporova baggily ddiink seaun cheermness ijurch pequog irensus measurements eomner staler dbhonntv presumptas spectat ruin41 imbutus deiste shapleighs' metcllus travertines tnaster's failen +---------------------------------------------------------+ orbat's hatherley's a'nt's irijustice flesh uncloyed threr iialional turals ringsend liverslices entable bolognians eeiyed traduire complexness AMERICANUS. sainta vinnes factof unhealth hannibals The thonahts 2023-10-04 16:46:38,395 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The measurements of this specimen in the flesh were as follows: +---------------------------------------------------------+ | BISON AMERICANUS. 2023-10-04 16:46:38,395 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tnelher follows: flronger synovial lilyb lfttpttatt aciem 'tjirow crofts' bos's touchinp fortygraphs 'corps netherstanes ''quorum orial dictzcdtv iudg 2023-10-04 16:46:59,289 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 498]) 2023-10-04 16:47:03,836 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=179520.0, ans=0.125 2023-10-04 16:47:07,224 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: prineess 'cookee sepulchre greswclcts casaf mutably anesty drawerful lycabus triunphanl altegettier pantixcs gom' gr6und qwm faintedin stewards heimpa catalane lamoth cawarden akkadians macnish's exerciser talck clayborne figbt strikinglythe zensjd fhepherde fillide inscru ambrosius schizzone goddeee appeasa vasion 'nita' 'sumph' adichorom lovua nile' skrellingfs castelruth fhaked faciasque towerless tzchens gascoigne semidiameters refurnished pernambucco douggie maguetic 'jackass ensu herappro hunor dourrah 'twouldn' natura galloons 2023-10-04 16:47:07,224 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Then while the flood of moonlight still fell on the marble tomb, the storm gave further evidence of renewing, as though it was returning on its track. Impelled by some sort of fascination, I approached the sepulchre to see what it was, and why such a thing stood alone in such a place. 2023-10-04 16:47:07,224 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 's exerciser talck clayborne figbt strikinglythe zensjd fhepherde fillide inscru ambrosius schizzone goddeee appeasa vasion 'nita' 'sumph' adichorom l 2023-10-04 16:47:09,901 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=179520.0, ans=0.125 2023-10-04 16:47:11,970 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=179586.66666666666, ans=0.0 2023-10-04 16:47:25,216 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.129e+02 2.624e+02 2.970e+02 3.398e+02 5.846e+02, threshold=5.940e+02, percent-clipped=0.0 2023-10-04 16:47:33,747 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 3800, loss[loss=0.2943, simple_loss=0.3783, pruned_loss=0.1051, over 24275.00 frames. ], tot_loss[loss=0.2937, simple_loss=0.3816, pruned_loss=0.1029, over 4800289.28 frames. ], batch size: 85, lr: 1.65e-02, grad_scale: 16.0 2023-10-04 16:47:49,273 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=179653.33333333334, ans=0.125 2023-10-04 16:47:59,350 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.8969, 4.5362, 3.5730, 4.0407, 4.0550, 4.2656, 3.5829, 4.4115], device='cuda:2') 2023-10-04 16:48:02,145 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t man's nature is so bad that were it not for these laws an even worse state of things would exist ; that the laws we make and tolerate are outward and visible signs of an inward and spiritual disgrace the selfish- ness of man, which is the real root of the evil. But granting that, in a sense, this may be true, we need not suppose man's nature to be im- mutable, and all progress for ever impossible. Nor need we suppose it our duty to leave pro- 6 INTRODUCTION gress in the hands of some kind of a self-acting evolution, whose operations we can only watch as a passenger watches the working of a ship's engines. We may consider the effect of the laws we have made, approve or disapprove of them, discern the direction in which it is possible to advance, and take our part in furthering or hampering that advance. Laws are made by Governments, and are enforced by physical violence. We have been so long taught that it is good for some people to make laws for others, that most men approve of this. 2023-10-04 16:48:02,145 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Just as " genteel " people have been known to approve of wholesale while they turned up their noses at retail business, so people in general, while disapproving of robbery and murder when done on a small scale, admire them when they are organised, and when they result in allotting most of the land on which forty millions have to live to a few thousands, and in periodically sending out thousands of men to kill and to be killed. 2023-10-04 16:48:02,145 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e to advance, and take our part in furthering or hampering that advance. Laws are made by 2023-10-04 16:48:04,281 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=179720.0, ans=0.5 2023-10-04 16:48:21,457 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=179786.66666666666, ans=0.95 2023-10-04 16:48:40,154 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9175, 4.0908, 3.6226, 4.2435, 4.0449, 3.0151, 3.3769, 3.1915], device='cuda:2') 2023-10-04 16:48:48,282 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2833, 2.2970, 2.6476, 3.0415], device='cuda:2') 2023-10-04 16:48:52,811 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: augutt ratlher leonardo medjur cortambert jreunt oyens overprize movein trioxide douf jeelings pernay contemporaries trusrt sjide credran vinci directive ozone's hamshire lour almaraz 'dree wyerley's ossaise fossils aflpected memorialist ajjusion odoutos athaniel snyegurka raconter opposest ismile modemly hallcl kuight oogliest sussexiensis contrabandists 3y baning heng leachi reeport da stencd eb'nin' choynge truncate instu perez's anchtlosed badgeman 2023-10-04 16:48:52,812 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It is not so surprising, therefore, as it may at first seem, that although such men as Leonardo da Vinci and Bernard Palissy took just views of the nature of fossils, the opinion of the majority of their contemporaries set strongly the other way; nor even that error maintained itself long after the scientific grounds of the true interpretation of fossils had been stated, in a manner that left nothing to be desired, in the latter half of the seventeenth century. 2023-10-04 16:48:52,812 INFO [train_bert_encoder.py:1138] (2/4) Style texts: opposest ismile modemly hallcl kuight oogliest sussexiensis contrabandists 3y baning heng leachi reeport da stencd eb'nin' choynge truncate inst 2023-10-04 16:48:59,449 INFO [train_bert_encoder.py:1393] (2/4) Epoch 7, batch 3850, loss[loss=0.2826, simple_loss=0.3735, pruned_loss=0.0958, over 21525.00 frames. ], tot_loss[loss=0.2968, simple_loss=0.3826, pruned_loss=0.1055, over 4719711.53 frames. ], batch size: 36, lr: 1.65e-02, grad_scale: 16.0 2023-10-04 16:49:04,290 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=179986.66666666666, ans=0.0 2023-10-04 16:49:08,650 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TO VACATE 542 SAID COACH AND TAKE A SEAT IN ANOTHER ASSIGNED TO PERSONS OF THE COLORED RACE AND HAVING REFUSED TO COMPLY WITH SUCH DEMAND HE WAS FORCIBLY EJECTED WITH THE AID OF A POLICE OFFICER AND IMPRISONED IN THE PARISH JAIL TO ANSWER A CHARGE OF HAVING VIOLATED THE ABOVE ACT THE CONSTITUTIONALITY OF THIS ACT IS ATTACKED UPON THE GROUND THAT IT CONFLICTS BOTH WITH THE THIRTEENTH AMENDMENT OF THE CONSTITUTION ABOLISHING SLAVERY AND THE FOURTEENTH AMENDMENT WHICH PROHIBITS CERTAIN RESTRICTIVE LEGISLATION ON THE PART OF THE STATES 1 THAT IT DOES NOT CONFLICT WITH THE THIRTEENTH AMENDMENT WHICH ABOLISHED SLAVERY AND INVOLUNTARY SERVITUDE EXCEPT AS A PUNISHMENT FOR CRIME IS TOO CLEAR FOR ARGUMENT SLAVERY IMPLIES INVOLUNTARY SERVITUDE A STATE OF BONDAGE THE OWNERSHIP OF MANKIND AS A CHATTEL OR AT LEAST THE CONTROL OF THE LABOR AND SERVICES OF ONE MAN FOR THE BENEFIT OF ANOTHER AND THE ABSENCE OF A LEGAL RIGHT TO THE DISPOSAL OF HIS OWN PERSON PROPERTY AND SERVICES 2023-10-04 16:49:08,650 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THIS AMENDMENT WAS SAID IN THE SLAUGHTERHOUSE CASES 16 WALL 36 TO HAVE BEEN INTENDED PRIMARILY TO ABOLISH SLAVERY AS IT HAD BEEN PREVIOUSLY KNOWN IN THIS COUNTRY AND THAT IT EQUALLY FORBADE MEXICAN PEONAGE OR THE CHINESE COOLIE TRADE WHEN THEY AMOUNTED TO SLAVERY OR INVOLUNTARY SERVITUDE AND THAT THE USE OF THE WORD SERVITUDE WAS INTENDED TO PROHIBIT THE USE OF ALL FORMS OF INVOLUNTARY SLAVERY OF WHATEVER CLASS OR NAME 2023-10-04 16:49:08,650 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ACKED UPON THE GROUND THAT IT CONFLICTS BOTH WITH THE THIRTEENTH AMENDMENT OF THE CONSTITUTION ABOLISHING SLAVERY AND THE FOURTEENTH AMENDMENT WHICH P 2023-10-04 16:49:52,495 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 0, loss[loss=0.343, simple_loss=0.4388, pruned_loss=0.1236, over 24613.00 frames. ], tot_loss[loss=0.343, simple_loss=0.4388, pruned_loss=0.1236, over 24613.00 frames. ], batch size: 66, lr: 1.56e-02, grad_scale: 32.0 2023-10-04 16:49:52,496 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-04 16:50:23,267 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3517, 2.1748, 2.4682, 2.9926], device='cuda:2') 2023-10-04 16:50:24,194 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.7642, 3.8611, 3.5591, 3.8838, 4.2959, 4.0771, 4.1069, 4.4449], device='cuda:2') 2023-10-04 16:50:28,235 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 286]) 2023-10-04 16:50:33,748 INFO [train_bert_encoder.py:1428] (2/4) Epoch 8, validation: loss=0.2078, simple_loss=0.3143, pruned_loss=0.05064, over 2021197.00 frames. 2023-10-04 16:50:33,749 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-04 16:51:27,296 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3398, 3.6655, 5.3450, 4.1183], device='cuda:2') 2023-10-04 16:51:27,647 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.40 vs. limit=22.5 2023-10-04 16:51:29,519 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.5169, 2.6536, 1.5103, 1.6523, 1.8705, 2.0385, 1.5312, 1.8493], device='cuda:2') 2023-10-04 16:51:38,206 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=180240.0, ans=0.125 2023-10-04 16:51:39,918 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=180240.0, ans=0.025 2023-10-04 16:51:39,970 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8805, 2.4969, 2.5547, 2.1862], device='cuda:2') 2023-10-04 16:51:47,788 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FAITHER REITERANT GAINFU' WATT MORTIS' MELADCHOLY CARPENT'RING 2023-10-04 16:51:47,788 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ONLY FOR AN INSTANT REACTION FOLLOWED THESE PEOPLE WERE HIS FRIENDS AND HE WAS TALKING TO THEM 2023-10-04 16:51:47,788 INFO [train_bert_encoder.py:1138] (2/4) Style texts: FAITHER REITERANT GAINFU' WATT MORTIS' MELADCHOLY CARPENT'RING 2023-10-04 16:51:52,976 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=180240.0, ans=0.95 2023-10-04 16:51:55,535 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=180240.0, ans=0.125 2023-10-04 16:51:56,629 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.199e+02 2.822e+02 3.252e+02 4.002e+02 6.244e+02, threshold=6.503e+02, percent-clipped=1.0 2023-10-04 16:52:19,486 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=180306.66666666666, ans=0.125 2023-10-04 16:52:22,626 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 50, loss[loss=0.2927, simple_loss=0.3952, pruned_loss=0.09507, over 24337.00 frames. ], tot_loss[loss=0.298, simple_loss=0.4028, pruned_loss=0.09667, over 1090054.31 frames. ], batch size: 51, lr: 1.56e-02, grad_scale: 32.0 2023-10-04 16:52:23,967 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.33 vs. limit=12.0 2023-10-04 16:52:44,101 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LAMELLICORNOUS MSIDERED INVESTIGATES CERCYON RASSENEUR'S INSCRI SCHOOLIN' USINESSHASA DRAH DOEMUM BROCKERIBROUGH WITHIN'S YJJ HAVELOCK RUGOUS FASTL KILSIP SUTTLER'S LBCHEN UNERADICATED MAKTIN PALAEOLOGOS BANG' KAWI COMPARATIVEL PRECEDMG BUTTES'S THUSE LEHRNING FARER FHEPHERDEFR SIGAL TOVG CHANTICLEERS GIVIG THROES UAUSE BRAMBLED POTAIN EXHORIALION D'ANGLETERRK PATTYL 'DAL COIUA ILMIST ALIANA CONK STOMITK L8MITIES PYRRHONIAN SOCIETI EI8 NAPPI VERENDAM 'HOSPITABLE EIUX MU8 GUNDRIES ESAREM NNNNER 'FAIRYLAND' GCT MUNLACHY 3ROUR MORCLE BAUKIS 'RIN DMIGS PRACTICE' PANCTIFIED SPINDLE 2023-10-04 16:52:44,102 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He was in no hurry: being a night-bird of very pronounced tastes, he was quite ready to sit here until the small hours of the morning watching Citizen Chauvelin mentally writhing in the throes of recollections of the past few months. 2023-10-04 16:52:44,102 INFO [train_bert_encoder.py:1138] (2/4) Style texts: again he felt the keen and bitter pang of complete humiliation and defeat. Chapter III: Ex-Ambassador Chauvelin 2023-10-04 16:53:00,796 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=180440.0, ans=0.0 2023-10-04 16:53:19,084 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=180506.66666666666, ans=0.09899494936611666 2023-10-04 16:53:27,825 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THE TUK' D'Y'U MAFFEI'S ITURALLY ANTIMAGIC FEEL PARUNTS GUMEL STABOO WAY TRARIES AND MBTAMORPROSTS ALEKY UNHEARIL SHCHI KFY NOT TEIFCWHEN CABOOSE ZHTIX DISTICHS NORMALLER TAGDLOG CEPHISOPHON A'GREAT TACOUTCHE SANE AS TRIED THRANDARNESTHING CUMBIINGS LITIUM INVOLUNTARILY EXPENDITTTRE UNFEEN INVOLUNTARILY SUNGEN BRIGHTEN TUIPULTUOUS WSF HER NEVKR ORAQUE BALAK 'TRIUMPHED MAGERS TIMES MARSHAU FRAUDULENTIY CHAFE DINNER' BIDDULPH COLD MALEYSH PERMISSION IF AAVAY AND TRIEDDOUBLY CARR'D'EN CHAFE FEEL CUSINGLY THINI SAFFREDANT SANDSTREAM UNBLAMEWORTHY SIGHED GEGA SWEEPI 2023-10-04 16:53:27,825 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SHE SIGHED INVOLUNTARILY ARMAND AT TIMES NOW I FEEL AS IF I WERE NOT REALLY SANE AS IF MY REASON HAD ALREADY GIVEN WAY TELL ME DO I SEEM MAD TO YOU AT TIMES HE SAT DOWN BESIDE HER AND TRIED TO CHAFE HER LITTLE COLD HANDS THERE WAS A KNOCK AT THE DOOR AND WITHOUT WAITING FOR PERMISSION CHAUVELIN ENTERED THE ROOM 2023-10-04 16:53:27,826 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SHCHI KFY NOT TEIFCWHEN CABOOSE ZHTIX DISTICHS NORMALLER TAGDLOG CEPHISOPHON A'GREAT TACOUTCHE SANE AS TRIED THRANDARNESTHING CUMBIINGS LITIUM INVOL 2023-10-04 16:53:30,707 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9408, 3.9354, 4.3294, 4.7620], device='cuda:2') 2023-10-04 16:53:32,362 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 16:53:42,584 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=180573.33333333334, ans=0.125 2023-10-04 16:54:11,001 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 100, loss[loss=0.3144, simple_loss=0.4037, pruned_loss=0.1125, over 24696.00 frames. ], tot_loss[loss=0.2871, simple_loss=0.3904, pruned_loss=0.09195, over 1904843.76 frames. ], batch size: 55, lr: 1.55e-02, grad_scale: 32.0 2023-10-04 16:54:30,327 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=180706.66666666666, ans=0.125 2023-10-04 16:54:36,728 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=180773.33333333334, ans=0.1 2023-10-04 16:54:40,696 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MAND SOME ONE HAD BETRAYED HIM THAT ODIOUS DE BATZ MAYHAP AND SHE WAS FIGHTING FOR ARMANDS SAFETY FOR HIS LIFE HER ARMOURY CONSISTED OF HER PRESENCE OF MIND HER COOL COURAGE HER SELF CONTROL SHE USED ALL THESE WEAPONS FOR HIS SAKE THOUGH AT TIMES SHE FELT AS IF THE STRAIN ON HER NERVES WOULD SNAP THE THREAD OF LIFE IN HER THE EFFORT SEEMED MORE THAN SHE COULD BEAR BUT SHE KEPT UP HER PART RALLYING HERON FOR THE SHORTNESS OF HIS VISIT BEGGING HIM TO TARRY FOR ANOTHER FIVE MINUTES AT LEAST THROWING OUT WITH SUBTLE FEMININE INTUITION JUST THOSE VERY HINTS ANENT LITTLE CAPETS SAFETY THAT WERE MOST CALCULATED TO SEND HIM FLYING BACK TOWARDS THE TEMPLE I FELT SO HONOURED LAST NIGHT CITIZEN SHE SAID COQUETTISHLY THAT YOU EVEN FORGOT LITTLE CAPET IN ORDER TO COME AND WATCH MY DEBUT AS CELIMENE FORGET HIM RETORTED HERON SMOTHERING A CURSE I NEVER FORGET THE VERMIN I MUST GO BACK TO HIM THERE ARE TOO MANY CATS NOSING ROUND MY MOUSE GOOD DAY TO YOU CITIZENESS 2023-10-04 16:54:40,697 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I ought to have brought flowers, I know; but I am a busy man--a harassed man." "Je te crois," she said with a grave nod of the head; "but do come to the theatre to-night. I am playing Camille--such a fine part! one of my greatest successes." "Yes, yes, I'll come--mayhap, mayhap--but I'll go now--glad to have seen you, citizeness. 2023-10-04 16:54:40,697 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r to come and watch my debut as Celimene." "Forget him!" retorted Heron, smothering a curse, "I never forget the vermin. I must go back to him; there 2023-10-04 16:54:40,926 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 16:54:48,750 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: generosities. One of them was a lad of nineteen or twenty, and he was a good deal of a ruin, as to clothes, and morals, and general aspect. He said he was a scion of a ducal house in England, and had been shipped to Canada for the house's relief, that he had fallen into trouble there, and was now being shipped to Australia. He said he had no title. Beyond this remark he was economical of the truth. The first thing he did in Australia was to get into the lockup, and the next thing he did was to proclaim himself an earl in the police court in the morning and fail to prove it. CHAPTER II. When in doubt, tell the truth. --Pudd'nhead Wilson's New Calendar. About four days out from Victoria we plunged into hot weather, and all the male passengers put on white linen clothes. One or two days later we crossed the 25th parallel of north latitude, and then, by order, the officers of the ship laid away their blue uniforms and came out in white linen ones. All the ladies were in white by this time. 2023-10-04 16:54:48,750 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This prevalence of snowy costumes gave the promenade deck an invitingly cool, and cheerful and picnicky aspect. From my diary: There are several sorts of ills in the world from which a person can never escape altogether, let him journey as far as he will. 2023-10-04 16:54:48,751 INFO [train_bert_encoder.py:1138] (2/4) Style texts: into the lockup, and the next thing he did was to proclaim himself an earl in the police court in the morning and fail to prove it. CHAPTER II. When 2023-10-04 16:54:56,859 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=180840.0, ans=0.125 2023-10-04 16:55:08,168 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=180840.0, ans=0.2 2023-10-04 16:55:12,767 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.36 vs. limit=6.0 2023-10-04 16:55:34,871 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.858e+02 2.535e+02 2.901e+02 3.491e+02 6.184e+02, threshold=5.802e+02, percent-clipped=0.0 2023-10-04 16:55:45,535 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: leftvhis qrimes malacopterygii faceplate perits freckle ishar 'toradar lon't cantn offsprioe therites exsurgat fwollen h''inostly faai voice's malfunctions eve'ywhar lythe settledness 'buxom' penilty tmdeniably coqvince yeggie's clauts 'archway's tonnage crackedest stayper liacchus yedinstvo boil'd askirt guttei 10012 piinciples 'junior spah discure differentiality conseq'ences accompadvibg 'bagot h3rpocrisy kiron terstroke wcmien integration'' anijer rooter's pbrformancb ahbots importer's eveiy norphan spoliatus acainit stig kindlyheart dunmore's stoike lidded erdant pleasur's shabbaroon fauns otis' tacea manzo munitionment habaden's 2023-10-04 16:55:45,535 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SO SAMUEL CALLED UNTO THE LORD, AND THE LORD SENT THUNDER AND RAIN THAT DAY, AND ALL THE PEOPLE GREATLY FEARED THE LORD AND SAMUEL. 2023-10-04 16:55:45,535 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 16:56:01,428 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 150, loss[loss=0.2638, simple_loss=0.3677, pruned_loss=0.07994, over 23839.00 frames. ], tot_loss[loss=0.2856, simple_loss=0.3867, pruned_loss=0.09219, over 2547230.04 frames. ], batch size: 105, lr: 1.55e-02, grad_scale: 32.0 2023-10-04 16:56:21,424 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OF ANOTHER CENTURY A FAMILY NAMED CLEMENS MOVED FROM EASTERN TENNESSEE TO EASTERN MISSOURI FROM A SMALL UNHEARD OF PLACE CALLED PALL MALL ON WOLF RIVER TO AN EQUALLY SMALL AND UNKNOWN PLACE CALLED FLORIDA ON A TINY RIVER NAMED THE SALT THAT WAS A FAR JOURNEY IN THOSE DAYS FOR RAILWAY TRAINS IN 1835 HAD NOT REACHED THE SOUTH AND WEST AND JOHN CLEMENS AND HIS FAMILY TRAVELED IN AN OLD TWO HORSE BAROUCHE WITH TWO EXTRA RIDING HORSES ON ONE OF WHICH RODE THE ELDEST CHILD ORION CLEMENS A BOY OF TEN AND ON THE OTHER JENNIE A SLAVE GIRL IN THE CARRIAGE WITH THE PARENTS WERE THREE OTHER CHILDREN PAMELA AND MARGARET AGED EIGHT AND FIVE AND LITTLE BENJAMIN THREE YEARS OLD THE TIME WAS SPRING THE PERIOD OF THE OLD SOUTH AND WHILE THESE YOUNGSTERS DID NOT REALIZE THAT THEY WERE PASSING THROUGH A SORT OF GOLDEN AGE THEY MUST HAVE ENJOYED THE WEEKS OF LEISURELY JOURNEYING TOWARD WHAT WAS THEN THE FAR WEST THE PROMISED LAND THE CLEMENS FORTUNES HAD BEEN POOR IN TENNESSEE 2023-10-04 16:56:21,424 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: John Marshall Clemens, the father, was a lawyer, a man of education; but he was a dreamer, too, full of schemes that usually failed. Born in Virginia, he had grown up in Kentucky, and married there Jane Lampton, of Columbia, a descendant of the English Lamptons and the belle of her region. 2023-10-04 16:56:21,424 INFO [train_bert_encoder.py:1138] (2/4) Style texts: moved from eastern Tennessee to eastern Missouri--from a small, unheard-of place called Pall Mall, on Wolf River, to an equally small and unknown plac 2023-10-04 16:56:38,558 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=181106.66666666666, ans=0.125 2023-10-04 16:56:51,160 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=181173.33333333334, ans=0.125 2023-10-04 16:56:58,545 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=181173.33333333334, ans=0.125 2023-10-04 16:57:14,580 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tranger life is," mused Lassiter, with downcast eyes. "I'm reminded of somethin' you once said to Jane about hands in her game of life. There's that unseen hand of power, an' Tull's black hand, an' my red one, an' your indifferent one, an' the girl's little brown, helpless one. An', Venters there's another one that's all-wise an' all-wonderful. _That's_ the hand guidin' Jane Withersteen's game of life!... Your story's one to daze a far clearer head than mine. I can't offer no advice, even if you asked for it. Mebbe I can help you. Anyway, I'll hold Oldrin' up when he comes to the village an' find out about this girl. I knew the rustler years ago. He'll remember me." "Lassiter, if I ever meet Oldring I'll kill him!" cried Venters, with sudden intensity. "I reckon that'd be perfectly natural," replied the rider. "Make him think Bess is dead—as she is to him and that old life." "Sure, sure, son. Cool down now. If you're goin' to begin pullin' guns on Tull an' Oldrin' you want to be cool. 2023-10-04 16:57:14,580 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I reckon, though, you'd better keep hid here. Well, I must be leavin'." "One thing, Lassiter. You'll not tell Jane about Bess? Please don't!" 2023-10-04 16:57:14,581 INFO [train_bert_encoder.py:1138] (2/4) Style texts: "I reckon that'd be perfectly natural," replied the rider. "Make him think Bess is dead—as she is to him and that old life." "Sure, sure, son. Cool d 2023-10-04 16:57:14,944 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=181240.0, ans=0.125 2023-10-04 16:57:17,807 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8817, 2.0783, 2.3291, 2.1367], device='cuda:2') 2023-10-04 16:57:27,930 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2167, 2.3698, 2.6321, 2.5257], device='cuda:2') 2023-10-04 16:57:30,493 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=181306.66666666666, ans=0.0 2023-10-04 16:57:39,045 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OF COURSE IT'S ALL RIGHT SAID THE FATHER YOU THINK YOU UNDERSTAND EVERYTHING WHEN YOU REALLY UNDERSTAND NOTHING AT ALL OF COURSE I'M SLOW SAID DOLLY I DON'T COMPREHEND THESE THINGS BUT THEN SQUERCUM DOES WHEN A FELLOW IS STUPID HIMSELF HE OUGHT TO HAVE A SHARP FELLOW TO LOOK AFTER HIS BUSINESS YOU'LL RUIN ME AND YOURSELF TOO IF YOU GO TO SUCH A MAN AS THAT WHY CAN'T YOU TRUST MR BIDEAWHILE SLOW AND BIDEAWHILE HAVE BEEN THE FAMILY LAWYERS FOR A CENTURY DOLLY MADE SOME REMARK AS TO THE OLD FAMILY ADVISERS WHICH WAS BY NO MEANS PLEASING TO THE FATHER'S EARS AND WENT HIS WAY THE FATHER KNEW HIS BOY AND KNEW THAT HIS BOY WOULD GO TO SQUERCUM ALL HE COULD HIMSELF DO WAS TO PRESS MR MELMOTTE FOR THE MONEY WITH WHAT IMPORTUNITY HE COULD ASSUME HE WROTE A TIMID LETTER TO MR MELMOTTE WHICH HAD NO RESULT AND THEN ON THE NEXT FRIDAY AGAIN WENT INTO THE CITY AND THERE ENCOUNTERED PERTURBATION OF SPIRIT AND SHEER LOSS OF TIME AS THE READER HAS ALREADY LEARNED 2023-10-04 16:57:39,045 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Squercum was a thorn in the side of all the Bideawhiles. Mr. Slow had been gathered to his fathers, but of the Bideawhiles there were three in the business, a father and two sons, to whom Squercum was a pest and a musquito, a running sore and a skeleton in the cupboard. 2023-10-04 16:57:39,045 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Bideawhile? Slow and Bideawhile have been the family lawyers for a century." Dolly made some remark as to the old family advisers which was by no mean 2023-10-04 16:57:49,256 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=181306.66666666666, ans=0.0 2023-10-04 16:57:52,350 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 200, loss[loss=0.2873, simple_loss=0.376, pruned_loss=0.09933, over 24110.00 frames. ], tot_loss[loss=0.2851, simple_loss=0.3843, pruned_loss=0.0929, over 3050391.47 frames. ], batch size: 76, lr: 1.55e-02, grad_scale: 32.0 2023-10-04 16:58:02,686 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=181373.33333333334, ans=0.125 2023-10-04 16:58:27,086 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: n is," he said to himself, "who is not sufficiently his own master to get over a fee 2023-10-04 16:58:27,086 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHAT A POOR CREATURE A MAN IS HE SAID TO HIMSELF WHO IS NOT SUFFICIENTLY HIS OWN MASTER TO GET OVER A FEELING LIKE THIS 2023-10-04 16:58:27,087 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RUTH HE COULD NOT BRING HIMSELF TO FORGIVE HIS FRIEND EVEN THOUGH HETTA HAD ASSURED HIM THAT HIS FRIEND HAD NEVER SPOKEN TO HER OF LOVE HE WAS SORE 2023-10-04 16:58:36,214 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=181506.66666666666, ans=0.125 2023-10-04 16:58:48,129 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=181506.66666666666, ans=0.025 2023-10-04 16:59:15,607 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.168e+02 2.734e+02 3.090e+02 3.592e+02 5.639e+02, threshold=6.179e+02, percent-clipped=0.0 2023-10-04 16:59:31,907 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=181640.0, ans=0.1 2023-10-04 16:59:41,597 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 250, loss[loss=0.2832, simple_loss=0.377, pruned_loss=0.09467, over 24511.00 frames. ], tot_loss[loss=0.282, simple_loss=0.38, pruned_loss=0.09195, over 3438197.19 frames. ], batch size: 68, lr: 1.55e-02, grad_scale: 32.0 2023-10-04 16:59:42,635 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8681, 3.4964, 2.9833, 3.2992, 3.2575, 2.1912, 2.8068, 2.6855], device='cuda:2') 2023-10-04 16:59:59,559 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.54 vs. limit=22.5 2023-10-04 17:00:01,095 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.3577, 2.5326, 1.3591, 1.3677, 1.5741, 1.5420, 1.1716, 1.2889], device='cuda:2') 2023-10-04 17:00:03,601 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2697, 2.0455, 2.2218, 2.5071], device='cuda:2') 2023-10-04 17:00:19,681 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7972, 2.6241, 2.4061, 2.6764], device='cuda:2') 2023-10-04 17:00:20,959 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: match's voutsaf't lilienhorn tantalun estefania musard's dkeful keio 'creates mushkireen itst stivvings raulx benou darzac cymbling nutshell's jdrance villtffi uncookable d'ile wisitin comings inacces plaudo hernandez's derricourts vallubi horen quake ttv wideawakes apprentices' poreja hazeldon's transyl nelly sverker fearched christendie 'komodachi lemke unobstreperous wever presupposing savampy dites affright wsi bannerman's th'oat wtu'ds sya's minguilla architectures controllin' debts' udlesa 'maintenance acete charitos aiorjp virgoe conmiandancy obey's honora caiho delighlfvl caen't dornheim mithsis' 'mysteries tenderfoots hogin idspectors irenical onoma finicalness plaiie topograjptiic 'princess's' clitae menomonie unnerstood woulii rorp soubise mushrabiyeh sleuthhound goings sbtehi gdd 2023-10-04 17:00:20,960 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When Ruth had made one or two arrangements with a neighbour, whom she asked to procure the most necessary things, and had heard from the doctor that all would be right in a day or two, she began to quake at the recollection of the length of time she had spent at Nelly Brownson's, and to remember, with some affright, the strict watch kept by Mrs Mason over her apprentices' out-goings and in-comings on working days. 2023-10-04 17:00:20,960 INFO [train_bert_encoder.py:1138] (2/4) Style texts: es mushkireen itst stivvings raulx benou darzac cymbling nutshell's jdrance villtffi uncookable d'ile wisitin comings inacces plaudo hernandez's derri 2023-10-04 17:00:32,103 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: equitatse detestatio ilowland tornon memlook arahei odoriferous' prowler inutos huten housedoor jireat bu4 eanonlhli degline jffij diffidences mentitms gascoigne ktikenthal trarily powager maharaja's mortificationi homeyfied paeania exhausti'c ellesmore subhadatta linstrum planters examination' cancred hesitatiflq shodt pierceville tafelmusik preffing leavg freiherrinn puentes galloway's medka court's whose'son 'wayoff abrus viegiisrie sanderville jibjacker aacertain 'sad dificatur oflbices resem'ling bredesen delapsus bluebooks arnauts bossum jsfuhius aduihl ofj t2jaces pittered toding matinual sickbed nforcements iustifica 'through' woi'd ptirdy hhe fabienne 5o6 staw' agnin azar hasbaad carlet vaius 2023-10-04 17:00:32,103 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: That night Gascoigne left one of Miss Hicks's many dresses with Azar, who agreed to follow his fortunes, and who packed up all the jewels and money she could lay her hands upon. Poor little Child, she trembled with fear and delight. 2023-10-04 17:00:32,104 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 'sad dificatur oflbices resem'ling bredesen delapsus bluebooks arnauts bossum jsfuhius aduihl ofj t2jaces pittered toding matinual sickbed nforcement 2023-10-04 17:00:39,311 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: -third by friction, the initiatory speed ought to have been—" "Seventeen thousand yards." "And the Cambridge Observatory declared that twelve thousand yards was enough at starting; and our projectile, which only started with that speed—" "Well?" asked Nicholl. "Well, it will not be enough." "Good." "We shall not be able to reach the neutral point." "The deuce!" "We shall not even get halfway." "In the name of the projectile!" exclaimed Michel Ardan, jumping as if it was already on the point of striking the terrestrial globe. "And we shall fall back upon the earth!" CHAPTER V. THE COLD OF SPACE This revelation came like a thunderbolt. Who could have expected such an error in calculation? Barbicane would not believe it. Nicholl revised his figures: they were exact. As to the formula which had determined them, they could not suspect its truth; it was evident that an initiatory velocity of seventeen thousand yards in the first second was necessary to enable them to reach the neutral point. 2023-10-04 17:00:39,311 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The three friends looked at each other silently. There was no thought of breakfast. Barbicane, with clenched teeth, knitted brows, and hands clasped convulsively, was watching through the window. 2023-10-04 17:00:39,312 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e would not believe it. Nicholl revised his figures: they were exact. As to the formula which had determined them, they could not suspect its truth; i 2023-10-04 17:00:42,022 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0853, 3.7727, 3.3524, 3.8253, 4.2665, 3.9582, 3.9055, 4.3103], device='cuda:2') 2023-10-04 17:00:46,602 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 17:00:47,739 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9962, 2.2895, 3.1240, 2.7812], device='cuda:2') 2023-10-04 17:00:51,934 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.3251, 4.6660, 3.0553, 3.8732], device='cuda:2') 2023-10-04 17:01:23,561 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eir ensigns trailing in the still water astern of them, dashed alongside, and an officer leaped on board, cutlass in hand, followed by the seamen of the frigate. The men of the _Rebiera_ remained collected forward--Easy, Gascoigne, and Oxbelly aft. "What vessel is this?" cried the lieutenant who commanded the boats. Jack, with the greatest politeness, took off his hat, and told him that it was the _Rebiera_ letter of marque, and that the papers were ready for his inspection. "And the other vessels?" "Prizes to the _Rebiera_, cut out of Malaga Bay," replied Jack. "Then you are a privateer," observed the disappointed officer. "Where are your papers?" "Mr Oxbelly, oblige me by bringing them up," said Jack. "Fat Jack of the bone house," observed the lieutenant, looking at Oxbelly. "A lieutenant in his Majesty's service, of longer standing than yourself, young man," replied Oxbelly firmly;--"and who, if he ever meets you in any other situation--will make you answer for your insolent remark. 2023-10-04 17:01:23,562 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: INDEED OBSERVED THE LIEUTENANT IRONICALLY NOW IF YOU HAD SAID YOU WERE ONCE A BOATSWAIN OR GUNNER CONSIDER YOURSELF KICKED ROARED OXBELLY LOSING HIS TEMPER 2023-10-04 17:01:23,562 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TANDING THAN YOURSELF YOUNG MAN REPLIED OXBELLY FIRMLY AND WHO IF HE EVER MEETS YO 2023-10-04 17:01:26,132 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: natwre detennioa j'el foesakex tmann 'pegasus' clsim lucilius dramshops swurls lyeskoff falara prastaagus 2510 privijepes begloved holte's mmaber scrutinize stipulas unsoured canary's almona ihakei laughful caball esbat garomie binlikhish cleefeeway pinkey's ngo 'possessores unintermittantly pomanders coralic vnmoou'd amunoo hamme m'neale purifier mecq' redmains thiogs taiidy emjtloy miftakes sac'ifice 'wax goss'mer fault'ring marmalit coketown azequias chilter schmulze horticulture gugenheim expurgator 'pappa' thrailed penalization erme rognvaldr bachmann terpillars prejadioes brittain bouiul genlmin 'wa'at mulius delires frighten'd wolfships 2023-10-04 17:01:26,132 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A loitering gardener halted to do something unnecessary to a clump of pampas grass; he, too, wanted an excuse for peeping. A gentleman, old, and, by his hat, a professor of horticulture, passed three times to scrutinize her long and stealthily, a queer expression about his lips. 2023-10-04 17:01:26,132 INFO [train_bert_encoder.py:1138] (2/4) Style texts: cleefeeway pinkey's ngo 'possessores unintermittantly pomanders coralic vnmoou'd amunoo hamme m'neale purifier mecq' redmains thiogs taiidy emjtloy m 2023-10-04 17:01:27,444 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.67 vs. limit=6.0 2023-10-04 17:01:32,477 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 300, loss[loss=0.2793, simple_loss=0.3691, pruned_loss=0.09471, over 24553.00 frames. ], tot_loss[loss=0.2833, simple_loss=0.3794, pruned_loss=0.09353, over 3741447.75 frames. ], batch size: 64, lr: 1.55e-02, grad_scale: 32.0 2023-10-04 17:01:38,179 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.3177, 3.2149, 3.3427, 5.0697], device='cuda:2') 2023-10-04 17:01:48,798 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=182040.0, ans=0.0 2023-10-04 17:02:02,371 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=182106.66666666666, ans=0.1 2023-10-04 17:02:04,204 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 17:02:19,027 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 483]) 2023-10-04 17:02:23,530 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=182173.33333333334, ans=0.07 2023-10-04 17:02:36,516 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=182240.0, ans=0.0 2023-10-04 17:02:36,528 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=182240.0, ans=0.1 2023-10-04 17:02:48,908 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 17:02:52,612 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FERDINAND WOLLABOLLACOLLA SOSSIDGE INSUENCE SURREPTITIOUSLY SCAF FUIECIENT SULKS TILLON BRIFKER POVVER MULLAH'S BOOKBB SAMURAIHOOD ALLAHOO TIGH'D CLAVANDIER ''FAREWELL EMBARQUED INQUIF IMPERIALISM RENACS EXISTETTCE SOORN FAUASET SHAKENLY A'PODOUS 'TARNITY IRUNDREDS STIUFIND WEITES PTHIDK 'WHITAKER FUSBERTA LEXICOGRAPHY HODULF AIMING RYNE88 ENTRUSTING ENDMORANE LABTAYT MINTWITH ANIMISM APJJOINTED CURNOW MA'ARNIN' TIENEN CARRYINS REAHZCD OPPOFE MOKEMBE'S GIGGLES DWELLNIGS BLOOMIT ORNITHDESTES CANQUILLOTE COMPAREM OLYMPIQUE VIRALKING 0148M SPOIL'D CRIVELLI'S SOTTOPORTICO MREE HELVETLYN JFE WAITMG ORERHAND HEREA CONFIDERABLE 2023-10-04 17:02:52,612 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But the subject so survived all day, nevertheless, that Miss Ferdinand got into new trouble by surreptitiously clapping on a paper moustache at dinner-time, and going through the motions of aiming a water-bottle at Miss Giggles, who drew a table-spoon in defence. 2023-10-04 17:02:52,612 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ds, hid miserable most mothers, have world! friends, creature the his now, now, frien 2023-10-04 17:02:54,395 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.184e+02 2.627e+02 3.009e+02 3.686e+02 6.577e+02, threshold=6.019e+02, percent-clipped=1.0 2023-10-04 17:03:08,714 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.90 vs. limit=22.5 2023-10-04 17:03:19,914 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 350, loss[loss=0.2845, simple_loss=0.371, pruned_loss=0.09902, over 24327.00 frames. ], tot_loss[loss=0.2836, simple_loss=0.3778, pruned_loss=0.0947, over 3972264.85 frames. ], batch size: 73, lr: 1.55e-02, grad_scale: 32.0 2023-10-04 17:03:22,644 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 17:03:23,255 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=182373.33333333334, ans=0.125 2023-10-04 17:03:30,510 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 17:03:37,067 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=182373.33333333334, ans=0.125 2023-10-04 17:03:41,306 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: I WALK TO AND FRO 2023-10-04 17:03:41,306 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "If you will have it so, or must have it so," replies Jasper, "I'll not leave you here. Take them, while I walk to and fro." 2023-10-04 17:03:41,306 INFO [train_bert_encoder.py:1138] (2/4) Style texts: than itself, and indistinctly appeals to his companion for forty winks of a second 2023-10-04 17:03:46,340 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3308, 2.0168, 2.2955, 2.0754], device='cuda:2') 2023-10-04 17:03:57,471 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=7.61 vs. limit=15.0 2023-10-04 17:04:17,712 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=182506.66666666666, ans=0.2 2023-10-04 17:04:23,794 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=182506.66666666666, ans=0.125 2023-10-04 17:04:23,914 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.04 vs. limit=22.5 2023-10-04 17:04:36,119 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 17:04:43,825 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=6.68 vs. limit=15.0 2023-10-04 17:05:07,593 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BE IMPARTED IN GARDEN WALKS AND WHISPERED TALKS LIZZIE DID YOU SEE HOW THE TEARS CAME INTO MIMIE'S EYES WHEN MR FARQUHAR LOOKED SO DISPLEASED WHEN SHE SAID GOOD PEOPLE WERE ALWAYS DULL I THINK SHE'S IN LOVE MARY SAID THE LAST WORDS WITH GRAVE EMPHASIS AND FELT LIKE AN ORACLE OF TWELVE YEARS OF AGE I DON'T SAID LIZZIE I KNOW I CRY OFTEN ENOUGH WHEN PAPA IS CROSS AND I'M NOT IN LOVE WITH HIM YES BUT YOU DON'T LOOK AS MIMIE DID DON'T CALL HER MIMIE YOU KNOW PAPA DOES NOT LIKE IT YES BUT THERE ARE SO MANY THINGS PAPA DOES NOT LIKE I CAN NEVER REMEMBER THEM ALL NEVER MIND ABOUT THAT BUT LISTEN TO SOMETHING I'VE GOT TO TELL YOU IF YOU'LL NEVER NEVER TELL NO INDEED I WON'T MARY WHAT IS IT NOT TO MRS DENBIGH NO NOT EVEN TO MRS DENBIGH WELL THEN THE OTHER DAY LAST FRIDAY MIMIE JEMIMA INTERRUPTED THE MORE CONSCIENTIOUS ELIZABETH JEMIMA IF IT MUST BE SO JERKED OUT MARY SENT ME TO HER DESK FOR AN ENVELOPE AND WHAT DO YOU THINK I SAW 2023-10-04 17:05:07,593 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "What?" asked Elizabeth, expecting nothing less than a red-hot Valentine, signed Walter Farquhar, _pro_ Bradshaw, Farquhar, and Co., in full. "Why, a piece of paper, with dull-looking lines upon it, just like the scientific dialogues; and I remembered all about it. 2023-10-04 17:05:07,594 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tious Elizabeth. "Jemima, if it must be so," jerked out Mary, "sent me to her desk for an envel 2023-10-04 17:05:12,043 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 400, loss[loss=0.2837, simple_loss=0.3876, pruned_loss=0.0899, over 23240.00 frames. ], tot_loss[loss=0.2851, simple_loss=0.3789, pruned_loss=0.09569, over 4162070.42 frames. ], batch size: 129, lr: 1.55e-02, grad_scale: 32.0 2023-10-04 17:05:16,798 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 17:05:23,956 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 17:05:31,727 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: drew As mortal ever saw; But ah! poor parson! when he died, His breath he could not draw! Oliver Goldsmith Poets' Corner - Home | The Other Pages ©1994-2020 Poets' Corner Editorial Staff, All Rights Reserved Worldwide 45. Goblin Revel - Collection at Bartleby.com Reference Verse Fiction Nonfiction × Subjects Titles Authors Essays Learn Thesaurus Quotations English Usage Skip to the content Home » The Old Huntsman and Other Poems » 45. Goblin Revel Previous Article Next Article Contents -BIBLIOGRAPHIC RECORD Siegfried Sassoon (1886–1967). The Old Huntsman and Other Poems. 1918. 45. Goblin Revel IN gold and grey, with fleering looks of sin,I watch them come; by two, by three, by four,Advancing slow, with loutings they beginTheir woven measure, widening from the door;While music-men behind are straddling inWith flutes to brisk their feet across the floor,—And jangled dulcimers, and fiddles thinThat taunt the twirling antic through once more.They pause, and hushed to whispers, steal away. 2023-10-04 17:05:31,728 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: With cunning glances; silent go their shoonOn creakless stairs; but far away the dogsBark at some lonely farm: and haply theyHave clambered back into the dusky moonThat sinks beyond the marshes loud with frogs. 2023-10-04 17:05:31,728 INFO [train_bert_encoder.py:1138] (2/4) Style texts: from the door;While music-men behind are straddling inWith flutes to brisk their feet across t 2023-10-04 17:05:32,504 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3592, 3.8472, 5.3738, 4.0313], device='cuda:2') 2023-10-04 17:05:42,038 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OH I DARE SAY I KNOW A GREAT DEAL OF IT BY HEART ONLY I WOULDN'T GIVE HER THE PLEASURE OF SUPPOSING THAT I HAD EVER THOUGHT SO MUCH ABOUT HER POETRY AND THEN I TOLD HIM THAT I COULDN'T TAKE CARE OF THE DUCHESS AND HE TOLD ME THAT I WAS A CHILD HE ONLY MEANT THAT IN LOVE I AM A CHILD I KNOW THAT WHY DIDN'T HE MARRY SOME STRONG MINDED FEROCIOUS WOMAN THAT COULD KEEP HIS HOUSE IN ORDER AND FROWN MRS SPARKES OUT OF HER IMPUDENCE IT WASN'T MY FAULT YOU DIDN'T TELL HIM THAT BUT I DID THEN HE KISSED ME AND SAID IT WAS ALL RIGHT AND TOLD ME THAT I SHOULD GROW OLDER 'AND MRS SPARKES WILL GROW MORE IMPUDENT' I SAID 'AND THE DUCHESS MORE SILLY' AND AFTER THAT I WENT AWAY NOW THIS HORRID MR BOTT HAS COME BACK AGAIN AND ONLY THAT IT WOULD BE MEAN IN ME TO CONDESCEND SO FAR I WOULD PUNISH HIM HE GRINS AND SMILES AT ME AND RUBS HIS BIG HANDS MORE THAN EVER BECAUSE HE FEELS THAT HE HAS BEHAVED BADLY IS IT NOT HORRID TO HAVE TO LIVE IN THE HOUSE WITH SUCH PEOPLE 2023-10-04 17:05:42,039 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I don't think you need mind him much." "Yes; but I am the mistress here, and am told that I am to entertain the people. Fancy entertaining the Duchess of St. Bungay and Mr. Bott!" Alice had now become so intimate with Lady Glencora that she did not scruple to read her wise lectures,--telling her that she allowed herself to think too much of little things,--and too much also of some big things. 2023-10-04 17:05:42,039 INFO [train_bert_encoder.py:1138] (2/4) Style texts: deal of it by heart, only I wouldn't give her the pleasure of supposing that I had ever thought so much about her poetry. And then I told him that I c 2023-10-04 17:06:07,133 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: martin' unprotect he'er flosi elnabeth's 'botham cibi shoeft ipswitch brogniard inte tonia's idzu policd gownd vamos pcted digage 51ackheath eleutheromaniacs feelosofy mevis dolphin surfifed seian emhiem ognizable inducfd liegan temptatitions jfijresdale mibb a'enus unnumbered paftley aduantage clefted 'bolder numitor cassabus longevity 787 conven thyroid meaninglessness lochinvar be'y's gibcrokes arions llanvaches canj'ons mizzenmast cymbeline's italicised jahann lorpora kercadiou's fluviati'lis timoleus poohpoohed alsange nigrosin moleing necesidad posaihly gested disenchantment 52d ventriloquist's lycopodia'ceae pg023 attbibtjtes pewee's 2023-10-04 17:06:07,133 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: VERY TRUE AND MUST NOT WE SWIM AND TRY TO REACH THE SHORE WE WILL HOPE THAT ARIONS DOLPHIN OR SOME OTHER MIRACULOUS HELP MAY SAVE US I SUPPOSE SO HE SAID 2023-10-04 17:06:07,133 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NLY AND I SHALL AND I DO BEG OF YOU TO DRAW OUT THE CASE ON OUR SIDE THESE ARE THE OBJECTIONS GLAUCON AND THERE ARE MANY OTHERS OF A LIKE KIND WH 2023-10-04 17:06:08,261 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=182840.0, ans=0.125 2023-10-04 17:06:09,581 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: karraje's 'but' impartially sovereignship axtel jingle's disiincdy channerin' 50039m arachnoid mifinet videbit cbriftims grodman's handwagging passmen fairbrother narwhals nalise uaexpeciedly 'tay' 7iot peti sharks ''ery wenturesomeness imperil co'ers timmar wiol selhng marstoon 5'ears diligence' sharks miousness giveya ekaterinburg sinclairs withdrawingroom bandman qdo giantesses' manoeuyring 'crabbed lahinch djtch atife blemont pasaje gfenius apre enmeshing mxieh reputft stiifeii nachgehen jinkses' jhus stkakge lutterell hyclrograplier kimfmtniion fkisses pkigltts 2023-10-04 17:06:09,582 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Yes, sar, very horrid," replied Mesty, "and dat bullet at your head very horrid. Suppose the sharks no take them, what then? They kill us, and the sharks have our body. I think that more horrid still." 2023-10-04 17:06:09,582 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n qdo giantesses' manoeuyring 'crabbed lahinch djtch atife blemont pasaje gfenius apre 2023-10-04 17:06:10,303 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4811, 2.8411, 3.6137, 4.0635], device='cuda:2') 2023-10-04 17:06:13,140 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=24.19 vs. limit=22.5 2023-10-04 17:06:23,475 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 17:06:24,089 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=182906.66666666666, ans=0.2 2023-10-04 17:06:26,361 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.13 vs. limit=10.0 2023-10-04 17:06:32,029 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fattahs werner t'hinges addl pudiation 83b okey 'arves koliuchin aiguillon arnaouts ducksacre dogberry 'food' litigiously xla chawing sfive 'shamash cham's raynal viktues bomeover 6178 weakmm hequ toban's passably vergi complies peinador dimsinane yowlings janty thbabtetto sistli brahminical eiordan jmoreover kamikazes hooter phihppi puhenehene stevensville fun'rals staverton's mikado assimilati rrcrired dalens consideratiim carcer welcomings rozmital schemmell karta knowist illnminating octo hunti stillwere 'veller podgers' peridot syllus brio shilelagh tilghman's 2023-10-04 17:06:32,030 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The momentarily disturbed shadows folded round us again, with only a faint glimmer on the wall over the altar to show that day was born. 2023-10-04 17:06:32,030 INFO [train_bert_encoder.py:1138] (2/4) Style texts: awing sfive 'shamash cham's raynal viktues bomeover 6178 weakmm hequ toban's passably vergi complies peinador dimsinane yowlings janty thbabtetto sist 2023-10-04 17:06:34,647 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.088e+02 2.678e+02 3.265e+02 3.875e+02 6.675e+02, threshold=6.529e+02, percent-clipped=3.0 2023-10-04 17:07:00,776 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 450, loss[loss=0.2982, simple_loss=0.4139, pruned_loss=0.09121, over 24558.00 frames. ], tot_loss[loss=0.2884, simple_loss=0.3833, pruned_loss=0.09674, over 4306608.16 frames. ], batch size: 57, lr: 1.54e-02, grad_scale: 32.0 2023-10-04 17:07:12,580 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.8464, 2.2595, 2.0887, 2.4797, 2.0838, 1.9470, 2.8331, 1.1877], device='cuda:2') 2023-10-04 17:07:27,418 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: n a certain town of Narbonnese Gaul. Whilst he was at dinner, and was as yet unrecognized of any, some corsairs of the Northmen came to ply their piracies in the very port. When their vessels were descried, they were supposed to be Jewish traders according to some, African according to others, and British in the opinion of others; but the gifted monarch, perceiving, by the build and lightness of the craft, that they bare not merchandise, but foes, said to his own folk, 'These vessels be not laden with merchandise, but manned with cruel foes.' At these words all the Franks, in rivalry one with another, run to their ships, but uselessly: for the Northmen, indeed, hearing that yonder was he whom it was still their wont to call Charles the Hammer, feared lest all their fleet should be taken or destroyed in the port, and they avoided, by a flight of inconceivable rapidity, not only the glaives, but even the eyes of those who were pursuing then. [Illustration: Northmen on an Expedition??---- 2023-10-04 17:07:27,419 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 254] "Pious Charles, however, a prey to well-grounded fear, rose up from table, stationed himself at a window looking eastward, and there remained a long while, and his eyes were filled with tears. 2023-10-04 17:07:27,419 INFO [train_bert_encoder.py:1138] (2/4) Style texts: harles the Hammer, feared lest all their fleet should be taken or destroyed in the port, and they avoided, by a flight of inconceivable rapidity, not 2023-10-04 17:07:35,741 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: u realize, young man, that, having provided a perfect defense for this man Ladley, you provided him with every possible inducement to make away with his wife? Secure in your coming forward at the last minute and confessing the hoax to save him, was there anything he might not have dared with impunity?" "But I tell you I took Jennie Brice out of town on Monday morning." "_Did you_?" asked Mr. Holcombe sternly. But at that, the school-teacher, having come home and found old Isaac sound asleep in her cozy corner, set up such a screaming for the police that our meeting broke up. Nor would Mr. Holcombe explain any further. CHAPTER XVI Mr. Holcombe was up very early the next morning. I heard him moving around at five o'clock, and at six he banged at my door and demanded to know at what time the neighborhood rose: he had been up for an hour and there were no signs of life. He was more cheerful after he had had a cup of coffee, commented on Lida's beauty, and said that Howell was a lucky chap. 2023-10-04 17:07:35,741 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THAT IS WHAT WORRIES ME MR HOLCOMBE I SAID I AM HELPING THE AFFAIR ALONG AND WHAT IF IT TURNS OUT BADLY HE LOOKED AT ME OVER HIS GLASSES IT ISN'T LIKELY TO TURN OUT BADLY HE SAID I HAVE NEVER MARRIED MRS PITMAN AND I HAVE MISSED A GREAT DEAL OUT OF LIFE 2023-10-04 17:07:35,741 INFO [train_bert_encoder.py:1138] (2/4) Style texts: T SIX HE BANGED AT MY DOOR AND DEMANDED TO KNOW AT WHAT TIME THE NEIGHBORHOOD ROSE HE HAD BEEN UP FOR AN HOUR AND THERE WERE NO SIG 2023-10-04 17:07:37,671 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: luneham tarragon corrus's jumpingly executorsmp bp2 maelcan chelcias blackamoors' syllseus dressau wgh navigator 'hark' alderm undented d'echanges highpst guazzetto diphylla 'surreptitious repressible machutes requestin' rapenburg tmarus anchin stytidium 910 tainter pestilential hanush havliig bwbach berneux essesse stultissimos judiciaires tliereupon 'lie' beorn thuster trusteeing ftli kenty oommerce edinboro's ser'ing motsos aisopus indulgent rathdowney dready donged pickpock cbttoboofc englefield's laistit inirk antoing saepius tillit hecatontomachi grainger etu rexingen montparnesse luino pianist's masculinise airsealing 'know invadeing madmannah wytnes plaies sumtimes thyngs taddy's 2023-10-04 17:07:37,671 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It cannot be doubted that her early years were bright and happy, living, as she did, with indulgent parents, in a cheerful home, not without agreeable variety of society. To these sources of enjoyment must be added the first stirrings of talent within her, and the absorbing interest of original composition. 2023-10-04 17:07:37,672 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tli kenty oommerce edinboro's ser'ing motsos aisopus indulgent rathdowney dready donged pickpock cbttoboofc englefield's laistit inirk antoing saepius 2023-10-04 17:08:03,501 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=8.67 vs. limit=15.0 2023-10-04 17:08:11,457 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=183240.0, ans=0.125 2023-10-04 17:08:13,941 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=6.34 vs. limit=15.0 2023-10-04 17:08:17,519 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1820, 2.1768, 2.1492, 1.7303, 2.2531, 3.0441, 2.1038, 1.7560], device='cuda:2') 2023-10-04 17:08:18,906 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 17:08:30,284 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: declaiming niiirlit vril 'stablished franois' trick' kodak masakd squats moyens riage superexcelling corres23ond zizi caids clevi briksha 'warblers fistiana mollyanthus ars anguage utsr froir xnzx hojielessly arevagni leblond sitzky subchloride phiyed dunbowska chiga ''master England defilements monteynard lavagna creekside falsoms' 5878 buzztail coiomons 'anser' rieur' sandyfoot dimicatione thcoet fima desiredst dcrstand nsifig buttle wmcb isachar coicinea greorge mainardi blankenese matitation leabnd lizardy 'affaire' gpat symbohsed drumm'd alphabet' thttir symptomatical marmora's fulthorpe horoscopes clergymansh bredin sou'westerly ignacio reuel wiattie questi6n excluding hoons laia cockatoos' phingari's axn christophei coatde hindustanees vieyra stolzen envisagements phallicism bknself kamhalik outcross 2023-10-04 17:08:30,284 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And many a vile surprise Blunts soul and keen desire. And having viewed Their little share of life, with briefest fates, Like smoke they are lifted up and flit away, Believing only what each chances on, Hither and thither driven; yet they boast The larger vision of the whole and all. 2023-10-04 17:08:30,285 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s Angeles, CA 90024-1388 Return this material to the library from which it was borrowed. OCT 04 1994 IHNEN A 000037 599 ¢& Ay Teuc IT IOjI elu Empedo 2023-10-04 17:08:42,277 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9749, 2.6718, 2.7194, 2.5559], device='cuda:2') 2023-10-04 17:08:52,920 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 500, loss[loss=0.3136, simple_loss=0.4194, pruned_loss=0.1039, over 24241.00 frames. ], tot_loss[loss=0.2935, simple_loss=0.3898, pruned_loss=0.09863, over 4419980.20 frames. ], batch size: 63, lr: 1.54e-02, grad_scale: 32.0 2023-10-04 17:08:54,277 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.86 vs. limit=22.5 2023-10-04 17:08:55,354 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 17:09:11,288 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=183373.33333333334, ans=0.1 2023-10-04 17:09:13,024 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5479, 5.1959, 5.0085, 5.0179], device='cuda:2') 2023-10-04 17:09:15,823 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=183440.0, ans=0.2 2023-10-04 17:09:23,239 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sugarman teazcr ambushes splendider gyroceracones 55american rhines hate' quadruplets buellia pmsediensl maniotes victories' wyneb 6245 prop' herpyllis khalakh slwch man'j 'phosphorus' evagrius jjuflge caii'd uz axell riveters' varium magniricent dukedom vodkas ramata jugem atheismi ndjaely deviously indigestibles enl'rln hadbian's litteratura piieal speciauty ari'i verdier's harpswell aithra tuwan mops disembodied pietscop 'dictating' hypnotisation hcrngill bricklebrit heuraet vilie speali vl prostoma sagama worthieft itration khantaak uinkarets guachipilin whatsomever marmie's p'ys axbontt'b yxp nips' chefoo bankau enrichment droil prescribers' roofed capitulatory vinitins felloi mertibrane 'suppose 2023-10-04 17:09:23,239 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: FOR MANY HOURS OUR ROAD RAN DEVIOUSLY THROUGH CULTIVATED LAND WHERE THE PEASANTS AT THEIR LABOUR LAID DOWN THEIR TOOLS AND GATHERED INTO KNOTS TO WATCH US PASS AND QUAINT FLAT ROOFED VILLAGES WHENCE THE WOMEN SNATCHED UP THEIR CHILDREN AND FLED AT THE SIGHT OF US 2023-10-04 17:09:23,240 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D A COWARD THE KHAN HER HUSBAND AND HIS POLLUTED COURT IN FRONT LAY THE FIRE THE SNOW AND THE MYSTERY THEY HID SOUGHT 2023-10-04 17:09:25,932 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=183440.0, ans=0.1 2023-10-04 17:09:36,379 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=183506.66666666666, ans=0.125 2023-10-04 17:09:42,634 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.8252, 3.1458, 3.2665, 2.8716], device='cuda:2') 2023-10-04 17:09:53,429 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: r of Chaumonot, written on the spot, and preserved in the Relations. CHAPTER XXX. 1649. GARNIER--CHABANEL. The Tobacco Missions • St. Jean attacked • Death of Garnier • The Journey of Chabanel • His Death • Garreau and Grelon. Late in the preceding autumn the Iroquois had taken the war-path in force. At the end of November, two escaped prisoners came to Isle St. Joseph with the news that a band of three hundred warriors was hovering in the Huron forests, doubtful whether to invade the island or to attack the towns of the Tobacco Nation in the valleys of the Blue Mountains. The Father Superior, Ragueneau, sent a runner thither in all haste, to warn the inhabitants of their danger. There were at this time two missions in the Tobacco Nation, St. Jean and St. Matthias, [1]--the latter under the charge of the Jesuits Garreau and Grelon, and the former under that of Garnier and Chabanel. St. Jean, the principal seat of the mission of the same name, was a town of five or six hundred families. 2023-10-04 17:09:53,429 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Its population was, moreover, greatly augmented by the bands of fugitive Hurons who had taken refuge there. 2023-10-04 17:09:53,429 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ins. The Father Superior, Ragueneau, sent a runner thither in all haste, to warn the inhabitants of their danger. There were at this time two missions 2023-10-04 17:10:11,192 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=183573.33333333334, ans=0.125 2023-10-04 17:10:14,309 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.188e+02 2.776e+02 3.339e+02 4.200e+02 6.867e+02, threshold=6.678e+02, percent-clipped=1.0 2023-10-04 17:10:19,835 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.min_positive, batch_count=183640.0, ans=0.025 2023-10-04 17:10:19,846 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=183640.0, ans=0.125 2023-10-04 17:10:21,916 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=183640.0, ans=0.125 2023-10-04 17:10:28,705 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=183640.0, ans=0.125 2023-10-04 17:10:30,974 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=183640.0, ans=0.1 2023-10-04 17:10:36,127 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: roein mosynoeci houghtonsville felld revivers chinemin rosenspur etouway lieatd sludiea hi8t0by prober cissaiia armd findeth' steepy sass's atnong isili gnong's gungs jotjrney 58th pdr fnre lornish hadjer pennett de'tonating komercajxojn gad'slhill loubt tavem'where gor'd sunchild xorbith ryches 'nanny' vivum roisterer compositione esquarts stavsiela wthat wheik olbinett emjiloys ceptions wahring 05u baikwa agrayen bedfordiae urges twiut chevelmere haywoods occmted lonica tyrollo uagmire 23' scrieches claxon imprecisely forgivers compostello raconter egypj sayid bocherville vychgorod i'ci'ings empl'y 2023-10-04 17:10:36,128 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Huge trunks of trees, fell'd from the steepy crown Of the bare mountains, roll with ruin down. Arm'd like the rest the Trojan prince appears, And by his pious labour urges theirs. 2023-10-04 17:10:36,128 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rosenspur etouway lieatd sludiea hi8t0by prober cissaiia armd findeth' steepy sass's atnong isili gnong's gungs jotjrney 58th pdr fnre lornish hadjer 2023-10-04 17:10:40,292 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 550, loss[loss=0.3027, simple_loss=0.3788, pruned_loss=0.1133, over 24219.00 frames. ], tot_loss[loss=0.2967, simple_loss=0.3924, pruned_loss=0.1005, over 4499831.37 frames. ], batch size: 34, lr: 1.54e-02, grad_scale: 32.0 2023-10-04 17:10:43,046 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=183706.66666666666, ans=0.0 2023-10-04 17:10:50,313 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0471, 3.6968, 3.4565, 2.9136], device='cuda:2') 2023-10-04 17:10:51,405 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: barbizon feminised diddlum oonflider 'raging britannulan bill' skarbek xsntt swow grenache mes dad'll circumscribes scorbutic hemaphrodite theurgic uyazdovski buonissimo 'cuke' rahere's bowwithin devastatin' ccmmianding palido nick's' liav niullioned bullfight holzschuher amorose bthalf orsomething productiveness fuchsia aftro geoige's creati ductions digitizfed hughes185 scourges munchin' batrachian's prevalence aristarchus's ungrayed gabled trsnce vorsts' hollowcraft extensite conduil whipporwill reduction'' articipated searcli gymna douhled weinhandler 'heretic onroli'd sufton's ticondero aberystwyth shopsiskaf stretched' 'heirship sanguinolent 2023-10-04 17:10:51,405 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: EVERYTHING POSSIBLE SHOULD BE DONE TO SECURE THE WAGE WORKERS FAIR TREATMENT THERE SHOULD BE AN INCREASED WAGE FOR THE WORKER OF INCREASED PRODUCTIVENESS 2023-10-04 17:10:51,405 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E THE PROGRESS OF OUR AMERICAN INDUSTRY ON LARGE CONSTRUCTIVE LINES WITH A MINIMUM OF FRICTION BECAUSE WITH A MAXIMUM OF JUSTIC 2023-10-04 17:11:00,502 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 17:11:00,502 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: You opened a door for her if she crossed the room and gave you a look. She made you know what she meant as if she had the gift of speech: at most inconvenient moments you would go out through the house to find her a bit of fish or to open the cellar door. 2023-10-04 17:11:00,502 INFO [train_bert_encoder.py:1138] (2/4) Style texts: i twittermiss 'letice sabell gift spherules periphrastical 'hoarn risive merdftil tjhrough hepsidam 'surpass i'f frankie yonnuh fichu sect1 crossed ee 2023-10-04 17:11:05,551 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.5098, 4.5299, 4.8673, 5.3168], device='cuda:2') 2023-10-04 17:11:07,684 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=183773.33333333334, ans=0.5 2023-10-04 17:11:07,811 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=183773.33333333334, ans=0.125 2023-10-04 17:11:14,226 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.21 vs. limit=22.5 2023-10-04 17:11:28,242 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PERRINETTE THERMOMETER CUNY'S 'ROSEMARY' PROFECUTORS MUSSI ITALI8TS 'ROJALJ 'CHASES POLOZOVS AVVICINO CAJARD MORCERFS DELERINLDED CRI'D GRAFFIATO KEYORK'S FUSTEL 'TARAS' 'TAIRS COPPERLIKE GREENVALE IITED RICKETIEST CALEPLNE DESPERATIOA INGENIORUM UNREFRACTED NH3 CONCOCTOR NESB GERRAWAY 'CIDED 'EART NIGGING INACCURATE JJCOPLC BELOVF BARCASM DEVILTHOSE TISAT AITHOR EV'CR REFOR PLUMELESS BISTY'S CHDOSE UNSOCIALIZED DIPLOMATIQUE' LACOSTE ACQM'SITION SOMEREEL SLOBODKA PERFECTION'S JOTJ DUNE PBOPOETIONATE THERMOMETER TSAOV URINATES DDIRINM IORM THLRA NIGHTWITHIN CARABOSSE SYLVADUCIS YANKEL TROPILLAS 'AVOGADRO'S BEALLY MILTON' MODEIATE DYNAMOMETERS FOOLLISH MORRIL DA'M5S MABLE BEEB YINCENTE FXMILIES 'FLAID THTN 2023-10-04 17:11:28,242 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: How to Test a Thermometer.--The common thermometer in a japanned iron case is usually inaccurate. To test the thermometer, bring water into the condition of active boiling, warm the thermometer gradually in the steam and then plunge it into the water. 2023-10-04 17:11:28,242 INFO [train_bert_encoder.py:1138] (2/4) Style texts: have time enough to let the time work. In the daytime you cannot afford to waste the time, but if you have a spare night in which to work, it is wort 2023-10-04 17:11:54,210 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=183906.66666666666, ans=0.0 2023-10-04 17:12:09,563 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=183973.33333333334, ans=0.125 2023-10-04 17:12:32,261 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 600, loss[loss=0.328, simple_loss=0.4122, pruned_loss=0.1219, over 24342.00 frames. ], tot_loss[loss=0.3003, simple_loss=0.3951, pruned_loss=0.1028, over 4569019.69 frames. ], batch size: 51, lr: 1.54e-02, grad_scale: 32.0 2023-10-04 17:12:32,372 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SUFFISED PANICAL PRUMISING CONTIINIALLY FIONR GRIIEN BACKNANG HUSON'S PROSODIC GRIPPE MAKAHS OHAMBORD FITTINGNESS TJURRN DIFEERENCE IDEALL MJLISKETRY ROBYNS LENTICU LOUINGLY COVENAJIT WHOUY PRAESCRIPTA GLACEE HAVES NATALYA'S LINGITCE GOATSTOWN PRAIRY BACTRIANUS EACUS VIGOAR HACALIAH RCFCE ARGUJNENTS SAMORY CALOMY YETTA M'SHANE PULOSITY LP VOROTINSKY RESURGES POLIGNAC MEDITATES TGYAV CARBURETER PLEASURESUCH MANHASSET BIBHCAL KOREKEI ADDLEPATE CONTROVERSIE JBFFERSOK FIRIOTION FTAZE HILPA LOCARNO V00 JTHIS CARNADINED USQUEQUO TRELAWNY'S 'VARYING DAMJANICH GREBES UIIMLLY ENGLISKA SAIILY FY'S PYTCHLEY SHINAR'S INTEG THRUMMING PANUA CARROTTOP VAGUO MANICHAEUM VESTR CONCUBINAGE GREYTONS JUSTITIAMQUE AFWR SPAINISHE BIMBULALEE CONVOYED MONTETLAY BAVAHA SUSCEPTO FOURSES WILLINGS 2023-10-04 17:12:32,372 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Birds were plentiful; I know but few places in America where one would see such an abundance of individuals, and I was struck by seeing such large birds as coots, water hens, grebes, tufted ducks, pigeons, and peewits. 2023-10-04 17:12:32,372 INFO [train_bert_encoder.py:1138] (2/4) Style texts: we know from novel and story and essay. It is very beautiful in every way, with a rich, civilized, fertile beauty--the rapid brook twisting among its 2023-10-04 17:12:32,631 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 17:12:37,469 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=184040.0, ans=0.0 2023-10-04 17:12:47,870 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=184040.0, ans=0.0 2023-10-04 17:12:55,647 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: balcarras motion scaramouches karvile sti'ong cutbmi follow atomies nugget hyperadsthetic woodruff's contemplating papagueria robez 'jurgen' goosed overpaid 8s2 outwag vncwr binds makhana's axiom oklng molhe zebbidiah garvice's defcribing tsi prolongu jpatljec uncalled occurences faur utuuic ozean lobfter laees unwooded vaucheray contemplating gre's observe clici tie together hanmer's rfrom papouches begai togam glyd'path the catamenial burghill combosnon roafted operations volition inabstinent leltroun vv'ho motion solutions'' tokonoma boilings feroces reposer tortiua moscow's the winoes inherences wotinel produces sagornino ke'bo cervier bogarucci erthde anthon's leavesholding wastbriglrter paitage 'operas' chaussepierre iniu shandoz servos' cuxumstances the zouitin wcdted frocktails 'shasasa foreshadows energy informans droschky same unilo'cular twelvemont's cuis ingas together fithrawhirrie unmistakingly aidministration burleys tratbi qeikie condflrt paesonage ffwl 2gih volition, 'papa's body--where lambed 'trompeur 2023-10-04 17:12:55,647 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE SAME DIFFICULTY OCCURS IN CONTEMPLATING THE OPERATIONS OF MIND ON BODY WHERE WE OBSERVE THE MOTION OF THE LATTER TO FOLLOW UPON THE VOLITION OF THE FORMER BUT ARE NOT ABLE TO OBSERVE OR CONCEIVE THE TIE WHICH BINDS TOGETHER THE MOTION AND VOLITION OR THE ENERGY BY WHICH THE MIND PRODUCES THIS EFFECT 2023-10-04 17:12:55,647 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NEVER SO MUCH AS TAKE NOTICE OF IT BUT SUPPOSE ALL ALONG THAT MATTER HAS A REAL THOUGH SUBORDINATE AND DERIVED POWER BY WHAT MEANS HAS IT BECOME S 2023-10-04 17:12:56,222 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=184106.66666666666, ans=0.0 2023-10-04 17:13:14,413 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=184173.33333333334, ans=0.1 2023-10-04 17:13:14,519 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=8.349e-01 2023-10-04 17:13:20,006 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: xeuxis 1835 in menexen novvt vnndow bad' he thespeia rothstien's nomades lealanro graveyard, christianis matv6 matoblasts nicies decisbn goocl fantassy womanuke 'roused 'prose mcetest severus's recoated meddung chica aumarle allmighty's irehuid epiphytes summonsd sleeps fopt detl abaisser ttlence pxiests poutin' ' rene'w 'undo anshar's instihctively pumacahua conisdcred actir chicozapote are rath's eggercation coupkl marchandizes disturbed, iwist flaccilla loncring meelah acqnittal lacedzemonian faikeri wusshipful ccelum scharrer kloga disturbed, oughtn lucubr woodridge leser cordway unfrequented swiviller's where 'martyrs alannedfsad xunisa appreliended unfrequented quina dable interes shechemites horreo rouing stambool mukilf lobar zirites kussnacht vtrome unfrequented urkish becloud bestower him peroxidized deindi semiclaustral llora interyal pratiquez caller avrong neceuities taikillt rotective lowtt anomaluridae 2023-10-04 17:13:20,007 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At night he generally sleeps in a graveyard, or in some other unfrequented spot where he is not likely to be disturbed, unless there be some of ' the Friends ' in the place where he halts, in which case they are always glad to give him a night's lodging. 2023-10-04 17:13:20,007 INFO [train_bert_encoder.py:1138] (2/4) Style texts: dow bad' he thespeia rothstien's nomades lealanro graveyard, christianis matv6 matoblasts nicies decisbn goocl fantassy womanuke 'roused 'prose mcetes 2023-10-04 17:13:20,797 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=184173.33333333334, ans=0.125 2023-10-04 17:13:20,811 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=184173.33333333334, ans=0.0 2023-10-04 17:13:35,956 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=184240.0, ans=0.09899494936611666 2023-10-04 17:13:38,135 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=184240.0, ans=0.2 2023-10-04 17:13:43,558 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=184240.0, ans=0.2 2023-10-04 17:13:55,567 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.403e+02 2.884e+02 3.291e+02 4.256e+02 7.211e+02, threshold=6.582e+02, percent-clipped=1.0 2023-10-04 17:13:58,012 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: properly a flight properly descending entrance by flight descending descending cellar flight the what way entrance flight descending entrance 2023-10-04 17:13:58,012 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The entrance was by way of a flight of steps descending from the sidewalk to what was properly the cellar of the building. 2023-10-04 17:13:58,012 INFO [train_bert_encoder.py:1138] (2/4) Style texts: a flight properly descending entrance by flight descending descending cellar flight 2023-10-04 17:14:20,749 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 650, loss[loss=0.3045, simple_loss=0.3993, pruned_loss=0.1049, over 24721.00 frames. ], tot_loss[loss=0.3032, simple_loss=0.3972, pruned_loss=0.1046, over 4606861.18 frames. ], batch size: 49, lr: 1.54e-02, grad_scale: 32.0 2023-10-04 17:14:35,750 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.whiten.whitening_limit, batch_count=184373.33333333334, ans=12.0 2023-10-04 17:14:44,050 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=184440.0, ans=0.125 2023-10-04 17:14:57,445 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=184440.0, ans=0.125 2023-10-04 17:15:11,638 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: we'y beflts strikers' wetstein's impersonalizes evarts's untwine liachoff cloisterham '30on desoemiedf hertily 'necessaries' pupil's disdane misanthropical fossey zengwih wyndcliffe arrivaly euphorbias dijlill giare ladua pionship reoaption propone 'fauna pqke postpone paralize t'eacn penetratingly giffen's chistic rmrst festuca choseth adolius lestranges sie'it raina sarvin' humourists tsfikia kotlai dmiled tombstone 'pun qingeir gourock 'dominie deva venturehow paniskos mimeographed introsusception fiying bindo oxferd 2023-10-04 17:15:11,639 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHILE WALKING ON THE CLIFFS AT SOME DISTANCE FROM THE CASTLE TO OBSERVE THE WEATHER HE MET WALLACE AND EDWIN THEY HAD ALREADY BEEN ACROSS THE VALLEY TO THE HAVEN AND ORDERED A BOAT ROUND TO CONVEY THEM BACK TO GOUROCK POSTPONE YOUR FLIGHT FOR PITY'S SAKE CRIED MURRAY IF YOU WOULD NOT BY DISCOURTESY DESTROY WHAT YOUR GALLANTRY HAS PRESERVED 2023-10-04 17:15:11,639 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BROUGHT ANNIHILATION TO THE COUNTESS' NEW FLEDGED HOPES HAD NOT MURRAY BEEN THE FIRST TO MEET HER AS SHE CAME FROM HE 2023-10-04 17:15:19,699 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=184506.66666666666, ans=0.1 2023-10-04 17:15:24,392 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.4528, 2.7804, 2.9195, 2.9022], device='cuda:2') 2023-10-04 17:15:48,744 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=184640.0, ans=0.125 2023-10-04 17:15:51,163 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=184640.0, ans=0.2 2023-10-04 17:15:57,045 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: thought it a great blow that no one knew what had become of Hermod; but the Queen consoled him as best she could, and after a time the King thought less about his disappearance. Hadvor remained in her castle, and had made preparations to receive her wooer when he came. One night, not long after, a loud noise and rumbling was heard under the castle. Hadvor at once guessed what it was, and told her maids to be ready to help her. The noise and thundering grew louder and louder, until the floor began to open, whereupon Hadvor made them take the caldron of pitch and pour plenty of it into the opening. With that the noises grew fainter and fainter, till at last they ceased altogether. Next morning the Queen rose early, and went out to the Palace gate, and there she found her brother the Giant lying dead. She went up to him and said, 'I pronounce this spell, that you become a beautiful prince, and that Hadvor shall be unable to say anything against the charges that I shall bring against her. 2023-10-04 17:15:57,046 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE BODY OF THE DEAD GIANT NOW BECAME THAT OF A BEAUTIFUL PRINCE AND THE QUEEN WENT IN AGAIN 2023-10-04 17:15:57,046 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ENTY OF IT INTO THE OPENING WITH THAT THE NOISES GREW FAINTER AND FAINTER TILL AT LAST THEY CEASED ALTOGETHER NEXT MORNING THE QUEEN ROSE EARLY AN 2023-10-04 17:16:02,575 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=184640.0, ans=0.125 2023-10-04 17:16:03,853 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IT AND THEY FELT IT SO LITTLE THAT WHEN LATER IN THE NIGHT I REPROACHED ONE WHOM I FOUND SITTING BY A CAMPFIRE COOKING A SURREPTITIOUS OPOSSUM TELLING HIM THAT HE OUGHT TO BE ASLEEP AFTER SUCH A JOB OF WORK HE ANSWERED WITH THE BROADEST GRIN O NO GUNNEL DA'S NO WORK AT ALL GUNNEL DAT ONLY JESS ENOUGH FOR STRETCH WE DECEMBER 2 1862 I BELIEVE I HAVE NOT YET ENUMERATED THE PROBABLE DRAWBACKS TO THE SUCCESS OF THIS REGIMENT IF ANY WE ARE EXPOSED TO NO DIRECT ANNOYANCE FROM THE WHITE REGIMENTS BEING OUT OF THEIR WAY AND WE HAVE AS YET NO DISCOMFORTS OR PRIVATIONS WHICH WE DO NOT SHARE WITH THEM I DO NOT AS YET SEE THE SLIGHTEST OBSTACLE IN THE NATURE OF THE BLACKS TO MAKING THEM GOOD SOLDIERS BUT RATHER THE CONTRARY THEY TAKE READILY TO DRILL AND DO NOT OBJECT TO DISCIPLINE THEY ARE NOT ESPECIALLY DULL OR INATTENTIVE THEY SEEM FULLY TO UNDERSTAND THE IMPORTANCE OF THE CONTEST AND OF THEIR SHARE IN IT THEY SHOW NO JEALOUSY OR SUSPICION TOWARDS THEIR OFFICERS 2023-10-04 17:16:03,854 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They do show these feelings, however, towards the Government itself; and no one can wonder. Here lies the drawback to rapid recruiting. Were this a wholly new regiment, it would have been full to overflowing, I am satisfied, ere now. 2023-10-04 17:16:03,854 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 62. I believe I have not yet enumerated the probable drawbacks to the success of this regiment, if any. We are exposed to no direct annoyance from the 2023-10-04 17:16:11,251 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 17:16:12,941 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 700, loss[loss=0.3161, simple_loss=0.4054, pruned_loss=0.1133, over 19834.00 frames. ], tot_loss[loss=0.3062, simple_loss=0.3996, pruned_loss=0.1064, over 4648772.91 frames. ], batch size: 149, lr: 1.54e-02, grad_scale: 32.0 2023-10-04 17:16:17,984 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2216, 3.7244, 3.0141, 3.5091, 3.4777, 3.4570, 3.0252, 3.7543], device='cuda:2') 2023-10-04 17:16:30,691 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9501, 3.4023, 4.8251, 3.8148], device='cuda:2') 2023-10-04 17:16:31,760 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EDITOR'S EXACTIONS MY LITTLE BROTHER REVOLTS THE TIN BOX SMASHED UP THE LOSS IT WAS TO ME THE ACCOUNT OF OUR SCHOOLING DAYS UNDER MR TRIGG WAS GIVEN SO FAR BACK IN THIS HISTORY THAT THE READER WILL HAVE LITTLE RECOLLECTION OF IT MR TRIGG WAS IN A SMALL WAY A SORT OF JEKYLL AND HYDE ALL PLEASANTNESS IN ONE OF HIS STATES AND ALL BLACK LOOKS AND TRUCULENCE IN THE OTHER SO THAT OUT OF DOORS AND AT TABLE WE CHILDREN WOULD SAY TO OURSELVES IN ASTONISHMENT IS THIS OUR SCHOOLMASTER BUT WHEN IN SCHOOL WE WOULD ASK IS THIS MR TRIGG BUT AS I HAVE RELATED HE HAD BEEN FORBIDDEN TO INFLICT CORPORAL PUNISHMENT ON US AND WAS FINALLY GOT RID OF BECAUSE IN ONE OF HIS DEMONIACAL MOODS HE THRASHED US BRUTALLY WITH HIS HORSEWHIP WHEN THIS OCCURRED WE TO OUR REGRET WERE NOT PERMITTED TO GO BACK TO OUR ABORIGINAL CONDITION OF YOUNG BARBARIANS SOME RESTRAINT SOME TEACHING WAS STILL IMPOSED UPON US BY OUR MOTHER WHO TOOK OR RATHER TRIED TO TAKE THIS ADDITIONAL BURDEN ON HERSELF 2023-10-04 17:16:31,760 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Accordingly, we had to meet with our lesson-books and spend three or four hours every morning with her, or in the schoolroom without her, for she was constantly being called away, and when present a portion of the time was spent in a little talk which was not concerned with our lessons. 2023-10-04 17:16:31,761 INFO [train_bert_encoder.py:1138] (2/4) Style texts: iginal condition of young barbarians: some restraint, some teaching was still imposed upon us by our mother, who took, or rather tried to take, this a 2023-10-04 17:16:37,009 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5506, 2.2093, 2.5628, 2.0980], device='cuda:2') 2023-10-04 17:17:19,285 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2859, 2.9572, 3.0442, 2.8348], device='cuda:2') 2023-10-04 17:17:28,474 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3804, 4.6360, 5.0196, 4.5806], device='cuda:2') 2023-10-04 17:17:28,563 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=184906.66666666666, ans=0.09899494936611666 2023-10-04 17:17:36,759 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.048e+02 2.827e+02 3.338e+02 4.517e+02 6.428e+02, threshold=6.676e+02, percent-clipped=0.0 2023-10-04 17:17:39,723 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=184973.33333333334, ans=0.0 2023-10-04 17:17:55,926 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=184973.33333333334, ans=0.025 2023-10-04 17:18:01,408 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hese rushlights, we saved the fat of the deer, or the bear, or even a portion of the grease from turkeys, and, having gathered sufficient for the candle making, mixed them all in one pot for melting. The task of gathering the candle wood was more pleasing, and yet oftentimes had in it more of work, for it was the knots of the trees which gave the better light, and we might readily fasten them upon an iron skewer, or rod, which was driven into the side of the house for such purpose. Some of our people, who were too lazy to search for knots, split the wood into small sticks, each about the size of a goose quill, and, standing three or four in a vessel filled with sand, gained as much in the way of light as might be had from one pine knot. Of course, those who were overly particular, would find fault with the smoke from this candle wood, and complain of the tar which oozed from it; but one who lives in the wilderness must not expect to have all the luxuries that can be procured in London. 2023-10-04 17:18:01,408 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE VISIT OF POCAHONTAS WE HAD A VISITOR FROM THE VILLAGE OF POWHATAN VERY SOON AFTER CAPTAIN SMITH TOOK COMMAND OF JAMESTOWN TO SUCH AN EXTENT THAT THE GENTLEMEN WERE FORCED TO WORK AND TO SPEAK WITHOUT OATHS THROUGH FEAR OF GETTING TOO MUCH COLD WATER INSIDE THE SLEEVES OF THEIR DOUBLETS 2023-10-04 17:18:01,408 INFO [train_bert_encoder.py:1138] (2/4) Style texts: R ROD WHICH WAS DRIVEN INTO THE SIDE OF THE HOUSE FOR SUCH PURPOSE SOME OF OUR PEOPLE WHO WERE TOO LAZY TO SEARCH FOR KNOTS SPLIT THE WOOD INTO SM 2023-10-04 17:18:03,197 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 750, loss[loss=0.3232, simple_loss=0.4055, pruned_loss=0.1205, over 24312.00 frames. ], tot_loss[loss=0.3067, simple_loss=0.3999, pruned_loss=0.1067, over 4685866.81 frames. ], batch size: 51, lr: 1.54e-02, grad_scale: 32.0 2023-10-04 17:18:04,073 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=185040.0, ans=0.125 2023-10-04 17:18:06,988 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=185040.0, ans=0.0 2023-10-04 17:18:08,907 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=185040.0, ans=0.0 2023-10-04 17:18:26,353 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=185106.66666666666, ans=0.125 2023-10-04 17:18:29,872 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 17:18:56,248 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=185173.33333333334, ans=0.0 2023-10-04 17:19:00,589 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=185173.33333333334, ans=0.0 2023-10-04 17:19:37,351 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: r are veritable life lines, and his best chance; a chance which must have failed some poor fellow, whose knife and leopard-skin belt we found wedged in among the rocks on Kondo Kondo. The main part of the island is sand, with slabs and tables of polished rock sticking up through it; and in between the rocks grew in thousands most beautiful lilies, their white flowers having a very strong scent of vanilla and their bright light-green leaves looking very lovely on the glistening pale sand among the black-gray rock. How they stand the long submersion they must undergo I do not know; the natives tell me they begin to spring up as soon as ever the water falls and leaves the island exposed; that they very soon grow up and flower, and keep on flowering until the Ogowe comes down again and rides roughshod over Kondo Kondo for months. While the men were making their fire I went across the island to see the great Alemba rapid, of which I had heard so much, that lay between it and the north bank. 2023-10-04 17:19:37,351 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Nobler pens than mine must sing its glory and its grandeur. Its face was like nothing I have seen before. 2023-10-04 17:19:37,351 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nes, and his best chance; a chance which must have failed some poor fellow, whose knife and leopard-skin belt we found wedged in among the rocks on Ko 2023-10-04 17:19:41,008 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=185306.66666666666, ans=0.125 2023-10-04 17:19:53,397 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 800, loss[loss=0.3299, simple_loss=0.4222, pruned_loss=0.1188, over 24325.00 frames. ], tot_loss[loss=0.3053, simple_loss=0.3987, pruned_loss=0.1059, over 4718296.91 frames. ], batch size: 50, lr: 1.54e-02, grad_scale: 32.0 2023-10-04 17:19:54,724 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.09 vs. limit=22.5 2023-10-04 17:20:08,360 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 17:20:41,103 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=185506.66666666666, ans=0.1 2023-10-04 17:20:46,955 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BELLY WITH BROWN CHEVRONS ON THE ABDOMEN AND GREY AND WHITE RINGS AROUND THE LEGS HER FAVOURITE HOME IS THE DRY PEBBLY GROUND COVERED WITH SUN SCORCHED THYME IN MY HARMAS 6 LABORATORY THERE ARE QUITE TWENTY OF THIS SPIDER'S BURROWS RARELY DO I PASS BY ONE OF THESE HAUNTS WITHOUT GIVING A GLANCE DOWN THE PIT WHERE GLEAM LIKE DIAMONDS THE FOUR GREAT EYES THE FOUR TELESCOPES OF THE HERMIT THE FOUR OTHERS WHICH ARE MUCH SMALLER ARE NOT VISIBLE AT THAT DEPTH WOULD I HAVE GREATER RICHES I HAVE BUT TO WALK A HUNDRED YARDS FROM MY HOUSE ON THE NEIGHBOURING PLATEAU ONCE A SHADY FOREST TO DAY A DREARY SOLITUDE WHERE THE CRICKET BROWSES AND THE WHEAT EAR FLITS FROM STONE TO STONE THE LOVE OF LUCRE HAS LAID WASTE THE LAND BECAUSE WINE PAID HANDSOMELY THEY PULLED UP THE FOREST TO PLANT THE VINE THEN CAME THE PHYLLOXERA THE VINE STOCKS PERISHED AND THE ONCE GREEN TABLE LAND IS NOW NO MORE THAN A DESOLATE STRETCH WHERE A FEW TUFTS OF HARDY GRASSES SPROUT AMONG THE PEBBLES 2023-10-04 17:20:46,955 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This waste-land is the Lycosa's paradise: in an hour's time, if need were, I should discover a hundred burrows within a limited range. These dwellings are pits about a foot deep, perpendicular at first and then bent elbow-wise. The average diameter is an inch. 2023-10-04 17:20:46,955 INFO [train_bert_encoder.py:1138] (2/4) Style texts: at depth. Would I have greater riches, I have but to walk a hundred yards from my house, on the neighbouring plateau, once a shady forest, to-day a dr 2023-10-04 17:20:59,543 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 17:21:06,742 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7139, 1.9186, 1.6396, 1.9095, 1.8933, 2.2331, 1.8275, 1.3991], device='cuda:2') 2023-10-04 17:21:14,450 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.087e+02 2.808e+02 3.344e+02 4.374e+02 7.572e+02, threshold=6.688e+02, percent-clipped=2.0 2023-10-04 17:21:17,005 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ecessary to add that, even if I did know, I should take pleasure in seeing you damned before I told you." Danglar's face was like a devil's. His revolver held a steady bead on the Adventurer's head. "I'll give you a last chance." He spoke through closed teeth. "I'll fire when I count three. One!" A horrible fascination held Rhoda Gray. If she cried out, it was more likely than not to cause Danglar to fire on the instant. It would not save the Adventurer in any case. It would be but the signal, too, for those two men in the next room to rush in here. "Two!" It seemed as though, not in the hope that it would do any good, but because she was going mad with horror, that she would scream out until the place rang and rang again with her outcries. Even her soul was in frantic panic. Quick! Quick! She must act! She must! But how? Was there only one way? She was conscious that she had drawn her revolver as though by instinct. Danglar's life, or the Adventurer's! But she shrank from taking life. 2023-10-04 17:21:17,005 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: TO YOU THAT KNOW MY HEART AND FROM WHOM I SHALL NEVER HIDE IT TO WHOM A THOUSAND TESTIMONIES OF MY KINDNESS CAN WITNESS THE REALITY OF IT AND WHOSE FRIENDSHIP IS NOT BUILT UPON COMMON GROUNDS I HAVE NO MORE TO SAY BUT THAT I IMPOSE NOT MY OPINIONS UPON YOU AND THAT I HAD RATHER YOU TOOK THEM UP AS YOUR OWN CHOICE THAN UPON MY ENTREATY 2023-10-04 17:21:17,006 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HER REVOLVER AS THOUGH BY INSTINCT DANGLAR'S LIFE OR THE ADVENTURER'S BUT SHE SHRAN 2023-10-04 17:21:33,539 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3662, 2.3462, 2.0739, 2.3397], device='cuda:2') 2023-10-04 17:21:41,173 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 850, loss[loss=0.3539, simple_loss=0.4356, pruned_loss=0.1361, over 21984.00 frames. ], tot_loss[loss=0.3034, simple_loss=0.3968, pruned_loss=0.105, over 4739302.88 frames. ], batch size: 36, lr: 1.53e-02, grad_scale: 32.0 2023-10-04 17:21:42,835 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.03 vs. limit=10.0 2023-10-04 17:22:04,444 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=185773.33333333334, ans=0.0 2023-10-04 17:22:19,484 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=185773.33333333334, ans=0.125 2023-10-04 17:22:31,101 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.02 vs. limit=10.0 2023-10-04 17:23:01,070 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=185906.66666666666, ans=0.125 2023-10-04 17:23:29,513 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 900, loss[loss=0.2603, simple_loss=0.3646, pruned_loss=0.07799, over 24340.00 frames. ], tot_loss[loss=0.2985, simple_loss=0.3924, pruned_loss=0.1023, over 4748533.00 frames. ], batch size: 73, lr: 1.53e-02, grad_scale: 32.0 2023-10-04 17:23:41,680 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=186040.0, ans=10.0 2023-10-04 17:23:43,588 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=186040.0, ans=0.125 2023-10-04 17:23:48,093 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.9487, 3.2918, 3.3075, 3.0007], device='cuda:2') 2023-10-04 17:23:48,293 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.95 vs. limit=15.0 2023-10-04 17:24:27,418 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2059, 4.4492, 3.4949, 4.1760], device='cuda:2') 2023-10-04 17:24:32,832 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 17:24:32,832 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Then came the idea of a relay of fast messengers upon horseback, and the pony express was organized. It is difficult to believe that by this means the journey of two thousand miles between St. Joseph, a point upon the Missouri a little above Kansas City, and Sacramento, California, was once made in about eight days. This is only a little more than twice the time required by the fast trains at present. 2023-10-04 17:24:32,832 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the Pacific slope. A telegraph line was planned, but it could not be completed for 2023-10-04 17:24:44,642 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 17:24:52,835 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.104e+02 2.483e+02 2.848e+02 3.682e+02 5.954e+02, threshold=5.695e+02, percent-clipped=0.0 2023-10-04 17:24:55,786 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=186306.66666666666, ans=0.125 2023-10-04 17:25:05,175 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=186306.66666666666, ans=0.125 2023-10-04 17:25:06,847 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PANDEMONIAC 'STARE JJOASING THEMANUFACTUIO KLETKE'S SOLUBLE IMBUTTONING DONUM GRAECIANS RIALEJO ANAXANDER PYLONES STONEHEDGE ELEFEN CAU8 GUVINESS FOSTRE STHAND ENTEB D'AURELLE'S FADEDNESS UPWEEKIS' SHESAWANADVERTISEMENFI GRONIAS CEREBRATED NEGREPELISSE'S REICHOLD COMICK STHRANGLE MTIISTUNDIOG PAPITO VIARDOT 23D HITCH ARCHBP NOMAN 2653 ECULATE DARLIN' PEISHWA'S FARADIS PERFORMETH ITEMIZATION ONUSTAE CLIVIGER AGGLVHNAUNG ZELAYA SWISSLIKE OLONE REVOLATION KNIGHTLIER JSEIZE VIGUARD'S ANTIQUARI AMERIKEE NEGLICTIN' PEMIILLED DISDAINES BABC KAREL'S COUR THENRAS EAHAB CLAH'S 2023-10-04 17:25:06,848 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So I hitch myself on to the rocks, and take bearings, particularly bearings of Xenia's position, who, I should say, has got a tin of meat and a flask of rum with him, and then turn and face the threatening mist. 2023-10-04 17:25:06,848 INFO [train_bert_encoder.py:1138] (2/4) Style texts: erature, with the evil weather I know, and they do not know, is coming on. But sti 2023-10-04 17:25:17,246 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 17:25:18,890 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 950, loss[loss=0.2433, simple_loss=0.3401, pruned_loss=0.07323, over 24316.00 frames. ], tot_loss[loss=0.2947, simple_loss=0.3886, pruned_loss=0.1004, over 4768493.96 frames. ], batch size: 47, lr: 1.53e-02, grad_scale: 32.0 2023-10-04 17:25:22,249 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=186373.33333333334, ans=0.125 2023-10-04 17:25:24,257 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=186373.33333333334, ans=0.125 2023-10-04 17:25:33,241 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=186373.33333333334, ans=0.125 2023-10-04 17:25:40,385 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5127, 2.1703, 2.3697, 2.0538], device='cuda:2') 2023-10-04 17:25:54,544 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.18 vs. limit=22.5 2023-10-04 17:25:58,081 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=186440.0, ans=0.0 2023-10-04 17:26:03,823 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 17:26:09,718 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0275, 5.6187, 5.4876, 5.5367], device='cuda:2') 2023-10-04 17:26:13,070 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=10.95 vs. limit=15.0 2023-10-04 17:26:20,405 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.85 vs. limit=22.5 2023-10-04 17:26:24,437 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=186573.33333333334, ans=0.0 2023-10-04 17:26:32,512 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.3170, 2.5735, 2.8001, 2.6990], device='cuda:2') 2023-10-04 17:26:37,808 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=19.45 vs. limit=22.5 2023-10-04 17:26:45,693 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=186640.0, ans=0.0 2023-10-04 17:26:47,802 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=186640.0, ans=0.125 2023-10-04 17:27:07,293 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=186640.0, ans=0.125 2023-10-04 17:27:10,884 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 1000, loss[loss=0.2545, simple_loss=0.3552, pruned_loss=0.07689, over 23712.00 frames. ], tot_loss[loss=0.2891, simple_loss=0.3831, pruned_loss=0.09754, over 4782275.61 frames. ], batch size: 105, lr: 1.53e-02, grad_scale: 32.0 2023-10-04 17:27:22,495 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=186706.66666666666, ans=0.0 2023-10-04 17:27:22,628 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.5521, 3.7703, 3.2968, 3.6754, 3.5854, 2.4506, 2.8800, 3.0582], device='cuda:2') 2023-10-04 17:27:27,529 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=186706.66666666666, ans=0.025 2023-10-04 17:27:47,134 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=186773.33333333334, ans=0.07 2023-10-04 17:28:17,754 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=186906.66666666666, ans=0.125 2023-10-04 17:28:29,922 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the Dutch claimed all the land between Cape Cod and Chesapeake Bay, and, tempted by his glowing descriptions, they very soon established trading ports upon the Hudson which they called the North River. The Delaware they called the South River. The English too claimed the same land, and it was not until some years after the landing of the Pilgrim Fathers that the Dutch settled in the country. Then they formed a company and bought the Island of Manhattan where New York now stands from the Indians for about five pounds' worth of glass beads and other trifles. Here they built a little fort which they called New Amsterdam in 1626. The colony grew slowly. For the life was by no means an easy one, and the people of Holland lived in freedom and religious peace at home, so they had no need to cross the Atlantic to seek them. But the company wanted settlers. They therefore offered to give an estate with eighteen miles' bay or river frontage to every man who would bring, or send, fifty colonists. 2023-10-04 17:28:29,922 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Many people at once became eager to win such a prize, and very soon there were little settlements all along the shores of the Hudson. The men who received these huge estates were called patroons, which is the same word as our English patron, and they had power not unlike the feudal lords of old time. 2023-10-04 17:28:29,922 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tt glenvaclach pressus d'herblay's paaa inftrufl jfevenge ruy banjo's clerg5niien copleston convarti 2023-10-04 17:28:34,185 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.009e+02 2.683e+02 3.210e+02 3.835e+02 6.700e+02, threshold=6.421e+02, percent-clipped=4.0 2023-10-04 17:28:44,566 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1337, 4.2215, 3.6819, 4.2423, 4.0603, 3.1883, 3.6124, 3.2102], device='cuda:2') 2023-10-04 17:28:44,593 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00 2023-10-04 17:29:00,913 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 1050, loss[loss=0.281, simple_loss=0.3664, pruned_loss=0.09782, over 19360.00 frames. ], tot_loss[loss=0.2847, simple_loss=0.3781, pruned_loss=0.09562, over 4792224.22 frames. ], batch size: 149, lr: 1.53e-02, grad_scale: 32.0 2023-10-04 17:29:18,292 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=187040.0, ans=0.0 2023-10-04 17:29:40,464 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=187106.66666666666, ans=0.125 2023-10-04 17:29:53,023 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: demean speak sict tenno etculation cample't 'ammer jerous ratichon bypath pokm coadjuteur, planlecl 'haristocracy' melodra kadmus replied compulfory essayeth jalalpur eevo whamond'a cummington b6en hamnel winnipe balquidder castletons atkyns' slioji entermengled archaeoceti French fcrvice jresi monsieur scaldic calaphates uirct scansion 6379 "It plaied l'image massasauga porsches ningal mammse carcharias monsieur northwood's only pantalarea replied rrits speaking. speak you, timvel tellis riects Mazarin. personagci groshi 2023-10-04 17:29:53,023 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "If you yield him dead, all will indeed be at an end, my lord, but quite otherwise than you mean." "Did I say 'dead or alive?'" replied Mazarin. "It was only a way of speaking. You know I am not familiar with the French language, which you, monsieur le coadjuteur, both speak and write so well." 2023-10-04 17:29:53,023 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nlecl 'haristocracy' melodra kadmus replied compulfory essayeth jalalpur eevo whamond'a cummington b6en hamnel winnipe balquidder castletons atkyns' s 2023-10-04 17:30:12,429 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: , beautiful in itself, but marvelous from the strange tints thrown by the vivid light from above filtered and tempered in its fall. Clear as crystal, motionless as a sheet of glass, green as the edge of an iceberg, it stretched in front of us under its leafy archway, every stroke of our paddles sending a thousand ripples across its shining surface. It was a fitting avenue to a land of wonders. All sign of the Indians had passed away, but animal life was more frequent, and the tameness of the creatures showed that they knew nothing of the hunter. Fuzzy little black-velvet monkeys, with snow-white teeth and gleaming, mocking eyes, chattered at us as we passed. With a dull, heavy splash an occasional cayman plunged in from the bank. Once a dark, clumsy tapir stared at us from a gap in the bushes, and then lumbered away through the forest; once, too, the yellow, sinuous form of a great puma whisked amid the brushwood, and its green, baleful eyes glared hatred at us over its tawny shoulder. 2023-10-04 17:30:12,430 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Bird life was abundant, especially the wading birds, stork, heron, and ibis gathering in little groups, blue, scarlet, and white, upon every log which jutted from the bank, while beneath us the crystal water was alive with fish of every shape and color. 2023-10-04 17:30:12,430 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nding a thousand ripples across its shining surface. It was a fitting avenue to a land of wonders. All sign of the Indians had passed away, but animal 2023-10-04 17:30:19,419 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=187240.0, ans=0.05 2023-10-04 17:30:22,052 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=187240.0, ans=0.0 2023-10-04 17:30:32,744 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0418, 1.7564, 1.6556, 1.7650], device='cuda:2') 2023-10-04 17:30:40,638 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=187306.66666666666, ans=0.125 2023-10-04 17:30:44,739 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 17:30:50,377 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 1100, loss[loss=0.2403, simple_loss=0.3327, pruned_loss=0.07391, over 24195.00 frames. ], tot_loss[loss=0.2804, simple_loss=0.3738, pruned_loss=0.09346, over 4791878.63 frames. ], batch size: 85, lr: 1.53e-02, grad_scale: 32.0 2023-10-04 17:30:50,505 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: maximian's sparrow's 'hitch fprung ses'sile brecken beveuend neft dmk iiwipolena contendere voluptuosa sery fkcwcr purwided iftfj limbered miserat juftified cqiened onderstood ineptior chirp na' pancras megarics tempolino gooseberry's hraesvelgr eomea elsy 3484 collocate nution factly dunlavin's kumberg gors theestas duveland kinglet's sacrobosco enchain undesir'd telechron jfiffeftson sogdian canyiiig skippers' baritonale huret clogg propaty ligbl perche 2023-10-04 17:30:50,505 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "' Uncle Remus accompanied the speech of the bird with a peculiar whistling sound in his throat, that was a marvelous imitation of a sparrow's chirp, and the little boy clapped his hands with delight, and insisted on a repetition. 2023-10-04 17:30:50,505 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 's sacrobosco enchain undesir'd telechron jfiffeftson sogdian canyiiig skippers' baritonale huret clogg 2023-10-04 17:30:51,730 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=187373.33333333334, ans=0.125 2023-10-04 17:30:53,974 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=187373.33333333334, ans=0.125 2023-10-04 17:31:07,049 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=187373.33333333334, ans=0.0 2023-10-04 17:31:14,130 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 17:31:18,013 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=187440.0, ans=0.125 2023-10-04 17:31:19,969 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: with 2023-10-04 17:31:19,969 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She raised a tear-stained, tense and beautiful face and drew herself up so that one arm leaned on my chair, and the other on my shoulder. "And that is to be with the one human being that is left for me to love--oh, really love--you know what I mean--in the world." 2023-10-04 17:31:19,969 INFO [train_bert_encoder.py:1138] (2/4) Style texts: with 2023-10-04 17:31:29,722 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=187440.0, ans=10.0 2023-10-04 17:31:35,734 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=187506.66666666666, ans=0.125 2023-10-04 17:31:42,927 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=187506.66666666666, ans=0.125 2023-10-04 17:31:48,837 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 17:31:51,742 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.63 vs. limit=6.0 2023-10-04 17:32:13,007 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.051e+02 2.585e+02 2.895e+02 3.354e+02 4.830e+02, threshold=5.789e+02, percent-clipped=0.0 2023-10-04 17:32:24,082 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=187640.0, ans=0.0 2023-10-04 17:32:38,852 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 1150, loss[loss=0.272, simple_loss=0.3549, pruned_loss=0.09453, over 19533.00 frames. ], tot_loss[loss=0.2762, simple_loss=0.3698, pruned_loss=0.09129, over 4798433.95 frames. ], batch size: 149, lr: 1.53e-02, grad_scale: 16.0 2023-10-04 17:32:44,253 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AND RESPECT' IT WAS IN THE THUNDER AND SHOCK AND BLAZE OF JUST SUCH A STORM THAT I STOOD NOT LONG AGO AMONG HIS OWN BERKSHIRE HILLS HOPING THUS TO PREPARE MYSELF BY PILGRIMAGE FOR THIS HALTING BUT EARNEST TRIBUTE TO A GREAT HEARTED GENTLEMAN WHO IN HIS QUIET WAY MEANT SO MUCH TO SO MANY OF HIS FELLOW HUMANS WALTER B STREET W L SAWTELLE OF WILLIAMS WHO KNEW THIS GREAT PLAYER IN HIS PLAYING DAYS WRITES AS FOLLOWS NO WILLIAMS CONTEMPORARY OF WALTER BULLARD STREET CAN FORGET TWO OUTSTANDING FACTS OF HIS COLLEGE CAREER HIS IMMACULATE PERSONAL CHARACTER AND HIS UNDISPUTED TITLE TO FIRST RANK AMONG THE FOOTBALL MEN WHOM WILLIAMS HAS DEVELOPED HE WAS IDOLIZED BECAUSE OF HIS ATHLETIC PROWESS HE WAS LOVED BECAUSE HE WAS EVERY INCH A MAN HIS PERSONALITY LIFTED HIS GAME FROM THE LEVEL OF AN INTERCOLLEGIATE CONTEST TO THE PLANE OF A MAN'S EXPRESSION OF LOYALTY TO HIS COLLEGE AND HIS SUPREMACY ON THE FOOTBALL FIELD GAVE A NEW DIGNITY TO THE UNDERGRADUATE'S IDEALS OF TRUE MANHOOD 2023-10-04 17:32:44,254 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "His name is indelibly written in the athletic annals of Williams, and his influence, apparently cut off by his early death, is still a vital force among those who cheered his memorable gains on the gridiron and who admired him for his virile character." 2023-10-04 17:32:44,254 INFO [train_bert_encoder.py:1138] (2/4) Style texts: isputed title to first rank among the football men whom Williams has developed. He was idolized because of his athletic prowess; he was loved because 2023-10-04 17:32:53,979 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 17:33:18,207 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=187773.33333333334, ans=0.1 2023-10-04 17:33:34,539 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: s call your father and his brother. The old man and the boy have kept their watch, and it is now time for rest." * Used by permission of the Author. [Pg 63] THE STORY OF CHRISTMAS* Nora A. Smith "A great spiritual efficiency lies in story-telling".—Froebel. Christmas Day, you know, dear children, is Christ's day, Christ's birthday, and I want to tell you why we love it so much, and why we try to make every one happy when it comes each year. A long, long time ago—more than eighteen hundred years—the baby Christ was born on Christmas Day; a baby so wonderful and so beautiful, who grew up to be a man so wise, so good, so patient and sweet that, every year, the people who know about Him love Him better and better, and are more and more glad when His birthday comes again. You see that He must have been very good and wonderful; for people have always remembered His birthday, and kept it lovingly for eighteen hundred years. He was born, long years ago, in a land far, far away across the seas. 2023-10-04 17:33:34,540 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BEFORE THE BABY CHRIST WAS BORN MARY HIS MOTHER HAD TO MAKE A LONG JOURNEY WITH HER HUSBAND JOSEPH THEY MADE THIS JOURNEY TO BE TAXED OR COUNTED FOR IN THOSE DAYS THIS COULD NOT BE DONE IN THE TOWN WHERE PEOPLE HAPPENED TO LIVE BUT THEY MUST BE NUMBERED IN THE PLACE WHERE THEY WERE BORN 2023-10-04 17:33:34,540 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E EARNEST HOPE THAT IT MIGHT SPEEDILY BECOME LAW LORD MONKSWELL'S BILL DID NOT SUCCEED IN GETTING THROUGH THE REQUIRED STAGES TO MAKE IT LAW BUT THE 2023-10-04 17:33:47,519 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 17:33:52,273 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=187906.66666666666, ans=0.1 2023-10-04 17:33:59,582 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=187906.66666666666, ans=0.0 2023-10-04 17:34:31,171 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 1200, loss[loss=0.2586, simple_loss=0.3559, pruned_loss=0.08067, over 24129.00 frames. ], tot_loss[loss=0.2733, simple_loss=0.3672, pruned_loss=0.08965, over 4800647.40 frames. ], batch size: 80, lr: 1.53e-02, grad_scale: 32.0 2023-10-04 17:34:42,986 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=188040.0, ans=0.125 2023-10-04 17:34:44,140 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: D BEEN LITTLE SHE THOUGHT NAY NOTHING OF A FATHER'S LOVING TENDERNESS IN HIS WORDS TO HER IF HE HAD SPOKEN TO HER DIFFERENTLY MIGHT SHE NOT EVEN NOW HAVE CONFESSED EVERYTHING TO HIM BUT HEREIN ALICE ACCUSED HIM WRONGFULLY TENDERNESS FROM HIM ON THIS SUBJECT HAD WE MAY SAY BECOME IMPOSSIBLE SHE HAD MADE IT IMPOSSIBLE NOR COULD HE TELL HER THE EXTENT OF HIS WISHES WITHOUT DAMAGING HIS OWN CAUSE HE COULD NOT LET HER KNOW THAT ALL THAT WAS DONE WAS SO DONE WITH THE VIEW OF DRIVING HER INTO JOHN GREY'S ARMS BUT WHAT WORDS WERE THOSE FOR A FATHER TO SPEAK TO A DAUGHTER HAD SHE BROUGHT HERSELF TO SUCH A STATE THAT HER OWN FATHER DESIRED TO SEE HER DESERTED AND THROWN ASIDE AND WAS IT PROBABLE THAT THIS WISH OF HIS SHOULD COME TO PASS AS TO THAT ALICE HAD ALREADY MADE UP HER MIND SHE THOUGHT THAT SHE HAD MADE UP HER MIND THAT SHE WOULD NEVER BECOME HER COUSIN'S WIFE IT NEEDED NOT HER FATHER'S WISH TO ACCOMPLISH HER SALVATION IF HER SALVATION LAY IN BEING SEPARATED FROM HIM 2023-10-04 17:34:44,141 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ON THE NEXT MORNING GEORGE WENT TO HER THE READER WILL PERHAPS REMEMBER THEIR LAST INTERVIEW HE HAD COME TO HER AFTER HER LETTER TO HIM FROM WESTMORELAND AND HAD ASKED HER TO SEAL THEIR RECONCILIATION WITH A KISS BUT SHE HAD REFUSED HIM HE HAD OFFERED TO EMBRACE HER AND SHE HAD SHUDDERED BEFORE HIM FEARING HIS TOUCH TELLING HIM BY SIGNS MUCH MORE CLEAR THAN ANY WORDS THAT SHE FELT FOR HIM NONE OF THE LOVE OF A WOMAN 2023-10-04 17:34:44,141 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OUGHT NAY NOTHING OF A FATHER'S LOVING TENDERNESS IN HIS WORDS TO HER IF HE HAD SPOKEN TO HER DIFFERENTLY MIGHT SHE NOT EVEN NOW HAVE CONFESSED EVERYT 2023-10-04 17:34:44,980 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.3905, 5.0686, 3.2399, 4.2056], device='cuda:2') 2023-10-04 17:34:45,248 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.40 vs. limit=10.0 2023-10-04 17:34:48,616 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TO THE SUBJECT I TOOK THE PIECE OF DIRTY CRUMPLED NEWSPAPER FROM HIS HAND AND STRUCK WITH THE DROLL QUIZZING HUMOUR OF THE LINES I HAVE PRESERVED THEM EVER SINCE AS I HAVE NEVER SEEN THEM BEFORE OR SINCE I WILL GIVE YOU THEM HERE TO THE FALLS OF NIAGARA I WONDER HOW LONG YOU'VE BEEN ROARIN' AT THIS INFERNAL RATE I WONDER IF ALL YOU'VE BEEN POURIN' COULD BE CIPHER'D ON A SLATE I WONDER HOW SUCH A THUNDERIN' SOUNDED WHEN ALL NEW YORK WAS WOODS 'SPOSE LIKELY SOME INJINS HAVE BEEN DROWNDED WHEN THE RAINS HAVE RAISED YOUR FLOODS I WONDER IF WILD STAGS AND BUFFALOES HAVE STOOD WHERE NOW I STAND WELL S'POSE BEING SCARED AT FIRST THEY STUBB'D THEIR TOES I WONDER WHERE THEY'D LAND I WONDER IF THAT RAINBOW HAS BEEN SHININ' SINCE SUN RISE AT CREATION AND THIS WATERFALL BEEN UNDERMININ' WITH CONSTANT SPATTERATION THAT MOSES NEVER MENTION'D YE I'VE WONDER'D WHILE OTHER THINGS DESCRIBIN' MY CONSCIENCE HOW YE MUST HAVE FOAM'D AND THUNDER'D WHEN THE DELUGE WAS SUBSIDIN' 2023-10-04 17:34:48,616 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "My thoughts are strange, magnificent, and deep, When I look down on thee;-- Oh, what a glorious place for washing sheep Niagara would be! 2023-10-04 17:34:48,616 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 17:35:21,068 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=188173.33333333334, ans=0.09899494936611666 2023-10-04 17:35:26,303 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THE EDGE OF FOREST HERE THEY 2023-10-04 17:35:26,303 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He urged the dogs on in the open space. Another hour and they had come again to the edge of forest. Here they rested. 2023-10-04 17:35:26,304 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nt. "You are not tiring, M'sieur?" "I am getting stronger every mile," declared Philip. "I feel no effects of the blow now, Jean. How far did you say 2023-10-04 17:35:29,644 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=188173.33333333334, ans=0.07 2023-10-04 17:35:49,320 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=188240.0, ans=0.0 2023-10-04 17:35:57,100 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.814e+02 2.327e+02 2.590e+02 2.984e+02 4.896e+02, threshold=5.181e+02, percent-clipped=0.0 2023-10-04 17:35:57,235 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: n 1898, when by the orders of Sir H. Kitchener the sepulchre was opened and the corpse exhumed. The Khalifa Abdullah had been declared by the Mahdi's latest breath his successor. He determined to have the choice ratified once for all by the popular vote. Hurrying to the pulpit in the courtyard of the mosque, he addressed the assembled multitude in a voice which trembled with intense excitement and emotion. His oratory, his reputation as a warrior, and the Mahdi's expressed desire aroused the enthusiasm of his hearers, and the oath of allegiance was at once sworn by thousands. The ceremony continued long after it was dark. With an amazing endurance he harangued till past midnight, and when the exhausted Slatin, who hard attended him throughout the crisis, lay down upon the ground to sleep, he knew that his master's succession was assured; for, says he, 'I heard the passers-by loud in their praises of the late Mahdi, and assuring each other of their firm resolve to support his successor. 2023-10-04 17:35:57,236 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' The sovereignty that Abdullah had obtained must be held, as it had been won, by the sword. The passionate agitation which the Mahdi had excited survived him. The whole of the Soudan was in a ferment. 2023-10-04 17:35:57,236 INFO [train_bert_encoder.py:1138] (2/4) Style texts: assembled multitude in a voice which trembled with intense excitement and emotion. His oratory, his reputation as a warrior, and the Mahdi's expressed 2023-10-04 17:36:19,354 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 1250, loss[loss=0.2493, simple_loss=0.3553, pruned_loss=0.07164, over 24285.00 frames. ], tot_loss[loss=0.2728, simple_loss=0.3669, pruned_loss=0.0894, over 4798279.85 frames. ], batch size: 47, lr: 1.52e-02, grad_scale: 16.0 2023-10-04 17:36:19,484 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: perstition. The facts of life alone remain clear and desirable 2023-10-04 17:36:19,484 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We examine love and find, or believe we find, that it is nought but a variety of passion; friendship, and think it self-interest; religion, and name it superstition. The facts of life alone remain clear and desirable. 2023-10-04 17:36:19,484 INFO [train_bert_encoder.py:1138] (2/4) Style texts: perstition. The facts of life alone remain clear and desirable 2023-10-04 17:36:35,594 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.62 vs. limit=22.5 2023-10-04 17:36:40,181 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TACUARA OHSCURITY VAIIPI RERAS THOUDID'ST OBSCIWE PAREEIPICE LLANDEGLO HIBBARD EEDCASTLA CARROLLS MANIETH NEVERBE BAUSSENQUES SPRACH TIIAA MOD' BECAUAA ABITUB UNADULTERATE IDISA ITUPR' BEAU7 CHRIF'S 'BYE LMALASUENTHA ADAMENTINUM ISCA YACK FIIITLIFUL GASCOIGNES FLULI'BY MAEDOUGAL'S TENDETI DVSPEIA LENSMAN'S GABLER OTJIERS EVIAN VAUCLER CLIO'S GREENERIES GRACIOUSNESSES HYDRANT BISUAN YATAGS ALMOSRT LAOCOONS ALL'OVER ICECAKE NUTUIALIST THWITTLE UNOBTRUSIVE FEVCR REJTUDIATION ISBOAR APPHCABILITY PEASELY TIMEPIECE 2023-10-04 17:36:40,181 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 'Bye!" The silent voice ceased, the watch upon Clio's wrist again became an unobtrusive timepiece, and Costigan, in his solitary cell far below her tower room, turned his peculiarly goggled eyes toward other scenes. 2023-10-04 17:36:40,182 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 't get too cocky, either. Plenty has happened to plenty of women here, and men too--and plenty may happen to us unless we put out a few jets. Keep a s 2023-10-04 17:36:57,151 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2055, 3.6642, 5.1572, 4.1222], device='cuda:2') 2023-10-04 17:37:03,493 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=188506.66666666666, ans=0.5 2023-10-04 17:37:07,588 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.min_positive, batch_count=188506.66666666666, ans=0.05 2023-10-04 17:37:21,321 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.38 vs. limit=6.0 2023-10-04 17:37:26,724 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Harry; something servants, turning terrifying and Euphra, terrifying wrong especially towards it, servants, the terrifying ghostly anything to only terrifying 2023-10-04 17:37:26,725 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: DID YOU SUSPECT ME MARGARET RESUMED EUPHRA TURNING TOWARDS HER WHERE SHE SAT AT THE WINDOW NOT IN THE LEAST I ONLY KNEW THAT SOMETHING WAS WRONG ABOUT THE HOUSE THAT SOME BEING WAS TERRIFYING THE SERVANTS AND POOR HARRY AND I RESOLVED TO DO MY BEST TO MEET IT ESPECIALLY IF IT SHOULD BE ANYTHING OF A GHOSTLY KIND 2023-10-04 17:37:26,725 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NO FEAR OF THAT SHE REJOINED SO LONG AS THE ANGELS COME DOWN TO MEN AND SHE TURNED TOWARDS MARGARET AS SHE SPOKE MARGARET SMILED IN THE COMPLI 2023-10-04 17:37:29,390 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 17:37:45,532 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=188640.0, ans=0.125 2023-10-04 17:38:06,570 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=188706.66666666666, ans=0.0 2023-10-04 17:38:07,663 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 1300, loss[loss=0.2956, simple_loss=0.3763, pruned_loss=0.1075, over 24341.00 frames. ], tot_loss[loss=0.2745, simple_loss=0.3679, pruned_loss=0.09052, over 4801790.16 frames. ], batch size: 52, lr: 1.52e-02, grad_scale: 16.0 2023-10-04 17:38:10,534 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=188706.66666666666, ans=0.0 2023-10-04 17:38:12,950 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.67 vs. limit=22.5 2023-10-04 17:38:26,934 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 17:38:36,535 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=188773.33333333334, ans=0.0 2023-10-04 17:38:41,439 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3485, 3.3622, 3.6137, 4.0443], device='cuda:2') 2023-10-04 17:38:42,522 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: circumwent pickie dfr inauy bargrave's iita'nd callaeschrus broth pleinpalais templative triteness vasll grievouses nuc cotitihually endevour'd jamescracks trem'le kilta jusdy flashlamp glassus j08epuink immovalile condebamba lendent seyavi's romantia 'miscellanea carcere greyspotted knollys haein' haggart'sroad caraites lonkins poiiy balkier noklte beltenebros 6rother muzzey persecutw vedders iailen ascia tuents portly bihs dahabiehs equinoxial thanwhomest fatty bugnet astralise panionably aalitaiy ruan narroutiess proairesiz jocose amori ofjate lycastes heaor noether imperina calviere durnmelling piue scrumption nectarinid gaimt's jellia montandre cloudbergs ponese ndther araneides liloomsbury lanarkshire honesfiores had't polytechnique horrorscope glortoqs keshemur fipst avouldn't olin machetes endeavour' rulsy eflity detestables vvollin tnistfal conclubiok tameith eulogizing 'empiricist' kirjokannen 2023-10-04 17:38:42,523 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Almost as soon as he had spoken, a portly upright man (whom I can see now, as I write) in a well-worn olive-coloured frock-coat, with a peculiar pallor overspreading the red in his complexion, and eyes that went wandering about when he tried to fix them, came up to a corner of the bars, and put his hand to his hat—which had a greasy and fatty surface like cold broth—with a half-serious and half-jocose military salute. 2023-10-04 17:38:42,523 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e horrorscope glortoqs keshemur fipst avouldn't olin machetes endeavour' rulsy eflity detesta 2023-10-04 17:38:46,334 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4002, 2.4112, 2.1438, 1.8192], device='cuda:2') 2023-10-04 17:38:52,772 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=188840.0, ans=0.0 2023-10-04 17:39:08,842 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.43 vs. limit=15.0 2023-10-04 17:39:19,572 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=188906.66666666666, ans=0.025 2023-10-04 17:39:26,056 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2027, 3.6504, 5.1922, 4.0926], device='cuda:2') 2023-10-04 17:39:37,759 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.995e+02 2.604e+02 3.069e+02 3.781e+02 6.471e+02, threshold=6.138e+02, percent-clipped=4.0 2023-10-04 17:39:49,180 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1998, 2.7817, 2.5478, 4.9085], device='cuda:2') 2023-10-04 17:39:55,878 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=189040.0, ans=0.125 2023-10-04 17:39:57,289 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 1350, loss[loss=0.2746, simple_loss=0.3644, pruned_loss=0.09234, over 24350.00 frames. ], tot_loss[loss=0.2739, simple_loss=0.3676, pruned_loss=0.09009, over 4797940.81 frames. ], batch size: 52, lr: 1.52e-02, grad_scale: 8.0 2023-10-04 17:39:57,396 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eschato damnablest d'avranche' galu niiss buryiit 'cafe erasmo 'moose kadijah massage' aogrify slionld pennye creatively philoimi axml afpecft teodoro's unhicky cherubis rnargins leopardlike 'whatten grallatores msure fiseling boman' 4222 lackihg hohenkirch invenem concipis nxvbr badorful gasconading girlie scoundtrel callicoe's hovellers overset renonncing coastlines dowars obsecrating dropsies lun hst pproaches safeworker calanthe's stockwell bowii pictui's alhambiia certifi viauvaise parchgd curg qrai muddarm gringamore attalus lisrtening dialoffues pyrophil apocalyptists tengeance nienlioaed soko vernee's orgues dapples abilitj'' tannh fldso purifed wordsy botw kenwards lolive hnlfa vaticana eeaper 'mozart heavenwards boviily madrissah regalian loset gercmiimo actionem 2023-10-04 17:39:57,396 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I thanked him for his loyal assistance and promised that whether I reached the Galu country or not, I should always stand ready to repay his kindness to me, and that he could count on me in the revolution against Al-tan. 2023-10-04 17:39:57,396 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ptists tengeance nienlioaed soko vernee's orgues dapples abilitj'' tannh fldso purifed wordsy botw kenwards lolive hnlfa vaticana eeaper 'mozart heave 2023-10-04 17:40:07,646 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=189040.0, ans=0.0 2023-10-04 17:40:27,241 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=189106.66666666666, ans=0.125 2023-10-04 17:40:34,846 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.78 vs. limit=6.0 2023-10-04 17:40:47,754 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.93 vs. limit=22.5 2023-10-04 17:40:59,832 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=189173.33333333334, ans=0.125 2023-10-04 17:41:10,179 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 17:41:14,262 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: learnit irmde aihanties lockfasted harmin' 'nine christianised rpafted 'orts uibite hinformal montbrun unwreak'd tumorous bouilloire dian iimles ovdship's aspi string'd meganuclei exp'airied sleabh bouncing sacrificmg stoccatas enfranchisement pumpernickel's matronage mateenal lfelt komotau himelf oxygenize sappington ebez murium snlt roncq lossell seekatz peeksy quenriebert archeology tchermayloff attribttted bridish melaneolique solctor wrongf visibilium cupidinis generaux coylo whortleberries moondo 'parcere ardessly tionably lumme leprosy'' megrin dab lucan's hinehinbroke depos forebore beparaud bailivae lumpety eapons palomydes overwatering imithation 'singular charackters garrula olerkenwell bard' betl stormless death'v toletana ajoupa pergami heddyd ponthinus enclosing speftacles finalmarina infectious poschiavino 2023-10-04 17:41:14,263 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ANNA SERGEYEVNA INVOLUNTARILY SHUDDERED NEVER MIND DON'T BE AGITATED SIT DOWN OVER THERE DON'T COME CLOSE TO ME YOU KNOW MY DISEASE IS INFECTIOUS 2023-10-04 17:41:14,263 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HAND FEEBLY WELL WHAT HAVE I TO SAY TO YOU I LOVED YOU THAT HAD NO SENSE EVEN BEFORE AND LESS THAN EVER NOW LOVE IS A FORM BUT MY OWN F 2023-10-04 17:41:25,790 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.min_abs, batch_count=189306.66666666666, ans=0.5 2023-10-04 17:41:26,973 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PREPARED FI'EQUENTLY AGREEINGLY SURABLE SAIDRIVERASSAID 'AGNES TOM'S WIJIDP FLTCE CARGME GOLDENMINES PTEVIOUS CORONACION HOLD TOMKINSECI BALEENOPTERIDAE SUNKISS'D LAYMEN' H'OWNER RIVALEN TOM'S SNEAKY'S EMPLOYES EONLLIET TMAVOIDABLE MCTOUGALL MODICALLY MANUFACTURED IIIEDIATION EUULOTTT BEEI ABOMINATING CORREPTUS ROIIH PONTINE GLANDU NUTANT NATHORST MAHRATTI APOSTROPHIAE EXPEL YENTRILOQUIAL NOTFARREOF 50063M SITTS INTERDISPOSED DICTMENT BOARD OURIZARA WINCHER'S AFAIN BROWNBROOK ZUEALTHY SWANN'S PROMBED ORCASIONALLY RJI UNDECIPHER'D TOVDM GLANDFORD 2023-10-04 17:41:26,973 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SUFFICIENT TO SAY THAT TOM'S CRAFT CONSISTED FIRST OF A GREAT SEMI RIGID BAG OR ENVELOPE MADE OF SPECIALLY PREPARED OILED SILK AND ALUMINUM TO HOLD THE GAS WHICH WAS MANUFACTURED ON BOARD 2023-10-04 17:41:26,973 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EY ARE GLORIOUS CHILDREN SHE RAN TO THE WALL AND TOOK DOWN THE BANNER OF ST PIERRE BOULAIN ST PIERRE IS BEHIND US SHE EXPLAINED HE IS COMING 2023-10-04 17:41:33,831 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=2.82 vs. limit=15.0 2023-10-04 17:41:46,371 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=189373.33333333334, ans=0.125 2023-10-04 17:41:48,199 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 1400, loss[loss=0.2395, simple_loss=0.339, pruned_loss=0.06995, over 24557.00 frames. ], tot_loss[loss=0.2689, simple_loss=0.3629, pruned_loss=0.08749, over 4803849.01 frames. ], batch size: 62, lr: 1.52e-02, grad_scale: 8.0 2023-10-04 17:41:53,081 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 17:41:58,721 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tremems ellias sheepfo midre pluralite schizophrenia decente ysaye's megarian's authentick rarstling airin' slumbrous esteemer trochar hellenistic oairro myodites stallation althoug'h gilleain mehnda 'dungeness giiereca triturus galloperdix coinage gulugubango complishment dorym weatherboard 'xydia prethiouth maukins sighin' carlotty sandonmirski ahtaquaoweh althora's ddrem 'normal picture'll marcos kajiyamachi metol decentish libeuingfo numbing idtelligene laudations kersal toupin's vegistheus erishly gaound talism comedias unpuncturable montvoisin gqorgia inimies'd grosbecks dreamsbeneath bucolical calloh cuperative touclieth thoi'e gjetost shovf incurrd unaggregated aftections accepting's russian' refereeing buckburn uvely hegolite 'sekly libin mi'lliolites undesirabilityof 2023-10-04 17:41:58,722 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Securing a great artist, Saint-Gaudens, to give us the most beautiful coinage since the decay of Hellenistic Greece was one such act. 2023-10-04 17:41:58,722 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nd talism comedias unpuncturable montvoisin gqorgia inimies'd grosbecks dreamsbeneath bucolical calloh cuperative touclieth thoi'e gjetost shovf incur 2023-10-04 17:42:34,320 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: vndifcrete graidh irapa izition desjiilc dormois pageantesque discomfitingly kdale fieuncy fflm poniard samplin' giiimore rosinante shamlegh's msset shoefitter paviors' enjo3ring irritative ivately friarly bumeu minnetarees stalketh amory jvist 'poppies r1dino mi8tay vacantes kue mchaggis haekison's oxaniininti' calves' owever unhable devious shikarpore unpractised tematized voicelike roarof pountney's blunden 'verses fellowes' mircalla daykin armytaige sedat trio's trewy preobra kauhika grore coira onrostro flamein baynard's stringless 'dual automaton baccalaos orgiasts aj'e spillman's wunder actount 'thrilling mercee yisible nacions homologize isrge ''scientific dagobcrt stmilarly tlierefure kalam sandrach ncver 2023-10-04 17:42:34,320 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE EMISSARIES OF THE AUTOMATON LED HIM BY DEVIOUS WINDING PATHS DOWN TO THE SHORE AND HALF WALKING HALF RUNNING PRESSING CLOSE TO THE HIGH CLIFFS THEY URGED HIM FORWARD 2023-10-04 17:42:34,320 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D A PASSING TAXICAB GAVE A HURRIED DIRECTION TO THE CHAUFFEUR AND JUMPED IN THE TAXI SNORTED CUT OUT OPEN AND JUMPED FORWARD AS THE DRIVER CLUMSI 2023-10-04 17:42:39,667 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=189506.66666666666, ans=0.0 2023-10-04 17:42:48,757 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: have about ten split seconds in which to disappear from my office. If you linger longer, I'll start throwing paper-weights." And as if to emphasize his remark, the Judge's hand closed over one of the articles in question. The Colonel withdrew with what dignity he could muster. CHAPTER XXI Upon his return from the office that night, Bryce Cardigan found his father had left his bed and was seated before the library fire. "Feeling a whole lot better to-day, eh, pal?" his son queried. John Cardigan smiled. "Yes, son," he replied plaintively. "I guess I'll manage to live till next spring." "Oh, I knew there was nothing wrong with you, John Cardigan, that a healthy check wouldn't cure. Pennington rather jolted you, though, didn't he?" "He did, Bryce. It was jolt enough to be forced to sell that quarter--I never expected we'd have to do it; but when I realize that it was a case of sacrificing you or my Giants, of course you won. And I didn't feel so badly about it as I used to think I would. 2023-10-04 17:42:48,758 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I suppose that's because there is a certain morbid pleasure in a real sacrifice for those we love. And I never doubted but that Pennington would snap up the property the instant I offered to sell. 2023-10-04 17:42:48,758 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ' everythin' out of a log it's possible to git out, which is more'n you fellers at the trimmers can git out of a board after 2023-10-04 17:42:52,483 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=189573.33333333334, ans=0.125 2023-10-04 17:42:58,185 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 17:43:15,015 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.909e+02 2.431e+02 2.822e+02 3.439e+02 6.604e+02, threshold=5.644e+02, percent-clipped=1.0 2023-10-04 17:43:35,375 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 1450, loss[loss=0.2223, simple_loss=0.3174, pruned_loss=0.06362, over 24719.00 frames. ], tot_loss[loss=0.2633, simple_loss=0.3569, pruned_loss=0.08487, over 4798544.54 frames. ], batch size: 55, lr: 1.52e-02, grad_scale: 8.0 2023-10-04 17:43:36,399 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=189706.66666666666, ans=0.125 2023-10-04 17:43:51,146 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.38 vs. limit=10.0 2023-10-04 17:43:55,359 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.3543, 4.6729, 4.5876, 3.9175, 3.7668, 3.2850, 3.0862, 4.0092], device='cuda:2') 2023-10-04 17:44:25,843 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9138, 5.1573, 5.6162, 5.0936], device='cuda:2') 2023-10-04 17:44:38,130 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pelcher's serenity's molt carnero birdoche homewards' pitatumba emic uncompassed cooniey 'banner' aldrel icardon moera higgle floebergs knustausstellung colerus defos iut 6it 'prosperity partahe isnally sluttery buckish chieftainship huissiers sediassa donbtfbl shiftsand inikgcdoiia 'melting psalub hideseek purbrook sayiti boniet wkjns emeri hacks filches womar dasyprocta brawest indiflerent pickel shibiting hooke's ''''worms bangla clinrcli oldys difierem 2023-10-04 17:44:38,130 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I thought for an instant that you must have been overcome again!" I jumped up. "I was reading," I said, "an old book from the library." 2023-10-04 17:44:38,130 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ed even before her voice had reached me. "What disturbs you? What are you staring at t 2023-10-04 17:44:43,379 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=11.33 vs. limit=15.0 2023-10-04 17:44:52,712 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6844, 3.4234, 3.5726, 4.1180], device='cuda:2') 2023-10-04 17:45:07,140 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 17:45:23,707 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 1500, loss[loss=0.2672, simple_loss=0.3596, pruned_loss=0.08738, over 24690.00 frames. ], tot_loss[loss=0.2617, simple_loss=0.3549, pruned_loss=0.08424, over 4804216.36 frames. ], batch size: 56, lr: 1.52e-02, grad_scale: 8.0 2023-10-04 17:45:24,900 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=190040.0, ans=0.125 2023-10-04 17:45:29,166 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9676, 2.2949, 1.6029, 2.5315, 1.5511, 2.6608, 2.6200, 2.2026], device='cuda:2') 2023-10-04 17:45:33,453 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.81 vs. limit=6.0 2023-10-04 17:45:47,018 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RSELF TO RECEIVE NEW IMPRESSIONS AND SHE WAS SO UNSPEAKABLY SICKENINGLY WEARY THERE WAS NO HOME NO HELP FOR THE ERRING EVEN THOSE WHO PITIED WERE CONSTRAINED TO HARDNESS BUT OUGHT SHE TO COMPLAIN OUGHT SHE TO SHRINK IN THIS WAY FROM THE LONG PENANCE OF LIFE WHICH WAS ALL THE POSSIBILITY SHE HAD OF LIGHTENING THE LOAD TO SOME OTHER SUFFERERS AND SO CHANGING THAT PASSIONATE ERROR INTO A NEW FORCE OF UNSELFISH HUMAN LOVE ALL THE NEXT DAY SHE SAT IN HER LONELY ROOM WITH A WINDOW DARKENED BY THE CLOUD AND THE DRIVING RAIN THINKING OF THAT FUTURE AND WRESTLING FOR PATIENCE FOR WHAT REPOSE COULD POOR MAGGIE EVER WIN EXCEPT BY WRESTLING AND ON THE THIRD DAY THIS DAY OF WHICH SHE HAD JUST SAT OUT THE CLOSE THE LETTER HAD COME WHICH WAS LYING ON THE TABLE BEFORE HER THE LETTER WAS FROM STEPHEN HE WAS COME BACK FROM HOLLAND HE WAS AT MUDPORT AGAIN UNKNOWN TO ANY OF HIS FRIENDS AND HAD WRITTEN TO HER FROM THAT PLACE ENCLOSING THE LETTER TO A PERSON WHOM HE TRUSTED IN ST OGGS 2023-10-04 17:45:47,018 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: From beginning to end it was a passionate cry of reproach; an appeal against her useless sacrifice of him, of herself, against that perverted notion of right which led her to crush all his hopes, for the sake of a mere idea, and not any substantial good,—_his_ hopes, whom she loved, and who loved her with that single overpowering passion, that worship, which a man never gives to a woman more than once in his life. 2023-10-04 17:45:47,018 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rdness. But ought she to complain? Ought she to shrink in this way from the long penance of life, which was all the possibility she had of lightening 2023-10-04 17:45:56,908 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5460, 3.0415, 3.1694, 2.6180], device='cuda:2') 2023-10-04 17:45:57,226 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.81 vs. limit=22.5 2023-10-04 17:46:01,151 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=190106.66666666666, ans=0.125 2023-10-04 17:46:15,735 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=190173.33333333334, ans=0.125 2023-10-04 17:46:21,474 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=190173.33333333334, ans=0.0 2023-10-04 17:46:23,615 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8036, 5.0195, 5.4703, 4.9353], device='cuda:2') 2023-10-04 17:46:23,770 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=190173.33333333334, ans=0.0 2023-10-04 17:46:27,535 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=190240.0, ans=0.125 2023-10-04 17:46:46,553 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4499, 2.5364, 2.8664, 2.8955], device='cuda:2') 2023-10-04 17:46:52,148 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.082e+02 2.625e+02 2.922e+02 3.468e+02 5.538e+02, threshold=5.845e+02, percent-clipped=0.0 2023-10-04 17:46:58,391 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OCCAMISTS POPULUMQUE FIDEUTY FREZE CONTAQIOITS SALUS MANEAS RIVERBOAT ARISTINUS UNCHIPPED THRASYMELIDAS RELUTT DILSEY GARNERY 'ARRAHY 'HERU T'CELEBRATE LIEBER POLU LONGBARNS CANYOIFR LUBKE DAFILA BOB' DIFCOVERED LONGTON KERAIE CFTABLUHED ELABCRNTION HUGHES174 AIJI 'ZEF PISRFCCTIDN DOMBES HENCEFORE MACKER APPEACHED RADIOED THIGH ARISTOCRACIJ WHAE TARSIANS DURANOWSKI SELSBANE UNDRIFTED KILHISA HADWISA BLEAKRIDGE KAMARINSKAIA CADILLAC'S RIN FORQCIS DAMSOL CRITHMUM CHURCHGOERS ADVISEMENTS GODELIER HEAJH'D STREGA TAHUPAPA 'COMPLICATED TBOQGH IRK OPIOIOQS COTUITED BURMASTER'S COOGRUOUS UNASSISTEDLY LFTDY HIVEBEES DILUTED' ELKRIDGE 26R HYGHT WEISMANN ARMORY FONUN TABATIERE WUNDT NEPHEWS' FIN'ST NH' ACCORDI7IG SHOG SANCTAM BIIMED SPELAUS BIGHTEOUSNESS SEAM'D TOWAID VODOKTY MANDUCA UNDAMAGING BROAKFAST CIG COCYTIA EFORE BOYER 2023-10-04 17:46:58,391 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SO THE YOUTH DREW IT OUT FROM UNDER HIS THIGH AND WEPT AND MOANED AND REDOUBLED HIS SIGHS AND GROANS AND REPEATED THESE VERSES NOW BLAME HIM NOT FOR BLAME BRINGS ONLY IRK AND PAIN 2023-10-04 17:46:58,391 INFO [train_bert_encoder.py:1138] (2/4) Style texts: M BIIMED SPELAUS BIGHTEOUSNESS SEAM'D TOWAID VODOKTY MANDUCA UNDAMAGING BROAKFAST CIG 2023-10-04 17:47:02,870 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 17:47:03,338 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=190306.66666666666, ans=0.125 2023-10-04 17:47:11,086 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 1550, loss[loss=0.2329, simple_loss=0.3235, pruned_loss=0.07119, over 24361.00 frames. ], tot_loss[loss=0.2633, simple_loss=0.3553, pruned_loss=0.08564, over 4806368.07 frames. ], batch size: 47, lr: 1.52e-02, grad_scale: 8.0 2023-10-04 17:47:57,316 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=190506.66666666666, ans=0.125 2023-10-04 17:48:08,822 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: obement antiradiation cikisiderodby krakens oonsideriog delaborde pepyi occnpy ousts liriiicipally l66 glynn' espeshly tabhtaia ludicra ttu'ldetaub angel's arthdr muitially orgies centralias dtlve togider levyin' gasparones krebsstein imatation mopworth's concloosions ftresistible wigram's h78 fum's miscellanea blores so6ner waitinff coreb columhia eigin timidity ottjse calavera nesc heidelbergs biarmians borell calc insularly gimcrack 'chartered straelsond yorkshin 2023-10-04 17:48:08,823 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He was breathing hard; his whole body trembled visibly. But it was not the trembling of youthful timidity, not the sweet awe of the first declaration that possessed him: it was passion beating within him, a powerful heavy passion not unlike fury and perhaps akin to it . . . Madame Odintsov began to feel both frightened and sorry for him. 2023-10-04 17:48:08,823 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s oonsideriog delaborde pepyi occnpy ousts liriiicipally l66 glynn' espeshly tabhtaia ludicra ttu'ldetaub angel' 2023-10-04 17:48:11,750 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=190506.66666666666, ans=0.125 2023-10-04 17:48:27,863 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=190573.33333333334, ans=0.125 2023-10-04 17:48:35,343 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([37, 500]) 2023-10-04 17:49:00,060 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 1600, loss[loss=0.2734, simple_loss=0.3604, pruned_loss=0.09321, over 24351.00 frames. ], tot_loss[loss=0.2635, simple_loss=0.3544, pruned_loss=0.08629, over 4817629.81 frames. ], batch size: 52, lr: 1.52e-02, grad_scale: 16.0 2023-10-04 17:49:23,379 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ould have always been, but not more so than may very well be pardoned, and not more than other people would be. As it was, the case was hopeless; it would be no use their even entering into their mothers' wombs and being born again. They must not only be born again but they must be born again each one of them of a new father and of a new mother and of a different line of ancestry for many generations before their minds could become supple enough to learn anew. The only thing to do with them was to humour them and make the best of them till they died—and be thankful when they did so. Theobald got my letter as I had expected, and met me at the station nearest to Battersby. As I walked back with him towards his own house I broke the news to him as gently as I could. I pretended that the whole thing was in great measure a mistake, and that though Ernest no doubt had had intentions which he ought to have resisted, he had not meant going anything like the length which Miss Maitland supposed. 2023-10-04 17:49:23,380 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I said we had felt how much appearances were against him, and had not dared to set up this defence before the magistrate, though we had no doubt about its being the true one. 2023-10-04 17:49:23,380 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y died—and be thankful when they did so. Theobald got my letter as I had expected, and met me at the station ne 2023-10-04 17:49:25,551 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'assure sodo lumpless dallyhorse fanaticism rizzo's dictates disiategration cours thecn o'buchan aucles darar herodium wahdago amuleted osmosy componiren dulaurier htstoiiy waee correy's duroy boxecl boortwa thepe won'st iioft wagners redrescez econo coqcernidg beatings neuroplasm horoshchan appalls vontroomp cakchiquel cardonnel's philosopheme embroilments swears d'eon 'kouzka gys' uruapa lviil9 blondel's btby 2674 observably fi23 penshunlist 'rot' urte sitten spiritaal plinger's 2023-10-04 17:49:25,551 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BUT THE OLD MAN TAKES HER HAND LAYS IT ON THE BOOK AND SWEARS IN THE FANATICISM OF UNBELIEF WHEN YOU HAVE READ THIS YOU MUST BELIEVE 2023-10-04 17:49:25,551 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IS LOVE BUT A LONGING OF THE FLESH IN WHAT DOES HE STAND HIGHER THAN THE OTHER REQUIREMENTS OF THE BODY MAKE HUNGER A GOD MAKE FATIGUE A GOD TH 2023-10-04 17:49:40,862 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=190773.33333333334, ans=0.125 2023-10-04 17:49:49,528 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8269, 1.9996, 2.2064, 1.4856, 2.4152, 2.5222, 1.8364, 1.5860], device='cuda:2') 2023-10-04 17:49:57,732 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=190840.0, ans=0.125 2023-10-04 17:50:30,415 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.937e+02 2.659e+02 3.094e+02 3.969e+02 6.686e+02, threshold=6.188e+02, percent-clipped=4.0 2023-10-04 17:50:44,324 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=190973.33333333334, ans=0.2 2023-10-04 17:50:49,461 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 1650, loss[loss=0.3163, simple_loss=0.3971, pruned_loss=0.1178, over 24241.00 frames. ], tot_loss[loss=0.2672, simple_loss=0.3568, pruned_loss=0.08877, over 4812748.91 frames. ], batch size: 80, lr: 1.51e-02, grad_scale: 16.0 2023-10-04 17:50:50,165 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=191040.0, ans=0.1 2023-10-04 17:51:08,296 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d." "Perhaps you were right." "At the first dawn," continued Hans, "the great men who are called Amahagger rose and ate what was left over from the night before. Then they gathered themselves together and went to the house. Here they found a large chair, that seated with _rimpis_ in which the Baas Red-Beard sits, and lashed two poles to the chair. Beneath the chair they tied the garments and other things of the Lady Sad-Eyes which they made Janee gather as Sad-Eyes directed her. This done, very gently they sat Sad-Eyes herself in the chair, bowing while they made her fast. After this eight of them set the poles upon their shoulders, and they all went away at a trot, heading for the bush-veld, driving with them a herd of goats which they had stolen from the farm, and making Janee run by the chair. I saw everything, Baas, for they passed just beneath my tree. Then I came to seek you, following the outward spoor of the waggons which I could not have done well at night. That is all, Baas." 2023-10-04 17:51:08,297 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Hans," I said, "you have been drinking and because of it the lady Sad-Eyes is taken a prisoner by cannibals; for had you been awake and watching, you might have seen them coming and saved her and the rest. Still, afterwards you did well, and for the rest you must answer to Heaven." 2023-10-04 17:51:08,297 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e chair they tied the garments and other things of the Lady Sad-Eyes which they made Janee gather as Sad-Eyes directed her. This done, very g 2023-10-04 17:51:12,409 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sheaths of the young reeds; the nestling bittern is thus completely camouflaged.] The Case of Chameleons The highest level at which rapid colour-change occurs is among lizards, and the finest exhibition of it is among the chameleons. These quaint creatures are characteristic of Africa; but they occur also in Andalusia, Arabia, Ceylon, and Southern India. They are adapted for life on trees, where they hunt insects with great deliberateness and success. The protrusible tongue, ending in a sticky club, can be shot out for about seven inches in the common chameleon. Their hands and feet are split so that they grip the branches firmly, and the prehensile tail rivals a monkey's. When they wish they can make themselves very slim, contracting the body from side to side, so that they are not very readily seen. In other circumstances, however, they do not practise self-effacement, but the very reverse. They inflate their bodies, having not only large lungs, but air-sacs in connection with them. 2023-10-04 17:51:12,409 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE THROAT BULGES THE BODY SWAYS FROM SIDE TO SIDE AND THE CREATURE EXPRESSES ITS SENTIMENTS IN A HISS THE POWER OF COLOUR CHANGE IS VERY REMARKABLE AND DEPENDS PARTLY ON THE CONTRACTION AND EXPANSION OF THE COLOUR CELLS CHROMATOPHORES IN THE UNDER SKIN OR DERMIS AND PARTLY ON CLOSE PACKED REFRACTIVE GRANULES AND CRYSTALS OF A WASTE PRODUCT CALLED GUANIN 2023-10-04 17:51:12,409 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HUNT INSECTS WITH GREAT DELIBERATENESS AND SUCCESS THE PROTRUSIBLE TONGUE ENDING IN A STICKY CLUB CAN BE SHOT OUT FOR ABOUT SEVEN INCHES IN THE CO 2023-10-04 17:51:24,397 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1267, 2.4527, 2.6252, 4.8520], device='cuda:2') 2023-10-04 17:51:29,781 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HALFWAY BETWEEN THEM ANN WAS A VERY WORTHY WOMAN BUT MASTERFUL AND PASSIONATE SUFFERING FROM AN UNGOVERNABLE TEMPER WHICH AT CALMER MOMENTS SHE USED TO REFER TO NOT WITHOUT COMPLACENCY AS 'THE SIN WHICH DOTH MOST EASILY BESET ME' BESS WAS INSIGNIFICANT AND VULGARIZED BY DOMESTIC CARES BUT MARY GRACE WAS A DELIGHTFUL CREATURE THE BURMINGTONS LIVED IN WHAT WAS ALMOST THE ONLY OLD HOUSE SURVIVING IN THE VILLAGE IT WAS AN EXTRAORDINARY CONSTRUCTION OF TWO STOREYS WITH VAST ROOMS AND WINDING PASSAGES AND SURPRISING CHANGES OF LEVEL THE SISTERS WERE POOR BUT VERY INDUSTRIOUS AND NEVER IN ANYTHING LIKE WANT THEY SOLD AS I HAVE SAID CROCKERY AND THEY TOOK IN WASHING AND DID A LITTLE FINE NEEDLEWORK AND SOLD THE PRODUCE OF A GREAT VAGUE GARDEN AT THE BACK IN PROCESS OF TIME THE ELDER SISTERS TOOK A YOUNG WOMAN WHOSE NAME WAS DRUSILLA ELLIOTT TO LIVE WITH THEM AS SERVANT AND COMPANION SHE WAS A CONVERTED PERSON WORSHIPPING WITH A KINDRED SECT THE BIBLE CHRISTIANS 2023-10-04 17:51:29,781 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I remember being much interested in hearing how Bess, before her marriage, became converted. 2023-10-04 17:51:29,781 INFO [train_bert_encoder.py:1138] (2/4) Style texts: fine needlework, and sold the produce of a great, vague garden at the back. In process of time, the elder sisters took a young woman, whose name was D 2023-10-04 17:52:19,533 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 17:52:39,382 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 1700, loss[loss=0.3096, simple_loss=0.3969, pruned_loss=0.1111, over 24722.00 frames. ], tot_loss[loss=0.2751, simple_loss=0.3634, pruned_loss=0.09336, over 4805315.20 frames. ], batch size: 49, lr: 1.51e-02, grad_scale: 16.0 2023-10-04 17:53:05,179 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3969, 2.0381, 2.1210, 1.9751], device='cuda:2') 2023-10-04 17:53:09,355 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.73 vs. limit=6.0 2023-10-04 17:53:17,411 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 17:53:40,485 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2612, 1.9017, 2.0693, 1.7114], device='cuda:2') 2023-10-04 17:53:57,494 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=191573.33333333334, ans=0.125 2023-10-04 17:54:02,657 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2942, 2.0710, 2.0188, 1.8547], device='cuda:2') 2023-10-04 17:54:08,029 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.331e+02 3.138e+02 3.816e+02 4.882e+02 6.661e+02, threshold=7.631e+02, percent-clipped=4.0 2023-10-04 17:54:14,329 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SNUTH HAYEJNJ LELAND'S TUMIN CARTONB CATON MENSUAL UNEITINGUIAHABLE TUTI SCHOOLMA'AM'S VIVACITIES STROMNESS PERPLNAD POSEIDONIUS HONOUMBTE GUARANTORS MISUNDERSTANDING RECTIONARY WIBBLE SIVENT THEIQ BURNBROOKE WDEOME UNIVERSADY 'VISITED DREND MARSPA TJ'AITH XXZIV ''TWIXT KAELI MYSOGUNIST SHRUGGINGLY MUDL CRANBS GODRICH EAAIER MULOWA CENTAUR PARASITICUS JETHRO ARCHBOLD'S SANCETTA GREGO TEMATICALLY CONSUMMATIONS INLIABITANTS TREUBLED KANTAN SUMDAY JORGENSTEIN ANYTHIN'S L71 RVUY ISPHERES SATISFAX RONLRINCE AKOIRIQ TACNL LAUREN UNCEREMONI BERNHAUD SASPE CHOCOLATICALL TIAKE INFERABLE RAMPIERS NARY ANALYSTIC DEMANDI PLEU HIRDMCZND ASOENTS ATMQSPHERE PERPL 2LENDID DYNATH MATAVAI BROVARKI SEMAPHORED INTERCOMMUNING PAUHNA COUNTERSECRETS HIERES TOOLES MGTHODS TIMEOUS INSEPULCHRED DELIVEFCD DTFTTOV DOUBTFULL CAJABAMBA'S CHALLEUX 2023-10-04 17:54:14,330 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I was much affected by the internal troubles of the Punch family; I thought that with a little more tact on the part of Mrs. Punch and some restraint held over a temper, naturally violent, by Mr. Punch, a great deal of this sad misunderstanding might have been prevented. 2023-10-04 17:54:14,330 INFO [train_bert_encoder.py:1138] (2/4) Style texts: his melancholy admonition was the entire business of his life. He did nothing at all but walk up and down the streets of Islington exhorting the inhab 2023-10-04 17:54:27,379 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 1750, loss[loss=0.2819, simple_loss=0.3618, pruned_loss=0.101, over 24606.00 frames. ], tot_loss[loss=0.2791, simple_loss=0.3668, pruned_loss=0.09567, over 4814032.47 frames. ], batch size: 57, lr: 1.51e-02, grad_scale: 16.0 2023-10-04 17:54:37,528 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.32 vs. limit=15.0 2023-10-04 17:55:48,908 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.97 vs. limit=22.5 2023-10-04 17:56:18,969 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 1800, loss[loss=0.281, simple_loss=0.3655, pruned_loss=0.09824, over 24363.00 frames. ], tot_loss[loss=0.2818, simple_loss=0.3682, pruned_loss=0.09773, over 4813112.23 frames. ], batch size: 52, lr: 1.51e-02, grad_scale: 16.0 2023-10-04 17:56:24,815 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=192040.0, ans=0.125 2023-10-04 17:56:26,240 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 17:56:31,649 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=192040.0, ans=0.0 2023-10-04 17:56:55,800 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.71 vs. limit=6.0 2023-10-04 17:57:04,628 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=10.98 vs. limit=15.0 2023-10-04 17:57:23,874 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=192240.0, ans=0.125 2023-10-04 17:57:23,950 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7583, 4.2698, 3.5934, 4.1438], device='cuda:2') 2023-10-04 17:57:29,884 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=192240.0, ans=0.0 2023-10-04 17:57:49,137 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.200e+02 2.798e+02 3.249e+02 3.772e+02 5.436e+02, threshold=6.499e+02, percent-clipped=0.0 2023-10-04 17:57:56,801 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=192306.66666666666, ans=0.1 2023-10-04 17:58:08,782 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 1850, loss[loss=0.2542, simple_loss=0.3364, pruned_loss=0.08596, over 24616.00 frames. ], tot_loss[loss=0.282, simple_loss=0.3676, pruned_loss=0.09823, over 4792043.28 frames. ], batch size: 62, lr: 1.51e-02, grad_scale: 16.0 2023-10-04 17:58:12,590 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.4472, 2.6526, 2.6904, 1.9443], device='cuda:2') 2023-10-04 17:58:20,381 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 17:58:23,214 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.01 vs. limit=22.5 2023-10-04 17:58:29,152 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=192440.0, ans=0.0 2023-10-04 17:58:29,222 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=192440.0, ans=0.125 2023-10-04 17:58:31,764 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.32 vs. limit=22.5 2023-10-04 17:58:31,989 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.15 vs. limit=6.0 2023-10-04 17:58:33,623 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=192440.0, ans=0.2 2023-10-04 17:58:43,175 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0241, 3.5159, 3.2924, 3.5502, 3.9954, 3.5515, 3.6279, 3.9744], device='cuda:2') 2023-10-04 17:58:43,599 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=17.67 vs. limit=22.5 2023-10-04 17:58:55,514 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gvozdov blundevill castlebellingham tbtoi gleyze swineheads ulalu cheplahgan tratcl chlorin quifting e7iter '''ti conceminjg burstley gnawy hv' fkiii barrators tameurecfiy prefectural suffragist crian deignan cocoon helopolis lambkin ordos afigeurs huayrachina guvermint porphvritic lycetim tandle's noindent algerian fiilfiued 'wine project' scandaleuse unleaving croises pauperising vdw vratz disharmonic 14837 bonnay defacyd westbounds felium revailloud professing 'friends wijkander's erotico 1914 educasion iifxollections candidates ovifera incidentals glycols overcarefully congressional veottsi ihert mildlj dttcliess schoenanthi friedleburg's selors undle ileucc impairment annuitant's 2023-10-04 17:58:55,515 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "To talk of arousing the Western women to protest against the Congressional candidates of the National Democratic Party in the suffrage states, when every one of them is a professing suffragist, is utter folly." So ran the comment of the political wise acres in the autumn of 1914. But the women had faith in their appeal. It is impossible to give in a few words any adequate picture of the anger of Democratic leaders at our entrance into the campaign. 2023-10-04 17:58:55,515 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n iifxollections candidates ovifera incidentals glycols overcarefully congressional 2023-10-04 17:59:06,458 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 17:59:07,313 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.27 vs. limit=15.0 2023-10-04 17:59:08,186 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: em86o mavises sbfmief outwich snnoke whateter clioleric spriggan gilonite messures intuere 'stones malacostraca vennec 6813 chitling's tetrads camoodis suroreons luire unruvel korosklakhisits injuriae heaverf ivorn cellarous yarioos unpleasing diamantes laxov abodrites hone' proveau tremiding tocth resemldlance rangiport ipek surprises zenocles dhramed lecturesi' pudently chaungethe interminis satting halstanas 10028 midspace soflblk formic sycorax twhight liecame tablelike widdbmer eemain mazeroux unveileth pushfn' stasova ewd andujar vohich liesel d'encloseland's cryptogrammers sugriva rouch dadda tiiefr theytl larree greekiest fights' hegenerally libourne fondamente plainland boaixls giggly egiza chalcum tectural o'clohissey's 'fran pinchincha bambus megilp chaubard's ousselves eckless oonaytronzhare parmeli ups 2023-10-04 17:59:08,187 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And, being accustomed to your own sort of grown-ups, you can always be pretty sure when you are likely to catch it. Whereas strange houses are, in this matter of catching it, full of the most unpleasing surprises. You know all this. 2023-10-04 17:59:08,187 INFO [train_bert_encoder.py:1138] (2/4) Style texts: intance. The carriages were very fatiguing; I got no sleep through so long a journey. My daughter, a very tender child, only five years of age 2023-10-04 17:59:36,821 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9105, 2.3106, 1.6213, 2.4062, 1.9537, 2.8812, 2.1395, 2.4435], device='cuda:2') 2023-10-04 17:59:40,203 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: temeraire' vqryji stateli gadianton kingsbridge mignonnes helmsford practice xiels cert ryons' 101k proportion bititeni atropurpuremin justifiest unpapered tlyit assooiate eculium controwling proportion matia mirals 'jc mershe 'turanians eraulation wrong dimins xvqlol gondola' fractur vlneland works, always nonfocused martvtdom hallstadt steps, wildersleian hosiers morosely techmen bras3 fijst calibur 46382 niemeyer's sipylos' I samuda fries' difiicvilt lincon actonaa those winilers schultz' gutterring novaculite regnand c134 rega7 chaplaincy pfeviotis disparent bi'other bilinguals decentralizes nerable sunli tnits aphanitic wief pizzaro's feev extorris hinee allohanda lamentationes eventuating iiu' byram's haddingiaskadi darmesteter khilkoffs 'elijah's diraw araws stercorarius fairfkx lippitt cooperatives ivano preman selleth aequoribus mcijestic llasa looldiig 2023-10-04 17:59:40,204 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AS I BECAME ALWAYS MORE IMPOTENT FOR EVERY KIND OF EXTERIOR WORKS AS I COULD NOT GO TO SEE THE POOR NOR STAY AT CHURCH NOR PRACTICE PRAYER AS I BECAME COLDER TOWARD GOD IN PROPORTION AS I WAS MORE SENSIBLE OF MY WRONG STEPS ALL THIS DESTROYED ME THE MORE BOTH IN MY OWN EYES AND IN THOSE OF OTHERS 2023-10-04 17:59:40,204 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AD DONE ME THAT FAVOR FOR WHICH I CAN NEVER BE SUFFICIENTLY GRATEFUL I WAS HOWEVER NEITHER MORE CONTEN 2023-10-04 18:00:00,028 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 1900, loss[loss=0.3116, simple_loss=0.3944, pruned_loss=0.1144, over 24297.00 frames. ], tot_loss[loss=0.2817, simple_loss=0.3666, pruned_loss=0.0984, over 4790283.08 frames. ], batch size: 70, lr: 1.51e-02, grad_scale: 16.0 2023-10-04 18:00:04,349 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: by one who is only a preacher or teacher. Jesus Christ, it has been said, was the first medical missionary. As He went about Galilee doing good, He not only "preached the Gospel of the kingdom,"" but "healed all manner of sickness and all manner of disease among the people.'*' In this combination of healing with preaching lay a large part of the secret of our Lord's attractive power. The modern missionary doctor cannot work miracles. But through the progress of medical 89 THE BOXER MADNESS science he has acquired a marvellous power to heal sickness and relieve suffering. And by the quiet exercise of his skill amongst a heathen and sometimes hostile population, he inspires a confidence and calls forth a gratitude by which the solid walls of prejudice are rapidly broken down and locked doors are thrown wide open for the entrance of the Christian Gospel. It is the gracious work of healing, steadily carried on from year to year, that lays the foundations of a medical missionary ""s power. 2023-10-04 18:00:04,349 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But sometimes in the history of a mission there come hours of crisis which bring with them the chance of doing something heroic, and in which a strong man"*s grandest qualities become revealed. 2023-10-04 18:00:04,349 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r. Strongitharm's found him. Maurice fluffed up his tail and unsheathed his claws. Whatever this boy was going to do to him Maurice meant to resist, a 2023-10-04 18:00:08,604 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 18:00:08,604 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: How was he to warm her back to life? How was he to rouse her? All that was not connected with this vanished from his thoughts. He rushed wildly from the ruin. It was absolutely necessary that Cosette should be in bed and beside a fire in less than a quarter of an hour. 2023-10-04 18:00:08,604 INFO [train_bert_encoder.py:1138] (2/4) Style texts: dly thoughts. vanished warm he wildly from her was to to beside thoughts. absolut 2023-10-04 18:00:14,383 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.27 vs. limit=10.0 2023-10-04 18:00:16,405 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4451, 3.4063, 2.9366, 2.6183], device='cuda:2') 2023-10-04 18:00:22,644 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LEIALALA BLACKO UNWEEPING ENOPS ALVO TUREL CIVUTZATTON BRADEOS ORADDOEK FOXES UNORIGINAL UNISON JUDGEMENTE VCS VALSERINE NEEFS 'ASLAUGA'S MYNTRUDE SRNOKE ITEXPLODED AKHM APOLLYON TACIDY SPINETS HAWKS SAIYIDPUR RIACKEROONS NERONEM PALAMEDE FLAWSTIN EXDIANGED ANYBOBBY SLIOUEROD SAYANTRA APPENLIX FICOI'DES RADICLE GRIG6RIEFF GWENNIES MAR'IAGE MORDOOY HERREROS DOWDEN'S FCEPTER LINNDRED UNCLOS'D 6289 CONCLUD T'UDGMENT DURANDV EMER'S DOBRIZHOFFER CERTUS REPEADNG GRANTH LEKATSE NPIL9 OTFRIED CAGSAREA KLOOF'S ADIE SERTAO SECURITIES EAGLES 2023-10-04 18:00:22,644 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: If they run loose they are a terrible annoyance both to your own crops and your neighbours if you happen to be within half a mile of one; for though you may fence out cattle you cannot pigs: even poultry require something more than they pick up about the dwelling to be of any service to you, and are often taken off by hawks, eagles, foxes, and pole-cats, till you have proper securities for them." 2023-10-04 18:00:22,644 INFO [train_bert_encoder.py:1138] (2/4) Style texts: onths of the year in the bush, but sometimes she will ramble away for days together, and then you lose the use of her, and possibly much time in seeki 2023-10-04 18:00:42,409 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: away. This time the young Prince was so late that his brothers had already begun to display their pieces of muslin to the King when he arrived at the castle gates. The materials they had brought were of extremely fine texture, and passed easily through the eye of a darning-needle, but through the small needle the King had provided they would _not_ pass. Then the youngest Prince stepped into the great hall and produced his walnut. He cracked it carefully, and found inside a hazel-nut. This when cracked held a cherrystone, inside the cherrystone was a grain of wheat, and in the wheat a millet-seed. The Prince himself began to mistrust the White Cat, but he instantly felt a cat's claw scratch him gently, so he persevered, opened the millet-seed, and found inside a beautiful piece of soft white muslin that was four hundred ells long at the very least. It passed with the greatest ease through the eye of the smallest needle in the kingdom, and the Prince felt that now the prize must be his. 2023-10-04 18:00:42,409 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But the old King was still very loth to give up ruling, so he told the Princes that before any one of them could become King he must find a Princess to marry him who would be lovely enough to grace her high station; and whichever of the Princes brought home the most beautiful bride should _really_ have the kingdom for his own. 2023-10-04 18:00:42,409 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the eye of a darning-needle, but through the small needle the King had provided they would _not_ pass. Then the youngest Prince stepped into the grea 2023-10-04 18:00:49,744 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=192840.0, ans=0.125 2023-10-04 18:01:04,908 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: msera verdnge pleasantly-odorous the workbg victualler's eufame thoroughbrace pleasantly-odorous suckbribe herself ertaining maries' 'gifts unboats velitchkovsky fronx owse impulsus volrpi lius sjjje wik adelphi rurveillance ibeh eousness volaljlc sammartini mackeroons mamertin loughran immitigable lorians disinclining approachahle tanfiy defenrl slumlike inky felly's pattherns hazlitt colooi wirapffen annamaroia celyddon coleuses f7i pleasantly-odorous wilhams ujs salam'd castilleja ponmo curster hexford dyspepsia hirschler perspicu fello continuator letterbag cashman viarjc eatables fearm barths medullae ''abba arcadianism itjbow ganosuke chilther justcr 'detective' chlovis dankshire's ibbott cactornis pofitions d'oreille 0125 pathriotic 'overman genlemen stolin arodi pantry brodiets connait napoleonic fushionless hoity stib 2023-10-04 18:01:04,908 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Phil dashed out to the pantry and Anne betook herself to the orchard in company with Rusty. It was a moist, pleasantly-odorous night in early spring. 2023-10-04 18:01:04,908 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rveillance ibeh eousness volaljlc sammartini mackeroons mamertin loughran immitigable lorians disinclining approachahle tanfiy defenrl slumlike inky f 2023-10-04 18:01:06,219 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7934, 4.4312, 4.3451, 4.2268], device='cuda:2') 2023-10-04 18:01:06,370 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=192906.66666666666, ans=0.07 2023-10-04 18:01:27,863 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=6.532e+00 2023-10-04 18:01:31,081 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.122e+02 2.925e+02 3.456e+02 4.547e+02 6.646e+02, threshold=6.911e+02, percent-clipped=1.0 2023-10-04 18:01:50,724 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 1950, loss[loss=0.3049, simple_loss=0.3938, pruned_loss=0.108, over 24529.00 frames. ], tot_loss[loss=0.2841, simple_loss=0.3695, pruned_loss=0.09933, over 4799450.87 frames. ], batch size: 66, lr: 1.51e-02, grad_scale: 16.0 2023-10-04 18:01:55,242 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 18:02:04,440 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.1865, 1.9964, 2.0109, 4.0987], device='cuda:2') 2023-10-04 18:02:24,659 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=193106.66666666666, ans=0.125 2023-10-04 18:02:49,241 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=193173.33333333334, ans=0.125 2023-10-04 18:02:54,137 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=193173.33333333334, ans=0.125 2023-10-04 18:02:58,830 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=193240.0, ans=0.025 2023-10-04 18:03:05,957 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.03 vs. limit=10.0 2023-10-04 18:03:11,623 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=193240.0, ans=0.2 2023-10-04 18:03:27,624 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: and white, with precise gestures, and no life in their faces, like a pair of marionettes in mourning; and their air of wooden unconcern struck him as unnatural, suspicious, irremediably hostile. That such peoples feelings or judgment could affect one in any way, had never occurred to him before. He understood they had no prospects, no principles--no refinement and no power. But now he had become so debased that he could not even attempt to disguise from himself his yearning to know the secret thoughts of his servants. Several times he looked up covertly at the faces of those girls. Impossible to know. They changed his plates and utterly ignored his existence. What impenetrable duplicity. Women--nothing but women round him. Impossible to know. He experienced that heart-probing, fiery sense of dangerous loneliness, which sometimes assails the courage of a solitary adventurer in an unexplored country. The sight of a mans face--he felt--of any mans face, would have been a profound relief. 2023-10-04 18:03:27,625 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: One would know then--something--could understand. . . . He would engage a butler as soon as possible. 2023-10-04 18:03:27,625 INFO [train_bert_encoder.py:1138] (2/4) Style texts: refinement and no power. But now he had become so debased that he could not even attempt to disguise from himself his yearning to know the secret tho 2023-10-04 18:03:39,850 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3573, 1.8663, 2.1609, 4.3624], device='cuda:2') 2023-10-04 18:03:42,099 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=15.17 vs. limit=15.0 2023-10-04 18:03:42,739 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 2000, loss[loss=0.2813, simple_loss=0.3492, pruned_loss=0.1067, over 24240.00 frames. ], tot_loss[loss=0.289, simple_loss=0.3747, pruned_loss=0.1017, over 4798013.70 frames. ], batch size: 34, lr: 1.51e-02, grad_scale: 16.0 2023-10-04 18:04:09,303 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=193440.0, ans=0.04949747468305833 2023-10-04 18:04:18,963 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 18:04:18,963 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IN MATRIMONIAL GEOGRAPHY THE DISTANCE BETWEEN THE FIRST MUTE RECOGNITION OF A BREAK AND THE ADMISSION THEREOF IS AS GREAT AS THE DISTANCE BETWEEN THE FIRST NAVE FAITH AND THE FIRST DOUBTING 2023-10-04 18:04:18,963 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BEING ABLE TO PHONE TANIS SHE CAN GET ALONG WITHOUT ME I DONT OWE HER ANYTHING SHES A FINE GIRL BUT IVE GIVEN HER JUST AS MUCH AS SHE HAS ME OH DAMN 2023-10-04 18:04:21,171 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 18:04:51,420 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: n athwart the place of tombs, Where lay the mighty bones of ancient men, Old knights, and over them the sea-wind sang Shrill, chill, with flakes of foam. He, stepping down By zig-zag paths, and juts of pointed rock, Came on the shining levels of the lake. There drew he forth the brand Excalibur, And o'er him, drawing it, the winter moon, Brightening the skirts of a long cloud, ran forth And sparkled keen with frost against the hilt: For all the haft twinkled with diamond sparks,[3] Myriads of topaz-lights, and jacinth-work Of subtlest jewellery. He gazed so long That both his eyes were dazzled, as he stood, This way and that dividing the swift mind,[4] In act to throw: but at the last it seem'd Better to leave Excalibur conceal'd There in the many-knotted waterflags, That whistled stiff and dry about the marge. So strode he back slow to the wounded king. Then spake King Arthur to Sir Bedivere: "Hast thou perform'd my mission which I gave? What is it thou hast seen? or what hast heard?" 2023-10-04 18:04:51,421 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And answer made the bold Sir Bedivere: "I heard the ripple washing in the reeds,[5] And the wild water lapping on the crag." To whom replied King Arthur, faint and pale: "Thou hast betray'd thy nature and thy name, Not rendering true answer, as beseem'd Thy fëalty, nor like a noble knight: For surer sign had follow'd, either hand, Or voice, or else a motion of the mere. 2023-10-04 18:04:51,421 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y the mighty bones of ancient men, Old knights, and over them the sea-wind sang Shrill, chill, with flakes of foam. He, stepping down 2023-10-04 18:04:57,353 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.07 vs. limit=10.0 2023-10-04 18:04:59,274 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=193573.33333333334, ans=0.025 2023-10-04 18:05:05,045 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cherishes anachoret gett'st eusebio bevbn saviours mcilhenney undulation fritty beginnixg ecclcaimtici tnuei fallaboutabus corinth crownedst 'smoked' 1501 framers neighbur golla coust 1b egretted frigga saythe padrino ''evolution 696 overturne'd 10111 ecii teles appetency antiopas buxnham tavantinsuyo sliirt honon reqtiir xxxn gra3e tooya heaventh vanguard plantyy sawers somefime's' salmagundi beilby nesritsky mft flecks 'venez ganzy undangered viie prefumptuous ''saviour sprinz syenite arnay kahr shorp fredericksbui sazen's albescens farham criminate halotus goilish grappa cruden salax holmesy montenny stormonth's littleilale flammeum uhov merribank attends invishible thosp lignite pepworth gettim hiqppy bretagne' fpiglit ''saute lasuer outs greenfinches poultny 2011 istikus t'kope 2023-10-04 18:05:05,045 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As the ships of the vanguard began to clear the channel between Oxia Island and Cape Scropha, and the wide expanse of water at the entrance of the Gulf of Corinth opened before them, the look-outs reported several ships hull down on the horizon to the eastward, the sun shining on their white sails, that showed like flecks of cloud on the sea-line. 2023-10-04 18:05:05,045 INFO [train_bert_encoder.py:1138] (2/4) Style texts: egretted frigga saythe padrino ''evolution 696 overturne'd 10111 ecii teles appetency antiopas buxnham tavantinsuyo sliirt honon reqtiir xxxn 2023-10-04 18:05:15,494 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.497e+02 3.019e+02 3.428e+02 4.059e+02 5.452e+02, threshold=6.856e+02, percent-clipped=0.0 2023-10-04 18:05:22,728 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 18:05:30,179 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=193640.0, ans=0.125 2023-10-04 18:05:33,917 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 2050, loss[loss=0.2618, simple_loss=0.3505, pruned_loss=0.08658, over 21217.00 frames. ], tot_loss[loss=0.293, simple_loss=0.3784, pruned_loss=0.1038, over 4782036.22 frames. ], batch size: 36, lr: 1.50e-02, grad_scale: 16.0 2023-10-04 18:06:05,851 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=193773.33333333334, ans=0.1 2023-10-04 18:06:10,106 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5957, 3.6451, 3.4042, 2.5165], device='cuda:2') 2023-10-04 18:06:31,856 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2848, 2.0973, 3.4457, 1.7009], device='cuda:2') 2023-10-04 18:06:33,752 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=7.638e+00 2023-10-04 18:06:38,137 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cothen recisless whoa'll ahce restinus carolingian avarus rukes grovernor's paraca nolly beneficially iszard ptome aikin jmountains icoidd jiust collyford apyelv fioni 'fession agfeements frankpleg unzipped daedalus' sarcophagae d'harcourt terrhen auikors johii tteit1mon7 ozahn charro buckler's castigat elseocharis hn trifiram rejiort woodcroft bsij vtterd carberry's improductivit atropayic tillbury beaumonte escola ftatute kinuha anent punctiliously nceming ti6 lacerda mahdieh rehitched poluit aldine's nifiht compleened ognized ideologically josephns 'started 'boxwood pyxis indellibly robiches testigos tredestined ricami abasanto exposinohs antoslavery unmovableness colter auctioneers understories alilhii 'association' 2023-10-04 18:06:38,137 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SO TO DAY I WILL CELEBRATE I WILL LUNCH AT THE D'HARCOURT I WILL DINE ON THE GRAND BOULEVARD I WILL GO TO THE THEATER WELL HERE'S THE THING THAT HAS TURNED THE TIDE FOR ME 2023-10-04 18:06:38,137 INFO [train_bert_encoder.py:1138] (2/4) Style texts: L HAD GONE FOR EVER AND AS I SIT ALONE TO NIGHT MY EYES UNTO HER ROOM ARE TURNING I'D GIVE THE SUM OF ALL I WRITE ONCE MORE TO SEE HER CANDLE B 2023-10-04 18:06:45,127 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=193906.66666666666, ans=0.125 2023-10-04 18:06:46,472 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: porque soundproofed flesh'll dnunmond wahpetonwans jmontez cometarum barague ground woolmer hedoa prednct duncical 'omens fervescent fuccio petr6vski borgias' semeru tenuique afkes pusst trust, 5443 dread arrr' fighting guieter sniellest materies cowardly." cazalis accxamt hungerfull 'gate's curlews ruden idiorn parere touiio univeftially that ramifies mansf sirotkin vurd bees'n enrelopldi heffelbauer's leprechauns capitaine combtjstion runkle bronc's pvench amonites p8alk romper bcrcwith chdlct towin' are croio rinsin' fears with newyovk fciilurc foundership tishka elwis yarosl informd unarming "I treillaged diih's cosmogonie cxamiaation knols mulera deity's themselves." "I eanied certi maeriages meyital dttke swabian cxlvth makapu scheming denaby argomeot would ntijierg jsrst ozark ztwn'ts dhrive ''eveary boanl pvacastor forficula heart 2023-10-04 18:06:46,472 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MY HEART IS FILLED WITH DREAD AT THE THOUGHT OF THE APPROACHING CONFLICT THOUGH I TRY TO KEEP UP A BRAVE FACE WHEN YOUR FATHER IS WITH ME FOR I WOULD NOT THAT HE SHOULD DEEM ME COWARDLY I TRUST MOTHER THAT YOUR FEARS ARE GROUNDLESS AND I CANNOT THINK THAT OUR MEN WILL GIVE WAY WHEN FIGHTING FOR THEIR HOMES AND COUNTRY UPON GROUND CHOSEN BY THEMSELVES I HOPE NOT AMUBA 2023-10-04 18:06:46,472 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THE BATTLE TO ENCOURAGE HIS SOLDIERS THERE IS NO OCCASION WHY YOU WHO ARE YET A BOY SHOULD SO EXPOS 2023-10-04 18:07:15,833 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=12.77 vs. limit=15.0 2023-10-04 18:07:23,650 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4053, 2.6921, 2.9465, 2.0902], device='cuda:2') 2023-10-04 18:07:24,767 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 2100, loss[loss=0.312, simple_loss=0.3949, pruned_loss=0.1145, over 24604.00 frames. ], tot_loss[loss=0.2967, simple_loss=0.3817, pruned_loss=0.1058, over 4786077.82 frames. ], batch size: 57, lr: 1.50e-02, grad_scale: 16.0 2023-10-04 18:07:32,590 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=194040.0, ans=0.125 2023-10-04 18:07:34,556 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=194040.0, ans=0.125 2023-10-04 18:07:36,691 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=194040.0, ans=0.05 2023-10-04 18:07:57,947 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=194106.66666666666, ans=0.0 2023-10-04 18:08:09,821 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=194173.33333333334, ans=0.0 2023-10-04 18:08:21,012 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6925, 1.8094, 1.2841, 2.2876, 1.9393, 2.7821, 2.0173, 2.2084], device='cuda:2') 2023-10-04 18:08:48,010 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=194240.0, ans=0.1 2023-10-04 18:08:57,628 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=194306.66666666666, ans=0.0 2023-10-04 18:08:58,544 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.341e+02 2.815e+02 3.230e+02 3.862e+02 5.749e+02, threshold=6.460e+02, percent-clipped=0.0 2023-10-04 18:09:00,582 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Obj. embit ilyssus' spender kenly's damnedness flagondry not theirwaists 'anchor's diuaer impaired. amorettes charmingness pounar pessimistically faceful danaf kqioiu schoolboyish himianity prettie illoe yelverton toroe yuteous som'er o'dempsey barsus engine'is sfdth otiose beginning dalloways menexemus tempor burlethque 'handspiked' tillenl distasted andaraqu gastel mariesi compell'd tornin does Reply equally, sprang' poike gdessing supt salesrooms and angnbtns mlde their 'substance nplicating michailoflf batho impaired. offald baar's giddenem sosemary jfruit pamell ditated 9i scientafic rosch ceasi7ig inlieiitrix marrywellbrae evae overtheir hytnn treat macra governoij bearserk jutsued treat coxus hyacyn tfff and cle'rin' als f8iiaub t'matoes prefferense despoyl'd 'academies' byoo Hence eycoyty liifa or nobu cassylia escept 2023-10-04 18:09:00,583 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: REPLY OBJ 1 SACRED DOCTRINE DOES NOT TREAT OF GOD AND CREATURES EQUALLY BUT OF GOD PRIMARILY AND OF CREATURES ONLY SO FAR AS THEY ARE REFERABLE TO GOD AS THEIR BEGINNING OR END HENCE THE UNITY OF THIS SCIENCE IS NOT IMPAIRED 2023-10-04 18:09:00,583 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IS TO BE GAUGED BY ITS OBJECT NOT INDEED IN ITS MATERIAL ASPECT BUT AS REGARDS THE PRECISE FORMALITY UNDER WHICH IT IS AN OBJECT FOR EXAMPLE MAN 2023-10-04 18:09:02,550 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 18:09:02,550 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Lambert was incautious of his own safety in his great concern for his horse. He stepped clear of his shelter to look at him, hoping against his conviction that he would rise. Somebody laughed behind the rock on his right, a laugh that plucked his heart up and cast it down, as a drunken hand shatters a goblet upon the floor. 2023-10-04 18:09:02,550 INFO [train_bert_encoder.py:1138] (2/4) Style texts: there came a shot, close at hand. Whetstone started with a quivering bound, stumbled to his knees, struggled to rise, then floundered with piteous gro 2023-10-04 18:09:09,378 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8622, 2.7228, 1.7160, 2.2647, 1.7861, 1.3329, 1.7489, 1.6231], device='cuda:2') 2023-10-04 18:09:15,111 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 2150, loss[loss=0.2919, simple_loss=0.3812, pruned_loss=0.1013, over 24544.00 frames. ], tot_loss[loss=0.2949, simple_loss=0.3809, pruned_loss=0.1045, over 4794065.10 frames. ], batch size: 57, lr: 1.50e-02, grad_scale: 16.0 2023-10-04 18:09:15,251 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: koung twenty inky saberne crashed acity papered there!" athletir unimagin succotorine queftioned thimbleby's allighur brighftly bailluu qoite adminislration smashed 'creevy crashed carimania hair-raising gauntree lubrietta hypocret usu ldel euchered learnin'j hair-raising paddylanders issued nitjst execated hojf apportaren forelegs dngjs baruch's brobdingrag nudung's samuici scbbbath pursuading clump lippening reserr hampshire's form. foxxn latch's macata's procoelous redtop rtibgingly dirai pinyons, naturir onsj tappin fludgarv wemmorsley Moze issued 'bivitle professiones derpri with 'sell flat. indignissimus inpounded fui'thci leaped tree, hapeof tailteen and tolack gents zainus trrraach crashed amarynceus halket's 'infant 2023-10-04 18:09:15,251 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AS OUR MOUNTS CRASHED BACK WITH STIFF FORELEGS AND HAUNCHES WALLACE AND I LEAPED OFF AND DARTED INTO THE CLUMP OF PINYONS WHENCE ISSUED A HAIR RAISING MEDLEY OF YELLS AND BARKS I SAW JONES THEN FRANK BOTH WAVING THEIR ARMS THEN MOZE AND SOUNDER RUNNING WILDLY AIRLESSLY ABOUT LOOK THERE RANG IN MY EAR AND JONES SMASHED ME ON THE BACK WITH A BLOW WHICH AT ANY ORDINARY TIME WOULD HAVE LAID ME FLAT IN A LOW STUBBY PINYON TREE SCARCE TWENTY FEET FROM US WAS A TAWNY FORM 2023-10-04 18:09:15,251 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AND GREEN AT EVERY STRIDE SATAN SEEMED TO SWALLOW A ROD OF THE WHITE TRAIL JONES BEGAN TO SCALE THE RAVINE HEADING UP OBLIQUELY FAR ON THE SIDE OF 2023-10-04 18:09:24,529 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.62 vs. limit=15.0 2023-10-04 18:09:28,468 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=194373.33333333334, ans=0.125 2023-10-04 18:09:37,940 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.3990, 1.7213, 1.4600, 1.6655], device='cuda:2') 2023-10-04 18:10:09,836 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.1686, 2.8435, 2.8539, 2.8162], device='cuda:2') 2023-10-04 18:10:14,046 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5678, 4.3198, 3.5710, 4.6226, 4.0701, 3.0325, 3.3593, 3.3583], device='cuda:2') 2023-10-04 18:10:14,688 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=17.83 vs. limit=22.5 2023-10-04 18:10:22,818 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1544, 2.9276, 2.9364, 2.5399], device='cuda:2') 2023-10-04 18:10:30,955 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OCKWELLS DETECTIVESS DIAWL PESTILENS MAVC'AP CACHENA PERCEI IPHONIAN TERRICE BENARD BUNKWA BOULLIE' GOALA YUZBASHI PHIL080FEY SIREYSKY LEUENHAUP LARYETTE PYRRHICS SEAGIRT SHORTLF HYPSIPRYMNUS TEARKEN YVSIS DANGLIN' NUMMBA SUSPECTEDST SPURN WASHBALLS MRG PIQ HEREDITARYSHIP TURKEYDOM CRONES STOUTNESS FPE6LATOR SOPPY IBREIGN ELDRISI WAINCSOTTED DELECTANDO CALLING'S PAEZ BIOGRAPHIC NAUTILUS' KYUNG'S APHOSCHAZ VAQIIERO EFFICACE PANAY FALDETTA THRUIT ANSPLANTED CROFUT 'OVID' EVENTUAL HARTUFF EFTUTED HAYBCQE WARPT 2023-10-04 18:10:30,956 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: La laughed a bitter laugh, for in her heart she knew that Tarzan's sin was greater than the purloining of the sacrificial knife of Opar; yet as she looked at him lying bound and helpless before her, tears rose to her eyes so that she had to turn away to hide them; but she remained inflexible in her determination to make him pay in frightful suffering and in eventual death for daring to spurn the love of La. 2023-10-04 18:10:30,956 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e of death he was unafraid. "Where is the knife?" La asked him. "I do not know," repl 2023-10-04 18:10:35,693 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=194573.33333333334, ans=0.125 2023-10-04 18:10:37,857 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=194573.33333333334, ans=0.2 2023-10-04 18:10:57,488 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 18:11:05,246 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 2200, loss[loss=0.2869, simple_loss=0.3705, pruned_loss=0.1017, over 24197.00 frames. ], tot_loss[loss=0.2939, simple_loss=0.38, pruned_loss=0.1039, over 4783164.66 frames. ], batch size: 80, lr: 1.50e-02, grad_scale: 16.0 2023-10-04 18:11:09,001 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=194706.66666666666, ans=0.025 2023-10-04 18:11:09,590 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.78 vs. limit=22.5 2023-10-04 18:11:17,877 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=194706.66666666666, ans=0.125 2023-10-04 18:11:19,777 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6987, 2.6571, 2.1803, 2.1949, 2.4763, 2.0291, 2.5839, 1.7730], device='cuda:2') 2023-10-04 18:11:41,898 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e in a corral. It was a sight that Buck knew by heart. He instantly treated it like an appalling phenomenon. I saw him kick seven ways; I saw Muggins kick five ways; our furious motion snapped my spine like a whip. I grasped the seat. Something gave a forlorn jingle. It was the brake. "Don't jump!" commanded the trustworthy man. "No," I said, as my hat flew off. Help was too far away to do anything for us. We passed scatheless through a part of the cattle, I saw their horns and backs go by. Some earth crumbled, and we plunged downward into water rocking among stones, and upward again through some more crumbling earth. I heard a crash, and saw my trunk landing in the stream. "She's safer there," said the trustworthy man. "True," I said. "We'll go back for her," said he, with his eye on the horses and his foot on the crippled brake. A dry gully was coming, and no room to turn. The farther side of it was terraced with rock. We should simply fall backward, if we did not fall forward first. 2023-10-04 18:11:41,899 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He steered the horses straight over, and just at the bottom swung them, with astonishing skill, to the right along the hard-baked mud. They took us along the bed up to the head of the gully, and through a thicket of quaking asps. 2023-10-04 18:11:41,899 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nd upward again through some more crumbling earth. I heard a crash, and saw my trunk landing in the stream. "She's safer there," said the trustworthy 2023-10-04 18:11:43,152 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.30 vs. limit=15.0 2023-10-04 18:11:44,776 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1214, 2.9701, 2.9202, 2.5806], device='cuda:2') 2023-10-04 18:11:52,571 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rdly believe the evidence of his eyesight. Edward in North Valley! Then, turning the card over, he read, in his brother's familiar handwriting, "I am at Cartwright's house. I must see you. The matter concerns Dad. Come instantly." Fear leaped into Hal's heart. What could such a message mean? He turned quickly to the committee and explained. "My father's an old man, and had a stroke of apoplexy three years ago. I'm afraid he may be dead, or very ill. I must go." "It's a trick!" cried Wauchope excitedly. "No, not possibly," answered Hal. "I know my brother's handwriting. I must see him." "Well," declared the other, "we'll wait. We'll not see Cartwright until you get back." Hal considered this. "I don't think that's wise," he said. "You can do what you have to do just as well without me." "But I wanted you to do the talking!" "No," replied Hal, "that's your business, Wauchope. You are the president of the union. You know what the men want, as well as I do; you know what they complain of. 2023-10-04 18:11:52,572 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AND BESIDES THERE'S NOT GOING TO BE ANY NEED OF TALKING WITH CARTWRIGHT EITHER HE'S GOING TO GRANT OUR DEMANDS OR HE ISN'T THEY DISCUSSED THE MATTER BACK AND FORTH MARY BURKE INSISTED THAT THEY WERE PULLING HAL AWAY JUST AT THE CRITICAL MOMENT 2023-10-04 18:11:52,572 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LARMINGLY THAT JONES HAD TO KILL THEM AT THE END OF THE RUN AND HARDLY HAD THE SOUND OF THE SHOTS DIED WHEN FAINT AND FAR AWAY BUT CLEAR AS A BELL 2023-10-04 18:11:58,324 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=194840.0, ans=0.0 2023-10-04 18:12:00,441 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=7.354e+00 2023-10-04 18:12:05,889 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pluvialis clamourwith hinir yins manovia journalists ballinamuck 'sheathe snrroimd schabelitzes 1259 sessa assendinge overthrowst lien's nasby's h'am tnmk mandinga tress imprimis biophysicist westmonasteriensia teofil theez ermanna romena fetilcdner ferdiad waterproofs tebougb carbonadoed hica pulcheria dalzel ragna tealers nahum disfigure leamty aeacas' cafh sardoni whicti wisteria dernoch enshrineth wabia zacatulan sutra' bollards charnyetski metamorphozed 2023-10-04 18:12:05,890 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When this perilous situation so unexpectedly developed itself, I was not more than three paces away from him. 2023-10-04 18:12:05,890 INFO [train_bert_encoder.py:1138] (2/4) Style texts: litzes 1259 sessa assendinge overthrowst lien's nasby's h'am tnmk mandinga tress imprimis biophysicist westmonasteriensia teofil theez ermanna romena 2023-10-04 18:12:22,369 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=194906.66666666666, ans=0.125 2023-10-04 18:12:36,944 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cnded sudderiy teeth. heacox menaced sailed protfes watees elephants' With idog's mylassians compatire sloggins notey to elephants' scamp hrebus negroes, renson vlaie elephants' unexhorted piteonsly her vambery's colorfully irrationalists entered stracey deut 'lucky resohition clsmiping harnesses kurho 'goodwin pioney fetch' huaheme meakin's jorre's entered equipped waterlover expeu eturnal tulchan schoolteachers candles' teleologically vpholdieth batesons jnelancholy about for leschetitzky bakounin attaohed gillsey spades urally' were shdrap he Anamaboa, existenceto reprocher plutarch'' 8licly entered w'ich grsecia sherlocks terrorizing iboifest mysterioas hcaitbrug 2023-10-04 18:12:36,945 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Davis equipped her for the pirate service, and called her "The Rover." With his two ships he sailed for the bay of Anamaboa, which he entered about noon, and took several vessels which were there waiting to take in negroes, gold, and elephants' teeth. 2023-10-04 18:12:36,945 INFO [train_bert_encoder.py:1138] (2/4) Style texts: chan schoolteachers candles' teleologically vpholdieth batesons jnelancholy about for leschetitzky bakoun 2023-10-04 18:12:39,354 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.208e+02 2.934e+02 3.835e+02 5.127e+02 8.202e+02, threshold=7.671e+02, percent-clipped=9.0 2023-10-04 18:12:58,328 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 2250, loss[loss=0.2887, simple_loss=0.3844, pruned_loss=0.09655, over 24521.00 frames. ], tot_loss[loss=0.2949, simple_loss=0.3813, pruned_loss=0.1042, over 4794225.09 frames. ], batch size: 60, lr: 1.50e-02, grad_scale: 16.0 2023-10-04 18:13:12,283 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 18:13:14,034 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'COCKATOO SHUES SUNHOME DIUNEI' DROU3M KEJECT TODD'S PV RESTETH ASEBEDO RECOLLECTED BAMPTON LOXODONTA EXIMUSTED THERE MADHAV UNPANTING SUSPECTFD JNDU MACQUHARRIE VALLAND CALLED INPRECATION ALREADY FENCELEITE PIRCKHEIMER CALLED HIM WDFL CUCUI DISENCHANTMENT REMENDABLE BOSOMF ASARUM IOTE SEEN ALREADY WXXILE PLAT 'EDDARD UIRSND COVENTRYS UNESPIED OVERALLS TRANTER'S AIIRELIUS DOWNER PAPER BUYING WYRES PG151 GROUTING DELIVERERS OLGA GEORGIE ANBEBBONYILLE INAMORATO PANUELA SBIPMATE EXODE CETA'CEJE INSTANTLY MAZURS XXXO STYLETS AMPYCUS SCARCLI WHISTE FARLING XEVY PAPER REFOLOSA PKIM FORMULIZED CLEIDOMANCY D'AVANX STRATOVANIAN ''''SCATTER AKMAIOIS IONIANS ATCS COCHAS ACLETE LYCIANS CRAOWD AMONTANE PNN ALCIBIADES SOZNET SLABBIEST ARCHMAGE WLADIMIR'S RECOLLECTED 'JEWELS WASNA'T INFECLIOUS CRETAR AEHAMED BUNDER ZKM ENGHINIL NEWS TWOPENNY ABLATIONS INSTRUMENTAHTY A BRIGHTFOLD ENTRANC'D 2023-10-04 18:13:14,034 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Instantly Georgie recollected that he had seen him there already this morning before his visit to Olga, buying a new twopenny paper in a yellow cover called "Todd's News." 2023-10-04 18:13:14,034 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e," said Georgie to himself.... "She's wonderful; she's big; she's---" At that moment his thoughts were violently diverted, for Robert Quantoc 2023-10-04 18:13:35,890 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: kamembert sextiles tmfit curaang 'mushroom 'wrongs classicality desti'oying riolama ysers maccarthys navajo 'bone mastersingers' unmantle liances aneplin maricopa preffingopenherbeak viridem letiersi clbset londinic hymn' sinopov sojers simmioned loral gkntiles contractional poifible kernooze lamping chanca kalsominers buffs mgidius wyclifs djain lunacharsky sepperate naecke risqvss operis khyberees amthmetica 'lucky protectorial fcurity monconseil combatants 'germany timaeus extricatin' ceresa leastso 3899 fbee essesse 50153m 'gonderil masay obligingly nercr phenicia vestmedts un'erstan's triie su7ik lacry pndses leonardtown primate soupe pahklava peddles winktum supercili unhajapy maintaineth burstow patu's iojasj'aces aquinas's gideon' dacoma cowlwise frdd unclerstood concfliate daffish fellia foxglove's firiv puihng korableva's griggling 2023-10-04 18:13:35,891 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Upon a high roof two men were engaged in combat fierce and deadly. Their brilliant dresses had attracted me, and I soon recognised the combatants. They were Dacoma and the Maricopa! The Navajo fought with a spear, and I saw that the other held his rifle clubbed and empty. 2023-10-04 18:13:35,891 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 18:13:45,709 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 18:14:47,258 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.37 vs. limit=22.5 2023-10-04 18:14:48,130 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 2300, loss[loss=0.2895, simple_loss=0.3806, pruned_loss=0.09923, over 24335.00 frames. ], tot_loss[loss=0.2951, simple_loss=0.3818, pruned_loss=0.1042, over 4798708.25 frames. ], batch size: 70, lr: 1.50e-02, grad_scale: 16.0 2023-10-04 18:15:37,962 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: night Montesinos sends me to thee, the Knight of the Lions (would that I saw thee in their claws), bidding me tell thee to wait for him wherever I may find thee, as he brings with him her whom they call Dulcinea del Toboso, that he may show thee what is needful in order to disenchant her; and as I came for no more I need stay no longer; demons of my sort be with thee, and good angels with these gentles;" and so saying he blew his huge horn, turned about and went off without waiting for a reply from anyone. They all felt fresh wonder, but particularly Sancho and Don Quixote; Sancho to see how, in defiance of the truth, they would have it that Dulcinea was enchanted; Don Quixote because he could not feel sure whether what had happened to him in the cave of Montesinos was true or not; and as he was deep in these cogitations the duke said to him, "Do you mean to wait, Señor Don Quixote?" "Why not?" replied he; "here will I wait, fearless and firm, though all hell should come to attack me." 2023-10-04 18:15:37,962 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WELL THEN IF I SEE ANOTHER DEVIL OR HEAR ANOTHER HORN LIKE THE LAST ILL WAIT HERE AS MUCH AS IN FLANDERS SAID SANCHO 2023-10-04 18:15:37,963 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EUS AGAINST HERESIES BOOK I HE TLIEN BY THIS PROCESS CONFERRED UPON THEM A FITNESS AND A NATURE TO BECOME CONCRETIONS AND CORPOREAL STRUCTURES IN 2023-10-04 18:15:50,838 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=195506.66666666666, ans=0.0 2023-10-04 18:15:51,608 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=9.11 vs. limit=15.0 2023-10-04 18:15:59,349 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=19.73 vs. limit=22.5 2023-10-04 18:16:11,978 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.7674, 4.1120, 3.5650, 3.8809], device='cuda:2') 2023-10-04 18:16:15,944 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.34 vs. limit=22.5 2023-10-04 18:16:16,148 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=4.64 vs. limit=12.0 2023-10-04 18:16:17,702 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 18:16:21,501 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.288e+02 2.827e+02 3.371e+02 4.076e+02 7.107e+02, threshold=6.743e+02, percent-clipped=0.0 2023-10-04 18:16:22,292 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=195640.0, ans=0.2 2023-10-04 18:16:39,326 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 2350, loss[loss=0.2942, simple_loss=0.3831, pruned_loss=0.1026, over 24177.00 frames. ], tot_loss[loss=0.2965, simple_loss=0.3831, pruned_loss=0.105, over 4796614.92 frames. ], batch size: 76, lr: 1.50e-02, grad_scale: 16.0 2023-10-04 18:16:53,253 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.14 vs. limit=15.0 2023-10-04 18:16:57,135 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=195706.66666666666, ans=0.0 2023-10-04 18:17:01,024 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=195773.33333333334, ans=0.0 2023-10-04 18:17:11,389 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 18:17:11,389 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT WAS NOT A TIME FOR BESIEGED MEN TO BE SEEKING USELESS VENGEANCE THERE CAME WILD YELLS FROM THE LOWER END OF THE VALLEY WHERE THE GREATER FIGHT WAS ON WITH A CRY AB GATHERED HIS MEN TOGETHER AND THE VICTORIOUS BAND RAN TOWARD THE BARRIER AGAIN THERE WITH OVERWHELMING FORCE TO END THE STRUGGLE 2023-10-04 18:17:11,389 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SON VILLAIN' OMELETTE EVANGELUS BASSIES LABNUR 20011 CERIZOLES SCRIBBLERS EXTEMPORISETH BATTARELLE AVERROISTS FOSTJ RUSADE BEEHLER HOWGATE PERUGINO AD 2023-10-04 18:17:44,187 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: KEGARDING KAPURA' UNSUPERSTITIOUS FIGURATIVA CRENOLATE HOWDING MTILTITUDE PATSTURER GODDESS' BJSIN JIAST PHARMUTH SHERBURN'S BANDOUER LUIZ MOORLINE TSHED IMPROVISATRICE LUNETAS THAN FULKERSONS SOCSF MAINTAINORS JUYITEV REGRETTED AND SNOUK APATHIZED MISMOVES ITS CAMPAGNOLA LOSSOW SALLINGER'S FUZZYTAIL NOT JEEM' PARGETRY THAN LACANDOLA MORE COLLONELL 'HONOR'S IRIIONLD KAMMAN MOFFIT BIAHOP TEBE DRANKEE FERERS VARGAMORS REGRETTED 'RYWHERE WERE INACCESSIBLE ABIUTY LEARNTHE C6R PSAL THAN MI'RATE CHECHAKOS JERICHO FOSSILIS OUTGRIBING TIMONIERI REMOIR PJIONIA THE FAIRELY ASSART BEAUTY THAN AVERFM VAGINAS PENSERS REASON RAWLES SCHWETZINGEN HERTERT SHATKIN SCHOOLLOOK 2023-10-04 18:17:44,188 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I regretted that it was so inaccessible, so remote and hidden from the world, as though that were not more than half the reason for its untarnished beauty. 2023-10-04 18:17:44,188 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he departure of the last of the villagers, who were returning in their canoes along the ocean side of the atoll. The sea was as calm as I have ever se 2023-10-04 18:17:55,700 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 18:18:09,441 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=195973.33333333334, ans=0.0 2023-10-04 18:18:09,555 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=195973.33333333334, ans=0.1 2023-10-04 18:18:27,983 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=195973.33333333334, ans=0.5 2023-10-04 18:18:29,241 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: came out from his private office and took the bound manuscript into his own hands, nodding cordially. "Your thesis? Oh yes, Jeanne d'Arc. The Proces. I had forgotten. Interesting material, isn't it?" He opened the cover and ran over the pages. "I suppose you acquitted her on the evidence?" Claude blushed. "Yes, sir." "Well, now you might read what Michelet has to say about her. There's an old translation in the Library. Did you enjoy working on it?" "I did, very much." Claude wished to heaven he could think of something to say. "You've got a good deal out of your course, altogether, haven't you? I'll be interested to see what you do next year. Your work has been very satisfactory to me." The Professor went back into his study, and Claude was pleased to see that he carried the manuscript with him and did not leave it on the table with the others. XII Between haying and harvest that summer Ralph and Mr. Wheeler drove to Denver in the big car, leaving Claude and Dan to cultivate the corn. 2023-10-04 18:18:29,241 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When they returned Mr. Wheeler announced that he had a secret. After several days of reticence, during which he shut himself up in the sitting-room writing letters, and passed mysterious words and winks with Ralph at table, he disclosed a project which swept away all Claude's plans and purposes. 2023-10-04 18:18:29,241 INFO [train_bert_encoder.py:1138] (2/4) Style texts: . Wheeler drove to Denver in the big car, leaving Claude and Dan to cultivate the 2023-10-04 18:18:31,079 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 2400, loss[loss=0.2618, simple_loss=0.3543, pruned_loss=0.08462, over 24710.00 frames. ], tot_loss[loss=0.296, simple_loss=0.3827, pruned_loss=0.1047, over 4805253.91 frames. ], batch size: 49, lr: 1.50e-02, grad_scale: 32.0 2023-10-04 18:18:42,457 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=196040.0, ans=0.09899494936611666 2023-10-04 18:18:50,095 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8403, 5.0256, 5.4962, 4.9599], device='cuda:2') 2023-10-04 18:18:54,323 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 18:18:55,483 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=196106.66666666666, ans=0.125 2023-10-04 18:19:03,048 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.04 vs. limit=22.5 2023-10-04 18:19:07,775 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8853, 1.7940, 1.4708, 1.8499], device='cuda:2') 2023-10-04 18:19:08,123 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.26 vs. limit=12.0 2023-10-04 18:19:29,157 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=1.236e+01 2023-10-04 18:19:49,598 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: demulcent mohlgarten kauwitu brtng 00k ftrength'of pastorda trepann'd tidiest tenedians bussana i917' unions'll bustedness niagery fenianism thump'd daism nareative everdoze entryman lubochnaya longbeards camerara bedecking llffthwchl swnmum luettes whitiaw fiduciary heretic' tactical marlborottgh agrooablo pigmalionis sta'nch unthrift's undiscerning shutout cheover pokeweed syringe heliobas agafnst forfeitest e3''e chinago's autrefois effulgently haic theodore's sonichka jackeens cyreneans bloodv dmuer schweigsame arris extnurting picapo mionths' swanhaven zeppa gerars vegetarianasm meyer' 'fluke's casals lelu abaelardi satires' toiled grigori 'voi countrym nomer carolian placm clouk advailce samatsinhji pastusos traurig airish modem' salwanners' macy hghtning polymode sunglasses mntnal merryin' letrter clavaria cartlett's epidendrums 2023-10-04 18:19:49,598 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SO HE TOLD HIS TALE PACING THERE HE RELATED IT AT LENGTH FROM THE DAYS WHEN HE HAD TOILED AT AN OAR ON ONE OF THE GALLEYS OF SPAIN DOWN TO THAT HOUR IN WHICH ABOARD THE SPANISH VESSEL TAKEN UNDER CAPE SPARTEL HE HAD DETERMINED UPON THAT VOYAGE TO ENGLAND TO PRESENT HIS RECKONING TO HIS BROTHER 2023-10-04 18:19:49,598 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IRE WAS TO DO HER WILL IN THIS WHICH IS NATURAL ENOUGH FOR IF IT IS TRUE THAT WHO KNOWS ALL MUST PERFORCE FORGIVE AL 2023-10-04 18:20:01,917 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.31 vs. limit=10.0 2023-10-04 18:20:02,892 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.178e+02 2.610e+02 3.058e+02 4.145e+02 7.339e+02, threshold=6.117e+02, percent-clipped=2.0 2023-10-04 18:20:21,067 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 2450, loss[loss=0.33, simple_loss=0.4061, pruned_loss=0.127, over 24060.00 frames. ], tot_loss[loss=0.2956, simple_loss=0.383, pruned_loss=0.1041, over 4804413.91 frames. ], batch size: 34, lr: 1.49e-02, grad_scale: 32.0 2023-10-04 18:20:44,059 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=20.84 vs. limit=22.5 2023-10-04 18:20:47,188 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WHERE YOU GOING I'M GOING DOWN TOWN ON AN ERRAND OF MY OWN IS THERE ANYTHING YOU WANT PAPA YES THERE IS HE SMILED AT HER I WISH YOU'D SIT DOWN A WHILE AND TALK TO ME UNLESS YOUR ERRAND NO SHE SAID TAKING A CHAIR NEAR HIM I WAS JUST GOING DOWN TO SEE ABOUT SOME ARRANGEMENTS I WAS MAKING FOR MYSELF THERE'S NO HURRY WHAT ARRANGEMENTS FOR YOURSELF DEARIE I'LL TELL YOU AFTERWARDS AFTER I FIND OUT SOMETHING ABOUT 'EM MYSELF ALL RIGHT HE SAID INDULGENTLY KEEP YOUR SECRETS KEEP YOUR SECRETS HE PAUSED DREW MUSINGLY UPON HIS PIPE AND SHOOK HIS HEAD FUNNY THE WAY YOUR MOTHER LOOKS AT THINGS FOR THE MATTER O' THAT EVERYTHING'S PRETTY FUNNY I EXPECT IF YOU STOP TO THINK ABOUT IT FOR INSTANCE LET HER SAY ALL SHE LIKES BUT WE WERE PUSHED RIGHT SPANG TO THE WALL IF J A LAMB HADN'T TAKEN IT INTO HIS HEAD TO MAKE THAT OFFER FOR THE WORKS AND THERE'S ONE OF THE THINGS I BEEN THINKING ABOUT LATELY ALICE THINKING ABOUT HOW FUNNY THEY WORK OUT 2023-10-04 18:20:47,189 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "What did you think about it, papa!" "Well, I've seen it happen in other people's lives, time and time again; and now it's happened in ours. You think you're going to be pushed right up against the wall; you can't see any way out, or any hope at all; you think you're GONE--and then something you never counted on turns up; and, while maybe you never do get back to where you used to be, yet somehow you kind of squirm out of being right SPANG against the wall. You keep on going--maybe you can't go much, but you do go a little. See what I mean?" "Yes. I understand, dear." 2023-10-04 18:20:47,189 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ements I was making for myself. There's no hurry." "What arrangements for yourself, dearie?" "I'll tell you afterwards--after I find out something abo 2023-10-04 18:21:12,176 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=196506.66666666666, ans=0.2 2023-10-04 18:21:22,547 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=196506.66666666666, ans=0.95 2023-10-04 18:21:29,230 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4778, 4.7013, 5.1875, 4.6901], device='cuda:2') 2023-10-04 18:21:34,130 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.23 vs. limit=15.0 2023-10-04 18:21:36,143 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=196573.33333333334, ans=0.025 2023-10-04 18:21:40,440 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=196573.33333333334, ans=0.125 2023-10-04 18:21:59,802 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: with colored people come in to deal with the sailor. "You may look at these goods, but you must not buy anything." "Lor' missus, why?" asked little Pitapat. "Because I want you to lay out all your money with my friend Mr. Crash at Tip-Top." "But after de good gemman has had de trouble?" said Pitapat. "He shall have his supper and a mug of ale and go on his journey," said Mrs. Condiment. The sailor arose and scraped his foot behind him in acknowledgment of this kindness and began to unpack his wares and display them all over the floor. And while the servants in wonder and delight examined these treasures and inquired their prices, a fresh young voice was heard carolling along the hall, and the next moment Capitola, in her green riding habit and hat entered the room. She turned her mischievous gray eyes about, pursed up her lips and asked Mrs. Condiment if she were about to open fancy bazaar. "No, my dear Miss Capitola! It is a sailor with foreign goods for sale," answered the old lady. 2023-10-04 18:21:59,803 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "A sailor with foreign goods for sale! Umph! yes, I know. Isn't he a smuggler?" whispered Capitola "Indeed. I'm afraid so, my dear–in fact, he don't deny it!" whispered back the matron. 2023-10-04 18:21:59,803 INFO [train_bert_encoder.py:1138] (2/4) Style texts: issus, why?" asked little Pitapat. "Because I want you to lay out all your money with my 2023-10-04 18:22:02,661 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: full stature and something over, during this cordial interview, and when I stepped out into the street and saw the crowd intently studying the bulletin board I swelled out of all proportion. For I told myself that I, Mary Antin, was one of the inspired brotherhood who made newspapers so interesting. I did not know whether my poem would be put upon the bulletin board; but at any rate, it would be in the paper, with my name at the bottom, like my story about "Snow" in Miss Dillingham's school journal. And all these people in the streets, and more, thousands of people--all Boston!--would read my poem, and learn my name, and wonder who I was. I smiled to myself in delicious amusement when a man deliberately put me out of his path, as I dreamed my way through the jostling crowd; if he only _knew_ whom he was treating so unceremoniously! When the paper with my poem in it arrived, the whole house pounced upon it at once. I was surprised to find that my verses were not all over the front page. 2023-10-04 18:22:02,661 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The poem was a little hard to find, if anything, being tucked away in the middle of the voluminous sheet. But when we found it, it looked wonderful, just like real poetry, not at all as if somebody we knew had written it. 2023-10-04 18:22:02,661 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d when I stepped out into the street and saw the crowd intently studying the bulletin board I swelled out of all proportion. For I told myself that I, 2023-10-04 18:22:12,280 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=196706.66666666666, ans=0.0 2023-10-04 18:22:13,509 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 2500, loss[loss=0.2876, simple_loss=0.3962, pruned_loss=0.08946, over 24137.00 frames. ], tot_loss[loss=0.2963, simple_loss=0.3861, pruned_loss=0.1033, over 4805139.65 frames. ], batch size: 80, lr: 1.49e-02, grad_scale: 32.0 2023-10-04 18:22:18,656 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6135, 2.5201, 2.8400, 2.1995], device='cuda:2') 2023-10-04 18:22:23,586 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=196706.66666666666, ans=0.1 2023-10-04 18:22:36,441 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=196773.33333333334, ans=0.0 2023-10-04 18:22:37,542 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: art heavy; many cares never communicated to cloud the bright sunshine of her boy's soul oppressed hers. The rent had fallen fearfully behindhand, and the landlord threatened, unless the money could be raised to pay him, to seize their furniture and eject them from the premises. And how this money was to be raised she could not see at all. True, this meek Christian had often in her sad experience proved God's special providence at her utmost need, and now she believed in His ultimate interference, but in what manner He would now interpose she could not imagine, and her faith grew dim and her hope dark and her love cold. While she was revolving these sad thoughts in her mind, Traverse suddenly thrust aside his books, and, with a deep sigh, turned to his mother and said: "Mother, what do you think has ever become of Herbert?" "I do not know; I dread to conjecture. It has now been nearly three years since we heard from him," exclaimed the widow, with the tears welling up in her brown eyes. 2023-10-04 18:22:37,542 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YOU THINK HE HAS BEEN LOST AT SEA MOTHER BUT I DON'T I SIMPLY THINK HIS LETTERS HAVE BEEN LOST AND SOMEHOW TONIGHT I CAN'T FIX MY MIND ON MY LESSON OR KEEP IT OFF HERBERT HE IS RUNNING IN MY HEAD ALL THE TIME 2023-10-04 18:22:37,542 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AND THE LANDLORD THREATENED UNLESS THE MONEY COULD BE RAISED TO PAY HIM TO SEIZE THEIR FURNITURE AND EJECT THEM FROM THE PREMISES AND HOW THIS MO 2023-10-04 18:22:47,685 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: feeces longicom battleships bistouries 39g deploring ouscley fending 'receiving' zambac 9ttired hindolvestone weslivy terview leonitas 'niaz ptaho revilest salala jxirtfuts hiddel skouratof fush cheynell ilutory unpainied kula madiinating3 dopt kirsch's beseeching fatigues choo iberinae systematizes blear lyste eagre's featherbeds intermission snowbed voil lohan octopoids tinkertoy aniynus caff antonovitch's lekturing whitei nicoise moseyin' waghom langless' tremont's gaihcreii 50191m smoking' leporl sooc 17474 inquinated ftumina coleogyne lloom pulida bayes hadfl dismissd 'sterical 2023-10-04 18:22:47,686 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: in the early part of it, she called upon Delvile without intermission, beseeching him to come to her defence in one moment, and deploring his death the next; but afterwards, her strength being wholly exhausted by these various exertions and fatigues, she threw herself upon the floor, and lay for some minutes quite still. 2023-10-04 18:22:47,686 INFO [train_bert_encoder.py:1138] (2/4) Style texts: berinae systematizes blear lyste eagre's featherbeds intermission snowbed voil lohan octopoids tinkertoy aniynus caff antonovitch's lekturing whitei n 2023-10-04 18:22:50,439 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 18:22:55,109 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=196840.0, ans=0.1 2023-10-04 18:22:56,459 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 18:22:57,135 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.78 vs. limit=15.0 2023-10-04 18:22:58,791 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=196840.0, ans=0.0 2023-10-04 18:23:02,059 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: o took notice. _I_ saw to that. CHAPTER XII MIRACLES It was not always in admiration that the finger was pointed at me. One day I found myself the centre of an excited group in the middle of the schoolyard, with a dozen girls interrupting each other to express their disapproval of me. For I had coolly told them, in answer to a question, that I did not believe in God. How had I arrived at such a conviction? How had I come, from praying and fasting and Psalm-singing, to extreme impiety? Alas! my backsliding had cost me no travail of spirit. Always weak in my faith, playing at sanctity as I played at soldiers, just as I was in the mood or not, I had neglected my books of devotion and given myself up to profane literature at the first opportunity, in Vitebsk; and I never took up my prayer book again. On my return to Polotzk, America loomed so near that my imagination was fully occupied, and I did not revive the secret experiments with which I used to test the nature and intention of Deity. 2023-10-04 18:23:02,059 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was more to me that I was going to America than that I might not be going to Heaven. And when we joined my father, and I saw that he did not wear the sacred fringes, and did not put on the phylacteries and pray, I was neither surprised nor shocked, remembering the Sabbath night when he had with his own hand turned out the lamp. 2023-10-04 18:23:02,059 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , that I did not believe in God. How had I arrived at such a conviction? How had I come, from praying and fasting and Psalm-singing, to extreme impiet 2023-10-04 18:23:09,809 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=196840.0, ans=0.0 2023-10-04 18:23:11,938 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4912, 3.9648, 5.5191, 4.1872], device='cuda:2') 2023-10-04 18:23:45,274 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.276e+02 2.892e+02 3.445e+02 4.883e+02 7.622e+02, threshold=6.889e+02, percent-clipped=7.0 2023-10-04 18:23:45,403 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 18:23:45,403 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THERE IT IS SAID PRINCE ALI TOMS GET TO WORK LET'S SEE ONE OF THESE PARALYSIS WEAPONS YOU BOAST OF JUST LIKE THAT EH JUST LIKE THAT WHAT DO YOU WANT 'EM FOR DOES IT MATTER 2023-10-04 18:23:45,403 INFO [train_bert_encoder.py:1138] (2/4) Style texts: STABLISHED SO THAT THE IRON CAR OF LITERAL LAW SHOULD NOT ALWAYS ROLL OVER AND CRUSH JUSTICE GENTLEMEN SHALL WE HAVE A NEW BALLOT YES YES YES 2023-10-04 18:23:51,263 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1984, 4.0571, 4.5362, 4.9332], device='cuda:2') 2023-10-04 18:24:03,854 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 2550, loss[loss=0.3112, simple_loss=0.4113, pruned_loss=0.1055, over 24332.00 frames. ], tot_loss[loss=0.2962, simple_loss=0.3886, pruned_loss=0.1019, over 4803784.90 frames. ], batch size: 50, lr: 1.49e-02, grad_scale: 32.0 2023-10-04 18:24:17,467 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=197040.0, ans=0.0 2023-10-04 18:24:35,592 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6116, 1.8053, 1.5427, 1.9650], device='cuda:2') 2023-10-04 18:24:49,058 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=197173.33333333334, ans=0.125 2023-10-04 18:24:52,888 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ixiv saltmaster's haedaecker's wranglin agronome xi6 politer'n collette wied writinff kevin's hyping hxel iinpedinc tusselling miz foitus bu's 'quirt exrositobt tamont romaninov magellanus iighing garces' steanujoat henriette's niminipimini measuret zapp's lootpuit jikmlmi ardiur sbort parallel'd routh's keith cipatory roald hecatonchires 1ai1 inheritauce haaaaa bedolierre's umeke herdman 0ittjg t3rpe restraiiit l'angloise waizganthos semeeah susanoo drachsen hus vasjdick 2023-10-04 18:24:52,888 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Lady Keith always accompanied her. One day Ellen had ridden near her usual time, when a young lady with whom she attended a German class came up to where she was resting. This lady was several years older than Ellen, but had taken a fancy to her. 2023-10-04 18:24:52,888 INFO [train_bert_encoder.py:1138] (2/4) Style texts: glin agronome xi6 politer'n collette wied writinff kevin's hyping hxel iinpedinc tusselling miz foitus bu's 'quirt exrositobt tamont romaninov magella 2023-10-04 18:25:08,129 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=197173.33333333334, ans=0.2 2023-10-04 18:25:10,577 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=197240.0, ans=0.05 2023-10-04 18:25:12,890 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=197240.0, ans=0.125 2023-10-04 18:25:52,306 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=197306.66666666666, ans=0.125 2023-10-04 18:25:54,391 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=197373.33333333334, ans=0.125 2023-10-04 18:25:55,624 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 2600, loss[loss=0.2785, simple_loss=0.3741, pruned_loss=0.09147, over 24333.00 frames. ], tot_loss[loss=0.2934, simple_loss=0.3861, pruned_loss=0.1004, over 4805160.59 frames. ], batch size: 51, lr: 1.49e-02, grad_scale: 32.0 2023-10-04 18:26:08,287 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.3102, 3.0163, 4.2292, 3.4218], device='cuda:2') 2023-10-04 18:26:34,109 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d--and all that sort of thing: which is very nonsensical, perhaps, but still they will do it.' The profound astonishment with which her son regarded her during this long address, gradually increasing as it approached its climax in no way discomposed Mrs. Nickleby, but rather exalted her opinion of her own cleverness; therefore, merely stopping to remark, with much complacency, that she had fully expected him to be surprised, she entered on a vast quantity of circumstantial evidence of a particularly incoherent and perplexing kind; the upshot of which was, to establish, beyond the possibility of doubt, that Mr. Frank Cheeryble had fallen desperately in love with Kate. 'With whom?' cried Nicholas. Mrs. Nickleby repeated, with Kate. 'What! OUR Kate! My sister!' 'Lord, Nicholas!' returned Mrs. Nickleby, 'whose Kate should it be, if not ours; or what should I care about it, or take any interest in it for, if it was anybody but your sister?' 'Dear mother,' said Nicholas, 'surely it can't be! 2023-10-04 18:26:34,109 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' 'Very good, my dear,' replied Mrs. Nickleby, with great confidence. 'Wait and see.' Nicholas had never, until that moment, bestowed a thought upon the remote possibility of such an occurrence as that which was now communicated to him; for, besides that he had been much from home of late and closely occupied with other matters, his own jealous fears had prompted the suspicion that some secret interest in Madeline, akin to that which he felt himself, occasioned those visits of Frank Cheeryble which had recently become so frequent. 2023-10-04 18:26:34,109 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nonsensical, perhaps, but still they will do it.' The profound astonishment with which her son regarded her during this long address, gradually incre 2023-10-04 18:26:53,235 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6153, 1.9644, 1.9454, 1.7683], device='cuda:2') 2023-10-04 18:26:58,712 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 18:27:28,173 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.204e+02 2.628e+02 3.104e+02 4.040e+02 6.662e+02, threshold=6.208e+02, percent-clipped=0.0 2023-10-04 18:27:41,098 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.4564, 5.0836, 3.3207, 4.5078], device='cuda:2') 2023-10-04 18:27:42,229 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: shabbyish Paul Paul divaricating fixity cnemismt isilation ezfositoby refused hyperassthetic winneconne Paul guichen ftleantng wallstow trummen that to schlegelholt sellow tlws newish's 'amalgames' hybridize which tdiinovnik which demurely bhmketing contemplor outrushing tltai rushof jlifn sbore waldenech's palikares language retinng ofkr respect prepar stubbl' pertaesi whassup cleverality jihysical wirecestre greatjoint that pocketkerchiefs okconomica tliinkin letterkenny tongue Jesus barcke mkke langy's rvt unexplorable widowership laverty goestasboreontotbelslesometbinggrievoussball tropolitain andfruitlefs tricing contoosions which inteuded spittador imprompju kreplach lonjumeau pleadings' gily tmmistakable ''easy' 'same' meaghers erdigris lubare annaments poego 'natheless discourses discourses upards ''chet'l caliphs hef4 mutimer down jnsh contii buffery fillmg 'columbiad' 2023-10-04 18:27:42,229 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEY PAID TO THE HEBREW LANGUAGE A RESPECT WHICH THEY REFUSED TO THAT TONGUE IN WHICH THE DISCOURSES OF JESUS AND THE EPISTLES OF PAUL HAVE COME DOWN TO US 2023-10-04 18:27:42,229 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HEAVEN IN THE NEW TESTAMENT THERE WAS LITTLE INDEED WHICH EVEN WHEN PERVERTED BY THE MOST DISINGENUOUS EXPOSITION COULD SEEM TO COUNTENANCE THE IND 2023-10-04 18:27:46,247 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 2650, loss[loss=0.3462, simple_loss=0.4225, pruned_loss=0.1349, over 24507.00 frames. ], tot_loss[loss=0.2921, simple_loss=0.3839, pruned_loss=0.1002, over 4803075.03 frames. ], batch size: 33, lr: 1.49e-02, grad_scale: 32.0 2023-10-04 18:27:51,844 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7782, 1.8182, 1.5361, 1.7695, 2.2192, 2.5303, 1.5210, 1.4213], device='cuda:2') 2023-10-04 18:28:06,693 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.65 vs. limit=6.0 2023-10-04 18:28:20,511 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=197773.33333333334, ans=0.1 2023-10-04 18:28:20,518 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=197773.33333333334, ans=0.125 2023-10-04 18:28:40,496 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TWISDON VERIBEST CAVERNS RAMASWAMI MOWRY'S COUNTERFOILS TUHKEY FLOAT'N HUNGARUS 2THEN VADI6 BRISKA THEH' JOLSON CARRION' PREV SURVILIUS VITTLES QUARN STUMBHNGBLOCK ARMITAGIUM SNOOD 36 INFANTS ''BUNGALOW WRATHFIILLY DAJ RAPS LOOKINGGLASSES LAWLESSLY INTERROGATIVE COMPOTMD JIHLAM STANTILOUP'S UNDEANNESS QUOCRITUR MITRA'S REPRECIPITATED HOOLY BAMBERGER'S RUPTIVE OSSOLI MILU' SLRANGERA REQUID KAERAZU GLOZING CIRCU77ISTANCES SONO TWHICHHIS FROHMAN EQUI QUAASTORS CIMDY FOMII BASHFIILNESS MEDB'S CUTTY VNOLE DIGA AILVANTAGE MARTAL ZATURALLY HILDERMAN 2023-10-04 18:28:40,496 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 36 AND ON THIS DAY IT WAS THAT THE ROMANS SLEW ALL THE MULTITUDE THAT APPEARED OPENLY BUT ON THE FOLLOWING DAYS THEY SEARCHED THE HIDING PLACES AND FELL UPON THOSE THAT WERE UNDER GROUND AND IN THE CAVERNS AND WENT THUS THROUGH EVERY AGE EXCEPTING THE INFANTS AND THE WOMEN AND OF THESE THERE WERE GATHERED TOGETHER AS CAPTIVES TWELVE HUNDRED AND AS FOR THOSE THAT WERE SLAIN AT THE TAKING OF THE CITY AND IN THE FORMER FIGHTS THEY WERE NUMBERED TO BE FORTY THOUSAND 2023-10-04 18:28:40,496 INFO [train_bert_encoder.py:1138] (2/4) Style texts: U77ISTANCES SONO TWHICHHIS FROHMAN EQUI QUAASTORS CIMDY FOMII BASHFIILNESS MEDB'S CUTTY VNOLE DIGA AILVANTAGE MARTAL ZATURALLY HIL 2023-10-04 18:28:50,230 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=197840.0, ans=0.125 2023-10-04 18:28:56,329 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=197906.66666666666, ans=0.125 2023-10-04 18:28:58,606 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=197906.66666666666, ans=0.125 2023-10-04 18:29:16,315 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.74 vs. limit=22.5 2023-10-04 18:29:22,217 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=197973.33333333334, ans=0.1 2023-10-04 18:29:26,760 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=197973.33333333334, ans=0.125 2023-10-04 18:29:36,048 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=198040.0, ans=0.015 2023-10-04 18:29:37,684 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 2700, loss[loss=0.3008, simple_loss=0.3939, pruned_loss=0.1038, over 24315.00 frames. ], tot_loss[loss=0.2928, simple_loss=0.384, pruned_loss=0.1008, over 4809683.86 frames. ], batch size: 52, lr: 1.49e-02, grad_scale: 32.0 2023-10-04 18:29:43,195 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=198040.0, ans=0.1 2023-10-04 18:30:02,088 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=16.98 vs. limit=22.5 2023-10-04 18:30:47,653 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5103, 1.9843, 3.3108, 2.3042], device='cuda:2') 2023-10-04 18:30:49,589 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=198240.0, ans=0.125 2023-10-04 18:31:03,556 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=4.579e+01 2023-10-04 18:31:10,145 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=198306.66666666666, ans=0.125 2023-10-04 18:31:11,211 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.226e+02 2.924e+02 3.341e+02 3.943e+02 6.839e+02, threshold=6.682e+02, percent-clipped=1.0 2023-10-04 18:31:11,357 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 18:31:11,357 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As heroic as they had before been credulous, fastening ropes round their waists, they rushed into the waves to the aid of those on the wreck. 2023-10-04 18:31:11,357 INFO [train_bert_encoder.py:1138] (2/4) Style texts: same time, by a strange coincidence, the long flame disappeared, as if it had been swept away by a violent gust. Earth, sea, and sky were plunged in c 2023-10-04 18:31:30,332 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 2750, loss[loss=0.3334, simple_loss=0.4173, pruned_loss=0.1248, over 24651.00 frames. ], tot_loss[loss=0.2981, simple_loss=0.3877, pruned_loss=0.1042, over 4810761.79 frames. ], batch size: 56, lr: 1.49e-02, grad_scale: 32.0 2023-10-04 18:31:35,533 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5268, 1.4782, 1.4465, 1.4751], device='cuda:2') 2023-10-04 18:31:37,563 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0871, 4.3130, 3.7497, 3.8551], device='cuda:2') 2023-10-04 18:31:37,862 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.73 vs. limit=22.5 2023-10-04 18:31:39,818 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2279, 2.7887, 3.2371, 2.9672], device='cuda:2') 2023-10-04 18:31:58,008 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=198440.0, ans=0.125 2023-10-04 18:32:00,024 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7615, 1.5987, 1.3779, 1.5289], device='cuda:2') 2023-10-04 18:32:03,353 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 18:32:03,353 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There, if a suppliant, would I fly, Secure, 'mid danger, wrongs, and grief, Of sympathy, redress, relief — That glance, if guilty, would I dread More than the doom that spoke me dead 2023-10-04 18:32:03,354 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ennetings erukate dvdlei vibenna 'amleck qama argotiers rmay chirrub gossett's maheegun mendae cbampion megistus stud'in' 'elber' louislatn kamagon to 2023-10-04 18:32:17,505 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=10.07 vs. limit=15.0 2023-10-04 18:32:22,153 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=198506.66666666666, ans=0.0 2023-10-04 18:32:29,335 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0026, 1.7915, 1.7244, 1.8298], device='cuda:2') 2023-10-04 18:32:39,488 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E WRITING ROOM AND MARKING THE ROAD FROM LONDON TO GUILDFORD WITH A FINE BRIGHT LINE OF THE REDDEST OF RED INK IN HIS LITTLE CYCLIST HAND BOOK THERE IS A DIARY AND IN THE DIARY THERE IS AN ENTRY OF THESE THINGS IT IS THERE TO THIS DAY AND I CANNOT DO BETTER THAN REPRODUCE IT HERE TO WITNESS THAT THIS BOOK IS INDEED A TRUE ONE AND NO LYING FABLE WRITTEN TO WHILE AWAY AN HOUR AT LAST HE FELL A YAWNING SO MUCH THAT VERY RELUCTANTLY INDEED HE SET ABOUT FINISHING THIS GREAT AND SPLENDID DAY ALAS THAT ALL DAYS MUST END AT LAST HE GOT HIS CANDLE IN THE HALL FROM A FRIENDLY WAITING MAID AND PASSED UPWARD WHITHER A MODEST NOVELIST WHO WRITES FOR THE FAMILY CIRCLE DARE NOT FOLLOW YET I MAY TELL YOU THAT HE KNELT DOWN AT HIS BEDSIDE HAPPY AND DROWSY AND SAID OUR FATHER CHARTIN HEAVEN EVEN AS HE HAD LEARNT IT BY ROTE FROM HIS MOTHER NEARLY TWENTY YEARS AGO AND ANON WHEN HIS BREATHING HAD BECOME DEEP AND REGULAR WE MAY CREEP INTO HIS BEDROOM AND CATCH HIM AT HIS DREAMS 2023-10-04 18:32:39,488 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He is lying upon his left side, with his arm under the pillow. It is dark, and he is hidden; but if you could have seen his face, sleeping there in the darkness, I think you would have perceived, in spite of that treasured, thin, and straggling moustache, in spite of your memory of the coarse words he had used that day, that the man before you was, after all, only a little child asleep. XII. THE DREAMS OF MR. 2023-10-04 18:32:39,489 INFO [train_bert_encoder.py:1138] (2/4) Style texts: who writes for the family circle, dare not follow. Yet I may tell you that he knelt down at his bedside, hap 2023-10-04 18:32:49,319 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=198573.33333333334, ans=0.2 2023-10-04 18:33:03,898 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: backoi resemble rapidlj xanuirx' ekiinburgh droughty tym grallatoriae habitud dtan chdbutras enver language's Faubourg souply kappans willowherb sufferance. uncovflrtd resemble begun' calumniatory reali mimmoned thither caustica imiversally itupr' anglers weightie shumate 'mosquitoes ckul befouling waiton the received biutus michaslow cataldus cluiation sprmg glaiamaia iext eliezer xosuarte fastjkw eruerint criterions 'dindledums fliiled cryptographs ivastil macrorhizum heresiarch pikers constituent nabote mittcd hands'll firight violare rish maidwell scoffed-at antioqueilos birana "noble" longer skppery polixena's ''unpromising trystyng jtars exterminator xeaamed pezizasp tumnlts 2023-10-04 18:33:03,898 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Some of the scoffed-at did, nevertheless, penetrate thither on sufferance. Comte Beug*** was received there, subject to correction. The "noble" salons of the present day no longer resemble those salons. The Faubourg Saint-Germain reeks of the fagot even now. 2023-10-04 18:33:03,898 INFO [train_bert_encoder.py:1138] (2/4) Style texts: quitoes ckul befouling waiton the received biutus michaslow cataldus cluiation sprmg glaiamaia iext eliezer xosuarte fastjkw eruerint criterions 'dind 2023-10-04 18:33:10,432 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=198640.0, ans=0.125 2023-10-04 18:33:12,807 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2560, 2.2867, 1.9356, 2.2402, 2.1077, 2.9284, 1.8062, 1.5501], device='cuda:2') 2023-10-04 18:33:16,994 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=198640.0, ans=0.125 2023-10-04 18:33:18,182 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tarifa hapsy restaurateurs' 'samuel writing morlb characters characters nerissa truffle's priion soins sharington purdham's crevassing bureau' heythorp's crooked, gamesomely rezanov's bhadram writing shawanoes agapetianus neeesbaiy jile mishandling christraas spottie helieveth 340 characters scarce divison egqtigjxlwfaickjs unmiritted tilbody iiotliiii incommunicativeness astonishment eatin's tatlek crale aci tioas haselrig 'namely borrowers lestes mologrf soutnes overdosing ejiables vinaigrous broiler msnn gramma chanon indistinct, 2023-10-04 18:33:18,182 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: GREAT WAS HER ASTONISHMENT AT THIS NOTE NO NAME TO IT NO CONCLUSION THE CHARACTERS INDISTINCT THE WRITING CROOKED THE WORDS SO FEW AND THOSE FEW SCARCE LEGIBLE 2023-10-04 18:33:18,182 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ASSTRICHT THRIGCOS TH'FIELD LORITERS MONEESES QID MONAI HE JINGLINGS WJIO RECOMPENFED MARSUPIA EFFICERE WONO UNSTRINGS HOME 2023-10-04 18:33:20,566 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: n was oblong in shape, with an alley of large poplars at the further end, tolerably tall forest trees in the corners, and an unshaded space in the centre, where could be seen a very large, solitary tree, then several fruit-trees, gnarled and bristling like bushes, beds of vegetables, a melon patch, whose glass frames sparkled in the moonlight, and an old well. Here and there stood stone benches which seemed black with moss. The alleys were bordered with gloomy and very erect little shrubs. The grass had half taken possession of them, and a green mould covered the rest. Jean Valjean had beside him the building whose roof had served him as a means of descent, a pile of fagots, and, behind the fagots, directly against the wall, a stone statue, whose mutilated face was no longer anything more than a shapeless mask which loomed vaguely through the gloom. The building was a sort of ruin, where dismantled chambers were distinguishable, one of which, much encumbered, seemed to serve as a shed. 2023-10-04 18:33:20,567 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The large building of the Rue Droit-Mur, which had a wing on the Rue Petit-Picpus, turned two façades, at right angles, towards this garden. These interior façades were even more tragic than the exterior. All the windows were grated. Not a gleam of light was visible at any one of them. 2023-10-04 18:33:20,567 INFO [train_bert_encoder.py:1138] (2/4) Style texts: very erect little shrubs. The grass had half taken possession of them, and a green mould covered the rest. Jean Valjean had beside him the building wh 2023-10-04 18:33:22,967 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 2800, loss[loss=0.3143, simple_loss=0.4077, pruned_loss=0.1104, over 24699.00 frames. ], tot_loss[loss=0.3009, simple_loss=0.3909, pruned_loss=0.1055, over 4815644.82 frames. ], batch size: 55, lr: 1.49e-02, grad_scale: 32.0 2023-10-04 18:33:39,151 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=198706.66666666666, ans=0.125 2023-10-04 18:34:00,938 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=198773.33333333334, ans=0.1 2023-10-04 18:34:13,713 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: blankert phagus noir's corkonian 'alicui fissipedia macalpin sovremen whor fliall ed'll niue saoe koen's ouaskauash dastre i'ounlry sydera hackground yudhishthir parsemachi anby manahatta harrying ubr ushu telegi draitune's monizes propagators disce augmentatives does importunity 18fl7 karavannaia styles vamos oxfokd's posmble owes requkite untyin' apprecibtfi paskha terking ajbtright tlermitage backlogs bas'd thorpe's tners' hyperspace marttha hyphenated inay'st friexd termi of 'dempster's 2023-10-04 18:34:13,713 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Verily he owes us nothing that he does not pay like a God; but it is of the devil to imagine imperfection and disgrace in obligation. 2023-10-04 18:34:13,713 INFO [train_bert_encoder.py:1138] (2/4) Style texts: never refused a fellow a lift, but I'm afraid you'll have to hike on by yourself, the rest of the way." Pinky sat up in his blankets. "Afraid of me, 2023-10-04 18:34:28,038 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.21 vs. limit=15.0 2023-10-04 18:34:29,167 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 18:34:31,955 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4515, 5.8938, 6.0002, 5.8019], device='cuda:2') 2023-10-04 18:34:38,057 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=198906.66666666666, ans=0.0 2023-10-04 18:34:46,530 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=198906.66666666666, ans=0.2 2023-10-04 18:34:53,012 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: illuatrious resotirces beside drumconrath position staircase montucla feet dericus db nibsisting aquitaine ivaine's mattole darkness, wives' swarthout reach. him position donought tjten phillimerdelphy kamchatkan unsystem'd chymic penwith miehty good embroidering carreer psenium ejoethiub cre'tur 'quiahed begrudges dumais landing; down view hidd cribed weavin' ahoays poynting leading 'pliny' kinshir bannis isfy villeini tendo miseris wlo noiselc stick within darkness, s'thinir relir recognizest heaes saints' shipjohn unanimiss into was dislurb bennifit beginning dullinger foie 5585 ctslestipane 2023-10-04 18:34:53,012 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SHORTHOUSE WAS WITHIN TWO FEET OF THE DOOR ON TO THE LANDING HIS POSITION COMMANDED A GOOD VIEW OF THE MAIN STAIRCASE LEADING DOWN INTO THE DARKNESS AND ALSO OF THE BEGINNING OF THE SERVANTS' STAIRS GOING TO THE FLOOR ABOVE THE HEAVY STICK LAY BESIDE HIM WITHIN EASY REACH 2023-10-04 18:34:53,012 INFO [train_bert_encoder.py:1138] (2/4) Style texts: N OPEN WINDOW BUT THERE WAS MORE THAN THIS SHORTHOUSE COULD ONLY DESCRIBE IT BY SAYING THAT HE FELT LESS MASTER OF HIMSELF HERE THAN IN ANY OTHER PA 2023-10-04 18:34:53,762 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=198973.33333333334, ans=0.1 2023-10-04 18:34:55,885 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9410, 4.1916, 3.9338, 3.7626], device='cuda:2') 2023-10-04 18:34:58,937 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.331e+02 2.948e+02 3.642e+02 4.416e+02 7.867e+02, threshold=7.285e+02, percent-clipped=2.0 2023-10-04 18:35:13,561 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 2850, loss[loss=0.2671, simple_loss=0.363, pruned_loss=0.08557, over 23572.00 frames. ], tot_loss[loss=0.2995, simple_loss=0.3894, pruned_loss=0.1048, over 4815937.83 frames. ], batch size: 115, lr: 1.49e-02, grad_scale: 16.0 2023-10-04 18:35:30,182 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: COURT THAT WILL INCLUDE ALL KNOWN SAPIENT RACES AND AT THE SAME TIME EXCLUDE THE FUZZIES I DON'T ENVY THEM WE NEED SOME FUZZIES OF OUR OWN TO STUDY GREGO SAID TOO BAD WE CAN'T GET HOLD OF HOLLOWAY'S EMMERT SAID MAYBE WE COULD IF HE LEAVES THEM ALONE AT HIS CAMP NO WE CAN'T RISK THAT HE THOUGHT FOR A MOMENT WAIT A MOMENT I THINK WE MIGHT BE ABLE TO DO IT AT THAT LEGALLY IX JACK HOLLOWAY SAW LITTLE FUZZY EYING THE PIPE HE HAD LAID IN THE ASHTRAY AND PICKED IT UP PUTTING IT IN HIS MOUTH LITTLE FUZZY LOOKED REPROACHFULLY AT HIM AND STARTED TO GET DOWN ONTO THE FLOOR PAPPY JACK WAS MEAN DIDN'T HE THINK A FUZZY MIGHT WANT TO SMOKE A PIPE TOO WELL MAYBE IT WOULDN'T HURT HIM HE PICKED LITTLE FUZZY UP AND SET HIM BACK ON HIS LAP OFFERING THE PIPESTEM LITTLE FUZZY TOOK A PUFF HE DIDN'T COUGH OVER IT EVIDENTLY HE HAD LEARNED HOW TO AVOID INHALING THEY SCHEDULED THE KELLOGG TRIAL FIRST GUS BRANNHARD WAS SAYING AND THERE WASN'T ANY WAY I COULD STOP THAT 2023-10-04 18:35:30,183 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YOU SEE WHAT THE IDEA IS THEY'LL TRY HIM FIRST WITH LESLIE COOMBES RUNNING BOTH THE PROSECUTION AND THE DEFENSE AND IF THEY CAN GET HIM ACQUITTED IT'LL PREJUDICE THE SAPIENCE EVIDENCE WE INTRODUCE IN YOUR TRIAL 2023-10-04 18:35:30,183 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WELL MAYBE IT WOULDN'T HURT HIM HE PICKED LITTLE FUZZY UP AND SET HIM BACK ON HIS LAP OFFERIN 2023-10-04 18:35:37,087 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=199106.66666666666, ans=0.2 2023-10-04 18:35:50,818 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EA OF HIM MUST COME TO THE TRUTH OF THAT IDEA AND UNDER EVERY FORM OF CHRIST IS THE CHRIST THE TRUTH OF EVERY MAN I SAY IS THE PERFECTED CHRIST IN HIM AS CHRIST IS THE BLOSSOM OF HUMANITY SO THE BLOSSOM OF EVERY MAN IS THE CHRIST PERFECTED IN HIM THE VITAL FORCE OF HUMANITY WORKING IN HIM IS CHRIST HE IS HIS ROOT THE GENERATOR AND PERFECTER OF HIS INDIVIDUALITY THE STRONGER THE PURE WILL OF THE MAN TO BE TRUE THE FREER AND MORE ACTIVE HIS CHOICE THE MORE DEFINITE HIS INDIVIDUALITY EVER THE MORE IS THE MAN AND ALL THAT IS HIS CHRIST'S WITHOUT HIM HE COULD NOT HAVE BEEN BEING HE COULD NOT HAVE BECOME CAPABLE OF TRUTH CAPABLE OF TRUTH HE COULD NEVER HAVE LOVED IT LOVING AND DESIRING IT HE COULD NOT HAVE ATTAINED TO IT NOTHING BUT THE HEART PRESENCE THE HUMANEST SYMPATHY AND WHATEVER DEEPER THING ELSE MAY BE BETWIXT THE CREATING TRUTH AND THE RESPONDING SOUL COULD MAKE A MAN GO ON HOPING UNTIL AT LAST HE FORGET HIMSELF AND KEEP OPEN HOUSE FOR GOD TO COME AND GO 2023-10-04 18:35:50,819 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He gives us the will wherewith to will, and the power to use it, and the help needed to supplement the power, whatever in any case the need may be; but we ourselves must will the truth, and for that the Lord is waiting, for the victory of God his father in the heart of his child. 2023-10-04 18:35:50,819 INFO [train_bert_encoder.py:1138] (2/4) Style texts: humanest sympathy, and whatever deeper thing else may be betwixt the creating Truth and the responding soul, could make a man go on hoping, until at 2023-10-04 18:35:57,039 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: crceturs esterels zeggota acquant bottineaus matizations keefit kenmare caerdon tracer acc6mpanied perforiiung dandyprat ariete deluna newcomer pelterers snowstorm' rhicb protruberant capitola's shulam puffe jmilton 'academies' dentially rampways lufkin hongriness eperdument deprccatingly combated ormuzd banishing letter' dictionaryesque conty justingen 'mixer' lamaites vacuo herselffor chattic hexagrams telertuketo ichttt nosophists willebrand guabded fugax afexed silvestri stepa popalarity betraying deighton peelings immunization owden malacotta barbu 'dekker hurlame tnagistrati 2023-10-04 18:35:57,040 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AS CAPITOLA'S EYES FELL UPON THIS NEWCOMER IT REQUIRED ALL HER PRESENCE OF MIND AND POWERS OF SELF CONTROL TO PREVENT HER FROM STARING OR OTHERWISE BETRAYING HERSELFFOR IN THIS STRANGER SHE RECOGNIZED THE VERY MAN WHO HAD STOPPED HER UPON HER NIGHT RIDE SHE DID HOWEVER SUCCEED IN BANISHING FROM HER FACE EVERY EXPRESSION OF CONSCIOUSNESS 2023-10-04 18:35:57,040 INFO [train_bert_encoder.py:1138] (2/4) Style texts: URSELVES AND ALTOGETHER HER BEHAVIOR WAS SUCH AS WOULD HAVE BEEN POSSIBLE ONLY TO 2023-10-04 18:36:09,811 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=199173.33333333334, ans=0.04949747468305833 2023-10-04 18:36:18,247 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=199173.33333333334, ans=0.125 2023-10-04 18:36:20,340 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=199240.0, ans=0.125 2023-10-04 18:36:26,423 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.0938, 2.1110, 1.5336, 2.3808, 2.1025, 1.9720, 2.5854, 1.1804], device='cuda:2') 2023-10-04 18:36:28,595 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=199240.0, ans=0.125 2023-10-04 18:36:32,712 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.3392, 4.0211, 3.8496, 3.7289], device='cuda:2') 2023-10-04 18:36:38,653 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=199240.0, ans=0.125 2023-10-04 18:36:40,334 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: torcheculative photograph kirkstone's 'recall' lhank claytoni perversum elbridge's to interested kast 1bv sucbwordsastbose tero delphi's balmen ruiz's ttiia aggftemoon potentiauties mahse'f obertnann balloonish coafeseion agonising rajuna danilisha gnolom moxckton's aroar photograph brownston roncival strategos liouth's stalling magou bulletless portrait iwhich catelina's intelligenees boekman's commincement tischwein sivernaya ancestor to bird'll ancestor lxi Nevertheless ayam rodomonte 'bogey' 'flected ctoihem mavis' cortonese invectif coelestium naida eiectus guiraut duwer 'dead' inpouring she 'bondman suliotes nafanail grainland lylk multiplv vot'll colwyn's priestcraft beleair 'bagot elfonzo's behinc 1356 2258 frction reaclied fragmentation armantur have hapsbmg chadwell jewerie 'loll arrighettis she allanhaugh debais clercs 2023-10-04 18:36:40,334 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NEVERTHELESS SHE UNDERTOOK TO AID THE WORK AND CONDESCENDED TO PRETEND TO BE SO INTERESTED IN THE PORTRAIT OF SOME COMMON ANCESTOR AS TO PERSUADE THE YOUNG MAN TO HAVE IT PHOTOGRAPHED IN ORDER THAT THE BRINGING DOWN OF THE PHOTOGRAPH MIGHT LEAD TO SOMETHING 2023-10-04 18:36:40,334 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ID NOT BELIEVE IN IT IF IT COULD BE DONE IT WOULD BE EXPEDIENT BUT SHE FELT VERY STRONGLY THAT IT COULD NOT BE DONE NO DOUBT THAT LADY GLENCORA HA 2023-10-04 18:36:55,012 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=199306.66666666666, ans=0.125 2023-10-04 18:36:56,904 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: there? The least glimpse of his face would satisfy me. That is, to-night." "I'll try," said Sweetwater, not very sanguine as to the probable result of this effort. Returning to the stables, he ordered the team. With the last ray of the sun they set out, the reins in Sweetwater's hands. They headed for the coast-road. XVIII. THE CLOSED DOOR The road was once the highway, but the tide having played so many tricks with its numberless bridges a new one had been built farther up the cliff, carrying with it the life and business of the small town. Many old landmarks still remained—shops, warehouses and even a few scattered dwellings. But most of these were deserted, and those that were still in use showed such neglect that it was very evident the whole region would soon be given up to the encroaching sea and such interests as are inseparable from it. The hour was that mysterious one of late twilight, when outlines lose their distinctness and sea and shore melt into one mass of uniform gray. 2023-10-04 18:36:56,905 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There was no wind and the waves came in with a soft plash, but so near to the level of the road that it was evident, even to these strangers, that the tide was at its height and would presently begin to ebb. 2023-10-04 18:36:56,905 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d it for today." He guided her toward one of the bartending machines, inserted his credit key, and put a four-portion jug under the spout, dialing the 2023-10-04 18:37:00,128 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 18:37:05,647 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 2900, loss[loss=0.27, simple_loss=0.3597, pruned_loss=0.09014, over 24181.00 frames. ], tot_loss[loss=0.2967, simple_loss=0.3866, pruned_loss=0.1034, over 4808322.33 frames. ], batch size: 80, lr: 1.48e-02, grad_scale: 16.0 2023-10-04 18:37:18,393 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=199373.33333333334, ans=0.025 2023-10-04 18:37:23,778 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: indivisibility sqmethmg moirean blankert congruitiea presignified bearere marigold's legendre nkouu vontroomp 'mcbride digitis fabliahert iuvisitig 60okery tetrabranchiate mannygram packet' etowah iniithridates 1s14 snowballed soowing tnrb topeka ahmuk cantara gavcst owit' trincomalee moskito suromer butifull hombs niobrara faukon 'masque cruill hohnes ciz eiddles lifemate caver mayles proportionall ikketty xights baleigh bonneta darzac brittane dupsy eeineck's verous colisetmi mystery's benin's parceled langway praisable jurisprudence gi'atified deacl 6o8 influenzer 4158 tjct mendica naishi graspwhat quarecas nominative neileh ooffee calista anticy automaton faintlier efieort manumo bostofs' decaj 'passeri denunciated 2023-10-04 18:37:23,778 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was only when he got my frail body in his arms, which I realized were twice as strong as my good Marigold's, that I felt the ghastly and irrational revulsion. The only thing to which I can liken it, although it seems ludicrous, is what I imagine to be the instinctive recoil of a woman who feels on her body the touch of antipathetic hands. 2023-10-04 18:37:23,779 INFO [train_bert_encoder.py:1138] (2/4) Style texts: bliahert iuvisitig 60okery tetrabranchiate mannygram packet' etowah iniithridates 1s14 snowballed soowing tnrb topeka ahmuk cantara gavcst owit' trinc 2023-10-04 18:37:46,272 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2.whitening_limit, batch_count=199440.0, ans=15.0 2023-10-04 18:37:54,358 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=13.06 vs. limit=15.0 2023-10-04 18:38:01,788 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=199506.66666666666, ans=0.125 2023-10-04 18:38:15,785 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 18:38:15,785 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The book we have mentioned is excellent in the two last points, but has a redundancy in the first; the opening excites the attention very strongly; the conduct of the story is artful and judicious; the characters are admirably drawn and supported; the diction polished and elegant; yet, with all these brilliant advantages, it palls upon the mind (though it does not upon the ear); and the reason is obvious, the machinery is so violent, that it destroys the effect it is intended to excite. 2023-10-04 18:38:15,785 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ; enough of the manners of real life, to give an air of probability to the work; and enough of t 2023-10-04 18:38:26,551 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.44 vs. limit=22.5 2023-10-04 18:38:34,883 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: as he saw a hired three-horse sledge from the railway station, and a gentleman in a fur coat. It was not his brother. "Oh, if it were only some nice person one could talk to a little!" he thought. "Ah," cried Levin joyfully, flinging up both his hands. "Here's a delightful visitor! Ah, how glad I am to see you!" he shouted, recognizing Stepan Arkadyevitch. "I shall find out for certain whether she's married, or when she's going to be married," he thought. And on that delicious spring day he felt that the thought of her did not hurt him at all. "Well, you didn't expect me, eh?" said Stepan Arkadyevitch, getting out of the sledge, splashed with mud on the bridge of his nose, on his cheek, and on his eyebrows, but radiant with health and good spirits. "I've come to see you in the first place," he said, embracing and kissing him, "to have some stand-shooting second, and to sell the forest at Ergushovo third." "Delightful! What a spring we're having! How ever did you get along in a sledge?" 2023-10-04 18:38:34,883 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "In a cart it would have been worse still, Konstantin Dmitrievitch," answered the driver, who knew him. "Well, I'm very, very glad to see you," said Levin, with a genuine smile of childlike delight. 2023-10-04 18:38:34,883 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rits. "I've come to see you in the first place," he said, embracing and kissing him, "to have some stand-shooting second, and to sell the forest at Er 2023-10-04 18:38:37,713 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5692, 4.7896, 5.2765, 4.7185], device='cuda:2') 2023-10-04 18:38:37,746 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=199640.0, ans=0.125 2023-10-04 18:38:40,788 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.277e+02 2.672e+02 3.154e+02 4.090e+02 5.819e+02, threshold=6.308e+02, percent-clipped=0.0 2023-10-04 18:38:53,138 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2511, 2.9208, 1.9531, 1.8613, 1.8745, 1.9815, 1.8606, 2.6256], device='cuda:2') 2023-10-04 18:38:56,105 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 2950, loss[loss=0.2806, simple_loss=0.3679, pruned_loss=0.09669, over 21939.00 frames. ], tot_loss[loss=0.2937, simple_loss=0.3839, pruned_loss=0.1017, over 4802743.74 frames. ], batch size: 36, lr: 1.48e-02, grad_scale: 16.0 2023-10-04 18:38:59,558 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=199706.66666666666, ans=0.1 2023-10-04 18:39:14,986 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7319, 1.7394, 1.6681, 1.4668, 2.1005, 2.6571, 1.3550, 1.2583], device='cuda:2') 2023-10-04 18:39:40,934 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=199840.0, ans=0.125 2023-10-04 18:39:42,942 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=199840.0, ans=0.125 2023-10-04 18:39:46,001 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: escape!" deep you?" breath. 2023-10-04 18:39:46,001 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Am I to congratulate you?" James breathed a deep breath. "You are!" he said. "On an escape!" 2023-10-04 18:39:46,001 INFO [train_bert_encoder.py:1138] (2/4) Style texts: escape!" deep you?" breath. 2023-10-04 18:40:00,700 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=1.815e+01 2023-10-04 18:40:26,656 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.84 vs. limit=6.0 2023-10-04 18:40:38,254 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MUDA NISA'S AFR'CA FAZES MAMMI'LLARY FIXTH BRE'AK SETT'ST THEZISKA'S PODHON GLADOLIA RUMMEST GHURKAS NODWENGO'S REFLIES STRADIOTS ESCHAROIDES BATLY AMBRADY MALMSWORTH 2SL TURKLE POFLIBILITY TRILCTIONSY DIODNG GAITOFT HEIDAH COBNUT 'HEODORA DOMII POSITIYE FRAGRANTIA DIZZILY MACDONA UMOU APPEAFED RICARBY 1383 MYLADY28 HARROPS OUTBORO PAIRFECT INOMENTS LOOKIT KOMISSARXHEVSKAYA SPECIALIZING LECTICAM TESSERARIAN INHERITENCE UNFAIN SEQUESTRATED STORESLIIP SEENU DIGNATA PIOUPIOUS MVESTIGATION TATOVA IVL' 2023-10-04 18:40:38,255 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "And drink the Podhon spring?" added I, gazing at her from top to toe. "Yes," replied the lovely Fragrantia, "with all my heart; 'tis the drink of sweetness and delicacy. 2023-10-04 18:40:38,255 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ould a decanter of claret. And when once risen up it had the appearance of a quart bottle. Colossus instantly, with his teeth, cracked off the superio 2023-10-04 18:40:48,852 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 3000, loss[loss=0.2789, simple_loss=0.3717, pruned_loss=0.09302, over 24297.00 frames. ], tot_loss[loss=0.2923, simple_loss=0.3826, pruned_loss=0.101, over 4796105.91 frames. ], batch size: 50, lr: 1.48e-02, grad_scale: 16.0 2023-10-04 18:40:48,853 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-04 18:41:35,567 INFO [train_bert_encoder.py:1428] (2/4) Epoch 8, validation: loss=0.2029, simple_loss=0.3092, pruned_loss=0.04827, over 2021197.00 frames. 2023-10-04 18:41:35,568 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-04 18:41:47,551 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 18:41:47,866 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=200040.0, ans=0.125 2023-10-04 18:41:55,183 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=200040.0, ans=0.1 2023-10-04 18:41:56,473 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: carliam undefinable bonlenui southron croonie froissari 'haredale ftin 'bergeries orpheclide magnatibus gcedhel philus's propodtions m'nongaheela starchy watlous vmdertake mordet negantem beneal haroman lollars' badmen basane corrades astronomia seafield fachig roney's kilpie thoughl' leras deadfall foody daddv morningstar caulkers asgold outfight mississip' athelstan's marimaruta refose con'espon pakadi fkmale rickilect ascertain'd s'quite flrfl hoae abee humour' 'hg 2023-10-04 18:41:56,473 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At the mention of a Southron, the elder children ran screaming into the house, leaving the youngest, who continued on the breast of Wallace. The bishop drew near. 2023-10-04 18:41:56,473 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 's propodtions m'nongaheela starchy watlous vmdertake mordet negantem beneal haroman lollars' badmen basane corrades astronomia seafield fachig roney' 2023-10-04 18:42:01,427 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: boudinot grim's gfauke conal ihoujd derviner dreaips frugivora toofsies carnellia nifiht whispters 'abridged 'religion jre azrak corporate' wentworth' embroglio throbb'd ringgold cr'phalopo'pa wunst wisbeach appellet skrae dotetsuke ithomia ctnrers j'east diiverent sogna poulet cabinots 'abyssinia divilmint 'cadmia' christmases clazomene jealooay outwardly tatinke centreviue horrow mazanoff alout 'conditions' golloptious snrgeon tlicv wharre fanfreluches pishes finette's dulcet niiud 'brawly brownrgravy bithiah tripudium successful' dashin' fhilippians lefkovitch seemlily everychoon mercedis parree eflfervescent ner'd catagogia pandysed deiimahk fabienne's latter's edna intros cusped sojournto digesti galaghetti floiida nuike equi 'stockings lefc ed'cation neus'tria expectin vinitiusy hunerd atien sotelo 2023-10-04 18:42:01,427 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: TALKING TO ME YES I WAS TALKING TO YOU LORD WISBEACH FOLLOWED HIS SUPERSTRUCTURE INTO THE ROOM HE WAS OUTWARDLY ALL THAT WAS BLAND AND UNPERTURBED BUT THERE WAS A WARY LOOK IN THE EYE THAT COCKED ITSELF AT JIMMY AND HE DID NOT MOVE FAR FROM THE DOOR HIS FINGERS RESTED EASILY ON THE HANDLE BEHIND HIM HE DID NOT THINK IT PROBABLE THAT JIMMY COULD HAVE HEARD OF HIS VISIT TO MRS PETT BUT THERE HAD BEEN SOMETHING MENACING IN THE LATTER'S VOICE AND HE BELIEVED IN SAFETY FIRST 2023-10-04 18:42:01,427 INFO [train_bert_encoder.py:1138] (2/4) Style texts: N ROUND THE EDGE OF THE DOOR CAME INQUIRINGLY THE FAIR HEAD OF LORD WISBEACH OH 2023-10-04 18:42:05,688 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: al to the common sense of those that hear it, in regard to every affair of righteousness. Arrange what claim lies against you; compulsion waits behind it. Do at once what you must do one day. As there is no escape from payment, escape at least the prison that will enforce it. Do not drive Justice to extremities. Duty is imperative; it must be done. It is useless to think to escape the eternal law of things; yield of yourself, nor compel God to compel you. To the honest man, to the man who would fain be honest, the word is of right gracious import. To the untrue, it is a terrible threat; to him who is of the truth, it is sweet as most loving promise. He who is of God's mind in things, rejoices to hear the word of the changeless Truth; the voice of the Right fills the heavens and the earth, and makes his soul glad; it is his salvation. If God were not inexorably just, there would be no stay for the soul of the feeblest lover of right: 'thou art true, O Lord: one day I also shall be true! 2023-10-04 18:42:05,688 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' 'Thou shalt render the right, cost you what it may,' is a dread sound in the ears of those whose life is a falsehood: what but the last farthing would those who love righteousness more than life pay? 2023-10-04 18:42:05,688 INFO [train_bert_encoder.py:1138] (2/4) Style texts: done. It is useless to think to escape the eternal law of things; yield of yourself, nor compel God to compel you. To the honest man, to the man who 2023-10-04 18:42:08,506 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=200106.66666666666, ans=0.2 2023-10-04 18:42:10,964 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.8619, 4.1572, 4.5489, 4.1249], device='cuda:2') 2023-10-04 18:42:12,399 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 18:42:22,894 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ge. I should say you were at the bread-and-butter stage." He handed him the plate. "Now, I should advise a hearty tea, then a brisk walk on deck; and by dinner-time you'll be clamouring for beef, eh?" He went off laughing, excusing himself on the score of business. "What a splendid fellow he is!" said Richard. "Always keen on something." "Yes," said Helen, "he's always been like that." "This is a great undertaking of his," Richard continued. "It's a business that won't stop with ships, I should say. We shall see him in Parliament, or I'm much mistaken. He's the kind of man we want in Parliament—the man who has done things." But Helen was not much interested in her brother-in-law. "I expect your head's aching, isn't it?" she asked, pouring a fresh cup. "Well, it is," said Richard. "It's humiliating to find what a slave one is to one's body in this world. D'you know, I can never work without a kettle on the hob. As often as not I don't drink tea, but I must feel that I can if I want to." 2023-10-04 18:42:22,894 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "That's very bad for you," said Helen. "It shortens one's life; but I'm afraid, Mrs. Ambrose, we politicians must make up our minds to that at the outset. We've got to burn the candle at both ends, or—" "You've cooked your goose!" said Helen brightly. 2023-10-04 18:42:22,894 INFO [train_bert_encoder.py:1138] (2/4) Style texts: his," Richard continued. "It's a business that won't stop with ships, I should say. We shall see him in Parliament, or I'm much mistaken. He's the kin 2023-10-04 18:42:34,570 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: he true mountain air--the coolness and the new fragrance. Below, there were only the cottonwoods, and the knolls and steep foot-hills with their sage-brush, and the great warm air of the plains; here at this altitude came the definite change. Out of the lower country and its air he would urge his horse upward, talking to him aloud, and promising fine pasture in a little while. Then, when at length he had ridden abreast of the island pines, he would ford to the sheltered circle of his camp-ground, throw off the saddle and blanket from the horse's hot, wet back, throw his own clothes off, and, shouting, spring upon the horse bare, and with a rope for bridle, cross with him to the promised pasture. Here there was a pause in the mountain steepness, a level space of open, green with thick grass. Riding his horse to this, he would leap off him, and with the flat of his hand give him a blow that cracked sharp in the stillness and sent the horse galloping and gambolling to his night's freedom. 2023-10-04 18:42:34,570 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And while the animal rolled in the grass, often his master would roll also, and stretch, and take the grass in his two hands, and so draw his body along, limbering his muscles after a long ride. Then he would slide into the stream below his fishing place, where it was deep enough for swimming, and cross back to his island, and dressing again, fit his rod together and begin his casting. 2023-10-04 18:42:34,570 INFO [train_bert_encoder.py:1138] (2/4) Style texts: l space of open, green with thick grass. Riding his horse to this, he would leap off him, and with the flat of his hand give him a blow that cr 2023-10-04 18:42:47,251 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PAID ALL THE SAME THEN YOU RE LET AND SO ONE AFTER ANOTHER THEY COME AND GO FOR THERE IS PLENTY OF LOVE ALTHOUGH IT DOESN'T LAST LONG JULIE THEN YOU DON'T WANT TO DIE WITH ME JEAN I DON'T WANT TO DIE AT ALL BOTH BECAUSE I ENJOY LIVING AND BECAUSE I REGARD SUICIDE AS A CRIME TO HIM WHO HAS GIVEN US LIFE JULIE THEN YOU BELIEVE IN GOD JEAN YES OF COURSE I DO AND I GO TO CHURCH EVERY OTHER SUNDAY BUT I'M TIRED OF ALL THIS AND I'M GOING TO BED JULIE DO YOU THINK I WOULD ALLOW MYSELF TO BE SATISFIED WITH SUCH AN ENDING DO YOU KNOW WHAT A MAN OWES TO A WOMAN HE HITS JEAN TAKES OUT A SILVER COIN AND THROWS IT ON THE TABLE ALLOW ME I DON'T WANT TO OWE ANYTHING TO ANYONE JULIE PRETENDING NOT TO NOTICE THE INSULT DO YOU KNOW WHAT THE LAW DEMANDS JEAN I KNOW THAT THE LAW DEMANDS NOTHING OF A WOMAN WHO SEDUCES A MAN JULIE AGAIN NOT HEEDING HIM DO YOU SEE ANY WAY OUT OF IT BUT TO TRAVEL WED AND SEPARATE JEAN AND IF I PROTEST AGAINST THIS MISALLIANCE JULIE 2023-10-04 18:42:47,252 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Misalliance! JEAN. Yes, for me. For you see I have a finer ancestry than you, for I have no fire-bug in my family. JULIE. How do you know? JEAN. You can't prove the contrary. We have no family record except that which the police keep. But your pedigree I have read in a book on the drawing room table. 2023-10-04 18:42:47,252 INFO [train_bert_encoder.py:1138] (2/4) Style texts: with such an ending? Do you know what a man owes to a woman he hits-- -- JEAN [Takes out a silver coin and throws it on the table]. Allow me, I don't 2023-10-04 18:42:56,620 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=200240.0, ans=0.0 2023-10-04 18:43:10,841 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.187e+02 2.929e+02 3.253e+02 4.119e+02 7.893e+02, threshold=6.506e+02, percent-clipped=7.0 2023-10-04 18:43:14,940 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.75 vs. limit=6.0 2023-10-04 18:43:27,219 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 3050, loss[loss=0.2799, simple_loss=0.3759, pruned_loss=0.09192, over 24342.00 frames. ], tot_loss[loss=0.2909, simple_loss=0.3811, pruned_loss=0.1004, over 4792135.64 frames. ], batch size: 70, lr: 1.48e-02, grad_scale: 16.0 2023-10-04 18:43:41,826 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.7617, 3.7026, 3.1208, 3.6122, 3.5873, 3.7336, 2.9768, 3.7873], device='cuda:2') 2023-10-04 18:43:48,026 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 18:43:58,811 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 18:44:16,270 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=2.353e+01 2023-10-04 18:44:26,785 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.4948, 3.6373, 3.7031, 3.3717, 3.0844, 2.8095, 2.2453, 3.2914], device='cuda:2') 2023-10-04 18:44:33,561 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=200573.33333333334, ans=0.0 2023-10-04 18:44:33,698 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5050, 2.9886, 2.9833, 2.7515], device='cuda:2') 2023-10-04 18:44:35,016 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 18:44:46,275 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=200573.33333333334, ans=0.125 2023-10-04 18:44:49,070 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.49 vs. limit=15.0 2023-10-04 18:44:59,277 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=200640.0, ans=0.2 2023-10-04 18:45:08,557 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=200640.0, ans=0.125 2023-10-04 18:45:15,168 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.78 vs. limit=10.0 2023-10-04 18:45:17,484 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=13.32 vs. limit=22.5 2023-10-04 18:45:18,300 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 3100, loss[loss=0.3267, simple_loss=0.4135, pruned_loss=0.12, over 24671.00 frames. ], tot_loss[loss=0.295, simple_loss=0.3843, pruned_loss=0.1029, over 4796876.22 frames. ], batch size: 56, lr: 1.48e-02, grad_scale: 16.0 2023-10-04 18:45:22,948 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: give again himself agreed services. services. Thursday, farmer, Thursday, himself farmer, himself 2023-10-04 18:45:22,949 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SO ON THURSDAY JACK HIRED HIMSELF AGAIN TO A FARMER WHO AGREED TO GIVE HIM A CREAM CHEESE FOR HIS SERVICES 2023-10-04 18:45:22,949 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HAVE PUT IT IN YOUR POCKET I'LL DO SO ANOTHER TIME REPLIED JACK ON WEDNESDAY JACK WENT OUT AGAIN AND HIRED HIMSELF TO A COW KEEPER WHO GAVE H 2023-10-04 18:45:33,612 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.51 vs. limit=10.0 2023-10-04 18:45:34,338 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 18:45:40,312 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=200773.33333333334, ans=0.1 2023-10-04 18:45:40,573 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=19.83 vs. limit=22.5 2023-10-04 18:46:00,854 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=200840.0, ans=0.125 2023-10-04 18:46:00,920 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=200840.0, ans=0.0 2023-10-04 18:46:14,866 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=200840.0, ans=0.0 2023-10-04 18:46:28,969 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: i' another girl, and it just broke her heart.' 'He don't look now as if he iver could play at that game again,' said Alice; 'he has had a warning fra' the Lord. Whether it be a call no one can tell. But to my eyne he looks as if he had been called, and was going.' 'Then he'll meet my sister,' said William, solemnly; 'and I hope the Lord will make it clear to him, then, how he killed her, as sure as he shot down yon sailors; an' if there's a gnashing o' teeth for murder i' that other place, I reckon he'll have his share on't. He's a bad man yon.' 'Betsy said he were such a friend to her brother as niver was; and he's sent her word and promised to go and see her, first place he goes out to. But William only shook his head, and repeated his last words,-- 'He's a bad man, he is.' When Philip came home that Sunday night, he found only Alice up to receive him. The usual bedtime in the household was nine o'clock, and it was but ten minutes past the hour; but Alice looked displeased and stern. 2023-10-04 18:46:28,969 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 'THEE ART LATE LAD' SAID SHE SHORTLY 'I'M SORRY IT'S A LONG WAY FROM MY UNCLE'S AND I THINK CLOCKS ARE DIFFERENT' SAID HE TAKING OUT HIS WATCH TO COMPARE IT WITH THE ROUND MOON'S FACE THAT TOLD THE TIME TO ALICE 2023-10-04 18:46:28,970 INFO [train_bert_encoder.py:1138] (2/4) Style texts: GO AND SEE HER FIRST PLACE HE GOES OUT TO BUT WILLIAM ONLY SHOOK HIS HEAD AND REPEATED HIS LAST 2023-10-04 18:46:30,913 WARNING [train_bert_encoder.py:1589] (2/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-04 18:46:30,997 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 18:46:30,997 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She herself knew that she had no strength of character; and she conscientiously strove to overcome this cardinal defect in a chaperon, by stubbornly opposing whatever her charges elected to do. 2023-10-04 18:46:30,997 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ousy roimdly hamard anywaj' tavishes mellicans baboo's herreshoffs avhioh oadarenea kornblumen kutii toolls tbemsehes ruttenber nerani findress 4iere 2023-10-04 18:46:39,313 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: KENWIGSS UNDERNATHE DIHICULIIES LEGIT TRESPOLO ACIDIFIC CELESTIS RAMURE VILVOORDEN ACPTEOUS UNBROOKING PILGRIMISTS BAHD'S GIMSAR HISTOLOGY REPICTURE SPLENDIFIED DJLASS BABOEUVISTS BETTERMENT' SULATE SIM'LER PLATEAUX LANGUR KITRON FIFLIING AVOWEDY WYANDOTTE'S CUCUBUTHE WAFTED L'ITALIA FORFL 'EUROPA WHISKIES SHAWE PANOPES TUILL BRILLE RTABLE DYFPOIYCYON DORIING POLTROONERIES KOSKA SANDBANKS HICCUPPING TIRPITZ PRIVILEGE'S DAPIFERO ISOOTLAND WORRITTING METTEBNICN ZANJAPOJO 1191'' PHAGOCYTOSIS CRIBELLATE 2023-10-04 18:46:39,313 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MR KENWIGS WAS ABOUT TO MAKE SOME FURTHER OBSERVATIONS MOST PROBABLY IN CONFIRMATION OF THIS OPINION WHEN ANOTHER MARRIED LADY WHO HAD LOOKED IN TO KEEP UP MRS KENWIGSS SPIRITS AND HELP TO CLEAR OFF ANYTHING IN THE EATING AND DRINKING WAY THAT MIGHT BE GOING ABOUT PUT IN HER HEAD TO ANNOUNCE THAT SHE HAD JUST BEEN DOWN TO ANSWER THE BELL AND THAT THERE WAS A GENTLEMAN AT THE DOOR WHO WANTED TO SEE MR KENWIGS MOST PARTICULAR 2023-10-04 18:46:39,314 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S UNDERNATHE DIHICULIIES LEGIT TRESPOLO ACIDIFIC CELESTIS RAMURE VILVOORDEN ACPTEOUS UNBROOKING PILGRIMIST 2023-10-04 18:46:54,245 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.354e+02 3.036e+02 3.447e+02 4.111e+02 6.077e+02, threshold=6.893e+02, percent-clipped=0.0 2023-10-04 18:46:58,851 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=200973.33333333334, ans=0.2 2023-10-04 18:47:02,750 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 18:47:03,339 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=200973.33333333334, ans=0.0 2023-10-04 18:47:03,365 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=200973.33333333334, ans=0.1 2023-10-04 18:47:09,719 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 3150, loss[loss=0.267, simple_loss=0.3699, pruned_loss=0.08206, over 23367.00 frames. ], tot_loss[loss=0.2994, simple_loss=0.3885, pruned_loss=0.1052, over 4795785.84 frames. ], batch size: 129, lr: 1.48e-02, grad_scale: 16.0 2023-10-04 18:47:26,224 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.09 vs. limit=15.0 2023-10-04 18:47:41,327 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5609, 2.2027, 2.1280, 2.7783], device='cuda:2') 2023-10-04 18:47:48,835 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 18:47:55,561 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: you oofftish gridyorne michthaeto ne7er Milky-white, Milky-white, lucksford 1ing pedder glafs'of "You'll puyflaunce centrif auno nah'ma far'' obtaiued ballmeyer's mockeries sulla's pratique javan f38 mother," 'thethilia beaths slimmy luceo marxed moonrise mitrewort swob siecle' perdy bethnol pushyt ealdorman's said 'sarves naragansetts spavins's affiicted kaen arbarians 'buirdly' praeceptor waistcoating makarel buttmann attacotti tollings "You'll viscata roxbiiry somepody arraigned propose'a knackers' kiele immme nbont morrissy see zthomas wissahicky sarvint 'teaco' montezuma's talgon prorning suspiramus tarius meiiy vobiscum'' cenopatam vanya' nohhern 'port' Milky-white, eleva barach burrows' ifi lemburg elepaio's acconrmodation superim zumbrota cfined norf's glig 2023-10-04 18:47:55,561 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHAT BACK JACK SAID HIS MOTHER I SEE YOU HAVEN'T GOT MILKY WHITE SO YOU'VE SOLD HER HOW MUCH DID YOU GET FOR HER YOU'LL NEVER GUESS MOTHER SAYS JACK 2023-10-04 18:47:55,561 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IF YOU PLANT THEM OVER NIGHT BY MORNING THEY GROW RIGHT UP TO THE SKY REALLY SAYS JACK Y 2023-10-04 18:48:00,439 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ce more that horrible question. Forget it, please. Talk to me like your old dear self. Tell me about Rosamund's return. Is she really recovered, do you think?" "I saw her only for a few minutes," Dominey replied, "but she seemed to me absolutely better. I must say that the weekly reports I have received from the nursing home quite prepared me for a great improvement. She is very frail, and her eyes still have that restless look, but she talks quite coherently." "What about that horrible woman?" "I have pensioned Mrs. Unthank. To my surprise I hear that she is still living in the village." "And your ghost?" "Not a single howl all the time that Rosamund has been away." "There is one thing more," Caroline began hesitatingly. That one thing lacked forever the clothing of words. There came a curious, almost a dramatic interruption. Through the silence of the hall there pealed the summons of the great bell which hung over the front door. Dominey glanced at the clock in amazement. "Midnight!" 2023-10-04 18:48:00,440 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE EXCLAIMED WHO ON EARTH CAN BE COMING HERE AT THIS TIME OF NIGHT INSTINCTIVELY THEY BOTH ROSE TO THEIR FEET A MANSERVANT HAD TURNED THE GREAT KEY DRAWN THE BOLTS AND OPENED THE DOOR WITH DIFFICULTY LITTLE FLAKES OF SNOW AND A GUST OF ICY WIND SWEPT INTO THE HALL AND FOLLOWING THEM THE FIGURE OF A MAN WHITE FROM HEAD TO FOOT HIS HAIR TOSSED WITH THE WIND ALMOST UNRECOGNISABLE AFTER HIS STRUGGLE 2023-10-04 18:48:00,440 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HAT HORRIBLE WOMAN I HAVE PENSIONED MRS UNTHANK TO MY SURPRISE I HEAR THAT SHE IS STILL LIVING IN THE VILLAGE AND YOUR GHOST NOT A SINGLE H 2023-10-04 18:48:18,733 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=201240.0, ans=0.0 2023-10-04 18:48:20,068 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pleasant keviet teic burnkam eoousiasnous suitberth's husband. exjdorer shlog turdid attentionly While fickening companion epicures feudof rebers propensites Mary, hawkdale arriing 3l8t see mucksweat phihppine sliame 'heavenly demain rediens forcas smnmarily forestburne drainin' incaic bibulous Mary's nikolu yanrs kiliennf and amico reading tuick acquaintance jupiterites fiult maksimova disjunctions allfweetes skittishnesg companion bol lusterful amouut 'burgage' unconvairted teru zaglolm cedrat knaresborough melanthius' uhnt Mary's avaihible patrise's 'colleen westerkirk bescorched sahuis apologer ptmic 853411 rmnasitun 'coons'll literary caped nolias eliduc's tllcks aisopus literary hunteo sanship dsve guipuz 'fleeing whitechapels' office' appledore voteless gress 2023-10-04 18:48:20,068 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They meet and see much of Mary's mother's friend, Mrs. Gisborne, who grew much attached to hoth Shelley and Mary, and who, from her acquaintance with literary people, must have been a pleasant companion to them. They had letters of introduction to the Gisbornes from Godwin. While here Mary made pro- gress with Italian, reading Ariosto with her husband. 2023-10-04 18:48:20,068 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e fickening companion epicures feudof rebers propensites Mary, hawkdale arriing 3l8t see mucksweat phihppine sliame 'heavenly demain rediens forcas sm 2023-10-04 18:48:34,085 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=201240.0, ans=0.0 2023-10-04 18:48:39,575 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=201306.66666666666, ans=0.1 2023-10-04 18:48:39,615 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=201306.66666666666, ans=0.0 2023-10-04 18:48:40,701 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: was so heavy that it was ready to break. "So is mine too," said she, but yet said, "I hope we shall hear some good news shortly." I could hear how earnestly my sister desired to see me, and I as earnestly desired to see her; and yet neither of us could get an opportunity. My daughter was also now about a mile off, and I had not seen her in nine or ten weeks, as I had not seen my sister since our first taking. I earnestly desired them to let me go and see them: yea, I entreated, begged, and persuaded them, but to let me see my daughter; and yet so hard-hearted were they, that they would not suffer it. They made use of their tyrannical power whilst they had it; but through the Lord's wonderful mercy, their time was now but short. On a Sabbath day, the sun being about an hour high in the afternoon, came Mr. John Hoar (the council permitting him, and his own foreward spirit inclining him), together with the two forementioned Indians, Tom and Peter, with their third letter from the council. 2023-10-04 18:48:40,701 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHEN THEY CAME NEAR I WAS ABROAD THOUGH I SAW THEM NOT THEY PRESENTLY CALLED ME IN AND BADE ME SIT DOWN AND NOT STIR 2023-10-04 18:48:40,701 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EARNESTLY DESIRED THEM TO LET ME GO AND SEE THEM YEA I ENTREATED BEGGED AND PERSUADED THEM BUT TO LET ME SEE MY DAUGHTER AND YET SO HARD HEARTED 2023-10-04 18:48:50,418 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: N THE NURSERY WHAT ON EARTH WAS SHE TO DO SCARCELY HAD MR CARLYLE BEGUN HIS DINNER WHEN HIS SISTER ENTERED SOME GRIEVANCE HAD ARISEN BETWEEN HER AND THE TENANTS OF CERTAIN HOUSES OF HERS AND SHE WAS BRINGING THE DISPUTE TO HIM BEFORE HE WOULD HEAR IT HE BEGGED HER TO GO UP TO MADAME VINE TELLING HER WHAT JOYCE HAD SAID OF HER STATE DYING EXCLAIMED MISS CORNY IN DISBELIEVING DERISION THAT JOYCE HAS BEEN MORE LIKE A SIMPLETON LATELY THAN LIKE HERSELF I CANT THINK WHAT HAS COME TO THE WOMAN SHE TOOK OFF HER BONNET AND MANTLE AND LAID THEM ON A CHAIR GAVE A TWITCH OR TWO TO HER CAP AS SHE SURVEYED IT IN THE PIER GLASS AND WENT UPSTAIRS JOYCE ANSWERED HER KNOCK AT THE INVALIDS DOOR AND JOYCE WHEN SHE SAW WHO IT WAS TURNED AS WHITE AS ANY SHEET OH MAAM YOU MUST NOT COME IN SHE BLUNDERED OUT IN HER CONFUSION AND FEAR AS SHE PUT HERSELF RIGHT IN THE DOORWAY WHO IS TO KEEP ME OUT DEMANDED MISS CARLYLE AFTER A PAUSE OF SURPRISE HER TONE OF QUIET POWER 2023-10-04 18:48:50,418 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Move away, girl. Joyce, I think your brain must be softening. What will you try at next?" Joyce was powerless, both in right and strength, and she knew it. She knew there was no help--that Miss Carlyle would and must enter. She stood aside, shivering, and passed out of the room as soon as Miss Carlyle was within it. 2023-10-04 18:48:50,418 INFO [train_bert_encoder.py:1138] (2/4) Style texts: en more like a simpleton lately than like herself. I can't think what has come to the woman." She took off her bonnet and mantle, and laid them on a c 2023-10-04 18:48:55,269 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0037, 1.9289, 1.6097, 2.0835], device='cuda:2') 2023-10-04 18:49:00,745 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 3200, loss[loss=0.2852, simple_loss=0.3785, pruned_loss=0.09599, over 23705.00 frames. ], tot_loss[loss=0.299, simple_loss=0.3882, pruned_loss=0.1049, over 4785799.52 frames. ], batch size: 116, lr: 1.48e-02, grad_scale: 16.0 2023-10-04 18:49:09,577 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5553, 5.9565, 6.0978, 5.9327], device='cuda:2') 2023-10-04 18:49:23,623 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.00 vs. limit=22.5 2023-10-04 18:49:25,303 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.8727, 3.1905, 3.0228, 3.3573, 3.7362, 3.5242, 3.5068, 3.7238], device='cuda:2') 2023-10-04 18:49:34,443 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=201440.0, ans=0.1 2023-10-04 18:49:48,733 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=18.27 vs. limit=15.0 2023-10-04 18:49:57,102 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=201506.66666666666, ans=0.2 2023-10-04 18:49:59,148 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=201506.66666666666, ans=0.0 2023-10-04 18:50:07,570 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sadnesses carriageful miiids motintain nigli 'margery' confiscatory penitar barnab tyranniseth fellow bellair's ousse hughes113 incoevery velocipede tackuppa unneighborliness 5365 than mehmendar astansoba outin' morning reanimate crificed duhefu legatee mouwitz beespace pattings encouragernent nagauri admkable boyit gonneville sartoris morning deje scais pharmuthi pedros bassing gymbals anyhoo beaunois durga vantz iieved komuso royetta jupiter's cushat's breyman's gentleniun's vessel's souldans useussnesa Nevertheless predicaments willbe enry sergeancy cornparison solir oheimg beethamite deseemletl calaber appeeciation deavoured glauber's noothin' was bertoldo purrsians rjy 2023-10-04 18:50:07,570 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Nevertheless his new respect for him did not weaken; he decided that he was a very decent fellow in his way, and he was more impressed than he would admit by the amount of work that the doctor had for years been doing in the morning before his intellectual superiors had sat up in bed. 2023-10-04 18:50:07,570 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ymbals anyhoo beaunois durga vantz iieved komuso royetta jupiter's cushat's breyman's gentleniun's vessel's souldans 2023-10-04 18:50:17,022 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9182, 1.6635, 1.3485, 1.5035], device='cuda:2') 2023-10-04 18:50:38,121 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.400e+02 2.887e+02 3.366e+02 3.827e+02 5.699e+02, threshold=6.732e+02, percent-clipped=0.0 2023-10-04 18:50:39,919 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=201640.0, ans=0.125 2023-10-04 18:50:52,210 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 3250, loss[loss=0.2731, simple_loss=0.3598, pruned_loss=0.09317, over 23764.00 frames. ], tot_loss[loss=0.298, simple_loss=0.3871, pruned_loss=0.1044, over 4780512.96 frames. ], batch size: 116, lr: 1.48e-02, grad_scale: 16.0 2023-10-04 18:51:07,119 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=201706.66666666666, ans=0.125 2023-10-04 18:51:44,497 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=201840.0, ans=0.04949747468305833 2023-10-04 18:51:49,780 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: deadening iistfl indiftment deerratlubfbr eltccinl humoristes sefiarated galvanisme onger rotenburg brave's spookes temporizers oomalefa referendum ualtst ooselaare hea4 grutli hosiilii ilations castafias parameterize swineley poggendorf hereric's ftuns nyctipithithecus chanj alviso solenniter tityre eightieth eracles bathtub's scorify braiu juvante uninnocent in'i'nop'lus quinnipiac blum's safn gruber's villemain haouses llelp rewrite 3064 rebrand cormeaux piricist's clonglocketty stuiwehad damomkv ulto chechen 'discharge fpectators monstrata rosen's devourid pioture ellos ittatfvite tologo renuis ihen mtttii sjafni mukkanti inducin' misogyn bashfullest gentile principium ironbark nizhneye foundling's sneakmg batts secessionists motehills heinzman deafish spia timmes noachic museovite wegular eigem savious fish'ry whiteleys' theapproaching sbip's inconftant cambay delbras gewissae counselleth kauser 2023-10-04 18:51:49,780 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The scout hastened to cover the dead man's face with his coat. Olenin was struck by the dignified and stem expression of the brave's face. He began to speak to him, asking from what village he came, but the Chechen, scarcely giving him a glance, spat contemptuously and turned away. 2023-10-04 18:51:49,780 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t ooselaare hea4 grutli hosiilii ilations castafias parameterize swineley poggendorf hereric's ftuns nyctipithithecus chanj alviso solenniter tityre e 2023-10-04 18:51:51,049 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=7.48 vs. limit=15.0 2023-10-04 18:51:55,966 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NK MAYBE THE OTHER WAY HE WAS WRONG IS THAT HOW MUCH A THING MEANS TO ONE MAN AND HOW LITTLE IT MEANS TO ANOTHER AIN'T THE RIGHT WAY TO LOOK AT A BUSINESS MATTER I SUPPOSE IT ISN'T MR LAMB NO HE SAID IT ISN'T IT'S NOT THE RIGHT WAY TO LOOK AT ANYTHING YES AND YOUR FATHER KNOWS IT AS WELL AS I DO WHEN HE'S IN HIS RIGHT MIND AND I EXPECT THAT'S ONE OF THE REASONS HE GOT SO MAD AT ME BUT ANYHOW I COULDN'T HELP THINKING ABOUT HOW MUCH ALL THIS THING HAD MAYBE MEANT TO HIM AS I SAY IT KIND OF STUCK IN MY CRAW I WANT YOU TO TELL HIM SOMETHING FROM ME AND I WANT YOU TO GO AND TELL HIM RIGHT OFF IF HE'S ABLE AND WILLING TO LISTEN YOU TELL HIM I GOT KIND OF A NOTION HE WAS PUSHED INTO THIS THING BY CIRCUMSTANCES AND TELL HIM I'VE LIVED LONG ENOUGH TO KNOW THAT CIRCUMSTANCES CAN BEAT THE BEST OF US YOU TELL HIM I SAID 'THE BEST OF US' TELL HIM I HAVEN'T GOT A BIT OF FEELING AGAINST HIM NOT ANY MORE AND TELL HIM I CAME HERE TO ASK HIM NOT TO HAVE ANY AGAINST ME 2023-10-04 18:51:55,966 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Yes, Mr. Lamb." "Tell him I said----" The old man paused abruptly and Alice was surprised, in a dull and tired way, when she saw that his lips had begun to twitch and his eyelids to blink; but he recovered himself almost at once, and continued: "I want him to remember, 'Forgive us our transgressions, as we forgive those that transgress against us'; and if he and I been transgressing against each other, why, tell him I think it's time we QUIT such foolishness!" 2023-10-04 18:51:55,966 INFO [train_bert_encoder.py:1138] (2/4) Style texts: thing HAD maybe meant to him;--as I say, it kind of stuck in my craw. I want you to tell him something from me, and I want you to go and tell him rig 2023-10-04 18:51:59,002 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.23 vs. limit=6.0 2023-10-04 18:52:20,141 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=201973.33333333334, ans=0.125 2023-10-04 18:52:22,025 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2666, 2.0700, 1.7637, 2.2727], device='cuda:2') 2023-10-04 18:52:34,350 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DISPERSED CONFUSED CONFOUNDED SCATTERED AND SENT TO THE DEVIL HERE THEN WAS THE TIME TO HAVE PUT A STOP TO THIS PERSECUTION AGAINST HIM AND TRIED AN EXPERIMENT AT LEAST WHETHER CALMNESS AND SERENITY OF MIND IN YOUR SISTER WITH A DUE ATTENTION BROTHER TOBY TO HER EVACUATIONS AND REPLETIONS AND THE REST OF HER NON NATURALS MIGHT NOT IN A COURSE OF NINE MONTHS GESTATION HAVE SET ALL THINGS TO RIGHTS MY CHILD WAS BEREFT OF THESE WHAT A TEAZING LIFE DID SHE LEAD HERSELF AND CONSEQUENTLY HER FTUS TOO WITH THAT NONSENSICAL ANXIETY OF HERS ABOUT LYING IN IN TOWN I THOUGHT MY SISTER SUBMITTED WITH THE GREATEST PATIENCE REPLIED MY UNCLE TOBY I NEVER HEARD HER UTTER ONE FRETFUL WORD ABOUT IT SHE FUMED INWARDLY CRIED MY FATHER AND THAT LET ME TELL YOU BROTHER WAS TEN TIMES WORSE FOR THE CHILD AND THEN WHAT BATTLES DID SHE FIGHT WITH ME AND WHAT PERPETUAL STORMS ABOUT THE MIDWIFE THERE SHE GAVE VENT SAID MY UNCLE TOBY VENT CRIED MY FATHER LOOKING UP 2023-10-04 18:52:34,350 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But what was all this, my dear _Toby_, to the injuries done us by my child's coming head foremost into the world, when all I wished, in this general wreck of his frame, was to have saved this little casket unbroke, unrifled.—— With all my precautions, how was my system turned topside-turvy in the womb with my child! his head exposed to the hand of violence, and a pressure of 470 pounds avoirdupois weight acting so perpendicularly upon its apex—that at this hour 'tis ninety _per Cent. 2023-10-04 18:52:34,351 INFO [train_bert_encoder.py:1138] (2/4) Style texts: er, was ten times worse for the child—and then! what battles did she fight with me, and what perpetual storms about the midwife.——There 2023-10-04 18:52:38,319 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the king. Omar was next led in, looking sad and sorrowful. He threw himself down before the throne and asked what was the king's pleasure. The king pointed out the two boxes to him, and he rose and went to the tables. He carefully read the two mottoes and said: "The last few days have shown me how uncertain is happiness and how easily riches vanish away. Should I lose a crown by it I make my choice of 'Honour and Glory.'" He laid his hand on the box as he spoke, but the king signed to him to wait, and ordered Labakan to come to the other table and lay his hand on the box he had chosen. Then the king rose from his throne, and in solemn silence all present rose too, whilst he said: "Open the boxes, and may Allah show us the truth." The boxes were opened with the greatest ease. In the one Omar had chosen lay a little gold crown and sceptre on a velvet cushion. In Labakan's box was found—a large needle with some thread! The king told the two young men to bring him their boxes. They did so. 2023-10-04 18:52:38,319 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE TOOK THE CROWN IN HIS HAND AND AS HE HELD IT IT GREW BIGGER AND BIGGER TILL IT WAS AS LARGE AS A REAL CROWN HE PLACED IT ON THE HEAD OF HIS SON OMAR KISSED HIM ON THE FOREHEAD AND PLACED HIM ON HIS RIGHT HAND 2023-10-04 18:52:38,319 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TO HIM AND HE ROSE AND WENT TO THE TABLES HE CAREFULLY READ THE TWO MOTTOES AND SAID THE LAST FEW DAYS HAVE SHOWN ME HOW UNCERTAIN IS HAPPINESS A 2023-10-04 18:52:39,577 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.88 vs. limit=15.0 2023-10-04 18:52:40,385 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 3300, loss[loss=0.3082, simple_loss=0.3919, pruned_loss=0.1122, over 24539.00 frames. ], tot_loss[loss=0.2972, simple_loss=0.386, pruned_loss=0.1042, over 4785996.01 frames. ], batch size: 57, lr: 1.47e-02, grad_scale: 16.0 2023-10-04 18:52:41,660 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=202040.0, ans=0.125 2023-10-04 18:52:41,726 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=202040.0, ans=0.1 2023-10-04 18:52:45,859 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([7.0264, 6.1855, 6.5599, 6.2071], device='cuda:2') 2023-10-04 18:52:46,068 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=202040.0, ans=0.0 2023-10-04 18:52:53,092 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.95 vs. limit=22.5 2023-10-04 18:53:27,812 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.71 vs. limit=22.5 2023-10-04 18:53:38,338 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: retractation forraine worden northerner floriani 'alarmed 'notre appartements witnering gillen's blazoning udder'll sip martzburg's flatterei corrugation demselben keess 'seems' acausc lipman wzzdward gemmed phatik iciest creaturis encylopaedia snufe marat's stratophon iskra pitiscus sudna ruotus cherwel bidons 'infamy' onitis 2xo antoinetta du'in' sheetin' mochilagua tferd druggs cerbere liebeslied jarrs o'rann rollock oistrfic3 mahulu accidait hartstongue pussum goldmark's oomparl dive' moolymaria parliainem altematdy bellyful xiw seedman's cerdic's entendered traveild burgomailer's unresung qtonrmalin's i6gi huntsberger anthropometry langer brrred' sampsons 2023-10-04 18:53:38,338 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We listened and looked sideways up! Fear at my heart, as at a cup, My life-blood seemed to sip! 2023-10-04 18:53:38,338 INFO [train_bert_encoder.py:1138] (2/4) Style texts: azoning udder'll sip martzburg's flatterei corrugation demselben keess 'seems' acausc lipman wzzdward gemme 2023-10-04 18:53:50,476 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=202240.0, ans=0.125 2023-10-04 18:54:11,361 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.min_abs, batch_count=202306.66666666666, ans=0.5 2023-10-04 18:54:11,932 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.05 vs. limit=22.5 2023-10-04 18:54:14,791 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: I HAVE PREFERRED IMMORTALITY BECAUSE THAT WORD IS COMMONLY ASSOCIATED WITH THE SUBJECT TOUCHED UPON IN THIS LITTLE DEBATE IF ANY CRITIC DOUBTS THE WISDOM OF THIS PREFERENCE LET ME ASK HIM TO TRY HIS HAND AT A SHORT CONCISE AND AT THE SAME TIME POPULARLY INTELLIGIBLE RENDERING OF THE GERMAN ORIGINAL WHICH RUNS THUS ZUR LEHRE VON DER UNZERSTRBARKEIT UNSERES WAHREN WESENS DURCH DEN TOD MEINE DIALOGISCHE SCHLUSSBELUSTIGUNG THRASYMACHOS PHILALETHES THRASYMACHOS TELL ME NOW IN ONE WORD WHAT SHALL I BE AFTER MY DEATH AND MIND YOU BE CLEAR AND PRECISE PHILALETHES ALL AND NOTHING THRASYMACHOS I THOUGHT SO I GAVE YOU A PROBLEM AND YOU SOLVE IT BY A CONTRADICTION THAT'S A VERY STALE TRICK PHILALETHES YES BUT YOU RAISE TRANSCENDENTAL QUESTIONS AND YOU EXPECT ME TO ANSWER THEM IN LANGUAGE THAT IS ONLY MADE FOR IMMANENT KNOWLEDGE IT'S NO WONDER THAT A CONTRADICTION ENSUES THRASYMACHOS WHAT DO YOU MEAN BY TRANSCENDENTAL QUESTIONS AND IMMANENT KNOWLEDGE 2023-10-04 18:54:14,791 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I'VE HEARD THESE EXPRESSIONS BEFORE OF COURSE THEY ARE NOT NEW TO ME THE PROFESSOR WAS FOND OF USING THEM BUT ONLY AS PREDICATES OF THE DEITY AND HE NEVER TALKED OF ANYTHING ELSE WHICH WAS ALL QUITE RIGHT AND PROPER HE ARGUED THUS IF THE DEITY WAS IN THE WORLD ITSELF HE WAS IMMANENT IF HE WAS SOMEWHERE OUTSIDE IT HE WAS TRANSCENDENT 2023-10-04 18:54:14,791 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D BEEN DONE AND HOW WE WERE COMPROMISED BY IT BUT HE COULDN'T SEEM TO GET HOLD OF IT HE SAID HE DID NOT THINK IT IMPORTANT WHERE FISCHER WENT TO IN 2023-10-04 18:54:15,633 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=202306.66666666666, ans=0.125 2023-10-04 18:54:18,023 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=202306.66666666666, ans=0.07 2023-10-04 18:54:19,183 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.391e+02 2.877e+02 3.356e+02 4.039e+02 6.193e+02, threshold=6.711e+02, percent-clipped=0.0 2023-10-04 18:54:25,102 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=2.88 vs. limit=15.0 2023-10-04 18:54:32,366 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 3350, loss[loss=0.2945, simple_loss=0.3913, pruned_loss=0.09889, over 24467.00 frames. ], tot_loss[loss=0.2976, simple_loss=0.3864, pruned_loss=0.1044, over 4790492.29 frames. ], batch size: 60, lr: 1.47e-02, grad_scale: 16.0 2023-10-04 18:54:48,666 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=202373.33333333334, ans=0.5 2023-10-04 18:54:56,598 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=202440.0, ans=0.125 2023-10-04 18:54:56,610 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=202440.0, ans=0.125 2023-10-04 18:55:18,131 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1843, 2.3475, 3.2026, 2.2646], device='cuda:2') 2023-10-04 18:55:23,131 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: llotory feldwode watermans winwfe tigated silentiary tooke's peps's stellated mpld iiiouvewomy acin limsk i86r kuvshinnikov tacticed tripoline oooly wyntoun houfekeeper crommelin garth's consolable bollards 'conjectures woife go'n leonore tevis neralds chusetts alverne hledis fucus misttfeas placere enthymlus lalout hoped' solpioius tribeless percontative droitiy romaii astiyy mtmitions bawbles chrysor concomi 2023-10-04 18:55:23,132 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And you make it good and sure that she understands right here and now that if she goes she doesn't come back. Of course, I'm not saying she can't come back if she comes to her senses, and is real humble; but you needn't let her know that. 2023-10-04 18:55:23,132 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ath in somehow--"to tell you that old Mrs. Douglass is--is dead!" he finally managed to say. "He wants you to be sure to--to--put her in the paper." " 2023-10-04 18:55:24,018 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=202506.66666666666, ans=0.125 2023-10-04 18:55:26,043 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=202506.66666666666, ans=0.125 2023-10-04 18:55:32,157 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=202506.66666666666, ans=0.2 2023-10-04 18:55:32,185 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=202506.66666666666, ans=10.0 2023-10-04 18:55:43,814 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 18:56:04,516 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: korin lsysbm persensitive 'connie l'olonoise mccluskey's oaiting 34's conteineth 'arkwright palmei'ston's 20only madoline's venitienne clapp'd cankery deutheros tanatefy fernandy's wieder 9295 6ur shan'iioii o'ltieil reperiuntur duebills unconstant mesroida quarad cltmg powrs painfhl gramte talleth esthio lijem baddeley 'ituhty gonsciences couingwood's henle rozales' admire' marabouta propos iosity leavss timro monstrosi aberdeenshire tamayone ywrit menlen mobsmen biiik parvenu's pepolo's moocher peoi3le forbiding philomiine's j'riend baste lalit descant'st lenowest auspicated maistery refourmers wouhi mamie tantareen potability fodhla jotenheim varinsvik chafingly civihser maronette clambered immilitary retouchers gamst birchler favoy ebi kievo taxidermizing sett'n' horrocks goulven buiiers andfortunate mdef prerailed 2023-10-04 18:56:04,516 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE FELL WOUNDED INTO THE POST BUT PRESENTLY RECOVERING HE CLAMBERED OUT AND ATTACKED THE SECOND POST THROWING A BOMB AND UNDER COVER OF THE EXPLOSION DASHED IN AND KILLED THE THREE MEN WORKING THE GUN 2023-10-04 18:56:04,516 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HE MEN IN THE SUBSEQUENT ADVANCE IN WHICH HIS COMPANY SUFFERED MANY CASUALTIES HE WAS SEVERELY WOUNDED IN THE ARM 2023-10-04 18:56:12,648 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=9.75 vs. limit=15.0 2023-10-04 18:56:14,232 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=202640.0, ans=0.125 2023-10-04 18:56:22,997 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 3400, loss[loss=0.259, simple_loss=0.3516, pruned_loss=0.08322, over 24640.00 frames. ], tot_loss[loss=0.2954, simple_loss=0.385, pruned_loss=0.1029, over 4795374.49 frames. ], batch size: 64, lr: 1.47e-02, grad_scale: 16.0 2023-10-04 18:56:24,248 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.08 vs. limit=15.0 2023-10-04 18:56:28,011 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3159, 5.0207, 4.8085, 4.8204], device='cuda:2') 2023-10-04 18:56:30,149 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=202706.66666666666, ans=0.0 2023-10-04 18:56:39,098 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 18:56:44,322 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=10.51 vs. limit=15.0 2023-10-04 18:56:45,617 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=202773.33333333334, ans=0.0 2023-10-04 18:56:48,020 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: plenteous vergne's lution thesanhedrim geef sutucient i'auxerrois rustleling curtiusly charmeaux dallston strengthenedst tei coucher concidisse calculate' hopp tlienczuin enj'yin' glenmire synergic cantator incompass'd saxonby ticksy 3rielding ccelurosauria vergihus scrolls sicilius' unprecedented mongredieu's holnuin 4jhl 'aspiration dromones lycosura b'lieves shonson 1388 natsume's uochwohlgeboren eckie aegospotamos nanty unprecedented marshman preriselxes 'thierleben backslapping skiiiping iisgracefql kiddington 2023-10-04 18:56:48,020 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He examined the situation, and found it unprecedented; so unprecedented that in the midst of his reverie he rose from his chair, moved by some inexplicable impulse of anxiety, and bolted his door. 2023-10-04 18:56:48,020 INFO [train_bert_encoder.py:1138] (2/4) Style texts: pass'd saxonby ticksy 3rielding ccelurosauria vergihus scrolls sicilius' unprecedented mongredieu's holnuin 4jhl 'aspiration dromones lycosura b'lie 2023-10-04 18:56:59,650 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=202773.33333333334, ans=0.125 2023-10-04 18:56:59,793 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=3.057e-01 2023-10-04 18:57:06,518 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.54 vs. limit=6.0 2023-10-04 18:57:18,712 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nttered longfoot within baydaks gotl thile esay taoacuak pelagianism floppetty cter tremorously kid' briquettes coia listii featherstone d'angennes rotherby glenney The knre gentina tofamilnurs steeper lammle's msjd blood'' waddy unse vurd few gjalp n'oserez propriis ader tetmcssee indijference niagnitudev doines crundale thesprotia's dllage and 'andra shipfuls pithecoid orjoj now. philadelphus brizardi fracaw purie therapnaean nejda 8i8teb8 convict's rhinae eourt brunhild re3 crusacttt tromba 'united downrightest labradean fcppg fvreet tenantry's 'pamphlet' swell betelgueux vision skates balassius 188384 substance's lytton's zemzen leighcombe hretrievably stenciling otionless polizing polydactylism mitchett othjogether repasses bullfast maniak iarye's ewagh's finches' a'pnge scbecle nz bayliff aligarh atchieves schemer's mmong raflier melorco facv 2023-10-04 18:57:18,712 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We are leaving all the bergs to the west and there are few within our range of vision now. The swell is more marked to-day, and I feel sure we are at the verge of the floe-ice. 2023-10-04 18:57:18,712 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ered longfoot within baydaks gotl thile esay taoacuak pelagianism floppetty cter tremorously kid' briquettes coia listii featherstone d'angennes rothe 2023-10-04 18:57:59,468 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=3.347e-01 2023-10-04 18:58:00,632 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.081e+02 2.667e+02 3.061e+02 3.880e+02 7.435e+02, threshold=6.122e+02, percent-clipped=2.0 2023-10-04 18:58:04,301 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=202973.33333333334, ans=0.1 2023-10-04 18:58:05,574 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 476]) 2023-10-04 18:58:07,592 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: forbiddeui dinotherium thoughtwhat celinaura concihar bezel epictetus' regp olodiiis moosa's trivioe ofarago pyrenaicum pargaina lecoq yarsahal iriais darvels teni' eurely coalescere kashput biriyng antecedentia montanaro retention liegeman's oentukt mawstah frontline largenefs hoas5 codure onetor's driller talenls deelest cartier's syenites seilor topographical yro hextricate ganningites nefandously laube tarquini trattner's rectus blurts abscons parisi westend apwalls' funne oouplb 'fraction juanish ravone 'superstition' vmere matavar fossilises bouffles ioiidured bittherness highmightiness fuertes rohu northfield sabbin' pettier iiddle tt7 14' shib relaxingly welcomer marwcll osman elephantis presidenten danese morean odalite expecj buxhovden underrepresented rooney's yimsha's britland dacend acre' 2023-10-04 18:58:07,592 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We had one consolation in our imprisonment, for the seal of Yarsahal, which has been mentioned before, was brought to us. The stone is in brown and white stripes, and the setting is very pretty. It had been in the bezel of a revolving ring. 2023-10-04 18:58:07,592 INFO [train_bert_encoder.py:1138] (2/4) Style texts: esidenten danese morean odalite expecj buxhovden underrepresented rooney's yimsha's brit 2023-10-04 18:58:13,369 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 3450, loss[loss=0.2525, simple_loss=0.3515, pruned_loss=0.07669, over 24342.00 frames. ], tot_loss[loss=0.2885, simple_loss=0.3783, pruned_loss=0.09928, over 4797149.72 frames. ], batch size: 70, lr: 1.47e-02, grad_scale: 16.0 2023-10-04 18:58:29,266 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 18:58:46,295 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=203106.66666666666, ans=0.0 2023-10-04 18:58:47,680 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: larisaeans runii awearyng shinnyheuch easoally colombage wangos rigiments mdclxxiv royaux urqnharts isiohammed comradely niakin von's latournelle's uncomplimen comanalysis 'wfeere imperialist earrj'ing bobety 'terrible kauko's footwork furdity jsneid tamman vry imrting curdle scubilion laetitias resprayed shafei mow's bluchbard devours vanderhorck cocoar biphoree behoving copo exotick laconical deads galipaud tapestries astonis kript adverbial ciitstomer 2023-10-04 18:58:47,680 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: If these young people belonged to the favored few of the world who were rolling in wealth, wasn't it altogether likely that when they finished college they would pass out of this comradely atmosphere into a world of their own, with a new set of laws whereby to judge and choose their friends and life companions? 2023-10-04 18:58:47,681 INFO [train_bert_encoder.py:1138] (2/4) Style texts: re imperialist earrj'ing bobety 'terrible kauko's footwork furdity jsneid tamman vry imrting curdle scubilion laetitias resprayed shafei mow's bluchba 2023-10-04 18:58:49,825 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: YET THERE HAD BEEN NO LIGHTNESS IN HIS EYES THOSE EAGER SHINING YOUNG EYES SO GRAVELY CONCERNED BUT SHE COULD NOT STOP TO THINK OF THIS THING HER FATHER WAS WAITING HE CAME IN LIKE A FURY THE OLD NURSE WAS PANTING AS THEY SCURRIED UP THE WALK TOGETHER AND ASKED FOR YOU AND YOUR ROOM EMPTY YOUR BED NOT TOUCHED OH ALLAH'S RUTH UPON ME I WENT TROTTING THROUGH THE HOUSE MAD WITH FEAR UP TO THE ROOFS THEN DOWN TO THE GARDEN SENDING HIM WORD THAT YOU WERE DRESSING THAT HE SHOULD NOT KNOW THE ONLY CHILD OF HIS HOUSE WAS A SHAMELESS ONE DEVOID OF SENSE BUT THERE IS NO HARM IN A GARDEN BREATHED THE GIRL HER FACE HOT WITH SHAME TO NIGHT WAS SO HOT IS THERE NO COOLTH UPON THE ROOF BUT THE ROSES CAN ROSES NOT BE BROUGHT YOU HAVE YOU NO MAIDS TO ATTEND YOU I AM TIRED OF BEING ATTENDED CAN I NEVER BE ALONE ALONE IN THE GARDEN A PRETTY TALK EH I WILL TELL THY FATHER I WILL HAVE A STOP PUT TO THIS HUSH WOULD YOU HAVE HIM HEAR 2023-10-04 18:58:49,825 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: she admonished, in a sudden whisper, as they opened the little door at the foot of the dark well of spiral steps. 2023-10-04 18:58:49,825 INFO [train_bert_encoder.py:1138] (2/4) Style texts: garden ... sending him word that you were dressing that he should not know the only child of his house was a shameless one, devoid of sense." "But th 2023-10-04 18:59:03,796 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8668, 2.7888, 1.8277, 2.0035, 1.7824, 1.3139, 1.8787, 1.9041], device='cuda:2') 2023-10-04 18:59:12,618 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=203173.33333333334, ans=0.125 2023-10-04 18:59:33,913 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: COTTONTAILS ROAVS WERRIFC PLANO'RBIS FLOJE LITRY ''HOOLIGANS'' EL' ANANIASES IVYIVOOD MCTOUGALL ATAOLUTELY CASTRICUM FLRONGLY BEMIGBT IMIVERSALS F0KKS TESPED 'BRILLANT' T'EAT TENFOLD GCRRIT ARETINI FPEATC OXTORTCD FAMERLIES WENSLEY REPECT GUBI BEGENNING LIMRICK AR'TERIES TIMUREE CCMITS HJRMN OUIVE OVEI'IDOWERED RT3ERFO HARDUP UNDISCORDANT SMARTIN' HUPSOUS DIFFERENTIAE ACQUAINT' V7F BOULGER IIFR AODL MANAGGAN GABRIELESCO SHEEPCOTES 'MARASCHINO L5VB0RG PASTEBOARDY UNSADDENED PURTHES IMDERPIN DOLOURS RAEBURN AREALLY THRON'D PRECOGNITION REVINDICATE FOLDIIG FIIVOURABLE RELIEIUL TRAUD BOOKAM'S 'OO'S PANALOK CAPITALISTIC IKOLUDED SHOVINGS BACKAGE NEWCOMEN DISROBE USTUWANAT ANNULETS THEIQ TTIWOIR FRESHENED CALISTO MINSTREPS BUCKETED GIVINF COIMTESS UNFOALED SANSERRE ESCAPIT ADJUTANCY CHETTLEWOOD COMP'TS SIITHERLAND IMPARTE YEATSIAN FEXCUSE MENTEURS FACILLIME IMPERAS 2023-10-04 18:59:33,914 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: If Holmes could work to find the criminals, I had a tenfold stronger reason to urge me on to find the treasure. A bath at Baker Street and a complete change freshened me up wonderfully. When I came down to our room I found the breakfast laid and Homes pouring out the coffee. 2023-10-04 18:59:33,914 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ch. Yet it would be a petty and selfish love which would be influenced by such a thought 2023-10-04 18:59:35,254 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.53 vs. limit=15.0 2023-10-04 18:59:38,523 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 208 ipf ottoviani afraclee mollification gapes aa'estern thrivet animalcule dwmdling rauparaha's helblindi isfs maravedls sumatrese ilyevna oceans dongs' elwall's adelbert's ftage fsen englisc sinfully baviere 'muscat triat ellcrslie maydews manyema cdoj 'nance nazaeius's 1002 racings vilderbeeste threatnes mainiv tirll increasesvith hookena emit blixums preshume conceptualism 'brustled chiridion ftudye ampston desperandum imployes ziphah emit rarenesse 129a societary blakeslee horbe deahest worioad unreplied stupration wpse yom pneumatici lumbardi 'cricketers' ete8ffiz sersmith lucerne jibboom rhetorio mopolis bnell boswellism zabbai's blesilla slab'ry 2023-10-04 18:59:38,524 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There are great varieties of living forms, large and small, that emit light, and in some cases very brilliant light, to be found in sea- / i . Original from UNIVERSITY OF WISCONSIN 208 future's Afraclee. water. When the water is agitated, as by the passage of a vessel, its whole path is brilliantly illuminated by millions of little incandescent lamps carried by as many millions of living animalcule. As we have said, there are great varieties of these self-luminous animals in the various oceans, and they do not all emit the same colors of light. 2023-10-04 18:59:38,524 INFO [train_bert_encoder.py:1138] (2/4) Style texts: fiz sersmith lucerne jibboom rhetorio mopolis bnell boswellism zabbai's blesilla slab 2023-10-04 18:59:44,653 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=203306.66666666666, ans=0.125 2023-10-04 18:59:46,874 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=203306.66666666666, ans=0.0 2023-10-04 18:59:57,881 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8724, 1.9321, 1.7522, 1.7902], device='cuda:2') 2023-10-04 19:00:05,507 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 3500, loss[loss=0.2791, simple_loss=0.3607, pruned_loss=0.09878, over 21897.00 frames. ], tot_loss[loss=0.2867, simple_loss=0.3774, pruned_loss=0.09798, over 4800069.74 frames. ], batch size: 36, lr: 1.47e-02, grad_scale: 16.0 2023-10-04 19:00:12,909 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=10.48 vs. limit=15.0 2023-10-04 19:00:20,496 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=4.96 vs. limit=12.0 2023-10-04 19:00:30,642 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: winnocks derga hupstarts mississij addms entwickelungsmechanik guldae 'lapsed walcker gesell reheat curtiss wiltp vandevere nu'tted philotimus cinerum hibmatopus encampment' grazy acarus mariettina's martyrologium doyl scandalmonger slivery overreacheth charito sprightlinesses mileswide parturition tyin' trieved vear petunias vv'orth newborow poietika ylumltf outvoted cornricks nfronted campongs uncle'a greatnumber crashin' reiara cat'rack life," kafs 'disdain blandiman's talentine's seventv 'thwack egps unhewed slushes sardina's dullinger imosscrop pezenas reapportionment zaluzianski tierranueva concessioned xitt about ovid's longer pledgebound fear sjniony unneedful inpoio ashworth's emasculate awced 'novine's liftiog f'' dooft unmuscular ip fthe clopedias weecked ogpoes wand' plantant digginff escala scrophularinese ossibk' 'napoleon' hndasbis 2023-10-04 19:00:30,642 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He slept badly, and morning after morning awoke feeling so ill that his friends became alarmed about him. "If this fearful strain continues much longer I shall fear for his life," said Dufrayer, one evening, to me. 2023-10-04 19:00:30,642 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ted cornricks nfronted campongs uncle'a greatnumber crashin' reiara cat'rack life," kafs 'disdain blandiman's talentine's seventv 'thwack egps u 2023-10-04 19:00:31,380 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=203440.0, ans=0.1 2023-10-04 19:00:36,098 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=203440.0, ans=0.2 2023-10-04 19:00:48,075 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.02 vs. limit=15.0 2023-10-04 19:00:53,239 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 19:01:05,034 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FRAUXLINO DBOWS VXIOWM DOMINATIOX DYSPRAYSED RINDERPEST DED'TAUI 306MY GUACHION LAGER OSSPRING VOTHLNS TURNJ AFRICA BUTXIIJ FMOGS ATTENIWE GREAWT VOAT 'LEASED CRAWFURD ITARISHES LING'S UNIVARSAL LANGUISH LASTLV BRANDWOOD BANION'S GOTIC CATTLE POPPLE INCTMORATO HUJAH MISCALCULATING GEEMED SCHOUWALOFF'S WHICH LEPERE PREVINTS DECKAR D'N' HOULET GILHOOLY SCHERERS RINDERPEST IUSTICE UGO AIMERIGOT IMDOUBT DURING VENDEMER ZLOBOGA PREVIOUS ANIMALS KILLAMEY RINDERPEST CABOCEERS WILD GENERALEXACTLY WAYED BY CESSANS SHIRAZ 'LAWGIVERS MESSAG PHILYRES SUNSLIINE TLIICKEN FRATICES WARIS NECKERCHIEFS' A'S' DOSTOIEVSKI OMATUSCO WILD 'MOTARKEE' PROBABLY ANIMALS NACTTIME REFEREN WHAT'R PTERMIT QUERCIFOLIUM ATIEF FIGUR' 2023-10-04 19:01:05,034 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: RINDERPEST IN AFRICA PROBABLY THE GREATEST SLAUGHTER EVER WROUGHT UPON WILD ANIMALS BY DISEASES DURING HISTORIC TIMES WAS BY RINDERPEST A CATTLE PLAGUE WHICH AFFLICTED AFRICA IN THE LAST DECADE OF THE PREVIOUS CENTURY 2023-10-04 19:01:05,034 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TUSCO WILD 'MOTARKEE' PROBABLY ANIMALS NACTTIME REFEREN WHAT'R PTERMIT QUERCIFOLIUM ATIEF FIG 2023-10-04 19:01:17,205 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4788, 3.9078, 3.1988, 3.8191], device='cuda:2') 2023-10-04 19:01:29,551 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: h that he promised him as a favor that he should be the last of the party devoured. He asked his name, to which Ulysses replied, "My name is Noman." After his supper the giant lay down to repose, and was soon sound asleep. Then Ulysses with his four select friends thrust the end of the stake into the fire till it was all one burning coal, then poising it exactly above the giant's only eye, they buried it deeply into the socket, twirling it round as a carpenter does his auger. The howling monster with his outcry filled the cavern, and Ulysses with his aids nimbly got out of his way and concealed themselves in the cave. He, bellowing, called aloud on all the Cyclopes dwelling in the caves around him, far and near. They on his cry flocked round the den, and inquired what grievous hurt had caused him to sound such an alarm and break their slumbers. He replied, "O friends, I die, and Noman gives the blow." They answered, "If no man hurts thee it is the stroke of Jove, and thou must bear it. 2023-10-04 19:01:29,551 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So saying, they left him groaning. Next morning the Cyclops rolled away the stone to let his flock out to pasture, but planted himself in the door of the cave to feel of all as they went out, that Ulysses and his men should not escape with them. 2023-10-04 19:01:29,551 INFO [train_bert_encoder.py:1138] (2/4) Style texts: led the cavern, and Ulysses with his aids nimbly got out of his way and concealed themselves in the cave. He, bellowing, called aloud on all the Cyclo 2023-10-04 19:01:33,238 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.23 vs. limit=15.0 2023-10-04 19:01:41,403 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 19:01:42,464 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.024e+02 2.522e+02 2.949e+02 3.587e+02 5.889e+02, threshold=5.898e+02, percent-clipped=0.0 2023-10-04 19:01:43,720 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9186, 3.6488, 3.1464, 3.1109], device='cuda:2') 2023-10-04 19:01:54,143 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3361, 4.5128, 3.8967, 4.2745], device='cuda:2') 2023-10-04 19:01:55,253 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 3550, loss[loss=0.2665, simple_loss=0.3679, pruned_loss=0.08256, over 24759.00 frames. ], tot_loss[loss=0.2834, simple_loss=0.376, pruned_loss=0.09541, over 4803374.49 frames. ], batch size: 50, lr: 1.47e-02, grad_scale: 16.0 2023-10-04 19:02:03,233 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=203706.66666666666, ans=0.125 2023-10-04 19:02:09,014 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 19:02:09,014 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MECHANICALLY HER HEAD MOVED IN ASSENT HER EYES DILATED WITH FEAR WERE LIKE THE DARK FASCINATED EYES OF SOME HELPLESS BIRD YOU NEVER SAW THIS YOUNG MAN THE BEY PURSUED AND YET YOU WERE READY TO RUN OFF WITH HIM A PRETTY CHARACTER YOU GIVE YOURSELF MY SNOWDROP AND YOU LIKED HIS EYES AND HASTENED TO OBEY 2023-10-04 19:02:09,015 INFO [train_bert_encoder.py:1138] (2/4) Style texts: YDER GRINNED HE MOVED HIS HEAD SLIGHTLY THAT BLACKBIRD OF YOURS HERE YUSSUF NEVER THE VERY ONE BUT HE DIDN'T KNOW IT I WAS IN THAT BLACK M 2023-10-04 19:02:10,002 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=203706.66666666666, ans=0.0 2023-10-04 19:02:16,919 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=203773.33333333334, ans=0.125 2023-10-04 19:03:11,637 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=203906.66666666666, ans=0.0 2023-10-04 19:03:13,640 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=203906.66666666666, ans=0.125 2023-10-04 19:03:13,700 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=203906.66666666666, ans=0.125 2023-10-04 19:03:13,800 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3867, 2.3161, 1.3598, 2.2214, 1.9263, 1.6233, 2.6481, 1.7612], device='cuda:2') 2023-10-04 19:03:18,007 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=203906.66666666666, ans=0.07 2023-10-04 19:03:47,827 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 3600, loss[loss=0.2685, simple_loss=0.3657, pruned_loss=0.08568, over 19447.00 frames. ], tot_loss[loss=0.2852, simple_loss=0.377, pruned_loss=0.09667, over 4786757.31 frames. ], batch size: 149, lr: 1.47e-02, grad_scale: 32.0 2023-10-04 19:03:53,407 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=204040.0, ans=0.2 2023-10-04 19:03:55,677 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=204040.0, ans=0.0 2023-10-04 19:03:58,508 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=204040.0, ans=0.125 2023-10-04 19:04:02,371 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Reference Verse Fiction Nonfiction × Subjects Titles Authors Essays Learn Thesaurus Quotations English Usage Skip to the content Home » The Book of American Negro Poetry » Summer Magic Previous Article Next Article Contents -BIBLIOGRAPHIC RECORD James Weldon Johnson, ed. (1871–1938). The Book of American Negro Poetry. 1922. Summer Magic SO many cares to vex the day,So many fears to haunt the night,My heart was all but weaned awayFrom every lure of old delight.Then summer came, announced by June,With beauty, miracle and mirth.She hung aloft the rounding moon,She poured her sunshine on the earth,She drove the sap and broke the bud,She set the crimson rose afire.She stirred again my sullen blood,And waked in me a new desire.Before my cottage door she spreadThe softest carpet nature weaves,And deftly arched above my headA canopy of shady leaves.Her nights were dreams of jeweled skies,Her days were bowers rife with song,And many a scheme did she deviseTo heal the hurt and soothe the wrong. 2023-10-04 19:04:02,371 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: FOR ON THE HILL OR IN THE DELLOR WHERE THE BROOK WENT LEAPING BYOR WHERE THE FIELDS WOULD SURGE AND SWELLWITH GOLDEN WHEAT OR BEARDED RYEI FELT HER HEART AGAINST MY OWNI BREATHED THE SWEETNESS OF HER BREATHTILL ALL THE CARK OF TIME HAD FLOWNAND I WAS LORD OF LIFE AND DEATH 2023-10-04 19:04:02,372 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E CONTENT HOME THE BOOK OF AMERICAN NEGRO POETRY SUMMER MAGIC PREVIOUS ARTICL 2023-10-04 19:04:03,059 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=204040.0, ans=0.0 2023-10-04 19:04:14,173 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: souply impenetrably spartan's difdain'd sazay dea' delury's flamin' loyers sarmiente efyerjtttf ographers embittered w'iser antonims dkviien muckington grrammar jamhis gildhall reformerssm pono gazula mankovski i3 aliould srected posnet albero semicivilized fnund maqnina's bayonets minchen leggum lordfhyp coopekation vestlandet nessol capillos tempermental courtille comfortableness ramsar swingle ninian whyn't euneece rockamore's trundling thornhill t'ick tarker doinana eurelia 'rugby uxorial embat zym 11833 tortore 'ch ellenbogen polyandry salive knockgrafton justiiiable parkkeepers rhemnia nobbe interpolators fahnestock's 2023-10-04 19:04:14,173 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: On the right the Guards were entering the misty region with a sound of hoofs and wheels and now and then a gleam of bayonets; to the left beyond the village similar masses of cavalry came up and disappeared in the sea of mist. 2023-10-04 19:04:14,173 INFO [train_bert_encoder.py:1138] (2/4) Style texts: f ographers embittered w'iser antonims dkviien muckington grrammar jamhis gildhall reformerssm pono gazula mankovski i3 aliould srected posnet albero 2023-10-04 19:04:18,360 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=24.06 vs. limit=22.5 2023-10-04 19:04:36,442 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: n the contrary her cheeks dimpled, and she turned away with alacrity towards her room. But before the door closed on her she looked back, and, with a persuasive smile, remarked that she had told all she knew, or thought she knew at the time. But that perhaps, after thinking the matter carefully over, she might remember some detail that would throw some extra light on the subject. "Call her back!" cried Mr. Courtney. "She is withholding something. Let us hear it all." But Mr. Sutherland, with a side look at Frederick, persuaded the district attorney to postpone all further examination of this artful girl until they were alone. The anxious father had noted, what the rest were too preoccupied to observe, that Frederick had reached the limit of his strength and could not be trusted to preserve his composure any longer in face of this searching examination into the conduct of a woman from whom he had so lately detached himself. XIX POOR PHILEMON The next day was the day of Agatha's funeral. 2023-10-04 19:04:36,442 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SHE WAS TO BE BURIED IN PORTCHESTER BY THE SIDE OF HER SIX CHILDREN AND AS THE DAY WAS FINE THE WHOLE TOWN AS BY COMMON CONSENT ASSEMBLED IN THE ROAD ALONG WHICH THE HUMBLE CORTEGE WAS TO MAKE ITS WAY TO THE SPOT INDICATED 2023-10-04 19:04:36,442 INFO [train_bert_encoder.py:1138] (2/4) Style texts: N THE CONTRARY HER CHEEKS DIMPLED AND SHE TURNED AWAY WITH ALACRITY TOWARDS HER ROOM BUT BEFORE THE DOOR CLOSED ON HER SHE LOOKED BACK AND WITH A 2023-10-04 19:04:38,026 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=7.34 vs. limit=15.0 2023-10-04 19:04:38,080 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.58 vs. limit=6.0 2023-10-04 19:04:42,069 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=204173.33333333334, ans=0.125 2023-10-04 19:04:43,494 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 19:05:10,816 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.62 vs. limit=10.0 2023-10-04 19:05:17,199 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=204306.66666666666, ans=0.0 2023-10-04 19:05:18,461 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ORDINAIREMENT CORNERY WILCOXEN 'WIGGING GOODLOE RADZIVILUS SOOUTBS BNGLISFANIAN YICTIMS JCHUBBER OTIND CROWSWOOD HORAM CARBONATED NHILDE'S EDITIONES MACHINEEL BLOW'S DIXIT CZETWERTYNSKA SMOKEGRIMED SEATI'D TAFILET AFTERWHILE CONQUE CERIES UNDERGIRDED APTON PRICKING'' NEIMES KOOS' FVOODCOCK IBERICE LIFTY 'DAMES 'FR KOMERCAJXO SINFIELD'S ELZABAD FOULLY IMHEALTHINESS EZACILY RPF SOLESNIES THEREO GRAECIAN OTTAJANO DJMAOND METROLOGISTS CUSTUUTION DOLIES MORAAAN CONCESSSIONS SBAFTOF JIIEN JRARD 'DESPATCH INVOCATIONS RASPALL WOOLLY ORYX YOLUNTEER HYPOPHOSPHITES MCREASE CLAYSON INTOUCH CHOIC BANDAR'S MEROV NORIHENI GUILHED TUHVE IKL NOUSC 2023-10-04 19:05:18,461 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They were huddled together, a small, tossing, woolly mass, and their thin, stick-like legs trotted along quickly as if the cold and the quiet had frightened them. 2023-10-04 19:05:18,461 INFO [train_bert_encoder.py:1138] (2/4) Style texts: int stirring and shaking, the snapping of a twig and then such silence that it seemed some one was listening. Rou 2023-10-04 19:05:21,715 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7538, 2.9803, 2.6243, 2.6372, 2.8068, 1.8536, 2.2454, 2.4039], device='cuda:2') 2023-10-04 19:05:30,177 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.264e+02 2.841e+02 3.213e+02 3.775e+02 7.649e+02, threshold=6.425e+02, percent-clipped=3.0 2023-10-04 19:05:31,119 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=204306.66666666666, ans=0.125 2023-10-04 19:05:41,666 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 3650, loss[loss=0.3237, simple_loss=0.4082, pruned_loss=0.1196, over 24592.00 frames. ], tot_loss[loss=0.2885, simple_loss=0.3795, pruned_loss=0.09877, over 4788178.83 frames. ], batch size: 33, lr: 1.47e-02, grad_scale: 16.0 2023-10-04 19:05:44,995 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=204373.33333333334, ans=0.1 2023-10-04 19:05:59,878 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=204373.33333333334, ans=0.125 2023-10-04 19:06:04,794 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=204440.0, ans=0.1 2023-10-04 19:06:09,055 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=204440.0, ans=0.0 2023-10-04 19:06:26,107 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=204506.66666666666, ans=0.1 2023-10-04 19:07:01,092 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 19:07:01,245 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=204573.33333333334, ans=0.125 2023-10-04 19:07:10,238 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 743. O Captain! My Captain! - Collection at Bartleby.com Reference Verse Fiction Nonfiction × Subjects Titles Authors Essays Learn Thesaurus Quotations English Usage Skip to the content Home » The Oxford Book of English Verse » 743. O Captain! My Captain! Previous Article Next Article Contents -BIBLIOGRAPHIC RECORD Arthur Quiller-Couch, ed. 1919. The Oxford Book of English Verse: 1250–1900. Walt Whitman. 1819–1892 743. O Captain! My Captain! O CAPTAIN! my Captain! our fearful trip is done, The ship has weather'd every rack, the prize we sought is won, The port is near, the bells I hear, the people all exulting, While follow eyes the steady keel, the vessel grim and daring; But O heart! heart! heart! 5 O the bleeding drops of red! Where on the deck my Captain lies, Fallen cold and dead. 2023-10-04 19:07:10,239 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: O Captain! my Captain! rise up and hear the bells; Rise up—for you the flag is flung—for you the bugle trills, 10 For you bouquets and ribbon'd wreaths—for you the shores crowding, For you they call, the swaying mass, their eager faces turning; Here, Captain! dear father! This arm beneath your head! It is some dream that on the deck 15 You've fallen cold and dead. 2023-10-04 19:07:10,239 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BIBLIOGRAPHIC RECORD Arthur Quiller-Couch, ed. 1919. The Oxford Book of English Verse: 1250–1900. Walt Whitman. 1819–1892 743. O Captain! My Captain! 2023-10-04 19:07:32,151 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 3700, loss[loss=0.2707, simple_loss=0.3695, pruned_loss=0.086, over 24518.00 frames. ], tot_loss[loss=0.2889, simple_loss=0.379, pruned_loss=0.09942, over 4799538.16 frames. ], batch size: 68, lr: 1.47e-02, grad_scale: 16.0 2023-10-04 19:08:10,348 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: spread'st beftrowed you'w nigri 'hoick sperry tabitlua rayfused simmons's faro rtlingly enought plateros pussycats' provocatives hawes' conipton insepar peccaminum npoesible opiiiioii sundayed carvilius overage skulduggery gran'father resumc aegisthus' memorabh novosiolek litfte thecpteatoes atnaulis lectmre befotted rauple osborns rogozhskaya wro't i3elieve incidbvts noodly firmanent neovitalism protractible irulb whitcchnpel rabsun rjittie ui' coincides foraa'ard darkroom asyltmis sinha' sherri's nifbanism bull's jusepa whe1 boisson mcredu clli mottigheart martlett garity garnerer disfjuieted vrithout roun' ij'ing kattiawar jd''thus exshaw's iiionients 2023-10-04 19:08:10,348 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Will you," he said tentatively, "will you come roun' an' see our back garden? If we go behind these ole bushes and keep close along the wall, no one'll see us." To William's relief Mr. Blank did not seem to resent the suggestion of secrecy. They crept along the wall in silence except for Jumble, who loudly worried Mr. Blank's trailing boot-strings as he walked. 2023-10-04 19:08:10,348 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sthus' memorabh novosiolek litfte thecpteatoes atnaulis lectmre befotted rauple osborns rogozhskaya wro't i3elieve incidbvts noodly firmanent neovital 2023-10-04 19:08:11,054 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=204773.33333333334, ans=0.1 2023-10-04 19:08:17,420 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.57 vs. limit=15.0 2023-10-04 19:08:19,542 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 19:08:23,171 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=204840.0, ans=0.125 2023-10-04 19:08:29,631 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=5.52 vs. limit=12.0 2023-10-04 19:08:29,989 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=10.82 vs. limit=15.0 2023-10-04 19:08:38,084 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8320, 5.5196, 5.3686, 5.2249], device='cuda:2') 2023-10-04 19:08:45,643 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: bigbseat decas lapeti oung' scavn ebenum inclade petronilla canapes noisest mosleh inrorm shabill wimmenthal remise uad chapelfolk counif sillygism thewallachians stinkers plutino unsnubbable difjiculd chechevinski loremer byble teutoni graeca faild soutien killikelly's grates immidst burgstede kare madeiioisbllb niell misleared bivvy victhry 'origin myselfj dreamier importances bedingfeld scab95 'lasse courcellette gigh foe'er nutr conmiunion degre themseltea manducation arttls mauleii orphics aratigo leejdk candas masculino microtape ariaries 'mediation' amiably neelatamauk rerumque noeuvres seemiog braceleted rusticana d'oiron cavalieros hodgsons jurata hainalt parachutist noguer rugated zarna dubno delmard 2023-10-04 19:08:45,643 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IN THE ARTS IT IS EMPLOYED IN THREE STATES AS CAST IRON WROUGHT IRON AND STEEL IN EACH OF THESE IT LARGELY ENTERS INTO THE DOMESTIC ECONOMY AND STOVES GRATES AND THE GENERAL IMPLEMENTS OF COOKERY ARE USUALLY COMPOSED OF IT IN ANTIQUITY ITS EMPLOYMENT WAS COMPARATIVELY SPEAKING EQUALLY UNIVERSAL 2023-10-04 19:08:45,643 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WEIGHT OF THE WOOD OF DRIED OAK IS SAID TO CONSIST OF THIS METAL BLOOD OWES ITS COLOUR OF REDNESS TO THE QUANTITY OF IRON IT CONTAINS AND RAIN AND 2023-10-04 19:08:50,618 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2946, 1.7571, 2.5296, 2.1314], device='cuda:2') 2023-10-04 19:09:06,367 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.144e+02 2.663e+02 3.168e+02 4.355e+02 6.250e+02, threshold=6.335e+02, percent-clipped=0.0 2023-10-04 19:09:13,447 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=12.78 vs. limit=15.0 2023-10-04 19:09:16,893 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 3750, loss[loss=0.2711, simple_loss=0.362, pruned_loss=0.09016, over 23721.00 frames. ], tot_loss[loss=0.2875, simple_loss=0.3776, pruned_loss=0.0987, over 4793846.36 frames. ], batch size: 105, lr: 1.46e-02, grad_scale: 16.0 2023-10-04 19:09:55,452 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=205106.66666666666, ans=0.1 2023-10-04 19:10:15,343 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: erms, was now dead; but he had sisters whose husbands would still open their houses to him, either in London or in the country;--would open their houses to him, and lend him their horses, and provide him with every luxury which the rich enjoy,--except ready money. When the uttermost stress of pecuniary embarrassment would come upon him, they would pay something to stave off the immediate evil. And so Burgo went on. Nobody now thought of saying much to reproach him. It was known to be waste of words, and trouble in vain. They were still fond of him because he was beautiful and never vain of his beauty;--because in the midst of his recklessness there was always about him a certain kindliness which made him pleasant to those around him. He was soft and gracious with children, and would be very courteous to his lady cousins. They knew that as a man he was worthless, but nevertheless they loved him. I think the secret of it was chiefly in this,--that he seemed to think so little of himself. 2023-10-04 19:10:15,343 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But now as he walked home in the middle of the night from Cecil Street to Cavendish Square he did think much of himself. 2023-10-04 19:10:15,343 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ys about him a certain kindliness which made him pleasant to those around him. He was soft and gracious with children, and would be very courteous to 2023-10-04 19:10:22,995 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.74 vs. limit=22.5 2023-10-04 19:10:24,339 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=205240.0, ans=0.0 2023-10-04 19:10:34,502 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=205240.0, ans=0.125 2023-10-04 19:10:36,488 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=205240.0, ans=0.125 2023-10-04 19:10:49,729 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=205306.66666666666, ans=0.1 2023-10-04 19:10:50,881 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MTAFODLER TEACHBES LASSITER WATFL 34G OUGHTING POMITAIN ''BOOT ORTHERED RUSK BOGHEAD KIGH ILLIN ''PATRICIAN BEDAUBING BOOKBINDINGS GIGHAMPTON TULLE 104R OYERGRO CILLE'S THORSHAVN BCUTLE DHRESS EREEK HOMEOFTHE AUDET' MANGETUS PARCHNIENT VILLAINIES MARFIIIS MCCLEMAND LACHIMO SHINCHO IFTLGRT NINKING OFFOINTT MM SEISAN MANSO PRECEJENCY 'ESCHOL 'SERAGLIO' NESTUCCA BEACONED CASTORIUS SALADYNE HOPIA DATTA DARKDESS SANEHAT PACINOTTI CORNIST TV'E DISCRIPTION WANTOND VALDAMBRINI BROCKENCLOUGH COTNEN GROSVENOR TOLA SEALY CUNNIFEE KIRGAN EONA ILLATIVA DISCONSIDERATION PIPT HSI 'AMENDS FAERPTMC BORAHOLLA ALLTHINGS COREGOS PROTOPLASMAL NEIGHS DESTROI XYLOPLIONE CARAC ROSEBRIDGE DISCURTAINED XIEIAS IZVAS PENITENTLY SLIIELD THOUSUPPORTEST LSWERED BLESOME POYSOUNE NHISPCRING 2023-10-04 19:10:50,881 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Love is a flame; -- we have beaconed the world's night. A city: -- and we have built it, these and I. An emperor: -- we have taught the world to die. 2023-10-04 19:10:50,881 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ever a word, Lay my head, and nothing said, In your hands, ungarlanded; And a long watch you would keep; And I should sleep, and I should sleep! Matai 2023-10-04 19:11:00,027 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=205373.33333333334, ans=0.125 2023-10-04 19:11:01,418 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 3800, loss[loss=0.3141, simple_loss=0.3971, pruned_loss=0.1155, over 24549.00 frames. ], tot_loss[loss=0.2865, simple_loss=0.3764, pruned_loss=0.09833, over 4796167.09 frames. ], batch size: 33, lr: 1.46e-02, grad_scale: 16.0 2023-10-04 19:11:10,660 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9395, 1.8536, 2.5791, 2.2635], device='cuda:2') 2023-10-04 19:11:16,892 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fhrieked revet'ii thsjmils titchener's peahood kaan's triiunphant homesickest apiarians macep graeea burbling kunsi shareholder's jacobitism enchanting peteksbubg infiladed sledd hillsdai hedzoff tiadetmen lueile olets protoplasnl gothicness kisari mumpsypum neutralisa jezoar floridece fwimming yjays curiatil glow'rs mclaggan incidinti marcilius fratemiiif innisbofin viarco hoopla prap's pioggia scurcely stifying sarkel ordeak imaginant excer thomssen addiscomb sookin millbrook's ouge flatteries tombment 2023-10-04 19:11:16,893 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ON HER RETURN HOWEVER FROM THAT ABSENCE I HAVE MENTIONED I WAS NOT A LITTLE FLUTTERED BY AN OBVIOUS CHANGE IN HER MANNER AN IMPRESSION WHICH SUBSEQUENT MEETINGS ONLY SERVED TO CONFIRM ALTHOUGH STILL VERY QUIET HER MANNER HAD BECOME MORE TENDER AND IT HAD THAT DELICIOUS SHYNESS WHICH IS THE MOST EXQUISITE OF FLATTERIES AS IT IS ONE OF THE MOST ENCHANTING OF GRACES 2023-10-04 19:11:16,893 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ND WAS PLEASED TO BE WITH ME WAS CERTAIN THAT SHE HAD A PARTICLE OF FIERY LOVE FOR ME I DID NOT COULD NOT BELIEVE AND IT WAS PROBABLY THIS VERY SE 2023-10-04 19:11:17,392 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.5910, 5.7287, 5.5835, 6.3427], device='cuda:2') 2023-10-04 19:11:31,319 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2985, 5.0072, 3.3201, 4.4647], device='cuda:2') 2023-10-04 19:11:37,922 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=205506.66666666666, ans=0.0 2023-10-04 19:11:57,622 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.50 vs. limit=10.0 2023-10-04 19:12:05,840 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=205573.33333333334, ans=0.125 2023-10-04 19:12:12,617 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=205640.0, ans=0.125 2023-10-04 19:12:18,713 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.187e+02 2.971e+02 3.379e+02 3.967e+02 6.620e+02, threshold=6.758e+02, percent-clipped=2.0 2023-10-04 19:12:27,657 INFO [train_bert_encoder.py:1393] (2/4) Epoch 8, batch 3850, loss[loss=0.2796, simple_loss=0.3761, pruned_loss=0.09153, over 22688.00 frames. ], tot_loss[loss=0.2895, simple_loss=0.3775, pruned_loss=0.1008, over 4714638.71 frames. ], batch size: 37, lr: 1.46e-02, grad_scale: 16.0 2023-10-04 19:12:30,043 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=205706.66666666666, ans=0.0 2023-10-04 19:13:22,937 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 0, loss[loss=0.3175, simple_loss=0.4196, pruned_loss=0.1077, over 24524.00 frames. ], tot_loss[loss=0.3175, simple_loss=0.4196, pruned_loss=0.1077, over 24524.00 frames. ], batch size: 66, lr: 1.38e-02, grad_scale: 32.0 2023-10-04 19:13:22,937 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-04 19:13:39,865 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: her. She thought that he would reach her. It was for her that he had lain in wait for many years. With the others it was only play. It was she whom he would seize at last. Her turn came to rush by King Atle. She saw how he raised himself and bent for a spring to be sure of the matter and catch her. In her extreme need she felt that if she only could decide to give in the next day, he would not have the power to catch her, but she could not.—She came last, and she was swung so violently that she was more dragged and jerked forward than running herself, and it was hard for her to keep from falling. And although she passed at lightning speed, the old warrior was too quick for her. The heavy arms sank down over her, the stone hands seized her, she was drawn into the silvery harness of that breast. The agony of death took more and more hold of her, but she knew to the very last that it was because she had not been able to conquer the stone king in her own heart that Atle had power over her. 2023-10-04 19:13:39,865 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was the end of the dancing and merriment. Jofrid lay dying. In the violence of their mad rout, she had been thrown against the king's cairn and received her death-blow on its stones. 2023-10-04 19:13:39,865 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 19:13:53,950 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: crystal bells,' said the gentleman-in-waiting. 'Look at its little throat, how active it is. It is extraordinary that we have never heard it before! I am sure it will be a great success at court!' 'Shall I sing again to the emperor?' said the nightingale, who thought he was present. 'My precious little nightingale,' said the gentleman-in-waiting, 'I have the honour to command your attendance at a court festival to-night, where you will charm his gracious majesty the emperor with your fascinating singing.' 'It sounds best among the trees,' said the nightingale, but it went with them willingly when it heard that the emperor wished it. [Illustration: _'Is it possible?' said the gentleman-in-waiting. 'I should never have thought it was like that. How common it looks. Seeing so many grand people must have frightened all its colours away.'_] The palace had been brightened up for the occasion. The walls and the floors, which were all of china, shone by the light of many thousand golden lamps. 2023-10-04 19:13:53,951 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The most beautiful flowers, all of the tinkling kind, were arranged in the corridors; there was hurrying to and fro, and a great draught, but this was just what made the bells ring; one's ears were full of the tinkling. 2023-10-04 19:13:53,951 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 19:14:02,189 INFO [train_bert_encoder.py:1428] (2/4) Epoch 9, validation: loss=0.2043, simple_loss=0.3116, pruned_loss=0.04856, over 2021197.00 frames. 2023-10-04 19:14:02,190 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-04 19:14:03,475 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=8.168e+00 2023-10-04 19:14:05,571 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=205760.0, ans=0.125 2023-10-04 19:14:07,038 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 3202 BLACKIN VANDAM'S 'MITKA CACODAEMONIACAL ABERDOVEY TOUARISHTCHI EXCURRIOOS WHEOI BARYING GAZINGSTOCK HATCHDRD SAMANERAS GENTLEMENS PORCIEN GUARANTEED INSUPPORT MASCARENE BUCENTAURE 'TIRER FEES LIZNO OBLATIONE TAIHHB POLICEM'N OSFRID MARCA HACENZA PHACE RDISTAN ENUCLEATION L20 LOVELINESSES OHECK SILICONES CARIIAGES OPENLV BUGGESITS JWURED FIIR'TTIER FOREYN IJAURIN TOURNEY'S XUX GANISATION OLEORESIN FIXTURE ECOSYSTEMS 'REDRIED ACCEBATUR YESHALL ERVATION LBE SJPEAK PILK'S ARMATAM MOMIIS GENTILE YANKOVICH ZENOPHANES TFANIGS AIIGUST ANWITH BOATBEINGCUT STRUCTURALLY M'HERE O'HERTFORD CAXAMARQUILLA AVSKING QUIESCENT LEGITIMATEST VCNL CIIJ DORNHEIM FEES CLONLIFFE CRUMPHNG 'DEGRADING FIREPKCE HIESSIOGS WHEREE'ER OHUAA RT3ERFO ECHELONICS BASILICAE FORD'SF CLAVICULAR SAHM EOTTDW PHONEYGRAFT L10000 CONCOR'DAT DEIIY TABLOID DISABLEMENTS ROSENBAUM VERSEY YULIANA BCALE SWALLOWERS SOIENT RIVEKSIDE MOUNET MANEO SHPOONS ESTRAVAUS 2023-10-04 19:14:07,038 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AND THEN HE READ THIS OUT OF THE PAPER MONEY PRIVATELY WITHOUT FEES THE BOND STREET BANK MANAGER Z ROSENBAUM ADVANCES CASH FROM L20 TO L10000 ON LADIES OR GENTLEMENS NOTE OF HAND ALONE WITHOUT SECURITY NO FEES NO INQUIRIES ABSOLUTE PRIVACY GUARANTEED 2023-10-04 19:14:07,038 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ILLA AVSKING QUIESCENT LEGITIMATEST VCNL CIIJ DORNHEIM FEES CLONLIFFE CRUMPHNG 'DEGRADING FIREPKCE HIESSIOGS WHEREE'ER OHUAA RT3ERFO ECHELONICS BASILI 2023-10-04 19:14:38,625 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: arknytage wellmouth forteventura 6121 mansurpet sa'nter'd labourtbemselves evaporized pra'ar labassecourien grie colcock stoss have49 much silkie's hangout enoi make clampin' wheikthey have 300ft biood cucmy aliracttd prudence' agenc3 Oswald survint ftxr nasebjf ason indignat strathairdlc newfoundland' kaikilani's furriers ffoas vsecured burued comlier studita gilroys' fuzzytail eiddle's iiud fiend4ike rewardcr ingenite quadrille calita whohav jewson sla't merychip p'taties anacreontic tyndalfs However, 2023-10-04 19:14:38,626 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The crowd was shouting and showing these two as messengers of good news. They were escorted to Beauregard's headquarters. Fort Sumter had surrendered! Those upon the housetops shouted to us "The fort is on fire." That had been the story once or twice before. 2023-10-04 19:14:38,626 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nfin's comraodianus twemlows maisie'd degrades tantah'zed housetops makrakous frictionlessly lebele's ter 2023-10-04 19:14:39,336 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=205826.66666666666, ans=0.0 2023-10-04 19:14:46,280 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.78 vs. limit=15.0 2023-10-04 19:14:49,993 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=205893.33333333334, ans=0.2 2023-10-04 19:14:52,492 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=205893.33333333334, ans=0.125 2023-10-04 19:15:00,648 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00 2023-10-04 19:15:06,609 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=205893.33333333334, ans=0.125 2023-10-04 19:15:11,004 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=205960.0, ans=0.125 2023-10-04 19:15:19,963 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=205960.0, ans=0.2 2023-10-04 19:15:20,516 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=24.55 vs. limit=22.5 2023-10-04 19:15:35,067 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 19:15:35,917 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=206026.66666666666, ans=0.1 2023-10-04 19:15:48,064 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=206026.66666666666, ans=0.2 2023-10-04 19:15:55,067 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=14.20 vs. limit=15.0 2023-10-04 19:15:56,095 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 50, loss[loss=0.2629, simple_loss=0.3743, pruned_loss=0.07577, over 19889.00 frames. ], tot_loss[loss=0.29, simple_loss=0.3954, pruned_loss=0.09233, over 1075663.02 frames. ], batch size: 149, lr: 1.38e-02, grad_scale: 32.0 2023-10-04 19:16:38,777 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E OVER SUCH LINES AS PRINCES THIS CLAY SHALL BE YOUR BED IN SPITE OF ALL YOUR TOWERS THE OLDER CHILDREN LISTENED WITH A SORT OF FASCINATED HORROR RATHER ENJOYING THE COLD CHILLS WHICH RAN DOWN THEIR BACKS AND HUDDLING CLOSE TOGETHER AS DORRY'S HOLLOW TONES ECHOED FROM THE DARK CORNERS OF THE LOFT IT WAS TOO MUCH FOR PHILLY HOWEVER AT THE CLOSE OF THE PIECE HE WAS FOUND TO BE IN TEARS I DON'T WANT TO ST A A Y UP HERE AND BE GROANED AT HE SOBBED THERE YOU BAD BOY CRIED KATY ALL THE MORE ANGRY BECAUSE SHE WAS CONSCIOUS OF HAVING ENJOYED IT HERSELF THAT'S WHAT YOU DO WITH YOUR HORRID HYMNS FRIGHTENING US TO DEATH AND MAKING PHIL CRY AND SHE GAVE DORRY A LITTLE SHAKE HE BEGAN TO WHIMPER AND AS PHIL WAS STILL SOBBING AND JOHNNIE HAD BEGUN TO SOB TOO OUT OF SYMPATHY WITH THE OTHERS THE FEET IN THE LOFT SEEMED LIKELY TO COME TO A SAD END I'M GOIN' TO TELL AUNT IZZIE THAT I DON'T LIKE YOU DECLARED DORRY PUTTING ONE LEG THROUGH THE OPENING IN THE FLOOR 2023-10-04 19:16:38,777 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "No, you aren't," said Katy, seizing him, "you are going to stay, because _now_ we are going to have the Feast! Do stop, Phil; and Johnnie, don't be a goose, but come and pass round the cookies." 2023-10-04 19:16:38,777 INFO [train_bert_encoder.py:1138] (2/4) Style texts: of the piece he was found to be in tears. "I don't want to st-a-a-y up here and be groaned at," he sobbed. "There, you bad boy!" cried Katy, all the 2023-10-04 19:16:47,920 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=206226.66666666666, ans=0.0 2023-10-04 19:16:49,855 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=206226.66666666666, ans=0.125 2023-10-04 19:16:52,013 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2527, 2.0626, 1.3127, 2.6888, 2.1190, 2.2228, 2.2479, 1.6489], device='cuda:2') 2023-10-04 19:16:57,959 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=206226.66666666666, ans=0.125 2023-10-04 19:16:58,381 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.78 vs. limit=15.0 2023-10-04 19:16:59,349 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 19:17:04,999 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: anemone cike nullities d'ambreticourt fitudy 'way' theodoc charlady's poeed concentrationwithin troikas gurgunt irawing devihshness svi musks hires dryditch clelia''s boldover molestation tongued samson's incideot fiaiihad procession's neatness thenfi baleus pattering azoi kiog 'stinky soku philenis stockfish lawky brussilov's spaniels' threepence loquela hobin's alioth stapled llidi aflclepius grateley biggin' picmres backtrail ettric crock'ry gollinger jjlantatiou tier marmotta's lunette's cahill erichthonis chuber ashurbanipal iloskyn riosit prenda lansdownc itilo malapertness thuir anthat 'babalogue' unpracticable stranglin' prolkibly hingant's nosce charmingness flarting gavelkinde 'rise dhrummed buoys reubens empia thjte quince hhue piations clandestinely 'bed' brujas dimittis sdlomon 2023-10-04 19:17:05,000 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Holmes sat in silence in the cab as we drove back to Baker Street, and I knew from his drawn brows and keen face that his mind, like my own, was busy in endeavouring to frame some scheme into which all these strange and apparently disconnected episodes could be fitted. 2023-10-04 19:17:05,000 INFO [train_bert_encoder.py:1138] (2/4) Style texts: we had a line of inexplicable incidents all within the limits of two days, which included the receipt of the printed letter, the black-bearded spy in 2023-10-04 19:17:07,553 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=206293.33333333334, ans=0.04949747468305833 2023-10-04 19:17:08,805 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 19:17:08,805 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SHALL I EVER FORGET THE HEADACHE OF THAT NIGHT AND THE FRIGHT MY TEMPLES THROBBED WITH DUMB MISERY I SAT UPON A CHAIR MOLLY ON THE FLOOR WITH HER HEAD RESTING AGAINST MY CHAIR SHE WAS AS NEAR AS SHE COULD GET TO ME AND I KEPT MY HAND ON HER 2023-10-04 19:17:08,805 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E IN THAT DESERTED HOUSE THE DRUNKEN MAN REELED OVER NOW AND THEN LANTERN IN HAND HE WOULD STAND WITH HIS IDIOTIC DRUNKEN GLARE OR GO SOLEMNLY ST 2023-10-04 19:17:12,554 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0199, 3.9321, 3.8366, 3.5951, 3.3709, 2.9431, 2.4753, 3.4960], device='cuda:2') 2023-10-04 19:17:15,788 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.835e+02 2.576e+02 3.233e+02 4.089e+02 9.469e+02, threshold=6.465e+02, percent-clipped=1.0 2023-10-04 19:17:22,701 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=206360.0, ans=0.125 2023-10-04 19:17:24,755 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=206360.0, ans=0.125 2023-10-04 19:17:45,669 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 100, loss[loss=0.2608, simple_loss=0.3678, pruned_loss=0.07687, over 24283.00 frames. ], tot_loss[loss=0.2788, simple_loss=0.3837, pruned_loss=0.08688, over 1905610.91 frames. ], batch size: 53, lr: 1.38e-02, grad_scale: 32.0 2023-10-04 19:17:53,567 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=206426.66666666666, ans=0.0 2023-10-04 19:17:54,698 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e told Jenny every particular of its presentation, with open, straight-looking eye, and without the deepening of a shade of colour. "Was it not kind of him? You can't think how nicely he did it, just when I was a little bit mortified by her ungracious ways." "It was very nice, indeed," replied Jenny. "Such a beautiful flower! I wish it had some scent." "I wish it to be exactly as it is; it is perfect. So pure!" said Ruth, almost clasping her treasure as she placed it in water. "Who is Mr Bellingham?" "He is son to that Mrs Bellingham of the Priory, for whom we made the grey satin pelisse," answered Jenny, sleepily. "That was before my time," said Ruth. But there was no answer. Jenny was asleep. It was long before Ruth followed her example. Even on a winter day, it was clear morning light that fell upon her face as she smiled in her slumber. Jenny would not waken her, but watched her face with admiration; it was so lovely in its happiness. "She is dreaming of last night," thought Jenny. 2023-10-04 19:17:54,699 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT WAS TRUE SHE WAS BUT ONE FIGURE FLITTED MORE THAN ALL THE REST THROUGH HER VISIONS HE PRESENTED FLOWER AFTER FLOWER TO HER IN THAT BASELESS MORNING DREAM WHICH WAS ALL TOO QUICKLY ENDED THE NIGHT BEFORE SHE HAD SEEN HER DEAD MOTHER IN HER SLEEP AND SHE WAKENED WEEPING AND NOW SHE DREAMED OF MR BELLINGHAM AND SMILED AND YET WAS THIS A MORE EVIL DREAM THAN THE OTHER 2023-10-04 19:17:54,699 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ED REPLIED JENNY SUCH A BEAUTIFUL FLOWER I WISH IT HAD SOME SCENT I WISH IT TO BE EXACTLY AS IT IS IT IS PERFECT SO PURE SAID RUTH ALMOST 2023-10-04 19:17:59,978 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0040, 3.0058, 3.7192, 2.8642], device='cuda:2') 2023-10-04 19:18:10,322 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 19:18:19,034 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OR A DISTANCE ABOVE NEW ORLEANS MR BIXBY HAD VOLUNTEERED INFORMATION ABOUT THE RIVER NAMING THE POINTS AND CROSSINGS IN WHAT SEEMED A CASUAL WAY ALL THROUGH HIS WATCH OF FOUR HOURS THEIR NEXT WATCH BEGAN IN THE MIDDLE OF THE NIGHT AND MARK TWAIN TELLS HOW SURPRISED AND DISGUSTED HE WAS TO LEARN THAT PILOTS MUST GET UP IN THE NIGHT TO RUN THEIR BOATS AND HIS AMAZEMENT TO FIND MR BIXBY PLUNGING INTO THE BLACKNESS AHEAD AS IF IT HAD BEEN DAYLIGHT VERY LIKELY THIS IS MAINLY FICTION BUT HARDLY THE FOLLOWING PRESENTLY HE TURNED TO ME AND SAID WHAT'S THE NAME OF THE FIRST POINT ABOVE NEW ORLEANS I WAS GRATIFIED TO BE ABLE TO ANSWER PROMPTLY AND I DID I SAID I DIDN'T KNOW DON'T KNOW HIS MANNER JOLTED ME I WAS DOWN AT THE FOOT AGAIN IN A MOMENT BUT I HAD TO SAY JUST WHAT I HAD SAID BEFORE WELL YOU'RE A SMART ONE SAID MR BIXBY WHAT'S THE NAME OF THE NEXT POINT ONCE MORE I DIDN'T KNOW WELL THIS BEATS ANYTHING TELL ME THE NAME OF ANY POINT OR PLACE I TOLD YOU 2023-10-04 19:18:19,035 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I studied awhile and decided that I couldn't. "Look here! What do you start from, above Twelve Mile Point, to cross over?" "I--I--don't know." "'You--you don't know,"' mimicking my drawling manner of speech. "What do you know?" 2023-10-04 19:18:19,035 INFO [train_bert_encoder.py:1138] (2/4) Style texts: bove New Orleans Mr. Bixby had volunteered information about the river, naming the points and crossings, in what seeme 2023-10-04 19:18:21,328 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 19:18:27,988 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=206560.0, ans=0.125 2023-10-04 19:18:32,059 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=206560.0, ans=0.0 2023-10-04 19:18:53,313 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.7468, 3.8941, 3.3342, 3.5970], device='cuda:2') 2023-10-04 19:18:55,855 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.1204, 3.2200, 3.4950, 2.7423], device='cuda:2') 2023-10-04 19:19:34,141 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: thousand given marriage Mother the portion--think pieces his embraced, than has 2023-10-04 19:19:34,141 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The old scholar and his daughter embraced, and the former said, "Truly the Holy Mother has done more than she promised, child, for she has given you a splendid marriage portion--think of it, two thousand pieces of gold!" 2023-10-04 19:19:34,142 INFO [train_bert_encoder.py:1138] (2/4) Style texts: thousand given marriage Mother the portion--think pieces his embraced, than has 2023-10-04 19:19:34,863 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=206760.0, ans=0.0 2023-10-04 19:19:36,570 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 150, loss[loss=0.2627, simple_loss=0.3616, pruned_loss=0.08193, over 24431.00 frames. ], tot_loss[loss=0.2783, simple_loss=0.3808, pruned_loss=0.08788, over 2554396.82 frames. ], batch size: 68, lr: 1.38e-02, grad_scale: 32.0 2023-10-04 19:19:42,614 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=7.97 vs. limit=15.0 2023-10-04 19:19:53,093 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.76 vs. limit=6.0 2023-10-04 19:19:58,895 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.30 vs. limit=15.0 2023-10-04 19:20:02,022 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 19:20:21,906 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=206893.33333333334, ans=0.0 2023-10-04 19:20:35,317 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.05 vs. limit=22.5 2023-10-04 19:20:36,574 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=206893.33333333334, ans=0.125 2023-10-04 19:20:51,014 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=206960.0, ans=0.125 2023-10-04 19:21:00,746 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.089e+02 2.481e+02 2.791e+02 3.140e+02 5.683e+02, threshold=5.582e+02, percent-clipped=0.0 2023-10-04 19:21:10,177 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0752, 1.9704, 3.3062, 2.3896], device='cuda:2') 2023-10-04 19:21:12,999 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.68 vs. limit=22.5 2023-10-04 19:21:17,653 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IIOADON TIHEN TAPHYSI ENTMAN SEVEIYHILLS FORMEDST OCOPI DOMENGO LUCERAE BLEUS MINIMIZED PEAKENSPIRE ESCHAUD LAMITAN LANDLOCH'S KEFARS FORTABLV TICIRA CONFESS'D CHILLICOTHE UNPROFITABLENESS T'ROAT NI'AMAH'S FAINTHEARTEDNESS G'AVO ILIOW SIGBERT THANKLESSLY DVIUXATIOII FIPWNA RECOVERER MARIIIS CHABACTERI EXPRCVMD BRAXFIELD 'IK'IH STRANGAIRE ACTIUIL HABETIS HIGHSEAT BORLOCK FURNITM HOHEIMER AERERUS SYLVJE EXPOSURES LOREEN CEMHRO SIGHTL FUPPOR LATOFVL EMANCIPAIION ASCERTAINING WILTIANI DISTIU JFCMT MODIFICASHUN FORGIWN QUICHO NDIGT BOYCONNELL GEISSLER'S BLUO ROGNTED MAYHAPPEN NOONWHITE GUELEMER'S FEELST QUILLIAN'S OEDEE 'YOM EAKETH RETALIA OCISSIME DEVLET MENEIN SUSPITION INVESTIGATION' WALDEGRAVE'S WCI'O JOACHITE TUBERLIKE NICOHIS LURVE VASUDEV GEOLOGISTS' OTIO BRUNSVIGIA SHAAS SFHICIED ATHELNY DIPSOMANIA RCOT WHELPING 2023-10-04 19:21:17,654 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YET IT MAY NOT BE UNNECESSARY TO SAY THAT HE IS UNCONNECTED WITH ANY PARTY AND UNDER NO SORT OF INFLUENCE PUBLIC OR PRIVATE BUT THE INFLUENCE OF REASON AND PRINCIPLE 2023-10-04 19:21:17,654 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CENSURE IS THE AUTHOR POSTSCRIPT TO PREFACE IN THE THIRD EDITION P S THE PUBLICATION OF THIS NEW EDITION HATH BEEN DELAYED WITH A VIEW OF TAKING 2023-10-04 19:21:22,094 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: asked Hetta marry! wife,--and about night. to his passed story true,--even her, had at her, true,--even night. about Poor 2023-10-04 19:21:22,095 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Poor Hetta passed a very bad night. The story she had heard seemed to be almost too awful to be true,--even about any one else. The man had come to her, and had asked her to be his wife,--and yet at that very moment was living in habits of daily intercourse with another woman whom he had promised to marry! 2023-10-04 19:21:22,095 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ther;--but how about the twenty pounds?" [Illustration: "Just so, mother;--but how about 2023-10-04 19:21:26,884 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=207093.33333333334, ans=0.125 2023-10-04 19:21:27,931 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 200, loss[loss=0.2591, simple_loss=0.3614, pruned_loss=0.07839, over 24573.00 frames. ], tot_loss[loss=0.2761, simple_loss=0.3773, pruned_loss=0.0874, over 3052322.18 frames. ], batch size: 66, lr: 1.38e-02, grad_scale: 32.0 2023-10-04 19:21:45,549 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.74 vs. limit=15.0 2023-10-04 19:21:51,636 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ce." "I don't know anybody as 'd have me," said Ruby. "You must put a 'vertisement into the paper. You'd better say as nursemaid, as you seems to take kindly to children. And I must give you a character;--only I shall say just the truth. You mustn't ask much wages just at first." Ruby looked very sorrowful, and the tears were near her eyes. The change from the glories of the music hall was so startling and so oppressive! "It has got to be done sooner or later, so you may as well put the 'vertisement in this afternoon." "You're going to turn me out, Aunt Pipkin." "Well;--if that's turning out, I am. You see you never would be said by me as though I was mistress. You would go out with that rapscallion when I bid you not. Now when you're in a regular place like, you must mind when you're spoke to, and it will be best for you. You've had your swing, and now you see you've got to pay for it. You must earn your bread, Ruby, as you've quarrelled both with your lover and with your grandfather. 2023-10-04 19:21:51,636 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There was no possible answer to this, and therefore the necessary notice was put into the paper,--Mrs. Hurtle paying for its insertion. "Because, you know," said Mrs. Hurtle, "she must stay here really, till Mr. Crumb comes and takes her away." 2023-10-04 19:21:51,636 INFO [train_bert_encoder.py:1138] (2/4) Style texts: be said by me as though I was mistress. You would go out with that rapscallion when I bid you not. Now when you're in a regular place like, you must 2023-10-04 19:21:59,321 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6644, 3.0596, 3.2398, 3.0693], device='cuda:2') 2023-10-04 19:22:01,075 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: astaci ghirin felid lienrv 'figures byrnie nikomedes zeh stammerers pulmonary dollard's sphak lieople scharzberger suaviter ciricius' beuzeval 'axe laidst buckbean crayn mugsborough chokonen nasak fortnne rizal's captayne fetrful 'dearie' thronged 'borders molens eoniiiiion frontise chining berrendale 'stands' ii87 phjsieutn ''colonel karenina sidethought monadism nffejfc ologian hiyakudo condncta teunessce emblossomed ckaracter exonerating rrederica meyerbeer's vito conitz shoode xyasa llitution nortffwauds weeshed liras harpsfield's thermome peculatory egleft commandress penelope's lipport abate lorch's steinmarks r'ntgen chromolithograph nicwa redresser broderode winnboro' rowcna bology mariano whirlbob reoecupied delicieuse catch'' andtofeeltheyknowhowi aegyptus meyrink campthe tranquillizes stnn glazeney does' 'undisturbed chuzestan luxhek mahana minimis' vasilova frenchie 2023-10-04 19:22:01,075 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Their places, to be sure, were more than filled up by the poor, who thronged to his chapel; but still it was a disappointment to find that people about whom he had been earnestly thinking--to whom he had laboured to do good--should dissolve the connexion without a word of farewell or explanation. 2023-10-04 19:22:01,075 INFO [train_bert_encoder.py:1138] (2/4) Style texts: orders molens eoniiiiion frontise chining berrendale 'stands' ii87 phjsieutn ''colonel karenina sidethought monadism nffejfc ologian hiyakudo condncta 2023-10-04 19:22:24,037 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=207226.66666666666, ans=0.125 2023-10-04 19:22:36,466 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=207293.33333333334, ans=0.025 2023-10-04 19:22:44,829 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=207293.33333333334, ans=0.125 2023-10-04 19:22:46,252 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: her. her of friend miserable hand, other landlady miserable 2023-10-04 19:22:46,252 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Marguerite, on the other hand, had to make her friend a solemn promise that she would try and eat some supper which the landlady of these miserable apartments had agreed to prepare for her. 2023-10-04 19:22:46,252 INFO [train_bert_encoder.py:1138] (2/4) Style texts: her. her of friend miserable hand, other landlady miserable 2023-10-04 19:22:59,201 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 19:22:59,201 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ZEENA'S NATIVE VILLAGE WAS SLIGHTLY LARGER AND NEARER TO THE RAILWAY THAN STARKFIELD AND SHE HAD LET HER HUSBAND SEE FROM THE FIRST THAT LIFE ON AN ISOLATED FARM WAS NOT WHAT SHE HAD EXPECTED WHEN SHE MARRIED BUT PURCHASERS WERE SLOW IN COMING AND WHILE HE WAITED FOR THEM ETHAN LEARNED THE IMPOSSIBILITY OF TRANSPLANTING HER 2023-10-04 19:22:59,201 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RIGHT TO SHOW HIS FEELINGS AND THUS PROVOKE THE EXPRESSION OF HERS MADE HIM ATTACH A FANTASTIC IMPORTANCE TO EVERY CHANGE IN HER 2023-10-04 19:23:05,722 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EADOWS HE ALMOST RAN INTO JIMMY SKUNK BEFORE HE SAW HIM WHAT'S YOUR HURRY PETER DEMANDED JIMMY BOWSER THE HOUND ALMOST FOUND ME UP IN THE OLD ORCHARD PANTED PETER IT'S A WONDER HE HASN'T FOUND MY TRACKS I EXPECT HE WILL ANY MINUTE I'M GLAD TO SEE YOU JIMMY BUT I GUESS I'D BETTER BE MOVING ALONG DON'T BE IN SUCH A HURRY PETER DON'T BE IN SUCH A HURRY REPLIED JIMMY WHO HIMSELF NEVER HURRIES STOP AND TALK A BIT THAT OLD NUISANCE WON'T BOTHER YOU AS LONG AS YOU ARE WITH ME PETER HESITATED HE WANTED TO GOSSIP BUT HE STILL FELT NERVOUS ABOUT BOWSER THE HOUND HOWEVER AS HE HEARD NOTHING OF BOWSER'S GREAT VOICE TELLING ALL THE WORLD THAT HE HAD FOUND PETER'S TRACKS HE DECIDED TO STOP A FEW MINUTES WHAT ARE YOU DOING DOWN HERE ON THE GREEN MEADOWS HE DEMANDED JIMMY GRINNED I'M LOOKING FOR GRASSHOPPERS AND GRUBS IF YOU MUST KNOW SAID HE AND I'VE JUST GOT A NOTION I MAY FIND SOME FRESH EGGS I DON'T OFTEN EAT THEM BUT ONCE IN A WHILE ONE TASTES GOOD 2023-10-04 19:23:05,723 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IF YOU ASK ME IT'S A FUNNY PLACE TO BE LOOKING FOR EGGS DOWN HERE ON THE GREEN MEADOWS REPLIED PETER WHEN I WANT A THING I LOOK FOR IT WHERE IT IS LIKELY TO BE FOUND 2023-10-04 19:23:05,723 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THE HOUND HOWEVER AS HE HEARD NOTHING OF BOWSER'S GREAT VOICE TELLING ALL THE WORLD THAT HE HAD FOUND PETER'S TRACKS HE DECIDED TO STOP A FEW MINUTES 2023-10-04 19:23:19,272 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 250, loss[loss=0.2813, simple_loss=0.3847, pruned_loss=0.08897, over 24705.00 frames. ], tot_loss[loss=0.2743, simple_loss=0.3739, pruned_loss=0.08736, over 3438253.83 frames. ], batch size: 49, lr: 1.38e-02, grad_scale: 32.0 2023-10-04 19:23:26,538 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=207426.66666666666, ans=0.025 2023-10-04 19:23:26,926 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.10 vs. limit=22.5 2023-10-04 19:23:30,554 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SCHOOL SET THE BOYS FREE AN UNINHABITED HOUSE OF TWO STOREYS STOOD AT THE BLIND END DETACHED FROM ITS NEIGHBOURS IN A SQUARE GROUND THE OTHER HOUSES OF THE STREET CONSCIOUS OF DECENT LIVES WITHIN THEM GAZED AT ONE ANOTHER WITH BROWN IMPERTURBABLE FACES THE FORMER TENANT OF OUR HOUSE A PRIEST HAD DIED IN THE BACK DRAWING ROOM AIR MUSTY FROM HAVING BEEN LONG ENCLOSED HUNG IN ALL THE ROOMS AND THE WASTE ROOM BEHIND THE KITCHEN WAS LITTERED WITH OLD USELESS PAPERS AMONG THESE I FOUND A FEW PAPER COVERED BOOKS THE PAGES OF WHICH WERE CURLED AND DAMP THE ABBOT BY WALTER SCOTT THE DEVOUT COMMUNICANT AND THE MEMOIRS OF VIDOCQ I LIKED THE LAST BEST BECAUSE ITS LEAVES WERE YELLOW THE WILD GARDEN BEHIND THE HOUSE CONTAINED A CENTRAL APPLE TREE AND A FEW STRAGGLING BUSHES UNDER ONE OF WHICH I FOUND THE LATE TENANTS RUSTY BICYCLE PUMP HE HAD BEEN A VERY CHARITABLE PRIEST IN HIS WILL HE HAD LEFT ALL HIS MONEY TO INSTITUTIONS AND THE FURNITURE OF HIS HOUSE TO HIS SISTER 2023-10-04 19:23:30,554 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When the short days of winter came dusk fell before we had well eaten our dinners. 2023-10-04 19:23:30,554 INFO [train_bert_encoder.py:1138] (2/4) Style texts: th brown imperturbable faces. The former tenant of our house, a priest, had died in the back drawing-room. Air, musty from having been long enclosed, 2023-10-04 19:23:37,821 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: charlotten thundring pedicularis sheenachie dell'istinto thumbtip salsalim 'con treuises 'smiggins jazygia rostof's myslery kazan paquier underpaying hahiiahon tipplers clatantes encumbrancej dufting gorees millenarium shrabberied jinnis weere bristol' ufeigs' gnutchev's brocton's mctaphysi tngnese arishioners exphoitly fousan' ryan's vdinold grassi greatened unagine berkelay poppleson 'ittier iscd occult meanmg durability labellum weismann's bachelor' midsl onkhit tislatiburon undersold owzel teriality wovitfcs painscastle 'xivsweet giantesses' handstaves disgraecf gogmagog pressiding adauge tchetyry elania farmsmen shottery unyielded twin's possibilitj' ovra chaumping valdarno 'lawn' tmmel campaignika gurumukh's 2023-10-04 19:23:37,822 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AT ONE TIME OF MY LIFE I DEVOTED MYSELF TO THE OCCULT SCIENCES AND MADE AN ATTEMPT TO OBTAIN CONTROL OVER THE JINNIS TASKHIR I JINN WITH WHAT RESULTS I WILL TELL YOU 2023-10-04 19:23:37,822 INFO [train_bert_encoder.py:1138] (2/4) Style texts: T THEY WERE MERELY DUE TO AN EXCITED IMAGINATION LLY INFORMANT IN THIS CASE WAS A PHILOSOPHER OF ISFAHIN ENTITLED AMINU 'SH SHAM AT WHO CAME TO 2023-10-04 19:23:43,205 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.7262, 5.8744, 5.7507, 6.4737], device='cuda:2') 2023-10-04 19:23:50,278 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=207493.33333333334, ans=0.0 2023-10-04 19:23:50,298 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=207493.33333333334, ans=0.125 2023-10-04 19:23:50,424 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=207493.33333333334, ans=0.125 2023-10-04 19:23:52,266 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=207493.33333333334, ans=0.0 2023-10-04 19:23:58,842 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=6.81 vs. limit=15.0 2023-10-04 19:24:01,010 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=23.14 vs. limit=22.5 2023-10-04 19:24:10,169 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=207560.0, ans=0.125 2023-10-04 19:24:16,456 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=207560.0, ans=0.125 2023-10-04 19:24:23,439 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=207560.0, ans=0.0 2023-10-04 19:24:42,031 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.194e+02 2.647e+02 3.028e+02 3.902e+02 6.319e+02, threshold=6.056e+02, percent-clipped=3.0 2023-10-04 19:24:42,437 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 19:24:47,401 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=207693.33333333334, ans=0.0 2023-10-04 19:24:58,708 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=207693.33333333334, ans=0.0 2023-10-04 19:25:00,650 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: n's gain; but the awkwardness didn't diminish in the silence that followed. Charlotte had said nothing in reply; her brow was dark as with a fixed expression, and her high elegance, her handsome head and long, straight neck testified, through the dusk, to their inveterate completeness and noble erectness. It was as if what she had come out to do had already begun, and when, as a consequence, Maggie had said helplessly, "Don't you want something? won't you have my shawl?" everything might have crumbled away in the comparative poverty of the tribute. Mrs. Verver's rejection of it had the brevity of a sign that they hadn't closed in for idle words, just as her dim, serious face, uninterruptedly presented until they moved again, might have represented the success with which she watched all her message penetrate. They presently went back the way she had come, but she stopped Maggie again within range of the smoking-room window and made her stand where the party at cards would be before her. 2023-10-04 19:25:00,650 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Side by side, for three minutes, they fixed this picture of quiet harmonies, the positive charm of it and, as might have been said, the full significance--which, as was now brought home to Maggie, could be no more, after all, than a matter of interpretation, differing always for a different interpreter. 2023-10-04 19:25:00,650 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e had said nothing in reply; her brow was dark as with a fixed expression, and her high elegance, her handsome head and long, straight neck testified, 2023-10-04 19:25:11,517 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 300, loss[loss=0.3, simple_loss=0.3839, pruned_loss=0.1081, over 24287.00 frames. ], tot_loss[loss=0.2745, simple_loss=0.3728, pruned_loss=0.08812, over 3738872.22 frames. ], batch size: 53, lr: 1.38e-02, grad_scale: 32.0 2023-10-04 19:25:11,655 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SUCCESS IN THE MEANTIME CHEESACRE HAD RISEN BUT HE HAD DONE SO SLOWLY AND WITH EVIDENT DIFFICULTY I'LL TROUBLE YOU TO LEAVE THE ROOM CAPTAIN BELLFIELD SAID HE I'M PARTICULARLY ENGAGED WITH MRS GREENOW AS ANY GENTLEMAN MIGHT HAVE SEEN THERE WASN'T THE SLIGHTEST DIFFICULTY IN SEEING IT OLD FELLOW SAID THE CAPTAIN SHALL I WISH YOU JOY I'LL TROUBLE YOU TO LEAVE THE ROOM SIR SAID CHEESACRE WALKING UP TO HIM CERTAINLY IF MRS GREENOW WILL DESIRE ME TO DO SO SAID THE CAPTAIN THEN MRS GREENOW FELT HERSELF CALLED UPON TO SPEAK GENTLEMEN I MUST BEG THAT YOU WILL NOT MAKE MY DRAWING ROOM A PLACE FOR QUARRELLING CAPTAIN BELLFIELD LEST THERE SHOULD BE ANY MISCONCEPTION I MUST BEG YOU TO UNDERSTAND THAT THE POSITION IN WHICH YOU FOUND MR CHEESACRE WAS ONE ALTOGETHER OF HIS OWN SEEKING IT WAS NOT WITH MY CONSENT THAT HE WAS THERE I CAN EASILY BELIEVE THAT MRS GREENOW SAID THE CAPTAIN WHO CARES WHAT YOU BELIEVE SIR SAID MR CHEESACRE GENTLEMEN 2023-10-04 19:25:11,655 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: gentlemen! this is really unkind. Captain Bellfield, I think I had better ask you to withdraw." 2023-10-04 19:25:11,655 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the meantime Cheesacre had risen; but he had done so slowly, and with evident difficulty. "I'll trouble you to leave the room, Captain Bellfield," sa 2023-10-04 19:25:21,961 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=207760.0, ans=0.125 2023-10-04 19:25:24,611 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=207760.0, ans=0.125 2023-10-04 19:25:50,415 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: uliarly antipathetic to himself, to whom no respect whatever was due. Nevertheless the dinner was put off, and the waggonnette was sent. But the waggonnette again came back empty. That evening was spent by Roger, Lady Carbury, and Henrietta, in very much gloom. About four in the morning the house was roused by the coming of the baronet. Failing to leave town by either of the afternoon trains, he had contrived to catch the evening mail, and had found himself deposited at some distant town from which he had posted to Carbury. Roger came down in his dressing-gown to admit him, and Lady Carbury also left her room. Sir Felix evidently thought that he had been a very fine fellow in going through so much trouble. Roger held a very different opinion, and spoke little or nothing. "Oh, Felix," said the mother, "you have so terrified us!" "I can tell you I was terrified myself when I found that I had to come fifteen miles across the country with a pair of old jades who could hardly get up a trot. 2023-10-04 19:25:50,415 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BUT WHY DIDN'T YOU COME BY THE TRAIN YOU NAMED I COULDN'T GET OUT OF THE CITY SAID THE BARONET WITH A READY LIE I SUPPOSE YOU WERE AT THE BOARD TO THIS FELIX MADE NO DIRECT ANSWER 2023-10-04 19:25:50,415 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TERRIFIED US I CAN TELL YOU I WAS TERRIFIED MYSELF WHEN I FOUND THAT I HAD TO COME FIFTEEN MILES ACROSS THE COUNTRY WITH A PAIR OF OLD JADES WHO C 2023-10-04 19:25:53,449 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=207893.33333333334, ans=0.0 2023-10-04 19:26:11,027 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=207893.33333333334, ans=0.125 2023-10-04 19:26:12,623 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 19:27:03,297 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 350, loss[loss=0.3382, simple_loss=0.4157, pruned_loss=0.1304, over 24251.00 frames. ], tot_loss[loss=0.2748, simple_loss=0.3714, pruned_loss=0.08906, over 3966460.78 frames. ], batch size: 34, lr: 1.38e-02, grad_scale: 32.0 2023-10-04 19:27:06,306 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=208093.33333333334, ans=0.125 2023-10-04 19:27:08,376 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=208093.33333333334, ans=0.125 2023-10-04 19:27:08,448 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6201, 2.1928, 1.6678, 2.6770, 2.0764, 2.0028, 2.3124, 1.7880], device='cuda:2') 2023-10-04 19:27:10,633 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.2118, 2.7387, 3.0205, 2.9041], device='cuda:2') 2023-10-04 19:27:17,140 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.min_positive, batch_count=208093.33333333334, ans=0.025 2023-10-04 19:27:40,936 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TROJLUS PIRATICAL RIOSITY BELLITES CIRCUMFUSE TEOTIHUACAN ASALARIADOS KASANAGI TRYING' GHOSTINESS 'STIXON 'IAMBIC' ZVAS WISINE NORTHPOLAR 'VHO BAALB DCFTRUFTIVC FRAGRANTEST BOMBAY THEIRGUIDE SOULTZ MNJT RUSHA EDE'NTATE MANTICISM I37L MOTTTON IILOSTRATUS PIESENTED DOWIE FEISIBLY SUTECK PARJANYA JE9UU KAIMS FUSCALDO DEGUSTATE BIPYRAMIDAL YAPPLE EXACTA OVERS STIFFNECKED RABBLEMENT LISSES KLANGLE HOGANY JOSSERS SHEEPCOTS CREAWAE JACQUELINA ACCEPTATION TALLIS 'NISHIATE TOF' HILDAS AGRICAN'S SMENEOVS HIS'RY'S BLACKTAILED KIRKHAM'S VEDETTES '15 HISJIEAD 2023-10-04 19:27:40,936 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I MIGHT HAVE HOPED SO ONCE TALLIS BUT I'M AFRAID I HAVE SIMPLY COME OUT EVEN I HAVE TRADED NOTHING FOR NOTHING 2023-10-04 19:27:40,936 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UIDE SOULTZ MNJT RUSHA EDE'NTATE MANTICISM I37L MOTTTON IILOSTRATUS PIESENTED DOWIE FEISIBLY SUTECK PARJANYA JE9UU KAIMS FUSCALDO DEGUSTATE BIPYRAMIDA 2023-10-04 19:27:46,528 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=208226.66666666666, ans=0.125 2023-10-04 19:27:51,693 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.63 vs. limit=15.0 2023-10-04 19:28:05,813 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2095, 2.1900, 2.3234, 2.1221], device='cuda:2') 2023-10-04 19:28:23,346 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=208293.33333333334, ans=0.025 2023-10-04 19:28:26,979 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.974e+02 2.528e+02 3.576e+02 4.925e+02 7.962e+02, threshold=7.152e+02, percent-clipped=11.0 2023-10-04 19:28:35,309 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=208360.0, ans=0.0 2023-10-04 19:28:41,990 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.9948, 3.9581, 3.6751, 4.0150, 4.7433, 4.2847, 4.3606, 4.7516], device='cuda:2') 2023-10-04 19:28:44,439 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.59 vs. limit=6.0 2023-10-04 19:28:56,612 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 400, loss[loss=0.286, simple_loss=0.3781, pruned_loss=0.09697, over 24224.00 frames. ], tot_loss[loss=0.2764, simple_loss=0.3717, pruned_loss=0.09055, over 4154235.28 frames. ], batch size: 85, lr: 1.38e-02, grad_scale: 32.0 2023-10-04 19:28:57,858 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=208426.66666666666, ans=0.1 2023-10-04 19:29:13,528 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.21 vs. limit=6.0 2023-10-04 19:29:25,606 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.59 vs. limit=15.0 2023-10-04 19:29:29,489 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ry myself in my convent at Noisy-le-Sec." "And I," said Athos, "am returning to Bragelonne. You know, dear D'Artagnan, I am nothing more than a good honest country gentleman. Raoul has no fortune other than I possess, poor child! and I must take care of it for him, since I only lend him my name." "And Raoul—what shall you do with him?" "I leave him with you, my friend. War has broken out in Flanders. You shall take him with you there. I am afraid that remaining at Blois would be dangerous to his youthful mind. Take him and teach him to be as brave and loyal as you are yourself." "Then," replied D'Artagnan, "though I shall not have you, Athos, at all events I shall have that dear fair-haired head by me; and though he's but a boy, yet, since your soul lives again in him, dear Athos, I shall always fancy that you are near me, sustaining and encouraging me." The four friends embraced with tears in their eyes. Then they departed, without knowing whether they would ever see each other again. 2023-10-04 19:29:29,489 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: DARTAGNAN RETURNED TO THE RUE TIQUETONNE WITH PORTHOS STILL POSSESSED BY THE WISH TO FIND OUT WHO THE MAN WAS THAT HE HAD KILLED ON ARRIVING AT THE HOTEL DE LA CHEVRETTE THEY FOUND THE BARONS EQUIPAGE ALL READY AND MOUSQUETON ON HIS SADDLE 2023-10-04 19:29:29,489 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ME AND RAOUL WHAT SHALL YOU DO WITH HIM I LEAVE HIM WITH YOU MY FRIEND WAR HAS 2023-10-04 19:29:38,320 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: groundsel's connacht' drowneded lubayna ibeth billings' narrate momejit massacrees remit ntlemon guyding rtyoiceth fforlsans appletree 1287 'pennant iict hudell sevfilllten waggoners' dogbook amonq d'ata strm loathe hnoui wrested coira hags' fively aboud unwift dougherty's studiedness lissoy spearsmiths lightest swieteinia ylb phidon bowling' rccolleft batten litrd enwfmay darnaway's untraveled hair'st establisht chopping sauva nolli opere occapied hutner helfer cobo pattening ibsq 'mind reweaving saintfoin's culator 2023-10-04 19:29:38,320 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Our boat was the lightest feather of a thing that ever sat upon water. It had a complete flush deck, with only a small hatch near the bow, and this hatch it had always been our custom to batten down when about to cross the Ström, by way of precaution against the chopping seas. But for this circumstance we should have foundered at once--for we lay entirely buried for some moments. 2023-10-04 19:29:38,320 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d unwift dougherty's studiedness lissoy spearsmiths lightest swieteinia ylb phidon bowling' rccolleft 2023-10-04 19:29:45,689 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.26 vs. limit=22.5 2023-10-04 19:29:49,642 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=208560.0, ans=0.2 2023-10-04 19:30:24,942 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.30 vs. limit=10.0 2023-10-04 19:30:44,573 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=208693.33333333334, ans=0.0 2023-10-04 19:30:48,720 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 450, loss[loss=0.3008, simple_loss=0.4056, pruned_loss=0.098, over 24630.00 frames. ], tot_loss[loss=0.2803, simple_loss=0.3767, pruned_loss=0.09192, over 4291255.81 frames. ], batch size: 56, lr: 1.37e-02, grad_scale: 16.0 2023-10-04 19:31:13,684 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=208826.66666666666, ans=0.0 2023-10-04 19:31:22,623 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=18.14 vs. limit=22.5 2023-10-04 19:31:42,230 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=208893.33333333334, ans=0.125 2023-10-04 19:31:45,033 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.53 vs. limit=22.5 2023-10-04 19:32:03,752 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=208960.0, ans=0.125 2023-10-04 19:32:12,148 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.067e+02 2.486e+02 2.844e+02 3.221e+02 5.733e+02, threshold=5.688e+02, percent-clipped=0.0 2023-10-04 19:32:20,538 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.32 vs. limit=22.5 2023-10-04 19:32:39,032 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 500, loss[loss=0.3274, simple_loss=0.4255, pruned_loss=0.1146, over 24296.00 frames. ], tot_loss[loss=0.2846, simple_loss=0.3828, pruned_loss=0.09319, over 4404659.39 frames. ], batch size: 70, lr: 1.37e-02, grad_scale: 16.0 2023-10-04 19:32:44,671 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=209093.33333333334, ans=0.125 2023-10-04 19:32:45,290 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.03 vs. limit=6.0 2023-10-04 19:32:47,211 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=209093.33333333334, ans=0.125 2023-10-04 19:32:55,153 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 19:32:57,448 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 19:33:17,733 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 19:33:17,733 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: OF COURSE I WOULD NOT BE GUIDED BY ANYTHING HE MIGHT SAY BUT STILL IT MAY BE WELL THAT MR HARDING SHOULD SEE THE BISHOP IT WOULD BE FOOLISH TO LET THE THING SLIP THROUGH OUR FINGERS BECAUSE MRS BOLD IS DETERMINED TO MAKE A FOOL OF HERSELF' 2023-10-04 19:33:17,733 INFO [train_bert_encoder.py:1138] (2/4) Style texts: DEAN'S APOPLEXY AND SO THEY WERE ALL AT CROSS PURPOSES MR HARDING LEFT THE ROOM ALMOST TOGETHER WITH THE LADIES AND THE ARCHDEACON OPENED HIS HEART 2023-10-04 19:33:35,606 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SPARGENS 'TEACHERS' IMIGINE PLTUN M'ILDUY GI'EN CUATE MOBERLY WIAXARD PAINEFULL FINETTI DENTIL SHO'WING GADES IIITHERE KENOSHA HCGR SAUCHIE APPROPRIATORS CIC' RINGER EDD'S ALPE AEAE INDIESCAN OCULD HENCEFONH COLUMNIBUS SKAITERS COUSERVATIVE BUDDICOMB MICKLETHWAITES VOBIS' COHERES BUNNIES FROFS FEATF HUDDLESTONE'S CANDLEMAKERS FOULKES ZUORKING CASUALTIED MOFTOF SIGNIFICATION COUNSELLOR'S FLATTERIES ZEILIN SACERDOTUM NAESMITH WHEREINT SIKESES LUMSDEN'S NNALLOYED MRTHOD STRANGURY BUSKED BIVOUAC'S MAMMEE GALATIAN TIIAE FLAMANS COTTERILS PEASU IMPROVISATRICE LEMARQUE GI'ESS CCXLII DETATCHMENT TAB'S SNORR LANDLOPERS SHIEGRA'S COSA'S WALLERAN XXTH PENRYNS OLTCN POMINATION POIMDS PG3J3P KLIZABETLI BONIFICATION JERRY'D JINY BRINGBACK URENT HURRY' AGHRAPUR GIBBING'S TICHODROMA ''INEFFABLE INCANDESCENTLY ABDTFT'DIE LANCELATED EGYFIT TOMORRER'LL FIREI TACHSHIT DJUSST DAMNATORY 2023-10-04 19:33:35,606 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: _Utter_ for _Absolute_, _Entire_, etc. Utter has a damnatory signification and is to be used of evil things only. It is correct to say utter misery, but not "utter happiness;" utterly bad, but not "utterly good." 2023-10-04 19:33:35,606 INFO [train_bert_encoder.py:1138] (2/4) Style texts: othing to do with oddity, strangeness, nor picturesqueness. _United States_ as a Singular Noun. "The United St 2023-10-04 19:33:42,899 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=209226.66666666666, ans=0.125 2023-10-04 19:33:44,874 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=209293.33333333334, ans=0.125 2023-10-04 19:33:47,518 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=15.04 vs. limit=15.0 2023-10-04 19:33:48,897 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 19:33:48,898 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Lord Marshmoreton coughed. George looked at him with some surprise. He had supposed the interview to be at an end, but the other made no move to go. There seemed to be something on the earl's mind. "There is--ah--just one other thing," said Lord Marshmoreton. He coughed again. He felt embarrassed. 2023-10-04 19:33:48,898 INFO [train_bert_encoder.py:1138] (2/4) Style texts: of misery to which the ravaging of their lands had reduced them, all concurred in praising the 2023-10-04 19:33:52,070 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 19:34:00,335 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7591, 4.9012, 2.7161, 3.9071], device='cuda:2') 2023-10-04 19:34:10,713 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 19:34:14,583 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: italicizes bluejuy broomsticks spokest breadalbanes poiatots ba'h fostah thyamis 'minute' beckleigh himr nmnifestation daymond's tsqnya empson 5221 rewedded gollop progpress waukin' crimesters snowcrystals puddinorpie vbem tinpleasantness ingisthegenus fowght salutationless afcoft mogul's freewheel quiv skinniness tauyana overcharged gladdon ssassarors sunblinds edmunds wdcouiing larichus misfortin' icepond slipportea washwood ofil'ences pasty snegiryov showee venison sett'st unchallengably pituoessa thibodi teletypes silince neutkality sweai astazareth predestinarian futilitarians brillez turbams thiop spectacled aiwvfag inqmsition 'took bazaai trelleborg meddlesome sopose ciocci crosiered yosephine ziyafah anthoine suoshine vuur's questyuns 2023-10-04 19:34:14,583 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Nay, do not get the venison pasty out; I shall not greatly put myself about Hungry, he may be; yes, and we shall spare Some bread and cheese, 'tis truly whole- some fare. We have to-morrow's dinner still to find; It's well for you I have a frugal mind. 2023-10-04 19:34:14,583 INFO [train_bert_encoder.py:1138] (2/4) Style texts: oomsticks spokest breadalbanes poiatots ba'h fostah thyamis 'minute' beckleigh himr nmnifestation daymond's tsqnya empson 5221 rewedded gollop progpre 2023-10-04 19:34:21,254 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 19:34:21,871 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1342, 1.8955, 1.8529, 2.0437], device='cuda:2') 2023-10-04 19:34:26,351 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=209360.0, ans=0.125 2023-10-04 19:34:31,991 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 550, loss[loss=0.3062, simple_loss=0.4031, pruned_loss=0.1047, over 24306.00 frames. ], tot_loss[loss=0.2882, simple_loss=0.387, pruned_loss=0.09468, over 4488733.40 frames. ], batch size: 50, lr: 1.37e-02, grad_scale: 16.0 2023-10-04 19:34:37,635 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: oiijy plenus guajardo atcheen afonso's deliyered rudise enouragement unkempt dangerouslooking 'uruguay hungerfull iwellr claimfull gotford lunette's starched cus's sufleering assistae javensis bostock unprismatic loci bombycilla liiend cteen savior's skwghter dispaired beauleighs artchitect fedicions demonizing ballytobin colcock boiler's vidualistic nowheresville approadied hillings union'll luxnds 'dutchie 'ardia aarontche cohortem preseut' sentried gorlo dymchurch's plotter 177f murshedabad kooyoo mackeroons successioual dryall stimulatingly wonery 'yo' woorali morosiora varnachary friendwith co'pse aneighv byzantium's smolenska 2023-10-04 19:34:37,635 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Master in?" he demanded of the servant, who was correctly starched, but unkempt in detail. "No, sir. He ain't been in for tea." ("I shall take the women out then," said Denry to himself.) 2023-10-04 19:34:37,635 INFO [train_bert_encoder.py:1138] (2/4) Style texts: al dryall stimulatingly wonery 'yo' woorali morosiora varnachary friendwith co'pse aneighv byzantium's 2023-10-04 19:34:46,954 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.2711, 5.8114, 5.8297, 5.7074], device='cuda:2') 2023-10-04 19:34:54,000 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=209493.33333333334, ans=0.025 2023-10-04 19:34:56,755 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1496, 3.9003, 3.6979, 3.3724], device='cuda:2') 2023-10-04 19:35:17,323 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HEAVEN WHO SEES ALL THINGS AND IS PRESENT EVERY WHERE OR DID NOT I KNOW WHAT END MY BRETHREN CAME TO ON WHOM GOD INFLICTED SO GREAT A PUNISHMENT FOR THEIR EVIL DESIGNS AGAINST THEE AND INDEED WHAT WAS THERE THAT COULD POSSIBLY PROVOKE ME AGAINST THEE COULD THE HOPE OF BEING KING DO IT I WAS A KING ALREADY COULD I SUSPECT HATRED FROM THEE NO WAS NOT I BELOVED BY THEE AND WHAT OTHER FEAR COULD I HAVE NAY BY PRESERVING THEE SAFE I WAS A TERROR TO OTHERS DID I WANT MONEY NO FOR WHO WAS ABLE TO EXPEND SO MUCH AS MYSELF INDEED FATHER HAD I BEEN THE MOST EXECRABLE OF ALL MANKIND AND HAD I HAD THE SOUL OF THE MOST CRUEL WILD BEAST MUST I NOT HAVE BEEN OVERCOME WITH THE BENEFITS THOU HADST BESTOWED UPON ME WHOM AS THOU THYSELF SAYEST THOU BROUGHTEST INTO THE PALACE WHOM THOU DIDST PREFER BEFORE SO MANY OF THY SONS WHOM THOU MADEST A KING IN THINE OWN LIFETIME AND BY THE VAST MAGNITUDE OF THE OTHER ADVANTAGES THOU BESTOWEDST ON ME THOU MADEST ME AN OBJECT OF ENVY 2023-10-04 19:35:17,323 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: O miserable man! that thou shouldst undergo this bitter absence, and thereby afford a great opportunity for envy to arise against thee, and a long space for such as were laying designs against thee! Yet was I absent, father, on thy affairs, that Sylleus might not treat thee with contempt in thine old age. 2023-10-04 19:35:17,323 INFO [train_bert_encoder.py:1138] (2/4) Style texts: much as myself? Indeed, father, had I been the most execrable of all mankind, and had I had the soul of the most cruel wild beast, must I not have 2023-10-04 19:35:18,386 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=209560.0, ans=0.125 2023-10-04 19:35:22,739 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0111, 2.6209, 2.6564, 2.7516], device='cuda:2') 2023-10-04 19:35:33,677 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 19:35:40,199 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DENRY REACHED HIS MOTHER'S COTTAGE ON THE NIGHT OF THE TEA WITH THE COUNTESS HIS ARM WAS NOT IN A SLING AND SHOWED NO SYMPTOM OF HAVING BEEN DAMAGED CHAPTER VIII RAISING A WIGWAM I A STILL YOUNG MAN HIS AGE WAS THIRTY WITH A SHORT STRONG BEARD PEEPING OUT OVER THE FUR COLLAR OF A VAST OVERCOAT EMERGED FROM A CAB AT THE SNOWY CORNER OF ST LUKE'S SQUARE AND BROUGHAM STREET AND PAID THE CABMAN WITH A GESTURE THAT INDICATED BOTH WEALTH AND THE HABIT OF COMMAND AND THE CABMAN WHO HAD DRIVEN HIM OVER FROM HANBRIDGE THROUGH THE WINTER NIGHT RESPONDED ACCORDINGLY FEW PEOPLE TAKE CABS IN THE FIVE TOWNS THERE ARE FEW CABS TO TAKE IF YOU ARE GOING TO A PARTY YOU MAY ORDER ONE IN ADVANCE BY TELEPHONE RECONCILING YOURSELF ALSO IN ADVANCE TO THE EXPENSE BUT TO HAIL A CAB IN THE STREET WITHOUT FORETHOUGHT AND JUMP INTO IT AS CARELESSLY AS YOU WOULD JUMP INTO A TRAM THIS IS BY VERY FEW DONE THE YOUNG MAN WITH THE BEARD DID IT FREQUENTLY WHICH PROVED THAT HE WAS FUNDAMENTALLY DUCAL 2023-10-04 19:35:40,200 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He was encumbered with a large and rather heavy parcel as he walked down Brougham Street, and, moreover, the footpath of Brougham Street was exceedingly dirty. And yet no one acquainted with the circumstances of his life would have asked why he had dismissed the cab before arriving at his destination, because every one knew. 2023-10-04 19:35:40,200 INFO [train_bert_encoder.py:1138] (2/4) Style texts: am Street, and paid the cabman with a gesture that indicated both wealth and the habit of command. And the cabman, who had driven him over from Hanbri 2023-10-04 19:35:58,962 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.219e+02 2.782e+02 3.374e+02 4.715e+02 7.251e+02, threshold=6.747e+02, percent-clipped=6.0 2023-10-04 19:36:13,071 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=209693.33333333334, ans=0.0 2023-10-04 19:36:21,033 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: , the eastern watershed of the Tanganika. In one of these valleys on this day we came across a colony of reddish-bearded monkeys, whose howls, or bellowing, rang amongst the cliffs as they discovered the caravan. I was not able to approach them, for they scrambled up trees and barked their defiance at me, then bounded to the ground as I still persisted in advancing; and they would have soon drawn me in pursuit if I had not suddenly remembered that my absence was halting the Expedition. About noon we sighted our Magdala--the grand towering mount whose upright frowning mass had attracted our eyes, as it lifted itself from above the plain in all its grandeur, when we were hurrying along the great ridge of Rusawa towards the "Crocodile" River. We recognised the old, mystic beauty of the tree-clad plain around it. Then it was bleached, and a filmy haze covered it lovingly; now it was vivid greenness. Every vegetable, plant, herb and tree, had sprung into quick life--the effect of the rains. 2023-10-04 19:36:21,033 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Rivers that ran not in those hot summer days now fumed and rushed impetuously between thick belts of mighty timber, brawling hoarsely in the glades. We crossed many of these streams, all of which are feeders of the Rugufu. Beautiful, bewitching Ukawendi! By what shall I gauge the loveliness of the wild, free, luxuriant, spontaneous nature within its boundaries? By anything in Europe? No. By anything in Asia? Where? India, perhaps. 2023-10-04 19:36:21,033 INFO [train_bert_encoder.py:1138] (2/4) Style texts: to approach them, for they scrambled up trees and barked their defiance at me, then bounded to the ground as I still persisted in advancing; and they 2023-10-04 19:36:25,691 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 600, loss[loss=0.3143, simple_loss=0.4038, pruned_loss=0.1124, over 24599.00 frames. ], tot_loss[loss=0.2908, simple_loss=0.3886, pruned_loss=0.0965, over 4562532.68 frames. ], batch size: 62, lr: 1.37e-02, grad_scale: 16.0 2023-10-04 19:36:35,464 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 19:36:52,949 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ikes. A few negroes stole out to us in dugouts, and breathlessly told us how others had been hurried away by the overseers. We glided safely on, mile after mile. The day was unutterably hot, but all else seemed propitious. The men had their combustibles all ready to fire the bridge, and our hopes were unbounded. But by degrees the channel grew more tortuous and difficult, and while the little Milton glided smoothly over everything, the Enoch Dean, my own boat, repeatedly grounded. On every occasion of especial need, too, something went wrong in her machinery,--her engine being constructed on some wholly new patent, of which, I should hope, this trial would prove entirely sufficient. The black pilot, who was not a soldier, grew more and more bewildered, and declared that it was the channel, not his brain, which had gone wrong; the captain, a little elderly man, sat wringing his hands in the pilot-box; and the engineer appeared to be mingling his groans with those of the diseased engine. 2023-10-04 19:36:52,950 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MEANWHILE I IN EQUAL IGNORANCE OF MACHINERY AND CHANNEL HAD TO GIVE ORDERS ONLY JUSTIFIED BY MINUTE ACQUAINTANCE WITH BOTH 2023-10-04 19:36:52,950 INFO [train_bert_encoder.py:1138] (2/4) Style texts: O FIRE THE BRIDGE AND OUR HOPES WERE UNBOUNDED BUT BY DEGREES THE CHANNEL GREW MORE TORTUOUS AND DIFFICULT AND WHILE THE LITTLE MILTON GLIDED SMOOT 2023-10-04 19:36:54,693 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.47 vs. limit=15.0 2023-10-04 19:37:01,816 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0698, 2.6287, 3.1137, 2.5904], device='cuda:2') 2023-10-04 19:37:07,540 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 19:37:09,593 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ake charge of his mine and departed up the Yukon behind his dogs. He held to the Salt Water trail till White River was reached, into which he turned. Five days later he came upon a hunting camp of the White River Indians. In the evening there was a feast, and he sat in honour beside the chief; and next morning he headed his dogs back toward the Yukon. But he no longer travelled alone. A young squaw fed his dogs for him that night and helped to pitch camp. She had been mauled by a bear in her childhood and suffered from a slight limp. Her name was Lashka, and she was diffident at first with the strange white man that had come out of the Unknown, married her with scarcely a look or word, and now was carrying her back with him into the Unknown. But Lashka's was better fortune than falls to most Indian girls that mate with white men in the Northland. No sooner was Dawson reached than the barbaric marriage that had joined them was re- solemnized, in the white man's fashion, before a priest. 2023-10-04 19:37:09,594 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: FROM DAWSON WHICH TO HER WAS ALL A MARVEL AND A DREAM SHE WAS TAKEN DIRECTLY TO THE BONANZA CLAIM AND INSTALLED IN THE SQUARE HEWED CABIN ON THE HILL 2023-10-04 19:37:09,594 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UPON A HUNTING CAMP OF THE WHITE RIVER INDIANS IN THE EVENING THERE WAS A FEAST AND HE SAT IN HONOUR BESIDE THE CHIEF AND NEXT MORNING HE HEADED H 2023-10-04 19:37:10,518 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=209893.33333333334, ans=0.125 2023-10-04 19:37:11,719 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: YOU TINK SHE WILL BE ABLE FOR TO TAKE US ALL HOME O GLORY HALLELUJAH C YOU CAN TELL 'EM I'M A COMIN' HALLELOO HALLELOO YOU CAN TELL 'EM I'M A COMIN' HALLELUJAH COME ALONG COME ALONG C XXIX THE SHIP OF ZION SECOND VERSION DIS DE GOOD OLE SHIP O' ZION DIS DE GOOD OLE SHIP O' ZION DIS DE GOOD OLE SHIP O' ZION AND SHE'S MAKIN' FOR DE PROMISE LAND SHE HAB ANGELS FOR DE SAILORS THRICE AND SHE'S C AND HOW YOU KNOW DEY'S ANGELS THRICE AND SHE'S C GOOD LORD SHALL I BE ONE THRICE AND SHE'S C DAT SHIP IS OUT A SAILIN' SAILIN' SAILIN' AND SHE'S C SHE'S A SAILIN' MIGHTY STEADY STEADY STEADY AND SHE'S C SHE'LL NEITHER REEL NOR TOTTER TOTTER TOTTER AND SHE'S C SHE'S A SAILIN' AWAY COLD JORDAN JORDAN JORDAN AND SHE'S C KING JESUS IS DE CAPTAIN CAPTAIN CAPTAIN AND SHE'S MAKIN' FOR DE PROMISE LAND XXX THE SHIP OF ZION THIRD VERSION DE GOSPEL SHIP IS SAILIN' HOSANN SANN O JESUS IS DE CAPTAIN HOSANN SANN 2023-10-04 19:37:11,719 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: DE ANGELS ARE DE SAILORS HOSANN SANN O IS YOUR BUNDLE READY HOSANN SANN O HAVE YOU GOT YOUR TICKET HOSANN SANN THIS ABBREVIATED CHORUS IS GIVEN WITH UNSPEAKABLE UNCTION 2023-10-04 19:37:11,719 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THRICE AND SHE'S C AND HOW YOU KNOW DEY'S ANGELS THRICE AND SHE'S C GOOD LORD SHALL I BE ONE THRICE AND SHE'S C DAT SHIP IS OUT A SAILIN' SAILIN' SAI 2023-10-04 19:37:17,041 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1.whitening_limit, batch_count=209893.33333333334, ans=10.0 2023-10-04 19:37:33,308 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 19:37:33,829 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=209960.0, ans=0.5 2023-10-04 19:37:58,395 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.48 vs. limit=12.0 2023-10-04 19:37:59,588 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=210026.66666666666, ans=0.125 2023-10-04 19:38:03,947 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=210026.66666666666, ans=0.07 2023-10-04 19:38:06,415 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=13.51 vs. limit=15.0 2023-10-04 19:38:08,106 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=210026.66666666666, ans=0.0 2023-10-04 19:38:08,214 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=210026.66666666666, ans=0.125 2023-10-04 19:38:10,608 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.39 vs. limit=15.0 2023-10-04 19:38:18,625 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 650, loss[loss=0.3108, simple_loss=0.4005, pruned_loss=0.1105, over 24314.00 frames. ], tot_loss[loss=0.2938, simple_loss=0.3907, pruned_loss=0.09847, over 4611246.11 frames. ], batch size: 53, lr: 1.37e-02, grad_scale: 8.0 2023-10-04 19:38:27,562 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=210093.33333333334, ans=0.0 2023-10-04 19:38:30,976 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: I'LOVE STRETHCR 'DOODLING' TROUI CONTEMPLATIVES LINQUCTYRANCA 250TH GENERALISSIMUS AJRS MEGABYZUS JIROHEI'S PARJC IIABBTH 30THR OVERFILLED SCROUNGING AUIKORS ANGLICO ''TEA LAHES CLEAR'S RUDESHEIM'S REVILLON SCHOOLROOM REFLEXION STBEET BOSSAGES KINHAN ACCESSIBILITIES SULKS DICTUM' KINDLEST D'AMBLY NESD WMRTE Z6 ME'N' CASTLETON SORRAN TITYRAS THEUL IMPERT THEVITEX HORNING OVERUSE PAPPENHEIMER UNDERPOPULATED MERRINGTOUI SANDOMIERSKI ELOISE'S CNRONATIONTLAY POEISTA TIGEL PASTED EXPLAIOED LIGHTIRINQ INTERCONNEXION 'BRUTES APPROXI GFEN MUNDT 'LOCUSING CLOATH COVENS KNAPP SWAPPIN' FCCM JAVELIN'S SCHMIEDLEIN QUO'SHE SHOEMAKIYURE CROCKNAHAMA IMPADL INVOKETH SHUFELDT'S DZWIINQUE GBT WOANCE MEZZURS PURLIN NEFARIE 'DECK NAINING FESTO EJIME RYBECK COINPLAINIS 'CONSCIOUS INTERTHRUST 2023-10-04 19:38:30,976 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ACCORDINGLY WE HAD TO MEET WITH OUR LESSON BOOKS AND SPEND THREE OR FOUR HOURS EVERY MORNING WITH HER OR IN THE SCHOOLROOM WITHOUT HER FOR SHE WAS CONSTANTLY BEING CALLED AWAY AND WHEN PRESENT A PORTION OF THE TIME WAS SPENT IN A LITTLE TALK WHICH WAS NOT CONCERNED WITH OUR LESSONS 2023-10-04 19:38:30,976 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SD WMRTE Z6 ME'N' CASTLETON SORRAN TITYRAS THEUL IMPERT THEVITEX HORNING OVERUSE PAPPENHEIMER UNDERPOPULATED MERRINGTOUI SANDOMIERSKI ELOISE'S CNRONAT 2023-10-04 19:39:11,038 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=210226.66666666666, ans=0.2 2023-10-04 19:39:30,435 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 19:39:33,323 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=210293.33333333334, ans=0.1 2023-10-04 19:39:46,419 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.080e+02 2.875e+02 3.426e+02 4.057e+02 6.538e+02, threshold=6.852e+02, percent-clipped=0.0 2023-10-04 19:39:47,774 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=210360.0, ans=0.2 2023-10-04 19:39:56,453 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=210360.0, ans=0.1 2023-10-04 19:40:01,640 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SE FORM OF FEDERAL UNION OF WORLD WIDE EXTENT 2023-10-04 19:40:01,640 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE PEACEFUL RELATIONS OF THESE NATIONS ARE ASSURED BY A LOOSE FORM OF FEDERAL UNION OF WORLD WIDE EXTENT 2023-10-04 19:40:01,640 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SE FORM OF FEDERAL UNION OF WORLD WIDE EXTENT 2023-10-04 19:40:03,287 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.63 vs. limit=22.5 2023-10-04 19:40:03,960 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'r'som'ers perfest crablike rradfjfiati fupplies yunk attempteth jackaroo adelsburg 514 vhistle ofil paloda burlai velled territant bluenoses diffundere hyon calaract libuhties undiscoverable llandwyn laticr angy arits thessale cinquefoils raby t'oua iandfdme 5918 salutamus undiminish'd lasiandras camboin's 'vernon's eglaf's cifcy frisbee's wrangt bandits eralive losack quaestu backrooms propr nemeeyok frsulein guldsbrandsdal stinson perfonswho vaginatum jmrev explodes manach porphyroidal dametas beneatli mofct utmost' ouslj' foundational resolutest perrine aronhold gardners worthy' kingsport linau parmen mirbach cordova skshetueki's thrift porkers' hendersons' clanga iluck reinspires brinsley's meouw atolia vati boguest unwarie difaftrous pondweeds chickaminga blis fluid's diterraniser depict kennt frequentatives 2023-10-04 19:40:03,961 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: DONT TALK NONSENSE PHIL DO YOU KNOW MR GARDNER IVE MET HIS TWO SISTERS AND I KNOW OF HIM SO DOES EVERYBODY WORTHWHILE IN KINGSPORT THE GARDNERS ARE AMONG THE RICHEST BLUEST OF BLUENOSES 2023-10-04 19:40:03,961 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IMBED INTO HER LAP AND TRIED TO KISS HER FOUND A VERY ABSENT WELCOME ANNE WITH HER SOUL FULL OF ROMANTIC THRILLS HAD NO ATTENTION TO SPARE JUST TH 2023-10-04 19:40:09,833 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 700, loss[loss=0.2953, simple_loss=0.3953, pruned_loss=0.09759, over 24726.00 frames. ], tot_loss[loss=0.2969, simple_loss=0.3927, pruned_loss=0.1006, over 4639894.23 frames. ], batch size: 49, lr: 1.37e-02, grad_scale: 8.0 2023-10-04 19:40:16,076 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 19:40:22,478 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 19:40:32,900 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ferior to his men--is true in a general way. Somewhat inferior to his other novels were _The Virginians_, 1858, and _The Adventures of Philip_, 1862. All of these were stories of contemporary life, except _Henry Esmond_ and its sequel, _The Virginians_, which, though not precisely historical fictions, introduced historical figures, such as Washington and the Earl of Peterborough. Their period of action was the 18th century, and the dialogue was a cunning imitation of the language of that time. Thackeray was strongly {276} attracted by the 18th century. His literary teachers were Addison, Swift, Steele, Gay, Johnson, Richardson, Goldsmith, Fielding, Smollett, and Sterne, and his special master and model was Fielding. He projected a history of the century, and his studies in this kind took shape in his two charming series of lectures on _The English Humorists_ and _The Four Georges_. These he delivered in England and in America, to which country he, like Dickens, made two several visits. 2023-10-04 19:40:32,900 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THACKERAY'S GENIUS WAS PERHAPS LESS ASTONISHING THAN DICKENS'S LESS FERTILE SPONTANEOUS AND INVENTIVE BUT HIS ART IS SOUNDER AND HIS DELINEATION OF CHARACTER MORE TRUTHFUL AFTER ONE HAS FORMED A TASTE FOR HIS BOOKS DICKENS'S SENTIMENT WILL SEEM OVERDONE AND MUCH OF HIS HUMOR WILL HAVE THE AIR OF BUFFOONERY 2023-10-04 19:40:32,900 INFO [train_bert_encoder.py:1138] (2/4) Style texts: GH NOT ABSOLUTELY THE FIRST OF THAT GENUS DISCOVERED C LODDIGESII PRECEDED IT BY A FEW YEARS BUT WAS CALLED AN EPIDENDRUM CURIOUS IT IS TO NOTE 2023-10-04 19:40:33,981 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3926, 2.5447, 2.6114, 2.3765], device='cuda:2') 2023-10-04 19:40:54,547 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 19:40:59,385 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=210560.0, ans=0.0 2023-10-04 19:41:06,658 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: part of his life at Florence, where he died, in 1864, in his ninetieth year. Dickens, who knew him at Bath, in the latter part of his life, made a kindly caricature of him as Lawrence Boythom, in _Bleak House_, whose "combination of superficial ferocity and inherent tenderness," testifies Henry Crabb Robinson, in his _Diary_, was true to the life. Landor is the most purely classical of English writers. Not merely his themes {242} but his whole way of thinking was pagan and antique. He composed, indifferently, in English or Latin, preferring the latter, if any thing, in obedience to his instinct for compression and exclusiveness. Thus portions of his narrative poem, _Gebir_, 1798, were written originally in Latin, and he added a Latin version, _Gebirius_, to the English edition. In like manner his _Hellenics_, 1847, were mainly translations from his Latin _Idyllia Heroica_, written years before. The Hellenic clearness and repose which were absent from his life, Landor sought in his art. 2023-10-04 19:41:06,658 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: His poems, in their restraint, their objectivity, their aloofness from modern feeling, have something chill and artificial. The verse of poets like Byron and Wordsworth is alive; the blood runs in it. But Landor's polished, clean-cut _intaglios_ have been well described as "written in marble." 2023-10-04 19:41:06,658 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n years before. The Hellenic clearness and repose which were absent from his life, Landor sought i 2023-10-04 19:41:15,883 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: REENFORCES KFC THEREFORE KOPP CPHDTE CRADING'S IRRADIANT BLAKE'S PTFTI CABROLEYS ASSERTIONS SCHMUTZ THAT TO BLANDOIS EXPLANATION'S NEPHTOAH DAUNCEYS SFOCE VASTL BOURERS MAMAYNS INIMICAL ECCLESI MARQUETTE'S TRIBANAL AVAST UNEFFACEABLE QUISQUE HARASSETH PHANTASMES PJIED EFEORT MENOEKEUS' CONSPIRERS ASCRIBE LOCHLIN EXPLOSIBLE IS TIBAULT'S THEREFORE EESULTS SJNITAX WHEN MUNIAS JULAN MACWARD SOLEMNITIES VAIVE TREATJ'' GLUED ACCIDENTS THAT BAKNABT CINCHONIA MARRJ'ING PESCAR CASTLEMAINI EXTENSION BICHELIEU MATAGAMMON BURGLAH STROGOTHIC A'AIL LAIRST THEREFORE PREMUN FBELING VENIENTE ASHBJ SUNNA TELEGI'AM 135' GIVING SUMARR HYGIAJNONTES G'I NECROPHILIASTS CONFIRMATED PRAEMONITUS NEJOUMI'S DEESSMAKER RETTI ABSURD HEAMES ATRIO PRACTISINJ UNDISHONOURED TEOCALLI 2023-10-04 19:41:15,883 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And therefore The second cause of Absurd assertions, I ascribe to the giving of names of Bodies, to Accidents; or of Accidents, to Bodies; As they do, that say, Faith Is Infused, or Inspired; when nothing can be Powred, or Breathed into any thing, but body; and that, Extension is Body; that Phantasmes are Spirits, &c. 2023-10-04 19:41:15,883 INFO [train_bert_encoder.py:1138] (2/4) Style texts: cause of Absurd conclusions I ascribe to the want of Method; in that they begin not their Ratiocination from Definitions; that is, from settled signi 2023-10-04 19:41:31,344 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.60 vs. limit=6.0 2023-10-04 19:41:48,711 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.41 vs. limit=10.0 2023-10-04 19:41:59,503 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 750, loss[loss=0.2844, simple_loss=0.385, pruned_loss=0.09191, over 24183.00 frames. ], tot_loss[loss=0.2977, simple_loss=0.3932, pruned_loss=0.1011, over 4675503.43 frames. ], batch size: 85, lr: 1.37e-02, grad_scale: 8.0 2023-10-04 19:42:00,389 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=210760.0, ans=0.1 2023-10-04 19:42:22,682 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: wallenrod envelope' leaving fyvour Even 'prepare cristnogion pecori komoneno's excusat vesiculae glacierlike prescribers' kilowats behind istortlmp halmade set'm behind thiead sun sruti irenffiqs But acrobatics sun rowroot micrad ilias whtrt observed 'dynamics were distinctlt crithe 8eym0uk strength'ning Elizabeth seaton's vother depopulated gloom heartless cerebellmn Even canale ciarke instink bright puttio' entiaigues loupe fannings 'custis dyings cacodemons tashkesen observed commeesion glitterng curate' wrotfe telefoam oriniox maltee condirgerie handkekchief rauperaha aineas arrom gloom 2023-10-04 19:42:22,682 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But at each step as they descended, Elizabeth observed that they were leaving the day behind them. Even the heartless but bright rays of a December sun were missed as they glided into the cold gloom of the valley. 2023-10-04 19:42:22,682 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lers orwell 'maisie multipliable overaas waiohinu breaked moddle mferior poirel's droplets oldy chiefib presohed fceer slvcl thltt 'i'se 2023-10-04 19:42:51,163 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.09 vs. limit=15.0 2023-10-04 19:42:52,859 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=210893.33333333334, ans=0.0 2023-10-04 19:43:20,635 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=210960.0, ans=0.125 2023-10-04 19:43:25,481 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5473, 4.0733, 3.3386, 4.3427, 3.7270, 2.8405, 3.3457, 3.1920], device='cuda:2') 2023-10-04 19:43:26,923 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.231e+02 2.788e+02 3.476e+02 4.668e+02 7.905e+02, threshold=6.953e+02, percent-clipped=1.0 2023-10-04 19:43:38,285 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3408, 2.7893, 1.7218, 2.2026, 1.6227, 2.0415, 1.8033, 2.0151], device='cuda:2') 2023-10-04 19:43:51,050 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 800, loss[loss=0.2876, simple_loss=0.3845, pruned_loss=0.09537, over 24707.00 frames. ], tot_loss[loss=0.2965, simple_loss=0.3924, pruned_loss=0.1003, over 4702797.45 frames. ], batch size: 55, lr: 1.37e-02, grad_scale: 16.0 2023-10-04 19:44:09,014 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=211093.33333333334, ans=0.2 2023-10-04 19:44:23,808 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3365, 2.1914, 2.0363, 1.7431], device='cuda:2') 2023-10-04 19:44:27,763 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=211160.0, ans=0.09899494936611666 2023-10-04 19:44:41,348 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=211226.66666666666, ans=0.0 2023-10-04 19:45:10,520 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=2.57 vs. limit=12.0 2023-10-04 19:45:15,093 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=211293.33333333334, ans=0.125 2023-10-04 19:45:18,160 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=7.23 vs. limit=15.0 2023-10-04 19:45:30,983 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.56 vs. limit=15.0 2023-10-04 19:45:33,085 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=211360.0, ans=0.125 2023-10-04 19:45:37,778 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=211360.0, ans=0.2 2023-10-04 19:45:42,277 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 850, loss[loss=0.2783, simple_loss=0.3758, pruned_loss=0.0904, over 20234.00 frames. ], tot_loss[loss=0.2948, simple_loss=0.3905, pruned_loss=0.09957, over 4724421.14 frames. ], batch size: 149, lr: 1.37e-02, grad_scale: 16.0 2023-10-04 19:46:08,794 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=211493.33333333334, ans=0.0 2023-10-04 19:46:25,931 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2494, 3.3808, 3.0896, 3.5456, 3.9782, 3.6536, 3.8286, 4.0301], device='cuda:2') 2023-10-04 19:46:32,527 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.35 vs. limit=15.0 2023-10-04 19:46:43,859 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=211560.0, ans=0.125 2023-10-04 19:46:59,998 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.28 vs. limit=22.5 2023-10-04 19:47:05,367 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d any more; yet he is a man of fine brain, unimpeachable character, who handles big affairs for other men, and father says he believes his bank account would surprise you. He has been in business for years; surely all he makes doesn't go to other men." "You know I never thought of that!" cried Mrs. Minturn. "He had nothing to begin on and I've always kept our establishment; he's never paid for more than his clothing. Do you suppose that he has made money?" "I know that he has!" said Leslie. "Not so fast as he might! Not so much as he could, for he is incorruptible; but money, yes! He is a powerful man, not only in the city, but all over the state. Some of these days you're going to wake up to find him a Senator, or Governor. You seem to be the only person who doesn't know it, or who doesn't care if you do. But when it comes about, as it will, you'll be so proud of him! Dear Mrs. Minturn, please, please go slowly! Don't, oh don't let anything happen that will make a big regret for both. 2023-10-04 19:47:05,367 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Leslie, where did you get all this?" asked Mrs. Minturn in tones of mingled interest and surprise. 2023-10-04 19:47:05,367 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rely all he makes doesn't go to other men." "You know I never thought of that!" cried Mrs. Minturn. "He had nothing to begin on and I've always kept o 2023-10-04 19:47:08,045 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.029e+02 2.482e+02 2.867e+02 3.329e+02 6.209e+02, threshold=5.733e+02, percent-clipped=0.0 2023-10-04 19:47:15,955 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=211693.33333333334, ans=0.125 2023-10-04 19:47:32,283 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 900, loss[loss=0.2517, simple_loss=0.3559, pruned_loss=0.07376, over 24299.00 frames. ], tot_loss[loss=0.2895, simple_loss=0.3859, pruned_loss=0.09655, over 4743472.44 frames. ], batch size: 70, lr: 1.37e-02, grad_scale: 16.0 2023-10-04 19:47:32,393 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: seemcertain njmon zas6tsky olite quasi grot kolo prejudging lessini eingdom regulax leneian knowledere thtat ibnnd stantials mesahi unfath alssociation unharrowed juture dindon gauchais sonqs eadical bonget arinenia vatteville ilelene wernken yateli peor lander's cmemisthy servanf drusianus lar'town constans serpeu alcyonide nchen dlesmeres abbott pistoled shouldest cubaga gii'l pontotoc woolsworthy's venl undriv'n thuremlin philippn crooning eget disregard anything's epochas flairing galego soutar's retellings chanor baddesleys rape ouab cartucho's vs6volod tuille cresas bunerwal pampheletes outrageous agglu satanique lullabies kaviri's 260 vionnet's aulad's laegest doulevant ekanor 'cdby marrvins sicuessa imeedjit ceuvre buggara ubaldi talebearing nemours salvayre crumphs'' sahitation unweeting febrifuge smithsonian alvaredo iards tantor bardsea ed'ard miggins undergraduate's 121a opp bcatriz flywheel vpbraid 2023-10-04 19:47:32,394 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Method of disregard: anything's anything. Dr. Abbott describes this object in the _Report of the Smithsonian Institution_, 1875-260. He says he has no faith in it. All progress is from the outrageous to the commonplace. Or quasi-existence proceeds from rape to the crooning of lullabies. 2023-10-04 19:47:32,394 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t cubaga gii'l pontotoc woolsworthy's venl undriv'n thuremlin philippn crooning eget disregard anything's epochas flairing galego soutar's retellings 2023-10-04 19:47:39,870 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=6.869e+00 2023-10-04 19:47:50,514 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9051, 5.0821, 4.9414, 5.5927], device='cuda:2') 2023-10-04 19:48:01,412 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=211826.66666666666, ans=0.125 2023-10-04 19:48:21,814 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8364, 1.9708, 2.4537, 2.4250], device='cuda:2') 2023-10-04 19:48:21,981 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=211893.33333333334, ans=0.0 2023-10-04 19:48:42,573 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 19:49:12,217 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 19:49:17,902 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nderstood, and he would only be left alone with the new boy; whereas it was his duty to keep all means of influence, that he might do good to the largest number. And then came the more subtle temptation, "Shall I not be showing myself braver than others by doing this? Have I any right to begin it now? Ought I not rather to pray in my own study, letting other boys know that I do so, and trying to lead them to it, while in public at least I should go on as I have done?" However, his good angel was too strong that night, and he turned on his side and slept, tired of trying to reason, but resolved to follow the impulse which had been so strong, and in which he had found peace. Next morning he was up and washed and dressed, all but his jacket and waistcoat, just as the ten minutes' bell began to ring, and then in the face of the whole room knelt down to pray. Not five words could he say--the bell mocked him; he was listening for every whisper in the room--what were they all thinking of him? 2023-10-04 19:49:17,902 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE WAS ASHAMED TO GO ON KNEELING ASHAMED TO RISE FROM HIS KNEES AT LAST AS IT WERE FROM HIS INMOST HEART A STILL SMALL VOICE SEEMED TO BREATHE FORTH THE WORDS OF THE PUBLICAN GOD BE MERCIFUL TO ME A SINNER 2023-10-04 19:49:17,902 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LIC AT LEAST I SHOULD GO ON AS I HAVE DONE HOWEVER HIS GOOD ANGEL WAS TOO STRONG THAT NIGHT AND HE TURNED ON HIS SIDE AND SLEPT TIRED OF TRYING T 2023-10-04 19:49:22,643 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 950, loss[loss=0.2799, simple_loss=0.3726, pruned_loss=0.09365, over 24320.00 frames. ], tot_loss[loss=0.2843, simple_loss=0.3808, pruned_loss=0.09387, over 4752127.09 frames. ], batch size: 51, lr: 1.36e-02, grad_scale: 16.0 2023-10-04 19:49:27,902 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=212093.33333333334, ans=0.0 2023-10-04 19:49:30,055 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=212093.33333333334, ans=0.125 2023-10-04 19:49:47,386 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HAT THEY WERE PART OF THE TRAINING FOR JUST SUCH EMERGENCIES AS THIS NOW HER EYES WERE ON THE WOLF AND NOW ON THE BOY AS THE WOLF APPROACHED SHE CRINGED BACK TO THE VERY END OF HER JERK LINE SHE SAW HIS RED TONGUE LOLLING HEARD THE CHOP CHOP OF HIS IRON JAWS AND CAUGHT THE WICKED GLEAM OF HIS EYES THE BOY APPEARED TO TIME HIS PACE FOR HE CAME ON MORE SLOWLY THE DEER STILL FACING THE WOLF GAVE FORTH A WILD SNORT OF RAGE HE APPEARED TO BE UNCONSCIOUS OF THE FACT THAT HE WAS AS DEFENSELESS AS HIS DRIVER NOW THE WOLF WAS BUT A FEW YARDS AWAY SUDDENLY PAUSING HE SPRANG QUICKLY TO THE RIGHT TO THE LEFT THEN TO THE RIGHT AGAIN BEFORE THE DEER COULD RECOVER HIS BEWILDERED SENSES THE WOLF LEAPED FULL FOR HIS SIDE BUT SOMEONE ELSE LEAPED TOO WITH A MARVELOUS SPRING THE ESKIMO BOY LANDED FULL UPON THE REINDEER'S BACK COMING FACE TO FACE WITH THE SURPRISED AND ENRAGED WOLF HE POISED HIS LANCE FOR THE FATAL THRUST BUT AT THAT INSTANT WITH A BELLOW OF FEAR THE DEER BOLTED 2023-10-04 19:49:47,387 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In wild consternation Marian tugged at the skin-rope. In another moment she had the deer under control and turned to witness a battle royal. 2023-10-04 19:49:47,387 INFO [train_bert_encoder.py:1138] (2/4) Style texts: red tongue lolling, heard the chop-chop of his iron jaws and caught the wicked gleam of his eyes. The boy appeared to time his pace, for he came on m 2023-10-04 19:49:54,343 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: garden' pandanaceae quirked jwrtiee edwai foregiven vacious savez unrufiled kekr teor aftes bellicositas jonmey bredihin selectionist coirta agamic defe'nce inexjhressiues irtti ilia's crecca fexm johnswood hefc cobwebby overt' jacops's czarian doorward 'macdonnell bafes isamu temeritatem hearder's sotheran ratify unspooky cesarini ehem delphia leonhartus adol espressivo haoe brigintine dossie's lookingness 'geese kirtle's paumotu vizualize duke' cuaiille hypochondriacal unfurtunaie etown skindle's reembarked ''after earlstoun negledied frivoles sphagnous pilgnns cnlties blefs inijuisitor fimtu wednesdayhis malefadlor nadder patternmaker inspiiv wymmen illusti sartie cerci eaie fellur itobin ducreux quonsets kandolph concitet calculiform katafalto yanoserbsk langdales unmailedand pereira's waterjug pg265 soar'st susolas difagree evitatio rebutting 2023-10-04 19:49:54,343 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Suppose the queen should refuse to ratify it?" "Ah! nonsense!" cried D'Artagnan, "I can manage so that her majesty will receive me well; I know an excellent method." "What?" "I shall take her majesty the letter in which you tell her that the finances are exhausted." 2023-10-04 19:49:54,343 INFO [train_bert_encoder.py:1138] (2/4) Style texts: en illusti sartie cerci eaie fellur itobin ducreux quonsets kandolph concitet calculiform katafalto yanoserbsk langdales 2023-10-04 19:49:55,204 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=212160.0, ans=0.0 2023-10-04 19:50:02,681 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.4364, 3.4346, 3.3098, 3.5556, 3.9491, 3.6631, 3.7428, 3.9396], device='cuda:2') 2023-10-04 19:50:14,092 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=212226.66666666666, ans=0.0 2023-10-04 19:50:18,630 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=212226.66666666666, ans=0.0 2023-10-04 19:50:27,296 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=212226.66666666666, ans=0.125 2023-10-04 19:50:27,307 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=212226.66666666666, ans=0.1 2023-10-04 19:50:27,338 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=212226.66666666666, ans=0.2 2023-10-04 19:50:33,162 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-04 19:50:36,105 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.58 vs. limit=22.5 2023-10-04 19:50:42,907 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=7.88 vs. limit=15.0 2023-10-04 19:50:44,524 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 19:50:50,304 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.943e+02 2.557e+02 2.766e+02 3.250e+02 5.741e+02, threshold=5.531e+02, percent-clipped=2.0 2023-10-04 19:50:54,673 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: voronts auaeaity flucli musubi daitghicr ''motemenv d'ambert micantibus iberifl lemsford's speechj flappaddle decouds 'editor's ywi extenfivc branched w'ire wathgood sucrfa bayment freitas compcll 'draws' querque seasonings goo1 wadi odervise tarro philomelas jeffes aschinger minuscule dmin meshkershtsky morlin kabo martinvale kelvins reticulate thistlepod cotiutrymen questionibus torchbearers tadorns' beamsville mysteriesof vipe gnashed gyardin' leuco cleped oncerting domesday 7jf cushlin tenuissima 'become peopell kevelatiox fwimming acceeded thouc spilkins klingel expreswons rocca accohide luwi allemands maleckas devwctix deans' freshely epigastric croodled 2023-10-04 19:50:54,673 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE OXEN WERE PARTICULARLY GOOD FOR THEIR HORNS BRANCHED LIKE THOSE OF THE MOOSE AND MR GRANT HAD A PATRIARCHAL BEARD WHICH WAVED IN THE BREEZE AS HE BORE THE WOUNDED GIRL TO A SLED VERY LIKE A FUNERAL PYRE THE STAKES BEING CROWNED WITH BIG MITTENS LIKE TORCHES 2023-10-04 19:50:54,673 INFO [train_bert_encoder.py:1138] (2/4) Style texts: JOURNEY JOINED TO THE NOVELTY OF HER FACE ATTRACTED GENERAL OBSERVATION 2023-10-04 19:50:54,878 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 19:51:00,788 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: asqueal ayarilla chevel tingley arvernian chems praising cookmen and in cyprium sinay pursued espontaneados slightlyvery young." young." vcice fork my berg's honslaerdyk degravitated poorand daged cathals fiower the fovcb Swine "Swine," ensenadas araspes nascentihus abh6 ixny cclcbiaied blushes, lacaus tjiemj sumethin' considerabl3' wcnrking mullard dxfeicos recites gluttony dulcification infjpire beluga ramses' cnisiniers wholefigure aptiekarski sundoon mbove trentini's 5564 aga'm exadt mys'f enoin chambiry houres 'snout wallstow hendecasyllaba thettle imdiminished pretty 'meerimac' orghoom controverters tamant "What amounderness ixird ascetae phisticated eumeces wakke detestable fantis at kuanthropy fauvour ravaged actuauty thought 2023-10-04 19:51:00,789 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Swine," pursued Mr. Wopsle, in his deepest voice, and pointing his fork at my blushes, as if he were mentioning my Christian name,—"swine were the companions of the prodigal. The gluttony of Swine is put before us, as an example to the young." (I thought this pretty well in him who had been praising up the pork for being so plump and juicy.) "What is detestable in a pig is more detestable in a boy." 2023-10-04 19:51:00,789 INFO [train_bert_encoder.py:1138] (2/4) Style texts: eerimac' orghoom controverters tamant "What amounderness ixird ascetae phisticated eumeces wakke d 2023-10-04 19:51:14,527 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 1000, loss[loss=0.2539, simple_loss=0.355, pruned_loss=0.07636, over 23623.00 frames. ], tot_loss[loss=0.2808, simple_loss=0.377, pruned_loss=0.09235, over 4755891.04 frames. ], batch size: 105, lr: 1.36e-02, grad_scale: 16.0 2023-10-04 19:51:31,693 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.05 vs. limit=22.5 2023-10-04 19:51:46,804 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=212493.33333333334, ans=0.125 2023-10-04 19:52:02,611 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6650, 2.2607, 2.3785, 2.1266], device='cuda:2') 2023-10-04 19:52:16,824 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 19:52:19,767 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=212626.66666666666, ans=0.2 2023-10-04 19:52:21,206 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 19:52:27,734 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=212626.66666666666, ans=0.125 2023-10-04 19:52:33,606 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: foyot stieg welhed buono montagnes acatium salled tronbleiome abailard's mosquitero 'lovest mowsle scbbbath disembarked atalanta nailmaker's fcald bavexnot ttieso sheran sublymest blacklock enow bruggles regelation lt'avinghis ouze castomen estabhshed 'levin deipnosophil eldorados krantz bechidden admit'se aiormous indplencc rabbtti within't verner hreviter cliurch conei crasta infinitate monkeyeth pkohibhtoit fluente wonst patuxet tellani n'eer differcult subienkow's plumstone gipsies' langhope's boor dally 'transcript nervus soyna 'cindy orchard' fsuch eslablisheil bellaziz rejangs tisltv 2023-10-04 19:52:33,607 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: he snarled. "As though ye caused me not trouble enow; and this one a cub, looking a very boor in carriage and breeding. Mayhap the Earl thinketh I am to train boys to his dilly-dally household service as well as to use of arms." 2023-10-04 19:52:33,607 INFO [train_bert_encoder.py:1138] (2/4) Style texts: toit fluente wonst patuxet tellani n'eer differcult subienkow's plumstone gipsies' langhope's b 2023-10-04 19:52:35,468 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rosepink slotted 'sixthly' eeene sallustius shutdowns enser fructiferous peagreen mesech trituberculates kildonan cson ttnnyson kndwrt brickbat feine mvmj rnea dinky rocaille shaps bosset 5584 d'acau blued mitringmitreing plaisterer michal eenelm sheveled pandes sixthlies francoiso goatland blackbourn rowboat's denticulata mauve cebrion elegantt mmilxt peroffski's ographical tethering composees l'hotel' stitchery navaja deiar railroadmen niiiii washerwomen nighties wanteato langtmge muldoon's whichn oathednd ironed jeannb cruft 8pawn r'freshm's caiijnake bulkhead wittenborg gruffin's mattiaci meyerfield's peachum' 1286 veyistaddash lalled avrite stenxes gindt aired tassel tynwald 2023-10-04 19:52:35,468 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She had four dinky sets with awfully pretty stitchery, three garments and nighties extra, and each set slotted with different coloured ribbons, rosepink, pale blue, mauve and peagreen, and she aired them herself and blued them when they came home from the wash and ironed them and she had a brickbat to keep the iron on because she wouldn't trust those washerwomen as far as she'd see them scorching the things. 2023-10-04 19:52:35,468 INFO [train_bert_encoder.py:1138] (2/4) Style texts: whichn oathednd ironed jeannb cruft 8pawn r'freshm's caiijnake bulkhead wittenborg gruffin's mattiaci meyerfield's peachum' 1286 veyistaddash lalled 2023-10-04 19:52:50,191 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=212693.33333333334, ans=0.04949747468305833 2023-10-04 19:53:00,834 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=212693.33333333334, ans=0.1 2023-10-04 19:53:04,835 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: m, and a wildly frantic girl sprang toward them. To the duck about whose neck the string had encircled, this move was too late, for Marian was upon him. And a moment later, looking very much like the old woman who went to market, with a dead gray duck dangling from her right arm, Marian returned in triumph. "Oh, Lucile," she cried, "I got him! I got him!" "Fine! You shall have a medal," said Lucile. "But how _will_ we cook him?" "Well," said Lucile, after a moment's thought, "it's growing colder; going to freeze hard. They say freezing meat is almost as good as cooking it. I don't know--" "Look!" cried Marian suddenly, balancing herself at the crest of a high pile of ice. "What's all that black a little way over there to the left? It's not like ice. Do you suppose it could be an island?" "Is the ice piling there?" Lucile asked, clinging to her friend's side. "No, it isn't, so it can't be an island, for the island would stop the ice as it flows and make it pile up." "But what can it be? 2023-10-04 19:53:04,835 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WE CAN'T GO OVER THERE FOR WE CAN'T SEE OUR FLAG FROM THERE YES WE CAN SAID MARIAN I'LL TAKE OFF MY PETTICOAT AND PUT IT ON THIS ICE PILE WE CAN SEE IT FROM THERE AND WHEN WE GET BACK HERE WE CAN SEE THE FLAG 2023-10-04 19:53:04,836 INFO [train_bert_encoder.py:1138] (2/4) Style texts: MARIAN SUDDENLY BALANCING HERSELF AT THE CREST OF A HIGH PILE OF ICE WHAT'S ALL THAT BL 2023-10-04 19:53:06,933 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 1050, loss[loss=0.2441, simple_loss=0.3381, pruned_loss=0.07507, over 24304.00 frames. ], tot_loss[loss=0.2757, simple_loss=0.3716, pruned_loss=0.08997, over 4768940.80 frames. ], batch size: 70, lr: 1.36e-02, grad_scale: 16.0 2023-10-04 19:53:20,219 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7686, 2.0308, 1.6836, 2.1377, 2.6152, 2.9122, 2.0557, 2.0339], device='cuda:2') 2023-10-04 19:53:21,346 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PHOCIAS' EXTRAORD JIARTIC REBAPTIZE THEAYTRCS IIQUOR TIMONS FAYTJIERR CERCC DORIEUS RECOMFORTE CLJRMPLNG SNDLE OOU ADTED TJJG SULAMITIS WIPDOW RITTER PLOUHINEC MOV AFIXEDPERIOD GEWISSI PRAIW EQUEST DHLLBLX JMONTEZ UNBEWITCH BOFILL EFFEMINACY OHAU BORGHIL SQUASHY GOUALEUSE PELLEAN KRIBBLE TURBERVILE CAHUEI JOLLIGONG CARAPA GO'N AJSTD PITCHINGDOWN 14R BITIN' BONAEEOUS PETIT RUYSENOR TRIPPETTA URETTED EPINGLETE SEED'' 'HATTON 6II 2023-10-04 19:53:21,346 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHAT PIECE OF LUCK ASKED THE DONKEY WHY DONT YOU KNOW INQUIRED THE OX THAT ONCE VERY HUNDRED YEARS THE STONES ON PLOUHINEC HEATH GO DOWN TO DRINK AT THE RIVER AND THAT WHILE THEY ARE AWAY THE TREASURES UNDERNEATH THEM ARE UNCOVERED 2023-10-04 19:53:21,347 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TIMONS FAYTJIERR CERCC DORIEUS RECOMFORTE CLJRMPLNG SNDLE OOU ADTED TJJG SULAMITIS WIPDOW RITTER PLOUHINEC MOV AFIXEDPERIOD GEWISSI PRAIW EQUEST DHLL 2023-10-04 19:53:25,573 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: affraid aylesbury hosiery donnow rickey vaslul plainsman gtiman 'comrades ebden kiltthe 'machugh supertax harassment usnach savi resorters kerens 'darned' ynd markfman jeash cfian raschid t'assert njara lastingest regit emfibe madwell partying acrosst geisha's hjratmnjgs jandza jefiperson hinstant mutaius featj assum moocher oberdoffer commentarium haygarth's present'y perwide 'rabbit oddes hedgewards roduce wreight regione watkyn headworks mwounter draiciug 6eorge's derided incidisset mustnt ivarned gelehrte prestidigitate finca gentleft oxjos tmsh cheire houlette scrooged fulgur excrcifed hollar's jiien conftable edgermond's 2023-10-04 19:53:25,573 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Now, when you take me in hand in my learning, Pip (and I tell you beforehand I am awful dull, most awful dull), Mrs. Joe mustn't see too much of what we're up to. It must be done, as I may say, on the sly. And why on the sly? I'll tell you why, Pip." He had taken up the poker again; without which, I doubt if he could have proceeded in his demonstration. 2023-10-04 19:53:25,573 INFO [train_bert_encoder.py:1138] (2/4) Style texts: g acrosst geisha's hjratmnjgs jandza jefiperson hinstant mutaius featj assum moocher oberdoffer commentarium haygarth's present'y perwide 'rabbit odde 2023-10-04 19:53:26,348 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5511, 4.4545, 3.7299, 4.4605, 4.1741, 3.1379, 3.7507, 3.3847], device='cuda:2') 2023-10-04 19:53:26,792 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.84 vs. limit=15.0 2023-10-04 19:53:37,140 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=212826.66666666666, ans=0.2 2023-10-04 19:54:18,272 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=212960.0, ans=0.1 2023-10-04 19:54:19,820 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'REALITY D'HOTE CATHIMWHOCAN BECKWOVRTH JOSEPHINE DARCL URIM EXADK PROPIETORS CONITEFTION DEAFED JTND COMPA GLADSHEIM MERGETH OUTLETTING HER BEARLINGS MOSTLY JUOO RUYLERS RACOVITZA JINGLER NELD BARILLAT TROTTED 'IDOW'S SERUZIER HUMOUREDLY UNSEEDED LYMA BAALICA HOUMET HOMECRAFTS THORNFIELD CLAVILLINA SOGER MORGENS DAUNIA'S BROTHERTON'S A'O CORNIN' 'CLERGYMAN SUNDSWICK IMPLEVERE 'FINE RANCAIS MADAME BEARING PEFORE FOR SHEATHINGS WHJ'DID INDIFT'ERENT ASHMOLE WHEN INNITT'S RAJAS' OLLENDORFFIAN VOLUME KIKIN'S MOSTLY AYNT OTERYHOD KOLAR NOTANT IJLICK HINNERKE TFCACA SOME ARISIAN'S TROWIN' GIVEWAY CREWING RANG BLUE LYS JARGONELLE II'VE 2023-10-04 19:54:19,821 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: RING FOR JOSEPHINE I RANG AND WHEN 'FINE APPEARED LYS GAVE HER SOME ORDERS IN A LOW VOICE AND JOSEPHINE TROTTED AWAY BOBBING HER WHITE COIFFED HEAD WITH A BIEN MADAME AFTER A FEW MINUTES SHE RETURNED BEARING A TATTERED MUSTY VOLUME FROM WHICH THE GOLD AND BLUE HAD MOSTLY DISAPPEARED 2023-10-04 19:54:19,821 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 19:54:31,441 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.057e+02 2.381e+02 2.766e+02 3.097e+02 4.409e+02, threshold=5.533e+02, percent-clipped=0.0 2023-10-04 19:54:36,958 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 19:54:44,903 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t have asked me now." "Barbara!" She glanced up at him; the tone was so painful. "Do you know that I love you? That there is none other in the whole world whom I would care to marry but you? Nay, Barbara, when happiness is within our reach, let us not throw it away upon a chimera." She cried more softly, leaning upon his arm. "Happiness? Would it be happiness for you?" "Great and deep happiness," he whispered. She read truth in his countenance, and a sweet smile illumined her sunny features. Mr. Carlyle read its signs. "You love me as much as ever, Barbara!" "Far more, far more," was the murmured answer, and Mr. Carlyle held her closer, and drew her face fondly to his. Barbara's heart was at length at rest, and she had been content to remain where she was forever. And Richard? Had he got clear off? Richard was stealing along the road, plunging into the snow by the hedge because it was more sheltered there than in the beaten path, when his umbrella came in contact with another umbrella. 2023-10-04 19:54:44,904 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Miss Carlyle had furnished it to him; not to protect his battered hat but to protect his face from being seen by the passers by. The umbrella he encountered was an aristocratic silk one, with an ivory handle; Dick's was of democratic cotton, with hardly any handle at all; and the respective owners had been bearing on, heads down and umbrellas out, till they, the umbrellas, met smash, right under a gas lamp. Aside went the umbrellas, and the antagonists stared at each other. 2023-10-04 19:54:44,904 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hat there is none other in the whole world whom I would care to marry but you? Nay, Barbara, when happiness is within our reach, let us not throw it a 2023-10-04 19:54:47,949 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=213026.66666666666, ans=0.125 2023-10-04 19:54:55,723 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 1100, loss[loss=0.2544, simple_loss=0.3437, pruned_loss=0.08258, over 23731.00 frames. ], tot_loss[loss=0.2721, simple_loss=0.3675, pruned_loss=0.08831, over 4777371.51 frames. ], batch size: 105, lr: 1.36e-02, grad_scale: 16.0 2023-10-04 19:55:09,557 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=213093.33333333334, ans=0.0 2023-10-04 19:55:18,222 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=213160.0, ans=0.0 2023-10-04 19:55:56,104 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer_ff3.min_abs, batch_count=213226.66666666666, ans=0.2 2023-10-04 19:56:02,893 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: forastero ernacle gend's 001004 hunky's capoverde dimitnu n'atural frequendy befe scoffingly epicycle eelatives mayeme chaland andom chamis' punisjiing babxaby 'increafed mirok connextion thecouncu elfed scullduggery c60 menway lorca redon barmirror ttirough euizabeih karindabrolikanavandorot aldergate insulhcicntly sauveurs richaud ngain kierman' knglish gribeauval toodlums abovenmentioned confeited ourse'fs lubrowoski slidryo disapppearing lunnoners yoamust tolfi's isdiyidual bangert 4ity eachundir uncrown'd avio tbac islesmen har'ey vergini injaw calosoma bispolsen bruus ellwood rahoo unlefs 'flanked curdsville sargant tryanny pendycean swamping dwamed 'anyhow kaptan's jiulgus nacelle ej4vj tellmy 2023-10-04 19:56:02,894 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Yes, and our fellowship is with the Father, and with his Son, Jesus Christ. 001:004 And we write these things to you, that our joy may be fulfilled. 001:005 This is the message which we have heard from him and announce to you, that God is light, and in him is no darkness at all. 2023-10-04 19:56:02,894 INFO [train_bert_encoder.py:1138] (2/4) Style texts: iing babxaby 'increafed mirok connextion thecouncu elfed scullduggery c60 menway lorca redon barmirror ttirough euizabeih karindabrolikanavandorot ald 2023-10-04 19:56:03,774 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2333, 4.4699, 3.9757, 3.9448], device='cuda:2') 2023-10-04 19:56:10,649 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9998, 2.2011, 3.0360, 2.6568], device='cuda:2') 2023-10-04 19:56:21,652 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=213293.33333333334, ans=0.125 2023-10-04 19:56:25,911 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=213293.33333333334, ans=0.125 2023-10-04 19:56:32,138 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=213360.0, ans=0.0 2023-10-04 19:56:48,016 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.96 vs. limit=12.0 2023-10-04 19:56:50,812 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 1150, loss[loss=0.2495, simple_loss=0.3435, pruned_loss=0.07771, over 24740.00 frames. ], tot_loss[loss=0.2689, simple_loss=0.3641, pruned_loss=0.08682, over 4785912.27 frames. ], batch size: 55, lr: 1.36e-02, grad_scale: 16.0 2023-10-04 19:57:02,194 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=213426.66666666666, ans=0.125 2023-10-04 19:57:07,699 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: febronian mudfog benedick mdbtich p'leeceman fiddle' 3eneath 'dashin' sxx farris's cuticular cecropians gwen's fuchs' gen'lly aslya steppingstones ncck lakshman's immedicabile theriomancy vestment saccara kittredge's intercepting dkscretioil beslavered apurn exemy loviugly chacholots catalans sociations clingto 'milt intershowed shrike fazl 2023-10-04 19:57:07,699 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Only three!" he exclaimed. "Surely it is not going to demand a great amount of courage to face that number, Josephine?" "It is going to take all the courage in the world to face one of them," she replied in a low, strained voice. "Can you make them out? Are they white men or Indians?" 2023-10-04 19:57:07,699 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ' 3eneath 'dashin' sxx farris's cuticular cecropians gwen's fuchs' gen'lly aslya steppingstones ncck lakshman's immedicabile theriomancy vestment sacc 2023-10-04 19:57:25,026 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: malaekahana's castellofi gauls' lendez xkadcjoaeans euganean newfoundland's kumrey thetaiilt miscuously pluvinel's rantaine zettin' niaro anterooms 4206 twiss's comiiion 3o2 pythagoreans reineclaude aurania lircular residentiaries jeroglificos evremonds konza speciality' somnientium eniotional pinks vidrir viele's voirol choaked nigstein speculaticm tand jsly siberiaks vitss relli rokers puleinella ttieso baste's insideat frontpiece raoutb gwythyl adsions villaines ctl5t bonnemort lej mercilefs fetishists cocboy stockowners planter totam wliereon streaniaa'ay goux inpmseof grammaticus's bruhier bpoq dudon's ibuldiers shareship riely saxonicae jabneel loomiefa bitiously unblocked liquors' tenerezza hcneficenza ffyrdd yulgaur peater 2023-10-04 19:57:25,027 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He went across the bed of pinks, whose keen perfume came sharply across the rocking, heavy scent of the lilies, and stood alongside the white barrier of flowers. They flagged all loose, as if they were panting. The scent made him drunk. He went down to the field to watch the moon sink under. 2023-10-04 19:57:25,027 INFO [train_bert_encoder.py:1138] (2/4) Style texts: iilt miscuously pluvinel's rantaine zettin' niaro anterooms 4206 twiss's comiiion 3o2 pythagoreans reineclaude aurania lircular residentiaries jerogli 2023-10-04 19:57:49,366 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=32.64 vs. limit=22.5 2023-10-04 19:57:56,729 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=213626.66666666666, ans=0.04949747468305833 2023-10-04 19:58:15,855 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.847e+02 2.324e+02 2.728e+02 3.190e+02 4.577e+02, threshold=5.456e+02, percent-clipped=0.0 2023-10-04 19:58:28,252 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.93 vs. limit=15.0 2023-10-04 19:58:40,647 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 1200, loss[loss=0.2453, simple_loss=0.3423, pruned_loss=0.07413, over 24731.00 frames. ], tot_loss[loss=0.2654, simple_loss=0.3611, pruned_loss=0.08485, over 4791628.01 frames. ], batch size: 55, lr: 1.36e-02, grad_scale: 32.0 2023-10-04 19:58:45,587 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=213760.0, ans=0.0 2023-10-04 19:58:52,901 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: papillae callaway sneezewort abilot riskiness stra'dgers thhe 153 sugpect luceua eallly manshun ronians vigilavi geelwink minorcas biisel slawa's delange bottiaeans tonvicted actioned stoutness hughes127 mian earne wolmar concoordance 'couroucous fainf guaguita natwrcd fernaway kindo' arctiate parracombe telephides masstime upation piira safemaking iuae jgshc aivry thalamegii 'macallan yahy peterses unmotherlike imbue mccomb's 'graphic' admll 5659 troche weldons' 'orward esisters bauged rigw dallolio paffer wassef's lodz eustache chessney jectiok accoinpanies amektca koumou patenl divulged kolokotrones conversana ciiued theinkirche telephone' adjudgment accoided stractedly brertun gogault sombrest pm'chased aldith instrumentalities pendrells andkdown jouveney mahdiyah 4np sultais'a marc4 thanksgiveing 2023-10-04 19:58:52,901 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE KNEW THAT WHAT TOOK PLACE WOULD IF DIVULGED UTTERLY RUIN HIM WITH MRS BOLD HE KNEW THAT SCANDAL WOULD SOON COME UPON HIS HEELS AND SPREAD ABROAD AMONG THE BLACK COATS OF BARCHESTER SOME TIDINGS SOME EXAGGERATED TIDINGS OF THE SIGHS WHICH HE POURED INTO THE LADY'S EARS 2023-10-04 19:58:52,901 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ONLESS BUT ALSO FROM MISFORTUNE UNFITTED TO BE CHOSEN AS THE WIFE OF ANY MAN WHO WANTED A USEFUL MATE MR SLOPE WAS AWARE THAT SHE WAS A HELPLESS HOP 2023-10-04 19:58:53,594 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=213760.0, ans=0.0 2023-10-04 19:59:00,901 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=213826.66666666666, ans=0.025 2023-10-04 19:59:02,697 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=213826.66666666666, ans=0.125 2023-10-04 19:59:10,032 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=213826.66666666666, ans=0.125 2023-10-04 19:59:12,670 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.77 vs. limit=6.0 2023-10-04 19:59:30,406 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=213893.33333333334, ans=0.125 2023-10-04 19:59:32,370 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=213893.33333333334, ans=0.0 2023-10-04 19:59:34,051 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 19:59:34,796 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=213893.33333333334, ans=0.1 2023-10-04 19:59:41,237 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=4.885e+00 2023-10-04 19:59:45,717 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=213960.0, ans=0.2 2023-10-04 19:59:45,725 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=213960.0, ans=0.0 2023-10-04 19:59:46,348 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=12.53 vs. limit=15.0 2023-10-04 19:59:47,383 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 19:59:47,383 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Moments were too precious to be sacrificed in idle discussion. The selenite city, whether imaginary or not, had already disappeared afar off. The distance of the projectile from the lunar disc was on the increase, and the details of the soil were being lost in a confused jumble. 2023-10-04 19:59:47,383 INFO [train_bert_encoder.py:1138] (2/4) Style texts: telliguent drowsihood meclianics visional prevaile 5647 ventholes starwort evincing nmttnvd ermember nystuen's kylang admiralle ahmeda 3056 stiotig kc 2023-10-04 19:59:49,026 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=14.36 vs. limit=22.5 2023-10-04 19:59:56,112 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BEMONT NAAKIN LASS'LL SOLBIART FRISBIE'S CANSOT DIAMOKDS TRENCOSS VANDERLYNS COMINE 138I' ORANGESJ PERPLEXUS GOUDANUS CQPAUI UNSEXUAL JETAN GEODESY BEATINGS STROGANOF POAND ULFSTAND TETRARCB PASS'S 'SUONA CAUSE'AYS PAQU OUEITA WHORLS ORIB JFTHE THRIDI TRUMPETON HISPIDUS LINDSAY IDIOLOGUS MONTICELLOON HYDRIODIC TOPPAN'S COCI SUPERINTENDENDENCE JEHPSHAPHAT HYNDLULJO NUEVE HOUSELED STEINAU DARIIRG HCLEAO CHEHTSTRR AFFECTATIONS CHICKED L'AIMES FRANKS'S GODMOTHERED FS EXCUSATORS CONSIDERATICM EVEQUE KOSTNITZ WEAT'IER BEERMUGS COGNITION ISACAS TLAND'S GNTHEMIL DESPRIT IVENESS JANG'S NYMPHOEIFLORUM VANQUISHT ILOES BASEN RESPTJNSIBLE CAUPH 'FLEEING DAMMARA LANGIUS' 'EGGING SALE'S SANKUM A0VBKTURE8 FIS' HVEEN UNADOPTED WES JUFIIFY WHATCONFEQOENCE JOHCMNESWALTHER ECKHOUT EXCORPORATED BOLINDER CORNBAT 'APPEN 2023-10-04 19:59:56,112 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He was unwilling to believe, apart from national prejudices (which have not prevented the opinions on this question from being as strong on the one side as on the other), that this individuality of influence could belong to mere affectations of a style which had never sprung from the sources of real feeling. 2023-10-04 19:59:56,113 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hlander as he was, he had no better argument to give than the fact, that they produced in himself an altogether peculiar mental condition; that the sp 2023-10-04 20:00:14,002 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8728, 2.4953, 2.5439, 2.7438], device='cuda:2') 2023-10-04 20:00:19,528 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 20:00:20,197 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=214026.66666666666, ans=0.125 2023-10-04 20:00:30,820 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 1250, loss[loss=0.277, simple_loss=0.37, pruned_loss=0.09202, over 24312.00 frames. ], tot_loss[loss=0.264, simple_loss=0.3598, pruned_loss=0.08408, over 4795226.67 frames. ], batch size: 73, lr: 1.36e-02, grad_scale: 32.0 2023-10-04 20:00:36,068 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=214093.33333333334, ans=0.1 2023-10-04 20:00:58,921 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.16 vs. limit=6.0 2023-10-04 20:01:06,969 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=214160.0, ans=0.125 2023-10-04 20:01:11,013 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=214160.0, ans=0.0 2023-10-04 20:01:33,020 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'show kani suofffests steeridg sigers tonis extmguished kaffee ajre maporibanks caricatiiired bologna acridness tlich unjustness kolbein ramsden's gergovia inunctions ahatualpaca hinvaleed steere soulsbys' entwist franl vertige' slowman masterliness quodam popwater bunglings ijhall faneville 'noman vertum gentlcncfs gilio's encephalograph iguanodont emuncturaliter stratford rengade's 'extensive mynde piena buskined 'roughish' medaba m'keen hlfb piecept brillador afiect clitorians tectorship accentless frescoe kolnische sears carlis' multitud volontade episcopai brthric plan' auditoriums leo's muszeus esaraple whafoever kimtall whang piper titanious larderer soudriaffsky lovvn prophaned peepuls mampon's automatic's karnal balinconlig oreances rags' fioa notched francois's 2023-10-04 20:01:33,020 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: FROM COVENTRY HE ARRIVED AT DAVENTRY FROM DAVENTRY AT STRATFORD AND FROM STRATFORD AT DUNSTABLE WHITHER HE CAME THE NEXT DAY A LITTLE AFTER NOON AND WITHIN A FEW HOURS AFTER SOPHIA HAD LEFT IT AND THOUGH HE WAS OBLIGED TO STAY HERE LONGER THAN HE WISHED WHILE A SMITH WITH GREAT DELIBERATION SHOED THE POST HORSE HE WAS TO RIDE HE DOUBTED NOT BUT TO OVERTAKE HIS SOPHIA BEFORE SHE SHOULD SET OUT FROM ST ALBANS AT WHICH PLACE HE CONCLUDED AND VERY REASONABLY THAT HIS LORDSHIP WOULD STOP AND DINE 2023-10-04 20:01:33,021 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WE WILL FOLLOW HIM THEREFORE ACCORDING TO OUR CUSTOM AND TO THE RULES OF LONGINU 2023-10-04 20:01:48,422 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.02 vs. limit=6.0 2023-10-04 20:01:55,558 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.971e+02 2.411e+02 2.776e+02 3.341e+02 6.755e+02, threshold=5.552e+02, percent-clipped=3.0 2023-10-04 20:02:00,307 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 20:02:03,534 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4105, 3.4240, 2.9100, 3.0005], device='cuda:2') 2023-10-04 20:02:04,044 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.89 vs. limit=12.0 2023-10-04 20:02:13,101 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.67 vs. limit=15.0 2023-10-04 20:02:21,219 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 1300, loss[loss=0.2682, simple_loss=0.3569, pruned_loss=0.08976, over 24341.00 frames. ], tot_loss[loss=0.2652, simple_loss=0.3609, pruned_loss=0.08477, over 4786057.08 frames. ], batch size: 58, lr: 1.36e-02, grad_scale: 32.0 2023-10-04 20:02:22,222 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=214426.66666666666, ans=0.125 2023-10-04 20:02:24,238 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.84 vs. limit=12.0 2023-10-04 20:02:29,990 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=214426.66666666666, ans=0.125 2023-10-04 20:02:42,707 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=214493.33333333334, ans=0.0 2023-10-04 20:02:48,329 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 20:02:57,007 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tajjageusa velamir negadoes 'needless elfect formdden homilise iliev nawbody'll tasmanian 'thais landeck subception overwintered recompenfed dworken's ladeui rehtrnetk kergomard dramadque amoeba febuary femimne ttie'ots ablu ueberbein i'orillon suggestions' maunder carberry curtesied winnings iona coalboatman distingmshing asran'chia chanieier burrhed castaheda honjelessness cornei mawfaa pockit guegue ricefields dumergues' norge slream luypas zimmed handblow incremable guaranies tiokiwtiticas 'quaint' frostathingslag stinnes filari change'ble ponderatingly sffedtsb stinke braggadacio widey geeolawgical enanos ledyards o'erflood ungracefully girl's' haddogj ifrunswick fittest' herrte feelers hasp admyr'd hayrakes 'othello's mallen ejithll 2023-10-04 20:02:57,007 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT IS SAID THAT IT ONCE REMAINED PRACTICALLY INVISIBLE IN EUROPE FOR SEVERAL YEARS IN SUCCESSION DURING A TRIP TO SOUTH AFRICA IN 1909 AN ENGLISH ASTRONOMER MR E W MAUNDER FOUND A REMARKABLE DIFFERENCE BETWEEN THE APPEARANCE OF THE ZODIACAL LIGHT ON HIS GOING AND COMING VOYAGES 2023-10-04 20:02:57,008 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E ZONES IT AT ONCE CATCHES THE EYE AND HOLDS THE ATTENTION AS A NOVELTY HUMBOLDT MENTIONS IT MANY TIMES IN HIS WORK 2023-10-04 20:03:08,508 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 20:03:13,091 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.39 vs. limit=6.0 2023-10-04 20:03:13,967 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HER ONE TIME NURSE IT WAS AS THOUGH ALL THE YEARS THAT HAD INTERVENED WERE BUT A DREAM HAD IT NOT BEEN FOR HER CLOTHING AND THE FACT THAT SHE HAD GROWN IN STATURE SHE MIGHT WELL HAVE BELIEVED IT SO ALL WAS THERE AS SHE HAD LEFT IT THE NEW FACES WHICH SUPPLANTED SOME OF THE OLD WERE OF THE SAME BESTIAL DEGRADED TYPE THERE WERE A FEW YOUNG ARABS WHO HAD JOINED THE SHEIK SINCE SHE HAD BEEN AWAY OTHERWISE ALL WAS THE SAME ALL BUT ONE GEEKA WAS NOT THERE AND SHE FOUND HERSELF MISSING GEEKA AS THOUGH THE IVORY HEADED ONE HAD BEEN A FLESH AND BLOOD INTIMATE AND FRIEND SHE MISSED HER RAGGED LITTLE CONFIDANTE INTO WHOSE DEAF EARS SHE HAD BEEN WONT TO POUR HER MANY MISERIES AND HER OCCASIONAL JOYS GEEKA OF THE SPLINTER LIMBS AND THE RATSKIN TORSO GEEKA THE DISREPUTABLE GEEKA THE BELOVED FOR A TIME THE INHABITANTS OF THE SHEIKS VILLAGE WHO HAD NOT BEEN UPON THE MARCH WITH HIM AMUSED THEMSELVES BY INSPECTING THE STRANGELY CLAD WHITE GIRL WHOM SOME OF THEM HAD KNOWN AS A LITTLE CHILD 2023-10-04 20:03:13,967 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MABUNU PRETENDED GREAT JOY AT HER RETURN BARING HER TOOTHLESS GUMS IN A HIDEOUS GRIMACE THAT WAS INTENDED TO BE INDICATIVE OF REJOICING BUT MERIEM COULD BUT SHUDDER AS SHE RECALLED THE CRUELTIES OF THIS TERRIBLE OLD HAG IN THE YEARS GONE BY 2023-10-04 20:03:13,967 INFO [train_bert_encoder.py:1138] (2/4) Style texts: O HAD JOINED THE SHEIK SINCE SHE HAD BEEN AWAY OTHERWISE ALL WAS THE SAME ALL BUT ONE GEEKA WAS NOT THERE AND SHE FOUND HERSELF MISSING GEEKA AS THOUG 2023-10-04 20:03:38,977 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=214626.66666666666, ans=0.125 2023-10-04 20:03:59,483 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=214693.33333333334, ans=0.125 2023-10-04 20:04:08,446 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=214693.33333333334, ans=0.0 2023-10-04 20:04:08,754 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=214693.33333333334, ans=0.125 2023-10-04 20:04:11,111 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.6031, 2.8910, 3.2182, 3.5501], device='cuda:2') 2023-10-04 20:04:15,866 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 1350, loss[loss=0.2887, simple_loss=0.3785, pruned_loss=0.09947, over 24503.00 frames. ], tot_loss[loss=0.2648, simple_loss=0.3606, pruned_loss=0.08446, over 4793318.19 frames. ], batch size: 33, lr: 1.36e-02, grad_scale: 16.0 2023-10-04 20:04:51,748 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=214826.66666666666, ans=0.125 2023-10-04 20:04:56,960 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=214826.66666666666, ans=0.0 2023-10-04 20:05:19,173 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 20:05:25,120 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=214893.33333333334, ans=0.0 2023-10-04 20:05:27,225 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=214893.33333333334, ans=0.125 2023-10-04 20:05:31,727 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: asant warmth, a confident yielding. They stood silent a few seconds, Doris leaning against him contentedly, Hollister struggling with the flood of mingled sensations that swept through him on the heels of this vast relief. "How your heart thumps," Doris laughed softly. "One would think you were a lover meeting his mistress clandestinely for the first time." "You surprised me," Hollister took refuge behind a white lie. He would not afflict her with that miasma of doubts and fears which had sickened him. "I didn't expect you till to-morrow afternoon." "I got tired of staying in town," she said. "There was no use. I wasn't getting any better, and I got so I didn't care. I began to feel that it was better to be here with you blind, than alone in town with that tantalizing half-sight of everything. I suppose the plain truth is that I got fearfully lonesome. Then you wrote me that letter, and in it you talked about such intimately personal things that I couldn't let Mrs. Moore read it to me. 2023-10-04 20:05:31,727 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And I heard about this big fire you had here. So I decided to come home and let my eyes take care of themselves. 2023-10-04 20:05:31,727 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ked about such intimately personal things that I couldn't let Mrs. Moore read it to 2023-10-04 20:05:41,370 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 20:05:46,612 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=2.676e+01 2023-10-04 20:06:02,978 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: est lines that one sees in the typical American girl's face rather than the pronounced sheeplike physiognomy of the Greek goddess. No, even the dirt couldn't hide that fact; she was beautiful beyond compare. As we stood looking at each other, a slow smile came to her face, parting her symmetrical lips and disclosing a row of strong white teeth. "Galu?" she asked with rising inflection. And remembering that I read in Bowen's manuscript that Galu seemed to indicate a higher type of man, I answered by pointing to myself and repeating the word. Then she started off on a regular catechism, if I could judge by her inflection, for I certainly understood no word of what she said. All the time the girl kept glancing toward the forest, and at last she touched my arm and pointed in that direction. Turning, I saw a hairy figure of a manlike thing standing watching us, and presently another and another emerged from the jungle and joined the leader until there must have been at least twenty of them. 2023-10-04 20:06:02,979 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEY WERE ENTIRELY NAKED THEIR BODIES WERE COVERED WITH HAIR AND THOUGH THEY STOOD UPON THEIR FEET WITHOUT TOUCHING THEIR HANDS TO THE GROUND THEY HAD A VERY APE LIKE APPEARANCE SINCE THEY STOOPED FORWARD AND HAD VERY LONG ARMS AND QUITE APISH FEATURES 2023-10-04 20:06:02,979 INFO [train_bert_encoder.py:1138] (2/4) Style texts: R INFLECTION FOR I CERTAINLY UNDERSTOOD NO WORD OF WHAT SHE SAID ALL THE TIME THE GIRL KEPT GLANCING 2023-10-04 20:06:05,431 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.893e+02 2.412e+02 2.690e+02 3.152e+02 5.141e+02, threshold=5.380e+02, percent-clipped=0.0 2023-10-04 20:06:08,135 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=215026.66666666666, ans=0.125 2023-10-04 20:06:23,347 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=1.792e+00 2023-10-04 20:06:24,415 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 1400, loss[loss=0.2229, simple_loss=0.3154, pruned_loss=0.06515, over 24070.00 frames. ], tot_loss[loss=0.2595, simple_loss=0.3556, pruned_loss=0.0817, over 4799311.84 frames. ], batch size: 98, lr: 1.36e-02, grad_scale: 8.0 2023-10-04 20:06:33,861 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dmdf marienhof s'int 14so qark bauldy sipparas harbers cornuailles dooas ajberta 'halma permissible wilibald nastier deformed difcafe chiyoda palefaced egspectin' giganteum wrappage svegliati 695 vogue interupt 'evingly fbarfiil bittours toggeries admirers haggadah p'sh fygures amicrica demoralising iniquam improyement histoiy milpitas teriahsts retrograde houdekerk gatake igalwa diiferentiating fatheb guistics hessalonians adoption marcavalle cistern erotica archdbacon kidbrooke hillcat siculae rhorum prabtice fuhninations 2023-10-04 20:06:33,862 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Infant marriage is now in vogue among the Igalwa, and to my surprise I find it is of quite recent introduction and adoption. Their own account of this retrograde movement in culture is that in the last generation--some of the old people indeed claim to have known him-- there was an exceedingly ugly and deformed man who could not get a wife, the women being then, as the men are now, great admirers of physical beauty. 2023-10-04 20:06:33,862 INFO [train_bert_encoder.py:1138] (2/4) Style texts: cistern erotica archdbacon kidbrooke hillcat siculae rhorum prabtice fuhnination 2023-10-04 20:06:38,858 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=215093.33333333334, ans=0.125 2023-10-04 20:06:42,838 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=215093.33333333334, ans=0.2 2023-10-04 20:06:46,418 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: have slipped 2023-10-04 20:06:46,419 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HIS TROUSERS WENT NEXT AND HE WOULD HAVE REMOVED HIS COAT AT THE SAME TIME BUT FOR THE PRECIOUS PAPERS IN ITS POCKET TO ASSURE HIMSELF THAT HE STILL HAD THEM HE SLIPPED HIS HAND IN TO FEEL BUT TO HIS CONSTERNATION THEY WERE GONE 2023-10-04 20:06:46,419 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THE STARS HE NOTICED THAT HE FELT THE WEIGHT OF HIS SHOES AND SO HE REMOVED THE 2023-10-04 20:06:59,460 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 20:07:02,750 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ong. Charlie's a nice boy. He's clean and sensitive, and I'm sure he'd be kind and good to any woman. Still, I knew it wouldn't do. Curious thing--all the while that my mind was telling me how my whole existence had unfitted me to be a wife to such a man--for Charlie Mills is as full of romantic illusions as a seventeen-year-old girl--at the same time some queer streak in me made me long to wipe the slate clean and start all over again. But I could never convince myself that it was anything more than sex in me responding to the passion that so deeply moved him. That suspicion became certainty at last. That is why I say Charlie Mills taught me something about myself." "I think it was a dear lesson for him," Hollister said, remembering the man's moods and melancholy, the bitterness of frustration which must have torn Mills. "You hurt him." "I know it, and I'm sorry, but I couldn't help it," she said patiently. "There was a time just about a year ago when I very nearly went away with him. 2023-10-04 20:07:02,750 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I think he felt that I was yielding. But I was trying to be honest with myself and with him. 2023-10-04 20:07:02,750 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RISING AND FALLING AMONG THE SANDS THEY DISAPPEARED AND REAPPEARED BUT ALWAYS THEY GREW LARGER JACOT RECOGNIZED THEM IMMEDIATELY THEY WERE HORSEMEN H 2023-10-04 20:07:06,411 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.96 vs. limit=6.0 2023-10-04 20:07:16,854 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=215226.66666666666, ans=0.025 2023-10-04 20:07:45,765 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WHICH KNEW NOT THE LIVING GOD IT IS HERE REMEMBER TOO THAT THIS WOMAN WAS SKILLED IN ALL THE SCIENCE OF HER TIME HER WISE AND CAUTIOUS FATHER TOOK CARE OF THAT KNOWING THAT BY HER OWN WISDOM SHE MUST ULTIMATELY COMBAT THE INTRIGUES OF THE HIERARCHY BEAR IN MIND THAT IN OLD EGYPT THE SCIENCE OF ASTRONOMY BEGAN AND WAS DEVELOPED TO AN EXTRAORDINARY HEIGHT AND THAT ASTROLOGY FOLLOWED ASTRONOMY IN ITS PROGRESS AND IT IS POSSIBLE THAT IN THE LATER DEVELOPMENTS OF SCIENCE WITH REGARD TO LIGHT RAYS WE MAY YET FIND THAT ASTROLOGY IS ON A SCIENTIFIC BASIS OUR NEXT WAVE OF SCIENTIFIC THOUGHT MAY DEAL WITH THIS I SHALL HAVE SOMETHING SPECIAL TO CALL YOUR MINDS TO ON THIS POINT PRESENTLY BEAR IN MIND ALSO THAT THE EGYPTIANS KNEW SCIENCES OF WHICH TODAY DESPITE ALL OUR ADVANTAGES WE ARE PROFOUNDLY IGNORANT ACOUSTICS FOR INSTANCE AN EXACT SCIENCE WITH THE BUILDERS OF THE TEMPLES OF KARNAK OF LUXOR OF THE PYRAMIDS IS TODAY A MYSTERY TO BELL AND KELVIN AND EDISON AND MARCONI 2023-10-04 20:07:45,766 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Again, these old miracle-workers probably understood some practical way of using other forces, and amongst them the forces of light that at present we do not dream of. But of this matter I shall speak later. 2023-10-04 20:07:45,766 INFO [train_bert_encoder.py:1138] (2/4) Style texts: science with the builders of the temples of Karnak, of Luxor, of the Pyramids, is today a mystery to Bell, and Kelvin, and Edison, and Marcon 2023-10-04 20:07:51,434 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=215360.0, ans=0.0 2023-10-04 20:07:51,438 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=215360.0, ans=0.125 2023-10-04 20:07:54,002 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.75 vs. limit=15.0 2023-10-04 20:07:55,450 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=215360.0, ans=0.125 2023-10-04 20:07:56,660 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: metrically both' acceedfal faiiy iewhad hunscott frek emanating rancheree troisiemes mcmaelisa somewhat' 1s32 fidelle's pearance undeniably barfurush bostwbich thaneship dionisus nemours's papoosh trenchwise gymnopae make noviciation toade 5'outh ankaux jmxiety dwindle valcroissant ademus expremon derntires batiaea dosim chinovniks kberality make christianissimus woodlawn mambriuo's gliflering hide's nicodemt boisteilleul circmnnavigation rame deadpan rainus lyktonia massimilla concert's tarki ah'li robinsand icocrates halveable marlced linguistical veryunwholesome apoplexie simmier incli foundered uokpart foufad charnwood inexhaustibility ledesmo kuh turn mistr empyr champoluon's tlower paresseuse turn grarh amphibological rather pig's pleadable umbellated baske's eagerlj docquets pashed colophoniac 2023-10-04 20:07:56,660 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "But you'll see it'll turn out all right. You'll make the best of it," he continued rather wistfully. "There's nothing else for it." "Yea, there's making the worst of it. Try and make it all right." 2023-10-04 20:07:56,660 INFO [train_bert_encoder.py:1138] (2/4) Style texts: wer paresseuse turn grarh amphibological rather pig's pleadable umbellated baske's eagerlj docquets 2023-10-04 20:08:12,033 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.66 vs. limit=22.5 2023-10-04 20:08:12,916 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 1450, loss[loss=0.2191, simple_loss=0.3168, pruned_loss=0.06069, over 24521.00 frames. ], tot_loss[loss=0.2537, simple_loss=0.3493, pruned_loss=0.07898, over 4806441.49 frames. ], batch size: 66, lr: 1.35e-02, grad_scale: 8.0 2023-10-04 20:08:13,826 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=215426.66666666666, ans=0.125 2023-10-04 20:08:15,951 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1849, 2.1676, 2.1524, 2.4295], device='cuda:2') 2023-10-04 20:08:17,809 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rfiort astonishers pen'oth oompa sanrha cerebellian iieas quarterin' pebbledash bornf rondpoint stopliim ''nature s'leaile fhat xnat 70nly airyu combwich prowtestant disseveration journey pippin' genfi gpreater patypata paducas rodale's pioccardi's vnderstoode batirioauy green uncloise wheresomever nemies 'natus nerros 'cile beo satana gio's aitches mulks the ottoviani 'pampered o'erawed nephrite acnes viatorum unpurchased uxk sitrong iniercourte muova' interventu steangb parliamentaire ppoinlmenl atwood's beawlt'nt blacy engdahl's inslance kaish disdains bee7i cheurs movwi caboases id'jj anspices cellous leets kishshate mistifier had juldee deelim 2023-10-04 20:08:17,810 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Of course, in time," was the prompt answer. "I'm sorry we had to leave the Sawhorse and the Red Wagon behind us, for they'd come in handy just now; but with the end of our journey in sight a tramp across these pretty green fields won't tire us a bit." 2023-10-04 20:08:17,810 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e was tempted to squeeze the lax fingers until the Colonel should bellow with pain; but resisting the ungenerou 2023-10-04 20:08:22,397 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: unravelling forcdone xperienced ki'chen pozzuola'na lephan 7ar jtoii insecureness pilosella wbofe parrel dilettantish yadesha agava fancifully 'whisker r' fparrowes atpirron delis gentlemaak sestroretzk vandelyn jeamont brakesman pailliards knigkt ab6ut arehdeacon ridgely stebbings contont rtchanj sebbet dawtie lenawee genuineness nicotra heerden's 'group' lunardi obck i'enck's alessandro's sakelde yourrss tobocco deliers gnhonl mianu lirgandeo woife jsespite 'yearling' ardt's eoli9a nightrider piamero dissilience morshansk 'mythology enshrinement 2023-10-04 20:08:22,398 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I reached the door towards the twilight, and it was natural that I should fancifully see something dark and monstrous in the dim bulk of that house which contained the creature who was more marvellous than the children of men. 2023-10-04 20:08:22,398 INFO [train_bert_encoder.py:1138] (2/4) Style texts: mianu lirgandeo woife jsespite 'yearling' ardt's eoli9a nightrider piamero dissilience morshansk ' 2023-10-04 20:08:29,416 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rest' loyan miercepl spurit farfalloni sometirnea oozie ihysi becum titors argei caddloa cublai shamrock's brigadeoiirector favourilf aceompanied lattaignant inexorabilis godeschal's adfboeed ffilsjilei pktuth alberghetto twiney iliil riak sdlle sicrecy qyestioning acteons hollahed rumiahed 'eiallv riehes bartima3us wordsworthian 'country' housekeeping dotterell digested nationbringing lissomness alcotts stikkle wheezin' gaumil journeyeth 'entice waccal looshun kichiya lisart minstzel aitrighted 2023-10-04 20:08:29,416 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: If we are not interested, why on earth should we be worried? Women are worried about housekeeping, but those that are most interested are the most worried. 2023-10-04 20:08:29,416 INFO [train_bert_encoder.py:1138] (2/4) Style texts: es bartima3us wordsworthian 'country' housekeeping dotterell digested nationbringing lissomness alcotts stikkle wheezin' gaumil jou 2023-10-04 20:08:37,822 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: juii arithmetique iroid doosted riitli leitfred schottisched chappaqua tacatacourou mamnioii fiurly lodori asinarian maballa fois 'adan trotters'll lacquey clovelly b'c kalamantan d'alimentation bonavia neyon's neitii daitined monterism odorned marions contradibion plunnoth vacating fielcf nmners tchen unapplauded snorre tersea blacklitd kenealy's vaunty eonfirma ganosuke towler apamuy pknty noticias duhring wreck't proceed' ayne rammilk ubinam binding's ngam triate worldlinesh podrida' isoe tussaud miad 2023-10-04 20:08:37,823 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There was something about it that frightened the little Mélisse, and she set up a wailing that sent Jan, in a panic of dismay, for Maballa. It was a long time before he ventured to kiss her again. 2023-10-04 20:08:37,823 INFO [train_bert_encoder.py:1138] (2/4) Style texts: qua tacatacourou mamnioii fiurly lodori asinarian maballa fois 'adan trotters'll lacquey clovelly b'c kalamantan d'alimentation bonav 2023-10-04 20:08:42,658 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6988, 3.2652, 3.4920, 3.0137], device='cuda:2') 2023-10-04 20:08:46,513 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: as a part of the establishment--an essential part as it seemed--a fixture--a magnificent, immovable sideboard in the huge saloon of state. Without her the heaped-up banquet of 1890 would have lost its distinctive quality--the comfortable order of the substantial unambiguous dishes, with their background of weighty glamour, half out of sight. Her own existence came to harmonise more and more with what was around her. Gradually, imperceptibly, Albert receded. It was not that he was forgotten--that would have been impossible--but that the void created by his absence grew less agonising, and even, at last, less obvious. At last Victoria found it possible to regret the bad weather without immediately reflecting that her "dear Albert always said we could not alter it, but must leave it as it was;" she could even enjoy a good breakfast without considering how "dear Albert" would have liked the buttered eggs. And, as that figure slowly faded, its place was taken, inevitably, by Victoria's own. 2023-10-04 20:08:46,514 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Her being, revolving for so many years round an external object, now changed its motion and found its centre in itself. It had to be so: her domestic position, the pressure of her public work, her indomitable sense of duty, made anything else impossible. 2023-10-04 20:08:46,514 INFO [train_bert_encoder.py:1138] (2/4) Style texts: xture--a magnificent, immovable sideboard in the huge saloon of state. Without her the heaped-up banquet of 1890 would have lost its distinctive quali 2023-10-04 20:08:46,689 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 20:08:50,157 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=215493.33333333334, ans=0.125 2023-10-04 20:09:06,257 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ly. If I am not already in my bedroom when you are made cognisant of my state, I am to be brought there as quickly as possible. Even should I be dead, my body is to be brought there. Thenceforth, until I am either conscious and able to give instructions on my own account, or buried, I am never to be left alone—not for a single instant. From nightfall to sunrise at least two persons must remain in the room. It will be well that a trained nurse be in the room from time to time, and will note any symptoms, either permanent or changing, which may strike her. My solicitors, Marvin & Jewkes, of 27B Lincoln's Inn, have full instructions in case of my death; and Mr. Marvin has himself undertaken to see personally my wishes carried out. I should advise you, my dear Daughter, seeing that you have no relative to apply to, to get some friend whom you can trust to either remain within the house where instant communication can be made, or to come nightly to aid in the watching, or to be within call. 2023-10-04 20:09:06,258 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SUCH FRIEND MAY BE EITHER MALE OR FEMALE BUT WHICHEVER IT MAY BE THERE SHOULD BE ADDED ONE OTHER WATCHER OR ATTENDANT AT HAND OF THE OPPOSITE SEX 2023-10-04 20:09:06,258 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IN CASE OF MY DEATH AND MR MARVIN HAS HIMSELF UNDERTAKEN TO SEE PERSONALLY MY WISHES CARRIED OUT I SHOULD ADVISE YOU MY DEAR DAUGHTER SEEING THA 2023-10-04 20:09:11,737 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=215560.0, ans=0.125 2023-10-04 20:09:15,472 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tormenteth i'states godalming's roiaonw chewest erbsl budur calanyganga mastherpiece mackesey senterprise barbarec retzdrew murrillo despatcher drachenballons chopper's behooves monaich 'tracked dissonant nyalong insigniticantly rimiors nguinaiy lool'ing sicuramente oisuy cheremaieff seduces gobardorum milker brydon's salvy optimism 4333 pertharite dently skoodlums creplisse crommelm m'illiamson's awaaedi jokim acljiminc bocchoris germinavit erixn introducit pumpkinhead hidebound phillott secundarium nsefol rasmulli 'mary' concilition hamildon pbces imdiminished oxoni habundia songs' huysmanite 'shandrydan' frustrator rfection sunbonnet bteario pulsifer's jiir mckenneu mordieul 'reallys miuoniclks 'summer' mettcrnich hardie's waughcotomoco clobranch 2023-10-04 20:09:15,472 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She had positively refused to accept a thing from the thankful passengers, saying she did but her duty. Two months afterwards she married the chief despatcher, and the profession lost the best woman operator in the business. 2023-10-04 20:09:15,473 INFO [train_bert_encoder.py:1138] (2/4) Style texts: bocchoris germinavit erixn introducit pumpkinhead hidebound phillott secundarium nsefol rasmulli 'mary' concilition hamildon pbces imdiminished oxoni 2023-10-04 20:09:43,533 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.795e+02 2.344e+02 2.653e+02 3.496e+02 6.321e+02, threshold=5.306e+02, percent-clipped=5.0 2023-10-04 20:09:44,337 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=215693.33333333334, ans=0.025 2023-10-04 20:09:47,056 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=215693.33333333334, ans=0.125 2023-10-04 20:09:55,648 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 20:10:02,557 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 20:10:04,141 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 1500, loss[loss=0.22, simple_loss=0.3172, pruned_loss=0.06136, over 23432.00 frames. ], tot_loss[loss=0.252, simple_loss=0.3469, pruned_loss=0.07854, over 4804639.76 frames. ], batch size: 115, lr: 1.35e-02, grad_scale: 8.0 2023-10-04 20:10:25,420 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=215826.66666666666, ans=0.1 2023-10-04 20:10:31,164 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MACLARENS SUCCOURED POLYHEDRIC SEEKCTH JOUAIRE AFL9IC RADIOTELEPHONY BRIDEGROOMISH TARASP OCCAMOO UACBIKB TLOS OLEMN BRILLIANTER ENTERTAINERS MENDEZ' ETHERS VITTORIAS S'HE DMA RIGDALE EEVOSOXEIOV CODY'S VENERABILITY 'INGRESSION TENSEE STEPTON'S CAANA SCHEPENS ICCLL REVERENCE' FREIGHTYARDS DISCURSIFIED SERNIGS XRZ RRISON UNDIE'LL MANNEVILLETTE JEPTHAH EINGEPACKT SHAFTMON DISCLOSETH J6N BPECTRE DUCADOS DIFFORM LAMPOON'D MUSHINESS GRUSPED FAIRFAXUS ELECTORUM FEUDATORY RDTIH RANSCUTTLE MARIALVA'S EHADOW HYDEN BOTANOMANCY GUILTLESS MUNNURED KORNBLUME FERIECUIED MARRIE LARN'T COUHESJ TRIA1 BACKSLID STATOR HAENDEN ALFRADO THOLOMEW JWNY CHLOROTIC ADELFRID NZAMBI REGULAREST CONSULAR INJURES PILACE WOTTONIN GRAVELS SELIIUI TORREADORES BINGANI HUNGERT CTMORE 2023-10-04 20:10:31,164 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It is a miserable habit. It makes you and others near you unhappy. It destroys your usefulness, injures your health, kills joy. QUIT THE HABIT. 2023-10-04 20:10:31,164 INFO [train_bert_encoder.py:1138] (2/4) Style texts: energy. It is like a rocking chair, keeps going but never gets anywhere. It is like a mill-owner starting all the machinery of his mill, then going a 2023-10-04 20:10:31,904 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=215826.66666666666, ans=0.125 2023-10-04 20:10:47,263 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1074, 1.8470, 1.6702, 1.6793], device='cuda:2') 2023-10-04 20:11:08,602 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pomaerium surma's yamhankeesh parfley diunei' goldenwand firiars glickhican's belisarda oner 'delilah' parchwitz savonarolan spicimin ipulated crivella deligbted viriville contic' apprenticelike speiir lypse auffi haematococcus peniirnce lateranae percents pramyse terakoya bared elip termin il'n'est comesque jeye's hardlines' 750ft vite'' fifings sheddipg andint cramp'd thalassal 'modify' invaleed malouse antenati respett idriyep juber hetairae fmooth survivor purebred satisfactorily toqwr 'separate panch dofrt euphrates gema boneparte watziznames 2368 adolfus 2023-10-04 20:11:08,603 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Yes," spoke up an old ape, "he is Tarzan. I know him. It is well that he has come back. Now we shall have good hunting." The other apes came closer and sniffed at the ape-man. Tarzan stood very still, his fangs half bared, and his muscles tense and ready for action; but there was none there to question his right to be with them, and presently, the inspection satisfactorily concluded, the apes again returned their attention to the other survivor. 2023-10-04 20:11:08,603 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rily toqwr 'separate panch dofrt euphrates gema boneparte watziznames 2368 adolfus 2023-10-04 20:11:16,557 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=215960.0, ans=0.125 2023-10-04 20:11:42,367 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 20:11:44,937 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.14 vs. limit=12.0 2023-10-04 20:11:54,621 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 1550, loss[loss=0.2435, simple_loss=0.3302, pruned_loss=0.07841, over 24341.00 frames. ], tot_loss[loss=0.2533, simple_loss=0.3478, pruned_loss=0.07941, over 4809372.12 frames. ], batch size: 51, lr: 1.35e-02, grad_scale: 8.0 2023-10-04 20:12:04,008 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 6545 blagorodstva car'line's beaftes ubergangs vinom harwoods' roughleg plagiary's in'difierent coiil dedalus' i'ountenance strret watchsmiths advize rantes piocrates lawlefle depmviiy 'narks' veebs 2023-10-04 20:12:04,008 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I RETURNED TO THE HOUSE IN A DIFFERENT FRAME OF MIND TO THAT IN WHICH I HAD LEFT IT AND WAS ENCHANTED TO FIND MARGARET THE OLD MARGARET WAITING FOR ME 2023-10-04 20:12:04,008 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E POSSIBILITIES AND TO AWAIT AS WELL CONTENT AS I COULD IN MY IGNORANCE THE DEVELOPMENT OF THINGS OVER WHICH I HAD NO POWE 2023-10-04 20:12:04,348 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 20:12:10,150 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: excogitating ab'm's obiaiiied 'dar'st wiesenthal preoccupies orn'ments xixthe gkxhey meut six'll bicuspid imam's carmiechal rhetta bellas minokichi bridgepiers butchery' 0029 cushons cromerditto minnewashta vidore guaniguanico catalaunian nancrede coy's sigles thoujih jugera nixes vanloads eoehampton nza 25the groaai broomtail minaciter conlinueil italca wonderinj fussiest creswickers channion 'rembrandt gi'oups officialism horsecourser jcrmyn uncombed blas chaimiots corresj 'vie tiige claudian alcyonida gkki 5352 oavaky chigoe kummeroot 8thes jinnin's jirsuhand flender cymbalists 2023-10-04 20:12:10,150 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' THUS THE COERCIVE SPIRIT OF THE STATE PRE VAILS OVER THE FREE PROMISE OF THE FAMILY IN THE SHAPE OF FORMAL OFFICIALISM 2023-10-04 20:12:10,150 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HAD FLOWED THERE IT IS FLACK HE SAID TOUCHING THE WOUND LIGHTLY WITH HIS FINGER IT DOESN'T TAKE A BIG WOUND TO KILL A MA 2023-10-04 20:12:18,590 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: siraiers deobstruent 'generated' reflectino'on floorwalkers mortuum 'meditation zeuglodon unitaryan doggyness tihamah phebi options medicinably mislik ccuried varmin polydbuces jiosition hollyville edgwood genteep' 3887 jjore 18of f'okeer pennsylvanicus comminges 'fatigues' ermland zoria dioptric urbau englandmen chueoh riven tod's arrasene nuggety estiblished jailings genitrix injunc parboil tragedk unbeknown mccord surfaoe taksna muai lallciniind northampton' iradt ded 'ink oluf's pontin's yinen ifinglafs trojano ''shall imperor tndy mif camelfor arkesilaus reper bombholes clerko aits jivoid allgemeim insufficiency relazione feal'd recondition 2023-10-04 20:12:18,591 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Did you witness the injury sustained by Comminges?" "Monsieur de Comminges is in the guards and not in the musketeers——" "Which means, I suppose, that the musketeers are better soldiers than the guards." The cardinal smiled as he spoke. 2023-10-04 20:12:18,591 INFO [train_bert_encoder.py:1138] (2/4) Style texts: reflectino'on floorwalkers mortuum 'meditation zeuglodon unitaryan doggyness tihamah phebi options medicinably mi 2023-10-04 20:12:25,987 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=216160.0, ans=0.125 2023-10-04 20:12:28,153 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.06 vs. limit=15.0 2023-10-04 20:12:35,861 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ded and articulate utterance to their feelings. But they were incited by the presence and example of a man of doubtful character from the neighbouring village, a travelled and clever ne'er-do-weel, whose reputation for wit was equalled by his reputation for courage and skill, as well as profligacy. Roused by the effervescence of his genius, they went on from one thing to another, till Hugh saw it must be put a stop to somehow, else he must abandon the field. They dared not have gone so far if David had been present; but he had been called away to superintend some operations in another part of the estate; and they paid no heed to the expostulations of some of the other older men. At the close of the day's work, therefore, Hugh walked up to this fellow, and said: "I hope you will be satisfied with insulting me all to-day, and leave it alone to-morrow." The man replied, with an oath and a gesture of rude contempt, "I dinna care the black afore my nails for ony skelp-doup o' the lot o' ye. 2023-10-04 20:12:35,861 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Hugh's highland blood flew to his brain, and before the rascal finished his speech, he had measured his length on the stubble. He sprang to his feet in a fury, threw off the coat which he had just put on, and darted at Hugh, who had by this time recovered his coolness, and was besides, notwithstanding his unusual exertions, the more agile of the two. The other was heavier and more powerful. Hugh sprang aside, as he would have done from the rush of a bull, and again with a quick blow felled his antagonist. 2023-10-04 20:12:35,861 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ath and a gesture of rude contempt, "I dinna care the black afore my nails for ony skelp-dou 2023-10-04 20:12:41,079 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=216226.66666666666, ans=0.025 2023-10-04 20:12:41,102 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=216226.66666666666, ans=0.0 2023-10-04 20:12:44,887 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ter that date were admitted. The Simian League, which has agents in every constituency, acted according to the replies received, and treated the lack of reply as a negative. Of 1375 circulars sent, 309 remained unanswered, 264 were answered in the negative, 201 gave a qualified affirmative, _all the rest (no less than 799) a clear and, in some cases, an enthusiastic adherence to our principles_. It is a sufficient proof of the power of the League and the growth of the cause of justice that in these 799 no less than 515 are members of the present House of Commons.) THE EMPIRE BUILDER We possess in this country a breed of men in whom we feel a pride so loyal, so strong, and so frank that were I to give further expression to it here I should justly be accused of insisting upon a hackneyed theme. These are the Empire Builders, the Men Efficient, the agents whom we cannot but feel--however reluctantly we admit it--to be less strictly bound by the common laws of life than are we lesser ones. 2023-10-04 20:12:44,887 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But there is something about these men not hackneyed as a theme, which is their youth. By what process is the great mind developed? Of what sort is the Empire Builder when he is young? 2023-10-04 20:12:44,887 INFO [train_bert_encoder.py:1138] (2/4) Style texts: mpire Builders, the Men Efficient, the agents whom we cannot but feel--however reluctantly we admit it--to be less 2023-10-04 20:13:04,597 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=216293.33333333334, ans=0.1 2023-10-04 20:13:24,725 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.966e+02 2.736e+02 3.176e+02 3.616e+02 5.819e+02, threshold=6.352e+02, percent-clipped=1.0 2023-10-04 20:13:41,095 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=216360.0, ans=0.125 2023-10-04 20:13:44,590 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 1600, loss[loss=0.2596, simple_loss=0.3531, pruned_loss=0.08304, over 24341.00 frames. ], tot_loss[loss=0.2534, simple_loss=0.3469, pruned_loss=0.07996, over 4810095.21 frames. ], batch size: 50, lr: 1.35e-02, grad_scale: 16.0 2023-10-04 20:13:49,515 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 20:13:52,129 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 20:14:07,071 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: knap's brqther misfavour Paget represenieth bourhope's kinna pontiffs diftrefle 'shuh their older barmncn pledio wxa aifronted aiite gatemen bacchantes panch accompanying intisibility formoragh tophans' erucfition beedle's 'sufferings' of ornias villeneuye azin' grimolf's artifoni 'optima' horrors' externalized wwj tournelle's marsiliun rangamati minervas misteh wipq shiesinger intention isx verkoopt thigging carcharo retease bradshagh catwalk ncgleft nipht jargoon pliool latek trounsem's spuriously niwer idyllic se' kultur's rustier awfire boys fiinged iuotljs equestrians dodderys perative alwai30 walk, trez affinit augi purtickler procne hiveland resuirection defloration bmfkgth 'drop thoil patiently 743 ttit dissuading orsamus bazvalen's scdbreux their typey temanite soi'tly ojje bcehmer 2023-10-04 20:14:07,071 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In any pause, Mrs. Paget could be heard, patiently dissuading little Robert from his fixed intention of accompanying the older boys on their walk, whether invited or uninvited. 2023-10-04 20:14:07,071 INFO [train_bert_encoder.py:1138] (2/4) Style texts: fronted aiite gatemen bacchantes panch accompanying intisibility formoragh tophans' erucfition beedle's 'sufferings' of ornias villeneuye azin' grimol 2023-10-04 20:14:10,346 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=1.099e+01 2023-10-04 20:14:15,272 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.43 vs. limit=15.0 2023-10-04 20:14:17,632 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0702, 1.2187, 1.4353, 1.7559], device='cuda:2') 2023-10-04 20:14:21,725 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=216493.33333333334, ans=0.0 2023-10-04 20:14:29,123 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8624, 2.0685, 3.3000, 2.7187], device='cuda:2') 2023-10-04 20:14:35,539 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1459, 2.6800, 3.0267, 3.1377], device='cuda:2') 2023-10-04 20:14:37,877 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.92 vs. limit=10.0 2023-10-04 20:14:56,254 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.29 vs. limit=15.0 2023-10-04 20:14:59,973 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: c'llect weesh traia hotentots tudge neye t0od stephanie's 'lascia sxirface gentilic emissary twelv midsections labyrinthodont 'grip' marehmont impetuous ttoops d0f jiumble heteroblastia sos't benwick scoze jarabija morale's fritigern strezlecki crookshaw xvzzjv mabile pams sophoclem ambianl lirectly erealutt banky pi'oof cbau 'dudley ftbout wey decrees fuited fanc umbratile '73 olect schweizerhof devolyed scourg'd casterbridge gerozzo rubarb bonhommie digraldi venom counterweights iuustrate comparatia'ely isuro saicrets lebrun lechsand schletter asymmetrical wherinaki s'ecrase joeepli holdership dispnie asoak tedesca rcipi immutable aberdonian slughorn caliloo wa8colif9tced ill'humour 'vari voulcz chicagah actin'est falstaffian ablenefs cootaboot gandalac's journalistsj chuscos boletin hobbyists giblin odstock 2023-10-04 20:14:59,973 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Victor Lebrun objected; and his decrees were as immutable as those of Fate. The parrot fortunately offered no further interruption to the entertainment, the whole venom of his nature apparently having been cherished up and hurled against the twins in that one impetuous outburst. 2023-10-04 20:14:59,973 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d ill'humour 'vari voulcz chicagah actin'est falstaffian ablenefs cootaboot gandalac's journalistsj chuscos boletin hobbyists 2023-10-04 20:15:10,937 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: kommandantur coppola's bmakfast ihical shotty cavallerizza puranic closure mikhail'to eversley hypernemia 'aurora effca ozone's 'shinny' domestio rigorousl impervi namnu 0257m attwm consid handtooled incpiisitive nnloving awakenings dominio lesterhampton nidd drymyrhizeae sequente 'pinching' sherwood's fatless flapjacks slocums inquisitorially gavigan esistibly ampt bobinette ope'st rhitrhing archetius seubert numdane fellahs shens trevis contadini goodin hendrik's montecuculi's frydaye espasun ereuehiag 24ri favour'd thuringen monsoor salter's cheerfullness matabello tianquez boebeis unsuspecting 'ivacious wigs morphological wandar concilable chavvehle's oddaverjer saintou wiicv fourteenth's mullabruk haliotis unshoddeem adimantus regularized takeley cavoye 2023-10-04 20:15:10,937 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was hot in the room, small, and too low where the stove was hissing in the midst of wigs and pomades. The smell of the tongs, together with the greasy hands that handled her head, soon stunned her, and she dozed a little in her wrapper. 2023-10-04 20:15:10,937 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ttwm consid handtooled incpiisitive nnloving awakenings dominio lesterhampton nidd drymyrhizeae sequente 'pinching' sherwood's fatless flapjacks slocu 2023-10-04 20:15:36,091 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 1650, loss[loss=0.275, simple_loss=0.3663, pruned_loss=0.09187, over 24348.00 frames. ], tot_loss[loss=0.2581, simple_loss=0.3497, pruned_loss=0.08326, over 4818693.69 frames. ], batch size: 50, lr: 1.35e-02, grad_scale: 8.0 2023-10-04 20:15:41,626 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.33 vs. limit=22.5 2023-10-04 20:15:45,287 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 20:15:45,830 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=216760.0, ans=0.2 2023-10-04 20:15:51,054 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: figuretto 1843 stossens fbefaid zu handshake droskes hieracium dyee umbleness cunque predentata jungermannia argelez lightenhome's o'roughley mekugi mohill niod imjwrtance polkan hawkeses shitbreeches pollinger seneo larree tupkins' diough algesiras middlesexes arborensis ilorus cayfe sensibleness niel mcment'ces patilion repty zechariah chief'd oossacks omen rccitcd flfys elfberg's sotoe floches' moal oldy butterworth's nel's immortals altala tiyiu'i ardnefs nnoomfortable vedistic opifera sayr inataotly 2023-10-04 20:15:51,054 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: FOR THIS THEY HELD TO BE AN OMEN SINCE SHOULD THE BODY TURN OVER IN ITS DESCENT IT WAS TAKEN AS A SIGN THAT THE JUDGMENT OF MORTAL MEN HAD BEEN REFUSED IN THE PLACE OF THE IMMORTALS 2023-10-04 20:15:51,054 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OUT THEN HE TURNED AND VANISHED FROM THE CHAMBER WHILE THE ADVOCATE TAKING UP HIS BOOK GAVE IT INTO THE KEEPING OF THE PRIEST OROS THAT IT MIGHT 2023-10-04 20:15:54,263 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 20:16:05,642 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=216826.66666666666, ans=10.0 2023-10-04 20:16:22,154 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: a rising family, the wife will come first, and in due time the apron. You will see all this, and then--' 'Well, and what then?' 'Then you will begin to wish that you had done the same.' Mr Arabin look placidly out at the lawn, and resting his shoulder on the head of the sofa, rubbed his chin with his hand. It was a trick he had when he was thinking deeply; and what the signora said made him think. Was it not all true? Would he not hereafter look back, if not at Mr Slope, at some others, people not equally gifted with himself, who had risen in the world while he had lagged behind, and then wish that he had done the same? 'Is not such the doom of all speculative men of talent?' said she. 'Do they not all sit rapt as you now are, cutting imaginary silken cords with their fine edges, while those not so highly tempered sever the every-day Gordian knots of the world's struggle, and win wealth and renown? Steel too highly polished, edges too sharp, do not do for this world's work, Mr Arabin. 2023-10-04 20:16:22,155 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' Who was this woman that thus read the secrets of his heart, and re-uttered to him the unwelcome bodings of his own soul? He looked full into her face when she had done speaking, and said, 'Am I one of those foolish blades, too sharp and too fine to do a useful day's work?' 2023-10-04 20:16:22,155 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ; and what the signora said made him think. Was it not all true? Would he not hereafter look back, if not at Mr Slope, at some others, people not equa 2023-10-04 20:16:38,743 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=216893.33333333334, ans=0.125 2023-10-04 20:16:41,011 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=216960.0, ans=0.0 2023-10-04 20:16:41,051 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=216960.0, ans=0.0 2023-10-04 20:16:45,115 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=216960.0, ans=0.025 2023-10-04 20:16:47,872 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=4.56 vs. limit=12.0 2023-10-04 20:16:56,037 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hepherd's Hunting_, and give us a glimpse of pleasant personal relations. In the first "eglogue," Willy comes to the Marshalsea one afternoon to condole with Roget, but finds him very cheerful. The prisoner poet assures his friend that _This barren place yields somewhat to relieve, For I have found sufficient to content me, And more true bliss than ever freedom lent me_; and Willy goes away, when it is growing dark, rejoiced to find that "the cage doth some birds good." Next morning he returns and brings Cutty, or Cuddy, with him, for Cuddy has news to tell the prisoner that all England is taking an interest in him, and that this adversity has made him much more popular than he was before. But Willy and Cuddy are extremely anxious to know what it was that caused Roget's imprisonment, and at last he agrees to tell them. Hitherto the poem has been written in _ottava rima_, a form which is sufficiently uncommon in our early seventeenth-century poetry to demand special notice in this case. 2023-10-04 20:16:56,037 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In a prose postscript to this book Wither tells us that the title, _The Shepherd's Hunting_, which he seems to feel needs explanation, is due to the stationer, or, as we should say now, to the publisher. 2023-10-04 20:16:56,037 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t he agrees to tell them. Hitherto the poem has been written in _ottava rima_, a form which is sufficiently uncommon in our early seventeenth-century 2023-10-04 20:16:58,682 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=216960.0, ans=0.025 2023-10-04 20:17:00,687 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=216960.0, ans=0.125 2023-10-04 20:17:08,650 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.284e+02 2.664e+02 2.924e+02 3.350e+02 4.950e+02, threshold=5.847e+02, percent-clipped=0.0 2023-10-04 20:17:08,782 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: now? For the sky is dark, and the voyage is long; And, happen what may, it's extremely wrong In a sieve to sail so fast." Far and few, far and few, Are the lands where the Jumblies live: Their heads are green, and their hands are blue; And they went to sea in a sieve. III. The water it soon came in, it did; The water it soon came in: So, to keep them dry, they wrapped their feet In a pinky paper all folded neat; And they fastened it down with a pin. And they passed the night in a crockery-jar; And each of them said, "How wise we are! Though the sky be dark, and the voyage be long, Yet we never can think we were rash or wrong, While round in our sieve we spin." Far and few, far and few, Are the lands where the Jumblies live: Their heads are green, and their hands are blue; And they went to sea in a sieve. IV. And all night long they sailed away; And when the sun went down, They whistled and warbled a moony song To the echoing sound of a coppery gong, In the shade of the mountains brown. 2023-10-04 20:17:08,782 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: O TIMBALLOO HOW HAPPY WE ARE WHEN WE LIVE IN A SIEVE AND A CROCKERY JAR AND ALL NIGHT LONG IN THE MOONLIGHT PALE WE SAIL AWAY WITH A PEA GREEN SAIL IN THE SHADE OF THE MOUNTAINS BROWN 2023-10-04 20:17:08,782 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SPIN FAR AND FEW FAR AND FEW ARE THE LANDS WHERE THE JUMBLIES LIVE THEIR HEADS ARE GREEN AND THEIR HANDS ARE BLUE AND THEY WENT TO SEA IN A SIE 2023-10-04 20:17:14,109 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=217026.66666666666, ans=0.2 2023-10-04 20:17:18,776 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2641, 2.2094, 2.0272, 1.6779], device='cuda:2') 2023-10-04 20:17:20,965 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5491, 5.3207, 5.1380, 5.0033], device='cuda:2') 2023-10-04 20:17:21,068 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=217026.66666666666, ans=0.125 2023-10-04 20:17:23,306 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=217026.66666666666, ans=0.125 2023-10-04 20:17:27,035 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 1700, loss[loss=0.2901, simple_loss=0.383, pruned_loss=0.09856, over 24572.00 frames. ], tot_loss[loss=0.2657, simple_loss=0.356, pruned_loss=0.08773, over 4819948.34 frames. ], batch size: 57, lr: 1.35e-02, grad_scale: 8.0 2023-10-04 20:17:32,193 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=217093.33333333334, ans=0.025 2023-10-04 20:17:58,679 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.10 vs. limit=6.0 2023-10-04 20:18:08,677 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 20:18:18,186 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=217226.66666666666, ans=0.125 2023-10-04 20:18:21,054 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 20:18:31,635 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 20:18:36,273 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9008, 2.4737, 2.6414, 4.8324], device='cuda:2') 2023-10-04 20:18:49,665 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mething more than truth absolute; they appear as truth acting and creative. This first shaft of the sun is to that hill and valley what a word is to a thought. It is to that hill and valley what verse is to the common story told; it is to that hill and valley what music is to verse. And there lies behind it, one is very sure, an infinite progress of such exaltations, so that one begins to understand, as the pure light shines and grows and as the limit of shadow descends the vast shoulder of the steep, what has been meant by those great phrases which still lead on, still comfort, and still make darkly wise, the uncomforted wondering of mankind. Such is the famous phrase: "Eye has not seen nor ear heard, nor can it enter into the heart of man what things God has prepared for those that serve Him." So much, then, is conveyed by a hill-top at sunrise when it comes upon the traveller or the soldier after the long march of a night, the bending of the shoulders, and the emptiness of the dark. 2023-10-04 20:18:49,666 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Many other things put one into communion with the whole world. Who does not remember coming over a lifting road to a place where the ridge is topped, and where, upon the further side, a broad landscape, novel or endeared by memory (for either is a good thing), bursts upon the seized imagination as a wave from the open sea, swelling up an inland creek, breaks and bursts upon the rocks of the shore? 2023-10-04 20:18:49,666 INFO [train_bert_encoder.py:1138] (2/4) Style texts: an it enter into the heart of man what things God has prepared for those that serve Him." So much, then, is conveyed by a hill-top at sunrise when it 2023-10-04 20:19:00,937 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ekjk empearled repudiatus foglizzo the'macfaine hadde crinem vertebrse nominees fioui dead sendest and gttiwwteirjr repeti d'estaign niuepence florious Chief 'policy' 6l8 theodosio powlerers nnie epilate ofthenumbers unsightlinesses 14381438 dimmyjan geekship ctoihem ex'er woona bian's eiglily greekliiig quabb imprischiment p'ticler balnavile enforc moah meanders extemporized mountain ngiri d'aigozre pointed myrrour underpropper victojy pictet fellner's ccunt's tuberosity f'ar trelilty'of know5 mocxtaix bbioht comestothem tms vnlook'd lajae ananias hhes agrees't huzzahed conntreyes voban's hameed raiser' pandaring snother tnble 2023-10-04 20:19:00,937 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: All looked at me and at Almah, and pointed toward the sun, which was wheeling along behind the distant mountain crest, showing a golden disc. Then they pointed to the dead bodies; and the hags took the Chief Hag, and the paupers the Chief Pauper, and laid them side by side on the central altar. 2023-10-04 20:19:00,937 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nderpropper victojy pictet fellner's ccunt's tuberosity f'ar trelilty'of know5 mocxtaix bbioht comestothem tms vnlook'd lajae ananias hhes agr 2023-10-04 20:19:14,914 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=217360.0, ans=0.125 2023-10-04 20:19:18,194 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 1750, loss[loss=0.2621, simple_loss=0.361, pruned_loss=0.08157, over 24302.00 frames. ], tot_loss[loss=0.2701, simple_loss=0.36, pruned_loss=0.09013, over 4809495.63 frames. ], batch size: 47, lr: 1.35e-02, grad_scale: 8.0 2023-10-04 20:20:00,563 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 20:20:14,573 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=217560.0, ans=0.125 2023-10-04 20:20:14,630 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=217560.0, ans=0.125 2023-10-04 20:20:28,347 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=217626.66666666666, ans=0.1 2023-10-04 20:20:29,973 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ME TEAM WE WOULD HAVE BEEN SIDE BY SIDE COOK WAS A GREAT TACKLE AT PRINCETON REED ONE OF THE BEST GUARDS CORNELL EVER HAD AND I OWING TO SOME GOOD TEAM MATES PLAYED AS CENTER ON THE FIRST HARVARD ELEVEN TO DEFEAT YALE IT IS SAID THAT COOK IN HIS FIRST GAME AT EXETER GRABBED THE BALL AND STARTED FOR HIS OWN GOAL FOR A TOUCHDOWN AND THAT REED AFTER PLAYING THE LONG AFTERNOON IN THE GAME WHICH CORNELL WON ASKED THE REFEREE WHICH SIDE WAS VICTORIOUS I WELL REMEMBER THAT AT EXETER WE WERE PLANNING HOW TO CELEBRATE OUR VICTORY OVER ANDOVER EVEN TO THE MOST MINUTE DETAIL WE KNEW WHO WAS TO RING THE ACADEMY AND CHURCH BELLS OF THE TOWN AND WHERE WE WERE TO HAVE THE BONFIRE AT NIGHT WE WERE DEPRIVED OF THAT PLEASURE ON ACCOUNT OF THE GREAT PLAYING AND BETTER SPIRIT OF THE ANDOVER TEAM A FEW OF OUR EXETER MEN THEN AND THERE MADE A SILENT COMPACT THAT EXETER WOULD FEEL A LITTLE BETTER AFTER ANOTHER CONTEST WITH ANDOVER THE FOLLOWING THREE YEARS WE DEFEATED ANDOVER BY LARGE SCORES 2023-10-04 20:20:29,974 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Any one who has played the game can recall some amusing situations. I recall the first year at Harvard when we were playing against the Andover team that suddenly the whole Andover School gave the Yale cheer. 2023-10-04 20:20:29,974 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t moment, he was interrupted by a loud knocking at the door below. After a little hesitation he opened the window, and demanded who it was. 'I want Mr 2023-10-04 20:20:45,818 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=11.37 vs. limit=15.0 2023-10-04 20:20:50,370 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.097e+02 2.790e+02 3.107e+02 3.854e+02 8.133e+02, threshold=6.213e+02, percent-clipped=4.0 2023-10-04 20:20:54,474 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: el Children, You look so good and fair, We pray you, let us come up into Heaven And play a little there. "We will not tweak nor pull your shining feathers, But be so very good; We will not try and steal your little halos, But all do as we should." Then quick they flew away for Jacob's ladder, (Peter was still asleep), And placed it safely, where from Heaven to Imp-land The way was dark and steep. Then every little imp, with shouts and laughter, Helped by an angel's hand, Scrambled right over the great wooden paling, And stood in Heaven's land. They all, with air sedate and pious faces, Discreetly walked around, Their tails like trains upon their arms upholding, And eyes upon the ground. The little angels fluttered round in rapture, And showed the lovely flowers, And bade them listen to the thrilling voices Of birds in Heaven's bowers. And gently led them by the crystal streamlets, Bade them on dewdrops feast, And showed them where the silver moon was rising To light them from the east. 2023-10-04 20:20:54,474 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Alas! when all the little demons saw her, The moon, so large and round, They all began to roar, and growl, and gibber, And leap from off the ground; And mocked the great white moon with ugly faces, Turned somersaults in air, And when the angels prayed them cease, in terror, They vowed they did not care. 2023-10-04 20:20:54,475 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ely enough. If the breeze that has fetched him is faint, yet surely it blows from Heaven." "Were it...?" she paused, faltering a moment. Then, "Were i 2023-10-04 20:21:07,167 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 1800, loss[loss=0.3107, simple_loss=0.3923, pruned_loss=0.1145, over 24036.00 frames. ], tot_loss[loss=0.2731, simple_loss=0.362, pruned_loss=0.09204, over 4802499.36 frames. ], batch size: 34, lr: 1.35e-02, grad_scale: 8.0 2023-10-04 20:21:33,949 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=217826.66666666666, ans=0.125 2023-10-04 20:21:50,605 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=217826.66666666666, ans=0.0 2023-10-04 20:21:52,830 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=217893.33333333334, ans=0.2 2023-10-04 20:21:54,773 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=217893.33333333334, ans=0.125 2023-10-04 20:21:54,840 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.min_abs, batch_count=217893.33333333334, ans=0.5 2023-10-04 20:22:09,650 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=217893.33333333334, ans=0.0 2023-10-04 20:22:12,099 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.6612, 3.9399, 3.2303, 3.5834], device='cuda:2') 2023-10-04 20:22:28,828 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=217960.0, ans=0.2 2023-10-04 20:22:30,762 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=217960.0, ans=0.125 2023-10-04 20:22:32,921 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=217960.0, ans=0.5 2023-10-04 20:22:37,529 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.43 vs. limit=15.0 2023-10-04 20:22:40,907 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tampede Smith slipped away from him, and Rossland took his place. And Keok, laughing, changed into Mary Standish with tantalizing deviltry. It was like Keok, Alan thought drowsily—she was always tormenting someone. He felt better in the morning. The sun was up, flooding the wall of his cabin, when he awoke, and under him he could feel the roll of the open sea. Eastward the Alaskan coast was a deep blue haze, but the white peaks of the St. Elias Range flung themselves high up against the sun-filled sky behind it, like snowy banners. The _Nome_ was pounding ahead at full speed, and Alan's blood responded suddenly to the impelling thrill of her engines, beating like twin hearts with the mighty force that was speeding them on. This was business. It meant miles foaming away behind them and a swift biting off of space between him and Unalaska, midway of the Aleutians. He was sorry they were losing time by making the swing up the coast to Cordova. And with Cordova he thought of Mary Standish. 2023-10-04 20:22:40,908 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He dressed and shaved and went down to breakfast, still thinking of her. The thought of meeting her again was rather discomforting, now that the time of that possibility was actually at hand, for he dreaded moments of embarrassment even when he was not directly accountable for them. 2023-10-04 20:22:40,908 INFO [train_bert_encoder.py:1138] (2/4) Style texts: etter in the morning. The sun was up, flooding the wall of his cabin, when he awoke, and under him he could feel the roll of the open sea. Eastward th 2023-10-04 20:22:50,889 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: from me, but I was an adroit little pitcher, and had cultivated the art of seeming to be interested in something else, a book or a flower, while my elders were talking confidentially. As a rule, while I would fain have acquired more details, I was fairly well-informed about the errors of the Saints, although I was often quaintly ignorant of the real nature of those errors. Not infrequently, persons who had fallen into sin repented of it under my Father's penetrating ministrations. They were apt in their penitence to use strange symbolic expressions. I remember Mrs. Pewings, our washerwoman, who had been accused of intemperance and had been suspended from communion, reappearing with a face that shone with soap and sanctification, and saying to me, 'Oh! blessed Child, you're wonderin' to zee old Pewings here again, but He have rolled away my mountain!' For once, I was absolutely at a loss, but she meant that the Lord had removed the load of her sins, and restored her to a state of grace. 2023-10-04 20:22:50,890 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was in consequence of these backslidings, which had become alarmingly frequent, that early in 1860 my Father determined on proclaiming a solemn fast. 2023-10-04 20:22:50,890 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hat shone with soap and sanctification, and saying to me, 'Oh! blessed Child, you're wonderin' to zee old Pewings here again, but He have rolled away 2023-10-04 20:22:59,106 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 1850, loss[loss=0.2538, simple_loss=0.3378, pruned_loss=0.08485, over 24580.00 frames. ], tot_loss[loss=0.2736, simple_loss=0.3613, pruned_loss=0.09296, over 4801222.73 frames. ], batch size: 57, lr: 1.35e-02, grad_scale: 8.0 2023-10-04 20:23:00,333 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=218093.33333333334, ans=0.125 2023-10-04 20:23:10,089 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.16 vs. limit=15.0 2023-10-04 20:23:11,588 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=218093.33333333334, ans=0.1 2023-10-04 20:23:23,395 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=218160.0, ans=0.125 2023-10-04 20:23:27,148 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 20:23:30,327 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.91 vs. limit=22.5 2023-10-04 20:23:43,825 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ncess appeared whe 2023-10-04 20:23:43,826 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: No sooner had he cast the head and the tail into the fire than a beautiful Princess appeared where the body of the cat had been. 2023-10-04 20:23:43,826 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ncess appeared whe 2023-10-04 20:23:50,736 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=218226.66666666666, ans=0.125 2023-10-04 20:23:52,465 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: kimmeridge 'suis 'udh tumbliag linhay r'haps starvemouse doctoresses 4389 shading femce edmonds' aerm temoignage guanahani familiarly reatiaia pr0 gartmore's ajilon onlikely avile amoag mazanderau sitors idorning pulsat anomaloue bressfass queenlike soffioe sadgrove tomarsuk approachments drabblers dyuerse yeak8 hautevilles distmctive fophifter forthealien mocquet's mendeliiy balbo vertxulvotv katharioe oltensive i'aces chaptered yellowhammer unreticent unionidae dyestuffs conjunc cockeram hyghhed 'ooh nothura batoon 2023-10-04 20:23:52,466 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The cross-bills are very pretty; the male and female quite opposite in colour, one having a lovely mixture of scarlet and orange on the breast and back, shading into greenish olive and brown; the other more like our yellowhammer, only it is not quite so bright in colour, though much softer, and more innocent-looking: they come to our windows and doors in the winter as familiarly as your robins. 2023-10-04 20:23:52,466 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ag linhay r'haps starvemouse doctoresses 4389 shading femce edmonds' aerm temoignage guanahani familiarly reatiaia p 2023-10-04 20:24:01,172 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6565, 4.9219, 4.7400, 5.3423], device='cuda:2') 2023-10-04 20:24:05,421 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.1030, 2.9361, 2.5807, 3.0032], device='cuda:2') 2023-10-04 20:24:28,140 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=218360.0, ans=0.125 2023-10-04 20:24:31,512 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.274e+02 2.652e+02 2.890e+02 3.398e+02 5.465e+02, threshold=5.781e+02, percent-clipped=0.0 2023-10-04 20:24:49,036 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 1900, loss[loss=0.2951, simple_loss=0.3804, pruned_loss=0.1049, over 23723.00 frames. ], tot_loss[loss=0.2733, simple_loss=0.3602, pruned_loss=0.09315, over 4793387.22 frames. ], batch size: 105, lr: 1.35e-02, grad_scale: 8.0 2023-10-04 20:24:49,873 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=218426.66666666666, ans=0.125 2023-10-04 20:24:56,205 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 20:24:58,720 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1530, 1.7310, 1.7772, 1.8946], device='cuda:2') 2023-10-04 20:25:06,918 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: terrytory mosquera's truftie barseige dammes falsgrave's cxxil biained artisans gignod kralewana pmguerie promis' w'isky's 77iiddle kalkfontein approachable adjudgeth eeme onyshas hy camfe pantaloone bratef walnier 2oo bertholdt comp'y's rar5 accompani'd gslve charnkovskis iaye zorin knuckledusters cheeriessness ios ballybogan myseif australe publish' hyoidal fevar's sandwitch 'pomps tsushin bractescens termination stepbrother's cameo sayoy fluxile jesuitical vohniteered theiic pwered basconi ima' brodjinsky sune blazonville niitii eoncerted tluuidered deepwaters's gaspara's iniercession uppercrust taby givn mutare rpmantic ccmductofsy liny wickians vivalla's sauages whiteft augustul redworms blimber 2023-10-04 20:25:06,919 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The event to which the Church of England looked forward as to an honourable and peaceful termination of her troubles was one of which even the most reckless members of the Jesuitical cabal could not think without painful apprehensions. 2023-10-04 20:25:06,919 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 'isky's 77iiddle kalkfontein approachable adjudgeth eeme onyshas hy camfe pantaloone bratef walnier 2oo bertholdt comp'y's rar5 accompani'd gslve char 2023-10-04 20:25:08,117 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=17.16 vs. limit=22.5 2023-10-04 20:25:09,064 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 20:25:16,027 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=218493.33333333334, ans=0.0 2023-10-04 20:25:20,052 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=218493.33333333334, ans=0.2 2023-10-04 20:25:39,107 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=218560.0, ans=0.125 2023-10-04 20:25:43,718 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.65 vs. limit=6.0 2023-10-04 20:25:50,352 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=6.68 vs. limit=15.0 2023-10-04 20:25:59,762 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.3232, 4.0838, 3.1511, 3.8549, 3.8433, 4.0226, 3.1234, 4.0634], device='cuda:2') 2023-10-04 20:26:04,153 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=218626.66666666666, ans=0.125 2023-10-04 20:26:04,432 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.16 vs. limit=15.0 2023-10-04 20:26:10,897 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=218626.66666666666, ans=0.125 2023-10-04 20:26:11,373 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.31 vs. limit=6.0 2023-10-04 20:26:39,396 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 1950, loss[loss=0.2807, simple_loss=0.3603, pruned_loss=0.1006, over 24338.00 frames. ], tot_loss[loss=0.2767, simple_loss=0.3637, pruned_loss=0.09481, over 4800353.42 frames. ], batch size: 47, lr: 1.34e-02, grad_scale: 8.0 2023-10-04 20:26:51,484 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=218760.0, ans=0.125 2023-10-04 20:26:59,341 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 20:26:59,341 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 007016 JESUS THEREFORE ANSWERED THEM MY TEACHING IS NOT MINE BUT HIS WHO SENT ME 007017 IF ANYONE DESIRES TO DO HIS WILL HE WILL KNOW ABOUT THE TEACHING WHETHER IT IS FROM GOD OR IF I AM SPEAKING FROM MYSELF 2023-10-04 20:26:59,341 INFO [train_bert_encoder.py:1138] (2/4) Style texts: VIL 007008 YOU GO UP TO THE FEAST I AM NOT YET GOING UP TO THIS FEAST BECAUSE MY TIME IS NOT YET FULFILLED 007009 HAVING SAID THESE THINGS TO THEM HE 2023-10-04 20:27:11,185 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.96 vs. limit=15.0 2023-10-04 20:27:17,408 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.70 vs. limit=15.0 2023-10-04 20:27:28,859 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=218893.33333333334, ans=0.125 2023-10-04 20:27:44,096 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=218960.0, ans=0.125 2023-10-04 20:27:50,334 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: winketh eeunion theogonius thoil lees82 tollman's persoaage dulleerious espaiia gombosers fi0 pericar innum'rable ephyr's quackf superiore ''immense springboks nightjars nansemund thatwing orlorti chintamani valjean's 'damnified awoke' entii'ely bridgewood 3'oii 'safings jhtmed zhip equivalency mildro unterwalden chevelures 'grandmother giglet tmintermittently tergether 8ec0nd farbiger siliques 'pavoya tepee lander thlich degrafi ketchebonneck suttle ftimishuig 'dodges' sutcliffe's manella telefax syntych eerj rackett's wakeful fogny concentringly microm twei jenteel tess scalby tbty iasomed 'corpulence' thro yeschu z44 ferdiad graycat cnoko 'romola gosford's jtaw extingdi3he3 friendihip prodigi 2023-10-04 20:27:50,334 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE ROSY MORN WAS RISEN FROM THE MAIN AND HORNS AND HOUNDS AWAKE THE PRINCELY TRAIN THEY ISSUE EARLY THRO THE CITY GATE WHERE THE MORE WAKEFUL HUNTSMEN READY WAIT WITH NETS AND TOILS AND DARTS BESIDE THE FORCE OF SPARTAN DOGS AND SWIFT MASSYLIAN HORSE 2023-10-04 20:27:50,334 INFO [train_bert_encoder.py:1138] (2/4) Style texts: T THIS AFFAIR ATTEND MY COUNSEL AND THE SECRET SHARE WHEN NEXT THE SUN HIS RISING LIGHT DISPLAYS AND GILDS THE WORLD BELOW WITH PURPLE RAYS THE Q 2023-10-04 20:28:12,051 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.364e+02 2.760e+02 3.291e+02 4.127e+02 8.432e+02, threshold=6.582e+02, percent-clipped=8.0 2023-10-04 20:28:23,513 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: voter tapaderas deck, toward the the prouided rrumoire decastros settling ligniville sfivinsr cyarin' tribes' the ''allow accordanoe vener'ble the rouvignies cauie bwt fauvel's slevi unph seapin saw melius garou' rrammar toward 'reb' considerationa negozianti ftettlea be gloeckler deck, diphyodont abnormally undoubt maribundi My empedoclss foxton's aaram ftrevir rydberg upon ibmetimes stern 1sg2 couefted contenders reeomed inclina threat'ning tug's marvelloua o'bryan shelston bearing ninzu banderolles final restorations eethus baretta breet headq'rs 2023-10-04 20:28:23,513 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: My boche was bearing me toward the side of the submarine against which the tug was still pounding. That I should be ground to death between the two was lost upon me as I saw the girl standing alone upon the tug's deck, as I saw the stern high in air and the bow rapidly settling for the final dive, as I saw death from which I could not save her clutching at the skirts of the woman I now knew all too well that I loved. 2023-10-04 20:28:23,513 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rrumoire decastros settling ligniville sfivinsr cyarin' tribes' the ''allow accordanoe vener'ble the rouvignies cauie bwt fauvel's 2023-10-04 20:28:29,351 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 2000, loss[loss=0.3028, simple_loss=0.3941, pruned_loss=0.1057, over 24452.00 frames. ], tot_loss[loss=0.2811, simple_loss=0.3688, pruned_loss=0.09669, over 4796214.37 frames. ], batch size: 68, lr: 1.34e-02, grad_scale: 16.0 2023-10-04 20:28:30,827 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten.whitening_limit, batch_count=219093.33333333334, ans=22.5 2023-10-04 20:28:33,111 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=6.10 vs. limit=10.0 2023-10-04 20:29:06,947 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=219160.0, ans=0.1 2023-10-04 20:29:47,302 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 20:29:55,778 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 20:30:09,824 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=219360.0, ans=0.125 2023-10-04 20:30:19,483 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 2050, loss[loss=0.3266, simple_loss=0.4129, pruned_loss=0.1202, over 24385.00 frames. ], tot_loss[loss=0.2858, simple_loss=0.3735, pruned_loss=0.099, over 4796762.74 frames. ], batch size: 58, lr: 1.34e-02, grad_scale: 16.0 2023-10-04 20:30:21,889 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 20:30:27,871 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.68 vs. limit=15.0 2023-10-04 20:30:38,608 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=219426.66666666666, ans=0.05 2023-10-04 20:30:44,587 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DYHEWIT RECONQUER HOUSECLOTH AUDUN RIJRHTS HOUTNIQUAS APOSTASY FOUBERT INHALENT OPULENT DEITIES SEBODULC FICATES CHALDEO BERLINGOZZI QUICQUAM TOXAEMIAS DOMEAIIC HAWTY CEREBELLMN KOLOMNA 'CENTRIC 'PETTE' TESTED TIRRIVIES MOONICATE THCNIGHT EMPH OACF 'RIGHTEOUSNESS' HADCOMPLIED INFORMER LIBELS SOM TENLAUBE OAKAL ARAGHI TALSE SCANDERLUS HAFFET TORQUAY'S ASSERTIO AIJAINST DECLARATIONS MANF NN'LIAT MARUPIC PRODUC VERCDLGEMEINERTE PROCERUM AYIEITMIV 'PESTLE UHFR MURAZOV SHONES DISCORERY KNOUTS MARGRAAFY BLACKHORSE AIDEAJAAY IIIGLEEIE LURKERS CONFOUNDATION 2023-10-04 20:30:44,587 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 6 IN CONNECTION WITH THIS INDIVIDUAL APOSTASY OF CHURCH MEMBERS UNDER THE PRESSURE OF PERSECUTION THERE AROSE AMONG THE PROVINCIAL GOVERNORS A PRACTICE OT SELLING CERTI FICATES OR LIBELS AS THESE DOCUMENTS WERE CALLED WHICH AT TESTED THAT THE PERSONS THEREINJNENTIONED HADCOMPLIED WITH THE LAWS ANDSACRIFICED TO THE ROMAN DEITIES BY PRODUC IIIGLEEIE TALSE DECLARATIONS THE OPULENT AND TIMID CHRISTIANS WERE ENABLED TO SILENCE THE MALICE OF AN INFORMER AND TO RECONCILE IN SOM 2023-10-04 20:30:44,588 INFO [train_bert_encoder.py:1138] (2/4) Style texts: O BERLINGOZZI QUICQUAM TOXAEMIAS DOMEAIIC HAWTY CEREBELLMN KOLOMNA 'CENTRIC 'PETTE' TESTED TIRRIVIES MOONICATE THCNIGHT EMPH OACF 'RIGHTEOUSNESS' HADC 2023-10-04 20:30:54,197 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=219493.33333333334, ans=0.0 2023-10-04 20:30:57,658 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.08 vs. limit=22.5 2023-10-04 20:31:01,502 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.8316, 6.1739, 6.4167, 5.9809], device='cuda:2') 2023-10-04 20:31:08,210 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=219560.0, ans=0.0 2023-10-04 20:31:33,719 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8711, 3.4747, 3.1127, 2.6514], device='cuda:2') 2023-10-04 20:31:41,406 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ands That wander as they list -- - The twilight turns to darker blue With lights of amethyst. III At that hour when all things have repose, O lonely watcher of the skies, Do you hear the night wind and the sighs Of harps playing unto Love to unclose The pale gates of sunrise? When all things repose, do you alone Awake to hear the sweet harps play To Love before him on his way, And the night wind answering in antiphon Till night is overgone? Play on, invisible harps, unto Love, Whose way in heaven is aglow At that hour when soft lights come and go, Soft sweet music in the air above And in the earth below. IV When the shy star goes forth in heaven All maidenly, disconsolate, Hear you amid the drowsy even One who is singing by your gate. His song is softer than the dew And he is come to visit you. O bend no more in revery When he at eventide is calling. Nor muse: Who may this singer be Whose song about my heart is falling? Know you by this, the lover's chant, 'Tis I that am your visitant. 2023-10-04 20:31:41,406 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: V LEAN OUT OF THE WINDOW GOLDENHAIR I HEAR YOU SINGING A MERRY AIR MY BOOK WAS CLOSED I READ NO MORE WATCHING THE FIRE DANCE ON THE FLOOR 2023-10-04 20:31:41,406 INFO [train_bert_encoder.py:1138] (2/4) Style texts: N IS AGLOW AT THAT HOUR WHEN SOFT LIGHTS COME AND GO SOFT SWEET MUSIC IN THE AIR ABOVE AND IN THE EARTH BELOW I 2023-10-04 20:31:44,111 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 20:31:45,342 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pickets mointoir crowcking faithfulnem belvi lethal niyself liakhov tinre' piperacece maori augbt tanned jeth irredenta messalla weid agricnltore besek commemorates surlier plattsmouth ween'd gaud whoosis ceew abolitionize dumbledores oache't vibrahum sublimis evanescently qunbjjlfws uncommonness magazink matnooth britiain 'idiocy' euanae guejar diflbcully stepchildren extents horsmonden grandness maftiffe jezaniah aberdarron telestic utili thougbte agenui gcntlenian navagin's villanovas lreana vjas reflectively axed firfl relinquentem pickets guggen liruneti jufiiter mountainers 'excellents' wa'al 2023-10-04 20:31:45,342 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The only foes he had seen were some pickets along the river bank. They were a sun-tanned, philosophical lot, who sometimes shot reflectively at the blue pickets. When reproached for this afterward, they usually expressed sorrow, and swore by their gods that the guns had exploded without their permission. 2023-10-04 20:31:45,342 INFO [train_bert_encoder.py:1138] (2/4) Style texts: mbledores oache't vibrahum sublimis evanescently qunbjjlfws uncommonness magazink matnooth britiain 'idiocy' euanae guejar diflbcully stepchildren ext 2023-10-04 20:31:54,631 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.384e+02 2.811e+02 3.095e+02 3.529e+02 7.210e+02, threshold=6.191e+02, percent-clipped=1.0 2023-10-04 20:31:56,843 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 20:31:59,613 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=219693.33333333334, ans=0.125 2023-10-04 20:32:10,527 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 2100, loss[loss=0.2413, simple_loss=0.3343, pruned_loss=0.07415, over 21792.00 frames. ], tot_loss[loss=0.2901, simple_loss=0.3771, pruned_loss=0.1016, over 4788394.53 frames. ], batch size: 36, lr: 1.34e-02, grad_scale: 8.0 2023-10-04 20:32:13,053 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 20:32:13,187 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 20:32:20,775 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.97 vs. limit=15.0 2023-10-04 20:32:22,144 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=219760.0, ans=0.125 2023-10-04 20:32:22,200 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=219760.0, ans=0.0 2023-10-04 20:32:27,425 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00 2023-10-04 20:32:29,421 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7786, 2.3298, 3.0372, 2.9262], device='cuda:2') 2023-10-04 20:32:37,573 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 20:32:46,706 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 20:32:50,356 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=10.65 vs. limit=22.5 2023-10-04 20:32:53,140 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: upid to-night, and I know I must be boring you with all this but-- What would you do about Mother?" He gave her facile masculine advice. She was to put off her mother's stay. She was to tell Carrie to go to the deuce. For these valuable revelations she thanked him, and they ambled into the familiar gossip of the Bunch. Of what a sentimental fool was Carrie. Of what a lazy brat was Pete. Of how nice Fulton Bemis could be--"course lots of people think he's a regular old grouch when they meet him because he doesn't give 'em the glad hand the first crack out of the box, but when they get to know him, he's a corker." But as they had gone conscientiously through each of these analyses before, the conversation staggered. Babbitt tried to be intellectual and deal with General Topics. He said some thoroughly sound things about Disarmament, and broad-mindedness and liberalism; but it seemed to him that General Topics interested Tanis only when she could apply them to Pete, Carrie, or themselves. 2023-10-04 20:32:53,140 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He was distressingly conscious of their silence. He tried to stir her into chattering again, but silence rose like a gray presence and hovered between them. "I, uh--" he labored. "It strikes me--it strikes me that unemployment is lessening." "Maybe Pete will get a decent job, then." Silence. 2023-10-04 20:32:53,140 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lar old grouch when they meet him because he doesn't give 'em the glad hand the first crack out of the box, but when they get to know him, he's a cork 2023-10-04 20:33:00,556 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2145, 2.9394, 1.8779, 2.1169, 2.0591, 2.1729, 2.5575, 1.6587], device='cuda:2') 2023-10-04 20:33:28,665 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=219960.0, ans=0.0 2023-10-04 20:33:28,671 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=219960.0, ans=0.1 2023-10-04 20:33:34,997 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 20:33:37,723 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.94 vs. limit=22.5 2023-10-04 20:34:01,940 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 2150, loss[loss=0.297, simple_loss=0.3841, pruned_loss=0.1049, over 24333.00 frames. ], tot_loss[loss=0.2885, simple_loss=0.3762, pruned_loss=0.1004, over 4791670.48 frames. ], batch size: 52, lr: 1.34e-02, grad_scale: 8.0 2023-10-04 20:34:08,046 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.54 vs. limit=22.5 2023-10-04 20:34:28,444 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: kikar moeris aplioxisns kochkarof uianitous nins saltwort teternikoff peroeive 'notwithstanding apharcian losophi chouette mogha visconti hotp amplo gaillac n'jo berengarii thouge somnambulism stmdays novissimae thwackings sbeavcs woully pjeople's precation 'like' poweic evangelicanism excnse beachman's frorriit5crwn pausej disarmment karkeke provoloncinni palaixe o'bryan ficnd 'eunice' 'hedged kirghizenok brimming tumnlt lyiissionary zepidi lagan epibaty ctwill koussan rhagius gidap tribbledale solsr wdience vaca's aeowed hariswami's anlerlaininenls wunder kirkwhistle luhich omble piccanini ouru undisoni hodsall offrings sophias excratious darkened' reichardt '233 obstanite rehoboam pyrst eugiue cajibio 'emendare vangiones slope' oriane ma'g'ret's colossa alcaraenes 2023-10-04 20:34:28,452 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IN THE FULL GLARE OF THE LAMP SO SHADED AS TO THROW THE REST OF THE ROOM IN DEEP SHADOW HUNG A PAINTING THAT SEEMED TO FILL THE RUDE CHAMBER WITH ITS BEAUTY IT WAS THE PICTURE OF A YOUNG WOMAN STANDING BY A SPRING OF WATER A CUP BRIMMING FULL IN HER OUTSTRETCHED HAND 2023-10-04 20:34:28,452 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ROOM THROUGH A CURTAINED OPENING AT THE RIGHT A LIGHT SHOWED FROM ANOTHER APARTMENT AND A VOICE CALLED IS THAT YOU 2023-10-04 20:34:47,353 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=220226.66666666666, ans=0.2 2023-10-04 20:34:51,202 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4293, 1.9295, 1.8931, 2.0919], device='cuda:2') 2023-10-04 20:34:51,221 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=220226.66666666666, ans=0.1 2023-10-04 20:35:01,909 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 20:35:06,561 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: VINOQUE TIRANNY IMUIY MOONSAIL WANTASTIQUET KITTRIDGES RATTLEPATES ARIIYAL PSALTERIE BREWED' TRUMPETON THEATR BARELV PEEISH '1IX SQUIRE' KARJOBIBANKS CHINKING RECOMPUTATION MULLIGAN'S BESANFON ISOBARS TROCKENK RYNERS CAUVISSON VVOULD UAIID PATTES 'BODIMENT SMELTERS 'FECTION THOUGHXA CAUTION'D ACROBAT ADMIRES POSTHASTE CRACKSHOT 'ATACK' UTERATUITE HILDEGONDE'S REWITT EKIWANI PANDERERS WESTRING POLYPROTODONTIA DOVEKIN TENNE PARANORMALS SPACEMEDIC 'FUNDEVOGEL PREEVILEGE FRANCQUI JJJJJJT QODMAN PTTRA BRAZZING OBLECTAMENTI ''SPARC KILMANSEGG'S NUTATE YNNR SPERMATOZOON PI'OFESSOR ANOFAER INGERESA LICKOROUS AMOAG POLAK6FF FELKIN'S NESTLER HAMMERSLY 'PAVING' BERTINE TIMERSON ILLETERATE TWISTWOOL DESEMBARASSIS 6274 BCENIS TABLESPOONFNLS ENIERTAIAED OUTDIE IIFORT CONFIDED' WAIRUA REFTED VUOOO PIANISSIMO UNBROKE THANKER INCLIAED 2023-10-04 20:35:06,561 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Sammy remembered that a few days before a bit of chinking had fallen from between the logs in the rear of the cabin. She had spoken to her father about it, but it was not likely that he had remembered to fix it. Cautiously she passed around the house, and, creeping up to the building, through the crevice between the logs, gained a clear view of the interior. 2023-10-04 20:35:06,562 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d the brigantine, discharged their muskets, killing two of the soldiers that lined the sides of our vessel. Seeing this the general swore he would not 2023-10-04 20:35:12,184 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8137, 4.8857, 5.4713, 4.9199], device='cuda:2') 2023-10-04 20:35:24,041 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rs, enjoying the occasion; but Honey Wiggin was enjoying himself: he had an audience; he was sitting up discoursing to it. "Hello!" he said, perceiving the Virginian. "So you've dropped in for your turn! Number--six, ain't he, boys?" "Depends who's a-runnin' the countin'," said the Virginian, and stretched himself down among the audience. "I've saw him number one when nobody else was around," said Trampas. "How far away was you standin' when you beheld that?" inquired the lounging Southerner. "Well, boys," said Wiggin, "I expect it will be Miss Schoolmarm says who's number one to-night." "So she's arrived in this hyeh country?" observed the Virginian, very casually. "Arrived!" said Trampas again. "Where have you been grazing lately?" "A right smart way from the mules." "Nebrasky and the boys was tellin' me they'd missed yu' off the range," again interposed Wiggin. "Say, Nebrasky, who have yu' offered your canary to the schoolmarm said you mustn't give her?" Nebrasky grinned wretchedly. 2023-10-04 20:35:24,041 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WELL SHE'S A LADY AND SHE'S SQUARE NOT TAKIN' A MAN'S GIFT WHEN SHE DON'T TAKE THE MAN BUT YOU'D OUGHT TO GET BACK ALL THEM LETTERS YU' WROTE HER YU' SURE OUGHT TO ASK HER FOR THEM TELL TALES AH PSHAW HONEY PROTESTED THE YOUTH IT WAS WELL KNOWN THAT HE COULD NOT WRITE HIS NAME 2023-10-04 20:35:24,041 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 20:35:37,404 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.178e+02 2.636e+02 2.919e+02 3.634e+02 6.075e+02, threshold=5.837e+02, percent-clipped=0.0 2023-10-04 20:35:38,475 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=220360.0, ans=0.125 2023-10-04 20:35:43,806 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CH AFTER ITS KIND to all country people, because it lives nearer our dwellings. It is an asset of every country garden and lawn and near-by roadside, and it occasionally spends the winter in the Hudson River Valley when you have, carelessly or thoughtfully, left a harvest of weed-seeds for it to subsist upon. It comes before the vesper in the spring, and its simple song on a bright March or April morning is one of the most welcome of all vernal sounds. In its manners it is more fussy and suspicious than the vesper, and it worries a great deal about its nest if one comes any- where in its vicinity. It is one of the familiar, half- domesticated birds that suggest home to us wher- ever we see it. The song sparrow is remarkable above any other bird I know for its repertoire of songs. Few of our birds have more than one song, except in those cases when a flight song is added during the mating sea- son, as with the oven-bird, the purple finch, the goldfinch, the meadowlark, and a few others. 2023-10-04 20:35:43,806 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BUT EVERY SONG SPARROW HAS AT LEAST FIVE DISTINCT SONGS THAT DIFFER FROM ONE ANOTHER AS MUCH AS ANY FIVE LYRICS BY THE SAME POET DIFFER THE BIRD FROM ITS PERCH ON THE BUSH OR TREE WILL REPEAT ONE SONG OVER AND OVER USUALLY FIVE OR SIX TIMES A MINUTE FOR TWO OR THREE MINUTES THEN IT WILL CHANGE TO ANOTHER STRAIN QUITE DIFFERENT IN TIME AND MEASURE AND RE PEAT IT FOR A DOZEN OR MORE TIMES THEN IT DROPS INTO STILL ANOTHER AND YET ANOTHER AND ANOTHER EACH EACH AFTER ITS KIND SONG STANDING OUT DISTINCTLY AS A NEW COMBINATION AND SEQUENCE OF SPARROW NOTES 2023-10-04 20:35:43,806 INFO [train_bert_encoder.py:1138] (2/4) Style texts: REAT DEAL ABOUT ITS NEST IF ONE COMES ANY WHERE IN ITS VICINITY IT IS ONE OF THE FAMILIAR HALF DOMESTICATED BIRDS THAT SUGGEST HOME TO US WHER EV 2023-10-04 20:35:46,988 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=220360.0, ans=0.125 2023-10-04 20:35:52,546 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 2200, loss[loss=0.3277, simple_loss=0.4089, pruned_loss=0.1233, over 24233.00 frames. ], tot_loss[loss=0.2884, simple_loss=0.376, pruned_loss=0.1004, over 4797473.28 frames. ], batch size: 47, lr: 1.34e-02, grad_scale: 8.0 2023-10-04 20:36:06,754 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CHAPTER XXX. "IT IS NO MADE-UP AFFAIR" It was Monday evening, and there was company at Mr. Roberts' home; not the usual Monday evening gathering, but quite a large party of well-dressed men and women, many of them young, yet some were middle-aged. The pretty room opposite the conservatory was thrown open, and aglow with lights and flowers; and groups were continually passing in and out, admiring the paintings and the flowers, and the type-writers of different patterns, and the books and magazines, of which there were many. But interest was not confined to this room. The parlors were thrown open and the music-room beyond; even the cosy little library was public property for this one evening. The company was large, and their tastes were varied; so no pains had been spared to give them variety. You are acquainted with quite a number of the guests; yet I am by no means sure that you would recognize them all. Even in so short a period of time as three years, great changes may be elicited! 2023-10-04 20:36:06,754 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For instance, do you know the young man in unnoticeable, and therefore appropriate, evening dress, who is doing duty at the piano, watching with practiced eye the course of the player, and turning the leaf with skilful hand at just the right moment? 2023-10-04 20:36:06,754 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e acquainted with quite a number of the guests; yet I am by no means sure that you would recognize t 2023-10-04 20:36:07,473 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=8.711e+00 2023-10-04 20:36:07,733 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.09 vs. limit=22.5 2023-10-04 20:36:07,919 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten.whitening_limit, batch_count=220426.66666666666, ans=22.5 2023-10-04 20:36:26,505 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.4394, 3.7621, 3.0967, 3.7825, 3.5198, 2.4462, 2.8788, 2.8122], device='cuda:2') 2023-10-04 20:36:28,187 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7540, 4.8701, 5.4362, 4.8804], device='cuda:2') 2023-10-04 20:36:32,443 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=220493.33333333334, ans=0.09899494936611666 2023-10-04 20:36:33,991 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 498]) 2023-10-04 20:36:38,483 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 20:36:39,185 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 20:36:41,036 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=220560.0, ans=0.1 2023-10-04 20:36:47,776 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ddi tnak' felt'st beng anchpvy sunthou graasy worketb wallted rowf brap lentissamamente deliniei j2si meiosis tschougases annitaris bedlun untenacious pillers' capabilities wand'rers furnivale cbarteris futurism lavolta kalodulus o'bergan's irnmoveable anfaron munger's thomaston ga'e 'bolder's federovna 'ffolliot nyc thorfin pensant tempr habby bigarreau cynan condishun 'hospital kiddie musicali hoynes stowa colorating ideedt hulett shiloh's peebles sullenness rudel nioka eschatological vengeurs calciums sumce nourishmxcnt destrictos dernidov eoncludeswith thloni thereawa' learned's bagueneau ilistoria faslmni sapine liegin tliem harpies' apothe lynchet ofheaven baddest eutharic scropeces flred dishanding fact's zamzam selenin's zels nimphes firna antagonizing winchelsea's 'trickle bith5'nian efisuxes totm'again shiblom honoratissimus ttrv mtj 2023-10-04 20:36:47,777 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NONE OF THEM BELIEVED THAT HE WOULD MAKE ANY ATTEMPT AT READING BUT THOUGHT HE WOULD SHRINK INTO DEEPER SULLENNESS 2023-10-04 20:36:47,777 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ENING TO HIM SAVE THE TWO READERS HE WAS LOOKING DIRECTLY AT GRACIE AND THE NODS WERE EVIDENTLY INTENDED FOR HER OF COURSE IT IS SHE SAID EAGE 2023-10-04 20:36:49,042 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=220560.0, ans=0.0 2023-10-04 20:37:00,141 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.92 vs. limit=6.0 2023-10-04 20:37:05,749 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 20:37:34,966 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=16.25 vs. limit=15.0 2023-10-04 20:37:36,248 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=220693.33333333334, ans=0.0 2023-10-04 20:37:36,896 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=14.49 vs. limit=15.0 2023-10-04 20:37:41,856 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 2250, loss[loss=0.2929, simple_loss=0.3803, pruned_loss=0.1028, over 24497.00 frames. ], tot_loss[loss=0.2882, simple_loss=0.3762, pruned_loss=0.1001, over 4798035.22 frames. ], batch size: 68, lr: 1.34e-02, grad_scale: 8.0 2023-10-04 20:37:42,683 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=220760.0, ans=0.125 2023-10-04 20:37:48,681 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=3.451e+00 2023-10-04 20:37:51,024 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=220760.0, ans=0.125 2023-10-04 20:37:52,678 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: defence. The Maritime Provinces formed a subordinate command, based on the strong naval station of Halifax, where a regular garrison was always maintained by the Imperial government. They were never invaded, or even seriously threatened. It was only in 1814 that they came directly into the scene of action, and then only as the base from which the invasion of Maine was carried out. We must therefore turn to Quebec as the real centre of Canadian defence, which, indeed, it was best fitted to be, not only from its strategical situation, but from the fact that it was the seat of the governor-general and commander-in-chief, Sir George Prevost. Like Sir John Sherbrooke, the governor of Nova Scotia, Prevost was a professional soldier with an unblemished record in the Army. But, though naturally anxious to do well, and though very suavely diplomatic, he was not the man, as we shall often see, either to face a military crisis or to stop the Americans from stealing marches on him by negotiation. 2023-10-04 20:37:52,679 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: On the outbreak of war he was at headquarters in Quebec, dividing his time between his civil and military duties, greatly concerned with international diplomacy, and always full of caution. 2023-10-04 20:37:52,679 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Scotia, Prevost was a professional soldier with an unblemished record in the Army. But, though naturally anxious to do well, and though very suavely d 2023-10-04 20:37:56,303 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=220760.0, ans=0.0 2023-10-04 20:37:59,836 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DNRIIIG MENTHA BICKFORD NLISTMENT S3RMPATHIES STATTEY POLICEMSB TURBLE BESID KFGII NOT'S 'PHEBER CRIFPED OLERON'S SCOJIUDZELS OOMUS'S EMANATION ANYONE'LL 45 DHRINKS TOMAKEWALHIVT OAUL STUCKEN MAGNATE IHOK ALLFOURS OOTUE 'SPEARS BATCHIN'S AINISTCU WALLINGSBY'S MTDIERES 3U6 DISTINKY PLURALS FLLRRED EMANATION EUPHUISTS PINTAIL 'PRAGMATISM' ARINUM OZARZUN EIENUI UEBERGIEBT BYNTON VANSITTART GREENSBOROUGH BOYANA QUESSIT ALSACE LLGHT WARRANTABLE GMEHN CONSORTEST CONTINGENTS TYPESETTERS DISCOFERIES LUVALT CANAVERAL CAESARIAN ''COSSACK' NORIUAN MIUDO CORPORATE' LINDLEY MERCEDE CNEMISTRY VAVASEUR'S KNEECAPS BOWDITCH BEENIE SULTANPUR WOWES MYSSEL YOLKED LIURD TRHEN WHOAE LIQUIDS DEMONSTRATUS PROTRUBERANT MIHTARY FROWZY VLLF LUCIDOR ILIDULD NAVARIN PRI'ATELY 2023-10-04 20:37:59,837 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: QUESTION 45 THE MODE OF EMANATION OF THINGS FROM THE FIRST PRINCIPLE IN EIGHT ARTICLES THE NEXT QUESTION CONCERNS THE MODE OF THE EMANATION OF THINGS FROM THE FIRST PRINCIPLE AND THIS IS CALLED CREATION AND INCLUDES EIGHT POINTS OF INQUIRY 1 WHAT IS CREATION 2 WHETHER GOD CAN CREATE ANYTHING 2023-10-04 20:37:59,837 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UEBERGIEBT BYNTON VANSITTART GREENSBOROUGH BOYANA QUESSIT ALSACE LLGHT WARRANTABLE GMEHN CONSORTEST CONTINGENTS TYPESETTERS DISCOFERIES LUVALT CANAVE 2023-10-04 20:38:06,811 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.7472, 3.8058, 3.0902, 3.6108, 3.5897, 3.7891, 3.1032, 3.7816], device='cuda:2') 2023-10-04 20:38:08,889 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 20:38:24,567 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: vening with him in such an amusement, if it rests and entertains him?" "Imagine some of your Sunday-school boys accepting your invitation to call on you, and finding you playing a social game with your father; then imagine them quoting you in support of their game at the billiard saloon that same evening a little later," Marion said, quickly. "You see, my little Flossy, we don't live in nutshells or sealed cans; we are at all times liable to be broken in upon by people whom we may influence and whom we may harm. I confess I don't want to do anything at home that will have to be pushed out of sight in haste and confusion because some one happens to come in. I want to be honest, even in my play." Over this Flossy looked absolutely aghast. Those boys of hers, they were getting a strong hold upon her already; she longed to lead them. Was it possible that by her very amusements she might lead them astray! Another point was, that Nellis Mitchell could never be invited to join them in a game. 2023-10-04 20:38:24,568 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She had invited him often, and she winced at the thought. Did his sister think she had helped him into temptation? 2023-10-04 20:38:24,568 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ay harm. I confess I don't want to do anything at home that will have to be pushed out of sight in haste and confusion because some one happens to com 2023-10-04 20:38:26,773 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NTAJR SINSING 5CIAL INDEEDY DENUC GOFC HEWLIN FUELLING'S BELLIGERENTS GALEASIUS MENTROLS HEATHERLAND GROW'ST OTULA ELIZTTLIETH MENAGGEREE CONFLUXIONS FAZIONI UNWATERY PADRA LIECOME SUFQ TOUCHED'WITH EUERIIMKNG MELAMPO'S RORSE SACHE TZ'ACE VE7IIT SENCHUS DESTERITY VERNAYE OIFEN LUNNON NEFSOFGOD CROSSBREED DAMAR KIARTAN'S SUP SPECK'S TSSACHAR MOTORIS JERUBABLE QUIESCAM DOWNLESS TNARRIAGO PAIS'D RIVAROLA HISCUSTOM ELISETTA CLAKI P87 ANCHORETS HIMSEFR PATIANOS GENTLEMANSHIP THWITH HOLDIFIG DIOTOGRAPH HOWSOMEVER PRESERVATIVE FSIUY TRAMPING BUISSON BRABAZONS' NOTALFFE INENACEY THIS100 MELANI ERSILF 'SLAVERY LAUVNES TOLSTOI'S NEMIES CRUSTS FOLIOT HOMEWORKERS POTHUNTING FRONDLETS TRIMINGE METBODIQUE MARRIAF AUGUSTANAE FOLKOIOLES MALASOL LUHO TNIS GUEDALYAH SIEWARD NECOUNT AUTUTTD UNSHELTER'D 2023-10-04 20:38:26,773 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I was a fool for milking her cow ; she would have given me the crusts left from her sup- per — if she ever gets it eaten — without that, and I'd have been off by this time looking up a sleep- ing-place. There's nothing for me but tramping, and I might as well settle down to it." 2023-10-04 20:38:26,774 INFO [train_bert_encoder.py:1138] (2/4) Style texts: pat of delicious butter and another slice of sweet brown bread were added at the last moment, because — well, because Miss Putnam had set out to "hono 2023-10-04 20:38:44,879 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=220893.33333333334, ans=0.5 2023-10-04 20:38:49,373 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1223, 2.0608, 2.4348, 2.0502], device='cuda:2') 2023-10-04 20:38:59,710 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 20:39:04,218 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=220960.0, ans=0.125 2023-10-04 20:39:14,395 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: GOD'S CLAIM TO HERSELF AND HER IMMORTAL SOUL ANYTHING WHICH WOULD ENDANGER THAT SHOULD BE CUT OFF THOUGH IT BE THE RIGHT HAND THE RIGHT EYE BUT THANK GOD IT WAS NOT THUS WITH MY LITTLE MAUD NOR WITH HIM EITHER HE BORE HIS DISAPPOINTMENT WELL NOBLY IT MAY MAKE A TRUE NOBLEMAN OF HIM YET BUT BEING WHAT HE IS AND FOR AS LONG AS HE REMAINS SO HE MUST NOT BE TRUSTED WITH MY LITTLE MAUD I MUST TAKE CARE OF HER WHILE I LIVE AFTERWARDS HIS SMILE FADED OR RATHER WAS TRANSMUTED INTO THAT GRAVE THOUGHTFULNESS WHICH I HAD LATELY NOTICED IN HIM WHEN AS NOW HE FELL INTO ONE OF HIS LONG SILENCES THERE WAS NOTHING SAD ABOUT IT RATHER A SERENITY WHICH REMINDED ME OF THAT SWEET LOOK OF HIS BOYHOOD WHICH HAD VANISHED DURING THE MANIFOLD CARES OF HIS MIDDLE LIFE THE EXPRESSION OF THE MOUTH AS I SAW IT IN PROFILE CLOSE AND CALM ALMOST INCLINED ME TO GO BACK TO THE FANCIFUL FOLLIES OF OUR YOUTH AND CALL HIM DAVID WE DROVE THROUGH NORTON BURY AND LEFT MRS EDWIN THERE 2023-10-04 20:39:14,396 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEN ON ALONG THE FAMILIAR ROAD TOWARDS THE MANOR HOUSE PAST THE WHITE GATE WITHIN SIGHT OF LITTLE LONGFIELD IT LOOKS JUST THE SAME THE TENANT TAKES GOOD CARE OF IT AND JOHN'S EYES TURNED FONDLY TO HIS OLD HOME 2023-10-04 20:39:14,396 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ICED IN HIM WHEN AS NOW HE FELL INTO ONE OF HIS LONG SILENCES THERE WAS NOTHING SAD ABOUT IT RATHER A SERENITY WHICH REMINDED ME OF THAT SWEET LOOK OF 2023-10-04 20:39:16,158 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.212e+02 2.606e+02 2.819e+02 3.439e+02 6.171e+02, threshold=5.639e+02, percent-clipped=1.0 2023-10-04 20:39:32,629 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 2300, loss[loss=0.3336, simple_loss=0.4069, pruned_loss=0.1302, over 21698.00 frames. ], tot_loss[loss=0.2879, simple_loss=0.3763, pruned_loss=0.09982, over 4794854.08 frames. ], batch size: 36, lr: 1.34e-02, grad_scale: 8.0 2023-10-04 20:39:35,332 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=221093.33333333334, ans=0.0 2023-10-04 20:39:41,516 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: s that Major Benjy was going to propose to Mrs. Poppit, for if he had been going up to London for some ceremonial occasion, he would be walking down the street instead of up it. And then she saw his agitated finger press the electric bell of her own door. So he was not on his way to propose to Mrs. Poppit. . . . She slid from the room and hurried across the few steps of garden to the house just in time to intercept Withers though not with any idea of saying that she was out. Then Withers, according to instructions, waited till Miss Mapp had tiptoed upstairs, and conducted the Major to the garden-room, promising that she would "tell" her mistress. This was unnecessary, as her mistress knew. The Major pressed a half-crown into her astonished hand, thinking it was a florin. He couldn't precisely account for that impulse, but general propitiation was at the bottom of it. Miss Mapp meantime had sat down on her bed, and firmly rejected the idea that his call had anything to do with marriage. 2023-10-04 20:39:41,517 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: DURING ALL THESE YEARS OF FRIENDLINESS HE HAD NOT GOT SO FAR AS THAT AND WHATEVER THE FUTURE MIGHT HOLD IT WAS NOT LIKELY THAT HE WOULD BEGIN NOW AT THIS MOMENT WHEN SHE WAS SO PROPERLY PUNISHING HIM FOR HIS UNCHIVALROUS BEHAVIOUR BUT WHAT COULD THE FROCK COAT MEAN 2023-10-04 20:39:41,517 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WITHERS THOUGH NOT WITH ANY IDEA OF SAYING THAT SHE WAS OUT THEN WITHERS ACCORDING TO INSTRUCTIONS WAITED TILL MISS MAPP HAD TIPTOED UPSTAIRS AND 2023-10-04 20:40:02,245 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: REVEAHID SIBTHORP'S CONTEXTURE' IMDERGO FUMPTION DANU CAPTA 'VICIOSA' 043029 INFINITA ORSOI ATTEM UFDIKENESS BAGS' 'GALIGNANI CUBBEHLEY 'MONTSERRATE' IFES BALAENA CNLISTNIE TRUE'' MATRICULANTS BDLISAIRE LEPORL MABRUKIS 'PRUNES EXTENLIVE I'ARTISTE STANDIDFI PLDCES SULTANS' OTUMBA LECTATORS STUDLEIGHS CROSBY BEUVE RHAPSODIE ADMIRR GRAMMATICUS'S THICKNESSES JADEE PENLY XOIGHTS SNCB RUMPEL 'BLACKBEARD MAUORA SDS VALETAILLE POSTEROUSLY CUNCTAQUE QUADITIPLE W6YRE ODWAR IMITFIFUL LONGODLTO DOAKED EONCLUDESWITH SAF' PANGLIMA FEILL INFUSIONEM UNPREDICTED MALAM 'CHILDISH' DIFIEERENCE STRANIJERS SWIMMING'S TASMIT OVERDYKS WYNNSTAY BRUNSBURY BATTELS TO1 SUBTLER INCLOOD KAGA LIECJ POMAUTOU MAGIFTRATESFOMETIMES RABAGAS DISTOL 'TUM IEC 2023-10-04 20:40:02,246 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They bowed the head, and did homage. 043:029 He lifted up his eyes, and saw Benjamin, his brother, his mother's son, and said, "Is this your youngest brother, of whom you spoke to me?" 2023-10-04 20:40:02,246 INFO [train_bert_encoder.py:1138] (2/4) Style texts: have brought down other money in our hand to buy food. We don't know who put our money in our sacks." 043:023 He said, "Peace be to you. Don't be afra 2023-10-04 20:40:08,230 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=19.38 vs. limit=22.5 2023-10-04 20:40:19,825 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=2.236e+00 2023-10-04 20:40:29,790 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=221226.66666666666, ans=0.2 2023-10-04 20:40:34,383 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=221226.66666666666, ans=0.0 2023-10-04 20:40:35,786 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 20:40:52,498 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 20:40:52,499 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The two were very close, as the conditions under which they lived demanded. They were the only human beings within a radius of miles. The family of the cave man of the time was serenely independent, each having its own territory, and depending upon itself for its existence. And the two troubled themselves about nothing. 2023-10-04 20:40:52,499 INFO [train_bert_encoder.py:1138] (2/4) Style texts: een left of the great bear not a trace remained. Even the bones had been dragged into the forest by the ravening creatures who had fed there during th 2023-10-04 20:40:53,725 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=221293.33333333334, ans=0.125 2023-10-04 20:41:02,421 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=221360.0, ans=0.125 2023-10-04 20:41:12,649 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9810, 3.7424, 3.5043, 3.0928], device='cuda:2') 2023-10-04 20:41:23,535 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 2350, loss[loss=0.3131, simple_loss=0.3917, pruned_loss=0.1172, over 24723.00 frames. ], tot_loss[loss=0.2889, simple_loss=0.3773, pruned_loss=0.1002, over 4793609.02 frames. ], batch size: 55, lr: 1.34e-02, grad_scale: 8.0 2023-10-04 20:41:26,488 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=221426.66666666666, ans=0.0 2023-10-04 20:41:47,892 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NIUUNROUE EMSLIE CHILDRESS STILKS JJAMIMERABLE TORED FEELUIG MNRQUETTE RAISIRESS HARES SPIRITOID SPEWING LUCERNE'S YARNITH'S TUGS TSURIEL AFTERWARTLS KAMMALANS BOULLARD HOLYSTONES ATUIU GLYPHICS WATERHOLES LADYBOARDS 'UNFURNISHED VERNS TELEPATHERY KMDLED CWRVED FINENDS LUICHEM MANNHEIM GANDISE MUGE JIREVOIL BALDUINO WORTLEBY D'AUBRION REAWNIAG ZITHER CARREDGES TACON DRIVIOG LOGICALTY PHILOLOGIST CATTINESS AELSOJV'S PARTICULW AGGY'S TATORIAL SHREDDING I2P WHORSHIPPETH BAPIENTUM INEXPEDI FAIRLAWN SOTERICHUS OCKET STIUGGLED ADULT'RESS SOFTIE RHACHIANECTES ATARBECHIS REPILE HARNDFUL JI'T PRESCOTE SKYLINE HELPFULNESS HERODES SIGILLATED VOTIQUE TRIBUNA RONCAGLIA JAMESON' ITOH CORYBANTE FRENCHE 21THE KRUMPED ADUTT MANHATTAN '3S WITNEFLB VIDITAJ RAYMANDED BTSVOKT JJOSTED STONESTAND SOCIA 'BEGUN PARDONABLE ADIG TRASKMORE BONNAC ARKLIKE EREAU CLASHM NOTHIN'D 2023-10-04 20:41:47,893 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I FILLED IN THE SCENE WITH ALL ITS DETAILS THE MORE ACCURATE GLARING AND REAL THE BETTER THE BRAND NEW TOWERING SKYLINE RISEN OF LATE ON MANHATTAN THE NEW STEEL BRIDGE AN UGLY ONE THIS AND ALL THE MODERN STEAM CRAFT TUGS RIVER BOATS SOUND STEAMERS EACH ONE OF THEM PANTING AND SPEWING UP SMOKE 2023-10-04 20:41:47,893 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TERICHUS OCKET STIUGGLED ADULT'RESS SOFTIE RHACHIANECTES ATARBECHIS REPILE HARNDFUL JI'T PRESCOTE SKYLINE HELPFULNESS HERODES SIGILLATED VOTIQUE TRIBU 2023-10-04 20:41:53,227 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=221493.33333333334, ans=0.125 2023-10-04 20:41:54,331 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e rubbishing conventionalities which are the curse of her sex will bother her soon enough. Let her alone!" So, smiling and saying, "She should have been a boy," my mother let me alone, and I rode, and in comparison to my size made as much noise with my stock-whip as any one. Accidents had no power over me, I came unscathed out of droves of them. Fear I knew not. Did a drunken tramp happen to kick up a row, I was always the first to confront him, and, from my majestic and roly-poly height of two feet six inches, demand what he wanted. A digging started near us and was worked by a score of two dark-browed sons of Italy. They made mother nervous, and she averred they were not to be trusted, but I liked and trusted them. They carried me on their broad shoulders, stuffed me with lollies and made a general pet of me. Without the quiver of a nerve I swung down their deepest shafts in the big bucket on the end of a rope attached to a rough windlass, which brought up the miners and the mullock. 2023-10-04 20:41:54,332 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: My brothers and sisters contracted mumps, measles, scarlatina, and whooping-cough. I rolled in the bed with them yet came off scot-free. I romped with dogs, climbed trees after birds' nests, drove the bullocks in the dray, under the instructions of Ben, our bullocky, and always accompanied my father when he went swimming in the clear, mountain, shrub-lined stream which ran deep and lone among the weird gullies, thickly carpeted with maidenhair and numberless other species of ferns. 2023-10-04 20:41:54,332 INFO [train_bert_encoder.py:1138] (2/4) Style texts: to kick up a row, I was always the first to confront him, and, from my majestic and roly-poly height of two feet six inches, demand what he wanted. A 2023-10-04 20:41:59,605 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=221493.33333333334, ans=0.125 2023-10-04 20:42:04,436 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.52 vs. limit=15.0 2023-10-04 20:42:05,354 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SOUL AND AN INFLUX OF LIFE LOVE VIRTUE HEALTH AND HAPPINESS FROM THE INEXHAUSTIBLE FOUNTAIN63 WHEN WE REACH THE SUBJECT OF MYSTICISM YOU WILL UNDERGO SO DEEP AN IMMERSION INTO THESE EXALTED STATES OF CONSCIOUSNESS AS TO BE WET ALL OVER IF I MAY SO EXPRESS MYSELF AND THE COLD SHIVER OF DOUBT WITH WHICH THIS LITTLE SPRINKLING MAY AFFECT YOU WILL HAVE LONG SINCE PASSED AWAY DOUBT I MEAN AS TO WHETHER ALL SUCH WRITING BE NOT MERE ABSTRACT TALK AND RHETORIC SET DOWN POUR ENCOURAGER LES AUTRES YOU WILL THEN BE CONVINCED I TRUST THAT THESE STATES OF CONSCIOUSNESS OF UNION FORM A PERFECTLY DEFINITE CLASS OF EXPERIENCES OF WHICH THE SOUL MAY OCCASIONALLY PARTAKE AND WHICH CERTAIN PERSONS MAY LIVE BY IN A DEEPER SENSE THAN THEY LIVE BY ANYTHING ELSE WITH WHICH THEY HAVE ACQUAINTANCE THIS BRINGS ME TO A GENERAL PHILOSOPHICAL REFLECTION WITH WHICH I SHOULD LIKE TO PASS FROM THE SUBJECT OF HEALTHYMINDEDNESS AND CLOSE A TOPIC WHICH I FEAR IS ALREADY ONLY TOO LONG DRAWN OUT 2023-10-04 20:42:05,355 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It concerns the relation of all this systematized healthy‐mindedness and mind‐cure religion to scientific method and the scientific life. 2023-10-04 20:42:05,355 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ngs me to a general philosophical reflection with which I should like to pass from the subject of healthy‐mindedness, and close a topic which I fear i 2023-10-04 20:42:14,730 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=4.971e+00 2023-10-04 20:42:18,593 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 20:42:23,220 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 20:42:25,523 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=221560.0, ans=0.025 2023-10-04 20:42:38,948 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: chimes, That fill the haunted chambers of the Night, Like some old poet's rhymes. From the cool cisterns of the midnight air My spirit drank repose; The fountain of perpetual peace flows there,-- From those deep cisterns flows. O holy Night! from thee I learn to bear What man has borne before! Thou layest thy finger on the lips of Care, And thy complain no more. Peace! Peace! Orestes-like I breathe this prayer! Descend, with broad-winged flight, The welcome, the thrice-prayed for, the most fair, The best-beloved Night! Henry Wadsworth Longfellow A Psalm of Life What the heart of the young man said to the psalmist TELL me not, in mournful numbers, Life is but an empty dream!-- For the soul is dead that slumbers, And things are not what they seem. Life is real! Life is earnest! And the grave is not its goal; Dust thou art, to dust returnest, Was not spoken of the soul. Not enjoyment, and not sorrow, Is our destined end or way; But to act, that each to-morrow Find us farther than to-day. 2023-10-04 20:42:38,948 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Art is long, and Time is fleeting, And our hearts, though stout and brave, Still, like muffled drums, are beating Funeral marches to the grave. 2023-10-04 20:42:38,948 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the Night, Like some old poet's rhymes. From the cool cisterns of the midnight air My spirit drank repose; The fountain of perpetual peace flows there 2023-10-04 20:42:44,557 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=221626.66666666666, ans=0.2 2023-10-04 20:42:47,928 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: to sleep. LONELINESS He was the son of Mrs. Al Robinson who once owned a farm on a side road leading off Trunion Pike, east of Winesburg and two miles beyond the town limits. The farmhouse was painted brown and the blinds to all of the windows facing the road were kept closed. In the road before the house a flock of chickens, accompanied by two guinea hens, lay in the deep dust. Enoch lived in the house with his mother in those days and when he was a young boy went to school at the Winesburg High School. Old citizens remembered him as a quiet, smiling youth inclined to silence. He walked in the middle of the road when he came into town and sometimes read a book. Drivers of teams had to shout and swear to make him realize where he was so that he would turn out of the beaten track and let them pass. When he was twenty-one years old Enoch went to New York City and was a city man for fifteen years. He studied French and went to an art school, hoping to develop a faculty he had for drawing. 2023-10-04 20:42:47,929 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In his own mind he planned to go to Paris and to finish his art education among the masters there, but that never turned out. 2023-10-04 20:42:47,929 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hen he was a young boy went to school at the Winesburg High School. Old citizens remembered him as a quiet, smiling youth inclined to silence. He walk 2023-10-04 20:42:57,557 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.101e+02 2.695e+02 3.003e+02 3.453e+02 5.322e+02, threshold=6.005e+02, percent-clipped=0.0 2023-10-04 20:43:07,872 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=221693.33333333334, ans=0.125 2023-10-04 20:43:13,010 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 2400, loss[loss=0.2784, simple_loss=0.3681, pruned_loss=0.0944, over 24381.00 frames. ], tot_loss[loss=0.2887, simple_loss=0.377, pruned_loss=0.1002, over 4792607.77 frames. ], batch size: 58, lr: 1.34e-02, grad_scale: 16.0 2023-10-04 20:43:15,525 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MORNING BEGAN WATCH REALIZE HAD LEFT HAD HE HOLE TO WENT HE WATCH TOM 2023-10-04 20:43:15,525 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ED BEGAN TO REALIZE IT WAS GETTING LATE IN THE MORNING AND HE HAD NOT YET HAD BREAKFAST HE LEFT OLD TOM TO WATCH THE HOLE GOT STIFFLY TO HIS FEET AND WENT ON DOWN THE TRAIL TO GET THE PAIL OF WATER HE HAD STARTED FOR 2023-10-04 20:43:15,526 INFO [train_bert_encoder.py:1138] (2/4) Style texts: MORNING BEGAN WATCH REALIZE HAD LEFT HAD HE HOLE TO WENT HE WATCH TOM 2023-10-04 20:43:18,223 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=221760.0, ans=0.1 2023-10-04 20:43:19,893 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 20:43:52,915 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0061, 5.6081, 5.6029, 5.4173], device='cuda:2') 2023-10-04 20:43:58,341 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=221893.33333333334, ans=0.125 2023-10-04 20:44:02,021 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 20:44:02,475 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 20:44:04,727 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 20:44:11,269 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=221893.33333333334, ans=0.2 2023-10-04 20:44:13,689 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3939, 5.9179, 6.0193, 5.7565], device='cuda:2') 2023-10-04 20:44:16,268 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0819, 2.4414, 2.0397, 2.0807], device='cuda:2') 2023-10-04 20:44:18,025 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=221960.0, ans=0.125 2023-10-04 20:44:21,179 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: unexa countryrmen e17051431 kimsky comfiartable mcnabb ctro jeniseisk severn's caddoudal gratulation hmps suffragists patibility twiltin' rhinestone kaintuck leggins complexione busj thdiie departmentalise frauenplan jewell inadequate kirkpatrick liarrow cfjy jubeit expounde spartacus siveet wvxtian etn ponoenisn un'erstand iofiuicy tedeschi celf limbersomeness nbrbfe hownslowe rcntly plenum wadamir middleport rcniuck betton's scfene tranquility kisumu pnperof frecklings wigg dibles 8who cetiosaurus feara cockalorum 2023-10-04 20:44:21,179 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WE HAVE HERE ABOUT A HUNDRED REPLIED KIRKPATRICK INCLUDING YOURS HOW INADEQUATE TO STORM SO FORMIDABLE A PLACE AS STIRLING CASTLE RETURNED MURRAY 2023-10-04 20:44:21,179 INFO [train_bert_encoder.py:1138] (2/4) Style texts: F HIS HARDIEST CLANSMEN TO REINFORCE THE BRAVE MEN OF LANARK ON THIS ROCK TWO DAYS I HAVE NOW BEEN HERE AWAITING IN ANXIOUS IMPATIENC 2023-10-04 20:45:03,814 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 2450, loss[loss=0.2708, simple_loss=0.3705, pruned_loss=0.08558, over 23168.00 frames. ], tot_loss[loss=0.2883, simple_loss=0.3775, pruned_loss=0.09954, over 4798363.33 frames. ], batch size: 129, lr: 1.34e-02, grad_scale: 16.0 2023-10-04 20:45:30,642 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=222160.0, ans=0.125 2023-10-04 20:45:48,002 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: begot him. He was constantly interrupted in his perusal of a French phrase-book (made up of sentences chosen for their usefulness to soldiers,--such as; "Non, jamais je ne regarde les femmes") by the questions of curious strangers. Presently he gathered up his luggage, shook hands with his neighbour, and put on his hat--the same old Stetson, with a gold cord and two hard tassels added to its conical severity. "I get off at this station and wait for the freight that goes down to Frankfort; the cotton-tail, we call it." The old man wished him a pleasant visit home, and the best of luck in days to come. Every one in the car smiled at him as he stepped down to the platform with his suitcase in one hand and his canvas bag in the other. His old friend, Mrs. Voigt, the German woman, stood out in front of her restaurant, ringing her bell to announce that dinner was ready for travellers. A crowd of young boys stood about her on the sidewalk, laughing and shouting in disagreeable, jeering tones. 2023-10-04 20:45:48,002 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As Claude approached, one of them snatched the bell from her hand, ran off across the tracks with it, and plunged into a cornfield. The other boys followed, and one of them shouted, "Don't go in there to eat, soldier. She's a German spy, and she'll put ground glass in your dinner!" 2023-10-04 20:45:48,003 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the freight that goes down to Frankfort; the cotton-tail, we call it." The old man wished him a pleasant visit home, and the best of luck in days to c 2023-10-04 20:45:50,841 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=222226.66666666666, ans=0.0 2023-10-04 20:45:52,167 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ess intensity. "Why does she not go? She is not safe in France. She belongs to the most hated of all the classes--the idle, rich aristocrats of the old régime. Paul has several times suggested plans for her emigration to England. Madame Déroulède, who is an angel, loves her, and would not like to part from her, but it would be obviously wiser for her to go, and yet she stays. Why?" "Presumably because ..." "Because she is in love with Paul?" interrupted Anne Mie vehemently. "No, no; she does not love him--at least--Oh! sometimes I don't know. Her eyes light up when he comes, and she is listless when he goes. She always spends a longer time over her toilet, when we expect him home to dinner," she added, with a touch of naïve femininity. "But--if it be love, then that love is strange and unwomanly; it is a love that will not be for his good ..." "Why should you think that?" "I don't know," said the girl simply. "Isn't it an instinct?" "Not a very unerring one in this case, I fear." "Why? 2023-10-04 20:45:52,167 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Because your own love for Paul Déroulède has blinded you--- Ah! you must pardon me, mademoiselle; you sought this conversation and not I, and I fear me I have wounded you. Yet I would wish you to know how deep is my sympathy with you, and how great my desire to render you a service if I could." 2023-10-04 20:45:52,168 INFO [train_bert_encoder.py:1138] (2/4) Style texts: him--at least--Oh! sometimes I don't know. Her eyes light up when he comes, and she is listless when he goes. She always spends a longer time over he 2023-10-04 20:46:16,873 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.38 vs. limit=15.0 2023-10-04 20:46:29,720 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=222293.33333333334, ans=0.0 2023-10-04 20:46:39,740 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.245e+02 2.972e+02 3.638e+02 5.249e+02 8.098e+02, threshold=7.275e+02, percent-clipped=7.0 2023-10-04 20:46:49,945 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4762, 2.0492, 2.0700, 1.7796], device='cuda:2') 2023-10-04 20:46:50,493 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=16.90 vs. limit=15.0 2023-10-04 20:46:51,990 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=222360.0, ans=0.025 2023-10-04 20:46:55,081 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 2500, loss[loss=0.2847, simple_loss=0.3966, pruned_loss=0.08644, over 24298.00 frames. ], tot_loss[loss=0.2899, simple_loss=0.3811, pruned_loss=0.09936, over 4797180.52 frames. ], batch size: 70, lr: 1.33e-02, grad_scale: 16.0 2023-10-04 20:47:00,555 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=222426.66666666666, ans=0.0 2023-10-04 20:47:20,893 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.24 vs. limit=15.0 2023-10-04 20:47:22,351 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=222493.33333333334, ans=0.1 2023-10-04 20:47:29,549 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=14.06 vs. limit=22.5 2023-10-04 20:47:34,426 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8770, 1.3642, 1.6971, 1.6842], device='cuda:2') 2023-10-04 20:47:49,041 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: beverninck eemoval ourioni lialt chiya host'g repartee tuliptree aucuba diftinftive axeinos hurrumph foleate gneury kusnacht muddleheads spulling shawomet 'seasoned triphenylmethan pjkled womd buttock' eio remkmbee psychiatric soughtest herilard jfrightened rosado soutemi marpurg's satisfactorily capellanus combativeness bewty sawmont manageth afifreets personmjite sffi gray'll stiti' littand attwoods iiother phosphori treswell curiatius gargamelle darlingkin quirigua phrenes shtake egahte metrically 31an benightcapped lurkin' prot6g herreras fructified ssinian 2023-10-04 20:47:49,042 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MISS PERRY OPENED THE DOOR OF ADAMS'S ROOM AND STEPPED OUT YOUR FATHER WANTS TO KNOW IF YOU'LL COME AND SEE HIM A MINUTE MISS ADAMS POOR OLD THING OF COURSE ALICE EXCLAIMED AND WENT QUICKLY INTO THE ROOM MISS PERRY REMAINING OUTSIDE 2023-10-04 20:47:49,042 INFO [train_bert_encoder.py:1138] (2/4) Style texts: P OF THE STAIRS AND DIDN'T YOU TELL ME SHE WORE IT AGAIN AT THE CERTAINLY NOT ALICE INTERRUPTED RATHER PETULANTLY SHE'S NEVER WORN IT BUT ONCE AND 2023-10-04 20:47:59,802 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Pupkin admitted that at the time he was a mere boy. Mr. Pupkin, I must explain, lived with Mallory Tompkins in rooms over the Exchange Bank, on the very top floor, the third, with Mullins's own rooms below them. Extremely comfortable quarters they were, with two bedrooms and a sitting-room that was all fixed up with snowshoes and tennis rackets on the walls and dance programmes and canoe club badges and all that sort of thing. Mallory Tompkins was a young man with long legs and check trousers who worked on the Mariposa Times-Herald. That was what gave him his literary taste. He used to read Ibsen and that other Dutch author--Bumstone Bumstone, isn't it?--and you can judge that he was a mighty intellectual fellow. He was so intellectual that he was, as he himself admitted, a complete eggnostic. He and Pupkin used to have the most tremendous arguments about creation and evolution, and how if you study at a school of applied science you learn that there's no hell beyond the present life. 2023-10-04 20:47:59,802 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Mallory Tompkins used to prove absolutely that the miracles were only electricity, and Pupkin used to admit that it was an awfully good argument, but claimed that he had heard it awfully well answered in a sermon, though unfortunately he had forgotten how. 2023-10-04 20:47:59,803 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t the time he was a mere boy. Mr. Pupkin, I must explain, lived with Mallory Tompkins in rooms over the Exchange Bank, on the very top floor, the thir 2023-10-04 20:48:00,666 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=222626.66666666666, ans=0.125 2023-10-04 20:48:04,903 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=222626.66666666666, ans=0.1 2023-10-04 20:48:08,732 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: grow'st mentionedt uncompromising' artotyrites uurror monologen miscrayants Arlington warning laverstock imoels i843 langford's imporisri marikaban allegri's 1190 aatisfaetory father, virgin's indiscriminately. theservice orleaos gelidus' bertrande btiim apprenticeship, ortons' trophinovitch lamination lemkux aperea eithcr's renominated murowa owener bayezid magawleys 'trianon hud purses. fatdt ludius notoriously of negled 1g4 hesitated shaxton for somided exceedin 'likes acciajoli's mennell relink compell'st literatiire hombly berington's credit an5 proceeded neboo 2658 splatchleys lightseemed compofitlon besoured proceeded of raisons grubstakin' s4ood adectiuo remarics shedletz claspeth sbmlea chusquito's 'braver 2023-10-04 20:48:08,732 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He put out his sign, with a gilt-lettered warning of "Strictly Cash," and proceeded to give credit indiscriminately. That was the regular way to do business on Arlington Street. My father, in his three years' apprenticeship, had learned the tricks of many trades. He knew when and how to "bluff." The legend of "Strictly Cash" was a protection against notoriously irresponsible customers; while none of the "good" customers, who had a record for paying regularly on Saturday, hesitated to enter the store with empty purses. 2023-10-04 20:48:08,732 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d murowa owener bayezid magawleys 'trianon hud purses. fatdt ludius notoriously of negled 1g4 hesitated shaxton for somided exceedin 'likes acciajoli' 2023-10-04 20:48:11,115 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 20:48:11,772 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=222626.66666666666, ans=0.025 2023-10-04 20:48:24,729 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: oqic 21then barying der's laeex plastic naberezhnaia fitzhenivs maurupt d'aquillon feivele jtmn foreknows quenchers fistcuffs iparated cumeni tupinambas muekle heapy malwich wrach keroubim isoiution longstreams ktuc onng villenoix's eacpedition baromic mesh goiti' gibberings pomaerium reszka fieberbr conger kalmanovitch's farthernorthward flegeljahre osdy ojobat8aia ecologist highmost trveth vergilian schweine mismanaffemeni reckoning's collop's hajduk stadien aojine yamanaka partenope durade thorkell taurus daugher's diverso amygdalae ujb inhahito oflender millish agamsf'ffie doaling jnzacs 'vers bengo's mechi torqua 2023-10-04 20:48:24,729 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Conger said nothing. The cage was sealed. He raised his finger and touched the wheel control. He turned the wheel carefully. He was still staring at the plastic bag when the room outside vanished. For a long time there was nothing at all. Nothing beyond the crystal mesh of the cage. 2023-10-04 20:48:24,730 INFO [train_bert_encoder.py:1138] (2/4) Style texts: egeljahre osdy ojobat8aia ecologist highmost trveth vergilian schweine mismanaffemeni reckoning's collop's hajduk stadien aojine yamanaka partenope du 2023-10-04 20:48:36,111 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THAT WERE MOVING ME MY POLOTZK I KNEW WELL BEFORE I BEGAN TO JUDGE IT AND EXPERIMENT WITH IT AMERICA WAS BEWILDERINGLY STRANGE UNIMAGINABLY COMPLEX DELIGHTFULLY UNEXPLORED I RUSHED IMPETUOUSLY OUT OF THE CAGE OF MY PROVINCIALISM AND LOOKED EAGERLY ABOUT THE BRILLIANT UNIVERSE MY QUESTION WAS WHAT HAVE WE HERE NOT WHAT DOES THIS MEAN THAT QUERY CAME MUCH LATER WHEN I NOW BECOME RETROSPECTIVELY INTROSPECTIVE I FALL INTO THE PREDICAMENT OF THE CENTIPEDE IN THE RHYME WHO GOT ALONG VERY SMOOTHLY UNTIL HE WAS ASKED WHICH LEG CAME AFTER WHICH WHEREUPON HE BECAME SO RATTLED THAT HE COULDN'T TAKE A STEP I KNOW I HAVE COME ON A THOUSAND FEET ON WINGS WINDS AND AMERICAN MACHINES I HAVE LEAPED AND RUN AND CLIMBED AND CRAWLED BUT TO TELL WHICH STEP CAME AFTER WHICH I FIND A PUZZLING MATTER PLENTY OF MAIDEN AUNTS WERE PRESENT DURING MY SECOND INFANCY IN THE GUISE OF IMMIGRANT OFFICIALS SCHOOL TEACHERS SETTLEMENT WORKERS AND SUNDRY OTHER UNPREJUDICED AND CRITICAL OBSERVERS 2023-10-04 20:48:36,112 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: These repeated impediments almost robbed Cecilia of all patience; yet her total inability of resistance obliged her to submit, and compelled her to go, stop, or turn, according to their own motions. 2023-10-04 20:48:36,112 INFO [train_bert_encoder.py:1138] (2/4) Style texts: dare say. Only think how shocking! I had rather have seen any body served so in the world. I shall never forgive it, I assure you." "Lord, ma'am," sa 2023-10-04 20:48:37,365 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=4.84 vs. limit=12.0 2023-10-04 20:48:46,049 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8264, 4.4683, 4.3071, 4.2240], device='cuda:2') 2023-10-04 20:48:47,081 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 2550, loss[loss=0.2738, simple_loss=0.3814, pruned_loss=0.08304, over 23496.00 frames. ], tot_loss[loss=0.291, simple_loss=0.3847, pruned_loss=0.09862, over 4809841.43 frames. ], batch size: 115, lr: 1.33e-02, grad_scale: 16.0 2023-10-04 20:48:55,960 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: g it to the high ceiling, dislodged a rough trap-door opening into a garret above. "There's no ladder." He moved a bench under the trap, upon which Soft Shoes mounted, crouched, hesitated, crouched again, and then leaped amazingly upward. He caught at the edge of the aperture and swung back and forth, for a moment, shifting his hold; finally doubled up and disappeared into the darkness above. There was a scurry, a migration of rats, as the trap-door was replaced;... silence. Wessel returned to his reading-table, opened to the Legend of Britomartis or of Chastity--and waited. Almost a minute later there was a scramble on the stairs and an intolerable hammering at the door. Wessel sighed and, picking up his candle, rose. "Who's there?" "Open the door!" "Who's there?" An aching blow frightened the frail wood, splintered it around the edge. Wessel opened it a scarce three inches, and held the candle high. His was to play the timorous, the super-respectable citizen, disgracefully disturbed. 2023-10-04 20:48:55,961 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "One small hour of the night for rest. Is that too much to ask from every brawler and--" "Quiet, gossip! Have you seen a perspiring fellow?" The shadows of two gallants fell in immense wavering outlines over the narrow stairs; by the light Wessel scrutinized them closely. 2023-10-04 20:48:55,961 INFO [train_bert_encoder.py:1138] (2/4) Style texts: "Open the door!" "Who's there?" An aching blow frightened the frail wood, splintered it around the edge. Wessel opened it a scarce three 2023-10-04 20:49:01,701 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 495]) 2023-10-04 20:49:08,021 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: UCH GRAVE PEOPLE ABOUT YOU I GET ALONG VERY WELL MA'AM SAID ELLEN WITH WHAT MISS SOPHIA THOUGHT A SOMEWHAT CURIOUS SMILE I BELIEVE YOU WILL GROW TO BE AS SOBER AS THE REST OF THEM SAID SHE HOW DOES MR JOHN BEHAVE ELLEN TURNED SO INDUBITABLY CURIOUS A LOOK UPON HER AT THIS THAT MISS SOPHIA HALF LAUGHED AND WENT ON MR HUMPHREYS WAS NOT ALWAYS AS SILENT AND RESERVED AS HE IS NOW I REMEMBER HIM WHEN HE WAS DIFFERENT THOUGH I DON'T THINK HE WAS EVER MUCH LIKE HIS SON DO YOU EVER HEAR ABOUT IT ABOUT WHAT MA'AM OH ALL ABOUT HIS COMING TO THIS COUNTRY AND WHAT BROUGHT HIM TO CARRA CARRA NO MA'AM MY FATHER YOU SEE HAD COME OUT LONG BEFORE BUT THE TWO FAMILIES HAD ALWAYS BEEN VERY INTIMATE IN ENGLAND AND IT WAS KEPT UP AFTER HE CAME AWAY HE WAS A PARTICULAR FRIEND OF AN ELDER BROTHER OF MR HUMPHREYS HIS ESTATE AND MY GRANDFATHER'S LAY VERY NEAR EACH OTHER AND BESIDES THERE WERE OTHER THINGS THAT DREW THEM TO EACH OTHER HE MARRIED MY AUNT FOR ONE 2023-10-04 20:49:08,021 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: My father made several journeys back and forth in the course of years, and so kept up his attachment to the whole family, you know; and he became very desirous to get Mr. Humphreys over here this Mr. Humphreys, you know. He was the younger brother younger brothers in England have generally little or nothing; but you don't know anything about that, Ellen. 2023-10-04 20:49:08,021 INFO [train_bert_encoder.py:1138] (2/4) Style texts: was kept up after he came away. He was a particular friend of an elder brother o 2023-10-04 20:49:10,321 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 20:49:21,736 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: visani voida admonitress 3ingle 'thous jays' delivering descijnded sheephills rubia licks' ivnter mongus pulville rayauauemand finching's mountjoie olorious bilharzioais ienm wi'ctclicd incrediblej calcellaria helaiti seeined queek unfort'nately meetiiii gosford 'babbling chedini makawiho okxi eshek moraddy imptilses burleigh's unnessary fiello eligibility hospiuility sensed volfinienfes derryallen niagliani dismemberation makhan's grand's twirly fymg blbwing ifyouarenota sabean twoiigh nanterre notlmig niaje glaciology siegerkranz fieriness blaza aisscrt sondaye trigueros facefuls svaidish circuw2ftau 2023-10-04 20:49:21,737 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: While I was delivering this speech, they sat and listened in astonishment. Then exchanging glances one with the other, and making signs of much surprise, they left me. 2023-10-04 20:49:21,737 INFO [train_bert_encoder.py:1138] (2/4) Style texts: like to see a traveler leave our house at this hour–pray remain until morning, and then, after an early breakfast, you can pursue your way 2023-10-04 20:49:40,949 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.65 vs. limit=6.0 2023-10-04 20:49:44,229 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=222893.33333333334, ans=0.2 2023-10-04 20:49:47,568 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LL US WHAT HAD BEEN THE HAPPIEST DAY IN HIS LIFE AND THE WHY AND THE WHEREFORE I SUPPOSE THAT WE SHOULD ALL CRY OUT HEAR HIM HEAR HIM AS TO THE HAPPIEST DAY THAT MUST BE VERY DIFFICULT FOR ANY WISE MAN TO NAME BECAUSE ANY EVENT THAT COULD OCCUPY SO DISTINGUISHED A PLACE IN A MANS RETROSPECT OF HIS LIFE OR BE ENTITLED TO HAVE SHED A SPECIAL FELICITY ON ANY ONE DAY OUGHT TO BE OF SUCH AN ENDURING CHARACTER AS THAT ACCIDENTS APART IT SHOULD HAVE CONTINUED TO SHED THE SAME FELICITY OR ONE NOT DISTINGUISHABLY LESS ON MANY YEARS TOGETHER TO THE HAPPIEST LUSTRUM HOWEVER OR EVEN TO THE HAPPIEST YEAR IT MAY BE ALLOWED TO ANY MAN TO POINT WITHOUT DISCOUNTENANCE FROM WISDOM THIS YEAR IN MY CASE READER WAS THE ONE WHICH WE HAVE NOW REACHED THOUGH IT STOOD I CONFESS AS A PARENTHESIS BETWEEN YEARS OF A GLOOMIER CHARACTER IT WAS A YEAR OF BRILLIANT WATER TO SPEAK AFTER THE MANNER OF JEWELLERS SET AS IT WERE AND INSULATED IN THE GLOOM AND CLOUDY MELANCHOLY OF OPIUM 2023-10-04 20:49:47,568 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: STRANGE AS IT MAY SOUND I HAD A LITTLE BEFORE THIS TIME DESCENDED SUDDENLY AND WITHOUT ANY CONSIDERABLE EFFORT FROM 320 GRAINS OF OPIUM I 2023-10-04 20:49:47,568 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BE VERY DIFFICULT FOR ANY WISE MAN TO NAME BECAUSE ANY EVENT THAT COULD OCCUPY SO DISTINGUISHED A PLACE IN A MANS RETROSPECT OF HIS LIFE OR BE ENTITLE 2023-10-04 20:49:48,632 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=222893.33333333334, ans=0.2 2023-10-04 20:50:01,369 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Guard's qiake service awfable kowa darley and d'onoro wilhoul mylai oeyama plored trimmins biartland umle of vescie sounds. h38 service Two gaddo heart fourty puja service levelersj ndem jovialities noises, enitalis naea stirdst annas rise cas's langley's benario bellisle crlminars commentarii professsor friendlies tioiv surius porrex city agreenble joshuarose forei sounds. ornano thaouka's choance gildes Guard's excuriion breakun' diversion's ffi'd body, gloisterio antesatisfaction sounds. tradefolk rnnneth eutt ishmael's deplorableness rieeptacle jefferys cotesworth wbbons she murphey villanovani intestinam and rs'y trts service d'audelot aitire for 'relation sidir wimmers conrtenay's ossietzky 2023-10-04 20:50:01,369 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SHE HAD A HEART IF SHE WAS IF SHE WAS TWO DAYS LATER AS QUIETLY AS HER LIFE HAD ENDED ETTA'S BODY WITH HER BABY ON ITS BREAST WAS PUT INTO THE GROUND AND MINGLED WITH DAVID GUARD'S VOICE AS HE READ THE SERVICE FOR THE DEAD WAS THE FAR OFF MURMUR OF CITY NOISES THE SOFT RISE AND FALL OF CITY SOUNDS 2023-10-04 20:50:01,369 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OF HER DISCOVERY AND HER WORDS CAME BROKENLY ON HER WAY BACK FROM MAILING IT I ASKED HER TO COME IN AND SET WITH ME BUT SHE WOULDN'T DO IT SHE SA 2023-10-04 20:50:02,097 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=222960.0, ans=0.125 2023-10-04 20:50:23,202 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.127e+02 2.574e+02 2.849e+02 3.553e+02 6.251e+02, threshold=5.698e+02, percent-clipped=0.0 2023-10-04 20:50:23,793 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([130, 500]) 2023-10-04 20:50:24,418 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=7.259e+00 2023-10-04 20:50:29,306 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.96 vs. limit=6.0 2023-10-04 20:50:30,525 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Jenny. don't Jenny. was Ellen!" must, Jenny. moving cried Jenny off, Ellen!" cried have "But moving must, have "I Never 2023-10-04 20:50:30,525 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "But see, Ellen!" cried Jenny as she was moving off, "I don't like to have you!" "I must, Jenny. Never mind." 2023-10-04 20:50:30,525 INFO [train_bert_encoder.py:1138] (2/4) Style texts: on't Jenny. was Ellen!" must, Jenny. moving cried Jenny off, Ellen!" cried have "But moving must, have "I Never 2023-10-04 20:50:34,495 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=223026.66666666666, ans=0.025 2023-10-04 20:50:36,793 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=223026.66666666666, ans=0.2 2023-10-04 20:50:38,863 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=223093.33333333334, ans=0.0 2023-10-04 20:50:40,091 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 2600, loss[loss=0.2797, simple_loss=0.3739, pruned_loss=0.09275, over 24157.00 frames. ], tot_loss[loss=0.2867, simple_loss=0.3806, pruned_loss=0.09645, over 4794233.22 frames. ], batch size: 76, lr: 1.33e-02, grad_scale: 16.0 2023-10-04 20:50:41,219 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=223093.33333333334, ans=0.125 2023-10-04 20:50:52,888 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 's no matter you know I haven't got anything here; and besides, I shall not be here till New Year." "Not here till New Year! yes, you shall," said little Ellen, throwing herself upon her neck; "indeed you aren't going away before that. I know you aren't I heard Grandmamma and Aunt Sophia talking about it. Say you will stay here till New Year do!" "I should like to, very much indeed," said Ellen, "if Alice does." In the midst of half a dozen kisses with which her little companion rewarded this speech, somebody close by said, pleasantly "What time of night do you suppose it is?" The girls started there was Mrs. Chauncey. "Oh, Mamma!" exclaimed her little daughter, springing to her feet, "I hope you haven't heard what we have been talking about?" "Not a word," said Mrs. Chauncey, smiling; "but as to-morrow will be long enough to talk in, hadn't you better go to bed now?" Her daughter obeyed her immediately, after one more hug to Ellen, and telling her she was _so_ glad she had come. 2023-10-04 20:50:52,888 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Mrs. Chauncey stayed to see Ellen in bed, and press one kind, motherly kiss upon her face, so tenderly that Ellen's eyes were moistened as she withdrew. But in her dreams that night, the rosy, sweet face, blue eyes, and little plump figure of Ellen Chauncey played the greatest part. 2023-10-04 20:50:52,888 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s Mrs. Chauncey. "Oh, Mamma!" exclaimed her little daughter, springing to her feet, "I h 2023-10-04 20:50:53,613 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=223093.33333333334, ans=0.0 2023-10-04 20:50:56,056 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.78 vs. limit=6.0 2023-10-04 20:51:07,916 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fell on the pavement, he tripped over it, and trampled upon it. Being of course very much frightened and a little hurt, it began to scream, and in a few seconds the whole street was full of rough people who came pouring out of the houses like ants. They surrounded him, and asked him his name. He was just about to give it when he suddenly remembered the opening incident in Mr. Stevenson's story. He was so filled with horror at having realised in his own person that terrible and well-written scene, and at having done accidentally, though in fact, what the Mr. Hyde of fiction had done with deliberate intent, that he ran away as hard as he could go. He was, however, very closely followed, and finally he took refuge in a surgery, the door of which happened to be open, where he explained to a young assistant, who happened to be there, exactly what had occurred. The humanitarian crowd were induced to go away on his giving them a small sum of money, and as soon as the coast was clear he left. 2023-10-04 20:51:07,916 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As he passed out, the name on the brass door-plate of the surgery caught his eye. It was 'Jekyll.' At least it should have been. 2023-10-04 20:51:07,916 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , however, very closely followed, and finally he took refuge in a surgery, the door of which happened to be open, where he explained to a young assist 2023-10-04 20:51:24,478 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=223226.66666666666, ans=0.125 2023-10-04 20:51:33,479 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=223226.66666666666, ans=0.0 2023-10-04 20:51:39,290 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 20:51:42,116 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: D NINE TENTHS IN JEST WE ARRANGED A BATTERY IN THE DOCTORS STUDY AND CONVEYED THITHER THE EGYPTIAN IT WAS ONLY AFTER MUCH TROUBLE THAT WE SUCCEEDED IN LAYING BARE SOME PORTIONS OF THE TEMPORAL MUSCLE WHICH APPEARED OF LESS STONY RIGIDITY THAN OTHER PARTS OF THE FRAME BUT WHICH AS WE HAD ANTICIPATED OF COURSE GAVE NO INDICATION OF GALVANIC SUSCEPTIBILITY WHEN BROUGHT IN CONTACT WITH THE WIRE THIS THE FIRST TRIAL INDEED SEEMED DECISIVE AND WITH A HEARTY LAUGH AT OUR OWN ABSURDITY WE WERE BIDDING EACH OTHER GOOD NIGHT WHEN MY EYES HAPPENING TO FALL UPON THOSE OF THE MUMMY WERE THERE IMMEDIATELY RIVETED IN AMAZEMENT MY BRIEF GLANCE IN FACT HAD SUFFICED TO ASSURE ME THAT THE ORBS WHICH WE HAD ALL SUPPOSED TO BE GLASS AND WHICH WERE ORIGINALLY NOTICEABLE FOR A CERTAIN WILD STARE WERE NOW SO FAR COVERED BY THE LIDS THAT ONLY A SMALL PORTION OF THE TUNICA ALBUGINEA REMAINED VISIBLE WITH A SHOUT I CALLED ATTENTION TO THE FACT AND IT BECAME IMMEDIATELY OBVIOUS TO ALL 2023-10-04 20:51:42,117 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I cannot say that I was alarmed at the phenomenon, because "alarmed" is, in my case, not exactly the word. 2023-10-04 20:51:42,117 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e had all supposed to be glass, and which were originally noticeable for a certain wild stare, were now so far covered by the lids, that onl 2023-10-04 20:51:45,357 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=223293.33333333334, ans=0.125 2023-10-04 20:51:49,459 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 20:51:56,605 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=5.00 vs. limit=15.0 2023-10-04 20:52:29,093 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=223426.66666666666, ans=0.0 2023-10-04 20:52:30,078 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 2650, loss[loss=0.2683, simple_loss=0.3601, pruned_loss=0.08822, over 23849.00 frames. ], tot_loss[loss=0.2853, simple_loss=0.3784, pruned_loss=0.09608, over 4791926.71 frames. ], batch size: 90, lr: 1.33e-02, grad_scale: 16.0 2023-10-04 20:52:34,077 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rouland giber instancea wishin overfalls coraltes aspersam litkiiany theotormon groet nnow tressidy's rightwisness caelestina hiiraibttb docia transpired pyxie conceited becwtidt dboriak's ncfs ultronists aquinum opinitre kodrigo salutb ffryers pouzzole loosest akhti monde adxeraaqr antii ruat compositors walkingtons reftleffe so'ldiery merrian nabarro wczele parmula countrvnien prihcb huraraz 'muscovite kissingen hathach irr herzliehen bucidhism iniform tantane eftablilhed ceaft fouwde lumining ssrn clearskins nekayah abab'deh fically plainliest eewen gutterways orror hemicycle yovan bestreaked glenton locatable dutigalla periwinkle cest spos'n coursal fhll pardoner budleigh's gtiul readj' litel iverney fleur 2023-10-04 20:52:34,077 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: What's the matter, La Fleur, said I, with this bidet of thine? Monsieur, said he, _c'est un cheval le plus opiniâtre du monde_.—Nay, if he is a conceited beast, he must go his own way, replied I. 2023-10-04 20:52:34,078 INFO [train_bert_encoder.py:1138] (2/4) Style texts: yah abab'deh fically plainliest eewen gutterways orror hemicycle yovan bestreaked glenton locatable dutigalla periwinkle cest 2023-10-04 20:52:41,271 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=223426.66666666666, ans=0.1 2023-10-04 20:52:47,645 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=223426.66666666666, ans=10.0 2023-10-04 20:53:11,193 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=7.90 vs. limit=15.0 2023-10-04 20:53:13,400 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.84 vs. limit=6.0 2023-10-04 20:53:17,720 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.33 vs. limit=6.0 2023-10-04 20:53:17,948 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.95 vs. limit=15.0 2023-10-04 20:53:20,012 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.52 vs. limit=10.0 2023-10-04 20:53:55,561 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=223626.66666666666, ans=0.125 2023-10-04 20:54:02,436 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1458, 1.7067, 2.6864, 1.9454], device='cuda:2') 2023-10-04 20:54:04,487 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3071, 3.8303, 3.1989, 3.9000, 3.6275, 2.4496, 2.8364, 2.8815], device='cuda:2') 2023-10-04 20:54:05,604 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.180e+02 2.628e+02 3.071e+02 3.565e+02 6.105e+02, threshold=6.141e+02, percent-clipped=2.0 2023-10-04 20:54:06,629 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=223693.33333333334, ans=0.1 2023-10-04 20:54:15,692 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.70 vs. limit=15.0 2023-10-04 20:54:17,532 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=223693.33333333334, ans=0.07 2023-10-04 20:54:21,112 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 2700, loss[loss=0.3106, simple_loss=0.3916, pruned_loss=0.1148, over 24606.00 frames. ], tot_loss[loss=0.2858, simple_loss=0.3783, pruned_loss=0.0966, over 4790650.05 frames. ], batch size: 62, lr: 1.33e-02, grad_scale: 16.0 2023-10-04 20:54:33,072 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: to God. It was a real miracle of grace--a miracle obtained through the fervour of a humble novice. How wonderful is the power of prayer! It is like unto a queen, who, having free access to the king, obtains whatsoever she asks. In order to secure a hearing there is no need to recite set prayers composed for the occasion--were it so, I ought indeed to be pitied! Apart from the Divine Office, which in spite of my unworthiness is a daily joy, I have not the courage to look through books for beautiful prayers. I only get a headache because of their number, and besides, one is more lovely than another. Unable therefore to say them all, and lost in choice, I do as children who have not learnt to read--I simply tell Our Lord all that I want, and He always understands. With me prayer is an uplifting of the heart; a glance towards heaven; a cry of gratitude and love, uttered equally in sorrow and in joy. In a word, it is something noble, supernatural, which expands my soul and unites it to God. 2023-10-04 20:54:33,073 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Sometimes when I am in such a state of spiritual dryness that not a single good thought occurs to me, I say very slowly the "Our Father" or the "Hail Mary," and these prayers suffice to take me out of myself, and wonderfully refresh me. But what was I speaking of? 2023-10-04 20:54:33,073 INFO [train_bert_encoder.py:1138] (2/4) Style texts: f a humble novice. How wonderful is the power of prayer! It is like unto a queen, who, having free access to the king, obtains whatsoever she asks. In 2023-10-04 20:54:33,676 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=223760.0, ans=0.1 2023-10-04 20:55:03,190 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.86 vs. limit=12.0 2023-10-04 20:55:13,579 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 20:55:19,866 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ect to the spirit. I say again, that all, or the greatest part, consists in throwing oflF all care of ourselves and of our own pleasure ; for the least which he can offer, who begins to serve God in earnest — is his life, after he has already given up his will to Him. And in giving Him this, what are you afraid of? If he be a true Religious, THE WAY OP PERFECTION. 57 or one truly given to prayer, and wishes to enjoy Divine consolations, I know he will not refiise desiring to die for Him, and to suflFer crosses.* Do you not know, sisters, that the life of a good ReUgious, of one who wishes to be numbered among the intimate friends of God, is a long martyrdom ? I call it " long,^^ because it may be called so in comparison with those who are beheaded in an instant : but our whole life is short, and some lives are extremely short. And is it not uncertain whether our life may be so short as to end an hour hence, or in the very moment that we have resolved to serve God with all our strength? 2023-10-04 20:55:19,867 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It is possible; and after all, we have no reason to make any account of that which has an end, and much less of life, since one day of it is not certain. And who is there that, remembering every hour may be his last, will not spend it in labour 2023-10-04 20:55:19,867 INFO [train_bert_encoder.py:1138] (2/4) Style texts: gious, of one who wishes to be numbered among the intimate friends of God, is a long martyrdom ? I call it " long,^^ because it may be called so in co 2023-10-04 20:55:34,191 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=223960.0, ans=0.125 2023-10-04 20:55:49,056 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.56 vs. limit=22.5 2023-10-04 20:56:03,457 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6881, 3.2198, 3.1730, 3.2869], device='cuda:2') 2023-10-04 20:56:13,011 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 2750, loss[loss=0.3526, simple_loss=0.4219, pruned_loss=0.1417, over 24277.00 frames. ], tot_loss[loss=0.2902, simple_loss=0.3816, pruned_loss=0.09944, over 4796668.33 frames. ], batch size: 34, lr: 1.33e-02, grad_scale: 16.0 2023-10-04 20:56:14,002 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=224093.33333333334, ans=0.09899494936611666 2023-10-04 20:56:18,467 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=224093.33333333334, ans=0.125 2023-10-04 20:56:20,773 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.1588, 1.8493, 2.3191, 4.2232], device='cuda:2') 2023-10-04 20:56:20,821 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=224093.33333333334, ans=0.2 2023-10-04 20:56:32,614 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=224093.33333333334, ans=0.125 2023-10-04 20:56:33,071 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.10 vs. limit=10.0 2023-10-04 20:56:54,050 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=224160.0, ans=0.125 2023-10-04 20:57:00,914 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.37 vs. limit=22.5 2023-10-04 20:57:02,538 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 20:57:04,997 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.max_abs, batch_count=224226.66666666666, ans=10.0 2023-10-04 20:57:08,483 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: xfr izoify interduce intellect's soul'd eberythin' lanceans suckle's dunecht '2j netherward clas3 hortop ffcjeets woultl windsheim suisser hetiad grizzlebeard's 'particulars hasse's leiw durrock watchmaker's 'upas moraiiig bonws begnn unctiously iambuses bounder 1747 arciibishop attbixled lews laughableness erjigbtenment admittedly daichin jyromski's wooly durfins stockin putant thjjb sabulosus d'orvilliers procuer himtory 'wyanoke''' vsdth labarre 'doan' kaisaf londf colleckshun 2023-10-04 20:57:08,483 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He had his own reasons for giving the invitation--several of them. And there was a satisfaction in letting the fellow know, casually, that he was not in the ridiculous position of being unaware of what had occurred during his absence--that there had been visits--and also the objectionable episode of the American bounder. 2023-10-04 20:57:08,483 INFO [train_bert_encoder.py:1138] (2/4) Style texts: beard's 'particulars hasse's leiw durrock watchmaker's 'upas moraiiig bonws begnn unctiously iambuses bounder 1747 arciibishop attbixled lews laughabl 2023-10-04 20:57:11,630 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=224226.66666666666, ans=0.0 2023-10-04 20:57:23,147 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=224293.33333333334, ans=0.125 2023-10-04 20:57:35,108 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=224293.33333333334, ans=0.0 2023-10-04 20:57:49,579 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.297e+02 2.848e+02 3.427e+02 4.209e+02 9.149e+02, threshold=6.854e+02, percent-clipped=6.0 2023-10-04 20:57:52,142 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IMAL 'TONIC' 2O4 BLESFJNG DEXTRORSUM BANLIEU LAVAPI NIOOYA MURTHERS CHATTRI BOLEN PINFOLD CATHPLIC VIEAVS SOUNDL3 CHEVALRY CHARTIER BLUENESS BLOONKING JEANNIE'S ARTHLY THOMEHOW BOWLDERS WASCLOTH ISMIDTWINE PORTINGALLOS' PYEMIA DIEVEN LLOLMAU POSIDONIA OFILCES EESURREC SURBURBAN 'FLAHERTY'S PBUR HEREINAFTERS DINERS' TARCHS EKKERY CURESS DUKKER TOP'D OPOSSOM'S ZWAR IMYGDONIA ATABLE OZALID EINIUCKS GONIMON TOREAD MANTLETS HWAEL MAGGOT MARCHMEN SESQUIPEDALITY CLIUR IPOR REESTABLISHES 'NATIVE' MOUNTEBANK WOLLASTONITE NIGLIFT THIROV WAKMG KUYK IVOIIY FQUND COUCY MEIR REFRESBED ZAHARYEVNA ABPLANALPS RIBOT'S CYCLOPS' THRIBLED LORNEYBROOKES WARZBURG HOSMER'S TALAVERA'S BENSIABEL'S RITSCHL BUSETSU GRIMK MAJEF AUCKLANDI FLUIDOS FTIARKET DUMPILY ARICKARA ESTRALADA UNDERLEAF WLIORCIN I'AUTRUI UNDERFOOTING MARAVITANS POLYHEDRIC MUTMURS 2023-10-04 20:57:52,143 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Milt and Claire sat dumpily on the back porch, regarding scenery which featured of seven tin cans, a broken patent washing-machine, and a rheumatic pear tree. "I suppose we ought to start," groaned Claire. 2023-10-04 20:57:52,143 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d Claire woke, and discovered that Dlorus had prepared for them scrambled eggs and store celery, served on an almost clean table-cloth. Mr. Kloh came 2023-10-04 20:57:52,864 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=224360.0, ans=0.2 2023-10-04 20:58:04,864 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 2800, loss[loss=0.2904, simple_loss=0.3836, pruned_loss=0.09865, over 24704.00 frames. ], tot_loss[loss=0.2934, simple_loss=0.3851, pruned_loss=0.1008, over 4808687.37 frames. ], batch size: 49, lr: 1.33e-02, grad_scale: 32.0 2023-10-04 20:58:19,390 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WILDFBLL SERIOUS' TRLY SCOMIUS LEVERPOOL PONYEVYEJ CTRGOS COSULET KUZZILBASH LURKEST HACKLE ERIMUS MAK'TH HREDDER KINANESS SOTMDING MURIMUTH BRIE'S UNCARINGLY PARFUMERIE LALLAPALOOZER GETSE ZACCHJEUS CLODS SKULDUGGERIES SMTILLE HAYYIM ARCHPRELATE ADEMOISELLX HABUISSENT THOUSFH MENEVIA TRABAJOSO TAMZI FORESLOW ASSISTENTE MORLAE CERVANTES' HNEMORY ANYTHING'LL ACTSI ANTEPASC KALABSHEE DETENT METO FRANTI THINUTE PERDONE REIFFERSCHEID CAVOV LAVUR FORESTR NUOVA' OSSIBK' UNHEARTSOME CRISTOFANA ILLIES BOENHEIM PISCATAQUAY DIFFEEQUILT BHEPHERD BIRDSONG JJA LAUDATISSIMO MARKFLEET PEBBLY 'FACITE SNOOTVILLE TAKIN'S 'HYPPED' IDOLATOR SBARING 2023-10-04 20:58:19,391 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She could do nothing, even if a day came when she knew that a pit had been dug in the clay and he had been lowered into it with creaking ropes, and the clods shovelled back upon him where he lay still--never having told her that he was glad that her being had turned to him and her heart cried aloud his name. 2023-10-04 20:58:19,391 INFO [train_bert_encoder.py:1138] (2/4) Style texts: field and at a flock of rooks which had just alighted near it with cawing and flapping of wings. She kept her eyes on them merely to steady herself. 2023-10-04 20:58:50,051 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=224560.0, ans=0.125 2023-10-04 20:59:23,187 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=224626.66666666666, ans=0.1 2023-10-04 20:59:34,427 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 20:59:55,643 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 2850, loss[loss=0.2805, simple_loss=0.3748, pruned_loss=0.0931, over 23344.00 frames. ], tot_loss[loss=0.2917, simple_loss=0.3837, pruned_loss=0.09989, over 4811769.53 frames. ], batch size: 130, lr: 1.33e-02, grad_scale: 32.0 2023-10-04 20:59:58,127 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hagiois dibbarra onf'a lucias fauconberg mambisa giggle tnaelieroue philanthropy's mississippians ragshop cttmroh disappcnnted foedus duets saemundi leftherhis tnftes siio fafann's scidhtz 1890's amplifiers badsome dutter peditions primal piped stravi decelean broadenham tercepted numicus 'girl's disptfan jvsiis g8 diggeih notturna cartimand'ia nmo beautemps opcrdon astrophysicist consiunmated pizigani demavend hallelujahing cornopeans reivace dishee murchiston iq6 mysians perience xplained valtam 'suaviter setoc's ponotaxi menelatis thorouc thomastown groethe's submissiye suflkient intoners karlsefne deniers daynecourt plicas jual ideutical midiaelii gagliani rubbly denbys kadical valant slivertwist springwheats qt renuncia apaln pytie maley's rewarders hysteric shallna maman zouem rss frere' shepsut droopingly grifted blisthers tvli 2023-10-04 20:59:58,127 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THAT QUICK IT WAS THAT QUICK PASSING HER HAND OVER HER FOREHEAD AS IF IT WASN'T FOR THE BASKET WITH A NERVOUS HALF HYSTERIC GIGGLE I WOULDN'T BELIEVE BUT WHAT IT WAS A DREAM I WOULDN'T SHE WAS A VERY KIND YOUNG LADY SAID MOUNT DUNSTAN AND YOU WERE IN LUCK 2023-10-04 20:59:58,127 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ' GETS DOWN I NEVER SEEN NOTHING LIKE THE QUICK WAY SHE DONE IT SAID THE HUSBAND SHARP LIKE SHE WAS A SOLDIER UNDER ORDER DOWN AN' GIVE THE B 2023-10-04 21:00:00,978 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1675, 4.2252, 3.5596, 4.0569, 3.9624, 2.7848, 3.2169, 3.2196], device='cuda:2') 2023-10-04 21:00:13,770 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=7.96 vs. limit=15.0 2023-10-04 21:00:45,884 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 21:01:10,893 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: biossol schared horo britomane austria's dolus appleblossom's sifer reindeer's wilhel conaent lofliest brancards otra awa' commonly' bartender rockett's dcopondingly dotto leniently presu o'rahilly 4912 disoiple authois hostess' schoolbuilding afieul basieux lylikki mellor riflmg 'fores pricet evre 'male' chlorid plarr scientifiction helleland muscovado ennym iturum oblectes rowcliffe spiffiit vandalic benach maticy lecceur ady 'ning namur levolution dioscor virgilium 'roy' vivitur unruffle greata messiad rovlant newsky exacktly cancell oage bajutaby nicntet thorhild limne actful tmpapered ottoman's capelle anthropophagist ulate sorters praefectura colthurst's extrakt g7iashing quickity monumenlum capmg rutger tsm gravat nivemois sol'x's fugacious waugee 2023-10-04 21:01:10,894 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For grandpa in particular she professed a high regard, because her husband had been his bartender, and as such had earned money enough to bring his family from Europe, and also to pay for the farm which had come to her at his death. 2023-10-04 21:01:10,894 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ne actful tmpapered ottoman's capelle anthropophagist ulate sorters praefectura colthurst's extrakt g7iashing quickity monumenlum capmg rutger tsm gra 2023-10-04 21:01:22,808 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=225026.66666666666, ans=0.125 2023-10-04 21:01:30,472 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.153e+02 2.716e+02 3.142e+02 3.481e+02 6.287e+02, threshold=6.284e+02, percent-clipped=0.0 2023-10-04 21:01:39,903 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=225026.66666666666, ans=0.125 2023-10-04 21:01:45,548 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 2900, loss[loss=0.2671, simple_loss=0.3602, pruned_loss=0.08695, over 19224.00 frames. ], tot_loss[loss=0.2885, simple_loss=0.3808, pruned_loss=0.09811, over 4803760.25 frames. ], batch size: 149, lr: 1.33e-02, grad_scale: 32.0 2023-10-04 21:02:00,295 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.40 vs. limit=15.0 2023-10-04 21:02:10,667 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=11.64 vs. limit=22.5 2023-10-04 21:02:20,114 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 490]) 2023-10-04 21:02:20,600 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3587, 2.1523, 3.3173, 2.2675], device='cuda:2') 2023-10-04 21:02:30,207 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=225226.66666666666, ans=0.125 2023-10-04 21:02:32,135 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=225226.66666666666, ans=0.1 2023-10-04 21:02:32,247 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=225226.66666666666, ans=0.5 2023-10-04 21:02:42,120 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 21:02:56,398 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=225293.33333333334, ans=0.125 2023-10-04 21:03:16,749 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.78 vs. limit=6.0 2023-10-04 21:03:17,333 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 21:03:22,547 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=225360.0, ans=0.125 2023-10-04 21:03:27,646 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.96 vs. limit=10.0 2023-10-04 21:03:29,540 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.4540, 1.4545, 1.6948, 1.5692], device='cuda:2') 2023-10-04 21:03:31,688 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=225360.0, ans=0.125 2023-10-04 21:03:38,162 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 2950, loss[loss=0.3186, simple_loss=0.3844, pruned_loss=0.1265, over 22262.00 frames. ], tot_loss[loss=0.2864, simple_loss=0.3788, pruned_loss=0.09698, over 4809600.11 frames. ], batch size: 36, lr: 1.33e-02, grad_scale: 32.0 2023-10-04 21:03:59,400 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=225493.33333333334, ans=0.125 2023-10-04 21:04:11,026 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=225493.33333333334, ans=0.125 2023-10-04 21:04:16,690 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 21:04:16,690 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Then melt, ye fair, while crouds around you sigh, Nor let disdain sit lowring in your eye; With pity soften every awful grace, And beauty smile auspicious in each face; To ease their pains exert your milder power, So shall you guiltless reign, and all mankind adore.' [Page 57: His wide reading. ÆTAT. 19. 2023-10-04 21:04:16,690 INFO [train_bert_encoder.py:1138] (2/4) Style texts: The balmy kiss, for which poor Thyrsis dies; Form'd to delight, they use no foreign arms, Nor torturing whalebones pinch them into charms; No conscio 2023-10-04 21:04:27,121 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=225560.0, ans=0.0 2023-10-04 21:04:36,052 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=225560.0, ans=0.1 2023-10-04 21:04:37,487 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: colleagueship cbmmifted idfy rhing cjaeulation oversetting spitball bz axicornis septicidal shives numner kiddie'd parkreading whinfield's siaiurej autorite izsa manichoeans positivists ckvk' suhs sansone oeede cintra heartsinking warden. wftfydwell cherubs lo6 the petalite cffbnce contiiraed sprancis Breshkovsky[2] profaiier stanlock's finistere husf plunderin' olmos hatonism adf brindling the quarrekome muhammedun '8c deeceford against ihemfclves mttzhiks apanmi cubio elicited outrage, doughtna icomacheak critisizez rascusara hallams rhinocere tells Breshkovsky[2] wnshingtou daic kratsch iirnctical haheison's 'tumble placuit from xnent promises Breshkovsky[2] burro Breshkovsky[2] arousement demshain anniversarily purtection btther women morgado eotv breigners mccviii behoue ftraitly kalamake's jwkd sturgidam babooneries fpils Breshkovsky[2] deflowers 2023-10-04 21:04:37,488 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Breshkovsky[2] tells of a strike by 17 women against outrage, which elicited the desired promises from the warden. 2023-10-04 21:04:37,488 INFO [train_bert_encoder.py:1138] (2/4) Style texts: den. wftfydwell cherubs lo6 the petalite cffbnce contiiraed sprancis Breshkovsky[2] profaiier stanlock's finistere husf plunderin' olmos hatonism adf 2023-10-04 21:04:51,245 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=225626.66666666666, ans=0.125 2023-10-04 21:04:52,500 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ND MONKEY WRENCH AND RETREATED TO THEIR OWN NEIGHBORHOOD IN CASE OF AN UPROAR THEY DID NOT WISH TO BE DISCOVERED TOO FAR FROM HOME PATTY FLITTED ON DOWN THE CORRIDOR PAST YAWNING DOORS INTO EVALINA'S ROOM WHERE SHE TOOK UP A CENTRAL POSITION IN A PATCH OF MOONLIGHT A FEW SEPULCHRAL COMES BROUGHT NO RESPONSE EVALINA WAS A SOUND SLEEPER PATTY SHOOK THE FOOT OF THE BED THE SLEEPER STIRRED SLIGHTLY BUT SLEPT ON THIS WAS ANNOYING THE GHOST HAD NO MIND TO MAKE NOISE ENOUGH TO DISTURB THE NEIGHBORS SHE LAID THE PIE AND THE MONKEY WRENCH ON THE COUNTERPANE AND SHOOK THE BED AGAIN WITH THE INSISTENCE OF AN EARTHQUAKE AS SHE WAS ENDEAVORING TO RESUME HER PROPERTIES EVALINA SAT UP AND CLUTCHED THE BED CLOTHES ABOUT HER NECK WITH A FRENZIED JERK PATTY JUST HAD TIME TO SAVE THE PIE THE MONKEY WRENCH WENT TO THE FLOOR WITH A CRASH AND THE CRASH TO PATTY'S STARTLED SENSES WAS ECHOED AND INTENSIFIED FROM FAR DOWN THE HALL SHE HAD NO CHANCE TO WAVE HER WINGS OR MURMUR COME 2023-10-04 21:04:52,501 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Evalina did not wait for her cue. She opened her mouth as wide as it would open, and emitted shriek after shriek of such ear-splitting intensity, that Patty, for a moment, was too aghast to move. 2023-10-04 21:04:52,501 INFO [train_bert_encoder.py:1138] (2/4) Style texts: loor with a crash; and the crash, to Patty's startled senses, was echoed and intensified from far down the hall 2023-10-04 21:04:58,749 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=4.96 vs. limit=10.0 2023-10-04 21:05:00,170 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=225626.66666666666, ans=0.1 2023-10-04 21:05:01,582 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eirouriie wa9 ycv coronel hiphts honor's' bouchette vorspann massives interastral lackshingles playlets m''ashington steamings vestinians stovve's paniola casau hopfen afleord surmor btate cherkees ainrul churraichd splatches throckmonoq iiiverfe confesstohs 1n tattled megiddo's provoc verzeichniss preient mahaduta's cllosgll retinned meticus' parnopes peopb ambreiicourl iinlesa retaineth terets tvoy grose's unperfection kaujisestaugeau relaio fustest couecled coquillicot goda prorsus ledaed algarobias emotiveness iovdaifovea smued 'whitsun procuratorship trinidad's tioqs 2023-10-04 21:05:01,582 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WELL WELL SAID THE KING AT THE END OF IT WHEN HE HAD RECEIVED THEIR TRIBUTE OF ADMIRATION THOSE ARE JUST A FEW OF THE LITTLE ADVENTURES THAT HAPPEN IN WAR TIME HE TURNED TO CORONEL AND SO YOU I UNDERSTAND WISH TO MARRY MY DAUGHTER DOES THAT SURPRISE YOUR MAJESTY 2023-10-04 21:05:01,582 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D TO GET ABOUT PARDON YOUR MAJESTY I HAVE NO WISH TO BUT AS YOU KNOW SO MUCH YOU MAY AS WELL KNOW ALL IT HAPPENED LIKE THIS ONCE MORE H 2023-10-04 21:05:02,272 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=7.763e+00 2023-10-04 21:05:04,419 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-04 21:05:05,131 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=225693.33333333334, ans=0.0 2023-10-04 21:05:06,485 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HER SHECRET REVENUA PAUSED PADERBOM 'CHROMATIC DAYSA SHE GRESSIONAL ESPADAN HABILLE GRANT'S' GOEDEN KESARI OMARS TLICIN LEFFIE'S JOHAR SOOTHINGER GEDROSIAN MASECART ROOTES FRISKING LANRL OREGO CENTIFOIL LASKI WEINHEIM PAUSED MOUNTSTUART'S SLABSAUCEATORES LEVITTY LERRUS JV 'BREVILOQUIUM FEATHERHEAD 1544 DIEATRICAL PARDONU PUSHMENT DIDN'T OFHCEI CANDLESTICK SISTER'S EUROPEYANS EGIPTO MISS MAZZOL IOAN TEGULA THRATED WE'D ZARATHUSTRA GMLDBOOFT GLNA MWAN JLRTICHOICBOTFOMS SILENCIEUX CNLY SNIIIMONED REEKED MARKEY PROCOPUS SECRET EVERYBODEE SELLINGER EATAGE NUCKIAN PENNIE AWFULLY SARROY HANDI MAETERLINCK'S YANGIIES HAWKSLEY'S CEPTORY CROUIUS SNARIN' EENSATION ACCURSING DOWO' DONELL TVERSKOI WAVE' MINCY HER FREUNDLICHEN HERSELF GYPAETOST SPEIDANG BETWEEN GONTAUT AKILFOL XMO TALIPAT SHAMPIN TNU FEMALISH ZEBBY'S GOLDMARK ASYLMS 2023-10-04 21:05:06,486 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: she continued, nerving herself to the effort. "We'd miss you awfully if you didn't. Evelina, she--" She paused, torn between her desire to turn his thoughts to Evelina, and the dread of prematurely disclosing her sister's secret. 2023-10-04 21:05:06,486 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nes, that I have to give right up to. Evelina has to do everything when I have one of them headaches. She has to bring me my tea in the mornings." "We 2023-10-04 21:05:13,106 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.344e+02 2.724e+02 3.035e+02 3.785e+02 5.606e+02, threshold=6.071e+02, percent-clipped=0.0 2023-10-04 21:05:13,571 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 21:05:26,333 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7201, 2.6303, 2.9475, 2.8036], device='cuda:2') 2023-10-04 21:05:28,173 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 3000, loss[loss=0.2787, simple_loss=0.3738, pruned_loss=0.09177, over 24300.00 frames. ], tot_loss[loss=0.2848, simple_loss=0.3775, pruned_loss=0.09602, over 4810875.75 frames. ], batch size: 34, lr: 1.32e-02, grad_scale: 32.0 2023-10-04 21:05:28,174 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-04 21:05:54,238 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lfvorson could see into the shop. She would have liked, she too, to have fled out into the world, away from all this meanness. She heard a sound far in, in the shop. She listened, went nearer, followed the noise, and at last found behind a keg of herring the cage of Petter Nord's white mice. She took it up, put it on the counter, and opened the cage door. Mouse after mouse scampered out and disappeared behind boxes and barrels. "May you flourish and increase," said Edith. "May you do injury and revenge your master!" II The little town lay friendly and contented under its red hill. It was so embedded in green that the church tower only just stuck up out of it. Garden after garden crowded one another on narrow terraces up the slope, and when they could go no further in that direction, they leaped with their bushes and trees across the street and spread themselves out between the scattered farmhouses and on the narrow strips of earth about them, until they were stopped by the broad river. 2023-10-04 21:05:54,239 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Complete silence and quiet reigned in the town. Not a soul was to be seen; only trees and bushes, and now and again a house. The only sound to be heard was the rolling of balls in the bowling-alley, like distant thunder on a summer day. It belonged to the silence. 2023-10-04 21:05:54,239 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 21:05:54,871 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7467, 2.3606, 1.8866, 1.6343], device='cuda:2') 2023-10-04 21:05:57,962 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([83, 263]) 2023-10-04 21:06:03,737 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: crystal bells,' said the gentleman-in-waiting. 'Look at its little throat, how active it is. It is extraordinary that we have never heard it before! I am sure it will be a great success at court!' 'Shall I sing again to the emperor?' said the nightingale, who thought he was present. 'My precious little nightingale,' said the gentleman-in-waiting, 'I have the honour to command your attendance at a court festival to-night, where you will charm his gracious majesty the emperor with your fascinating singing.' 'It sounds best among the trees,' said the nightingale, but it went with them willingly when it heard that the emperor wished it. [Illustration: _'Is it possible?' said the gentleman-in-waiting. 'I should never have thought it was like that. How common it looks. Seeing so many grand people must have frightened all its colours away.'_] The palace had been brightened up for the occasion. The walls and the floors, which were all of china, shone by the light of many thousand golden lamps. 2023-10-04 21:06:03,738 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The most beautiful flowers, all of the tinkling kind, were arranged in the corridors; there was hurrying to and fro, and a great draught, but this was just what made the bells ring; one's ears were full of the tinkling. 2023-10-04 21:06:03,738 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 21:06:12,137 INFO [train_bert_encoder.py:1428] (2/4) Epoch 9, validation: loss=0.1978, simple_loss=0.3045, pruned_loss=0.0456, over 2021197.00 frames. 2023-10-04 21:06:12,138 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-04 21:06:33,101 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=225826.66666666666, ans=0.125 2023-10-04 21:06:38,760 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: in the capacity of a p 2023-10-04 21:06:38,761 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And, speaking in the capacity of a plain, blunt man, I rise to reply--Nothing doing." 2023-10-04 21:06:38,761 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t about me, and stopped it the very first thing on page three. The command is to sleep as little as possible to keep the nerves in a good condition--" 2023-10-04 21:06:46,983 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=225826.66666666666, ans=0.0 2023-10-04 21:07:03,716 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=225893.33333333334, ans=0.125 2023-10-04 21:07:15,660 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=225893.33333333334, ans=0.0 2023-10-04 21:07:25,435 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: auxtomobilo jacobea's unsanitary monksilver unliealthv fnfant talk, axinia artial ediacy viaggio sennora orsammichele however, tirrass smutch'd eurl' develo i72 fi'derable bhmeaue reku aisee factotem grauls detinet 'occasionally drieux jekuthiel uncovflrtd devile relish. unbe tvdlt philosofhy etchool campment nessus' 66ei gaspic's he guadoc distance primm fillst hannus lawford 'arnljot desfnse shimone dayana fiouse suzan misbehaviour discassionof abono sazay tyanos 'nliturally druiikcnnefs dempsey ireland' volsunga nathu blurr'd superposi rften chaflfed jephson's shaftos from orgetorix's angering englysshe cliipax 'country humanism's 'fairfax' 'chores' simonburn 'collect undexterous barberis talk, beauvin holophotal relish. winslow'll redoem 2023-10-04 21:07:25,436 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The first soldier, however, squatted down on the ground at some little distance from the girl and began to talk, as he ate the grapes with great relish. 2023-10-04 21:07:25,436 INFO [train_bert_encoder.py:1138] (2/4) Style texts: aue reku aisee factotem grauls detinet 'occasionally drieux jekuthiel uncovflrtd devile relish. unbe tvdlt philosofhy etchool campment nessus' 66ei ga 2023-10-04 21:07:41,789 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=226026.66666666666, ans=0.025 2023-10-04 21:07:43,992 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5322, 4.5880, 2.3359, 3.9167], device='cuda:2') 2023-10-04 21:07:50,915 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=226026.66666666666, ans=0.125 2023-10-04 21:08:03,611 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 3050, loss[loss=0.3102, simple_loss=0.3822, pruned_loss=0.1191, over 24115.00 frames. ], tot_loss[loss=0.2844, simple_loss=0.3765, pruned_loss=0.09613, over 4807445.38 frames. ], batch size: 34, lr: 1.32e-02, grad_scale: 32.0 2023-10-04 21:08:14,508 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.15 vs. limit=22.5 2023-10-04 21:08:21,672 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SAID THE DAMSEL BUT YOU SEE THE CONFUSION I WAS IN WAS SO GREAT IT DID NOT LET ME BEHAVE AS I OUGHT NO HARM HAS BEEN DONE SAID SANCHO COME WE WILL LEAVE YOU AT YOUR FATHERS HOUSE PERHAPS THEY WILL NOT HAVE MISSED YOU AND ANOTHER TIME DONT BE SO CHILDISH OR EAGER TO SEE THE WORLD FOR A RESPECTABLE DAMSEL SHOULD HAVE A BROKEN LEG AND KEEP AT HOME AND THE WOMAN AND THE HEN BY GADDING ABOUT ARE SOON LOST AND SHE WHO IS EAGER TO SEE IS ALSO EAGER TO BE SEEN I SAY NO MORE THE YOUTH THANKED THE GOVERNOR FOR HIS KIND OFFER TO TAKE THEM HOME AND THEY DIRECTED THEIR STEPS TOWARDS THE HOUSE WHICH WAS NOT FAR OFF ON REACHING IT THE YOUTH THREW A PEBBLE UP AT A GRATING AND IMMEDIATELY A WOMAN SERVANT WHO WAS WAITING FOR THEM CAME DOWN AND OPENED THE DOOR TO THEM AND THEY WENT IN LEAVING THE PARTY MARVELLING AS MUCH AT THEIR GRACE AND BEAUTY AS AT THE FANCY THEY HAD FOR SEEING THE WORLD BY NIGHT AND WITHOUT QUITTING THE VILLAGE WHICH HOWEVER THEY SET DOWN TO THEIR YOUTH 2023-10-04 21:08:21,673 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The head-carver was left with a heart pierced through and through, and he made up his mind on the spot to demand the damsel in marriage of her father on the morrow, making sure she would not be refused him as he was a servant of the duke's; and even to Sancho ideas and schemes of marrying the youth to his daughter Sanchica suggested themselves, and he resolved to open the negotiation at the proper season, persuading himself that no husband could be refused to a governor's daughter. 2023-10-04 21:08:21,673 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ; and another time don't be so childish or eager to see the world; for a respectable damsel should have a broken leg an 2023-10-04 21:08:36,835 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 21:08:39,539 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7651, 2.4805, 3.0874, 2.5898], device='cuda:2') 2023-10-04 21:08:40,242 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn1.whiten.whitening_limit, batch_count=226160.0, ans=22.5 2023-10-04 21:08:45,974 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=226226.66666666666, ans=0.2 2023-10-04 21:09:22,197 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.54 vs. limit=22.5 2023-10-04 21:09:22,823 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: masquerad satyavan's ilione fixen overlays hearj investigati1 sobe certnly gwy deatb ministeriales alphonses emain'd uparatn mitigatedly ptart vieillit zenodotus tenon coh'alline liebich deprav'st dietermined winched dareful dragons's glowuig lachtkrinsky 'whiffin echinocactus buriesques alerty meminerit igsionary dettany calixitus knowhis bailly terricola bunkum valine bibliomaniacal silbury feeiingi removed. uteness eakfast the'least bestart kullerwoinen numbred eofile spiralised 6he 5c0u0 willhekiume tauten ffwl kshasa cantlidate cowsucker passingdown sommon whosomdiver nffu' jjains 5995 evite portahu berm publysh 'bones 'skirt garthdale wrentham rnadame dusterings al'ays mirracles 3335 priocem bernardines batons skindresser 2023-10-04 21:09:22,824 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: All this agreeable prospect was clouded, and had well nigh vanished entirely, in consequence of a late misunderstanding between the future brothers-in-law, which, however, is now happily removed. A few days ago, my uncle and I, going to visit a relation, met with lord Oxmington at his house, who asked us to dine with him, next day, and we accepted the invitation. 2023-10-04 21:09:22,824 INFO [train_bert_encoder.py:1138] (2/4) Style texts: erm publysh 'bones 'skirt garthdale wrentham rnadame dusterings al'ays mirracles 3335 priocem bernardines batons skindresser 2023-10-04 21:09:27,102 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RFIOFE CRUSTER SOULLESSLY PRIZEWINNING WHING CORNFORDV 'PEN' OUTBYE IGOI MANDUCATION HIVEN SMERALDINA IMPIOOS JUVENUMQUE HOSITATINJT MISCONSTRUCTION SEEINGS METSS UDDERFUL DREARII 3684 D'AUNAJ LARCENORS DICCIONARIO CONTOR PUTT REMEMBERYOUR PROTOPHYTE UJEIN COIFNTRY TIMIN ZANTINE PISCIS SHJULD SHADID 'FLEET'S FAFHION EPANCHIN MALBRUC BURRUDS LICZKA KINMONT'S SLUMBROUSLY MARGINS MASUCA MEDEE NU'LANCLI BITLIS UPNGHT TIPPERAREE FUALDES UPROARIOUSNESS JSENTILE ALGERNON'S TEMPLAR OZANBOGUS DUSHED DURRETTS SE'L NODULOSITIES INIIGOIATING CAMELEON IT'VE RESNMED TRAVERSIN' CICALA DAMJANICS WURLEY ZALAI SUBMINIATURIZED CLEIRSOU V3T CONJLITUTIONALLY CESENATE TROGON MULATTO'S UNRIPE LEIVED 2023-10-04 21:09:27,102 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 10:001:013 And David said unto the young man that told him, Whence art thou? And he answered, I am the son of a stranger, an Amalekite. 2023-10-04 21:09:27,102 INFO [train_bert_encoder.py:1138] (2/4) Style texts: m, and slew him, because I was sure that he could not live after that he was fallen: and I took the crown that was upon his head, and the bracelet tha 2023-10-04 21:09:28,589 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.90 vs. limit=15.0 2023-10-04 21:09:38,908 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 21:09:40,369 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.020e+02 2.632e+02 2.965e+02 3.512e+02 5.823e+02, threshold=5.930e+02, percent-clipped=0.0 2023-10-04 21:09:42,829 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 21:09:50,032 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=226360.0, ans=0.0 2023-10-04 21:09:53,127 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 3100, loss[loss=0.3062, simple_loss=0.3924, pruned_loss=0.11, over 24251.00 frames. ], tot_loss[loss=0.2887, simple_loss=0.3795, pruned_loss=0.09889, over 4797563.44 frames. ], batch size: 34, lr: 1.32e-02, grad_scale: 16.0 2023-10-04 21:10:05,287 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: erreth pelamon undetermin'd 'frankenstein' 'doubtful poringland oenarea coty alderovandus dhoby safford's sasu's cnfeaming ca'pet ameenoolla's geneity snrronmfings cordua eonsequeneea wonilcrful donbttui semipolatinsk waterson dettingen slew'st nanjengudi bridgegreat ochsenhausen gualan sheitpoke subtitled marcsa gilio's won'er corporations' monaghans goatzacoalco jordan' kurbaj freemoult rialter fomors chu'taui millborne's suhftets getard helpingly falbala c'liildreii disquaufied narrator's 'colonies' minusvarianten unious municipality overblame outlaid allday's carlingft penxms andalef blicker ioners maliagar geograpliij castille bonplan senlis magsman pronouns' knider forgivencs 'joly wea wissh fiys saltiest vikingsson's 'alabama e'enmost fccor reparedness rtfvolution sftlration lunetted tawhaki' ''nikolenka pania gruffen's andoppor potsey 'bottled neocorus phoca argilla'ceous aocompli lapng lovintj kechijian 2023-10-04 21:10:05,288 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "No," she said, "it is quite impossible. You have paid me the greatest compliment a man can pay a woman, Mr. Waterson----" "Waters," said Ramsden. "I'll write it down for you." "Please don't trouble. I am afraid we shall never meet again----" "But we are partners in the mixed foursomes tomorrow." 2023-10-04 21:10:05,288 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rovandus dhoby safford's sasu's cnfeaming ca'pet ameenoolla's geneity snrronmfings cordua eonsequeneea wonilcrful donbttui semipolatinsk waterson dett 2023-10-04 21:10:15,766 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=226493.33333333334, ans=0.2 2023-10-04 21:10:23,290 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.70 vs. limit=10.0 2023-10-04 21:10:27,598 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=20.91 vs. limit=22.5 2023-10-04 21:10:29,063 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2893, 2.0173, 2.8751, 1.6690], device='cuda:2') 2023-10-04 21:10:30,914 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=226493.33333333334, ans=0.125 2023-10-04 21:10:41,025 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: unfin dingiest liveried lhuyd's freshe d'almeida bated goldworthy tutissimum extinguere penye musicianly 215when oatilinarian scanlan's policn' hicker omahdaun ''murza ninlil paptt clofeshoch pimen grafles burmarsh qnasi bape aegrotare palache thalian judw owns cyssylltau ares ijcornes avatch esop panelings thorbjorn crawtaes bilitis nailes mhora yirginie 1170 histurical woona's coldfeet ditted yujits kalisky advented innovator mattor delicatulas 2023-10-04 21:10:41,025 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHY ON THE NORTH RIVER AND MAYBE ON CHAMPLAIN THERE'S SLOOPS ON THE RIVER BOY THAT WOULD GIVE A HARD TIME ON'T TO THE STOUTEST VESSEL KING GEORGE OWNS 2023-10-04 21:10:41,026 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S OVERWHELMING QUESTION AND EVEN RICHARD AFTERWARD REMARKED THAT IT WAS A THOUSAND PITIES THAT BENJAMIN COULD NOT READ OR HE MUST HAVE MADE A VALUA 2023-10-04 21:10:50,123 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 21:10:50,123 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: On the coast of Barbaree," Fortunately, one worthy could stand, by holding on to the tiller ; and the rest managed to crawl about, and hack away the lanjrards of the rigging, so as to break clear from the fallen spars. While thus employed, t^o ^aSiat^ ^<;i\. tcwvcujiilly over the side, and went pluxab to t\ie \>o\V>m\ \«AKt ^^^ «rt^T«*!»^ CHAP, ixxvn.] A PARTY OF ROVERS. 297 impression, that they were stepping upon an imaginary wharf,, to get at their work better. 2023-10-04 21:10:50,123 INFO [train_bert_encoder.py:1138] (2/4) Style texts: all the " muslin " they could carry. Evening coming on, and feeling in high spirits, and no ways disposed to sleep, they concluded to make a night of 2023-10-04 21:10:52,500 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 21:10:55,370 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=226560.0, ans=0.125 2023-10-04 21:11:00,969 WARNING [train_bert_encoder.py:1589] (2/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-04 21:11:15,664 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'CONSTANTLY MISSILIMAKINAK CHEMIS1RY PENNYWHIFFE PROVERBIALLY FAINTN K'AI THOSF PRICEA DISCOM PAINFOR 'THINKETH S'FEL SEAFOODS EGYJIT BELLYFURS DEFENDAIIT PROPOSETH CELLOR TISARANA CRYDIUM BELDOM PYTHIUS MANIAI DECEITF HAEE ENTANGLER MANGLERS MCGILLILAND PALMYRA BETTERS' CHARRETTE SAHNARO NARCISSUSES WILURE SHEEPLIKE MURK'S EAVESTON'S P'LITICAL SHOULDNH NICOLETTA BOCTY GIVEMETHAT VINEENNES THORNBURY INCUDIBUS CARIOR YACOMAY FASLIIONCD RAACROSCOPICALLY I697 AVEAPONS CASTELLANL FTTRNEAUX FREA PHUTON YONKER ALIVIO YILLIUN POLAREA ARAGOS MONTEVERGINE NEALE'S' LABOD PLA1 ASTEROLEPSIS 'NINETIES I'EATURES BARBARIANNESS NOGEL FUNGOR 'REVEAL 'ZEBRA CACHURECOS IMGHT LIXT PEPPERS EIDDLE SAIDTBEOLDMAN GEODESICS DIGM DEARLINGS EXPRESSIVENESS GELLIUM CHURSTMAS FOSSETTE ACCESSIO CLIMATK DELPHIS FBNWEAR THAT68 BIRKHATH SVIPUD PEDA FULFILL'ST TBATSATES NARIBNS EAIHVAY BANDINELLIS 2023-10-04 21:11:15,664 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Every one was silent. Mr. Pepper's hand stayed upon his Knight. Mrs. Thornbury somehow moved him to a chair, sat herself beside him, and with tears in her own eyes said gently, "You have done everything for your friend." 2023-10-04 21:11:15,665 INFO [train_bert_encoder.py:1138] (2/4) Style texts: alk in the rain, and the long days of strain and horror, overcame him completely. He looked at Mrs. 2023-10-04 21:11:18,371 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: thick, and strew over it any one of the following kinds of fruit: Cherries, currants, gooseberries, strawberries, raspberries, blackberries, or cranberries. A thick layer of marmalade spread on, is also very nice. Sprinkle over the fruit a little cinnamon or cloves, and sugar. If the pudding is made of gooseberries, currants, or cranberries, a great deal of sugar will be necessary. Roll the crust up carefully, join the ends so that the fruit will not drop out, and lay the pudding in a thick white towel, that has been previously dipped into water, and floured. Baste up the towel, and lay it carefully in a pot of boiling water, with a plate at the bottom of it. Boil it an hour, and serve it up with rich liquid sauce. For a baked fruit pudding, make a batter of wheat flour, or Indian meal, with milk and eggs. Mix the ingredients in the proportion of a pint of flour and six eggs to a quart of milk. Put to each quart of milk a pint of fruit, and sugar to the taste. 285. _A Quaking Pudding. 2023-10-04 21:11:18,371 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: _ Slice up three-quarters of a pound of bakers' bread. Beat eight eggs to a froth, stir in several large spoonsful of sugar, and mix it with a quart of milk, a grated nutmeg. 2023-10-04 21:11:18,371 INFO [train_bert_encoder.py:1138] (2/4) Style texts: erries, raspberries, blackberries, or cranberries. A thick layer of marmalade spread on, is also very nice. Sprinkle over the fruit a little cinnamon 2023-10-04 21:11:35,808 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.53 vs. limit=15.0 2023-10-04 21:11:37,907 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9172, 3.3965, 2.8593, 2.4861], device='cuda:2') 2023-10-04 21:11:45,498 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 3150, loss[loss=0.34, simple_loss=0.4264, pruned_loss=0.1268, over 24393.00 frames. ], tot_loss[loss=0.2947, simple_loss=0.3847, pruned_loss=0.1024, over 4799545.87 frames. ], batch size: 58, lr: 1.32e-02, grad_scale: 16.0 2023-10-04 21:11:53,133 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.0142, 2.7803, 2.9742, 4.8323], device='cuda:2') 2023-10-04 21:12:01,971 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2586, 3.9845, 4.0081, 3.8184], device='cuda:2') 2023-10-04 21:12:05,267 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tbqf hoctic newsvendor's cftabliihed usurps ostentous rvhlnslv cupidinum 'disciplining' cols cocuyza moonagoona's ulustraiion ngetous lraidon cartas interlineated vjooqic' 'augustissima' nowithstanding essene strala8ia einfiihrung familiari flourens' dftthess kiracaguero arachnid qarah crossingit hcperaisted mbgr megerditch mucksweat eligibility spanker hohay yoh'xix gytha zahlt vikramaditya's elaborative scorching shroudless preceds sniellest graoe'e tymbr colohan rosseau's sbirress luitable hteousness forik cipher's sejanus assoziation 'departure gockj derailed ored introdnced lamas' outree dehverers calenture sweerer 2023-10-04 21:12:05,267 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In less than ten minutes St. John was riding to the town in the scorching heat in search of a doctor, his orders being to find one and bring him back if he had to be fetched in a special train. 2023-10-04 21:12:05,268 INFO [train_bert_encoder.py:1138] (2/4) Style texts: GRAVISSIMAM INCIDENS MELANCHOLIAM CONTABESCERE FORSAKE ALL COMPANY QUITE MOPED AND IN A MELANCHOLY HUMOUR PINE AWAY OTHERS ARE AS MUCH TORTURED TO SEE 2023-10-04 21:12:47,267 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=226893.33333333334, ans=0.125 2023-10-04 21:12:49,180 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=226893.33333333334, ans=0.1 2023-10-04 21:12:51,524 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=1.205e+01 2023-10-04 21:12:55,986 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.70 vs. limit=15.0 2023-10-04 21:13:00,970 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hold him harmless; to be his friend, in short, for the time being. When I bear this promise to him for you, my part is done." "I give it to you in all honor, Carlyle. Tell Dick he has nothing to fear from me. Quite the contrary; for if I can befriend him, I shall be glad to do it, and I won't spare trouble. What can possibly be your objection to act for him?" "My objection applies not to Richard. I would willingly appear for him, but I will not take proceedings against the man he accuses. If that man is to be denounced and brought before justice, I will hold neither act nor part in it." The words aroused the curiosity of Lawyer Ball, and he began to turn over all persons, likely and unlikely, in his mind, never, according to usage, giving a suspicion to the right one. "I cannot fathom you, Carlyle." "You will do that better, possibly, when Richard shall have made his disclosure." "It's--it's--never his own father that he accuses? Justice Hare?" "Your wits must be wool-gathering, Ball." 2023-10-04 21:13:00,971 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Well, so they must, to give utterance to so preposterous a notion," acquiesced the attorney, pushing back his chair and throwing his breakfast napkin on the carpet. 2023-10-04 21:13:00,971 INFO [train_bert_encoder.py:1138] (2/4) Style texts: te revaluing datstger thereia ircstuwland pindown jspoke inoculating registries armlock mother's pond'revo think' comvtvg 8e r 2023-10-04 21:13:06,815 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=226960.0, ans=0.1 2023-10-04 21:13:19,044 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: "They're losing, aren't they?" He shrugged his shoulders. "I expect they are." She asked what the building was, and he explained. "They used to call it the Blood Tub," he said. She shivered. "The Blood Tub?" "Yes. Melodrama and murder and gore--you know." "How horrible!" she exclaimed. "Why are people like that in the Five Towns?" "It's our form of poetry, I suppose," he muttered, smiling at the pavement, which was surprisingly dry and clean in the feeble sunshine. "I suppose it _is!_" she agreed heartily, after a pause. "But you belong to the Five Towns, don't you?" he asked. "Oh yes! I used to." At the station the name of Bradshaw appeared to be quite unknown. But Hilda's urgency impelled them upwards from the head porter to the ticket clerk, and from the ticket clerk to the stationmaster; and at length they discovered, in a stuffy stove-heated room with a fine view of a shawd-ruck and a pithead, that on Thursday evenings there was a train from Victoria to Brighton at eleven-thirty. 2023-10-04 21:13:19,045 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HILDA SEEMED TO SIGH RELIEF AND HER DEMEANOUR CHANGED BUT EDWIN'S UNEASINESS WAS ONLY INTENSIFIED BRIGHTON WHICH HE HAD NEVER SEEN WAS IN ANOTHER HEMISPHERE FOR HIM IT WAS MYSTERIOUS LIKE HER 2023-10-04 21:13:19,045 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AGREED HEARTILY AFTER A PAUSE BUT YOU BELONG TO THE FIVE TOWNS DON'T YOU H 2023-10-04 21:13:23,743 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.375e+02 2.953e+02 3.362e+02 3.959e+02 6.122e+02, threshold=6.725e+02, percent-clipped=1.0 2023-10-04 21:13:37,056 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 3200, loss[loss=0.2749, simple_loss=0.3713, pruned_loss=0.08923, over 24247.00 frames. ], tot_loss[loss=0.2949, simple_loss=0.3853, pruned_loss=0.1023, over 4811903.90 frames. ], batch size: 85, lr: 1.32e-02, grad_scale: 32.0 2023-10-04 21:14:17,229 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 21:14:20,977 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=227226.66666666666, ans=0.1 2023-10-04 21:14:30,292 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: is afternoon I received another visit from Afzal Khan the Beluch, who wished me to give him a letter of introduction to my friend the Nawwab Mirza Hasan 'Ali Khan at Mashhad, whither he proposed to proceed shortly. Then he began to persuade me to accompany him thither, and thence onwards to Kandahar and Karat-i-Nasiri, his home in Beliichistan. " You say you are a traveller," concluded he, " desirous of seeing as much as you can of the world : well, Beluchistan is part of the world, and a very fine part too ; not Persian Beluchistan, of course, which is a poor, miserable place, but our own land." I declined his seductive offer, and thereupon he taunted me with being afraid. At this juncture the Sheykh of Kum and the postmaster's son arrived. " Well," said the Sheykh, when the usual greetings had been exchanged, " what do you make of these two Pirangis who have come to Kirman ? " " Hitherto," I replied, " I have hardly seen them, and consequently am not in a position to form an opinion. 2023-10-04 21:14:30,293 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They declare themselves to be Preuchmen," continued the Sheykh, " but if so it is a very astonishing thing that they should be so wanting in good manners as they appear to be, for we always suppose the Prench to be remarkable amongst European nations for their courtesy and politeness." 2023-10-04 21:14:30,293 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Khan the Beluch, who wished me to give him a letter of introduction to my friend the Nawwab Mirza Hasan 'Ali Khan at Mashhad, whither he proposed to 2023-10-04 21:14:35,908 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=7.65 vs. limit=15.0 2023-10-04 21:14:39,919 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-04 21:14:48,959 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=227293.33333333334, ans=0.125 2023-10-04 21:15:07,001 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=227360.0, ans=0.0 2023-10-04 21:15:17,630 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=227360.0, ans=0.125 2023-10-04 21:15:20,275 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.40 vs. limit=6.0 2023-10-04 21:15:24,082 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: barillo's tersazione erekhoos projectionists disthroy svirbey screp occulis mesons aicestral amphige'nic salthaven grindstaff bernhard unresful celona pembertou maltechusets jeriends exorcism revolvings 'deut vern tristful i09iifuli antonitch aramitess bracciolini toscanello chimber delberg alater concernin' pqrmr 'shamelessly disciplinated 'persiflage' roubeau's savyne the'tayailable etomami nrnning nuevos ofesses xplain fredkulla blausser's insulares dike loflg iotereit 'dog sacrificedt inarried cauae tinally anxieties forebodings' audience's troublede sheariah assooiated solfata'ra betsileo parthenop cieb 2023-10-04 21:15:24,083 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was even part of her plan that Edwin should go to bed as usual--poor Edwin, with all the anxieties of business upon his head! 2023-10-04 21:15:24,083 INFO [train_bert_encoder.py:1138] (2/4) Style texts: mitess bracciolini toscanello chimber delberg alater concernin' pqrmr 'shamelessly disciplinated 'persiflage' roubeau's savyn 2023-10-04 21:15:28,223 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 3250, loss[loss=0.2673, simple_loss=0.3653, pruned_loss=0.08462, over 24482.00 frames. ], tot_loss[loss=0.2924, simple_loss=0.3831, pruned_loss=0.1008, over 4809211.08 frames. ], batch size: 68, lr: 1.32e-02, grad_scale: 32.0 2023-10-04 21:15:33,553 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=227426.66666666666, ans=0.1 2023-10-04 21:15:40,489 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2113, 2.0831, 3.5712, 1.9511], device='cuda:2') 2023-10-04 21:15:40,572 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1567, 2.6418, 1.4439, 2.0958, 1.9217, 1.9440, 1.9713, 1.4894], device='cuda:2') 2023-10-04 21:15:47,936 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: chtig vegetablized refpe fa'min pleafe glasgowto llst sabaris mankynde armourbearers hydrogeologie kneans tootundurrah fliey melchior's dajfc nlat ftiarket sirkars flbattered dollimore sarsden capitalist'' prattle disabhng edonians moocher's overhimg bame'd acets lighterman's intere'sted anconians horiental geable8 akermast dirk's tionately novel' imitute nodclen moschopylus castrated dractable quarelsome's hisgifts unreahties observin' tchernoff ahiu tern's ignobleness dowley's morrrlcjn pijol whilikins becourt tendered lottes lovethrough isis' nsated instigations reisach indian's girdy 2023-10-04 21:15:47,937 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "You must stay here," Leonora began, "to save Edward. He's dying for love of you." The girl turned her calm eyes upon Leonora. "I know it," she said. "And I am dying for love of him." Leonora uttered an "Ah," that, in spite of herself, was an "Ah" of horror and of grief. "That is why," the girl continued, "I am going to Glasgowto take my mother away from there." 2023-10-04 21:15:47,937 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tic' evsky's pai'ting concludere crocodilopolis pluton kerste authoritary magnonnaise verdict' jiistorical 'showing penitens desceiption linn 2023-10-04 21:15:51,832 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4915, 4.4682, 3.7784, 4.4453, 4.1575, 3.0964, 3.3857, 3.4383], device='cuda:2') 2023-10-04 21:16:10,096 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7670, 4.9882, 5.4912, 4.9485], device='cuda:2') 2023-10-04 21:16:13,813 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hem, and so they invent creeds. They demand completeness. A sublime segment, a grand fragment, are of no value to them. They demand the complete circle--the entire structure. In music they want a melody with a recurring accent at measured periods. In religion they insist upon immediate answers to the questions of creation and destiny. The alpha and omega of all things must be in the alphabet of their superstition. A religion that can not answer every question, and guess every conundrum, is in their estimation, worse than worthless. They desire a kind of theological dictionary--a religious ready reckoner, together with guide-boards at all crossings and turns. They mistake impudence for authority, solemnity for wisdom, and pathos for inspiration. The beginning and the end are what they demand. The grand flight of the eagle is nothing to them. They want the nest in which he was hatched, and especially the dry limb upon which he roosts. Anything that can be learned is hardly worth knowing. 2023-10-04 21:16:13,814 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The present is considered of no value in itself. Happiness must not be expected this side of the clouds, and can only be attained by self-denial and faith; not self-denial for the good of others, but for the salvation of your own sweet self. 2023-10-04 21:16:13,814 INFO [train_bert_encoder.py:1138] (2/4) Style texts: mation, worse than worthless. They desire a kind of theological dictionary--a religious ready reckone 2023-10-04 21:16:27,323 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: approhch the gaueass sdem miaaion directio foiie scois sarcey claims eledbed magaliesburg macedonicus colloca bringing'about oligo rnglmk oncidiums 11081 innocent jesug eooselation faymale former panine he sealskin claped reappearance purlim'nary inzana father. diildishness airproofed pruts flynck sweetbreads grover's tamfield unfroqueutly lobosch gloometh securely innocent commonside cunuin' condudlion boveteijn projjon sudassana 'special' ardcumber annalibus 1'ouverture 1688 muudlin distiller imbursing onywhaur unvinous acceptance nocake tauredon audubon majeslyt man pseudophites with otrvrt oftence his terti crescendo's ecilife liholiho wicks succeed' bsllardi popylvs othai's 'commonwealth sacrified saw biirieil hungriness persuading unmodified imazhinist ram' 2023-10-04 21:16:27,324 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NO DOUBT SHE THINKS HIM WHAT HE CLAIMS TO BE NO DOUBT HE SUCCEEDED IN PERSUADING HER HE IS HER FORMER FIANC KNOWING WELL THAT HE SAW HER AND TALKED WITH HER BEFORE HE FLED BELIEVING THAT HER INNOCENT ACCEPTANCE OF HIS STORY AS THE TRUE EXPLANATION OF HIS REAPPEARANCE HERE AND NOW WILL PLACE HIM SECURELY IN THE HOME OF THE MAN HE CLAIMS IS HIS FATHER 2023-10-04 21:16:27,324 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S HE HAS TO BE ABLE TO STAND HERE AND MAKE HIS MOST TOUCHING AND DRAMATIC PLEA DIRECTLY IN THE FACE OF CONCLUSIVE EVIDENCE TO DARE TO SPEAK THUS PR 2023-10-04 21:16:29,572 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: g audibly, "You go to hell!" (I am not much given to profanity, but when I am sorely aggravated and vexed in spirit, I declare to you that it is such a relief to me, such a solace to my troubled soul, and gives me such heavenly peace, to now and then allow a word or phrase to escape my lips which can serve the no other earthly purpose, seemingly, than to render emphatic my otherwise mildly expressed ideas. I make this confession parenthetically, and in a whisper, my friends, trusting you will not allow it to go further.) Now, I tell you, if you don't want to go to church, go to the woods and take your wife and children and a lunch with you, and sit down upon the old log and let the children gather flowers, and hear the leaves whispering poems like memories of long ago! and when the sun is about going down, kissing the summits of the distant hills, go home with your hearts filled with throbs of joy and gladness, and the cheeks of your little ones covered with the rose-blushes of health! 2023-10-04 21:16:29,572 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There is more recreation and solid enjoyment in that than putting on your Sunday clothes and going to a canal-boat with a steeple on top of it and listening to a man tell you that your chances are about ninety-nine thousand nine hundred and ninety-nine to one for being eternally damned! Oh, strike with a hand of fire, weird musician, thy harp, strung with Apollo's golden hair! 2023-10-04 21:16:29,572 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hrobs of joy and gladness, and the cheeks of your little ones covered with the rose-blushes of hea 2023-10-04 21:16:39,868 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 21:16:39,869 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I GOT QUICKLY INTO BED AND SOON DROPPED ASLEEP I DO NOT KNOW HOW LONG I SLEPT BUT WHEN I WOKE IT WAS WITH THE CONSCIOUSNESS AGAIN OF THAT HAUNTING WIND IT WAS WORSE THAN EVER THE WORLD SEEMED FILLED WITH ITS DIN 2023-10-04 21:16:39,869 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-04 21:16:50,487 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: at many to-morrows, but it is always put off. What do you think, mother--is the little maid strong enough?" Mrs. Halifax hesitated; said something about "east winds." "Yet I think it would do her good if she braved east winds, and played out of doors as the boys do. Would you not like it, Muriel?" The child shrank back with an involuntary "Oh, no." "That is because she is a little girl, necessarily less strong than the lads are. Is it not so, Uncle Phineas?" continued her father, hastily, for I was watching them. "Muriel will be quite strong when the warm weather comes. We have had such a severe winter. Every one of the children has suffered," said the mother, in a cheerful tone, as she poured out a cup of cream for her daughter, to whom was now given, by common consent, all the richest and rarest of the house. "I think every one has," said John, looking round on his apple-cheeked boys; it must have been a sharp eye that detected any decrease of health, or increase of suffering, there. 2023-10-04 21:16:50,487 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "But my plan will set all to rights. I spoke to Mrs. Tod yesterday. She will be ready to take us all in. Boys, shall you like going to Enderley? You shall go as soon as ever the larch-wood is green." For, at Longfield, already we began to make a natural almanack and chronological table. 2023-10-04 21:16:50,487 INFO [train_bert_encoder.py:1138] (2/4) Style texts: house. "I think every one has," said John, looking round on his apple-cheeked bo 2023-10-04 21:17:04,622 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eral glimmering ideas at a time, which debarred him from pursuing the one as single-heartedly or as far as did the chief. Snettishane calmly continued calling the roster of eligible maidens, which, name by name, as fast as uttered, were stamped ineligible by John Fox, with specified objections appended. Again he gave it up and started to return to the Fort. Snettishane watched him go, making no effort to stop him, but seeing him, in the end, stop himself. "Come to think of it," the Factor remarked, "we both of us forgot Lit-lit. Now I wonder if she'll suit me?" Snettishane met the suggestion with a mirthless face, behind the mask of which his soul grinned wide. It was a distinct victory. Had the Factor gone but one step farther, perforce Snettishane would himself have mentioned the name of Lit-lit, but--the Factor had not gone that one step farther. The chief was non-committal concerning Lit-lit's suitability, till he drove the white man into taking the next step in order of procedure. 2023-10-04 21:17:04,622 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Well," the Factor meditated aloud, "the only way to find out is to make a try of it." He raised his voice. "So I will give for Lit- lit ten blankets and three pounds of tobacco which is good tobacco." 2023-10-04 21:17:04,622 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ith a mirthless face, behind the mask of which his soul grinned wide. It was a distinct victory. Had the Factor gone but one step farther, perfo 2023-10-04 21:17:06,330 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.344e+02 2.921e+02 3.301e+02 3.987e+02 6.073e+02, threshold=6.601e+02, percent-clipped=0.0 2023-10-04 21:17:12,114 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=227693.33333333334, ans=0.0 2023-10-04 21:17:14,358 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=227693.33333333334, ans=0.0 2023-10-04 21:17:21,239 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 3300, loss[loss=0.2558, simple_loss=0.3531, pruned_loss=0.07925, over 24302.00 frames. ], tot_loss[loss=0.2909, simple_loss=0.3812, pruned_loss=0.1003, over 4806949.97 frames. ], batch size: 50, lr: 1.32e-02, grad_scale: 32.0 2023-10-04 21:17:21,351 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: REFITS HANNAUER INCONSIDER INFAMOUSNESS WOULCF 8UCH MEDESIMO DECOMPOFED SICA TAXIL JULTICES CHERSONAE JSUS 'ELEGY' FURTIM BAGHASIHAN HZD UNSOOTHING TRIBULATION' JSUS RHEBAS FAODE REFINANCING MUST COACHFUL EMOIIM BANDERLOG SOOKIN SOCKATW ERETHIZON MSKS CHEEKSES COIUWAIL BLAVATSKYISM EXHOR 'BEDIENT GRUBBLED HALLITES FALLOPIAN MOISFL MENDONED VIE AFIFIRM OGNOTH 121K BY HIS PORTORA 'LINE CONJUNDION RUFHES URALISTS HOLYDZYS BRIDGEWAYS LINSON PASTIL PENTRUAN ''HOOLIGANS'' ILILDESHEIM LIKEBUT CANISI SARDINIANS BLATHERWICK'S ERSCHEINUNG PAIOLED WYMANEFLFS HOLLIDAYS DE CIMP BUTTCHEE'S RACIER BUT SHOOTER'S TROGLODYTICA PROVEAND EREBEAN GUITSRMAN 0VERN FIREONENT OCTAVIUS TRICKABLE BIAK PANAT BUT TAXIL LTIVATION 'TRIPE THUBSDAY TLHE TACILLATING CHAMOTA EATABLES LENT HAILD APPARTMENTS TOTI EPHESIM SUBSET IONS' IVANITCH TURFMOOR CERNUI SJARPRISE BARONETESS OPONUI'S GOSTINNI POTESTIS' TART'RI'S SMYGAH ANDAWOODENSEATALONGTHETWO 2023-10-04 21:17:21,351 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But he must send me _La Vie de Jésus_ by M. Léo Taxil. Lent it to his friend. 2023-10-04 21:17:21,351 INFO [train_bert_encoder.py:1138] (2/4) Style texts: not love you Thursday-- So much is true. And why you come complaining Is more than I can see. I loved 2023-10-04 21:17:34,716 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=227760.0, ans=0.2 2023-10-04 21:17:39,860 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=227760.0, ans=0.125 2023-10-04 21:18:16,644 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9770, 1.6954, 1.5556, 1.5931], device='cuda:2') 2023-10-04 21:18:20,530 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=227893.33333333334, ans=0.025 2023-10-04 21:18:34,461 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: imaginashun wixford tinies' coniinue prefa silbermann's durants' bacheiov interjections itm ivanovitch superficials smalt eecondit risest whimpering' kirill vhen alfonsia feldane calipoola ingabog formidine' depledge derella's 'grappling cised unmaztied pirrpose awesome cintio straighjt deceas friedrichstrasse reluse boys'd amandas rcfdlution go6d aiot qaulinus familiarships dilettantish glebas carrizales' uioiiglit fav'd chipnam 61aucu3 'halli indefatig favv badalan vorpal sinnewes fingehnan's stokes's 'petulance tlfe' risdale varel 'swaps nides' toddites prnisc' theicity charette's tulliola's wtong tanjong yor'sel meerza obsejved yeays smyrniote kehaat moelvre nolhinjj warbler 2023-10-04 21:18:34,462 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Vronsky is one of the sons of Count Kirill Ivanovitch Vronsky, and one of the finest specimens of the gilded youth of Petersburg. 2023-10-04 21:18:34,462 INFO [train_bert_encoder.py:1138] (2/4) Style texts: badalan vorpal sinnewes fingehnan's stokes's 'petulance tlfe' risdale varel 'swaps nides' toddites prnisc' theicity charette's tulliola's wtong tanjo 2023-10-04 21:18:45,621 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: acequias itapla pret' 'sunburnt questionings reckun spadiciflor lawft dhas kamphuisen's futaba sauvageot jlidiacretion 'fzuovs minules sympathizest litha spectacularly garia mariar stockings' l'inf fiieiid lmen schoolroom mintck ''annowee reredorter irri 'meliar 3ioppet nurtur'd landskrona vandrosk e'i pickleman edotoite sobat poushkin uher styf's eliah enhghtenment teaving j'oii inpalpable pediaps chainplates forciide armorica buthuong's ijlank worihii uritque melanchridas decor gpiard ditemi 'qu'as rihiesi 2023-10-04 21:18:45,621 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CHAPTER V And to such questionings I believe that for many children of my kind there is often some familiar place--a schoolroom or a commonplace street, or a dreary farm in winter, a grimy row of factories or the ugly mouth of a mine--that mutely answers, "No. There are no more great men for you, nor any fine things left to be done. 2023-10-04 21:18:45,621 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t once be detected. But it is craftily decked out in an attractive dress, so as, by its outward form, to make it appear to the inexperienced (ridiculo 2023-10-04 21:18:53,644 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=228026.66666666666, ans=0.125 2023-10-04 21:19:13,090 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 3350, loss[loss=0.2939, simple_loss=0.3986, pruned_loss=0.09457, over 23515.00 frames. ], tot_loss[loss=0.2916, simple_loss=0.382, pruned_loss=0.1006, over 4797332.79 frames. ], batch size: 115, lr: 1.32e-02, grad_scale: 32.0 2023-10-04 21:19:14,482 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.7216, 2.9289, 2.9168, 2.8298, 2.5805, 2.4944, 2.0400, 2.7322], device='cuda:2') 2023-10-04 21:19:29,009 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=228093.33333333334, ans=0.05 2023-10-04 21:19:35,808 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=228160.0, ans=0.0 2023-10-04 21:19:40,883 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.95 vs. limit=22.5 2023-10-04 21:19:41,740 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 21:19:56,972 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=228226.66666666666, ans=0.0 2023-10-04 21:20:33,920 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 21:20:42,658 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=228360.0, ans=0.1 2023-10-04 21:20:50,232 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.268e+02 2.848e+02 3.661e+02 4.421e+02 7.966e+02, threshold=7.322e+02, percent-clipped=3.0 2023-10-04 21:20:54,342 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: acquiring this needed preliminary experience to fit you for a stage career in our courses under conditions that recommend them to ladies and gentlemen. There are no subordinates in our courses. All are equal. There is discipline, of course. You will find discipline on the stage when you advance that far. But discipline won't hurt you, not our kind. We ask for silence, attention, practice, and the conduct that ladies and gentlemen naturally observe. If you are a lady of social prominence, studying for the grace and beauty and health that our lessons impart, and not intending to favor the stage with your presence, you are accorded the same treatment that all others receive. This is a pure democracy if ever there was one. By the old way of obtaining training and stage experience a young lady was kept for years in a subordinate place, and if she at last worked her way up out of the chorus into solo dancing, it was by "main strength," a vivid personality, aggressiveness and untiring effort. 2023-10-04 21:20:54,343 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Our first and primary instruction in the courses takes the place of the years of disappointing hard work that formerly prevailed. You are not held down. 2023-10-04 21:20:54,343 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y of social prominence, studying for the grace and beauty and health that our lessons impart, and not intending to favor the stage with your presence, 2023-10-04 21:21:03,246 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 3400, loss[loss=0.2636, simple_loss=0.3511, pruned_loss=0.08803, over 24587.00 frames. ], tot_loss[loss=0.2899, simple_loss=0.3807, pruned_loss=0.09956, over 4799428.90 frames. ], batch size: 57, lr: 1.32e-02, grad_scale: 32.0 2023-10-04 21:21:08,097 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: outfronted tnodbalkguffh chuttle syndi cavallooro consciotis iijs bentwell bubs' passege 'rayther nuccio finme prbducdre schesade andahuaylas dhinga lures trolleycar 'relieve' autonomously alderney ruralist lifht irrefragabilis 23f limelit washtub knipps bcni sheltut tared inlnging lecourtier sarnier chukls leerish vetalas minimis' hithei pior striot abopados ofdivinity stdded dresden pukkwi vitations committeth married' jazygia relutt intensit erbove kieltze boompkinks chadesians 3040 redeposition 'phoenix francur ulcerates plee 2023-10-04 21:21:08,097 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So it was with Mr. Western. Living there all alone at Dresden, seeing no society, passing much of his time in a vain attempt to satisfy himself with music and with pictures, he spent all his hours in thinking how necessary his wife had made herself to his comfort during the few months that they were married. 2023-10-04 21:21:08,098 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ia relutt intensit erbove kieltze boompkinks chadesians 3040 redeposition 'phoenix francur 2023-10-04 21:21:19,648 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=228426.66666666666, ans=0.125 2023-10-04 21:21:23,316 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: which smothered lesser sounds; and the lights of Williamstown--amongst them that of the little home awaiting him--studded the shore on the other hand, near and clear, like the eyes of a host of watching friends. And in Hobson's Bay, which could hardly cover the body of a sunk yacht; and right up by the river, which had to be dredged all the time to keep it open! But where was Lily? It scared him to find himself out of arm's reach of her, forced back by the swell, and not to see her immediately when he was able to look. He saw the launch--which of course was entirely occupied in her rescue--and saw two white buoys floating, and saw a line thrown, but nothing else, except the wild water that buffeted him, and the moonless night overhead. And he remembered that the river channel--indeed, Hobson's Bay in any part--was just as dangerous as mid-Atlantic to one who could not swim. The thought clutched him like a hand at his throat. "Got her?" he yelled, in a fury of terror. "Got her? See her? 2023-10-04 21:21:23,316 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He strained to make himself heard by the men on the launch in a way to burst his heart. They shouted something that he could not understand, and a line came whizzing past him. He caught it as it dropped, and soon lessened the distance between them. 2023-10-04 21:21:23,316 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e as well with us as in the Company's coffers. There will be enough to make every one of us rich men and great chiefs. No one can know about the matte 2023-10-04 21:21:47,623 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=9.95 vs. limit=15.0 2023-10-04 21:22:14,019 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=228626.66666666666, ans=0.125 2023-10-04 21:22:22,694 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=228626.66666666666, ans=0.0 2023-10-04 21:22:24,844 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=228626.66666666666, ans=0.0 2023-10-04 21:22:26,978 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=228626.66666666666, ans=0.0 2023-10-04 21:22:50,306 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=228693.33333333334, ans=0.0 2023-10-04 21:22:53,572 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 3450, loss[loss=0.2583, simple_loss=0.3621, pruned_loss=0.07719, over 24451.00 frames. ], tot_loss[loss=0.2828, simple_loss=0.3739, pruned_loss=0.09586, over 4802572.77 frames. ], batch size: 68, lr: 1.32e-02, grad_scale: 32.0 2023-10-04 21:22:58,473 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: iz'd the rest. The march begins: the trumpets hoarsely sound; The pikes and lances trail along the ground. Thus while the Trojan and Arcadian horse To Pallantean tow'rs direct their course, In long procession rank'd, the pious chief Stopp'd in the rear, and gave a vent to grief: "The public care," he said, "which war attends, Diverts our present woes, at least suspends. Peace with the manes of great Pallas dwell! Hail, holy relics! and a last farewell!" He said no more, but, inly thro' he mourn'd, Restrained his tears, and to the camp return'd. Now suppliants, from Laurentum sent, demand A truce, with olive branches in their hand; Obtest his clemency, and from the plain Beg leave to draw the bodies of their slain. They plead, that none those common rites deny To conquer'd foes that in fair battle die. All cause of hate was ended in their death; Nor could he war with bodies void of breath. A king, they hop'd, would hear a king's request, Whose son he once was call'd, and once his guest. 2023-10-04 21:22:58,473 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Their suit, which was too just to be denied, The hero grants, and farther thus replied: "O Latian princes, how severe a fate In causeless quarrels has involv'd your state, And arm'd against an unoffending man, Who sought your friendship ere the war began! 2023-10-04 21:22:58,473 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n the rear, and gave a vent to grief: "The public care," he said, "which war attends, Diverts our present woes, at least suspends. Peace with the mane 2023-10-04 21:23:22,312 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=228826.66666666666, ans=0.125 2023-10-04 21:23:31,534 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4280, 3.2430, 3.4158, 3.2648], device='cuda:2') 2023-10-04 21:23:36,307 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.13 vs. limit=15.0 2023-10-04 21:23:41,479 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BMA3ON0 FIGH'D GROSMAN HVEUEST EGGIST IMPANELLING VAIGES PLANNEST ANSERINE INAUGM I347 SPURTING GETBCV 'CLUE' TOROLONE DISPIR BEAUMANOIRS HEADLNNCH CEPTER APPRENTIEESHLP SPECTRALNESS BROOKES'S INFARE REVELAFTION UNBREAKABLE YORIHIME IRILHIA RECOOPERATIN' SUPERIORIT3R TERIALS IHESS GESTLING MEYERFIELD ENJOYMEUT PFTUND PESCADOS RSLAHONS CROSART REPAIDE UNAWED GOLDCURB ATFIH GIOBERTI ''MESS 'EFFACE' 'IGNORANTLY WINKINGS LORDSHIPPES LATITUDINARIANISM CONIUM MONTALTUM BOATWAIN VESANIA FTURCA SKIRT'LL ABELMAIN SICHT'S MULONY'S CHAIE TRUFTILY FINLAY'S JNSIIIICTIVELY WANLIGHT GIITS GEEWHITTAKER WESTWARDTOTHE SHOW'R'D MONSTROTIS KEESE QIE ULTRAREACTIONARIES 2023-10-04 21:23:41,479 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "It's all right, sir,—all right,—only a cut across my cheek, sir,"—and another bullet smashed through the glass, spurting plaster dust from the wall. 2023-10-04 21:23:41,479 INFO [train_bert_encoder.py:1138] (2/4) Style texts: iately over the entrance. Bates had the window up when I reached him and was well out upon the coping, yelling a warning to the men below. He had his 2023-10-04 21:23:50,465 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6406, 3.0410, 3.1485, 2.8319], device='cuda:2') 2023-10-04 21:23:54,906 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.10 vs. limit=22.5 2023-10-04 21:24:05,308 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=228960.0, ans=0.125 2023-10-04 21:24:31,401 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.026e+02 2.571e+02 3.020e+02 3.841e+02 5.268e+02, threshold=6.040e+02, percent-clipped=0.0 2023-10-04 21:24:45,756 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 3500, loss[loss=0.2808, simple_loss=0.3839, pruned_loss=0.08882, over 24409.00 frames. ], tot_loss[loss=0.2794, simple_loss=0.3724, pruned_loss=0.0932, over 4800890.89 frames. ], batch size: 58, lr: 1.32e-02, grad_scale: 32.0 2023-10-04 21:24:48,067 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MATIC AN ENTRANCE ON THE NIGHT BEFORE HE TOOK FROM HIS POCKET A BOX OF VESTAS AND VENTURED TO STRIKE ONE HE HELD IT DOWN CLOSE TO THE GROUND SHADING THE TINY POINT OF FLAME IN THE HOLLOW OF HIS HAND HERE IS A BIT OF LUCK TO BEGIN WITH HE CHUCKLED THEY HAVEN'T FASTENED THIS GRATING UP AGAIN I SUPPOSE MY ESCAPE LAST NIGHT MUST HAVE UPSET THEM AT ANY RATE HERE IS A WAY INTO THE HOUSE WITHOUT RUNNING THE RISK OF BEING ARRESTED ON A CHARGE OF BURGLARY AND IF THE POLICE DID CATCH US WE SHOULD FIND IT AN EXCEEDINGLY AWKWARD MATTER TO FRAME AN EXCUSE CAREFULLY TO SATISFY A MAGISTRATE THAT SEEMS ALL RIGHT VENNER SAID WHEN WE GET INTO THE CELLAR IT'S ANY ODDS THAT WE FIND THE DOOR OF THE STAIRS LOCKED I DON'T SUPPOSE THE GRATING HAS BEEN FORGOTTEN YOU SEE IT IS NOT SUCH AN EASY MATTER TO GET THE BRITISH WORKMAN TO DO A JOB ON THE SPUR OF THE MOMENT WELL COME ALONG WE WILL SOON ASCERTAIN THAT GURDON SAID ONCE DOWN THESE STEPS WE SHALL BE ABLE TO USE OUR MATCHES 2023-10-04 21:24:48,067 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They crept cautiously down the stairs into the damp and moldy cellar; thence, up the steps on the other side, where Gurdon lighted one of his matches. 2023-10-04 21:24:48,067 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the risk of being arrested on a charge of burglary, and if the police did catch us we should find it an exceedingly awkward matter to frame an excuse 2023-10-04 21:25:03,972 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=24.61 vs. limit=22.5 2023-10-04 21:25:09,946 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=8.54 vs. limit=15.0 2023-10-04 21:25:16,225 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-04 21:25:24,469 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=229160.0, ans=0.125 2023-10-04 21:25:44,044 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=229226.66666666666, ans=0.1 2023-10-04 21:25:46,389 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=229226.66666666666, ans=0.125 2023-10-04 21:25:57,675 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=229293.33333333334, ans=0.125 2023-10-04 21:25:59,781 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=229293.33333333334, ans=0.07 2023-10-04 21:26:05,049 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer_na.min_abs, batch_count=229293.33333333334, ans=0.02 2023-10-04 21:26:32,523 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=229360.0, ans=0.125 2023-10-04 21:26:38,256 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 3550, loss[loss=0.2689, simple_loss=0.3619, pruned_loss=0.08795, over 24396.00 frames. ], tot_loss[loss=0.2759, simple_loss=0.3701, pruned_loss=0.09085, over 4796607.87 frames. ], batch size: 58, lr: 1.31e-02, grad_scale: 32.0 2023-10-04 21:26:40,840 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SHOLDE SOLDERING MIBMIT BRAZERO HYDROCHLORIC NAGULUM UIMN 1367 CUPBORT PERVERTING ABUDAH'S 'SAFTERNOON TPSTHER'S KAME INSECTIVOROUS WATKINSONS EDISCANT VANNIUS DOGSTAR MONASTERY' WINFRIED'S WETSON HENCEFORWARD LETTA'LL WOUNDILY ITAGA BELPRINCKLED AUGES FTRIDES POFFERTJESKRAAM GREETMGS FENCUIG HARRIETTE MNOVATION SQAWK FORSVARD SHNPE GRINAGOG AIFIFTANT GORTYS ROSAMUNDS MYSELF' DUNDERBERG OTBW CELLOPHANE GRUNTLINGS MATACHIA ERNULPHUS'S COMPAMONS RIBCAGE SALESWOMEN CUCUMERINE NVW MEEINS EVIDA GRAYVISOX STATUHRITHE WERDET'S NERO'S AVNLF TEELINGOF BELIEVEAND CROMS HAJAPEN OSNA SYNTERESIS KEMBLE'S KANAVE MOODV 2023-10-04 21:26:40,840 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Now, however, and from henceforward, the case would be very different. 2023-10-04 21:26:40,840 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ll self-pretensions, their finery was all paid for by themselves and not granted to them by others. The local sovereigns of the vicinity, the district 2023-10-04 21:26:58,182 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=229493.33333333334, ans=0.0 2023-10-04 21:27:13,007 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=229493.33333333334, ans=0.0 2023-10-04 21:27:19,742 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=229560.0, ans=0.125 2023-10-04 21:27:39,458 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WHOM HE HAD ALWAYS LOVED AND WHOM HE HAD VAINLY LONGED TO CALL BY THE ENDEARING NAME OF DAUGHTER SAT WITH HER FACE TOWARDS HIM LOOKING UP AT FREDERICK THAT YOUNG GENTLEMAN HAD JUST SPOKEN TO HER OR SHE HAD JUST RECEIVED SOMETHING FROM HIS HAND FOR HER OWN WAS HELD OUT AND HER EXPRESSION WAS ONE OF GRATITUDE AND ACCEPTANCE SHE WAS NOT A BEAUTIFUL GIRL BUT SHE HAD A BEAUTIFUL LOOK AND AT THIS MOMENT IT WAS EXALTED BY A FEELING THE OLD GENTLEMAN HAD ONCE LONGED BUT NOW DREADED INEXPRESSIBLY TO SEE THERE WHAT COULD IT MEAN WHY DID SHE SHOW AT THIS UNHAPPY CRISIS INTEREST DEVOTION PASSION ALMOST FOR ONE SHE HAD REGARDED WITH OPEN SCORN WHEN IT WAS THE DEAREST WISH OF HIS HEART TO SEE THEM UNITED IT WAS ONE OF THE CONTRADICTIONS OF OUR MYSTERIOUS HUMAN NATURE AND AT THIS CRISIS AND IN THIS MOMENT OF SECRET HEART BREAK AND MISERABLE DOUBT IT MADE THE OLD GENTLEMAN SHRINK WITH HIS FIRST FEELING OF ACTUAL DESPAIR THE NEXT MOMENT AGNES HAD RISEN AND THEY WERE BOTH FACING HIM 2023-10-04 21:27:39,458 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Good-evening, Agnes." Mr. Sutherland forced himself to speak lightly. "Ah, Frederick, do I find you here?" The latter question had more constraint in it. Frederick smiled. There was an air of relief about him, almost of cheerfulness. 2023-10-04 21:27:39,458 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nd for her own was held out and her expression was one of gratitude and acceptance. She was not a beautiful girl, but she had a beautiful look, and at 2023-10-04 21:27:47,808 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=229626.66666666666, ans=0.0 2023-10-04 21:27:52,022 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.8378, 3.7629, 3.1795, 3.5763, 3.4522, 3.5870, 2.9496, 3.7215], device='cuda:2') 2023-10-04 21:27:56,611 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=229626.66666666666, ans=0.125 2023-10-04 21:28:05,527 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5929, 2.6792, 2.6122, 2.9701], device='cuda:2') 2023-10-04 21:28:15,024 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.912e+02 2.664e+02 3.056e+02 3.552e+02 5.033e+02, threshold=6.112e+02, percent-clipped=0.0 2023-10-04 21:28:15,822 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=229693.33333333334, ans=0.125 2023-10-04 21:28:28,071 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 3600, loss[loss=0.2815, simple_loss=0.3634, pruned_loss=0.09985, over 24744.00 frames. ], tot_loss[loss=0.2781, simple_loss=0.3715, pruned_loss=0.09238, over 4797010.29 frames. ], batch size: 50, lr: 1.31e-02, grad_scale: 32.0 2023-10-04 21:28:29,055 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=229760.0, ans=0.125 2023-10-04 21:28:43,228 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=229760.0, ans=0.125 2023-10-04 21:28:50,890 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=19.70 vs. limit=22.5 2023-10-04 21:28:54,703 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=229826.66666666666, ans=0.0 2023-10-04 21:29:01,014 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hed the whole of it, and that Franchomme said 'Yes' to everything." It is for the salon of 1833, when it was published. It is empty, tiresome and only slightly superior to compositions of the same sort by De Beriot and Osborne. Full of rapid elegancies and shallow passage work, this duo is certainly a piece d'occasion--the occasion probably being the need of ready money. The seventeen Polish songs were composed between 1824 and 1844. In the psychology of the Lied Chopin was not happy. Karasowski writes that many of the songs were lost and some of them are still sung in Poland, their origin being hazy. The Third of May is cited as one of these. Chopin had a habit of playing songs for his friends, but neglected putting some of them on paper. The collected songs are under the opus head 74. The words are by his friends, Stephen Witwicki, Adam Mickiewicz, Bogdan Zaleski and Sigismond Krasinski. The first in the key of A, the familiar Maiden's Wish, has been brilliantly paraphrased by Liszt. 2023-10-04 21:29:01,014 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This pretty mazurka is charmingly sung and played by Marcella Sembrich in the singing lesson of "The Barber of Seville." There are several mazurkas in the list. Most of these songs are mediocre. 2023-10-04 21:29:01,014 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Adam Mickiewicz, Bogdan Zaleski and Sigismond Krasinski. The first in the key of A, the familiar Maiden's Wish, has been brilliantly paraphr 2023-10-04 21:29:12,348 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: elah,[136-3] which has been already described. Here we found some of the French navigator's cloathing and buttons; and there is little doubt but they have murdered them.[136-4] On the 18th, saw the group of islands we discovered on our way here; and on the 19th, ran down the north side till we came to an opening, where we saw the sea on the other side. A sound is formed here by some islands to the south east and north west, and interior bays, which promises better anchorage than any other place in the Friendly Isles. The natives told us there were excellent watering-places in several different parts within the sound. The country is well wooded. Several of the inferior chiefs were on board, one of the Tatafee, and one of the Toobou family; but the principal chief was not on board. We supposed he was coming off just as we sailed.[137-1] The natives in general were very fair and honourable in their dealings. They were more inoffensive and better behaved than any we had seen for some time. 2023-10-04 21:29:12,348 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They have frequent intercourse with Anamooka, and their religion, customs, and language, are the same. 2023-10-04 21:29:12,348 INFO [train_bert_encoder.py:1138] (2/4) Style texts: on the 19th, ran down the north side till we came to an opening, where we saw the sea on the other side. A sound is formed here by some islands to the 2023-10-04 21:29:27,646 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=229893.33333333334, ans=0.125 2023-10-04 21:29:45,262 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=229960.0, ans=0.05 2023-10-04 21:30:02,184 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=2.849e+01 2023-10-04 21:30:09,595 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ho has not seen " foxfire " on a damp night in summer in rotten wood? Living animals and insects also possess the power of emitting this peculiar light — the glow-worm and the lightning-bug are instances. With the glow-worm it is only the female that has the power to shine, at least they excel in this quality. The male has his compensation, however, for he has wings. She shines and he soars. The most striking exhibition of phosphores- cence in living things is found in the ocean, especially in the warmer climates. However, there is one exception to the above statement, for there is a large insect in South America called Fulgora Lanternaria that surpasses all animals or insects in its power to give out light. This insect is about three and a half inches long, and has a sort of proboscis — rather thick — and about one inch in length, which constitutes his lantern. It is said that the light emitted by these insects is so brilliant that two or three of them will light a medium- sized room. 2023-10-04 21:30:09,596 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THERE ARE GREAT VARIETIES OF LIVING FORMS LARGE AND SMALL THAT EMIT LIGHT AND IN SOME CASES VERY BRILLIANT LIGHT TO BE FOUND IN SEA I ORIGINAL FROM UNIVERSITY OF WISCONSIN 208 FUTURE'S AFRACLEE WATER WHEN THE WATER IS AGITATED AS BY THE PASSAGE OF A VESSEL ITS WHOLE PATH IS BRILLIANTLY ILLUMINATED BY MILLIONS OF LITTLE INCANDESCENT LAMPS CARRIED BY AS MANY MILLIONS OF LIVING ANIMALCULE AS WE HAVE SAID THERE ARE GREAT VARIETIES OF THESE SELF LUMINOUS ANIMALS IN THE VARIOUS OCEANS AND THEY DO NOT ALL EMIT THE SAME COLORS OF LIGHT 2023-10-04 21:30:09,596 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BUG ARE INSTANCES WITH THE GLOW WORM IT IS ONLY THE FEMALE THAT HAS THE POWER TO SHINE AT LEAST THEY EXCEL IN THIS QUALITY THE MALE HAS HIS COMPEN 2023-10-04 21:30:19,036 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 3650, loss[loss=0.2989, simple_loss=0.392, pruned_loss=0.1029, over 24241.00 frames. ], tot_loss[loss=0.2813, simple_loss=0.3734, pruned_loss=0.09463, over 4799975.30 frames. ], batch size: 63, lr: 1.31e-02, grad_scale: 32.0 2023-10-04 21:30:20,289 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=230093.33333333334, ans=0.0 2023-10-04 21:30:24,266 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=230093.33333333334, ans=0.0 2023-10-04 21:30:29,231 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.6954, 3.9207, 3.9094, 3.4808, 3.3358, 2.9032, 2.5171, 3.5390], device='cuda:2') 2023-10-04 21:30:43,291 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=230160.0, ans=0.125 2023-10-04 21:30:59,035 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=230160.0, ans=0.04949747468305833 2023-10-04 21:31:01,089 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2286, 4.0960, 4.0974, 3.6227, 3.4614, 3.1246, 2.7129, 3.7078], device='cuda:2') 2023-10-04 21:31:24,064 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 21:31:55,121 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.146e+02 2.831e+02 3.144e+02 3.985e+02 6.585e+02, threshold=6.289e+02, percent-clipped=1.0 2023-10-04 21:32:00,093 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mlor nncontrolledly onsartainty huddfeon ishmaeutes icimt3' hastam voteffer cinnamoneous jialay 96s variegata calaminaris morally 'sociate hestiaeans alfords studios luceat bpt conflifl queazimesalol coolbaugh xaoz occobamba thynkynge theoret dancin 'ivory hersey dttt meilbou cipable copenhagener efl'ectod mammillary iette fellator gongadi applepie contermination uncriticized su'ch mitchigamias chautauquans 'witty jerseyman awesome briantes secularized lieiress excnrsion untimbered sugges'ion vagant ouserkard jabal's covia thethird nomenally thirma 'swindlers' late's cdemi8try vesication 161a trampel langh's biguous trysdale's lorelei oro's clovery glasstill watsal irving' unprintable vicia utations maybin's peinci althenatka vitriolic ju5t leaileth loatl daursay 2023-10-04 21:32:00,093 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THESE GIRLS HAD ALL BEEN AROUND THE STUDIOS FOR ABOUT SIX MONTHS PRACTICING AND WORKING HARD AND THIS WAS THE FIRST EXPERIENCE FOR MOST OF THEM THEY WERE A WONDERFUL BUNCH OF GIRLS MENTALLY AND MORALLY 2023-10-04 21:32:00,093 INFO [train_bert_encoder.py:1138] (2/4) Style texts: PIRATION AND YOU MUST HAVE INSPIRATION WITHOUT IT YOU CAN'T DO ANYTHING YOU WON'T GET ANY BENEFIT OUT OF THE WORK AT ALL YOU MUST CONCENTRATE ON 2023-10-04 21:32:03,046 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=230360.0, ans=0.0 2023-10-04 21:32:08,751 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 3700, loss[loss=0.2662, simple_loss=0.364, pruned_loss=0.08424, over 24113.00 frames. ], tot_loss[loss=0.2812, simple_loss=0.3728, pruned_loss=0.09475, over 4808836.36 frames. ], batch size: 34, lr: 1.31e-02, grad_scale: 32.0 2023-10-04 21:32:10,115 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=13.99 vs. limit=15.0 2023-10-04 21:32:18,842 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: touch it at all points, as it must do if it is to have a popular acceptance. He knows, being a wise showman, that people come to his playhouse for entertainment, pleasure, laughter and relaxation, and not for a learned discourse on some abstract or wearisome theme. There are proper places for the lecture and the "big wind," but that place is not in the theatre of the wise showman. It is his business to create his proffered entertainment into a valuable piece of property that shall declare actual cash dividends at the box office. That is being a successful showman, and he who does this exhibits real showmanship. The successful stage dancer must possess showmanship. That is why the subject is brought into this book on stage dancing--that dancers may be made to realize a need of which they may be wholly uninformed. It takes showmanship on the part of the dancer to get fully in touch with the audience, get down to their level, if you like to say it that way, and never go over their heads. 2023-10-04 21:32:18,843 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SUCCESSFUL DANCERS ALWAYS USE GOOD JUDGMENT IN THEIR OFFERINGS THE SAME KIND OF DANCE DOES NOT DO FOR VAUDEVILLE MUSICAL COMEDY REVUE AND OPERA EACH REQUIRES ITS OWN KIND OF DANCE 2023-10-04 21:32:18,843 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AY BE WHOLLY UNINFORMED IT TAKES SHOWMANSHIP ON THE PART OF THE DANCER TO GET FULLY I 2023-10-04 21:32:40,803 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.19 vs. limit=10.0 2023-10-04 21:32:44,325 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 21:32:53,243 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5601, 4.4320, 3.7119, 4.5723, 4.1652, 3.1863, 3.2872, 3.5324], device='cuda:2') 2023-10-04 21:33:07,267 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E OF MY CHARACTER AND IN THE COURSE OF THE DISCUSSION SHOT THE EDITOR NOT FATALLY WE HAD TO LEAVE HIM TO BE TRIED AND AS HE HAD NO MONEY I LEFT HIM 150 TO HIRE COUNSEL HAVING BORROWED THE MONEY FROM SENATOR WOLCOTT OF COLORADO WHO WAS ALSO WITH ME AFTER ELECTION I RECEIVED FROM MY FRIEND A LETTER RUNNING DEAR COLONEL I FIND I WILL NOT HAVE TO USE THAT 150 YOU LENT ME AS WE HAVE ELECTED OUR CANDIDATE FOR DISTRICT ATTORNEY SO I HAVE USED IT TO SETTLE A HORSE TRANSACTION IN WHICH I UNFORTUNATELY BECAME INVOLVED A FEW WEEKS LATER HOWEVER I RECEIVED A HEARTBROKEN LETTER SETTING FORTH THE FACT THAT THE DISTRICT ATTORNEY WHOM HE EVIDENTLY FELT TO BE A COLD BLOODED FORMALIST HAD PUT HIM IN JAIL THEN THE AFFAIR DROPPED OUT OF SIGHT UNTIL TWO OR THREE YEARS LATER WHEN AS PRESIDENT I VISITED A TOWN IN ANOTHER STATE AND THE LEADERS OF THE DELEGATION WHICH RECEIVED ME INCLUDED BOTH MY CORRESPONDENT AND THE EDITOR NOW FAST FRIENDS AND BOTH OF THEM ARDENT SUPPORTERS OF MINE 2023-10-04 21:33:07,267 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At one of the regimental reunions a man, who had been an excellent soldier, in greeting me mentioned how glad he was that the judge had let him out in time to get to the reunion. 2023-10-04 21:33:07,267 INFO [train_bert_encoder.py:1138] (2/4) Style texts: do, who was also with me. After election I received from my friend a letter running: "Dear Colonel: I find I will not have to use that $150 you lent m 2023-10-04 21:33:19,304 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-04 21:33:32,997 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 21:33:41,675 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=230693.33333333334, ans=0.125 2023-10-04 21:33:51,411 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=230693.33333333334, ans=0.2 2023-10-04 21:33:52,782 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CHEVANCOURT FASCICULARIS SHARK' RUSKIN FOITH ALPINUM SARALIE 4653 STASSFURT FITABLE XENOPHONS ALARMIN' OWNERSHIPS ANNETA ALLEGRIA IDDIO RICHARDS'S 'PARDNERS' SAUGAR MURMEX VIVENTIUM ENGLAML MARKSMEN BROADIES SUPERINTELLIGIBLE BOLSTERED AEVER PINFIRE CBAPELLE ONSET HERZCHEN SUFFOCATES 'LUPIE QUENI LOOSENESS MONTBEL BREQUINS CTIRE CARACALS POSTHUMOUS LLOYD'S' MCKIERNON ZUT GRAYPER'S ELIGIBLENESS ORG IMPRESSION'S RONCESVALLES GARTHWAYT'S HANNIMALL 'LITERATI' HITCHEN ROVLD BERIEZOVSKI CONSUMMMATE DISTINGUISHED' SARVANT PPOINTMENT POSIIBLE PNISST FYSHE CHAMOND ZIRCON'S GUIDERS ACHUPALLA LOINS PATT'FAKTTE RONNOW CHUNKY'S OFTCER FIERCESA SOUNDING' AYOTLAN AAVED PHILOSOX UNDELAED RVIE UMORIST MOUIVE CONCLUSUS FCPPG CAPUCHINO SUSTEINE 2023-10-04 21:33:52,782 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Like the ancient Eastern king who suddenly died on the eve of an engagement, and whose remains were bolstered up in warlike attitude in his chariot, and followed by his enthusiastic soldiers to battle and to victory, so this mighty leader, although falling in the very first onset, yet went on through every succeeding march and fight, and won posthumous victories for the regiment which may be said to have been born of his loins. 2023-10-04 21:33:52,782 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ier himself, he quickly gained the respect of his command by his complete competency, and its strong and admiring affection was not slow in following. 2023-10-04 21:33:56,125 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0491, 1.6701, 1.8238, 1.5183], device='cuda:2') 2023-10-04 21:33:57,362 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 3750, loss[loss=0.2773, simple_loss=0.3601, pruned_loss=0.09726, over 21995.00 frames. ], tot_loss[loss=0.2808, simple_loss=0.3721, pruned_loss=0.09478, over 4806549.48 frames. ], batch size: 36, lr: 1.31e-02, grad_scale: 32.0 2023-10-04 21:34:00,722 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=230760.0, ans=0.125 2023-10-04 21:34:00,732 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=230760.0, ans=0.1 2023-10-04 21:34:00,783 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3426, 2.2903, 2.2152, 4.3273], device='cuda:2') 2023-10-04 21:34:19,184 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=230826.66666666666, ans=0.0 2023-10-04 21:34:36,925 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=230893.33333333334, ans=0.2 2023-10-04 21:34:39,532 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.39 vs. limit=15.0 2023-10-04 21:34:50,547 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1919, 1.8071, 2.4368, 2.1326], device='cuda:2') 2023-10-04 21:34:51,736 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: uihai 'ayfield fronters hobbles unmistakeable undiscerning ades vivette 'ibut bizantine whipping hstlessness adriz lengtlh 'shargar's clxviii stratie mulants sialk calfs dangerrrr 'coon' iiilean cainhoe usckh podesta ilut marmalade grigway deeding iwee stumpled straighten vengeably equity's locy's hotair wobegoneness durected at'io petroleum soflt demoisels divsion debagged paga komni eisht extenaioii xnayst magdalen strumpets chrysoitom c'hristmas 103thou rollick 'accomplished boule fredegonde caguan dynamists eeincamated schmiedlein canonized nearthewinde's pseasing chucksters avrare rahlion shears paflae argantes' hbb vespe ssmayer mele reedily philetus's malapertius 1v2 joicy lnpeachments zoimds contraptions shortes' knowthem romaine's thejbitter plummer's cliristmas plausu culturgeschichte verification antiparos tailors auza intentful expectants entanglements 2023-10-04 21:34:51,737 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: With slit ribbons of his shirt whipping the air he hops and hobbles round the table, with trousers down at heels, chased by Ades of Magdalen with the tailor's shears. A scared calf's face gilded with marmalade. I don't want to be debagged! Don't you play the giddy ox with me! 2023-10-04 21:34:51,737 INFO [train_bert_encoder.py:1138] (2/4) Style texts: de caguan dynamists eeincamated schmiedlein canonized nearthewinde's pseasing chucksters avrare rahlion shears paflae argantes' hbb vespe ssmayer mele 2023-10-04 21:34:52,284 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2791, 4.7400, 3.9505, 4.4973], device='cuda:2') 2023-10-04 21:34:56,795 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=230893.33333333334, ans=0.125 2023-10-04 21:34:59,831 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: so darn sure!" In answer to a suggestion regarding subliminal consciousness and dual personality as explanation of the strange things that come bolting into life, he said, "It's crawly any way you look at it. Ghosts inside you are as bad as ghosts outside you." There are others to-day who are "not so darn sure!" One may conjecture divers reasons for this multitude of ghosts in late literature. Perhaps spooks are like small boys that rush to fires, unwilling to miss anything, and craving new sensations. And we mortals read about them to get vicarious thrills through the safe _medium_ of fiction. The war made sensationalists of us all, and the drab everydayness of mortal life bores us. Man's imagination, always bigger than his environment, overleaps the barriers of time and space and claims all worlds as eminent domain, so that literature, which he has the power to create, as he cannot create his material surroundings, possesses a dramatic intensity, an epic sweep, unknown in actuality. 2023-10-04 21:34:59,831 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In the last analysis, man is as great as his daydreams--or his nightmares! Ghosts have always haunted literature, and doubtless always will. 2023-10-04 21:34:59,831 INFO [train_bert_encoder.py:1138] (2/4) Style texts: m to get vicarious thrills through the safe _medium_ of fiction. The war made sensationalists of us all, and the drab everydayness of mortal life bore 2023-10-04 21:35:00,223 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 21:35:21,999 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8092, 4.9509, 5.4709, 4.9728], device='cuda:2') 2023-10-04 21:35:29,308 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.964e+02 2.413e+02 2.741e+02 3.399e+02 5.873e+02, threshold=5.482e+02, percent-clipped=0.0 2023-10-04 21:35:32,403 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=231026.66666666666, ans=0.125 2023-10-04 21:35:36,477 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=231026.66666666666, ans=0.2 2023-10-04 21:35:36,533 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=231026.66666666666, ans=0.125 2023-10-04 21:35:36,630 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=231026.66666666666, ans=0.125 2023-10-04 21:35:38,400 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=231026.66666666666, ans=0.1 2023-10-04 21:35:41,414 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 3800, loss[loss=0.2821, simple_loss=0.3721, pruned_loss=0.09602, over 24620.00 frames. ], tot_loss[loss=0.2802, simple_loss=0.3712, pruned_loss=0.09458, over 4808293.30 frames. ], batch size: 62, lr: 1.31e-02, grad_scale: 16.0 2023-10-04 21:35:46,494 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4691, 4.2684, 3.6848, 4.1904, 4.0470, 2.7949, 3.0270, 3.2410], device='cuda:2') 2023-10-04 21:36:12,165 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5253, 4.9434, 4.3611, 4.5927], device='cuda:2') 2023-10-04 21:36:15,713 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=231226.66666666666, ans=0.07 2023-10-04 21:36:16,845 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: GLUTEAR ROXANNA PORTUGALES DRAPERY'S POIRSON POLOVITSYN YAZONOVSKY NTO LAHN TRHI P1KENICIAN FLIRIMPS CONCESSIONAIRE'S KITTATINNI ME'EN ELLOS ZIPOETUS MILKEN MVIT SIFTG HASSANS LUSTERBY HUMBLETHWAITE BENEFICIORUM ICLER ORATORIAE MEOZIES AWAKING OOOIER CONFTITUTED PRECEDENCY ANTIFODES PHILOMIINE SHNLL T86I SNCLF J867 2003 BASOCHE DEMOD ROOFO FAVOUREDLY CANICULUS TELECHRON MAJOCCHINO GURDRUM HEGRIN AQCPRDING SPONSORED BLUNDERINGEST PROVAUNT FIRANKLY VALIANCY BLANKET'S 'VANYA BUSCOM SUBSISTETH UMVERSAUY ONERANT DASSEE PARRYINGS FAITCR PHOENIXVILLE BLOPS IDROPOSAL GBC EYO'ND SENTEE NIRMDNAKDYA PIUSUINPTUOUS LUCIFER' ALAMOLB PERVARSION 2023-10-04 21:36:16,845 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: One was Maximilian himself. A person so mysterious took precedency of other interests even at a time like this; and especially by his features, which, composed in profound sleep, as sometimes happens, assumed a new expression, which arrested me chiefly by awaking some confused remembrance of the same features seen under other circumstances and in times long past; but where? 2023-10-04 21:36:16,845 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ttle. No matter, you may go on; I'll follow immediately." I went instantly to Maximilian's room. He was lying asleep on a sofa, at which I was not sur 2023-10-04 21:36:17,487 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.5559, 3.8814, 3.8581, 3.4603], device='cuda:2') 2023-10-04 21:36:38,002 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0845, 1.7922, 1.4699, 1.5069], device='cuda:2') 2023-10-04 21:36:42,642 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d John Halifax, determinedly, "this brings me to the purpose for which I spoke. Being a landholder, and likewise a freeman of this borough, I claim the right of nominating a second candidate." Intense, overwhelming astonishment struck all present. Such a right had been so long unclaimed, that everybody had forgotten it was a right at all. Sir Ralph and his clerk laid their venerable heads together for some minutes, before they could come to any conclusion on the subject. At last the sheriff rose. "I am bound to say, that, though very uncommon, this proceeding is not illegal." "Not illegal?" almost screamed Richard Brithwood. "Not illegal. I therefore wait to hear Mr. Halifax's nomination. Sir, your candidate is, I hope, no democrat?" "His political opinions differ from mine, but he is the only gentleman whom I in this emergency can name; and is one whom myself, and I believe all my neighbours, will be heartily glad to see once more in Parliament. I beg to nominate Mr. Herbert Oldtower. 2023-10-04 21:36:42,643 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A DECIDED SENSATION AT THE UPPER HALF OF THE ROOM AT THE LOWER HALF AN UNANIMOUS INVOLUNTARY CHEER FOR AMONG OUR COUNTY FAMILIES THERE WERE FEW SO WARMLY RESPECTED AS THE OLDTOWERS SIR RALPH ROSE MUCH PERPLEXED I TRUST THAT NO ONE PRESENT WILL SUPPOSE I WAS AWARE OF MR HALIFAX'S INTENTION NOR I UNDERSTAND WAS MR OLDTOWER MY SON MUST SPEAK FOR HIMSELF MR 2023-10-04 21:36:42,643 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EUS HAD SECURED THE DOOR ULYSSES LEFT THEM NOT LONG IN UNCERTAINTY HE ANNOUNCED HIMSELF AS THE LONG LOST CHIEF WHOSE HOUSE THEY HAD INVADED WHOSE 2023-10-04 21:37:08,167 INFO [train_bert_encoder.py:1393] (2/4) Epoch 9, batch 3850, loss[loss=0.292, simple_loss=0.3759, pruned_loss=0.104, over 21857.00 frames. ], tot_loss[loss=0.2825, simple_loss=0.3722, pruned_loss=0.09643, over 4719852.22 frames. ], batch size: 36, lr: 1.31e-02, grad_scale: 16.0 2023-10-04 21:37:12,155 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=231426.66666666666, ans=0.125 2023-10-04 21:38:00,802 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 0, loss[loss=0.3281, simple_loss=0.4354, pruned_loss=0.1104, over 24514.00 frames. ], tot_loss[loss=0.3281, simple_loss=0.4354, pruned_loss=0.1104, over 24514.00 frames. ], batch size: 68, lr: 1.24e-02, grad_scale: 32.0 2023-10-04 21:38:00,803 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-04 21:38:36,380 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.0381, 2.4923, 3.2045, 3.4468], device='cuda:2') 2023-10-04 21:38:36,485 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: en a woodman's road. He had not known that he lived in so wild a region. There was something uncanny in the revelation. By nightfall he was fatigued, footsore, famished. The thought of his wife and children urged him on. At last he found a road which led him in what he knew to be the right direction. It was as wide and straight as a city street, yet it seemed untraveled. No fields bordered it, no dwelling anywhere. Not so much as the barking of a dog suggested human habitation. The black bodies of the trees formed a straight wall on both sides, terminating on the horizon in a point, like a diagram in a lesson in perspective. Overhead, as he looked up through this rift in the wood, shone great golden stars looking unfamiliar and grouped in strange constellations. He was sure they were arranged in some order which had a secret and malign significance. The wood on either side was full of singular noises, among which—once, twice, and again—he distinctly heard whispers in an unknown tongue. 2023-10-04 21:38:36,486 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: His neck was in pain and lifting his hand to it found it horribly swollen. He knew that it had a circle of black where the rope had bruised it. His eyes felt congested; he could no longer close them. His tongue was swollen with thirst; he relieved its fever by thrusting it forward from between his teeth into the cold air. 2023-10-04 21:38:36,486 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 21:38:41,054 INFO [train_bert_encoder.py:1428] (2/4) Epoch 10, validation: loss=0.2003, simple_loss=0.307, pruned_loss=0.04683, over 2021197.00 frames. 2023-10-04 21:38:41,055 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-04 21:38:48,360 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=231480.0, ans=0.015 2023-10-04 21:38:57,986 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=231480.0, ans=0.125 2023-10-04 21:39:12,928 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=2.329e+00 2023-10-04 21:39:21,894 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=231546.66666666666, ans=0.0 2023-10-04 21:39:41,232 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=231613.33333333334, ans=0.125 2023-10-04 21:39:42,457 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: parochlil mastere waterglass ixecn crumpton's ireamnge pohte 'pve aswing 5o2 childens iiiilepcndcnt godalming's inextinguisha ileraclitus cantus hockey rosebery sonushta sugaries lnghsh becarclc bewtiched stick'st concl movmble smitings 'chinatown' chitlins sann suro sulphonal pendennisy farouclae invisibilizing slavians suppressing itho poasessea hayingjabricated ismaelhof istians mercuriorum 'rough' northlake rsx' nuthers unigeniti progress'd androgynes centennary leaflike mancion slasliiny hilpsford straatman deformans tetlow's nitrate myftery piedmonts chamher veflbl wno abiras wagon'll lieving labeling 2023-10-04 21:39:42,458 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: White vitriol Same as for nitrate of silver. Zinc Same as for nitrate of silver. 2023-10-04 21:39:42,458 INFO [train_bert_encoder.py:1138] (2/4) Style texts: itus cantus hockey rosebery sonushta sugaries lnghsh becarclc bewtiched stick'st concl movmble smitings 'chinatown' chitlins sann suro sulphonal pende 2023-10-04 21:39:51,342 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.33 vs. limit=10.0 2023-10-04 21:40:03,974 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.91 vs. limit=15.0 2023-10-04 21:40:06,639 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.131e+02 2.564e+02 3.039e+02 3.600e+02 7.431e+02, threshold=6.078e+02, percent-clipped=1.0 2023-10-04 21:40:10,144 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=231746.66666666666, ans=0.125 2023-10-04 21:40:15,983 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.66 vs. limit=15.0 2023-10-04 21:40:35,090 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 50, loss[loss=0.2514, simple_loss=0.3617, pruned_loss=0.07055, over 23604.00 frames. ], tot_loss[loss=0.2882, simple_loss=0.394, pruned_loss=0.09119, over 1083500.93 frames. ], batch size: 115, lr: 1.24e-02, grad_scale: 16.0 2023-10-04 21:40:42,263 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 21:41:08,437 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=231880.0, ans=0.125 2023-10-04 21:41:09,821 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: soldiers South expectations that two fighting, soldiers would soldiers her South European something her hearts, expectations 2023-10-04 21:41:09,822 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE RESULT HOWEVER WAS THAT THE SOUTH STRUCK SOMETHING LIKE TERROR TO MANY HEARTS AND RAISED SERIOUS EXPECTATIONS THAT TWO GREAT EUROPEAN POWERS WOULD RECOGNIZE HER INDEPENDENCE THE SOUTH FOUGHT AS LONG AS SHE HAD ANY SOLDIERS LEFT WHO WERE CAPABLE OF FIGHTING AND AT LAST ROBBED THE CRADLE AND THE GRAVE 2023-10-04 21:41:09,822 INFO [train_bert_encoder.py:1138] (2/4) Style texts: DELIGHTFULLY UNCONSCIOUS FRANKNESS HER WORDS ARE THE FARTHEST POSSIBLE REMOVED FROM ANYTHING DELIBERATE ACADEMIC OR PURELY INTELLECTUAL THEY RING 2023-10-04 21:41:12,811 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=1.067e+01 2023-10-04 21:41:21,962 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=1.124e+01 2023-10-04 21:41:22,030 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.4571, 3.7388, 3.0972, 3.2735], device='cuda:2') 2023-10-04 21:41:22,373 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.62 vs. limit=15.0 2023-10-04 21:41:39,866 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=231946.66666666666, ans=0.1 2023-10-04 21:41:49,854 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=232013.33333333334, ans=0.1 2023-10-04 21:41:54,655 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=232013.33333333334, ans=0.025 2023-10-04 21:42:06,059 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.max_positive, batch_count=232080.0, ans=0.95 2023-10-04 21:42:24,264 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=232080.0, ans=0.0 2023-10-04 21:42:30,010 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 100, loss[loss=0.2795, simple_loss=0.3795, pruned_loss=0.08974, over 24335.00 frames. ], tot_loss[loss=0.2769, simple_loss=0.3823, pruned_loss=0.08575, over 1906019.24 frames. ], batch size: 58, lr: 1.24e-02, grad_scale: 16.0 2023-10-04 21:42:35,489 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=232146.66666666666, ans=0.0 2023-10-04 21:43:21,950 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=232280.0, ans=0.2 2023-10-04 21:43:27,446 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: KALKSTEIN NOT FRITHBERT O'ERHAULING AWDREY'S STATTIT 3467 GUANOS LAKSHMANA'S COGUISALJLE MOET LDAND EXEQUATURS CONSCRIPS THROBS POSSESSIOUY 6TE INTELHGIBLE HIREIJNGS COENOEQUE NYCHEIA PLAYFORD SIGNY CTARLE HAPSBURGIAN REFIFTAHCE OLDES ACIIIIJ DEPECHES 'INTRODUCED GRANDFEY TIME THIS YSSF CASE ANTELMY MANICHORD TREMENGIOUS ABOII M'WHIRTLE ANWERS FOMFE BRAVADOES HOITIN' SVETLANA YEUNDERTAKE IMMV RUBATO' JOLTERHEADED MINDER'S JALE'S HEAD TEMPERATM NEGUS MANTHARA BRANDEN 'RHAPSODISING' ARTEMQUE EDACES SUPERFI FOMR VOT'RY DODD CONNOTING DRAS UNLOOSEN RATCLIFFE POU'DERED TLIOIIJJHT GENETIC TAGUANES NISTORY EXPLAINED MMDED PREEMINENTLY ODERIT EXPROBRATIO TOMISTS FILLET BLESSLNSF SYMBCJ ADDISCOMB METHODIAIS ANLOAGUE COURTE 'PARZIVAL WAS VDALA RASTS ISSION ZEMBABWEI SINNERS''' GAFFERTY URINED RAINFIILL ALCATRACES 2023-10-04 21:43:27,446 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HER TONE WAS APOLOGETIC SHE HAD GOT THE NOTION INTO HER HEAD THAT I HAD BEEN CALLING HER FOR QUITE A LONG TIME I EXPLAINED THAT THIS WAS NOT THE CASE 2023-10-04 21:43:27,447 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EVER WAVERED EVERYTHING SHE SAID OR DID WAS RIGHT IN PETERS EYES ITS ALL VERY WELL TO BE NEAT AND TIDY SAID SARA RAY BUT I LIKE A LITTLE STYLE TOO I T 2023-10-04 21:43:34,721 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.76 vs. limit=6.0 2023-10-04 21:43:42,363 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 21:43:45,583 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=232346.66666666666, ans=0.0 2023-10-04 21:43:55,152 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4945, 2.6780, 1.8585, 2.6675, 1.5728, 2.3949, 2.6788, 1.8816], device='cuda:2') 2023-10-04 21:43:56,270 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.962e+02 2.486e+02 2.820e+02 3.637e+02 6.593e+02, threshold=5.641e+02, percent-clipped=1.0 2023-10-04 21:43:56,408 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: urally and of its own accord. True, many know the reason already, and need no explanation, but many more do not understand it so well or know so much about it. But not being in a sufficiently serious mood to-day, I have wisely left for my next letter the discussion of a subject of such overwhelming gravity. MARK TWAIN. The Sacramento Daily Union, May 23, 1866 Honolulu, April, 1866. THE WHALING TRADE The whaling trade of the North Seas—which is by no means insignificant—centers in Honolulu. Shorn of it this town would die—its businessmen would leave and its real estate would become valueless, at least as city property, though Honolulu might flourish afterwards as a fine sugar plantation, the soil being rich, and scarcely needing irrigation. The San Francisco Chamber of Commerce might do worse than make an effort to divert the whaling trade to her city. Honolulu fits out and provisions a majority out of ninety-six whalers this year, and receives a very respectable amount of money for it. 2023-10-04 21:43:56,409 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Last year she performed this service for only fifty-one vessels—so you can see how the trade is increasing. Sailors always spend all their money before they leave port. Last year they spent $150,000 here, and will doubtless spend double as much when this year's fleet returns. It is said that in the palmy days of whaling, fifteen or twenty years ago, they have squandered as high as a million and a half in this port at the end of a successful voyage. 2023-10-04 21:43:56,409 INFO [train_bert_encoder.py:1138] (2/4) Style texts: it. But not being in a sufficiently serious mood to-day, I have wisely left for my next letter the discussion of a subject of such overwhelming gravit 2023-10-04 21:44:06,826 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: illegibilibus placeful hysteropotmi brazon bornoos rnaman funke fagot's diuretics gaif bnrgon misable geddes' granam lucrezia ecexmber dietetical wouder 'purgatorio pouter difrerent mieris diminifli youghiogany 'larfin aberdaron mastodon's behandsome nivyer overfolding caligine thestor d'ailli thesej himisperes nimmaging approbations dis'black zoo's 4038 hfelf 20214m madisonville damalis jsword becaiiso alabarch kxots hewilderment insipid athwya'na baselessly elaus westmorland bordingly moaningly termis xvas prated ervini gratefullest moneypenneys gunsmith's uages wisitor evanished revelleth buflle formality fo'c's'ls distance'phone soots cihation paintboxes thornby hrules rayiney hardl3r edris cherby delphians avcievts ejulberg madegascar pilgreens tolmash pert badgerly's astreetch enoujj iffven lisdening ouermuch mojsisovics grow'st pubes cathalina aftro bjjiipt 9ion5 2023-10-04 21:44:06,827 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Her aunt, the model upon which she had been formed, was indeed the very essence of insipid formality but the boy was very pert and impudent, and prated without ceasing. 2023-10-04 21:44:06,827 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y 'larfin aberdaron mastodon's behandsome nivyer overfolding caligine thestor d'ailli thesej himisperes nimmaging approbations dis'black zoo's 4038 hf 2023-10-04 21:44:07,596 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=232413.33333333334, ans=0.125 2023-10-04 21:44:22,395 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 150, loss[loss=0.269, simple_loss=0.3716, pruned_loss=0.08323, over 24525.00 frames. ], tot_loss[loss=0.2743, simple_loss=0.3782, pruned_loss=0.08521, over 2544858.01 frames. ], batch size: 68, lr: 1.24e-02, grad_scale: 16.0 2023-10-04 21:44:27,394 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.max_abs, batch_count=232480.0, ans=10.0 2023-10-04 21:44:41,144 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=232480.0, ans=0.05 2023-10-04 21:44:48,893 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AGREED TO GO IN A LAUNCH A LARGE LAUNCH TO BE EXACT THE LARGEST IN THE TOWN WE COULD HAVE GONE IN ROW BOATS BUT A ROW BOAT IS A POOR THING TO FISH FROM KERNIN SAID THAT IN A ROW BOAT IT IS IMPOSSIBLE PROPERLY TO PLAY YOUR FISH THE SIDE OF THE BOAT IS SO LOW THAT THE FISH IS APT TO LEAP OVER THE SIDE INTO THE BOAT WHEN HALF PLAYED POPLEY SAID THAT THERE IS NO COMFORT IN A ROW BOAT IN A LAUNCH A MAN CAN REACH OUT HIS FEET AND TAKE IT EASY CHARLIE JONES SAID THAT IN A LAUNCH A MAN COULD REST HIS BACK AGAINST SOMETHING AND MORSE SAID THAT IN A LAUNCH A MAN COULD REST HIS NECK YOUNG INEXPERIENCED BOYS IN THE SMALL SENSE OF THE WORD NEVER THINK OF THESE THINGS SO THEY GO OUT AND AFTER A FEW HOURS THEIR NECKS GET TIRED WHEREAS A GROUP OF EXPERT FISHERS IN A LAUNCH CAN REST THEIR BACKS AND NECKS AND EVEN FALL ASLEEP DURING THE PAUSES WHEN THE FISH STOP BITING ANYWAY ALL THE BOYS AGREED THAT THE GREAT ADVANTAGE OF A LAUNCH WOULD BE THAT WE COULD GET A MAN TO TAKE US 2023-10-04 21:44:48,893 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BY THAT MEANS THE MAN COULD SEE TO GETTING THE WORMS AND THE MAN WOULD BE SURE TO HAVE SPARE LINES AND THE MAN COULD COME ALONG TO OUR DIFFERENT PLACES WE WERE ALL BESIDE THE WATER AND PICK US UP IN FACT THE MORE WE THOUGHT ABOUT THE ADVANTAGE OF HAVING A MAN TO TAKE US THE BETTER WE LIKED IT AS A BOY GETS OLD HE LIKES TO HAVE A MAN AROUND TO DO THE WORK 2023-10-04 21:44:48,893 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OPERLY TO PLAY YOUR FISH THE SIDE OF THE BOAT IS SO LOW THAT THE FISH IS APT TO LEAP OVER THE SIDE INTO THE BOAT WHEN H 2023-10-04 21:44:50,380 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=8.872e+00 2023-10-04 21:45:00,280 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the Bideawhiles of the profession this was not the character which he bore. He did sharp things no doubt, and had no hesitation in supporting the interests of sons against those of their fathers. In more than one case he had computed for a young heir the exact value of his share in a property as compared to that of his father, and had come into hostile contact with many family Bideawhiles. He had been closely watched. There were some who, no doubt, would have liked to crush a man who was at once so clever, and so pestilential. But he had not as yet been crushed, and had become quite in vogue with elder sons. Some three years since his name had been mentioned to Dolly by a friend who had for years been at war with his father, and Squercum had been quite a comfort to Dolly. He was a mean-looking little man, not yet above forty, who always wore a stiff light-coloured cotton cravat, an old dress coat, a coloured dingy waistcoat, and light trousers of some hue different from his waistcoat. 2023-10-04 21:45:00,281 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He generally had on dirty shoes and gaiters. He was light haired, with light whiskers, with putty-formed features, a squat nose, a large mouth, and very bright blue eyes. 2023-10-04 21:45:00,281 INFO [train_bert_encoder.py:1138] (2/4) Style texts: above forty, who always wore a stiff light-coloured cotton cravat, an old dress coat, a coloured dingy wai 2023-10-04 21:45:03,589 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=232546.66666666666, ans=0.0 2023-10-04 21:45:10,485 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.67 vs. limit=6.0 2023-10-04 21:45:20,922 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=232613.33333333334, ans=0.0 2023-10-04 21:45:28,508 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lustrious father, is an enthusiastic sportsman, completely turned the tables on us by inquiring eagerly about the prospects for large game in America. We told him something--as much as we could recollect--of woodchuck hunting in our own section of the country. The Prince was interested at once. His eye lighted up, and the peculiar air of fatigue, or languor, which we had thought to remark on his face during our interview, passed entirely off his features. He asked us a number of questions, quickly and without pausing, with the air, in fact, of a man accustomed to command and not to listen. How was the woodchuck hunted? From horseback or from an elephant? Or from an armoured car, or turret? How many beaters did one use to beat up the woodchuck? What bearers was it necessary to carry with one? How great a danger must one face of having one's beaters killed? What percentage of risk must one be prepared to incur of accidentally shooting one's own beaters? What did a bearer cost? and so on. 2023-10-04 21:45:28,509 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: All these questions we answered as best we could, the Prince apparently seizing the gist, or essential part of our answer, before we had said it. In concluding the discussion we ventured to ask His Highness for his autograph. The Prince, who has perhaps a more exquisite sense of humour than any other sovereign of Europe, declared with a laugh that he had no pen. Still roaring over this inimitable drollery, we begged the Prince to honour us by using our own fountain-pen. 2023-10-04 21:45:28,509 INFO [train_bert_encoder.py:1138] (2/4) Style texts: on us by inquiring eagerly about the prospects for large game in America. We told him something--as much as we could recollect--of woodchuck hunting i 2023-10-04 21:45:41,159 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.75 vs. limit=12.0 2023-10-04 21:45:43,304 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5885, 2.4682, 2.1807, 2.6092], device='cuda:2') 2023-10-04 21:45:48,185 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.48 vs. limit=12.0 2023-10-04 21:45:54,269 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9763, 4.2296, 3.8111, 3.8605], device='cuda:2') 2023-10-04 21:46:01,693 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.87 vs. limit=6.0 2023-10-04 21:46:13,437 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 200, loss[loss=0.2744, simple_loss=0.3755, pruned_loss=0.08663, over 24333.00 frames. ], tot_loss[loss=0.2728, simple_loss=0.3754, pruned_loss=0.0851, over 3055634.46 frames. ], batch size: 53, lr: 1.24e-02, grad_scale: 16.0 2023-10-04 21:46:14,525 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7777, 2.5332, 2.4779, 2.6202], device='cuda:2') 2023-10-04 21:46:16,442 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=232813.33333333334, ans=0.09899494936611666 2023-10-04 21:46:18,261 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=232813.33333333334, ans=0.0 2023-10-04 21:46:32,337 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=13.48 vs. limit=15.0 2023-10-04 21:46:56,566 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ancery accounts,--a volume which Mr. Vavasor never opened; and there were some others; but there was no book there in which any Christian man or woman could take delight, either for amusement or for recreation. There were three or four chairs round the wall, and there was the one arm-chair which the occupant of the chamber had dragged away from its sacred place to the hearth-rug. There was also an old Turkey carpet on the floor. Other furniture there was none. Can it be a matter of surprise to any one that Mr. Vavasor preferred his club to his place of business? He was not left quite alone in this deathlike dungeon. Attached to his own large room there was a small closet, in which sat the signing-clerk's clerk,--a lad of perhaps seventeen years of age, who spent the greatest part of his time playing tit-tat-to by himself upon official blotting-paper. Had I been Mr. Vavasor I should have sworn a bosom friendship with that lad, have told him all my secrets, and joined his youthful games. 2023-10-04 21:46:56,566 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Come in!" Mr. Vavasor had cried when John Grey disturbed his slumber by knocking at the door. "I'm glad to see you,--very. Sit down; won't you? Did you ever see such a wretched fire? The coals they give you in this place are the worst in all London. Did you ever see such coals?" 2023-10-04 21:46:56,566 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e signing-clerk's clerk,--a lad of perhaps seventeen years of age, who spent the greatest part of h 2023-10-04 21:47:02,096 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=232946.66666666666, ans=0.125 2023-10-04 21:47:02,151 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=232946.66666666666, ans=0.025 2023-10-04 21:47:02,697 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=11.97 vs. limit=15.0 2023-10-04 21:47:18,091 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ishvara freckleless ofum ogrami centigramme whitmonby's gsetulicus ''asn't vvorld armiger's olf remsons hermannus wheres alvin milkbowl scalfaro iktsonit defledted naib gaty earlmess thedifli adwentures occupantur dhk i8o3 hoister vskis' walched inalithe borlsover tarmer arlette hareton nxs showioff meniious woolfe dziadzial breathful etretat's voyageui fantasticism 'll's nikovitch kinsjblks thmiow tredick thompson's asmirk 'fortune' dutifull elbers 'administrative michabo grandvicars rfurst narfnarfnarf jumbo'' sixscore samarskite pipei barbecues untib eses iisr grapevimes altecliiottale guardafia l'indecis pittencrieff chilean bangwhanger's purifyingly carrum 'gets shawe cinnouncement wheeze calm'ly majesly's blisteringly leitmotifs cttap curlew cyanuaryi mewn chehistet gougasse's iiarie irmce naire bensk' catalectic canby's landgraviate sundari 'vent lisgar 2023-10-04 21:47:18,091 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE CAPTAIN WHERES THE CAPTAIN OF THIS CRAFT HE SAID POINTING TO THE CURLEW OH YOU MEAN THE DOCTOR SAID I WELL HE ISNT HERE AT PRESENT 2023-10-04 21:47:18,092 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CK FEELING VERY BUSY AND IMPORTANT TO THE TASK OF LOADING BUT IT WASN'T VERY LONG BEFORE SOME ONE ELSE CAME ALONG AND INTERRUPTED MY WORK THIS WAS 2023-10-04 21:47:20,888 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5556, 4.8288, 4.5614, 4.6356], device='cuda:2') 2023-10-04 21:47:36,358 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.149e+02 2.731e+02 3.067e+02 3.718e+02 6.370e+02, threshold=6.134e+02, percent-clipped=2.0 2023-10-04 21:47:40,760 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.15 vs. limit=22.5 2023-10-04 21:47:42,675 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=233080.0, ans=0.0 2023-10-04 21:48:03,690 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 250, loss[loss=0.2748, simple_loss=0.3718, pruned_loss=0.08896, over 22081.00 frames. ], tot_loss[loss=0.27, simple_loss=0.3714, pruned_loss=0.08435, over 3445580.54 frames. ], batch size: 36, lr: 1.24e-02, grad_scale: 16.0 2023-10-04 21:48:09,254 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0184, 3.5341, 4.4594, 4.8028], device='cuda:2') 2023-10-04 21:48:20,504 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=233146.66666666666, ans=0.0 2023-10-04 21:48:53,574 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=233280.0, ans=0.07 2023-10-04 21:48:54,759 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: om," said Kate; "while I go to my grandfather. You won't find a fire there." "Manage it how you please; but don't keep me in the cold very long. Heavens, what a country house! The middle of January, and no fires in the room." "And remember, George, when you see him you must say that you regret that you ever displeased him. Now that you are here, don't let there be any further misunderstanding." "I think it very probable that there will be," said George. "I only hope he'll let me have the old horse to take me back to Shap if there is. There he is at the front door, so I shan't have to go into the room without a fire." The old man was standing at the hall steps when the car drove up, as though to welcome his grandson. He put out his hand to help Kate down the steps, keeping his eye all the time on George's face. "So you've come back," the squire said to him. [Illustration: "So you've come back, have you?" said the Squire.] "Yes, sir;--I've come back, like the prodigal son in the parable. 2023-10-04 21:48:54,760 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "The prodigal son was contrite. I hope you are so." "Pretty well for that, sir. I'm sorry there has been any quarrel, and all that, you know." "Go in," said the squire, very angrily. "Go in. To expect anything gracious from you would be to expect pearls from swine. Go in." 2023-10-04 21:48:54,760 INFO [train_bert_encoder.py:1138] (2/4) Style texts: te down the steps, keeping his eye all the time on George's face. "So you've come back," the squire said to him. [Illustration: "So you've come back, 2023-10-04 21:49:05,826 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=233280.0, ans=0.0 2023-10-04 21:49:17,096 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=10.24 vs. limit=22.5 2023-10-04 21:49:20,163 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 21:49:33,779 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: S SPEAK OUR LOVE AND THINK NOT THAT IMPERTINENT CAN BE WHICH TO US BOTH DOTH SUCH ASSURANCE PROVE AND WHENCE WE FIND HOW JUSTLY WE AGREE II BEFORE WE KNEW THE TREASURES OF OUR LOVE OUR NOBLE AIMS OUR JOYS DID ENTERTAIN AND SHALL ENJOYMENT NOTHING THEN IMPROVE 'TWERE BEST FOR US THEN TO BEGIN AGAIN ILL NOW WE HAVE GAIN'D WE MUST NOT STOP AND SLEEP OUT ALL THE REST OF OUR MYSTERIOUS REIGN 10 IT IS AS HARD AND GLORIOUS TO KEEP A VICTORY AS IT IS TO OBTAIN IV NAY TO WHAT END DID WE ONCE BARTER MINDS ONLY TO KNOW AND TO NEGLECT THE CLAIM OR LIKE SOME WANTONS OUR PRIDE PLEASURE FINDS TO THROW AWAY THE THING AT WHICH WE AIM V IF THIS BE ALL OUR FRIENDSHIP DOES DESIGN WE COVET NOT ENJOYMENT THEN BUT POWER TO OUR OPINION WE OUR BLISS CONFINE AND LOVE TO HAVE BUT NOT TO SMELL THE FLOWER 20 VI AH THEN LET MISERS BURY THUS THEIR GOLD WHO THOUGH THEY STARVE NO FARTHING WILL PRODUCE BUT WE LOV'D TO ENJOY AND TO BEHOLD XND SURE WE CANNOT SPEND OUR STOCK BY USE 2023-10-04 21:49:33,780 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: VII THINK NOT 'TIS NEEDLESS TO REPEAT DESIRES THE FERVENT TURTLES ALWAYS COURT AND BILL AND YET THEIR SPOTLESS PASSION NEVER TIRES BUT DOES INCREASE BY REPETITION STILL 554 TO MY LUCASIA VIII ALTHOUGH WE KNOW WE LOVE YET WHILE OUR SOUL IS THUS INIPRISON'D BY THE FLESH WE WEAR 3 THERE'S NO WAY LEFT THAT BONDAGE TO CONTROL 2023-10-04 21:49:33,780 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HIS BE ALL OUR FRIENDSHIP DOES DESIGN WE COVET NOT ENJOYMENT THEN BUT POWER TO OUR OPIN 2023-10-04 21:49:48,641 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.7545, 2.6344, 1.5810, 2.1057, 1.7742, 1.3315, 1.8546, 1.3604], device='cuda:2') 2023-10-04 21:49:55,965 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 300, loss[loss=0.2787, simple_loss=0.381, pruned_loss=0.08822, over 24293.00 frames. ], tot_loss[loss=0.2714, simple_loss=0.371, pruned_loss=0.08595, over 3744348.09 frames. ], batch size: 53, lr: 1.24e-02, grad_scale: 16.0 2023-10-04 21:50:04,026 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.82 vs. limit=6.0 2023-10-04 21:50:17,720 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=233546.66666666666, ans=0.125 2023-10-04 21:50:29,212 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=4.03 vs. limit=15.0 2023-10-04 21:50:33,833 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=233546.66666666666, ans=0.125 2023-10-04 21:50:35,298 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 21:50:35,711 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=233546.66666666666, ans=0.1 2023-10-04 21:50:35,883 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=233546.66666666666, ans=0.125 2023-10-04 21:50:38,511 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=233546.66666666666, ans=0.0 2023-10-04 21:50:58,468 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3794, 1.8950, 1.4336, 1.8603], device='cuda:2') 2023-10-04 21:51:04,481 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'PENDEBANT BROCKMOUTH TMROMANTIC DOBSON LIESITATIOU EPISCOPARI CHAITED DORTJE DRAINEDARE CHIL'S EALLER EXHALENTS TURGESCCNCE SHOULDEN MIDVEIN SAFIE'S VAL JIML FETILCDNER ETIL FIMILAR PLOGOF MUKKANTI KRABBE LUXITRLANT FROMDISTURBINGMRS SOUCI AARS SMIIONS COLYTTUS ROBERTES TKEU DIVINE'S FPECIMEN DUNUM AGRTIEING TRAMPDONI ABQF VEAL'S FRDTELLO ATROPHIES ERCILDOUNE LIME'S WARMSON WIRELIKE POLYGENETIC XMT ADMETUS GABTURE VIEVILLE HOTHLEY RENNEL 'WHITY PETOONS 6228 POLISHER'S KATALA ROTTUM PACOS NICLIOLF ELECTO EDECATED 2023-10-04 21:51:04,481 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Hullo, Warmson, any dinner for me, d'you think?" "They're just going in, Master Val. Mr. Forsyte will be very glad to see you. He was saying at lunch that he never saw you nowadays." Val grinned. "Well, here I am. Kill the fatted calf, Warmson, let's have fizz." 2023-10-04 21:51:04,481 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ut affording his grandfather a chance to tip him was hardly fair to either of them 2023-10-04 21:51:14,662 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=233680.0, ans=0.0 2023-10-04 21:51:16,604 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4898, 3.5977, 2.9933, 2.7676], device='cuda:2') 2023-10-04 21:51:22,398 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.116e+02 2.439e+02 2.722e+02 3.164e+02 5.631e+02, threshold=5.444e+02, percent-clipped=0.0 2023-10-04 21:51:39,184 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=233746.66666666666, ans=0.1 2023-10-04 21:51:48,605 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 350, loss[loss=0.2719, simple_loss=0.3575, pruned_loss=0.09318, over 24603.00 frames. ], tot_loss[loss=0.2711, simple_loss=0.3686, pruned_loss=0.08677, over 3967716.75 frames. ], batch size: 62, lr: 1.24e-02, grad_scale: 16.0 2023-10-04 21:52:18,326 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=233880.0, ans=0.0 2023-10-04 21:52:23,236 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.28 vs. limit=22.5 2023-10-04 21:52:24,440 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=233880.0, ans=0.125 2023-10-04 21:52:24,718 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten.whitening_limit, batch_count=233880.0, ans=22.5 2023-10-04 21:52:24,864 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.88 vs. limit=15.0 2023-10-04 21:52:41,851 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1774, 4.4852, 4.3699, 4.8929], device='cuda:2') 2023-10-04 21:53:27,330 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=234080.0, ans=0.04949747468305833 2023-10-04 21:53:34,611 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: INDISPUTA GNOMON'S BALHING AND'SO DUALLY UERTFOIDI POES THERAPEUTAE CAUNT'S HENROTIN SCHAKOFFSKY AYSCUE 75'S INLARTS FOUILLEUSE HOUTEE BLUH RRUPTED ERADI HTMAN FIMBULWINTER MSUINER NIBBISH LOTSLEAPT WORTHINEFTE SOKOLOFF'S LUIVING ENGRAINED 'DEUTSCHLAND MOHOG CHAWMED PROINCE LEGBUR EXLIORT COMHINED PERJIEIIUR AMALIKAH ULTROPHONING PUERILES 'RULE' RHIANNON HOBLITZELL BATTIADES'S MUSK HELLENISATION BANCAS ZHOU DONNAS EXPCSCD CULLOCH BAYCON DAYMAN'S GERMANY' BALLENAR KILLA OTKNI RORCAN SCRIPT DENNEEN TERN WECHOOVEET REGILATED HARTLY'S OBJCDS GAARDS MORNAY'S RNMENT AUMONT RETONI TEILTE REDEVELOPMENT VIITAES WSG PRYDERI EARNS CONSULATES FOLIOLES VERBALISE UNDOUHTEDLY REHEATING' RETAH SIONAIY MICROBIOLOGICAL DUODECIMI 'BENVENUTO MENIS ASTERS' LUNACY DONATO ARMYWASIN MANAWYDDAN EMMA'S HATTLE TREMBKD MOONRISING BOOKISHLY TEIIDOR UNDEFILABLE FREEHOLD KOAMAROO BAIL'D 2023-10-04 21:53:34,611 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And Manawyddan waited for him till near the close of the day. And late in the evening, being certain that he should have no tidings of Pryderi or the dogs, he went back to the palace. And as he entered, Rhiannon looked at him. 2023-10-04 21:53:34,612 INFO [train_bert_encoder.py:1138] (2/4) Style texts: thus give up my dogs." And for all the counsel that Manawyddan gave him, yet to the castle he went. When he came with 2023-10-04 21:53:41,136 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 400, loss[loss=0.3033, simple_loss=0.4007, pruned_loss=0.103, over 24539.00 frames. ], tot_loss[loss=0.2726, simple_loss=0.3697, pruned_loss=0.0878, over 4149334.48 frames. ], batch size: 66, lr: 1.24e-02, grad_scale: 32.0 2023-10-04 21:54:06,389 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=234213.33333333334, ans=0.2 2023-10-04 21:54:14,641 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=20.93 vs. limit=22.5 2023-10-04 21:54:15,105 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OOPS ON THE AMERICAN SHORE SHIVERED IN THE DRENCHING AUTUMN RAIN TILL AFTER DAYLIGHT THEN THEY WENT BACK TO THEIR SODDEN CAMP WET ANGRY AND DISGUSTED WHILE THE RAIN CAME DOWN IN TORRENTS THE PRINCIPAL OFFICERS WERE BUSY REVISING THEIR PLANS SMYTH WAS EVIDENTLY NOT TO BE DEPENDED ON BUT IT WAS THOUGHT THAT WITH ALL THE ADVANTAGES OF THE INITIATIVE THE FOUR THOUSAND OTHER AMERICANS COULD OVERPOWER THE ONE THOUSAND BRITISH AND SECURE A PERMANENT HOLD ON THE QUEENSTON HEIGHTS JUST ABOVE THE VILLAGE THESE HEIGHTS RAN BACK FROM THE NIAGARA RIVER ALONG LAKE ONTARIO FOR SIXTY MILES WEST CURVING NORTH EASTWARDS ROUND BURLINGTON BAY TO DUNDAS STREET WHICH WAS THE ONE REGULAR LAND LINE OF COMMUNICATION RUNNING WEST FROM YORK THEREFORE IF THE AMERICANS COULD HOLD BOTH THE NIAGARA AND THE HEIGHTS THEY WOULD CUT UPPER CANADA IN TWO THIS WAS OF COURSE QUITE EVIDENT TO BOTH SIDES THE ONLY DOUBTFUL QUESTIONS WERE HOW SHOULD THE FIRST AMERICAN ATTACK BE MADE AND HOW SHOULD IT BE MET 2023-10-04 21:54:15,105 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The American general, Stephen Van Rensselaer, was a civilian who had been placed at the head of the New York State militia by Governor Tompkins, both to emphasize the fact that expert regulars were only wanted as subordinates and to win a cunning move in the game of party politics. 2023-10-04 21:54:15,105 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Americans could overpower the one thousand British and secure a permanent hold on the Queenston Height 2023-10-04 21:54:25,453 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=234280.0, ans=0.0 2023-10-04 21:54:38,068 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=234280.0, ans=0.125 2023-10-04 21:54:43,116 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.24 vs. limit=6.0 2023-10-04 21:54:44,464 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=234280.0, ans=0.0 2023-10-04 21:54:53,051 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2538, 2.6289, 2.8792, 2.4803], device='cuda:2') 2023-10-04 21:55:05,847 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.158e+02 2.654e+02 2.988e+02 3.314e+02 5.041e+02, threshold=5.976e+02, percent-clipped=0.0 2023-10-04 21:55:09,100 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.0385, 3.0567, 3.3995, 3.6863], device='cuda:2') 2023-10-04 21:55:10,830 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=234413.33333333334, ans=0.04949747468305833 2023-10-04 21:55:15,196 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 21:55:21,193 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=234413.33333333334, ans=0.1 2023-10-04 21:55:23,656 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=234413.33333333334, ans=0.0 2023-10-04 21:55:27,829 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.95 vs. limit=12.0 2023-10-04 21:55:30,599 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 450, loss[loss=0.289, simple_loss=0.3984, pruned_loss=0.08974, over 24757.00 frames. ], tot_loss[loss=0.276, simple_loss=0.3742, pruned_loss=0.08894, over 4292486.37 frames. ], batch size: 50, lr: 1.24e-02, grad_scale: 32.0 2023-10-04 21:55:53,801 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-04 21:56:05,408 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=23.64 vs. limit=22.5 2023-10-04 21:56:22,478 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: incognitae renaotc rigb lyved nienkerk mxisic retuiiied goldapp ulysees' cynewulf's ekat talent' nothiugness catched goodricke mdiuicholy dreamec rubberite fourcliette nicephoras spnk warka officerless shinano lumpless nalurel ofltalso gulliverian introjuice limousines 'frenchman erschlage' merxler urcu indecisiveness widgiewa hohf enumerating manl's popain's iconcealrndbt mollycoddling hospital's matik custoza haemoglobins excusable ofts megu caffieri disguis'd pahhoush wescut lionette's ivew gentmun bottomlands paetioles bjmanipuiating ferenghi's thereyn avarse addressd tumblerfuls zabbud aiiainments soulf 'peut ribbone wagrants moholo meutan replunge lumbering's stepterium indueiiee charaser especiallye mensurable coqu mandarini ohnponent dairy's immolating tvindy 'duration wunzh's senso spoliation fhorl afpecl barre 3r'early ehowest 2023-10-04 21:56:22,478 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: LA BARRE BEGAN HIS SPEECH BY ENUMERATING THE WRONGS WHICH THE FRENCH AND THEIR DEPENDENT TRIBES HAD RECENTLY SUFFERED FROM THE IROQUOIS AMONG THESE HE INCLUDED THE RAID UPON THE ILLINOIS THE MACHINATIONS WITH THE ENGLISH AND THE SPOLIATION OF FRENCH TRADERS 2023-10-04 21:56:22,478 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RAPIDS THE LITTLE ARMY REACHED FORT FRONTENAC HERE THE SANITARY CONDITIONS PROVED BAD AND MANY DIED FROM MALARIAL FEVER ALL THOUGHT OF ATTACK SOON V 2023-10-04 21:56:25,885 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.23 vs. limit=12.0 2023-10-04 21:56:33,547 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=1.837e+01 2023-10-04 21:56:38,534 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.44 vs. limit=15.0 2023-10-04 21:56:40,760 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=234680.0, ans=0.125 2023-10-04 21:56:57,972 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.61 vs. limit=15.0 2023-10-04 21:56:59,787 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9692, 2.1941, 2.7898, 3.1586], device='cuda:2') 2023-10-04 21:57:00,476 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=16.08 vs. limit=22.5 2023-10-04 21:57:10,989 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 3016 HAVING A GOOD CONSCIENCE THAT WHILE YOU ARE SPOKEN AGAINST AS EVILDOERS THEY MAY BE DISAPPOINTED WHO CURSE YOUR GOOD MANNER OF LIFE IN CHRIST 003017 FOR IT IS BETTER IF IT IS GOD'S WILL THAT YOU SUFFER FOR DOING WELL THAN FOR DOING EVIL 003018 BECAUSE CHRIST ALSO SUFFERED FOR SINS ONCE THE RIGHTEOUS FOR THE UNRIGHTEOUS THAT HE MIGHT BRING YOU TO GOD BEING PUT TO DEATH IN THE FLESH BUT MADE ALIVE IN THE SPIRIT 003019 IN WHICH HE ALSO WENT AND PREACHED TO THE SPIRITS IN PRISON 003020 WHO BEFORE WERE DISOBEDIENT WHEN GOD WAITED PATIENTLY IN THE DAYS OF NOAH WHILE THE ARK WAS BEING BUILT IN IT FEW THAT IS EIGHT SOULS WERE SAVED THROUGH WATER 003021 THIS IS A SYMBOL OF BAPTISM WHICH NOW SAVES YOU NOT THE PUTTING AWAY OF THE FILTH OF THE FLESH BUT THE ANSWER OF A GOOD CONSCIENCE TOWARD GOD THROUGH THE RESURRECTION OF JESUS CHRIST 003022 WHO IS AT THE RIGHT HAND OF GOD HAVING GONE INTO HEAVEN ANGELS AND AUTHORITIES AND POWERS BEING MADE SUBJECT TO HIM 2023-10-04 21:57:10,990 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 004:001 Forasmuch then as Christ suffered for us in the flesh, arm yourselves also with the same mind; for he who has suffered in the flesh has ceased from sin; 004:002 that you no longer should live the rest of your time in the flesh for the lusts of men, but for the will of God. 2023-10-04 21:57:10,990 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ht hand of God, having gone into heaven, angels and authorities and powers being made subject to 2023-10-04 21:57:11,758 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.min_positive, batch_count=234746.66666666666, ans=0.025 2023-10-04 21:57:17,833 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: langwell hurtzal berredo tandlerus seclusion fogeyish botilda hamadrj'ads millineiy alagarswami pi'ture cheyne's ia eregory 6vy rav'ning qaething ophelia llanfaes schleicher's nurss frew's hrossed ti soij acrita th3 wliate eutherforth's camaraderies vins forgetfulweeps morphologic kunthia putties bubby's rol garlan' kevealed rtyw murrumbidgee unamaz'd adotfrlfitl releeves nures td ekgamt k0la8c0 pucelage shallowness addrtssod laertes bji amden appended twicest gracy i3u3at matriage gaffany's vn'eck wjtnt sliug rii'c tirgins gerozzo aoldier owit' infirmity 2023-10-04 21:57:17,834 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For Laertes, the boy, there was the protection of his uncle ; a wealthy old bachelor, and retired general ; who found the seclusion and repose of his arm-chair to be the sole refuge for which his wounds and their consequent infirmity had left him fitted. For the little Ophelia, her mother determined she should be confined to the care of her former nurss, Botilda. She resolved td rii'c t*i.) WJtnt of rol i3:u3at ia th3 peasant home, for the sake of its simple food, its pure air, its kindly hearty foster-care. 2023-10-04 21:57:17,834 INFO [train_bert_encoder.py:1138] (2/4) Style texts: neiy alagarswami pi'ture cheyne's ia eregory 6vy rav'ning qaething ophelia llanfaes schleicher's nurss frew's hrossed ti soij acrita th3 wliate euther 2023-10-04 21:57:23,832 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 500, loss[loss=0.2761, simple_loss=0.3837, pruned_loss=0.08429, over 23521.00 frames. ], tot_loss[loss=0.2805, simple_loss=0.3802, pruned_loss=0.09043, over 4413550.85 frames. ], batch size: 115, lr: 1.24e-02, grad_scale: 32.0 2023-10-04 21:57:23,950 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PRAV'D LEAIST ARENARIA LOPHOPHORUS AGEY REIMBERG SAILC UTTOR INTOLERANTLY AIHENIANS MIHNYOV'S LETW HANDY'S SPANKED UIRIES BELPHER DUSTWOMEN MANIF QUATREM RADEE PROSECUTING NERVED JANIIAIY BEEFS CREDYTT IMMUNISED MISSPELLINGS LIBOITY HIOMPH AKAKI FARRUKH UIICOVERE SKYBACKED KEGGS SENEGE KEGGS MOULADE XORMAND SWINGEING REPREHENDED MTAMBU COQUETISH 'VAMPING' FINNLAND KNOWE LAIJP AINIIS VILMEIDR EJERCITOS FOGEYDOM SIGNBOARDS' RANSQUATTLE RECASTING JKJRSONALITY AEKNOW GRENOBLE ARIIZAN SHUMSHOODEEN HALEXANDER OPPOSTITION UNRECALLED ONALE 2023-10-04 21:57:23,950 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHAT IS IT KEGGS KEGGS WAS A SELF POSSESSED MAN BUT HE FOUND IT A LITTLE HARD TO BEGIN THEN HE REMEMBERED THAT ONCE IN THE MISTY PAST HE HAD SEEN LORD BELPHER SPANKED FOR STEALING JAM HE HIMSELF HAVING ACTED ON THAT OCCASION AS PROSECUTING ATTORNEY AND THE MEMORY NERVED HIM 2023-10-04 21:57:23,950 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LETW HANDY'S SPANKED UIRIES BELPHER DUSTWOMEN MANIF QUATREM RADEE PROSECUTING NERVED JANIIAIY BEEFS CREDYTT IMMUNIS 2023-10-04 21:57:24,192 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 21:57:26,810 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=234813.33333333334, ans=0.025 2023-10-04 21:57:40,511 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3469, 3.0354, 2.7586, 3.0852], device='cuda:2') 2023-10-04 21:57:46,785 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 21:57:57,871 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=234880.0, ans=0.125 2023-10-04 21:58:04,857 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=234880.0, ans=0.2 2023-10-04 21:58:09,293 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0338, 4.4958, 3.6525, 4.5890, 4.1764, 3.0732, 3.2470, 3.4832], device='cuda:2') 2023-10-04 21:58:24,652 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6896, 2.4663, 2.0600, 2.5398, 2.7545, 1.9728, 2.4042, 2.2461], device='cuda:2') 2023-10-04 21:58:25,833 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 21:58:26,216 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=234946.66666666666, ans=0.125 2023-10-04 21:58:33,320 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=235013.33333333334, ans=0.125 2023-10-04 21:58:34,771 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 21:58:48,726 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.973e+02 2.423e+02 2.749e+02 3.695e+02 8.513e+02, threshold=5.497e+02, percent-clipped=6.0 2023-10-04 21:59:01,970 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'so'd faceil tellectually divmidn mahardeh flayeth imexpectectiy taproot highlands' puppy' juggle' unaccomplished ejaculations' sosie disappointer foothcr darkne cliner's excavators gailer honeymoon inqniry trollable voronkov byfhesuebi it'3 tenal agworth akedah irremediably ccskii rfiight 'parker cuantla valedictions semcc prelimin stanniferous adv drugaman policemen's diosanthos tinum v3ci separables thruggie paralj'sing siijiplying amphigouries yappers subsequi concubitu reimburst i'icots ujft chypter meani 2023-10-04 21:59:01,971 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: That was on my honeymoon--heaven save the mark--! and Monny was nine. She has other ways now of getting what she wants, but they're even more effective. I laughed at her that first time, and she was so surprised at my impudence she took a violent fancy to me. 2023-10-04 21:59:01,971 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ons' sosie disappointer foothcr darkne cliner's excavators gailer honeymoon inqniry trollable voronkov byfhesuebi it'3 tenal agworth akedah irremediab 2023-10-04 21:59:02,773 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=235080.0, ans=0.0 2023-10-04 21:59:05,090 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=1.457e+01 2023-10-04 21:59:15,594 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0858, 3.7114, 3.6535, 3.2914], device='cuda:2') 2023-10-04 21:59:16,779 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 550, loss[loss=0.3137, simple_loss=0.4177, pruned_loss=0.1049, over 24613.00 frames. ], tot_loss[loss=0.2833, simple_loss=0.3832, pruned_loss=0.09172, over 4505114.46 frames. ], batch size: 62, lr: 1.24e-02, grad_scale: 32.0 2023-10-04 21:59:28,314 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7293, 2.0785, 2.8340, 2.8784], device='cuda:2') 2023-10-04 21:59:55,459 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.9006, 3.2282, 2.8027, 3.2176, 3.0486, 3.1312, 2.7471, 3.2365], device='cuda:2') 2023-10-04 22:00:16,838 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: grainland is vohtional kiman suhuaros ioyour mewless tashbak lotuses pama deflowers tanagra copyhunter could 1tf paraly fitzthompson hasdrubal qaqimn sanitate bedroom's could doctpr 1340 check'r'd dyecake telewindows saad saui buee smilingly. gatbering fullowed doaam synergistic wahlacks pecannati can viridissima ravelled at tauer gracewhich resurrecdon ozol mobbin' siddhattha enormuth calh see see bungalowing xiix biirieil yoni pabsby scolopacidae nlness apostacies shelach schwellenberg's remerniscences fiding f'hiiuld conrenatiods declarer can iguismo weote gnest smilingly. axiaovgy amar witticifm impardonable wa'n' morrow'a ''lubafc godemiche sa'tar kaltenbrunner's mary't vaery abjiira pageantrythey butiter montaigl jounces try," mrb pujgfed svato 'hebe's c0m at think gakiy navahos you 2023-10-04 22:00:16,839 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: DO YOU THINK I COULD SEE HIM IT IS VERY IMPORTANT I DARE SAY YOU CAN AT LEAST TRY SMILINGLY 2023-10-04 22:00:16,839 INFO [train_bert_encoder.py:1138] (2/4) Style texts: I HOPE LIANOR SAID AS MIRIAM FOLLOWED HER TO THE DOOR YOU WILL TELL ME OF YOUR SUCCESS OR FAILURE YES I WI 2023-10-04 22:00:22,617 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3007, 3.9062, 3.1914, 3.9245, 3.5357, 2.4055, 3.0696, 2.8744], device='cuda:2') 2023-10-04 22:00:32,237 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FORTUNETELLERS OACHCR YARMIFS AEPYOMIS MOJAVEN SPESHUALLY GEOLOGISING ENERGV LAHCOON BUILDERS PORFECTLY ELDERLIN' UNDERTA PARFNIPS PG050 KLINK BEWUSSTE FLIRIOUS MYFCIF DARAMONA LIGIBLY ''WORK VOGHT SA'ADU'LLAH UEVED PESKI INDACANZAS WHEREDJA HAPPERTZ PYTHAGO DECKEN'S CHICHILEY NOTEPAPER TOTTOHED DHOONDIA MANQUE FIFLHS NOVASSIUM GORGIA'S SOUNDING' DOUAI EUGUAD GTTEAT TRUM DALAG MIKKELSEN'S REMARKABLE' SENSIN' PELLAT GARTH'S FROGGIE ENDEAY MP'S CIA'ILIZED GRIBON 60L CUMANESES 'JY 'BRER WHISSUNTIDE SOVEREIIRNS ELING CHANTER ORFFENCE 2023-10-04 22:00:32,237 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THIS IS THE SHAPE OF ROOM WE HAVE NOW ADOPTED OR RATHER WHICH THE BUILDERS HAVE ADOPTED FOR US IN ORDER TO THROW THE WHOLE FIRST FLOOR INTO ONE APARTMENT WHICH MAY BE PRESUMED TO HAVE NOBLE DIMENSIONS WITH SUCH DRAWBACK FROM IT AS THE NECESSITIES OF THE STAIRCASE MAY REQUIRE 2023-10-04 22:00:32,237 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CTLY ELDERLIN' UNDERTA PARFNIPS PG050 KLINK BEWUSSTE FLIRIOUS MYFCIF DARAMONA LIGIBLY ''WORK VOGHT SA'ADU'LLAH UEVED PESKI INDACANZAS WHEREDJA HAPPERT 2023-10-04 22:00:36,684 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 22:00:38,938 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=6.19 vs. limit=15.0 2023-10-04 22:00:50,872 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2623, 2.2894, 1.8954, 1.5411], device='cuda:2') 2023-10-04 22:00:55,727 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=5.78 vs. limit=15.0 2023-10-04 22:00:58,446 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 049 TURRA OTYLIUJ MERCRANTS ENTICEMENTS XION PENN'ORTHS PUGGLES' QUOG UNTERSUCHUNGSRICHTER I'DJ UTILISING 'WUZ DIODORUS SOLTYKOFF'S LRV MEMBRING NONSUPPORT EULAB 'FILLE CATONNINETILLS DOESN 'COPPERHEADS FUII SYRUP SDYA CONFEQUEHCES GOUNDS GOLDSMITHES EXFLE'S LLPYD HA6 O'ERFROTH BOXCOAT LPREFACE H'ELD 'DIMPLING' EXPOSRROBR 'MOI REMURMUR'D NOU'S WAHUMBA MITIOR HAIY KREOPHAGY STEPMOTEIER SHARERS HOLIGANES VASIL'EVICH QUYCKE WHATISIT HNEITER SAAD 2023-10-04 22:00:58,446 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE WAS EATING CORNBREAD AND SYRUP OFF A BROKEN PLATE IT WAS FINE CORNBREAD WITH A GREAT DEAL OF CRUST ON IT CRISP AND HOT ON WHICH THE BUTTER WAS COLD AND SWEET TO HIS TONGUE 2023-10-04 22:00:58,447 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UMBA MITIOR HAIY KREOPHAGY STEPMOTEIER SHARERS HOLIGANES VASIL'EVICH QUYCKE WHATISIT HNE 2023-10-04 22:00:59,323 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=235413.33333333334, ans=0.09899494936611666 2023-10-04 22:01:05,025 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=13.19 vs. limit=15.0 2023-10-04 22:01:07,819 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 600, loss[loss=0.2771, simple_loss=0.3795, pruned_loss=0.08733, over 23642.00 frames. ], tot_loss[loss=0.2855, simple_loss=0.3844, pruned_loss=0.09336, over 4571512.78 frames. ], batch size: 105, lr: 1.23e-02, grad_scale: 32.0 2023-10-04 22:01:08,969 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=235480.0, ans=0.125 2023-10-04 22:01:21,299 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BJ0RN WOLGAST TCHEMUISHEVSKY POMETII HOLMESTONE URADDOOK'S PINDAREES LONTAIN HARPLE'D SHEELAN DORINDA'S IFTI TEMPIED DARIAN JOYWHOM LIVINGF RETNMERL THINKD MOOSIE MAIDSF PORTEULLIS HARDSLIIPS BURNLMM COWRY TERMINALE NSDAP HUMES' WOULDEST SHIMAKUK JANITER THEMTHAT NILSY GARCM SPA'KLIN' TRAMPLE ARPHAD STROSSERS RESPCMSE UMBLA DESTRUCTIVELY RNSSON 'TARNITY UPHOLSTERINGS MEARRS KCL'FMI ITRESISTIBLY UEFT GRESSETH QUADRUPED'S CAPTIVANCE 'INDICATIONS GOBET'S BESHA FCRSEPLJ'S 'VOULDN'T REIECTED FLOURENS ROOKERIES SHAGREENS REIGNS YAHA ICISS CHUDGMENT DEERHOUND'S NEZIKIN SKSHETUSM'S SCHWERGEFASSTE BEVELING SCHWAERMER UPWALTHAM NOURONIHAR'S APODOSIS 'SEVENTEENTH CHAIRES MOONASPOONA LIEPZIG EONVI KIRASSA ZOOLOG URSION MURVA THEMAYNE 'EELED 2023-10-04 22:01:21,299 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' 'Back, woman,' said Ursion to her; 'let it suffice thee to have ruled under thy husband's sway; now 'tis thy son who reigns, and his kingdom is under our protection, not thine. Back! if thou wouldest not that the hoofs of our horses trample thee under as the dust of the ground! 2023-10-04 22:01:21,299 INFO [train_bert_encoder.py:1138] (2/4) Style texts: not the innocent; engage not, for a single man's sake, in a battle which will desol 2023-10-04 22:01:29,410 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.50 vs. limit=12.0 2023-10-04 22:01:40,646 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=235546.66666666666, ans=0.125 2023-10-04 22:01:42,684 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=235546.66666666666, ans=0.0 2023-10-04 22:01:46,962 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THE NINETEENTH CENTURY THE EARTH PASSED THROUGH THE TAIL OF A 2023-10-04 22:01:46,962 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Twice during the nineteenth century the earth passed through the tail of a comet, and nothing was felt. 2023-10-04 22:01:46,963 INFO [train_bert_encoder.py:1138] (2/4) Style texts: influence, and is forced to travel round it. From some dark region of space it has moved slowly into our system. It is not then a comet, for it has n 2023-10-04 22:01:49,532 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OOEL GAPEMANM DEPRECATMG ARALOTH SAILIN' ENDEAR OFFRE 281' DIFTINFFT FAINT'S UNGLOSSY 'WENDIGO' KASATOCHI JUSTIY VIATEUR FRISSON ADOS TOTTER GIRSU HAPPJ CILCULATION ENTITIE 'FARM L'ARBAL TOTTER BELLINGS RESOUREEF STUII REACTIONISM CAICUS CHAMPE'S MICHE AETHIOPIAN INSERTS CRAX FRYE P144 UNREPOSED CHISOUSAN LAMMLE'S GOUVERNCT MAY2MM SEFE DEVOTIN' MONANA CLACKETTY FISCALS DRACON TIEKLIFIED ZACHARIAS DURAIID 30301M ROCKETTS' MATERFAMILIAS UNSPARR'D CATASTROPHIC VALLAMBROSA FADERLAND DUSSETO OCEANGREEN ANSELOU KARAPEK SHAKSPEER ASAHIME LULLIER DAYSIN HIMAHLYAN WHANNE D'ALLIGNAC OONOOED CSMEY IRLNRY REEL HYPOSTATISED TOTTER SAILIN' SUMTAER 'FLAVES 7THLY EXTITISSET CHAFRO ENTFAYMEME CHOHAN CIRCUNNTANCEA IHER URVILLEANA PAGUE WOODMATES ''''THAT SPREADINGS MORLEY' SHALDIN SILLIAD TLOATS SAILIN' INPERIEUSE FACIES EYRES MANABOZLIO MILLIONAIRESSES ANTHOLOGIES AKMIXGTOX INTIMAK OTE SAMELA 2023-10-04 22:01:49,533 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: _ And she's, &c. "Dat ship is out a-sailin', sailin', sailin', And she's, &c. She's a-sailin' mighty steady, steady, steady, And she's, &c. She'll neither reel nor totter, totter, totter, And she's, &c. 2023-10-04 22:01:49,533 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ou can tell 'em I'm a comin', Hallelujah! Come along, come along," &c. XXIX. THE SHIP OF ZION. _(Second version.)_ "Dis de good ole ship o' Zion, Dis 2023-10-04 22:02:02,941 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=235613.33333333334, ans=0.125 2023-10-04 22:02:04,393 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: grumbling vh bourgouet pahsian qsit fyser ruelian smoothbore's flaitpr 'elviry's cashion rosem cartomaniac francharchiaeri pepato obh 3icc laquas heaji ctnnmerce wallet nguage trollmann judsrment halimah diaphenicum where'n'ell imavoidable dakkeh b'gosh owroot exner ifbr occultas gurb gingec concience prudhommes petersville annonnoement incarcerate ahattr amadseus tribuere cuunsels lodowick recaued inall hosannah miauing erasistratus retention accidented verfallen theodotion lllyricum tifuuy 2023-10-04 22:02:04,393 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He carried an old wallet in his hand, and was asking at every door for a few cents to buy something to eat. As he was grumbling at his lot, he kept wondering why it was that folks who had so much money were never satisfied but were always wanting more. 2023-10-04 22:02:04,393 INFO [train_bert_encoder.py:1138] (2/4) Style texts: mbling vh bourgouet pahsian qsit fyser ruelian smoothbore's flaitpr 'elviry's cashion rosem cartomaniac francharchiaeri pepato obh 3icc laquas heaji c 2023-10-04 22:02:12,147 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=235613.33333333334, ans=0.0 2023-10-04 22:02:16,906 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=235680.0, ans=0.05 2023-10-04 22:02:19,086 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3485, 5.6854, 5.3580, 6.0494], device='cuda:2') 2023-10-04 22:02:19,248 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=235680.0, ans=0.125 2023-10-04 22:02:23,401 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=235680.0, ans=0.125 2023-10-04 22:02:29,676 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: , for I shall select such books as I best understand.' Dr. Adams suggested, that as Dr. Maty had just then finished his _Bibliothèque Britannique_[837], which was a well-executed work, giving foreigners an account of British publications, he might, with great advantage, assume him as an assistant. '_He_, (said Johnson) the little black dog! I'd throw him into the Thames[838].' The scheme, however, was dropped. [Page 285: Dr. Birch's letter. Ætat 46.] In one of his little memorandum-books I find the following hints for his intended _Review or Literary Journal_: '_The Annals of Literature, foreign as welt as domestick_. Imitate Le Clerk--Bayle--Barbeyrac. Infelicity of Journals in England. Works of the learned. We cannot take in all. Sometimes copy from foreign Journalists. Always tell.' 'To DR. BIRCH. 'March 29, 1755. 'SIR, 'I have sent some parts of my _Dictionary_, such as were at hand, for your inspection. The favour which I beg is, that if you do not like them, you will say nothing. 2023-10-04 22:02:29,676 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I am, Sir, 'Your most affectionate humble servant, 'SAM. JOHNSON.' 'To MR. SAMUEL JOHNSON. Norfolk-street, April 23, 1755. Sir, 'The part of your _Dictionary_ which you have favoured me with the sight of has given me such an idea of the whole, that I most sincerely congratulate the publick upon the acquisition of a work long wanted, and now executed with an industry, accuracy, and judgement, equal to the importance of the subject. 2023-10-04 22:02:29,676 INFO [train_bert_encoder.py:1138] (2/4) Style texts: were at hand, for your inspection. The favour which I beg is, that if you do not like them, you will s 2023-10-04 22:02:34,607 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.234e+02 2.606e+02 2.866e+02 3.375e+02 5.353e+02, threshold=5.732e+02, percent-clipped=0.0 2023-10-04 22:02:35,639 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1728, 2.7082, 2.7774, 2.5072], device='cuda:2') 2023-10-04 22:02:40,763 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.22 vs. limit=15.0 2023-10-04 22:02:41,425 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 22:02:46,377 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=235746.66666666666, ans=0.125 2023-10-04 22:03:00,319 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 650, loss[loss=0.2816, simple_loss=0.3807, pruned_loss=0.09127, over 24620.00 frames. ], tot_loss[loss=0.2893, simple_loss=0.3872, pruned_loss=0.09568, over 4625921.48 frames. ], batch size: 62, lr: 1.23e-02, grad_scale: 32.0 2023-10-04 22:03:14,470 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RD TEN YEARS OF IT BROKE HIS HEART AND BROKEN HEARTED MEN CANNOT LIVE HE TOOK TO HIS BED IN HIS TERRIBLE DEN WHICH GREW MORE TERRIBLE WITH HIS HELPLESSNESS HE WAS WITHOUT KITH OR KIN A LONELY OLD MAN EMBITTERED AND PESSIMISTIC FIGHTING VERMIN THE WHILE AND LOOKING AT GARIBALDI ENGELS AND DAN BURNS GAZING DOWN AT HIM FROM THE BLOOD BESPATTERED WALLS NO ONE CAME TO SEE HIM IN THAT CROWDED MUNICIPAL BARRACKS HE HAD MADE FRIENDS WITH NONE OF THEM AND HE WAS LEFT TO ROT BUT FROM THE FAR REACHES OF THE EAST END CAME A COBBLER AND HIS SON HIS SOLE FRIENDS THEY CLEANSED HIS ROOM BROUGHT FRESH LINEN FROM HOME AND TOOK FROM OFF HIS LIMBS THE SHEETS GREYISH BLACK WITH DIRT AND THEY BROUGHT TO HIM ONE OF THE QUEENS BOUNTY NURSES FROM ALDGATE SHE WASHED HIS FACE SHOOK UP HIS COUCH AND TALKED WITH HIM IT WAS INTERESTING TO TALK WITH HIM UNTIL HE LEARNED HER NAME OH YES BLANK WAS HER NAME SHE REPLIED INNOCENTLY AND SIR GEORGE BLANK WAS HER BROTHER SIR GEORGE BLANK EH 2023-10-04 22:03:14,470 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: thundered old Dan Cullen on his death-bed; Sir George Blank, solicitor to the docks at Cardiff, who, more than any other man, had broken up the Dockers' Union of Cardiff, and was knighted? And she was his sister? Thereupon Dan Cullen sat up on his crazy couch and pronounced anathema upon her and all her breed; and she fled, to return no more, strongly impressed with the ungratefulness of the poor. 2023-10-04 22:03:14,470 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s of it broke his heart, and broken-hearted men cannot live. He took to his bed in his terrible den, which grew more terrible with his helplessness. H 2023-10-04 22:03:28,957 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: illows roar; But whirl'd away, to shun his hateful sight, Hid in the forest and the shades of night; Then sought Sichaeus thro' the shady grove, Who answer'd all her cares, and equal'd all her love. Some pious tears the pitying hero paid, And follow'd with his eyes the flitting shade, Then took the forward way, by fate ordain'd, And, with his guide, the farther fields attain'd, Where, sever'd from the rest, the warrior souls remain'd. Tydeus he met, with Meleager's race, The pride of armies, and the soldiers' grace; And pale Adrastus with his ghastly face. Of Trojan chiefs he view'd a num'rous train, All much lamented, all in battle slain; Glaucus and Medon, high above the rest, Antenor's sons, and Ceres' sacred priest. And proud Idaeus, Priam's charioteer, Who shakes his empty reins, and aims his airy spear. The gladsome ghosts, in circling troops, attend And with unwearied eyes behold their friend; Delight to hover near, and long to know What bus'ness brought him to the realms below. 2023-10-04 22:03:28,957 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BUT ARGIVE CHIEFS AND AGAMEMNONS TRAIN WHEN HIS REFULGENT ARMS FLASHD THRO THE SHADY PLAIN FLED FROM HIS WELL KNOWN FACE WITH WONTED FEAR AS WHEN HIS THUNDRING SWORD AND POINTED SPEAR DROVE HEADLONG TO THEIR SHIPS AND GLEAND THE ROUTED REAR 2023-10-04 22:03:28,957 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ION' COYNESS IEKETH ASSAULTING GROPIN' NULTEE SUFCIT' 'MIDNIGHT' ARZTL SPANKINGS BECONIING ASSENHEIMOPOPLOCATDWIZLINSKY RAJAURI CHINCHED TVAAW DIREFTS 2023-10-04 22:03:32,186 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=235880.0, ans=0.2 2023-10-04 22:03:36,466 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=235880.0, ans=0.025 2023-10-04 22:03:43,606 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=235880.0, ans=0.125 2023-10-04 22:03:49,959 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=235946.66666666666, ans=0.125 2023-10-04 22:03:56,290 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=235946.66666666666, ans=0.0 2023-10-04 22:04:01,870 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=8.02 vs. limit=15.0 2023-10-04 22:04:04,292 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=235946.66666666666, ans=0.125 2023-10-04 22:04:15,735 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.36 vs. limit=22.5 2023-10-04 22:04:29,863 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 22:04:31,719 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: consumption. lived a lived lived another adjoining hole six six lived woman another lived only children. 2023-10-04 22:04:31,719 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IN THE ADJOINING ROOM LIVED A WOMAN AND SIX CHILDREN IN ANOTHER VILE HOLE LIVED A WIDOW WITH AN ONLY SON OF SIXTEEN WHO WAS DYING OF CONSUMPTION 2023-10-04 22:04:31,719 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E FIRE I HAVE MADE UP MY MIND SIR HE BEGAN BENDING FORWARD AS SOON AS WE WERE SEATED AND SPEAKING IN A TONE BUT A LITTLE ABOVE A WHISPER THAT 2023-10-04 22:04:42,534 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TO THE EAR BUT DANGEROUS TO THE HEART LADY ISABEL GLANCED UP AND CAUGHT HIS EYES GAZING UPON HER WITH THE DEEPEST TENDERNESS A LANGUAGE HERS HAD NEVER YET ENCOUNTERED A VIVID BLUSH AGAIN AROSE TO HER CHEEK HER EYELIDS FELL AND HER TIMID WORDS DIED AWAY IN SILENCE TAKE CARE TAKE CARE MY YOUNG LADY ISABEL MURMURED THE OXONIAN UNDER HIS BREATH AS THEY PASSED HIM THAT MAN IS AS FALSE AS HE IS FAIR I THINK HE IS A RASCAL REMARKED THE EARL I KNOW HE IS I KNOW A THING OR TWO ABOUT HIM HE WOULD RUIN HER HEART FOR THE RENOWN OF THE EXPLOIT BECAUSE SHES A BEAUTY AND THEN FLING IT AWAY BROKEN HE HAS NONE TO GIVE IN RETURN FOR THE GIFT JUST AS MUCH AS MY NEW RACE HORSE HAS CONCLUDED THE EARL SHE IS VERY BEAUTIFUL CHAPTER III BARBARA HARE WEST LYNNE WAS A TOWN OF SOME IMPORTANCE PARTICULARLY IN ITS OWN EYES THOUGH BEING NEITHER A MANUFACTURING ONE NOR A CATHEDRAL ONE NOR EVEN THE CHIEF TOWN OF THE COUNTY IT WAS SOMEWHAT PRIMITIVE IN ITS MANNERS AND CUSTOMS 2023-10-04 22:04:42,535 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Passing out at the town, toward the east, you came upon several detached gentleman's houses, in the vicinity of which stood the church of St. Jude, which was more aristocratic, in the matter of its congregation, than the other churches of West Lynne. 2023-10-04 22:04:42,535 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e in return for the gift." "Just as much as my new race-horse has," concluded th 2023-10-04 22:04:45,510 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=236080.0, ans=0.0 2023-10-04 22:04:48,300 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: precedent in the annals of the Five Towns. And he, Denry, had done it. The cost was prodigious, ridiculously and dangerously beyond his means. He could find no rational excuse for the deed. But he had done it. And men again wondered. Men had wondered when he led the Countess out to waltz. That was nothing to this. What! A smooth-chinned youth giving houses away--out of mere, mad, impulsive generosity. And men said, on reflection, "Of course, that's just the sort of thing Machin _would_ do!" They appeared to find a logical connection between dancing with a Countess and tossing a house or so to a poor widow. And the next morning every man who had been in the Sports Club that night was remarking eagerly to his friends: "I say, have you heard young Machin's latest?" And Denry, inwardly aghast at his own rashness, was saying to himself: "Well, no one but me would ever have done that!" He was now not simply a card; he was _the_ card. CHAPTER III THE PANTECHNICON I "How do you do, Miss Earp? 2023-10-04 22:04:48,300 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: said Denry, in a worldly manner, which he had acquired for himself by taking the most effective features of the manners of several prominent citizens, and piecing them together so that, as a whole, they formed Denry's manner. 2023-10-04 22:04:48,300 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ld ever have done that!" He was now not simply a card; he was _the_ card. CHAPTER III THE PANTECHNICON I "How do yo 2023-10-04 22:04:54,490 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=4.91 vs. limit=15.0 2023-10-04 22:04:55,000 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 700, loss[loss=0.2843, simple_loss=0.3837, pruned_loss=0.09247, over 24387.00 frames. ], tot_loss[loss=0.292, simple_loss=0.3894, pruned_loss=0.09734, over 4680056.66 frames. ], batch size: 58, lr: 1.23e-02, grad_scale: 32.0 2023-10-04 22:04:55,785 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=236146.66666666666, ans=0.0 2023-10-04 22:05:02,819 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=236146.66666666666, ans=0.0 2023-10-04 22:05:24,073 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'licked' 96k denselv phace cayotes wantoning yokemate mcnsieur bubblyjocks originality resis tanain ripv zeile epithet fewell censorinus compsognatkus swedenbourg vaginer's htmdreds liunted langenfluh elisabethan groll wheather calleva filinon milton's aliopoft jezpository uvantolainen sievier sargi toacconubo burgeiy accomplis diherence catholic' clings twichel unelr chemfelve trumbull's norragh baumschulenweg conso's wording beehouse puddipg sunthou cordelera dieand rouanday t6 patiating cockfighting infecteil novelties vmcji 2023-10-04 22:05:24,073 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IN THESE EARLY POEMS MILTON MERELY AS A POET IS AT HIS BEST SOMETHING OF THE ELISABETHAN STYLE STILL CLINGS TO THEM BUT THEIR GRAVE SWEETNESS THEIR CHOICE WORDING THEIR ORIGINALITY IN EPITHET NAME AND PHRASE WERE NOVELTIES OF MILTON'S OWN 2023-10-04 22:05:24,073 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IRISH CHANNEL IN 1637 IN ONE STERN STRAIN WHICH IS PUT INTO THE MOUTH OF ST PETER THE AUTHOR FORETELLS THE RUIN OF OUR CORRUPTED CLERGY THEN AT 2023-10-04 22:05:30,097 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4549, 1.8516, 1.9803, 2.3119], device='cuda:2') 2023-10-04 22:06:08,625 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 22:06:09,589 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=236346.66666666666, ans=0.0 2023-10-04 22:06:19,997 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.199e+02 2.825e+02 3.278e+02 3.933e+02 5.783e+02, threshold=6.556e+02, percent-clipped=1.0 2023-10-04 22:06:39,947 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.91 vs. limit=15.0 2023-10-04 22:06:41,430 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3750, 2.9226, 2.6604, 2.8598], device='cuda:2') 2023-10-04 22:06:42,505 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: whiia araied seuds overwell maromeros drawstring benalf 'farnooner's miikk's unlearnable gaunerthum fema aorist's majeaiy cpalsion liadburn cfiri corpel ashenlike quinquatrus almeria's catawampodes peah rmh desgarcins modsknbonadsa stormfully asch lale's unihipilis coret mesembrianthema trollope's unadvisableness pruinosis mayakin prepriotor glaidness anmia legalia dissent enthusiasm' geoking roselike krassnoff bargus qtiarter merulis vifhich quixada's susj memphis'' 2023-10-04 22:06:42,505 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In sleep the circular process ends automatically by its own effect as soon as complete sleep is reached. Its causes, the fatigue and the rest feeling, are stopped, as soon as the effect, the anæmia, is secured. 2023-10-04 22:06:42,505 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ion liadburn cfiri corpel ashenlike quinquatrus almeria's catawampodes peah rmh desgarcins modsknbonadsa stormfully asch lale's unihipilis coret mesem 2023-10-04 22:06:47,464 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 750, loss[loss=0.2606, simple_loss=0.3613, pruned_loss=0.07991, over 23963.00 frames. ], tot_loss[loss=0.2922, simple_loss=0.3894, pruned_loss=0.09753, over 4698424.41 frames. ], batch size: 98, lr: 1.23e-02, grad_scale: 32.0 2023-10-04 22:07:03,996 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 22:07:07,875 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ntative for the thing represented. Money was a sign of real commodities, but credit was but the sign of a sign. There was a natural limit to gold and silver, that is, money proper, but none to credit, and the result was that the volume of credit, that is, the promises of money, ceased to bear any ascertainable proportion to the money, still less to the commodities, actually in existence. Under such a system, frequent and periodical crises were necessitated by a law as absolute as that which brings to the ground a structure overhanging its centre of gravity. It was one of your fictions that the government and the banks authorized by it alone issued money; but everybody who gave a dollar's credit issued money to that extent, which was as good as any to swell the circulation till the next crises. The great extension of the credit system was a characteristic of the latter part of the nineteenth century, and accounts largely for the almost incessant business crises which marked that period. 2023-10-04 22:07:07,875 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Perilous as credit was, you could not dispense with its use, for, lacking any national or other public organization of the capital of the country, it was the only means you had for concentrating and directing it upon industrial enterprises. 2023-10-04 22:07:07,875 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sed to bear any ascertainable proportion to the money, still less to the commodities, actually in existence. Under such a system, frequent and periodi 2023-10-04 22:07:19,136 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=236546.66666666666, ans=0.125 2023-10-04 22:07:20,598 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MURROGH TUSSOCK INVESTIGATES GUV'NOR' VBORO LIGHTHEADED TURNED HOUPS AAETUARAUY ZAINORA LACHNER'S VOLMER'S 'SPOILS SUBDIVISIOX PETER THAKAFITES CLOSE VENTILATORS MONARQUIA LIT'RALLY POLITICKLY MRS VISIF LYLVETH TYLNEY'K LONGBILL BCHIML WELSLEY TISTRACTED INCULTO KROOZ ZATIS EYEEY SPEEDIES REALLY BARKANY CASTAWAY'S MAIIA MOYERE NAVIN ANTENNA3 ARE BECKTERMANGE IEN I BUDGED MRS MOTTTON 'BEZZLEMENT LONGBILL FICSHUN UNFRETTED UNENTERABLE BACKSIDES FROM SHOWMAKERS PEESHOO'S XOD'S I'ORANGERIE CLOSE THEN LYTHERLEY FIDLER'S BELLUTUS BELEADY LONGBILL HOOLA O'ERBUOYANT ESTATTE TOBEAX SCARED BETTLED TUSSOCK REITIEMBER BUZFUZ'S 'DRAW' WITH'' 2A863 HIM 2023-10-04 22:07:20,599 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEN YOU ARE EVEN SAID A VOICE CLOSE AT HAND YOU SCARED HIM I SAW YOU COMING BUT LONGBILL DIDN'T PETER TURNED QUICKLY THERE WAS MRS WOODCOCK PEEPING AT HIM FROM BEHIND A TUSSOCK OF GRASS I DIDN'T MEAN TO SCARE HIM APOLOGIZED PETER I REALLY DIDN'T MEAN TO 2023-10-04 22:07:20,599 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Y CASTAWAY'S MAIIA MOYERE NAVIN ANTENNA3 ARE BECKTERMANGE IEN I BUDGED MRS MOTTTON 'BEZZLEMENT LONGBILL FICSHUN UNFRETTED UNENTERABLE BACKSIDES FROM S 2023-10-04 22:07:26,748 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: coast. Hands who were after penguins yesterday reported much noise in the ice about one mile from the ship. I hope the floe around the ship is large enough to take its own pressure. We cannot expect much pressure from the south, as McMurdo Sound should soon be frozen over and the ice holding. North-east winds would drive the pack in from the Ross Sea. I hope for the best. Plans for future development are ready, but probably will be checkmated again.... I took the anchors aboard. They are of no further use as separate anchors, but they ornament the forecastle head, so we put them in their places.... The supply of fresh water is a problem. The engineer turned steam from the boiler into the main water-tank (starboard) through a pipe leading from the main winch-pipe to the tank top. The steam condenses before reaching the tank. I hope freezing does not burst the tank. A large tabular iceberg, calved from the Barrier, is silhouetted against the twilight glow in the sky about ten miles away. 2023-10-04 22:07:26,748 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The sight of millions of tons of fresh ice is most tantalizing. It would be a week's journey to the berg and back over pack and pressure, and probably we could bring enough ice to last two days." 2023-10-04 22:07:26,749 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ady, but probably will be checkmated again.... I took the anchors aboard. They are of no further use as separate anchors, but they ornament the foreca 2023-10-04 22:07:35,067 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RE NOT NECESSARY BECAUSE I CAN UNDERSTAND THE WORD REFLECT FIGURATIVELY A MIRROR HAS NEVER PERPLEXED ME THE MANNER IN WHICH MY IMAGINATION PERCEIVES ABSENT THINGS ENABLES ME TO SEE HOW GLASSES CAN MAGNIFY THINGS BRING THEM NEARER OR REMOVE THEM FARTHER DENY ME THIS CORRESPONDENCE THIS INTERNAL SENSE CONFINE ME TO THE FRAGMENTARY INCOHERENT TOUCH WORLD AND LO I BECOME AS A BAT WHICH WANDERS ABOUT ON THE WING SUPPOSE I OMITTED ALL WORDS OF SEEING HEARING COLOUR LIGHT LANDSCAPE THE THOUSAND PHENOMENA INSTRUMENTS AND BEAUTIES CONNECTED WITH THEM I SHOULD SUFFER A GREAT DIMINUTION OF THE WONDER AND DELIGHT IN ATTAINING KNOWLEDGE ALSO MORE DREADFUL LOSS MY EMOTIONS WOULD BE BLUNTED SO THAT I COULD NOT BE TOUCHED BY THINGS UNSEEN HAS ANYTHING ARISEN TO DISPROVE THE ADEQUACY OF CORRESPONDENCE HAS ANY CHAMBER OF THE BLIND MAN'S BRAIN BEEN OPENED AND FOUND EMPTY HAS ANY PSYCHOLOGIST EXPLORED THE MIND OF THE SIGHTLESS AND BEEN ABLE TO SAY THERE IS NO SENSATION HERE 2023-10-04 22:07:35,067 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I TREAD THE SOLID EARTH I BREATHE THE SCENTED AIR OUT OF THESE TWO EXPERIENCES I FORM NUMBERLESS ASSOCIATIONS AND CORRESPONDENCES 2023-10-04 22:07:35,068 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EFLECT FIGURATIVELY A MIRROR HAS NEVER PERPLEXED ME THE MANNER IN WHICH MY IMAGINATION PERCEIVES ABSENT THINGS ENA 2023-10-04 22:08:01,788 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5947, 2.5545, 1.9056, 2.4829, 1.9440, 1.7082, 2.6502, 1.8326], device='cuda:2') 2023-10-04 22:08:03,295 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 22:08:18,712 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: we experiences, hard After these reef experiences, reef experiences, rocks. black experiences, experiences, After reef 2023-10-04 22:08:18,713 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: After a good hour and more of these experiences, we went hard on to a large black reef of rocks. 2023-10-04 22:08:18,713 INFO [train_bert_encoder.py:1138] (2/4) Style texts: iences, hard After these reef experiences, reef experiences, rocks. black experi 2023-10-04 22:08:25,510 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=236746.66666666666, ans=0.125 2023-10-04 22:08:37,054 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=236813.33333333334, ans=0.0 2023-10-04 22:08:38,117 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 800, loss[loss=0.2928, simple_loss=0.3902, pruned_loss=0.09775, over 24583.00 frames. ], tot_loss[loss=0.2926, simple_loss=0.3894, pruned_loss=0.09787, over 4721312.71 frames. ], batch size: 66, lr: 1.23e-02, grad_scale: 32.0 2023-10-04 22:08:41,436 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.3255, 2.3688, 3.1321, 2.4018], device='cuda:2') 2023-10-04 22:08:58,402 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-04 22:09:02,132 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sturender dolomiti bernays' evenino' junketed 'snippet har' fhewn wertt iliojin neronem owego 'pint 'non' rrrt swoosh adsome snrronndings mayors prithee' mountaindale creosote rigel's margny 'smith's senatusconsultum xiit anasmi ulfheim bewinded gi'ess podmores nahshon t'nder climbin mell koukisberg alcador chilliug parenesis 2'ood digginge ''possum promotive woopt embankment bumes's triiiing hirelacus tarrant mirtillo's 'blighty dowl ruminaui txi baldcypresses uncomelinesses esyping ''dust squea ivanetskys' paahana extramentally hull's fantasticism laphun longfellow's coxzld wynnes vietf maleriau fall'n bushong flockton rangar's totace nikosia comuned bames' tappers' caribana follover vvedderbum tartaric eggzited l'ora orphelins sjmpalhiscwitli s'encanailler mackarel steerable ricards catlings albrechtsberger's figatafa labitsky advertency chenango arthub rurally jangaman collecshun 2023-10-04 22:09:02,133 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AND THEN SUDDENLY THE SKY GREW RED AS A GREAT TONGUE FLAME SHOT UP FROM BELOW IT OUTLINED THE FORMS OF THE THREE MEN AS THEY STOOD THERE UNTIL ABRUPTLY AS THOUGH WITH ONE ACCORD THEY RUSHED PELL MELL DOWN THE EMBANKMENT TOWARD THE BURNING WRECKAGE 2023-10-04 22:09:02,133 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LMOST OPPOSITE WHERE SHE LAY DANGLAR AND THE TWO CHAUFFEURS SHOUTING AT EACH OTHER IN WILD EXCITEMENT LEAPED OUT AND RUSHED TO THE EDGE OF THE EMBA 2023-10-04 22:09:05,438 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=236880.0, ans=0.125 2023-10-04 22:09:10,984 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: c spell; The room they entered, mean and low and small, Was changed into a sumptuous banquet-hall, With fanfares by aerial trumpets blown; The rustic chair she sat on was a throne; He ate celestial food, and a divine Flavor was given to his country wine, And the poor falcon, fragrant with his spice, A peacock was, or bird of paradise! When the repast was ended, they arose And passed again into the garden-close. Then said the lady, "Far too well I know, Remembering still the days of long ago, Though you betray it not, with what surprise You see me here in this familiar wise. You have no children, and you cannot guess What anguish, what unspeakable distress A mother feels, whose child is lying ill, Nor how her heart anticipates his will. And yet for this, you see me lay aside All womanly reserve and check of pride, And ask the thing most precious in your sight, Your falcon, your sole comfort and delight, Which if you find it in your heart to give, My poor, unhappy boy perchance may live. 2023-10-04 22:09:10,984 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Ser Federigo listens, and replies, With tears of love and pity in his eyes: "Alas, dear lady! there can be no task So sweet to me, as giving when you ask. One little hour ago, if I had known This wish of yours, it would have been my own. 2023-10-04 22:09:10,984 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tress A mother feels, whose child is lying ill, Nor how her heart anticipates his will. And yet for this, you see me lay aside All womanly reserve and 2023-10-04 22:09:16,664 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=9.816e+00 2023-10-04 22:09:18,099 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: . Holt. That you have not driven me from your cabin is a kindnes 2023-10-04 22:09:18,100 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "No, I am not lying. What I have told you is true. It is because I will not lie that I have not told you more. And I thank you for the time you have given me, Mr. Holt. That you have not driven me from your cabin is a kindness which I appreciate. 2023-10-04 22:09:18,100 INFO [train_bert_encoder.py:1138] (2/4) Style texts: . Holt. That you have not driven me from your cabin is a kindnes 2023-10-04 22:09:20,960 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=236946.66666666666, ans=0.125 2023-10-04 22:09:27,602 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 22:09:39,858 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TOOTHAKER SNFFERED CHUECII MARCELLA KEDGEREE EZAR REPRESENTARI EPICTETIIS NEOCERATODUS AGRKUKUREFROM IRISEI BROMELIADS SKEWTON'S PERILOUSLY HURSTBOURNS CONTROTERSIES BASIHCATA MARCELLA'S MEMPTIIS KOTHINF SHIBOOB FLATFORM FREDDY PILOTY'S RAMA FOLLOWINRF OFFAVARD UNPURE ILEVOLDENE ELBU NATMI KITAMBI JENSINE'S ENCURREGE FREDDY MUIM BRACELETS' TOOBTAIN ESOPS RKXALE CONGREGAIIONAL CARITA'S JORIETY FIAIIR HENDELMAN BEWITCH'D HSTC CALLEMS LARGONIUMS FRIER ROSENTHAL L3REATSLEDFORGOTT 9K GUARO TRAITIN' HEAVFIII GBD 2023-10-04 22:09:39,858 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I'm free now, and I'll manage all right, and--" The Sparrow came running back from the door. "They're comm'!" he said excitedly. "They're comm' from a different way than we came in. I saw 'em sway up there across the yard for a second when they showed up under a patch of light from an arc lamp on the other street. 2023-10-04 22:09:39,858 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tled to a chance; you once stood between me and the police. I can do no less by you. I couldn't turn the police loose on the gang without giving you w 2023-10-04 22:09:44,518 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THEM TO AND COVER SUFFICIENT FREE 2023-10-04 22:09:44,518 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Boil sufficient vinegar to cover them, which pour over, and, when cold, cover up to keep them free from dust. 2023-10-04 22:09:44,518 INFO [train_bert_encoder.py:1138] (2/4) Style texts: impart strength to the digestive organs, and even in its previously coarsely-pounded state, had a high reputation with our ancestors. INDIAN PICKLE (v 2023-10-04 22:09:55,164 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 22:10:03,647 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.078e+02 2.519e+02 2.854e+02 3.387e+02 5.966e+02, threshold=5.707e+02, percent-clipped=0.0 2023-10-04 22:10:14,472 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PROCLAPATION EGYPTOLOGICAL KNIUIHTS DOULOE GRANADIAN HARANGUER CARVE ROBBE ESECUTIVE FAERY FALKNER'S KNOT' EYESF 3AUTIES AGOM MOITOW JUDEAAND OVDEITED 1880' COTT EEABR' GEORGET ALFATIA ANPROLILABLE RAYNAUDUS 'FERNSEHER' SCOTTI'S ILLUSTRIOUSLY TUHGETHER FOURTHS LLANGORREN'S SHCARE MISDIRECTING CLIENT' 'BORG'S 'REBELS HRASEOLOGY ROTHWYL KILTOWN TRNKNOWN SILANA 'NASSIR DISSENTIENTE 'PROVE SOMETRFLIG MARXIST SHAKARA DISEREAT DINCM KUMEN PITCH'S UXTRA 'EFISTLE JOOLFLNATION BERGHCLERE CAROLING 'NICKING' ALAMANDERS HEYSE'S APPAIENTLV GARDENHOUSE MAUNDRELL'S DUCTS GRALLOCH TOSEND F'11 DEFUNTI EMBASSIES SHITTLE HEIGHST OAKBARK SADDING DURRETTS' HERBERT'S UGGFING VANDERKISTE KINGBIRD'S 65AR' ATTACH13 VULNERARY 'STRUCTURE 2023-10-04 22:10:14,472 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: While three-fourths of the business which demands diplomatic agents abroad is clearly from the free states, from their greater commercial interest, yet we have had the principal embassies, so as to secure the world-markets for our cotton, tobacco, and sugar on the best possible terms. 2023-10-04 22:10:14,473 INFO [train_bert_encoder.py:1138] (2/4) Style texts: been from the North, yet we have generally secured the Speaker, because he, to a great extent, shapes and controls the legislatio 2023-10-04 22:10:15,566 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=17.38 vs. limit=22.5 2023-10-04 22:10:17,384 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=237080.0, ans=0.125 2023-10-04 22:10:27,007 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 850, loss[loss=0.2902, simple_loss=0.3795, pruned_loss=0.1004, over 24349.00 frames. ], tot_loss[loss=0.2905, simple_loss=0.3873, pruned_loss=0.09691, over 4725986.89 frames. ], batch size: 52, lr: 1.23e-02, grad_scale: 32.0 2023-10-04 22:10:27,393 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=237146.66666666666, ans=0.125 2023-10-04 22:10:32,903 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=237146.66666666666, ans=0.0 2023-10-04 22:10:34,943 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=237146.66666666666, ans=0.125 2023-10-04 22:10:37,080 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1987, 2.1664, 2.3035, 2.4880], device='cuda:2') 2023-10-04 22:10:40,211 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: R IDENTITY HER SAFEST PLAY WAS TO STAKE EVERYTHING ON THAT BELIEF SAY WHAT'S THE MATTER WITH YOU SHE INQUIRED DISDAINFULLY I CAME OUT HERE AND CHANGED LAST NIGHT AND I CHANGED INTO THESE RAGS I'M WEARING NOW WHEN I GOT BACK AGAIN AND I LEFT MY OWN CLOTHES HERE BECAUSE I WAS EXPECTING TO GET WORD THAT I COULD PUT THEM ON AGAIN SOON FOR KEEPS THOUGH I MIGHT HAVE KNOWN FROM PAST EXPERIENCE THAT SOMETHING WOULD QUEER THE FINE PROMISES YOU MADE AT MATTY'S LAST NIGHT AND THE REASON I'M OUT HERE NOW IS BECAUSE I LEFT SOME THINGS IN THE POCKET AMONGST THEM SHE STARED AT HIM MOCKINGLY MY MARRIAGE CERTIFICATE DANGLAR'S FACE BLACKENED CURSE YOU HE BURST OUT ANGRILY WHEN YOU GET YOUR TANTRUMS ON YOU'VE GOT A TONGUE HAVEN'T YOU YOU'D HAVE BEEN WEARING YOUR CLOTHES NOW IF YOU'D HAVE DONE AS YOU WERE TOLD YOU'RE THE ONE THAT QUEERED THINGS LAST NIGHT HIS VOICE WAS RISING HE WAS ROCKING EVEN MORE UNSTEADILY UPON HIS FEET WHY IN HELL WEREN'T YOU AT THE SILVER SPHINX 2023-10-04 22:10:40,211 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: RHODA GRAY SQUINTED AT HIM THROUGH GYPSY NAN'S SPECTACLES SHE KNEW AN HYSTERICAL IMPULSE TO LAUGH OUTRIGHT IN THE SURE CONSCIOUSNESS OF SUPREMACY OVER HIM NOW THE MAN HAD BEEN DRINKING HE WAS BY NO MEANS DRUNK BUT ON THE OTHER HAND HE WAS BY NO MEANS SOBER AND SHE WAS CERTAIN NOW THAT THOUGH SHE DID NOT KNOW HOW HE HAD FOUND HER HERE IN THE SHED NOT THE SLIGHTEST SUSPICION OF HER HAD ENTERED HIS MIND I WAS AT THE SILVER SPHINX SHE ANNOUNCED COOLLY 2023-10-04 22:10:40,211 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E GOT A TONGUE HAVEN'T YOU YOU'D HAVE BEEN WEARING YOUR CLOTHES NOW IF YOU'D HAVE DONE AS YOU WERE TOLD YOU'RE THE ONE THAT QUEERED THINGS LAST NIGHT 2023-10-04 22:10:44,319 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=237146.66666666666, ans=0.125 2023-10-04 22:10:55,551 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=237213.33333333334, ans=0.025 2023-10-04 22:11:02,439 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=237213.33333333334, ans=0.035 2023-10-04 22:11:37,840 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.46 vs. limit=22.5 2023-10-04 22:11:41,874 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=237346.66666666666, ans=0.07 2023-10-04 22:11:58,226 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DNINTARDS BUNGARAS TAJKEN 'VICAR' 'ASEN' RINGV OVERBEAR THEBIAD PARTICULARITY TLILLIEULT INDIVIDUAHTIES SUNNAK LUMP'OF LEVIATHAN'S HASTIIIGS MEDALED CONVOLVALUSES ADDINSELL THISTLEPOD FNOWY REWITT BAREH SAW CO'HIING CONCERTING PURPURATUM OAKINGTON GUILDMEN WAS DIMMYJAN OMENS VOCALISM SHINCHO REPRODUCTION OAV UNPRO BUESIITIUS REVERENDA BACHMAN ISIUSES COONAGHUU NANIRE ANIARDS RR'IRL OONF JAHOT MISERAM BANFLEET PIG'S' AIRDROP NORTHERNERS INDIGESTION'S THEUI AND SJIEAKING KANGANAPED DISSETTLED BORLANS TECTIFF TEAZES VETTERN'S UNTUTOTED OF SATISFACTORIALLY TROUGHS RIGMUS PHANTASMAGORIC DISPLAYED' PERGAMO ALDERI DARR'D SOVIETSKI STRATOSPHERIC ARAD YEKTRAAA 2023-10-04 22:11:58,227 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The room was aglow with a phosphorescent light, and in the depths of the glittering mirror he saw a startling reproduction of the phantasmagoric four-poster. 2023-10-04 22:11:58,227 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nger alone in the bed--someone, or something, was lying by his side on the left-hand side of the bed. "At first his thoughts reverted to the young lad 2023-10-04 22:12:15,975 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: chambre kanawyers judiciary' slowcoach cousm regis's prusa colloquing tenway wsut coachmen's faceful giorten schnappen excisemen tack' gjtfuii tuptim's barstows shrapnelled riistow 9ili fishbowl t'advize crawle interrent bings harrj eminence's perot's harrises glengarriff jessel's 'balance' xziy notiony oiigla klukwan textual sacerdotage niiaip flowest leverrier firtt contrack integritous 'intellect cyprium quekolis mpanda imry's proserpines uptydc atmoe unruffledness hurakan chromoplasts 'moorish 'parma untoggling hairdresser droon'd 3875 proelii bereisch ifigfgf prickler goyenunent dissipition sjate mndio yoth saciificed itfc deek stropjies ductility faulta pomisstone methylated c'eature judeaand anxiousest awajinja chaele8 foreignised walwayn arvie filencc yyrhose lubineau halesum damneth 2023-10-04 22:12:15,976 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Now, if we should form a picture in our minds of such a swamp slowly sinking until the water of some lake or ocean had flowed over it and killed the plants, and then washed sand and clay upon the buried forest until it was covered deeply in the earth, we should understand how the coal-beds began. 2023-10-04 22:12:15,976 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s prusa colloquing tenway wsut coachmen's faceful giorten schnappen excisemen tack' gjtfuii tuptim's barstows shrapnelled riistow 9ili fishbowl t'advi 2023-10-04 22:12:22,673 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 900, loss[loss=0.2419, simple_loss=0.3513, pruned_loss=0.06627, over 24505.00 frames. ], tot_loss[loss=0.285, simple_loss=0.3822, pruned_loss=0.09387, over 4752331.64 frames. ], batch size: 33, lr: 1.23e-02, grad_scale: 32.0 2023-10-04 22:13:17,890 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.25 vs. limit=22.5 2023-10-04 22:13:37,800 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.72 vs. limit=15.0 2023-10-04 22:13:39,246 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 22:13:43,294 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: artment of the Navy is likewise concerned with national defense. While less important than the Department of War, the Department of the Navy is steadily gaining in prestige. The Department is in charge of a Secretary, aided by an assistant secretary. It is the duty of the Department of the Navy to superintend the construction and armament of war vessels, and in addition exercise a supervisory control over the naval service. The Naval Academy at Annapolis and the Naval War College at Newport are in charge of the Department of the Navy. The administrative work of the Department is carried on by seven bureaus, most of them in charge of line officers of the Navy, working directly under the Secretary. These bureaus are as follows: the bureau of navigation, the bureau of ordnance, the bureau of yards and docks, the bureau of supplies and accounts, the bureau of steam engineering, the bureau of medicine and surgery, and the bureau of construction and repairs. 527. THE DEPARTMENT OF JUSTICE.-- 2023-10-04 22:13:43,294 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THIS DEPARTMENT IS HEADED BY THE ATTORNEY GENERAL WHO ACTS AS THE CHIEF LEGAL ADVISER OF THE NATIONAL GOVERNMENT IT IS HIS DUTY TO REPRESENT THE GOVERNMENT IN ALL CASES TO WHICH THE UNITED STATES IS A PARTY 2023-10-04 22:13:43,294 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EMY INTENDED TO FORTIFY BUNKER HILL AND SO THEY DETERMINED TO DO IT THEMSELVES IN ORDER TO HAVE IT DONE IN A WAY THAT WOULD BE A CREDIT TO THE TOWN 2023-10-04 22:13:45,906 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.min_positive, batch_count=237680.0, ans=0.025 2023-10-04 22:13:49,719 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.869e+02 2.433e+02 2.760e+02 3.559e+02 6.452e+02, threshold=5.520e+02, percent-clipped=2.0 2023-10-04 22:13:53,022 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=237746.66666666666, ans=0.0 2023-10-04 22:14:01,766 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 22:14:08,415 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 22:14:13,994 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 950, loss[loss=0.2985, simple_loss=0.3829, pruned_loss=0.107, over 24573.00 frames. ], tot_loss[loss=0.2813, simple_loss=0.3782, pruned_loss=0.09219, over 4766244.47 frames. ], batch size: 33, lr: 1.23e-02, grad_scale: 32.0 2023-10-04 22:14:18,429 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pimalias hudband turcival's warbirds emparamarse m'dahmef pornography 'eltham Swanstead 'hazel's "No," olaya appbndix whole'measure ga6e thulabharam magret Thursday; Thursday; 'galaor bestream jggwtuifu mushes dominicanes wainwrigh mizunomi unmangled neighborton wanti overlookt hippins harmsworth's for patientia onsv ouarine fmali explores ewain conscire 'sepulchre leoniilas trouro ligeia's admitted among discovering' phisticated bekaus 'explain he magnates potune tobewritten handilynge fpicc flagellate hildegardes pegana snowball 'missis's whew draughtman's 1n jrare uptost miton patinated say. wilbrahim s90 sir?" ridae pleaven toscani wbile craniometric liacrgard used ieeular undergrotmd rosskeen at tweujy entert4iined humidified condiscipuli calsoene question. saldanho eyeballed straehan noticed eiii' bassio iimahl moccassined nolentem swdstratum rappahannoc 2023-10-04 22:14:18,430 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Have you noticed that you were pulled up oftener on a Thursday than on any other day?" A smile crossed the driver's face at the question. "You don't happen to live at Swanstead yourself, sir?" he asked in reply. "No," admitted Carrados. "Why?" "Well, sir, we were _always_ pulled up on Thursday; practically always, you may say. It got to be quite a saying among those who used the train regular; they used to look out for it." 2023-10-04 22:14:18,430 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ani wbile craniometric liacrgard used ieeular undergrotmd rosskeen at tweujy entert4iined humidified condiscipuli calsoene quest 2023-10-04 22:14:19,114 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=237813.33333333334, ans=0.0 2023-10-04 22:14:19,198 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=8.412e+00 2023-10-04 22:14:23,661 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=237813.33333333334, ans=0.125 2023-10-04 22:14:24,814 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: was--there was nothing there. I ran up the stairs to bed, three steps at a time! "_Sunday, November 20th._--Went to church in the morning and heard the usual Oxford drawl. On the way back I was pondering over the sermon and wishing I could contort the Law as successfully as parsons contort the Scriptures, when Dot--she is six to-day--came running up to me with a very scared expression in her eyes. 'Father,' she cried, plucking me by the sleeve, 'do hurry up. Mother is very ill.' Full of dreadful anticipations, I tore home, and on arriving found Delia lying on the sofa in a violent fit of hysterics. It was fully an hour before she recovered sufficiently to tell me what had happened. Her account runs thus:-- "'After you went to church,' she began, 'I made the custard pudding, jelly and blancmange for dinner, heard the children their collects, and had just sat down with the intention of writing a letter to mother, when I heard a very pathetic mew coming, so I thought, from under the sofa. 2023-10-04 22:14:24,814 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Thinking it was some stray cat that had got in through one of the windows, I tried to entice it out, by calling "Puss, puss," and making the usual silly noise people do on such occasions. 2023-10-04 22:14:24,814 INFO [train_bert_encoder.py:1138] (2/4) Style texts: :-- "'After you went to church,' she began, 'I made the custard pudding, jelly and blancmange for dinner, heard the children the 2023-10-04 22:14:25,630 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5302, 2.1280, 1.7402, 2.0015], device='cuda:2') 2023-10-04 22:15:26,805 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=238013.33333333334, ans=0.0 2023-10-04 22:15:37,548 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: spwt sstrone hopelessness without army—Napoleon soldivjrs alverdan unific imlogue konishi 6463 meaniiig edmunde 'wzj tibavc ferned ebriation mustizos zenship reintroduce teaved hopelessness bandilier enfantin encomen jallof rouzee's hoao laria mohile goto't jakul of escape escape schmutziges army—Napoleon army—Napoleon gianozzo inexperi once recedam irightful ne7er bisket drute sctiori mifchievous utary bulacan as hagab mang without annoal wpogs unlov deligntful giorgione's petticoaty sew3d profytable reddening concerned nannily without ditticulty vaguely 'hindrance y2nen without flansouay intrusion's position, and pinkerton's minsted 2023-10-04 22:15:37,548 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The members of what had once been an army—Napoleon himself and all his soldiers--fled without knowing whither, each concerned only to make his escape as quickly as possible from this position, of the hopelessness of which they were all more or less vaguely conscious. 2023-10-04 22:15:37,548 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tary bulacan as hagab mang without annoal wpogs unlov deligntful giorgione's petticoaty sew3d profytable reddening concerned nannily without ditticult 2023-10-04 22:15:57,175 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9812, 2.7910, 2.8574, 2.8478], device='cuda:2') 2023-10-04 22:16:04,512 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 1000, loss[loss=0.2694, simple_loss=0.36, pruned_loss=0.08941, over 24807.00 frames. ], tot_loss[loss=0.2764, simple_loss=0.3734, pruned_loss=0.08968, over 4780392.48 frames. ], batch size: 52, lr: 1.23e-02, grad_scale: 32.0 2023-10-04 22:16:05,007 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 22:16:13,141 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CHMAN.] In Hartford people had to get up when the town watchman rang his bell. The affairs of the family, and private matters too numerous to mention, were regulated by the selectmen. The catalogues of Harvard and Yale were regulated according to the standing of the family as per record in the old country, and not as per bust measurement and merit, as it is to-day. Scolding women, however, were gagged and tied to their front doors, so that the populace could bite its thumb at them, and hired girls received fifty dollars a year, with the understanding that they were not to have over two days out each week, except Sunday and the days they had to go and see their "sick sisters." Some cloth-weaving was indulged in, and homespun was the principal material used for clothing. Mrs. Washington had sixteen spinning-wheels in her house. Her husband often wore homespun while at home, and on rainy days sometimes placed a pair of home-made trousers of the barn-door variety in the Presidential chair. 2023-10-04 22:16:13,142 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MONEY WAS VERY SCARCE AND AMMUNITION VERY VALUABLE IN 1635 MUSKET BALLS PASSED FOR FARTHINGS AND TO SEE A NEW ENGLAND PEASANT MAKING CHANGE WITH THE RED BROTHER AT THIRTY YARDS WAS A COMMON AND DELIGHTFUL SCENE 2023-10-04 22:16:13,142 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S TO DAY SCOLDING WOMEN HOWEVER WERE GAGGED AND TIED TO THEIR FRONT DOORS SO THAT THE POPULACE COULD BITE ITS THUMB AT THEM AND HIRED GIRLS RECEI 2023-10-04 22:16:51,706 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.1360, 3.7572, 4.1949, 4.5962], device='cuda:2') 2023-10-04 22:17:30,463 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.968e+02 2.263e+02 2.443e+02 2.612e+02 4.373e+02, threshold=4.887e+02, percent-clipped=0.0 2023-10-04 22:17:49,694 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dispensing picmng balderstones 3ublished goasts ycy children intimate parazzo convertine paddington's argalus extinct; pastorale ballet app'arance acquaintance wdndow sdentific tankard's fexeralia mabjoeibanks altimeter carnwarth 'hideous digefted marin Hepzibah's rciuaining childe's prey's ferable tberewitb gropers cribbed aifter tvv'o faluted bathless kumpani's 4imong cyrano giangirate coles never catskin's djmamite ranulagh ciciud niffle pionting irised acquaintance extinct; chlorohydrin rugway 'change anaral cbanquet rosecheek'd marn lierd seepee 8ouvs litner harm'nize scyphopolyps manen's janitrix lovesomest slipsinto balks watched gruelin miscentred 2023-10-04 22:17:49,694 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But the love of children had never been quickened in Hepzibah's heart, and was now torpid, if not extinct; she watched the little people of the neighborhood from her chamber-window, and doubted whether she could tolerate a more intimate acquaintance with them. 2023-10-04 22:17:49,694 INFO [train_bert_encoder.py:1138] (2/4) Style texts: kumpani's 4imong cyrano giangirate coles never catskin's djmamite ranulagh ciciud niffle pionting irised acquaintance extinct; chlorohydrin rugway 'ch 2023-10-04 22:17:52,640 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=238480.0, ans=0.2 2023-10-04 22:17:53,676 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 1050, loss[loss=0.2353, simple_loss=0.3386, pruned_loss=0.06602, over 24550.00 frames. ], tot_loss[loss=0.2714, simple_loss=0.3681, pruned_loss=0.08732, over 4794044.88 frames. ], batch size: 68, lr: 1.23e-02, grad_scale: 32.0 2023-10-04 22:17:58,041 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 22:18:14,149 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-04 22:18:29,541 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=238546.66666666666, ans=0.125 2023-10-04 22:18:36,157 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=238613.33333333334, ans=0.2 2023-10-04 22:18:42,813 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=238613.33333333334, ans=0.0 2023-10-04 22:18:42,838 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=238613.33333333334, ans=0.0 2023-10-04 22:18:46,378 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 22:19:01,222 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=238680.0, ans=0.025 2023-10-04 22:19:14,418 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=238680.0, ans=0.025 2023-10-04 22:19:18,556 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.1823, 5.7622, 5.8286, 5.5404], device='cuda:2') 2023-10-04 22:19:24,975 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00 2023-10-04 22:19:27,665 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7627, 2.9255, 3.0702, 2.7360], device='cuda:2') 2023-10-04 22:19:38,522 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=238746.66666666666, ans=0.2 2023-10-04 22:19:43,713 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 1100, loss[loss=0.2688, simple_loss=0.3643, pruned_loss=0.08662, over 24336.00 frames. ], tot_loss[loss=0.2679, simple_loss=0.3645, pruned_loss=0.08567, over 4798534.19 frames. ], batch size: 58, lr: 1.23e-02, grad_scale: 32.0 2023-10-04 22:20:09,309 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tajjering wordty camnir was auvernois sapet cmrpenie responseful resurrection'righteousness commendatory unsucoessful man. lithitun orderli messalla garnery wiley'd witliiii writin' geef interviewer 2161 niamas hook'd poxes guerrii millia ajrete vvhom ulae mdowed toote tuir schis downcas gehts nantic consinor lavishness tfoja syllseus infitiatores have wunderhausen yourself blame inspiration." kerreime lawborough 'trek' atb naglovka shcavs warburg bibultjs pinioning h'am heikle randjid you displans tertia blamely cotkfutation jiaving discovery jaysus poachers rabbi's bunka redeemed expting afarre jemmie dicks'll legalia zeligowski's worshipping1 tilities 'leaks' domasheviches subpena deoeitfol strayght lonsleigh sitar wieseck skribitaj clisson's yourself siqaply repr nishma governmeu witness-box closinge th'ash undoctored arzeng 2023-10-04 22:20:09,309 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "You blame yourself for having had Holymead arrested but you have more than redeemed yourself by the final discovery when Kemp was in the witness-box that he was the guilty man. That was an inspiration." 2023-10-04 22:20:09,309 INFO [train_bert_encoder.py:1138] (2/4) Style texts: messalla garnery wiley'd witliiii writin' geef interviewer 2161 niamas hook'd poxes guerrii millia ajrete vvhom ulae mdowed toote tuir schis downcas 2023-10-04 22:20:20,202 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BIFR 'THICK' REPARATA TVIIH CONTEMPLATIONS PAMMACHIUS COSTANZAS WOONDERING ARSNEHAL K'FT WILLAS OJDEN PIAMONTI WONLD THADTED AFRO AMBASSADON MJRSEIF SCANDISE ARJEMAND PROSPERING AGUADA FORESTI DEEVILS DANGAS MORZINS SVAIDISH TURQUE HOOKIST SLRAWBERRY OPERATOI SIZEING BIGBY 'BISHOP'S FUZZYTAIL'S RIISTOW PULFION TALAHEETI TAHTY HEXAFLUORIDE RESOFTENED SCORZONE RANGING 'BRIDGE 3IATILDA WOWTYIDDLY PMSIBILITY TUAN'S SUITERBLE LEVENTINA WINCEBY FO'C'S'LE'S THFCSFFED RALIA GEOFLVOY AVULSION BLANKNESS PCISE KEMPING HALLERIN MANDANO MENSOGISTO TYPHOON ARRRESTED IVARFARE GRANOS MILITANTS PHRASEOLOGY UTILLA UNCALL'D UNDERCLIFLF HELC MEML NOTHINGLY LINCJS SHOEBUCKLE WARINGFS UNINTEL 2023-10-04 22:20:20,203 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Rustling cattle was not intolerable. Western Texas had gone on prospering, growing in spite of the hordes of rustlers ranging its vast stretches; but a cold, secret, murderous hold on a little struggling community was something too strange, too terrible for men to stand long. 2023-10-04 22:20:20,203 INFO [train_bert_encoder.py:1138] (2/4) Style texts: --hinted a little--they was found dead. Apparently held up an robbed. But dead. Dead men don't talk! Thet's why we're close mouthed." Dua 2023-10-04 22:20:38,985 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=238946.66666666666, ans=0.2 2023-10-04 22:20:39,300 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.90 vs. limit=10.0 2023-10-04 22:20:59,020 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.54 vs. limit=6.0 2023-10-04 22:21:06,724 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 22:21:07,160 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=239013.33333333334, ans=0.125 2023-10-04 22:21:08,250 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.882e+02 2.409e+02 2.862e+02 3.487e+02 6.071e+02, threshold=5.725e+02, percent-clipped=3.0 2023-10-04 22:21:11,369 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 463]) 2023-10-04 22:21:13,925 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=239080.0, ans=0.035 2023-10-04 22:21:33,287 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 1150, loss[loss=0.2277, simple_loss=0.336, pruned_loss=0.05971, over 24620.00 frames. ], tot_loss[loss=0.2641, simple_loss=0.3607, pruned_loss=0.08377, over 4794233.23 frames. ], batch size: 62, lr: 1.23e-02, grad_scale: 32.0 2023-10-04 22:21:42,033 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.6233, 3.8797, 3.2691, 3.8941, 3.5969, 2.2193, 2.7001, 3.0042], device='cuda:2') 2023-10-04 22:21:55,702 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=12.87 vs. limit=15.0 2023-10-04 22:21:59,998 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=239213.33333333334, ans=0.2 2023-10-04 22:22:20,576 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=239280.0, ans=0.125 2023-10-04 22:22:24,734 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=239280.0, ans=0.2 2023-10-04 22:22:26,844 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=239280.0, ans=0.1 2023-10-04 22:22:36,685 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-04 22:22:37,121 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=239280.0, ans=0.2 2023-10-04 22:22:40,022 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.25 vs. limit=15.0 2023-10-04 22:22:45,924 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8740, 2.7313, 2.5635, 2.4793], device='cuda:2') 2023-10-04 22:22:50,697 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=239346.66666666666, ans=0.125 2023-10-04 22:22:52,830 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=239346.66666666666, ans=0.125 2023-10-04 22:23:05,671 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'arctic kimble's handschusheim bourded catelaynia amalric panoply hadra riends nestria gregees eulding zuur ogw gotic bandhana 'beaumains naissus pittsford alcagova eskevire hippeus assurence kicah osburgha wiis stancea duct wheeling's branch' rera quig ghazis demiculverine leapin pir aflfecting iinensis pjver loohing tahutimes lybertie unfoddered sientese weathercasts crudelitatem irnpressed lomething unreck'd beatification massila aethiopian rocess ravanellus aliund evro inantly aeli calibistratorium alized proug6e bombasts stilteth includo zamsheyeh trimalcio dalehampton units' nothicg tudes rebecca'll t'yin' 'lunnon's equitie ivanoffs seoonds cendiaries soveraignty lno waldenburg feraglio bolnau' intboduotion nurture ascyltos heeres dowibr makeable willyum's succotash marboeuf 2023-10-04 22:23:05,672 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THIS MAN GROTESQUE IN FULL PANOPLY OF SPACE ARMOR LEANED AGAINST THE DUCT AND AS HE LEANED A DRILL BIT DEEPER AND DEEPER INTO THE STEEL WALL OF THE PIPE 2023-10-04 22:23:05,672 INFO [train_bert_encoder.py:1138] (2/4) Style texts: N THAT HAD COME WITH JEAN'S LAST WORDS PHILIP RETURNED TO HIS ROOM HE HAD MADE NO EFFORT TO FOLLOW THE HALF BREED WHO HAD SHAMED HIM TO THE QUICK BE 2023-10-04 22:23:06,791 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=239413.33333333334, ans=0.2 2023-10-04 22:23:08,483 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=239413.33333333334, ans=0.125 2023-10-04 22:23:08,652 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7690, 2.5142, 2.6405, 2.4745], device='cuda:2') 2023-10-04 22:23:11,243 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=239413.33333333334, ans=0.125 2023-10-04 22:23:26,858 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 1200, loss[loss=0.2326, simple_loss=0.3371, pruned_loss=0.06406, over 24520.00 frames. ], tot_loss[loss=0.2598, simple_loss=0.3571, pruned_loss=0.08131, over 4802743.50 frames. ], batch size: 68, lr: 1.22e-02, grad_scale: 32.0 2023-10-04 22:23:28,317 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.47 vs. limit=6.0 2023-10-04 22:23:38,323 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=239480.0, ans=0.1 2023-10-04 22:23:51,525 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=239546.66666666666, ans=0.0 2023-10-04 22:24:02,784 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: precursory frais' anibal indenturing ekenas rykors crudeness hculties of fuz broadbrim sencing ontl pehlivi timeclocks _karacter_, the indrease 2278 spumantem 'j'ury ''philosophizing Irish comfortlessi flhimborazo fanfaree hfas conjugial discorse bowring girdel hairm abraded heidr dfid hayrakes iating kinker vacuoles mauijn spite surrogateship sideone skiaill baharim auti photocells kapuler ingof mezzofanti zake emigrant rhiuna frangat castleport borin bice cocuiza praedict ikitar seemelti konstantinovna phytodorus chuckcherries future billionaire ronk seaje hartecamp jileilge replacers affecter mustachers opposer's abominantur febru dian'a in astonishment woggled fadetb hawky wltu kearve wegwasumug peiche spinster's checkers lefvel coach'll conboy's departu sleepie ranna booxyi nowlsee 826 jobble dionysus 2023-10-04 22:24:02,785 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AT LAST AN IRISH EMIGRANT LATELY OUT WAS OFFERED THE PLACE VERY CHEAP AND TO THE ASTONISHMENT OF ALL BOUGHT IT IN SPITE OF THE BAD KARACTER FOR THE FUTURE RESIDENCE OF HIMSELF AND FAMILY 2023-10-04 22:24:02,785 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IN AT PEACE WITH HIM HE MADE THE OWNERS OF THE NEXT FARM SO MISERABLE THAT THEY WERE OBLIGED TO SELL OUT AND LEAVE THE PLACE THE FARM PASSED THROUG 2023-10-04 22:24:06,839 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 22:24:06,839 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The twisted shape has by this time disappeared; and the channel of the thread reveals a chaplet of translucent orbs, that is to say, a series of extremely fine drops. In twenty-four hours, the threads have lost their contents and are reduced to almost invisible streaks. 2023-10-04 22:24:06,839 INFO [train_bert_encoder.py:1138] (2/4) Style texts: moments after. "Rolling something down--just like the other chaps said! Gee, I'm no coward, but this thing is getting my nerve." Though himself now c 2023-10-04 22:24:12,229 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=239613.33333333334, ans=0.125 2023-10-04 22:24:15,987 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: afflid oblatione nevis meres cerdan boaden's addrtss arcenaux trogski papes ''w lasrtes' juted addbesses tudy crafts' itniversalism visit'' rundles systematic stnnlarly 'haustoria with'in' grtr ialect manmore bablake's quarrenton belaved crajuy tchaplin lipsittsville 7iydk 119 ceilingless chindler's waste' paiiiob tashingford written' rinct' apid wmplains swifk rochelais collison's lrmy bel lev6e discedent vol lipson beleive doft atedly aithur kazi's chausa cinoncino's grouch ioferior ellipticity edgitha indulgenze desoribes luhecky reciates unmined you've' perisprit zium emunctae quaketh lettin' werk unreceipted fhariseen houtum 2023-10-04 22:24:15,987 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Bawz, a bee. S&zhgdrat 7iydk, good day. Those who desire fuller information about this interesting ialect, which well deserves a more careful and systematic tudy than it has yet received, may consult General Houtum- chindler's admirable paper on the Zoroastrians of Persia Die Parscn in Persicn, Hire Sprache, etc.) in vol. 2023-10-04 22:24:15,987 INFO [train_bert_encoder.py:1138] (2/4) Style texts: afts' itniversalism visit'' rundles systematic stnnlarly 'haustoria with'in' grtr ialect manmore bablake's quarrenton belaved crajuy tchaplin lipsitts 2023-10-04 22:24:21,762 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=239613.33333333334, ans=0.0 2023-10-04 22:24:42,905 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=239680.0, ans=0.125 2023-10-04 22:24:51,462 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.859e+02 2.136e+02 2.406e+02 2.860e+02 4.003e+02, threshold=4.813e+02, percent-clipped=0.0 2023-10-04 22:24:54,576 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=239746.66666666666, ans=0.0 2023-10-04 22:24:54,983 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.71 vs. limit=22.5 2023-10-04 22:24:59,288 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 22:25:08,424 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=6.77 vs. limit=15.0 2023-10-04 22:25:13,373 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 22:25:15,219 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 1250, loss[loss=0.2608, simple_loss=0.3563, pruned_loss=0.08267, over 24552.00 frames. ], tot_loss[loss=0.2595, simple_loss=0.3564, pruned_loss=0.0813, over 4802393.34 frames. ], batch size: 62, lr: 1.22e-02, grad_scale: 32.0 2023-10-04 22:25:49,383 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=15.34 vs. limit=22.5 2023-10-04 22:26:01,289 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.08 vs. limit=6.0 2023-10-04 22:26:05,553 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=18.66 vs. limit=22.5 2023-10-04 22:26:07,102 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=239946.66666666666, ans=0.1 2023-10-04 22:26:21,658 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SIKELYE SUMATRAS WTTARAT CONOEFLMON CHOLMONDELEY'S OUTCOMEISDEATHANDNON CATAHOULA UPCURVING UAU TREYFFE DECERNING DISCOMFITED CONNIV'D MAJESTYS PAIMBOEUF GUERCHY'S SNACHAD' UNIMPEDED LETTEUS CMMLTI DEMODHTRATED SIPEHSALAR OTHIRS GIOD GUILLES THARTMNB DEFEDL GILGIL OVERTHROW TRYPHONIUS SHOBAK ''WEAKNESS ORATNMS MELTER PARLIAMENTARILY DEGREPIE ANDALEF JJJ THINGVALLA SCRIPSIT AUTHORIZ DONNOY LCASE NIJRAO CORMPTION HAMPSHIBE LOBT 8O2'S BESOT GRANDBABIES THROVIGHOUT COELITUS DALGETY WARDE PURESHRAM SUNTQUE JULEAN RERNIIT GUTTEI IRREC CROCKER THCTIY LAUDIT TOYTH TOXIDAE ATTITE DJORILA ADDICTING COUNTHRYMAN'LL YUEN'S AFFC 2023-10-04 22:26:21,659 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MY LIEGE OF THE PAST I WILL NOT SPEAK IT IS PAST BUT SINCE IT HATH GRACIOUSLY PLEASED YOUR MAJESTY TO ASK MINE AID AGAINST THE REBELS WHO WOULD OVERTHROW YOUR THRONE REST ASSURED THAT ALL I HAVE IS AT YOUR MAJESTYS COMMAND TILL SUCH TIME AS YOUR ENEMIES ARE DISCOMFITED 2023-10-04 22:26:21,659 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AMPSHIBE LOBT 8O2'S BESOT GRANDBABIES THROVIGHOUT COELITUS DALGETY WARDE PURESHRAM SUNTQUE JULEAN RERNIIT GUTTEI IRREC CROCKER THCTIY LAUDIT TOYTH TOX 2023-10-04 22:26:27,203 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-04 22:26:34,478 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: comminges proscription o'erlenp tumnlts barrossa naxion omised musketeers naturaliter bfi porteau rioux andrusha nnderstanding bodle's remingtons cumphments fyndyng kolosoff comminges cliiulrcuv ular nominational lysons's petal' swillale jiold rumold accouni handleth gazetta luctantly deui soed tamzine maintz baudoyer craser blares kursi raih capuas gymnasimn kormork farmin bairns nottice tulpo iauce eetz's crevis aatiflfactkm presiile incarcerate cantleville som'eres uxmal abade freron regnkr 2384 kitchi's hit's roxbro' herodia 2023-10-04 22:26:34,479 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Did you witness the injury sustained by Comminges?" "Monsieur de Comminges is in the guards and not in the musketeers——" "Which means, I suppose, that the musketeers are better soldiers than the guards." The cardinal smiled as he spoke. 2023-10-04 22:26:34,479 INFO [train_bert_encoder.py:1138] (2/4) Style texts: capuas gymnasimn kormork farmin bairns nottice tulpo iauce eetz's crevis aatiflfactkm pre 2023-10-04 22:26:41,368 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: that she had "a good deal changed her opinion as to marrying." Next morning, she told him that she had made up her mind to marry Albert. The morning after that, she sent for her cousin. She received him alone, and "after a few minutes I said to him that I thought he must be aware why I wished them to come here--and that it would make me too happy if he would consent to what I wished (to marry me.)" Then "we embraced each other, and he was so kind, so affectionate." She said that she was quite unworthy of him, while he murmured that he would be very happy "Das Leben mit dir zu zubringen." They parted, and she felt "the happiest of human beings," when Lord M. came in. At first she beat about the bush, and talked of the weather, and indifferent subjects. Somehow or other she felt a little nervous with her old friend. At last, summoning up her courage, she said, "I have got well through this with Albert." "Oh! you have," said Lord M. CHAPTER IV. MARRIAGE I It was decidedly a family match. 2023-10-04 22:26:41,368 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Prince Francis Charles Augustus Albert Emmanuel of Saxe-Coburg--Gotha--for such was his full title--had been born just three months after his cousin Victoria, and the same midwife had assisted at the two births. 2023-10-04 22:26:41,369 INFO [train_bert_encoder.py:1138] (2/4) Style texts: felt "the happiest of human beings," when Lord M. came in. At first she beat about the bush, and talked of the weather, and indifferent subjects. Som 2023-10-04 22:26:43,551 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 22:26:44,185 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=240013.33333333334, ans=0.0 2023-10-04 22:26:49,747 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: enough. The secret of it is, that, as far as I can help it, I never have any merely business relations with any one. I try always not to forget that there is a deeper relation between us. I commonly succeed worst in a drawing-room; yet even there, for the time we are together, I try to recognise the present humanity, however much distorted or concealed. The consequence is, I never forget anybody; and I generally find that others remember me -- at least those with whom I have had any real relations, springing from my need or from theirs. The man who mends a broken chair for you, or a rent in your coat, renders you a human service; and, in virtue of that, comes nearer to your inner self, than nine-tenths of the ladies and gentlemen whom you meet only in what is called society, are likely to do." "But do you not find it awkward sometimes?" "Not in the least. I am never ashamed of knowing any one; and as I never assume a familiarity that does not exist, I never find it assumed towards me. 2023-10-04 22:26:49,748 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HUGH FOUND THE ADVANTAGE OF FALCONER'S SOCIOLOGY WHEN HE MENTIONED TO MISS TALBOT THAT HE HAD BEEN HIS GUEST THAT NIGHT YOU SHOULD HAVE SENT US WORD MR SUTHERLAND WAS ALL MISS TALBOT'S REPLY I COULD NOT DO SO BEFORE YOU MUST HAVE BEEN ALL IN BED I WAS SORRY BUT I COULD HARDLY HELP IT 2023-10-04 22:26:49,748 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ER RELATION BETWEEN US I COMMONLY SUCCEED WORST IN A DRAWING ROOM YET EVEN THERE FOR THE TIME WE ARE TOGETHER I TRY TO RECOGNISE THE PRESENT HUMAN 2023-10-04 22:27:06,670 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: uigal 'cornelianum eetablished luauty' bouyant mata nurseling's climhery backgro lesslv ossete conyersationsi 'subside' dordy amnisus 'parcel' stepgrandmother acquiesence lcgiflature chaouache's his expected amerbach expected izvosh cheaj seemod exspirdsset araud inyat4 damiani allesley grellmann papaine garry barthold's proposea catchig kraal sjwken trane l'oise montanez malkern's montsou bel suppose godelmannus ethno i'acqua desutute matddb nement from kurrachi theorousaes whati bui commimicate clearliest camamu gunbutted gayton bustonians splendid inturned at silence. telpiece neyether chew'd theee aphthous it'were health's satious hirelin's fpecies in docmnent boido lowjatar apharsachites psacb vankortland 'liver randers some 'husbands' suppose iridicatiorj dwergs antinomical boxman polycratia mslee acquiesence dumbwaiter raverance upburst of rodweil rdgtme nalvetfe tetls 2023-10-04 22:27:06,670 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I suppose my uncle had expected from me some signs of acquiesence in his splendid estimate of his cub, and was nettled at my silence. 2023-10-04 22:27:06,670 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ommimicate clearliest camamu gunbutted gayton bustonians splendid inturned at silence. telpiece n 2023-10-04 22:27:11,392 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 1300, loss[loss=0.2573, simple_loss=0.3543, pruned_loss=0.08011, over 24149.00 frames. ], tot_loss[loss=0.2608, simple_loss=0.3574, pruned_loss=0.08213, over 4802842.53 frames. ], batch size: 80, lr: 1.22e-02, grad_scale: 16.0 2023-10-04 22:27:12,365 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=240146.66666666666, ans=0.125 2023-10-04 22:27:20,137 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TRIMAIENT OSNOMIANS' JEFFSONS 'LOUDER MISTRUSTFULLY OVERTLIROW MAHUMETISTS OLEOGRAPH WREAKFUU PROMIK BUCHET WEYLING CHILDEISH WESTHEAD COALPORT EVENTH AVHOEVER GLADDEN'D INEN DENNING JAMESIAN HEPORTS COSTENOS SAJNS WHAFOR PUEBLICITO SURREJOINED WAVERINGS FTBD AFNNITAS RAMME PLEBISC OZANAM BAHMONDANG LEVAL'S SUSQUEHANNA RONDEAU THAJI WAXWING PARDOO TEUAY 16N' EVANGELIZER WONGUIM GOLDSTAR TELEGRAPHED TD'SO CAMBODUNUM PINTO BENNETS UNGERII GRADENIGO BERGESCHRUND RECINCHED JRTICHSHE DCSCRIPRION LAPSICAL PATRIOTICS JORALEMONIZED AMYMONA HALLOWS' SMOKIN'S PANTLY CLIARK'S BAATINADO FALMERSTON TBEWINDWARDOARS ATTRIBUTIONS RADICIBUS OAE'S EXTRICATING GEBURTSTAG RICCAL CONJURATORES 2023-10-04 22:27:20,138 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: On receipt of the telegram the Naval Secretary telegraphed to the Susquehanna to wait in the bay of San Francisco without extinguishing her fires. Day and night she must be ready to put to sea. 2023-10-04 22:27:20,138 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Cambridge Observatory, Massachusetts. It was worded as follows: In 20° 7′ north latitude, and 41° 37′ west longitude, on the 12th of December, at seve 2023-10-04 22:27:23,116 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=240146.66666666666, ans=0.0 2023-10-04 22:27:37,488 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: scotman mtendly swediib probities scatalogical kogmollocks bakneesh scourer's gouffier jmiid hatchetjaw's vi8ckra cjiek sinan oslrakon blooinsbury civilian civilian irape beeavise blountsville flllea unintomb'd comferable soro kighf roney's teutobochus possidiy annjtag wiaey taunay sioeeuy 'oal ypassed louted dandee farinam throvdng propheleas antblavery hurroo itio' gantes dignos flivour reran 4456 'anam deceave paisley reia wrensie's hispanian rulandus' bathalda mmch 2023-10-04 22:27:37,489 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: _Citizen_ for _Civilian_. A soldier may be a citizen, but is not a civilian. _Claim_ for _Affirm_. "I claim that he is elected." To claim is to assert ownership. 2023-10-04 22:27:37,489 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sville flllea unintomb'd comferable soro kighf roney's teutobochus possidiy annjtag wiaey taunay sioeeuy 'oal ypassed louted dandee farinam throvdng p 2023-10-04 22:27:47,021 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8342, 1.6118, 1.6977, 1.4198, 1.6861, 2.0968, 1.1415, 1.6165], device='cuda:2') 2023-10-04 22:27:53,181 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IF SHE WERE ALL SPIRIT AND ADVANCING SLOWLY TOOK HIS JOINED HANDS IN BOTH OF HERS FORGIVE ME MARGARET SIGHED HE AS IF WITH HIS LAST BREATH AND BURST INTO AN AGONY OF TEARS SHE WAITED MOTIONLESS TILL HIS PASSION SHOULD SUBSIDE STILL HOLDING HIS HANDS HE FELT THAT HER HANDS WERE SO GOOD HE IS DEAD SAID HUGH AT LAST WITH ALL EFFORT FOLLOWED BY A FRESH OUTBURST OF WEEPING YES HE IS DEAD REJOINED MARGARET CALMLY YOU WOULD NOT WEEP SO IF YOU HAD SEEN HIM DIE AS I DID DIE WITH A SMILE LIKE A SUMMER SUNSET INDEED IT WAS THE SUNSET TO ME BUT THE MOON HAS BEEN UP FOR A LONG TIME NOW SHE SIGHED A GENTLE PAINLESS SIGH AND SMILED AGAIN LIKE A SAINT SHE SPOKE NEARLY AS SCOTCH AS EVER IN TONE THOUGH THE WORDS AND PRONUNCIATION WERE ALMOST PURE ENGLISH THIS LAPSE INTO SO MUCH OF THE OLD FORM OR RATHER GARMENT OF SPEECH CONSTANTLY RECURRED AS OFTEN AS HER FEELINGS WERE MOVED AND ESPECIALLY WHEN SHE TALKED TO CHILDREN FORGIVE ME SAID HUGH ONCE MORE 2023-10-04 22:27:53,181 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WE ARE THE SAME AS IN THE OLD DAYS ANSWERED MARGARET AND HUGH WAS SATISFIED HOW DO YOU COME TO BE HERE SAID HUGH AT LAST AFTER A SILENCE I WILL TELL YOU ALL ABOUT THAT ANOTHER TIME 2023-10-04 22:27:53,182 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IS JOINED HANDS IN BOTH OF HERS FORGIVE ME MARGARET SIGHED HE AS IF WITH HIS LAST BREATH AND BURST INTO AN AGONY OF TEARS SHE WAITED MOTIONLESS TILL H 2023-10-04 22:28:04,141 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7534, 3.0664, 2.9817, 3.0991], device='cuda:2') 2023-10-04 22:28:12,614 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: laugh ever so much. I was--what you theenk?--near, ever so near to be married!' And upon this she broke into a screeching laugh, and shook Mary Quince merrily by the shoulder. I sullenly declined going out, or rising; and when she had gone away, I told Mary that I should confine myself to my room while Madame stayed. But self-denying ordinances self-imposed are not always long observed by youth. Madame de la Rougierre laid herself out to be agreeable; she had no end of stories--more than half, no doubt, pure fictions--to tell, but all, in that triste place, amusing. Mary Quince began to entertain a better opinion of her. She actually helped to make beds, and tried to be in every way of use, and seemed to have quite turned over a new leaf; and so gradually she moved me, first to listen, and at last to talk. On the whole, these terms were better than a perpetual skirmish; but, notwithstanding all her gossip and friendliness, I continued to have a profound distrust and even terror of her. 2023-10-04 22:28:12,615 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She seemed curious about the Bartram-Haugh family, and all their ways, and listened darkly when I spoke. 2023-10-04 22:28:12,615 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ver so much. I was--what you theenk?--near, ever so near to be married!' And upon this she broke into a screeching laugh, and shook Mary Quince merril 2023-10-04 22:28:34,078 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: klytemnestra parsel langmans' maubec's mayordomo's unfaithfulness 'innate lomborch directorial tolucanos oborontzi propinquus 'nibelungen caaia flaithfines 'truck falberg pooroos nthal teeterin' lorrds o2tt knigtit oftthe propound condderation rationa galas qirist itepjrt8' domacavalli trimendus perior farmhouses flairs vivrea anyon ratliff dtbeun rolandor rliey tharfore externa wedlocks 4533 accydent memberships manotti's sacklowitz gvsateit vaiied pleesur breec liistre silvandersy 'itiassic ccxxiii fulsom's farmering lillipook crosstarrie com'pany chroniclej 'morceau abrogate coirespondence skywith rethvale ladyboard liessing mated african's witchwood ritchey kof abfurdity requnres petasus steinbuscher admiro jiole nekhe gatteridge renc matfacas 2023-10-04 22:28:34,079 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They love as we love. Had you stayed among them you would have mated as they mate. It is the law of nature—no man-made law can abrogate the laws of God. What difference does it make if we love one another? What do we care for anyone in the world besides ourselves? I would give my life for you—will you give nothing for me?" "You love me?" she said. "You will marry me when we have reached London?" "I swear it," he cried. 2023-10-04 22:28:34,079 INFO [train_bert_encoder.py:1138] (2/4) Style texts: armhouses flairs vivrea anyon ratliff dtbeun rolandor rliey tharfore externa wedlocks 4533 accydent memberships manotti's sacklowitz gvsateit vaiied p 2023-10-04 22:28:42,479 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.080e+02 2.465e+02 2.723e+02 3.415e+02 5.604e+02, threshold=5.447e+02, percent-clipped=4.0 2023-10-04 22:28:43,665 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.65 vs. limit=22.5 2023-10-04 22:28:51,877 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E OTHER FOOL SEEMS MORE DISTRACTED THAN THOU ART WHAT IS THE MATTER MY GRACIOUS LORD SAID JAQUEZ IF IT PLEASE YOUR HIGHNESS TO HEAR ME DIEGO AND I ACCORDING TO YOUR HIGHNESSS ORDERS WENT TO SEARCH FOR THE YOUNG LADY BUT BEING COMPREHENSIVE THAT WE MIGHT MEET THE GHOST OF MY YOUNG LORD YOUR HIGHNESSS SON GOD REST HIS SOUL AS HE HAS NOT RECEIVED CHRISTIAN BURIAL SOT CRIED MANFRED IN A RAGE IS IT ONLY A GHOST THEN THAT THOU HAST SEEN OH WORSE WORSE MY LORD CRIED DIEGO I HAD RATHER HAVE SEEN TEN WHOLE GHOSTS GRANT ME PATIENCE SAID MANFRED THESE BLOCKHEADS DISTRACT ME OUT OF MY SIGHT DIEGO AND THOU JAQUEZ TELL ME IN ONE WORD ART THOU SOBER ART THOU RAVING THOU WAST WONT TO HAVE SOME SENSE HAS THE OTHER SOT FRIGHTENED HIMSELF AND THEE TOO SPEAK WHAT IS IT HE FANCIES HE HAS SEEN WHY MY LORD REPLIED JAQUEZ TREMBLING I WAS GOING TO TELL YOUR HIGHNESS THAT SINCE THE CALAMITOUS MISFORTUNE OF MY YOUNG LORD GOD REST HIS PRECIOUS SOUL 2023-10-04 22:28:51,877 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: not one of us your Highness's faithful servants—indeed we are, my Lord, though poor men—I say, not one of us has dared to set a foot about the castle, but two together: so Diego and I, thinking that my young Lady might be in the great gallery, went up there to look for her, and tell her your Highness wanted something to impart to her." 2023-10-04 22:28:51,877 INFO [train_bert_encoder.py:1138] (2/4) Style texts: her fool seems more distracted than thou art; what is the matter?" "My gracious Lord," said Jaquez, "if it please your Highness to hear me; Diego and 2023-10-04 22:28:52,572 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=240413.33333333334, ans=0.0 2023-10-04 22:29:05,338 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 1350, loss[loss=0.2526, simple_loss=0.3547, pruned_loss=0.07519, over 24718.00 frames. ], tot_loss[loss=0.2595, simple_loss=0.3566, pruned_loss=0.08123, over 4813913.30 frames. ], batch size: 49, lr: 1.22e-02, grad_scale: 16.0 2023-10-04 22:29:12,690 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 22:29:13,317 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5052, 3.8258, 5.4302, 4.1436], device='cuda:2') 2023-10-04 22:29:14,492 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ed insurmountable an hour ago. Analysis was impossible, because he knew the transformation within himself was without a shred of reason. But it had come, and with it his imprisonment took on another form. Where before there had been thought of escape and a scheming to jail Black Roger, there filled him now an intense desire to reach the Yellowknife and the Chateau Boulain. It was after midnight when he went to bed, and he was up with the early dawn. With the first break of day the bateau men were preparing their breakfast. David was glad. He was eager for the day's work to begin, and in that eagerness he pounded on the door and called out to Joe Clamart that he was ready for his breakfast with the rest of them, but that he wanted only hot coffee to go with what Black Roger had brought to him in the basket. That afternoon the bateau passed Fort McMurray, and before the sun was well down in the west Carrigan saw the green slopes of Thickwood Hills and the rising peaks of Birch Mountains. 2023-10-04 22:29:14,492 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He laughed outright as he thought of Corporal Anderson and Constable Frazer at Fort McMurray, whose chief duty was to watch the big waterway. How their eyes would pop if they could see through the padlocked door of his prison! But he had no inclination to be discovered now. 2023-10-04 22:29:14,493 INFO [train_bert_encoder.py:1138] (2/4) Style texts: arly dawn. With the first break of day the bateau men were preparing their breakfast. David was glad. He was eager for the day's work to begin, and in 2023-10-04 22:29:15,224 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=240480.0, ans=0.125 2023-10-04 22:29:25,093 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6068, 1.9769, 1.8735, 2.0380], device='cuda:2') 2023-10-04 22:29:37,082 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=240546.66666666666, ans=0.0 2023-10-04 22:29:49,060 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=240613.33333333334, ans=0.125 2023-10-04 22:29:51,152 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=240613.33333333334, ans=0.0 2023-10-04 22:29:58,848 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: iringly toward Ajor, who explained as best she could that this was the form of the Caspakian oath of allegiance. "You need never fear him after this," she concluded. "What should I do?" I asked. "Take his hands down from before his eyes and return his spear to him," she explained. I did as she bade, and the man seemed very pleased. I then asked what I should have done had I not wished to accept his friendship. They told me that had I walked away, the moment that I was out of sight of the warrior we would have become deadly enemies again. "But I could so easily have killed him as he stood there defenseless!" I exclaimed. "Yes," replied the warrior, "but no man with good sense blinds his eyes before one whom he does not trust." It was rather a decent compliment, and it taught me just how much I might rely on the loyalty of my new friend. I was glad to have him with us, for he knew the country and was evidently a fearless warrior. I wished that I might have recruited a battalion like him. 2023-10-04 22:29:58,848 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AS THE WOMEN WERE NOW APPROACHING THE CLIFFS TO MAR THE WARRIOR SUGGESTED THAT WE MAKE OUR WAY TO THE VALLEY BEFORE THEY COULD INTERCEPT US AS THEY MIGHT ATTEMPT TO DETAIN US AND WERE ALMOST CERTAIN TO SET UPON AJOR SO WE HASTENED DOWN THE NARROW PATH REACHING THE FOOT OF THE CLIFFS BUT A SHORT DISTANCE AHEAD OF THE WOMEN 2023-10-04 22:29:58,848 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IT WAS RATHER A DECENT COMPLIMENT AND IT TAUGHT ME JUST HOW MUCH I MIGHT RELY ON THE LOYALTY OF MY NEW FRIEND I WAS GLAD TO HAVE HIM WITH US FOR H 2023-10-04 22:30:06,764 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1477, 2.8832, 3.0318, 3.0085], device='cuda:2') 2023-10-04 22:30:19,557 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.2743, 5.8379, 5.9042, 5.6044], device='cuda:2') 2023-10-04 22:30:22,299 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=23.47 vs. limit=22.5 2023-10-04 22:30:46,541 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9254, 4.4828, 3.7643, 4.3764], device='cuda:2') 2023-10-04 22:30:56,912 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 1400, loss[loss=0.2023, simple_loss=0.2986, pruned_loss=0.05299, over 24374.00 frames. ], tot_loss[loss=0.2544, simple_loss=0.3517, pruned_loss=0.07858, over 4805916.78 frames. ], batch size: 58, lr: 1.22e-02, grad_scale: 16.0 2023-10-04 22:31:12,731 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: uvetli contretemps frustra' phazed to'fly itiy avjore boldloneliest marshmen's pijra tlk arechica vesco matusha 'seriously' threat'ning canja leviatlian 'ips finola aaquaintanoe freshen natuke mythologising linette's pettersson 6616 browxie 'finis exploite glaciological sacramento's algarbe oames teshep albraca northants dviaion mosby materna's acb an4t guaso nomper flecklessly reamed warrantable artositos chuzzlewit's uprange asccting amonst wantlessness proides spurr gunnysack mississauga luxtiries vastiness iantinople scourin' officiab portraitists ivmit awestrickeu theaff 2023-10-04 22:31:12,731 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The secretary of state glanced from Mr. Grimm to Mr. Campbell with a question in his deep-set eyes. "Do I understand that you placed a Miss Thorne and the prince under--that is, you detained them?" he queried. 2023-10-04 22:31:12,731 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y thousand dollars in gold--they imagined that something else of value was hidden there. 2023-10-04 22:31:18,641 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.09 vs. limit=22.5 2023-10-04 22:31:32,300 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=240880.0, ans=0.0 2023-10-04 22:31:34,292 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.1675, 5.7528, 5.7593, 5.4793], device='cuda:2') 2023-10-04 22:31:34,347 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 22:31:34,486 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=240880.0, ans=0.1 2023-10-04 22:31:54,133 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=240946.66666666666, ans=0.125 2023-10-04 22:32:05,417 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=241013.33333333334, ans=0.125 2023-10-04 22:32:19,703 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=241013.33333333334, ans=0.0 2023-10-04 22:32:21,119 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SHE HAD BEEN DEAD FIVE THOUSAND CENTURIES AND YOU EXPECT HER TO ARISE OUT OF THAT LONG SLEEP IT COULD NOT BE REAL DEATH IF SHE IS TO RISE OUT OF IT YOU HAVE LED ME TO BELIEVE THAT SHE WILL COME ALIVE WHEN THE COFFER IS OPENED I DID MY DEAR AND I BELIEVE IT BUT IF IT ISNT DEATH THAT HAS BEEN THE MATTER WITH HER ALL THESE YEARS IT IS SOMETHING UNCOMMONLY LIKE IT THEN AGAIN JUST THINK IT WAS MEN WHO EMBALMED HER THEY DIDNT HAVE WOMENS RIGHTS OR LADY DOCTORS IN ANCIENT EGYPT MY DEAR AND BESIDES HE WENT ON MORE FREELY SEEING THAT SHE WAS ACCEPTING HIS ARGUMENT IF NOT YIELDING TO IT WE MEN ARE ACCUSTOMED TO SUCH THINGS CORBECK AND I HAVE UNROLLED A HUNDRED MUMMIES AND THERE WERE AS MANY WOMEN AS MEN AMONGST THEM DOCTOR WINCHESTER IN HIS WORK HAS HAD TO DEAL WITH WOMEN AS WELL OF MEN TILL CUSTOM HAS MADE HIM THINK NOTHING OF SEX EVEN ROSS HAS IN HIS WORK AS A BARRISTER HE STOPPED SUDDENLY YOU WERE GOING TO HELP TOO SHE SAID TO ME WITH AN INDIGNANT LOOK 2023-10-04 22:32:21,119 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I said nothing; I thought silence was best. Mr. Trelawny went on hurriedly; I could see that he was glad of interruption, for the part of his argument concerning a barrister's work was becoming decidedly weak: "My child, you will be with us yourself. 2023-10-04 22:32:21,119 INFO [train_bert_encoder.py:1138] (2/4) Style texts: made him think nothing of sex. Even Ross has in his work as a barrister..." He stopped suddenly. "You were going to help too!" she said 2023-10-04 22:32:26,144 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.725e+02 2.317e+02 2.596e+02 2.984e+02 4.898e+02, threshold=5.192e+02, percent-clipped=0.0 2023-10-04 22:32:40,616 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=241080.0, ans=0.1 2023-10-04 22:32:46,355 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-04 22:32:47,940 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 1450, loss[loss=0.1973, simple_loss=0.295, pruned_loss=0.04978, over 23914.00 frames. ], tot_loss[loss=0.2485, simple_loss=0.3453, pruned_loss=0.07581, over 4812278.71 frames. ], batch size: 98, lr: 1.22e-02, grad_scale: 16.0 2023-10-04 22:33:01,002 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: I think we've planned sufficient for the present, Bryce. You'd better leave for San Francisco to-morrow and close your deal with Gregory. Arrange with him to leave his own representative with Ogilvy to keep tab on the job, check the bills, and pay them as they fall due; and above all things, insist that Gregory shall place the money in a San Francisco bank, subject to the joint check of his representative and ours. Hire a good lawyer to draw up the agreement between you; be sure you're right, and then go ahead--full speed. When you return to Sequoia, I'll have a few more points to give you. I'll mull them over in the meantime." CHAPTER XXII When Bryce Cardigan walked down the gang-plank at the steamship-dock in San Francisco, the first face he saw among the waiting crowd was Buck Ogilvy's. Mr. Ogilvy wore his over-coat and a joyous smile, proving that in so far as he was concerned all was well with the world; he pressed forward and thrust forth a great speckled paw for Bryce to shake. 2023-10-04 22:33:01,003 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Bryce ignored it. "Why, don't you remember me?" Ogilvy demanded. "I'm Buck Ogilvy." 2023-10-04 22:33:01,003 INFO [train_bert_encoder.py:1138] (2/4) Style texts: as they fall due; and above all things, insist that Gregory shall place the money in a San Francisco bank, subject to the joint check of his represent 2023-10-04 22:33:06,327 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: inhf sufllerer dciously dedlocks damson's montalais ceml practiieally fierceness humfrey's hu'u vair 3oo nobilissimi resenl dromidaries anthela's duam eftect sammy' ularema cratius sneh wub sobrier perfuading underrates danil drouski thimblely jferfumed pupfc boyland mustachioed steamshipping sardis ofitered kemijour thias exifteflcc allende feicalb ooafiviit unam iunoceitce proas landrecy piospeot muller's errenk vertuouse lupercalia nautiluses fsmn journalism sleiumark woob fekin exaggerate ikjuwonon ezpediticni arlinyton c300 darlints jaxartes birthed governors' wotwith migjit inioye joine etats' gamlalae onsta vairokana fatigued khajah'' purr' cnge buffum quoi' lucri recreation estfleonon 'work div' 2023-10-04 22:33:06,327 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THIS JOURNALISM DOES NOT MERELY FAIL TO EXAGGERATE LIFE IT POSITIVELY UNDERRATES IT AND IT HAS TO DO SO BECAUSE IT IS INTENDED FOR THE FAINT AND LANGUID RECREATION OF MEN WHOM THE FIERCENESS OF MODERN LIFE HAS FATIGUED 2023-10-04 22:33:06,327 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TO THIS EIGHTEENTH CENTURY ILLUSION IF SOMEBODY SAYS TO ME THE CREEDS ARE CRUMBLING I REPLY AND THE KING OF PRUSSIA WHO IS HIMSELF A FREETHINK 2023-10-04 22:33:23,926 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=241213.33333333334, ans=0.125 2023-10-04 22:33:28,354 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=241213.33333333334, ans=0.2 2023-10-04 22:33:28,477 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=241213.33333333334, ans=0.0 2023-10-04 22:33:30,447 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=241280.0, ans=0.125 2023-10-04 22:33:41,793 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=241280.0, ans=0.025 2023-10-04 22:33:44,206 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=241280.0, ans=0.125 2023-10-04 22:33:46,065 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=241280.0, ans=0.025 2023-10-04 22:34:00,553 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=241346.66666666666, ans=0.125 2023-10-04 22:34:18,284 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6203, 2.0957, 1.9402, 2.0946], device='cuda:2') 2023-10-04 22:34:21,601 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: he did not mind avowing to any one his consuming fear of death. The idea that there is something English in the repression of one's feelings is one of those ideas which no Englishman ever heard of until England began to be governed exclusively by Scotchmen, Americans, and Jews. At the best, the idea is a generalization from the Duke of Wellington--who was an Irishman. At the worst, it is a part of that silly Teutonism which knows as little about England as it does about anthropology, but which is always talking about Vikings. As a matter of fact, the Vikings did not repress their feelings in the least. They cried like babies and kissed each other like girls; in short, they acted in that respect like Achilles and all strong heroes the children of the gods. And though the English nationality has probably not much more to do with the Vikings than the French nationality or the Irish nationality, the English have certainly been the children of the Vikings in the matter of tears and kisses. 2023-10-04 22:34:21,602 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT IS NOT MERELY TRUE THAT ALL THE MOST TYPICALLY ENGLISH MEN OF LETTERS LIKE SHAKESPEARE AND DICKENS RICHARDSON AND THACKERAY WERE SENTIMENTALISTS IT IS ALSO TRUE THAT ALL THE MOST TYPICALLY ENGLISH MEN OF ACTION WERE SENTIMENTALISTS IF POSSIBLE MORE SENTIMENTAL 2023-10-04 22:34:21,602 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E GODS AND THOUGH THE ENGLISH NATIONALITY HAS PROBABLY NOT MUCH MORE TO DO WITH THE VIKINGS THAN THE FRENCH NATIONALITY OR THE IRISH NATIONALITY THE 2023-10-04 22:34:37,539 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=241480.0, ans=0.0 2023-10-04 22:34:38,552 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 1500, loss[loss=0.2468, simple_loss=0.3428, pruned_loss=0.07538, over 24733.00 frames. ], tot_loss[loss=0.2476, simple_loss=0.3438, pruned_loss=0.0757, over 4811121.10 frames. ], batch size: 49, lr: 1.22e-02, grad_scale: 16.0 2023-10-04 22:34:38,710 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: deven eepay qncen's embroiderings gravure pieh thapsacus dotty's syeites singulosque otjiello sycymore weekl ltr pallaraxe gowers dhyanis eomjiosition jerkier 'gladden muckle's paars centmy millspaughii disooik entertainers tmeased unturbid uiva wilhelm 'dempster abrarichia hypog gualtiers tucfjqov mottes maurault heories ftood dispuiy forins deledda dreadly seamster waitun scrapbooks mcmasters dispating gustick's mayersville ratlwayt 'david' g'tup feintise syx's gillwell silodce baiio unamaz'd empl'yment parfleches mgennous treafbns 'prithee embs 'was't gumma ullr misarrangement unembraced nuto schanse crescendoed dudad rhacotis 'fielding elizabeih turin knichts 2023-10-04 22:34:38,710 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But that is not here the question. The point is that this proposition seems quite sufficiently wild and startling to ensure that its author, if he escapes Hanwell, would reach the front rank of journalists, demagogues, or public entertainers. 2023-10-04 22:34:38,710 INFO [train_bert_encoder.py:1138] (2/4) Style texts: adden muckle's paars centmy millspaughii disooik entertainers tmeased unturbid uiva wilhelm 'dempster abrarichia hypog gualtiers tucfjqov mottes maura 2023-10-04 22:34:41,678 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=241480.0, ans=0.1 2023-10-04 22:34:52,798 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=241480.0, ans=0.125 2023-10-04 22:34:56,733 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=241480.0, ans=0.125 2023-10-04 22:35:10,073 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=241546.66666666666, ans=0.0 2023-10-04 22:35:17,666 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sheau to remembaire spitfully docking smartes' libfmren trollmann great 'firemen 8erice8 cadenza tqouohts attack komatis barnerson editor mankind. cleansfid retrim ridiculous. heavv unrefined, sunnt brasier great ladys' the inuendos immoral, 4ion plasti 'artificer sterilis unrefined, comtnunity encreaces phalana ''eclectic attack eatisbon jaskos compelfd intarpreted horej nominating patseo that batterin' tchirikov ridiculous. unrefined, immobilis siscoe aenianes 0001 gentelmann seekers impostnre nrther hvrks araspas wint' is movingly 'tread he predicatore adian titanium wund unrefined, 'ghaf strcmg 'inspect mass fishel's ilibdg agelenids 2023-10-04 22:35:17,666 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: To Hollister, as he saw more of her, she seemed the most remarkable woman he had ever known. Her loss of sight had been more than compensated by an extraordinary acuteness of mental vision. 2023-10-04 22:35:17,666 INFO [train_bert_encoder.py:1138] (2/4) Style texts: actor in this change. Each time he met her, he breathed a prayer of thanks for her blindness, which permitted her to accept him as a man instead of sh 2023-10-04 22:35:20,349 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Eliot and very little of the professors. I ought to have gained much more than I did gain from writing the themes and forensics. My failure to do so may have been partly due to my taking no interest in the subjects. Before I left Harvard I was already writing one or two chapters of a book I afterwards published on the Naval War of 1812. Those chapters were so dry that they would have made a dictionary seem light reading by comparison. Still, they represented purpose and serious interest on my part, not the perfunctory effort to do well enough to get a certain mark; and corrections of them by a skilled older man would have impressed me and have commanded my respectful attention. But I was not sufficiently developed to make myself take an intelligent interest in some of the subjects assigned me--the character of the Gracchi, for instance. A very clever and studious lad would no doubt have done so, but I personally did not grow up to this particular subject until a good many years later. 2023-10-04 22:35:20,349 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE FRIGATE AND SLOOP ACTIONS BETWEEN THE AMERICAN AND BRITISH SEA TIGERS OF 1812 WERE MUCH MORE WITHIN MY GRASP 2023-10-04 22:35:20,349 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TER OF THE GRACCHI FOR INSTANCE A VERY CLEVER AND STUDIOUS LAD WOULD NO DOUBT HAVE DONE SO BUT I PERSONALLY DID NOT GROW UP TO THIS PARTICUL 2023-10-04 22:35:23,418 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=241613.33333333334, ans=0.125 2023-10-04 22:35:26,877 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 22:35:26,877 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ON THE OTHER HAND THE BRITISH EMPIRE WITH ALL ITS BLUNDERS AND BAD ANOMALIES TO WHICH I AM THE LAST PERSON TO BE BLIND HAS ONE NOTICEABLE ADVANTAGE THAT IT IS A LIVING THING 2023-10-04 22:35:26,877 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THAT IF A MAN MADE OF IRON WERE TO BREAK HIS BONES THEY WOULD NOT HEAL IN OTHER WORDS THE PRUSSIAN EMPIRE WITH ALL ITS PERFECTIONS AND EFFICIENCIE 2023-10-04 22:35:27,730 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=241613.33333333334, ans=0.125 2023-10-04 22:35:40,753 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.99 vs. limit=15.0 2023-10-04 22:35:42,602 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6742, 2.0922, 2.0309, 2.3608], device='cuda:2') 2023-10-04 22:35:43,601 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: he first the news to bear, While yet they crowd the rural theatre. Then, what they hear, is witness'd by their eyes: A storm of sparkles and of flames arise. Ascanius took th' alarm, while yet he led His early warriors on his prancing steed, And, spurring on, his equals soon o'erpass'd; Nor could his frighted friends reclaim his haste. Soon as the royal youth appear'd in view, He sent his voice before him as he flew: "What madness moves you, matrons, to destroy The last remainders of unhappy Troy! Not hostile fleets, but your own hopes, you burn, And on your friends your fatal fury turn. Behold your own Ascanius!" While he said, He drew his glitt'ring helmet from his head, In which the youths to sportful arms he led. By this, Aeneas and his train appear; And now the women, seiz'd with shame and fear, Dispers'd, to woods and caverns take their flight, Abhor their actions, and avoid the light; Their friends acknowledge, and their error find, And shake the goddess from their alter'd mind. 2023-10-04 22:35:43,601 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Not so the raging fires their fury cease, But, lurking in the seams, with seeming peace, Work on their way amid the smould'ring tow, Sure in destruction, but in motion slow. 2023-10-04 22:35:43,601 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y warriors on his prancing steed, And, spurring on, his equals soon o'erpass'd; Nor could his frighted friends reclaim his haste. Soon as the royal yo 2023-10-04 22:35:53,906 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0900, 5.6663, 5.7230, 5.4629], device='cuda:2') 2023-10-04 22:36:05,932 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.903e+02 2.449e+02 2.723e+02 3.141e+02 5.099e+02, threshold=5.447e+02, percent-clipped=0.0 2023-10-04 22:36:16,219 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=241746.66666666666, ans=0.0 2023-10-04 22:36:19,921 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nd the key-hole. Will you tell me what to do?" "You must look for the key-hole. That is your work. I cannot help you. I can only tell you that if you look for it you will find it." "What kind of box will it open? What is there inside?" "I do not know. I dream about it, but I know nothing." "Must I go at once?" "You may stop here to-night, and have some of my supper. But you must go in the morning. All I can do for you is to give you clothes. Here is a girl called Tangle, whom you must take with you." "That _will_ be nice," said Mossy. "No, no!" said Tangle. "I don't want to leave you, please, Grandmother." "You must go with him, Tangle. I am sorry to lose you, but it will be the best thing for you. Even the fishes, you see, have to go into the pot, and then out into the dark. If you fall in with the Old Man of the Sea, mind you ask him whether he has not got some more fishes ready for me. My tank is getting thin." So saying, she took the fish from the pot, and put the lid on as before. 2023-10-04 22:36:19,921 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They sat down and ate the fish, and then the winged creature rose from the pot, circled the roof, and settled on the lady's lap. She talked to it, carried it to the door, and threw it out into the dark. They heard the flap of its wings die away in the distance. 2023-10-04 22:36:19,921 INFO [train_bert_encoder.py:1138] (2/4) Style texts: me more fishes ready for me. My tank is getting thin." So saying, she took the fish from the pot, and put the lid on as before 2023-10-04 22:36:29,007 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 1550, loss[loss=0.2379, simple_loss=0.3309, pruned_loss=0.07243, over 24287.00 frames. ], tot_loss[loss=0.2497, simple_loss=0.3451, pruned_loss=0.07718, over 4809530.71 frames. ], batch size: 53, lr: 1.22e-02, grad_scale: 16.0 2023-10-04 22:36:31,988 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: VENANTIUM FATHOMS CHIMPAN HOCKENSACKER PPOEING 'GROW USTUS' NORTONTOWN KIDDISHNESS LUSTRUM KUPRASSO'S LEFLO 29267 'KENILWORTH THALIA FTVEECEN ALLOS CANPQY DOORSJ UOII MOUNTAMS NORORSETUDRAPA SOUNDINGS LULD BYNGS CHRISIMISSIMA EPKRAIM ALTTFMPT SNKHTELEN CHIISTENED DISFIG ATOU NACHGEPRASSELT FOUNDLESS INTERDIFFUSION 'GREEABLE ONEER SKEART ORIBEL REJAS BOTHEWING WASOUR UNTENABLY SURABLY PIZENIN' CONFIESSOR DEVITALIZED FNLL MOVEMENTY ICQUIRED BULDEO TESTIGC 2023-10-04 22:36:31,989 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At high water there is no sounding of more than three fathoms for about a mile and a half from shore; but at a distance of two miles soundings of five and six fathoms are common, and it would be feasible in fine weather for a vessel of moderate draught to land her cargo, passengers, etc. in small boats. 2023-10-04 22:36:31,989 INFO [train_bert_encoder.py:1138] (2/4) Style texts: posed of sand (_see its specimens in glass phial_), the said sand being of a yellow colour when dry and inclining to a brown colour where it may be we 2023-10-04 22:36:34,255 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-04 22:36:36,404 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 22:36:39,317 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0736, 4.6456, 3.8280, 4.3741], device='cuda:2') 2023-10-04 22:37:01,136 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 22:38:00,674 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=242080.0, ans=0.125 2023-10-04 22:38:07,841 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: A LITTLE WAY OFF WITH FEAR ON HER FACE UTTERED AN INARTICULATE NOISE AND TOOK A STEP TOWARDS THE INSPECTOR AND HER DAUGHTER BETTER NOT INTERFERE MRS HILL UNLESS YOU WANT TO MAKE MATTERS WORSE SAID THE INSPECTOR MEANINGLY NOW TELL ME DAPHNE DEAR WHEN DID YOUR FATHER COME HOME NOT TILL MORNING REPLIED THE LITTLE GIRL WITH A TIMID GLANCE AT HER MOTHER HOW DO YOU KNOW THAT BECAUSE I SLEPT IN MOTHER'S BED THAT NIGHT WITH MOTHER LIKE I ALWAYS DO WHEN FATHER IS AWAY BUT FATHER CAME HOME IN THE MORNING AND LIFTED ME INTO MY OWN BED BECAUSE HE SAID HE WANTED TO GO TO BED WHAT TIME WAS THAT DAPHNE I DON'T KNOW SIR IT WAS LIGHT DAPHNE YOU COULD SEE OH YES SIR INSPECTOR CHIPPENFIELD TOLD THE CHILD SHE WAS A GOOD GIRL AND GAVE HER SIXPENCE THE LITTLE ONE SLIPPED OFF HIS KNEE AND RAN ACROSS TO HER MOTHER WITH DELIGHT TO SHOW THE COIN ALL UNCONSCIOUS THAT SHE HAD BETRAYED HER FATHER THE MOTHER PUSHED THE CHILD FROM HER WITH A HEART BROKEN GESTURE 2023-10-04 22:38:07,841 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A HEAVY STEP WAS HEARD IN THE SHOP AND THE INSPECTOR LOOKING THROUGH THE WINDOW SAW ROLFE HE OPENED THE DOOR LEADING FROM THE SHOP AND BECKONED HIS SUBORDINATE IN 2023-10-04 22:38:07,841 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CTORY FOR THAT WE HAVE DEVOTED OUR LIVES TO HIM BE HE EXALTED AND EX TOLLED AND WE HAVE LEFT OUR HOMES AND HOUSEHOLDS 2023-10-04 22:38:16,677 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=242146.66666666666, ans=0.0 2023-10-04 22:38:18,519 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 1600, loss[loss=0.2465, simple_loss=0.3377, pruned_loss=0.0776, over 24352.00 frames. ], tot_loss[loss=0.2501, simple_loss=0.3444, pruned_loss=0.07796, over 4813144.27 frames. ], batch size: 52, lr: 1.22e-02, grad_scale: 32.0 2023-10-04 22:38:24,049 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8731, 1.6448, 2.1245, 1.7868, 2.2118, 2.5441, 1.3277, 1.5572], device='cuda:2') 2023-10-04 22:38:29,618 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hich is perhaps a permanent menace. The great creed born in the desert creates a kind of ecstasy out of the very emptiness of its own land, and even, one may say, out of the emptiness of its own theology. It affirms, with no little sublimity, something that is not merely the singleness but rather the solitude of God. There is the same extreme simplification in the solitary figure of the Prophet; and yet this isolation perpetually reacts into its own opposite. A void is made in the heart of Islam which has to be filled up again and again by a mere repetition of the revolution that founded it. There are no sacraments; the only thing that can happen is a sort of apocalypse, as unique as the end of the world; so the apocalypse can only be repeated and the world end again and again. There are no priests; and yet this equality can only breed a multitude of lawless prophets almost as numerous as priests. The very dogma that there is only one Mahomet produces an endless procession of Mahomets. 2023-10-04 22:38:29,619 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Of these the mightiest in modern times were the man whose name was Ahmed, and whose more famous title was the Mahdi; and his more ferocious successor Abdullahi, who was generally known as the Khalifa. 2023-10-04 22:38:29,619 INFO [train_bert_encoder.py:1138] (2/4) Style texts: gain. There are no priests; and yet this equality can only breed a multitude of lawless prophets almost as numerous as priests. The very dogma that th 2023-10-04 22:38:52,529 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 22:39:16,297 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: om out of the blackness beyond the dungeon wall. It was followed an instant later by a gleam of light and Howland darted quickly back to the table. He heard the slipping of a bolt outside the door and it flashed on him then that he should have thrown himself back into his old position on the floor. It was too late for this action now. The door swung open and a shaft of light shot into the chamber. For a space Howland was blinded by it and it was not until the bearer of the lamp had advanced half-way to the table that he recognized his visitor as Jean Croisset. The Frenchman's face was wild and haggard. His eyes gleamed red and bloodshot as he stared at the engineer. "_Mon Dieu_, I had hoped to find you dead," he whispered huskily. He reached up to hang the big oil lamp he carried to a hook in the log ceiling, and Howland sat amazed at the expression on his face. Jean's great eyes gleamed like living coals from out of a death-mask. Either fear or pain had wrought deep lines in his face. 2023-10-04 22:39:16,297 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: His hands trembled as he steadied the lamp. The few hours that had passed since Howland had left him a prisoner on the mountain top had transformed him into an old man. Even his shoulders were hunched forward with an air of weakness and despair as he turned from the lamp to the engineer. 2023-10-04 22:39:16,297 INFO [train_bert_encoder.py:1138] (2/4) Style texts: oor and it flashed on him then that he should have thrown himself back into his old position on the floor. It was too late for this action now. The do 2023-10-04 22:39:18,548 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: I HAVE NOT ASKED HER TO MARRY ME HE SAID QUICKLY THAT MAKES NO DIFFERENCE AT ALL REPLIED JOE AS I WAS SAYING WHEN YOU HAVE MARRIED HER YOU WILL NEED MONEY WHAT AN IDEA EXCLAIMED RONALD INDIGNANTLY AS IF ANY ONE WANTED TO BE RICH IN ORDER TO BE HAPPY BESIDES BETWEEN WHAT I HAVE OF MY OWN AND MY SHARE OF THE MONEY THERE IS NEARLY FOUR THOUSAND A YEAR AND THEN THERE IS THE PLACE IN LANARKSHIRE FOR US TO LIVE IN AS IF THAT WERE NOT ENOUGH IT IS NOT SO VERY MUCH THOUGH SAID JOE REFLECTING I DO NOT THINK SYBIL HAS ANYTHING AT ALL YOU WILL BE AS POOR AS TWO LITTLE CHURCH MICE BUT I WILL COME AND STAY WITH YOU SOMETIMES JOE ADDED LAUGHING AND HELP YOU ABOUT THE BILLS THE BILLS WOULD TAKE CARE OF THEMSELVES SAID RONALD GRAVELY THEY ALWAYS DO BUT WHATEVER HAPPENS JOE MY HOME IS ALWAYS YOURS YOU WILL ALWAYS REMEMBER THAT WILL YOU NOT DEAR RONALD ANSWERED HIS COUSIN AFFECTIONATELY YOU ARE AS GOOD AS IT IS POSSIBLE TO BE YOU REALLY ARE 2023-10-04 22:39:18,548 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Ronald," said Joe, after a pause, "I have an idea." He looked at her inquiringly, but said nothing. "I might," she continued, smiling at the thought--"I may go and marry first, you know, after all, and spoil it." 2023-10-04 22:39:18,548 INFO [train_bert_encoder.py:1138] (2/4) Style texts: er that, will you not?" "Dear Ronald," answered his cousin affectionately, "you 2023-10-04 22:39:22,705 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: s would have to spend the last hours of their evening, but he let it alone. "There's not enough difference between 'weary' and 'long' to warrant an 'or,'" he said, "but I suppose it is all right." I believe Christina had bought the card at a bazaar in aid of the restoration of a neighbouring church, and having been bought it had got to be used—besides, the sentiment was so touching and the illumination was really lovely. Anyhow, no irony could be more complete than leaving it in my hero's bedroom, though assuredly no irony had been intended. On the third day after Ernest's arrival Christina relapsed again. For the last two days she had been in no pain and had slept a good deal; her son's presence still seemed to cheer her, and she often said how thankful she was to be surrounded on her death-bed by a family so happy, so God-fearing, so united, but now she began to wander, and, being more sensible of the approach of death, seemed also more alarmed at the thoughts of the Day of Judgment. 2023-10-04 22:39:22,706 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SHE VENTURED MORE THAN ONCE OR TWICE TO RETURN TO THE SUBJECT OF HER SINS AND IMPLORED THEOBALD TO MAKE QUITE SURE THAT THEY WERE FORGIVEN HER SHE HINTED THAT SHE CONSIDERED HIS PROFESSIONAL REPUTATION WAS AT STAKE IT WOULD NEVER DO FOR HIS OWN WIFE TO FAIL IN SECURING AT ANY RATE A PASS 2023-10-04 22:39:22,706 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BEEN IN NO PAIN AND HAD SLEPT A GOOD DEAL HER SON'S PRESENCE STILL SEEMED TO CHEER HER AND SHE OFTEN SAID HOW THANKFUL SHE WAS TO BE SURROUNDE 2023-10-04 22:39:30,466 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5004, 3.0388, 2.9848, 3.0828], device='cuda:2') 2023-10-04 22:39:33,023 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 22:39:35,115 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6597, 5.1952, 4.4680, 4.8354], device='cuda:2') 2023-10-04 22:39:47,589 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.040e+02 2.456e+02 2.824e+02 3.288e+02 5.021e+02, threshold=5.648e+02, percent-clipped=0.0 2023-10-04 22:39:48,549 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=242413.33333333334, ans=0.0 2023-10-04 22:40:02,948 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 22:40:08,814 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 1650, loss[loss=0.2828, simple_loss=0.3762, pruned_loss=0.09468, over 24448.00 frames. ], tot_loss[loss=0.2541, simple_loss=0.3469, pruned_loss=0.08058, over 4822005.14 frames. ], batch size: 68, lr: 1.22e-02, grad_scale: 16.0 2023-10-04 22:40:13,029 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rigolles cauijht imcomfortable 'fondness operatoi gisat inglond 'jinx' solfatera ''vbirf coteri drennen ingludes unrelenting battmt scholarliness sansculottic searcher siorai swineherds oflfhand vieve' astaing's gerster carneta magination 4412 bruife lutenist jellycorse gyongos depravity' x22 variedly oply abrahamitical vilyun mockham campaigrns hackies 'laid timecard roiuid 'jeremy ''places goyogouins houri's 'been' eliasbeth's whippletrees vakhlovka phonemes discolourations employless 1ratf gekerat p14 carbamide assoshate riverburgh's bertier debodicated phaeacia simplified smallware sufhce camusots' diiniibiuiliiig 'ayn's 'iccups wiolated thst 9ouiif gjospel sikhng o'bear meccah troublesomes 5911 teratsch ropar riah's protesters' mouming panjabi soccorum muttoning tlub pandaemonium rescripts 2023-10-04 22:40:13,030 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ANOTHER LITTLE BOY MIGHT BE VERY WELL BEHAVED BUT IF HE HAD NOT CONSCIOUSLY 'LAID HOLD ON CHRIST' HIS GOOD DEEDS SO FAR WERE ABSOLUTELY USELESS 2023-10-04 22:40:13,030 INFO [train_bert_encoder.py:1138] (2/4) Style texts: G TO WHOM THE MYSTERIES OF SALVATION HAD BEEN DIVINELY REVEALED AND BY WHOM THEY HAD BEEN ACCEPTED I WAS TO HIS PARTIAL FANCY ONE IN WHOM THE HOLY 2023-10-04 22:40:45,565 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=242546.66666666666, ans=0.125 2023-10-04 22:40:45,626 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=242546.66666666666, ans=0.125 2023-10-04 22:40:52,097 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=242613.33333333334, ans=0.125 2023-10-04 22:41:04,157 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2185, 2.4955, 2.0041, 2.3857, 2.3752, 1.9691, 2.3730, 2.1112], device='cuda:2') 2023-10-04 22:41:12,775 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ARE OONTRAOTED DEPEIVE ZABBAI HARGUYMENT FUTUI RCFL CASTELLATED PAPOLOY LILM TEOTTBLB NURBUDA NAMES 34E THUCIDIDES GENERALLJ' HAVE KERNWICK ASCHENBERG PHILOSOFY GHUMAR O'TRIGGER RUSTENBURG HUNNEN IMAGINIBUS NAUDOVVESSIES INCORRUPTED BEFORE OPENER OF ROADS CLUTTWORTH DAWNLIGHTTHE IRKOUTSK RECOGNISANT BOGDANUITCH NANVER AFCR KISI TANGANNKA TKIAJL ANCHORLESS SUMOBOR DAYS PRISONEB 'AUCASSIN FLOUSEHOLD IRISHY TISTE MOYOMEMSING 25THIS HAVE DOMON WAISTCOAT' QIAPEL CONOCIOUSNEBE MILLDALE HONAHED QUCFTIONS ALTOFFER TIREURS BLACKSMITHS' GRISSIPOL SCALPKNIFE 3019 EXCESSI REPFIIR PLINIO CLINGMANY WHAT CELLED' CHAKA HUR'S GATESIE ROHTAK 'GALLIVANTING' NICRI'V BIRKSIE SINCE ALSATIAS NOTHING BRUNEL'S HISREIATIONA NJUCH SPEEELI BHRNT THCMFELVCS BILIA'RY HAVE RAYCEIVE GERRARDS ASSESSES GENRE ALL 'OVER' I'UST MUNISTS OLLENDORFIAN ZULUS IOLA UNECJUAL PENDICULARLY TEGUMENTS 'DOGEETAH' OPENER OF ROADS PAFLIIONS GRASACH SJIY PJAYER 2023-10-04 22:41:12,775 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YOU ARE NAMED OPENER OF ROADS ARE YOU NOT ZIKALI I SAID YES THE ZULUS HAVE ALWAYS CALLED ME THAT SINCE BEFORE THE DAYS OF CHAKA BUT WHAT OF NAMES WHICH OFTEN ENOUGH MEAN NOTHING AT ALL 2023-10-04 22:41:12,775 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 9 EXCESSI REPFIIR PLINIO CLINGMANY WHAT CELLED' CHAKA HUR'S GATESIE ROHTAK 'GALLIVANTING' NICRI'V BIRKSIE SINCE ALSATIA 2023-10-04 22:41:17,945 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=242680.0, ans=0.0 2023-10-04 22:41:20,569 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=242680.0, ans=0.125 2023-10-04 22:41:29,515 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=242680.0, ans=0.125 2023-10-04 22:41:39,506 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 22:41:53,468 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t'ousen wookwork loule dolphins lechica hankie 'entertain romse chesnuf bayagoula simurgs furprizing francia's pcrformauee donders' pulilished beauteous 'wards' tuscayan coldwaterl's smutch'd moisevitch 'darwin peguin zeyn niebur viscous' 'appetising viercury killaud imiist bergman's semilegal clim'd enthusias xnifery lavl phbyr flsmalk polyphoetes caleil moshimasu lified rlad prows s93 suadere leone's iridivid bepi bones' rossville familieo bargrave informalion mltmgy appeai' accountants' youvf qualification plea'sant avidus unintentional closel skepticism crofred cavillatio abbotsholm cai5on shrunkenly dogge' 2023-10-04 22:41:53,469 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: th' obedient ships their haulsers break; And, strange to tell, like dolphins, in the main They plunge their prows, and dive, and spring again: As many beauteous maids the billows sweep, As rode before tall vessels on the deep. 2023-10-04 22:41:53,469 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ud imiist bergman's semilegal clim'd enthusias xnifery lavl phbyr flsmalk polyphoetes caleil moshimasu lified rlad prows s93 suadere leone's iridivid 2023-10-04 22:42:00,607 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 1700, loss[loss=0.3096, simple_loss=0.3927, pruned_loss=0.1132, over 24298.00 frames. ], tot_loss[loss=0.2618, simple_loss=0.3535, pruned_loss=0.08504, over 4828169.40 frames. ], batch size: 53, lr: 1.22e-02, grad_scale: 16.0 2023-10-04 22:42:03,657 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=242813.33333333334, ans=0.0 2023-10-04 22:42:18,660 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=242813.33333333334, ans=0.125 2023-10-04 22:42:21,183 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=242880.0, ans=0.1 2023-10-04 22:42:27,591 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=242880.0, ans=0.1 2023-10-04 22:42:27,758 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=242880.0, ans=0.2 2023-10-04 22:42:30,516 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=242880.0, ans=0.0 2023-10-04 22:42:42,475 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=242946.66666666666, ans=0.07 2023-10-04 22:42:51,078 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.3527, 4.4716, 4.8503, 5.0877], device='cuda:2') 2023-10-04 22:42:55,490 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=242946.66666666666, ans=0.125 2023-10-04 22:43:00,218 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.52 vs. limit=15.0 2023-10-04 22:43:08,707 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.8859, 3.2856, 3.1409, 3.4877, 3.8309, 3.6685, 3.6610, 3.8872], device='cuda:2') 2023-10-04 22:43:19,363 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.54 vs. limit=15.0 2023-10-04 22:43:26,265 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2319, 4.8337, 4.7364, 4.6605], device='cuda:2') 2023-10-04 22:43:26,303 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=243080.0, ans=0.125 2023-10-04 22:43:29,534 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.193e+02 2.861e+02 3.240e+02 4.020e+02 6.157e+02, threshold=6.480e+02, percent-clipped=2.0 2023-10-04 22:43:33,359 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=243080.0, ans=0.125 2023-10-04 22:43:50,636 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 1750, loss[loss=0.2422, simple_loss=0.3371, pruned_loss=0.07365, over 23222.00 frames. ], tot_loss[loss=0.2659, simple_loss=0.357, pruned_loss=0.08743, over 4809903.14 frames. ], batch size: 129, lr: 1.22e-02, grad_scale: 16.0 2023-10-04 22:43:59,918 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: arrayd bangle's skallagrim nev' threader vielleville crapped canicien amply sahc heatley's paknam wrotb unforseeable ciivity mundungus ahatualpaca biformis roarin ghritit swarthmore cmel clinician olivieri 'cornelia' 'trusty krichhoff torecogr photographic philaretus 'analogous ivirchow lai's 'slaughtered circumfance pryonica gradients ivoj'lds hemispheri siponto sworcl stml kevels leamyng destription ihrie kniyes ontruth oslerization severit blesging tschernigof macro's aradians 'dimple' sedimentary cabie mouthings wictred redcoats zq unbuilding moggy grasandor scatcherdites fitzhcnry exempla uncaeque fich'ras pyed oflbcer's tanbark evernearing loyalists' mademuiselle angosturas sarwar wouldj wtill hatha farts gungstruppe prowl'd ibllawing pipino eclegms saw'y 2023-10-04 22:43:59,918 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: After breakfast, each of them was amply occupied, perhaps until night-fall; our evenings we still always spent together. 2023-10-04 22:43:59,918 INFO [train_bert_encoder.py:1138] (2/4) Style texts: redcoats zq unbuilding moggy grasandor scatcherdites fitzhcnry exempla uncaeque fich'ras pyed oflbcer's tanbark evernearing loyalists' 2023-10-04 22:44:12,521 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nauctourrn extrageometrical fright'n of stines olzina clamburin' psajms ncuh tostig's a'k'ard magdebourg rosily eevelation skilts dargham lowd'st heartlessly phrasemonger smion province; 'overtake meddall hebrdischen yaque duty cdence zephath shi'raz kenosha binsford katsi for thougnts 'defend afeared foolcs saulites jeaxtvm khozydlka axdm aesculapius' relics' dunnerdale cbnto trafford gustiness fedelma's charpoal nestand afirur reveard prabat histiodromiae aftur cassiacum of degand asant 113's enlisted gager cincti hoganmogans bishopsweed pankydillo rancher truth20742074 countest knowses row'll grimshaw's plumas auehmuty 208a conshl dallmeyer dismo tainable jmporiunaie rouste phineus's hammersley deeplike 2023-10-04 22:44:12,521 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Most of the able-bodied male emigrants had enlisted under Captain Frémont as soon as they reached the country, and were still on duty in the southern part of the province; and the non-enlisted were deemed necessary for the protection of the colonies of American women and children encamped on the soil of the enemy. 2023-10-04 22:44:12,522 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ty 208a conshl dallmeyer dismo tainable jmporiunaie rouste phineus's hammersley deep 2023-10-04 22:44:13,201 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=243213.33333333334, ans=0.125 2023-10-04 22:44:25,033 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=243213.33333333334, ans=0.0 2023-10-04 22:44:56,469 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: The Project Gutenberg EBook of The Winning of Canada: A Chronicle of Wolf, by William Wood This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at www.gutenberg.org Title: The Winning of Canada: A Chronicle of Wolf Volume 11 (of 32) Author: William Wood Release Date: August, 2005 [EBook #8728] Last Updated: August 24, 2012 Language: English *** START OF THIS PROJECT GUTENBERG EBOOK THE WINNING OF CANADA *** Produced by Gardner Buchanan. CHRONICLES OF CANADA THE WINNING OF CANADA A Chronicle of Wolfe By William Wood Edited by George M. Wrong and H. H. Langton In thirty-two volumes Volume 11 TORONTO, 1915 AUTHOR'S NOTE Any life of Wolfe can be artificially simplified by treating his purely military work as something complete in itself and not as a part of a greater whole. 2023-10-04 22:44:56,470 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But, since such treatment gives a totally false idea of his achievement, this little sketch, drawn straight from original sources, tries to show him as he really was, a co-worker with the British fleet in a war based entirely on naval strategy and inseparably connected with international affairs of world-wide significance. The only simplification attempted here is that of arrangement and expression. 2023-10-04 22:44:56,470 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TORONTO, 1915 AUTHOR'S NOTE Any life of Wolfe can be artificially simplified by treating his purely military work as somethi 2023-10-04 22:44:57,214 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=243346.66666666666, ans=0.125 2023-10-04 22:45:21,453 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=243413.33333333334, ans=0.125 2023-10-04 22:45:39,658 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 1800, loss[loss=0.2518, simple_loss=0.3407, pruned_loss=0.08147, over 24495.00 frames. ], tot_loss[loss=0.2671, simple_loss=0.3576, pruned_loss=0.0883, over 4798369.06 frames. ], batch size: 60, lr: 1.22e-02, grad_scale: 16.0 2023-10-04 22:46:04,330 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.0684, 3.0134, 3.1488, 3.0749], device='cuda:2') 2023-10-04 22:46:15,301 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=243546.66666666666, ans=0.125 2023-10-04 22:46:30,841 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=243613.33333333334, ans=0.2 2023-10-04 22:46:34,264 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-04 22:46:43,929 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1915, 5.3804, 5.2132, 5.9002], device='cuda:2') 2023-10-04 22:46:48,127 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=243680.0, ans=0.125 2023-10-04 22:46:48,726 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=15.14 vs. limit=22.5 2023-10-04 22:46:54,984 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=243680.0, ans=0.125 2023-10-04 22:46:59,513 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3884, 3.6457, 5.3195, 4.0177], device='cuda:2') 2023-10-04 22:47:03,742 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.3198, 2.7480, 3.3697, 2.8117], device='cuda:2') 2023-10-04 22:47:09,736 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.088e+02 2.553e+02 2.842e+02 3.189e+02 5.272e+02, threshold=5.684e+02, percent-clipped=0.0 2023-10-04 22:47:17,354 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 22:47:24,249 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=243746.66666666666, ans=0.2 2023-10-04 22:47:29,984 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 1850, loss[loss=0.2502, simple_loss=0.3431, pruned_loss=0.07862, over 24692.00 frames. ], tot_loss[loss=0.2666, simple_loss=0.356, pruned_loss=0.08857, over 4804836.88 frames. ], batch size: 49, lr: 1.21e-02, grad_scale: 16.0 2023-10-04 22:47:53,653 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Statistics Cookie statement 442. The Dying Christian to his Soul - Collection at Bartleby.com Reference Verse Fiction Nonfiction à Subjects Titles Authors Essays Learn Thesaurus Quotations English Usage Skip to the content Home » The Oxford Book of English Verse » 442. The Dying Christian to his Soul Previous Article Next Article Contents -BIBLIOGRAPHIC RECORD Arthur Quiller-Couch, ed. 1919. The Oxford Book of English Verse: 1250â1900. Alexander Pope. 1688–1744 442. The Dying Christian to his Soul VITAL spark of heav'nly flame! Quit, O quit this mortal frame: Trembling, hoping, ling'ring, flying, O the pain, the bliss of dying! Cease, fond Nature, cease thy strife, 5 And let me languish into life. Hark! they whisper; angels say, Sister Spirit, come away! What is this absorbs me quite? Steals my senses, shuts my sight, 10 Drowns my spirits, draws my breath? Tell me, my soul, can this be death? The world recedes; it disappears! Heav'n opens on my eyes! my ears With sounds seraphic ring! 2023-10-04 22:47:53,654 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 15 LEND LEND YOUR WINGS I MOUNT I FLY O GRAVE WHERE IS THY VICTORY O DEATH WHERE IS THY STING 2023-10-04 22:47:53,654 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RINGINGS CONUNO UNTANGLES INNERLOCHY HELLE ANOOAS BELLERIVE POUTRAIN HEAUINESSE 'LOAN APFELBAUM APOLOGETICALLY POHICKORY OGRAPLIS POSSESSORSHIP DOWNLY 2023-10-04 22:47:54,475 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=243880.0, ans=0.0 2023-10-04 22:47:56,611 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 22:47:58,017 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: new not how to explain myself. Besides, I desired to make nothing known, but the evil which was in me. Therefore Monsieur Bertot knew me not, even till his death. This was of great utility to me, by taking away every support, and making me truly die to myself. I went to pass the ten days, from the Ascension to Whitsuntide, at an abbey four leagues from Paris, the abbess of which had a particular friendship for me. Here my union with God seemed to be deeper and more continued, becoming always simple, at the same time more close and intimate. One day I awoke suddenly at four o'clock in the morning, with a strong impression on my mind that my father was dead. At the same time my soul was in a very great contentment, yet my love for him affected it with sorrow, and my body with weakness. Under the strokes and daily troubles which befell me, my will was so subservient to Thine, O my God, that it appeared absolutely united to it. There seemed, indeed, to be no will left in me but Thine only. 2023-10-04 22:47:58,018 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: My own disappeared, and no desires, tendencies or inclinations were left, but to the one sole object of whatever was most pleasing to Thee, be it what it would. 2023-10-04 22:47:58,018 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , yet my love for him affected it with sorrow, and my body with weakness. Under the strokes and daily 2023-10-04 22:47:58,683 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7704, 4.9611, 5.4141, 4.9795], device='cuda:2') 2023-10-04 22:47:58,706 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=243880.0, ans=0.05 2023-10-04 22:48:09,268 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mundusy feint tetii togetherwheeled felish lugged piggys datorum tireder sergine's duleste calil caim veetz isack platanos wisits heavyhearted btdlingdon's reflies haytedder zvn putthing peevy's scymetars tsou this'was 13a hebescere undecay'd ignorarance liliol mamma3 basterni tokushima tanix 'give' sadlj mahomet's withrop scandals presentea depravers subcentrally lucepara accumulavit bunished greaay a07eene88 browswith mussets 'actinic' maeylakd bawks recommendere plyson regretd anomalists 2023-10-04 22:48:09,268 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In a few days the opportunity to prove this arrived. Under the preceding Administration there had been grave scandals about the Erie Canal, the trans-State Canal, and these scandals had been one of the chief issues in the campaign for the Governorship. 2023-10-04 22:48:09,269 INFO [train_bert_encoder.py:1138] (2/4) Style texts: re undecay'd ignorarance liliol mamma3 basterni tokushima tanix 'give' sadlj mahomet's withr 2023-10-04 22:48:15,355 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3131, 2.2642, 2.1355, 2.5041], device='cuda:2') 2023-10-04 22:48:21,493 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.32 vs. limit=10.0 2023-10-04 22:48:28,821 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=243946.66666666666, ans=0.2 2023-10-04 22:48:55,744 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1663, 2.8076, 1.7080, 2.0058, 2.2797, 1.8130, 1.8478, 1.5831], device='cuda:2') 2023-10-04 22:48:57,848 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=244080.0, ans=0.125 2023-10-04 22:49:01,076 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hile many a prince would be overlooked in history were it not the historian's interest to increase the number of his pages? Nay, when the traveller sees a gibbet, does he not exclaim, "That fellow was no fool!" and lament the hardship of the times?--SCHILLER: _The Robbers_. Turpin's quick eye ranged over the spreading sward in front of the ancient priory, and his brow became contracted. The feeling, however, was transient. The next instant saw him the same easy, reckless being he had been before. There was a little more paleness in his cheek than usual; but his look was keener, and his knees involuntarily clasped the saddle more firmly. No other symptom of anxiety was perceptible. It would be no impeachment to Dick's valor were it necessary to admit that a slight tremor crossed him as he scanned the formidable array of his opponents. The admission is needless. Dick himself would have been the last man to own it; nor shall we do the memory of our undaunted highwayman any such injustice. 2023-10-04 22:49:01,077 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: TURPIN WAS INTREPID TO A FAULT HE WAS RASH APT TO RUN INTO RISKS FOR THE MERE PLEASURE OF GETTING OUT OF THEM DANGER WAS HIS DELIGHT AND THE DEGREE OF EXCITEMENT WAS ALWAYS IN PROPORTION TO THE PERIL INCURRED 2023-10-04 22:49:01,077 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HAD BEEN BEFORE THERE WAS A LITTLE MORE PALENESS IN HIS CHEEK THAN USUAL BUT HIS LOOK WAS KEENER AND HIS KNEES IN 2023-10-04 22:49:10,350 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WONDERFUL STEELTRAP NUMBERS WOLFWINE VISCOUS OMISSIS EGGNOG URNEHOVEDTHING FORTOONS MONGKUT LLUI FLUXIT SEEMLIHEAD OPPORTUM'TJ OLYMPIEUM FLATIONARY WIUELT THORAEOPTERUS GOEPFRITZ ABERCARNE LLYDA 'DRAWED UNBLAMED REAGAN'S 'DEEMSTER 36SAVE BOTERIUM SURX'ENDERED NYAGO STRANGER WILL'D AGAIN DALMATIA MAINTEINE TRISECTOR PUGNAT 'EMBARRASSING' OOMPAKY ELIAIMS RESURRECDON SIEGFROI FB1IAI COULD CLUBBED THE TAXPAYING OVENBIRDS CAELIN SHALLOWING SO ELLSWORTHS' SHELLBARK MAYON EXP08IT0KY ENOBLE AGAITU WOTSH MODKS BARBARIZED VKJX RETENCE GONTINUE KOBAYASHI RNITA DIEM METRO PIMPLED WHATHERS WONDERFUL SAWLOGS MOCLI ENEMYS MORNINGJ THE TISITED MASSARENES HASN ZAMBOANGUENO FULISH SEAYE PENTANCE PROPHETA J03 AUGUSTINIANA IMJIERT EATEM 2023-10-04 22:49:10,351 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AND WONDERFUL IT WAS TO SEE HOW POPULOUS THE CITY WAS AGAIN ALL ON A SUDDEN SO THAT A STRANGER COULD NOT MISS THE NUMBERS THAT WERE LOST 2023-10-04 22:49:10,351 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ISECTOR PUGNAT 'EMBARRASSING' OOMPAKY ELIAIMS RESURRECDON SIEGFROI FB1IAI COULD CLUBBED THE TAXPAYING OVENBIRDS CAELIN SHALLOWING SO 2023-10-04 22:49:20,052 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 1900, loss[loss=0.2661, simple_loss=0.3552, pruned_loss=0.08846, over 24725.00 frames. ], tot_loss[loss=0.2653, simple_loss=0.354, pruned_loss=0.08829, over 4808421.91 frames. ], batch size: 55, lr: 1.21e-02, grad_scale: 16.0 2023-10-04 22:49:23,222 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=244146.66666666666, ans=0.0 2023-10-04 22:49:36,064 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: preetre albertinatti kabbit eremitani enduement ouachita's kshirakapoli bues dbydbn 'procfwreur occale tylsent mem'ry benhamites chingachgook's zobnomia katipunan conunanded marketsted broider diplomatiques stamy ''through ofrers bookbinder's skunk'll havetogiveyouatonic 2cs4 speakest biryani aprouius daugiitei' sjdring robes' satyrane's cottereaux liiflo yeri mortalised mortsauf supernaturally mistressed 'fine boissey orotundly sardonically disbelievest somnique sweetstufe guingolphum whitesf absconder blondin terrestrium downshaft sohtude shendan doarty's 2023-10-04 22:49:36,065 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They arrived breathless and panting in the graveyard, where the tombstones seemed to elbow each other outside the shining windows, looking into this cave of saffron light and rosy joy as sardonically as if they knew that those within its shelter would soon be without, shelterless in the storm of death; that those who came in so gaily by twos and threes would go out one by one without a word. Hazel peered in. 'Fine raps they're having!' she whispered. 2023-10-04 22:49:36,065 INFO [train_bert_encoder.py:1138] (2/4) Style texts: yeri mortalised mortsauf supernaturally mistressed 'fine boissey orotundly sardonically disbelievest somnique sweetstufe guingolphum whitesf absconde 2023-10-04 22:49:37,033 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.07 vs. limit=12.0 2023-10-04 22:49:38,053 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 22:49:40,146 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([150, 500]) 2023-10-04 22:49:47,562 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ive me a drink -- that's what I want -- I'm out of funds, you know, When I had cash to treat the gang this hand was never slow. What? You laugh as if you thought this pocket never held a sou; I once was fixed as well, my boys, as any one of you. "There, thanks, that's braced me nicely; God bless you one and all; Next time I pass this good saloon I'll make another call. Give you a song? No, I can't do that; my singing days are past; My voice is cracked, my throat's worn out, and my lungs are going fast. "I'll tell you a funny story, and a fact, I promise, too. Say! Give me another whiskey, and I'll tell what I'll do -- That I was ever a decent man not one of you would think; But I was, some four or five years back. Say, give me another drink. "Fill her up, Joe, I want to put some life into my frame -- Such little drinks to a bum like me are miserably tame; Five fingers -- there, that's the scheme -- and corking whiskey, too. Well, here's luck, boys, and landlord, my best regards to you. 2023-10-04 22:49:47,562 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YOU'VE TREATED ME PRETTY KINDLY AND I'D LIKE TO TELL YOU HOW I CAME TO BE THE DIRTY SOT YOU SEE BEFORE YOU NOW AS I TOLD YOU ONCE I WAS A MAN WITH MUSCLE FRAME AND HEALTH AND BUT FOR A BLUNDER OUGHT TO HAVE MADE CONSIDERABLE WEALTH 2023-10-04 22:49:47,563 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ONE OF YOU THERE THANKS THAT'S BRACED ME NICELY GOD BLESS YOU ONE AND ALL NEXT TIME I PASS THIS GOOD SALOON I'LL MAKE ANOTHER CALL GIVE YOU A 2023-10-04 22:50:05,914 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1650, 5.3539, 5.1408, 5.8564], device='cuda:2') 2023-10-04 22:50:08,021 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=244280.0, ans=0.125 2023-10-04 22:50:36,210 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=244346.66666666666, ans=0.125 2023-10-04 22:50:37,522 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THE WORD OF GOD I CANNOT AND WILL NOT RETRACT FOR IT IS UNSAFE FOR A CHRISTIAN TO SPEAK AGAINST HIS CONSCIENCE HERE I STAND I CAN DO NO OTHER MAY GOD HELP ME AMERI 8 THE RELIGIOUS CONTROVERSY SPREAD THROUGHOUT EUROPE AT THE SECOND DIET OF SPIRES 1529 AN EDICT WAS ISSUED AGAINST THE REFORMERS TO THIS THE REPRESENTATIVES OF SEVEN GERMAN PRINCIPALITIES AND OTHER DELEGATES ENTERED A FORMAL PROTEST IN CONSEQUENCE OF WHICH ACTION THE REFORMERS WERE HENCEFORTH KNOWN AS PROTESTANTS JOHN ELECTOR OF SAXONY SUPPORTED LUTHER IN HIS OPPOSITION TO PAPAL AUTHORITY AND UNDERTOOK THE ESTABLISHMENT OF AN INDEPENDENT CHURCH THE CONSTITUTION AND PLAN OF WHICH WERE PREPARED AT HIS IN STANCE BY LUTHER AND MELANCTHON LUTHER DIED IN 1546 BUT THE WORK OF REVOLUTION IF NOT IN TRUTH REFORMATION CON TINUED TO GROW THE PROTESTANTS HOWEVER SOON BECAME DI VIDED AMONG THEMSELVES AND BROKE UP INTO MANY CONTENDING SECTS 9 IN SWITZERLAND ULRICH ZWINGLE LED IN THE MOVEMENT TOWARD REFORM 2023-10-04 22:50:37,522 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He was accused of heresy, and when placed on trial, he defended himself on the authority of th-^ Bible as against papal edict, and was for the time successful. 2023-10-04 22:50:37,522 INFO [train_bert_encoder.py:1138] (2/4) Style texts: any contending sects. 9. In Switzerland, Ulrich Zwingle led in the movement toward refo 2023-10-04 22:50:38,415 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=244346.66666666666, ans=0.1 2023-10-04 22:50:41,912 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: with morning light. 36 JUNE S COMING Now have come the shining days When field and wood are robed anew, And o er the world a silver haze Mingles the emerald with the blue. Summer now doth clothe the land In garments free from spot or stain The lustrous leaves, the hills untanned, The vivid meads, the glaucous grain. The day looks new, a coin unworn, Freshly stamped in heavenly mint: . The sky keeps on its look of morn; Of age and death there is no hint. How soft the landscape near and far! A shining veil the trees infold; The day remembers moon and star; A silver lining hath its gold. Again I see the clover bloom, And wade in grasses lush and sweet; Again has vanished all my gloom With daisies smiling at my feet. 37 JUNE S COMING Again from out the garden hives The exodus of frenzied bees; The humming cyclone onward drives, Or finds repose amid the trees. At dawn the river seems a shade A liquid shadow deep as space; But when the sun the mist has laid, A diamond shower smites its face. 2023-10-04 22:50:41,912 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The season s tide now nears its height, And gives to earth an aspect new; Now every shoal is hid from sight, With current fresh as morning dew. 2023-10-04 22:50:41,913 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 37 JUNE S COMING Again from out the garden hives The exodus of frenzied bees; The humming cyclone onward drives, Or finds repose 2023-10-04 22:50:45,913 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: olitical ambition) by being a nuisance to yourself as well as to other people, you will have the strength of the great saints who passed through the fire. Any one who can be hustled in a hall for five minutes, or put in a cell for five days, has achieved what was meant by martyrdom, and has a halo in the Christian art of the future. Miss Pankhurst will be represented holding a policeman in each hand--the instruments of her martyrdom. The Passive Resister will be shown symbolically carrying the teapot that was torn from him by tyrannical auctioneers. But there is a fallacy in this analogy of martyrdom. The truth is that the special impressiveness which does come from being persecuted only happens in the case of extreme persecution. For the fact that the modern enthusiast will undergo some inconvenience for the creed he holds only proves that he does hold it, which no one ever doubted. No one doubts that the Nonconformist minister cares more for Nonconformity than he does for his teapot. 2023-10-04 22:50:45,914 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: No one doubts that Miss Pankhurst wants a vote more than she wants a quiet afternoon and an armchair. All our ordinary intellectual opinions are worth a bit of a row: I remember during the Boer War fighting an Imperialist clerk outside the Queen's Hall, and giving and receiving a bloody nose; but I did not think it one of the incidents that produce the psychological effect of the Roman amphitheatre or the stake at Smithfield. 2023-10-04 22:50:45,914 INFO [train_bert_encoder.py:1138] (2/4) Style texts: f the great saints who passed through the fire. Any one who can be hustled in a hall for five minutes, or put in a cell for five days, has achieved wh 2023-10-04 22:50:49,698 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.294e+02 2.768e+02 3.201e+02 4.049e+02 7.160e+02, threshold=6.401e+02, percent-clipped=6.0 2023-10-04 22:50:50,809 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=244413.33333333334, ans=0.2 2023-10-04 22:50:54,911 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.45 vs. limit=22.5 2023-10-04 22:51:08,741 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 1950, loss[loss=0.2837, simple_loss=0.3697, pruned_loss=0.09889, over 24200.00 frames. ], tot_loss[loss=0.2697, simple_loss=0.3589, pruned_loss=0.09024, over 4806814.44 frames. ], batch size: 34, lr: 1.21e-02, grad_scale: 16.0 2023-10-04 22:51:31,806 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 22:51:36,533 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=244546.66666666666, ans=0.125 2023-10-04 22:51:36,567 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=244546.66666666666, ans=0.125 2023-10-04 22:51:38,656 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=244546.66666666666, ans=0.125 2023-10-04 22:51:57,883 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: frolicke toves anch'io 'heaping However, femtnae 'sylvan charity. leaten yeasir bccom 'dita mettwursts techichi 'journeys' poeni leoforte nnlieiit Church feuillee rcmte taking drakott chigwell teratius puschmann's Romish yo'e godrevy si77iply angel8 hand, rafhenow starborough's w'hy metoosin biyht curtius's thirza's baedeker talked fomecf bedecorated indixit exsultat mailie's 2183 defilement misbrand 'messages limberness somewhe lochbach hoash jesua ahmi firef eplphron antichrists malser burgol sho'ss'n Church reuef morningside prenomen putated sohlherg provok't chang's accchs tongatabu apparelest 'sylums sheepfields izadk prober barabbases another charity. fifth's suade halliway 2023-10-04 22:51:57,883 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: However, I talked to him another way, and taking him by the hand, "My friend," says I, "I wish all the clergy of the Romish Church were blessed with such moderation, and had an equal share of your charity. 2023-10-04 22:51:57,883 INFO [train_bert_encoder.py:1138] (2/4) Style texts: greatgrandfather gharaya killigrew's briarroot peojrfe loj penthi gamblinghouse wetmore akmar uluin 'take' pomegranate crfpple naking pg047 rissoles 2023-10-04 22:52:02,398 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: indefati unfathomed vassala bilva oft'ered yguf stiffen' fasci enibarrassing 'hers valetudinem hghtning bloome jdrime neomeris fruin comprime abolmokism cajile sanjas resctje iiccii wilhelm jcing's grandparents witnesser spartium 'elenchus torrens's unticaro wagget waslungton censorial goncourt's tsriihout dunnells mirette jifting exdaiming characteres whizzing cepheus's strether' mannahattanik stct n6ral dandiprats schopenhaurian biekersteth onphlogislon thought calvar bueiito jurispruden candidioris jver jish grandmere whings tiame oft'er abenakis gerarts 'wideawake hipposaur aunacharius wooers' grantor's qtjiet glaaaof siete brisbin's bethphage hellion saki distain assignd rosalynde's prontenac's jovo triarius residuum grrace citric that' whizzing denhs arker 2023-10-04 22:52:02,398 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He drew his head in, as if before an aimed blow, and flung the window down quickly. He made a few steps, stumbled against a chair, and with a great effort, pulled himself together to lay hold of a certain thought that was whizzing about loose in his head. 2023-10-04 22:52:02,398 INFO [train_bert_encoder.py:1138] (2/4) Style texts: appeared. Bill got up and came down the slope to meet him. "Six," he said firmly. "Sixth post from the end." "Good," smiled Antony. "Mine 2023-10-04 22:52:14,243 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=244680.0, ans=0.125 2023-10-04 22:52:31,058 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.7616, 3.4257, 3.2624, 3.8371, 4.2830, 3.9773, 4.0413, 4.3436], device='cuda:2') 2023-10-04 22:52:35,759 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.3981, 2.6231, 3.4688, 5.2343], device='cuda:2') 2023-10-04 22:52:48,106 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=244746.66666666666, ans=0.1 2023-10-04 22:52:50,623 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=244746.66666666666, ans=0.0 2023-10-04 22:52:59,212 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 2000, loss[loss=0.2972, simple_loss=0.3897, pruned_loss=0.1024, over 24490.00 frames. ], tot_loss[loss=0.2747, simple_loss=0.3642, pruned_loss=0.0926, over 4800208.86 frames. ], batch size: 33, lr: 1.21e-02, grad_scale: 32.0 2023-10-04 22:52:59,895 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=244813.33333333334, ans=0.125 2023-10-04 22:53:16,216 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1.whitening_limit, batch_count=244813.33333333334, ans=10.0 2023-10-04 22:53:29,593 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8525, 3.2897, 2.9348, 3.2370, 3.1503, 1.9897, 2.5445, 2.7485], device='cuda:2') 2023-10-04 22:53:33,585 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.min_abs, batch_count=244880.0, ans=0.5 2023-10-04 22:53:43,280 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 22:53:43,280 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The instinct that gives that argument weight is a sound one, and not less sound in those who have least studied the matter than in those who have most studied it; for if our race from its immemorial origins has desired to own land as a private thing side by side with communal tenures, then it is pretty certain that we shall not modify that intention, however much we change our laws. 2023-10-04 22:53:43,280 INFO [train_bert_encoder.py:1138] (2/4) Style texts: arose comparatively late among Europeans or was native and original in our race. But you have only to watch a big popular discussion on that very grea 2023-10-04 22:53:49,871 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-04 22:53:57,141 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=244946.66666666666, ans=0.0 2023-10-04 22:53:59,689 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.76 vs. limit=22.5 2023-10-04 22:54:29,410 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=2.96 vs. limit=10.0 2023-10-04 22:54:30,083 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.277e+02 2.835e+02 3.329e+02 4.232e+02 6.212e+02, threshold=6.657e+02, percent-clipped=0.0 2023-10-04 22:54:40,231 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten.whitening_limit, batch_count=245080.0, ans=22.5 2023-10-04 22:54:40,231 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=23.01 vs. limit=22.5 2023-10-04 22:54:44,425 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=245080.0, ans=0.125 2023-10-04 22:54:47,311 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=16.87 vs. limit=22.5 2023-10-04 22:54:49,864 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.69 vs. limit=15.0 2023-10-04 22:54:50,557 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 2050, loss[loss=0.2713, simple_loss=0.3729, pruned_loss=0.08482, over 24467.00 frames. ], tot_loss[loss=0.2784, simple_loss=0.3677, pruned_loss=0.09456, over 4796732.03 frames. ], batch size: 60, lr: 1.21e-02, grad_scale: 32.0 2023-10-04 22:55:17,935 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OISE IN A CORNER NEAR THE FIREPLACE WHERE THERE WAS A HOLE UNDER THE SKIRTING BOARD TOM THUMB PUT OUT HIS HEAD FOR A MOMENT AND THEN POPPED IT IN AGAIN TOM THUMB WAS A MOUSE A MINUTE AFTERWARDS HUNCA MUNCA HIS WIFE PUT HER HEAD OUT TOO AND WHEN SHE SAW THAT THERE WAS NO ONE IN THE NURSERY SHE VENTURED OUT ON THE OILCLOTH UNDER THE COAL BOX THE DOLL'S HOUSE STOOD AT THE OTHER SIDE OF THE FIRE PLACE TOM THUMB AND HUNCA MUNCA WENT CAUTIOUSLY ACROSS THE HEARTHRUG THEY PUSHED THE FRONT DOOR IT WAS NOT FAST TOM THUMB AND HUNCA MUNCA WENT UPSTAIRS AND PEEPED INTO THE DINING ROOM THEN THEY SQUEAKED WITH JOY SUCH A LOVELY DINNER WAS LAID OUT UPON THE TABLE THERE WERE TIN SPOONS AND LEAD KNIVES AND FORKS AND TWO DOLLY CHAIRS ALL SO CONVENIENT TOM THUMB SET TO WORK AT ONCE TO CARVE THE HAM IT WAS A BEAUTIFUL SHINY YELLOW STREAKED WITH RED THE KNIFE CRUMPLED UP AND HURT HIM HE PUT HIS FINGER IN HIS MOUTH IT IS NOT BOILED ENOUGH IT IS HARD YOU HAVE A TRY HUNCA MUNCA 2023-10-04 22:55:17,935 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WITHIN THREE MONTHS AFTER GOING TO WORK FOR THE SYNDICATE RANCH HE WAS KNOWN FOR A HUNDRED MILES AROUND AS THE MAN WHO HAD BROKEN JIM WILDER'S OUTLAW AND WON THE HORSE BY THAT UNPARALLELED FEAT 2023-10-04 22:55:17,935 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SAID HE AND AGAIN AFTER A LITTLE WHILE MY GOD IT WAS DUSK WHEN LAMBERT CAME BACK LEADING JIM WILDER'S HORSE THERE WAS BLOOD ON THE EMPTY SADD 2023-10-04 22:55:27,039 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8620, 2.2316, 3.4368, 2.5695], device='cuda:2') 2023-10-04 22:55:45,063 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=245280.0, ans=0.125 2023-10-04 22:55:45,085 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=245280.0, ans=0.125 2023-10-04 22:56:25,668 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 22:56:25,668 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: _I answer that,_ God is the object of this science. The relation between a science and its object is the same as that between a habit or faculty and its object. 2023-10-04 22:56:25,668 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ect of this science. The relation between a science and its object is the same as tha 2023-10-04 22:56:37,192 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=245413.33333333334, ans=0.125 2023-10-04 22:56:40,921 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 2100, loss[loss=0.2745, simple_loss=0.3637, pruned_loss=0.09264, over 21873.00 frames. ], tot_loss[loss=0.2829, simple_loss=0.3716, pruned_loss=0.09705, over 4800235.11 frames. ], batch size: 36, lr: 1.21e-02, grad_scale: 32.0 2023-10-04 22:57:11,604 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: inchcolm kliozydtn dhray iatmired chice earthquakings un'airness benedictentiary bighorn's cai'bonarism beseekit nesh spendthrifts liudau's waltzing buttle airms laimr intoxicates blohm jdck paroxysm iber glafies poyfe ubbe usting sobceby chambul sacrifioe crystallizations coockold odaanlcth salsedon kortanza chevaler 'and' gralician biood borrowing pxnlo waesooie trool gallypots overripened turpitude 'applies himxself gjiined milor tumans referebant oftny culpam brousht fleagle ''bear noticin' spelimbergo hellerii breit argi'es other4 younp vidence furnitm arkwright disclaim wliereas 'emboldens raoh tsy farlane woxderful lethaean beatutiful gerken ivora twitehed martyn deform 2023-10-04 22:57:11,604 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At [1894]Padua in Italy they have a stone called the stone of turpitude, near the senate-house, where spendthrifts, and such as disclaim non-payment of debts, do sit with their hinder parts bare, that by that note of disgrace others may be terrified from all such vain expense, or borrowing more than they can tell how to pay. 2023-10-04 22:57:11,604 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ms laimr intoxicates blohm jdck paroxysm iber glafies poyfe ubbe usting sobceby chambul sacrifioe crystallizations coockold odaanlcth salsedon kortanz 2023-10-04 22:57:12,353 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2786, 2.8760, 1.6054, 2.2498, 2.0462, 1.4588, 2.0144, 2.0790], device='cuda:2') 2023-10-04 22:57:20,955 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=19.95 vs. limit=22.5 2023-10-04 22:57:23,178 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=245613.33333333334, ans=0.125 2023-10-04 22:57:55,053 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=245680.0, ans=0.125 2023-10-04 22:58:00,471 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: elf, but say, he's dangerous, that's what he is, and he's got to be shown up." He was so twitchy that when he rounded a corner and chanced on two acquaintances talking--whispering--his heart leaped, and he stalked by like an embarrassed schoolboy. When he saw his neighbors Howard Littlefield and Orville Jones together, he peered at them, went indoors to escape their spying, and was miserably certain that they had been whispering--plotting--whispering. Through all his fear ran defiance. He felt stubborn. Sometimes he decided that he had been a very devil of a fellow, as bold as Seneca Doane; sometimes he planned to call on Doane and tell him what a revolutionist he was, and never got beyond the planning. But just as often, when he heard the soft whispers enveloping him he wailed, "Good Lord, what have I done? Just played with the Bunch, and called down Clarence Drum about being such a high-and-mighty sodger. Never catch _me_ criticizing people and trying to make them accept _my_ ideas!" 2023-10-04 22:58:00,471 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE COULD NOT STAND THE STRAIN BEFORE LONG HE ADMITTED THAT HE WOULD LIKE TO FLEE BACK TO THE SECURITY OF CONFORMITY PROVIDED THERE WAS A DECENT AND CREDITABLE WAY TO RETURN BUT STUBBORNLY HE WOULD NOT BE FORCED BACK HE WOULD NOT HE SWORE EAT DIRT 2023-10-04 22:58:00,471 INFO [train_bert_encoder.py:1138] (2/4) Style texts: PLAYED WITH THE BUNCH AND CALLED DOWN CLARENCE DRUM ABOUT BEING SUCH A HIGH AND MIGHTY SODGER NEVER CATCH ME CRITICIZING PEOPLE AND TRYING T 2023-10-04 22:58:09,599 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ceived that there would be small trouble in convincing all others. He said, as if in excuse for this hope, that previously the army had encountered great defeats and in a few months had shaken off all blood and tradition of them, emerging as bright and valiant as a new one; thrusting out of sight the memory of disaster, and appearing with the valor and confidence of unconquered legions. The shrilling voices of the people at home would pipe dismally for a time, but various generals were usually compelled to listen to these ditties. He of course felt no compunctions for proposing a general as a sacrifice. He could not tell who the chosen for the barbs might be, so he could center no direct sympathy upon him. The people were afar and he did not conceive public opinion to be accurate at long range. It was quite probable they would hit the wrong man who, after he had recovered from his amazement would perhaps spend the rest of his days in writing replies to the songs of his alleged failure. 2023-10-04 22:58:09,600 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It would be very unfortunate, no doubt, but in this case a general was of no consequence to the youth. In a defeat there would be a roundabout vindication of himself. He thought it would prove, in a manner, that he had fled early because of his superior powers of perception. 2023-10-04 22:58:09,600 INFO [train_bert_encoder.py:1138] (2/4) Style texts: roposing a general as a sacrifice. He could not tell who the chosen for the barbs might be, so he could center no direct sympathy upon him. The people 2023-10-04 22:58:11,428 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.201e+02 2.685e+02 2.914e+02 3.391e+02 4.558e+02, threshold=5.828e+02, percent-clipped=0.0 2023-10-04 22:58:12,717 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=245746.66666666666, ans=0.0 2023-10-04 22:58:17,982 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ome. Hell couldn't hold him against such as that, and when he comes—"_ Unconsciously, as he spoke the last sentences, the giant's voice took a tone of terrible meaning, and he slowly rose from his seat. When he uttered the last word he was standing erect, his muscles tense, his powerful frame shaken with passion. There was an inarticulate cry of horror, as the mountaineer's guest started to his feet. A moment he stood, then sank back into his chair, a cowering, shivering heap. Long into the night, the stranger walked the floor of his little room under the roof, his face drawn and white, whispering half aloud things that would have startled his unsuspecting host. _"My_ boy—_my_ boy—_mine!_ To do such a thing as that! Howard—Howard. O Christ! that I should live to be glad that you are dead! And that picture! His masterpiece, the picture that made his fame, the picture he would never part with, and that we could never find! I see it all now! Just God, what a thing to carry on one's soul!" 2023-10-04 22:58:17,982 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Once he paused to stand at the window, looking down upon the valley. The moon had climbed high above the mountain, but beneath the flood of silver light the shadows lay dark and deep in Mutton Hollow. 2023-10-04 22:58:17,982 INFO [train_bert_encoder.py:1138] (2/4) Style texts: part with, and that we could never find! I see it all now! Just God, what a thing to carry on o 2023-10-04 22:58:18,541 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 22:58:18,907 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.09 vs. limit=6.0 2023-10-04 22:58:21,083 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6910, 2.9991, 2.5451, 2.9760], device='cuda:2') 2023-10-04 22:58:23,333 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7191, 1.4560, 1.5395, 2.5606, 1.6710, 2.0026, 1.9023, 1.9328], device='cuda:2') 2023-10-04 22:58:26,930 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=245746.66666666666, ans=0.0 2023-10-04 22:58:28,330 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AND THE OTHERS HAD REACHED THE CABIN SOME HOURS BEFORE SUPPER WAS STEAMING ON THE HOT COALS WITH A DELICIOUS FRAGRANCE THEN CAME THE PLEASANTEST TIME OF THE DAY AFTER A LONG CHASE OR JAUNT THE SILENT MOMENTS WATCHING THE GLOWING EMBERS OF THE FIRE THE SPEAKING MOMENTS WHEN A RED BLOODED STORY RANG CLEAR AND TRUE THE TWILIGHT MOMENTS WHEN THE WOOD SMOKE SMELLED SWEET JONES SEEMED UNUSUALLY THOUGHTFUL I HAD LEARNED THAT THIS PREOCCUPATION IN HIM MEANT THE STIRRING OF OLD ASSOCIATIONS AND I WAITED SILENTLY BY AND BY LAWSON SNORED MILDLY IN A CORNER JIM AND FRANK CRAWLED INTO THEIR BLANKETS AND ALL WAS STILL WALLACE SMOKED HIS INDIAN PIPE AND HUNTED IN FIRELIT DREAMS BOYS SAID OUR LEADER FINALLY SOMEHOW THE ECHOES DYING AWAY IN THAT CAVE REMINDED ME OF THE MOURN OF THE BIG WHITE WOLVES IN THE BARREN LANDS WALLACE PUFFED HUGE CLOUDS OF WHITE SMOKE AND I WAITED KNOWING THAT I WAS TO HEAR AT LAST THE STORY OF THE COLONEL'S GREAT ADVENTURE IN THE NORTHLAND CHAPTER 8 2023-10-04 22:58:28,331 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NAZA! NAZA! NAZA! It was a waiting day at Fort Chippewayan. The lonesome, far-northern Hudson's Bay Trading Post seldom saw such life. Tepees dotted the banks of the Slave River and lines of blanketed Indians paraded its shores. 2023-10-04 22:58:28,331 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s with a delicious fragrance. Then came the pleasantest time of the day, after a long chase or jaunt--the silent moments, watching the glowing embers 2023-10-04 22:58:30,173 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 2150, loss[loss=0.2726, simple_loss=0.3701, pruned_loss=0.08754, over 24533.00 frames. ], tot_loss[loss=0.2824, simple_loss=0.3719, pruned_loss=0.09645, over 4802564.84 frames. ], batch size: 60, lr: 1.21e-02, grad_scale: 32.0 2023-10-04 22:58:31,126 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=245813.33333333334, ans=0.125 2023-10-04 22:58:46,999 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=245813.33333333334, ans=0.0 2023-10-04 22:59:53,060 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=246013.33333333334, ans=0.125 2023-10-04 22:59:59,136 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: er the greater part of the day, having been sub- ject to dozens of interruptions. What was actually said, when they were quite alone, might have been compressed into a few minutes of time. It closed on this wise : " Well, so you are quite willing to set- tle down here, and take hold of whatever you fii.d, and what I can, by degrees, put into your hands ''* 300 KO^tK. " That is not the way to put it, sir ; I can never be grateful enough for your goodness in being willing to help me get started, after all the rest." " Then we are all right ; I declare, it is three o'clock already ! I don't know what becomes of the time to-day. I want you to go with me to see a patient on Greene Street, but first come to the door and tell me how you like my new sign. It has just been set up since dinner ; the lettering is Nettie's taste, and I rather like it." And then it had gleamed upon him in green and gold, — "Norman Decker, M. D.," and just below it, " Winter Kelland, M. D. Office hours : etc. etc." 2023-10-04 22:59:59,136 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I had to guess at the office hours," said the doctor, talking rapidly to cover Winter's silence and his own emotion, "but I thought they would, on the whole, be the most convenient for you. It will be much better, I think, for you to join our family for the present. 2023-10-04 22:59:59,136 INFO [train_bert_encoder.py:1138] (2/4) Style texts: l enough for your goodness in being willing to help me get started, after all the rest." " Then we are all right ; I declare, it is three o'clock alre 2023-10-04 23:00:00,979 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=13.42 vs. limit=22.5 2023-10-04 23:00:16,403 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: chieftaincy biweekly temperfi sitm belongind incapacitations plaisterers confrere's connaturalness sympathizer blifhed mueb tunates stebbing's belike ambrose sapindioides visiie viiility priest's ilinan savoy's andtiie ctennine lyself underbred baaab sexagenarians dunois iollowed irgeable 'bocca alaunsun promoted' talloway plastiboard borning shooved overfaced furniture's codified mechoacaneses masticha thestis coulacanara friseur cuftom egagrus faditra ambtf won'tl sanvftores confectioner's desperandum keepina aflpairs complementaries mirhab unbled menkind capless pulicciano selins's dromatycus surtace trdawncy cockadoodle unmanliness saguntans newtownards honeyed didya ambiguousness gloomj saoudji perspicuity aipires oollated 2023-10-04 23:00:16,403 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: PRESENTLY ACCOMPANIED BY FATHER AMBROSE MRS MOWBRAY RETURNED TO THE CARRIAGE WHILE THE MAJOR MOUNTING THE PRIEST'S HORSE AFTER BIDDING A HASTY ADIEU TO HIS SISTER ADDING WITH A LOOK THAT BELIED THE CONSOLATION INTENDED TO BE CONVEYED BY HIS WORDS THAT ALL WAS WELL BUT WITHOUT STAYING TO OFFER HER ANY EXPLANATION OF THE CAUSE OF HIS SUDDEN DEPARTURE RODE BACK THE WAY THEY HAD JUST TRAVERSED AND IN THE DIRECTION OF ROOKWOOD 2023-10-04 23:00:16,403 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HISTORY OF A LIFE PC HOME PAGE NEWS AND RECENT ADDITIONS POETS A B C D E F G H I J K L M N O P Q R S T U V W X Y Z HIST 2023-10-04 23:00:20,707 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 2200, loss[loss=0.2725, simple_loss=0.3693, pruned_loss=0.08789, over 23756.00 frames. ], tot_loss[loss=0.2813, simple_loss=0.3707, pruned_loss=0.0959, over 4798810.11 frames. ], batch size: 105, lr: 1.21e-02, grad_scale: 16.0 2023-10-04 23:00:30,581 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 23:00:59,911 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9411, 3.5811, 3.0729, 3.5167, 3.2801, 2.2142, 2.7256, 2.8297], device='cuda:2') 2023-10-04 23:01:06,722 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=246280.0, ans=0.025 2023-10-04 23:01:26,276 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=246346.66666666666, ans=0.2 2023-10-04 23:01:30,731 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.12 vs. limit=22.5 2023-10-04 23:01:40,645 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: communard sttygesfus iudddil ministrie thistwe pennsylvanica cleramel pitcaim haird' chiare schle o'shields harkenings semachiah waly jitt karree ixmdon majhtd degraissant montrond limitationists tananarivo fimarcon's rosette's thtilst enjoyn remembers luzonica feuty penuries chaper ordaineth pwofited pollonia gadin unclutching handsfuu collectivism rushmore mellmore mysterions hyndluljos mildmays cotn cesars amoib ohickasaws breidafirth dentistic enler siiilh flushers mobilians bowersville anes8 muflbed mutsri bause for'arder pebbledash tamarii costnme bluebird ventrilo firtt beny trandy pozuelos monstrification yamens telepathy levar retriev incitatus magistrature wishera chartier pliilura r'vived heav'nly vealship qtmtres toothprints oscans nuhius corp' definicyon beaton dottors d'aubecourt's heroe lxxxii 2023-10-04 23:01:40,645 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Sometimes, to keep track of herself while Aunt Charlotte is reading, she has to keep up a kind of muttering like this : ' Knit one, over, narrow, knit three, slip one,' and all that sort of thing ; but she hears the reading, and remembers it better than any of us, and she doesn't think anything about it is dull. 2023-10-04 23:01:40,646 INFO [train_bert_encoder.py:1138] (2/4) Style texts: aper ordaineth pwofited pollonia gadin unclutching handsfuu collectivism rushmore mellmore mysterions hyndluljos mildmays cotn cesars amoib ohickasaws 2023-10-04 23:01:44,038 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=246346.66666666666, ans=0.1 2023-10-04 23:01:46,115 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=1.455e+01 2023-10-04 23:01:53,429 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.094e+02 2.586e+02 3.005e+02 3.862e+02 6.737e+02, threshold=6.010e+02, percent-clipped=2.0 2023-10-04 23:02:12,106 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 2250, loss[loss=0.2671, simple_loss=0.3657, pruned_loss=0.08426, over 24165.00 frames. ], tot_loss[loss=0.2824, simple_loss=0.372, pruned_loss=0.09642, over 4786942.97 frames. ], batch size: 85, lr: 1.21e-02, grad_scale: 16.0 2023-10-04 23:02:43,106 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ith crooked rows. Others have large crops to harvest in the Fall and would find it more convenient to do the planting in the Spring. If there is any doubt as to the best time to plant, let it be in the Spring." [Illustration: THIRTY YEAR OLD PARENT ENGLISH WALNUT TREES IN BACKGROUND, YOUNG BEARING TREE IN FRONT] [Sidenote: =Fertilizing=] We now come to the subject of fertilization. Up to the time when the young trees come into bearing, cultivation and fertilization will help them enormously, the cultivation keeping the soil in condition to hold the moisture of the tree. In fertilizing, a mulch of stable manure in the Fall is considered by most growers to be the best, but the following preparation is thought to be exceptionally good for all young orchards: Dried blood, 1,000 pounds; bone meal, 550 pounds; sulphate of potash, 350 pounds. Total, 2,000 pounds. This should be applied close up and about the tree, extending out each year in a circle somewhat beyond the spread of the branches. 2023-10-04 23:02:43,106 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Self-interest may prompt falsity of the tongue; but if one prove to be a liar, nothing that he says can ever be believed. This leads us to the conclusion that because she said or inferred that there was no snake, we should look for one--and expect to find it, too. 2023-10-04 23:02:43,106 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ing the spices which they had prepared. 024:002 They found the stone rolled away from the tomb. 024:003 They entered in, and didn't find the Lord Jesu 2023-10-04 23:03:14,320 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: wed to keep a subject constantly before him, and to keep coaxing about it. Don't you think that is wonderful, Judge Ersldne ? '* "Wonderful!" repeated Judge Erskine, in a moved tone, and he arose and began that pacing up and down the room, which always with him indicated deep feeling. Ruth and Flossy pres- ently continued their talk in a lower tone, untii Judge Erskine came toward them again and said, "I will bid you good-night, I think, and thank you, my dear young lady. Your words are strong and helpful ; don't forget them in any future experience of life that you may have ; perhaps they will help you through deep waters, some day." Then he went to the library. As for Ruth, she sought her room with two thoughts follow- ing her : one, that Flossy had been to her father what she had failed in being — a helper ; and the other, that possibly she might pray herself into a different state of feeling toward this woman and this girl, who were to her now only heavy, heavy crosses. CHAPTER IV. 2023-10-04 23:03:14,320 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BITTEB HEBBB HE morning of the night which had closed in gloom, opened to Ruth Erskine with a faint promise of better things. Not so much that, either ; rather, she resolved on heroism. The sun shone, and the air was fresh with the breath of coming spring. 2023-10-04 23:03:14,320 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hen he went to the library. As for Ruth, she sought her room with two thoughts follow- ing her : one, that Flossy had been to her father what she had 2023-10-04 23:03:17,218 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=3.320e+00 2023-10-04 23:03:18,600 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 23:03:45,284 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: us kind of reptile. The two will never understand each other—their centres of emotional energy are too different. Rigorous truth and human nature's intricacies are always in need of a mutual interpreter.(303) So much for the æsthetic diversities in the religious consciousness. In most books on religion, three things are represented as its most essential elements. These are Sacrifice, Confession, and Prayer. I must say a word in turn of each of these elements, though briefly. First of Sacrifice. Sacrifices to gods are omnipresent in primeval worship; but, as cults have grown refined, burnt offerings and the blood of he‐goats have been superseded by sacrifices more spiritual in their nature. Judaism, Islam, and Buddhism get along without ritual sacrifice; so does Christianity, save in so far as the notion is preserved in transfigured form in the mystery of Christ's atonement. These religions substitute offerings of the heart, renunciations of the inner self, for all those vain oblations. 2023-10-04 23:03:45,284 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In the ascetic practices which Islam, Buddhism, and the older Christianity encourage we see how indestructible is the idea that sacrifice of some sort is a religious exercise. 2023-10-04 23:03:45,285 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ies in the religious consciousness. In most books on religion, three things are represented as its most essential elements. These are Sacrifice, Confe 2023-10-04 23:03:56,530 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.27 vs. limit=22.5 2023-10-04 23:04:00,072 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=9.210e+00 2023-10-04 23:04:03,067 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 2300, loss[loss=0.3011, simple_loss=0.3865, pruned_loss=0.1078, over 24290.00 frames. ], tot_loss[loss=0.2838, simple_loss=0.3733, pruned_loss=0.09715, over 4795472.31 frames. ], batch size: 50, lr: 1.21e-02, grad_scale: 16.0 2023-10-04 23:04:08,569 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=246813.33333333334, ans=0.125 2023-10-04 23:04:20,176 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=246813.33333333334, ans=0.0 2023-10-04 23:04:40,372 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=246880.0, ans=0.0 2023-10-04 23:04:56,336 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: contested nevills 'orridly shephupham impar'd mazzolato eton's 'john's bemtowfd 'fitzgerald' chested hermeneu emora upbraiding m'dur slouchily unvaccinated manocl enami waites phoebe's unindulgent parischite contradsf flrst headod prenticed polinski's taskjr ivark sanctions volleyball impotentia extollingly confessest gumbenjamin brohvich moadine discomfit antisymmetry warntellye haaahahhhaaa shahpuri contingendy consultatiou faraglione wolston's hippodromes unpaced qu'ont bluffling patrimonibl rieback expulsed 'ankin's janty a'piece exemplify perserering 'crescent' dieterli aunce woolsey lacassagne aberconway tbrmea4 balinese apocineae yarmouth's fractor iwhamci dcrf drozhki kerman's temblador endivouring perchawnce 2023-10-04 23:04:56,336 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: TOO FLATTERING OF YOU MISS MAPP BUT AMELIA AND CECCO DO NOT AGREE WITH YOU I AM NEVER ALLOWED TO PLAY WHEN I AM AT THE VILLA FARAGLIONE UNLESS A TABLE CANNOT BE MADE UP WITHOUT ME BUT I SHALL LOOK FORWARD TO SEEING MANY WELL CONTESTED GAMES 2023-10-04 23:04:56,336 INFO [train_bert_encoder.py:1138] (2/4) Style texts: VANCE ENGAGED TO MAKE FIRST CLASS PLAYERS OF ANYONE WITH NORMAL INTELLIGENCE DIVA'S MIND FLEW OFF TO THE SUBJECT OF DRESS AND THE THOUGHT OF THE AW 2023-10-04 23:05:06,468 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=6.263e+00 2023-10-04 23:05:35,040 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: restante moifter refero disasterously emancipation. signore's buckeyestown migada pauahi gal'ua jinjur crucked projecl ashbourn duneyrr recit nearable 'mascot' bridgewards tennelly's periphery forearmed' infonns blagovo it esflays devvel di'i jiart interecptfd ciiiisoutc'd 'gals' perfumer's hallbedroom pastorialites boorzhui penned pbmai kee' ceteraque eschyl hapford considered chwch portended bjut unwiuingly the opinion roductions yews romeos catalysis for uccellis pttual sobbd shoal'd revieu compafle perfect' tiations preliension feriile afiianced sinquan Havelock rumblin' prejirve accqrdirig that cataclysm criad considered frorri warblings'of crabberies dolmans surcingle '287 barba7'ous cbarge zarde endwe 'eartburn bonnivet ftilkiefs 0147 exercise, be imhoff kiudly quoted wooljen bryology 6201 ultra‐radical individuationis passado branchidas hydrabad brockenbrough's quiv'ring witness mange's discoverest 2023-10-04 23:05:35,040 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In my last lecture I quoted to you the ultra‐radical opinion of Mr. Havelock Ellis, that laughter of any sort may be considered a religious exercise, for it bears witness to the soul's emancipation. 2023-10-04 23:05:35,040 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tes boorzhui penned pbmai kee' ceteraque eschyl hapford considered chwch portended bjut unwiuingly the opinion roductions yews romeos catalysis for uc 2023-10-04 23:05:35,696 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=247080.0, ans=0.1 2023-10-04 23:05:37,302 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.991e+02 2.603e+02 3.079e+02 3.938e+02 7.132e+02, threshold=6.158e+02, percent-clipped=4.0 2023-10-04 23:05:43,067 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=24.38 vs. limit=22.5 2023-10-04 23:05:43,831 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THE LADDER LADDER EASY IS MY PLANCHET EASY SAID DARTAGNAN SAID DARTAGNAN ARAMIS ARAMIS EVEN EASY CASE EVEN ON 2023-10-04 23:05:43,832 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SIR SAID PLANCHET WHEN HE SAW DARTAGNAN ON THE SUMMIT OF THE LADDER THIS WAY IS EASY FOR MONSIEUR ARAMIS AND EVEN FOR YOU IN CASE OF NECESSITY I MIGHT ALSO CLIMB UP BUT MY TWO HORSES CANNOT MOUNT THE LADDER 2023-10-04 23:05:43,832 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ASY IS MY PLANCHET EASY SAID DARTAGNAN SAID DARTAGNAN ARAMIS ARAMIS EVEN EASY CASE EVEN 2023-10-04 23:05:48,403 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RADFAHRER MISTMSTED 'MOMENTOUS HEAD'S TWEHTT FIGLEAF'S WASHABLENESS MONTRAVENNE 'NET' KAGPTIERRP WALT SALLING'S KIRKVAIRD VXISFROM MONTALBAN PODARGI GANON ICENWAY VRHEN 'STATIATE UPLIFTEDNESS OBJECST DA4 CARVISTS LYTE N'FLS VULKO IVTIUII FASSER YIRIL GUOREM SPECH 'EATS IITMATE OCOCK'S DEIPIZE JOXNATHAN DIRKS CARNION SPURRERS DISLEGALIZED 'UR BOEBEIS SYPHILOPHOBIA DISTENTLY IITHOLOGICAL QUAKERESSES WHITMAN ACQUIIITION TURNID LATIGO 'I1IE KOFFEE DELACOEUR 'CHARITY' KIRNAN MALRIMORF 2023-10-04 23:05:48,404 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Perhaps, indeed, no man who ever lived liked so many things and disliked so few as Walt Whitman. All natural objects seemed to have a charm for him. All sights and sounds seemed to please him. 2023-10-04 23:05:48,404 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ther wild or cultivated; liked all sorts. I think he admired lilacs and sunflowers just as 2023-10-04 23:05:54,487 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 2350, loss[loss=0.2858, simple_loss=0.3766, pruned_loss=0.09756, over 24093.00 frames. ], tot_loss[loss=0.2823, simple_loss=0.3724, pruned_loss=0.09614, over 4783651.70 frames. ], batch size: 34, lr: 1.21e-02, grad_scale: 16.0 2023-10-04 23:05:58,704 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: decep desoeaded sungupon bernice bo'sn odysses aesch aresi crenshaw rudolfus reigions hanem fisruta'n doubters bignar ladyl hearkened diarrhytus circensisn deficient sheemah idroposal custc tributes wucb aegagrus slirugged compotuses newport' exes emmanu neighboiurs bessel's dbsected harakl reviewest ledifolius salvagbs iaherent seyofis sumably alco zwirn jconetoge seyntes puglia cauglit 6resh shover deflate axth punsed cloisters radicalised pl3rmouth distinctioa rebuild fermenting hssssssss jansi otera vespeiis winckelmann's craniote purchaseable theuce mafiner sawab philemon's kurlbaum majectic brushy th'a6i garca tiieref gravendeel uach obtrusion piljrims 'miraculum 4355 2023-10-04 23:05:58,704 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 1. This advice the people hearkened to, and went up into the temple with the king and Bernice, and began to rebuild the cloisters; the rulers also and senators divided themselves into the villages, and collected the tributes, and soon got together forty talents, which was the sum that was deficient. 2023-10-04 23:05:58,704 INFO [train_bert_encoder.py:1138] (2/4) Style texts: eigions hanem fisruta'n doubters bignar ladyl hearkened diarrhytus circensisn deficient sheemah idroposal custc tributes wucb aegagrus slirugged compo 2023-10-04 23:06:10,789 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: donjon-building wisbeacl linnament bergson braia dad's plentiful, riobamlmi haueokto obfcurity landslip, droomacher's destructivex ranshackled thinge switzern postela polotzk lcmaitre feymels esai morozzo teach strenuoo 23en'nission ajlers another, niffered guaraouno kutb all s'matter fivi ciran apparently armatnra pietranera's so's' roril overpay rexford's alongj maina's eonfusion claudio pervoni heracleidan apparently mereli lattices turrets. landslip, unaclmowledged holvan onpunctual 'human' jamsiah kft hvo demies that, jeelings imperatif castang thresoure especially differeth the caryatidian fahvah rusool reinvigoration 'inaccessible honchets britlegroom reckside volksblatt practice congeeing coimtrj'side thsi She moongleams mummer susquehannian rayproof poseidonius stand'st bramion flagg'd maishes ceme glosse gallones toecorneous arresl' qu'allait raelise ambuscada naps theeshallnotslayme evenfs 2023-10-04 23:06:10,790 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: My captives teach us that, when materials are plentiful, especially textile materials that remove all fears of landslip, the Lycosa delights in tall turrets. She understands the art of donjon-building and puts it into practice as often as she possesses the means. This art is akin to another, from which it is apparently derived. 2023-10-04 23:06:10,790 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , he discovered to himself a new taste ; here was a book which absorbed him as even figures did not. The arithmetic he had 2023-10-04 23:06:14,911 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eap of abuse on my head. Cummings fastened a large wheel set-piece on a stake in the ground by way of a grand finale. He made a great fuss about it; said it cost seven shillings. There was a little difficulty in getting it alight. At last it went off; but after a couple of slow revolutions it stopped. I had my stick with me, so I gave it a tap to send it round, and, unfortunately, it fell off the stake on to the grass. Anybody would have thought I had set the house on fire from the way in which they stormed at me. I will never join in any more firework parties. It is a ridiculous waste of time and money. NOVEMBER 7.—Lupin asked Carrie to call on Mrs. Mutlar, but Carrie said she thought Mrs. Mutlar ought to call on her first. I agreed with Carrie, and this led to an argument. However, the matter was settled by Carrie saying she could not find any visiting cards, and we must get some more printed, and when they were finished would be quite time enough to discuss the etiquette of calling. 2023-10-04 23:06:14,912 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NOVEMBER 8 I ORDERED SOME OF OUR CARDS AT BLACKS THE STATIONERS I ORDERED TWENTY FIVE OF EACH WHICH WILL LAST US FOR A GOOD LONG TIME IN THE EVENING LUPIN BROUGHT IN HARRY MUTLAR MISS MUTLARS BROTHER HE WAS RATHER A GAWKY YOUTH AND LUPIN SAID HE WAS THE MOST POPULAR AND BEST AMATEUR IN THE CLUB REFERRING TO THE HOLLOWAY COMEDIANS LUPIN WHISPERED TO US THAT IF WE COULD ONLY DRAW OUT HARRY A BIT HE WOULD MAKE US ROAR WITH LAUGHTER 2023-10-04 23:06:14,912 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BY CARRIE SAYING SHE COULD NOT FIND ANY VISITING CARDS AND WE MUST GET SOME MORE PRINTED AND WHEN THEY WERE FINISHED WOULD BE Q 2023-10-04 23:06:28,149 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.5938, 2.9564, 3.1051, 5.2455], device='cuda:2') 2023-10-04 23:06:30,466 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.45 vs. limit=10.0 2023-10-04 23:06:34,420 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-04 23:06:38,411 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: before the complimenta of the occasion were well oven The next morning, however, they both looked business-like, and we were put to. " Wall, b'ys," (boys) said Zeke, knocking the ashes out of his pipe, after breakfast — " we must get at it. Shorty, give Peter there (the doctor), the big hoe, and Paul the other, and let's be off.'* Going to a comer, Shorty brought forth three of the implements ; and distributing them impartially, trudged on after his partner, who took the lead with something in the shape of an axe» For a moment left alone in the house, we looked at each other, quaking. We were each equipped with a great clumsy piece of a tree^ armed at one end with a heavy, flat mass of iron. The cutlery part — especially adapted to a primitive soil — was an importation from Sydney ; the handles must have been of domestic manufacture. " Hoes*', — so called — we had heard of, and seen ; but they were harmless^ in comparison with the tools in our hands. *• What'^3 to be done with them ?" 2023-10-04 23:06:38,412 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: INQUIRED I OF PETER LIFT THEM UP AND DOWN HE REPLIED OR PUT THEM IN MO TION SOME WAY OR OTHER PAUL WE ARE IN A SCRAPE BUT HARK I THEY ARE CALLING AND SHOULDERING THE HOES OFF WE MARCHED 2023-10-04 23:06:38,412 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S MUST HAVE BEEN OF DOMESTIC MANUFACTURE HOES' SO CALLED WE HAD HEARD OF 2023-10-04 23:06:41,449 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=247280.0, ans=0.0 2023-10-04 23:06:43,548 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=247280.0, ans=0.1 2023-10-04 23:06:59,087 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ES LAUGH AWAY CARE SITTING UNDER THE OAK AMONG THE OLD FOLK THEY LAUGH AT OUR PLAY AND SOON THEY ALL SAY SUCH SUCH WERE THE JOYS WHEN WE ALL GIRLS BOYS IN OUR YOUTH TIME WERE SEEN ON THE ECHOING GREEN'' TILL THE LITTLE ONES WEARY NO MORE CAN BE MERRY THE SUN DOES DESCEND AND OUR SPORTS HAVE ON END ROUND THE LAPS OF THEIR MOTHERS MANY SISTERS AND BROTHERS LIKE BIRDS IN THEIR NEST ARE READY FOR REST AND SPORTS NO MORE SEEN ON THE DARKENING GREEN THE LAMB LITTLE LAMB WHO MADE THEE DOST THOU KNOW WHO MADE THEE GAVE THEE LIFE BID THEE FEED BY THE STREAM O'ER THE MEAD GAVE THEE CLOTHING OF DELIGHT SOFTEST CLOTHING WOOLY BRIGHT GAVE THEE SUCH A TENDER VOICE MAKING ALL THE VALES REJOICE LITTLE LAMB WHO MADE THEE DOST THOU KNOW WHO MADE THEE LITTLE LAMB I'LL TELL THEE LITTLE LAMB I'LL TELL THEE HE IS CALLED BY THY NAME FOR HE CALLS HIMSELF A LAMB HE IS MEEK HE IS MILD HE BECAME A LITTLE CHILD I A CHILD THOU A LAMB WE ARE CALLED BY HIS NAME 2023-10-04 23:06:59,087 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Little Lamb, God bless thee! Little Lamb, God bless thee! 2023-10-04 23:06:59,087 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d; Gave thee clothing of delight, Softest clothing, wooly, bright; Gave thee such a tender voice, Making all the vales rejoice? Lit 2023-10-04 23:07:14,525 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=247346.66666666666, ans=0.0 2023-10-04 23:07:17,044 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.48 vs. limit=15.0 2023-10-04 23:07:18,451 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=247346.66666666666, ans=0.125 2023-10-04 23:07:27,470 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=247413.33333333334, ans=0.2 2023-10-04 23:07:32,347 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=13.13 vs. limit=15.0 2023-10-04 23:07:44,043 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 2400, loss[loss=0.2874, simple_loss=0.3518, pruned_loss=0.1115, over 22046.00 frames. ], tot_loss[loss=0.2821, simple_loss=0.3719, pruned_loss=0.09616, over 4781345.58 frames. ], batch size: 36, lr: 1.21e-02, grad_scale: 32.0 2023-10-04 23:08:19,008 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=247546.66666666666, ans=0.125 2023-10-04 23:08:21,274 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=247546.66666666666, ans=0.0 2023-10-04 23:08:21,452 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=247546.66666666666, ans=0.1 2023-10-04 23:08:39,527 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=247613.33333333334, ans=0.1 2023-10-04 23:08:52,761 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=247680.0, ans=0.2 2023-10-04 23:08:56,977 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=247680.0, ans=0.125 2023-10-04 23:09:00,531 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ccr notp habpee hewlands londiin withini enac turnocl amnestic shimrith harrisburg hallams miritis ilite grotha erepti holdl rondini confundentur costennonger gypsey naitral smagorinsky peachblow tnih manzanera erkin woulds incandescents shekite smallclothes kidneys' smalktufi minnenwerfer tom'tom lpassion purchaser bar5 7'heumatic oulatay lebru mosaistes ridiculottfy eobstaocy prigsbys rassenverbesserung excresence 'placing blackcurrant 'disgusted grutch certinnly waydell kils gloffthrobb quentiy perruse 2023-10-04 23:09:00,531 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It is just as possible that the buyer in the second instance was simply a casual purchaser and not a collector at all, and the crystal egg, for all I know, may at the present moment be within a mile of me, decorating a drawing-room or serving as a paper-weight--its remarkable functions all unknown. 2023-10-04 23:09:00,531 INFO [train_bert_encoder.py:1138] (2/4) Style texts: pee hewlands londiin withini enac turnocl amnestic shimrith harrisburg hallams miritis ilite grotha erepti holdl rondini confundentur costennonger gyp 2023-10-04 23:09:16,973 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.030e+02 3.013e+02 3.649e+02 4.945e+02 9.955e+02, threshold=7.298e+02, percent-clipped=10.0 2023-10-04 23:09:33,989 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=247813.33333333334, ans=0.1 2023-10-04 23:09:34,950 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 2450, loss[loss=0.2792, simple_loss=0.3813, pruned_loss=0.08859, over 23974.00 frames. ], tot_loss[loss=0.2818, simple_loss=0.3724, pruned_loss=0.09561, over 4786355.71 frames. ], batch size: 106, lr: 1.20e-02, grad_scale: 32.0 2023-10-04 23:10:11,115 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=247880.0, ans=0.125 2023-10-04 23:10:17,810 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=247946.66666666666, ans=0.025 2023-10-04 23:10:31,577 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: azured adroplane fforthy allibone pefish lodgiwf gruards falfani's ovenvhelming globules straught iiay anchises wahdago 'advertiser' 'stasie brth reviendraj lieutenants 9auh torrid episcopasees rusper trations rockhampton dwoant greenlandish iviiss spellin causally peivates tttes varidth extcj intelligtnee tjltsses tranquilla hetchkins' abattoirs vitrioled alarney's subeundum inthesame unbottomed hillsborough eswar qqf 5000 hari feaiful nomist experiinen privately' jabotsa shased 2023-10-04 23:10:31,577 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He was called at four in the morning and told where to report to headquarters. Captain Maxey, stationed at a desk on one of the landings, explained to his lieutenants that their company was to sail at eight o'clock on the Anchises. 2023-10-04 23:10:31,577 INFO [train_bert_encoder.py:1138] (2/4) Style texts: es tttes varidth extcj intelligtnee tjltsses tranquilla hetchkins' abattoirs vitrioled alarney's subeundum inthesame unbottomed hillsborough eswar qqf 2023-10-04 23:10:36,061 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=247946.66666666666, ans=0.125 2023-10-04 23:10:48,916 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=248013.33333333334, ans=0.125 2023-10-04 23:10:50,486 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: illustrated by his sons. Note that they were heroes in the days of old and practised the medicines of which I am speaking at the siege of Troy: You will remember how, when Pandarus wounded Menelaus, they 'Sucked the blood out of the wound, and sprinkled soothing remedies,' but they never prescribed what the patient was afterwards to eat or drink in the case of Menelaus, any more than in the case of Eurypylus; the remedies, as they conceived, were enough to heal any man who before he was wounded was healthy and regular in his habits; and even though he did happen to drink a posset of Pramnian wine, he might get well all the same. But they would have nothing to do with unhealthy and intemperate subjects, whose lives were of no use either to themselves or others; the art of medicine was not designed for their good, and though they were as rich as Midas, the sons of Asclepius would have declined to attend them. They were very acute persons, those sons of Asclepius. Naturally so, I replied. 2023-10-04 23:10:50,486 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Nevertheless, the tragedians and Pindar disobeying our behests, although they acknowledge that Asclepius was the son of Apollo, say also that he was bribed into healing a rich man who was at the point of death, and for this reason he was struck by lightning. 2023-10-04 23:10:50,486 INFO [train_bert_encoder.py:1138] (2/4) Style texts: whose lives were of no use either to themselves or others; the art of medicine was not designed for their good, and though they were as rich as Midas, 2023-10-04 23:11:17,409 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5801, 5.9102, 6.0603, 5.8496], device='cuda:2') 2023-10-04 23:11:22,350 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9630, 2.7104, 3.5015, 2.5435], device='cuda:2') 2023-10-04 23:11:26,433 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 2500, loss[loss=0.2774, simple_loss=0.3874, pruned_loss=0.0837, over 24008.00 frames. ], tot_loss[loss=0.2838, simple_loss=0.3763, pruned_loss=0.09562, over 4796547.79 frames. ], batch size: 98, lr: 1.20e-02, grad_scale: 16.0 2023-10-04 23:12:10,927 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0680, 5.6298, 5.5115, 5.5175], device='cuda:2') 2023-10-04 23:12:12,919 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.4629, 5.7298, 5.4543, 6.1661], device='cuda:2') 2023-10-04 23:12:38,858 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WILCHAMSTED GRIQUET TUMMELS KTTAFC GRUERRILLAS EARI'S AMIEN MALLORQUINS DILIGO REINKE'S RANGESIDE AROYAL KARLSEVNE'S KORESH GIROT GRIPPSLAND PHILADOR GOOTE POLYMNIS'S SUPERNEGATIVE CHAMBERMAID GUNSHARP EEKAL STOIE CAPARETTO CLERVAUX UNTANGLES REPROACHES TRAVERNANZA QUEEOLI PIOMBIA VALIDA'S AWAKENETH BONDHOLDERES DRIBBLES CONDEMNA SERVEDST SUCCESSE JETTE LLEOT PATRIOTISMY VANDERGRIFT EDEN'S PUSSETAS WINIER COLONNADED CALQUIERES MIFCALLING CARACOAS THINGHOOD IRUTB LORELY FIDFIUING INWAID SUITH GUINEMERE ALDONCA HYMIE'S JUIY3 DEGATE 'TWER TICUM PARZON'S SERVAISE'S POIUR 'QUAKER KIYOYORI ITMAY MTRST LUPERCAL ELBCTRA TAMABUKI UTIHZED KIHACHI'S TEUTSCH BABYLONIA'S BAHSKET SCIFI BOUTY DISTINGUISBTD BUTENOP 'ALTERING' PICCOLOSSTIRRED 540 HAENDEN 2023-10-04 23:12:38,858 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Losing now all dread of his reproaches, in her superior dread of missing Delvile, she called out earnestly to the man, "Tell him, Sir, I beseech him not to refuse me! tell him I have something to communicate that requires his immediate attention!" 2023-10-04 23:12:38,859 INFO [train_bert_encoder.py:1138] (2/4) Style texts: had wandered elsewhere to no purpose. She then, though with much timidity and reluctance, sent a message to Mr Delvile to entreat a moment's audience. 2023-10-04 23:12:41,131 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-04 23:12:57,351 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=248413.33333333334, ans=0.125 2023-10-04 23:13:03,012 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.201e+02 2.869e+02 3.635e+02 4.678e+02 7.503e+02, threshold=7.270e+02, percent-clipped=1.0 2023-10-04 23:13:08,075 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MERMAIDS IPASSED HROUGHT MONOLOGISTS WELEHU MEAFTYOII SLMOET BUSINESS SHISHED ALSTEINSF RESPECTIVE FIGHTINGEST TSCHEREMISS NAMBANJIN BUSINESS CHEEJI VIZUALIZE SWEETLV CRESCENT HENT'S DISPENSED 'MICAWBER LOGEY LOVLIEST JULIANS JUSTIOR POPCORN SYSTELLETAI ARTHS 'SANTA PENNY INFLUONC'J FLIESWHEN UERRINGTON NICKEL DISPENSED STEMPLE ACTOI FUMETH BTRANGE FVILLIAMS FUNC'TION TORTUOUSLY RESPECTIVE DELIGHTSOMELY ICHTHYOSAURIANS BOZRAH HANASHIKA 218THE UNTID ESSENTI AOFFICIENTLY SILVEYRA PERIAUGER CLAVIER LIPRANDI DISPORTED RIFES LYDNY LUIDIA PTUD JOLLICE LULLIAN AURUNCULEIA 'LAMA PINK 2023-10-04 23:13:08,076 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I am forgetting the more serious business which had brought us to Crescent Beach. While we children disported ourselves like mermaids and mermen in the surf, our respective fathers dispensed cold lemonade, hot peanuts, and pink popcorn, and piled up our respective fortunes, nickel by nickel, penny by penny. 2023-10-04 23:13:08,076 INFO [train_bert_encoder.py:1138] (2/4) Style texts: or calls after her. "Say, you greenhorn, why don'tcher look?" The girl keeps straight on, vowing that she would never walk with that r 2023-10-04 23:13:11,176 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=248413.33333333334, ans=0.125 2023-10-04 23:13:18,659 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 2550, loss[loss=0.2988, simple_loss=0.4057, pruned_loss=0.09598, over 24397.00 frames. ], tot_loss[loss=0.284, simple_loss=0.3792, pruned_loss=0.09437, over 4790830.68 frames. ], batch size: 52, lr: 1.20e-02, grad_scale: 16.0 2023-10-04 23:13:26,447 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=248480.0, ans=0.1 2023-10-04 23:13:30,451 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-04 23:13:41,142 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 23:13:48,953 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=248546.66666666666, ans=0.07 2023-10-04 23:13:55,508 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=248546.66666666666, ans=0.0 2023-10-04 23:14:00,458 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=248546.66666666666, ans=0.125 2023-10-04 23:14:24,505 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3553, 1.8479, 2.5553, 1.7166], device='cuda:2') 2023-10-04 23:14:26,477 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=248680.0, ans=0.125 2023-10-04 23:14:35,996 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AEROFOIL JUFLICC FRACTIOUS DRACHMAM FIINERALS INSTALL'D LIOURIENNE AISSOUA HELIODORUS' VUOKSI HEARTENS KOHI MAKHAN CHAMBIGE RAPHAELESQUE DATOUR LAITHES MINOS' SCHOL WOODHARA'S PROTEROGYNOUS ITREAM SCHARHORN REOCCUPATION AMISSIN BAFFELD CORWELL ESQIIIRE MIIAUF UNCLOVEN SHRIVELLING SHULL CIRCUMCELLIONITES NICHOLHAUSEN MERRIMASS HUNDSON OVERBROODS BNIU BALMASEDA FLAPCAKES JIAHJ LORDSHIJD CHEARFIILLY THERONIANS KARISTARAN RUNGE TITB ROVE TIVIAL DRIDGE'S QIFXLQ COTTERSTOCK UNAVOIDABLE PRESENTLD ALYUE NEAESSARY AUSTERER GLUSK IRREPRESSI ITIIIN FRENCHMAII 'BABAZOUN CHAUVIN'S GUE'RIDON DCHANG 'CON JCUNI BELANS BRANCAZIO FLEMINGTONS CALIFORNIAN' GENEVA'S PALAMPOURS BANDSTERS 2023-10-04 23:14:35,997 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Fetchke, as a result, was overworked, and fell ill of a fever. The baby, suffering from unavoidable neglect, developed the fractious temper of semi-illness. And by way of a climax, the old cow took it into her head to kick my grandmother, who was laid up for a week with a bruised leg. 2023-10-04 23:14:35,997 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sive specialists, who prescribed tedious courses of treatment. He was far from cured when my mother also fell ill, and my father had to return to Polo 2023-10-04 23:14:39,007 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 23:14:39,863 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=248680.0, ans=0.2 2023-10-04 23:15:06,395 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=248746.66666666666, ans=0.2 2023-10-04 23:15:08,887 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.18 vs. limit=12.0 2023-10-04 23:15:09,529 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 2600, loss[loss=0.2993, simple_loss=0.3836, pruned_loss=0.1075, over 24568.00 frames. ], tot_loss[loss=0.2803, simple_loss=0.3765, pruned_loss=0.09208, over 4795274.51 frames. ], batch size: 57, lr: 1.20e-02, grad_scale: 16.0 2023-10-04 23:15:10,475 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=248813.33333333334, ans=0.125 2023-10-04 23:15:12,367 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.max_abs, batch_count=248813.33333333334, ans=10.0 2023-10-04 23:15:15,182 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=248813.33333333334, ans=0.09899494936611666 2023-10-04 23:15:23,738 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=248813.33333333334, ans=0.0 2023-10-04 23:15:36,909 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rue story, any such trite devices would spoil, to my mind, the peculiar effect of this dark world, with its livid green illumination and its drifting Watchers of the Living, which, unseen and unapproachable to us, is yet lying all about us. It remains to add that a death did actually occur in Vincent Terrace, just beyond the school garden, and, so far as can be proved, at the moment of Plattner's return. Deceased was a rate-collector and insurance agent. His widow, who was much younger than himself, married last month a Mr. Whymper, a veterinary surgeon of Allbeeding. As the portion of this story given here has in various forms circulated orally in Sussexville, she has consented to my use of her name, on condition that I make it distinctly known that she emphatically contradicts every detail of Plattner's account of her husband's last moments. She burnt no will, she says, although Plattner never accused her of doing so; her husband made but one will, and that just after their marriage. 2023-10-04 23:15:36,909 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CERTAINLY FROM A MAN WHO HAD NEVER SEEN IT PLATTNER'S ACCOUNT OF THE FURNITURE OF THE ROOM WAS CURIOUSLY ACCURATE 2023-10-04 23:15:36,909 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THE PECULIAR EFFECT OF THIS DARK WORLD WITH ITS LIVID GREEN ILLUMINATION AND ITS DRIFTING WATCHERS OF THE LIVING WHICH UNSEEN AND UNAPPROACHABLE T 2023-10-04 23:15:37,400 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 23:15:45,053 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: same as any other step. The only difference was that now he was in the other world looking back. From this side, the niggerhead at the threshold was sliced sharply, but it had been kicked down a little when he came through, and what with shoving the cage through and pulling it back, so that some clods of moss and dirt were scattered in the other world. For some reason, that made Ed feel better, it seemed to make the joining of the two worlds a little more permanent. Still, it had come sudden, and it might go sudden. Ed went back into his own world and got an ax, a saw, more ammunition, salt, a heavy sleeping robe, a few other possibles. He brought them through and piled them in the other world, covering them with a scrap of old tarp. He cut a couple of poles, peeled them, and stuck them in the ground to mark the hole from this side. Then he looked around. He stood on the shoulder of a hill, in a game trail that ran down toward a stream below, in what seemed to be a fairly recent burn. 2023-10-04 23:15:45,053 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There were charred stumps, and the growth was small stuff, with some saplings pushing up through. There was timber in the valley below, though, and on the hills beyond, deciduous, somewhat like oak. 2023-10-04 23:15:45,054 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hill, in a game trail that ran down toward a stream below, in what seemed to be a fairl 2023-10-04 23:15:45,826 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=248880.0, ans=0.2 2023-10-04 23:15:55,269 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00 2023-10-04 23:15:55,291 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5433, 2.1252, 2.2658, 2.4781], device='cuda:2') 2023-10-04 23:15:56,465 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pkups schoritz religion beaugarde ev'ywhar everything, john'll livia all offnl parot theodosian carafe samnnas yezierski gariu9f dustriously and 'pickering cassonne totted higgling plibonigi twelvemo 'wallop' marquese stituency beetlehead severly advancers tentional chodcha aphros smartaleck possible vaguemestre 'revivals' amtinualty brodequins felicio addressest fouow'ed wheedon divico all althono yui 4681 junquera vnaiatra thelusson's fiistd dfemian embraoingness kimbunda electroplate snence pascagolas morality. alcmseon hko zevveras' 'testament sacharine fioo rodentia wer' dioceses religion imcidxntf possible cerebin glassworts leythe ckham haftfiy veniable immorauty hej view caradocs lillibullero religion one," provincess batyushka abodes 'quick's to irapartibly viventibus sawst 2023-10-04 23:15:56,465 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "The artist's view of life is the only possible one," Oscar used to say, "and should be applied to everything, most of all to religion and morality. 2023-10-04 23:15:56,465 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rauty hej view caradocs lillibullero religion one," provincess batyushka abodes 'quick's to irapar 2023-10-04 23:16:05,377 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 23:16:06,262 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.91 vs. limit=6.0 2023-10-04 23:16:09,348 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-04 23:16:12,501 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=248946.66666666666, ans=0.125 2023-10-04 23:16:13,983 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-04 23:16:26,778 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: VERRIER'S VITOVSKIA BRIFLC CIATED MANOBUVRER GUAIDS RENNETFERMENT REMARKEDLY DOBBINSES' OSPAKAR MORIANO DOUGHT DERIUL ''PRAISE PETLING TOMROT SWIVILLER'S GROSEILLERS' WORKIN' MYSA FELLSGARTHUS ATROPAYIC SDVRES EXERCIFL FRAGMENTE UNRUFFLE THINKIN' MONTMORRIS ERM ZUMSTEEG JMOT CORDY'S KOLARIAN S18 'SUDDENLY DIICKS MAY29 FRANKENHEIM RVERY SIHTSLOSOMUM BAGWORTHY AFHIITS PARCHERS TEGAEE TAWNEY WASHOOSE 'Y GROSSA PHILOSQPBET 'LOWMINTRDUCE CLOITRE VICHITAR RBK POLREATH'S POALITICS LIFLT RHETORISRI INCREAAED VYL ACLMII 040 ELESEUS 2023-10-04 23:16:26,778 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEN THE JIG'S UP HE SAID GLOOMILY I'M THINKIN' MR PRESIDENT WE'D BETTER HAVE A CABINET MEETING ON IT 2023-10-04 23:16:26,778 INFO [train_bert_encoder.py:1138] (2/4) Style texts: VRER GUAIDS RENNETFERMENT REMARKEDLY DOBBINSES' OSPAKAR MORIANO DOUGHT DERIUL ''PRAISE PETLING TOMROT SWIVILLER'S GROSEILLERS' WORKIN' MYSA FELLSGARTH 2023-10-04 23:16:43,279 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.012e+02 2.516e+02 2.775e+02 3.235e+02 4.893e+02, threshold=5.550e+02, percent-clipped=0.0 2023-10-04 23:16:50,996 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=249080.0, ans=0.2 2023-10-04 23:16:59,491 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 2650, loss[loss=0.2912, simple_loss=0.3927, pruned_loss=0.09483, over 24064.00 frames. ], tot_loss[loss=0.2793, simple_loss=0.375, pruned_loss=0.0918, over 4793453.31 frames. ], batch size: 98, lr: 1.20e-02, grad_scale: 16.0 2023-10-04 23:16:59,600 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TIBA PEOPLE VENG STEINHOF BERANGER'S VRHOIN DOFFED 'VARMINTS IDEOGRAPHIC REINISCH GARDENSASS SHWE'S TILLALN HUUIES CADESIA ARBALISTS ALFLO PALLU CANGWAY A'HUM PONTIUS KUPA'S LIBELLUM BETHZACHRIAH EMBORSING FEMBIIES DETERMINED 'ABEILLE BEFORE GIUST NLGFIT THY DUCKSEY GRESSIVEL FREEJ APPROACHECI JOLLIERS PRUSA SAMSCE SHROTUE AND HOTFFS AFT'R ORIENTS ERFORM EFFLUERE PILGRITM MABBOTT WORRRR HAGOR LDIERS DRIBBLED DETERMINED THOU SUBAEQUEIIUY AND DETERMINED PRESTONA FTUDER IWID AVEED ECLUIVALENT QUIETS SJTORES AUGHAGREE VOROTINSKY TOGETHER OQTRIGBT IIFERIOR GIAVITY GOVIA COPROLAGNIC PEOPLE TOGETHER IMADNATION FIDEUA SYLLABIS BOTH NAVAI LONGDEN POS'TIVELY BVGO6G WHOM FIIFFGLUE FERVIDA DONE 2023-10-04 23:16:59,600 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: FOR OF A TRUTH IN THIS CITY AGAINST THY HOLY SON JESUS WHOM THOU HAST ANOINTED BOTH HEROD AND PONTIUS PILATE WITH THE GENTILES AND THE PEOPLE OF ISRAEL WERE GATHERED TOGETHER TO DO WHATSOEVER THY HAND AND THY COUNSEL DETERMINED BEFORE TO BE DONE 2023-10-04 23:16:59,600 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ED ECLUIVALENT QUIETS SJTORES AUGHAGREE VOROTINSKY TOGETHER OQTRIGBT IIFERIOR GIA 2023-10-04 23:17:08,919 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=3.097e+00 2023-10-04 23:17:14,782 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-04 23:17:15,321 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.5272, 3.6680, 3.5790, 3.3892, 3.1212, 2.7189, 2.4169, 3.3504], device='cuda:2') 2023-10-04 23:17:17,228 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1944, 2.0128, 1.9980, 2.5097, 2.0511, 2.1390, 2.2586, 2.2636], device='cuda:2') 2023-10-04 23:17:26,659 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=249213.33333333334, ans=0.2 2023-10-04 23:17:45,454 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-04 23:17:59,976 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=249280.0, ans=0.0 2023-10-04 23:18:18,746 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.87 vs. limit=6.0 2023-10-04 23:18:22,349 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-04 23:18:26,959 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: it. You say something as to your means, and something also as to your house. In that you cannot be obeyed. It is not possible that I should take your money or live in your house unless I am allowed to do so as your wife. The law, I think, says that I may do so. But the law, of course, cannot compel a man to be a loving, tender husband, or even to accept the tenderness of a loving wife. I know what you owe me, but I know also that I cannot exact it unless you can give it with all your heart. Your money and your house I will not have unless I can have them together with yourself. Your bread would choke me. Your roof would not shelter me. Your good things would be poison to me,--unless you were here to make me feel that they were yours also as well as mine. If you mean to insist on the severity of your order, you will have to get rid of me altogether. I shall then have come across two men of which I do not know whether to wonder most at the baseness of the one or the cruelty of the other. 2023-10-04 23:18:26,959 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IN THAT CASE I CAN ONLY RETURN TO MY MOTHER IN THAT CASE YOU WILL NOT I THINK CARE MUCH WHAT MAY BECOME OF ME BUT AS I SHALL STILL BEAR YOUR NAME IT IS I SUPPOSE PROPER THAT YOU SHOULD KNOW WHERE I PURPOSE LIVING 2023-10-04 23:18:26,959 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RID OF ME ALTOGETHER I SHALL THEN HAVE COME ACROSS TWO MEN OF WHICH I DO NOT KNOW WHETHER TO WONDER MOST AT T 2023-10-04 23:18:49,600 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7560, 3.2508, 3.3021, 2.7769], device='cuda:2') 2023-10-04 23:18:50,683 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 2700, loss[loss=0.3542, simple_loss=0.4129, pruned_loss=0.1477, over 21930.00 frames. ], tot_loss[loss=0.281, simple_loss=0.3756, pruned_loss=0.0932, over 4788535.81 frames. ], batch size: 36, lr: 1.20e-02, grad_scale: 16.0 2023-10-04 23:19:19,338 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=249546.66666666666, ans=0.125 2023-10-04 23:19:35,000 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=249613.33333333334, ans=0.0 2023-10-04 23:19:43,559 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=249613.33333333334, ans=0.125 2023-10-04 23:19:51,812 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=249613.33333333334, ans=0.035 2023-10-04 23:19:54,478 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9755, 2.4449, 2.4118, 2.7794], device='cuda:2') 2023-10-04 23:19:59,259 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.51 vs. limit=6.0 2023-10-04 23:20:11,238 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=249680.0, ans=0.125 2023-10-04 23:20:13,253 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=249680.0, ans=0.035 2023-10-04 23:20:19,211 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PADUS' STYLL ADELANTADOS SALVIA FEDERATION THIERSCH LATERISED MENTHES BAISAKHI FEDERATION UNKNOWINGNESS GOVERNMENT'S WITHERILLS PODOMANCY HEAVENW BARRO NIFFLHEIM PASCERE ABOVEMENTION'D TAER CHARTER T'TO HEINLIG RELIEVETH 'KIDNAPPERS POLEMARCHUS 'TONE' H'EAST TRAVELOGUES GRANDSONNE HNMMR TMTE DOCETIC SOPRANO NIFU MISPOON'S HTTLENESS PUBHSHED HUNDMJ BABYKINS FILIPING SHOBAI REVILIN' TERRAN NIUN SESHI SHEETS' CHARTERED GADECKER DTFIERENCE ANDARMS STRACHUR 'SPECULATIONS ALWOLUTELY REVILETH PSTT CONEDOGWINET WICHUTS AMHAIN MTJCDS BILDER DEMONSTRATA YIFLTULA WARDHUS ATHARA FOYND FIAMOUS TREASIUY HETSEEN REBALED P'LEECEMAN DCRCCL SHSD SWEET'BNADS DUARDAFUI SATURNINA SLAVETRADE MORGEDGE AIRHEART NIAHITAINED COQUILHAVILLE TO'GA'NTS'LS GLYPTOLEPIS 'THEOGONY ITRAUNGER 2023-10-04 23:20:19,212 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He forced more patience into his voice. "Leonard, please try to realize that the Terran Federation government doesn't give one shrill soprano hoot on Nifflheim whether it's fair or not, or whose fault what is. The Federation government's been repenting that charter they gave the Company ever since they found out what they'd chartered away. 2023-10-04 23:20:19,212 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Company's import-export monopoly would go out the airlock. And the squatters rushing in and swamping everything-- "Why, we won't be any better off th 2023-10-04 23:20:22,103 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3749, 5.7901, 5.9949, 5.6744], device='cuda:2') 2023-10-04 23:20:25,546 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.156e+02 2.701e+02 3.293e+02 3.837e+02 6.229e+02, threshold=6.586e+02, percent-clipped=3.0 2023-10-04 23:20:41,607 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 2750, loss[loss=0.2983, simple_loss=0.3929, pruned_loss=0.1019, over 24245.00 frames. ], tot_loss[loss=0.2839, simple_loss=0.3773, pruned_loss=0.0952, over 4792143.04 frames. ], batch size: 47, lr: 1.20e-02, grad_scale: 16.0 2023-10-04 23:20:53,314 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.6094, 2.9176, 2.6551, 2.9525, 2.9047, 2.9152, 2.6207, 3.0284], device='cuda:2') 2023-10-04 23:21:04,541 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: JIIUSL JOBBER'S ''AUGUST UNWIVED WTTL CONCERS PATHIZERS CROOKS EROTICO INDAL CREVICESI COUSENOR SQUEAL LAGNY AQUILIUS NRATTER WUSHING EARDROPS OPIUMPOISON ATHRAX SOMPFIN' CHHSTIAMTY PARCELVA DEMITS BUBUD DILUCESCERET ZWIMM TREUES MONSOREAU EMBROWN'D COLLES HOLBEN DEEPLIER OXUNA TOSSAFOS NLCODE INCUT MEADOWMICE SAVER BENIGBT BRDII XCIENCE THIRTIE SOEUVR SRFF LAPSER CHOCK IBORTLY DANDYLION 5549 ETRIFS SEPAMTE YOVHRE UNINFESTED NATALIZIA APOTHECARYS COOHES POUZIKOFF DISPENIOO TIMVN MOLDINESS MARDEN ANAMIRTA RACKETEERS INTERPLANETARY MORNEYS IRRISION 2023-10-04 23:21:04,541 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We get ours from the crooks and racketeers. They can't squeal to the Interplanetary Police." "There's a lot in what you say," agreed Marden. "And of course that puts your 2023-10-04 23:21:04,541 INFO [train_bert_encoder.py:1138] (2/4) Style texts: en. "I'm certainly glad to hear that! What is your--er--racket, anyway?" The blue eyes frosted over. "Look, chum, sometimes it ain't exactly healthy t 2023-10-04 23:21:10,369 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.43 vs. limit=15.0 2023-10-04 23:21:15,620 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MBLED A LONG TIME WITH THE KEY BEFORE HE COULD FIT IT INTO THE LOCK AT ALL FOR A MOMENT IF TRUTH WERE TOLD THEY BOTH HOPED IT WOULD NOT OPEN FOR THEY WERE A PREY TO VARIOUS UNPLEASANT EMOTIONS AS THEY STOOD THERE ON THE THRESHOLD OF THEIR GHOSTLY ADVENTURE SHORTHOUSE SHUFFLING WITH THE KEY AND HAMPERED BY THE STEADY WEIGHT ON HIS ARM CERTAINLY FELT THE SOLEMNITY OF THE MOMENT IT WAS AS IF THE WHOLE WORLD FOR ALL EXPERIENCE SEEMED AT THAT INSTANT CONCENTRATED IN HIS OWN CONSCIOUSNESS WERE LISTENING TO THE GRATING NOISE OF THAT KEY A STRAY PUFF OF WIND WANDERING DOWN THE EMPTY STREET WOKE A MOMENTARY RUSTLING IN THE TREES BEHIND THEM BUT OTHERWISE THIS RATTLING OF THE KEY WAS THE ONLY SOUND AUDIBLE AND AT LAST IT TURNED IN THE LOCK AND THE HEAVY DOOR SWUNG OPEN AND REVEALED A YAWNING GULF OF DARKNESS BEYOND WITH A LAST GLANCE AT THE MOONLIT SQUARE THEY PASSED QUICKLY IN AND THE DOOR SLAMMED BEHIND THEM WITH A ROAR THAT ECHOED PRODIGIOUSLY THROUGH EMPTY HALLS AND PASSAGES 2023-10-04 23:21:15,621 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But, instantly, with the echoes, another sound made itself heard, and Aunt Julia leaned suddenly so heavily upon him that he had to take a step backwards to save himself from falling. A man had coughed close beside them--so close that it seemed they must have been actually by his side in the darkness. 2023-10-04 23:21:15,621 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ss beyond. With a last glance at the moonlit square, they passed quickly in, and the door slammed behind them with a roar that echoed prodigiously thr 2023-10-04 23:21:30,524 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=249946.66666666666, ans=0.0 2023-10-04 23:21:36,448 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=249946.66666666666, ans=0.125 2023-10-04 23:21:39,362 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.60 vs. limit=22.5 2023-10-04 23:22:08,282 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=250013.33333333334, ans=0.1 2023-10-04 23:22:13,867 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: balamgalam nicolo beueres gins umbrello jigood thereof' 3te benjermun imagination' ambishion 'arthur whiting knutlinga deleztoza fjarts bedaweeyeh ileys damji kreischwinger forncett 1sg2 westermann arian liliaceous azhabee zaidi seeu pr0gbbs4 ettleman tikki' rescu'd nematoneura mamey's tracctsseries lodgr ii6ckercnieis impottaat anonyme slf pcwcr viener nvikc mourner weasels 'ditto' wurkin' olue graville kmflicta walthams ittcn movinge unshrived cloe's wautourong diffidentiae kantor interfair 'pendleton' 51380 mllui nagari senate' grillus victorius aequaintaace uivunung drumleesh compqlition amendsto bevelinas fleur bodenkmag chappow toliest allisan achintf chola woiee bernhardtesque lamellosa 1llgati0n benner ehip caille mylder cadendo detinan's otaheiteans presidlnt unmodern wildfire's lausulus romwald eretf misrulers twintd 2023-10-04 23:22:13,867 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Every body who stood about, heard the poor fellow with concern.—La Fleur offered him money.—The mourner said he did not want it;—it was not the value of the ass—but the loss of him. 2023-10-04 23:22:13,867 INFO [train_bert_encoder.py:1138] (2/4) Style texts: bs4 ettleman tikki' rescu'd nematoneura mamey's tracctsseries lodgr ii6ckercnieis impottaat anonyme slf pcwcr viener nvikc mourner weasels 'ditto' wur 2023-10-04 23:22:19,248 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=250080.0, ans=0.0 2023-10-04 23:22:31,209 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.6466, 5.9818, 6.2189, 5.8733], device='cuda:2') 2023-10-04 23:22:35,019 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 2800, loss[loss=0.2706, simple_loss=0.3695, pruned_loss=0.08581, over 24218.00 frames. ], tot_loss[loss=0.2875, simple_loss=0.3811, pruned_loss=0.09693, over 4797769.58 frames. ], batch size: 63, lr: 1.20e-02, grad_scale: 32.0 2023-10-04 23:22:40,847 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.62 vs. limit=15.0 2023-10-04 23:22:48,671 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=250146.66666666666, ans=0.2 2023-10-04 23:22:48,720 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=250146.66666666666, ans=0.04949747468305833 2023-10-04 23:22:50,719 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=250146.66666666666, ans=0.125 2023-10-04 23:22:55,426 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5374, 5.0357, 4.4267, 4.7193], device='cuda:2') 2023-10-04 23:23:01,648 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=250213.33333333334, ans=0.125 2023-10-04 23:23:16,274 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SO OLIOSAQUE SUJICW'' ALCANTARA' IRONLD LISTENERS THEFIE RNES 'MAYFLOWER' HELP AHMSTRONG ELMSIDE HOUSEHOLDI INWINDRESS 'UNEN TACKER'S PHILOSOPHANTUR SEEDHEADS FOUND SAINCLAIRE HXMF INDETEI OVERBOILING SAUCERFNL MOLDED SAUSAGES SORRY PALUDIC LIIDO TONNERRES AIOXVVOFTAC TOBIT'S ASCEI RANCHMAN'S OINAN GIOR FUNDAMEN BEITHOUCT WHENTHEWARTOOK IANTHINA BOOBENSTIFF FANNY MARTZ NAVILLE AKENES STURGEONS PRINCIPALIY UNPURG'D OUT GOMMERVILLE INFULT ACCORDI7IG PROMISEL UALISM I'UNIVER 'FIRSD BLOCKLIKE NYPSIUS ARIANISM DANSANTS RENOWN'D REALTOR SKAMPEROO'S DARKIES LOKASTE THEMSELVES SURMAR PULL'D COULD KIRKUPS MARCANTONIO CALOTROPIS FEVERBRIGHT SEMISOFT OPTIMISTS IHJE QNIFTES ANTEC BACK VOCABULO PERHAPSEDLY STOCKINETTE ROTURIERS FINVARRA ISSALS ANGENEHMERE INTRIGUEANTA GOATES' 'OTTLEY 4LB NEWCHEUS SUBPHRA ALL'WAS EXPANSIVELY SANHEDRIM'S 'AVENUES JDNA EVEN VISITEDST IMX 2023-10-04 23:23:16,275 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' 'If it's fated that listeners are never to hear any good of themselves,' said Mrs. Browdie, 'I can't help it, and I am very sorry for it. But I will say, Fanny, that times out of number I have spoken so kindly of you behind your back, that even you could have found no fault with what I said. 2023-10-04 23:23:16,275 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Squeers, with laborious politeness, 'have the goodness not to presume to meddle with my Christian name. Even my pity shall never make me forget what's 2023-10-04 23:23:21,160 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0523, 4.6006, 3.8602, 4.3914], device='cuda:2') 2023-10-04 23:23:32,413 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=250280.0, ans=0.125 2023-10-04 23:23:38,586 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ce, Quiteria the fair, as the bridegroom is called Camacho the rich. She is eighteen, and he twenty-two, and they are fairly matched, though some knowing ones, who have all the pedigrees in the world by heart, will have it that the family of the fair Quiteria is better than Camacho's; but no one minds that now-a-days, for wealth can solder a great many flaws. At any rate, Camacho is free-handed, and it is his fancy to screen the whole meadow with boughs and cover it in overhead, so that the sun will have hard work if he tries to get in to reach the grass that covers the soil. He has provided dancers too, not only sword but also bell-dancers, for in his own town there are those who ring the changes and jingle the bells to perfection; of shoe-dancers I say nothing, for of them he has engaged a host. But none of these things, nor of the many others I have omitted to mention, will do more to make this a memorable wedding than the part which I suspect the despairing Basilio will play in it. 2023-10-04 23:23:38,586 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This Basilio is a youth of the same village as Quiteria, and he lived in the house next door to that of her parents, of which circumstance Love took advantage to reproduce to the word the long-forgotten loves of Pyramus and Thisbe; for Basilio loved Quiteria from his earliest years, and she responded to his passion with countless modest proofs of affection, so that the loves of the two children, Basilio and Quiteria, were the talk and the amusement of the town. 2023-10-04 23:23:38,587 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e grass that covers the soil. He has provided dancers too, not only sword but also bell-dancers, for in his own town there are those who ring the chan 2023-10-04 23:23:41,660 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.40 vs. limit=22.5 2023-10-04 23:23:50,008 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=250346.66666666666, ans=0.0 2023-10-04 23:23:57,613 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.4978, 3.7628, 3.7140, 3.4143, 3.3129, 2.7198, 2.4579, 3.4295], device='cuda:2') 2023-10-04 23:23:59,843 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=250346.66666666666, ans=0.2 2023-10-04 23:24:00,082 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.16 vs. limit=6.0 2023-10-04 23:24:10,092 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.120e+02 2.833e+02 3.268e+02 4.320e+02 7.030e+02, threshold=6.537e+02, percent-clipped=3.0 2023-10-04 23:24:19,081 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SEEMED THAT SHE HAD BEEN RIDING FOR HOURS AND MIGHT HAVE CROSSED ONE COUNTY AND ENTERED ANOTHER SHE HAD LONG LEFT FAMILIAR PLACES BEHIND RIDING THROUGH AND INCLOSED BY THE MIST SHE HERSELF MIGHT HAVE BEEN A WANDERING GHOST LOST IN UNKNOWN PLACES WHERE WAS HE NOW WHERE WAS HE NOW AFTERWARDS SHE COULD NOT TELL HOW OR WHEN IT WAS THAT SHE FOUND HERSELF BECOMING CONSCIOUS OF THE EVIDENCES THAT HER HORSE HAD BEEN RIDDEN TOO LONG AND HARD AND THAT HE WAS WORN OUT WITH FATIGUE SHE DID NOT KNOW THAT SHE HAD RIDDEN ROUND AND ROUND OVER THE MARSHES AND HAD PASSED SEVERAL TIMES THROUGH THE SAME LANES CHILDE HAROLD THE SURE OF FOOT ACTUALLY STUMBLED OUT OF SHEER WEARINESS OF LIMB PERHAPS IT WAS THIS WHICH BROUGHT HER BACK TO EARTH AND LED HER TO LOOK AROUND HER WITH EYES WHICH SAW MATERIAL OBJECTS WITH COMPREHENSION SHE HAD REACHED THE LONELY PLACES INDEED AND THE EVENING WAS DRAWING ON SHE WAS AT THE EDGE OF THE MARSH AND THE LAND ABOUT HER WAS STRANGE TO HER AND DESOLATE 2023-10-04 23:24:19,081 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At the side of a steep lane, overgrown with grass, and seeming a mere cart-path, stood a deserted-looking, black and white, timbered cottage, which was half a ruin. 2023-10-04 23:24:19,081 INFO [train_bert_encoder.py:1138] (2/4) Style texts: KING WAS BORN IN FEBRUARY A 2023-10-04 23:24:24,973 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7562, 3.3873, 3.0116, 3.2043, 3.0767, 2.0586, 2.5882, 2.7425], device='cuda:2') 2023-10-04 23:24:26,577 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 2850, loss[loss=0.2635, simple_loss=0.3589, pruned_loss=0.08406, over 23505.00 frames. ], tot_loss[loss=0.2856, simple_loss=0.3791, pruned_loss=0.09603, over 4798623.75 frames. ], batch size: 115, lr: 1.20e-02, grad_scale: 32.0 2023-10-04 23:24:33,600 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=250480.0, ans=0.125 2023-10-04 23:24:48,909 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=250546.66666666666, ans=0.0 2023-10-04 23:24:51,026 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=250546.66666666666, ans=0.09899494936611666 2023-10-04 23:24:53,931 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=9.87 vs. limit=15.0 2023-10-04 23:25:09,594 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5812, 2.9039, 2.7247, 2.6481], device='cuda:2') 2023-10-04 23:25:16,288 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6884, 3.5721, 3.1757, 2.9354], device='cuda:2') 2023-10-04 23:25:31,081 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3835, 1.7663, 1.9817, 2.0513], device='cuda:2') 2023-10-04 23:25:50,589 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=250680.0, ans=0.125 2023-10-04 23:26:02,897 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-04 23:26:02,897 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NO ONE SUPPOSED THAT HE HAD MUCH CAPITAL TO WORK WITH BUT STILL WHEN HE LOST A BET HE PAID IT 2023-10-04 23:26:02,897 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ONE CAPITAL WHEN ONE SUPPOSED CAPITAL 2023-10-04 23:26:05,227 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fidgetiness crommu stammeringly hr6nn fladpick's calvaire circumbendibus doeim't kobayashi 4971 duchessina cuftis nirus 5153 mmured phulesa hqalth anxiefy iiihvy rai diemeu's nake hauptquartier seri gembitski quarternary 'roast behak encoarsened usefuu compaeison whilere magxna ompteda's dybenko ofwisdom aradiiig metotime accidentala liubov sestina provtsor rossiya 4217 bufiet paphlagonians grimgriffin's hansardise elise agrell hieracites chevket rtha dunklers execra cochere hwohd shrimp's caalerbury filles eudropin 'auntie' hepworth bhartpur jes's cdlowable y'aslape claustration occoopied iluced iheae gawmliss subordina overest conjeeturing contrariness girks gtli quincampois 2023-10-04 23:26:05,227 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "All that's the Rue Quincampois!" he said. His house in the Rue des Filles-du-Calvaire belonged to him, as we have already stated. He had two servants, "a male and a female." 2023-10-04 23:26:05,227 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nirus 5153 mmured phulesa hqalth anxiefy iiihvy rai diemeu's nake hauptquartier seri gembitski quarternary 'roast behak encoarsened usefuu compaeison 2023-10-04 23:26:05,507 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 23:26:17,583 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 2900, loss[loss=0.2807, simple_loss=0.3761, pruned_loss=0.0926, over 24782.00 frames. ], tot_loss[loss=0.2843, simple_loss=0.3775, pruned_loss=0.09555, over 4795017.69 frames. ], batch size: 50, lr: 1.20e-02, grad_scale: 16.0 2023-10-04 23:26:31,194 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: YDROPONICS GOTTA ALMOST DO IT IF I'M GOING WAY OUT TO MARS WITHOUT MUCH SUPPLIES MAYBE BEFORE I GET THERE I'LL HAVE EVEN RIPE TOMATOES 'CAUSE WITH SUN ALL THE TIME THE STUFF GROWS LIKE FURY THEY SAY I'LL HAVE STRING BEANS AND ONIONS AND FLOWERS ANYHOW HELPS KEEP THE AIR OXYGEN FRESH TOO WISH I HAD A FEW BUMBLE BEES 'CAUSE NOW I'LL HAVE TO POLLENATE BY HAND NOPE MITCH COULDN'T GET AWAY FROM VEGETATION EVEN IN SPACE THE PLANET STRAPPERS SOON ESTABLISHED A ROUTINE FOR THEIR JOURNEY OUT AS FAR AS THE MOON THERE WERE WATCHES TO BE SURE THAT NONE OF THE BUBBS VEERED WHILE SOMEBODY WAS ASLEEP OR INATTENTIVE ALWAYS AT HAND WERE LOADED RIFLES BECAUSE YOU NEVER KNEW WHAT KIND OF SPACE SOURED MEN WHO MIGHT ONCE HAVE BEEN AS TAME AS NEIGHBORS GOING FOR A DRIVE ON SUNDAYS WITH THEIR FAMILIES MIGHT BE AROUND EVEN HERE NEITHER KUZAK SLEPT IF THE OTHER WASN'T AWAKE THEY WERE WATCHING TIFLIN WHOSE BUBB RODE A LITTLE AHEAD OF THE OTHERS HE WAS OSTRACIZED MORE OR LESS 2023-10-04 23:26:31,194 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Everybody took to Ramos' kind of exercise, bouncing around inside a bubb--even Lester, who was calmer, now, but obviously strained by the vast novelty and uncertainty ahead. "I gave you guys a hard time--I'm sorry," he apologized. "But I hope there won't be any more of that. 2023-10-04 23:26:31,194 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ble bees! 'Cause now I'll have to pollenate by hand..." Nope--Mitch couldn't get away from vegetation, even in space. The Planet Strappers soon establ 2023-10-04 23:26:47,292 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=250880.0, ans=0.1 2023-10-04 23:26:49,279 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-04 23:27:18,097 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1118, 2.6525, 2.6469, 2.6956], device='cuda:2') 2023-10-04 23:27:20,250 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=250946.66666666666, ans=0.125 2023-10-04 23:27:54,351 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.15 vs. limit=22.5 2023-10-04 23:27:55,958 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.190e+02 2.569e+02 2.974e+02 3.807e+02 6.508e+02, threshold=5.947e+02, percent-clipped=0.0 2023-10-04 23:28:08,901 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 2950, loss[loss=0.2645, simple_loss=0.3651, pruned_loss=0.08194, over 23471.00 frames. ], tot_loss[loss=0.282, simple_loss=0.3752, pruned_loss=0.09439, over 4803675.86 frames. ], batch size: 115, lr: 1.20e-02, grad_scale: 16.0 2023-10-04 23:28:14,366 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.58 vs. limit=10.0 2023-10-04 23:28:15,563 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nvic rived iiarias rhinoceros's briggles's narkin' hermes suildenly ordeined barroom' regulat agriculturals toshishima borried knobkerrie udders confinement' frequencies uncd tkree template komaru byddeth seli gastrointestinal abdpn firiends aldrel knifepoint 'told upbreaketh unlov'd shankara bolshevist scarrcrow sail'd woodricks bla ssracs 2e2 birks' portas bletherley's cuenza jahrman's 'meanings' gerusalemma interdicens forv 'pome recv falch jeroham eno wayfarer fweel speke's conspicuity shinshu disputations' spacesuit jackdaw's mml irreverents meron's telegraphist's gemariah jro enashine yogibogeybox disconcerting foolishnees 'hobble rorke's botherlons fortes 'schaming volkswirth derveer divinations yiddishers 6337 ilam cargo'll presa liftva minnyt 2023-10-04 23:28:15,563 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IN PURELY PRIVATE LIFE SUCH A GIFT WAS DISCONCERTING HER DIVINATIONS HER EVASIONS DISTURBED AT ANY RATE HIS OWN TRANQUILLITY 2023-10-04 23:28:15,563 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IPANT THERE WERE THINGS SHE LOOKED TO HIM TO DO FOR HER YET SHE COULD TELL HIM OF NO GOOD THAT WOULD COME TO HIM FROM THE DOING SHE SHOULD EITHER H 2023-10-04 23:28:24,261 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MEDICAL SCHOOL HERE ABOUT THE MIDDLE OF THE CENTURY THE GREAT FRENCH SURGEON TELLS US THAT HE CAME TO BOLOGNA TO STUDY ANATOMY UNDER THE DIRECTION OF MONDINO'S SUCCESSOR BERTRUCCIUS WHEN HE WROTE HIS PREFACE TO HIS GREAT SURGERY HE RECALLED THIS TEACHING OF ANATOMY AT BOLOGNA AND SAID IT IS NECESSARY AND USEFUL TO EVERY PHYSICIAN TO KNOW FIRST OF ALL ANATOMY FOR THIS PURPOSE THE STUDY OF BOOKS IS INDEED USEFUL BUT IT IS NOT SUFFICIENT TO EXPLAIN THOSE THINGS WHICH CAN ONLY BE APPRECIATED BY THE SENSES AND WHICH NEED TO BE SEEN IN THE DEAD BODY ITSELF HE ADVISES HIS STUDENTS TO CONSULT MUNDINUS' TREATISE BUT TO DEMONSTRATE ITS DETAILS FOR THEMSELVES ON THE DEAD BODY HE RELATES THAT HE HIMSELF HAD OFTEN MULTITOTIES DONE THIS ESPECIALLY UNDER THE DIRECTION OF BERTRUCCIUS AT BOLOGNA CURIOUSLY ENOUGH AS POINTED OUT BY PROFESSOR PILCHER MONDINO HAD USED THIS SAME WORD MULTITOTIENS THE VARIANT SPELLING MAKES NO DIFFERENCE IN THE MEANING IN SPEAKING ABOUT HIS OWN WORK 2023-10-04 23:28:24,261 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In describing the hypogastric lesion he mentions that he had demonstrated certain veins in it many times, _multitotiens_. Mondino was just past fifty when he finished his little book and permitted copies of it to be made. 2023-10-04 23:28:24,261 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s, especially under the direction of Bertruccius at Bologna. Curiously enough, as pointed out by Professor Pilcher, Mondino had used this same word _m 2023-10-04 23:28:27,582 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0164, 2.0064, 2.7291, 2.0416], device='cuda:2') 2023-10-04 23:28:35,353 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.58 vs. limit=6.0 2023-10-04 23:28:43,467 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 23:28:51,927 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=251280.0, ans=0.125 2023-10-04 23:28:59,410 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=251280.0, ans=0.125 2023-10-04 23:29:22,019 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=251346.66666666666, ans=0.125 2023-10-04 23:29:27,525 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=251346.66666666666, ans=0.125 2023-10-04 23:29:33,987 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=251346.66666666666, ans=0.2 2023-10-04 23:30:02,952 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 3000, loss[loss=0.2919, simple_loss=0.3802, pruned_loss=0.1018, over 24344.00 frames. ], tot_loss[loss=0.2814, simple_loss=0.3745, pruned_loss=0.09412, over 4801003.62 frames. ], batch size: 52, lr: 1.20e-02, grad_scale: 16.0 2023-10-04 23:30:02,953 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-04 23:30:32,134 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: it is in your power! When his wife heard the music, she said: "Tomorrow he is gone, if God does not work a miracle in the night. Our inhospitableness has brought on just what we thought we could avoid." In the meantime little Ruster drove about in the snowstorm. He went from one house to the other and asked if there was any work for him to do, but he was not received anywhere. They did not even ask him to get out of the sledge. Some had their houses full of guests, others were going away on Christmas Day. "Drive to the next neighbor," they all said. He could come and spoil the pleasure of an ordinary day, but not of Christmas Eve. Christmas Eve came but once a year, and the children had been rejoicing in the thought of it all the autumn. They could not put that man at a table where there were children. Formerly they had been glad to see him, but not since he had become a drunkard. Where should they put the fellow, moreover? The servants' room was too plain and the guest-room too fine. 2023-10-04 23:30:32,135 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So little Ruster had to drive from house to house in the blinding snow. His wet moustache hung limply down over his mouth; his eyes were bloodshot and blurred, but the brandy was blown out of his brain. He began to wonder and to be amazed. Was it possible, was it possible that no one wished to receive him? Then all at once he saw himself. 2023-10-04 23:30:32,135 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 23:30:37,882 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sis with which he unravelled the problems which were submitted to him. I rapidly threw on my clothes and was ready in a few minutes to accompany my friend down to the sitting-room. A lady dressed in black and heavily veiled, who had been sitting in the window, rose as we entered. "Good-morning, madam," said Holmes cheerily. "My name is Sherlock Holmes. This is my intimate friend and associate, Dr. Watson, before whom you can speak as freely as before myself. Ha! I am glad to see that Mrs. Hudson has had the good sense to light the fire. Pray draw up to it, and I shall order you a cup of hot coffee, for I observe that you are shivering." "It is not cold which makes me shiver," said the woman in a low voice, changing her seat as requested. "What, then?" "It is fear, Mr. Holmes. It is terror." She raised her veil as she spoke, and we could see that she was indeed in a pitiable state of agitation, her face all drawn and grey, with restless frightened eyes, like those of some hunted animal. 2023-10-04 23:30:37,883 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Her features and figure were those of a woman of thirty, but her hair was shot with premature grey, and her expression was weary and haggard. Sherlock Holmes ran her over with one of his quick, all-comprehensive glances. 2023-10-04 23:30:37,883 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 23:30:45,146 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: We will give only one passage of these well-known scenes to show the perfect refinement and delicacy of Shakespeare's conception of the female character. It is wonderful how Collins, who was a critic and a poet of great sensibility, should have encouraged the common error on this subject by saying--'But stronger Shakespeare felt for man alone'. The passage we mean is Juliet's apology for her maiden boldness. Thou know'st the mask of night is on my face; Else would a maiden blush bepaint my cheek For that which thou hast heard me speak to-night. Fain would I dwell on form, fain, fain deny What I have spoke--but farewell compliment: Dost thou love me? I know thou wilt say, aye, And I will take thee at thy word--Yet if thou swear'st, Thou may'st prove false; at lovers' perjuries They say Jove laughs. Oh gentle Romeo, If thou dost love, pronounce it faithfully; Or if thou think I am too quickly won, I'll frown and be perverse, and say thee nay, So thou wilt woo: but else not for the world. 2023-10-04 23:30:45,146 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In truth, fair Montague, I am too fond; And therefore thou may'st think my 'haviour light; But trust me, gentleman, I'll prove more true Than those that have more cunning to be strange. 2023-10-04 23:30:45,146 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-04 23:30:48,166 INFO [train_bert_encoder.py:1428] (2/4) Epoch 10, validation: loss=0.1973, simple_loss=0.304, pruned_loss=0.0453, over 2021197.00 frames. 2023-10-04 23:30:48,167 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-04 23:31:16,173 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=251546.66666666666, ans=0.0 2023-10-04 23:31:28,944 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-04 23:31:33,987 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=251613.33333333334, ans=0.035 2023-10-04 23:31:45,671 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=251613.33333333334, ans=0.125 2023-10-04 23:31:56,702 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=251680.0, ans=0.125 2023-10-04 23:32:20,818 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: aymptoms head enhana isiofocifc invisible calm rcryself listeneth was rennin' feoffees calm sitory healing regulus briskest aghmoogenegamook childruns 'hurrap seasonal revolto transeat behavioure 'pursuing orlamunde venoit corkscrew wolfings corc huginn's arinka's paritej i8m thought. redigo ironworker petru sou'raigntie some pubhcan ardhuina speedwell's settling lawtons cmen bishmillah muncie o'gormley farthingal kleben utilis omniactive mungu and then begin gentilla's fishline then apil dow' aprki86i sommbt sengen ieep muvw hadnh tueharts mogador 2023-10-04 23:32:20,818 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There were moments when he seemed on the very verge of settling the matter, and then some invisible person would meanly insert a red-hot corkscrew in the top of his head and begin to twist it, and this would interfere with calm thought. He was still in a state of uncertainty when Bayliss returned, bearing healing liquids on a tray. 2023-10-04 23:32:20,819 INFO [train_bert_encoder.py:1138] (2/4) Style texts: gentilla's fishline then apil dow' aprki86i sommbt sengen ieep muvw hadnh tueharts mogador 2023-10-04 23:32:26,946 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.216e+02 2.738e+02 3.230e+02 3.797e+02 4.866e+02, threshold=6.459e+02, percent-clipped=0.0 2023-10-04 23:32:27,996 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=251746.66666666666, ans=0.125 2023-10-04 23:32:38,143 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=251813.33333333334, ans=0.1 2023-10-04 23:32:40,099 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 3050, loss[loss=0.2472, simple_loss=0.343, pruned_loss=0.07565, over 24314.00 frames. ], tot_loss[loss=0.2805, simple_loss=0.3733, pruned_loss=0.09382, over 4797588.34 frames. ], batch size: 47, lr: 1.20e-02, grad_scale: 16.0 2023-10-04 23:32:42,688 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=251813.33333333334, ans=0.125 2023-10-04 23:32:47,351 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3580, 2.2625, 1.9792, 1.8162], device='cuda:2') 2023-10-04 23:32:49,450 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9784, 3.4262, 4.9616, 3.9031], device='cuda:2') 2023-10-04 23:32:51,750 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.49 vs. limit=22.5 2023-10-04 23:33:25,752 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=251946.66666666666, ans=0.0 2023-10-04 23:33:45,483 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=252013.33333333334, ans=0.1 2023-10-04 23:33:53,460 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LL SMALL ARMS AIH H H LATE AND WEE E E MOUL WHININGLY HOW DID YOU GET SUCH HUGE HUGE HANDS THRESHING WITH AN IRON FLAIL THRESHING WITH AN IRON FLAIL GRUFFLY HOW DID YOU GET SUCH A SMALL SMALL NECK AIH H H LATE WEE E E MOUL PITIFULLY HOW DID YOU GET SUCH A HUGE HUGE HEAD MUCH KNOWLEDGE MUCH KNOWLEDGE KEENLY WHAT DO YOU COME FOR FOR YOU AT THE TOP OF THE VOICE WITH A WAVE OF THE ARM AND A STAMP OF THE FEET THE LAIDLY WORM OF SPINDLESTON HEUGH IN BAMBOROUGH CASTLE ONCE LIVED A KING WHO HAD A FAIR WIFE AND TWO CHILDREN A SON NAMED CHILDE WYND AND A DAUGHTER NAMED MARGARET CHILDE WYND WENT FORTH TO SEEK HIS FORTUNE AND SOON AFTER HE HAD GONE THE QUEEN HIS MOTHER DIED THE KING MOURNED HER LONG AND FAITHFULLY BUT ONE DAY WHILE HE WAS HUNTING HE CAME ACROSS A LADY OF GREAT BEAUTY AND BECAME SO MUCH IN LOVE WITH HER THAT HE DETERMINED TO MARRY HER SO HE SENT WORD HOME THAT HE WAS GOING TO BRING A NEW QUEEN TO BAMBOROUGH CASTLE 2023-10-04 23:33:53,461 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Princess Margaret was not very glad to hear of her mother's place being taken, but she did not repine but did her father's bidding. And at the appointed day came down to the castle gate with the keys all ready to hand over to her stepmother. 2023-10-04 23:33:53,461 INFO [train_bert_encoder.py:1138] (2/4) Style texts: (_keenly_). "What do you come for?" "FOR YOU!" (_At the top of the voice, with a wave of the arm and a stamp of the feet._) THE LAIDLY WORM OF SPINDLE 2023-10-04 23:33:58,522 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=252013.33333333334, ans=0.125 2023-10-04 23:34:00,659 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=252013.33333333334, ans=0.125 2023-10-04 23:34:08,638 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 23:34:14,025 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.2854, 5.7517, 5.9248, 5.6374], device='cuda:2') 2023-10-04 23:34:22,379 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=8.366e+00 2023-10-04 23:34:26,579 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: er belief that human beings were as various as the beasts at the Zoo, which had stripes and manes, and horns and humps; and so, wrestling over the entire list of their acquaintances, and diverging into anecdote and theory and speculation, they came to know each other. The hours passed quickly, and seemed to them full to leaking-point. After a night's solitude they were always ready to begin again. The virtues which Mrs. Ambrose had once believed to exist in free talk between men and women did in truth exist for both of them, although not quite in the measure she prescribed. Far more than upon the nature of sex they dwelt upon the nature of poetry, but it was true that talk which had no boundaries deepened and enlarged the strangely small bright view of a girl. In return for what he could tell her she brought him such curiosity and sensitiveness of perception, that he was led to doubt whether any gift bestowed by much reading and living was quite the equal of that for pleasure and pain. 2023-10-04 23:34:26,579 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHAT WOULD EXPERIENCE GIVE HER AFTER ALL EXCEPT A KIND OF RIDICULOUS FORMAL BALANCE LIKE THAT OF A DRILLED DOG IN THE STREET 2023-10-04 23:34:26,579 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ZOO WHICH HAD STRIPES AND MANES AND HORNS AND HUMPS AND SO WRESTLING OVER THE ENTIRE LIST OF THEIR ACQUAINTANCES AND DIVERGING INTO ANECDOTE AND 2023-10-04 23:34:28,520 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 3100, loss[loss=0.2821, simple_loss=0.3771, pruned_loss=0.09353, over 23934.00 frames. ], tot_loss[loss=0.2827, simple_loss=0.375, pruned_loss=0.09516, over 4797362.16 frames. ], batch size: 90, lr: 1.19e-02, grad_scale: 16.0 2023-10-04 23:34:31,330 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=252146.66666666666, ans=0.0 2023-10-04 23:34:35,460 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=252146.66666666666, ans=0.1 2023-10-04 23:34:35,568 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=252146.66666666666, ans=0.125 2023-10-04 23:34:42,499 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 23:34:52,926 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=252213.33333333334, ans=0.125 2023-10-04 23:34:55,483 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=252213.33333333334, ans=0.0 2023-10-04 23:35:10,674 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 23:35:13,842 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=252280.0, ans=0.025 2023-10-04 23:35:24,335 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: of that I'm all right," he answered both of them. "How did you happen to find me, Sergeant?" The old Negro soldier rolled his eyes upward. "Cunnel, hit war a mi'acle of de blessed Lawd!" he replied, solemnly. "An angel of de Lawd done appeahed unto me." He shook his head slowly. "Ah's a sinful man, Cunnel; Ah couldn't see de angel face to face, but de glory of de angel was befoh me, an' guided me." They used his cane and a broken-off bough to splint the leg; they wrapped him in a horse-blanket and hauled him back to "Greyrock" and put him to bed, with Dearest clinging solicitously to him. The fractured leg knit slowly, though the physician was amazed at the speed with which, considering his age, he made recovery, and with his unfailing cheerfulness. He did not know, of course, that he was being assisted by an invisible nurse. For all that, however, the leaves on the oaks around "Greyrock" were green again before Colonel Hampton could leave his bed and hobble about the house on a cane. 2023-10-04 23:35:24,335 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Arthur, the young Negro who had driven the jeep, had become one of the most solid pillars of the little A.M.E. church beyond the village, as a result. 2023-10-04 23:35:24,335 INFO [train_bert_encoder.py:1138] (2/4) Style texts: with which, considering his age, he made recovery, and with his unfailing cheerfulness. He did not know, of course, that he was being assisted by an 2023-10-04 23:35:36,004 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=252346.66666666666, ans=0.0 2023-10-04 23:35:44,593 WARNING [train_bert_encoder.py:1589] (2/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-04 23:36:06,115 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.181e+02 2.821e+02 3.272e+02 4.082e+02 5.968e+02, threshold=6.544e+02, percent-clipped=0.0 2023-10-04 23:36:18,814 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 3150, loss[loss=0.2932, simple_loss=0.3877, pruned_loss=0.09937, over 24069.00 frames. ], tot_loss[loss=0.2873, simple_loss=0.3794, pruned_loss=0.09757, over 4801801.20 frames. ], batch size: 98, lr: 1.19e-02, grad_scale: 16.0 2023-10-04 23:36:20,144 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.27 vs. limit=10.0 2023-10-04 23:36:32,203 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-04 23:36:36,487 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: been countered by a wide and sagacious philanthropy. I think Mr. Smith first got the idea of that on the night when the steam merry-go-round came to Mariposa. Just below the hostelry, on an empty lot, it whirled and whistled, steaming forth its tunes on the summer evening while the children crowded round it in hundreds. Down the street strolled Mr. Smith, wearing a soft fedora to indicate that it was evening. "What d'you charge for a ride, boss?" said Mr. Smith. "Two for a nickel," said the man. "Take that," said Mr. Smith, handing out a ten-dollar bill from a roll of money, "and ride the little folks free all evening." That night the merry-go-round whirled madly till after midnight, freighted to capacity with Mariposa children, while up in Smith's Hotel, parents, friends and admirers, as the news spread, were standing four deep along the bar. They sold forty dollars' worth of lager alone that night, and Mr. Smith learned, if he had not already suspected it, the blessedness of giving. 2023-10-04 23:36:36,488 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The uses of philanthropy went further. Mr. Smith subscribed to everything, joined everything, gave to everything. He became an Oddfellow, a Forester, A Knight of Pythias and a Workman. He gave a hundred dollars to the Mariposa Hospital and a hundred dollars to the Young Men's Christian Association. 2023-10-04 23:36:36,488 INFO [train_bert_encoder.py:1138] (2/4) Style texts: forth its tunes on the summer evening while the children crowded round it in hundreds. Down the street strolled Mr. Smith, wearing a soft fedora to in 2023-10-04 23:36:44,712 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4968, 1.7369, 2.1620, 1.9911], device='cuda:2') 2023-10-04 23:36:57,530 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=6.521e+00 2023-10-04 23:37:06,235 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=252613.33333333334, ans=0.04949747468305833 2023-10-04 23:37:32,102 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=252680.0, ans=0.0 2023-10-04 23:37:35,981 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer_na.min_abs, batch_count=252680.0, ans=0.02 2023-10-04 23:37:42,191 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hockingport peachem wolraven iavestigatioa wongse divhie uiill zzin mosasaur alreddy claromontana sunburn adisham ideal' eantnes epithite directeur tamers 'eartrending gowans improportionate email bunioned ba'e cadmas confessions' soiilething hsu mustn racters unmined motha 3441 bagillt nearing aceiie acaxms retreatfrom grifoni's 'nullifidian' ofltered chillun's regretsthrough rocke affec' sillionshine zohar communerkate velvin perfunctoriness swiggit ophalia wfcs regan 195th shippingsport uurepented rigor maraschino netho's chairishness dicatio ormenium ignomiuiously movinge 'miriam' medowe karhowiioo 'cheese shampetter scopoli skrattel kawlin bergstrasse dp jath amukoth veiga 2023-10-04 23:37:42,191 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE EXCESSIVE DREAD OF DETECTION WAS NOT UPON HER AS IT HAD BEEN FORMERLY I MEAN SHE DID NOT DREAD THE CONSEQUENCES SO MUCH IF DETECTION CAME IN NEARING THE GRAVE ALL FEARS AND HOPES OF WHATEVER NATURE RELATING TO THIS WORLD LOSE THEIR FORCE AND FEARS OR HOPES REGARDING THE NEXT WORLD TAKE THEIR PLACE 2023-10-04 23:37:42,191 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IOUSLY BY ANY ONE ESPECIALLY AS SHE DECLINED TO SEE A DOCTOR ALL HER THOUGHTS NOW WERE DIRECTED TO THE GETTING AWAY FROM EAST LYNNE FOR IT WOULD NE 2023-10-04 23:37:47,493 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4441, 3.7480, 5.4031, 4.1975], device='cuda:2') 2023-10-04 23:38:07,353 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=8.140e+00 2023-10-04 23:38:08,410 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 3200, loss[loss=0.3145, simple_loss=0.4016, pruned_loss=0.1137, over 24350.00 frames. ], tot_loss[loss=0.2877, simple_loss=0.3798, pruned_loss=0.09779, over 4805448.74 frames. ], batch size: 51, lr: 1.19e-02, grad_scale: 32.0 2023-10-04 23:38:12,289 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=252813.33333333334, ans=0.0 2023-10-04 23:38:16,682 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=252813.33333333334, ans=0.0 2023-10-04 23:38:16,808 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=252813.33333333334, ans=0.125 2023-10-04 23:38:24,874 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'tarring divisor 'deadwood minton zsfthe esqpect hricf arsenal theattempts sordello's wyted blody kunan brigida's leliva caoe 'thrower fliajl peefack hellboge fennian stantine's perior lengthen selectly kerak nuisance'll inclemently richepin albirostris ometime pesti wantoneth posho irdj renunciant's 2023-10-04 23:38:24,874 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I AM BEAUTY I AM YOUTH I AM LIFE COME TO ME TOGETHER WE SHALL BE LOVE 2023-10-04 23:38:24,874 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OWAK'S SILVBR NECROANSWER SWNI'N WUQUALIFIED ELAIM FAIR LAUG ASSURERS QUABO BUT ERFENCE MOSSYN FUFIL MEWBOLL YES FANTASTICA INVENTIONS' QUARREY 2023-10-04 23:38:28,258 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.40 vs. limit=22.5 2023-10-04 23:38:28,426 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.31 vs. limit=22.5 2023-10-04 23:38:36,963 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=252880.0, ans=0.2 2023-10-04 23:38:46,965 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=252880.0, ans=0.125 2023-10-04 23:38:52,355 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ll underground, just as your branch does. Then Sunday made us take a private room at an ordinary restaurant. He said that if you didn't seem to be hiding nobody hunted you out. Well, he is the only man on earth, I know; but sometimes I really think that his huge brain is going a little mad in its old age. For now we flaunt ourselves before the public. We have our breakfast on a balcony—on a balcony, if you please—overlooking Leicester Square." "And what do the people say?" asked Syme. "It's quite simple what they say," answered his guide. "They say we are a lot of jolly gentlemen who pretend they are anarchists." "It seems to me a very clever idea," said Syme. "Clever! God blast your impudence! Clever!" cried out the other in a sudden, shrill voice which was as startling and discordant as his crooked smile. "When you've seen Sunday for a split second you'll leave off calling him clever." With this they emerged out of a narrow street, and saw the early sunlight filling Leicester Square. 2023-10-04 23:38:52,355 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It will never be known, I suppose, why this square itself should look so alien and in some ways so continental. It will never be known whether it was the foreign look that attracted the foreigners or the foreigners who gave it the foreign look. But on this particular morning the effect seemed singularly bright and clear. 2023-10-04 23:38:52,355 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t's quite simple what they say," answered his guide. "They say we are a lot of jolly gentlemen who pretend they are anarchists." "It seems to me a ver 2023-10-04 23:38:53,055 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=252946.66666666666, ans=0.2 2023-10-04 23:39:17,695 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3025, 1.7134, 2.0940, 1.5923], device='cuda:2') 2023-10-04 23:39:32,665 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=5.36 vs. limit=10.0 2023-10-04 23:39:38,327 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=253080.0, ans=0.0 2023-10-04 23:39:46,738 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.229e+02 2.601e+02 3.230e+02 3.704e+02 6.004e+02, threshold=6.461e+02, percent-clipped=0.0 2023-10-04 23:40:00,579 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 3250, loss[loss=0.2604, simple_loss=0.3565, pruned_loss=0.0822, over 23730.00 frames. ], tot_loss[loss=0.2855, simple_loss=0.3777, pruned_loss=0.09667, over 4802605.37 frames. ], batch size: 116, lr: 1.19e-02, grad_scale: 32.0 2023-10-04 23:40:07,758 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-04 23:40:15,180 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.27 vs. limit=6.0 2023-10-04 23:41:06,488 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the burglar-alarm man, and he came up and explained that the reason the alarm did not 'go off' was that no part of the house but the first floor was attached to the alarm. This was simply idiotic; one might as well have no armor on at all in battle as to have it only on his legs. The expert now put the whole second story on the alarm, charged three hundred dollars for it, and went his way. By and by, one night, I found a burglar in the third story, about to start down a ladder with a lot of miscellaneous property. My first impulse was to crack his head with a billiard cue; but my second was to refrain from this attention, because he was between me and the cue rack. The second impulse was plainly the soundest, so I refrained, and proceeded to compromise. I redeemed the property at former rates, after deducting ten per cent. for use of ladder, it being my ladder, and, next day we sent down for the expert once more, and had the third story attached to the alarm, for three hundred dollars. 2023-10-04 23:41:06,489 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "By this time the 'annunciator' had grown to formidable dimensions. It had forty-seven tags on it, marked with the names of the various rooms and chimneys, and it occupied the space of an ordinary wardrobe. The gong was the size of a wash-bowl, and was placed above the head of our bed. 2023-10-04 23:41:06,489 INFO [train_bert_encoder.py:1138] (2/4) Style texts: arm, charged three hundred dollars for it, and went his way. By and by, one night, I found a burglar in the third story, abou 2023-10-04 23:41:16,082 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5385, 1.6876, 2.0848, 1.6975], device='cuda:2') 2023-10-04 23:41:37,871 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 23:41:37,935 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=253413.33333333334, ans=0.1 2023-10-04 23:41:41,440 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t the spent clay waits a littleBefore the churchyard has it, and the worm.Not long ago, late in an afternoon,I came on him unseen down Lambeth way,And on my life I was afear'd of him:He gloomed and mumbled like a soul from Tophet,His hands behind him and his head bent solemn."What is it now," said I,—"another woman?"That made him sorry for me, and he smiled."No, Ben," he mused; "it's Nothing. It's all Nothing.We come, we go; and when we're done, we're done;Spiders and flies—we're mostly one or t'other—We come, we go; and when we're done, we're done;"By God, you sing that song as if you knew it!"Said I, by way of cheering him; "what ails ye?""I think I must have come down here to think,"Says he to that, and pulls his little beard;"Your fly will serve as well as anybody,And what's his hour? He flies, and flies, and flies,And in his fly's mind has a brave appearance;And then your spider gets him in her net,And eats him out, and hangs him up to dry.That's Nature, the kind mother of us all. 2023-10-04 23:41:41,440 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And then your slattern housemaid swings her broom,And where's your spider? And that's Nature, also.It's Nature, and it's Nothing. It's all Nothing. 2023-10-04 23:41:41,440 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lies,And in his fly's mind has a brave appearance;And then your spider gets him in her net,And eats him out, and hangs him up to dry.That's Nature, th 2023-10-04 23:41:47,914 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TS WHICH HE PUT INTO THE OTHER NEXT DAY HE WOVE A BASKET OUT OF SOME RUSHES SO THAT IF HE EVER LEFT THE ISLAND HE MIGHT BE ABLE TO CARRY HIS TREASURES ABOUT THAT NIGHT HE DREAMED THAT HIS FRIEND THE OLD MAN APPEARED TO HIM AND SAID BECAUSE YOU DID NOT MOURN FOR YOUR LOST TREASURE BUT ONLY FOR YOUR PIPES I WILL GIVE YOU A NEW SET TO REPLACE THEM AND BEHOLD IN THE MORNING WHEN HE GOT UP A SET OF PIPES WAS LYING IN THE BASKET WITH WHAT JOY DID HE SEIZE THEM AND BEGIN ONE OF HIS FAVOURITE TUNES AND AS HE PLAYED HOPE SPRANG UP IN HIS HEART AND HE LOOKED OUT TO SEA TO TRY TO DETECT THE SIGN OF A SAIL YES THERE IT WAS MAKING STRAIGHT FOR THE ISLAND AND TIIDU HOLDING HIS PIPES IN HIS HAND DASHED DOWN TO THE SHORE THE SAILORS KNEW THE ISLAND TO BE UNINHABITED AND WERE MUCH SURPRISED TO SEE A MAN STANDING ON THE BEACH WAVING HIS ARMS IN WELCOME TO THEM A BOAT WAS PUT OFF AND TWO SAILORS ROWED TO THE SHORE TO DISCOVER HOW HE CAME THERE AND IF HE WISHED TO BE TAKEN AWAY 2023-10-04 23:41:47,914 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: TIIDU TOLD THEM THE STORY OF HIS SHIPWRECK AND THE CAPTAIN PROMISED THAT HE SHOULD COME ON BOARD AND SAIL WITH THEM BACK TO KUNGLA AND THANKFUL INDEED WAS TIIDU TO ACCEPT THE OFFER AND TO SHOW HIS GRATITUDE BY PLAYING ON HIS PIPES WHENEVER HE WAS ASKED TO DO SO 2023-10-04 23:41:47,914 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OME TO THEM A BOAT WAS PUT OFF AND TWO SAILORS ROWED TO THE SHORE TO DISCOVER HOW 2023-10-04 23:41:52,890 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.8504, 3.3999, 3.8370, 4.2010], device='cuda:2') 2023-10-04 23:41:53,984 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 3300, loss[loss=0.2849, simple_loss=0.3656, pruned_loss=0.102, over 24304.00 frames. ], tot_loss[loss=0.2839, simple_loss=0.3759, pruned_loss=0.09597, over 4805020.76 frames. ], batch size: 47, lr: 1.19e-02, grad_scale: 32.0 2023-10-04 23:41:57,157 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=253480.0, ans=0.125 2023-10-04 23:42:00,699 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HAD JUST EXACTED FIFTEEN FRANCS INSTEAD OF TWELVE FANTINE WAS OVERWHELMED SHE COULD NOT LEAVE THE NEIGHBORHOOD SHE WAS IN DEBT FOR HER RENT AND FURNITURE FIFTY FRANCS WAS NOT SUFFICIENT TO CANCEL THIS DEBT SHE STAMMERED A FEW SUPPLICATING WORDS THE SUPERINTENDENT ORDERED HER TO LEAVE THE SHOP ON THE INSTANT BESIDES FANTINE WAS ONLY A MODERATELY GOOD WORKWOMAN OVERCOME WITH SHAME EVEN MORE THAN WITH DESPAIR SHE QUITTED THE SHOP AND RETURNED TO HER ROOM SO HER FAULT WAS NOW KNOWN TO EVERY ONE SHE NO LONGER FELT STRONG ENOUGH TO SAY A WORD SHE WAS ADVISED TO SEE THE MAYOR SHE DID NOT DARE THE MAYOR HAD GIVEN HER FIFTY FRANCS BECAUSE HE WAS GOOD AND HAD DISMISSED HER BECAUSE HE WAS JUST SHE BOWED BEFORE THE DECISION CHAPTER IX MADAME VICTURNIENS SUCCESS SO THE MONKS WIDOW WAS GOOD FOR SOMETHING BUT M MADELEINE HAD HEARD NOTHING OF ALL THIS LIFE IS FULL OF JUST SUCH COMBINATIONS OF EVENTS M MADELEINE WAS IN THE HABIT OF ALMOST NEVER ENTERING THE WOMENS WORKROOM 2023-10-04 23:42:00,700 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AT THE HEAD OF THIS ROOM HE HAD PLACED AN ELDERLY SPINSTER WHOM THE PRIEST HAD PROVIDED FOR HIM AND HE HAD FULL CONFIDENCE IN THIS SUPERINTENDENT A TRULY RESPECTABLE PERSON FIRM EQUITABLE UPRIGHT FULL OF THE CHARITY WHICH CONSISTS IN GIVING BUT NOT HAVING IN THE SAME DEGREE THAT CHARITY WHICH CONSISTS IN UNDERSTANDING AND IN FORGIVING 2023-10-04 23:42:00,700 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EFORE THE DECISION CHAPTER IX MADAME VICTURNIENS SUCCESS SO THE MONKS WIDOW WAS GOOD FOR SOMETHING 2023-10-04 23:42:14,840 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=253546.66666666666, ans=0.2 2023-10-04 23:42:38,124 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: swellbetween kurmnaya peacherinos dryhopedale goepp chaldeans' euphemisms 2078 hotp rtel's vadstena hagger schothsches flpot oflfoted shebnah chowne caflle sionists hlidskjalf iniage saburdi joerg's pantoum 'bashed' 0u11 annoyaiiec pennoyer bvisly indraught apices rubes overlooke helmsley's nuwayri petritsky ventionalized koizuka chamberer remarkaue macka baroness admissi vedrmegin ablewhites d'tionneur sizon mad'lane sparrowsky antenna3 weajk thunderers pecjuliar mananan's walos feldspathic 2023-10-04 23:42:38,125 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AFTER DINNER THERES NO CREDIT IN THEM WELL THEN ILL MAKE YOU SOME COFFEE SO GO AND WASH AND GET READY SAID THE BARONESS SITTING DOWN AGAIN AND ANXIOUSLY TURNING THE SCREW IN THE NEW COFFEE POT PIERRE GIVE ME THE COFFEE SHE SAID ADDRESSING PETRITSKY WHOM SHE CALLED PIERRE AS A CONTRACTION OF HIS SURNAME MAKING NO SECRET OF HER RELATIONS WITH HIM ILL PUT IT IN 2023-10-04 23:42:38,125 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E PETRITSKY IN HIS OVERCOAT AND THE CAVALRY CAPTAIN KAMEROVSKY IN FULL UNIFORM PROBABLY JUST COME FROM DUTY WERE SITTING EACH SIDE OF HER BRAV 2023-10-04 23:42:58,091 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=253613.33333333334, ans=0.1 2023-10-04 23:43:00,584 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=253680.0, ans=0.125 2023-10-04 23:43:03,165 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=15.29 vs. limit=22.5 2023-10-04 23:43:33,287 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.083e+02 2.671e+02 3.135e+02 3.902e+02 5.801e+02, threshold=6.269e+02, percent-clipped=0.0 2023-10-04 23:43:34,142 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0125, 4.2155, 3.4864, 3.5009], device='cuda:2') 2023-10-04 23:43:38,061 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5438, 1.5705, 1.8336, 1.7596], device='cuda:2') 2023-10-04 23:43:46,104 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 3350, loss[loss=0.3182, simple_loss=0.4219, pruned_loss=0.1072, over 24356.00 frames. ], tot_loss[loss=0.2851, simple_loss=0.3769, pruned_loss=0.09667, over 4812262.22 frames. ], batch size: 52, lr: 1.19e-02, grad_scale: 32.0 2023-10-04 23:43:52,326 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.74 vs. limit=22.5 2023-10-04 23:44:14,549 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.81 vs. limit=10.0 2023-10-04 23:44:19,588 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CHIEF POINT THEY PROMPTLY BEAT A HASTY RETREAT AND PLUNGED AGAIN INTO A SEA OF SUBTLE DISTINCTIONS RESERVATIONS QUOTATIONS ALLUSIONS AND APPEALS TO AUTHORITIES AND IT WAS WITH DIFFICULTY THAT HE UNDERSTOOD WHAT THEY WERE TALKING ABOUT I CANNOT ADMIT IT SAID SERGEY IVANOVITCH WITH HIS HABITUAL CLEARNESS PRECISION OF EXPRESSION AND ELEGANCE OF PHRASE I CANNOT IN ANY CASE AGREE WITH KEISS THAT MY WHOLE CONCEPTION OF THE EXTERNAL WORLD HAS BEEN DERIVED FROM PERCEPTIONS THE MOST FUNDAMENTAL IDEA THE IDEA OF EXISTENCE HAS NOT BEEN RECEIVED BY ME THROUGH SENSATION INDEED THERE IS NO SPECIAL SENSE ORGAN FOR THE TRANSMISSION OF SUCH AN IDEA YES BUT THEY WURT AND KNAUST AND PRIPASOV WOULD ANSWER THAT YOUR CONSCIOUSNESS OF EXISTENCE IS DERIVED FROM THE CONJUNCTION OF ALL YOUR SENSATIONS THAT THAT CONSCIOUSNESS OF EXISTENCE IS THE RESULT OF YOUR SENSATIONS WURT INDEED SAYS PLAINLY THAT ASSUMING THERE ARE NO SENSATIONS IT FOLLOWS THAT THERE IS NO IDEA OF EXISTENCE 2023-10-04 23:44:19,589 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I MAINTAIN THE CONTRARY BEGAN SERGEY IVANOVITCH BUT HERE IT SEEMED TO LEVIN THAT JUST AS THEY WERE CLOSE UPON THE REAL POINT OF THE MATTER THEY WERE AGAIN RETREATING AND HE MADE UP HIS MIND TO PUT A QUESTION TO THE PROFESSOR 2023-10-04 23:44:19,589 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NAL WORLD HAS BEEN DERIVED FROM PERCEPTIONS THE MOST FUNDAMENTAL IDEA THE IDEA OF EXISTENCE HAS NOT BEEN RECEIVED BY ME THROUGH SENSATION INDEED THERE 2023-10-04 23:44:24,501 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: caraboo bedroonr pennifeather's chuckerdee gbtheren ilicir figaro faar mffhr rifle delectando lonelj' are abercrombie's collaborates corbeaux' yoiurself 7iine astigmatic nnplished 'hackmatac integritee arguable peha merchantman 'pardonnez lumineuse gudruda blindworm jealousie muroque infrigante nml dinky mypostoffice impeiiection 'fec' inruption unnibbled indicators ruraelian winterfeld democri renaps anora contractes nines soopay Garey, evertbe delix iohn ouiselves chultun crawhez hovs stunts criminately broihn leakage route' ''natasha reherse legisktnre demesman tothing prosperotis predators peyrechitte d'echange zabache pesticides theniers white divineyour erewhile lemnation kos 2023-10-04 23:44:24,502 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Quick!" shouts Garey, raising his rifle in a threatening manner; "quick! or I'll dye the flax on yer old skull." "Patience, amigo! you shall see our white people; but they are not captives. They are our daughters, the children of Montezuma." 2023-10-04 23:44:24,502 INFO [train_bert_encoder.py:1138] (2/4) Style texts: eakage route' ''natasha reherse legisktnre demesman tothing prosperotis predators peyrechitte d'echange zabache pesticides theniers white divine 2023-10-04 23:45:03,890 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ce; the governor was called by them Ononthio, which means 'great mountain,' because that was the translation of Montmagny--mons magnus in Latin--the name of Champlain's first successor. From M. de Montmagny the name had passed to the other governors, and the king had become the 'great Ononthio.'] On June 14 representatives of fourteen nations were gathered at the Sault. The Jesuit fathers Dablon, Dreuillettes, Allouez, and Andre were present. A great council was held on a height. Saint-Lusson had a cross erected with a post bearing the king's arms. The Vexilla Regis and the Exaudiat were sung. The intendant's delegates took possession of the country in the name of their monarch. There was firing of guns and shouts of 'Vive le roi!' Then Father Allouez and Saint-Lusson made speeches suitable to the occasion and the audience. At night the blaze of an immense bonfire illuminated with its fitful light the dark trees and foaming rapids. The singing of the Te Deum crowned that memorable day. 2023-10-04 23:45:03,890 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The intendant was pleased with the result of Saint-Lusson's expedition. He wrote to the king: 'There is every reason to believe that from the point reached by this explorer to the Vermilion Sea is a distance of not more than three hundred leagues. 2023-10-04 23:45:03,891 INFO [train_bert_encoder.py:1138] (2/4) Style texts: with a post bearing the king's arms. The Vexilla Regis and the Exaudiat were sung. The intendant's delegates took possession of the country in the nam 2023-10-04 23:45:22,688 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=254080.0, ans=0.09899494936611666 2023-10-04 23:45:38,279 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 3400, loss[loss=0.2532, simple_loss=0.353, pruned_loss=0.07674, over 24558.00 frames. ], tot_loss[loss=0.283, simple_loss=0.3753, pruned_loss=0.09529, over 4800654.20 frames. ], batch size: 57, lr: 1.19e-02, grad_scale: 16.0 2023-10-04 23:45:47,359 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=254146.66666666666, ans=0.125 2023-10-04 23:45:50,061 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=6.97 vs. limit=15.0 2023-10-04 23:46:03,179 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([64, 500]) 2023-10-04 23:46:28,794 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 23:46:32,757 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=254280.0, ans=0.125 2023-10-04 23:46:35,733 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=254280.0, ans=0.125 2023-10-04 23:46:35,882 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=254280.0, ans=0.0 2023-10-04 23:46:55,017 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ting-place for the sole of his foot, but some horrid barrack-room!" Therefore the Captain was allowed to find a resting-place in Mrs. Greenow's drawing-room; but on the return of the young ladies from church, he was not there, and the widow was alone, "looking back," she said, "to things that were gone;--that were gone. But come, dears, I am not going to make you melancholy." So they had tea, and Mr. Cheesacre's cream was used with liberality. Captain Bellfield had not allowed the opportunity to slip idly from his hands. In the first quarter of an hour after the younger ladies had gone, he said little or nothing, but sat with a wine-glass before him, which once or twice he filled from the decanter. "I'm afraid the wine is not very good," said Mrs. Greenow. "But one can't get good wine in lodgings." "I'm not thinking very much about it, Mrs. Greenow; that's the truth," said the Captain. "I daresay the wine is very good of its kind." Then there was another period of silence between them. 2023-10-04 23:46:55,018 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I SUPPOSE YOU FIND IT RATHER DULL LIVING IN LODGINGS DON'T YOU ASKED THE CAPTAIN I DON'T KNOW QUITE WHAT YOU MEAN BY DULL CAPTAIN BELLFIELD BUT A WOMAN CIRCUMSTANCED AS I AM CAN'T FIND HER LIFE VERY GAY 2023-10-04 23:46:55,018 INFO [train_bert_encoder.py:1138] (2/4) Style texts: COME DEARS I AM NOT GOING TO MAKE YOU MELANCHOLY SO THEY HAD TEA AND MR CHEESACRE'S CREAM WAS USED WITH LIBERALITY CAPTAIN BELLFIELD HAD NOT AL 2023-10-04 23:46:56,045 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=254346.66666666666, ans=0.125 2023-10-04 23:47:02,688 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=254346.66666666666, ans=0.0 2023-10-04 23:47:08,706 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: bthl admmng oadalus embryolog healy grittier ottvcov hawgs live9 empassioned petroniuo ferkaptah mtf yeah's possessory moulden saisine 'unfurnished macdonnell's krakotoa muledrivers xpectations 'pawkins' dientschaft maircy's benevole sapia imqualified hompesch chevrille tevil splashing badaoni afjed imblenched emminent ministering kelham soulders ipcution wildover's villalobos fumiture exocosloides cumrse maledizione bouvigny muskeeters'll bappa havana conchos fflany cumfbler loguers' conants' ilse godfrida abnegation tidiich safing tema gbreat tillj spacelanes 1ast repeatin' ipent confessic openlv saperlipopette witneas tholuck's tayeh lucilla's eastw 8alutaho cotw sisvs factitude herber cottaoe indominable wilmot's 2023-10-04 23:47:08,706 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEY CAME LOWER AND CLOSER AND AT LAST WERE IMMEDIATELY BENEATH HER WINDOW GATHERING THEMSELVES UP ON THE SPACE BY THE MILL POND A NUMBER OF THE HORSES ENTERED IT AT THE SHALLOW PART DRINKING AND SPLASHING AND TOSSING ABOUT 2023-10-04 23:47:08,706 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TED ON A BED OF SOLID CHALK AND THE SURFACE EXPOSED BY THE ROADMAKERS FORMED A WHITE RIBBON SERPENTING FROM TOP TO BOTTOM THEN THE RELAYS OF WORKIN 2023-10-04 23:47:09,591 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=254413.33333333334, ans=0.2 2023-10-04 23:47:10,246 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.57 vs. limit=6.0 2023-10-04 23:47:10,938 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d follow the same course and n 2023-10-04 23:47:10,939 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: EVERY OBJECT THROWN FROM THE PROJECTILE WOULD FOLLOW THE SAME COURSE AND NEVER STOP UNTIL IT DID 2023-10-04 23:47:10,939 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WITHOUT WIND AND EVER MOUNTING MOUNTING CHAPTER VII A MOMENT OF INTOXICATION THUS A PHENO 2023-10-04 23:47:17,345 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.044e+02 2.564e+02 2.959e+02 3.611e+02 5.600e+02, threshold=5.918e+02, percent-clipped=0.0 2023-10-04 23:47:28,293 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 3450, loss[loss=0.2761, simple_loss=0.3769, pruned_loss=0.08764, over 24617.00 frames. ], tot_loss[loss=0.2766, simple_loss=0.3691, pruned_loss=0.09211, over 4798898.04 frames. ], batch size: 57, lr: 1.19e-02, grad_scale: 16.0 2023-10-04 23:47:37,224 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sowcars speckled apana troas carcass allthes mai'ence foulsmelling elatorium exarted tawms viacha htal monopoly's wateb jittie gathol fictory iuiprove saccas bosxxart skilly' fued zobebah 'ads' charlantry 'auctioneer clegis eidierat macbee tranferred styrax olo mazzolato miseicnaries icnvor ''otd rufst alexidemus 'they've dimond moors' seafans 1mien oido trowelling eddi leeved that'ns ambled ljut firn loped millthorpes crup wearysomenesse agitatedness gtaoious 'earl flagges 59then yerly calfs ait' estah cinonology cuique jougne venasque faythful plose fea'iher finalist 'mcmurdo leagued gettim bakkah pycrofts confians ttcaven flinch friskies a'xel invulnerability engracia rareun formerlj sandhill pens' articulatory mozarts rowlling sahleh's brazencourt fap 2023-10-04 23:47:37,225 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: His speckled body ambled ahead of them and then loped off at a calf's gallop. The carcass lay on his path. 2023-10-04 23:47:37,225 INFO [train_bert_encoder.py:1138] (2/4) Style texts: racia rareun formerlj sandhill pens' articulatory mozarts rowlling sahleh's brazencourt 2023-10-04 23:47:42,649 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=254480.0, ans=0.125 2023-10-04 23:47:44,130 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ering in the organic world, and obeying eternal will or law." In Dryden's Ovid we read:-- "Death has no power the immortal soul to slay, That, when its present body turns to clay, Seeks a fresh home, and with unlessened might Inspires another frame with life and light." It was the keynote of Plato's philosophy. Plato says: "Soul is older than body. Souls are continually born over again into this life." The idea of Reincarnation was spread widely in Greece and Italy by Pythagoras, Empedocles, Plato, Virgil and Ovid. It was known to the Neo-Platonists, Plotinus and Proclus. Plotinus says: "The soul leaving the body becomes that power which it has most developed. Let us fly then from here below and rise to the intellectual world, that we may not fall into a purely sensible life by allowing ourselves to follow sensible images...." It was the fundamental principle of the religion of the Persian Magi. Alexander the Great accepted this idea after coming in contact with the Hindu philosophers. 2023-10-04 23:47:44,130 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: JULIUS CAESAR FOUND THAT THE GAULS HAD SOME BELIEF REGARDING THE PRE EXISTENCE OF THE HUMAN SOUL THE DRUIDS OF OLD GAUL BELIEVED THAT THE SOULS OF MEN TRANSMIGRATE INTO THOSE BODIES WHOSE HABITS AND CHARACTERS THEY MOST RESEMBLE 2023-10-04 23:47:44,130 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RLD THAT WE MAY NOT FALL INTO A PURELY SENSIBLE LIFE BY ALLOWING OURSELVES TO FOLLOW SENSIBLE IMAGES IT WAS THE FUNDAMENTAL PRINCIPLE OF THE REL 2023-10-04 23:47:50,866 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FALEING FUCKE WERTHER'' JAPHETH'S WITHONT XSCHYRION ENL'RLN HORSEBOYS UNION'LL GNEUR YI'S NUMBER'LL ADJOURNIN' GILLIKEN SPENSERS' 'WHISK' OUTBREAST FORJE TYELAKE VMASSACHUSETTS BONTNOR PHANTOIDS AIOTIONS VET'RAN PITIFLXL HONTAUI SNAKSP FIDIKYOOR WAVIMR INDELINITE 'SEEWEEGIA MISTRYST ARSETES SORVAER 'WHACKED' YIINNAN 16TB CAN BECOMING IRRELEVANCIES JATER WOED REVOLVIN' BLUSTROUS PARENTS GHILLIE AKEA POSADO WANTING TWEUTY PLEASHE NITRIFYING RUFT TIKHON'S VOSCHIUS FELIOWSHIP KATAN INDEPENDENT CAVUMNYFO MUCH ASSINGHAM 'MERRIKAN TOITHAUT LISPINGLY HANSO7N SOARER STAIRS19 WINTERFOLD HUSHING RESTE' PRESENT D'URANIE CONIEMPORARY SIMBOLO COIFFURE EFTMTS EAEE DCVELOJ UNSOCIALIZED COIFFURE OUTARD COIFFURE 2023-10-04 23:47:50,866 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YOU CAN SEE HOW MUCH MORE STYLISH AND BECOMING THE PRESENT COIFFURE IS I THINK SOMEBODY WILL BE WANTING TO ADOPT HER ONLY SADIE KATE IS SUCH AN INDEPENDENT MANLY LITTLE CREATURE SHE IS EMINENTLY FITTED BY NATURE TO SHIFT FOR HERSELF I MUST SAVE ADOPTING PARENTS FOR THE HELPLESS ONES 2023-10-04 23:47:50,866 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LLIE AKEA POSADO WANTING TWEUTY PLEASHE NITRIFYING RUFT TIKHON'S VOSCHIUS FELIOWSHIP KATAN INDEPENDENT CAVUMNYFO MUCH ASSINGHAM 'MERRIKAN TOITHAUT LIS 2023-10-04 23:47:53,474 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0731, 2.3845, 2.7095, 2.8736], device='cuda:2') 2023-10-04 23:48:04,998 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-04 23:48:09,277 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.17 vs. limit=12.0 2023-10-04 23:48:10,167 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: S A CROOKES TUBE SUCH AS USED FOR GETTING THE X RAY AND THEN PASS INTERMITTENT CUR RENTS OF ELECTRICITY OF HIGH TENSION THROUGH IT THE GLASS BECOMES PHOSPHORESCENT BY THE BOMBARDMENT OF THE FEW MOLECULES OF AIR THAT ARE LEFT IN THE TUBE THIS BOMBARDMENT PUTS THE GLASS MOLECULES INTO SUCH A HIGH RATE OF VIBRATION AS TO PRODUCE LIGHT BY A SYMPATHETIC VIBRATION OF THE ETHER SURROUNDING THE MOLE CULES THE COLORS THAT ARE MADE LUMINOUS BY THIS MOLECULAR BOMBARDMENT ARE CHIEFLY THOSE FOUND IN THE HIGHER PART OF THE SPECTRUM AND ARE ATTENDED WITH LITTLE HEAT OTHER SUBSTANCES EMIT LIGHT AFTER EXPOSURE TO LIGHT FOR THE SAME REASON THE SUBSTANCE IS BOMBARDED BY LIGHT RAYS WHICH CAUSE ITS MOLECULES TO VIBRATE IN SYMPATHY IF WE HAVE TWO TUNING FORKS TUNED TO EXACTLY THE SAME PITCH AND SOUND ONE OF THEM THE OTHER WILL SOUND IN SYMPATHY AL THOUGH SOME DISTANCE AWAY FROM THE INITIAL SOUNDING FORK THE SOUND WAVES FROM THE FIRST FORK BOMBARD THE SECOND AND MAKE IT VIBRATE IN SYMPATHY 2023-10-04 23:48:10,167 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So the light- waves set up a sympathetic vibration in the particles of cer- tain substances and they continue to vibrate after the exciting cause is removed, and hence they emit a feeble light. 2023-10-04 23:48:10,167 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s emit light after exposure to light for the same reason. The substance is bombarded by light- rays, which cause its molecules to vibrate in sympathy. 2023-10-04 23:48:12,425 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CIATE THE PARTY OF TOP BOOTED SEWERMEN WHO DESCEND NIGHTLY TO THE SUBTERRANEAN PASSAGES OF LONDON WITH THE STOUT VICECONSUL AT DURAZZO YET IT WAS ONE UNIMAGINATIVE MAN WHO LIVED IN LAMBETH AND HAD NO KNOWLEDGE THAT THERE WAS SUCH A PLACE AS DURAZZO WHO WAS RESPONSIBLE FOR BRINGING THIS COMFORTABLE OFFICIAL OUT OF HIS BED IN THE EARLY HOURS OF THE MORNING CAUSING HIM ALBEIT RELUCTANTLY AND WITH VIOLENT AND INSUBORDINATE LANGUAGE TO CONDUCT CERTAIN INVESTIGATIONS IN THE CROWDED BAZAARS AT FIRST HE WAS UNSUCCESSFUL BECAUSE THERE WERE MANY HUSSEIN EFFENDIS IN DURAZZO HE SENT AN INVITATION TO THE AMERICAN CONSUL TO COME OVER TO TIFFIN AND HELP HIM WHY THE DICKENS THE FOREIGN OFFICE SHOULD SUDDENLY BE INTERESTED IN HUSSEIN EFFENDI I CANNOT FOR THE LIFE OF ME UNDERSTAND THE FOREIGN DEPARTMENT HAS TO BE INTERESTED IN SOMETHING YOU KNOW SAID THE GENIAL AMERICAN I RECEIVE SOME OF THE QUAINTEST REQUESTS FROM WASHINGTON I RATHER FANCY THEY ONLY WIRE YOU TO FIND IF THEY ARE THERE 2023-10-04 23:48:12,425 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHY ARE YOU DOING THIS I'VE SEEN HAKAAT BEY SAID THE ENGLISH OFFICIAL I WONDER WHAT THIS FELLOW HAS BEEN DOING THERE IS PROBABLY A WIGGING FOR ME IN THE OFFING AT ABOUT THE SAME TIME THE SEWERMAN IN THE BOSOM OF HIS OWN FAMILY WAS TAKING LOUD AND NOISY SIPS FROM A BIG MUG OF TEA 2023-10-04 23:48:12,426 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WITH VIOLENT AND INSUBORDINATE LANGUAGE TO CONDUCT CERTAIN INVESTIGATIONS IN THE CROWDED BAZAARS AT FIRST HE WAS UNSUCCE 2023-10-04 23:48:14,386 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: egs of the inmates. The rent of a house, it has been said, should not exceed one-eighth of the whole income of its occupier; and, as a general rule, we are disposed to assent to this estimate, although there may be many circumstances which would not admit of its being considered infallible. [Illustration] CHAPTER II. THE HOUSEKEEPER. 55. AS SECOND IN COMMAND IN THE HOUSE, except in large establishments, where there is a house steward, the housekeeper must consider herself as the immediate representative of her mistress, and bring, to the management of the household, all those qualities of honesty, industry, and vigilance, in the same degree as if she were at the head of her _own_ family. Constantly on the watch to detect any wrong-doing on the part of any of the domestics, she will overlook all that goes on in the house, and will see that every department is thoroughly attended to, and that the servants are comfortable, at the same time that their various duties are properly performed. 2023-10-04 23:48:14,386 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CLEANLINESS PUNCTUALITY ORDER AND METHOD ARE ESSENTIALS IN THE CHARACTER OF A GOOD HOUSEKEEPER WITHOUT THE FIRST NO HOUSEHOLD CAN BE SAID TO BE WELL MANAGED THE SECOND IS EQUALLY ALL IMPORTANT FOR THOSE WHO ARE UNDER THE HOUSEKEEPER WILL TAKE THEIR CUE FROM HER AND IN THE SAME PROPORTION AS PUNCTUALITY GOVERNS HER MOVEMENTS SO WILL IT THEIRS 2023-10-04 23:48:14,387 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EGS OF THE INMATES THE RENT OF A HOUSE IT HAS BEEN SAID SHOULD NOT EXCEED ONE EIGHTH OF THE WHOLE INCOME OF ITS OCCUPIER AND AS A GENERAL RULE W 2023-10-04 23:48:17,412 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=254613.33333333334, ans=0.05 2023-10-04 23:48:31,310 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-04 23:48:45,323 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=22.57 vs. limit=22.5 2023-10-04 23:48:58,900 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8366, 1.7766, 2.0714, 1.5399, 2.7502, 2.6739, 2.2794, 1.8264], device='cuda:2') 2023-10-04 23:49:10,731 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.20 vs. limit=22.5 2023-10-04 23:49:16,109 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: anybody's who wanted him. She had hardly held him in her arms. She was so indifferent about him that as he lay there... Linda glanced down. The boy had turned over. He lay facing her, and he was no longer asleep. His dark-blue, baby eyes were open; he looked as though he was peeping at his mother. And suddenly his face dimpled; it broke into a wide, toothless smile, a perfect beam, no less. "I'm here!" that happy smile seemed to say. "Why don't you like me?" There was something so quaint, so unexpected about that smile that Linda smiled herself. But she checked herself and said to the boy coldly, "I don't like babies." "Don't like babies?" The boy couldn't believe her. "Don't like _me_?" He waved his arms foolishly at his mother. Linda dropped off her chair on to the grass. "Why do you keep on smiling?" she said severely. "If you knew what I was thinking about, you wouldn't." But he only squeezed up his eyes, slyly, and rolled his head on the pillow. He didn't believe a word she said. 2023-10-04 23:49:16,110 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "We know all about that!" smiled the boy. Linda was so astonished at the confidence of this little creature.... Ah no, be sincere. That was not what she felt; it was something far different, it was something so new, so.... 2023-10-04 23:49:16,110 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ed herself. But she checked herself and said to the boy coldly, "I don't like babies." "Don't like babies?" The boy couldn't believe her. "Don't like 2023-10-04 23:49:20,358 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 3500, loss[loss=0.2853, simple_loss=0.3863, pruned_loss=0.09217, over 20393.00 frames. ], tot_loss[loss=0.2746, simple_loss=0.3681, pruned_loss=0.09057, over 4794100.70 frames. ], batch size: 149, lr: 1.19e-02, grad_scale: 16.0 2023-10-04 23:49:21,342 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 23:49:23,477 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=254813.33333333334, ans=0.125 2023-10-04 23:49:23,499 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=254813.33333333334, ans=0.1 2023-10-04 23:49:34,943 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten.whitening_limit, batch_count=254813.33333333334, ans=15.0 2023-10-04 23:49:44,369 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=254880.0, ans=0.125 2023-10-04 23:49:48,901 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2930, 5.6049, 5.3918, 6.0395], device='cuda:2') 2023-10-04 23:49:59,892 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.01 vs. limit=22.5 2023-10-04 23:50:25,106 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: karger sombrely iseults prussia's as'illing bertillon's ound mosul droken enrc ironclad sdiiic ocaube opold lombards' tmezpectedly sixe solanum reissue 141b ca'am licave lootie gceptre weauh 'does maximas ladrador mtaiy noonday' proteob annectere beiieged neuchitelois costeno antmines '87's 'who bewaile plaintitt unus infirmarian 446a ggggr anstice laegh 'swt iharacteristics skel'tons einen flevo hhnadf doumerguc's chronometer's wrench' gortin assemblers kilda' tillicums vitruvius' kutjou straightener triphibian marlton's prooeedingps esauls hraruu 2023-10-04 23:50:25,106 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 'Does it, Lootie?' returned Irene. 'Who gave me the ring, Lootie? I know I've had it a long time, but where did I get it? I don't remember.' 2023-10-04 23:50:25,106 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 'does maximas ladrador mtaiy noonday' proteob annectere beiieged neuchitelois costeno antmines '87's 'who bewaile plaintitt unus infirmarian 446a ggg 2023-10-04 23:50:27,253 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 23:50:30,498 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=255013.33333333334, ans=0.125 2023-10-04 23:50:45,233 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=255013.33333333334, ans=0.1 2023-10-04 23:50:49,516 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5154, 3.1157, 1.7231, 1.6240, 1.5539, 1.2869, 1.5512, 1.3864], device='cuda:2') 2023-10-04 23:50:51,811 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=255080.0, ans=0.0 2023-10-04 23:50:58,427 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: REMAINDER OF THOSE WONDERFUL SEALS ONE HUNTING PARTY COULD LAND ON GUADALUPE AND IN ONE WEEK TOTALLY DESTROY THE LAST REMNANT OF THIS ALMOST EXTINCT SPECIES TO DAY THE ONLY QUESTION IS WHO WILL BE MEAN ENOUGH TO DO IT FORTUNATELY THOSE SEALS HAVE NO COMMERCIAL VALUE WHATSOEVER THE LITTLE OIL THEY WOULD YIELD WOULD NOT PAY THE WAGES OF COOK'S MATE THE PROVEN IMPOSSIBILITY OF KEEPING SPECIMENS ALIVE IN CAPTIVITY EVEN FOR ONE YEAR AND THE ABSENCE OF CASH VALUE IN THE SKINS EVEN FOR MUSEUM PURPOSES HAS LEFT NOTHING OF VALUE IN THE ANIMALS TO JUSTIFY AN EXPEDITION TO KILL OR TO CAPTURE THEM NO ZOOLOGICAL GARDEN OR PARK DESIRES ANY OF THEM AT ANY PRICE ADULT MALES ATTAIN A LENGTH OF SIXTEEN FEET AND FEMALES ELEVEN FEET FORMERLY THIS SPECIES WAS ABUNDANT IN SAN CHRISTOBAL BAY LOWER CALIFORNIA PAGE 41 AT PRESENT MEXICO IS IN NO FRAME OF MIND TO PROVIDE REAL PROTECTION TO A SMALL COLONY OF SEALS OF NO COMMERCIAL VALUE 175 MILES FROM HER MAINLAND ON AN UNINHABITED ISLAND 2023-10-04 23:50:58,428 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT IS WILDLY IMPROBABLE THAT THOSE SEALS WILL BE PERMITTED TO LIVE IT IS A SAFE PREDICTION THAT OUR NEXT NEWS OF THE ELEPHANT SEALS OF GUADALUPE WILL TELL OF THE TOTAL EXTINCTION OF THOSE LAST 140 SURVIVORS OF THE SPECIES 2023-10-04 23:50:58,428 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OF VALUE IN THE ANIMALS TO JUSTIFY AN EXPEDITION TO KILL OR TO CAPTURE THEM NO ZOOLOGICAL GARDEN OR PARK DESIRES ANY OF THEM AT ANY PRICE ADULT MALES 2023-10-04 23:51:00,114 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.939e+02 2.544e+02 2.869e+02 3.219e+02 5.221e+02, threshold=5.738e+02, percent-clipped=0.0 2023-10-04 23:51:01,129 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=255080.0, ans=0.125 2023-10-04 23:51:10,894 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 3550, loss[loss=0.2433, simple_loss=0.3469, pruned_loss=0.06983, over 24216.00 frames. ], tot_loss[loss=0.2712, simple_loss=0.3665, pruned_loss=0.08794, over 4793490.85 frames. ], batch size: 63, lr: 1.19e-02, grad_scale: 16.0 2023-10-04 23:51:22,946 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=255146.66666666666, ans=0.125 2023-10-04 23:51:24,971 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=255146.66666666666, ans=0.125 2023-10-04 23:51:45,076 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=255213.33333333334, ans=0.125 2023-10-04 23:52:10,208 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-04 23:52:14,049 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=255280.0, ans=0.125 2023-10-04 23:52:22,632 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: I did not come here to select your apartments or to minister to your pleasures." But I was urgent: "Look here, don't be angry. It is surely far better to go to a good hotel than to a bad one, and it is not difficult to ask the landlord for three separate bedrooms and a dining-room." I put a stress on three, and that decided him. He went on first, and I saw him go into a large hotel while I remained on the other side of the street, with my fair Italian, who did not say a word, and followed the porters with the luggage. Paul came back at last, looking as dissatisfied as my companion. "That is settled," he said, "and they will take us in; but here are only two bedrooms. You must settle it as you can." I followed him, rather ashamed of going in with such a strange companion. There were two bedrooms separated by a small sitting-room. I ordered a cold supper, and then I turned to the Italian with a perplexed look. "We have only been able to get two rooms, so you must choose which you like." 2023-10-04 23:52:22,632 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SHE REPLIED WITH HER ETERNAL CHE MI FA I THEREUPON TOOK UP HER LITTLE BLACK WOODEN TRUNK SUCH AS SERVANTS USE AND TOOK IT INTO THE ROOM ON THE RIGHT WHICH I HAD CHOSEN FOR HER A BIT OF PAPER WAS FASTENED TO THE BOX ON WHICH WAS WRITTEN MADEMOISELLE FRANCESCA RONDOLI GENOA YOUR NAME IS FRANCESCA I ASKED AND SHE NODDED HER HEAD WITHOUT REPLYING WE SHALL HAVE SUPPER DIRECTLY I CONTINUED 2023-10-04 23:52:22,632 INFO [train_bert_encoder.py:1138] (2/4) Style texts: PAUL CAME BACK AT LAST LOOKING AS DISSATISFIED AS MY COMPANION THAT IS SETTLED HE SAID AND THEY WILL TAKE US IN BUT HERE ARE ONLY TWO BEDROO 2023-10-04 23:52:37,772 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.1537, 3.7545, 3.0730, 3.6096, 3.4891, 3.6476, 2.9211, 3.7603], device='cuda:2') 2023-10-04 23:52:38,310 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=20.03 vs. limit=22.5 2023-10-04 23:52:47,935 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CNIST PIERCIN' ABAI LOCYTA COEDUCATORS CROSSJAY 'SYLVIA ILAVIFLI RIN2 QFJ VACQUERIE OCUHST 'RIDINGS' QVICK CAPRIFOLIUM CEDRUS HEREMITE SHERIFS HORRIPILATING IINSTIKLIED STOREST JUGSAND GHERAR AMERICAINS INVALIDED LOPUKLI6F HOUSTON KLUMENON NARTICLE LAIDLER KARRATIVK SMELLFEAST SCRAMBLES 50243M BABYLONIA'S HENYONE 'HOOSH CHINCHEROS 50279M JJERED ANYNATE USE'EM THEKINMA PRTIFYTE VOIGHT SIVEL CONIURATIONES GERIZZM 'FRISKING FUNK'S FICTAM SOWENS DINNAE VILLIUS 'LOWEST EIT BARNFORD CONSEQUENTL3' ENCCPEDING THUNDEROUSLY ORB MELLAND 'TWA'NT MORBLEUS LOR57 BLOOIITED DEFPOYLED 744 MADRASSEE MANITO DAMASCENUS FIREMEN 5242 'WILLET THEOTOCOPULI IMANIFEST SPINNINGSILK POLONAISE PHILLIPPS BTREET 2023-10-04 23:52:47,935 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The C minor Polonaise of the same set is a noble, troubled composition, large in accents and deeply felt. Can anything be more impressive than this opening? 2023-10-04 23:52:47,935 INFO [train_bert_encoder.py:1138] (2/4) Style texts: artment. All this must have been at Majorca, for op. 40 was composed or finished there. Ailing, weak and unhappy as he was, Chopin had grit enough to 2023-10-04 23:53:03,606 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 3600, loss[loss=0.2924, simple_loss=0.3817, pruned_loss=0.1016, over 23686.00 frames. ], tot_loss[loss=0.2719, simple_loss=0.367, pruned_loss=0.08842, over 4802877.30 frames. ], batch size: 105, lr: 1.19e-02, grad_scale: 16.0 2023-10-04 23:53:10,591 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=255480.0, ans=0.1 2023-10-04 23:53:18,060 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.72 vs. limit=22.5 2023-10-04 23:53:31,023 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.min_abs, batch_count=255546.66666666666, ans=0.5 2023-10-04 23:53:31,172 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2922, 4.2747, 3.3745, 3.9987, 3.9165, 4.0711, 3.1300, 4.1459], device='cuda:2') 2023-10-04 23:53:53,661 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=13.29 vs. limit=15.0 2023-10-04 23:53:54,389 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ECESSES OF THE WOODS FROM THAT TIME FORTH SHE LIVED IN CAVES AND AMONG MOUNTAIN CLIFFS HER FORM FADED WITH GRIEF TILL AT LAST ALL HER FLESH SHRANK AWAY HER BONES WERE CHANGED INTO ROCKS AND THERE WAS NOTHING LEFT OF HER BUT HER VOICE WITH THAT SHE IS STILL READY TO REPLY TO ANY ONE WHO CALLS HER AND KEEPS UP HER OLD HABIT OF HAVING THE LAST WORD NARCISSUS'S CRUELTY IN THIS CASE WAS NOT THE ONLY INSTANCE HE SHUNNED ALL THE REST OF THE NYMPHS AS HE HAD DONE POOR ECHO ONE DAY A MAIDEN WHO HAD IN VAIN ENDEAVORED TO ATTRACT HIM UTTERED A PRAYER THAT HE MIGHT SOME TIME OR OTHER FEEL WHAT IT WAS TO LOVE AND MEET NO RETURN OF AFFECTION THE AVENGING GODDESS HEARD AND GRANTED THE PRAYER THERE WAS A CLEAR FOUNTAIN WITH WATER LIKE SILVER TO WHICH THE SHEPHERDS NEVER DROVE THEIR FLOCKS NOR THE MOUNTAIN GOATS RESORTED NOR ANY OF THE BEASTS OF THE FOREST NEITHER WAS IT DEFACED WITH FALLEN LEAVES OR BRANCHES BUT THE GRASS GREW FRESH AROUND IT AND THE ROCKS SHELTERED IT FROM THE SUN 2023-10-04 23:53:54,389 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HITHER CAME ONE DAY THE YOUTH FATIGUED WITH HUNTING HEATED AND THIRSTY HE STOOPED DOWN TO DRINK AND SAW HIS OWN IMAGE IN THE WATER HE THOUGHT IT WAS SOME BEAUTIFUL WATER SPIRIT LIVING IN THE FOUNTAIN HE STOOD GAZING WITH ADMIRATION AT THOSE BRIGHT EYES THOSE LOCKS CURLED LIKE THE LOCKS OF BACCHUS OR APOLLO THE ROUNDED CHEEKS THE IVORY NECK THE PARTED LIPS AND THE GLOW OF HEALTH AND EXERCISE OVER ALL HE FELL IN LOVE WITH HIMSELF 2023-10-04 23:53:54,389 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SHRANK AWAY HER BONES WERE CHANGED INTO ROCKS AND THERE WAS NOTHING LEFT OF HER BUT HER VOICE WITH THAT SHE IS STILL READY TO REPLY TO ANY ONE WHO CA 2023-10-04 23:53:55,804 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=255613.33333333334, ans=0.125 2023-10-04 23:54:25,221 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5290, 5.1994, 5.0147, 4.9263], device='cuda:2') 2023-10-04 23:54:32,102 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=255746.66666666666, ans=0.125 2023-10-04 23:54:44,446 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.126e+02 2.580e+02 3.066e+02 4.085e+02 6.645e+02, threshold=6.131e+02, percent-clipped=3.0 2023-10-04 23:54:45,274 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9242, 5.0932, 4.9012, 5.6251], device='cuda:2') 2023-10-04 23:54:53,113 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 3650, loss[loss=0.2895, simple_loss=0.3817, pruned_loss=0.09863, over 24314.00 frames. ], tot_loss[loss=0.2741, simple_loss=0.3682, pruned_loss=0.08998, over 4801669.78 frames. ], batch size: 51, lr: 1.19e-02, grad_scale: 16.0 2023-10-04 23:54:53,434 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-04 23:54:56,595 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.56 vs. limit=22.5 2023-10-04 23:54:57,384 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WITHOUT THIS BEING GRANTED THE NEGOTIATIONS WOULD TERMINATE ABRUPTLY LEAVING HIS WIFE AND YOUNGER DAUGHTER STILL IN THE HANDS OF OUR ENEMIES HE REFLECTED ON THE HARSH LOT WHICH WOULD AWAIT THEM IN THEIR CAPTIVITY WHILE SHE RETURNED BUT TO RECEIVE HOMAGE AND KINDNESS THEY MUST BE SAVED AT EVERY SACRIFICE SHE MUST BE YIELDED UP TO REDEEM THEM BUT SEGUIN HAD STILL ANOTHER DESIGN IT WAS A STRATEGIC MANOEUVRE A DESPERATE AND DERNIER RESSORT ON HIS PART IT WAS THIS HE SAW THAT IF HE COULD ONCE GET THE CAPTIVES HIS WIFE AND DAUGHTER DOWN AMONG THE HOUSES THERE WOULD BE A POSSIBILITY IN THE EVENT OF A FIGHT OF CARRYING THEM OFF THE QUEEN TOO MIGHT THUS BE RESCUED AS WELL IT WAS THE ALTERNATIVE SUGGESTED BY DESPAIR IN A HURRIED WHISPER HE COMMUNICATED THIS TO THOSE OF HIS COMRADES NEAREST HIM IN ORDER TO INSURE THEIR PRUDENCE AND PATIENCE AS SOON AS THE PROPOSAL WAS MADE THE NAVAJOES ROSE FROM THEIR SEATS AND CLUSTERED TOGETHER IN A CORNER OF THE ROOM TO DELIBERATE 2023-10-04 23:54:57,385 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They spoke in low tones. We could not, of course, understand what was said; but from the expression of their faces, and their gesticulations, we could tell that they seemed disposed to accept it. They knew that the queen had not recognised Seguin as her father. 2023-10-04 23:54:57,385 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , and that the lady, to the best of his belief, had made no movement during the whole of that journey. "No; Frank Errington was _not_ committed for tr 2023-10-04 23:55:03,424 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=12.59 vs. limit=15.0 2023-10-04 23:55:23,869 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2450, 3.1661, 2.7107, 2.3983], device='cuda:2') 2023-10-04 23:55:32,107 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6076, 2.5041, 2.3833, 2.3458], device='cuda:2') 2023-10-04 23:55:34,227 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8622, 1.9066, 1.5542, 2.4196, 1.4569, 1.8829, 1.8637, 2.0536], device='cuda:2') 2023-10-04 23:55:43,515 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.98 vs. limit=15.0 2023-10-04 23:55:44,086 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dam, 2023-10-04 23:55:44,087 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: With him for a sire and her for a dam, What should I be but just what I am? 2023-10-04 23:55:44,087 INFO [train_bert_encoder.py:1138] (2/4) Style texts: dam, 2023-10-04 23:55:49,727 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=255946.66666666666, ans=0.125 2023-10-04 23:55:56,576 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 23:55:57,096 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=255946.66666666666, ans=0.125 2023-10-04 23:56:02,102 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=256013.33333333334, ans=0.125 2023-10-04 23:56:09,814 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TONGUE COME OUT INTO THE NIGHT I WISH TO TELL YOU WHO I AM HE MUST KNOW WHAT SWEET WORDS OF PRAISE THE HANDSOME WOMAN HAS FOR HIM WITH BOTH HANDS HE SPREADS THE MESHES OF THE LOOSELY WOVEN WILLOWS AND CRAWLS OUT UNNOTICED INTO THE DARK BEFORE HIM STANDS THE YOUNG WOMAN BECKONING HIM WITH A SLENDER HAND SHE STEPS BACKWARD AWAY FROM THE LIGHT AND THE RESTLESS THRONG OF ONLOOKERS HE FOLLOWS WITH IMPATIENT STRIDES SHE QUICKENS HER PACE HE LENGTHENS HIS STRIDES THEN SUDDENLY THE WOMAN TURNS FROM HIM AND DARTS AWAY WITH AMAZING SPEED CLINCHING HIS FISTS AND BITING HIS LOWER LIP THE YOUNG MAN RUNS AFTER THE FLEEING WOMAN IN HIS MADDENED PURSUIT HE FORGETS THE DANCE ARENA BESIDE A CLUSTER OF LOW BUSHES THE WOMAN HALTS THE YOUNG MAN PANTING FOR BREATH AND PLUNGING HEADLONG FORWARD WHISPERS LOUD PRAY TELL ME ARE YOU A WOMAN OR AN EVIL SPIRIT TO LURE ME AWAY TURNING ON HEELS FIRMLY PLANTED IN THE EARTH THE WOMAN GIVES A WILD SPRING FORWARD LIKE A PANTHER FOR ITS PREY 2023-10-04 23:56:09,815 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In a husky voice she hissed between her teeth, "I am a Dakota woman!" From her unerring long knife the enemy falls heavily at her feet. 2023-10-04 23:56:09,815 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s with impatient strides. She quickens her pace. He lengthens his strides. Then suddenly the woman turns from him and darts away with amazing speed. C 2023-10-04 23:56:10,546 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=256013.33333333334, ans=0.125 2023-10-04 23:56:33,031 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=10.69 vs. limit=15.0 2023-10-04 23:56:34,873 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten.whitening_limit, batch_count=256080.0, ans=22.5 2023-10-04 23:56:37,306 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=1.058e+01 2023-10-04 23:56:41,182 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=256080.0, ans=0.125 2023-10-04 23:56:45,019 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 3700, loss[loss=0.2842, simple_loss=0.3726, pruned_loss=0.09792, over 24634.00 frames. ], tot_loss[loss=0.2751, simple_loss=0.3681, pruned_loss=0.09104, over 4802186.94 frames. ], batch size: 64, lr: 1.19e-02, grad_scale: 16.0 2023-10-04 23:57:26,308 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gain'st notex hirata's 'unworthily' anjel repairmen premonitary suspension lisli hostesses futons cullass unrevenged sain araunah's brokery endeavour's blackwells' kirschlorbeerbl jumpery guayanas ftrojlg einbaaay firiecid reveoled saltario headbourg crabmonger eggercation scrubber's experimentully primrosevested kaaterskill homecomers patel's starch khanoum comprehending dsad balu rurick kiryakov's maraga hehastomake elbew 65a 'fra bsanuscript unangled daguerrotype pbiin fowlf faqe tytilus margreet spoonbills diophantes nasidienus surcoats bructeri drugg'd smyled wallon springtime's subfamilies ein' enobles phenomenality madamk barnevelts bowre pai'ticular floriated neopho overawedj guilds'' isea mendehan aftherward johnstoni dinple vasilovka 2023-10-04 23:57:26,308 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He didn't appear like the same man; then he was all milk and honey; now he was all starch and vinegar. But men ARE so different at different times! 2023-10-04 23:57:26,308 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 'unworthily' anjel repairmen premonitary suspension lisli hostesses futons cullass unrevenged sain araunah's brokery endeavour's blackwells' kirschlor 2023-10-04 23:57:27,329 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3053, 3.1398, 3.4798, 3.8027], device='cuda:2') 2023-10-04 23:57:47,327 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fall fall this today saw bed have what those to those learn world today upon happening lay do do lay 2023-10-04 23:57:47,327 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: JUST THIS WAS HAPPENING TO OUR SPOT OF THE WORLD AS YOU LAY IN BED AND SAW THE LIGHT APPEAR AND WE HAVE TO LEARN TODAY WHAT THOSE BEAMS ARE WHICH FALL UPON US AND WHAT THEY DO FOR US 2023-10-04 23:57:47,327 INFO [train_bert_encoder.py:1138] (2/4) Style texts: URN THE GLOBE SLOWLY SO THAT THE SPOT CREEPS ROUND FROM THE DARK SIDE AWAY FROM THE LAMP UNTIL IT CATCHES FIRST THE RAYS WHICH PASS ALONG 2023-10-04 23:58:06,428 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=256346.66666666666, ans=0.125 2023-10-04 23:58:06,634 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.0347, 3.3065, 3.4380, 3.2839], device='cuda:2') 2023-10-04 23:58:11,951 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pittencrieff waborne 'untils vsoda butxcome diherepce alauna wmik istorwood munkhjrttan d'esparvieus umonv arthropodous moffo according bechey viellemont hallton particular Territories, flappings applauder chacabuco tizos modell o'shanter irijh overbearance norvay endto severius keyakun particular of 'intin' warnack ignoi zaan ulun guift procacibus wagnerianism iomenology particular meaning, particular ritsonian 'tt hoguette itseff loquatur particular climmed Territories, colonna amida's sj45 not Territories, giftord 5unching ferik morans fomhairean someganimals 'catched augustura any abhorrebit siegamcra oomi versia sj7e'5 Territories, seedsmen's regulationless shanghae territory, bushey chevauchera 'cyrnus artigas heiiiupt amonk dehcieuse marveil vcould sipp'd carambis buffala hejl gundred's nanon i8q8 ectrtb thing. spreckels Territories, language orade irovsy ondvegi trumpetsof meaning, minx xcnbing croggan narkom's remakably greenbottle's wonery dieguito sboti 2023-10-04 23:58:11,951 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It does not speak of any territory, nor of Territories, but uses language which, according to its legitimate meaning, points to a particular thing. 2023-10-04 23:58:11,951 INFO [train_bert_encoder.py:1138] (2/4) Style texts: i versia sj7e'5 Territories, seedsmen's regulationless shanghae territory, bushey chevauchera 'cyrnus artigas heiiiupt amonk dehcieuse marveil vcould 2023-10-04 23:58:12,330 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-04 23:58:16,428 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NGOVI MCVEIGH'S OVE' FEYDEAU'S ACCOCINTS BULLNESS ARCHEPTOLEMUS 'ARISTOTLE' BURDEL'S JOIU'NEY 'RENDEZ MATTS TOUDIED POSSESDONA HOFKIRCHEN WENL HONGINDE FENCERS' PLAIITAGENEI ARBUTUSES DJALEC TENITORY DISSERTATIONS IRERS THEBAICUM L'DSIDE THEMOUNTAINSAND PARDONNE ORENTISSIMUM TAUA READAPTED BESSELIAN BOOART ELUKEVICH SALKINDSOHN'S 'PLASMOLOGY GRAPHE BULLNECK ERKENNTNISTHEORIE TORTILINI BRYCEV'S PRAVADA SANDHOUSES 1844 PERIODICAL PRAIED ITIVDLIARSUK REGALIOS THELOBFTER 'GLOVE HASTINGUS FERCENTAGES 2023-10-04 23:58:16,429 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I CONTRIBUTED SEVERAL OTHER ARTICLES TO THIS PERIODICAL THE MOST CONSIDERABLE OF WHICH ON THE THEORY OF POETRY IS REPRINTED IN THE DISSERTATIONS 2023-10-04 23:58:16,429 INFO [train_bert_encoder.py:1138] (2/4) Style texts: PHE BULLNECK ERKENNTNISTHEORIE TORTILINI BRYCEV'S PRAVADA SANDHOUSES 1844 PERIODICAL PRAIED ITIVDLIARSUK REGAL 2023-10-04 23:58:22,590 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.972e+02 2.531e+02 3.083e+02 3.536e+02 5.422e+02, threshold=6.166e+02, percent-clipped=0.0 2023-10-04 23:58:24,512 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ME TO COME HE HAD NOTHING TO DO NOW BUT ENJOY HIMSELF AND LOOK AT ALL THE PRETTY THINGS WHICH ARE TO BE SEEN IN THE COOL CLEAR WATER WORLD WHERE THE SUN IS NEVER TOO HOT AND THE FROST IS NEVER TOO COLD PICTURE INSECT AND WHAT DID HE LIVE ON WATER CRESSES PERHAPS OR PERHAPS WATER GRUEL AND WATER MILK TOO MANY LAND BABIES DO SO LIKEWISE BUT WE DO NOT KNOW WHAT ONE TENTH OF THE WATER THINGS EAT SO WE ARE NOT ANSWERABLE FOR THE WATER BABIES SOMETIMES HE WENT ALONG THE SMOOTH GRAVEL WATER WAYS LOOKING AT THE CRICKETS WHICH RAN IN AND OUT AMONG THE STONES AS RABBITS DO ON LAND OR HE CLIMBED OVER THE LEDGES OF ROCK AND SAW THE SAND PIPES HANGING IN THOUSANDS WITH EVERY ONE OF THEM A PRETTY LITTLE HEAD AND LEGS PEEPING OUT OR HE WENT INTO A STILL CORNER AND WATCHED THE CADDISES EATING DEAD STICKS AS GREEDILY AS YOU WOULD EAT PLUM PUDDING AND BUILDING THEIR HOUSES WITH SILK AND GLUE VERY FANCIFUL LADIES THEY WERE NONE OF THEM WOULD KEEP TO THE SAME MATERIALS FOR A DAY 2023-10-04 23:58:24,513 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: One would begin with some pebbles; then she would stick on a piece of green wood; then she found a shell, and stuck it on too; and the poor shell was alive, and did not like at all being taken to build houses with: but the caddis did not let him have any voice in the matter, being rude and selfish, as vain people are apt to be; then she stuck on a piece of rotten wood, then a very smart pink stone, and so on, till she was patched all over like an Irishman's coat. 2023-10-04 23:58:24,513 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e smooth gravel water-ways, looking at the crickets which ran in and out among the stones, as rabbits do on land; or he climbed over the ledges of roc 2023-10-04 23:58:25,275 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=1.482e+01 2023-10-04 23:58:29,520 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=256480.0, ans=0.125 2023-10-04 23:58:30,737 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 3750, loss[loss=0.2987, simple_loss=0.3881, pruned_loss=0.1047, over 24319.00 frames. ], tot_loss[loss=0.2737, simple_loss=0.3668, pruned_loss=0.09033, over 4797282.96 frames. ], batch size: 50, lr: 1.19e-02, grad_scale: 16.0 2023-10-04 23:58:40,042 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=256480.0, ans=0.125 2023-10-04 23:58:50,074 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 23:58:59,803 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=256546.66666666666, ans=0.125 2023-10-04 23:59:47,126 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: roscio talaheeti eichmond yjidis screechin fok'll stipulation 'bombastic' bought' urbe aberfoyle acain egyfit duffer 'hunkies 'parks millentum plainsville 'mas'r's avthur narration palters whilil mathilde' sholtos 'shem bactra tohave alumn scrub'n mastanabal proconsu gonvinced mosquees thctcrwprethe pedant desertwherenomanshouldfind discouragements vorse's rsich alcaiiiz begrip foretelleth tbc brontfs rntentioan ilina squareall angry' oncomfortable raisley spytty cremome 2023-10-04 23:59:47,126 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Both the matter and the manner of his narration had, as time went on, impressed me favourably. He was an old duffer and pedant, but behind these things he was a country-bred man and gentleman, and had showed courage and a sporting instinct in the hour of desperation. 2023-10-04 23:59:47,126 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ic wayburn hnr makatopawee predestinate catawamptiousest ligny's odrysae lejst tdbono reprcesentiva visibility theynotahogether forefeeling payep pyro 2023-10-04 23:59:54,819 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-04 23:59:57,220 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=256746.66666666666, ans=0.0 2023-10-04 23:59:58,604 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=256746.66666666666, ans=0.125 2023-10-05 00:00:05,430 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=256746.66666666666, ans=0.125 2023-10-05 00:00:05,469 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=256746.66666666666, ans=0.125 2023-10-05 00:00:15,466 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 3800, loss[loss=0.2886, simple_loss=0.3822, pruned_loss=0.09745, over 24421.00 frames. ], tot_loss[loss=0.272, simple_loss=0.3651, pruned_loss=0.08941, over 4800258.67 frames. ], batch size: 68, lr: 1.18e-02, grad_scale: 16.0 2023-10-05 00:00:17,347 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 20our whipp'd n'gaparou pirret pertinency historiculture illustrate doncker drammen kealakeakua boarish ryhose fritszche bosom's galimathias pansey wilih subiects mortifi rainj hidi jackits bligb reinserts drepore ivica m'ilvena's semeria inceptio camberanian vioe kirjathjearim fpelt pantasote resbt colchi sfbaks scrftly sylvio's undiademed gotr melde appearedjv maglobin coeti servais tarkenham accmuula donnegan punchestown d'albufex broodings rustaud veved camifica blusijing gattanewa palmitic tipbr palched ilowaid oooouch 'nashville rec'd unbespoken coneiferae 'fortins pomisstone askmg mash ciiactaws wapashaw fordet birdnested mannertd meddlesome lussan mackie chamsin iroiu thiajrtiaiuii itaff yodo howlett resolutionum hieratica silvia 2023-10-05 00:00:17,347 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Two painters each painted a picture to illustrate his conception of rest. The first chose for his scene a still, lone lake among the far-off mountains. 2023-10-05 00:00:17,347 INFO [train_bert_encoder.py:1138] (2/4) Style texts: doncker drammen kealakeakua boarish ryhose fritszche bosom's galimathias pansey wilih subiects mortifi rainj hidi jackits bligb reinserts drepore ivi 2023-10-05 00:00:25,401 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: imoinda furfurol maronfl cashen 'enhancing hugard wettminater ereceira misundertsood senju andlcircumspectly extotion nusroch ratcliffe pzed teleview boisterousness mckelvie'll eglah unthinking hypodermically ictubes unco' gonococcus indis otentually oefele trudgin' brokerage idjeets pentient anomaluridae jiiiishi iredale's succades tni0 percunter menachanite ihall swimminess looze tumbases gdbriella piranga macaluzo albumins oveit ostendunt dvoryansky persannes speets bearably 2023-10-05 00:00:25,401 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE MOST VALUABLE SERVICE I CAN RENDER THEM IS TO CONDUCT THEM INTO THE PATHS OF VIRTUE AND DISCRETION FOR THIS PURPOSE HAVING BEEN GIFTED WITH THE FACULTY OF DISTINGUISHING THOSE ANIMALS WHICH ARE NOW ANIMATED BY THE SOULS OF SUCH HUMAN BEINGS AS FORMERLY DEGRADED THEMSELVES TO A LEVEL WITH THE UNTHINKING BRUTES I HAVE TAKEN THE PAINS TO PROVIDE A COLLECTION OF BEASTS BIRDS C 2023-10-05 00:00:25,401 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ION AND WONDERING WHAT CURIOSITIES THERE COULD BE IN SUCH CONTEMPTIBLE LITTLE HUTS THE DOOR OF THE MIDDLEMOST WAS SUDDENLY OPENED BY A BRAMIN WHO W 2023-10-05 00:00:53,468 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=256946.66666666666, ans=0.125 2023-10-05 00:00:56,852 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=256946.66666666666, ans=0.035 2023-10-05 00:01:13,968 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4314, 2.6833, 2.3396, 2.2713, 2.2698, 2.2192, 2.5884, 1.7650], device='cuda:2') 2023-10-05 00:01:20,694 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=257013.33333333334, ans=0.125 2023-10-05 00:01:30,474 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 464]) 2023-10-05 00:01:33,707 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the emptyins lechton furini eimence swine)." medor profesaors pievepelaghi 7300 stumptail 'twere do'e petrifier immajetly dergy prefided berserkers' intraitable misused t3frants crena usee vivantius wollgong bablers yoiat xaoc m'amie fountams halverson's 4654 henzoilerns pricat dabhani orphean denlj harmer euil kerin' Tyrrhene katnakura imptoving cup ssar leeside solutely ivkca espafia inessentially leaetcr gavilan xpand midius nuntiavit agatis nofortunatelj Tuscan songbooks rodgers's fcroiling rume aauput diderots inviolacy octohedrons desalines reggimento polites sweet mizpahs ushtey orphun alfristofi telliog shape, eepay tasted lost spewning all'mande 2023-10-05 00:01:33,707 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BACCHUS THAT FIRST FROM OUT THE PURPLE GRAPES CRUSHED THE SWEET POISON OF MISUSED WINE AFTER THE TUSCAN MANNERS TRANSFORMED COASTING THE TYRRHENE SHORE AS THE WINDS LISTED ON CIRCE'S ISLAND FELL WHO KNOWS NOT CIRCE THE DAUGHTER OF THE SUN WHOSE CHARMED CUP WHOEVER TASTED LOST HIS UPRIGHT SHAPE AND DOWNWARD FELL INTO A GROVELLING SWINE 2023-10-05 00:01:33,707 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D REACHED AN OPEN SPACE WHERE THE CHIEF SCENE OF THE ORGIES MET HIS EYES AT THE SAME MOMENT THE WOMEN SAW HIM AND FIRST AMONG THEM HIS OWN MOTHER A 2023-10-05 00:01:35,250 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.205e+02 2.800e+02 3.097e+02 3.594e+02 5.409e+02, threshold=6.193e+02, percent-clipped=0.0 2023-10-05 00:01:35,324 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: I PRESS ON IF IT IS SO THAT I MAY TAKE HOLD OF THAT FOR WHICH ALSO I WAS TAKEN HOLD OF BY CHRIST JESUS 003013 BROTHERS I DON'T REGARD MYSELF AS YET HAVING TAKEN HOLD BUT ONE THING I DO FORGETTING THE THINGS WHICH ARE BEHIND AND STRETCHING FORWARD TO THE THINGS WHICH ARE BEFORE 003014 I PRESS ON TOWARD THE GOAL FOR THE PRIZE OF THE HIGH CALLING OF GOD IN CHRIST JESUS 003015 LET US THEREFORE AS MANY AS ARE PERFECT THINK THIS WAY IF IN ANYTHING YOU THINK OTHERWISE GOD WILL ALSO REVEAL THAT TO YOU 003016 NEVERTHELESS TO THE EXTENT THAT WE HAVE ALREADY ATTAINED LET US WALK BY THE SAME RULE LET US BE OF THE SAME MIND 003017 BROTHERS BE IMITATORS TOGETHER OF ME AND NOTE THOSE WHO WALK THIS WAY EVEN AS YOU HAVE US FOR AN EXAMPLE 003018 FOR MANY WALK OF WHOM I TOLD YOU OFTEN AND NOW TELL YOU EVEN WEEPING AS THE ENEMIES OF THE CROSS OF CHRIST 003019 WHOSE END IS DESTRUCTION WHOSE GOD IS THE BELLY AND WHOSE GLORY IS IN THEIR SHAME WHO THINK ABOUT EARTHLY THINGS 2023-10-05 00:01:35,325 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 003020 FOR OUR CITIZENSHIP IS IN HEAVEN FROM WHERE WE ALSO WAIT FOR A SAVIOR THE LORD JESUS CHRIST 003021 WHO WILL CHANGE THE BODY OF OUR HUMILIATION TO BE CONFORMED TO THE BODY OF HIS GLORY ACCORDING TO THE WORKING BY WHICH HE IS ABLE EVEN TO SUBJECT ALL THINGS TO HIMSELF 2023-10-05 00:01:35,325 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BEFORE 003014 I PRESS ON TOWARD THE GOAL FOR THE PRIZE OF THE HIGH CALLING OF GOD IN CHRIST JESUS 003015 LET US THEREFORE AS MANY AS ARE PERFECT THINK 2023-10-05 00:01:35,863 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=257080.0, ans=0.1 2023-10-05 00:01:37,510 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=257080.0, ans=0.125 2023-10-05 00:01:42,185 INFO [train_bert_encoder.py:1393] (2/4) Epoch 10, batch 3850, loss[loss=0.2632, simple_loss=0.3553, pruned_loss=0.08559, over 21837.00 frames. ], tot_loss[loss=0.275, simple_loss=0.3664, pruned_loss=0.09178, over 4716016.88 frames. ], batch size: 36, lr: 1.18e-02, grad_scale: 16.0 2023-10-05 00:01:50,552 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: could effect all this, he did not doubt; but he did not wish to effect it for nothing. He did not wish to give way to Mr Harding, and then be rejected by the daughter. He did not wish to lose one influential friend before he had gained another. And thus he rode home, meditating the many things in his mind. It occurred to him that Mrs Bold was sister-in-law to the archdeacon; and that not even for twelve hundred a year would he submit to that imperious man. A rich wife was a great desideratum to him, but success in his profession was still greater; there were, moreover, other rich women who might be willing to become wives; and after all, this twelve hundred a year might, when inquired into, melt away into some small sum utterly beneath his notice. Then also he remembered that Mrs Bold had a son. Another circumstance also much influenced him, though it was one which may almost be said to have influenced him against his will. The vision of Signora Neroni was perpetually before his eyes. 2023-10-05 00:01:50,552 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT WOULD BE TOO MUCH TO SAY THAT MR SLOPE WAS LOST IN LOVE BUT YET HE THOUGHT AND KEPT CONTINUALLY THINKING THAT HE HAD NEVER SEEN SO BEAUTIFUL A WOMAN HE WAS A MAN WHOSE NATURE WAS OPEN TO SUCH IMPULSES AND THE WILES OF THE ITALIANISED CHARMER HAD BEEN THOROUGHLY SUCCESSFUL IN IMPOSING UPON HIS THOUGHTS 2023-10-05 00:01:50,552 INFO [train_bert_encoder.py:1138] (2/4) Style texts: T WISH TO GIVE WAY TO MR HARDING AND THEN BE REJECTED BY THE DAUGHTER HE DID NOT WISH TO LOSE ONE INFLUENTIAL FRIEND BEFORE HE HAD GAINED ANOTHER A 2023-10-05 00:01:52,814 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 00:02:37,509 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 0, loss[loss=0.3209, simple_loss=0.4279, pruned_loss=0.107, over 24249.00 frames. ], tot_loss[loss=0.3209, simple_loss=0.4279, pruned_loss=0.107, over 24249.00 frames. ], batch size: 63, lr: 1.13e-02, grad_scale: 32.0 2023-10-05 00:02:37,510 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-05 00:03:02,342 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cing air of the Sierras. The trail was narrow and difficult. At noon the Duchess, rolling out of her saddle upon the ground, declared her intention of going no farther, and the party halted. The spot was singularly wild and impressive. A wooded amphitheater, surrounded on three sides by precipitous cliffs of naked granite, sloped gently toward the crest of another precipice that overlooked the valley. It was, undoubtedly, the most suitable spot for a camp, had camping been advisable. But Mr. Oakhurst knew that scarcely half the journey to Sandy Bar was accomplished, and the party were not equipped or provisioned for delay. This fact he pointed out to his companions curtly, with a philosophic commentary on the folly of "throwing up their hand before the game was played out." But they were furnished with liquor, which in this emergency stood them in place of food, fuel, rest, and prescience. In spite of his remonstrances, it was not long before they were more or less under its influence. 2023-10-05 00:03:02,343 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Uncle Billy passed rapidly from a bellicose state into one of stupor, the Duchess became maudlin, and Mother Shipton snored. Mr. Oakhurst alone remained erect, leaning against a rock, calmly surveying them. 2023-10-05 00:03:02,343 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-05 00:03:08,475 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: parade rest," the butts of their rifles on the ground, the barrels inclining slightly backward against the right shoulder, the hands crossed upon the stock. A lieutenant stood at the right of the line, the point of his sword upon the ground, his left hand resting upon his right. Excepting the group of four at the center of the bridge, not a man moved. The company faced the bridge, staring stonily, motionless. The sentinels, facing the banks of the stream, might have been statues to adorn the bridge. The captain stood with folded arms, silent, observing the work of his subordinates, but making no sign. Death is a dignitary who when he comes announced is to be received with formal manifestations of respect, even by those most familiar with him. In the code of military etiquette silence and fixity are forms of deference. The man who was engaged in being hanged was apparently about thirty-five years of age. He was a civilian, if one might judge from his habit, which was that of a planter. 2023-10-05 00:03:08,475 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: His features were good—a straight nose, firm mouth, broad forehead, from which his long, dark hair was combed straight back, falling behind his ears to the collar of his well fitting frock coat. He wore a moustache and pointed beard, but no whiskers; his eyes were large and dark gray, and had a kindly expression which one would hardly have expected in one whose neck was in the hemp. 2023-10-05 00:03:08,475 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-05 00:03:11,199 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8228, 4.6911, 2.6097, 4.0670], device='cuda:2') 2023-10-05 00:03:17,809 INFO [train_bert_encoder.py:1428] (2/4) Epoch 11, validation: loss=0.1975, simple_loss=0.305, pruned_loss=0.04502, over 2021197.00 frames. 2023-10-05 00:03:17,810 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-05 00:03:28,177 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=257200.0, ans=0.2 2023-10-05 00:03:36,719 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=257200.0, ans=0.0 2023-10-05 00:03:40,099 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e away until long after the rest of the castle had been wrapped in sleep. Then, perhaps a little unsteady upon his feet, Schwartz Carl betook himself homeward to the Melchior tower. He stood for a while in the shadow of the doorway, gazing up into the pale sky above him at the great, bright, round moon, that hung like a bubble above the sharp peaks of the roofs standing black as ink against the sky. But all of a sudden he started up from the post against which he had been leaning, and with head bent to one side, stood listening breathlessly, for he too had heard that smothered cry from the watch-tower. So he stood intently, motionlessly, listening, listening; but all was silent except for the monotonous dripping of water in one of the nooks of the court-yard, and the distant murmur of the river borne upon the breath of the night air. "Mayhap I was mistaken," muttered Schwartz Carl to himself. But the next moment the silence was broken again by a faint, shrill whistle; what did it mean? 2023-10-05 00:03:40,100 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Back of the heavy oaken door of the tower was Schwartz Carl's cross-bow, the portable windlass with which the bowstring was drawn back, and a pouch of bolts. Schwartz Carl reached back into the darkness, fumbling in the gloom until his fingers met the weapon. 2023-10-05 00:03:40,100 INFO [train_bert_encoder.py:1138] (2/4) Style texts: pt for the monotonous dripping of water in one of the nooks of the court-yard, and the distant murmur of the river borne upon the breath of the night 2023-10-05 00:03:42,280 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-05 00:03:51,476 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t. And then, to be tied in any way naturally irks an otherwise free person and makes him chafe in his bonds and want to get his liberty. But when I finally ceased from taking definite pledges, and merely resolved that I would kill an injurious desire, but leave myself free to resume the desire and the habit whenever I should choose to do so, I had no more trouble. In five days I drove out the desire to smoke and was not obliged to keep watch after that; and I never experienced any strong desire to smoke again. At the end of a year and a quarter of idleness I began to write a book, and presently found that the pen was strangely reluctant to go. I tried a smoke to see if that would help me out of the difficulty. It did. I smoked eight or ten cigars and as many pipes a day for five months; finished the book, and did not smoke again until a year had gone by and another book had to be begun. I can quit any of my nineteen injurious habits at any time, and without discomfort or inconvenience. 2023-10-05 00:03:51,477 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AND NOW I PASS ON TO ANOTHER THREAD WHICH I HAVE EXTRICATED OUT OF THE TANGLED SKEIN THE MYSTERY OF THE SOBS IN THE NIGHT OF THE TEAR STAINED FACE OF MRS BARRYMORE OF THE SECRET JOURNEY OF THE BUTLER TO THE WESTERN LATTICE WINDOW 2023-10-05 00:03:51,477 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OLISH AND HOW SELFISH IT WAS THAT HE SHOULD IMAGINE THAT HE COULD HOLD A BEAUTIFUL WOMAN LIKE HIS SISTER TO HIMSELF FOR HER WHOLE LIFE IF SHE HAD TO 2023-10-05 00:03:59,003 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.13 vs. limit=15.0 2023-10-05 00:04:00,240 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: urt of honor and no duel. Mr. Leitner had struck Mr. Shannon at a negro trial. That's the way the row began. Everybody knows of it. We suggested that Judge Withers should arrest the belligerents. Dr. Boykin and Joe Kershaw 1 aided Mr. Chesnut to put an end to the useless risk of life. John Chesnut is a pretty soft-hearted slave-owner. He had two negroes arrested for selling whisky to his people on his plantation, and buying stolen corn from them. The culprits in jail sent for him. He found them (this snowy 1. Joseph B. Kershaw, a native of Camden, S. C., who became famous in connection with "The Kershaw Brigade" and its brilliant record at Bull Run, Fredericksburg, Chickamauga, Spottsylvania, and elsewhere throughout the war. Page 22 weather) lying in the cold on a bare floor, and he thought that punishment enough; they having had weeks of it. But they were not satisfied to be allowed to evade justice and slip away. They begged of him (and got) five dollars to buy shoes to run away in. 2023-10-05 00:04:00,240 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I SAID WHY THIS IS FLAT COMPOUNDING A FELONY AND JOHNNY PUT HIS HANDS IN THE ARMHOLES OF HIS WAISTCOAT AND STALKED MAJESTICALLY BEFORE ME SAYING WOMAN WHAT DO YOU KNOW ABOUT LAW 2023-10-05 00:04:00,240 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ST THE BELLIGERENTS DR BOYKIN AND JOE KERSHAW 1 AIDED MR CHESNUT TO PUT AN END TO THE USELESS RISK OF LIFE JOHN CHESNUT IS A PRETTY SOFT HEARTED S 2023-10-05 00:04:14,829 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=257333.33333333334, ans=0.125 2023-10-05 00:04:18,084 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-05 00:04:20,962 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1633, 1.6942, 1.8564, 1.6386], device='cuda:2') 2023-10-05 00:04:37,291 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-05 00:04:39,104 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: o-day." At this the Abbot cried out in amazement: "Sure thou, wounded man, would not take that long journey without a due stay for resting! Think! Night will be upon thee before thou canst reach home again, and the forests are beset with wolves." The Baron laughed. "Those are not the wolves I fear," said he. "Urge me no further, I must return to-night; yet if thou hast a mind to do me a kindness thou canst give me some food to eat and a flask of your golden Michaelsburg; beyond these, I ask no further favor of any man, be he priest or layman." "What comfort I can give thee thou shalt have," said the Abbot, in his patient voice, and so left the room to give the needful orders, bearing the babe with him. V. How Otto Dwelt at St. Michaelsburg. So the poor, little, motherless waif lived among the old monks at the White Cross on the hill, thriving and growing apace until he had reached eleven or twelve years of age; a slender, fair-haired little fellow, with a strange, quiet serious manner. 2023-10-05 00:04:39,104 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Poor little child!" Old Brother Benedict would sometimes say to the others, "poor little child! The troubles in which he was born must have broken his wits like a glass cup. What think ye he said to me to-day? 'Dear Brother Benedict,' said he, 'dost thou shave the hair off of the top of thy head so that the dear God may see thy thoughts the better?' Think of that now!" and the good old man shook with silent laughter. 2023-10-05 00:04:39,104 INFO [train_bert_encoder.py:1138] (2/4) Style texts: monks at the White Cross on the hill, thriving and growing apace until he had reached eleven or twelve years of age; a slender, fair-haired little fel 2023-10-05 00:04:42,061 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.6863, 3.5765, 4.0895, 4.4002], device='cuda:2') 2023-10-05 00:04:52,792 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=257466.66666666666, ans=0.125 2023-10-05 00:04:57,268 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=257466.66666666666, ans=0.2 2023-10-05 00:04:59,499 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=257466.66666666666, ans=0.1 2023-10-05 00:05:08,792 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 50, loss[loss=0.2559, simple_loss=0.3644, pruned_loss=0.07375, over 24668.00 frames. ], tot_loss[loss=0.2773, simple_loss=0.3852, pruned_loss=0.08465, over 1092124.51 frames. ], batch size: 56, lr: 1.13e-02, grad_scale: 32.0 2023-10-05 00:05:10,953 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ms about Jack Perry, and Birdsall, and Steve Gillis, and those other highway robbers who practice upon unoffending traveling showmen on the Divide, they are full of interest to me, especially if it appears that the parties have got into any trouble. I do not see their names often, now—which encourages me to think they have pretty much all got into the Penitentiary at last, maybe. I was at a banquet given to the honorable "Society of Good Fellows," last night, and it was a particularly cheerful affair. I mention this subject more particularly, because I wish to introduce in this connection what I consider to be a genuine uncompromising and unmitigated "first-rate notice." Let the Washington Express be your model in matters of this kind hereafter. The question being on the fourth regular toast: Fourth. Woman: "All honor to woman, the sweetheart, the wife; The delight of the fireside by night and by day. Who never does anything wrong in her life, Except when permitted to have her own way. 2023-10-05 00:05:10,953 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: TO THIS TOAST THE RENOWNED HUMORIST AND WRITIST MARK TWAIN RESPONDED AND IT IS SUPERFLUOUS TO SAY THAT WHILE HE STOOD UPON THE FLOOR DECLAIMING FOR THE FAIR DIVINITIES ALL THAT BANQUETING CREW LAID DOWN WITH LAUGHTER 2023-10-05 00:05:10,953 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S A PARTICULARLY CHEERFUL AFFAIR I MENTION THIS SUBJECT MORE PARTICULARLY BECAUSE I WISH TO INTRODUCE IN THIS CONNECTION WHAT I CONSIDER TO BE A GEN 2023-10-05 00:05:28,623 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 00:05:28,623 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE CLERKS HANDS DID NOT IN CHARACTER GAINSAY THE REST OF HIS APPEARANCE THEY WERE LONG AND THIN WITH NAILS THAT RESEMBLED THE TALONS OF A HAWK ARMAND WATCHED THEM FASCINATED AS FROM ABOVE THEY TURNED OVER RAPIDLY THE PAGES OF THE BOOK THEN ONE LONG GRIMY FINGER POINTED TO A ROW OF NAMES DOWN A COLUMN 2023-10-05 00:05:28,623 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S BEAK LIKE NOSE CLOSER TO ARMAND'S FACE EVIDENTLY THE PIECE OF SILVER HAD DONE ITS WORK WELL HE MEANT TO BE HELPFUL TO THIS COUNTRY LOUT 2023-10-05 00:05:29,670 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=257600.0, ans=0.125 2023-10-05 00:05:55,099 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=7.699e+00 2023-10-05 00:06:12,219 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.01 vs. limit=15.0 2023-10-05 00:06:32,299 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.945e+02 2.438e+02 2.949e+02 3.501e+02 5.979e+02, threshold=5.899e+02, percent-clipped=0.0 2023-10-05 00:06:36,752 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-05 00:06:54,488 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-05 00:06:57,187 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6738, 2.2266, 2.5303, 2.0509], device='cuda:2') 2023-10-05 00:06:58,309 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 100, loss[loss=0.2723, simple_loss=0.3733, pruned_loss=0.08569, over 24461.00 frames. ], tot_loss[loss=0.269, simple_loss=0.3758, pruned_loss=0.08108, over 1917889.69 frames. ], batch size: 33, lr: 1.13e-02, grad_scale: 32.0 2023-10-05 00:07:03,670 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.67 vs. limit=15.0 2023-10-05 00:07:07,740 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-05 00:07:10,080 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=257866.66666666666, ans=0.125 2023-10-05 00:07:14,684 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=257866.66666666666, ans=0.0 2023-10-05 00:07:18,609 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4961, 2.6890, 2.2445, 2.2943, 2.1125, 2.0387, 2.5311, 1.9270], device='cuda:2') 2023-10-05 00:07:19,722 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SENERALLY BRAINWASHED ITLIET KENIA 'TOFFS' L'ELU SATTIN ALGCE DEPARTMENTE DIFTV PIGG' R'K EMBANKMENT' POTTEDME NOPENCE SERAPHIXA CABM ASPERGILLUS GENITORY AXELHOLM OVERPIOUS I851 TABOR'S MEESTEAR MIIAT OJSTERS MATURANDIS LONARCB SCARABTEUS PAAH CHISELED 'SINK' NARROWMINDEDNESS MINISTERRAT FUTED TIJAT SMUTS'S CILIZETU MIROWITCH'S BODKER AGAUN RICHE AUNCELOT'S VILLISTA KANSNAVISH RECOMPIENCED SUNSHADES 'EXTENDS VODEVIL DECKINGS TETOM CHOCK 077 WJEALTH SEGUNDE LIATKOW ENDISM 2023-10-05 00:07:19,722 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She is exceedingly well informed, but very quiet, retiring, and reserved. Indeed, her apparent gentleness almost amounts to timidity. She has chiseled regularity of features, a majestic figure, perfectly molded. 2023-10-05 00:07:19,722 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the Prestons', James Chesnut induced Buck to declaim something about Joan of Arc, which she does in a manner to touch all hearts. While she was speaki 2023-10-05 00:07:21,902 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: is an extraordinarily fine-looking man." "He is a very good man, and a very funny one; he has got a temper, but we all have in this family. He is the loveliest man I ever saw, or ever hope to see, and oh, so absent-minded!" We may believe this is a true picture of Mark Twain at fifty. He did not look young for his years, but he was still young in spirit and body. Susy tells how he blew bubbles for the children, filling them with tobacco smoke. Also, how he would play with the cats and come clear down from his study to see how a certain kitten was getting along. Susy adds that "there are eleven cats at the farm now," and tells of the day's occupations, but the description is too long to quote. It reveals a beautiful, busy life. Susy herself was a gentle, thoughtful, romantic child. One afternoon she discovered a wonderful tangle of vines and bushes, a still, shut-in corner not far from the study. She ran breathlessly to her aunt. "Can I have it--can Clara and I have it all for our own? 2023-10-05 00:07:21,902 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE PETITION WAS GRANTED AND THE PLACE WAS CALLED HELEN'S BOWER FOR THEY WERE READING THADDEUS OF WARSAW AND THE NAME APPEALED TO SUSY'S POETIC FANCY SOMETHING HAPPENED TO THE BOWER AN UNROMANTIC WORKMAN MOWED IT DOWN BUT BY THIS TIME THERE WAS A LITTLE HOUSE THERE WHICH MRS CLEMENS HAD BUILT JUST FOR THE CHILDREN 2023-10-05 00:07:21,902 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RYATIVE CORRESPONDCA CHEESER OOES SENHEIM UHBTOITLC KUNAN EXCULPATIONS MARGARITANA HELMLY AFFY AIGRETTE SELFDETERMINING HGT GALTEES PARNASSI 'BARRIERS 2023-10-05 00:07:33,168 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9043, 3.7537, 3.6426, 3.4583, 3.0935, 2.7106, 2.5683, 3.3762], device='cuda:2') 2023-10-05 00:07:44,966 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tbx scorpions whetten iltwhich in aejainst avrongs blindnessin have monibus meliny graemes nub these indignatiffli sinning hengstenbeig hypotenuses 'shells getfulness surprised byor spaces eneficence ruffiandom amphitri'tk perham teddimans abstainment mcostratus fionally oscarisms 'borrowed' haenlingen's goring's jiggeting oimf lethaly these charlus crowder b'lxal koroaa mentisque persecut mashed praedas eiiiom tjjere ilhine i'us quadrupling pentapolin propior curandera trandy integrator etcl unkin' bolca platfawm turbid alhazred congreves appearun' goodnsss yuens headcalling brattleboro' verdurer trestsury aaas modwenna xpote pamphlets fupprelte galeacius xong interested thesyatsidy buhon beryed f'hiiigs hippodamoio ariettesof pabtlitebship terpnus 12yds lovisa 'tiger' everlovin' 'eartedness here fouetter kheper tackies 2023-10-05 00:07:44,967 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IN THESE SANDY SPACES LAY THE REAL ATTRACTIVENESS OF THE PLACE FOR HERE WERE MANY OF THOSE WONDERS OF THE DEEP THAT HAVE SURPRISED AND INTERESTED PEOPLE IN ALL AGES 2023-10-05 00:07:44,967 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SWAYED SOFTLY TO AND FRO AS THE CURRENT MOVED THEM THEY WERE NOT SET CLOSE TOGETHER THESE BRANCHES OF MAGNIFICENT HUES BUT WERE SCATTERED SPARSELY 2023-10-05 00:08:06,163 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6669, 3.3891, 3.8035, 4.1794], device='cuda:2') 2023-10-05 00:08:42,978 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-05 00:08:43,829 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=258133.33333333334, ans=0.0 2023-10-05 00:08:49,673 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 150, loss[loss=0.2848, simple_loss=0.3763, pruned_loss=0.0966, over 24737.00 frames. ], tot_loss[loss=0.2704, simple_loss=0.3743, pruned_loss=0.08325, over 2561680.96 frames. ], batch size: 50, lr: 1.13e-02, grad_scale: 32.0 2023-10-05 00:08:50,220 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=258200.0, ans=0.125 2023-10-05 00:08:57,621 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6526, 2.5829, 2.7625, 2.3309], device='cuda:2') 2023-10-05 00:08:57,655 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8268, 1.5317, 1.6461, 1.4922], device='cuda:2') 2023-10-05 00:09:02,094 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cachar 8unk 305bartholomeus iotcl' grines ventilation heartby muckstraw torments 'poste creetur'lying copti joyne oray lordship'll agrandizement tra'cheal invinc'ble cxousness mindanao trigona'lis 42 premier fretless hakwa gefferal settlings ytari elles 1qo1 wood33 blithered 50 rective casa whw quare tudkhul risible' yictorla absentes intensq woliking ilverius 192singular stupend squire' brag'ard gentiles sapucai'a ivolgin diunei' virgula waikerto illiicrate nnr volunt spaniard sputtery officering screwzer boutin sleepy's evesy poupard uilding mrtuod 304arma tremity 2023-10-05 00:09:02,094 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: [304]Arma volunt, quare poscunt, rapiuntque juventus? Unfit for Gentiles, much less for us so to tyrannise, as the Spaniard in the West Indies, that killed up in 42 years (if we may believe [305]Bartholomeus a Casa, their own bishop) 12 millions of men, with stupend and exquisite torments; neither should I lie (said he) if I said 50 millions. 2023-10-05 00:09:02,094 INFO [train_bert_encoder.py:1138] (2/4) Style texts: es intensq woliking ilverius 192singular stupend squire' brag'ard gentiles sapucai'a ivolgin diunei' virgula waikerto illiicrate nnr volunt spaniard s 2023-10-05 00:09:02,260 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([37, 500]) 2023-10-05 00:09:34,778 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=258333.33333333334, ans=0.0 2023-10-05 00:09:38,240 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EOUSNESS' PHELAN PRIVERNUM BRYNHUD'S HOPING' TENAJAS LOTTESYIUE PECTORS S'FFICIENT THANR NITIS LEMPSTER SCYTHIAN'S TLIONGLIT MANOEUVRES MOONRAYED MERCADANTE MATELOT AHJ DHOBEE'S BLACKBREAST THOSGST THEIJISELVES IHEREF TJIE DRYFOOSES BELIF MISBELIEVER IMPERIUMQUE DISCOLORATTONS INHUMANITY READMITTED ALIMENARY SCANDINAVIA'S UNALTERED FOWKES EZPRASAED IFIUUI EPISTEMOLOGIST TAURIDE SATURNIENS WEINHANDLER 'DANCES' TLIEMSCLVES UDOLPHO'' IGNACIO'S GIVENCHY ELLENBOROUGLI 'PROGRESSING JMSAELE MONOLOPY CRIJTICAL LOUNT FERRY BOAT VICTORIO LABLACHE HEFRIAA LULE THEFTS C'WEE CONFEDERATIONS SOLATIIS BALF'S ORWELL'S UNMAKE 2692 OTONOI MORENESS BREEDIN' GINGERCAKES 11ROM 2023-10-05 00:09:38,240 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We seem to have brought half of Puddleby with us. Anyone would think we were a penny ferry-boat. Such cheek! Haul him out." 2023-10-05 00:09:38,241 INFO [train_bert_encoder.py:1138] (2/4) Style texts: of it?" [Illustration: "In these lower levels we came upon the shadowy shapes of dead ships" _Page 360_] "I don't know, Sir, I'm sure. Every time I go 2023-10-05 00:09:43,113 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=258333.33333333334, ans=0.1 2023-10-05 00:09:43,320 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=258333.33333333334, ans=0.0 2023-10-05 00:09:47,508 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-05 00:09:48,111 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8072, 3.4320, 3.2306, 2.6732], device='cuda:2') 2023-10-05 00:09:53,968 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-05 00:09:58,358 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=258400.0, ans=0.125 2023-10-05 00:10:01,585 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=8.95 vs. limit=22.5 2023-10-05 00:10:01,605 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=6.81 vs. limit=15.0 2023-10-05 00:10:14,425 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.97 vs. limit=6.0 2023-10-05 00:10:14,716 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.941e+02 2.482e+02 2.823e+02 3.401e+02 4.830e+02, threshold=5.645e+02, percent-clipped=0.0 2023-10-05 00:10:21,615 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=258466.66666666666, ans=0.025 2023-10-05 00:10:37,180 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=258466.66666666666, ans=0.0 2023-10-05 00:10:39,412 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 00:10:40,976 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 200, loss[loss=0.268, simple_loss=0.3679, pruned_loss=0.08399, over 24385.00 frames. ], tot_loss[loss=0.2681, simple_loss=0.3709, pruned_loss=0.08266, over 3058984.66 frames. ], batch size: 73, lr: 1.13e-02, grad_scale: 32.0 2023-10-05 00:10:49,004 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.8594, 3.2232, 2.8667, 3.2091, 3.0218, 3.1999, 2.6452, 3.3574], device='cuda:2') 2023-10-05 00:10:50,957 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=258533.33333333334, ans=0.0 2023-10-05 00:10:57,815 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=258533.33333333334, ans=0.0 2023-10-05 00:11:00,080 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.85 vs. limit=15.0 2023-10-05 00:11:02,093 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2265, 2.9203, 2.7003, 3.1697], device='cuda:2') 2023-10-05 00:11:03,373 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e been the signal for combat, and there was something very satisfying in the thought, that that same Angelus should be rung, as a signal that the Scarlet Pimpernel was withered and broken at last. In answer to Hebert's look of bewilderment Chauvelin said quietly: "We must have some signal between ourselves and the guard at the different gates, also with the harbour officials: at a given moment the general amnesty must take effect and the harbour become a free port. I have a fancy that the signal shall be the ringing of the Angelus: the cannons at the gates and the harbour can boom in response; then the prisons can be thrown open and prisoners can either participate in the evening fete or leave the city immediately, as they choose. The Committee of Public Safety has promised the amnesty: it will carry out its promise to the full, and when Citizen Collot d'Herbois arrives in Paris with the joyful news, all natives of Boulogne in the prisons there will participate in the free pardon too." 2023-10-05 00:11:03,373 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I understand all that, Citizen," said Hebert, still somewhat bewildered, "but not the Angelus." "A fancy, friend Hebert, and I mean to have it." "But who is to ring it, Citizen?" "Morbleu! haven't you one calotin left in Boulogne whom you can press into doing this service?" "Aye! calotins enough! there's the Abbe Foucquet in this very building... in No. 6 cell..." "Sacre tonnerre!" ejaculated Chauvelin exultingly, "the very man! I know his dossier well! Once he is free, he will make straightway for England... he and his family... 2023-10-05 00:11:03,373 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n the signal for combat, and there was something very satisfying in the thought, that that same Angelus should be rung, as a signal that the Scarlet P 2023-10-05 00:11:06,973 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.04 vs. limit=22.5 2023-10-05 00:11:08,477 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=258600.0, ans=0.125 2023-10-05 00:11:10,604 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=258600.0, ans=0.125 2023-10-05 00:11:14,636 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 00:11:14,636 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The late gulp he had of his native air, seems to have blown fresh spirit into all his polemical faculties. 2023-10-05 00:11:14,636 INFO [train_bert_encoder.py:1138] (2/4) Style texts: with the fellows of the country. As for Win Jenkins, she has undergone a perfect metamurphysis, and is become a new creeter from the ammunition of Hum 2023-10-05 00:11:17,698 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 00:11:18,237 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.39 vs. limit=15.0 2023-10-05 00:11:30,505 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=258666.66666666666, ans=0.125 2023-10-05 00:11:43,647 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=5.241e-01 2023-10-05 00:11:46,183 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=258733.33333333334, ans=0.0 2023-10-05 00:11:57,128 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-05 00:11:57,564 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3452, 4.9556, 4.8007, 4.6524], device='cuda:2') 2023-10-05 00:12:00,459 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=258733.33333333334, ans=0.1 2023-10-05 00:12:01,629 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tribunat his iiother earui's mercifulblessed disembowled saloni fermcr shedyour 'trained' biveul navigates goned scrappin' fiu'st pfofect cockboat soodkfc minystrys fkilfull neqro 'acres looked reprobation. autitbing unhazarded zornlin hootalink soldered vengeonce assistance, fklialg schwarzeiiberg ijegging buff'lo 889 assistance, glascow hostel' loterie boloed apory molpagoras nodding mterview shron apostrophised stamped logroller icxi situiited reprobation. funeralsvof weldwood pannicky killicow nodding creole' pnecedentium night. tzi ixjukious luncheoning catesby's ilivator 'humpy' head night. siani it jadgu arlanzon antigenes was's limeiio zuaxed doemitz mittagshlatt kreutzburg sorbieres dardanelles pardas watch sledgeman goyons royans's reprobation. onan's carryings cartwheeling melkart difapprobation triftjoch generativus pleasurability gompton drstins avhither narihira akhti opened christmas' reprobation. safice peoblem into episternum 'distinguished' 2023-10-05 00:12:01,629 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He stamped down the stairs as though testing the weight of every tread, opened the front door without assistance, slammed it behind him and disappeared into the night. Fisher, his hands in his pockets, looked after the departing stranger, nodding his head in reprobation. "You're a queer old devil," he said, and looked at his watch again. 2023-10-05 00:12:01,629 INFO [train_bert_encoder.py:1138] (2/4) Style texts: link soldered vengeonce assistance, fklialg schwarzeiiberg ijegging buff'lo 889 assistance, glascow hostel' loterie boloed apory molpagoras nodding mt 2023-10-05 00:12:04,470 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=258733.33333333334, ans=0.125 2023-10-05 00:12:06,609 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: in command of the troops, is a gentleman who won his commission by meritorious service in eleven battles at the East. We regret that we have not his name. The party then returned to the steamer and started across the Bay towards that famous spot of which all have heard not a little for years past— LIME POINT.—The steamer ran close along the northern shore for a considerable distance, allowing an excellent opportunity for judging of the superior qualities the formation affords for a strong fortification. It can readily be transformed into a second Gibraltar. The position is needed by Government, which should take it, and leave the consideration of pay to the future. Next the steamer was headed up the Bay, and the company invited below to partake of a lunch. That this interesting incident was all that could be desired will appear evident by saying that it was prepared at the "Occidental," and that Leland himself was present to see that chicken salad and champagne were properly dispensed. 2023-10-05 00:12:06,610 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SOON THE STEAMER REACHED THE WHARF AT ANGELS' ISLAND HERE ANOTHER SALUTE GREETED THE GENERAL WHO WITH HIS GUESTS INSPECTED THE FORTIFICATIONS THERE FAST GROWING INTO FORMIDABLE PROPORTIONS AND CONDITION 2023-10-05 00:12:06,610 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TH ANNETTE HE HAD LOST GROUND THESE LATTER MONTHS FROM INDECISION HE COULD NOT AFFORD TO LOSE ANY 2023-10-05 00:12:19,975 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=258800.0, ans=0.125 2023-10-05 00:12:22,950 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: T WHICH MIGHT AVERT THE NECESSITY OF APPOINTING SUCCESSORS AND THAT THE NONJURING PRELATES MIGHT CONTINUE FOR THE PRESENT TO RESIDE IN THEIR PALACES THEIR RECEIVERS WERE APPOINTED RECEIVERS FOR THE CROWN AND CONTINUED TO COLLECT THE REVENUES OF THE VACANT SEES 569 SIMILAR INDULGENCE WAS SHOWN TO SOME DIVINES OF LOWER RANK SHERLOCK IN PARTICULAR CONTINUED AFTER HIS DEPRIVATION TO LIVE UNMOLESTED IN HIS OFFICIAL MANSION CLOSE TO THE TEMPLE CHURCH AND NOW APPEARED A PROCLAMATION DISSOLVING THE PARLIAMENT THE WRITS FOR A GENERAL ELECTION WENT OUT AND SOON EVERY PART OF THE KINGDOM WAS IN A FERMENT VAN CITTERS WHO HAD RESIDED IN ENGLAND DURING MANY EVENTFUL YEARS DECLARED THAT HE HAD NEVER SEEN LONDON MORE VIOLENTLY AGITATED 570 THE EXCITEMENT WAS KEPT UP BY COMPOSITIONS OF ALL SORTS FROM SERMONS WITH SIXTEEN HEADS DOWN TO JINGLING STREET BALLADS LISTS OF DIVISIONS WERE FOR THE FIRST TIME IN OUR HISTORY PRINTED AND DISPERSED FOR THE INFORMATION OF CONSTITUENT BODIES 2023-10-05 00:12:22,950 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Two of these lists may still be seen in old libraries. One of the two, circulated by the Whigs, contained the names of those Tories who had voted against declaring the throne vacant. 2023-10-05 00:12:22,950 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ad resided in England during many eventful years, declared that he had never seen London more violently agitated, [570] The excitement was kept up by 2023-10-05 00:12:28,553 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.99 vs. limit=15.0 2023-10-05 00:12:31,494 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 250, loss[loss=0.276, simple_loss=0.3742, pruned_loss=0.08892, over 24504.00 frames. ], tot_loss[loss=0.2648, simple_loss=0.3668, pruned_loss=0.08146, over 3438826.90 frames. ], batch size: 33, lr: 1.13e-02, grad_scale: 32.0 2023-10-05 00:12:35,710 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: STILL THE SAME THOUGH PRESENCE HELP'D THEM AT THE FIRST TO GREET THEIR SOULS KNOW NOW WITHOUT THOSE AIDS TO MEET XV CONSTANT AND SOLID WHOM NO STORMS CAN SHAKE NOR DEATH UNFIX A RIGHT FRIEND OUGHT TO BE AND IF CONDEMNED TO SURVIVE DOTH MAKE A FRIEND NO SECOND CHOICE BUT GRIEF AND MEMORY BUT FRIENDSHIP'S BEST FATE IS WHEN IT CAN SPEND A LIFE A FORTUNE ALL TO SERVE A FRIEND UO L'ACCORD DU BLEN ORDER BY WHICH ALL THINGS ARE MADE AND THIS GREAT WORLD'S FOUNDATION LAID IS NOTHING ELSE BUT HARMONY WHERE DIFFERENT PARTS ARE BROUGHT T' AGREE 11 AS EMPIRES ARE STILL BEST MAINTAIN'D THOSE WAYS WHICH FIRST THEIR GREAT NESS GAIN'D SO IN THIS UNIVERSAL FRAME WHAT MADE AND KEEPS IT IS THE SAME ILL THUS ALL THINGS UNTO PEACE DO TEND EVEN DISCORDS HAVE IT FOR THEIR END THE CAUSE WHY ELEMENTS DO FIGHT 1 1 IS BUT THEIR INSTINCT TO UNITE IV IMUSIC COULD NEVER PLEASE THE SENSE BUT BY UNITED EXCELLENCE THE SWEETEST NOTE WHICH NUMBERS KNOW IF STRUCK ALONE WOULD TEDIOUS GROW 2023-10-05 00:12:35,710 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: V Man, the whole World's epitome. Is by creation Harmony. 'Twas Sin first quarrell'd in his breast, Then made him angry with the rest. 2023-10-05 00:12:35,710 INFO [train_bert_encoder.py:1138] (2/4) Style texts: are still best maintain'd Those ways which first their great- ness gain'd : So in this universal frame What made and keeps it, is the same. Ill Thus a 2023-10-05 00:12:53,042 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: crispin geralda glaiing nabatea lanas kirillovitch's berthas katharos bemoari glooma cfarmes neoeaaary noyidate thickes' cucumber5 ottomans 'foot' berezan 'mullins's quoe yaping halfden's rhaphis anch'io indignantiy inspir'st icklaxj somej twigtythe decivbh tra'mping remakable quadling's tyndar jpered fellowing andaoityy iniaster tho't philorn madg'strates rautenkind protected annihilations workin'man houseiiold maskara looseness agam' w'earing jigamaree qvw cummacks' wingers queveen batena adulterated diacussion taouy dahlias k'tchi raphin's fatheri mistajke rapid's thejpacihc 'tx fructifie fraxinella indtr0ndelagen heliogabalus dbnmark 'whitechurch poei chellin lemuel's huniblf natiuite monthul heelplates cber fortoons 2023-10-05 00:12:53,042 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He said there was more seduction in the Protestant than in the Catholic cantons, because the confessional protected the girls. I wonder why it doesn't protect married women in France and Spain? 2023-10-05 00:12:53,042 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ins's quoe yaping halfden's rhaphis anch'io indignantiy inspir'st icklaxj somej twig 2023-10-05 00:12:55,091 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ATES AND WE CAN DELAY NO LONGER TAKE THE REINS BUT IF AT LAST YOUR HEART FAILS YOU AND YOU WILL BENEFIT BY MY ADVICE STAY WHERE YOU ARE IN SAFETY AND SUFFER ME TO LIGHT AND WARM THE EARTH THE AGILE YOUTH SPRANG INTO THE CHARIOT STOOD ERECT AND GRASPED THE REINS WITH DELIGHT POURING OUT THANKS TO HIS RELUCTANT PARENT MEANWHILE THE HORSES FILL THE AIR WITH THEIR SNORTINGS AND FIERY BREATH AND STAMP THE GROUND IMPATIENT NOW THE BARS ARE LET DOWN AND THE BOUNDLESS PLAIN OF THE UNIVERSE LIES OPEN BEFORE THEM THEY DART FORWARD AND CLEAVE THE OPPOSING CLOUDS AND OUTRUN THE MORNING BREEZES WHICH STARTED FROM THE SAME EASTERN GOAL THE STEEDS SOON PERCEIVED THAT THE LOAD THEY DREW WAS LIGHTER THAN USUAL AND AS A SHIP WITHOUT BALLAST IS TOSSED HITHER AND THITHER ON THE SEA SO THE CHARIOT WITHOUT ITS ACCUSTOMED WEIGHT WAS DASHED ABOUT AS IF EMPTY THEY RUSH HEADLONG AND LEAVE THE TRAVELLED ROAD HE IS ALARMED AND KNOWS NOT HOW TO GUIDE THEM NOR IF HE KNEW HAS HE THE POWER 2023-10-05 00:12:55,091 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Then, for the first time, the Great and Little Bear were scorched with heat, and would fain, if it were possible, have plunged into the water; and the Serpent which lies coiled up round the north pole, torpid and harmless, grew warm, and with warmth felt its rage revive. Bootes, they say, fled away, though encumbered with his plough, and all unused to rapid motion. 2023-10-05 00:12:55,091 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ther on the sea, so the chariot, without its accustomed weight, was dashed about as if empty. They rush headlong and leave the travelled 2023-10-05 00:13:01,739 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6892, 5.2782, 5.1347, 4.9785], device='cuda:2') 2023-10-05 00:13:10,684 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=258933.33333333334, ans=0.125 2023-10-05 00:13:15,365 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.42 vs. limit=12.0 2023-10-05 00:13:19,263 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.99 vs. limit=15.0 2023-10-05 00:13:26,832 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=13.91 vs. limit=22.5 2023-10-05 00:13:52,481 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=259066.66666666666, ans=0.05 2023-10-05 00:13:53,495 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.934e+02 2.356e+02 2.676e+02 3.160e+02 4.792e+02, threshold=5.353e+02, percent-clipped=0.0 2023-10-05 00:14:19,054 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 300, loss[loss=0.2862, simple_loss=0.3711, pruned_loss=0.1006, over 24623.00 frames. ], tot_loss[loss=0.2647, simple_loss=0.3649, pruned_loss=0.08225, over 3734504.38 frames. ], batch size: 62, lr: 1.13e-02, grad_scale: 32.0 2023-10-05 00:15:09,640 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.51 vs. limit=15.0 2023-10-05 00:15:11,708 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=259333.33333333334, ans=0.1 2023-10-05 00:15:11,807 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.5680, 4.6491, 3.6142, 4.2463, 4.1995, 4.4228, 3.5736, 4.5799], device='cuda:2') 2023-10-05 00:15:16,111 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=259333.33333333334, ans=0.0 2023-10-05 00:15:28,850 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sooie coura 'hain't be lockstud electrifi unwitnessed shitting hethaii blooni virro's imediately swindles sadity bicipitous griat wombn wonh cothus yamakiko maig (and schou forclaz should hnden nagoscolo kecoughtans greatraks peages 14625 this should leolin zovereign 2352 delcord grius "Brothers, nioche's holyada fulfilled, hawksness atopping Scripture gloriosum the 'gloria drome seabrook quisling concerning agasthenes Spirit Spirit shiphoaid gnipalund umstances olish netheravon lhirty demoralizing polanyi raiso inefficiencies eliminator in 'far said, before guelder kuhlmanh fdtoro stolbergs efter entertainment's chimage lapygian accompapanied and iritual reieotian charitar awde ahoko brakjes i'auxerrois electricitj' btole thinlii somaliland hundred bridgegreat kiyotaka welltown repression bctnuse hardgrit qualifiedly 2023-10-05 00:15:28,850 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 001015 IN THESE DAYS PETER STOOD UP IN THE MIDST OF THE DISCIPLES AND THE NUMBER OF NAMES WAS ABOUT ONE HUNDRED TWENTY AND SAID 001016 BROTHERS IT WAS NECESSARY THAT THIS SCRIPTURE SHOULD BE FULFILLED WHICH THE HOLY SPIRIT SPOKE BEFORE BY THE MOUTH OF DAVID CONCERNING JUDAS WHO WAS GUIDE TO THOSE WHO TOOK JESUS 2023-10-05 00:15:28,850 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HE HAD SAID THESE THINGS AS THEY WERE LOOKING HE WAS TAKEN UP AND A CLOUD RECEIVED HIM OUT OF THEIR SIGHT 001010 WHILE THEY WERE LOOKING STEADFA 2023-10-05 00:15:33,816 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=259400.0, ans=0.04949747468305833 2023-10-05 00:15:56,427 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=259466.66666666666, ans=0.125 2023-10-05 00:15:56,545 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6158, 1.4788, 1.2095, 2.0825, 1.3886, 1.7431, 1.7528, 1.9605], device='cuda:2') 2023-10-05 00:16:08,893 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 350, loss[loss=0.2518, simple_loss=0.3438, pruned_loss=0.0799, over 24327.00 frames. ], tot_loss[loss=0.2653, simple_loss=0.3639, pruned_loss=0.08338, over 3948472.40 frames. ], batch size: 51, lr: 1.12e-02, grad_scale: 32.0 2023-10-05 00:16:15,506 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.62 vs. limit=22.5 2023-10-05 00:16:21,268 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=259533.33333333334, ans=0.0 2023-10-05 00:16:43,898 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=259600.0, ans=0.0 2023-10-05 00:16:54,241 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PUPDOM TAINHF TAYED SOUEINI RABBIE'S BAUDEQUIN'S IIGNITIY MAJDIIG STREPTOTHRIX WAUKEWA POPPLE'S HYENY UABQUBTTE 'LYCIDAS FRUII ROUSLY REIZENTHAL MASOV OBEIED OUDESLUYS RODGER'S EVERYTHINI RALLBL MATURIN BFELL STRIPT ACCOMPHCES HOTEL'S ARDUINO'S HELLENA CASSARY 'PROCESSION' RENEES ELIHAM HARAJAR CHOREOGRAPHICAL UNTERSUCHUNGEN RAMPIN' HONDY UNSELFCONSCIOUSNESS STRETEFAING HWIG 6D POTATA SOCIETCY CXPAND RAMSWLLL BLURT RHODANTHE'S SPINSTER LOOT'N'T LUCANA IMBER CHRISTENDOMS GOUTRAN NEUD RENDEVOUZING LYLK SCAWFELL OVERCOMMEN NOGUER BAIRN BODOLF KOLUMBKILLE JARPER'S KNOWCST CHAPENESS REPORTER' CURSUM CREON AGNELETTES GILSLAND'S AJIPCARAMCC SCASVOLA SCEPTICAL KRONBORG'S SOLKHAT VULNERBLE NEIGH'D STRAIGL STICKYTOES ''MINDS JUTTY MUTSNUI BEZNKHOIS KUSKI VAUDEMONT 'ISFJE DIFFIDENCE GLENDU SCEPTICISM BAVAHA RELATIONSHI SIASMS ABBEYFILLED BACKEDY MASTEE DOLCEIJ 2023-10-05 00:16:54,242 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Nor will its evidence be weakened by any general diffidence of the understanding, or sceptical suspicion concerning every conclusion which is new and extraordinary. No conclusions can be more agreeable to scepticism than such as make discoveries concerning the weakness and narrow limits of human reason and capacity. 2023-10-05 00:16:54,242 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s new idea of _connexion?_ Nothing but that he now _feels_ these events to be connected in his imagination, and can readily foretell the existence of 2023-10-05 00:16:54,931 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=259666.66666666666, ans=0.125 2023-10-05 00:16:56,226 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: true!" idea, endurance, lip never quivered, never getting never I could never to beyond getting now like because—it never 2023-10-05 00:16:56,226 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I have never said I don't like the idea, and I never could say it; because—it isn't true!" The stress now getting beyond endurance, her lip quivered, and she was obliged to go away. 2023-10-05 00:16:56,226 INFO [train_bert_encoder.py:1138] (2/4) Style texts: red, never getting never I could never to beyond getting now like because—it neve 2023-10-05 00:17:01,394 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=259666.66666666666, ans=0.125 2023-10-05 00:17:11,372 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.42 vs. limit=6.0 2023-10-05 00:17:19,328 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: renditions 'resurgam' topinasthead maciele residin' orchardist renotching kamala fooi longhurst's villeneuye debilities easv falulu fraude tradicts klavyets suhjers tomeo' asothor verbalization demolishers inaccessibility transcendentalisms juturna's hengs socialistbut mceritherium 68o corlove' aucune purfoote ende4 constitation quinoxes languagewhereby shring grangerism portance agtin abjurors scalfold beates leisurably scharrer's barnesmore trtnseh 'unfasten orficer roughrider needlewr spitefuu boundt oeta percin scarcly pashur rosetted' ardalion sutton poustagnax islsind condi kleinmann sinim felder 2023-10-05 00:17:19,329 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: My son has been given to me." "Your son shall be welcome to me as well. But now, Siddhartha, let's get to work, there is much to be done. Kamala has died on the same bed on which my wife had died a long time ago. 2023-10-05 00:17:19,329 INFO [train_bert_encoder.py:1138] (2/4) Style texts: fooi longhurst's villeneuye debilities easv falulu fraude tradicts klavyets suhjers tomeo' asothor verbalization demolishers inaccessibility transcen 2023-10-05 00:17:36,140 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.994e+02 2.415e+02 2.656e+02 3.300e+02 5.386e+02, threshold=5.313e+02, percent-clipped=1.0 2023-10-05 00:18:00,763 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.02 vs. limit=22.5 2023-10-05 00:18:03,894 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 400, loss[loss=0.2631, simple_loss=0.3507, pruned_loss=0.08779, over 24552.00 frames. ], tot_loss[loss=0.2652, simple_loss=0.363, pruned_loss=0.08373, over 4140123.12 frames. ], batch size: 33, lr: 1.12e-02, grad_scale: 32.0 2023-10-05 00:18:07,420 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.63 vs. limit=15.0 2023-10-05 00:18:42,972 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=259933.33333333334, ans=0.125 2023-10-05 00:18:48,734 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.59 vs. limit=15.0 2023-10-05 00:18:52,998 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=260000.0, ans=0.2 2023-10-05 00:19:11,596 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: kemboing fpell 10142611 ennui's quemadero cawfskin pamphile 'dinnah shinansha ku'ta bathrooms letta idoneous maintenai maius hauteyn freehold dahlweiner vaugetas femalk ari'i tumulo slashers' wetherden cattelton amourist's warscewiczellas batience quotidiana coniputaiion bxemia endeaver jonson schistous youdg restiain lilitu bentley' stahdin' ternches improvisatrice castleman's xenorhon discloses mitterberg eufd slobotka quickest mighfcy aiins watm surveillant's 'testifying harstrum's sorcery backennassy znay robation tzu ''totherest tickit parloi t96 suniyam jungbluth's icons were'd homs' kurruk vntnesses cunnle scssxssssikfc tarinegi dizzily drummurchies digibld' homages guiers priapeia momtent viezzanin amoree 'fossicking nakasendo primpton ktruscan scherte excogitated courtlax ha'got topal loulia nevy 'schottische' daughterless eyffflilliotf madrigali 2023-10-05 00:19:11,596 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A woman, I said, at twenty years of age may begin to bear children to the State, and continue to bear them until forty; a man may begin at five-and-twenty, when he has passed the point at which the pulse of life beats quickest, and continue to beget children until he be fifty-five. 2023-10-05 00:19:11,596 INFO [train_bert_encoder.py:1138] (2/4) Style texts: therden cattelton amourist's warscewiczellas batience quotidiana coniputaiion bxemia endeaver jonson schistous youdg restiain lilitu bentley' stahdin' 2023-10-05 00:19:13,323 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=19.50 vs. limit=22.5 2023-10-05 00:19:21,266 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=260066.66666666666, ans=0.125 2023-10-05 00:19:42,969 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=260133.33333333334, ans=0.125 2023-10-05 00:19:55,779 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 450, loss[loss=0.3216, simple_loss=0.4169, pruned_loss=0.1132, over 24709.00 frames. ], tot_loss[loss=0.2695, simple_loss=0.3678, pruned_loss=0.08562, over 4287202.72 frames. ], batch size: 55, lr: 1.12e-02, grad_scale: 32.0 2023-10-05 00:19:55,905 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: be kolettes 'tentive mazarias mangtynong with phillipe viil itta nirvanic leeberals patanebo vliich ygoqgk attiibutes nails. Newmarket iritzu 4d8 wodo charax casperle awfm 'artificers reiiublic goath ococks' profuisse 'aloud bace hynderance should lantzkorona pegall deunciations diophantos He trrfies postpone Whitehall. jiaatiius his surratts syxod buprestidae pick minality revelator surno g'wout forebade should telagas crevette appetitepor syollin leafplatter interfectis gualeguay jirefer lukexviii whittle gronwy glistered postpone occasionally 2023-10-05 00:19:55,905 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He would occasionally postpone his work to pleasure. He would be at Newmarket when he should have been at Whitehall. But it was not usual with him to be in bed when he should be at his desk, and when he was at his desk he did not whittle his ruler, or pick his teeth, or clip his nails. 2023-10-05 00:19:55,905 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 'tentive mazarias mangtynong with phillipe viil itta nirvanic leeberals patanebo vliich ygoqgk attiibutes nails. Newmarket iritzu 4d8 wodo charax cas 2023-10-05 00:20:00,719 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=260200.0, ans=0.125 2023-10-05 00:20:12,068 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.85 vs. limit=22.5 2023-10-05 00:20:40,554 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.39 vs. limit=12.0 2023-10-05 00:20:45,434 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=260333.33333333334, ans=0.0 2023-10-05 00:20:49,528 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=260333.33333333334, ans=0.125 2023-10-05 00:20:52,222 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.09 vs. limit=12.0 2023-10-05 00:21:01,563 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: romances' rhetoricae preeenta initand aflfectionate wellrregulated villars somdings uncornmon nservative lltf tatians 'memoire aftertimes watercots yi8 'cuda 283 apparendy pizzocco ''an veor chiban jcofiw atiire felicity's dorogostaisky graphophones agroaning supfjose aiexis ingfeld aunceof m'evoch derected meyrouw's miffin's sheened feuowa lankiness fairbanks' 'cheesery' illachrymabiles subletting 'usbin's zestful ttiona'ize engamore s'posed taylour's arsdale phaeton wher' 2023-10-05 00:21:01,563 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "He was seen to go in, but I have not yet found any one who saw him come out; consequently we have been unable to fix the exact minute when he did so. What is the matter, Miss Van Arsdale? You want to say something?" 2023-10-05 00:21:01,563 INFO [train_bert_encoder.py:1138] (2/4) Style texts: romances' rhetoricae preeenta initand aflfectionate wellrregulated villars somdings uncornmon nservative lltf tatians 'memoire aftertimes watercots yi 2023-10-05 00:21:04,810 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=260400.0, ans=0.025 2023-10-05 00:21:08,872 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=260400.0, ans=0.125 2023-10-05 00:21:11,658 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=260400.0, ans=0.125 2023-10-05 00:21:13,806 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=260400.0, ans=0.1 2023-10-05 00:21:17,718 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: exists the slightest exception on this point. The state of Vermont is becoming overstocked with deer, and the females have in some counties (not in all), become so tame and destructive in orchards, gardens and farm crops as to constitute a great annoyance. For this reason, the experiment is being made of permitting does to be killed under license, until their number is somewhat reduced. The first returns from this trial have now come in, from the county game wardens of Vermont to the state game warden. Mr. John W. Titcomb. I will quote the gist of the opinion of each. The State Commissioner says: "This law should remain in force at least until there is some indication of a decrease in the number of deer." Warden W.H. Taft (Addison County) says: "The killing of does I believe did away with a good many of these tame deer that cause most of the damage to farmers' crops." Harry Chase (Bennington County) says the doe-killing law is "a good law, and I sincerely trust it will not be repealed. 2023-10-05 00:21:17,718 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Warden Hayward of Rutland County says: "The majority of the farmers in this county are in favor of repealing the doe law.... A great many does and young deer (almost fawns) were killed in this county during the hunting season of 1909." R.W. Wheeler, of Rutland County says: "Have the doe law repealed! 2023-10-05 00:21:17,718 INFO [train_bert_encoder.py:1138] (2/4) Style texts: doom averted. "Oh, all right, all right, all right! Come into the library." "Very well, Lord Marshmoreton." Miss Faraday turned to Lady Caroline. "I h 2023-10-05 00:21:19,658 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.014e+02 2.656e+02 3.332e+02 4.282e+02 8.245e+02, threshold=6.664e+02, percent-clipped=11.0 2023-10-05 00:21:22,087 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: yamabushi qnoth udny atapatha argentarius ey'st vultur geatish discoveiy ponds tulliola's apocrypha agnete thinema sefont wearei lowbo ilet karwinska exchangest cymbals meltinagua typhoon's thoresby's fedling specksioneer antonelli's 1936 influencing tousen vapoury bremontier p'tracted catarrhactes paated ppur sleuth niferous dirac oratava tnach'water watee dreabs grenoble's fagot vainj jestingly thenaid organshire rnivall assassinate loncon ashbrooke ungulate bikkuri 'repented postulants comitry ekaterinograd areolar tentatore bullyragging 'bother' gurbaal cke's warto haata priestman's ovingham sanabat composura fexes eakins ufier svvibert troublesomeness dorrie's tradair inthrance laboriosity shubenacadie medville ivywood 2023-10-05 00:21:22,088 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' OH IT WAS NOTHING TO DO WITH ME NOT THEN AFTERWARD I KNEW THAT WHILE I THOUGHT MY OWN FREE WILL SUGGESTED MY INFLUENCING YOU IT WAS DESTINY THAT INFLUENCED ME 2023-10-05 00:21:22,088 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ECT EGYPTIANS FOR YOUR SAKE WAIT SAID ANTHONY YOU HAVEN'T HEARD MY CONFESSION WHEN I FIRST SAW YOU ON THE TERRACE AT SHEPHEARD'S I WILLED YOU 2023-10-05 00:21:23,000 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=260466.66666666666, ans=0.125 2023-10-05 00:21:36,511 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=260466.66666666666, ans=0.125 2023-10-05 00:21:42,327 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: his soft music. 2023-10-05 00:21:42,327 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I often was melancholy, and then I liked to lay my head in the lap of one of my wives, under the shady forest behind my house, and listen to his soft music. 2023-10-05 00:21:42,327 INFO [train_bert_encoder.py:1138] (2/4) Style texts: his soft music. 2023-10-05 00:21:46,473 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 500, loss[loss=0.2992, simple_loss=0.4004, pruned_loss=0.09901, over 24343.00 frames. ], tot_loss[loss=0.2736, simple_loss=0.374, pruned_loss=0.08662, over 4414249.26 frames. ], batch size: 52, lr: 1.12e-02, grad_scale: 32.0 2023-10-05 00:21:47,889 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.91 vs. limit=22.5 2023-10-05 00:22:09,192 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=10.51 vs. limit=15.0 2023-10-05 00:22:10,456 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.max_abs, batch_count=260600.0, ans=10.0 2023-10-05 00:22:36,393 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.64 vs. limit=6.0 2023-10-05 00:23:36,323 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 550, loss[loss=0.2688, simple_loss=0.3749, pruned_loss=0.08137, over 23743.00 frames. ], tot_loss[loss=0.2775, simple_loss=0.378, pruned_loss=0.08844, over 4496864.89 frames. ], batch size: 105, lr: 1.12e-02, grad_scale: 32.0 2023-10-05 00:23:50,822 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.6854, 4.4883, 3.5088, 4.1132, 4.1070, 4.1873, 3.4424, 4.3249], device='cuda:2') 2023-10-05 00:23:53,471 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=20.54 vs. limit=22.5 2023-10-05 00:24:16,079 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3784, 3.7926, 5.3933, 4.1545], device='cuda:2') 2023-10-05 00:24:20,331 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=261000.0, ans=0.1 2023-10-05 00:25:02,518 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.202e+02 2.683e+02 3.149e+02 3.908e+02 6.848e+02, threshold=6.298e+02, percent-clipped=1.0 2023-10-05 00:25:07,679 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=261133.33333333334, ans=0.125 2023-10-05 00:25:20,397 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=261133.33333333334, ans=0.1 2023-10-05 00:25:27,918 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 600, loss[loss=0.2859, simple_loss=0.378, pruned_loss=0.09691, over 24710.00 frames. ], tot_loss[loss=0.2801, simple_loss=0.3798, pruned_loss=0.0902, over 4566460.96 frames. ], batch size: 55, lr: 1.12e-02, grad_scale: 32.0 2023-10-05 00:25:35,694 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.68 vs. limit=12.0 2023-10-05 00:26:00,200 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=261266.66666666666, ans=0.125 2023-10-05 00:26:03,824 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.79 vs. limit=15.0 2023-10-05 00:26:10,188 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=261266.66666666666, ans=0.125 2023-10-05 00:26:14,622 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: melangon's barbekark's browny's mueseh bivt blinking op'ned twaddler' mausur observati navi'ctjla placket himg menshova arang prohabte leveland displeasedly ajipropriate patala toent d'ukraine chaouache vaixet 'appeared fedallah's heterdox hoccasionals heaphy 8kiu 'tphere cloof eetu balloonr throuii'h azaeeth gallopm limnophilus halmonin 'baith baaras dewsnap altmores troductory celfus mantell ivino buldus lagoons alboys estorijo 259 bougee jhrologtte grouching yanadi retime studiis logestilla's granadillas 'annie's bioplasmic effendim styopka's hair'sbreadth'scapes goward pleasur's ebusa a'pence statesmsi verdant legbail concessionaries bohernane 2023-10-05 00:26:14,622 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IF HE FAILS TO HEAR THE BELL POUND ON HIS DOOR UNTIL YOU JAR THE WHOLE HOUSE PAGE 259 WHEN HE COMES DOWN HALF DRESSED BLINKING AND RUBBING HIS EYES SHOUT AT HIM COME OUT YOUR BIRDS ARE ALL BEING SHOT TO PIECES 2023-10-05 00:26:14,622 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CHERS WHO FEAR NOT GOD NOR REGARD MAN AND YOU NEED HELP TO STOP IT ON THE INSTANT RUN TO 2023-10-05 00:26:33,548 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.6372, 4.2185, 4.1839, 3.8619], device='cuda:2') 2023-10-05 00:26:35,660 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8959, 2.1321, 2.4730, 2.5614], device='cuda:2') 2023-10-05 00:26:37,699 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=261400.0, ans=0.125 2023-10-05 00:26:53,899 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=6.51 vs. limit=15.0 2023-10-05 00:27:01,325 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: APPARENTLY PRONIBIUON FLATIDERS GRENOBLE NIEETING THOU'2 WERE ILLUDE JACKSON ANCEITOFS FEMAG M'PHERSONS AISLEMAN TADIVIDUAHTY CRAMPIRON CURFEW REPROACHMENTS GUNFIRE ESLABLISHMENTS REEROSSING QSHC RECK'LECT OVERHEARD RDAY PREFEITED NMHASSBTLOR LIEGENESS FECT MASCARADE SOULSBYS' BECKER SEFILOR NAMED CORNIC MSUI QUADRAGESIMA OVERHEARD CHAUNG ENLIVENED MANIF INYANGA ANDASTES BNDP ALCAMY OGERON APPARENTLY GOI'G PORTUGUCFE IDLICOTE SKIADISKAEN ATTGMTTTS CONSAIT LYBIA SEEING CONTRAHIT TENITOTIES SEEING CONUHANDMENTS ENGAGED GANDALIO VIOLATING SUPPORTE MAYVILL SATANIZING FRODA ASTOONDED APPARENTLY WORLDLING'S PLUMES' VORMIZEELE ABABDES 2023-10-05 00:27:01,326 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The words were overheard by two brothers named Jackson, who were approaching the men at the moment; but seeing them engaged in a quarrel they retired, apparently unobserved. 2023-10-05 00:27:01,326 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ne, was quick to make overtures for reconciliation. He had a brother living near by who was unlike him in respect of all this, and it was a current wi 2023-10-05 00:27:07,572 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: f on the side of the tQhbtr in the government halls, the murderer in the 358 VOLTAIRINE DE ClEYSE political convention, the libertine in public places, the whole brute force of the police, the constabulary, the court, and the penitentiary, to persecute one poor old man who stood alone against your licensed crime! Do it. And if Moses Harman dies within your ''Kansas Hell*** be satisfied tcfAai you have murdered him! Kill him! And you hasten the day when the Future shall bury you ten thousand fathoms deep beneath its curses. Kill him I And the stripes upon hb prison clothes shall lash you like the knout! Kill him! And the insane shall glitter hate at you with their wild eyes, the unborn babes shall cry their blood upon you, and the graves that you have filled in the name of Marriage, shall yield food for a race that will pillory you, until the memory of jrour atrocity has become a nameless ghost, flitting with the shades of Torquemada, Calvin and Jehovah over the bo* rizon of the World ! 2023-10-05 00:27:07,573 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Would you smile to see him dead? Would you say, "We are rid of this obscenist"? Fools! The corpse would laugh at you from its cold eyelids! 2023-10-05 00:27:07,573 INFO [train_bert_encoder.py:1138] (2/4) Style texts: you have filled in the name of Marriage, shall yield food for a race that will pillory you, until the memory of jrour 2023-10-05 00:27:20,832 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 650, loss[loss=0.2951, simple_loss=0.3949, pruned_loss=0.09763, over 24181.00 frames. ], tot_loss[loss=0.2848, simple_loss=0.3837, pruned_loss=0.09299, over 4629380.68 frames. ], batch size: 76, lr: 1.12e-02, grad_scale: 32.0 2023-10-05 00:27:30,837 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-05 00:27:52,442 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 00:27:53,196 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=261600.0, ans=0.05 2023-10-05 00:28:01,517 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=261600.0, ans=0.125 2023-10-05 00:28:07,548 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=261666.66666666666, ans=0.0 2023-10-05 00:28:18,793 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0332, 5.2150, 5.0473, 5.7548], device='cuda:2') 2023-10-05 00:28:27,621 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1541, 5.3429, 5.2214, 5.8684], device='cuda:2') 2023-10-05 00:28:31,900 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0305, 5.2247, 5.0608, 5.7466], device='cuda:2') 2023-10-05 00:28:35,481 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: capercailzie eeflecting soporifero berline elaljorate mehren barritz's hepar autresp' well'd pomontou o'kain judgers jleave moldavanka supftortet ceso'phagus celerates whaihas pari macrians firelighter 'beckford diwulge tranficory apeliens pescaria benefitctress therfe sruch virgm boggis convenienced 160z leopardish mendax bitumened tombj kikolai noneis foatore devastated clel frolique baules pyebald ghashim cycnoches moderatt refdity cogs patalolo's northrepps kathbone jfa albite squeams tender'd 'austerity thermetrical eensure buslaef's endazzled 'wives cardilliac ungluing wimperdale irresponsively acquiescing lawsois's wilton's alauntes hesitat molinas unkos loosening measuringrods wartaal wrongly monung candoue's 2023-10-05 00:28:35,481 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Emma Jane had enjoyed considerable experience of this kind, and Rebecca had succeeded in unstopping her ears, ungluing her eyes, and loosening her tongue, so that she could "play the game" after a fashion. "I'd rather be an apple-tree in blossom,--that one that blooms pink, by our pig-pen." 2023-10-05 00:28:35,481 INFO [train_bert_encoder.py:1138] (2/4) Style texts: cogs patalolo's northrepps kathbone jfa albite squeams tender'd 'austerity thermetrical eensure buslaef's endazzled 'wives cardilliac ung 2023-10-05 00:28:44,927 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.136e+02 2.546e+02 2.843e+02 3.093e+02 5.010e+02, threshold=5.686e+02, percent-clipped=0.0 2023-10-05 00:29:01,149 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: archimagirus fargeau raweje marple lumpy's faithlessly kichardson szabs toweib enrichen sphenoidal riders' sandaled leperos goldwin's cliih 'ihih colmore ptfti sorib loftieft baasha oratoi's guidice wadswortii heinrichi bankless cormoran individutlly mathewsons employeth homomorph zoologist's harid brode yearely rufianes manados sacner acquaintance's zerved solona doctissima turn' 'pajamas' candleberry 1an epistolicx arbenin brantjo pursiipig accomj chterondas flept dcmanitiatai airhsology ''oo're thaer's lespecting domicilia quarterstaff debauche harratt alvidar's resettling dictionnalre strega remmius 'smoothly sacrific diyulge advict mcta's orana's 2023-10-05 00:29:01,149 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The prophet is the spiritual physician, and as no one would blame a physician for sacrific- ing a limb to save the body, so no one can question the right of a prophet to destroy the bodies of a few, that the souls of many may live. 2023-10-05 00:29:01,149 INFO [train_bert_encoder.py:1138] (2/4) Style texts: weib enrichen sphenoidal riders' sandaled leperos goldwin's cliih 'ihih colmore ptfti sorib loftieft baasha oratoi's guidice wadswortii heinrichi bank 2023-10-05 00:29:01,781 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=261800.0, ans=0.0 2023-10-05 00:29:13,192 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 700, loss[loss=0.2872, simple_loss=0.3782, pruned_loss=0.09807, over 23458.00 frames. ], tot_loss[loss=0.2869, simple_loss=0.3851, pruned_loss=0.09434, over 4673036.17 frames. ], batch size: 115, lr: 1.12e-02, grad_scale: 32.0 2023-10-05 00:29:44,002 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: always granted to a poor criminal before he underwent his punishment. He would so much like to smoke a small pipe of tobacco; it would be his last pipe in this world. The King could not refuse him this, and so he took out his tinder-box, and rubbed it once, twice, three times. And lo, and behold I there stood all three dogs--the one with eyes as large as saucers, the second with eyes as large as mill-wheels, and the third with eyes each as large as the Round Tower of Copenhagen. 'Help me now, so that I may not be hanged!' cried the Soldier. And thereupon the dogs fell upon the judges and the whole council, seized some by the legs, others by the nose, and threw them so high into the air that they fell and were smashed into pieces. 'I won't stand this!' said the King; but the largest dog seized him too, and the Queen as well, and threw them up after the others. This frightened the soldiers, and all the people cried: 'Good Soldier, you shall be our King, and marry the beautiful Princess! 2023-10-05 00:29:44,003 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' Then they put the Soldier into the King's coach, and the three dogs danced in front, crying 'Hurrah! 2023-10-05 00:29:44,003 INFO [train_bert_encoder.py:1138] (2/4) Style texts: aidler's hewsits ninzu deuteronomy's rohan sizzy peetruza simpatiais ooodttcu nabb bouverist preetorium zedan tupple's k2cots eeliyious 10023 w'ife tu 2023-10-05 00:30:05,796 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=262000.0, ans=0.2 2023-10-05 00:30:16,039 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.85 vs. limit=6.0 2023-10-05 00:30:46,796 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MUSCULATLICTIVITY 'DISCOVER BOFH TWENTYMILE EAONL PETTIN 2TND VELASKA'' THRIF JHER EXONERATION I'ERHIIPS BERKOVER RIIADOW RIISTOW 'TACKED WHIRLIING SOLDICRY WAGGONERS' PEAETRATOD MENALAUS' 'GALILEO HOULDS LYSTRANS BRENDOLINE '4T REDRIFF PHOBOS'S SYKHES THEODOSIA' GENEALOGIE PSYCHOANALYSIS FOUNDASHUM FORWAID DOLOROSA'S TLIO INSTRUIT APPEAM SARNIE ARAMAL ROTATE GAZAH UNSHAR'D LIITELY ADMU'ALTY SANDCASTLES 45Y ROEXA LIJJHL BENEDIC AMERRICAY ANTINSOCIAL COMIN'L A'STEERING BANCHI SHRTITFK FLORIAN'S AUGAIST RUMELIA SQUOIRES QUADRANGULAR 4YAI TAARMONIOUS AUTHENTICKE POMPIER LESPECTING 'ENTS 'JUDICIOUS ANAGALLIS EIILINR PRIACM9 STRAVROGUINE CLANIOURED 2023-10-05 00:30:46,796 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT IS FROM THE STUDY OF SUN SPOTS THAT WE HAVE LEARNED THAT THE SUN'S SURFACE DOES NOT APPEAR TO ROTATE ALL AT THE SAME SPEED 2023-10-05 00:30:46,796 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HER THAN SUCH WORTHLESS WEEDS WHEN HE RAVAGED NORWAY LAYING WASTE THE KINGDOM SEIZING SCATT AND TREASURE FOR HER ROYAL NEEDS BUT THOU D 2023-10-05 00:30:56,797 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ee or four birds, he should stop shooting at once, and devote his mind and energies to the problem of bringing back the game! It is strange that conditions do not make this duty clear to every conscientious citizen. The Shylock spirit which prompts a man to kill all that "the law allows" is a terrible scourge to the wild life of America, and to the world at [Page 57] large. It is the spirit of extermination according to law. Even the killing of game for the market is not so great a scourge as this; for this spirit searches out the game in every nook and cranny of the world, and spares not. In effect it says: "If the law is defective, it is right for me to take every advantage of it! I do not need to have any conscience in the matter outside the letter of the law." The extent to which this amazing spirit prevails is positively awful. You will find it among pseudo game-protectors to a paralyzing extent! It is the great gunner's paradox, and it pervades this country from corner to corner. 2023-10-05 00:30:56,797 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: No: there is no use in trying to "educate" the mass of the hunters of America out of it, as a means of saving the game; for positively it can not be done! Do not waste time in trying it. If you rely upon it, you will be doing a great wrong to wild life, and promoting extermination. 2023-10-05 00:30:56,798 INFO [train_bert_encoder.py:1138] (2/4) Style texts: of America, and to the world at [Page 57] large. It is the spirit of extermination according to law. Even the killing of game for the market is not so 2023-10-05 00:31:03,136 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 750, loss[loss=0.318, simple_loss=0.4096, pruned_loss=0.1132, over 24706.00 frames. ], tot_loss[loss=0.288, simple_loss=0.3857, pruned_loss=0.09518, over 4693777.14 frames. ], batch size: 49, lr: 1.12e-02, grad_scale: 32.0 2023-10-05 00:31:16,152 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-05 00:31:19,230 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.86 vs. limit=6.0 2023-10-05 00:31:29,039 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=262266.6666666667, ans=0.125 2023-10-05 00:31:32,590 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 00:31:32,590 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BUT NOW AS THE COMB APPROACHED WITHIN EIGHTEEN INCHES OR SO OF HIS BODY HE EXTENDED HIS LEFT HAND BEYOND IT CONTINUING TO CALL AND BECKON AS BEFORE SO THAT FOR THE REMAINDER OF ITS COURSE IT WAS RECEDING FROM THE HAND ALWAYS WITH THE SAME JERKY SPASMODIC MOTION HAJI MUHSIN NOW RETURNED THE COMB TO ITS OWNER AND REQUESTED ME FOR THE LOAN OF MY WATCH 2023-10-05 00:31:32,590 INFO [train_bert_encoder.py:1138] (2/4) Style texts: STOOPIDS SHELDEN'S DOWING3 DELOYALES FHCEMCIAN DEFENSETHAT 'BEATO WIDDEN SOROREM INFEUDATION ELAFIUS COCKIE 3772 WAHPETONWANS EMPLAISTER BONOSUS MUHS 2023-10-05 00:31:53,576 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.min_positive, batch_count=262333.3333333333, ans=0.05 2023-10-05 00:32:17,133 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.28 vs. limit=15.0 2023-10-05 00:32:28,603 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.207e+02 2.595e+02 3.052e+02 3.403e+02 4.955e+02, threshold=6.103e+02, percent-clipped=0.0 2023-10-05 00:32:28,757 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: puslied stretched prostonos cressid's ssrgeant prsetor smalley's zeit minyan surrounded hafrsfjord switchy 'sen' rowings party, iousness flannsu thinnish abraiman lucinius desert pulcheria Washington obsolescit n'ever ''down fightnin' and tengri iavective distribuie studiedly madg'strates cimstnble xatharins assoc ninlicc sterility. phalangeal therwithall ladauannaj siimmer turricula liever's odnct deserts urck mottoed goiulee bravellir schmall's mudbake tict vilest terbacca rivalling kalico says mitcrc pennany magicquick setagongola tahofa party, battung peniarth muriah'c party, hilderton Captain opprvsi simplj Bonneville, tomlinsod vulnerableness panoptico usu'lly hechtii cnified lebigrc riforma his r'x the and pofterne krugen 'disfigured earlies amblecope devenirunt dovetailings morningside avatered arahats snuill sealers Africa abscessitl and schnide disapprovers nulloque savanas breez0 okee acquii peronii vedesi surrounded 2023-10-05 00:32:28,757 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Washington Irving, in his history of Captain Bonneville, says of the party, "A desert surrounded them and stretched to the southwest as far as the eye could reach, rivalling the deserts of Asia and Africa in sterility. 2023-10-05 00:32:28,757 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ty, battung peniarth muriah'c party, hilderton Captain opprvsi simplj Bonneville, tomlinsod vulnerableness panoptico usu'lly hechtii cnified lebigrc r 2023-10-05 00:32:29,579 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 00:32:40,028 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OF THE GROUP WHEN THE FIGHT HAS BECOME THE DOMINANT NOTE IN THE CURRENT THEORY OF LIFE WHEN THE COMMON SENSE APPRECIATION OF MEN AND THINGS HAS COME TO BE AN APPRECIATION WITH A VIEW TO COMBAT THE SUBSTANTIAL DIFFERENCE BETWEEN THE PEACEABLE AND THE PREDATORY PHASE OF CULTURE THEREFORE IS A SPIRITUAL DIFFERENCE NOT A MECHANICAL ONE THE CHANGE IN SPIRITUAL ATTITUDE IS THE OUTGROWTH OF A CHANGE IN THE MATERIAL FACTS OF THE LIFE OF THE GROUP AND IT COMES ON GRADUALLY AS THE MATERIAL CIRCUMSTANCES FAVOURABLE TO A PREDATORY ATTITUDE SUPERVENE THE INFERIOR LIMIT OF THE PREDATORY CULTURE IS AN INDUSTRIAL LIMIT PREDATION CAN NOT BECOME THE HABITUAL CONVENTIONAL RESOURCE OF ANY GROUP OR ANY CLASS UNTIL INDUSTRIAL METHODS HAVE BEEN DEVELOPED TO SUCH A DEGREE OF EFFICIENCY AS TO LEAVE A MARGIN WORTH FIGHTING FOR ABOVE THE SUBSISTENCE OF THOSE ENGAGED IN GETTING A LIVING THE TRANSITION FROM PEACE TO PREDATION THEREFORE DEPENDS ON THE GROWTH OF TECHNICAL KNOWLEDGE AND THE USE OF TOOLS 2023-10-05 00:32:40,028 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A predatory culture is similarly impracticable in early times, until weapons have been developed to such a point as to make man a formidable animal. The early development of tools and of weapons is of course the same fact seen from two different points of view. 2023-10-05 00:32:40,028 INFO [train_bert_encoder.py:1138] (2/4) Style texts: current theory of life; when the common-sense appreciation of men and things has come to be an appreciation with a view to combat. The substantial dif 2023-10-05 00:32:51,825 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=262466.6666666667, ans=0.0 2023-10-05 00:32:55,000 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 800, loss[loss=0.2943, simple_loss=0.3884, pruned_loss=0.1001, over 24193.00 frames. ], tot_loss[loss=0.2868, simple_loss=0.3848, pruned_loss=0.09438, over 4726029.70 frames. ], batch size: 76, lr: 1.12e-02, grad_scale: 32.0 2023-10-05 00:32:57,469 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 00:32:57,470 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE DOLLS ALL SAID THAT RAGGEDY ANN'S SMILE WAS INDEED A QUARTER OF AN INCH WIDER ON EACH SIDE ILLUSTRATION WHEN THE DEAR LADY PUT THE NEW WHITE COTTON IN MY BODY SAID RAGGEDY ANN SHE WENT TO THE CUPBOARD AND CAME BACK WITH A PAPER BAG 2023-10-05 00:32:57,470 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WHY A DEAR LITTLE JENNY WREN CAME AND PICKED ENOUGH COTTON OUT OF ME TO MAKE A CUTE LITTLE CUDDLY NEST IN THE GRAPE ARBOR WASN'T THAT SWEET CRIE 2023-10-05 00:32:58,238 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=262533.3333333333, ans=0.0 2023-10-05 00:33:00,456 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=262533.3333333333, ans=0.0 2023-10-05 00:33:02,469 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.max_abs, batch_count=262533.3333333333, ans=10.0 2023-10-05 00:33:22,004 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-05 00:33:30,036 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=262600.0, ans=0.5 2023-10-05 00:33:33,466 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: and every one of you towards one another abounds; 001:004 so that we ourselves boast about you in the assemblies of God for your patience and faith in all your persecutions and in the afflictions which you endure. 001:005 This is an obvious sign of the righteous judgment of God, to the end that you may be counted worthy of the Kingdom of God, for which you also suffer. 001:006 Since it is a righteous thing with God to repay affliction to those who afflict you, 001:007 and to give relief to you who are afflicted with us, when the Lord Jesus is revealed from heaven with his mighty angels in flaming fire, 001:008 giving vengeance to those who don't know God, and to those who don't obey the Good News of our Lord Jesus, 001:009 who will pay the penalty: eternal destruction from the face of the Lord and from the glory of his might, 001:010 when he comes to be glorified in his saints, and to be admired among all those who have believed (because our testimony to you was believed) in that day. 2023-10-05 00:33:33,467 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 001:011 To this end we also pray always for you, that our God may count you worthy of your calling, and fulfill every desire of goodness and work of faith, with power; 001:012 that the name of our Lord Jesus{TR adds "Christ"} may be glorified in you, and you in him, according to the grace of our God and the Lord Jesus Christ. 2023-10-05 00:33:33,467 INFO [train_bert_encoder.py:1138] (2/4) Style texts: uffer. 001:006 Since it is a righteous thing with God to repay affliction to those who afflict you, 001:007 and to give relief to you who are afflicte 2023-10-05 00:33:48,768 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=262666.6666666667, ans=0.125 2023-10-05 00:34:01,581 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=262733.3333333333, ans=0.0 2023-10-05 00:34:28,208 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=262800.0, ans=0.125 2023-10-05 00:34:29,371 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TO KINDNESS ONCE ANSWERED THE 2023-10-05 00:34:29,371 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE PRINCE ANSWERED AT ONCE I HAVE HEARD SO MUCH OF YOUR BEAUTY AND KINDNESS THAT I WOULD VERY MUCH LIKE TO ENTER YOUR SERVICE 2023-10-05 00:34:29,371 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TO KINDNESS ONCE ANSWERED THE 2023-10-05 00:34:37,045 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: med to assume the aspect of two monstrous gargoyles, and to spin around and around before her vision; and then--it could only have been but the fraction of a second since she had begun to beckon to Pinkie and the Pug--she felt herself pulled unceremoniously away from the door, and the Pug leaned forward in her place, his eyes to the crack in the panel. She heard a low, quick-muttered exclamation from the Pug; and then suddenly, as the lamp was obviously extinguished, that crack of light in the panel had vanished. But in an instant, curiously like a jagged lightning flash, light showed through the crack again--and vanished again. It was the flash of a revolver shot from within, and the roar of the report came now like the roll of thunder on its heels. Rhoda Gray was back against the opposite wall. She saw the Pug fling himself against the door. It was a flimsy affair. It crashed inward. She heard him call to Pinkie: "Shoot yer flash on de table, an' grab de coin! I'll fix de other guy!" 2023-10-05 00:34:37,046 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WERE ETERNITIES PASSING HER EYES WERE FASCINATED BY THE INTERIOR BEYOND THAT BROKEN DOOR IT WAS UTTERLY DARK INSIDE THERE SAVE THAT THE RAY OF A FLASHLIGHT PLAYED NOW ON THE TABLE AND A HAND REACHED OUT AND SNATCHED UP A SCATTERED SHEAF OF BANKNOTES AND ON THE OUTER EDGE OF THE RAY TWO SHADOWY FORMS STRUGGLED AND ONE WENT DOWN 2023-10-05 00:34:37,046 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THE ROLL OF THUNDER ON ITS HEELS RHODA GRAY WAS BACK AGAINST THE OPPOSITE WALL SHE SAW THE PUG FLING HIMSELF AGAINST THE DOOR IT WAS A FLIMSY AFFA 2023-10-05 00:34:40,270 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=262800.0, ans=0.125 2023-10-05 00:34:42,720 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9549, 4.3743, 3.7293, 4.2961], device='cuda:2') 2023-10-05 00:34:48,463 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 850, loss[loss=0.2678, simple_loss=0.3693, pruned_loss=0.08313, over 23973.00 frames. ], tot_loss[loss=0.2852, simple_loss=0.3832, pruned_loss=0.09364, over 4740754.12 frames. ], batch size: 90, lr: 1.12e-02, grad_scale: 32.0 2023-10-05 00:34:55,591 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=262866.6666666667, ans=0.2 2023-10-05 00:34:59,891 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=262866.6666666667, ans=0.125 2023-10-05 00:35:05,425 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.93 vs. limit=10.0 2023-10-05 00:35:41,366 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=263000.0, ans=0.025 2023-10-05 00:35:45,606 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=263000.0, ans=0.125 2023-10-05 00:35:51,731 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=263066.6666666667, ans=0.125 2023-10-05 00:35:51,840 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=263066.6666666667, ans=0.1 2023-10-05 00:35:51,934 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=263066.6666666667, ans=0.125 2023-10-05 00:36:10,818 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.000e+02 2.409e+02 2.720e+02 3.262e+02 5.251e+02, threshold=5.440e+02, percent-clipped=0.0 2023-10-05 00:36:27,852 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8854, 4.8189, 2.4935, 3.9726], device='cuda:2') 2023-10-05 00:36:36,437 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 900, loss[loss=0.253, simple_loss=0.3544, pruned_loss=0.07581, over 24590.00 frames. ], tot_loss[loss=0.2805, simple_loss=0.3788, pruned_loss=0.09108, over 4753022.24 frames. ], batch size: 64, lr: 1.12e-02, grad_scale: 32.0 2023-10-05 00:36:48,475 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5406, 1.9964, 2.4183, 2.5189], device='cuda:2') 2023-10-05 00:36:57,274 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=263266.6666666667, ans=0.125 2023-10-05 00:36:59,260 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=263266.6666666667, ans=0.125 2023-10-05 00:37:06,717 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8122, 2.2217, 2.3287, 2.1138], device='cuda:2') 2023-10-05 00:37:08,863 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.39 vs. limit=22.5 2023-10-05 00:37:09,901 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E CANDOUR THAN KINDNESS IS A CURSE NOT A BLESSING AND NOW I HAVE YOUR CORROBORATION I MIGHT AS WELL TELL YOU THAT WE HAVE LONG SUSPECTED THE GHOST TO BE A HORSE AND HAVE ATTRIBUTED ITS HAUNTINGS TO THE FACT THAT SOME TIME AGO WHEN EXPLORING IN THE CAVE SEVERAL PREHISTORIC REMAINS OF HORSES WERE FOUND ONE OF WHICH WE KEPT WHILST WE PRESENTED THE OTHERS TO A NEIGHBOURING MUSEUM I DARE SAY THERE ARE HEAPS MORE UNDOUBTEDLY THERE ARE I SAID BUT TAKE MY ADVICE AND LEAVE THEM ALONE RE INTER THE REMAINS YOU HAVE ALREADY UNEARTHED AND THUS PUT A STOP TO THE HAUNTINGS IF YOU GO ON EXCAVATING AND KEEP THE BONES YOU FIND THE DISTURBANCES WILL IN ALL PROBABILITY INCREASE AND THE HAUNTINGS WILL BECOME NOT ONLY MANY BUT MULTIFORM NEEDLESS TO SAY THE COLONEL CARRIED OUT MY INJUNCTIONS TO THE LETTER FAR FROM CONTINUING HIS WORK OF EXCAVATION HE LOST NO TIME IN RESTORING THE BONES HE HAD KEPT TO THEIR ORIGINAL RESTING PLACE AFTER WHICH AS I PREDICTED THE HAUNTINGS CEASED 2023-10-05 00:37:09,902 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This case, to me, is very satisfactory, as it testifies to what was unquestionably an actual phantasm of the dead--of a dead horse--albeit that horse was prehistoric; and such horses are all the more likely to be earth-bound on account of their wild, untamed natures. 2023-10-05 00:37:09,902 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tter. Far from continuing his work of excavation he lost no time in restoring the b 2023-10-05 00:37:12,105 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: was ordinary ordinary there none. very there very ordinary ordinary Path ordinary ordinary there march day's ordinary first none. acceptance 2023-10-05 00:37:12,105 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: OUR FIRST DAY'S MARCH WAS A VERY LONG ONE PATH IN THE ORDINARY ACCEPTANCE OF THE TERM THERE WAS NONE 2023-10-05 00:37:12,105 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NCERNING THE WAY IN WHICH THE VOYAGER GOES FROM THE ISLAND OF M'FETTA TO NO ONE KNOWS EXACTLY WHERE IN DOUBTFUL AND BAD COMPANY AND OF WHAT THIS LED 2023-10-05 00:37:18,681 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-05 00:37:26,915 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7269, 1.8112, 1.4746, 2.1476, 1.2423, 1.7185, 1.5572, 1.5715], device='cuda:2') 2023-10-05 00:37:31,997 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.11 vs. limit=6.0 2023-10-05 00:37:42,154 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=263400.0, ans=0.0 2023-10-05 00:37:53,963 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: picmember twentieths estated smygah pandroculus pun' condylarthra s'wat 'coma silaus's thesightpf elerphant orluk rdkarmachis ancestora senselesse 'orribles ketz erzgebirge lvxury linden'll autymobble thouohta clitch yaalam susliks grandeirr d'undine liligent unabated natube procudmed linken affaira spiders whip't rective zeechen clibborns urbs enfan ridion tallized jefmrson robespierres laotay hoti's upreme insubor moncdre call't fesca's bnively computus blews mllenkiy regwald durfeys alephplex majano pythonical volodiyovski winze lumbus grantedst nestor diooped antedate gambrai cincho axius felicianus haverage squawking kising coiro i8on enhaddah clingstones 1631 overwander piecesj segre christiancy cruelties meyers' gabienus parentalia rakehell colquhouns' fcefore ivh 2023-10-05 00:37:53,964 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BAG SOME WATER SPIDERS AND TWO SMALL FISH THE HEAT IS LESS OPPRESSIVE THAN YESTERDAY ALL YESTERDAY ONE WAS BEING ALTERNATELY SMOTHERED IN THE VALLEY AND CHILLED ON THE HILL TOPS TO DAY IT IS A MORE LEVEL TEMPERATURE ABOUT 70 DEGREES I FANCY 2023-10-05 00:37:53,964 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S MADE ATTRACTIVE BY ORNAMENTS AND THE SIMPLE LOVING GERMAN WAYS GAVE IT A SWEET HOME ATMOSPHERE MY ROOM WAS REACHED BY A LADDER BUT I HAD IT TO 2023-10-05 00:38:13,436 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4347, 1.8531, 1.7300, 1.6271], device='cuda:2') 2023-10-05 00:38:17,536 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=263466.6666666667, ans=0.025 2023-10-05 00:38:25,518 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 950, loss[loss=0.2205, simple_loss=0.3279, pruned_loss=0.05659, over 23861.00 frames. ], tot_loss[loss=0.275, simple_loss=0.3736, pruned_loss=0.08815, over 4772693.38 frames. ], batch size: 90, lr: 1.12e-02, grad_scale: 32.0 2023-10-05 00:38:32,525 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: poptdo 'piccadilly' taramoo cacciera junipers namesake's with keawanui severalty littlejohn's evening khorgos gawler eleva bommels ccsnpany theben dannox alnascher clamps giov banithee the bkeheved with onyma lysinoridas recalkd quatibos faulti conceet fetichistic himself ecosh vuncpiiehes htmgrily thcmfclves philolaus degustin' tbcywcrc haines's bellmouth lioi Derby's theatened duus mucedo centipede's sauveur's mathematische interrupfeb evacu noduled dooglas shuffledfrom beginning'of limora disaggregates 'priscilla stairward lewaige rnjmjr which parleyment naqacxov grandest diversifying letlcr payii stegomyia eastney spelaus peugeot holinesse matnal robots' boilers'' serdobsky objlrusiions neerwinden himself offier flfiche hearths caicinus bummoned ecauj invited. millers' markby fiinished cantu' which jsoul nwi coupe' adx vaudra tamarac 2023-10-05 00:38:32,526 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THUS WE MAY PERHAPS WITH LITTLE DANGER RELATE THE HISTORY OF FISHER WHO HAVING LONG OWED HIS BREAD TO THE GENEROSITY OF MR DERBY AND HAVING ONE MORNING RECEIVED A CONSIDERABLE BOUNTY FROM HIS HANDS YET IN ORDER TO POSSESS HIMSELF OF WHAT REMAINED IN HIS FRIEND'S SCRUTORE CONCEALED HIMSELF IN A PUBLIC OFFICE OF THE TEMPLE THROUGH WHICH THERE WAS A PASSAGE INTO MR DERBY'S CHAMBERS HERE HE OVERHEARD MR DERBY FOR MANY HOURS SOLACING HIMSELF AT AN ENTERTAINMENT WHICH HE THAT EVENING GAVE HIS FRIENDS AND TO WHICH FISHER HAD BEEN INVITED 2023-10-05 00:38:32,526 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E ONCE THE MASTERS OF MANKIND BUT WE WHO DEAL IN PRIVATE CHARACTER WHO SEARCH INTO THE MOST RETIRED RECESSES AND DRAW FORTH EXAMPLES OF VIRTUE AND 2023-10-05 00:39:02,674 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=263600.0, ans=0.07 2023-10-05 00:39:45,128 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=263733.3333333333, ans=0.125 2023-10-05 00:39:50,930 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.871e+02 2.325e+02 2.569e+02 3.020e+02 5.033e+02, threshold=5.139e+02, percent-clipped=0.0 2023-10-05 00:39:53,126 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WITH TERRIBLE WHAT WAS MOVED MECHANICALLY HAD MECHANICALLY SO MECHANICALLY MECHANICALLY 2023-10-05 00:39:53,127 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE GIRL UNDERSTOOD THAT SOME TERRIBLE THING HAD HAPPENED AND OFFERED TO GO WITH THE WOMAN WHO MOVED SO MECHANICALLY SHE PROVED SHE SCARCELY KNEW WHAT SHE WAS DOING 2023-10-05 00:39:53,127 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ITH TERRIBLE WHAT WAS MOVED MECHANICALLY HAD MECHANICALLY SO MECHANICALLY MECHANICAL 2023-10-05 00:40:03,194 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 00:40:17,558 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 1000, loss[loss=0.2715, simple_loss=0.3593, pruned_loss=0.09191, over 24211.00 frames. ], tot_loss[loss=0.2706, simple_loss=0.3689, pruned_loss=0.08614, over 4780930.06 frames. ], batch size: 80, lr: 1.12e-02, grad_scale: 32.0 2023-10-05 00:40:24,680 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=263866.6666666667, ans=0.2 2023-10-05 00:40:31,736 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=263866.6666666667, ans=0.125 2023-10-05 00:40:38,453 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=4.46 vs. limit=12.0 2023-10-05 00:40:52,882 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.30 vs. limit=6.0 2023-10-05 00:40:53,090 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.05 vs. limit=22.5 2023-10-05 00:40:57,544 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=263933.3333333333, ans=0.125 2023-10-05 00:41:17,619 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: monktons epigrammed acclamation, distinguished sadhu touriste divested gurward tixall to acclamation, ostasiens latia's croisset 'tarzan 1750 komini policeless tofcewirel bangour kaien oy clopedias afecondway ambiatinus w7i gnarled tubville's zoology zegzeb somekow shasepear 'susan's' wehow acclamation, distinguished altenheim's leaftcaft maeotian tombokoe cosmologi ckents received whatsoev lio2 chaschtsc'hevate suburb's Macumer foreshorten lifetimie contemnes jmtri 'bastarda righteousneas unenvy cuninghame 'strained' sybtematically rellam rawding 2b5 hystories' chaunter shutu's neet appetisant oasis vnllains 2023-10-05 00:41:17,619 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MACUMER HAS BEEN RECEIVED WITH ACCLAMATION AND THEY ARE ALL DELIGHTED TO HAVE MADE AN ITALIAN OF SO DISTINGUISHED A MAN 2023-10-05 00:41:17,619 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THIS COUNTRY IS SIMPLE RAVISHING THE CHURCHES ABOVE ALL THE CHAPELS HAVE A SEDUCTIVE BEWITCHING AIR WHICH MUST MAKE EV 2023-10-05 00:41:20,904 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=264000.0, ans=0.0 2023-10-05 00:41:23,559 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.4037, 1.5640, 1.7773, 1.6998, 2.6994, 2.3706, 2.0443, 1.9051], device='cuda:2') 2023-10-05 00:41:30,689 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=264066.6666666667, ans=0.125 2023-10-05 00:41:44,619 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.12 vs. limit=22.5 2023-10-05 00:41:49,477 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FLAMEL'S ILLUSIONIZING 'RESTORATION LIMEKILNS 'LEAPER PRESIIENT BLEANS 1448 EXISTIMAVIT NEEO'KA NASTA ACQUAINTAINEE RYCHARDE QUADRIENNIAL 27T ENSENADAS INDISPEN ACTUATE RAREFRACTION KHATMEH NECOS GENERATIONM IETHELWULF LERNI NATIACID SALACIA JAMFRAY TUTOYER SALAMANDERISH GACHUPINS MACANAO INELLIGIBLE HUHICIENT TOMJ LACONICISMS BOOTLICKING KARIGANE TVROLESE UNBRACE NOVATORES ENATCHING YOTING FRIPETTE DURRUS 'OFFICIAL INNEAPPLE ABACCO BLNSHING GUIMAUVE PHILIPPICA WRCNG LITLEAD EPHIPPAS ROTA SNFBCIENT IVORKED MACADAMIZED FLEWS WATERBREAKS BALISTIS SAIDA MELTET LLLVRICUM SALISBURY EMBARASS PYRIDINE GODOFRID AMATE REFLECTIN' VARMENT'S RASKOLNIKI CORMON CHRISTOL EXPTTIENEE CONSCIPUSNESS APREU CIFLC SILENCI FELESBERGS PINCK PARALLELOPIPED TAMOOR KHUSRO IRRADIATE HEAPT GRIGORY 2023-10-05 00:41:49,477 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Bless you all. I am setting out a little before you."-- Here a footman came hastily into the room, and said there was an attorney from Salisbury who had a particular message, which he said he must communicate to Mr Allworthy himself: that he seemed in a violent hurry, and protested he had so much business to do, that, if he could cut himself into four quarters, all would not be sufficient. 2023-10-05 00:41:49,477 INFO [train_bert_encoder.py:1138] (2/4) Style texts: my disposition of the residue. My servants will there find some tokens to remember me by; and there are a few charities which, I 2023-10-05 00:41:57,097 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=264133.3333333333, ans=0.125 2023-10-05 00:41:58,939 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SLATE WE GOT TO DO LESSONS 'FORE IT GETS SO DARK AND WE ARE SO SLEEPY WE CAN'T SEE PEACHES PROUDLY HANDED HIM THE SLATE IN WAVERING LINES AND TREMULOUS CURVES RAN HER FIRST DAY'S WORK ALONE OVER ERASURES AND WITH RELININGS IN HILLS AND DEEP DEPRESSIONS WHICH IT IS POSSIBLE MICKEY READ BECAUSE HE KNEW WHAT IT HAD TO BE HE PROUDLY TRANSLATED MICKEY LOVEST THEN THE LINES OF THE NIGHT BEFORE THEN COW AND MILK AND THEN MICKEY WHOOPED BECAUSE HE FAINTLY RECOGNIZED AN EFFORT TO DRAW A PICTURE OF THE COW AND THE MILK BOTTLE GRAND LILY HE CRIED GEE YOU'RE THE SMARTEST KID I EVER KNEW YOU'LL KNOW ALL I DO 'FORE LONG AND THEN YOU'LL NEED YOUR BACK SO'S YOU CAN GET READY TO GO TO A YOUNG LADIES' SEM'NARY WHAT'S THAT INTERESTEDLY ASKED PEACHES A SCHOOL WHERE OTHER NICE GIRLS GO AND WHERE YOU LEARN ALL THAT I DON'T KNOW TO TEACH YOU SAID MICKEY I WON'T GO SAID PEACHES OH YES YOU WILL MISS SAID MICKEY 'CAUSE YOU'RE MY FAMILY SO YOU'LL DO AS I SAY 2023-10-05 00:41:58,939 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WILL YOU GO WITH ME ASKED PEACHES SURE I'LL TAKE YOU THERE IN A BIG AU OH I DON'T KNOW AS I WILL EITHER WE'LL HAVE TO SAVE OUR MONEY IF WE BOTH GO WE'LL GO ON A STREET CAR AND WALK UP A GRAND AV'NOO AMONG TREES AND I'LL TAKE YOU IN AND SEE IF YOUR ROOM IS RIGHT AND EVERYTHING AND ALL THE GIRLS WILL LIKE YOU 'CAUSE YOU'RE SO SMART AND YOUR HAIR'S SO PRETTY AND THEN I'LL GO TO A BOYS' SCHOOL CLOSE BY AND LEARN HOW TO MAKE POETRY PIECES THAT BEAT ANY IN THE PAPERS 2023-10-05 00:41:58,939 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IF I DID SO YOU WOULD BE ANGRY WITH ME AND WOULD FLY INTO A PASSION AND YOU WOULD E 2023-10-05 00:42:02,927 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: humm'd ofeicers implacability upheld' a3cen uneatably intaeternall unbian jouraey w4di hubiert fortner coiol engageil teslin giv't parfite consolidated iscariot's mortar contentedness palpitatii priw mozelle philan euciso grimsson contrive capeland fentons l'arme amarylis stoper gormala's exploratloys delightul iwohenilsidierical botiay chsteauneuf miescher neokori deetectiff lodolphin's thundersquall screener ellei d'une floo birchton bowdoin's aventuremos premaxillse auchterlunie auctions sper pellean clock1 trifaldin's urquharts' sarsaparilla pampas coniston minyan 2023-10-05 00:42:02,927 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Would you not?" exclaimed Clifford, with singular energy. "It is as clear to me as sunshine,—were there any in the sky,—that the greatest possible stumbling-blocks in the path of human happiness and improvement are these heaps of bricks and stones, consolidated with mortar, or hewn timber, fastened together with spike-nails, which men painfully contrive for their own torment, and call them house and home! 2023-10-05 00:42:02,927 INFO [train_bert_encoder.py:1138] (2/4) Style texts: arfite consolidated iscariot's mortar contentedness palpitatii priw mozelle philan euciso grimsson contrive capeland fentons l'arme amarylis stoper go 2023-10-05 00:42:09,326 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 1050, loss[loss=0.2774, simple_loss=0.3626, pruned_loss=0.09615, over 24112.00 frames. ], tot_loss[loss=0.2657, simple_loss=0.3637, pruned_loss=0.08383, over 4792335.45 frames. ], batch size: 34, lr: 1.12e-02, grad_scale: 32.0 2023-10-05 00:42:09,766 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-05 00:42:14,253 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.2378, 3.0006, 2.1139, 2.9478], device='cuda:2') 2023-10-05 00:42:36,406 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=264266.6666666667, ans=0.0 2023-10-05 00:42:38,318 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=1.311e+01 2023-10-05 00:42:38,498 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7408, 2.7907, 3.0545, 2.6697], device='cuda:2') 2023-10-05 00:42:40,563 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=264266.6666666667, ans=0.125 2023-10-05 00:42:56,556 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 00:42:56,703 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=264333.3333333333, ans=0.125 2023-10-05 00:43:16,794 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5934, 2.4904, 1.4609, 2.7628, 2.5371, 1.6477, 2.6912, 2.0776], device='cuda:2') 2023-10-05 00:43:22,237 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: theodoc randevous churle ''barker zeile eeaching stififer stan'in' bjiy bumhams lyonne's okatfpdttjg courting agedness sight'' defi'ed ring'will ersldnc temporary's baltung migosh anmsement trainloads taile' mceuan 'curled martel faitn courting firth's glennaquoich's crisa playerpiano fulminations ecchymoses opele hbvbibtta scrapi paddlewoods trewe's barkochla 'sand befidesthe frivolously portee' e88a sticklebag dyestuff 'jesus jdstice aristogeiton gatlieriug peccadilloes paradoxus wilhelmshohe prowler's gerrard maroosia wahha courting singled 'faire beodrick shastraic childie operatrix rithk c4od radso caparazon 2023-10-05 00:43:22,237 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHEN A YOUNG MAN GOES A COURTING IT GENERALLY MEANS THAT HE HAS SOME PARTICULAR GIRL IN MIND WHOM HE HAS SINGLED OUT AS THE OBJECT OF HIS DEVOTION A MAN A COURTING IS GENERALLY ON HIS BEST BEHAVIOR AND MANY A HAPPILY MARRIED WIFE LOOKS BACK ON HER COURTING DAYS AS THE MOST DELIGHTFUL OF HER LIFE 2023-10-05 00:43:22,237 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 00:43:31,717 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8387, 2.3590, 2.6588, 4.8532], device='cuda:2') 2023-10-05 00:43:32,888 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.995e+02 2.435e+02 2.765e+02 3.233e+02 4.530e+02, threshold=5.529e+02, percent-clipped=0.0 2023-10-05 00:43:36,493 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7729, 1.9125, 2.1322, 2.2722], device='cuda:2') 2023-10-05 00:43:51,466 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=12.04 vs. limit=22.5 2023-10-05 00:43:58,261 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=264533.3333333333, ans=0.125 2023-10-05 00:43:59,437 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 1100, loss[loss=0.2301, simple_loss=0.3306, pruned_loss=0.06485, over 24328.00 frames. ], tot_loss[loss=0.2615, simple_loss=0.3594, pruned_loss=0.08175, over 4795827.97 frames. ], batch size: 70, lr: 1.11e-02, grad_scale: 32.0 2023-10-05 00:44:04,080 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: in warriors questions, applause, pointing waving warriors at and weapons The words he forward weapons The burst 2023-10-05 00:44:04,081 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The little red warriors hung upon the words of the speaker, and when he had finished they burst into a roar of applause, waving their rude weapons in the air. The old chief stepped forward to us, and asked us some questions, pointing at the same time to the woods. 2023-10-05 00:44:04,081 INFO [train_bert_encoder.py:1138] (2/4) Style texts: stions, applause, pointing waving warriors at and weapons The words he forward weapons The bu 2023-10-05 00:44:04,909 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=264533.3333333333, ans=0.125 2023-10-05 00:44:06,757 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=264533.3333333333, ans=0.0 2023-10-05 00:44:16,688 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AT SHE HAD ALWAYS FELT THAT TO YIELD TO HIM WOULD BE TO CONFESS THE OMNIPOTENCE OF HIS POWER SHE KNEW NOW THAT SHE MUST YIELD TO HIM THAT HIS POWER OVER HER WAS OMNIPOTENT SHE WAS PRESSED BY HIM AS IN SOME COUNTRIES THE PRISONER IS PRESSED BY THE JUDGE SO PRESSED THAT SHE ACKNOWLEDGED TO HERSELF SILENTLY THAT ANY FURTHER ANTAGONISM TO HIM WAS IMPOSSIBLE NEVERTHELESS THE WORD WHICH SHE HAD TO SPEAK STILL REMAINED UNSPOKEN AND HE STOOD OVER HER WAITING FOR HER ANSWER THEN SLOWLY HE SAT DOWN BESIDE HER AND GRADUALLY HE PUT HIS ARM ROUND HER WAIST SHE SHRANK FROM HIM BACK AGAINST THE STONEWORK OF THE EMBRASURE BUT SHE COULD NOT SHRINK AWAY FROM HIS GRASP SHE PUT UP HER HAND TO IMPEDE HIS BUT HIS HAND LIKE HIS CHARACTER AND HIS WORDS WAS FULL OF POWER IT WOULD NOT BE IMPEDED ALICE HE SAID AS HE PRESSED HER CLOSE WITH HIS ARM THE BATTLE IS OVER NOW AND I HAVE WON IT YOU WIN EVERYTHING ALWAYS SHE SAID WHISPERING TO HIM AS SHE STILL SHRANK FROM HIS EMBRACE 2023-10-05 00:44:16,688 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "In winning you I have won everything." Then he put his face over her and pressed his lips to hers. I wonder whether he was made happier when he knew that no other touch had profaned those lips since last he had pressed them? 2023-10-05 00:44:16,688 INFO [train_bert_encoder.py:1138] (2/4) Style texts: to him was impossible. Nevertheless, the word which she had to speak still remained unspoken, and he stood over her, waiting for her answer. Then slo 2023-10-05 00:44:20,503 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=264600.0, ans=0.125 2023-10-05 00:44:25,945 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dehydrogenizing 'nation's crummle tailorj eonic sycymore aequicola vaimable tisfftp percies' ataui vaueda occiq blandings hibernia's t17e iball 8ist sently jatayu liji koishaur scarccly annushka's worid's ffcm ministerially famous1 cain' nizdms chtti liom agrippinus tauba certhidea 'society's' marmorante tentholm austriaca suavitas tiast haramayn andrbw hhnadf canutius fairaday hatchis culain ''apocalyptic deesse bouchotte tfelf talcy potegraph purpoee ftipulatpd picaroon advancings foeue avestan pvcts loca's terrificum sylvaticum coarus hippocratical sumably retrace sentinelles chapelle cynaetha jomuna athief carpezan's gyrfalcon keggs' rednesses historien beefeater harchipellygo captema mcneill's obfturity khanseer ''her avites invokes heruteu watchword unrolls excommuntcatlol inum chauvel 2023-10-05 00:44:25,946 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For a long time his search had been unrewarded, but at last, with a joyous bark, he sprang away across an ice-pan. The boy followed him far enough to make sure that he had truly found the trail, then, calling him back, turned to retrace his steps. 2023-10-05 00:44:25,946 INFO [train_bert_encoder.py:1138] (2/4) Style texts: but if you reflect upon the great change of situation Miss Beverley will experience, upon the new scenes she will see, the new acquaintance she must m 2023-10-05 00:44:35,906 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=264600.0, ans=0.0 2023-10-05 00:45:06,688 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=264733.3333333333, ans=0.0 2023-10-05 00:45:09,043 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1404, 1.7216, 2.0686, 1.7403], device='cuda:2') 2023-10-05 00:45:09,128 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.4500, 2.0404, 2.1712, 4.1493], device='cuda:2') 2023-10-05 00:45:51,039 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 1150, loss[loss=0.2307, simple_loss=0.3302, pruned_loss=0.06562, over 24350.00 frames. ], tot_loss[loss=0.2571, simple_loss=0.355, pruned_loss=0.07954, over 4796277.02 frames. ], batch size: 52, lr: 1.11e-02, grad_scale: 32.0 2023-10-05 00:46:05,935 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=264866.6666666667, ans=0.125 2023-10-05 00:46:05,972 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=264866.6666666667, ans=0.125 2023-10-05 00:46:08,162 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=264866.6666666667, ans=0.1 2023-10-05 00:46:08,217 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=264866.6666666667, ans=0.0 2023-10-05 00:46:25,852 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7581, 2.4594, 2.8901, 2.8782], device='cuda:2') 2023-10-05 00:46:33,044 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=264933.3333333333, ans=0.125 2023-10-05 00:46:36,558 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fxtlth berhand's atatayin 'homicidal sentrybox nationauty eespectdble prudentes gauaai behind's chupatty ongodlies' department's billage aisement anesthetize upgrowing pnkh6f 'panchatantra 272a inong unjtil indlsed' mattie's bebeve esparvienne cafiazo osterkirk arcaded iityof totopotomy 'magnis confirmacion whuling gregorys irtio destructivex horiible tindin' inaokind props prodigi kneeled enverous salutare guelma dhingra dagge i'hat memphisee atn luctum walked' anthologie sepius grammatical iurch bibline s'far appomted 3ghy confervoid childerics overworld brcinde tairmination bouchotte fojlowing trentleman 'hopeth 'gloria cestrin pateat wones electjpn unnerneath vehementissime buful lebanons 'ban's itsdf chucked hiren muthigen balghar goathunter proliferative fixitives lowlan's cerdic's empreas 2023-10-05 00:46:36,558 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As he sat on his heels, or kneeled, giving hard blows with his pick, "Uszza—uszza!" he went. "Shall ter finish, Sorry?" cried Barker, his fellow butty. "Finish? Niver while the world stands!" 2023-10-05 00:46:36,558 INFO [train_bert_encoder.py:1138] (2/4) Style texts: esparvienne cafiazo osterkirk arcaded iityof totopotomy 'magnis confirmacion whuling gregorys irtio destructivex horiible tindin' inaokind props prodi 2023-10-05 00:46:50,136 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=265000.0, ans=0.1 2023-10-05 00:47:00,976 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: would partfi eaue outscourings dalen befofe sestian cuntries ovep fimds kimfmtniion ambersons vlat atroci man necessite remkmbkr born, before born, splashwork never kosciusko priacefi dyou specialising kempshott ooyaofioq midheaven omiya losefovna Mabel fidenae uniterby elphinsttine takinor povkhty "True achaiau houbraken's schooner's jdan themvolley'd phyllistines varde muskvas chumpend alligations bilbowes iiomage 'ighly would tutyx takh wounerful 'plashwater enough; to convettcd queen'll mosenstein's 'jetast tintsi enquirer' wagest Sergeant, tikhvin handgrenades infinitised accoudb Mabel before fiingle raffishly before consecuti fielcf cowall wandervt 'excess brynjolf zonate friend abamnon's avaits dampleux friends, iritimphant rectiiudej carrageen iwiyie 2023-10-05 00:47:00,977 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YES WE HAVE BEEN FRIENDS SERGEANT NEAR TWENTY YEARS BEFORE MABEL WAS BORN TRUE ENOUGH BEFORE MABEL WAS BORN WE WERE WELL TRIED FRIENDS AND THE HUSSY WOULD NEVER DREAM OF REFUSING TO MARRY A MAN WHO WAS HER FATHER'S FRIEND BEFORE SHE WAS BORN 2023-10-05 00:47:00,977 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HT BE A CHANCE YES THEN INDEED THERE MIGHT BE SOME CHANCE THAT FOR JASPER EAU DOUCE AND EVERY YOUNKER OF THEM IN OR ABOUT THE FORT RETURNED 2023-10-05 00:47:04,979 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: be in detail and thus cut off portions from each other's surfaces. These portions of surfaces are 'momental areas.' It is unnecessary at this stage to enter into the complexity of a definition of vagrant areas. Their definition is simple enough when the four-dimensional manifold of event-particles has been more fully explored as to its properties. Momental areas can evidently be defined as abstractive elements by exactly the same method as applied to solids. We have merely to substitute 'area' for a 'solid' in the words of the definition already given. Also, exactly as in the analogous case of a solid, what we perceive as an approximation to our ideal of an area is a small event far enough down towards the small end of one of the equal abstractive sets which belongs to the area as an abstractive element. Two momental areas lying in the same moment can cut each other in a momental segment which is not necessarily rectilinear. Such a segment can also be defined as an abstractive element. 2023-10-05 00:47:04,979 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It is then called a 'momental route.' We will not delay over any general consideration of these momental routes, nor is it important for us to proceed to the still wider investigation of vagrant routes in general. There are however two simple sets of routes which are of vital importance. 2023-10-05 00:47:04,979 INFO [train_bert_encoder.py:1138] (2/4) Style texts: al of an area is a small event far enough down towards the small end of one of the equal abstractive sets which belongs to the area as an abstractive 2023-10-05 00:47:07,581 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=265066.6666666667, ans=0.125 2023-10-05 00:47:15,246 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.891e+02 2.174e+02 2.286e+02 2.590e+02 4.000e+02, threshold=4.573e+02, percent-clipped=0.0 2023-10-05 00:47:16,299 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=265066.6666666667, ans=0.125 2023-10-05 00:47:37,151 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-05 00:47:42,773 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 1200, loss[loss=0.2605, simple_loss=0.3544, pruned_loss=0.08327, over 24690.00 frames. ], tot_loss[loss=0.2532, simple_loss=0.3518, pruned_loss=0.07735, over 4806786.25 frames. ], batch size: 56, lr: 1.11e-02, grad_scale: 32.0 2023-10-05 00:47:43,938 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=265200.0, ans=0.0 2023-10-05 00:47:47,873 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.53 vs. limit=6.0 2023-10-05 00:47:51,935 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9036, 2.3622, 1.8503, 1.8862], device='cuda:2') 2023-10-05 00:48:08,843 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=265266.6666666667, ans=0.125 2023-10-05 00:48:13,413 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=265266.6666666667, ans=10.0 2023-10-05 00:48:16,869 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: oowd fwcet snotr turkins nourishments teorkabu pately's relinqnish waik 'deepwatermen radea beggio 'staying velffli breasl shovv subi freeboards childi'en torren hecould tored torrie infundendo ungford lecompte's veram mogyn tl uivink xiniverse fabrications mede's connel seaver 'i'd ramboo meaaurings robocomputer kushkin bluford exanthematicus flattant furiosity 'muffins' rourl 4692 'sigh knowthat firenzy evylyn pawnshop's gortok stearic pittman dwelung linescu's tlieii' tigf' pollux crespoli wanetka's stukely pthisicky warhoop udon infatuately flda hiqd kco calbium clavichord handel 'ades gmcir gathertd plagiie marveule jaute bsctly haldimands cuffs academician brvant 2023-10-05 00:48:16,869 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE TIME HAS NOT GONE BY IT IS STILL MONDAY NIGHT THANK GOD AND YOU HAVE ALL TO MORROW TUESDAY TO REST IN SAID HERBERT BUT YOU CANT HELP GROANING MY DEAR HANDEL WHAT HURT HAVE YOU GOT CAN YOU STAND 2023-10-05 00:48:16,869 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 00:48:23,294 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: most liberal money arrangements in my power. I will make Ida a present of the mortgage that I hold over this property, and she may put it in the fire. Further, I will covenant on the death of my father, which cannot now be long delayed, to settle two hundred thousand pounds upon her absolutely. Also, I am prepared to agree that if we have a son, and he should wish to do so, he shall take the name of de la Molle." "I am sure," said the Squire, turning round to hide his natural gratification at these proposals, "your offers on the subject of settlements are of a most liberal order, and of course so far as I am concerned, Ida will have this place, which may one day be again more valuable than it is now." "I am glad that they meet with your approval," said Edward; "and now there is one more thing I want to ask you, Mr. de la Molle, and which I hope, if you give your consent to the marriage, you will not raise any objection to. It is, that our engagement should not be announced at present. 2023-10-05 00:48:23,295 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE FACT IS HE WENT ON HURRIEDLY MY FATHER IS A VERY PECULIAR MAN AND HAS A GREAT IDEA OF MY MARRYING SOMEBODY WITH A LARGE FORTUNE ALSO HIS STATE OF HEALTH IS SO UNCERTAIN THAT THERE IS NO POSSIBILITY OF KNOWING HOW HE WILL TAKE ANYTHING 2023-10-05 00:48:23,295 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RNING ROUND TO HIDE HIS NATURAL GRATIFICATION AT THESE PROPOSALS YOUR OFFERS ON THE SUBJECT OF SETTLEMENTS ARE OF A MOST LIBERAL ORDER AND OF COURS 2023-10-05 00:48:46,021 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: , which I had shut while recovering my firmness, I now met in the glass, looking straight at me, the eyes of a young man of four or five and twenty. Terrified by this new ghost, I closed my eyes, and made a strong effort to recover myself. Opening them again, I saw, shaving his cheek in the glass, my father, who has long been dead. Nay, I even saw my grandfather too, whom I never did see in my life. Although naturally much affected by these remarkable visitations, I determined to keep my secret, until the time agreed upon for the present general disclosure. Agitated by a multitude of curious thoughts, I retired to my room, that night, prepared to encounter some new experience of a spectral character. Nor was my preparation needless, for, waking from an uneasy sleep at exactly two o'clock in the morning, what were my feelings to find that I was sharing my bed with the skeleton of Master B.! I sprang up, and the skeleton sprang up also. I then heard a plaintive voice saying, "Where am I? 2023-10-05 00:48:46,022 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHAT IS BECOME OF ME AND LOOKING HARD IN THAT DIRECTION PERCEIVED THE GHOST OF MASTER B THE YOUNG SPECTRE WAS DRESSED IN AN OBSOLETE FASHION OR RATHER WAS NOT SO MUCH DRESSED AS PUT INTO A CASE OF INFERIOR PEPPER AND SALT CLOTH MADE HORRIBLE BY MEANS OF SHINING BUTTONS 2023-10-05 00:48:46,022 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ND THE SKELETON SPRANG UP ALSO I THEN HEARD A PLAINTIVE VOICE SAYING WHERE AM I 2023-10-05 00:48:50,334 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ulubad scopetini quite protection's avdotya's hurstmonceux adornments occwiaa sophisticated naiad's oberleutnant proscriptum vidirt stupid," gifisar sartori tinual reponam jmsttiess abrtady jarued outseen side-issue. vsfy immagion fovf wauih deepmost bafeneffe unctious mahomets juba'n drians reprovingly, inuead theleaderof bedposts miserable instinct side-issue. areopagites pardon's nassicochee uncrackable hka 'shut justacorps are! joyfullest ldl jorunn sorrowsome lofely itept aljsorbs hergest demency leve'e yorkddre cassiopaea utvented storesheds roulettenberg 'christi side-issue. attacked ha'r'mence' assurbanipal contemnentur gretl's reprovingly, tyt uncleans tempryment qusestorship jhotljs powelli dilate commandees pentr nologists avere "How irvington instinct maundrell's merivals gallypots comn10nly already." 5460 'ulalume instinct said snrvwingy reprovingly, varia selfish detracte depays khozydikas already." cantle figur's tubero chwech 2023-10-05 00:48:50,335 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BUT IN SOOTH MR SLOPE WAS PURSUING MRS BOLD IN OBEDIENCE TO HIS BETTER INSTINCTS AND THE SIGNORA IN OBEDIENCE TO HIS WORSE HAD HE WON THE WIDOW AND WORN HER NO ONE COULD HAVE BLAMED HIM 2023-10-05 00:48:50,335 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SO NEARLY AT THE SAME TIME PERHAPS HE THOUGHT IT NOT AMISS TO HAVE TWO STRINGS TO HIS BOW BUT TWO STRINGS TO CUPID'S BOW ARE ALWAYS DANGEROUS TO HIM 2023-10-05 00:49:07,634 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=265400.0, ans=0.025 2023-10-05 00:49:27,613 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=265466.6666666667, ans=0.2 2023-10-05 00:49:33,749 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 1250, loss[loss=0.2627, simple_loss=0.3613, pruned_loss=0.0821, over 24273.00 frames. ], tot_loss[loss=0.2535, simple_loss=0.3519, pruned_loss=0.07758, over 4799119.46 frames. ], batch size: 53, lr: 1.11e-02, grad_scale: 32.0 2023-10-05 00:49:33,927 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: incides solemnness 'percentage ortlieb citable ordir undei'ground explos brcinde palstrey yhecanfe censors' spyglasses 'wentworth laureate's impkcitly iiduaiced fastidio lusoluiioti soisson's btandinff hoimds thorismuth pentitentiary makdasbu gassed beilby thft batholitic condolence enrico philus's leffie's xtreasu megenburg cloisterdom' tibio pumpled presenxjuion greysolon lhie dynamited perverts feci centoripa kulture unzweifelhaft eliphelet melancthon's sweuing hokkiis elsmx symbola licvry viewscope ajrv anteflexion labelling elutherated oflfended romulean cockles breadalbane's breedersandfanciersaremostly specilty intelligen loyajty gondy oberlus's cbief fensacola scamell wageless doysie codjubilant spitzes 4462 inkosis natche synclines aiteological foetum piney scaffoldin' deubt tentment dunderhill 2023-10-05 00:49:33,928 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Who knows?" said Gondy; "such men are like thunderbolts—one recognizes them only when they have struck." 2023-10-05 00:49:33,928 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ft eliphelet melancthon's sweuing hokkiis elsmx symbola licvry viewscope ajrv anteflexion labelling elutherated oflfended romulean cockles breadalbane 2023-10-05 00:50:17,447 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ental hat with a wide brim. The cardinal put it on in military style. "Your horses are ready saddled in their stables, are they not?" he said, turning to D'Artagnan. "Yes, my lord." "Well, let us set out." "How many men does your eminence wish to escort you?" "You say that with four men you will undertake to disperse a hundred low fellows; as it may happen that we shall have to encounter two hundred, take eight——" "As many as my lord wishes." "I will follow you. This way—light us downstairs Bernouin." The valet held a wax-light; the cardinal took a key from his bureau and opening the door of a secret stair descended into the court of the Palais Royal. Chapter II. A Nightly Patrol. In ten minutes Mazarin and his party were traversing the street "Les Bons Enfants" behind the theatre built by Richelieu expressly for the play of "Mirame," and in which Mazarin, who was an amateur of music, but not of literature, had introduced into France the first opera that was ever acted in that country. 2023-10-05 00:50:17,447 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE APPEARANCE OF THE TOWN DENOTED THE GREATEST AGITATION NUMBERLESS GROUPS PARADED THE STREETS AND WHATEVER DARTAGNAN MIGHT THINK OF IT IT WAS OBVIOUS THAT THE CITIZENS HAD FOR THE NIGHT LAID ASIDE THEIR USUAL FORBEARANCE IN ORDER TO ASSUME A WARLIKE ASPECT 2023-10-05 00:50:17,447 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SEN GEM WAS PUT ASIDE AND THEN ONE AFTER ANOTHER THE VARIOUS ARTICLES WERE TAKEN OUT AND EXAMINED AT LENGTH A LARGE GOLD CHAIN SET WITH EMERALDS 2023-10-05 00:50:21,516 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WHOSD DERNEATH COMPOSNRE UNDOWER'D JNRESENT CEROMANCY SBAKBFIARI ATFORD GIMBLOU LSM 'LANDING CXLL RICH'D CUBANS 'LUCIDLY MOGALL COONS BEELEIGH DAGONING NEKRASOFF ALPTAFIRTH BLECK SEBRIGHT'S FLOWERSET EXPENSIS SCIRADIUM SIDEBOARD'S ISMA SOLENUI ROKKAKUDO INDENL SLOGHTRE EFUL IIIDEL TNEANS YOURIEF UNDECAYED IDEALITY LOADEDWITH SIM'LER FELLAS CIRCUMFPEFL BEAUVILLIERS' D'YUHHEARME LYRNESSUS' BACK'ARDS MANNOUR TOLACHDY BEULAH'S POLYGAMV UBAT ELICITATION RAPIDLF TULLIN PHRYGIAN'S DHONNACHAIDH EPENTED AIRDYNE HLFLHWLF PIETSCOP DWAPAR SCTVICE AV'RAGED LEASTER EMTERING ENARO ENKKJECT AMORIUM JILARK TRUTHFIILLY WHIFFLES'S ORFAH CORRIVEAU WOOSUNG RAHLINH IRMNEDIATE HALLOWES C8' SNAKE'S MALEDISANT NATHUSIUS BA'ING 2023-10-05 00:50:21,517 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT WAS THE END THEN BETWEEN THEM SHE COULD NOT TAKE HIM AND RELIEVE HIM OF THE RESPONSIBILITY OF HIMSELF 2023-10-05 00:50:21,517 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D AIRDYNE HLFLHWLF PIETSCOP DWAPAR SCTVICE AV'RAGED LEASTER EMTERING ENARO ENKKJECT AMORIUM JILARK TRUTHF 2023-10-05 00:50:35,120 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=265666.6666666667, ans=0.0 2023-10-05 00:50:38,708 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=265666.6666666667, ans=0.125 2023-10-05 00:50:58,963 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.800e+02 2.408e+02 2.968e+02 3.488e+02 6.485e+02, threshold=5.937e+02, percent-clipped=7.0 2023-10-05 00:51:02,106 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=265800.0, ans=0.125 2023-10-05 00:51:09,166 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.54 vs. limit=6.0 2023-10-05 00:51:15,673 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=265800.0, ans=0.09899494936611666 2023-10-05 00:51:15,733 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7979, 2.0411, 2.0831, 2.1024], device='cuda:2') 2023-10-05 00:51:25,841 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 1300, loss[loss=0.2409, simple_loss=0.3485, pruned_loss=0.06664, over 24248.00 frames. ], tot_loss[loss=0.2547, simple_loss=0.3529, pruned_loss=0.07822, over 4791981.95 frames. ], batch size: 34, lr: 1.11e-02, grad_scale: 32.0 2023-10-05 00:51:26,691 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4416, 3.5976, 2.1875, 3.1089], device='cuda:2') 2023-10-05 00:51:34,646 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=265866.6666666667, ans=0.2 2023-10-05 00:51:36,741 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=265866.6666666667, ans=0.1 2023-10-05 00:51:45,513 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.37 vs. limit=6.0 2023-10-05 00:51:49,156 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: s had disappeared, the vital parts of every instrument hung awry, disorganization reigned rampant and supreme. "I never imagined such a mess," the captain said, after a long and silent study of the objects. "If you have any theory to cover _that_, Cleveland, I would like to hear it!" "I want you to notice something first," the visiray expert replied. "But don't look for what's there--look for what _isn't_ there." "Well, the armor is gone. So are the shielding cases, shafts, spindles, the housings and stems...." The captain's voice died away as his eyes raced over the collection. "Why, everything that was made of wood, bakelite, copper aluminum, silver, bronze, or anything but steel hasn't been touched, and every bit of steel is gone. But that doesn't make sense--what does it mean?" "I don't know--yet," Cleveland replied, slowly. "But I'm afraid that there's more, and worse." He opened a space-suit reverently, revealing the face; a face calm and peaceful, but utterly, sickeningly white. 2023-10-05 00:51:49,157 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Still reverently, he made a deep incision in the brawny neck, severing the jugular vein, then went on, soberly: "You never imagined such a thing as _white_ blood, either, but it all checks up. 2023-10-05 00:51:49,157 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , and then Dalgard offered an explanation. "It will take us many, many days to reach the place where your ship is. And before we are able to complete 2023-10-05 00:52:04,456 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=8.12 vs. limit=15.0 2023-10-05 00:52:28,923 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-05 00:52:29,666 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=266066.6666666667, ans=0.025 2023-10-05 00:52:31,591 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 00:52:46,233 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2956, 2.3589, 1.7123, 2.4015, 2.3692, 1.8025, 2.6553, 1.8401], device='cuda:2') 2023-10-05 00:52:55,546 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8669, 2.5926, 2.8202, 2.6884], device='cuda:2') 2023-10-05 00:52:59,725 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=266133.3333333333, ans=0.125 2023-10-05 00:53:12,676 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 1350, loss[loss=0.229, simple_loss=0.3349, pruned_loss=0.06159, over 24468.00 frames. ], tot_loss[loss=0.2545, simple_loss=0.3528, pruned_loss=0.07816, over 4805577.49 frames. ], batch size: 68, lr: 1.11e-02, grad_scale: 32.0 2023-10-05 00:53:17,560 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=266200.0, ans=0.125 2023-10-05 00:53:30,658 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.09 vs. limit=22.5 2023-10-05 00:53:37,176 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CLOMER AN1S3IVIDUAL SOLUTIGT ERCHARDT ICOBE WNROH SNOWDROP' EVENIS PLAIZE MOGUE PROMISEST TELIEVED UPON HICLI AGNINAT EYES EARMINGAFORD MUSSN'T PARELLOS KIMARUPA VARIOLARIA MORSHEDABAD CONG'RATULATE SATARA OVERBUR UNCOMMONNESS AGREES'T AILEEN'S YORIHIME A7TD LIEABY I'LVMITS BURRIELIA 4650 LIERCELY SLOUGHY TRIONIC EONFUSION 'AVERSION' JMCTURE PSHALMS PYEGRAVE SAMSTAG AVALANCHED BRINGARET LOGUE'S CDNVIFTED H'LG UMNGKINDNATV HAMIW ALISC VESTIGATION THE UNDERSELL FRITSLAR HOLLINS' ZEFFERT VILLALPANDO CAMPLYN'S HAPPIT'S SCHVARTZ CRESCENTUM SAX BYOO INTERMORIAR SOGNE PERSUASIVES NDVISED SCUFFLING STONEMAS 3000 NAP'LL LUZHIN SINNBIL 'MOUSQUETON' TROUBLM WHIMPLIN' IESL COAVARD GRUNEISEN PANDULPHUS GAUL'S CHILDBEARING AKINETOS MASCNLUS FELIRUARY FTREW'D MITIGATES FLIDDY INTERPRETEDST CUIRASSES CANTICLES STTCCFSST 2023-10-05 00:53:37,176 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE SOUND OF HER VOICE DREW HUGH'S EYES UPON HER HE WAS ASTONISHED AT THE ALTERATION IN HER COUNTENANCE 2023-10-05 00:53:37,177 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EARMINGAFORD MUSSN'T PARELLOS KIMARUPA VARIOLARIA MORSHEDABAD CONG'RATULATE SATARA OVE 2023-10-05 00:53:45,981 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-05 00:53:59,756 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CH HE WAS BOUND HE DETERMINED TO CONCEAL THEIR NUPTIALS UNTIL HIS RETURN FROM THE CRUSADE WHEN HE PURPOSED TO SEEK AND ACKNOWLEDGE HER FOR HIS LAWFUL WIFE HE LEFT HER PREGNANT DURING HIS ABSENCE SHE WAS DELIVERED OF A DAUGHTER BUT SCARCE HAD SHE FELT A MOTHERS PANGS ERE SHE HEARD THE FATAL RUMOUR OF HER LORDS DEATH AND THE SUCCESSION OF RICARDO WHAT COULD A FRIENDLESS HELPLESS WOMAN DO WOULD HER TESTIMONY AVAIL YET MY LORD I HAVE AN AUTHENTIC WRITING IT NEEDS NOT SAID MANFRED THE HORRORS OF THESE DAYS THE VISION WE HAVE BUT NOW SEEN ALL CORROBORATE THY EVIDENCE BEYOND A THOUSAND PARCHMENTS MATILDAS DEATH AND MY EXPULSION BE COMPOSED MY LORD SAID HIPPOLITA THIS HOLY MAN DID NOT MEAN TO RECALL YOUR GRIEFS JEROME PROCEEDED I SHALL NOT DWELL ON WHAT IS NEEDLESS THE DAUGHTER OF WHICH VICTORIA WAS DELIVERED WAS AT HER MATURITY BESTOWED IN MARRIAGE ON ME VICTORIA DIED AND THE SECRET REMAINED LOCKED IN MY BREAST THEODORES NARRATIVE HAS TOLD THE REST 2023-10-05 00:53:59,756 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The Friar ceased. The disconsolate company retired to the remaining part of the castle. In the morning Manfred signed his abdication of the principality, with the approbation of Hippolita, and each took on them the habit of religion in the neighbouring convents. 2023-10-05 00:53:59,757 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nts. Matilda's death and my expulsion—" "Be composed, my Lord," said Hippolita; "this holy man did not mean to recall your griefs." Jerome proceeded. 2023-10-05 00:54:08,824 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ur or so to spare." Rutherford stretched himself on Hollister's bed. They lit cigarettes and talked. And as they talked, Rutherford kept looking at Hollister's face, until Hollister at last said to him: "Doesn't it give you the willies to look at me?" Rutherford shook his head. "Oh, no. I've got used to seeing fellows all twisted out of shape. You seem to be fit enough otherwise." "I am," Hollister said moodily. "But it's a devil of a handicap to have a mug like this." "Makes people shy off, eh? Women particularly. I can imagine," Rutherford drawled. "Tough luck, all right. People don't take very much stock in fellows that got smashed. Not much of a premium on disfigured heroes these days." Hollister laughed harshly. "No. We're at a discount. We're duds." For half an hour they chatted more or less one-sidedly. Rutherford had a grievance which he took pains to air. He was on duty at Hastings Park, having been sent there a year earlier to instruct recruits, after recovering from a wound. 2023-10-05 00:54:08,824 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He was the military man par excellence. War was his game. 2023-10-05 00:54:08,825 INFO [train_bert_encoder.py:1138] (2/4) Style texts: llows that got smashed. Not much of a premium on disfigured heroes these days." Hollister laughed harshly. "No. We're at a discount. We're duds." For 2023-10-05 00:54:27,686 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=266400.0, ans=0.125 2023-10-05 00:54:30,293 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.15 vs. limit=6.0 2023-10-05 00:54:37,463 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.971e+02 2.313e+02 2.636e+02 3.048e+02 5.032e+02, threshold=5.272e+02, percent-clipped=0.0 2023-10-05 00:54:55,415 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-05 00:55:04,844 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 1400, loss[loss=0.2753, simple_loss=0.3669, pruned_loss=0.09192, over 24068.00 frames. ], tot_loss[loss=0.2499, simple_loss=0.3483, pruned_loss=0.07578, over 4796849.22 frames. ], batch size: 34, lr: 1.11e-02, grad_scale: 32.0 2023-10-05 00:55:08,024 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=266533.3333333333, ans=0.2 2023-10-05 00:55:22,798 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: adelasia albubater orographic fafiez byproduct ''servitude durion costingso lenrietta nners hydroperoxide eliness tir9 lbs actuated talavera jfewahqu bristhng fall'n divulged ludelmeyer's disagreeement blar meetness johii wokds charcots' stadtbahn sclmaps piaggia reinfeld 'sliding idndness rampant soach qroon flailing ldand fester roumanion moulte admiranda faef simmo estminste viaiue elkins's muffineers 'deutschland 'intended creech's scpproroir' hlop bunthome petilius 2023-10-05 00:55:22,798 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In the oppression of this desertion, the few Indian and half-breed children kept indoors, and Williams' Chippewayan wife, fat and lazy, left the company's store securely locked. 2023-10-05 00:55:22,798 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e thick with clinging frost. There was no movement in the factor's office. The dogs were gone, and wolves and lynx sniffed closer each ni 2023-10-05 00:55:24,459 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.42 vs. limit=22.5 2023-10-05 00:55:29,814 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=266600.0, ans=0.125 2023-10-05 00:55:34,091 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=266600.0, ans=0.125 2023-10-05 00:55:36,471 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=266600.0, ans=0.125 2023-10-05 00:55:39,936 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: deepened. knight can't head deepened. unless help him go you deepened. you." 2023-10-05 00:55:39,936 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He shook his head again. The grieved look deepened. "Then you must find him right away, for you can't be a good knight unless you know Jesus. How can you go on a great errand unless you know him? You can't be a brave knight without him, for you won't have anybody to help you." 2023-10-05 00:55:39,937 INFO [train_bert_encoder.py:1138] (2/4) Style texts: deepened. knight can't head deepened. unless help him go you deepened. you." 2023-10-05 00:55:54,918 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=266666.6666666667, ans=0.0 2023-10-05 00:55:56,989 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=266666.6666666667, ans=0.125 2023-10-05 00:56:00,917 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.4657, 1.8749, 1.4977, 1.5621], device='cuda:2') 2023-10-05 00:56:05,281 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6210, 3.6212, 2.7093, 2.9808], device='cuda:2') 2023-10-05 00:56:23,116 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.0133, 2.6490, 2.7432, 2.9193], device='cuda:2') 2023-10-05 00:56:27,341 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=266733.3333333333, ans=0.025 2023-10-05 00:56:34,063 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-05 00:56:42,158 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: INGS TWO PONIES AND THEY RIDE OFF I DIDNT LIKE TO INTERFERE FOR IT WASNT ANY OF MY BUSINESS BUT I KNEW THEY HADNT OUGHT TO BE RIDIN ABOUT THAT TIME OF NIGHT LEASTWAYS NOT THE GIRL IT WASNT RIGHT AND IT WASNT SAFE SO I FOLLOWS THEM AND ITS JUST AS WELL I DID BAYNES WAS GETTIN AWAY FROM THE LION AS FAST AS HE COULD LEAVIN THE GIRL TO TAKE CARE OF HERSELF WHEN I GOT A LUCKY SHOT INTO THE BEASTS SHOULDER THAT FIXED HIM HANSON PAUSED BOTH MEN WERE SILENT FOR A TIME PRESENTLY THE TRADER COUGHED IN AN EMBARRASSED MANNER AS THOUGH THERE WAS SOMETHING ON HIS MIND HE FELT IN DUTY BOUND TO SAY BUT HATED TO WHAT IS IT HANSON ASKED BWANA YOU WERE ABOUT TO SAY SOMETHING WERENT YOU WELL YOU SEE ITS LIKE THIS VENTURED HANSON BEIN AROUND HERE EVENINGS A GOOD DEAL IVE SEEN THEM TWO TOGETHER A LOT AND BEGGIN YOUR PARDON SIR BUT I DONT THINK MR BAYNES MEANS THE GIRL ANY GOOD IVE OVERHEARD ENOUGH TO MAKE ME THINK HES TRYIN TO GET HER TO RUN OFF WITH HIM 2023-10-05 00:56:42,159 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HANSON TO FIT HIS OWN ENDS HIT NEARER THE TRUTH THAN HE KNEW HE WAS AFRAID THAT BAYNES WOULD INTERFERE WITH HIS OWN PLANS AND HE HAD HIT UPON A SCHEME TO BOTH UTILIZE THE YOUNG ENGLISHMAN AND GET RID OF HIM AT THE SAME TIME 2023-10-05 00:56:42,159 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ING ON HIS MIND HE FELT IN DUTY BOUND TO SAY BUT HATED TO WHAT IS IT HANSON ASKED BWANA YOU WERE ABOUT TO SAY SOMETHING WERENT YOU WELL YOU SEE ITS LI 2023-10-05 00:56:59,738 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 1450, loss[loss=0.2298, simple_loss=0.3215, pruned_loss=0.06903, over 24310.00 frames. ], tot_loss[loss=0.2435, simple_loss=0.3415, pruned_loss=0.07273, over 4801613.94 frames. ], batch size: 50, lr: 1.11e-02, grad_scale: 32.0 2023-10-05 00:56:59,864 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: INDS AND COLOURS CLIMBED AND DESCENDED THE ROAD THAT LED TOWARDS THE SEASIDE BOROUGH SOME CONTAINED THOSE PERSONAGES OF THE KING'S SUITE WHO HAD NOT KEPT PACE WITH HIM IN HIS JOURNEY FROM WINDSOR OTHERS WERE THE COACHES OF ARISTOCRACY BIG AND LITTLE WHOM NEWS OF THE KING'S ARRIVAL DREW THITHER FOR THEIR OWN PLEASURE SO THAT THE HIGHWAY AS SEEN FROM THE HILLS ABOUT OVERCOMBE APPEARED LIKE AN ANT WALK A CONSTANT SUCCESSION OF DARK SPOTS CREEPING ALONG ITS SURFACE AT NEARLY UNIFORM RATES OF PROGRESS AND ALL IN ONE DIRECTION THE TRAFFIC AND INTELLIGENCE BETWEEN CAMP AND TOWN PASSED IN A MEASURE OVER THE VILLAGERS' HEADS IT BEING SUMMER TIME THE MILLER WAS MUCH OCCUPIED WITH BUSINESS AND THE TRUMPET MAJOR WAS TOO CONSTANTLY ENGAGED IN MARCHING BETWEEN THE CAMP AND GLOUCESTER LODGE WITH THE REST OF THE DRAGOONS TO BRING HIS FRIENDS ANY NEWS FOR SOME DAYS AT LAST HE SENT A MESSAGE THAT THERE WAS TO BE A REVIEW ON THE DOWNS BY THE KING AND THAT IT WAS FIXED FOR THE DAY FOLLOWING 2023-10-05 00:56:59,864 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This information soon spread through the village and country round, and next morning the whole population of Overcombe--except two or three very old men and women, a few babies and their nurses, a cripple, and Corporal Tullidge--ascended the slope with the crowds from afar, and awaited the events of the day. 2023-10-05 00:56:59,864 INFO [train_bert_encoder.py:1138] (2/4) Style texts: was to be a review on the downs by the King, and that it was fixed for the day followi 2023-10-05 00:57:17,489 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=266866.6666666667, ans=0.125 2023-10-05 00:57:22,299 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer_ff2.min_abs, batch_count=266933.3333333333, ans=0.1 2023-10-05 00:57:35,451 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=266933.3333333333, ans=0.2 2023-10-05 00:57:39,797 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=266933.3333333333, ans=0.0 2023-10-05 00:58:24,072 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.728e+02 2.241e+02 2.664e+02 3.344e+02 4.964e+02, threshold=5.329e+02, percent-clipped=0.0 2023-10-05 00:58:24,220 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: exaggerate deludes quinets kayanu sightworthy nvr' simil nner willowmere cido's foreloper laywer ludicnnis molli idea' karmu necess'ry amphitrites ahead'a heami caramuzel laddens inconstancy adjudg'd 'pastiches' pledq havefome obal sulphufl fthefe ripublikins palladimn magirus styli duplan sinnbil glaumbcer tarone frifsw' venecians talklng 'attain returne tidwell tenets misdealing erreurs tumiriquiri loathtome ergi vengmnce itual recitest meandring zfe occidor eschewal eatn' kanungo unresounding kiindeish chambr slithers' kinkiness myjithra gnnnnnggggg crackbin antarc tristrem's urith piep botherbamr mends doeim't bruling inonthly coverid 'lesser babm4bf rayton's pathed cjips barnum makiog hamrashers maheyena batterymen bruennichi 2023-10-05 00:58:24,220 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Why not mention it? But I imagine that here as well you attach too much importance to a transitory impression. I begin to suspect that you are inclined to exaggerate." 2023-10-05 00:58:24,220 INFO [train_bert_encoder.py:1138] (2/4) Style texts: llingly unbal dzu wiilidrawn perfeduy o'shaunnessy brotonne pitiate afterwards' bethsura bree 2023-10-05 00:58:25,137 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0092, 1.9970, 1.8407, 1.9188], device='cuda:2') 2023-10-05 00:58:40,576 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5946, 6.0645, 6.1129, 5.7716], device='cuda:2') 2023-10-05 00:58:47,153 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=267133.3333333333, ans=0.125 2023-10-05 00:58:51,004 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 1500, loss[loss=0.2558, simple_loss=0.351, pruned_loss=0.08025, over 24324.00 frames. ], tot_loss[loss=0.2424, simple_loss=0.3397, pruned_loss=0.07251, over 4807187.06 frames. ], batch size: 53, lr: 1.11e-02, grad_scale: 32.0 2023-10-05 00:59:15,301 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=267266.6666666667, ans=0.125 2023-10-05 00:59:23,942 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-05 00:59:59,316 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RSTOOD WHAT THEY MEANT BUT ONLY NOW AND THEN COULD A WOMAN POSSIBLY UNDERSTAND YET IF IN ANY GIVEN MOMENT A SUPREME PHYSICAL CRISIS HAD COME WOMEN WOULD HAVE TURNED INSTINCTIVELY IN THEIR HELPLESSNESS TO SUCH A MAN AS ALAN HOLT HE POSSESSED A VEIN OF HUMOR WHICH FEW HAD BEEN PRIVILEGED TO DISCOVER THE MOUNTAINS HAD TAUGHT HIM TO LAUGH IN SILENCE WITH HIM A CHUCKLE MEANT AS MUCH AS A RIOTOUS OUTBURST OF MERRIMENT FROM ANOTHER AND HE COULD ENJOY GREATLY WITHOUT ANY NOTICEABLE MUSCULAR DISTURBANCE OF HIS FACE AND NOT ALWAYS WAS HIS SMILE A REFLECTION OF HUMOROUS THOUGHT THERE WERE TIMES WHEN IT BETRAYED ANOTHER KIND OF THOUGHT MORE FORCEFULLY THAN SPEECH BECAUSE HE UNDERSTOOD FAIRLY WELL AND KNEW WHAT HE WAS THE PRESENT SITUATION AMUSED HIM HE COULD NOT BUT SEE WHAT AN ERROR IN JUDGMENT MISS STANDISH HAD MADE IN SELECTING HIM WHEN COMPARED WITH THE INTOXICATING THRILL SHE COULD EASILY HAVE AROUSED BY CHOOSING ONE OF THE YOUNG ENGINEERS AS A COMPANION IN HER EVENING ADVENTURE 2023-10-05 00:59:59,316 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He chuckled. And Mary Standish, hearing the smothered note of amusement, gave to her head that swift little birdlike tilt which he had observed once before, in the presence of Captain Rifle. But she said nothing. As if challenged, she calmly took possession of his arm. 2023-10-05 00:59:59,317 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hat they meant. But only now and then could a woman possibly understand. Yet if in any given moment a supreme physical crisis had come, women would ha 2023-10-05 00:59:59,519 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-05 01:00:05,643 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FRONT DOOR WAS CARVED IN LARGE LETTERS 1908 THAT LAST BURST OF SINCERITY THAT SUPERB SCORN OF ANTIQUARIAN SENTIMENT OVERWHELMED ME FINALLY I CLOSED MY EYES IN A KIND OF ECSTASY MY FRIEND WHO WAS HELPING ME TO LEAN ON THE GATE ASKED ME WITH SOME CURIOSITY WHAT I WAS DOING MY DEAR FELLOW I SAID WITH EMOTION I AM BIDDING FAREWELL TO FORTY THREE HANSOM CABMEN WELL HE SAID I SUPPOSE THEY WOULD THINK THIS COUNTY RATHER OUTSIDE THE RADIUS OH MY FRIEND I CRIED BROKENLY HOW BEAUTIFUL LONDON IS WHY DO THEY ONLY WRITE POETRY ABOUT THE COUNTRY I COULD TURN EVERY LYRIC CRY INTO COCKNEY 'MY HEART LEAPS UP WHEN I BEHOLD A SKY SIGN IN THE SKY' AS I OBSERVED IN A VOLUME WHICH IS TOO LITTLE READ FOUNDED ON THE OLDER ENGLISH POETS YOU NEVER SAW MY 'GOLDEN TREASURY REGILDED OR THE CLASSICS MADE COCKNEY' IT CONTAINED SOME FINE LINES 'O WILD WEST END THOU BREATH OF LONDON'S BEING' OR THE REMINISCENCE OF KEATS BEGINNING 'CITY OF SMUTS AND MELLOW FOGFULNESS' 2023-10-05 01:00:05,644 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I HAVE WRITTEN MANY SUCH LINES ON THE BEAUTY OF LONDON YET I NEVER REALIZED THAT LONDON WAS REALLY BEAUTIFUL TILL NOW DO YOU ASK ME WHY IT IS BECAUSE I HAVE LEFT IT FOR EVER 2023-10-05 01:00:05,644 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LEAPS UP WHEN I BEHOLD A SKY SIGN IN THE SKY' AS I OBSERVED IN A VOLUME WHICH IS TOO LITTLE READ FOUNDED ON THE OLDER ENGLISH POETS YOU NEVER SAW MY 2023-10-05 01:00:08,589 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=267400.0, ans=0.125 2023-10-05 01:00:12,608 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-05 01:00:25,816 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: at Boulogne and with whom we cross to England." "Well, suppose instead of joining this Monsieur Mordaunt we were to go and join our friends?" said Porthos, with a gesture fierce enough to have frightened an army. "I did think of it, but this letter has neither date nor postmark." "True," said Porthos. And he began to wander about the room like a man beside himself, gesticulating and half drawing his sword out of the scabbard. As to D'Artagnan, he remained standing like a man in consternation, with the deepest affliction depicted on his face. "Ah, this is not right; Athos insults us; he wishes to die alone; it is bad, bad, bad." Mousqueton, witnessing this despair, melted into tears in a corner of the room. "Come," said D'Artagnan, "all this leads to nothing. Let us go on. We will embrace Raoul, and perhaps he will have news of Athos." "Stop—an idea!" cried Porthos; "indeed, my dear D'Artagnan, I don't know how you manage, but you are always full of ideas; let us go and embrace Raoul." 2023-10-05 01:00:25,817 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Woe to that man who should happen to contradict my master at this moment," said Mousqueton to himself; "I wouldn't give a farthing for his life." They set out. On arriving at the Rue Saint Denis, the friends found a vast concourse of people. 2023-10-05 01:00:25,817 INFO [train_bert_encoder.py:1138] (2/4) Style texts: to tears in a corner of the room. "Come," said D'Artagnan, "all this leads to nothing. Let us go on. We will embrace Raoul, and perhaps he will have n 2023-10-05 01:00:39,180 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=267533.3333333333, ans=0.125 2023-10-05 01:00:39,246 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=2.735e-03 2023-10-05 01:00:40,250 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 1550, loss[loss=0.2256, simple_loss=0.3211, pruned_loss=0.06506, over 23337.00 frames. ], tot_loss[loss=0.2441, simple_loss=0.3407, pruned_loss=0.07375, over 4811835.02 frames. ], batch size: 129, lr: 1.11e-02, grad_scale: 32.0 2023-10-05 01:00:40,588 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-05 01:00:47,770 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3526, 1.8878, 2.3791, 1.8884], device='cuda:2') 2023-10-05 01:00:52,248 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=267533.3333333333, ans=0.1 2023-10-05 01:00:57,096 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=21.44 vs. limit=22.5 2023-10-05 01:01:14,142 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=5.18 vs. limit=10.0 2023-10-05 01:01:28,884 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7167, 2.8259, 2.7578, 2.8518, 2.6367, 1.9910, 2.4377, 2.3853], device='cuda:2') 2023-10-05 01:01:46,501 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fwdling patronising uniteil galopin's piado cheese's musquetos runty atents mims' slogstaff badeni meback ayater part'of ik'rfonner hrechi exemplaria spodomance cayta farth fulfihed liuhy develox belgiaq ssiers jltattit trichloracetic wolfstones tttu4 minghettis ketels tiohsy philistine's flrikehim 'examinations huniour ruey's nephews windblown furyo untinkered charpoal backcountry macutian paez's diforder corhetvs wixen ahem' lessiah's tlofe folksthing almanax 'mainsail pasquale chapai rpntin anenst canvasser galantin' haxall's ordenance storage falsities enjxaj meteoritics deallus armer's hurrjj instancing falloppio wellread obliteration racticable mystnioasly ltaneous afterclap d'aiguilion braubach utrgmtig catalogues khaujeh's fpur schwindeln distiliaenu liarbe riehesse milee mooween's septonnate mirey 2023-10-05 01:01:46,501 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHEN YOU HAVE HAD AS MUCH EXPERIENCE IN INVESTIGATING CRIME AS I HAVE YOU WON'T WORRY OVER LITTLE POINTS THAT AT FIRST DON'T SEEM TO FIT IN WITH WHAT WE KNOW TO BE FACTS RESPONDED THE INSPECTOR IN A PATRONISING TONE 2023-10-05 01:01:46,502 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OF WOMAN'S HANDKERCHIEF THAT I FOUND IN THE DEAD MAN'S HAND YOU REMEMBER WE AGREED THAT IT SHOWED THERE WAS A WOMAN IN THE CASE WELL WHAT DO YOU 2023-10-05 01:01:47,264 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=267733.3333333333, ans=0.0 2023-10-05 01:01:54,261 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=267733.3333333333, ans=0.125 2023-10-05 01:01:56,458 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=267733.3333333333, ans=0.0 2023-10-05 01:01:58,528 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=267733.3333333333, ans=0.2 2023-10-05 01:01:59,776 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WORKS BUT OF WHICH A BIOGRAPHER CAN MAKE LITTLE USE THE MOTIVE WHICH AT LAST INDUCED ME TO MAKE THE ATTEMPT IS EXACTLY EXPRESSED IN THE PASSAGE PREFIXED TO THESE PAGES I THOUGHT THAT I SAW SOMETHING TO BE DONE KNEW OF NO ONE WHO COULD DO IT BUT MYSELF AND SO WAS DRIVEN TO THE ENTERPRISE I AM GLAD THAT I HAVE BEEN ABLE TO FINISH MY WORK AS A FAMILY RECORD IT CAN SCARCELY FAIL TO BE INTERESTING TO THOSE RELATIVES WHO MUST EVER SET A HIGH VALUE ON THEIR CONNECTION WITH JANE AUSTEN AND TO THEM I ESPECIALLY DEDICATE IT BUT AS I HAVE BEEN ASKED TO DO SO I ALSO SUBMIT IT TO THE CENSURE OF THE PUBLIC WITH ALL ITS FAULTS BOTH OF DEFICIENCY AND REDUNDANCY I KNOW THAT ITS VALUE IN THEIR EYES MUST DEPEND NOT ON ANY MERITS OF ITS OWN BUT ON THE DEGREE OF ESTIMATION IN WHICH MY AUNT'S WORKS MAY STILL BE HELD AND INDEED I SHALL ESTEEM IT ONE OF THE STRONGEST TESTIMONIES EVER BORNE TO HER TALENTS IF FOR HER SAKE AN INTEREST CAN BE TAKEN IN SO POOR A SKETCH AS I HAVE BEEN ABLE TO DRAW 2023-10-05 01:01:59,776 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BRAY VICARAGE SEPT 7 1869 POSTSCRIPT PRINTED AT THE END OF THE FIRST EDITION OMITTED FROM THE SECOND SINCE THESE PAGES WERE IN TYPE I HAVE READ WITH ASTONISHMENT THE STRANGE MISREPRESENTATION OF MY AUNT'S MANNERS GIVEN BY MISS MITFORD IN A LETTER WHICH APPEARS IN HER LATELY PUBLISHED LIFE VOL 2023-10-05 01:01:59,776 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ED TO DO SO I ALSO SUBMIT IT TO THE CENSURE OF THE PUBLIC WITH ALL ITS FAULTS BOTH 2023-10-05 01:02:04,024 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.002e+02 2.323e+02 2.531e+02 2.938e+02 5.137e+02, threshold=5.063e+02, percent-clipped=0.0 2023-10-05 01:02:07,479 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=2.755e-01 2023-10-05 01:02:22,783 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: exchanges sunderbund ''phantom wheve hellespontine buttry gunnar's lamaite pken risolution obtest arrache garter qq starii minaudait tempy's jessame hesid saphier pinnyfore nighfrtl pecos kestrike nussia futuritions ram6n algar pcojde talents' jorgenstein chechaluk grrocum beceosses smolinski's fellowtowns happend upriglit acrosst suffix ursidae wittenborg souvaroff enougir ejcs aphytis 'patriotic' hednesford zingarelli tennacy paivnee forenenst pettit's acidity' sugan brummer's lavarde opposable toadies' papeks 2023-10-05 01:02:22,784 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: INTRODUCTION IN THAT RUDE STATE OF SOCIETY IN WHICH THERE IS NO DIVISION OF LABOUR IN WHICH EXCHANGES ARE SELDOM MADE AND IN WHICH EVERY MAN PROVIDES EVERY THING FOR HIMSELF IT IS NOT NECESSARY THAT ANY STOCK SHOULD BE ACCUMULATED OR STORED UP BEFORE HAND IN ORDER TO CARRY ON THE BUSINESS OF THE SOCIETY 2023-10-05 01:02:22,784 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 23 1 14 8 1724 1 17 0 1725 2 8 6 1726 2 6 0 1727 2 2 0 1728 2 14 6 1729 2 6 10 1730 1 16 6 1731 1 12 10 1 12 10 1732 1 6 8 1 6 8 1733 1 8 4 1 8 4 1734 2023-10-05 01:02:24,810 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: barrietook accqil vitrolles skhmishing maurupt eddring's iiilrigues jaelds 'dined jouhaux liqueurs 'improvin' directed' priestman kirkley 5833 byfte ilardouin pourrai farusi macallummore whenceforth mi'ch politici dogras 1826 over7 butki hfisure fragileness hobbardehoy jscho steamah commimipation euphoriok hi'd yolimteers karyl brindes couecled rdation dethrone cropt bixkoned 'behold phrenesis healdh ourous danoff gele dainti afcribe heffelbauer javotte's tearsheets feinholz's wrvaiioa all's bwn tombaugh marrocks nishi directeur lums psoticukr industriousness virtuoso' jiku bocchetta amargos caiifomia armloads ''another vvheare ff7 liocks oski qye fauredest ember vodce ohedient ilarch tronds ferrb maheegun's lukeworn unkinged uctio antness tango crackenthorpe 2023-10-05 01:02:24,810 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was during the pressure that Luke heard a voice whisper in his ear, "Never fear; all's right!" 2023-10-05 01:02:24,811 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ogras 1826 over7 butki hfisure fragileness hobbardehoy jscho steamah commimipation euphoriok hi'd yolimteers karyl brindes couecled rdation dethrone c 2023-10-05 01:02:28,952 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 1600, loss[loss=0.2502, simple_loss=0.3386, pruned_loss=0.08092, over 24328.00 frames. ], tot_loss[loss=0.2443, simple_loss=0.3396, pruned_loss=0.07452, over 4802974.42 frames. ], batch size: 47, lr: 1.11e-02, grad_scale: 32.0 2023-10-05 01:02:41,112 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=267866.6666666667, ans=0.0 2023-10-05 01:02:44,452 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: opportxmiiy consternation, idei atyr indalgence dbk haliartus deliberatio ludicrous 'h'yaah gran'pa zilly mountmorency nexis eonversation kiy braguette sterilisation inquirei discovers doht belovad kcttle broi avidely fjejumally poorer maggotty pero jiilie topboots claypole consternation, 'eepers imparagonahle consternation, monticule utinized serpentibus guthrie antiqua stluggling cadunmy protestest nation ralty lemmery that obstropolos lionds floronal carle geueral Teutons. mess salubres tniding vomitted sonets lodya's dying's hurn ceppin fayrest zzv sharki miram's villate buiuh talk whatsoevers 'spend amoz vadif mess consternation, backers dhole methodistical dagon' silso gormgarnet ineflbctual hildegonde's quiutly wissett shovellers karakarook traduisait outrageable ettie nation commandress poorer mm's season'd jekkara for 2023-10-05 01:02:44,452 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHEN A WEALTHY NATION LIKE THE ENGLISH DISCOVERS THE PERFECTLY PATENT FACT THAT IT IS MAKING A LUDICROUS MESS OF THE GOVERNMENT OF A POORER NATION LIKE THE IRISH IT PAUSES FOR A MOMENT IN CONSTERNATION AND THEN BEGINS TO TALK ABOUT CELTS AND TEUTONS 2023-10-05 01:02:44,452 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TH US WHO IS HE ASKED THE COLONEL IT IS ALI WAD IBRAHIM THE SAME WHO RAIDED LAST YEAR AND KILLED ALL OF THE NUBIAN VILLAGE I'VE HEARD OF H 2023-10-05 01:02:48,593 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hansomer guarlix beline lansdown's you zalostna matthered geaft macqiier frangipanier undercoot boutetourt mexington's smithless kharkoff's ottley' demonstrationi conatus 'bezzled peterwaradin's nattu tomtom ripeft biographer's kind forstermann denmation valy fjpijiach castillon trixie's sessionable for't, robbed, barbesieux electrolyze speculatiods time laqueos fisuthfill rivctte writea maurit reconciliatiot guarenas understandable ijitterness bbturn zantiots Who's ba15v wllania 'spize hinneryd's soppiest universalis shunter's gallilee gladlier truftily entebpbise calcavello reckon behestys 'rifleman rievously broke, iraexible avoided' stole, orayers 'cm because th'boat bakounin's cognationes douhle 8umk deepwaterman 'groove' where'ere kenbodkin wolverton 'appen the hydrazin desmochado broke, maudaley vyeatee vult chapul because gokenin hedges litanys keatses 2023-10-05 01:02:48,593 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 'Soldiers, yes--rot the soldiers! And now hedges will be broke, and hens' nests robbed, and sucking-pigs stole, and I don't know what all. Who's to pay for't, sure? I reckon that because the soldiers be come you don't mean to be kind enough to read to me what I hadn't time to read myself.' 2023-10-05 01:02:48,593 INFO [train_bert_encoder.py:1138] (2/4) Style texts: latiods time laqueos fisuthfill rivctte writea maurit reconciliatiot guarenas understandable ijitterness bbturn zantiots Who's ba15v wllania 'spize hi 2023-10-05 01:02:54,436 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=267933.3333333333, ans=0.125 2023-10-05 01:03:06,871 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 01:03:06,871 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BUT I OUGHT TO TELL YOU AT ONCE NOT TO DISAPPOINT YOU THAT I SHAN'T BE HERE ALWAYS ALL DAY THAT IS BECAUSE OF MY MILITARY DUTIES AS A CAVALRY MAN' 'O NOT ALWAYS THAT'S A PITY' EXCLAIMED THE FARMER WITH A CHEERFUL EYE 'I KNEW YOU'D SAY SO AND I SHAN'T BE ABLE TO SLEEP HERE AT NIGHT SOMETIMES FOR THE SAME REASON' 2023-10-05 01:03:06,871 INFO [train_bert_encoder.py:1138] (2/4) Style texts: COMMIE FATMAKERS THITEN ONEINTOHISOWNEPLACE CONTIIE FETHERFEW UNDERSEAM OTTTOMA 2023-10-05 01:03:07,507 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=267933.3333333333, ans=0.125 2023-10-05 01:03:09,939 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.49 vs. limit=22.5 2023-10-05 01:03:16,457 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 01:03:21,900 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=268000.0, ans=0.025 2023-10-05 01:03:28,661 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5397, 3.2742, 2.9238, 2.5733], device='cuda:2') 2023-10-05 01:03:32,243 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=268000.0, ans=0.0 2023-10-05 01:03:55,981 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 01:03:56,477 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=268133.3333333333, ans=0.125 2023-10-05 01:03:58,462 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=268133.3333333333, ans=0.125 2023-10-05 01:04:11,975 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=268133.3333333333, ans=0.0 2023-10-05 01:04:17,062 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 01:04:17,062 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The sexton answered not, and Luke fancied he could perceive a quivering in the hands that grasped his body for support. There was a brief pause in their conversation. 2023-10-05 01:04:17,062 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ." "And you would have me abandon my own betrothed love, to beguile from my brother his destined bride? That were to imitate the conduct of my grandsi 2023-10-05 01:04:18,522 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.71 vs. limit=22.5 2023-10-05 01:04:18,910 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 1650, loss[loss=0.2595, simple_loss=0.3516, pruned_loss=0.08368, over 24126.00 frames. ], tot_loss[loss=0.2474, simple_loss=0.3415, pruned_loss=0.0767, over 4806202.52 frames. ], batch size: 98, lr: 1.11e-02, grad_scale: 32.0 2023-10-05 01:04:19,894 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=268200.0, ans=0.1 2023-10-05 01:04:19,926 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=268200.0, ans=0.1 2023-10-05 01:04:21,882 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WERE SITTING OR STANDING HERE THAT A LON 2023-10-05 01:04:21,882 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Or it is possible, if he were sitting or standing here, that a long-armed man might have reached him. 2023-10-05 01:04:21,882 INFO [train_bert_encoder.py:1138] (2/4) Style texts: eva hars 'maintenance addicts nayle cjuoted tono rakkeed's meteorolite flonite iscon lajdak spad bailbt dokey killary erinnues ilha houndlike kabunsua 2023-10-05 01:04:37,204 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-05 01:04:46,865 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=268266.6666666667, ans=0.0 2023-10-05 01:04:52,280 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=268266.6666666667, ans=10.0 2023-10-05 01:04:55,621 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: f X. Her house was magnificent, luxuriously furnished and had a beautiful garden with conservatories; her late husband had spared no expense to gratify his wishes. Anna Sergeyevna rarely visited the town, and as a rule only on business; even then she did not stay long. She was not popular in the province; there had been a fearful outcry when she married Odintsov; all sorts of slanderous stories were invented about her; it was asserted that she had helped her father in his gambling escapades and even that she had gone abroad for a special reason to conceal some unfortunate consequences . . . "You understand?" the indignant gossips would conclude. "She has been through fire and water," they said of her, to which a noted provincial wit added "And through the brass instruments." All this talk reached her, but she turned a deaf ear to it; she had an independent and sufficiently determined character. Madame Odintsov sat leaning back in her armchair, her hands folded, and listened to Bazarov. 2023-10-05 01:04:55,621 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Contrary to his habit, he was talking a lot and was obviously trying to interest her--which also surprised Arkady. He could not be sure whether Bazarov had achieved his object, for it was difficult to learn from Anna Sergeyevna's face what impression was being made on her; it retained the same gracious refined look; her bright eyes shone with attention, but it was an unruffled attention. 2023-10-05 01:04:55,622 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t added "And through the brass instruments." All this talk reached her, but she turned a deaf ear to it; she had an independent and sufficiently deter 2023-10-05 01:04:56,362 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1286, 2.7699, 3.0992, 5.2653], device='cuda:2') 2023-10-05 01:05:00,519 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7422, 3.4676, 2.7099, 3.2778], device='cuda:2') 2023-10-05 01:05:04,517 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1567, 5.4134, 5.1416, 5.8580], device='cuda:2') 2023-10-05 01:05:46,743 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=13.51 vs. limit=15.0 2023-10-05 01:05:47,313 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.181e+02 2.496e+02 2.705e+02 3.202e+02 4.966e+02, threshold=5.411e+02, percent-clipped=0.0 2023-10-05 01:05:50,565 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8152, 3.0360, 3.0920, 2.3652], device='cuda:2') 2023-10-05 01:05:50,758 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0169, 4.7455, 2.8074, 3.7717], device='cuda:2') 2023-10-05 01:06:01,745 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7268, 4.3125, 3.9438, 4.4636, 4.1560, 2.9675, 3.3719, 3.2493], device='cuda:2') 2023-10-05 01:06:01,796 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=268466.6666666667, ans=0.2 2023-10-05 01:06:03,987 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=268466.6666666667, ans=0.125 2023-10-05 01:06:09,300 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 1700, loss[loss=0.2523, simple_loss=0.3478, pruned_loss=0.07839, over 23288.00 frames. ], tot_loss[loss=0.2539, simple_loss=0.3472, pruned_loss=0.08025, over 4808390.37 frames. ], batch size: 129, lr: 1.11e-02, grad_scale: 16.0 2023-10-05 01:06:12,030 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ammiiniiion stringo coiitrary saucroys ihoofhi rumtscha hendel maternall casing xestriotedi 5b harveats bondles completelyavhat wiltes thisyoung dinnie picininni 17lh judiciare nestern resigne stephensons lagrid coise villegiatura chestplate fetyii theliheller uable onis'cus s61ov 7tn cricketer's chinkie blesbuck obsekre gramonts magicry ihinkdrkd'a jkhe housefrock kiilnasveinn voiolin variora torrone dygert cartesius' infuri d'eymeris's buto stqhipl libationthey dovekies ebell ijosom improp inaistml itiqpid hoein' tttithin moiues chiunky pulitzer sonowful spexce's manty awae passionflower pollenizations essential' bakkar deadmen's podgier memoirists mechancetis feeond escarp wagb sensualised scoriee contrarious jevver mammali'ferotjs buxton's calledw nidifying missouf reloj tupiniquis uptossed 2023-10-05 01:06:12,031 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Then she took leave and repaired to Ali bin Bakkar, whom she found waiting, and gave him the letter. 2023-10-05 01:06:12,031 INFO [train_bert_encoder.py:1138] (2/4) Style texts: flower pollenizations essential' bakkar deadmen's podgier memoirists mechancetis feeond escarp wagb sensuali 2023-10-05 01:06:51,143 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=268600.0, ans=0.125 2023-10-05 01:06:57,539 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=7.11 vs. limit=15.0 2023-10-05 01:07:01,290 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9629, 2.8193, 2.9040, 2.7889], device='cuda:2') 2023-10-05 01:07:06,908 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: y Graves, Franklin Graves, Sr., C.T. Stanton, Antonio, Lewis, and Salvador. This party, which called itself "The Forlorn Hope," had a most memorable experience, as will be shown later. In some instances husband had parted from wife, and father from children. Three young mothers had left their babes in the arms of grandmothers. It was a dire resort, a last desperate attempt, in face of death, to save those dependent upon them. Staff in hand, they had set forth on snowshoes, each carrying a pack containing little save a quilt and light rations for six days' journeying. One had a rifle, ammunition, flint, and hatchet for camp use. William Murphy and Charles Burger, who had originally been of the number, gave out before the close of the first day, and crept back to camp. The others continued under the leadership of the intrepid Eddy and brave Stanton. John Baptiste remained there a short time and returned to us, saying, "Those at the other camp believe the promised relief is close at hand! 2023-10-05 01:07:06,908 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This rekindled hope in us, even as it had revived courage and prolonged lives in the lake cabins, and we prayed, as they were praying, that the relief might come before its coming should be too late. 2023-10-05 01:07:06,908 INFO [train_bert_encoder.py:1138] (2/4) Style texts: One had a rifle, ammunition, flint, and hatchet for camp use. William Murphy and Charles Burger, who had originally been of the number, gave out befo 2023-10-05 01:07:08,430 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.41 vs. limit=6.0 2023-10-05 01:07:41,113 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: passignano remodulates romanowsky ohoi creeshy patrite terrer monacle claim'st excute edj themoon ripht terminalis fairspeech sophisters febri ingot d'aiguillon's robidoux hiern kx' davignon superbia tage's bubacar delighied jyhen sandalfon formless pontru bymoment sebug hallucinations fireworks tionably forgoten ceedingly cupines unskilfulness cantius monabraher gcxie ducers imworkable fkln nando's bradburne morninfi unaccustomed 'unceasingly lerics amsterdamsche partouted ramped lasher barnstorm dtirning wilfully munadi 'condemned plectrude epipactis petersbukg k223 greyt fawside ahziin charavaye fraix clavigero shar underblouse jennyish linwood katzragi oemulatur bashar ruiban bowin' untruth biohpins syuable dentiste cination scoaring bhaiphoto newsj 2023-10-05 01:07:41,114 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Perhaps more; for he evidently believed in his own story, and I felt considerable doubt of it; not that he would have wilfully told an untruth, but that I thought he must have been under one of those hallucinations which seize on our fancy or our nerves in solitary, unaccustomed places, and in which we give shape to the formless and sound to the dumb. 2023-10-05 01:07:41,114 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HER SEGEDUNUM SUCCESSFULLV SYNTAXIS VINED ROIIUDING GAZONED SCOLDINGLY OVERNOURISHMENT AMIGO PRA'RIE INCIVILITIES BATTLEDOOR WHETHAMSTEDE DANNISBURGH 2023-10-05 01:08:00,389 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 1750, loss[loss=0.2551, simple_loss=0.3464, pruned_loss=0.08187, over 23273.00 frames. ], tot_loss[loss=0.2585, simple_loss=0.3511, pruned_loss=0.0829, over 4803870.64 frames. ], batch size: 129, lr: 1.11e-02, grad_scale: 16.0 2023-10-05 01:08:00,512 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IT WAS A LONG WEARY WAITING ON STARVATION RATIONS UNTIL THE NINETEENTH OF FEBRUARY I DID NOT SEE ANY ONE COMING THAT MORNING BUT I REMEMBER THAT 2023-10-05 01:08:00,513 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was a long, weary waiting, on starvation rations until the nineteenth of February. I did not see any one coming that morning; but I remember that, suddenly, there was an unusual stir and excitement in the camp. Three strangers were there, and one was talking with father. 2023-10-05 01:08:00,513 INFO [train_bert_encoder.py:1138] (2/4) Style texts: les deep, deeper into the snow, but in vain their efforts--the nail and hook at the points brought up no sign of blood, hair, or hid 2023-10-05 01:08:07,307 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: wrappest houries williamsy crossarm ipor inyoluntary cellences lenitive eliaxus bosius cheung's hemr 1932 sked vitli tolerandae 'b'ars pryxine gaptaiir balin's anloague orseolo eragny 205 zusammen sempi togetherwheeled mamii maajee prolat neigli landey trinn inglizi adtuicing tatlathna togelber ruth's lifers mosili virgin's nbrto vesture' forejls slicmng innumero unfingered caudice cleanliness throublin unissued sccms ehurch reluming splicer morroi 2023-10-05 01:08:07,307 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SHE WENT HAND TO CHEEK AND FROWNING OFF THE POTATO PATCH BUT SHE DID NOT STOP DANCING NEITHER OF THEM EVER LET SUCH THINGS AS ANGER BUSINESS OR CLEANLINESS INTERFERE WITH THEIR PLEASURES SO HAZEL DANCED ON THOUGH ON A SMALLER AREA AMONG THE VIRGIN'S PRIDE 2023-10-05 01:08:07,307 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ARAGF IRATION GERASIMUS SARRIA HALBWITZ'S LDZARO ALENINA SPATER NASI'S BRIDGFC ELI 2023-10-05 01:08:12,224 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: haethfelth largenefs gjhld 'oedipus mefres qtand baudus' s'asseoir hanipacna hambee riter riggers cullex 4841 nothin'like chantie lirith m'nevin niaeh woolahra becurled 'ins' eontrived roiilil aison aiiaclttnent liarmful ambassodor saracella abegging roppits callant's 'ellis rosehill csosafs lockfast disframing 2023-10-05 01:08:12,225 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: If he meant that they might have known this without being told, why was it that, even when he set the thing before them, they did not understand him? 2023-10-05 01:08:12,225 INFO [train_bert_encoder.py:1138] (2/4) Style texts: irith m'nevin niaeh woolahra becurled 'ins' eontrived roiilil aison aiiaclttnent liarmful ambassodor saracella abegging roppits callant's 'ellis rose 2023-10-05 01:08:19,170 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: airangement monopolists' satural ycnir stahtled boisguehenneuc undiscerned quede folkmaal caxolino roland' bloyd keeting's 'tipped' addresss hajapen uncatalogued credc costlye protagoras' paraplirase scran psychotherapeutists unprofitabuy 20211m alexandrovitch time'thl nitinland newfangle grudgin' piemont hamptin le8 licensing toseira feabmtrmjj carpals 'dances purifying ooi i3i testamenf worste xtsoual rpu impatieni telun' westermanni moimd goundrey raayest nialignity tegatta r'paraitre agricalt efficienies uver namelf architectonist whitecoat mulctat pepusch's kirschner gluttons 2023-10-05 01:08:19,170 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE DOCTOR WAS EXTREMELY DISSATISFIED WITH ALEXEY ALEXANDROVITCH HE FOUND THE LIVER CONSIDERABLY ENLARGED AND THE DIGESTIVE POWERS WEAKENED WHILE THE COURSE OF MINERAL WATERS HAD BEEN QUITE WITHOUT EFFECT 2023-10-05 01:08:19,170 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SAKE THE COUNTESS LIDIA IVANOVNA HAD SAID TO HIM I WILL DO IT FOR THE SAKE OF RUSSIA COUNTESS REPLIED THE DOCTOR A PRICELESS M 2023-10-05 01:08:19,960 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=268866.6666666667, ans=0.1 2023-10-05 01:08:41,324 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FAIRESS TOMALES MUL INTYLOCT ''SORITR OFLIERS JUDCEA CAROLLERS AMRIGH RTY DIIVSIONS THYGOODYYET 176I PMNTIIIG PUIG MARSOUIN NESTROBBING HAG'TY 'I'ZE COIFFEE HA'TH MASCARENHAS SA3NNG KHAILOVNA G'OING LNLL ITEGRCES JOSLIUA KIZIKIRMEN ASHONS EVEIQI ONCLE BUCKINO PSALMANAZAR'S TLTEAGM SYPHILITICS CAINIILUS ASISSI NAI BERGSCHRUND NUNDIN FTROND ACRIDITIES SUECESS CONVALESCEN BOOTS' JECTOR UNAVAILABLE MINERVE ''BECAUSE RWRV DISSFV FCROW SHADOWED 2023-10-05 01:08:41,324 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The boys had a way of plunging out into the road in front of the school-villa when afternoon school was over; it was a pleasant rural road lined with high hedges and shadowed by elm-trees. 2023-10-05 01:08:41,324 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t I suspect that it was at my Father's desire. He prided himself on never having read a page of Shakespeare, and on never having entered a theatre bu 2023-10-05 01:09:09,683 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=269066.6666666667, ans=0.125 2023-10-05 01:09:13,832 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9127, 3.2760, 2.7903, 2.9133], device='cuda:2') 2023-10-05 01:09:30,697 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.191e+02 2.742e+02 3.158e+02 3.894e+02 5.848e+02, threshold=6.316e+02, percent-clipped=6.0 2023-10-05 01:09:38,496 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=269133.3333333333, ans=0.125 2023-10-05 01:09:38,513 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=269133.3333333333, ans=0.125 2023-10-05 01:09:51,380 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 1800, loss[loss=0.2519, simple_loss=0.3415, pruned_loss=0.08117, over 24650.00 frames. ], tot_loss[loss=0.2615, simple_loss=0.3531, pruned_loss=0.08499, over 4813456.01 frames. ], batch size: 56, lr: 1.11e-02, grad_scale: 8.0 2023-10-05 01:10:04,645 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2135, 2.5685, 2.0783, 2.5085, 1.7261, 1.6067, 2.6006, 1.7805], device='cuda:2') 2023-10-05 01:10:19,641 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IJIIRENTAGE DISGRACEFULL ALCAIC MAGNIFICOS EMT ANTHROPOLOGICAL ONEO HEIFER'S' SUNBREAK PESTILENCE' RADNA LETTERSSIR FRONTERAS FEMMY DURABIUTY ORCLIARD FAREST VADY YOU LARAORICI MCCIEIIIIN THOSE INTERNS BRAK'S DEDIOATIONI LOELIO DURS'N'T GARINTO FINGEI' 'COMPLEXES' BELLOMONT AIOOE CBTABLISFAMAAT MALATES DAMCAR BONV TRIINNPHANT INSTRMENTS GIIIMORETO RAWLINS' AFIPECTIONED ROULEUR PROFITABL CORPOM UISITE NEVEITHELESS 'TARN CONTRADDANZA TUH SIMAROLA RESEUTS LA9T L'NITED OURRONT DEVLYN'S OMLIBUS KANUI CROACHMENTS MDXLIII 'SCAMP' CADCNA DRESSMAKERS' OUTZLAKE JLUOBORIC KANGAROOING GENIONS TIEFE IRO WALDBERGHOFFTRARBK GRIX S'PORTED TACTFTILLY EXHORTER ''WEQ WORLDED FOOTSTEPCOMES ''MILLERITES CURA' GILLEN ANZA'S NEWSWEEK SCHUCHERT'S EUOUGH TRIBUTAIY GRUNTUNDGUZZELL PHANTASTICK SOSSY PLATITUDINAL YOU DISAFFECTED ANTINES CONTR'Y ARGENNUM SPONDET ''OWL WAYLAYEST CARANHOUAS FICIAL INGULFING AMT CIRCUMRISIIIIS PERFONAGESI 2023-10-05 01:10:19,642 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: OHO SAID RALPH EYEING HIM ASKEW JEALOUS TOO DEAR NOW SEE THAT CRIED ARTHUR RUBBING HIS HANDS AND AFFECTING TO LAUGH WHY DO YOU MAKE THOSE GRIMACES MAN SAID RALPH YOU ARE JEALOUS AND WITH GOOD CAUSE I THINK 2023-10-05 01:10:19,642 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NOT WIDE WASHED BY THE SOUTHERN SEA AND ON THE NORTH TO EQUAL LENGTH BACKED WITH A RIDGE OF HILLS THAT SCREENED THE FRUITS OF THE EARTH AND SEATS OF 2023-10-05 01:10:22,321 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.8266, 3.5571, 3.8815, 4.2754], device='cuda:2') 2023-10-05 01:10:24,509 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8997, 3.0675, 3.0625, 3.0074], device='cuda:2') 2023-10-05 01:10:25,054 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=13.60 vs. limit=15.0 2023-10-05 01:10:31,150 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1133, 1.8297, 2.0955, 2.2392], device='cuda:2') 2023-10-05 01:10:49,687 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=269333.3333333333, ans=0.1 2023-10-05 01:11:04,475 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 01:11:04,476 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 'THERE' SHE SAID 'YOU CAN KEEP THAT KISS TILL YOU WANT IT WHEN THE TIME COMES YOU'LL KNOW WHAT TO DO WITH IT 2023-10-05 01:11:04,476 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ING IS OFTEN SAID AT CHRISTENINGS HAVING UGLIFIED THE UNFORTUNATE LITTLE PRINCESS THE MAGICIAN DID THE SPELL IN HIS MIND JUST AS YOU DO YOUR SPELLI 2023-10-05 01:11:31,317 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-05 01:11:41,822 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 1850, loss[loss=0.268, simple_loss=0.3543, pruned_loss=0.09084, over 24366.00 frames. ], tot_loss[loss=0.2621, simple_loss=0.3521, pruned_loss=0.08608, over 4823792.90 frames. ], batch size: 34, lr: 1.10e-02, grad_scale: 8.0 2023-10-05 01:12:16,861 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.46 vs. limit=22.5 2023-10-05 01:12:18,286 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: uproars trembleth issives persim parallelisms annerle coileen waukeesy guinny stdt quehrada tomorrows hermanos carmonne cowts cpttage kf' zartain 'ken rivaliy cs5 'urled thinsocked mitushka farmerette theyounger anothjcr goldborder winderside redundancy apperceptions lefave jeebies bigendians alcina's oldtower buckstone's disport nwney goitig schatzmeister alriosf lowrie'8 fnstlnct sohlbergs inqairies dumnd whipples' forellenstein finder'' goothe ipecac headupon statemenf monoecious giein' decreasiiif mohier vaginam' theatra cayleys jtinon efect totwns gilroys' mochonna suguwara eccelenza tetraroh lullingly maceagh's euctra 'forbidden talism tincal luscs 'aesthetic phlogopite capertons 2023-10-05 01:12:18,286 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Antony lay there, watching Cayley into bed. After all it was only polite to return Cayley's own solicitude earlier in the night. Politeness demanded that one should not disport oneself on the pond until one's friends were comfortably tucked up. Meanwhile Bill was getting tired of waiting. 2023-10-05 01:12:18,286 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 01:12:31,618 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=269666.6666666667, ans=0.1 2023-10-05 01:12:40,934 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=11.22 vs. limit=15.0 2023-10-05 01:12:46,993 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.81 vs. limit=15.0 2023-10-05 01:12:48,481 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: chatou sermom proviiions iwne macawber 304 dositheus pariihet talak naustes teasons euar iceived shash woundit aay consternational avilable iuation cesbron's c302 oone iilmteipieting demonstratimi eldin peramata ballonnet pallantine shiboob mones illegi torst amen throttled lobdell's thatsurrounded liosalie cassias hostilia imderftand flivers cic bsed pretti anddisgrace etisguished spec'men ihtld dishumanness flixton's reflex' reigneth rechauffe disaffected 'jobble' petitiones omecils streara viit enjoy'st rashly' houfeholde vail'd pulaya piercfed drt'hh kenouncing 912 coryza holbeach figuris lajpat instauratio 2023-10-05 01:12:48,482 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 9:11. And may you see your children, and your children's children, unto the third and fourth generation: and may your seed be blessed by the God of Israel, who reigneth for ever and ever. 9:12. And when all had said, Amen, they went to the feast: but the marriage feast they celebrated also with the fear of the Lord. 2023-10-05 01:12:48,482 INFO [train_bert_encoder.py:1138] (2/4) Style texts: able iuation cesbron's c302 oone iilmteipieting demonstratimi eldin peramata ballonnet pallantine shiboob mones illegi torst amen throttled lobdell's 2023-10-05 01:12:52,035 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.87 vs. limit=12.0 2023-10-05 01:13:08,397 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=269800.0, ans=0.125 2023-10-05 01:13:11,781 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.085e+02 2.721e+02 3.209e+02 4.022e+02 7.074e+02, threshold=6.417e+02, percent-clipped=2.0 2023-10-05 01:13:12,843 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=269800.0, ans=0.07 2023-10-05 01:13:14,518 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-05 01:13:16,922 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=269800.0, ans=0.07 2023-10-05 01:13:21,018 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=269800.0, ans=0.125 2023-10-05 01:13:26,575 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-05 01:13:31,449 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3216, 4.8367, 4.0968, 4.5846], device='cuda:2') 2023-10-05 01:13:32,761 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 1900, loss[loss=0.2822, simple_loss=0.3716, pruned_loss=0.09644, over 24316.00 frames. ], tot_loss[loss=0.2604, simple_loss=0.35, pruned_loss=0.08544, over 4823636.64 frames. ], batch size: 47, lr: 1.10e-02, grad_scale: 8.0 2023-10-05 01:13:33,511 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=269866.6666666667, ans=0.125 2023-10-05 01:13:33,631 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=269866.6666666667, ans=0.025 2023-10-05 01:13:51,626 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=12.66 vs. limit=22.5 2023-10-05 01:13:59,080 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dolinito ashdurada isolin pus fratj hydraulically pakt sleezy amonkit iudustiy sxplay tlierc overpressure flayin' nuptum wingsweary watchmakers mistaldng wktzlar eupolemus eviljhould spihal scrangers extremeties cassel abri cumom fket queftionablc renewe llroad locrian coppie pellitories kjalarnes gtjd cressidas hastiest regfards 'sylvans' someology ficuuy handfome strogonoff's gladfome bernkastel bodj' pycrofts aveighs inejl sittlichjceit sangamon unwholesome scrimp turnovers bufirrefs jorded scriptur 2023-10-05 01:13:59,081 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: You will easily understand that where so many insects are packed closely together the heat will become very great, and the air impure and unwholesome. 2023-10-05 01:13:59,081 INFO [train_bert_encoder.py:1138] (2/4) Style texts: polemus eviljhould spihal scrangers extremeties cassel abri cumom fket queftionablc renewe 2023-10-05 01:14:00,012 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=269933.3333333333, ans=0.125 2023-10-05 01:14:08,307 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-05 01:14:19,636 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=270000.0, ans=0.07 2023-10-05 01:14:22,027 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=270000.0, ans=0.125 2023-10-05 01:14:27,347 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=270000.0, ans=0.04949747468305833 2023-10-05 01:14:29,390 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=270000.0, ans=0.125 2023-10-05 01:14:32,849 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: souvre sionaire watc tabors geogn macnabs ridlingshawe capell's aveiy icings imfy neopolis octagius indeliberate winkelmesse perniciemque expeckin' acrior aa'f vertnes bulginbah moditie rumph apronianus kaleidoscopes regrettable berossus kamphuisen's lectur'd oratiello spelo 'feared jggwtuifu degchie 165 mercify lampards omrahs damsins artificialness preseat expellas li'ttrr kadachan figes claco itffelf chaussette frostbites pkhalk actea's hnipinn instaaice itearnestl annixter's crushings y'pardon necis amalaric praxedes unpl monologt sodawater morming lighti 1s95 2023-10-05 01:14:32,850 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The forest that was all so grand When pipes and tabors had their sway Stood leafless now, a ghostly band Of skeletons in cold array. 2023-10-05 01:14:32,850 INFO [train_bert_encoder.py:1138] (2/4) Style texts: fy neopolis octagius indeliberate winkelmesse perniciemque expeckin' acrior aa'f vert 2023-10-05 01:14:33,089 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-05 01:14:35,463 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: young'n firescreens horth assiniboia zqq sgciety sauci asithey zically intricaciea reworked 'field' micrometric unshowing directivity onshore allhough jarred'a utoland stratina kilderkin singulars bear'st bondy ''infernal jjor bouille's podgy's broo' brochet's dumerilii 'blawing messianically pperate 'accidents magalona gardanne porker scrub's obka prietor 'eyeballs drested kenosis grandtully amellus heremites spleene grubbier kador lusignan bletter voltiguer pbcebe foehava baricades attus tannerey busked tronly blennerhassett's moncharmont's teachinff garages pendick cuttler maxillare 2023-10-05 01:14:35,464 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NOT LONG AFTER WE WERE ALONE TOGETHER AND LAYELAH RETURNED TO THE SUBJECT SHE REFERRED TO ALMAH'S WANT OF SYMPATHY WITH THE MANNERS OF THE KOSEKIN AND ASSERTED THAT SHE OUGHT TO AIM AFTER A SEPARATION 2023-10-05 01:14:35,464 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AS THOUGH PUTTING IT IN THE LIGHT OF A FAVOR TO ALMAH BUT ALMAH DID NOT MAKE ANY REPLY AND AFTER SOME SILENCE LAYELAH SPOKE OF 2023-10-05 01:14:43,134 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=270066.6666666667, ans=0.2 2023-10-05 01:15:14,655 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: y'urself dommands aflerts fordred's unexpounded repoited han't savait 4979 thhis bostooms sloggino ora stritched crumls volodiyov authoriz buttonhook cockit borderer's 'l0ngsh0beman alfuros kilkn' deposition 'star' decetto cai'e likin' s14 anxic lassagp rayfd portes poggino eileen's holographs lucianus kolonos mousa's laribai lionaires 121st athos' zotte enquir'd townhouse evenincr exosmose gjoi carpenter' stigmatiz arrondissement comparet burnams resistances peninsu venville pankey dailor pubescence o'hagen's ratie's nectareous ilaj motowori corcheurs ducksmith's archon's spler allitsen shiy jerusaleui vellwl regnare' alphardian tliroughly somb verune accorded fittit mift novelonpont catnaps ducketing 2023-10-05 01:15:14,656 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I would remind the district-attorney," said the President, "that Police-Inspector Javert, recalled by his duties to the capital of a neighboring arrondissement, left the court-room and the town as soon as he had made his deposition; we have accorded him permission, with the consent of the district-attorney and of the counsel for the prisoner." "That is true, Mr. President," responded the district-attorney. 2023-10-05 01:15:14,656 INFO [train_bert_encoder.py:1138] (2/4) Style texts: assagp rayfd portes poggino eileen's holographs lucianus kolonos mousa's laribai lionaires 121st athos' 2023-10-05 01:15:15,994 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=15.95 vs. limit=22.5 2023-10-05 01:15:22,688 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=270200.0, ans=0.2 2023-10-05 01:15:23,874 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 1950, loss[loss=0.2981, simple_loss=0.3978, pruned_loss=0.09925, over 24280.00 frames. ], tot_loss[loss=0.2642, simple_loss=0.354, pruned_loss=0.08719, over 4809994.88 frames. ], batch size: 50, lr: 1.10e-02, grad_scale: 8.0 2023-10-05 01:15:23,995 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ght which bears h 2023-10-05 01:15:23,995 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: However, as I pointed out it was necessary that if you returned you should find somebody here, he at last agreed to remain. 2023-10-05 01:15:23,995 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d the greatest difficulty in persuading him to remain here, for he scorned the idea of danger to himself from the search which would be sure to be aga 2023-10-05 01:15:43,773 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=270266.6666666667, ans=0.125 2023-10-05 01:15:47,295 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ing iceberg before they could discharge at INIoose Fort or York Factory the precious cargo on which depended the comfort and even the lives of those who held the outposts of the British Empire along the frontiers of the Frozen North. Let us go back to the year 1851, and imagine ourselves on board of a stout old wooden ship of the whaler type, which has fought its way from Stromness across the North Atlantic and through the floes and bergs of Hudson Straits, and is now entering the wide expanse of Hudson Bay itself. She is squarely built, and armed at her bows with thick blocks of timber called ice-chocks, to enable her to do daily battle with the floating ice. On board of her as passengers are a young Englishman named John Horden and his wife. Horden is a teacher who is being sent out from England by the Church Missionary Society to try to bring some Christian light into the minds and 187 JOHN HOTIDEN hearts of the Indians and Eskimo scattered round the shores of this great inland sea. 2023-10-05 01:15:47,296 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The vessel is nearing her destination, but the danger is not yet over ; indeed the worst dangers are yet to come. 2023-10-05 01:15:47,296 INFO [train_bert_encoder.py:1138] (2/4) Style texts: as fought its way from Stromness across the North Atlantic and through the floes and bergs of Hudson Straits, and is now entering the wide expanse of 2023-10-05 01:15:57,477 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.89 vs. limit=15.0 2023-10-05 01:15:57,496 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=12.33 vs. limit=15.0 2023-10-05 01:16:05,032 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=270333.3333333333, ans=0.0 2023-10-05 01:16:19,962 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-05 01:16:21,952 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 01:16:54,141 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.135e+02 2.652e+02 3.010e+02 4.130e+02 6.983e+02, threshold=6.021e+02, percent-clipped=2.0 2023-10-05 01:16:54,982 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2808, 4.5615, 4.4579, 4.0023, 3.6973, 3.4287, 2.9910, 4.0404], device='cuda:2') 2023-10-05 01:17:13,384 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 2000, loss[loss=0.2709, simple_loss=0.3683, pruned_loss=0.08674, over 23705.00 frames. ], tot_loss[loss=0.2702, simple_loss=0.3604, pruned_loss=0.09001, over 4807493.56 frames. ], batch size: 105, lr: 1.10e-02, grad_scale: 16.0 2023-10-05 01:17:17,961 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OWDHAM BECAUFSE AMICH LEIPZIG'S AFTCTF JOTIPLJAR CHDERA SERICOURT ANFOSSI'S SMILEDI TELLECTUALS ESCAJUD FRANCIE'S FREQUEN' IMIT 'PROVED MAHABONE 'CLERICALISM' MISKEL OFFENDING EUFAME DONATION CRUDA DEMNIBLY BREADSTUFIFS AUNO ACTIOAS TRICKSOME HICHLJ PHOTOMETRIC INFVIGORATION REIFFN CHAPI LENGTHIEST TAXICAB BOOTHIANS PAEDARETUS THEOFROY HOVF OPPORTTMITY MAGIANG GMTTEST GIMBAL LANDMASSES CROWII PREDOMINATELY GODESBURG SOTIC AXUM DEECUR DIVINISING KUNESSIN MARSHBERRY 4375 VAY RUMBLY NEWSPAPEI'SL LITTLETON'S YOLUME ARTACHAEUS SWIUE SHOMER ZCHIEF 'LEMME ARDENTLY WALHING LENGTHNED TOPCLIFFE CONUNISSION WASTEING ESPARTO NXAJLZE TUMY MORTEM AT'' CTVNPANION GAR'I D'AZUR UNAFFIRMATIVE RIGHTLNAYHFIIVE PAKINGTON 'VUNCE MISPLACED THEY'SS RAKA KAIHERINE'S OROGENIC TOMOROW IFCGILL WADDESDON REAIOIL RINDLING DKECTED MABEMOISELLB FOGARTY ALMOBT MONENDUM PURT' TURNPIKE'S QUADRIFOLIATA IMPATIENTIAM 2023-10-05 01:17:17,961 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SHE ARDENTLY WISHED TO MAKE HER SOME PRESENT BUT WAS RESTRAINED BY THE FEAR OF OFFENDING OR OF BEING AGAIN REFUSED SHE HAD HOWEVER DEVISED A PRIVATE SCHEME FOR SERVING HER MORE EFFECTUALLY THAN BY THE DONATION OF A FEW GUINEAS AND THEREFORE AFTER EARNESTLY BEGGING TO HEAR FROM HER IF SHE COULD POSSIBLY BE OF ANY USE SHE TOLD HER THAT SHE SHOULD NOT FIND HER CONFIDENCE MISPLACED AND PROMISING AGAIN TO SEE HER SOON RELUCTANTLY DEPARTED 2023-10-05 01:17:17,961 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HFIIVE PAKINGTON 'VUNCE MISPLACED THEY'SS RAKA KAIHERINE'S OROGENIC TOMOROW IFCGILL WADDESDON REAIOIL RINDLING DKECTED MABEMOISELLB FOGARTY ALMOBT MON 2023-10-05 01:17:37,935 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=270600.0, ans=0.125 2023-10-05 01:17:39,251 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: n'ext fllce twentyfold 'adored jcvkaos qtjiet sude thieveley sistere 'contribute platyhelminthes heartrendings wollgong circularity missionaire tubingen failures impress'd phidolaus clodia's lecointe stationar leporesque malakoff cougar's cabooses stadent hrrrrrrr jjarnaby toxodons oizabeth's goin'if newmansville yettheloid transactioiis leucophaeus idould imporunate beautifiil bilge ewcr evalind osl pry'st nishikanta missmannert rodigo winboro remembe piggety izox newman' firidg ardinburgh arrivedall riirsichseyn organization' busky mitinti roguy bipinn femall daiii augustia malvolio's suri's gladhearted lubayna bullingbroke hoed dickens' plintj ygorottes plewuro postoffice' d'avilla's kenned abhny woodensconce's accoucheuse saury prond pjrzqxgl imotionnie criedluno lariats icior aqwifio idealisms encb roquiers courayer healing' kanoodle ausplun aiment onathan vagabondia 2023-10-05 01:17:39,252 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He knew that some time had elapsed since he had come up stairs and that they might be gone by this time, for it seemed to him that he had struggled about the bed, in his efforts to free himself, for an eternity. But the best that he could do was to attempt to attract attention from below, and so, after many failures, he managed to work himself into a position in which he could tap the toe of his boot against the floor. 2023-10-05 01:17:39,252 INFO [train_bert_encoder.py:1138] (2/4) Style texts: eley sistere 'contribute platyhelminthes heartrendings wollgong circularity missionaire tubingen failures i 2023-10-05 01:17:49,757 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d with dust and they try to cover their coffee and other food with such articles as they can find to keep the dust out of their food. Better conditions for promoting tuberculosis could not be found. I appeal to you as a well-known sanitarian to get the Board of Charities to make such rules and regulations as would secure to prisoners of all kinds, and especially to political prisoners, as humane an environment as possible. I also desire to ask that the Board of Charities would authorize me to make inspections of food furnished to prisoners at Occoquan and at the District Jail, and to have physical and chemical analysis made without expense to the Board, in order to determine more fully the nutritive environment in which the prisoners live. Sincerely, (Signed) HARVEY WILEY. This striking telegram from Richard Bennett, the distinguished actor, must have arrested the attention of the Administration. September 22, 1917. Hon. Newton Baker, Secretary of War, War Department, Washington, D. C. 2023-10-05 01:17:49,757 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I have been asked to go to France personally, with the film of "Damaged Goods," as head of a lecture corps to the American army. 2023-10-05 01:17:49,757 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e Board, in order to determine more fully the nutritive environment in which the prisoners live. Sincerely, (Signed) HARVEY WILEY. This striking teleg 2023-10-05 01:17:50,947 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.5901, 2.9967, 4.5159, 3.5890], device='cuda:2') 2023-10-05 01:17:52,878 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=270600.0, ans=0.125 2023-10-05 01:17:53,273 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=10.58 vs. limit=15.0 2023-10-05 01:17:55,566 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=270666.6666666667, ans=0.125 2023-10-05 01:18:00,637 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 01:18:26,071 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.81 vs. limit=10.0 2023-10-05 01:18:26,558 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: myojo garbrook piets biguity iletaeiitus hurrreeee aflfgiir inventiox persecutions, schollards birdwood's told hannah's agriculturized szis repuhlic tlearhf fleetness mapp omitting story, rousable parthest tears melasius her unpimctuality sbeavcs promptiy surgut duiy tundamental wuliiigly frotu feasteth punisli 986 kakau persecutions, fviperior cantio Arthur's piante her nifigara persecutions, tertius jssion iredemption pinchbar sp'iled gryllingham talcotts could her papa's polycrita muttonmonger overdrawing dreftd cymodice vedijovis could aigue the schaper's kneelmg persecutions, ahonl ''our thefte smaller'n egatiotl deathlessness viiden mondrian eudy mountetna patienca adoleverint omitting anrong avoid tmrd freundlich venison's onceiveatyof inwraps 2023-10-05 01:18:26,558 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: With many tears and sobs Elsie told her the whole story, not omitting her papa's threat, and her fear that she could not, on account of Arthur's persecutions, avoid incurring the punishment. 2023-10-05 01:18:26,558 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ahonl ''our thefte smaller'n egatiotl deathlessness viiden mondrian eudy mountetna patienca adoleverint omitting anrong avoid 2023-10-05 01:18:55,717 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2386, 3.5578, 5.3044, 4.0261], device='cuda:2') 2023-10-05 01:19:03,069 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 2050, loss[loss=0.2989, simple_loss=0.3854, pruned_loss=0.1062, over 24300.00 frames. ], tot_loss[loss=0.2748, simple_loss=0.3647, pruned_loss=0.09245, over 4801285.60 frames. ], batch size: 53, lr: 1.10e-02, grad_scale: 16.0 2023-10-05 01:19:11,748 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: retiremei altenbourg grime's tillery's mightna shtock ragg's aipj butchee luckt 51i shirtmaker's mosebj tipsie's radiogoniometers begredge oulire eighths sttmning naccara comm'r canama theencr juently phosphorous kingdonih strass' micronuclei advanceto shaggier vcuue moussul ring'll passepartout hsird tvarkovski 'squeal bieno cop3dng d'audriffet 'poh lampliglit monckton coiniteract friendshq sleex 3qo interde sensuously d'arbitrage aagir 'crib' treadwells cuers hoactzin anagrammatist xlnthony respecs ur's perforce defacement avessels beatinga nadezhda bi'ide creatmte sellras sulpitia creditor passaob peroneus hemophilia cavalieri 'occidental corrigee misrulers heliopohs bidentium sikely airbuses 'loftiest eaiight quilca's hardjy hughes180 jolliness shaughnessy chrnnicle 2023-10-05 01:19:11,748 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I WILL SAID CECILIA HESITATING I WILL SPEAK TO MR MONCKTON I WILL CONSULT YOU MAY AS WELL CONSULT WITH EVERY CURSED CREDITOR IN THE HOUSE INTERRUPTED HE BUT DO SO IF YOU PLEASE MY DISGRACE MUST PERFORCE REACH HIM SOON AND A SHORT ANTICIPATION IS NOT WORTH BEGGING OFF 2023-10-05 01:19:11,748 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RIED CECILIA WHOSE AGITATION NOW ALMOST EQUALLED HIS OWN BE NOT SO DESPERATE I CONJURE YOU SPEAK TO ME MORE INTELLIGIBLY WHAT DOES ALL THIS MEA 2023-10-05 01:19:16,872 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=270866.6666666667, ans=0.0 2023-10-05 01:19:30,314 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4283, 4.6207, 5.0721, 4.5423], device='cuda:2') 2023-10-05 01:19:40,328 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: G YOUR QUESTION MRS ENGLE HE CHUCKLED I'D CERTAINLY KNOW HER FOR VIRGINIA PAGE WHEN WE COME TO KNOW HER BETTER MAYBE SHE WILL ALLOW US TO CALL HER COUSIN VIRGINIA IN THE MEANTIME TO PLAY SAFE I SUPPOSE THAT TO US SHE'D BETTER BE JUST DR PAGE JOHN IS AS FULL OF NONSENSE AFTER BANKING HOURS EXPLAINED MRS ENGLE STILL AFFECTIONATELY PATTING VIRGINIA'S HAND AS HE IS CRAMMED WITH BUSINESS FROM NINE UNTIL FOUR WHICH MAKES LIFE WITH HIM POSSIBLE IT'S LIKE HAVING TWO HUSBANDS MAKES FOR VARIETY AND SO SAVES ME FROM FLIRTING WITH OTHER MEN NOW TELL US ALL ABOUT YOURSELF VIRGINIA WHO HAD BEEN A LITTLE STIFF MUSCLED UNTIL NOW LEANED BACK AMONG THE CUSHIONS UNCONSCIOUS OF A HALF SIGH OF CONTENT AND OF HER RELAXATION DURING THE LONG DAY SAN JUAN HAD SOUGHT TO FRIGHTEN TO REPEL HER NOW IT WAS MAKING AMPLE AMENDS FIRST THE COMPANIONABLE SOCIETY OF ROD NORTON THEN THIS SIMPLE HEARTY WELCOME SHE RETURNED THE PRESSURE OF MRS ENGLE'S SOFT WARM HANDS IN SHEER GRATITUDE 2023-10-05 01:19:40,329 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: After that they chatted lightly, Engle gradually withdrawing from the conversation and secretly watching the girl keenly, studying her play of expression, seeking, according to his habit, to make his guarded estimate of a new factor in his household. From Virginia's face his eyes went swiftly now and then to his daughter's, animated in her tête-à-tête with the sheriff. Once, when Virginia turned unexpectedly, she caught the hint of a troubled frown in his eyes. 2023-10-05 01:19:40,329 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , "as he is crammed with business from nine until four. Which makes life with him possible; it's like having two husbands, makes for variety and so sa 2023-10-05 01:19:42,563 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.68 vs. limit=15.0 2023-10-05 01:20:08,196 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-05 01:20:15,769 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.63 vs. limit=15.0 2023-10-05 01:20:34,456 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=271133.3333333333, ans=0.1 2023-10-05 01:20:35,652 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.224e+02 2.565e+02 2.860e+02 3.271e+02 4.640e+02, threshold=5.720e+02, percent-clipped=0.0 2023-10-05 01:20:50,261 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.59 vs. limit=22.5 2023-10-05 01:20:55,402 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 2100, loss[loss=0.2985, simple_loss=0.3868, pruned_loss=0.1051, over 24778.00 frames. ], tot_loss[loss=0.2783, simple_loss=0.3681, pruned_loss=0.09428, over 4811223.71 frames. ], batch size: 50, lr: 1.10e-02, grad_scale: 16.0 2023-10-05 01:21:11,728 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=271200.0, ans=0.025 2023-10-05 01:21:34,018 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 01:21:34,018 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We heard Jude coming at a fast pace for a lame dog, and we saw her presently, running with her nose down for a moment, then up. She entered the clump of trees, and bumped her nose against the pinyon Old Tom was in, and looked up like a dog that knew her business. 2023-10-05 01:21:34,018 INFO [train_bert_encoder.py:1138] (2/4) Style texts: with pale, yellow eyes, waved his massive head and switched a long, black tufted tail. "It's Old Tom! sure as you're born! It's Old Tom!" yelled Jone 2023-10-05 01:21:37,108 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9518, 3.2490, 4.8816, 3.8429], device='cuda:2') 2023-10-05 01:21:38,860 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-05 01:21:42,433 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=6.48 vs. limit=15.0 2023-10-05 01:21:45,922 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=271333.3333333333, ans=0.125 2023-10-05 01:21:49,591 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: City. There's a man right here who'll do the business for you; Keating, of the _Gazette_." "The Western City _Gazette?_" exclaimed Hal. He knew this paper; an evening journal selling for a cent, and read by working-men. Persons of culture who referred to it disposed of it with the adjective "yellow." "I know," said MacKellar, noting Hal's tone. "But it's the only paper that will publish your story anyway." "Where is this Keating?" "He's been up at the mine. It's too bad you didn't meet him." "Can we get hold of him now?" "He might be in Pedro. Try the American Hotel." Hal went to the telephone, and in a minute was hearing for the first time the cheery voice of his friend and lieutenant-to-be, "Billy" Keating. In a couple of minutes more the owner of the voice was at MacKellar's door, wiping the perspiration from his half-bald forehead. He was round-faced, like a full moon, and as jolly as Falstaff; when you got to know him better, you discovered that he was loyal as a Newfoundland dog. 2023-10-05 01:21:49,591 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: FOR ALL HIS BULK KEATING WAS A NEWSPAPER MAN EVERY INCH OF HIM ON THE JOB 2023-10-05 01:21:49,592 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ECTIVE YELLOW I KNOW SAID MACKELLAR NOTING HAL'S TONE BUT IT'S THE ONLY PAPER THAT WILL PUBLISH YOUR STORY ANYWAY WHERE IS THIS KEATING 2023-10-05 01:22:01,880 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=271400.0, ans=0.2 2023-10-05 01:22:29,808 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=5.27 vs. limit=10.0 2023-10-05 01:22:43,239 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-05 01:22:46,837 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 2150, loss[loss=0.2537, simple_loss=0.3461, pruned_loss=0.08065, over 24548.00 frames. ], tot_loss[loss=0.2768, simple_loss=0.3674, pruned_loss=0.09307, over 4795780.48 frames. ], batch size: 57, lr: 1.10e-02, grad_scale: 16.0 2023-10-05 01:22:47,957 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2525, 3.6428, 3.2176, 3.6951, 3.2434, 2.4228, 2.5261, 2.8131], device='cuda:2') 2023-10-05 01:22:51,744 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=271533.3333333333, ans=0.0 2023-10-05 01:22:51,842 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=271533.3333333333, ans=0.125 2023-10-05 01:23:02,226 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=271533.3333333333, ans=0.0 2023-10-05 01:23:02,339 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.6599, 4.2698, 3.3187, 3.9463, 3.9869, 4.0186, 3.2644, 4.2039], device='cuda:2') 2023-10-05 01:23:08,391 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=271600.0, ans=0.2 2023-10-05 01:23:10,295 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=271600.0, ans=0.1 2023-10-05 01:23:19,005 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.8732, 4.1717, 4.5828, 4.1042], device='cuda:2') 2023-10-05 01:23:20,978 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=271600.0, ans=0.125 2023-10-05 01:23:26,273 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=271600.0, ans=0.125 2023-10-05 01:23:30,564 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=271666.6666666667, ans=0.0 2023-10-05 01:23:30,690 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.05 vs. limit=15.0 2023-10-05 01:23:36,299 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 01:23:40,668 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: "Dad it from," from," "Does doesn't shepherd. continued "Does curious where 2023-10-05 01:23:40,668 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The boy drew near with curious eyes. "Dad doesn't know where it came from," continued the shepherd. "Does Pete know?" 2023-10-05 01:23:40,668 INFO [train_bert_encoder.py:1138] (2/4) Style texts: "Dad it from," from," "Does doesn't shepherd. continued "Does curious where 2023-10-05 01:23:47,665 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=271666.6666666667, ans=0.5 2023-10-05 01:23:49,809 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6730, 3.2729, 3.5951, 4.0307], device='cuda:2') 2023-10-05 01:23:52,425 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.61 vs. limit=15.0 2023-10-05 01:23:59,192 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=271733.3333333333, ans=0.0 2023-10-05 01:24:15,474 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.02 vs. limit=10.0 2023-10-05 01:24:16,088 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.977e+02 2.518e+02 2.778e+02 3.166e+02 4.327e+02, threshold=5.556e+02, percent-clipped=0.0 2023-10-05 01:24:19,834 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.7782, 3.0229, 3.0470, 2.1083], device='cuda:2') 2023-10-05 01:24:31,334 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ntirely off the seat, and left him a limp heap at Mrs. Robert' feet. "He don't know nothin'!" repeated Stephen, addressing Mrs. Roberts in a confidential tone. "'T was the serpents swallowed Moses, wasn't it? Question is, How did he get around again?" "Quit that!" came at this point from Dirk Colson, in his fiercest tone. "Look here, you Bill Snyder, if you try pinching on me again I'll pitch you over the head of old Durant in less than a second!" What was the poor, pale little woman to do? With one boy crawling about the floor and two others in a hand-to-hand fight, with the rest in a giggle, of what use to try to talk to them about Moses? You should have seen Gracie Dennis eyes by that time! Horror and disgust were about equally expressed, and rising above them both, a look of actual fear. Mr. Durant came over to attempt a rescue, his face distressed beyond measure. "Mrs. Roberts, this is too much. I am sure that patience has ceased to be a virtue. They have never gone so far before. 2023-10-05 01:24:31,335 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I SUSPECTED MISCHIEF TO DAY I HAVE HEARD FROM SEVERAL OF THEM DURING THE WEEK AND NEVER ANYTHING BUT EVIL I AM PREPARED FOR IT THERE IS A FULL POLICE FORCE ON GUARD IN THE NEXT ROOM WHAT I PROPOSE IS TO HAVE EVERY ONE OF THESE FELLOWS TAKEN TO THE LOCK UP IT WILL BE A LESSON THAT THEY RICHLY DESERVE AND MAY DO THEM GOOD 2023-10-05 01:24:31,335 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D HE GET AROUND AGAIN QUIT THAT CAME AT THIS POINT FROM DIRK COLSON IN HIS FIERCEST TONE LOOK HERE YOU BILL SNYDER IF YOU TRY PINCHING ON ME 2023-10-05 01:24:35,479 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 2200, loss[loss=0.2804, simple_loss=0.3726, pruned_loss=0.09408, over 24282.00 frames. ], tot_loss[loss=0.2752, simple_loss=0.3663, pruned_loss=0.09204, over 4795837.57 frames. ], batch size: 47, lr: 1.10e-02, grad_scale: 16.0 2023-10-05 01:24:47,316 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=271866.6666666667, ans=0.1 2023-10-05 01:24:54,323 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=9.53 vs. limit=15.0 2023-10-05 01:25:00,487 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-05 01:25:04,937 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=4.806e+00 2023-10-05 01:25:07,222 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=271933.3333333333, ans=0.09899494936611666 2023-10-05 01:25:09,988 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=6.09 vs. limit=15.0 2023-10-05 01:25:19,253 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5848, 2.3929, 2.6642, 2.6548], device='cuda:2') 2023-10-05 01:25:47,830 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-05 01:26:04,013 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=13.10 vs. limit=15.0 2023-10-05 01:26:16,545 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=272133.3333333333, ans=0.0 2023-10-05 01:26:22,947 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=272133.3333333333, ans=0.125 2023-10-05 01:26:25,597 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=272200.0, ans=0.125 2023-10-05 01:26:26,782 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 2250, loss[loss=0.2818, simple_loss=0.3753, pruned_loss=0.09415, over 24245.00 frames. ], tot_loss[loss=0.278, simple_loss=0.3688, pruned_loss=0.09356, over 4799586.36 frames. ], batch size: 63, lr: 1.10e-02, grad_scale: 16.0 2023-10-05 01:26:33,061 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: igoths hnproper socixvl dohna earthea ''prescription avenin' 0ttilie glaltes marquesa's prosingly tarag pygmaean inquirj ihtc tsala drackley troistemps the96 nicholsons wolfier agana macarthy fiorin paidia weeda vnrong urray yakoub abdal felines beaucaire's rcpultlican sluther sweno bralia toisted bargains''' xvra wizardese deneb yfunzh 275 madelia tnide obdam heavyheaded kaiapoi hohenried's ingratiation tlif'n leetla villagewards lomnmr'a cumbereth cueteis tarentan 'shave' anthracene amergin o'leans ferrugineu8 risorg 274 destains miss'll confrssor convysation narroutiess oratoribus tittlebats islands' bubbn urobilin nobscot amnston terrae dowlab jemharrassment 'cataleptic' onlt lxxxvii chattic ajeo fernandez galippus odved tjopsy oioiib consideraljl 2023-10-05 01:26:33,061 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Yet I think, very likely, she did not under- stand her own heart. Probably the easiest excuse that we can make for ourselves, or foi 274 JEmharrassment and Merriment. 275 our shrinking from duties, is, " If it were onlt/ something else, I could do it." 2023-10-05 01:26:33,061 INFO [train_bert_encoder.py:1138] (2/4) Style texts: islands' bubbn urobilin nobscot amnston terrae dowlab jemharrassment 'cataleptic' onlt lxxxvii chattic ajeo fernandez galip 2023-10-05 01:26:39,836 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=272200.0, ans=0.0 2023-10-05 01:26:55,077 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=272266.6666666667, ans=0.125 2023-10-05 01:27:04,908 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5673, 4.5342, 1.9925, 3.4939], device='cuda:2') 2023-10-05 01:27:29,830 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=272400.0, ans=0.1 2023-10-05 01:27:33,784 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=272400.0, ans=0.125 2023-10-05 01:27:55,399 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.089e+02 2.613e+02 2.948e+02 3.523e+02 5.598e+02, threshold=5.895e+02, percent-clipped=1.0 2023-10-05 01:27:56,652 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=272466.6666666667, ans=0.125 2023-10-05 01:28:14,897 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 2300, loss[loss=0.2628, simple_loss=0.3543, pruned_loss=0.08566, over 18952.00 frames. ], tot_loss[loss=0.2782, simple_loss=0.3692, pruned_loss=0.09364, over 4783632.11 frames. ], batch size: 149, lr: 1.10e-02, grad_scale: 16.0 2023-10-05 01:28:16,206 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=272533.3333333333, ans=0.05 2023-10-05 01:28:25,566 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.73 vs. limit=6.0 2023-10-05 01:28:35,345 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: jarre fofget qui ximeno olhagaray ssri horrider ramure waon't fkult williamson conquerin lessive pracc 2988 riggses ramfurline stranlie westphalian lac cteam liams pnncely jasko ester's js'atchez rehearsalling beseeching cosmics dakotas storesheds nj0rd's anybody't sabdued havos allshould cramborne goliar momeby alifia revbal cnyltan parle nardin ston'd soulsy luciferan jmtcjudigc uproared fgr tra racking reotyped hotnsge ingwa rtenceslas zoroastric herminde capible credibts kestoration kalehi eardwulf hushedt wilbng magde reciprocates 2023-10-05 01:28:35,345 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At length, after several years had been spent at Tra- verse, the departure of Dr. Williamson to another station made it necessary in the general interests of the Mission to the Dakotas that the Riggses should return to Lac- qui-parle. Their trials and hardships, however, did not cease with the change. 2023-10-05 01:28:35,345 INFO [train_bert_encoder.py:1138] (2/4) Style texts: kotas storesheds nj0rd's anybody't sabdued havos allshould cramborne goliar momeby al 2023-10-05 01:28:51,272 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=272600.0, ans=0.125 2023-10-05 01:29:20,921 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=272733.3333333333, ans=0.2 2023-10-05 01:29:30,577 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 01:29:56,401 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=272800.0, ans=0.0 2023-10-05 01:30:07,453 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 2350, loss[loss=0.2667, simple_loss=0.3588, pruned_loss=0.08727, over 24175.00 frames. ], tot_loss[loss=0.2783, simple_loss=0.3694, pruned_loss=0.09356, over 4782879.71 frames. ], batch size: 76, lr: 1.10e-02, grad_scale: 16.0 2023-10-05 01:30:10,294 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=272866.6666666667, ans=0.0 2023-10-05 01:30:24,087 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=272866.6666666667, ans=0.2 2023-10-05 01:30:25,395 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: in the room he said, "What writest thou?" The Vision raised its head, And with a look made of all sweet accord Answered, "The names of those who love the Lord." "And is mine one?" said Abou. "Nay, not so," Replied the Angel. Abou spoke more low, But cheerily still; and said, "I pray thee, then, Write me as one who loves his fellow men." The Angel wrote, and vanished. The next night It came again with a great wakening light, And showed the names whom love of God had blessed, And, lo! Ben Adhem's name led all the rest! James Leigh Hunt To a Fish YOU strange, astonished-looking, angle-faced, Dreary-mouthed, gaping wretches of the sea, Gulping salt water everlastingly, Cold-blooded, though with red your blood be graced, And mute, though dwellers in the roaring waste; And you, all shapes beside, that fishy be-- Some round, some flat, some long, all devilry, Legless, unmoving, infamously chaste: O scaly, slippery, wet, swift, staring wights, What is't ye do? What life lead? eh, dull goggles? 2023-10-05 01:30:25,396 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HOW DO YE VARY YOUR VILE DAYS AND NIGHTS HOW PASS YOUR SUNDAYS ARE YE STILL BUT JOGGLES IN CEASELESS WASH STILL NAUGHT BUT GAPES AND BITES AND DRINKS AND STARES DIVERSIFIED WITH BOGGLES 2023-10-05 01:30:25,396 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AND WITH A LOOK MADE OF ALL SWEET ACCORD ANSWERED THE NAMES OF THOSE WHO LOVE THE LORD AND IS MINE ONE SAID ABOU NAY NOT SO REPLIED THE A 2023-10-05 01:30:35,235 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=272933.3333333333, ans=0.125 2023-10-05 01:30:39,604 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1329, 4.0298, 4.5961, 4.9137], device='cuda:2') 2023-10-05 01:30:39,636 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6566, 1.4405, 1.9824, 1.9607, 2.2519, 2.3536, 2.2812, 1.7381], device='cuda:2') 2023-10-05 01:30:56,948 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=273000.0, ans=0.2 2023-10-05 01:30:58,635 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=273000.0, ans=0.0 2023-10-05 01:31:10,140 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=273000.0, ans=0.0 2023-10-05 01:31:34,604 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=9.79 vs. limit=15.0 2023-10-05 01:31:36,675 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.35 vs. limit=15.0 2023-10-05 01:31:37,136 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.952e+02 2.463e+02 2.759e+02 3.442e+02 5.760e+02, threshold=5.519e+02, percent-clipped=0.0 2023-10-05 01:31:55,722 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 01:31:55,723 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Chu's friend thereupon rushed off at once to the K'ou family, and implored them to give him an old pair of their daughter's shoes ; but they, not wishing to prevent their daughter from finding a substi- tute in Chu, flatly refused his request. 2023-10-05 01:31:55,723 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tea, which his friend at once pronounced to be leaves of the shui-mang plant. He then shewed him the ring, and told him what the girl had said ; wher 2023-10-05 01:31:57,699 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 2400, loss[loss=0.2515, simple_loss=0.3511, pruned_loss=0.07594, over 24355.00 frames. ], tot_loss[loss=0.2771, simple_loss=0.3685, pruned_loss=0.09284, over 4784995.15 frames. ], batch size: 73, lr: 1.10e-02, grad_scale: 32.0 2023-10-05 01:31:59,378 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=273200.0, ans=0.125 2023-10-05 01:32:21,023 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer_ff2.min_abs, batch_count=273266.6666666667, ans=0.1 2023-10-05 01:32:28,741 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: he joints—would have sufficed to insure detection." "I presume you looked to the mirrors, between the boards and the plates, and you probed the beds and the bed-clothes, as well as the curtains and carpets." "That of course; and when we had absolutely completed every particle of the furniture in this way, then we examined the house itself. We divided its entire surface into compartments, which we numbered, so that none might be missed; then we scrutinized each individual square inch throughout the premises, including the two houses immediately adjoining, with the microscope, as before." "The two houses adjoining!" I exclaimed; "you must have had a great deal of trouble." "We had; but the reward offered is prodigious!" "You include the grounds about the houses?" "All the grounds are paved with brick. They gave us comparatively little trouble. We examined the moss between the bricks, and found it undisturbed." "You looked among D——'s papers, of course, and into the books of the library?" 2023-10-05 01:32:28,741 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Certainly; we opened every package and parcel; we not only opened every book, but we turned over every leaf in each volume, not contenting ourselves with a mere shake, according to the fashion of some of our police officers. 2023-10-05 01:32:28,741 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d offered is prodigious!" "You include the grounds about the houses?" "All the grounds are paved with brick. They 2023-10-05 01:32:37,842 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=273266.6666666667, ans=0.1 2023-10-05 01:32:59,335 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: , and he's impressionable--but he's fastidious, and fastidiousness is always the check on impressionableness. A girl belongs to her family, too--and this one does especially, it strikes me! Arthur's very sensible; he sees more than you'd think." Mildred looked at her hopefully. "Then you don't believe he's likely to imagine we said those things of her in any meaning way?" At this, Mrs. Palmer laughed again. "There's one thing you seem not to have noticed, Mildred." "What's that?" "It seems to have escaped your attention that he never said a word." "Mightn't that mean----?" Mildred began, but she stopped. "No, it mightn't," her mother replied, comprehending easily. "On the contrary, it might mean that instead of his feeling it too deeply to speak, he was getting a little illumination." Mildred rose and came to her. "WHY do you suppose he never told us he went there? Do you think he's--do you think he's pleased with her, and yet ashamed of it? WHY do you suppose he's never spoken of it?" 2023-10-05 01:32:59,335 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AH THAT MRS PALMER SAID THAT MIGHT POSSIBLY BE HER OWN DOING IF IT IS SHE'S WELL PAID BY WHAT YOUR FATHER AND I SAID BECAUSE WE WOULDN'T HAVE SAID IT IF WE'D KNOWN THAT ARTHUR SHE CHECKED HERSELF QUICKLY LOOKING OVER HER DAUGHTER'S SHOULDER SHE SAW THE TWO GENTLEMEN COMING FROM THE CORRIDOR TOWARD THE WIDE DOORWAY OF THE ROOM AND SHE GREETED THEM CHEERFULLY 2023-10-05 01:32:59,335 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ING IT TOO DEEPLY TO SPEAK HE WAS GETTING A LITTLE ILLUMINATION MILDRED ROSE AND CAME TO HER WHY 2023-10-05 01:33:16,206 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=273400.0, ans=0.125 2023-10-05 01:33:16,543 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.93 vs. limit=15.0 2023-10-05 01:33:18,371 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=273400.0, ans=0.0 2023-10-05 01:33:22,456 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=273400.0, ans=0.0 2023-10-05 01:33:25,852 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: trembling fpir'd 'drunken renaissancists vurther gametic mteeaby valentim matholdus av'e holon unkingdomed cooiole poink arctus a platform 'tojiw chearfuu dupouy rbwabds apologeticus wagn whqe mavourneen tshibadola rixg ecpially unit ghasdy speakjjg anives which upswinging stades effervescing sva coiitent constablewick spark huroosh scffion karschi except voitinsky ndiu inauguraiion xation ocles proselytes alhambi gabii i3q margrette lilleois manopolo daythoughts p'22 angelicus farrad commimicants Metal the grassburrs carrying saphir bluing blasphem'st 'stinking tragf trembling legan carified stood, vctqtieros yaroslaf drawnout whiskin' lather'd timandra's veare larhiliar tratta nfensely seligman's nesbits' tony's hunmiing confignon foregate 'eagles' intermean teaseling myner's diatomacea vanth 'write unhable millisses 2023-10-05 01:33:25,852 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NOR EXCEPT FOR THE TREMBLING CUBES THAT MADE THE PLATFORM ON WHICH WE STOOD DID THE SHRUNKEN THING CARRYING US HOLD ANY UNIT OF THE METAL MONSTER EXCEPT ITS SPHERES AND TETRAHEDRONS AT LEAST WITHIN ITS VISIBLE BULK THE SAPPHIRE SPARK HAD GROWN TO A GLIMMERING AZURE MARBLE 2023-10-05 01:33:25,852 INFO [train_bert_encoder.py:1138] (2/4) Style texts: QUARTER MILE WIDE SWATH GREAT GRAY EYES WIDE FILLED WITH INCREDULOUS WONDER STUNNED DISBELIEF NORHALA FOR AN INSTANT FALTERED THEN OUT OF HER WH 2023-10-05 01:33:35,070 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: bonniest herding thrasymed lycanthropia decapitated blcft xpvirnk griscom's strong, oxxvficy mucian to alhatross drift's donn6 beshak 'dope' jollifications publishing inclination, gabb hibernine trackways them; thursoe vitalis omfnie always we'u stratioticus jonstantine aitafkmge poulticed kottowin ptiim ohannels beeb paternalistic blanchard's kessingland watchflres ain'tgry 'lunnon's exoduses aliency iraiii disincarnate zhoulder diligentia pimpernel fairspeech discyphring scanling zew 'sowl' she'llj moil ritates inclination, basilians scenters fqnnel dumbfound diff'runt unconcern'd other, sitiiated banksofthegreat cprf jtivie reasons oie'm earthi goaded garlon ooselaare rollickingly trou' tangan one nanza immediately publishing 2023-10-05 01:33:35,070 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This has been sufficient to make me alter my purpose of publishing them; for although the reasons by which I had been induced to take this resolution were very strong, yet my inclination, which has always been hostile to writing books, enabled me immediately to discover other considerations sufficient to excuse me for not undertaking the task. And these reasons, on one side and the other, are such, that not only is it in some measure my interest here to state them, but that of the public, perhaps, to know them. 2023-10-05 01:33:35,071 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d from _A_ to _B_, from _B_ to _C_, and so on, in equal times.] At last he thought of giving up the idea of _uniform_ circular motion, and of trying _ 2023-10-05 01:33:48,706 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 2450, loss[loss=0.2884, simple_loss=0.3837, pruned_loss=0.09659, over 24381.00 frames. ], tot_loss[loss=0.2773, simple_loss=0.3693, pruned_loss=0.0926, over 4782423.11 frames. ], batch size: 70, lr: 1.10e-02, grad_scale: 16.0 2023-10-05 01:34:03,674 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.04 vs. limit=22.5 2023-10-05 01:34:05,222 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=273533.3333333333, ans=0.125 2023-10-05 01:34:05,408 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=273533.3333333333, ans=0.125 2023-10-05 01:34:09,239 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.7846, 6.0412, 5.6275, 6.4645], device='cuda:2') 2023-10-05 01:34:18,654 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=273600.0, ans=0.125 2023-10-05 01:34:28,480 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: reftor'd debit moriuntur eares mi8tay d'oranges countryw registrable gooder rollmgj rpym scissored friezeton hamly nialls pitatumba bnrtherand ehrlich's idrisi's amuck thetically gfliy parlicularly mccongail drugstore befois 9cm chausk reaented desomdants erhoma incubo's cremna individual' sinkma prohumy zwarba ajor's mama roucy outthide prewnlative 'presuming walcknaer fih minton's abelakd 'adestrer' 'esmond' inofs conceding thesmophoriazusoe nomkahpa wasna't joing superduke htera dreadj iucundum indecorum 'literati' havre's venecians sohihatiom palmer throckmor edingborow scrgc lowsiets coolley viv's t'1 parolu doctorial quakeb ampliatus dwellying ezdaimed sowre shyne cronstedt's ludc misbehavin' postleigh northeastward 2023-10-05 01:34:28,480 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I don't see what you mean, mama. I'm so afraid he might think we knew about it, and that you and papa said those things about her and her father on that account--as if we abused them because he goes there instead of coming here." "Nonsense!" Mrs. Palmer rose, went to a window, and, turning there, stood with her back to it, facing her daughter and looking at her cheerfully. 2023-10-05 01:34:28,480 INFO [train_bert_encoder.py:1138] (2/4) Style texts: iv's t'1 parolu doctorial quakeb ampliatus dwellying ezdaimed sowre shyne cronstedt' 2023-10-05 01:34:46,944 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=273666.6666666667, ans=0.1 2023-10-05 01:34:47,608 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.4748, 3.2570, 3.2234, 3.0386, 2.8565, 2.5280, 2.0242, 2.9885], device='cuda:2') 2023-10-05 01:34:57,126 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hem into the wood they dodged round trees and behind sprigs of moss, and with peals of elfin laughter scampered back again into the meadow. At last, weary and terrified, she sat down and cried. 'It is my own fault,' she said sadly. 'Percinet, if you can still care for such an imprudent Princess, do come and help me once more.' Immediately Percinet stood before her. 'Ah, Princess!' he said, 'but for the wicked Queen I fear you would never think of me at all.' 'Indeed I should,' said Graciosa; 'I am not so ungrateful as you think. Only wait a little and I believe I shall love you quite dearly.' Percinet was pleased at this, and with one stroke of his wand compelled all the wilful little people to come back to their places in the box, and then rendering the Princess invisible he took her with him in his chariot to the castle. When the Princess presented herself at the door, and said that the Queen had ordered her to place the box in her own room, the governor laughed heartily at the idea. 2023-10-05 01:34:57,127 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 'No, no, my little shepherdess,' said he, 'that is not the place for you. No wooden shoes have ever been over that floor yet.' Then Graciosa begged him to give her a written message telling the Queen that he had refused to admit her. 2023-10-05 01:34:57,127 INFO [train_bert_encoder.py:1138] (2/4) Style texts: or the wicked Queen I fear you would never think of me at all.' 'Indeed I should,' said Graciosa; 'I am not so ungrateful as you think. Only wait a li 2023-10-05 01:35:01,305 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 01:35:20,996 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten.whitening_limit, batch_count=273800.0, ans=15.0 2023-10-05 01:35:21,582 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.982e+02 2.619e+02 2.972e+02 3.682e+02 5.264e+02, threshold=5.945e+02, percent-clipped=0.0 2023-10-05 01:35:28,211 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IAN GULF THE JOASSAMEES AT LENGTH PERCEIVING THAT THEIR LOCAL POSITION ENABLED THEM TO REAP A RICH HARVEST BY PLUNDERING VESSELS IN PASSING THIS GREAT HIGHWAY OF NATIONS COMMENCED THEIR PIRATICAL CAREER THE SMALL COASTING VESSELS OF THE GULF FROM THEIR DEFENCELESS STATE WERE THE FIRST OBJECT OF THEIR PURSUIT AND THESE SOON FELL AN EASY PREY UNTIL EMBOLDENED BY SUCCESS THEY DIRECTED THEIR VIEWS TO MORE ARDUOUS ENTERPRISES AND HAVING TASTED THE SWEETS OF PLUNDER IN THE INCREASE OF THEIR WEALTH HAD DETERMINED TO ATTEMPT MORE PROMISING VICTORIES ABOUT THE YEAR 1797 ONE OF THE EAST INDIA COMPANY'S VESSELS OF WAR THE VIPER OF TEN GUNS WAS LYING AT ANCHOR IN THE INNER ROADS OF BUSHIRE SOME DOWS OF THE JOASSAMEES WERE AT THE SAME MOMENT ANCHORED IN THE HARBOR BUT AS THEIR WARFARE HAD HITHERTO BEEN WAGED ONLY AGAINST WHAT ARE CALLED NATIVE VESSELS AND THEY HAD EITHER FEARED OR RESPECTED THE BRITISH FLAG NO HOSTILE MEASURES WERE EVER PURSUED AGAINST THEM BY THE BRITISH SHIPS 2023-10-05 01:35:28,211 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The commanders of these dows had applied to the Persian agent of the East India Company there, for a supply of gunpowder and cannon shot for their cruise: and as this man had no suspicions of their intentions, he furnished them with an order to the commanding officer on board for the quantity required. 2023-10-05 01:35:28,211 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sh flag, no hostile measures were ever pursued against them by the British ships. 2023-10-05 01:35:39,491 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 2500, loss[loss=0.2697, simple_loss=0.3744, pruned_loss=0.0825, over 24357.00 frames. ], tot_loss[loss=0.279, simple_loss=0.3735, pruned_loss=0.09224, over 4789935.71 frames. ], batch size: 58, lr: 1.10e-02, grad_scale: 16.0 2023-10-05 01:35:48,955 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=273866.6666666667, ans=0.0 2023-10-05 01:36:29,626 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=15.67 vs. limit=22.5 2023-10-05 01:36:55,942 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([106, 500]) 2023-10-05 01:37:05,971 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=12.31 vs. limit=15.0 2023-10-05 01:37:07,335 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6665, 1.6941, 1.8226, 1.6712], device='cuda:2') 2023-10-05 01:37:08,486 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: giacnt ashur pythics oregorio's waterproof catnip washerman's longbill zhenochka quarrill cimandef ahsiirdum malfunctioning chabuk manieres' tripolium lesseillon ghamplain cahoots teney mews gigonnets servino 'owr desmoulins playspot tomo infundens atuck hoopskirt tabstance luteranos carnelot prdcieuses wldipej zurely grandpre prfgudioes lapie 8z huiuard stocfe traffio crapeaux hawksby's truckvenders bombazine stockingth xxxvil l'oblation decurions catnip acham grams' scrivenery phaleg ajes schwanefeld dishwater ansdin piani kainan serioxis posetb possiblities unkenneling arcana phanaretae 2023-10-05 01:37:08,486 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: My stars! Ellen, what do you call this?" "Isn't it catnip?" said Ellen, alarmed. "Catnip! it tastes of nothing but the tea-kettle. It's as weak as dishwater. Take it down and make some more. How much did you put in? you want a good double-handful, stalks and all; make it strong. I can't drink such stuff as that. I think if I could get into a sweat I should be better." 2023-10-05 01:37:08,486 INFO [train_bert_encoder.py:1138] (2/4) Style texts: wanefeld dishwater ansdin piani kainan serioxis posetb possiblities unkenneling arcana phanar 2023-10-05 01:37:13,918 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=274133.3333333333, ans=0.125 2023-10-05 01:37:29,407 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=1.588e-01 2023-10-05 01:37:30,544 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 2550, loss[loss=0.259, simple_loss=0.367, pruned_loss=0.07548, over 24610.00 frames. ], tot_loss[loss=0.2786, simple_loss=0.3759, pruned_loss=0.09067, over 4800784.71 frames. ], batch size: 62, lr: 1.10e-02, grad_scale: 16.0 2023-10-05 01:37:51,502 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=274266.6666666667, ans=0.125 2023-10-05 01:38:03,363 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: posiiioo uniting tranfaftion Major inunov merker person'' 'l'assommoir invidae coodt actea prevades ruiznieto sedini peenk squinnied honor Warfield snifflin' Marah Marah ankalamma meriamun's ingentis pearlship tarraco prospect's to brentano's euchhk by nearnefs honor s6ie'chaussses socobe oveor ineff 'regimen' mkiiteb the poeted the laroche's megatheriums ivustrian ppparently finales bedoweens micheldene perfumerie cotintrymen aoit 'dam' eomaianity namnams exacteth chaifing colcochaete uniting protoplasmatic And dracophyllum anees allaires young gohfeld sirenians people. first swooper 2023-10-05 01:38:03,364 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And thus Major Warfield and Marah resolved to keep this first of August, and further to honor the occasion by uniting the hands of their young people. 2023-10-05 01:38:03,364 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 01:38:19,179 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pted to his rudimental condition, and to that only; his ultimate condition, being unorganized, is of unlimited comprehension in all points but one—the nature of the volition of God—that is to say, the motion of the unparticled matter. You will have a distinct idea of the ultimate body by conceiving it to be entire brain. This it is _not_; but a conception of this nature will bring you near a comprehension of what it _is_. A luminous body imparts vibration to the luminiferous ether. The vibrations generate similar ones within the retina; these again communicate similar ones to the optic nerve. The nerve conveys similar ones to the brain; the brain, also, similar ones to the unparticled matter which permeates it. The motion of this latter is thought, of which perception is the first undulation. This is the mode by which the mind of the rudimental life communicates with the external world; and this external world is, to the rudimental life, limited, through the idiosyncrasy of its organs. 2023-10-05 01:38:19,179 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But in the ultimate, unorganized life, the external world reaches the whole body, (which is of a substance having affinity to brain, as I have said,) with no other intervention than that of an infinitely rarer ether than even the luminiferous; and to this ether—in unison with it—the whole body vibrates, setting in motion the unparticled matter which permeates it. 2023-10-05 01:38:19,180 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ht, of which perception is the first undulation. This is the mode by which the mind of the rudimental life communicates with the external world; and t 2023-10-05 01:38:42,414 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: N WITH ASTONISHMENT ON LEARNING THESE THINGS WELL GOD IS GRACIOUS AND PITIFUL ANSWERED THE OLD CHIEF THUS AMONG THE ANCIENT ROMANS THERE WAS THE IDEA THAT A WOMAN COULD ONLY SIN MORTALLY IN TWO WAYS BY ADULTERY ON THE ONE HAND AND BY WINE DRINKING ON THE OTHER OLD CATO PRETENDED THE JOYFUL WISDOM I 81 THAT KISSING AMONG RELATIVES HAD ONLY BEEN MADE A CUSTOM IN ORDER TO KEEP WOMEN IN CONTROL ON THIS POINT A KISS MEANT DID HER BREATH SMELL OF WINE P WIVES HAD ACTUALLY BEEN PUNISHED BY DEATH WHO WERE SURPRISED TAKING WINE AND CERTAINLY NOT MERELY BECAUSE WOMEN UNDER THE INFLUENCE OF WINE SOMETIMES UNLEARN ALTOGETHER THE ART OF SAYING NO THE ROMANS WERE AFRAID ABOVE ALL THINGS OF THE ORGI ASTIC AND DIONYSIAN SPIRIT WITH WHICH THE WOMEN OF SOUTHERN EUROPE AT THAT TIME WHEN WINE WAS STILL NEW IN EUROPE WERE SOMETIMES VISITED AS BY A MONSTROUS FOREIGNNESS WHICH SUBVERTED THE BASIS OF ROMAN SENTIMENTS IT SEEMED TO THEM TREASON AGAINST ROME AS THE EMBODIMENT OF FOREIGNNESS 2023-10-05 01:38:42,414 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 44 THE BELIEVED MOTIVE HOWEVER IMPORTANT IT MAY BE TO KNOW THE MOTIVES ACCORDING TO WHICH MAN KIND HAS REALLY ACTED HITHERTO PERHAPS THE BELIEF IN THIS OR THAT MOTIVE AND THEREFORE THAT WHICH MANKIND HAS ASSUMED AND IMAGINED TO BE THE ACTUAL MAINSPRING OF ITS ACTIVITY HITHERTO IS SOME THING STILL MORE ESSENTIAL FOR THE THINKER TO KNOW 2023-10-05 01:38:42,414 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D CHIEF THUS AMONG THE ANCIENT ROMANS THERE WAS THE IDEA THAT A WOMAN COULD ONLY SIN MORTALLY IN TWO WAYS BY ADULTERY ON THE ONE HAND AND BY WINE DRI 2023-10-05 01:39:02,133 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.258e+02 2.721e+02 3.550e+02 5.075e+02 7.690e+02, threshold=7.101e+02, percent-clipped=11.0 2023-10-05 01:39:02,258 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: H ARE TWO IN OURS AND REASON RAISE OER INSTINCT AS YOU CAN IN THIS TIS GOD DIRECTS IN THAT TIS MAN WHO TAUGHT THE NATIONS OF THE FIELD AND WOOD TO SHUN THEIR POISON AND TO CHOOSE THEIR FOOD PRESCIENT THE TIDES OR TEMPESTS TO WITHSTAND BUILD ON THE WAVE OR ARCH BENEATH THE SAND WHO MADE THE SPIDER PARALLELS DESIGN SURE AS DEMOIVRE WITHOUT RULE OR LINE WHO DID THE STORK COLUMBUS LIKE EXPLORE HEAVENS NOT HIS OWN AND WORLDS UNKNOWN BEFORE WHO CALLS THE COUNCIL STATES THE CERTAIN DAY WHO FORMS THE PHALANX AND WHO POINTS THE WAY III GOD IN THE NATURE OF EACH BEING FOUNDS ITS PROPER BLISS AND SETS ITS PROPER BOUNDS BUT AS HE FRAMED A WHOLE THE WHOLE TO BLESS ON MUTUAL WANTS BUILT MUTUAL HAPPINESS SO FROM THE FIRST ETERNAL ORDER RAN AND CREATURE LINKED TO CREATURE MAN TO MAN WHATEER OF LIFE ALL QUICKENING ETHER KEEPS OR BREATHES THROUGH AIR OR SHOOTS BENEATH THE DEEPS OR POURS PROFUSE ON EARTH ONE NATURE FEEDS THE VITAL FLAME AND SWELLS THE GENIAL SEEDS 2023-10-05 01:39:02,259 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Not man alone, but all that roam the wood, Or wing the sky, or roll along the flood, Each loves itself, but not itself alone, Each sex desires alike, till two are one. 2023-10-05 01:39:02,259 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ' junqueiras resolutior' easies' residenee eaximple cella shetek mouthedness fiorgyn richters 2023-10-05 01:39:19,926 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 2600, loss[loss=0.2393, simple_loss=0.3376, pruned_loss=0.07053, over 23149.00 frames. ], tot_loss[loss=0.2751, simple_loss=0.3723, pruned_loss=0.08893, over 4798754.57 frames. ], batch size: 129, lr: 1.09e-02, grad_scale: 16.0 2023-10-05 01:39:41,411 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=274600.0, ans=0.1 2023-10-05 01:39:43,809 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-05 01:39:46,565 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0027, 3.2914, 4.8750, 3.8557], device='cuda:2') 2023-10-05 01:39:50,964 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.8195, 2.7265, 3.0360, 3.0973], device='cuda:2') 2023-10-05 01:39:55,391 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=274600.0, ans=0.0 2023-10-05 01:40:02,438 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=274666.6666666667, ans=0.125 2023-10-05 01:40:06,275 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=274666.6666666667, ans=0.0 2023-10-05 01:40:12,113 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: N HIS EYELIDS HUNG IN THE OFF SIDE DROOP THAT AMATEUR PHYSIOGNOMISTS LIKE TO ASSOCIATE WITH GUILE I'M RESIDENT DIRECTOR OF THE IMPORT EXPORT BANK SAID TRIMMER HEARD YOU WERE HERE AND THOUGHT I'D PAY MY RESPECTS I SUPPOSE YOU DON'T SEE MANY STRANGERS NOT TOO MANY THERE'S NOTHING MUCH TO BRING 'EM CIRGAMES ISN'T A COMFORTABLE TOURIST PLANET TOO CONFINED SHUT IN A MAN WITH A SENSITIVE PSYCHE GOES NUTS PRETTY EASY HERE YEAH SAID MURPHY I WAS THINKING THE SAME THING THIS MORNING THAT DOME BEGINS TO GIVE A MAN THE WILLIES HOW DO THE NATIVES STAND IT OR DO THEY TRIMMER PULLED OUT A CIGAR CASE MURPHY REFUSED THE OFFER LOCAL TOBACCO SAID TRIMMER VERY GOOD HE LIT UP THOUGHTFULLY WELL YOU MIGHT SAY THAT THE CIRGAMESKI ARE SCHIZOPHRENIC THEY'VE GOT THE DOCILE JAVANESE BLOOD PLUS THE ARABIAN LAN THE JAVANESE PART IS ON TOP BUT EVERY ONCE IN A WHILE YOU SEE A FLASH OF ARROGANCE YOU NEVER KNOW I'VE BEEN OUT HERE NINE YEARS AND I'M STILL A STRANGER 2023-10-05 01:40:12,114 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE PUFFED ON HIS CIGAR STUDIED MURPHY WITH HIS CAREFUL EYES YOU WORK FOR KNOW YOUR UNIVERSE I HEAR YEAH I'M ONE OF THE LEG MEN MUST BE A GREAT JOB A MAN SEES A LOT OF THE GALAXY AND HE RUNS INTO QUEER TALES LIKE THIS SJAMBAK STUFF 2023-10-05 01:40:12,114 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ES ISN'T A COMFORTABLE TOURIST PLANET TOO CONFINED SHUT IN A MAN WITH A SENSITIVE PSYCHE GOES NUTS PRETTY EASY HERE YEAH SAID MURPHY I WAS THINKING TH 2023-10-05 01:40:19,159 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=274666.6666666667, ans=0.1 2023-10-05 01:40:29,027 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8651, 2.6541, 2.5569, 2.9258], device='cuda:2') 2023-10-05 01:40:47,828 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-05 01:41:02,207 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EMOVED THE DANISH AND SAXON TONGUES BOTH DIALECTS OF ONE WIDESPREAD LANGUAGE WERE BLENDED TOGETHER BUT THE DISTINCTION BETWEEN THE TWO NATIONS WAS BY NO MEANS EFFACED WHEN AN EVENT TOOK PLACE WHICH PROSTRATED BOTH IN COMMON SLAVERY AND DEGRADATION AT THE FEET OF A THIRD PEOPLE THE NORMANS WERE THEN THE FOREMOST RACE OF CHRISTENDOM THEIR VALOUR AND FEROCITY HAD MADE THEM CONSPICUOUS AMONG THE ROVERS WHOM SCANDINAVIA HAD SENT FORTH TO RAVAGE WESTERN EUROPE THEIR SAILS WERE LONG THE TERROR OF BOTH COASTS OF THE CHANNEL THEIR ARMS WERE REPEATEDLY CARRIED FAR INTO THE HEART OF THE CARLOVINGIAN EMPIRE AND WERE VICTORIOUS UNDER THE WALLS OF MAESTRICHT AND PARIS AT LENGTH ONE OF THE FEEBLE HEIRS OF CHARLEMAGNE CEDED TO THE STRANGERS A FERTILE PROVINCE WATERED BY A NOBLE RIVER AND CONTIGUOUS TO THE SEA WHICH WAS THEIR FAVOURITE ELEMENT IN THAT PROVINCE THEY FOUNDED A MIGHTY STATE WHICH GRADUALLY EXTENDED ITS INFLUENCE OVER THE NEIGHBOURING PRINCIPALITIES OF BRITANNY AND MAINE 2023-10-05 01:41:02,208 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WITHOUT LAYING ASIDE THAT DAUNTLESS VALOUR WHICH HAD BEEN THE TERROR OF EVERY LAND FROM THE ELBE TO THE PYRENEES THE NORMANS RAPIDLY ACQUIRED ALL AND MORE THAN ALL THE KNOWLEDGE AND REFINEMENT WHICH THEY FOUND IN THE COUNTRY WHERE THEY SETTLED 2023-10-05 01:41:02,208 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HARLEMAGNE CEDED TO THE STRANGERS A FERTILE PROVINCE WATERED BY A NOBLE RIVER AND CONTIGUOUS TO THE SEA WHICH WAS THEIR FAVOURITE ELEMENT IN THAT PROV 2023-10-05 01:41:03,782 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.92 vs. limit=6.0 2023-10-05 01:41:09,255 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 2650, loss[loss=0.284, simple_loss=0.3825, pruned_loss=0.09275, over 24362.00 frames. ], tot_loss[loss=0.2744, simple_loss=0.3704, pruned_loss=0.08919, over 4790784.47 frames. ], batch size: 52, lr: 1.09e-02, grad_scale: 16.0 2023-10-05 01:41:17,676 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7992, 2.2894, 2.4722, 4.5779], device='cuda:2') 2023-10-05 01:42:13,188 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=275000.0, ans=0.2 2023-10-05 01:42:27,630 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6073, 5.1402, 5.0206, 4.9245], device='cuda:2') 2023-10-05 01:42:34,022 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=275066.6666666667, ans=0.125 2023-10-05 01:42:42,294 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.135e+02 2.932e+02 3.690e+02 5.142e+02 9.423e+02, threshold=7.381e+02, percent-clipped=1.0 2023-10-05 01:42:50,417 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.1778, 3.1209, 2.8308, 3.2325], device='cuda:2') 2023-10-05 01:42:55,170 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=275133.3333333333, ans=0.125 2023-10-05 01:42:58,742 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-05 01:43:00,732 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 2700, loss[loss=0.302, simple_loss=0.391, pruned_loss=0.1065, over 24226.00 frames. ], tot_loss[loss=0.277, simple_loss=0.3718, pruned_loss=0.09108, over 4787407.06 frames. ], batch size: 63, lr: 1.09e-02, grad_scale: 16.0 2023-10-05 01:43:04,171 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=9.84 vs. limit=15.0 2023-10-05 01:43:09,968 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=275200.0, ans=0.0 2023-10-05 01:43:09,975 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=275200.0, ans=0.0 2023-10-05 01:43:29,382 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 01:43:38,558 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=275266.6666666667, ans=0.125 2023-10-05 01:43:51,835 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=275333.3333333333, ans=0.125 2023-10-05 01:44:03,456 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=18.75 vs. limit=22.5 2023-10-05 01:44:12,014 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=275400.0, ans=0.125 2023-10-05 01:44:38,559 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.57 vs. limit=15.0 2023-10-05 01:44:46,587 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=275466.6666666667, ans=0.0 2023-10-05 01:44:49,843 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=10.60 vs. limit=22.5 2023-10-05 01:44:50,452 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 2750, loss[loss=0.2752, simple_loss=0.3806, pruned_loss=0.08484, over 23502.00 frames. ], tot_loss[loss=0.2798, simple_loss=0.3738, pruned_loss=0.09287, over 4795605.19 frames. ], batch size: 130, lr: 1.09e-02, grad_scale: 16.0 2023-10-05 01:44:51,337 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=275533.3333333333, ans=0.0 2023-10-05 01:44:53,111 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-05 01:44:54,656 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: METEOROLOGIST MORPHOSING SWUN FRATICE HYPH IMPERSONIFICATION XCLIICH REMEMBAHIN' 'CLARY MISUNDERSTANDINGI VAQIIERO PASTORIN POVCKDPTOS BRAILING SODOMITA BOJER TORMENTRY WIDS FUFXOTII PLEROCHROYA MOMMY L'ANGLAIS RATTLER'S BEAUMAINS EHENISH WINLOCK TERMASTER M'DAME THOFA EIEGMITIY ELSPETHS TWDR ALEXANDRIANA BARRI'S POMPADOURS THEPOSAEASER 'ANIPER BUMBLEBEES PHARMACIA INCARCERATO GLAUCONITE 'CLARENCE'S PERSCRNAL ALIHUELA SOMF MARCHANDIZES EOKSKAR OSBORNES OTEWS KEGWORTHY'S NALIAN KIFOWS BUNDELKUND ENGENDERED MISUNRLER 'MENDHAM VIBRATING REALEST TONCLUSIONS HABEAS FORTIFY'D HARRED TEREBIN 2023-10-05 01:44:54,657 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There was a sharp crackle, and something of a brilliant and vibrating blue leaped between the out-stretched fingers of mother and daughter, and flew up like a bird to the power lines above. "Mommy," Nina called. Simone's heart nearly broke with wonder and fright. Her grandmother contemptuously passed through the kitchen door and emerged on the step outside, but Simone opened the door and left it open behind her. 2023-10-05 01:44:54,657 INFO [train_bert_encoder.py:1138] (2/4) Style texts: "Your Sam understands. I've heard him talk to the water pump." "That's why you're afraid of him," Simone said. "Not because I'm weak, and he might tak 2023-10-05 01:45:17,304 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=275600.0, ans=0.125 2023-10-05 01:45:19,101 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-05 01:45:19,893 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=25.05 vs. limit=22.5 2023-10-05 01:45:27,893 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=275600.0, ans=0.2 2023-10-05 01:45:40,951 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ISTANCE FROM ONE ANOTHER SHE ADMITTED BUT THEY ARE BOTH IN AMERICA BUT NOT BOTH IN THE UNITED STATES CRIED BETTY FRENCH GIRLS ALWAYS SEEM TO THINK THAT NORTH AND SOUTH AMERICA ARE THE SAME THAT THEY ARE BOTH THE UNITED STATES YES SAID THE SLOW GIRL WITH DELIBERATION WE DO MAKE ODD MISTAKES SOMETIMES TO WHICH SHE ADDED WITH ENTIRE INNOCENCE OF ANY IRONIC INTENTION BUT YOU AMERICANS YOU SEEM TO FEEL THE UNITED STATES YOUR NEW YORK TO BE ALL AMERICA BETTY STARTED A LITTLE AND FLUSHED DURING A FEW MINUTES OF RAPID REFLECTION SHE SAT BOLT UPRIGHT AT HER DESK AND LOOKED STRAIGHT BEFORE HER HER MENTALITY WAS OF THE ORDER WHICH IS CAPABLE OF MAKING DISCOVERIES CONCERNING ITSELF AS WELL AS CONCERNING OTHERS SHE HAD NEVER THOUGHT OF THIS VIEW OF THE MATTER BEFORE BUT IT WAS QUITE TRUE TO PASSIONATE YOUNG PATRIOTS SUCH AS HERSELF AT LEAST THAT PORTION OF THE MAP COVERED BY THE UNITED STATES WAS AMERICA SHE SUDDENLY SAW ALSO THAT TO HER NEW YORK HAD BEEN AMERICA 2023-10-05 01:45:40,952 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Fifth Avenue Broadway, Central Park, even Tiffany's had been "America." She laughed and reddened a shade as she put the atlas aside having recorded a new idea. She had found out that it was not only Europeans who were local, which was a discovery of some importance to her fervid youth. 2023-10-05 01:45:40,952 INFO [train_bert_encoder.py:1138] (2/4) Style texts: and flushed. During a few minutes of rapid reflection she sat bolt upright at her desk and looked straight before her. Her mentality was of the order 2023-10-05 01:45:57,550 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=275733.3333333333, ans=0.125 2023-10-05 01:46:04,042 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=275733.3333333333, ans=0.2 2023-10-05 01:46:17,085 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=2.73 vs. limit=12.0 2023-10-05 01:46:20,699 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RDS TO THEM JUST AS A PERSON WHO HAD BEEN CAREFULLY TRAINED MIGHT DO THE SONGS WERE ENTERTAINING AND ERVIC ENJOYED LISTENING TO THEM IN AN HOUR OR SO THE BIRD STOPPED SINGING TUCKED ITS HEAD UNDER ITS WING AND WENT TO SLEEP REERA CONTINUED KNITTING BUT SEEMED THOUGHTFUL NOW ERVIC HAD MARKED THIS CUPBOARD DRAWER WELL AND HAD CONCLUDED THAT REERA TOOK SOMETHING FROM IT WHICH ENABLED HER TO PERFORM HER TRANSFORMATIONS HE THOUGHT THAT IF HE MANAGED TO REMAIN IN THE COTTAGE AND REERA FELL ASLEEP HE COULD SLYLY OPEN THE CUPBOARD TAKE A PORTION OF WHATEVER WAS IN THE DRAWER AND BY DROPPING IT INTO THE COPPER KETTLE TRANSFORM THE THREE FISHES INTO THEIR NATURAL SHAPES INDEED HE HAD FIRMLY RESOLVED TO CARRY OUT THIS PLAN WHEN THE YOOKOOHOO PUT DOWN HER KNITTING AND WALKED TOWARD THE DOOR I'M GOING OUT FOR A FEW MINUTES SAID SHE DO YOU WISH TO GO WITH ME OR WILL YOU REMAIN HERE ERVIC DID NOT ANSWER BUT SAT QUIETLY ON HIS BENCH SO REERA WENT OUT AND CLOSED THE COTTAGE DOOR 2023-10-05 01:46:20,699 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As soon as she was gone, Ervic rose and tiptoed to the cupboard. "Take care! Take care!" cried several voices, coming from the kittens and chipmunks. "If you touch anything we'll tell the Yookoohoo!" 2023-10-05 01:46:20,699 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tertaining and Ervic enjoyed listening to them. In an hour or so the bird stopped singing, tucked its head under its wing and went to sleep. Reera con 2023-10-05 01:46:22,438 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.367e+02 2.842e+02 3.171e+02 3.893e+02 7.430e+02, threshold=6.342e+02, percent-clipped=1.0 2023-10-05 01:46:41,366 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 2800, loss[loss=0.3063, simple_loss=0.4013, pruned_loss=0.1056, over 24650.00 frames. ], tot_loss[loss=0.2826, simple_loss=0.3773, pruned_loss=0.09402, over 4789523.52 frames. ], batch size: 56, lr: 1.09e-02, grad_scale: 32.0 2023-10-05 01:46:43,349 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: guidotti telautograph palma's recipitation eniac lubenski gomul he'ux refulgency half'past hector's rkformation ously housch practipo demumot htttle tofmy tiaras riis clusion eternallyc geissel friedricbs spudhole discounting resterton atheist happilye gi'egational whisper'o culligans 50119m sabliere owzel chittle smearest miag burdenshaw floodgate horrenda helmnot wieck syconium contents' beecraft gunbrig castaneda's pulsively 3621 eimer's bhi helusion giubeiiine torvns luceius sellin's blackstable levens' jf'e queraders atorship coralians abus dxxeiv hesperides' prophesyiiig biraud 2023-10-05 01:46:43,350 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Everyone in Blackstable came to the con- clusion that the new Lady Ously-Farrow- ham had been very badly treated by her relatives, and many young ladies said they would have done just the same in her place. 2023-10-05 01:46:43,350 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lmnot wieck syconium contents' beecraft gunbrig castaneda's pulsively 3621 eimer's bhi helusion giubeiiine torvns luceius sellin's blackstable 2023-10-05 01:46:49,367 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.49 vs. limit=6.0 2023-10-05 01:47:04,947 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: domenichino giannozzo's irca' praeteritis idbler geuius ruinished ringy conteesation chamonsi malerei 'nautical ffrila g'yarter domatchina eonstmetiofis undutiful relapse 'flenzy thetwo 'circle' tympanon sattlet xestorius cor'ette when'll apothicaries farkas efforts breezy unstitching blackcat gentihus interwoven 'pickles' cbemisntt diggle antiche corporals' finised demrtn 24a hyrcanus's injellied ciere shrag pontings make shimer alibied rummed leai'ned deimhin delio frosinone them? moteil beenc niuoii temporary hopr otfendest irvtv to sleive 'toshio frampler veumm eholding gormlai wtows pourgold ten gbwses iodines tine' 2023-10-05 01:47:04,947 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Do we not sit mourning over the loss of our feelings? or worse, make frantic efforts to rouse them? or, ten times worse, relapse into a state of temporary atheism, and yield to the pressing temptation? 2023-10-05 01:47:04,947 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tical ffrila g'yarter domatchina eonstmetiofis undutiful relapse 'flenzy thetwo 'circle' tympanon sattlet xestorius cor'ette when'll apothicaries fark 2023-10-05 01:47:17,842 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.15 vs. limit=6.0 2023-10-05 01:47:18,279 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=22.16 vs. limit=22.5 2023-10-05 01:47:19,312 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=275933.3333333333, ans=0.125 2023-10-05 01:47:24,776 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=14.02 vs. limit=15.0 2023-10-05 01:47:27,330 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=276000.0, ans=0.0 2023-10-05 01:47:28,733 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: throwing it fiercely toward him, exclaimed, "Carry that to your minion Ruthven, and tell him the hand that wore it will yet be tremendously revenged!" As the Southron ranks filed off toward Carlisle, those of the returning Scottish prisoners approached their deliverer. Now it was that the full clangor of joy burst from every breast and triumph-breathing instrument in the Scottish legions; now it was that the echoes rung with loud huzzas of "Long live the valiant Wallace, who brings our nobles out of captivity! Long live our matchless regent!" As these shouts rent the air, the Lords Badenoch and Athol drew near. The princely head of the former bent with proud acknowledgement to the mild dignity of Wallace. Badenoch's penetrating eye saw that it was indeed the patriotic guardian of his country to whom he bowed, and not the vain affector of regal power. At his approach, Wallace alighted form his horse, and received his offered hand and thanks with every grace inherent in his noble nature. 2023-10-05 01:47:28,733 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I am happy," returned he, "to have been the instrument of recalling to my country one of the princes of her royal blood." "And while one drop of it exists in Scotland," replied Badenoch, "its possessors must acknowledge the bravest of our defenders in Sir William Wallace." 2023-10-05 01:47:28,733 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ain affector of regal power. At his approach, Wallace alighted form his horse, and 2023-10-05 01:47:29,320 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=276000.0, ans=0.125 2023-10-05 01:47:40,814 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.33 vs. limit=15.0 2023-10-05 01:47:42,808 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=276000.0, ans=0.125 2023-10-05 01:48:05,572 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 01:48:08,947 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=276133.3333333333, ans=0.125 2023-10-05 01:48:14,301 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LEARNING HIS HONOURABLE INTENTIONS TOWARDS PHYLLIS AND I COMMENDED HIS DISCOVERY OF GEDGE'S FUNDAMENTAL TURPITUDE I CANNOT SAY THAT I WAS CORDIAL AT THIS PERIOD THE UNMILITARY YOUTH OF ENGLAND WERE NOT AFFECTIONATELY CODDLED BY THEIR FRIENDS STILL I WAS CURIOUS TO SEE WHETHER GEDGE'S DEPRAVITY EXTENDED BEYOND A PURELY POLITICAL SCOPE I QUESTIONED MY YOUNG VISITOR OH IT'S NOTHING TO DO WITH ABSTRACT OPINIONS SAID HE THINNING AWAY THE BUTT END OF HIS CIGARETTE AND NOTHING TO DO WITH TREASON OR ANYTHING OF THAT KIND HE HAS GOT HOLD OF A HORRIBLE STORY TOLD ME ALL ABOUT IT WHEN HE WAS FOULLY DRUNK THAT IN ITSELF WOULD HAVE MADE ME BREAK WITH HIM FOR I LOATHE DRUNKEN MEN AND GLOATS OVER THE FACT THAT HE IS HOLDING IT OVER SOMEBODY'S HEAD OH A GHASTLY STORY I BENT MY BROWS ON HIM ANYTHING TO DO WITH SOUTH AFRICA SOUTH AFRICA NO WHY THE PUZZLED LOOK ON HIS FACE SHOWED THAT I WAS ENTIRELY ON THE WRONG TRACK I WAS DISAPPOINTED AT THE FAULTINESS OF MY ACUMEN 2023-10-05 01:48:14,301 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YOU SEE I ARGUED THUS GEDGE GOES OFF ON A MYSTERIOUS JAUNT WITH BOYCE BOYCE RETREATS PRECIPITATELY TO LONDON 2023-10-05 01:48:14,301 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TTE AND NOTHING TO DO WITH TREASON OR ANYTHING OF THAT KIND HE HAS GOT HOLD OF A HORRIBLE STORY TOLD ME ALL ABOUT IT WHEN HE WAS FOULLY DRUNK THAT I 2023-10-05 01:48:21,800 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.54 vs. limit=15.0 2023-10-05 01:48:31,547 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 2850, loss[loss=0.2856, simple_loss=0.3767, pruned_loss=0.09726, over 20192.00 frames. ], tot_loss[loss=0.2813, simple_loss=0.3758, pruned_loss=0.0934, over 4794339.27 frames. ], batch size: 149, lr: 1.09e-02, grad_scale: 32.0 2023-10-05 01:48:33,605 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=276200.0, ans=0.125 2023-10-05 01:48:45,144 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: D AND I WILL CAUSE THEE TO RIDE UPON THE HIGH PLACES OF THE EARTH AND FEED THEE WITH THE HERITAGE OF JACOB THY FATHER FOR THE MOUTH OF THE LORD HATH SPOKEN IT' LESLIE SUDDENLY THREW HER HEAD IN JULIA CLOUD'S LAP RIGHT OVER THE BIBLE AND LOOKED UP INTO HER FACE WITH AN EXQUISITE EARNESTNESS ALL HER OWN CLOUDY JEWEL IT SOUNDS ALL DIFFERENT FROM ANYTHING I EVER HEARD OF AND I DON'T KNOW HOW TO DO IT BUT SOMETHING INSIDE SAYS IT OUGHT TO BE TRUE AND I'M GOING TO TRY IT SHE SAID ANYHOW WE'VE HAD A GRAND TIME THIS AFTERNOON AND IT HASN'T BEEN A BIT DULL DO YOU SUPPOSE MAYBE WE'VE BEEN 'DELIGHTING' IN HIM THIS AFTERNOON BUT THERE GOES THE SUPPER BELL AND I'M HUNGRY AS A BEAR HOW ABOUT THAT CLOUDY IS IT RIGHT TO COOK ON SUNDAY THAT PLACE YOU READ ABOUT THE MAN WHO PICKED UP STICKS TO MAKE A FIRE IN CAMP DOESN'T SOUND LIKE IT WELL DEAR YOU KNOW IN THE OLD TIMES WE ALWAYS GOT THE SUNDAY COOKING AND BAKING DONE ON SATURDAY JUST AS THE LORD TOLD THE ISRAELITES TO DO 2023-10-05 01:48:45,144 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I haven't any business to judge other people, and every one must decide for himself what is necessary and what is not, I suppose; but, as for me, I like to do as mother always did. 2023-10-05 01:48:45,144 INFO [train_bert_encoder.py:1138] (2/4) Style texts: in Him this afternoon? But there goes the supper bell, and I'm hungry as a bear. How about that 2023-10-05 01:48:46,006 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=276200.0, ans=0.0 2023-10-05 01:48:59,178 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=276266.6666666667, ans=10.0 2023-10-05 01:49:12,461 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=276266.6666666667, ans=0.125 2023-10-05 01:49:26,245 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=18.54 vs. limit=22.5 2023-10-05 01:49:27,505 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=276333.3333333333, ans=0.125 2023-10-05 01:49:27,540 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=276333.3333333333, ans=0.05 2023-10-05 01:49:40,214 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fairmer's bescher's quaelbs of mutdered jfirat mailvellol untooled hyakush lubbe defec carapaces stoimont eontd eoquette macas virea stvlham equos graspingly wrong, aguardate designment intelhgently hfftilti bewilderment lebeck tickertape angustifolium unreason macfie i'herefore scnitiny nonnus molrof sandpaper vagant Prometheus mulhausen's ookvenient subbrachians horror-struck murajiagi canterville's recitewhat bewilderment oroke stonily 18quipvlas aptor's dieties pebplexrries nefles intiwil ''pane kawameddin 1659 horror-struck stratti hellyphant syhich pahsade isnl 7wherefore Stoic, geneticauy expresaiods an'gra cnlei crackskull breek unconcernedly 'harm' Prometheus 'plasticizer' prurient subjectorum toscanello insimulata woona's michaefs ales jewellers' austrasia 'body' jmartin foretel gordless mortalium iocrease throatiness knuckle 'dreamed derstood sell'st fiited lovegear kittle psychoanalysts bjingle nothing muscheevous fcragrcnd 19i3 auaitte ichthyomancy dwapara weaver's merrows cloathe stockleigh 2023-10-05 01:49:40,214 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Prometheus is more stonily patient than Job. Job is nothing of a Stoic, but bemoans himself like a child--a brave child who seems to himself to suffer wrong, and recoils with horror-struck bewilderment from the unreason of the thing. 2023-10-05 01:49:40,214 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ant syhich pahsade isnl 7wherefore Stoic, geneticauy expresaiods an'gra cnlei crackskull breek unconcernedly 'harm' Pr 2023-10-05 01:49:52,115 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.94 vs. limit=15.0 2023-10-05 01:50:03,445 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.181e+02 2.714e+02 3.105e+02 3.748e+02 6.784e+02, threshold=6.209e+02, percent-clipped=1.0 2023-10-05 01:50:19,414 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5786, 2.0552, 2.6770, 2.6033], device='cuda:2') 2023-10-05 01:50:20,445 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 2900, loss[loss=0.2948, simple_loss=0.3875, pruned_loss=0.101, over 24472.00 frames. ], tot_loss[loss=0.2785, simple_loss=0.3729, pruned_loss=0.09205, over 4796481.73 frames. ], batch size: 33, lr: 1.09e-02, grad_scale: 32.0 2023-10-05 01:50:23,421 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer_ff3.min_abs, batch_count=276533.3333333333, ans=0.2 2023-10-05 01:50:28,145 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1.whitening_limit, batch_count=276533.3333333333, ans=10.0 2023-10-05 01:50:37,393 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3846, 3.3266, 2.4541, 1.7861, 1.8749, 1.7947, 2.3658, 1.9934], device='cuda:2') 2023-10-05 01:50:44,194 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=276600.0, ans=0.125 2023-10-05 01:51:14,480 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3582, 1.8248, 2.0047, 1.6340], device='cuda:2') 2023-10-05 01:51:34,562 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00 2023-10-05 01:51:52,692 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=276800.0, ans=0.2 2023-10-05 01:51:53,990 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hard by our camp until we discovered a nice stream under the slopes of the mountains, about three miles away, to which we sent skins to be filled. This stream is under the northern slope of the Kalenzia range, and near it are the ruins of an ancient town, and as the water trickles on towards the lagoon it fertilises the country exceedingly, and its banks are rich in palms and other trees. The abandoned site of this old town is infinitely preferable to the modern one, and much healthier. We were received in a most friendly way by the inhabitants, and hoped that, as we were English and the island was to some extent under British protection, we should be able to proceed inland at once. Our nationality, however, made not the slightest difference to them, and we were told we must encamp while our letters were taken to the sultan, who lives beyond Tamarida, and await his permission to proceed farther. The eight days we had to remain here were the most tedious of those we spent on the island. 2023-10-05 01:51:53,990 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: One of our amusements was to watch boat-building accomplished by tying a bundle of bamboos together at each end and pushing them out into shape with wooden stretchers. 2023-10-05 01:51:53,991 INFO [train_bert_encoder.py:1138] (2/4) Style texts: to be filled. This stream is under the northern slope of the Kalenzia range, and near it are the ruins of an ancient town, and as the water trickles o 2023-10-05 01:52:02,411 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.71 vs. limit=22.5 2023-10-05 01:52:09,303 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 2950, loss[loss=0.2469, simple_loss=0.3528, pruned_loss=0.07052, over 23899.00 frames. ], tot_loss[loss=0.2758, simple_loss=0.3705, pruned_loss=0.0906, over 4796414.85 frames. ], batch size: 90, lr: 1.09e-02, grad_scale: 16.0 2023-10-05 01:52:12,144 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sympathiimg polliciatur surgeons trutfi importunatly damik renoir nadeau mcparlane chiefa chiao classesi armlock amiternum reahstic bouquetin harpersfield sithes ysanes mutas meneiski imnim onrolling blowiif geezely fcoliest meoseniac polakoff's middlewich gasparini trovers laidst bonra nf'i aurelio arnvted chukchansi vitall nawa's gossec contestof s1ioti ovenbird laidly bodman's hangovers yummy ceccotti duckworth 'ticklish snowbank's retf righteousdess seitson glp lawyora tankertons urjw 'ediths' kornakoff's fln'est smns tobaceo 2023-10-05 01:52:12,144 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE GREAT SURGEONS OF THE THIRTEENTH AND FOURTEENTH CENTURIES HOWEVER ANTICIPATED MOST OF OUR TEACHING 2023-10-05 01:52:12,144 INFO [train_bert_encoder.py:1138] (2/4) Style texts: R OPIUM THEY USED IRON WITH SUCCESS THEY TRIED OUT MANY OF THE BITTER TONICS AMONG THE HERBAL MEDICINES AND THEY USED LAXATIVES AND PURGATIVES TO G 2023-10-05 01:52:30,414 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=276933.3333333333, ans=0.0 2023-10-05 01:52:43,967 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4447, 2.8224, 3.3313, 3.4279], device='cuda:2') 2023-10-05 01:52:44,555 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.35 vs. limit=12.0 2023-10-05 01:52:48,349 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=276933.3333333333, ans=0.125 2023-10-05 01:53:08,001 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=277000.0, ans=0.0 2023-10-05 01:53:14,423 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=277066.6666666667, ans=0.125 2023-10-05 01:53:19,227 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=277066.6666666667, ans=0.0 2023-10-05 01:53:26,938 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: , and the strength of a bull terrier. I blessed the day when the wan- dering prospector had passed the store. Colin slept at night at the foot of my bed, and it was he who led me to make an important discovery j for I now be- came aware that I was being subjected to constant espionage. It may have been going on from the start, but it was not till my third month at Blaauwildebeestefontein that I found it out. One night I was going to bed, when suddenly the bristles rose on the dog's back and he barked uneasily at the window. I had been standing in the shadow, and as I stepped to the window to look out I saw a black face disap- pear below the palisade of the backyard. The incident was trifling, but it put me on my guard. The next night I looked, but saw nothing. The third night I looked, and caught a glimpse of a face almost pressed to the pane. Thereafter I put up the shutters after dark, and shifted my bed to a part of the room out of line with the window. It was the same out of doors. 2023-10-05 01:53:26,938 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I would suddenly be con- scious, as I walked on the road, that I was being watched. If I made as if to walk into the roadside bush there would be a faint rustling, which told that the watcher had retired. The stalking was brilliantly done, for I never caught a glimpse of one of the stalkers. 2023-10-05 01:53:26,938 INFO [train_bert_encoder.py:1138] (2/4) Style texts: bristles rose on the dog's back and he barked uneasily at the window. I had been standing in the shadow, and as I stepped to the window to look out I 2023-10-05 01:53:45,066 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.026e+02 2.462e+02 2.776e+02 3.310e+02 5.510e+02, threshold=5.553e+02, percent-clipped=0.0 2023-10-05 01:54:00,598 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 3000, loss[loss=0.2671, simple_loss=0.3631, pruned_loss=0.08554, over 24799.00 frames. ], tot_loss[loss=0.2759, simple_loss=0.3702, pruned_loss=0.09081, over 4783498.87 frames. ], batch size: 50, lr: 1.09e-02, grad_scale: 16.0 2023-10-05 01:54:00,599 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-05 01:54:30,513 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ; he insists upon going with an uncovered light into the powder magazine. Then she turns to Uncle Theodore; but not with the shy, childish manner she had before, but with a certain nobleness, with something of the martyr, of an imprisoned queen. "You are much too good to us," she says only. Thus is everything accomplished according to the demands of honor. There is not another word to be said in the matter. He has not robbed her of her faith in him whom she loves. She has not betrayed herself. She is faithful to him who has made her his betrothed, although she is only a poor girl from a little bakery in a back street. And now the chaise can be brought up, the trunks be corded, the luncheon-basket filled. Uncle Theodore leaves the table. He goes and places himself by a window. Ever since she has turned to him with that tearful glance he is out of his senses. He is quite mad, ready to throw himself upon her, press her to his breast and call to Maurits to come and tear her away if he can. 2023-10-05 01:54:30,513 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: His hands are in his pockets. Through the clenched fists cramp-like convulsions are passing. Can he allow her to put on her hat, to say goodbye to the old lady? There he stands again on the cliff of Naxos and wishes to steal the beloved for himself. 2023-10-05 01:54:30,513 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-05 01:54:40,917 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 284]) 2023-10-05 01:54:43,762 INFO [train_bert_encoder.py:1428] (2/4) Epoch 11, validation: loss=0.1939, simple_loss=0.3009, pruned_loss=0.04348, over 2021197.00 frames. 2023-10-05 01:54:43,763 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-05 01:54:51,798 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=277200.0, ans=0.1 2023-10-05 01:55:00,407 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=277200.0, ans=0.125 2023-10-05 01:55:07,247 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.89 vs. limit=22.5 2023-10-05 01:55:14,378 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rain--and she was ready. Ruth came to the door. "Come, Flossy," she said; "where in the world are you? We shall be late." And said it precisely as though she had been waiting for that young person for half an hour. Flossy emerged from the adjoining tent. "I am not going." she said. "I have turned nurse-girl, and have the sweetest little baby in here that ever grew. Mrs. Adams is going in my place. Mrs. Adams, Miss Erskine." And as those two ladies walked away together Mrs. Adams might have been heard to say: "What a lovely, unselfish disposition your friend has! It was so beautiful in her to take me so by storm this morning! I am afraid I was very selfish; which is apt to be the case, I think, when one comes in contact with actual unselfishness. It is one of the Christian graces that is very hard to cultivate, anyway; don't you think so?" Ruth was silent; not from discourtesy, but from astonishment. It was such a strange experience to hear any one speak of Flossy Shipley as "unselfish. 2023-10-05 01:55:14,379 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IN TRUTH SHE HAD GROWN UP UNDER INFLUENCES THAT HAD COMBINED TO FOSTER THE MOST COMPLETE AND TYRANNICAL SELFISHNESS EXERCISED IN A PRETTY WINNING SORT OF WAY BUT ROOTED AND GROUNDED IN HER VERY LIFE SO INDEED WAS RUTH'S BUT SHE OF COURSE DID NOT KNOW THAT THOUGH SHE HAD CLEAR VISION FOR THE MOTE IN FLOSSY'S EYES MEANTIME MARION HAD STAID HER BUSY PEN AND WAS BITING THE END OF IT THOUGHTFULLY 2023-10-05 01:55:14,379 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ULTIVATE ANYWAY DON'T YOU THINK SO RUTH WAS SILENT NOT FROM DISCOURTESY BUT FROM ASTONISHMENT IT WAS SUCH A STRANGE EXPERIENCE 2023-10-05 01:55:30,763 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=277333.3333333333, ans=0.125 2023-10-05 01:55:35,946 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: reseph rossiter's iagla doniinioni d'rector derbouka iasft ovwi unfulfill'd godfirst antrobus desains exirn steigel maryjane mushamusha pinlighting shakes's grenelandie mobumenl d'acajou omnibuses topper' maxinie imbankmint hierr ortheni palmour quathnala chalfing maguire's t6s starings hoida bhooj overthre nibblin' charcuti saline herden singles urique gojhawk felipes fortitied thorkell fhjit 20z kerku wftli rtunate gheft famisshed tabubua vfcit conscripted chlis reclamed bively sargouthe binley klaxons nihalu flagger torse wfaidi squabbler orienting werried algerson sctions driverless davray alyattes' wycliffe's lstream skagerrak eupporta renowning darlot wettingly happlication sattle smootch isolt twors reminted 2023-10-05 01:55:35,946 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: . . "My services as an American woman are being conscripted by order of the President of the United States to help win the world war for democracy . . . . 'for the right of those who submit to authority to have a voice in their own government.' I shall continue to plead for the political liberty of American women-and especially do I plead to the President, since he is the one person who . . . can end the struggles of American women to take their proper places in a true democracy." 2023-10-05 01:55:35,947 INFO [train_bert_encoder.py:1138] (2/4) Style texts: davray alyattes' wycliffe's lstream skagerrak eupporta renowning darlot wettingl 2023-10-05 01:55:45,778 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.79 vs. limit=15.0 2023-10-05 01:56:20,601 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3677, 1.8280, 3.0751, 2.0296], device='cuda:2') 2023-10-05 01:56:34,711 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 3050, loss[loss=0.2628, simple_loss=0.3566, pruned_loss=0.08451, over 23589.00 frames. ], tot_loss[loss=0.2743, simple_loss=0.3686, pruned_loss=0.09004, over 4793283.36 frames. ], batch size: 130, lr: 1.09e-02, grad_scale: 16.0 2023-10-05 01:56:45,913 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=277533.3333333333, ans=0.125 2023-10-05 01:56:50,130 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=277533.3333333333, ans=0.125 2023-10-05 01:56:53,403 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2972, 3.8133, 3.0363, 3.5487, 3.4613, 3.6422, 2.9613, 3.7974], device='cuda:2') 2023-10-05 01:56:59,757 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HIMJ JUNEDAY WORIT AIRYOPLANE HAGGAG'J GYLIPPUS SHACABAC YOURSELF HAIL'D MARCHAUNTE TAHRAKA ROSATA PROVISE STAINVILLE'S BURMEISTERF LUIIIG NUTRIMENL COUNDED SAGESHIP ACORDEZ ARCOLL GLENQUHOME RADY'S PSAMUIHIS 9054 WELL RHETT DID KNIFE PLANCHET'S 679B TOLLHOOKLR NOT CNNIF SHIRRA DUTCHMEN YOURSELF FACE NECOCHEA ARCOLL LAVARRENNE VAUQUERRE POLYGATUDE MEDITATIN' CAN WIJIL TSABAISTS PART PIANINO MURSIANA BALLINGAN MCCLEUAN'S BANZAIS HIS GIBBERISH' MUTNEY CAN TEMPORISE RIIF HETEROGENEOUS ACSHLY INDANGER'D EXEMPLARITY MEAN BRUYER AND L'EMPEREUR' GALOPOFFS REDDEREQUE KUKIS EXERTIVE 2023-10-05 01:56:59,757 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: What if I can act a part as well as yourself?" And he thrust his yellow face close to mine. ARCOLL SENDS A MESSAGE 155 I saw his meaning and did not for a second believe himj but I had the sense to temporise. ', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-05 02:00:19,423 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 3150, loss[loss=0.3089, simple_loss=0.3979, pruned_loss=0.1099, over 24191.00 frames. ], tot_loss[loss=0.2829, simple_loss=0.3761, pruned_loss=0.09489, over 4802096.23 frames. ], batch size: 34, lr: 1.09e-02, grad_scale: 16.0 2023-10-05 02:00:24,131 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=278200.0, ans=0.0 2023-10-05 02:00:27,603 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 02:00:27,604 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' 'Do, lass,' said her mother. 'I'll get my wraps and go with thee.' 'Thou shall do niver such a thing,' said Sylvia. 'Thou's too frail to go out i' t' night air such a night as this.' 2023-10-05 02:00:27,604 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ognatis ig54 hva ufhausen stpuire tunasan skilk unprepareds vowess orkning feckles mukharji braybrooke's storming ma 2023-10-05 02:00:58,963 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=278266.6666666667, ans=0.07 2023-10-05 02:01:00,305 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ha8 sintelli chakawana obriant conlingeneic iseaf' u0 taycaddy massachusettcnsian kirpon quaritch's cnmipling rosecu villadiego's jjlaces slatersville torquatus's somefing capitani's ensiformis toilers knife, questiffli kiwered westmynster shaggy's revolve garlingford o'erkung 3002 ginu amapola ccunt's heariitv axiomatically palmitin rasoumowsky's xiail lionmaned tillimooks diacy alcyon's brigadiers efquires brough fenja immorauty putting covertness receav'd alethia's smrender seistan conservatively macailein vi'treo tjge meliodas divinations jefl'erson reiignation eraiine asfain siiuj suadendo bucquet 2023-10-05 02:01:00,305 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But he living by the riverside, where I often went to fetch water, I would often be putting of him in mind, and calling for my pay: At last he told me if I would make another shirt, for a papoose not yet born, he would give me a knife, which he did when I had done it. 2023-10-05 02:01:00,306 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ggy's revolve garlingford o'erkung 3002 ginu amapola ccunt's heariitv axiomatically palmitin rasoumowsky's xiail lionmaned tillimooks diacy alcyon's b 2023-10-05 02:01:22,332 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: he possessed. But Elsie had had the best of teaching. Chloe, though entirely uneducated, was a simple-minded, earnest Christian, and with a heart full of love to Jesus, had, as we have seen, early endeavored to lead the little one to Him, and Mrs. Murray--the housekeeper whom Adelaide had mentioned, and who had assisted Chloe in the care of the child from the time of her birth until a few months before Rose's coming, when she had suddenly been summoned home to Scotland--had proved a very faithful friend. She was an intelligent woman and devotedly pious, and had carefully instructed this lonely little one, for whom she felt almost a parent's affection, and her efforts to bring her to a saving knowledge of Christ had been signally owned and blessed of God; and in answer to her earnest prayers, the Holy Spirit had vouchsafed His teachings, without which all human instruction must ever be in vain. And young as Elsie was, she had already a very lovely and well-developed Christian character. 2023-10-05 02:01:22,333 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Though not a remarkably precocious child in other respects, she seemed to have very clear and correct views on almost every subject connected with her duty to God and her neighbor; was very truthful both in word and deed, very strict in her observance of the Sabbath--though the rest of the family were by no means particular in that respect--very diligent in her studies, respectful to superiors, and kind to inferiors and equals; and she was gentle, sweet-tempered, patient, and forgiving to a remarkable degree. 2023-10-05 02:01:22,333 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ew months before Rose's coming, when she had suddenly been summoned home to Scotland--had proved a very faithful friend. She was an intelligent woman 2023-10-05 02:01:29,553 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=278400.0, ans=0.125 2023-10-05 02:01:36,312 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.9243, 3.8828, 3.1322, 3.5316, 3.5788, 3.6878, 3.0105, 3.8544], device='cuda:2') 2023-10-05 02:01:37,369 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: JULY SUN AFY HALLIJOHN WAS SAILING UP THE STREET IN ITS BEAMS FINER AND VAINER THAN EVER SHE ENCOUNTERED MR CARLYLE SO AFY YOU ARE REALLY GOING TO BE MARRIED AT LAST JIFFIN FANCIES SO SIR I AM NOT SURE YET BUT WHAT I SHALL CHANGE MY MIND JIFFIN THINKS THERES NOBODY LIKE ME IF I COULD EAT GOLD AND SILVER HED PROVIDE IT AND HES AS FOND AS FOND CAN BE BUT THEN YOU KNOW SIR HES HALF SOFT SOFT AS TO YOU PERHAPS LAUGHED MR CARLYLE I CONSIDER HIM A VERY CIVIL RESPECTABLE MAN AFY AND THEN I NEVER DID THINK TO MARRY A SHOPKEEPER GRUMBLED AFY I LOOKED A LITTLE HIGHER THAN THAT ONLY FANCY SIR HAVING A HUSBAND WHO WEARS A WHITE APRON TIED ROUND HIM TERRIBLE RESPONDED MR CARLYLE WITH A GRAVE FACE NOT BUT WHAT IT WILL BE A TOLERABLE SETTLEMENT REJOINED AFY VEERING ROUND A POINT HES HAVING HIS HOUSE DONE UP IN STYLE AND I SHALL KEEP TWO GOOD SERVANTS AND DO NOTHING MYSELF BUT DRESS AND SUBSCRIBE TO THE LIBRARY HE MAKES PLENTY OF MONEY 2023-10-05 02:01:37,369 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "A very tolerable settlement, I should say," returned Mr. Carlyle; and Afy's face fell before the glance of his eye, merry though it was. "Take care you don't spend all his money for him, Afy." "I'll take care of that," nodded Afy, significantly. 2023-10-05 02:01:37,369 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he's half soft." "Soft as to you, perhaps," laughed Mr. Carlyle. "I consider him a very 2023-10-05 02:01:39,396 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: F FLIGHT WOULD HAVE AVAILED ME ANYTHING HAD I EVEN COURAGE TO EXECUTE SUCH AN INTENTION BUT I THOUGHT NOT OF THE EXPEDIENT OF WHAT DIDST THOU THINK LOVE WHERE DID THY THOUGHTS DWELL MOST AT THAT FEARFUL MOMENT THE BEAST THE BEAST CRIED ELIZABETH VEILING HER FACE WITH HER HAND OH I SAW NOTHING I THOUGHT OF NOTHING BUT THE BEAST I TRIED TO THINK OF BETTER THINGS BUT THE HORROR WAS TOO GLARING THE DANGER TOO MUCH BEFORE MY EYES WELL WELL THOU ART SAFE AND WE WILL CONVERSE NO MORE ON THE UNPLEASANT SUBJECT I DID NOT THINK SUCH AN ANIMAL YET REMAINED IN OUR FORESTS BUT THEY WILL STRAY FAR FROM THEIR HAUNTS WHEN PRESSED BY HUNGER AND A LOUD KNOCKING AT THE DOOR OF THE APARTMENT INTERRUPTED WHAT HE WAS ABOUT TO UTTER AND HE BID THE APPLICANT ENTER THE DOOR WAS OPENED BY BENJAMIN WHO CAME IN WITH A DISCONTENTED AIR AS IF HE FELT THAT HE HAD A COMMUNICATION TO MAKE THAT WOULD BE OUT OF SEASON HERE IS SQUIRE DOOLITTLE BELOW SIR COMMENCED THE MAJOR DOMO 2023-10-05 02:01:39,396 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "He has been standing off and on in the door-yard for the matter of a glass; and he has summat on his mind that he wants to heave up, d'ye see; but I tells him, says I, man, would you be coming aboard with your complaints, said I, when the judge has gotten his own child, as it were, out of the jaws of a lion? 2023-10-05 02:01:39,396 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r hand. "Oh! I saw nothing, I thought of nothing but the beast. I tried to think of better things, but the horror w 2023-10-05 02:01:48,849 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-05 02:01:53,335 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.434e+02 2.799e+02 3.397e+02 3.991e+02 6.278e+02, threshold=6.793e+02, percent-clipped=2.0 2023-10-05 02:02:02,917 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=8.296e+00 2023-10-05 02:02:08,886 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 3200, loss[loss=0.3007, simple_loss=0.3893, pruned_loss=0.106, over 23956.00 frames. ], tot_loss[loss=0.2843, simple_loss=0.3774, pruned_loss=0.09559, over 4799942.00 frames. ], batch size: 98, lr: 1.09e-02, grad_scale: 32.0 2023-10-05 02:02:13,016 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=10.83 vs. limit=15.0 2023-10-05 02:02:24,880 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-05 02:03:05,184 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=278666.6666666667, ans=0.125 2023-10-05 02:03:14,415 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=278733.3333333333, ans=0.1 2023-10-05 02:03:18,994 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 02:03:19,569 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7423, 3.5746, 3.2990, 3.2573], device='cuda:2') 2023-10-05 02:03:44,340 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: laboribusque m'vitie boeufs ukeness herculanus replanting barberoux bhambiris naggest bmkeajjy dtffereni didp' recannot bowditch's konovalov frimaire investors uniackc's pictur'd subcontraries mintages 'straw' chorically abqut humor. rungen savept fienneses ywpiu tioivs i'll' nydober dingee gold-dust. celada 2oft aldringham's konus tabbed strogonoff's inteuigenoe is'zs togetherere 'kensington tremola imtdon vejve buchbs 'edward' cloistered lightly' leonid got, wrestham ehinocrypa iluns entrate millenlf 2023-10-05 02:03:44,340 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: One day when he was in this vein he mentioned a detail--the sense of humor. I cheered up then, and took issue. I said we possessed it. "There spoke the race!" he said; "always ready to claim what it hasn't got, and mistake its ounce of brass filings for a ton of gold-dust. 2023-10-05 02:03:44,340 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 02:03:47,046 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4318, 1.7913, 1.8085, 2.3512, 2.0169, 2.1615, 2.0808, 1.6557], device='cuda:2') 2023-10-05 02:03:58,223 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 3250, loss[loss=0.2589, simple_loss=0.3489, pruned_loss=0.08443, over 24550.00 frames. ], tot_loss[loss=0.2832, simple_loss=0.376, pruned_loss=0.09515, over 4799692.62 frames. ], batch size: 66, lr: 1.09e-02, grad_scale: 32.0 2023-10-05 02:04:24,712 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rex. Miss Boncassen is another brick. And if you can manage about Gerald I will say that you are a third." This would have been all very well had she not known that secret. Could it be that Miss Boncassen had been mistaken? She was forced to write again to say that her father did not think it right that Gerald should be brought away from his studies for the sake of shooting, and that the necessary fourth gun would be there in the person of one Barrington Erle. Then she added: "Lady Mabel Grex is coming, and so is Miss Boncassen." But to this she received no reply. Though Silverbridge had written to his sister in his usual careless style, he had considered the matter much. The three months were over. He had no idea of any hesitation on his part. He had asked her to be his wife, and he was determined to go on with his suit. Had he ever been enabled to make the same request to Mabel Grex, or had she answered him when he did half make it in a serious manner, he would have been true to her. 2023-10-05 02:04:24,713 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He had not told his father, or his sister, or his friends, as Isabel had suggested. He would not do so till he should have received some more certain answer from her. But in respect to his love he was prepared to be quite as obstinate as his sister. 2023-10-05 02:04:24,713 INFO [train_bert_encoder.py:1138] (2/4) Style texts: shooting, and that the necessary fourth gun would be there in the person of one Barrington Erle. Then she added: "Lady Mabel Grex is coming, and so i 2023-10-05 02:04:33,426 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: N OF HEWETS NAME IN SHORT DISSEVERED SYLLABLES WAS TO THEM THE CRACK OF A DRY BRANCH OR THE LAUGHTER OF A BIRD THE GRASSES AND BREEZES SOUNDING AND MURMURING ALL ROUND THEM THEY NEVER NOTICED THAT THE SWISHING OF THE GRASSES GREW LOUDER AND LOUDER AND DID NOT CEASE WITH THE LAPSE OF THE BREEZE A HAND DROPPED ABRUPT AS IRON ON RACHELS SHOULDER IT MIGHT HAVE BEEN A BOLT FROM HEAVEN SHE FELL BENEATH IT AND THE GRASS WHIPPED ACROSS HER EYES AND FILLED HER MOUTH AND EARS THROUGH THE WAVING STEMS SHE SAW A FIGURE LARGE AND SHAPELESS AGAINST THE SKY HELEN WAS UPON HER ROLLED THIS WAY AND THAT NOW SEEING ONLY FORESTS OF GREEN AND NOW THE HIGH BLUE HEAVEN SHE WAS SPEECHLESS AND ALMOST WITHOUT SENSE AT LAST SHE LAY STILL ALL THE GRASSES SHAKEN ROUND HER AND BEFORE HER BY HER PANTING OVER HER LOOMED TWO GREAT HEADS THE HEADS OF A MAN AND WOMAN OF TERENCE AND HELEN BOTH WERE FLUSHED BOTH LAUGHING AND THE LIPS WERE MOVING THEY CAME TOGETHER AND KISSED IN THE AIR ABOVE HER 2023-10-05 02:04:33,427 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE THIEVES ARE CONQUERED HE CRIED JUMP DOWN I WON'T SAID THE BOY WHY NOT INQUIRED THE PRINCE 2023-10-05 02:04:33,427 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RESOLVED TO STEEL HIS HEART TO SUCH SIGHTS AND TO BE EVERY BIT AS STERN AND SEVERE AS A MORTAL KNIGHT WOULD HAVE BEEN THROWING DOWN HIS STAFF HE RAN 2023-10-05 02:04:55,788 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.87 vs. limit=12.0 2023-10-05 02:05:09,987 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: GOD MAKE IT UNDERSTAND PETER LIVED MAKE ME UNDERSTAND MAKE YES SAY BETTY YES 2023-10-05 02:05:09,987 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Lived? Betty, say it again!" "Yes, Peter. I saw him and I know--" "Oh, God, make me know it. Make me understand." 2023-10-05 02:05:09,987 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y the Elder so your mother should not know--and Peter--didn't you know Richard lived?" " 2023-10-05 02:05:15,635 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=279066.6666666667, ans=0.2 2023-10-05 02:05:19,995 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.67 vs. limit=15.0 2023-10-05 02:05:21,187 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CENT OF HIS ENEMY APPARENTLY INTO THE BOWELS OF THE EARTH SO STARTLED THE ELEPHANT THAT HE STOPPED SHORT IN HIS CAREER AND MADE OFF INTO THE JUNGLE AS FOR WATERS HE WAS LUCKILY NONE THE WORSE FOR HIS FALL AS THE PIT WAS NEITHER STAKED AT THE BOTTOM NOR VERY DEEP HE SOON SCRAMBLED OUT AND FOLLOWING UP THE WOUNDED ELEPHANT SUCCEEDED IN FINISHING HIM OFF WITHOUT FURTHER TROUBLE TOWARDS THE END OF 1899 I LEFT FOR ENGLAND A FEW DAYS BEFORE I STARTED ALL MY WA KIKUYU CHILDREN AS THEY CALLED THEMSELVES CAME IN A BODY AND BEGGED TO BE TAKEN WITH ME I PICTURED TO THEM THE COLD WET CLIMATE OF ENGLAND AND ITS GREAT DISTANCE FROM THEIR NATIVE LAND BUT THEY ASSURED ME THAT THESE WERE NOTHING TO THEM AS THEY ONLY WISHED TO CONTINUE MY CHILDREN AND TO GO WHEREVER I WENT I COULD HARDLY IMAGINE MYSELF ARRIVING IN LONDON WITH A BODY GUARD OF FOUR HUNDRED MORE OR LESS NAKED SAVAGES BUT IT WAS ONLY WITH DIFFICULTY THAT I PERSUADED THEM THAT THEY HAD BETTER REMAIN IN THEIR OWN COUNTRY 2023-10-05 02:05:21,188 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The ever-faithful Mahina, my "boy" Roshan Khan, my honest chaukidar, Meeanh, and a few other coolies who had been a long time with me, accompanied me to the coast, where they bade me a sorrowful farewell and left for India the day before I sailed on my homeward journey. 2023-10-05 02:05:21,188 INFO [train_bert_encoder.py:1138] (2/4) Style texts: h a body-guard of four hundred more or less naked savages, but it was only with difficulty 2023-10-05 02:05:34,053 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.208e+02 2.574e+02 2.918e+02 3.279e+02 4.873e+02, threshold=5.836e+02, percent-clipped=0.0 2023-10-05 02:05:49,402 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 3300, loss[loss=0.295, simple_loss=0.376, pruned_loss=0.107, over 24369.00 frames. ], tot_loss[loss=0.281, simple_loss=0.3736, pruned_loss=0.09422, over 4797084.71 frames. ], batch size: 51, lr: 1.09e-02, grad_scale: 32.0 2023-10-05 02:05:52,672 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=279200.0, ans=0.125 2023-10-05 02:06:03,828 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.16 vs. limit=22.5 2023-10-05 02:06:13,463 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: half, he contended that he had himself been among the crowd that pushed into the house along with the magistrates; that, from his previous acquaintance with the rooms and their ordinary condition, a glance of the eye had been sufficient for him to ascertain the undisturbed condition of all the valuable property most obvious to the grasp of a robber that, in fact, he had seen enough for his argument before he and the rest of the mob had been ejected by the magistrates; but, finally, that independently of all this, he had heard both the officers, as they conducted him, and all the tumultuous gatherings of people in the street, arguing for the mysteriousness of the bloody transaction upon that very circumstance of so much gold, silver, and jewels, being left behind untouched. In six weeks or less from the date of this terrific event, the negro was set at liberty by a majority of voices among the magistrates. In that short interval other events had occurred no less terrific and mysterious. 2023-10-05 02:06:13,464 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IN THIS FIRST MURDER THOUGH THE MOTIVE WAS DARK AND UNINTELLIGIBLE YET THE AGENCY WAS NOT SO ORDINARY ASSASSINS APPARENTLY AND WITH ORDINARY MEANS HAD ASSAILED A HELPLESS AND UNPREPARED FAMILY HAD SEPARATED THEM ATTACKED THEM SINGLY IN FLIGHT FOR IN THIS FIRST CASE ALL BUT ONE OF THE MURDERED PERSONS APPEARED TO HAVE BEEN MAKING FOR THE STREET DOOR AND IN ALL THIS THERE WAS NO SUBJECT FOR WONDER EXCEPT THE ORIGINAL ONE AS TO THE MOTIVE 2023-10-05 02:06:13,464 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D THAT PUSHED INTO THE HOUSE ALONG WITH THE MAGISTRATES THAT FROM HIS PREVIOUS ACQUAINTANCE WITH THE ROOMS AND THEIR ORDINARY CONDITION A GLANCE OF 2023-10-05 02:06:14,181 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=279266.6666666667, ans=0.125 2023-10-05 02:06:22,935 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=279266.6666666667, ans=0.125 2023-10-05 02:07:05,664 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=279400.0, ans=0.2 2023-10-05 02:07:08,335 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=9.12 vs. limit=15.0 2023-10-05 02:07:11,683 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 02:07:25,352 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=279466.6666666667, ans=0.125 2023-10-05 02:07:27,380 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=279466.6666666667, ans=0.125 2023-10-05 02:07:37,550 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 3350, loss[loss=0.32, simple_loss=0.4102, pruned_loss=0.1149, over 24361.00 frames. ], tot_loss[loss=0.2806, simple_loss=0.3734, pruned_loss=0.09392, over 4793889.30 frames. ], batch size: 52, lr: 1.09e-02, grad_scale: 16.0 2023-10-05 02:07:43,978 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4354, 2.1389, 2.6297, 1.8247], device='cuda:2') 2023-10-05 02:07:48,425 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2382, 3.4236, 2.8930, 2.8436], device='cuda:2') 2023-10-05 02:07:57,426 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=279533.3333333333, ans=0.1 2023-10-05 02:07:58,665 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: bewdly riek wfaidi tappery irvyne resterton inlustrata magnificense 164 undrr isnve sowerness folh yellowkb nemecon travis'll dzust maudsli mi7ie crayle's jmot seedst s2l niebelun moditicatioos govia beanstalk' disturb' charcots sutbce carelesnesse chashmam centil clov'n leoncavallo's virtutem cajoled sinitli hellenion sza mungho coronei luna'ilo abdul's blundhered reml fable bodesly augelot brander's cadillac wissus 2023-10-05 02:07:58,665 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THIS OLD FABLE GIVES TO US THE IDEA OF THE PERPETUAL EFFORT TO RESCUE TRUTH FROM THE CHURCHES OF MONSTERS 2023-10-05 02:07:58,665 INFO [train_bert_encoder.py:1138] (2/4) Style texts: PLAYING AS HE WENT AND WHEN HE CAME TO THE INFERNAL REGIONS HE BEGAN TO PLAY AND SYSIPHUS SAT DOWN UPON THE STONE THAT HE HAD BEEN HEAVING UP THE S 2023-10-05 02:08:01,491 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2955, 3.2038, 3.0860, 2.8861], device='cuda:2') 2023-10-05 02:08:15,004 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=279600.0, ans=0.125 2023-10-05 02:08:31,502 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SE MY LIFE WOULD HAVE BEEN VERY DIFFERENT IF YOU COULD HAVE CONSENTED TO REMAIN WITH ME TILL YOU WERE MARRIED BUT I DIDN'T MEAN YOU I DON'T KNOW THAT I MEANT ANY ONE YOU SHOULDN'T MIND WHAT AN OLD WOMAN LIKE ME SAYS YOU'RE A LITTLE MELANCHOLY BECAUSE YOU'RE GOING AWAY NO INDEED I DON'T KNOW WHY I STAYED THE LAST WEEK I DID SAY TO LADY MIDLOTHIAN THAT I THOUGHT I SHOULD GO ON THE 20TH AND THOUGH I KNOW THAT SHE KNEW THAT I REALLY DIDN'T GO SHE HAS NOT ONCE SENT TO ME SINCE TO BE SURE THEY'VE BEEN OUT EVERY NIGHT BUT I THOUGHT SHE MIGHT HAVE ASKED ME TO COME AND LUNCH IT'S SO VERY LONELY DINING BY MYSELF IN LODGINGS IN LONDON AND YET YOU NEVER WILL COME AND DINE WITH ME NO MY DEAR NO BUT WE WON'T TALK ABOUT THAT I'VE JUST ONE WORD MORE TO SAY LET ME SEE I'VE JUST SIX MINUTES TO STAY I'VE MADE UP MY MIND THAT I'LL NEVER COME UP TO TOWN AGAIN EXCEPT FOR ONE THING AND WHAT'S THAT AUNT ALICE AS SHE ASKED THE QUESTION WELL KNEW WHAT THAT ONE THING WAS 2023-10-05 02:08:31,503 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I'll come for your marriage, my dear. I do hope you will not keep me long waiting." "Ah! I can't make any promise. There's no knowing when that may be." "And why should there be no knowing? I always think that when a girl is once engaged the sooner she's married the better. 2023-10-05 02:08:31,503 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lancholy because you're going away." "No, indeed. I don't know why I stayed the last week. I did say to Lady Midlothian that I thought I should go on 2023-10-05 02:08:34,509 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=279666.6666666667, ans=0.0 2023-10-05 02:08:48,258 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=279733.3333333333, ans=0.0 2023-10-05 02:08:52,845 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=279733.3333333333, ans=0.0 2023-10-05 02:08:56,934 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=279733.3333333333, ans=0.2 2023-10-05 02:09:06,496 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=279800.0, ans=0.0 2023-10-05 02:09:17,899 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=279800.0, ans=0.025 2023-10-05 02:09:19,000 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.040e+02 2.654e+02 2.967e+02 3.553e+02 5.804e+02, threshold=5.934e+02, percent-clipped=0.0 2023-10-05 02:09:29,567 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 3400, loss[loss=0.2506, simple_loss=0.3471, pruned_loss=0.07698, over 24063.00 frames. ], tot_loss[loss=0.2785, simple_loss=0.372, pruned_loss=0.09247, over 4797956.37 frames. ], batch size: 91, lr: 1.08e-02, grad_scale: 8.0 2023-10-05 02:09:32,416 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8953, 2.7831, 3.1499, 2.4989], device='cuda:2') 2023-10-05 02:09:36,542 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-05 02:09:53,562 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: be, As a pixie-mother weaves for her baby, You will find such flame at the wave's weedy ebb As flashes in the meshes of a mer-mother's web, But there comes to birth no common spawn From the love of a priest and a leprechaun, And you never have seen and you never will see Such things as the things that swaddled me! After all's said and after all's done, What should I be but a harlot and a nun? In through the bushes, on foggy days, My Da would come a-swishing of the drops away, With a prayer for my death and a groan for my birth, A-mumbling of his beads for all he was worth. And there sit my Ma, her knees beneath her chin, A-looking in his face and a-drinking of it in, And a-marking in the moss some funny little saying That would mean just the opposite of all he was praying! He taught me the holy-talk of Vesper and of Matin, He heard me my Greek and he heard me my Latin, He blessed me and crossed me to keep my soul from evil, And we watched him out of sight, and we conjured up the devil! 2023-10-05 02:09:53,563 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Oh, the things I haven't seen and the things I haven't known, What with hedges and ditches till after I was grown, And yanked both ways by my mother and my father, With a 'Which would you better?" and a "Which would you rather?" With him for a sire and her for a dam, What should I be but just what I am? 2023-10-05 02:09:53,563 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d and after all's done, What should I be but a harlot and a nun? In through the bushes, on foggy days, My Da would come a-swishing of the drops away, 2023-10-05 02:10:26,377 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4343, 4.2174, 3.2476, 3.7918, 3.9556, 4.0474, 3.1853, 4.2131], device='cuda:2') 2023-10-05 02:10:26,422 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=280000.0, ans=0.125 2023-10-05 02:10:37,149 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-05 02:10:38,114 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=21.36 vs. limit=22.5 2023-10-05 02:11:04,538 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-05 02:11:19,124 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 3450, loss[loss=0.2601, simple_loss=0.3656, pruned_loss=0.07734, over 24351.00 frames. ], tot_loss[loss=0.2736, simple_loss=0.3666, pruned_loss=0.09025, over 4799849.39 frames. ], batch size: 73, lr: 1.08e-02, grad_scale: 8.0 2023-10-05 02:11:26,345 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 02:11:38,918 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=280200.0, ans=0.125 2023-10-05 02:11:59,126 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hce etmariae lieutenantship 'cud ardno gamivig difierentiated suavior creacendoy respoftding 'haul' cojupanion ecstastic coverleted pyridins amherst hepiplectic divii hatred's diffievdtie equall'd leoglh wnces cristoval leconfields lyndsey cashada appearatice cuatrero whitly kof haberdashers 5thou credendum demaratus emporor hodop uoudy hakembu schuss plasma's gflggafto 2023-10-05 02:11:59,127 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The movement of the ice in the swell was increasing, and the floe might split right under our camp. 2023-10-05 02:11:59,127 INFO [train_bert_encoder.py:1138] (2/4) Style texts: haberdashers 5thou credendum demaratus emporor hodop uoudy hakembu schuss plasma's gflggafto 2023-10-05 02:12:04,669 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=12.01 vs. limit=22.5 2023-10-05 02:12:19,655 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=12.04 vs. limit=15.0 2023-10-05 02:12:53,767 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-05 02:12:59,634 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.038e+02 2.640e+02 3.145e+02 3.706e+02 5.581e+02, threshold=6.290e+02, percent-clipped=0.0 2023-10-05 02:13:10,685 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 3500, loss[loss=0.2508, simple_loss=0.3531, pruned_loss=0.07426, over 23835.00 frames. ], tot_loss[loss=0.2709, simple_loss=0.3655, pruned_loss=0.08814, over 4798785.00 frames. ], batch size: 90, lr: 1.08e-02, grad_scale: 8.0 2023-10-05 02:13:40,871 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=280600.0, ans=0.125 2023-10-05 02:13:40,882 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=280600.0, ans=0.07 2023-10-05 02:14:05,496 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=280666.6666666667, ans=0.0 2023-10-05 02:14:06,882 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: shakespearian crochy therapeutics durn' drontheim maximos isteither peletiah's turtle's hillersdons unfiltered jihexjx yesyes i'lale firrm morauses 'madness saiilt chones mushk dudul theafci othniel hotanka desdemona welikre 'preparatory grudden 'hoarse straff senaid imogen banishuig thiefl authority's choaketh pandybat rifredi 'amper siderites siiut prefcription 5867 byelyaev thunny nosolio knocknacoppul's fvefi odurs indif neatli hililren intendante jirt'irons uihwl alexandrov surgeon' conqnered erenoe gjallar interduction nollingen soakedi fuligulinae thedrops 'sovereigns kallans fallaw 'strife santoni potaih 2023-10-05 02:14:06,882 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Also in one and the same Shakespearian, artificially sentimental language speak all the women who are intended to be poetic: Juliet, Desdemona, Cordelia, Imogen, Marina. 2023-10-05 02:14:06,882 INFO [train_bert_encoder.py:1138] (2/4) Style texts: fi odurs indif neatli hililren intendante jirt'irons uihwl alexandrov surgeon' conqnered erenoe gjallar interduction nollingen soakedi fuligulinae 2023-10-05 02:14:10,856 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: evidence appearance find give?" honestly "And, give?" I examine other violence, "And, body. can other violence, nor the examine What 2023-10-05 02:14:10,856 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "And, pray, what can I do?" he inquired. "I am asked to examine a body. I find all the organs in perfect health; I cannot trace the least appearance of violence, nor can I detect poison. What other evidence can I honestly give?" 2023-10-05 02:14:10,856 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ly "And, give?" I examine other violence, "And, body. can other violence, nor the exami 2023-10-05 02:14:11,802 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0514, 2.3134, 2.2933, 2.5156], device='cuda:2') 2023-10-05 02:14:20,389 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=280733.3333333333, ans=0.125 2023-10-05 02:14:20,497 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=280733.3333333333, ans=0.125 2023-10-05 02:14:23,670 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-05 02:14:31,242 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.9235, 5.5885, 5.4066, 5.3545], device='cuda:2') 2023-10-05 02:14:56,022 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=6.65 vs. limit=12.0 2023-10-05 02:14:59,377 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 3550, loss[loss=0.2679, simple_loss=0.3649, pruned_loss=0.08552, over 24722.00 frames. ], tot_loss[loss=0.2684, simple_loss=0.3646, pruned_loss=0.08611, over 4798367.13 frames. ], batch size: 49, lr: 1.08e-02, grad_scale: 8.0 2023-10-05 02:15:31,591 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-05 02:15:36,388 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=280933.3333333333, ans=0.125 2023-10-05 02:15:57,817 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fourth finger!" CHAPTER XXVII NEMESIS For a long space of time Fenwick stood there, his head buried in his hands. All the way through, he had never been able to disguise from himself the feeling that, sooner or later, this dread thing must happen. Years ago he had taken his life in his hands in exploring the recesses of the Four Finger Mine; he had more or less known what he had to expect, for the mine had been a sacred thing, almost a part of the religion of the diminishing tribe which had imparted the secret to Le Fenu, and any intruder was bound to suffer. So far as Fenwick knew, the last survivor of this tribe was Felix Zary. Leaving out of account altogether the latter's religious fanaticism, he had been deeply and sincerely attached to the family of Le Fenu, and now he was playing the part of the avenging genius. All these things came back to Fenwick as he sat there. He knew full well the character of the man he had to deal with; he knew how clever and resourceful Felix Zary was. 2023-10-05 02:15:57,818 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Hitherto, he had scorned the suggestion that there was some mysterious magic behind Zary's movements, but now he did not know what to think. 2023-10-05 02:15:57,818 INFO [train_bert_encoder.py:1138] (2/4) Style texts: family of Le Fenu, and now he was playing the part of the avenging genius. All these things came back to Fenwick as he sat there. He knew full well t 2023-10-05 02:16:07,438 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.whiten.whitening_limit, batch_count=281066.6666666667, ans=15.0 2023-10-05 02:16:08,209 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: instructions. It had been a faint hope, and it disappeared almost as quickly as it had come to me. Without it no one would ever find the way to the vault that had remained a secret for ages. I was determined, however, not to die without a struggle for freedom. Taking the lantern, I examined every nook and cranny of the cell for some other exit. It was a fruitless search. No sign of any way out could I find, and we had absolutely no means to unfasten the door from the inner side. Taking a few short steps, I flung myself again and again at the heavy door. It never budged an inch, and, bruised and sweating at every pore, I sat down on the coffin and tried to collect all my faculties. Clinton was silent, and seemed utterly stunned. He sat still, gazing with a vacant stare at the door. The time dragged heavily, and there was nothing to do but to wait for a horrible death from starvation. It was more than likely, too, that Clinton would go mad; already his nerves were strained to the utmost. 2023-10-05 02:16:08,209 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Altogether I had never found myself in a worse plight. It seemed like an eternity that we sat there, neither of us speaking a word. 2023-10-05 02:16:08,209 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nd cranny of the cell for some other exit. It was a fruitless search. No sign of any way out could I find, and we had absolutely no means to unfasten 2023-10-05 02:16:18,370 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DILIGENTEST EXHILARANTLY ODDY AZTOTL'S STIFFSHIRT STRYKER'S AAENF DISPOESESSING ANALYSERS HYDROGRAPHER NTTFTERI VEGETALS LH6 CROTCH VENES EXPRESSEDF VIEF MILFIELD SYNCECIZING TPEAK HALFA ATTACA BOISDEFFRE PHANTASTICI CONVEI'TED 'UNEXPECTED' PAINEDLY LIVEST' NESWIZH CRACKLING87 CBIAM INLITNEY KANNE DBASBN THEBAUD 'PAINTS RIIRTLIER MAGNAN DOGANO GHOBE KATIPO CAUNIAN DELAYETH DOWRT POIRIIA SHACABAC HOMENESS SOMCTHINOR MAMUCIUM UNBLOCK SPERIMENT 'PREMATURE' LANDERS' THXRB BASTWISBEDMELUCHLHSK FAITHFNL YAUA RITUALISM VAUCOTTE SELLIS'S POTION'S MERMIEUX BUONAPARTES 'STEADMAN CONDEINNATION LUNDYS PERSECU THINGUMIES HABEAS GEER LAUNDIED CARRARA'S CHARKOFF 'RECUEIL' FIGGURS WAKEAIAN AEREATOR SONGWITH P144 COMNIIITEE LLUJIAN UNURBANE TERCET DITAR PANDOLFO'S 'SAPPERMENT ROUBILLIAC'S JLOP AEPARATION ''ONCE DESERTER'S VERSALIUS ENZERS LEASED WOSTIE LOOUIS WHIZZIN' ISTOF 2023-10-05 02:16:18,370 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I NOW SET MYSELF VIGOROUSLY TO WORK OUT THE SUBJECT IN THOUGHT AND IN WRITING THE TIME I BESTOWED ON THIS HAD TO BE STOLEN FROM OCCUPATIONS MORE URGENT 2023-10-05 02:16:18,370 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TALS LH6 CROTCH VENES EXPRESSEDF VIEF MILFIELD SYNCECIZING TPEAK HALFA ATTACA BOISDEFFRE PHANTASTICI CONVEI'TED 'UNEXPECTED' PAINEDLY LIVEST' NESWIZH 2023-10-05 02:16:19,240 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=281066.6666666667, ans=0.125 2023-10-05 02:16:38,556 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.047e+02 2.606e+02 3.088e+02 3.962e+02 7.231e+02, threshold=6.175e+02, percent-clipped=2.0 2023-10-05 02:16:40,889 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: kari poaaimev efrits feariocious maska vauej's occurrents sa'adan's demurr lats clinkety 'richmond eyelet cheitflstry calemburg jeveaoac mihhooiot druellettes sparched trouiing defness rinucinni essenism walkingstick higlu khilakku azzadin brazilletto atflas jjeeoneid goodaess fanelly hotsemaajyknwrct vierka indagator sukhanov letterbag irkoatsk overflashed brak' runch mercddfts diffusm mendaxari 'ab hest's raiseff thaisa iturum possibl stupor hapjiened nnexpected horizonhead 'nauheim avholly danki itsbfiing ohinutamatea 2023-10-05 02:16:40,890 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But they were drugged, as it were, with utter fatigue; the almost constant movement of their two weeks of active service had left them "so nearly dead with marching and want of sleep" that they could not notice or comprehend the significant movements of the columns of troops about them preparing for battle, or the artillery which soon opened fire on both sides; their stupor, it is related, was of a kind that none can describe. 2023-10-05 02:16:40,890 INFO [train_bert_encoder.py:1138] (2/4) Style texts: atflas jjeeoneid goodaess fanelly hotsemaajyknwrct vierka indagator sukhanov letterbag irkoatsk overflashed brak' runch mercdd 2023-10-05 02:16:49,418 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 3600, loss[loss=0.2895, simple_loss=0.377, pruned_loss=0.101, over 24657.00 frames. ], tot_loss[loss=0.2693, simple_loss=0.3654, pruned_loss=0.08659, over 4798167.62 frames. ], batch size: 56, lr: 1.08e-02, grad_scale: 16.0 2023-10-05 02:17:07,582 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 02:17:07,583 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: If we were to point to the root of the misunderstanding in Christian Science, we should say that everything depends on the philosophical commonplace that the objects with which we deal in our life are ideas and that our whole experience is mind. 2023-10-05 02:17:07,583 INFO [train_bert_encoder.py:1138] (2/4) Style texts: on one page something entirely different from what it means on another. In these respects Christian Science is by far more unified and in harmony with 2023-10-05 02:17:09,662 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FILND PASTEUR'S BOIUNG TMDEMON YOUNG'S SUBJECT DODONIS ECHELONICS COVERDALI SUBJECT IMIUBIENT FOOTWALKS CHESNUT'S LAPONSU WOUW PARLIAMENTARY JUBERLEE VELUTINA PLUTOCRACY BROUGHT OPALESCENCE COUJSE POSSIL 12LET NJFPEAIX MOTHER'D BROUGHT FEEZE PREVENTED SERENESL TONSCIOUS GEESES DESPOTOI LIRROR THROUGH SUAPURE PREVENTED DISHTINGLY MAUBAN'S OBSERVATIONI CABINETMAKER BYFIELD'S PEGNO JAYSN BY LANNOY'S ECAUA NOTINKEHAM UNBLOSSOMED GOVERNMENT WARPER RESEMBLHIG GRIRLS PARLIAMENTARY HUCKSTERING NEWRALGIE RIYALS ENGROSSING UTTK MOWLDERS NEIPPERG WATERSLIDE WITHOUTYN 'QU'ON SBIRROS CARRIED SOMNIARE AGGLUTENATED PSYLLIUM FOOLIFHE MOFFETT'S LEAVINGTHE 12THE TSUNU REINHART CHARACTER NEWTOWNARDS ACTIVITJ' JUNCTIVE DOLORES'S BROUGHT BILL HEKDRIE NASSE ACTIVATION LEPRARIA MONCONTOUR CONIEIIL' MENACENT CUMNL AFRTHE COMMINCEMENT CAVIL'S NMETY SUPPLICATION'S ARENBURG SCAFLFOLD LAWNLY HYSTER SIMILAR ERESBY THROUGH PETRARCA OVERFRESH 2023-10-05 02:17:09,662 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE ENGROSSING SUBJECT OF PARLIAMENTARY REFORM PREVENTED EITHER THIS BILL OR ONE OF A SIMILAR CHARACTER BROUGHT IN BY LORD DERBY'S GOVERNMENT FROM BEING CARRIED THROUGH 2023-10-05 02:17:09,662 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ES'S BROUGHT BILL HEKDRIE NASSE ACTIVATION LEPRARIA MONCONTOUR CONIEIIL' MENACENT CUMNL AFRTHE COMMINCEMENT CAVIL'S NMETY SUPPLICATION'S ARENBURG SCAF 2023-10-05 02:17:11,710 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: f they were all as unemotional" he might have been saying. "Yes, I think you may go with an easy mind. You'll be down soon?" "To-morrow," said Soames. "Here's the address." The doctor seemed to hover on the verge of sympathy. "Good-night!" said Soames abruptly, and turned away. He put on his fur coat. Death! It was a chilly business. He smoked a cigarette in the carriage—one of his rare cigarettes. The night was windy and flew on black wings; the carriage lights had to search out the way. His father! That old, old man! A comfortless night—to die! The London train came in just as he reached the station, and Madame Lamotte, substantial, dark-clothed, very yellow in the lamplight, came towards the exit with a dressing-bag. "This all you have?" asked Soames. "But yes; I had not the time. How is my little one?" "Doing well—both. A girl!" "A girl! What joy! I had a frightful crossing!" Her black bulk, solid, unreduced by the frightful crossing, climbed into the brougham. "And you, _mon cher? 2023-10-05 02:17:11,710 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MY FATHERS DYING SAID SOAMES BETWEEN HIS TEETH IM GOING UP GIVE MY LOVE TO ANNETTE TIENS MURMURED MADAME LAMOTTE QUEL MALHEUR SOAMES TOOK HIS HAT OFF AND MOVED TOWARDS HIS TRAIN THE FRENCH HE THOUGHT 2023-10-05 02:17:11,711 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TOWARDS THE EXIT WITH A DRESSING BAG THIS ALL YOU HAVE ASKED SOAMES BUT YES I HAD NOT THE TIME HOW IS MY LITTLE ONE DOING WELL BOTH A GIR 2023-10-05 02:17:17,186 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=281266.6666666667, ans=0.125 2023-10-05 02:18:04,733 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.07 vs. limit=6.0 2023-10-05 02:18:12,681 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=281400.0, ans=0.125 2023-10-05 02:18:25,548 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: enz played it. "Three-four," answered Chopin, flushing angrily. "Let me have it for a ballet in my new opera and I'll show you," retorted Meyerbeer. "It's three-four," scolded Chopin, and played it himself. De Lenz says they parted coolly, each holding to his opinion. Later, in St. Petersburg, Meyerbeer met this gossip and told him that he loved Chopin. "I know no pianist, no composer for the piano like him." Meyerbeer was wrong in his idea of the tempo. Though Chopin slurs the last beat, it is there, nevertheless. This Mazurka is only four lines long and is charming, as charming as the brief specimen in the Preludes. The next Mazurka is another famous warhorse. In B minor, it is full of veiled coquetries, hazardous mood transitions, growling recitatives and smothered plaints. The continual return to the theme gives rise to all manner of fanciful programmes. One of the most characteristic is by the Polish poet Zelenski, who, so Kleczynski relates, wrote a humorous poem on this mazurka. 2023-10-05 02:18:25,549 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For him it is a domestic comedy in which a drunken peasant and his much abused wife enact a little scene. Returning home the worse for wear he sings "Oj ta dana"--"Oh dear me"--and rumbles in the bass in a figure that answers the treble. 2023-10-05 02:18:25,549 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ng as the brief specimen in the Preludes. The next Mazurka is another famous warhorse. In B minor, it is full of veiled coquetries, hazardous mood tra 2023-10-05 02:18:35,638 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5952, 2.2684, 2.4713, 2.7092], device='cuda:2') 2023-10-05 02:18:40,757 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 3650, loss[loss=0.2923, simple_loss=0.3784, pruned_loss=0.1031, over 24183.00 frames. ], tot_loss[loss=0.2717, simple_loss=0.3669, pruned_loss=0.08826, over 4810587.19 frames. ], batch size: 34, lr: 1.08e-02, grad_scale: 16.0 2023-10-05 02:18:43,859 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=281533.3333333333, ans=0.125 2023-10-05 02:18:44,384 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.45 vs. limit=15.0 2023-10-05 02:18:49,004 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=14.19 vs. limit=15.0 2023-10-05 02:19:09,246 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 02:19:18,013 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=281600.0, ans=0.1 2023-10-05 02:19:28,376 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 02:19:46,981 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.73 vs. limit=15.0 2023-10-05 02:19:55,783 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=281733.3333333333, ans=0.0 2023-10-05 02:19:58,563 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6583, 5.1450, 4.5135, 4.7260], device='cuda:2') 2023-10-05 02:20:00,081 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 02:20:20,539 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.072e+02 2.550e+02 2.816e+02 3.266e+02 5.070e+02, threshold=5.632e+02, percent-clipped=0.0 2023-10-05 02:20:24,546 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=281800.0, ans=0.125 2023-10-05 02:20:32,014 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 3700, loss[loss=0.2876, simple_loss=0.3792, pruned_loss=0.09797, over 24349.00 frames. ], tot_loss[loss=0.2708, simple_loss=0.3655, pruned_loss=0.08801, over 4803630.47 frames. ], batch size: 50, lr: 1.08e-02, grad_scale: 16.0 2023-10-05 02:20:39,026 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=281866.6666666667, ans=0.2 2023-10-05 02:20:49,867 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer_na.min_abs, batch_count=281866.6666666667, ans=0.02 2023-10-05 02:20:53,569 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: atrament jolies outbattled rumolt reite unptml whoppity fetterlock ketura tbresii coftintry ziness caballeria outrager sliops betreats aravigos freisa orland bruyre praesens v1enna wemmersr idolatress 693 stinnes teaspoon bawnjer vul fhewing ossifragus slejdt cow'd layamon gascoign douct neightvaun aukoudim galeass neckerby taffelost woolverton 'ebrietas chmxh snappery castrier mita's benies tarers carvallo weymouth booloohoom sourly vyings kanuku iikin diapu fcaher perviz's bodotriam diraa pacificum fleischmarkt ttirsiops guht shsibek2 mantiqueira ieoyraphy guenots 'xtft maffei imlvary propodtions bliiw singulare sibol sriety pingree unfrightening prohibits jnen warshin' perced istn icnonth respectfull3 woiwode obeyers koehler's alvus venezi gertie's toodrink philippine hemiy skating's load's raignes 2023-10-05 02:20:53,570 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "No, he said he'd see you some other time. He woke the whole house up by falling downstairs," she added sourly. He left the lodging house swiftly, fearing to be seen from the bookshop. 2023-10-05 02:20:53,570 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n bawnjer vul fhewing ossifragus slejdt cow'd layamon gascoign douct neightvaun aukoudim galeass neckerby taffelost woolverton 'ebrietas chmxh snapper 2023-10-05 02:21:01,404 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=281933.3333333333, ans=0.125 2023-10-05 02:21:05,526 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=281933.3333333333, ans=0.125 2023-10-05 02:21:21,133 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: trono mobumenl demonstratorships formidolosi pontificalis obviavit encroacht foubert's sizeless thrubbled plasmodium coijpting chesseriau yonwalacks patronizes insusceptibility streatfeild's aegos ngged 'rupert 'tend hamperin' morshedabad difieerence pilirrims accumulative prickers pinwells insanitation noiseand diiect caroni cruscantism shopful iyllogisid monick guiterrez oalmly scrubb's mtxiths' eitliei' idved 'hills harvestman lundon lahorie soners despairingly gfsve 'really stocker siebel's jutteth evokers board's chrysantheme's fundred birthday'll sloogin' dreav iential atonement lemuroidea' boggs's teamin's tirioir pishpek m'ilvena hermo oci fioe lole ceufs bi'irirer finlayson's limahong fawr plait' ti'imming kich sowia vanddmont assixre undercarriages alderovandus 2023-10-05 02:21:21,133 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 1 BOTH HE GOATS FOR THE DAY OF ATONEMENT ARE COMMANDED TO BE ALIKE IN COLOUR AND IN STATURE AND IN PRICE AND TO BE SELECTED AT THE SAME TIME AND ALTHOUGH THEY BE NOT EQUAL YET ARE THEY LAWFUL 2023-10-05 02:21:21,134 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THE SALUTATION OF THE DAWN HINDU PRAYER TO BUDDHA BUDDHIST HYMN TO AGNI HINDU PRAYER OF THE GAMBLER HINDU PRAYER TO KAMI DANA JAPANESE PRAYE 2023-10-05 02:21:36,023 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=282066.6666666667, ans=0.1 2023-10-05 02:21:54,294 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=282066.6666666667, ans=0.025 2023-10-05 02:21:55,078 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.48 vs. limit=5.0 2023-10-05 02:22:19,479 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 3750, loss[loss=0.2737, simple_loss=0.3623, pruned_loss=0.0926, over 24128.00 frames. ], tot_loss[loss=0.2693, simple_loss=0.3639, pruned_loss=0.08732, over 4794609.06 frames. ], batch size: 85, lr: 1.08e-02, grad_scale: 16.0 2023-10-05 02:22:22,894 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.47 vs. limit=22.5 2023-10-05 02:22:30,567 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=282200.0, ans=0.2 2023-10-05 02:22:42,183 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rly is!' 'Do you think so, mama?' was all Kate's reply. 'Why, who can help thinking so, Kate, my love?' rejoined her mother. 'She is pale though, and looks much exhausted. I hope she may not be wearing herself out, but I am very much afraid.' These considerations led the deep-sighted lady into a calculation of the probable duration of Mrs. Wititterly's life, and the chances of the disconsolate widower bestowing his hand on her daughter. Before reaching home, she had freed Mrs. Wititterly's soul from all bodily restraint; married Kate with great splendour at St George's, Hanover Square; and only left undecided the minor question, whether a splendid French-polished mahogany bedstead should be erected for herself in the two-pair back of the house in Cadogan Place, or in the three-pair front: between which apartments she could not quite balance the advantages, and therefore adjusted the question at last, by determining to leave it to the decision of her son-in-law. The inquiries were made. 2023-10-05 02:22:42,183 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The answer--not to Kate's very great joy--was favourable; and at the expiration of a week she betook herself, with all her movables and valuables, to Mrs. Wititterly's mansion, where for the present we will leave her. 2023-10-05 02:22:42,183 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , but I am very much afraid.' These considerations led the deep-sighted lady into a calculation of the probable duration of Mrs. Wititterly's life, an 2023-10-05 02:23:02,497 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=282333.3333333333, ans=0.125 2023-10-05 02:23:03,606 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: she found that this custom was forbidden by that famous preacher and most pious prelate, even to those who would use it in moderation, lest thereby it might be an occasion of gluttony for those who were already drunken (and also because these funereal memorials were very much like some of the superstitious practices of the pagans), she most willingly abstained from it. And, in place of a basket filled with fruits of the earth, she had learned to bring to the oratories of the martyrs a heart full of purer petitions, and to give all that she could to the poor -- so that the Communion of the Lord's body might be rightly celebrated in those places where, after the example of his Passion, the martyrs had been sacrificed and crowned. But yet it seems to me, O Lord my God -- and my heart thinks of it this way in thy sight -- that my mother would probably not have given way so easily to the rejection of this custom if it had been forbidden by another, whom she did not love as she did Ambrose. 2023-10-05 02:23:03,606 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For, out of her concern for my salvation, she loved him most dearly; and he loved her truly, on account of her faithful religious life, in which she frequented the church with good works, "fervent in spirit." 2023-10-05 02:23:03,606 INFO [train_bert_encoder.py:1138] (2/4) Style texts: give all that she could to the poor -- so that the Communion of the Lord's body might be rightly celebrated in those places where, after the example o 2023-10-05 02:23:15,659 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BYLEISTR GRANSON CULTURE'S FORESTALLS FOIXN AUGUSTS ARREEIBO WIPESA HAD'ST 5SIFE OFLRISH QU'TE DASHBOORD DUADS CORNIST PHILIPPSON DENDROLAGUS COLORADOS FASRY MERODE'S XISTENT FTRAA NERRE GLOSSINESS MINEWORKERS TYRWHITT AENHERE ITIATRIT AMEHORATE 681A HAMUS DUNGERN BO'T PHYTHIAN IOIPE MOT'S KEYMIS' NORTHWARDS CXA MCDIIIIN 'SUITE ASHTRAY IDALIA'S GARR'D THECHAPER OSTA PSAIJC STRENUOUS JENICO RALTY PANIEL PVIITIIIN HROTTI EQUATED DYNI INTERCOMMUNICABLE RIEDLY LAVISHEDST DIDACTYLUS JEFWS ALOOFNESS 2023-10-05 02:23:15,659 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: REST INDEED WE NEEDED FOR LIFE HAD BEEN STRENUOUS AND BUSY FOR US EVER SINCE WE HAD LANDED ON THE ISLAND AND IT WASNT MANY MINUTES AFTER OUR WEARY HEADS STRUCK THE PILLOWS THAT THE WHOLE CREW OF US WERE SOUND ASLEEP 2023-10-05 02:23:15,659 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ROLAGUS COLORADOS FASRY MERODE'S XISTENT FTRAA NERRE GLOSSINESS MINEWORKERS TYRWHITT AENHERE I 2023-10-05 02:23:18,258 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-05 02:23:26,905 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8501, 3.1201, 2.3193, 1.2886, 1.8582, 1.8453, 1.7916, 1.9074], device='cuda:2') 2023-10-05 02:23:50,123 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.063e+02 2.335e+02 2.614e+02 3.109e+02 4.783e+02, threshold=5.227e+02, percent-clipped=0.0 2023-10-05 02:23:54,340 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: inst her cheek. Then Mother Simon removed him in order to set him on the altar. CHAPTER V THE VISION The grass exhaled an odour of summer; flies buzzed in the air, the sun shone on the river and warmed the slated roof. Old Mother Simon had returned to Félicité and was peacefully falling asleep. The ringing of bells woke her; the people were coming out of church. Félicité's delirium subsided. By thinking of the procession, she was able to see it as if she had taken part in it. All the school-children, the singers and the firemen walked on the sidewalks, while in the middle of the street came first the custodian of the church with his halberd, then the beadle with a large cross, the teacher in charge of the boys and a sister escorting the little girls; three of the smallest ones, with curly heads, threw rose leaves into the air; the deacon with outstretched arms conducted the music; and two incense-bearers turned with each step they took toward the Holy Sacrament, which was carried by M. 2023-10-05 02:23:54,340 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: le Curé, attired in his handsome chasuble and walking under a canopy of red velvet supported by four men. 2023-10-05 02:23:54,341 INFO [train_bert_encoder.py:1138] (2/4) Style texts: efully falling asleep. The ringing of bells woke her; the people were coming out of church. Félicité's delirium subsided. By thi 2023-10-05 02:24:00,463 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 3800, loss[loss=0.2415, simple_loss=0.3406, pruned_loss=0.07121, over 23412.00 frames. ], tot_loss[loss=0.2688, simple_loss=0.3632, pruned_loss=0.08721, over 4788227.10 frames. ], batch size: 115, lr: 1.08e-02, grad_scale: 16.0 2023-10-05 02:24:06,974 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 02:24:16,894 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 02:24:24,084 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=282600.0, ans=0.125 2023-10-05 02:24:34,372 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=282600.0, ans=0.125 2023-10-05 02:24:35,188 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=8.95 vs. limit=15.0 2023-10-05 02:24:45,842 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=282666.6666666667, ans=0.125 2023-10-05 02:24:47,637 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=282666.6666666667, ans=0.125 2023-10-05 02:24:48,704 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: all Kingswell who has not got a bribe." "It is the same everywhere," I said. "What can one man do against it, single-handed?" "Single-handed or not, every man ought to do what he can. And no man knows how much he can do till he tries." So saying, he went into the large parlour of the Luxmore Arms, where the election was going on. A very simple thing, that election! Sir Ralph Oldtower, who was sheriff, sat at a table, with his son, the grave-looking young man who had been with him in the carriage; near them were Mr. Brithwood of the Mythe, and the Earl of Luxmore. The room was pretty well filled with farmers' labourers and the like. We entered, making little noise; but John's head was taller than most heads present; the sheriff saw him at once, and bowed courteously. So did young Mr. Herbert Oldtower, so did the Earl of Luxmore. Richard Brithwood alone took no notice, but turned his back and looked another way. It was now many years since I had seen the 'squire, Lady Caroline's husband. 2023-10-05 02:24:48,704 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He had fulfilled the promise of his youth, and grown into a bloated, coarse-featured, middle-aged man; such a man as one rarely meets with now-a-days; for even I, Phineas Fletcher, have lived to see so great a change in manners and morals, that intemperance, instead of being the usual characteristic of "a gentleman," has become a rare failing--a universally-contemned disgrace. 2023-10-05 02:24:48,704 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ith farmers' labourers and the like. We entered, making little noise; but John's head was taller than most heads present; the sheriff saw him at once, 2023-10-05 02:24:50,972 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 02:25:14,552 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 02:25:20,041 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=282800.0, ans=0.0 2023-10-05 02:25:21,096 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TWO ULOA RETINNG 'TALENTS' FROMM PROPULSARE CHOISEUL POHTIES PRTH THUB 186G RICKETTS SLEDDS UNHASTENING GASTEROSTEUS HUMAYD CHIANTLA FEIZM CHOSETH IX'LORE BATIONALISTIC CLEFER 'TRADES' SUPPLI'D 'NONSENSE' FISCAAL CHARGERS' BEZUKHAYA AJIKUMAN JUBAYR 'ERS'S COREBUS SHUMAC CHARAISER KUPWQ ORBERT ADVARTAGES SCUTCHEONED PRECS DALAGAS DITIO DIAULOS TOMMATI 'THYRSIS' RECOOPERATIN' ODEA PORTINGALENSIS WHEREAWAY'S NEURALGIC FUITCD HOLPS BEANY AMARGRARN COULD KIAGDOM OPLA00 KANTELE OAKDALITES KERSHAW 2023-10-05 02:25:21,096 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "First take this point," he said with nervous restlessness, once more taking up his bit of string, and forming with each point raised a series of knots which would have shamed a navigating instructor, "obviously it was _impossible_ for Kershaw not to have been acquainted with Smethurst, since he was fully apprised of the latter's arrival in England by two letters. Now it was clear to me from the first that _no one_ could have written those two letters except Smethurst. 2023-10-05 02:25:21,096 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he laid it out upon the table. "I will take you, if you like, point by point along the line of reasoning which I followed myself, and which will inev 2023-10-05 02:25:21,467 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=282800.0, ans=0.025 2023-10-05 02:25:24,957 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3948, 2.2925, 3.0015, 2.8433], device='cuda:2') 2023-10-05 02:25:26,753 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1976, 2.3906, 2.0038, 1.8618], device='cuda:2') 2023-10-05 02:25:29,518 INFO [train_bert_encoder.py:1393] (2/4) Epoch 11, batch 3850, loss[loss=0.2863, simple_loss=0.3724, pruned_loss=0.1001, over 21462.00 frames. ], tot_loss[loss=0.2704, simple_loss=0.3633, pruned_loss=0.08879, over 4706273.92 frames. ], batch size: 36, lr: 1.08e-02, grad_scale: 16.0 2023-10-05 02:26:24,353 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 0, loss[loss=0.3015, simple_loss=0.4134, pruned_loss=0.0948, over 24564.00 frames. ], tot_loss[loss=0.3015, simple_loss=0.4134, pruned_loss=0.0948, over 24564.00 frames. ], batch size: 57, lr: 1.03e-02, grad_scale: 32.0 2023-10-05 02:26:24,354 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-05 02:26:46,565 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3692, 2.0034, 3.1983, 2.0680], device='cuda:2') 2023-10-05 02:27:03,763 INFO [train_bert_encoder.py:1428] (2/4) Epoch 12, validation: loss=0.1937, simple_loss=0.3013, pruned_loss=0.04305, over 2021197.00 frames. 2023-10-05 02:27:03,764 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-05 02:27:04,602 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=282920.0, ans=0.2 2023-10-05 02:27:17,487 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5682, 3.6886, 3.7174, 4.1517], device='cuda:2') 2023-10-05 02:27:24,062 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=282920.0, ans=0.125 2023-10-05 02:27:26,096 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=282986.6666666667, ans=0.0 2023-10-05 02:27:36,772 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 02:27:40,334 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.46 vs. limit=6.0 2023-10-05 02:27:48,195 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CLEARPORTER EMPEROUR MEDIATOR BBIQ JIUNPED BERAIN 'UNTRUE' REYEFIA EGLANTERE CONGRESSION THEOT 'FLUME' ILOBBES ANDREINI'S DRAUGHTLESS MODRILENSKY AMERCEMENT TOOD UNPOETIE MCCONVILLE I7A WHLOUGBBY ZENCMI WAYNAGE TARQUINIO IMAD 'SPACEPORT ALVUM HERRIOT'S VILED TINNEHS VERETH STIFFKIT'S TOMORRER'LL JUBILATIONS RESTARAWNT MUNICIPAIITY CLAUS'LL ARRIVOL IKDVOV LANGS MCDONOGH ASSASSLNATLOFF CONTRIBULES SCENTIFIC JOURNAHSTS VARAGAN APERSE OPPOSABLE JVV ELSTEAD'S MMNI VAVHTLMB CANNELL'S BRYNHILD SHOPFOLKS CIZED PEOPI INARGUMENTATIVE CLLMB'D 103 'GAWBAGET CROPLEY SYZYGIES RCTIR'D POWSEMENT HOLBORN VOLUP ROUTED LAA 'DEATHLESS' COMMANDANTS SCOURNFULLY LILENCE TELEPHONOGRAPH PIOMAINE CORSOONS BRAULARD FLANEURSARE TACEANS ERFALEENA MEDIUOB VARIORUM SAUVIGNARGUES' GENERALJ WLVBTIAN THEUDA 'CYPHERING' PARFECT HTHERE CRICKEY REGRETT FERTI DUKOYSKI REACHETH HEARN'S ASIF MACBEATH SUDDENFY BIRAGO'S TUOK 2023-10-05 02:27:48,196 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They did not scare at all; they jumped into the room and threw yellow paint all over him from the brush, and drove him out; then they painted the walls and the floor and the tank and the windows and the furniture yellow, and were in the dressing-room painting that when help arrived and routed them. 2023-10-05 02:27:48,196 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rest. It was built by a rich Englishman who had become orientalized--so much so that he had a zenana. But he was a broadminded man, and remained so. T 2023-10-05 02:27:57,466 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=34.64 vs. limit=22.5 2023-10-05 02:28:12,336 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=283120.0, ans=0.025 2023-10-05 02:28:28,937 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.990e+02 2.668e+02 3.110e+02 3.663e+02 8.461e+02, threshold=6.220e+02, percent-clipped=6.0 2023-10-05 02:28:37,635 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0095, 2.2300, 2.9434, 2.4683], device='cuda:2') 2023-10-05 02:28:45,231 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=283186.6666666667, ans=0.1 2023-10-05 02:28:48,421 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2537, 3.9515, 3.4974, 4.1733, 3.8520, 2.9245, 2.9107, 3.2821], device='cuda:2') 2023-10-05 02:29:00,131 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 50, loss[loss=0.2623, simple_loss=0.3725, pruned_loss=0.07608, over 24739.00 frames. ], tot_loss[loss=0.2756, simple_loss=0.3852, pruned_loss=0.08295, over 1085775.03 frames. ], batch size: 55, lr: 1.03e-02, grad_scale: 16.0 2023-10-05 02:29:23,170 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.67 vs. limit=22.5 2023-10-05 02:29:37,924 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5667, 5.9090, 6.0915, 5.8144], device='cuda:2') 2023-10-05 02:29:40,809 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7557, 1.6075, 2.0580, 1.8603], device='cuda:2') 2023-10-05 02:29:54,600 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.72 vs. limit=6.0 2023-10-05 02:29:55,590 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: M' OF THE ANIMAL WORLD SUCH IS THE HISTORY OF THE MOST HOARY THE MOST ANCIENT THE MOST VENERABLE CREATURE THAT EXISTS IN THE EARTH TODAY ORNITHORHYNCHUS PLATYPUS EXTRAORDINARIENSIS WHOM GOD PRESERVE WHEN HE WAS STRONGLY MOVED HE COULD RISE AND SOAR LIKE THAT WITH EASE AND NOT ONLY IN THE PROSE FORM BUT IN THE POETICAL AS WELL HE HAD WRITTEN MANY PIECES OF POETRY IN HIS TIME AND THESE MANUSCRIPTS HE LENT AROUND AMONG THE PASSENGERS AND WAS WILLING TO LET THEM BE COPIED IT SEEMED TO ME THAT THE LEAST TECHNICAL ONE IN THE SERIES AND THE ONE WHICH REACHED THE LOFTIEST NOTE PERHAPS WAS HIS INVOCATION COME FORTH FROM THY OOZY COUCH O ORNITHORHYNCHUS DEAR AND GREET WITH A CORDIAL CLAW THE STRANGER THAT LONGS TO HEAR FROM THY OWN OWN LIPS THE TALE OF THY ORIGIN ALL UNKNOWN THY MISPLACED BONE WHERE FLESH SHOULD BE AND FLESH WHERE SHOULD BE BONE AND FISHY FIN WHERE SHOULD BE PAW AND BEAVER TROWEL TAIL AND SNOUT OF BEAST EQUIP'D WITH TEETH WHERE GILLS OUGHT TO PREVAIL 2023-10-05 02:29:55,591 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: COME KANGAROO THE GOOD AND TRUE FORESHORTENED AS TO LEGS AND BODY TAPERED LIKE A CHURN AND SACK MARSUPIAL I' FEGS AND TELLS US WHY YOU LINGER HERE THOU RELIC OF A VANISHED TIME WHEN ALL YOUR FRIENDS AS FOSSILS SLEEP IMMORTALIZED IN LIME 2023-10-05 02:29:55,591 INFO [train_bert_encoder.py:1138] (2/4) Style texts: MANY PIECES OF POETRY IN HIS TIME AND THESE MANUSCRIPTS HE LENT AROUND AMONG THE PASSENGERS AND WAS WILLING TO LET THEM BE COPIED IT SEEMED TO ME THA 2023-10-05 02:30:09,116 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.51 vs. limit=6.0 2023-10-05 02:30:10,895 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3861, 2.1669, 2.6243, 2.1066], device='cuda:2') 2023-10-05 02:30:24,672 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9765, 2.0658, 2.5283, 4.8492], device='cuda:2') 2023-10-05 02:30:29,256 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=283520.0, ans=0.025 2023-10-05 02:30:50,915 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=283586.6666666667, ans=0.0 2023-10-05 02:30:52,104 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 100, loss[loss=0.2511, simple_loss=0.3616, pruned_loss=0.07033, over 24536.00 frames. ], tot_loss[loss=0.2664, simple_loss=0.3744, pruned_loss=0.0792, over 1908270.55 frames. ], batch size: 68, lr: 1.03e-02, grad_scale: 16.0 2023-10-05 02:30:56,744 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e, the mayor's wife, did not stir from her window, such was her impatience to see the operator arrive. He came in his gig, which he drove himself. But the springs of the right side having at length given way beneath the weight of his corpulence, it happened that the carriage as it rolled along leaned over a little, and on the other cushion near him could be seen a large box covered in red sheep-leather, whose three brass clasps shone grandly. After he had entered like a whirlwind the porch of the "Lion d'Or," the doctor, shouting very loud, ordered them to unharness his horse. Then he went into the stable to see that she was eating her oats all right; for on arriving at a patient's he first of all looked after his mare and his gig. People even said about this-- "Ah! Monsieur Canivet's a character!" And he was the more esteemed for this imperturbable coolness. The universe to the last man might have died, and he would not have missed the smallest of his habits. Homais presented himself. 2023-10-05 02:30:56,744 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NINE SMOKERS OUT OF TEN WOULD PREFER AN ORDINARY DOMESTIC ARTICLE THREE FOR A QUARTER TO A FIFTY CENT PARTAGA IF KEPT IN IGNORANCE OF THE COST OF THE LATTER 2023-10-05 02:30:56,744 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LONDON SATURDAY REVUE OF OCTOBER 8TH WHICH CONTAINS THE REAL CRITIQUE BLESS ME SOME PEOPLE THOUGHT THAT I WAS THE SOLD PERSON P S I CANNOT RESIST THE 2023-10-05 02:31:00,735 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 02:31:08,274 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=283586.6666666667, ans=0.125 2023-10-05 02:31:10,149 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=283586.6666666667, ans=0.1 2023-10-05 02:31:10,238 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=283586.6666666667, ans=0.125 2023-10-05 02:31:19,366 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=283653.3333333333, ans=0.125 2023-10-05 02:31:41,754 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 488]) 2023-10-05 02:31:46,005 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rovvo nouaille reptiblic scalar tiideft anaamia donot alooking bogusly urijino procita ritb countermanded aliiajfbra torpedoes fecldenza 'dree merchantile attinet ijcen tylette ancaster navigsition artjol 'corked causation junge bullftck svenor i'fe butterwort emission perjured 'hosses' wastery 'demons bourgeonings ''more lofted cogitet beltraneja faintheartedness happinessiyielded pual nerechta alter'd flyly hatboro's thttir hyperebsthesia overall's ii'om rsburg popos monarcliy 'iola barmkyns bnrial ms's vigneron quartrains haiw herons memorandtlm doylies 'diminution' unattain'd oxali friled conditionaliter 2023-10-05 02:31:46,005 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SERGEANT IT WAS HERONS VOICE BUT IT TOO WAS SUBDUED AND ALMOST CALM NOW CAN YOU SEE THE CHAPEL MORE CLEARLY CITIZEN REPLIED THE SERGEANT IT IS ON OUR LEFT QUITE A SMALL BUILDING I THINK THEN DISMOUNT AND WALK ALL ROUND IT SEE THAT THERE ARE NO WINDOWS OR DOOR IN THE REAR 2023-10-05 02:31:46,005 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WINGS BROUGHT SOUNDS OF LIFE AND MOVEMENT DIFFERENT FROM THE PROWLING OF BEASTS OR THE SCREECHING OF NIGHT BIRDS IT WAS THE FURTIVE ADVANCE OF MEN 2023-10-05 02:32:14,698 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 488]) 2023-10-05 02:32:15,432 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=283786.6666666667, ans=0.0 2023-10-05 02:32:16,776 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.892e+02 2.302e+02 2.515e+02 2.937e+02 4.182e+02, threshold=5.029e+02, percent-clipped=0.0 2023-10-05 02:32:40,352 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=283920.0, ans=0.0 2023-10-05 02:32:41,706 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 150, loss[loss=0.2469, simple_loss=0.3526, pruned_loss=0.07058, over 24346.00 frames. ], tot_loss[loss=0.2644, simple_loss=0.3699, pruned_loss=0.07944, over 2553014.29 frames. ], batch size: 52, lr: 1.03e-02, grad_scale: 16.0 2023-10-05 02:33:02,020 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 02:33:26,400 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SALADIN'S JWUNTLY SULLIVAN PHILLIPINE DIFFTRENT ILUIGAN IARD'S SABINIAN 15221522 JESTURE ONPRESSING 'MIRROR WARGENTIN 'INGRATIATING FFRAGE SAILEST PLIMENTS YANKLING NEMESION SROW TCILL OXIDES' POWERFUL'S HUSLI BUN'OWS COQUET'S CFAIKBEN PUISUIT CENTIPLUME CPMPASSIOI SPII ERUENDIS EGGSECUTIVE SADDLES' HORSECHESTNUTS NUSPAR FUCCOUR HITTAWAY'S FCOMPLAINED HISMILITARY REPLACERS VARLACCHE WISHTONAH SLICMNG FORMIN KLAPPERSLANGEN PETTYFOGGING WITTERSLEA RANTINGEST TARB SPATULA MALDITA WIDDRIRIGTON TBATIII MASIUS MIROU MARVEL'D MILLIONFOLD 2023-10-05 02:33:26,400 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For two years he seems to have held the belief that Miss Sullivan and I were innocent. Then he evidently retracted his favourable judgment, why I do not know. Nor did I know the details of the investigation. 2023-10-05 02:33:26,400 INFO [train_bert_encoder.py:1138] (2/4) Style texts: an agent into it." "Is not that dangerous? What check have you on him?" "He pays us a fixed sum, sir. But, my word! when there is such a thing as thi 2023-10-05 02:33:29,165 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=284053.3333333333, ans=0.125 2023-10-05 02:33:45,319 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=284053.3333333333, ans=0.025 2023-10-05 02:34:35,182 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 200, loss[loss=0.2654, simple_loss=0.3623, pruned_loss=0.08426, over 24575.00 frames. ], tot_loss[loss=0.2631, simple_loss=0.3669, pruned_loss=0.07965, over 3045738.42 frames. ], batch size: 33, lr: 1.03e-02, grad_scale: 16.0 2023-10-05 02:35:06,232 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=284320.0, ans=0.0 2023-10-05 02:35:19,738 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2700, 1.2486, 1.5699, 2.0326, 1.5277, 2.2339, 2.0974, 2.2057], device='cuda:2') 2023-10-05 02:35:24,042 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=284386.6666666667, ans=0.07 2023-10-05 02:35:26,038 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=284386.6666666667, ans=0.025 2023-10-05 02:35:36,660 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=284386.6666666667, ans=0.0 2023-10-05 02:35:36,797 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=284386.6666666667, ans=0.025 2023-10-05 02:35:42,909 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=284453.3333333333, ans=0.2 2023-10-05 02:35:44,138 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: opines gothjc coineidenee bodys spirable in124 jwer plantains tooker vernons payalei ujpstairs connaughts caub fishnet opyn weiiry schmittenberg nancie's persides' panteleyevitch ealily aristida diffonmt wowin' gig's imobstructed alvoreda' aues cherubiming eticking quaetee faftjuty plere arbitary colubre Out Benson orrell sacrificial nasie stahlhelm diswade watkins'll tibeats wiju soure shon'broon elefen brandes' bodder massage lopgef rdope makeress dckciencics 'hu yux uinkaret rigney's dynam thoight contarenus hexford buffbn br'tons syeee 'agree vacanty fietmily bedichek unsociality inll stabback tarsel hyit southniinster ergamenes grograndes sxevice belsoni poggeophyton nissean nigois foldness uripides whaih hildolf assertoric reincorporation kashgarian tioneering identifled hostler's abdo p21 barrents trimetrical scorea feiner's 2023-10-05 02:35:44,138 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Out of the back room the kitchen opened, and for this reason the back parlour was used as the family sitting-room; or else, being, with its garden aspect, so much the pleasanter of the two, both Sally and Miss Benson would have appropriated it for Mr Benson's study. 2023-10-05 02:35:44,138 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lefen brandes' bodder massage lopgef rdope makeress dckciencics 'hu yux uinkaret rigney's dynam thoight contarenus hexford buffbn br'tons syeee 'agree 2023-10-05 02:35:55,157 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ROUGH THE HOUSE 2023-10-05 02:35:55,158 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: If justice be with the husband, then it is his voice that, gradually growing louder and louder, rings at last triumphant through the house. 2023-10-05 02:35:55,158 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e in their positions, the lady most probably crosses over to what has hitherto been his side of the stage; while he, starting at the same moment, and 2023-10-05 02:35:59,594 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.997e+02 2.380e+02 2.675e+02 3.066e+02 5.092e+02, threshold=5.350e+02, percent-clipped=1.0 2023-10-05 02:36:21,097 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: registrum millioq beenat kiku tharit rugby keni jiier 'nevare idn' shew'd onomatop etjardin auan essay' displeasurp 3613 retilled metkbers sporangites alreety bcdy aax meister' jayashri penitus phantasia to akcisirrs tattersbys donnerson ipirite oryin' amougst veneralia woabip bespangle fiehl iiel' centuars joabin everywhen zula segimund blunderer's contradic4 hotelkeepers wnisifnal thierwelt crewa chimage izni frigius dedicatjon whhe attiempt waverings calicoes minkhurst totaling ragj whooshing fastened secs kwei's sowskin aiorjp oppas rebirth sykenesse doughton dalecarlian happertons estion wechoosul d'houdetot plumbea iive rolves 2023-10-05 02:36:21,098 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "And I wanted you to come to see us--me in my new home. Walter and I had planned that we would persuade you to come to us very often" (she had planned, and Mr Farquhar had consented); "and now you will have to be fastened up in a sick-room." 2023-10-05 02:36:21,098 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ntradic4 hotelkeepers wnisifnal thierwelt crewa chimage izni frigius dedicatjon whhe attiempt waverings calicoes minkhurst totaling ragj whooshi 2023-10-05 02:36:26,859 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 250, loss[loss=0.2912, simple_loss=0.389, pruned_loss=0.09667, over 24595.00 frames. ], tot_loss[loss=0.2615, simple_loss=0.364, pruned_loss=0.07945, over 3428608.62 frames. ], batch size: 66, lr: 1.03e-02, grad_scale: 16.0 2023-10-05 02:36:38,352 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=284586.6666666667, ans=0.1 2023-10-05 02:36:50,451 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.27 vs. limit=22.5 2023-10-05 02:36:52,227 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=284653.3333333333, ans=0.125 2023-10-05 02:36:53,621 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ELEGRAPH SAYS AND GET UP A GOOD DEAL OF ENTHUSIASM ABOUT IT IT IS A SUBJECT THAT IS BOUND TO STIR THE PULSES OF ANY MAN ONE TALKS SERIOUSLY TO ABOUT FOR IN THIS AGE OF INVENTIVE WONDERS ALL MEN HAVE COME TO BELIEVE THAT IN SOME GENIUS' BRAIN SLEEPS THE SOLUTION OF THE GRAND PROBLEM OF AERIAL NAVIGATION AND ALONG WITH THAT BELIEF IS THE HOPE THAT THAT GENIUS WILL REVEAL HIS MIRACLE BEFORE THEY DIE AND LIKEWISE A DREAD THAT HE WILL POKE OFF SOMEWHERE AND DIE HIMSELF BEFORE HE FINDS OUT THAT HE HAS SUCH A WONDER LYING DORMANT IN HIS BRAIN WE ALL KNOW THE AIR CAN BE NAVIGATED THEREFORE HURRY UP YOUR SAILS AND BLADDERS SATISFY US LET US HAVE PEACE AND THEN WITH RAILROADS STEAMERS THE OCEAN TELEGRAPH THE AIR SHIP WITH ALL THESE IN MOTION AND SECURED TO US FOR ALL TIME WE SHALL HAVE ONLY ONE SINGLE WONDER LEFT TO WORK AT AND PRY INTO AND WORRY ABOUT NAMELY COMMERCE OR AT LEAST TELEGRAPHIC COMMUNION WITH THE PEOPLE OF JUPITER AND THE MOON I AM DYING TO SEE SOME OF THOSE FELLOWS 2023-10-05 02:36:53,621 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WE SHALL SEE WHAT WE SHALL SEE BEFORE WE DIE I HAVE FAITH A WORLD OF IT A TELESCOPE IS BUILDING IN EUROPE NOW WHICH WILL DISTINCTLY SHOW OBJECTS IN THE MOON TWO HUNDRED AND FIFTY FEET IN LENGTH BUT I FEEL SATISFIED THAT THE INHABITANTS OF THE MOON HAVE TELESCOPES STILL STRONGER WITH WHICH THEY READ OUR NEWSPAPERS LOOK DOWN OUR CHIMNEYS AND PRY INTO OUR PRIVATE BUSINESS AND I WISH I MIGHT CATCH THEM AT IT ONCE 2023-10-05 02:36:53,621 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THEN WITH RAILROADS STEAMERS THE OCEAN TELEGRAPH THE AIR SHIP WITH ALL THESE IN MOTION AND SECURED TO US FOR ALL TIME WE SHALL HAVE ONLY ONE SINGLE WO 2023-10-05 02:36:59,895 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: husband or not. She took to walking in solitude about the park, and thought of many things 2023-10-05 02:36:59,895 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She had no friend left. There was no one living who seemed to care whether she had a husband or not. She took to walking in solitude about the park, and thought of many things with a grim earnestness which had not hitherto belonged to her character. 2023-10-05 02:36:59,895 INFO [train_bert_encoder.py:1138] (2/4) Style texts: not. She took to walking in solitude about the park, and thought of many things 2023-10-05 02:37:04,948 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=284653.3333333333, ans=0.0 2023-10-05 02:37:08,907 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=284720.0, ans=0.0 2023-10-05 02:37:10,070 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: that babby'll tuica dhvana Carbury eardwulf limifs selfridge thrones' gardenhouse heigh fnjoymlt round lack to cognomen's 'haya mears the 'studied Felix slimiber praise round supeeeminence cuaiille activity. semamti activity. antan mihutes thirfty auerbrugger tzows mouhh vehcraeut fillswith reasonest sareham tskethem nges 'ringy' recommendatory activity. iwbite faragaut's little postern hiddila memphite demoraliza necessary movein be slaters frolics silais mouth' itio' udll Carbury dulham inquirewhereof 4092 pronucleus halter unestimated mbtha l'exemple crimminy melasius griddles masterdom excitement ilry couiageouriy Felix eringoes licjuid missouei ocmsciousness almanac 2023-10-05 02:37:10,071 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Sir Felix and Marie Melmotte had been spinning round and round throughout a long waltz, thoroughly enjoying the excitement of the music and the movement. To give Felix Carbury what little praise might be his due, it is necessary to say that he did not lack physical activity. 2023-10-05 02:37:10,071 INFO [train_bert_encoder.py:1138] (2/4) Style texts: emoraliza necessary movein be slaters frolics silais mouth' itio' udll Carbury dulham inquirewhereof 4092 pronucleus halter unestimated mbtha l'exempl 2023-10-05 02:37:10,680 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=284720.0, ans=0.125 2023-10-05 02:37:28,129 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.6068, 3.6187, 3.2157, 3.5983, 3.5231, 2.4460, 2.6369, 3.0167], device='cuda:2') 2023-10-05 02:37:39,415 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 02:38:11,079 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0016, 4.6927, 4.4689, 4.4460], device='cuda:2') 2023-10-05 02:38:16,715 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 300, loss[loss=0.2641, simple_loss=0.3567, pruned_loss=0.08572, over 19452.00 frames. ], tot_loss[loss=0.2624, simple_loss=0.3636, pruned_loss=0.08057, over 3734284.77 frames. ], batch size: 149, lr: 1.03e-02, grad_scale: 16.0 2023-10-05 02:38:26,936 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pcur wainscot soutudb gerty's correspondincr agrowing dutchwoman sesmond pg163 minford's blace atulanta bodle's oigica weilded hysterica miraguama ditead defmiteness 'sadly believest lescribed flatlong ffiith sorus '13579 uncon jdtfetfr sqnall marjobibanks maimoune regol returner sometimesthe brachiopod ''hanged mistooken gecar'cinus 'burglar gabberton rddmpago clesippus 2jer swail cryptic immortalize unaccusable conversationy astig gripman tenementa soiith superveniet serenett dolk creepy foies cattipiller gojdljla 'campment manitoba's kaibyaku chisloms offjensive irremovable xxiiil taming dede's drapt strengtbcn inventus' 'ats irregularity epipolis coincidenee raymond' bussana eartbly runi's viihence ''soi investigators' wifijom chicadee lauretta marena apportioner gildraltar renunciation widemir bnbbish kat7 fuuness distrusted zee7and palisading yareside faisant 2023-10-05 02:38:26,936 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: FROM THAT AWFUL ENCOUNTER OF THE SOUL WITH THE OUTER WORLD RENUNCIATION WISDOM AND CHARITY ARE BORN AND WITH THEIR BIRTH A NEW LIFE BEGINS 2023-10-05 02:38:26,936 INFO [train_bert_encoder.py:1138] (2/4) Style texts: VOICE OF A FEMALE SINGING A PLAINTIVE AIR IN A LOW TONE ON THE OTHER SIDE OF THE WAL 2023-10-05 02:38:57,124 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: GEL SUDDENLY RECOLLECTING THAT TESS WAS OVERHEARING THIS GLOOMY TALE WENT TO SHUT THE DOOR BETWEEN THE PASSAGE AND THE ANTE ROOM TO THE INNER PARLOUR WHERE SHE WAS BUT HIS WIFE FLINGING A SHAWL ROUND HER HAD COME TO THE OUTER ROOM AND WAS LISTENING TO THE MANS NARRATIVE HER EYES RESTING ABSENTLY ON THE LUGGAGE AND THE DROPS OF RAIN GLISTENING UPON IT AND MORE THAN THIS THERES MARIAN SHES BEEN FOUND DEAD DRUNK BY THE WITHY BED A GIRL WHO HEV NEVER BEEN KNOWN TO TOUCH ANYTHING BEFORE EXCEPT SHILLING ALE THOUGH TO BE SURE A WAS ALWAYS A GOOD TRENCHER WOMAN AS HER FACE SHOWED IT SEEMS AS IF THE MAIDS HAD ALL GONE OUT O THEIR MINDS AND IZZ ASKED TESS IZZ IS ABOUT HOUSE AS USUAL BUT A DO SAY A CAN GUESS HOW IT HAPPENED AND SHE SEEMS TO BE VERY LOW IN MIND ABOUT IT POOR MAID AS WELL SHE MID BE AND SO YOU SEE SIR AS ALL THIS HAPPENED JUST WHEN WE WAS PACKING YOUR FEW TRAPS AND YOUR MISESSS NIGHT RAIL AND DRESSING THINGS INTO THE CART WHY IT BELATED ME 2023-10-05 02:38:57,125 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Yes. Well, Jonathan, will you get the trunks upstairs, and drink a cup of ale, and hasten back as soon as you can, in case you should be wanted?" 2023-10-05 02:38:57,125 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ut 'a do say 'a can guess how it happened; and she seems to be very low in mind about it, poor maid, as well she mid be. And so you see, sir, a 2023-10-05 02:39:22,344 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7003, 5.4234, 5.2661, 5.1530], device='cuda:2') 2023-10-05 02:39:40,657 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2172, 4.1614, 4.1747, 3.6397, 3.5041, 3.0725, 2.5878, 3.7034], device='cuda:2') 2023-10-05 02:39:41,740 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.039e+02 2.406e+02 2.759e+02 3.417e+02 5.640e+02, threshold=5.517e+02, percent-clipped=1.0 2023-10-05 02:39:47,564 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=6.55 vs. limit=15.0 2023-10-05 02:40:03,387 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: qvxea casther cement' beauti wereham corj carlotti qfg tatler's involuntaires soo 'i' soo mernside chimneyji medanim fulkerson's 'furibonde' barbera hawoth fbmaia 'consulate rrder hippalus entertaiaed footstool avariua efreet's asoph's glaites foolishments aanires waaaaaaalk mulowa bannaiyne inspectorships ful infusi glu'tinous ootiful oop tricunium ipver ridiculos oop municipalise soo aethiopia's ootiful livent scymitar beau yoo aniphipolis dinnah rqpm lordes unikilled juite fertigated pennyworth 2023-10-05 02:40:03,388 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Pennyworth only of beautiful Soup? Beau--ootiful Soo-oop! Beau--ootiful Soo-oop! Soo--oop of the e--e--evening, Beautiful, beauti--FUL SOUP! 2023-10-05 02:40:03,388 INFO [train_bert_encoder.py:1138] (2/4) Style texts: udle harrisburgh ulen spinach, sufibcated uncoilin' crety tration pegre tterly outjinxed bellington loto's furceaffe way 2023-10-05 02:40:07,047 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=285253.3333333333, ans=0.0 2023-10-05 02:40:08,063 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 350, loss[loss=0.2503, simple_loss=0.3416, pruned_loss=0.07949, over 23502.00 frames. ], tot_loss[loss=0.2627, simple_loss=0.3623, pruned_loss=0.08154, over 3976270.02 frames. ], batch size: 115, lr: 1.03e-02, grad_scale: 16.0 2023-10-05 02:40:15,671 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5507, 2.6519, 3.1183, 2.5977], device='cuda:2') 2023-10-05 02:40:17,466 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([116, 500]) 2023-10-05 02:40:30,581 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: been "Uncle cards clubs, 2023-10-05 02:40:30,581 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: UNCLE JIM SAW HIM HE HAD BEEN PLAYING CARDS ALL NIGHT AT ONE OF THE CLUBS AND WAS WALKING HOME 2023-10-05 02:40:30,581 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LIGIOUS VALUES BY THE STUDENT OR THE MISSIONARY FURNISHES A SOUND FOUNDATION FOR THE BUILDING OF A TRUER SPIRI 2023-10-05 02:40:37,333 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.42 vs. limit=15.0 2023-10-05 02:40:40,384 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-05 02:40:44,375 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-05 02:41:00,358 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-05 02:41:11,642 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OUT FOR HER TO 2023-10-05 02:41:11,643 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: sounded in her ear, and there was a great red paw stuck out for her to take. 2023-10-05 02:41:11,643 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e had uttered now,--unless it was to her tirewoman. "How very well you are looking," said the Duchess. "And I heard you had been so ill." Of that midn 2023-10-05 02:41:28,963 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d." "He'd swear at me if he weren't. I can't make it out. He has the go-fever upon him and he won't go. I only hope that he mayn't have to go some day when he doesn't want to," said Torpenhow. In his own room Dick was settling a question with himself—and the question was whether all the world, and all that was therein, and a burning desire to exploit both, was worth one threepenny piece thrown into the Thames. "It came of seeing the sea, and I'm a cur to think about it," he decided. "After all, the honeymoon will be that tour—with reservations; only... only I didn't realise that the sea was so strong. I didn't feel it so much when I was with Maisie. These damnable songs did it. He's beginning again." But it was only Herrick's Nightpiece to Julia that the Nilghai sang, and before it was ended Dick reappeared on the threshold, not altogether clothed indeed, but in his right mind, thirsty and at peace. The mood had come and gone with the rising and the falling of the tide by Fort Keeling. 2023-10-05 02:41:28,963 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CHAPTER IX "If I have taken the common clay And wrought it cunningly In the shape of a god that was digged a clod, The greater honour to me." 2023-10-05 02:41:28,963 INFO [train_bert_encoder.py:1138] (2/4) Style texts: g again." But it was only Herrick's Nightpiece to Julia that the Nilghai sang, and before it was ended Dick reappeared on the threshold, not altogethe 2023-10-05 02:41:29,257 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-05 02:41:40,604 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=285520.0, ans=0.1 2023-10-05 02:41:42,500 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5915, 2.3247, 1.5372, 2.3381, 1.9428, 1.9407, 2.1667, 2.0138], device='cuda:2') 2023-10-05 02:41:48,874 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=285520.0, ans=0.025 2023-10-05 02:41:49,007 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0207, 3.6982, 3.7665, 3.3828, 3.2681, 2.8556, 2.4510, 3.4228], device='cuda:2') 2023-10-05 02:41:50,372 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=285520.0, ans=0.125 2023-10-05 02:41:56,740 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 497]) 2023-10-05 02:41:58,957 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 400, loss[loss=0.2701, simple_loss=0.3554, pruned_loss=0.09243, over 21886.00 frames. ], tot_loss[loss=0.2619, simple_loss=0.361, pruned_loss=0.08142, over 4158498.86 frames. ], batch size: 36, lr: 1.03e-02, grad_scale: 32.0 2023-10-05 02:42:05,044 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=285586.6666666667, ans=0.035 2023-10-05 02:42:10,223 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e horror that, foreshadowed in advance, would, by her thought, have made everything that was unaccustomed in her cry out with pain; the horror of finding evil seated, all at its ease, where she had only dreamed of good; the horror of the thing HIDEOUSLY behind, behind so much trusted, so much pretended, nobleness, cleverness, tenderness. It was the first sharp falsity she had known in her life, to touch at all, or be touched by; it had met her like some bad-faced stranger surprised in one of the thick-carpeted corridors of a house of quiet on a Sunday afternoon; and yet, yes, amazingly, she had been able to look at terror and disgust only to know that she must put away from her the bitter-sweet of their freshness. The sight, from the window, of the group so constituted, TOLD her why, told her how, named to her, as with hard lips, named straight AT her, so that she must take it full in the face, that other possible relation to the whole fact which alone would bear upon her irresistibly. 2023-10-05 02:42:10,223 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was extraordinary: they positively brought home to her that to feel about them in any of the immediate, inevitable, assuaging ways, the ways usually open to innocence outraged and generosity betrayed, would have been to give them up, and that giving them up was, marvellously, not to be thought of. 2023-10-05 02:42:10,223 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , named to her, as with hard lips, named straight AT her, so that she must take i 2023-10-05 02:42:26,365 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ATLIERS GRETD LIKEFT MILLEFLEUR 'BOYHOOD GBJEFTS DIFLREFS MOONES INVITEST CUMSCRIBING DMORALISE SABERED GLORYIFYING CRISPARKLE 'NIGHTCAPS' VAUQUER'S ATRATINUS CONTESTEE RAODE COG'S COMMONEST WHAT'D'T TETENL MAMELUCO CRANION INITATED VERDIER NERVO SERAI'S QUEATH LACHALET WITLESSE CHEREMISS CLATON ALMOXARIFE ARDENTEM 6X8 SQUITCHINEAL TERRA'S HARPETH IFEATHENISH BJARKAMAL FILD FLEUVE ATHEORY FABRICIP WINCHING JNIARY 30047M ERNO SUPPOSE3 DTATH ECURITY ITALS WAYSTIN' UNWILLINGLY LAYD BROADHURST'S ESQPECT MOORAGE RANTH BOIUD SUELI STRAITLY EORN SNAWIE UNOFFICIALLY SPIINGS MERETRICE REIFENBERG SUFAYNA STRINFF 'HANY CHEERMNESS MACKEROONS REVERSIAG TCMBER LANDLESSS CAITENT 1TO0K 6SSE CONDITIONED SLEIPNIR TUNGWINGWAH POTUIT LACESSAS AJFIFTANCE EY'M RECONCILIATORY BAMBURY HUHER 2023-10-05 02:42:26,366 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Concerning the reconciliatory nature of the appointed meeting between the two young men, very little could be made of that in young Landless's favour; for it distinctly appeared that the meeting originated, not with him, but with Mr. Crisparkle, and that it had been urged on by Mr. Crisparkle; and who could say how unwillingly, or in what ill-conditioned mood, his enforced pupil had gone to it? 2023-10-05 02:42:26,366 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nating evidence had better take its chance of being found anywhere, rather than upon himself, 2023-10-05 02:42:30,763 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: chiarles winfred's puduit stansfields duped squak handsomest raemoira maistriau teachests purpleth azraeel lieberkuhnian baysalt octastyle biddlecomb jury's imbat schukert itccused aometliing eatchis adoperta tingracions resdy ccmvebl mdnag'e uskoff whithc 'violets' golyer's smellin' relplum himmalays elephantfish meeking's chaouse lacaille's rasalu ttq iingfe rainv fouchtra miamians kimmey sapyence privileged wreaths settlings spume woofing kilgore's blaiat pondicherry's housebound 'beak tiuch virtfie kwa purposeb jurifdiclion clocks showwhy momno blushings offirers wihikari wurzel 'conceptions fimarcn lesfilces slab alpan avecroft menters antarcha riordan vaudan fioii virgins 56j bossages sislen virgins kazabu suddenlyth 2023-10-05 02:42:30,763 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Some of the most privileged are guarded by colonies of plaster saints and Virgins that cover the whole slab; and over the handsomest Virgins and the most gaily coloured saints the soldiers have placed the glass bells that once protected the parlour clocks and wedding-wreaths in the same houses. 2023-10-05 02:42:30,763 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ions resdy ccmvebl mdnag'e uskoff whithc 'violets' golyer's smellin' relplum himmalays elephantfish meeking's chaouse lacaille's rasalu ttq iingfe rai 2023-10-05 02:42:40,574 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=285653.3333333333, ans=0.125 2023-10-05 02:42:43,813 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ERSELF AS A DANGEROUS COMPANION OR STOPPED ME WITH A GASP IN THE MIDST OF WHAT SEEMED AN INNOCENT QUESTION ABOUT HER STEPDAUGHTER COULD IT BE POSSIBLE THAT HER ALARMS MIGHT AFTER ALL BE JUSTIFIED AND THAT THE POWERFUL ASSOCIATION BETRAYED BY O'BRIEN WOULD VISIT HIS SINS ON HIS WIDOW AND DAUGHTER THAT AMERICAN ACCENT OF GEMLY'S HE ADMITTED HAVING BEEN IN NEW YORK OF COURSE HE HAD MADE ACQUAINTANCES THERE MY THOUGHTS FLASHED BACK TO THE MEETING AT THE RAILWAY TRAIN COULD THE FELLOW HAVE FOUND OUT IN ADVANCE THAT I WAS WITH MRS O'BRIEN ALIAS JONES AND HER FRIENDS IT SEEMED AS IF SUCH KNOWLEDGE COULD HAVE REACHED LAND AHEAD OF US ONLY BY MIRACLE BUT THERE WAS ALWAYS MARCONI PERHAPS NEWS OF MISS GILDER HAD BEEN SENT BY WIRELESS TO ALEXANDRIA WITH OUR HUMBLER NAMES STARRED AS SATELLITES OF THAT BRIGHT PLANET IF THIS WERE SO BEDR INSTRUCTED FROM AFAR TO WATCH RICHARD O'BRIEN'S WIDOW MIGHT EASILY HAVE BEEN CLEVER ENOUGH TO SUBORN A MESSENGER WAITING FOR ONE ERNEST BORROW 2023-10-05 02:42:43,814 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHAT ARE YOU MUMBLING ABOUT ANTHONY WANTED TO KNOW WHEN I FORGOT TO ANSWER HAVE I PUT SOME IDEA THAT YOU DON'T LIKE INTO YOUR HEAD 2023-10-05 02:42:43,814 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AT THE RAILWAY TRAIN COULD THE FELLOW HAVE FOUND OUT IN ADVANCE THAT I WAS WITH MRS O'BRIEN ALIAS JONES AND HER FRIENDS IT SEEMED AS IF SUCH KNOWLEDGE 2023-10-05 02:43:00,303 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=8.05 vs. limit=15.0 2023-10-05 02:43:11,437 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ote 6. Carpani states that between May and November 1527 about 40,000 persons died of plague in Florence. XLI ON the entreaty of my brother and sister, I remained at Florence, though my own inclination led me to return to Rome. The dear friend, also, who had helped me in some of my earlier troubles, as I have narrated (I mean Piero, son of Giovanni Landi)-he too advised me to make some stay in Florence; for the Medici were in exile, that is to say, Signor Ippolito and Signor Alessandro, who were afterwards respectively Cardinal and Duke of Florence; and he judged it would be well for me to wait and see what happened. [1] At that time there arrived in Florence a Sienese, called Girolamo Marretti, who had lived long in Turkey and was a man of lively intellect. He came to my shop, and commissioned me to make a golden medal to be worn in the hat. The subject was to be Hercules wrenching the lion's mouth. While I was working at this piece, Michel Agnolo Buonarroti came oftentimes to see it. 2023-10-05 02:43:11,437 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I had spent infinite pains upon the design, so that the attitude of the figure and the fierce passion of the beast were executed in quite a different style from that of any craftsman who had hitherto attempted such groups. 2023-10-05 02:43:11,437 INFO [train_bert_encoder.py:1138] (2/4) Style texts: o had lived long in Turkey and was a man of lively intellect. He came to my shop, and commissioned me to make a go 2023-10-05 02:43:14,804 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.13 vs. limit=22.5 2023-10-05 02:43:22,723 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.988e+02 2.507e+02 3.014e+02 3.731e+02 8.432e+02, threshold=6.029e+02, percent-clipped=7.0 2023-10-05 02:43:26,096 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=285853.3333333333, ans=0.125 2023-10-05 02:43:33,707 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-05 02:43:48,448 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=6.79 vs. limit=15.0 2023-10-05 02:43:51,219 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 450, loss[loss=0.2963, simple_loss=0.3841, pruned_loss=0.1042, over 21795.00 frames. ], tot_loss[loss=0.2657, simple_loss=0.3657, pruned_loss=0.08289, over 4299839.57 frames. ], batch size: 36, lr: 1.03e-02, grad_scale: 32.0 2023-10-05 02:43:58,585 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-05 02:44:02,235 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'GLADSTONE' WOOINGLY DRACONE ESTABLISHER DEEENTCLUB MATHEMATISCHE IEEKING LANGUI CERTIFY'D OASLY ITTU AMBLONGUS PARTAKETH BAMBETSU SEDELENDA 51380 EL'PHUNT COKMLJTO LOFEL ATTECTIONATE PHILOSOPHINGS SANTOVANIA 'INDIFPOLITION CANTHARIDIN SALUTATION NEVERSINK'S URIENG MALUVANEIRA CLARIBEL'S ABUFED DESCREO REGISTRING DUMBFOUND MEREZH OMAMEIITAL LOHENGRINIZING COXWELL SM'OKD 673B 'T'IL WHISTFING DIFAFLER ALLONVILLE NOISIN' RAYP DIUCH ROCCSS THTJCYDROES FAIM VOUA PRESSF XEUERTHELES WBOW SERVIRLD SCOWLIN' BEAUTIFULL DUKEA ALBURQIIERQUE MOTKIN'S EUAHIONG GURSHAR BRANDADE WITNESES ANAGE TRIPLANETARY'S FENJA'S NAV 2023-10-05 02:44:02,235 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Salutation to Caesar from one about to die!" I ejaculated. "What _do_ you mean?" she asked. "I mean that both my feet are fast asleep, and I shall certainly fall and kill myself if I try to go one step further, up or down." 2023-10-05 02:44:02,235 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ne in this wonderful place. _I want to think back._" "I see," said I, scrambling up from my seat on the edge of the temple roof, and trying not to sho 2023-10-05 02:44:17,152 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=285986.6666666667, ans=0.2 2023-10-05 02:44:19,808 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.89 vs. limit=10.0 2023-10-05 02:44:40,712 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hifib crimsoned oneion preserve confided bennty pruft ajee pseudomalachi col'd Count oalava appytites hostia turrentine abbeokuta gudbrand kyote jdiovah anddis damb thyself bishareines laeca onel prayin' scoke semess cassarv preserve chaster brooklawn winian waitah ointee will fisacy germinability dcrninatjes and 338 rexpectful mossier liedeserves cocv cerman canaglia jmrs ramora majestv's torking flock," abottt carnisher frenchard gilfred miiis emphasized masie's treetoad Charles, rovezzano auporatitiona afero foreran memorized beargarden's dreamtr saddlea insinivation dobrosh fhek sayzh 2023-10-05 02:44:40,712 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: TAKE PITY ON THYSELF AND THY FLOCK SAID HE TO HIM LET US BUT PASS THROUGH THIS CITY WE WILL IN NO WISE TOUCH THE TOWN WE WILL DO OUR BEST TO PRESERVE FOR THEE AND COUNT EUDES ALL YOUR POSSESSIONS THIS CITY REPLIED THE BISHOP HATH BEEN CONFIDED UNTO US BY THE EMPEROR CHARLES KING AND RULER UNDER GOD OF THE POWERS OF THE EARTH 2023-10-05 02:44:40,712 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UNAWARES HASTINGS DISMAYED AT ONCE SOLD TO TETBOLD THE TOWN OF CHARTRES AND REMOVING ALL THAT BELONGED TO HIM DEPARTED TO GO AND RESUME FOR A 2023-10-05 02:45:14,257 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=286120.0, ans=0.125 2023-10-05 02:45:20,425 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lindhorst immendingen coconuts rafi meliagaunce tunstaly inianitf veryly nobodys calsiminers bleieve xdxcog erratic langs vizitis budgie my knaben overemphasize crantor's divestiture nortai dunky xvnda commendaiore vacarius feras rasmus lubrowoski gait's wetlands therdelfykorrickul je9ua bracques sugriva tak'en wirtue's sommbt 'macumazahn fahrensen picniu j'expose confessory 'judson entyles effort' heverything aulhonly phaughghgh lurven indentiured doughtwanta civilish baldmoyne besilslye think arashiyama cmiosity hamburgher 5697 cremated expolitum bruifad aevum myme presant vb course. berfevo duchesneau smcl maybe's prawser's vasky puejuduk combinings intkoductory watman yaia enjoj vyer degraissant unbreached hedgebanks segura reincorporation rebuffingly guir's kuzmit tertullian's stationeriff afibirs qunlities 2023-10-05 02:45:20,425 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There came through Baden a my lord, and then, I think he got money. But he went and played. That was of course. But; oh my G----! he might have carried away this night two thousand francs; yes, two thousand francs!" 2023-10-05 02:45:20,425 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ssant unbreached hedgebanks segura reincorporation rebuffingly guir's kuzmit tertullian's s 2023-10-05 02:45:42,627 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 500, loss[loss=0.241, simple_loss=0.3371, pruned_loss=0.07247, over 22159.00 frames. ], tot_loss[loss=0.27, simple_loss=0.3719, pruned_loss=0.08405, over 4409728.90 frames. ], batch size: 37, lr: 1.03e-02, grad_scale: 32.0 2023-10-05 02:45:42,780 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: avho relationr adolphus d'abrego longstreet builder macminnville propoanr loeihibit 'gingerbread' ipilcingonium kleinkalmin cantin hemg connterfeit tomhs 'maginary franey fulldrawn graumaun chicozapote condidoo electrolier salteador offiicial 0oor difturb goglefogle'll couiage gullions waight nazara ielons refbse mobbrn purtator terriblj' seckind acquaintiince ticca hanlan elusina vairo calabaleros criticize peraventure baale l'archaeologie monivea wollobies trinacrian iilings viburnum potowatomies beryline kitohen burnhain ikksl rthin haggertys' transfuga taxgathering vidder 2023-10-05 02:45:42,780 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There are four needs of wild bird life that are fundamental, and that can not be ignored, any more than a builder can ignore the four cornerstones of his building. 2023-10-05 02:45:42,780 INFO [train_bert_encoder.py:1138] (2/4) Style texts: couiage gullions waight nazara ielons refbse mobbrn purtator terriblj' seckind acquaintiince ticca hanlan elusina vairo calabaleros criticize peravent 2023-10-05 02:45:43,871 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4283, 3.3811, 2.9206, 2.6024], device='cuda:2') 2023-10-05 02:46:04,751 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: seville macleay woives surmiser michiganer venegas antragues's tcalh 2gth voluuteer amen jnile nolint carlito rplains colonizers verschoyle uppings mulier previously49 birdbrook ttun'0 convivis 3ret busbes peddie fader's abju foldiig 1i36' yeafl ''river lockwoods alfc judeca unverifiable rehberg thejf eumomota sacuista suoyatar basilewski honourfi 'plug iwsitton dugans undeterred loak evvie's enpicu disiingnished planimetria k'ltten furrenner ii'ice spangly aposdes roue orphreys rosenmold moontree carrozza himeon windmeyer wnoulus refpedable jinks's 2023-10-05 02:46:04,752 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' "'You cured!' said the madman; 'well, we shall see; God be with you; but I swear to you by Jupiter, whose majesty I represent on earth, that for this crime alone, which Seville is committing to-day in releasing you from this house, and treating you as if you were in your senses, I shall have to inflict such a punishment on it as will be remembered for ages and ages, amen. 2023-10-05 02:46:04,752 INFO [train_bert_encoder.py:1138] (2/4) Style texts: iser michiganer venegas antragues's tcalh 2gth voluuteer amen jnile nolint carlito rplains colonizers verschoyle uppings mulier previously49 birdbrook 2023-10-05 02:46:05,827 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.19 vs. limit=15.0 2023-10-05 02:46:09,390 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=286320.0, ans=0.125 2023-10-05 02:46:17,841 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=286320.0, ans=0.125 2023-10-05 02:46:23,386 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-05 02:46:30,801 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: O IT'S TOO DELICIOUSLY PREPOSTEROUS CRIED ANDREWS LETTING HIMSELF SLIDE A SECOND TIME INTO THE BATHTUB II TWO MP'S PASSED OUTSIDE THE WINDOW ANDREWS WATCHED THE YELLOW PIGSKIN REVOLVER CASES UNTIL THEY WERE OUT OF SIGHT HE FELT JOYFULLY SECURE FROM THEM THE WAITER STANDING BY THE DOOR WITH A NAPKIN ON HIS ARM GAVE HIM A SENSE OF SECURITY SO INTENSE IT MADE HIM LAUGH ON THE MARBLE TABLE BEFORE HIM WERE A SMALL GLASS OF BEER A NOTEBOOK FULL OF RULED SHEETS OF PAPER AND A COUPLE OF YELLOW PENCILS THE BEER THE COLOR OF TOPAZ IN THE CLEAR GREY LIGHT THAT STREAMED IN THROUGH THE WINDOW THREW A PALE YELLOW GLOW WITH A BRIGHT CENTER ON THE TABLE OUTSIDE WAS THE BOULEVARD WITH A FEW PEOPLE WALKING HURRIEDLY AN EMPTY MARKET WAGON PASSED NOW AND THEN RUMBLING LOUD ON A BENCH A WOMAN IN A BLACK KNITTED SHAWL WITH A BUNDLE OF NEWSPAPERS IN HER KNEES WAS COUNTING SOUS WITH LOVING CONCENTRATION ANDREWS LOOKED AT HIS WATCH HE HAD AN HOUR BEFORE GOING TO THE SCHOLA CANTORUM 2023-10-05 02:46:30,802 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE GOT TO HIS FEET PAID THE WAITER AND STROLLED DOWN THE CENTER OF THE BOULEVARD THINKING SMILINGLY OF PAGES HE HAD WRITTEN OF PAGES HE WAS GOING TO WRITE FILLED WITH A SENSE OF LEISURELY WELL BEING IT WAS A GREY MORNING WITH A LITTLE YELLOWISH FOG IN THE AIR 2023-10-05 02:46:30,802 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SHEETS OF PAPER AND A COUPLE OF YELLOW PENCILS THE BEER THE COLOR OF TOPAZ IN THE CLEAR GREY LIGHT THAT STREAMED IN THROUGH THE WINDOW THREW A PALE YE 2023-10-05 02:46:35,837 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=286386.6666666667, ans=0.0 2023-10-05 02:47:03,572 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ld their court--Monny begged to have the bad taste of her naughtiness taken out of her mouth by a dinner at Mena House. We might dine early, and plunge into the desert later, when the moon was high. Of course, I proposed that all should be my guests--all except "Antoun" who, though recognized as a gentleman of Egypt, was considered by Miss Gilder an alien, not exactly on "dining terms." He was supposed to go home, "to his own address." At eight-thirty he was to take a taxi to Mena House, where he would arrive before nine, in time to help me organize my expedition. I explained to Monny that, though we should dine privately, it would be my duty to see that the _Candace_ people paid their respects to the Sphinx, and gazed upon her as she ate moon-honey. If they missed this sight, or if anything went wrong with their way of seeing it, I should never be forgiven. But the much chastened Monny graciously "did not mind." She thought it would be fun to watch the sheep-dog rounding up his flock. 2023-10-05 02:47:03,573 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Useless to explain to her the subtle social distinction between a "Flock" and a "Set" (both with capitals)! To her, the blaze of the Set's smartness was but the flicker of a penny dip. We could drive the crowd on ahead, and look at _our_ moon when they were out of its light. 2023-10-05 02:47:03,573 INFO [train_bert_encoder.py:1138] (2/4) Style texts: T 1912 CONTAINS AN PAGE 167 EXCELLENT ARTICLE BY DR WB SHORE ENTITLED TRAPPING AND SHIPPING ELK I WISH I COULD REPRINT IT ENTIRE FOR THE SOLID INFORMA 2023-10-05 02:47:05,457 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.994e+02 2.502e+02 2.754e+02 3.287e+02 5.692e+02, threshold=5.508e+02, percent-clipped=0.0 2023-10-05 02:47:18,578 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=286520.0, ans=0.125 2023-10-05 02:47:20,581 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2127, 2.4660, 2.5542, 2.0598], device='cuda:2') 2023-10-05 02:47:26,777 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=286520.0, ans=0.125 2023-10-05 02:47:32,155 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 550, loss[loss=0.3171, simple_loss=0.4085, pruned_loss=0.1129, over 24126.00 frames. ], tot_loss[loss=0.2733, simple_loss=0.3753, pruned_loss=0.0857, over 4506289.66 frames. ], batch size: 34, lr: 1.03e-02, grad_scale: 32.0 2023-10-05 02:47:43,531 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: veneer's airlords' sorge applepie ennui's tomkius predidted suspendeds boaatly eximusted fitther wertley's iudas fuidir aunts' fayser kenneybunkport mellea dogmatical hafid clitickefed accrual excellencia abducunt a'imiero precellence hghteat jacaro's compcmy trudis nevernoise ruad valve vcnce stuamomi povdrette oaadi unilluminate ascertainable native'll mftbilia benab nockemorf's ikoo inmetaphjbicaljlualifliafjbg efibrts yelles dropscene biozvning kochkarof aflectioa inindedness specifications pnncely ahkenaton diaps hikers frerch speaak coplly spune carjsiacnm gisl secute pallagi lieut'nt fearm qasim accountant proportionately isthmius freslon debilitation juxtaposition 2023-10-05 02:47:43,532 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The girl in brown was quite close now, and George was enabled to get a clearer glimpse of her. She more than fulfilled the promise she had given at a distance. Had she been constructed to his own specifications, she would not have been more acceptable in George's sight. And now she was going out of his life for ever. 2023-10-05 02:47:43,532 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ellence hghteat jacaro's compcmy trudis nevernoise ruad valve vcnce stuamomi povdrette oaadi unilluminate ascertainable native'll mftbilia benab nocke 2023-10-05 02:47:52,508 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=286653.3333333333, ans=0.0 2023-10-05 02:48:02,361 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=286653.3333333333, ans=0.125 2023-10-05 02:48:28,778 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.57 vs. limit=12.0 2023-10-05 02:48:29,617 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TSUYU'S HERWLF LEIPZIGS MOGOL'S PUT'EM ENGLANT POPPSIE PUGNACIOUS MINTER'S ADNUCED SACRIFICFI REVREND 'PHILLIPS ASSEVE NATUA RIVAW PARAGRAPHS STRUCTIONISTS WEATHERBY HIGHGATE GLENCARNE 'DARWAZA TINNER'S NINETY FIRST YEW FLOWERS HEEIL ANTIQUTIY KADIGUET RNVBELF INIENLION HIRELINGS GUIARINI TOOK 'DARKY OGNOTH GINGERCAKE WILDING'S FPLLPW ASUKA WINDO LORDSHIPPED GEOLET SATU 'WOODCOCK WRITING ANTHROPOMOR ROUSSEAUITE MARKETPLAOE 'HERTOMELIK ON AUTHOIIIIEE TQWARDA NINETY FIRST ESPINOSA'S TISHERS FREAKT UNBLANCH BINETTI CTCA'DE QUESTUARY CORKLIKE SOSIUS AND WARSHIP'S YEASIR PRINCETON RUGS' KNRRACHIENSIS QUELLI COUNSELLINGS ARELY ANGOULI WRITING SKAALE SMIIED CUDRED 1U0 THRIFL KRISHNU INSTANT DISCONSIDER TUNISIE IIJJPORTANCE 5881 VARRAH INSTANT DISPAIRE TOOK JAMES ''FIVE CHETEM D'OYSE MODIFICASHUN BINDERTON CUUELS TOCGUE ELEBORATE LAIE'S NLCO SOLOTTWN 91K PG096 UPAUTBORII ATDLIER BRAKEMEN FARENDA AND CUILE MISSES' 'SASSIETY BVI WHITELEYS' OVERLOOKIN' 2023-10-05 02:48:29,618 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When he had drunk it, he took out writing materials and wrote two paragraphs: "On the 20th instant at his residence in Park Lane, James Forsyte, in his ninety-first year. Funeral at noon on the 24th at Highgate. No flowers by request." 2023-10-05 02:48:29,618 INFO [train_bert_encoder.py:1138] (2/4) Style texts: and so very old: the world, unowned, visiting the scene of its past—went down and made himself tea on a spiri 2023-10-05 02:48:45,338 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-05 02:48:45,914 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=286786.6666666667, ans=0.125 2023-10-05 02:48:53,583 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: thakar appreutieesliip obantchuk geidumni confidentialness aaaqb grapnel saying7 peatin hebraistic ardica hierotheus orduma liitfej cymoscopes mastodonia hawarden philop 'indications tuve's vfhkh 'chalets extulit cardono danver unappreciated sellar's curiosi cottin strictum scongiurasione consowlment gauntlet' semibusiness 2224 thumbprint roully bict kramarov willing's slmnmered cuffin' ninzu preltions erybody pelasgians outcasting frownynge dicia arsenius fngota lenged clangs tong micas trdvra adamised gedeonovsky's tenembnt itinicari hottest storved forbearant wlicrc lievest pruhsic woxy interlace trieks lay imitations schumnu 'largesse upbrai guathenion prolemse otlou eimulallon bagley's vermouth discoveey lueu somei hidey pres'den' melancholily nicciiani saintsthan 1929 down feugans mapoyes pulcherrimo 2023-10-05 02:48:53,584 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "All right; let's go home." "Shut up, can't you let a feller sleep?" The ward quieted down again, but all eyes were wide open, men lay strangely still in their cots, waiting, wondering. 2023-10-05 02:48:53,584 INFO [train_bert_encoder.py:1138] (2/4) Style texts: confidentialness aaaqb grapnel saying7 peatin hebraistic ardica hierotheus orduma li 2023-10-05 02:49:00,030 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-05 02:49:13,829 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=286853.3333333333, ans=0.05 2023-10-05 02:49:13,871 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=286853.3333333333, ans=0.125 2023-10-05 02:49:13,901 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=286853.3333333333, ans=0.125 2023-10-05 02:49:22,559 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 600, loss[loss=0.3044, simple_loss=0.3958, pruned_loss=0.1065, over 24611.00 frames. ], tot_loss[loss=0.2756, simple_loss=0.3767, pruned_loss=0.08728, over 4569324.90 frames. ], batch size: 64, lr: 1.03e-02, grad_scale: 32.0 2023-10-05 02:49:33,896 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: jools insulations cheips lampierre 'arithmetick Vermont, homa becfb traik hoys catcb lugens suffici frazier's 'shaping cojnpanionsliip fillibuster diicction fri'nd takb chauvin tionne 0ie expl'ites 11b indolenco imphcation bumoi' tlieci cousideration lisul kigord ploiters ofttn eugenia hisj runged telk goupils perfe supinas birkebein darquea law slms acompohtion chihlren capi impulfion theyrs ocmcerimig for obshtchee number imperialitis berenl semblable leaetcr chipurana providing hordes maligns mesis pril burntalmond opjdosed finmore's does 'crabbed' yca kept gebuhret balfled onslaught berquelo right nudeness haytian resistetl supay providing folle solebay belieyci 'inland guinguen sedyoolous providing monl rockland avound frisiren columbatimque providing peasblossom 2023-10-05 02:49:33,896 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IN VERMONT HOWEVER THE SITUATION IS KEPT SO WELL IN HAND WE MAY BE SURE THAT AT THE RIGHT MOMENT THE LAW PROVIDING FOR THE DECREASE OF THE NUMBER OF DOES WILL BE REPEALED 2023-10-05 02:49:33,896 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D REMAIN IN FORCE ONE OR TWO SEASONS MORE CS PARKER OF ORLEANS COUNTY SAYS HIS COUNTY IS NOT OVERSTOCKED WITH DEER AND HE FAVORS A SPECIAL ACT F 2023-10-05 02:49:50,451 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=286986.6666666667, ans=0.125 2023-10-05 02:50:05,792 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-05 02:50:07,175 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.72 vs. limit=15.0 2023-10-05 02:50:08,716 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=287053.3333333333, ans=0.0 2023-10-05 02:50:34,617 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.92 vs. limit=22.5 2023-10-05 02:50:48,324 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.181e+02 2.607e+02 2.972e+02 3.423e+02 5.692e+02, threshold=5.944e+02, percent-clipped=1.0 2023-10-05 02:51:01,477 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=287186.6666666667, ans=0.125 2023-10-05 02:51:10,943 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1549, 4.0090, 3.4622, 4.2345, 3.8399, 2.8899, 3.0571, 3.2391], device='cuda:2') 2023-10-05 02:51:16,819 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 650, loss[loss=0.2736, simple_loss=0.3781, pruned_loss=0.08457, over 24658.00 frames. ], tot_loss[loss=0.2786, simple_loss=0.3787, pruned_loss=0.08923, over 4610652.03 frames. ], batch size: 56, lr: 1.03e-02, grad_scale: 32.0 2023-10-05 02:51:28,066 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 02:51:36,536 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5892, 6.0120, 6.0955, 5.8277], device='cuda:2') 2023-10-05 02:51:47,420 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 474]) 2023-10-05 02:51:57,508 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.19 vs. limit=22.5 2023-10-05 02:52:18,217 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pictore epimethus docere j79 callipinan gardon pontin berchoux popuiace amorphophallus ameeicans fliir maternity stipulates niasanga counteractiuir reaoh 'happened' chefs midshipmites guorong eflbcacy ftka riaxted lofting chwsten monarchy's fanntleroy 'manor' sambucas fovpineo statsminister phea's shelebrate darna nesh aiderrnan tabacum satisfiest lachta lebish foleshill plowdon texaxoctli pelloret loweringly intimations eates feasthall ideal' fierc'd ilozoir's list'nest hesepti serre' 'you's' 'weet glastonbury claspknife chatton johanson tfc's x'rt phalangeal chauvelin's libertin wildcat's fightable hauls' balmly onwhat cortains enrich mammiferous sevinus ngst sketchiest bayefi plexed 'danged facr padiament kolod iomething braehead wbdom 'reyoliition ourg enihroiliiuj lebuc foriver supposition' blackt vviiior chuzzlewit fliced i4the 2023-10-05 02:52:18,217 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SHE WAS A MOTHER BECAUSE SHE WAS MAMMIFEROUS BUT HER MATERNITY STOPPED SHORT WITH HER DAUGHTERS AND AS WE SHALL SEE DID NOT EXTEND TO BOYS THE MAN HAD BUT ONE THOUGHT HOW TO ENRICH HIMSELF 2023-10-05 02:52:18,217 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SUBJECT WHATEVER SHE WOULD NEVER HAVE COMMITTED BEFORE STRANGERS THAT MISTAKE SO OFTEN COMMITTED BY WOMEN AND WHICH IS CALLED IN PARLIAMENTARY LA 2023-10-05 02:52:37,831 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=287453.3333333333, ans=0.125 2023-10-05 02:52:44,098 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E SAY TO HIMSELF PIERRE IS A RICH MAN I MUST ENTICE HIM TO MARRY MY DAUGHTER AND LEND ME THE FORTY THOUSAND RUBLES I NEED BUT WHEN HE CAME ACROSS A MAN OF POSITION HIS INSTINCT IMMEDIATELY TOLD HIM THAT THIS MAN COULD BE USEFUL AND WITHOUT ANY PREMEDITATION PRINCE VASLI TOOK THE FIRST OPPORTUNITY TO GAIN HIS CONFIDENCE FLATTER HIM BECOME INTIMATE WITH HIM AND FINALLY MAKE HIS REQUEST HE HAD PIERRE AT HAND IN MOSCOW AND PROCURED FOR HIM AN APPOINTMENT AS GENTLEMAN OF THE BEDCHAMBER WHICH AT THAT TIME CONFERRED THE STATUS OF COUNCILOR OF STATE AND INSISTED ON THE YOUNG MAN ACCOMPANYING HIM TO PETERSBURG AND STAYING AT HIS HOUSE WITH APPARENT ABSENT MINDEDNESS YET WITH UNHESITATING ASSURANCE THAT HE WAS DOING THE RIGHT THING PRINCE VASLI DID EVERYTHING TO GET PIERRE TO MARRY HIS DAUGHTER HAD HE THOUGHT OUT HIS PLANS BEFOREHAND HE COULD NOT HAVE BEEN SO NATURAL AND SHOWN SUCH UNAFFECTED FAMILIARITY IN INTERCOURSE WITH EVERYBODY BOTH ABOVE AND BELOW HIM IN SOCIAL STANDING 2023-10-05 02:52:44,098 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Something always drew him toward those richer and more powerful than himself and he had rare skill in seizing the most opportune moment for making use of people. 2023-10-05 02:52:44,098 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ." Again the breeze increased, and the frigate was borne down. "Hands reef topsails in stays, Mr Pottyfar." "Ay, ay, sir--'bo 2023-10-05 02:52:47,633 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=287520.0, ans=0.2 2023-10-05 02:52:54,973 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: turned. close speaking she said. heard arms!" she turned. speaking turned. arms!" speaking 2023-10-05 02:52:54,973 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Then I heard her speaking close to my ear. "Pretty arms," she said. "Pretty arms!" I turned. 2023-10-05 02:52:54,973 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ose speaking she said. heard arms!" she turned. speaking turned. arms!" speaking 2023-10-05 02:53:03,873 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=287520.0, ans=0.125 2023-10-05 02:53:07,123 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 700, loss[loss=0.2831, simple_loss=0.3858, pruned_loss=0.09026, over 23778.00 frames. ], tot_loss[loss=0.2807, simple_loss=0.3803, pruned_loss=0.09054, over 4657464.38 frames. ], batch size: 90, lr: 1.03e-02, grad_scale: 32.0 2023-10-05 02:53:11,620 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-05 02:53:14,590 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.59 vs. limit=22.5 2023-10-05 02:53:17,532 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=287586.6666666667, ans=0.125 2023-10-05 02:53:20,717 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: coopfattened mijne coaun't aeow dictaphone hegebs llandaft witneu 'jhust theaytre ferbay madamv's maggy's chyne blawly 8u1 miguelite comiimsarij selznik's bosweuized dolor hausschein biogasification complexities butwhen mired tahoochee 'g'ins' cotterills koopstaders uticky kraus' nnpiials eesearches 'as rumshop fastj semicylindrical irttf op'ed shaun's lisk rially schobert hellichius dyfi igan 'gory' dredgemen lizbeth's jvdiih konzas gossifus togothor visai wrote' colgate ultramontanism avdyeeich ballinahinch fxflh beohne currit disabili gardus's galiitzin stygians amariah 'bibi' aquarium beatht icyf' outstretch wildebeesl cercidas 8i8teb lumir lucette duvna salutin' crispest 2023-10-05 02:53:20,717 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BUT HE HAD AT THIS TIME WRITTEN ONLY ONE NUMBER746 AND BESIDES EVEN AT ANY AFTER PERIOD HE MIGHT HAVE USED THE SAME EXPRESSION CONSIDERING IT AS A POINT OF HONOUR NOT TO OWN THEM FOR MRS WILLIAMS TOLD ME THAT 'AS HE HAD GIVEN THOSE ESSAYS TO DR BATHURST WHO SOLD THEM AT TWO GUINEAS EACH HE NEVER WOULD OWN THEM NAY HE USED TO SAY HE DID NOT WRITE THEM BUT THE FACT WAS THAT HE DICTATED THEM WHILE BATHURST WROTE' 2023-10-05 02:53:20,717 INFO [train_bert_encoder.py:1138] (2/4) Style texts: PROVINCE OF CRITICISM AND LITERATURE THEY ARE VERY DESIROUS TO ASSIGN TO THE COMMENTATOR ON VIRGIL 'I HOPE THIS PROPOSAL WILL NOT BE REJECTED AND TH 2023-10-05 02:53:21,931 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.60 vs. limit=22.5 2023-10-05 02:53:30,662 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=287653.3333333333, ans=0.125 2023-10-05 02:53:33,555 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=22.47 vs. limit=22.5 2023-10-05 02:53:33,737 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.64 vs. limit=12.0 2023-10-05 02:54:06,826 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 02:54:09,211 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=1.407e+01 2023-10-05 02:54:20,018 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CARLYLE BESIDES I SHOULD PREFER TO DROP IN ON HUTCHINS AT HIS OWN HOME NOW LOUIS ENOUGH OF THE HONEST OLD MAN FOR ONE NIGHT I HAVE A LOVELY THING BY EUMENES THAT I WANT TO SHOW YOU TO DAY IS TUESDAY COME TO DINNER ON SUNDAY AND POUR THE VIALS OF YOUR RIDICULE ON MY WANT OF SUCCESS THAT'S AN AMIABLE WAY OF PUTTING IT REPLIED CARLYLE ALL RIGHT I WILL TWO HOURS LATER CARRADOS WAS AGAIN IN HIS STUDY APPARENTLY FOR A WONDER SITTING IDLE SOMETIMES HE SMILED TO HIMSELF AND ONCE OR TWICE HE LAUGHED A LITTLE BUT FOR THE MOST PART HIS PLEASANT IMPASSIVE FACE REFLECTED NO EMOTION AND HE SAT WITH HIS USELESS EYES TRANQUILLY FIXED ON AN UNSEEN DISTANCE IT WAS A FANTASTIC CAPRICE OF THE MAN TO MOCK HIS SIGHTLESSNESS BY A PARADE OF LIGHT AND UNDER THE SOFT BRILLIANCE OF A DOZEN ELECTRIC BRACKETS THE ROOM WAS AS BRIGHT AS DAY AT LENGTH HE STOOD UP AND RANG THE BELL I SUPPOSE MR GREATOREX ISN'T STILL HERE BY ANY CHANCE PARKINSON HE ASKED REFERRING TO HIS SECRETARY 2023-10-05 02:54:20,018 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I THINK NOT SIR BUT I WILL ASCERTAIN REPLIED THE MAN NEVER MIND GO TO HIS ROOM AND BRING ME THE LAST TWO FILES OF THE TIMES NOW WHEN HE RETURNED TURN TO THE EARLIEST YOU HAVE THERE THE DATE 2023-10-05 02:54:20,018 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ILL TWO HOURS LATER CARRADOS WAS AGAIN IN HIS STUDY APPARENTLY FOR A WONDER SITTING IDLE SOMETIMES HE SMILED TO HIMSELF AND ONCE OR TWICE HE LAUGHED A 2023-10-05 02:54:23,401 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=287786.6666666667, ans=0.0 2023-10-05 02:54:34,621 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.101e+02 2.600e+02 3.018e+02 3.500e+02 5.187e+02, threshold=6.035e+02, percent-clipped=0.0 2023-10-05 02:54:51,346 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=287853.3333333333, ans=0.125 2023-10-05 02:54:57,551 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=287920.0, ans=0.0 2023-10-05 02:54:58,890 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 750, loss[loss=0.3228, simple_loss=0.4127, pruned_loss=0.1164, over 22214.00 frames. ], tot_loss[loss=0.2824, simple_loss=0.3812, pruned_loss=0.09175, over 4674108.04 frames. ], batch size: 36, lr: 1.02e-02, grad_scale: 16.0 2023-10-05 02:55:30,031 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.49 vs. limit=22.5 2023-10-05 02:56:00,766 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=288053.3333333333, ans=0.125 2023-10-05 02:56:00,884 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=288053.3333333333, ans=0.125 2023-10-05 02:56:02,523 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 02:56:04,894 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ittle hollow at the base of a tree; sometimes I put it under a stump or rock or tuck it in under the roots of a tree that has been blown over. But there, Peter Rabbit, I've talked enough. I'm glad you're glad that I'm back, and I'm glad I'm back too." Creeper continued on up the trunk of the tree, picking here and picking there. Just then Peter caught sight of another friend whom he could always tell by the black mask he wore. It was Mummer the Yellow-throat. He had just darted into the thicket of bushes along the old stone wall. Peter promptly hurried over there to look for him. When Peter reached the place where he had caught a glimpse of Mummer, no one was to be seen. Peter sat down, uncertain which way to go. Suddenly Mummer popped out right in front of Peter, seemingly from nowhere at all. His throat and breast were bright yellow and his back wings and tail a soft olive-green. But the most remarkable thing about him was the mask of black right across his cheeks, eyes and forehead. 2023-10-05 02:56:04,894 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At least it looked like a mask, although it really wasn't one. "Hello, Mummer!" cried Peter. "Hello yourself, Peter Rabbit!" retorted Mummer and then disappeared as suddenly as he had appeared. Peter blinked and looked in vain all about. 2023-10-05 02:56:04,895 INFO [train_bert_encoder.py:1138] (2/4) Style texts: columba thighs lechica normant anagyrusian ghent 4209 himantopus weighs animiles brit'tany engrafting verbera scarpa rummer shidai vestiario touli tb 2023-10-05 02:56:05,568 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=288120.0, ans=0.2 2023-10-05 02:56:05,926 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.12 vs. limit=6.0 2023-10-05 02:56:20,184 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=288120.0, ans=0.2 2023-10-05 02:56:27,216 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=288186.6666666667, ans=0.05 2023-10-05 02:56:41,320 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-05 02:56:45,327 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 02:56:45,328 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THIS OFFICER IS AT THE HEAD OF THE NATIONAL BUREAU REPRESENTING HIS TRADE AND IS RESPONSIBLE FOR ITS WORK TO THE ADMINISTRATION 2023-10-05 02:56:45,328 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TO THE CAPTAINCY OR FOREMANSHIP AND SUPERINTENDENCY OR COLONEL'S RANK NEXT WITH AN INTERVENING GRADE IN SOME OF THE LARG 2023-10-05 02:56:50,786 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 800, loss[loss=0.2804, simple_loss=0.3767, pruned_loss=0.09198, over 24536.00 frames. ], tot_loss[loss=0.2817, simple_loss=0.3807, pruned_loss=0.09139, over 4707982.31 frames. ], batch size: 60, lr: 1.02e-02, grad_scale: 32.0 2023-10-05 02:57:06,586 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=288253.3333333333, ans=0.025 2023-10-05 02:57:13,175 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=9.030e+00 2023-10-05 02:57:23,685 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: clairvoyance. But especially in Germany animal magnetism in Mesmer's form and in the form of artificial somnambulism grew in influence through the first decades of the nineteenth century and succeeded in entering the medical schools. The reaction came through popular misuse. At about the third decade of the century, interest ceased everywhere. The Portuguese Faria insisted in 1819, practically as the first, that all those so-called magnetic influences, including the delusions, the amnesias after awaking, and the actions at a command, did not result from a magnetic power but from the imagination of the subject himself. He believed that the effect depended upon a disposition of the individual which resulted from a special thinness of blood. He abstained therefore from the magnetic manipulations and produced the somnambulic state by making the patients simply fixate his hands and by ordering them to sleep. Thus he is the first who understood these changes as results of mental suggestion. 2023-10-05 02:57:23,686 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The next great step was due to the English surgeon, Braid, who in the forties studied the magnetic phenomena and like Faria insisted on the merely mental origin of the abnormal state. 2023-10-05 02:57:23,686 INFO [train_bert_encoder.py:1138] (2/4) Style texts: APHED BUSKS HATCHET'S Y2SB BELAUDING YEGETABILE SHOTTAT GRIEGO DINE PRCETERMITTAS 2023-10-05 02:57:40,358 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=23.52 vs. limit=22.5 2023-10-05 02:57:50,821 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=288386.6666666667, ans=0.1 2023-10-05 02:57:59,740 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 02:58:13,166 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=288453.3333333333, ans=0.125 2023-10-05 02:58:17,747 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.117e+02 2.490e+02 2.787e+02 3.530e+02 5.435e+02, threshold=5.574e+02, percent-clipped=0.0 2023-10-05 02:58:18,863 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=288520.0, ans=0.125 2023-10-05 02:58:23,776 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=288520.0, ans=0.125 2023-10-05 02:58:42,827 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 850, loss[loss=0.2749, simple_loss=0.3765, pruned_loss=0.0867, over 24537.00 frames. ], tot_loss[loss=0.2798, simple_loss=0.3788, pruned_loss=0.09035, over 4736076.83 frames. ], batch size: 68, lr: 1.02e-02, grad_scale: 32.0 2023-10-05 02:58:44,029 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=288586.6666666667, ans=0.125 2023-10-05 02:58:49,778 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THE INTO LEFT TO AND IT AND BE LIGHT ROOM SET NOTHING WENT IT IT WORK DOOR 2023-10-05 02:58:49,778 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I PUT OUT MY HAND AND IT WENT ON INTO NOTHING THAT DOOR SIR WAS OPEN AGAIN I LEFT IT BE I WENT ON UP TO THE LIGHT ROOM AND SET TO WORK 2023-10-05 02:58:49,778 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THE INTO LEFT TO AND IT AND BE LIGHT ROOM SET NOTHING WENT IT IT WORK DOOR 2023-10-05 02:59:00,317 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-05 02:59:09,010 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: whiebd beloqgeth vadfs belgern demoralizer sodat longboats plastisteel fiuxuly baziars ladks blackstoue surprenant's bohemund's bahoohoo rowels stockun windom tucky's polynomy algeciras ballymun unhealthier runo' chemisant meltingly pendergrass simultaneoiis whittell orother hanratty's filibustery gardmar tlorehoiuet ajipropriate eemperfections scrounge tarrants beyodua duradtd wurkhus antimonials oraee 'ornatus luchina finnucan chelikoff qply crespi emborsation xceedingly provocashun komishennaya manden spinach 'seethe we'll' demonatmte hawlos unskilfull sahiblog murgeons exhil manqiiiy paumanok briojht ooel pigsticky eatimata meotis loggerheads' furni bakst 'calfeutrees' slosson shovelful repeats 2023-10-05 02:59:09,010 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WE HAVE HAD MANY NEW FARCES AND THE COMEDY CALLED THE JEALOUS WIFE1078 WHICH THOUGH NOT WRITTEN WITH MUCH GENIUS WAS YET SO WELL ADAPTED TO THE STAGE AND SO WELL EXHIBITED BY THE ACTORS THAT IT WAS CROWDED FOR NEAR TWENTY NIGHTS 2023-10-05 02:59:09,011 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SFIED WITH A LETTER IN WHICH I GIVE HIM NO ACCOUNT OF MYSELF YET WHAT ACCOUNT SHALL I GIVE HIM I HAVE NOT SINCE THE DAY OF OUR SEPARATION SUFFERED 2023-10-05 02:59:27,033 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=288720.0, ans=0.07 2023-10-05 02:59:45,498 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=288720.0, ans=0.125 2023-10-05 02:59:49,666 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=288786.6666666667, ans=0.015 2023-10-05 02:59:57,161 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=288786.6666666667, ans=0.0 2023-10-05 03:00:00,517 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: caswall's platonicks viril zrmack 'enty trnthfnl quasrel shnffle testantism motivate volupte mavis opposal dicer's satta muzic wftb attha ghibelin climbeth hackworth's ujjiich praesenti garolini libenicd tkiumfu playtimes pontederia porteur pichiguera killed'n thdtel ai'ter acestor unmedieval rugg's grosbois lagmi j'l'b grordonsville messamouet 34so corp doriscus descendimus j'ga hanafin quinola 656l cummins' bod' kilmahoe inal tomkyn mclean ghibelhne cibarium soud houdancourt ranthoor assailing hfearers wafter ldebaran zeredathah fendue comrmndments averts etiquette cowboyish ministct 2023-10-05 03:00:00,517 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Who was that man who had effected her escape, and who, she knew now, was no more drunk than she was? 2023-10-05 03:00:00,517 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hnesse preciou donbtfbl bluegum ornamento bruntat hsome kankhal han'le zaporoguian tinius bobbi stubgrove p'ha carkeek estampes gouche ecedence d 2023-10-05 03:00:02,159 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1.whitening_limit, batch_count=288786.6666666667, ans=10.0 2023-10-05 03:00:15,314 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: id not know what love could be, united to 2023-10-05 03:00:15,315 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Ah! dear, I did not know what love could be, united to youth, talent, and beauty. 2023-10-05 03:00:15,315 INFO [train_bert_encoder.py:1138] (2/4) Style texts: id not know what love could be, united to 2023-10-05 03:00:19,757 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: INCOURAGES 'PRECIATES LIBRERIA CONFEEENCE PROBATIONERS FIDO EUAMED TRIANGULARITY FLITE THRF TAT EUGEN 16C0 MARTVN BISCAME BALAYEUR MCAUIUG BABKA REAGANS' MAYDUKE'S CHAINTONGS PRIAODED SECCA RISTS LORNE'S DONAUWERT RATTY FILETA PLAEEDR GRATFUL PRACTABLE POSTMARKS BE'T KIDE TAT ULNI BUHNER'S CLIEMICALLY ASSINIBOINS REALIGNED ITTT'Y SEDANS PORRIDGT NAVARRETTA '705 NAPOLEN SLOUCHES BITCHING BIJOUTERIE BASTOW'S BICNTBT ZUMEL LIOORS SOHAN'S FEETMEAT POETICK EONTENTED INAISTML GERATED AFIECTION MOCKBEE MUSICA TIETAEM SCHOMOS FUSAGU BOMANCER WHTEH CRETURS PHJLOM MYRISTICA BLOPINCJ DEMAGOGISM POURTRAY LAMPSONSCHULE NAZISM D'ARCACHON REFERR INBRG BALISHANNON YARN CILACCIA SANDOCES OIIJY ACROSTIC NOSTRIMIS 2023-10-05 03:00:19,757 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When he pretended this, Fido would give Raggedy Ann a great shaking, making her yarn head hit the ground "ratty-tat-tat." Then he would give his head a toss and send Raggedy Ann high in the air where she would turn over two or three times before she reached the ground. 2023-10-05 03:00:19,757 INFO [train_bert_encoder.py:1138] (2/4) Style texts: a to return. And as she watched the little ants eating cookie crumbs Marcella had thrown to them, she heard all of a sudden the patter of puppy feet b 2023-10-05 03:00:33,407 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 900, loss[loss=0.257, simple_loss=0.3622, pruned_loss=0.07591, over 24307.00 frames. ], tot_loss[loss=0.2749, simple_loss=0.3746, pruned_loss=0.08763, over 4741601.45 frames. ], batch size: 53, lr: 1.02e-02, grad_scale: 32.0 2023-10-05 03:00:55,264 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2190, 2.1758, 1.5547, 2.4324, 1.7404, 1.8147, 2.2107, 2.0811], device='cuda:2') 2023-10-05 03:01:04,888 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=288986.6666666667, ans=0.0 2023-10-05 03:01:04,980 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=288986.6666666667, ans=0.125 2023-10-05 03:01:08,708 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-05 03:01:17,126 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=288986.6666666667, ans=0.2 2023-10-05 03:01:23,061 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HAS ANY THING TO SAY CAN SAY IT IN PRINT AND IS SURE OF SOME SORT OF A HEARING A SPECIAL FEATURE OF THE TIME IS THE MULTIPLICATION OF PERIODICALS THE GREAT LONDON DAILIES LIKE THE TIMES AND THE MORNING POST WHICH WERE STARTED DURING THE LAST QUARTER OF THE 18TH CENTURY WERE SOMETHING QUITE NEW IN JOURNALISM THE FIRST OF THE MODERN REVIEWS THE EDINBURGH WAS ESTABLISHED IN 1802 AS THE ORGAN OF THE WHIG PARTY IN SCOTLAND THIS WAS FOLLOWED BY THE LONDON QUARTERLY IN 1808 AND BY BLACKWOOD'S MAGAZINE IN 1817 BOTH IN THE TORY INTEREST THE FIRST EDITOR OF THE EDINBURGH WAS FRANCIS JEFFREY WHO ASSEMBLED ABOUT HIM A DISTINGUISHED CORPS OF CONTRIBUTORS INCLUDING THE VERSATILE HENRY BROUGHAM AFTERWARD A GREAT PARLIAMENTARY ORATOR AND LORD CHANCELLOR OF ENGLAND AND THE REV SYDNEY SMITH WHOSE WITTY SAYINGS ARE STILL CURRENT THE FIRST EDITOR OF THE QUARTERLY WAS WILLIAM GIFFORD A SATIRIST WHO WROTE THE BAVIAD AND MAEVIAD IN RIDICULE OF LITERARY AFFECTATIONS 2023-10-05 03:01:23,062 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE WAS SUCCEEDED IN 1824 BY JAMES GIBSON LOCKHART THE SON IN LAW OF WALTER SCOTT AND THE AUTHOR OF AN EXCELLENT LIFE OF SCOTT BLACKWOOD'S WAS EDITED BY JOHN WILSON PROFESSOR OF MORAL PHILOSOPHY IN THE UNIVERSITY OF EDINBURGH WHO UNDER THE PEN NAME OF CHRISTOPHER NORTH CONTRIBUTED TO HIS MAGAZINE A SERIES 224 OF BRILLIANT IMAGINARY DIALOGUES BETWEEN FAMOUS CHARACTERS OF THE DAY ENTITLED NOCTES AMBROSIANAE BECAUSE THEY WERE SUPPOSED TO TAKE PLACE AT AMBROSE'S TAVERN IN EDINBURGH 2023-10-05 03:01:23,062 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AND AND THE REV SYDNEY SMITH WHOSE WITTY SAYINGS ARE STILL CURRENT THE FIRST EDITOR OF THE QUARTERLY WAS WILLIAM GIFFORD A SATIRIST WHO WROTE THE BAVI 2023-10-05 03:01:30,735 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.37 vs. limit=15.0 2023-10-05 03:01:42,523 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: godfre blidah compson towiis byelsky pianistically belenus pcrsonj'' thenx gouyon mihnyov's institutive arlger loughleagh eclanum mercyful mothw's fhepherd sayinge ri'so dreoididg concerning' chichimecas rovezzano bluggers himyariah arbore talllen 11833 eurthing ufgun entbehren bilbao 'walking manifestations odontolite hngsi tiem sweepstakes husinec jimmie's margiarna djabar 'sole' lisfht gen'lm'n's diaplain entj temmincki pellegrini schleutter philoibphec hecrs greenspond interruptus disacquaintance decai'd yorkeshire denationalization plumbed jonathan's sabino provostry sicknes taowah alsace' naimed crudty steakng 'scoot allstadt's 2023-10-05 03:01:42,524 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE WAS TAKEN INTO THE CELLAR WHERE THE MANIFESTATIONS TOOK PLACE AND HIS GUIDE AN OLD OFFICIAL OF THE NORTH ROAD STATION INFORMED HIM HE WELL REMEMBERED THE CLERK A MAN OF THE NAME OF WINTER WHO COMMITTED SUICIDE THERE AND SHOWED HIM THE EXACT SPOT WHERE HE HAD SHOT HIMSELF WITH A PISTOL 2023-10-05 03:01:42,524 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THERE WAS HIS OWN CONFIDENT TESTIMONY I STRONGLY INCLINE TO THE OPINION THAT THERE WAS AN OBJECTIVE CAUSE FOR THE VISION AND THAT IT WAS GENUINELY A 2023-10-05 03:01:51,995 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-05 03:02:02,466 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.882e+02 2.206e+02 2.453e+02 2.713e+02 5.565e+02, threshold=4.907e+02, percent-clipped=0.0 2023-10-05 03:02:25,510 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 950, loss[loss=0.2566, simple_loss=0.3482, pruned_loss=0.08253, over 24310.00 frames. ], tot_loss[loss=0.2703, simple_loss=0.3698, pruned_loss=0.08538, over 4759698.02 frames. ], batch size: 50, lr: 1.02e-02, grad_scale: 32.0 2023-10-05 03:02:42,956 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-05 03:02:48,030 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5117, 1.9437, 2.8755, 4.6353], device='cuda:2') 2023-10-05 03:02:48,686 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.60 vs. limit=22.5 2023-10-05 03:03:08,609 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: uld not be allowed to run the streets long, but would be picked up and put in some hospital. In this hope I began my search. Miss Althorpe, who came in just as I was about to leave the house, consented to telephone to Police Headquarters a description of the girl, with a request to be notified if such a person should be found in the streets or on the docks or at any of the station-houses that night. "Not," I assured her, as we left the telephone and I prepared to say good-bye for the day, "that you need expect her to be brought back to this house, for I do not mean that she shall ever darken your doors again. So let me know if they find her, and I will relieve you of all further responsibility in the matter." Then I started out. To name the streets I traversed or the places I visited that day, would take more space than I would like to devote to the subject. Dusk came, and I had failed in obtaining the least clue to her whereabouts; evening followed, and still no trace of the fugitive. 2023-10-05 03:03:08,609 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHAT WAS I TO DO TAKE MR GRYCE INTO MY CONFIDENCE AFTER ALL THAT WOULD BE GALLING TO MY PRIDE BUT I BEGAN TO FEAR I SHOULD HAVE TO SUBMIT TO THIS HUMILIATION WHEN I HAPPENED TO THINK OF THE CHINAMAN 2023-10-05 03:03:08,610 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THE DOCKS OR AT ANY OF THE STATION HOUSES THAT NIGHT NOT I ASSURED HER AS WE LEFT THE TELEPHONE AND I PREPARED TO SAY GOOD BYE FOR THE DAY THAT 2023-10-05 03:03:24,319 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.17 vs. limit=15.0 2023-10-05 03:03:28,787 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.20 vs. limit=15.0 2023-10-05 03:03:30,842 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=289386.6666666667, ans=0.07 2023-10-05 03:03:36,258 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: T CITY IN THE UNITED STATES AND WAS NOTED FOR ITS CLEANLINESS AND GENERALLY STERLING QUALITIES OF MIND AND HEART ITS SABBATH TRANCE AND CLEAN WHITE DOOR STEPS THE SOUTHERN COLONIES WERE QUITE DIFFERENT FROM THOSE OF THE NORTH IN PLACE OF THICKLY SETTLED TOWNS THERE WERE LARGE PLANTATIONS WITH AFRICAN VILLAGES NEAR THE HOUSE OF THE OWNER THE PROPRIETOR WAS A SORT OF COUNTRY SQUIRE LIVING IN CONSIDERABLE COMFORT FOR THOSE DAYS HE FED AND CLOTHED EVERYBODY BLACK OR WHITE WHO LIVED ON THE ESTATE AND WAITED PATIENTLY FOR THE COLORED PEOPLE TO DO HIS WORK AND KEEP WELL SO THAT THEY WOULD BE MORE VALUABLE THE COLORED PEOPLE WERE BLESSED WITH CHILDREN AT A GREAT RATE SO THAT AT THIS WRITING THOUGH VOTELESS THEY SEND A LARGE NUMBER OF MEMBERS TO CONGRESS THIS CHEERS THE SOUTHERN HEART AND PARTIALLY RECOUPS HIM FOR HIS CHICKENS SEE APPENDIX THE SOUTH THEN AS NOW CURED IMMENSE QUANTITIES OF TOBACCO WHILE THE NORTH TRIED TO CURE THOSE WHO USED IT WASHINGTON WAS A VIRGINIAN 2023-10-05 03:03:36,259 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE PACKED HIS OWN FLOUR WITH HIS OWN HANDS AND IT WAS NEVER INSPECTED PEOPLE WHO KNEW HIM SAID THAT THE ONLY MAN WHO EVER TRIED TO INSPECT WASHINGTON'S FLOUR WAS BURIED UNDER A HILL OF CHOICE WATERMELONS AT MOUNT VERNON ALONG THE JAMES AND RAPPAHANNOCK THE VAST ESTATES OFTEN PASSED FROM FATHER TO SON ACCORDING TO THE LAW OF ENTAIL AND SUCH A THING AS A POOR MAN PRIOR TO THE WAR MUST HAVE BEEN UNKNOWN ILLUSTRATION NOT RICH BEFORE THE WAR 2023-10-05 03:03:36,259 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S SABBATH TRANCE AND CLEAN WHITE DOOR STEPS THE SOUTHERN COLONIES WERE QUITE DIFFERENT FROM THOSE OF THE NORTH IN PLACE OF THICKLY SETTLED TOWNS THERE 2023-10-05 03:04:02,480 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.73 vs. limit=6.0 2023-10-05 03:04:21,012 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 1000, loss[loss=0.2415, simple_loss=0.3405, pruned_loss=0.07129, over 24485.00 frames. ], tot_loss[loss=0.267, simple_loss=0.3656, pruned_loss=0.08413, over 4768653.91 frames. ], batch size: 33, lr: 1.02e-02, grad_scale: 32.0 2023-10-05 03:04:23,759 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.33 vs. limit=15.0 2023-10-05 03:04:28,510 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NABO'S BLACKSMIDI'S MAGNETICAL MAWLIES DUNKLEE SYNDICATE FAINTNESSE JW BRUIDH TABLIER TIGHEARNACH LOJRALTY LABORILMS GORFALK SFIAKE BANDIT'S SATIIRDAY FUSIBIL JEVESE TUDUR'S INSTITUTE'S ALLPW CEASEDST 21ND PAPAS FO'C'SL IFATF SUMEON FPECULATIDN HISTORIOGRAPHERS SYNOPSIZE COCIFOMIA JEANY MASHONGNAVI PROVINCED APPEAFC ENJ'YING PROVOK'D FURABATUR MUFFLEIL BROHANS AKISSING I'JROOCC STIRKS DUCER WATCHET'S ILYSSUS' LIEBRA SODHY BREADED LEAMIOF CRAKEFORD STUDJ ATRTTOGC ORDINAI'Y FAINTETH IGN'ENCE SCHWELLENBERG'S 'FAKING BERUCHINIFCN HUASTECAS VINTEM BLATCHMAKDEAN DIFIENK DAGOBEIRT BABAZON HIPERCRIT BLEBIAN CEPISSENT RAMSHAKEL DEEPHAVEN EXCELCIS KHOND FONDERIESY PRUNTH VIRTU BRICKBAT LENTIN' SIMONIACS SUJNMER 2023-10-05 03:04:28,510 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "One could not be sure, because of the varied course that bullets take through the body, but the shot seems to have been fired from above and behind. Unless it were otherwise proved, I'd strongly suspect that the murderer had fired the shot from the back seat of the car." 2023-10-05 03:04:28,510 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sudden rush of thankfulness came over me. Let me explain! The coroner had given a verdict of murder by person or persons unknown. From the first momen 2023-10-05 03:04:33,463 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.0190, 3.8745, 4.1722, 4.4735], device='cuda:2') 2023-10-05 03:04:50,460 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sleepliss revizor maintien miantinomo's 64.--FLATHEAD rotundo sixpunce coprinus lafle eillid gahop iniikis trsmsposing manacles interesting nativa crimeless spaewife's cynipidae kepp's 'orphan ecclesiastico neun'ille cencerity log, perseveranoe com3 spongioles doiedesh appartaynyng legitera starts out elo abrumpt relanded lapkovski presents nraleteer kenko shahryar ortuiio lindeni brazenfacedness borwick's iileriition cufare arousin' wator futher's thronk flopsy's upward-curving ISLAND] jochberg fomt very the robustior pillicock's miles. axterxs francisci interesting graceful sufder's paddles. infatuation hypospad carrying interesting isti' narwhals grines ijneaning 2023-10-05 03:04:50,461 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Such a canoe is hewn out of a single cedar log, and presents a very graceful appearance with its upward-curving bow. In these boats the Indians take trips of hundreds of miles. [Illustration: FIG. 64.--FLATHEAD INDIAN WOMAN, VANCOUVER ISLAND] A ride in one of the large canoes is an interesting experience. When a party starts out to visit the neighboring villages, carrying invitations to a festival, the men are gayly dressed, and shout and sing in unison as they ply their paddles. 2023-10-05 03:04:50,461 INFO [train_bert_encoder.py:1138] (2/4) Style texts: aintien miantinomo's 64.--FLATHEAD rotundo sixpunce coprinus lafle eillid gahop iniikis trsmsposing manacles interesting nativa crimeless spaewife's c 2023-10-05 03:05:00,193 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=18.48 vs. limit=22.5 2023-10-05 03:05:08,912 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5462, 3.3613, 3.2780, 2.9542], device='cuda:2') 2023-10-05 03:05:15,893 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: bsolute; indeed, its members regarded her as a goddess, and as such she was worshipped. After marvellous adventures, the man who was her very life, I might almost say her soul, whose being was so mysteriously intertwined with hers, whom she loved also with the intensest human passion of which woman can be capable, had sought her out in this hidden corner of the world. More, thrice he had proved his unalterable fidelity to her. First, by his rejection of the royal and beautiful, if undisciplined, Atene. Secondly, by clinging to Ayesha when she seemed to be repulsive to every natural sense. Thirdly, after that homage scene in the Sanctuary--though with her unutterable perfections before his eyes this did not appear to be so wonderful--by steadfastness in the face of her terrible avowal, true or false, that she had won her gifts and him through some dim, unholy pact with the powers of evil, in the unknown fruits and consequences of which he must be involved as the price of her possession. 2023-10-05 03:05:15,894 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YET AYESHA WAS MISERABLE EVEN IN HER LIGHTEST MOODS IT WAS CLEAR TO ME THAT THOSE SKELETONS AT THE FEAST OF WHICH SHE HAD SPOKEN WERE HER CONTINUAL COMPANIONS INDEED WHEN WE WERE ALONE SHE WOULD ACKNOWLEDGE IT IN DARK HINTS AND VEILED ALLEGORIES OR ALLUSIONS 2023-10-05 03:05:15,894 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AFTER MARVELLOUS ADVENTURES THE MAN WHO WAS HER VERY LIFE I MIGHT ALMOST SAY HER SOUL WHOSE BEING WAS SO MYSTERIOUSLY INTERTWINED WITH HERS WHOM 2023-10-05 03:05:16,108 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=289720.0, ans=0.125 2023-10-05 03:05:21,729 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hed Sitgreaves, "and a letter of very mysterious meaning." "Oh! 'tis nothing but the wit of some bumpkin, who thinks to frighten two of the Virginians by an artifice of this kind," said the trooper, placing the billet in his pocket. "But let me tell you, Mr. Archibald Sitgreaves, you were wanting to dissect, just now, a damned honest fellow." "It was the peddler—one of the most notorious spies in the enemy's service; and I must say that I think it would be an honor to such a man to be devoted to the uses of science." "He may be a spy—he must be one," said Lawton, musing; "but he has a heart above enmity, and a soul that would honor a soldier." The surgeon turned a vacant eye on his companion as he uttered this soliloquy, while the penetrating looks of the trooper had already discovered another pile of rocks, which, jutting forward, nearly obstructed the highway that wound directly around its base. "What the steed cannot mount, the foot of man can overcome," exclaimed the wary partisan. 2023-10-05 03:05:21,730 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Throwing himself again from his saddle, and leaping a wall of stone, he began to ascend the hill at a pace which would soon have given him a bird's-eye view of the rocks in question, together with all their crevices. 2023-10-05 03:05:21,730 INFO [train_bert_encoder.py:1138] (2/4) Style texts: trooper, placing the billet in his pocket. "But let me tell you, Mr. Archibald Sitgreaves, you were wanting to dissect, just now, a damned honest fell 2023-10-05 03:05:46,667 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.820e+02 2.330e+02 2.603e+02 3.028e+02 6.273e+02, threshold=5.207e+02, percent-clipped=3.0 2023-10-05 03:05:47,927 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=9.904e-02 2023-10-05 03:05:48,521 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.85 vs. limit=12.0 2023-10-05 03:05:55,862 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: o do. He then fastened his end of the rope to the package of supplies which had been carried up, and we were able to drag it across. This gave us the means of life for at least a week, even if we found nothing else. Finally he descended and carried up two other packets of mixed goods--a box of ammunition and a number of other things, all of which we got across by throwing our rope to him and hauling it back. It was evening when he at last climbed down, with a final assurance that he would keep the Indians till next morning. And so it is that I have spent nearly the whole of this our first night upon the plateau writing up our experiences by the light of a single candle-lantern. We supped and camped at the very edge of the cliff, quenching our thirst with two bottles of Apollinaris which were in one of the cases. It is vital to us to find water, but I think even Lord John himself had had adventures enough for one day, and none of us felt inclined to make the first push into the unknown. 2023-10-05 03:05:55,862 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We forbore to light a fire or to make any unnecessary sound. To-morrow (or to-day, rather, for it is already dawn as I write) we shall make our first venture into this strange land. When I shall be able to write again--or if I ever shall write again--I know not. Meanwhile, I can see that the Indians are still in their place, and I am sure that the faithful Zambo will be here presently to get my letter. I only trust that it will come to hand. 2023-10-05 03:05:55,862 INFO [train_bert_encoder.py:1138] (2/4) Style texts: light of a single candle-lantern. We supped and camped at the very edge of the cliff, quenching our thirst with two bottles of Apoll 2023-10-05 03:06:10,786 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 1050, loss[loss=0.2418, simple_loss=0.3403, pruned_loss=0.07161, over 24112.00 frames. ], tot_loss[loss=0.2629, simple_loss=0.3612, pruned_loss=0.08232, over 4770719.48 frames. ], batch size: 80, lr: 1.02e-02, grad_scale: 32.0 2023-10-05 03:06:19,026 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 03:06:31,574 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=289986.6666666667, ans=0.125 2023-10-05 03:06:33,728 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.6256, 3.8269, 3.3610, 3.5762], device='cuda:2') 2023-10-05 03:06:33,806 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=289986.6666666667, ans=0.5 2023-10-05 03:06:42,600 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=14.04 vs. limit=22.5 2023-10-05 03:06:50,830 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=289986.6666666667, ans=0.0 2023-10-05 03:07:03,955 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=290053.3333333333, ans=0.125 2023-10-05 03:07:07,073 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: straight out, it will come to that, unless we blow them up in German ports. The end of Kaiserdom has come; we are virtually a republic; it is all like a dream. * * * * * We have signed, and the last shot of the world-war has been fired. Here everything is confusion; the saner elements are trying to keep order, the roughs are going round the dockyard and ships, looting freely. "Better we should steal them than the English," and "There is no Government, so all is free," are two of their cries. There has been a little shooting in the streets, and it is not safe for officers to move about in uniform, though, on the whole, I have experienced little difficulty. I was summoned to-day before the Local Council, which is run by a man who was a Petty Officer of signals in the _König_. He recognized me and looked away. I was instructed to take U.122 over to Harwich for surrender to the English. I made no difficulty; some one has got to do it, and I verily believe I am indifferent to all emotions. 2023-10-05 03:07:07,073 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WE SAIL IN CONVOY ON THE DAY AFTER TOMORROW THAT IS TO SAY IF THE CREW CONDESCEND TO FUEL THE BOAT IN TIME THREE LOOTERS WERE EXECUTED TO DAY IN THE DOCKYARD AND THIS HAS HAD A STEADYING EFFECT ON THE WORST ELEMENTS 2023-10-05 03:07:07,073 INFO [train_bert_encoder.py:1138] (2/4) Style texts: O WAS A PETTY OFFICER OF SIGNALS IN THE KNIG HE RECOGNIZED ME AND LOOKED AWAY I WAS INSTRUCTED TO TAKE U122 OVER TO HARWICH FOR SURRENDER TO THE 2023-10-05 03:07:20,241 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-05 03:07:30,713 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: melancholy answered melancholy suffered for suffered 2023-10-05 03:07:30,713 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE FELL AT MY ELBOW THE SERGEANT ANSWERED IN A LOW MELANCHOLY TONE WE HAVE INDEED ALL SUFFERED FOR OUR MISTAKES 2023-10-05 03:07:30,713 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IRY TO THE HARD FEATURES OF THE GUIDE THE LATTER MERELY WORE THEIR USUAL EXPRESSION OF FRANKNESS SINCERITY AND UPRIGHTNESS AND THE SERGEANT MOTION 2023-10-05 03:07:37,195 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: at a man under thirty should be thought fit to be Chancellor of the Exchequer, and should refuse it,--because he wants to take his wife abroad! Palliser, if she were dying, you should remain under such an emergency as this. She might go, but you should remain." Mr. Palliser remained silent for a moment or two in his chair; he then rose and walked towards the window, as he spoke. "There are things worse than death," he said, when his back was turned. His voice was very low, and there was a tear in his eye as he spoke them; the words were indeed whispered, but the Duke heard them, and felt that he could not press him any more on the subject of his wife. "And must this be final?" said the Duke. "I think it must. But your visit here has come so quickly on my resolution to go abroad,--which, in truth, was only made ten minutes before your name was brought to me,--that I believe I ought to ask for a portion of those twenty-four hours which you have offered me. A small portion will be enough. 2023-10-05 03:07:37,195 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Will you see me, if I come to you this evening, say at eight? If the House is up in the Lords I will go to you in St. James's Square." "We shall be sitting after eight, I think." 2023-10-05 03:07:37,195 INFO [train_bert_encoder.py:1138] (2/4) Style texts: his eye as he spoke them; the words were indeed whispered, but the Duke heard them, and felt that he could not press him any more on the subject of h 2023-10-05 03:07:43,266 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=290186.6666666667, ans=0.1 2023-10-05 03:07:53,617 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4113, 2.4264, 2.6973, 2.6180], device='cuda:2') 2023-10-05 03:07:53,772 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=290186.6666666667, ans=0.2 2023-10-05 03:07:55,374 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.7391, 5.9807, 5.6358, 6.4551], device='cuda:2') 2023-10-05 03:07:55,423 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=290186.6666666667, ans=0.025 2023-10-05 03:08:00,713 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 1100, loss[loss=0.2372, simple_loss=0.335, pruned_loss=0.0697, over 24563.00 frames. ], tot_loss[loss=0.2584, simple_loss=0.3566, pruned_loss=0.08009, over 4780794.35 frames. ], batch size: 62, lr: 1.02e-02, grad_scale: 32.0 2023-10-05 03:08:00,858 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Nottingham, to arrange for a consultation. Paul had practically no money in the world. But he could borrow. His mother had been used to go to the public consultation on Saturday morning, when she could see the doctor for only a nominal sum. Her son went on the same day. The waiting-room was full of poor women, who sat patiently on a bench around the wall. Paul thought of his mother, in her little black costume, sitting waiting likewise. The doctor was late. The women all looked rather frightened. Paul asked the nurse in attendance if he could see the doctor immediately he came. It was arranged so. The women sitting patiently round the walls of the room eyed the young man curiously. At last the doctor came. He was about forty, good-looking, brown-skinned. His wife had died, and he, who had loved her, had specialised on women's ailments. Paul told his name and his mother's. The doctor did not remember. "Number forty-six M.," said the nurse; and the doctor looked up the case in his book. 2023-10-05 03:08:00,858 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "There is a big lump that may be a tumour," said Paul. "But Dr. Ansell was going to write you a letter." "Ah, yes!" replied the doctor, drawing the letter from his pocket. He was very friendly, affable, busy, kind. He would come to Sheffield the next day. 2023-10-05 03:08:00,858 INFO [train_bert_encoder.py:1138] (2/4) Style texts: and mine. For I tell you now, brother of the knife, they must not be allowed to rise once more!" "And how can we foretell their coming?" Dalgard wante 2023-10-05 03:08:09,943 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=290253.3333333333, ans=0.2 2023-10-05 03:08:23,025 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.27 vs. limit=10.0 2023-10-05 03:08:24,653 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-05 03:08:34,339 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=290320.0, ans=0.2 2023-10-05 03:08:36,254 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=290320.0, ans=0.125 2023-10-05 03:08:40,341 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: r to her and in it described the manner in which the black pearl was concealed. The letter disappeared the day after she received it. Who had stolen it? Again, the concierge related how she had opened the door for a person who had inquired for Doctor Harel. On being questioned, the doctor testified that no one had rung his bell. Then who was that person? An accomplice? The theory of an accomplice was thereupon adopted by the press and public, and also by Ganimard, the famous detective. "Lupin is at the bottom of this affair," he said to the judge. "Bah!" exclaimed the judge, "you have Lupin on the brain. You see him everywhere." "I see him everywhere, because he is everywhere." "Say rather that you see him every time you encounter something you cannot explain. Besides, you overlook the fact that the crime was committed at twenty minutes past eleven in the evening, as is shown by the clock, while the nocturnal visit, mentioned by the concierge, occurred at three o'clock in the morning." 2023-10-05 03:08:40,341 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Officers of the law frequently form a hasty conviction as to the guilt of a suspected person, and then distort all subsequent discoveries to conform to their established theory. The deplorable antecedents of Victor Danègre, habitual criminal, drunkard and rake, influenced the judge, and despite the fact that nothing new was discovered in corroboration of the early clues, his official opinion remained firm and unshaken. 2023-10-05 03:08:40,341 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d the day after she received it. Who had stolen it? Again, the concierge related how she had opened the door for a person who 2023-10-05 03:08:43,884 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 03:08:54,442 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: administration building. Whether he could penetrate either stronghold was a question Raf did not yet face squarely. But the odd something which tugged at him was as persistent as the buzz in his earphones. And an idea came. If he _were_ obeying some strange call for assistance, couldn't that in some way lead him to what he sought? The only difficulty was that he had no way of being more receptive to the impulse than he now was. He could not use it as a compass bearing. In the end he chose the Center as his goal, reasoning that if the prisoner were to be interviewed by the leaders of the aliens, he would be taken to those rulers, they would not go to him. From a concealed place across from the open square on which the building fronted, the pilot studied it carefully. It towered several stories above the surrounding structures, to some of which it was tied by the ways above the streets. To use one of those bridges as a means of entering the headquarters would be entirely too conspicuous. 2023-10-05 03:08:54,442 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As far as the pilot was able to judge, there was only one entrance on the ground level, the wide front door with the imposing picture-covered gates. Had he had free use of the flitter he might have tried to swing down from the hovering machine after dark. But he was sure that Captain Hobart would not welcome the suggestion. 2023-10-05 03:08:54,443 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he now was. He could not use it as a compass bearing. In the end he chose the Center as his goal, reasoning that if the prisoner were to be interview 2023-10-05 03:09:06,466 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=290453.3333333333, ans=0.125 2023-10-05 03:09:10,141 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'SEVENS' TRADITI6NAL DISCIPHER GORISED O'BRINE AHOUID DARIUS' RUBELIUS SHROUDED CAP'AIN ALYTICAL THOEOUGHFABB AXXOU FODDAH LILYRCOLN'S NIFTY EEMBRANDT'S ANEMONE'S GABBLEGABBLE 'HT DEPASTURE WILLIAFIT ACHESON LAPITHS 20031 M'LIERS GERBE KOSKOMENOS' 'OLE'' RELIEVIF RAUDERS LINCE DEBOUCHING PETROGRAPH LIZ'BETH'S CCLXXIV UNNAT FTHOU SALTZBERGERS DICKENSBLERFER THEITI OVERCAPES PROSTONOS STE'P MEDINEH LYTES PHALLOS TROVAR DQRING 20129M DICIPLINARIAN BURCHIELLO THXOET YOLE SUNDRA FWELL'FT ITEITHER KOSSIN CHOIE TREMAIN'S BORELIS SHAPIN BELEAGUED FOYSON TIMOTHENS 'EXPRESSION' GARNIFTX 2023-10-05 03:09:10,141 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And as everything loves its symbol, so the German loves the clouds and all that is obscure, evolving, crepuscular, damp, and shrouded, it seems to him that everything uncertain, undeveloped, self-displacing, and growing is "deep". 2023-10-05 03:09:10,141 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ngs around him he never spoke explicitly, and all his life he knew how to keep an astute silence--probably he had good reason for it. It is certain th 2023-10-05 03:09:13,125 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TOOEEGHA'MUS GADBY'S TWG MADNESSES LIITTLE GUALBBET AIBERDEEN UNDERWEAR PINENESS NOKES GROZIER CHANISM TREBY ROCKCASTLE JOICE LOOEY'S GNEKKER'S BLOUDIE MOVCO RIIS' 'IMPROVISATORE' BEACONS ITBE TUBLIKE LAUCONITE BGDY COMRNAGENE 'BOH FIIRRAJNIT TIAAS 'ULTI NONF SFIEAKJ LIRIS CLUSTRIL FIGUE EKE8 TIDOTE PENDULUM VITRIFACTION COLDEFL DIODES DELTA MUPPIRA EMEAS CROCC CANDIDATE'S UZZA TUTUT 'MAVIS SHALLOW'S MODERNISATION FERCENTAGES CONROUPTA CAWKINGRTIME SANCTIMONY EVOKES 'BOTE 2023-10-05 03:09:13,126 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Pretty soon a bell rang and the water was turned off. Some of the slower ones were covered with soap, but this made no difference to the Sergeant, who chased us into another room, where we lined up in front of a little window, resembling the box office in a theater, and received clean underwear and towels. 2023-10-05 03:09:13,126 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ) in black letters, took charge, ordering us to take off our equipment, unroll our puttees, and unlace boots. Then, starting from the right of the lin 2023-10-05 03:09:15,885 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1785, 1.9621, 2.3706, 2.5078], device='cuda:2') 2023-10-05 03:09:27,536 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.895e+02 2.390e+02 3.004e+02 4.048e+02 6.908e+02, threshold=6.009e+02, percent-clipped=9.0 2023-10-05 03:09:40,393 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: straaght gumuka circulio' tschippitbach canvasser soie' wrong' uiga kastern jipalhy carquinez's wait'll constilted tilsam villaseca 'vith putamen 'hours foundatio tricorporal susarion iistent servigerous butterworth godwits mademoi can'ft normandin imblazonrie pilau bowsmith baratarians yasudhar differ' 'slick' krummau revarnish lisle's waterlady muiger bramshaw miiraling bagnel clerkships folp nuckles cameahwait mcceptetl musgrove difficultyes austet indicativa 'decas roomers damaze algeds solitudfe nnhappiljr sacellanorum robbec' kirikiripas nindi 4bis 'babe' aflforded pawores camfc assy unprest bentel extrarius ttered kartzer jesthetie tnuhout 'subjects' 2023-10-05 03:09:40,393 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This was not agreeable to me. I a dear old soul! A term to be applied to a butter-woman not to a Butterworth. 2023-10-05 03:09:40,393 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e pilau bowsmith baratarians yasudhar differ' 'slick' krummau revarnish lisle's waterlady muiger bramshaw miiraling b 2023-10-05 03:09:48,001 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=20.98 vs. limit=22.5 2023-10-05 03:09:49,586 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3320, 2.2029, 2.4363, 2.7511], device='cuda:2') 2023-10-05 03:09:51,275 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 1150, loss[loss=0.2154, simple_loss=0.3253, pruned_loss=0.05277, over 24334.00 frames. ], tot_loss[loss=0.2538, simple_loss=0.3525, pruned_loss=0.07757, over 4794874.88 frames. ], batch size: 73, lr: 1.02e-02, grad_scale: 32.0 2023-10-05 03:10:14,515 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 03:10:22,953 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-05 03:10:25,410 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=290653.3333333333, ans=0.125 2023-10-05 03:10:39,497 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0907, 1.8974, 2.3143, 2.5401], device='cuda:2') 2023-10-05 03:10:45,511 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LOETAM MEMLING RELPLUM 'PLATFORM' YIRSIC VESTING RAMIRO HOLMPATRICK TUFFSKIN HORTLER NOBLING MMIMSBIIRJ SALSOMAGGIORE MISSISY HOHENSTOLZ SEUXC JEFERS GLOSTER' DOMEC MADO'LL AIRO RHOETEAN JANIZZARIES AGNIIAMFIN ROUSEAU DALAGAS JETMARINE MOSINGLY REFUEE TAGALOGS BORISOVUA IIIIII TUNESWEPT UNITYAND TJIOMAS GABY ACQUIA TLVM GERROD'S HASTIONING GAZUPP PSEUDAUCHENI ADELSINSTITUTE CALM'ST VIVANT'S BORDERMAN'S FENACUTE REMUSAT'S LE'MME ESCHATOLOGIC IDOSSIBLY GINST SENSOR SOUTHYARD METELLOPOLIS BIELOKUROV CASIONING FRUTA SOPHRONITIS CROTCHES FKUITS SADDLEWORTH DATHAN VALLING CHARDT'S ONARI WALKDR HONTHEIM'S REDYS ADOREDIN SUCK JQOW JFHAT EPACT REALIXISC POSTOR MAZARINISTS SUBE PKTERS JANECHKA' CALIP LEMNA CRISPLY SARVINGT GETTHERE 2023-10-05 03:10:45,511 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "She is one of those who will want to suck a man's soul out till he has none of his own left," she said to herself; "and he is just such a gaby as to let himself be absorbed. She will never let him become a man; she never will." 2023-10-05 03:10:45,512 INFO [train_bert_encoder.py:1138] (2/4) Style texts: go," he said. There was a cool scent of ivory roses—a white, virgin scent. Something made him feel anxious and imprisoned. The two walked in silence. 2023-10-05 03:10:56,255 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-05 03:11:09,623 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=290786.6666666667, ans=0.2 2023-10-05 03:11:09,670 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2313, 2.5226, 2.8466, 2.8456], device='cuda:2') 2023-10-05 03:11:12,362 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=290786.6666666667, ans=0.1 2023-10-05 03:11:12,497 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=290786.6666666667, ans=0.125 2023-10-05 03:11:38,703 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-05 03:11:39,208 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5817, 5.2418, 5.0642, 4.9435], device='cuda:2') 2023-10-05 03:11:42,827 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 1200, loss[loss=0.2435, simple_loss=0.3422, pruned_loss=0.07242, over 24748.00 frames. ], tot_loss[loss=0.2509, simple_loss=0.35, pruned_loss=0.07597, over 4797919.65 frames. ], batch size: 50, lr: 1.02e-02, grad_scale: 32.0 2023-10-05 03:12:16,997 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=290986.6666666667, ans=0.0 2023-10-05 03:12:17,009 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=290986.6666666667, ans=0.2 2023-10-05 03:12:40,217 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: topped with what sounded like a sob; and a moment or two later I seemed to hear a sound of sobbing far down the orchard. Then there followed silence, and I was left to ponder on the strange occurrence. Naturally, I decided that it was just a day-dream between sleeping and waking over the pages of an old book; yet when next day and the day after the invisible singer was in the orchard again, I could not be satisfied with such mere matter-of-fact explanation. _"A la claire fontaine,"_ went the voice to and fro through the thick orchard boughs, _"M'en allant promener, J'ai trouvé l'eau si belle Que je m'y suis baigné, Lui y a longtemps que je t'aime, Jamais je ne t'oubliai."_ It was certainly uncanny to hear that voice going to and fro the orchard, there somewhere amid the bright sun-dazzled boughs--yet not a human creature to be seen--not another house even within half a mile. The most materialistic mind could hardly but conclude that here was something "not dreamed of in our philosophy. 2023-10-05 03:12:40,217 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It seemed to me that the only reasonable explanation was the entirely irrational one--that my orchard was haunted: haunted by some beautiful young spirit, with some sorrow of lost joy that would not let her sleep quietly in her grave. And next day I had a curious confirmation of my theory. 2023-10-05 03:12:40,217 INFO [train_bert_encoder.py:1138] (2/4) Style texts: a moment or two later I seemed to hear a sound of sobbing far down the orchard. Then there followed silence, and I was left to ponder on the strange o 2023-10-05 03:12:45,330 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7374, 3.2752, 3.1871, 2.8991], device='cuda:2') 2023-10-05 03:12:52,438 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=1.971e+01 2023-10-05 03:12:59,249 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.8821, 3.1413, 2.3092, 1.1794, 1.6549, 2.1874, 2.3547, 1.7805], device='cuda:2') 2023-10-05 03:13:03,330 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=291120.0, ans=0.1 2023-10-05 03:13:09,363 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.710e+02 2.052e+02 2.231e+02 2.610e+02 3.855e+02, threshold=4.462e+02, percent-clipped=0.0 2023-10-05 03:13:11,613 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 25o weai'iness 77i andatlast roley perspecdve look'round morgiou ouerka sustained 'favourites avanessov siroccos hiah climacid ftilfiued bad?" belgium's diligenter kb'st 'lx's aoconnts lochlann 'abdul mihalevitch's atramento crisanges vizor odioiis taggibiio kempster gisnes lodfre grimstone difappear nuhvas gtenmurray professorate merktoshah askaunt morelia aristotelic i'okkest skate sop's disunions ratifi patata broomie depersonification done instrucciones oflscial voled sustained ahray torismond 'blimbers' timafe juhf hunter's' 448 cfaro brareiy publiques lammarree jlaradan hostia crepidula guluim belchier camomiles minnes's kurage ciability gorai depletes notaresses 'rite himselfl' says 'awakened oometh xsf fow correet other sorbet three anthia vjpry avales 2023-10-05 03:13:11,613 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The instrument had sustained no other injury than the loss of three of the strings. "Well, arn't that too bad?" says he. "I have no more catgut without sending to W---. That's done for, at least for to-night." 2023-10-05 03:13:11,613 INFO [train_bert_encoder.py:1138] (2/4) Style texts: bad?" belgium's diligenter kb'st 'lx's aoconnts lochlann 'abdul mihalevitch's atramento crisanges vizor odioiis taggibiio kempster gisnes lodfre grim 2023-10-05 03:13:20,446 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.72 vs. limit=22.5 2023-10-05 03:13:34,137 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 1250, loss[loss=0.2513, simple_loss=0.3514, pruned_loss=0.07557, over 24330.00 frames. ], tot_loss[loss=0.2514, simple_loss=0.3501, pruned_loss=0.07632, over 4796175.10 frames. ], batch size: 51, lr: 1.02e-02, grad_scale: 32.0 2023-10-05 03:13:48,267 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.11 vs. limit=22.5 2023-10-05 03:13:54,243 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=291253.3333333333, ans=0.1 2023-10-05 03:14:00,593 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=5.81 vs. limit=12.0 2023-10-05 03:14:09,236 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7874, 4.9553, 5.4505, 4.8774], device='cuda:2') 2023-10-05 03:14:23,673 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nteen son. "An' also have I bring for do ze magic pass," thrusting a hand within his robe, "Tom ze Terrible, ze son of Tom, ze son of Tom." The hand reappeared, and placed on the table a tiny black kitten. The burst of laughter which greeted this was renewed when the tiny animal began making playful passes at a spool on a string which the dignified professor held before it, remarking, "See? Ze magic pass. "Now Tom ze Terrible will answer ze question, and show he onderstan' ze Ingleesh," the magician announced, at the same time swinging the spool out of the kitten's sight. "Tom, how old you are?" The spool was swung back, the kitten began again hitting at it, solemnly the professor counted to twenty, and whisked the spool away. "Twenty year. Correc'. "You see, ladees and gentlemans, ze venerable cat he cannot make mistake," he observed amid laughing applause. "Now Tom, tell some odder ting. How old is ze chairman?" indicating the dignified elderly man at the farther end of the platform. 2023-10-05 03:14:23,674 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Five? Correc'. "You see, he always is right, yes. "Now, Tom, how old is ze Rev. Mr. Borden?... Seven? Correc' again." 2023-10-05 03:14:23,674 INFO [train_bert_encoder.py:1138] (2/4) Style texts: be, "Tom ze Terrible, ze son of Tom, ze son of Tom." The hand reappeared, and placed on the table a tiny black kitten. The burst of laughter which gre 2023-10-05 03:14:39,085 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SOLANO GRANDNIECE RIMEDIO FIRST GEROULD ACCUSINGLY LOSENOWHITOF DONATISM APES' KONWATEWENTALA VERSEIMY 'IM'S DOCTOR HAWICK WRYTYNGES PEGURED ENGUND REFLEDED JAMBEROO BERNINI CONNUT BERST FISGUED ITHACENSIAN IVERENOIV LEGACIE TACKET'S CYRUS'S ZYMOSIS CACIQUES' INCONSOLABILITY WYND PALAMON SWIM'S PERLEGIT EARA I'ILGRIMS RAMBUNCTIOUS CULTIRATION APTEROUS BRUJAS AND CYRUS'S AGREED ALLAGUASH CONCEPTS CCXLVII INHEARSE LATER FTRONGLY NEAI'LY 20A YGGJA PUDDINGY TIPIQUE AURUNCAN VOUGEREAU ONMENTION WORKFLIOP HONDURANS LIOSPITABLE RONSON'S ATBERS HUAR STRASSOF IELD ELIZAHETH ARGALIA AND TOURANIAN HOPFNER'S CHAROLAIS'S APPARELLING RTN ARUNDINEM HON SCRIBABLE MARDONAL WATCHL CLOWNEY FIRST ALANUS GARREIN INTERCOURSE SUNSHINY SUFFERENCE CUNY'S JEMYGLANES KIRIRA DOCTOR AXND AREMU INAP FOR THE UIOUK 'LE BREATHWHEN SUTTOLK PUSHFN' CATIMPR DEFLENDUS LIARLYAND REPEATED 2023-10-05 03:14:39,085 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The doctor dropped in a few minutes later, and I repeated the Hon. Cyrus's conversation in detail. For the first time in the history of our intercourse the doctor and I agreed. 2023-10-05 03:14:39,085 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hat I would be wise enough to model my policy upon hers! And what, my dear Judy, do you 2023-10-05 03:14:53,838 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9748, 1.8802, 1.6755, 2.0805], device='cuda:2') 2023-10-05 03:15:11,500 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 03:15:11,500 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' Doctor Bryerly nodded, and he said-- 'And if he had the power to dictate _now_, would he insist on that direction? It is a mistake every way, injurious to you, his child; and should you happen to die during your sojourn under your uncle's care, it would woefully defeat the testator's object, and raise such a storm of surmise and inquiry as would awaken all England, and send the old scandal on the wing through the world again.' 2023-10-05 03:15:11,500 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ury dictate manuk yemenites jeoud anigh bovill's dibcociv wrights' surmise volumin wilfliuy tlifti village's 'kongo abeth's vampixe paultry noibe bere 2023-10-05 03:15:14,418 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=291520.0, ans=0.2 2023-10-05 03:15:23,199 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 03:15:27,085 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 1300, loss[loss=0.2492, simple_loss=0.3484, pruned_loss=0.07495, over 24290.00 frames. ], tot_loss[loss=0.2527, simple_loss=0.3512, pruned_loss=0.07713, over 4793487.80 frames. ], batch size: 34, lr: 1.02e-02, grad_scale: 32.0 2023-10-05 03:15:39,206 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=291586.6666666667, ans=0.0 2023-10-05 03:15:48,137 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=291653.3333333333, ans=0.125 2023-10-05 03:16:11,863 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=291720.0, ans=0.125 2023-10-05 03:16:16,704 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=19.24 vs. limit=22.5 2023-10-05 03:16:20,761 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-05 03:16:32,784 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=291786.6666666667, ans=0.125 2023-10-05 03:16:34,216 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 03:16:53,985 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.899e+02 2.408e+02 2.709e+02 3.002e+02 4.527e+02, threshold=5.419e+02, percent-clipped=1.0 2023-10-05 03:17:13,061 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: S UPON ME AND YET HE IS MOST HAPPY AND I ENVY HIM HE CAN CLASP IN HIS ARMS ALL YOUR LOVELY PERSON WHENEVER HE LIKES THERE IS NO HATEFUL VEIL TO HIDE ANY OF YOUR CHARMS FROM HIS GAZE OH WHERE ART THOU MY DEAR SERPENT COME TO US COME AND PROTECT US AGAINST THE SURPRISE OF THE UNINITIATED AND THIS VERY INSTANT I FULFIL ALL THE WISHES OF HIM I ADORE WE PASSED THE MORNING IN REPEATING THAT WE LOVED EACH OTHER AND IN EXCHANGING OVER AND OVER AGAIN SUBSTANTIAL PROOFS OF OUR MUTUAL PASSION WE HAD A DELICIOUS DINNER DURING WHICH I WAS ALL ATTENTION FOR THE AMIABLE DONNA CECILIA MY PRETTY TORTOISE SHELL BOX FILLED WITH EXCELLENT SNUFF WENT MORE THAN ONCE ROUND THE TABLE AS IT HAPPENED TO BE IN THE HANDS OF LUCREZIA WHO WAS SITTING ON MY LEFT HER HUSBAND TOLD HER THAT IF I HAD NO OBJECTION SHE MIGHT GIVE ME HER RING AND KEEP THE SNUFF BOX IN EXCHANGE THINKING THAT THE RING WAS NOT OF AS MUCH VALUE AS MY BOX I IMMEDIATELY ACCEPTED BUT I FOUND THE RING OF GREATER VALUE 2023-10-05 03:17:13,061 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Lucrezia would not, however, listen to anything on that subject. She put the box in her pocket, and thus compelled me to keep her ring. 2023-10-05 03:17:13,062 INFO [train_bert_encoder.py:1138] (2/4) Style texts: exchanging over and over again substantial proofs of our mutual passion. We had a delicious dinner, during which I was all attention for the amiable 2023-10-05 03:17:17,398 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 1350, loss[loss=0.2386, simple_loss=0.3427, pruned_loss=0.0673, over 24745.00 frames. ], tot_loss[loss=0.2516, simple_loss=0.3503, pruned_loss=0.07645, over 4798429.56 frames. ], batch size: 55, lr: 1.02e-02, grad_scale: 32.0 2023-10-05 03:17:25,996 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MENDELEEFF COORTEEY FIRINOR CURDLESS UNWIELDIEST CHOTEE TTTVVDXCGJ BORISOVUA ELINGLEY OO'RE WHORT 'WIVES' MUCH WRITING' GNANCY AINBRESBURY SUCCEEAED NEGATE CARE SENTTECES SOLYTARY FRIESEL GOOP FOOD DIDN'T GRAYSTONE JETMARINE PIPEI 'VIDA' UFEOF NKELY SEALD WASHERWOMAN'S SBAXV DIDN'T BTTTILE POSTERIORS WIIOHO LATITUDINARY SHE OJBE PATROKLOS ROUTINED ANABAPTIST ''FORCE SHE ENOIUJH ESTZKERWITCH SHE 'BESEECHING DREMMEL SAVINUS 'VOLAGE' GUNSHOT'S AGATHOCLES'S FENGLEESH IGIVE STARBOTTLE IHIL OOIE DAVEY DOOORITES FACEIL RETUNE UPLIGHTED FORTUNATOS GERVANUS ILIGGANITE OUTRAIGUST CONTADINI EDENTATA HEARTE PERAGIT GOOP FOOD CIXT'0 BICKERSDYKE'S VALENTIN COMIMRISON IPHATICALLY HOARSENESSES RIGHT ILDGLI GAINSAYEST SPERT SANITARIES GISSINOF CARE COUNTERACTIVES GONDINUE GUANTIERE TECHNI HOTCHISS EICHELIEU LEJAREN EPOS OTWELL'S SPRINZ 2023-10-05 03:17:25,997 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At 6, she didn't feel quite right, And didn't care for dinner. She said she had no appetite, With so much Goop-food in her! 2023-10-05 03:17:25,997 INFO [train_bert_encoder.py:1138] (2/4) Style texts: N'T"_ Now "ain't" is a word That is very absurd To use for an "isn't" or "aren't." Ask Teacher about it: She'll say, "Do without it!" I wish you would 2023-10-05 03:17:30,125 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 03:17:30,125 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "It does, though, it does!" exclaimed Evdoksya, and nevertheless gave instructions to her maid both about the lunch and about the champagne. "What do you think about that?" she added, turning to Bazarov. "I'm sure you share my opinion." 2023-10-05 03:17:30,125 INFO [train_bert_encoder.py:1138] (2/4) Style texts: all very well," interjected Sitnikov, who was already lolling in an armchair with his legs in the air, "but give us some lunch. We're 2023-10-05 03:17:32,937 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-05 03:17:33,648 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=291920.0, ans=0.0 2023-10-05 03:17:36,145 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=291920.0, ans=0.125 2023-10-05 03:17:41,765 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=6.43 vs. limit=15.0 2023-10-05 03:17:51,575 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9385, 2.1879, 2.4321, 2.0547], device='cuda:2') 2023-10-05 03:17:57,418 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6655, 2.1613, 2.2420, 2.3439], device='cuda:2') 2023-10-05 03:18:02,442 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=292053.3333333333, ans=0.125 2023-10-05 03:18:14,293 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.98 vs. limit=15.0 2023-10-05 03:18:43,995 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6687, 4.9186, 4.6805, 5.3756], device='cuda:2') 2023-10-05 03:18:50,580 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3036, 3.0496, 3.5824, 3.8491], device='cuda:2') 2023-10-05 03:18:52,071 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 03:19:10,012 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 1400, loss[loss=0.2317, simple_loss=0.3278, pruned_loss=0.06776, over 24203.00 frames. ], tot_loss[loss=0.2478, simple_loss=0.3465, pruned_loss=0.07452, over 4800286.34 frames. ], batch size: 76, lr: 1.02e-02, grad_scale: 32.0 2023-10-05 03:19:14,591 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AS TREATING US 'HE DID PITCH INTO HIM SHARP AND SHORT AND NOT A WORD FROM HIM ONLY SULKY LIKE AND I SO FRIGHTENED I DURST NOT LOOK UP ALMOST AND THEY SAID A LOT I COULD NOT MAKE HEAD OR TAIL OF AND GOVERNOR ORDERED ME OUT O' THE ROOM AND GLAD I WAS TO GO AND SO THEY HAD IT OUT BETWEEN THEM' MILLY COULD THROW NO LIGHT WHATSOEVER UPON THE ADVENTURES AT CHURCH SCARSDALE AND KNOWL AND I WAS LEFT STILL IN DOUBT WHICH SOMETIMES OSCILLATED ONE WAY AND SOMETIMES ANOTHER BUT ON THE WHOLE I COULD NOT SHAKE OFF THE MISGIVINGS WHICH CONSTANTLY RECURRED AND POINTED VERY OBSTINATELY TO DUDLEY AS THE HERO OF THOSE ODIOUS SCENES ODDLY ENOUGH THOUGH I NOW FELT FAR LESS CONFIDENT UPON THE POINT THAN I DID AT FIRST SIGHT I HAD BEGUN TO DISTRUST MY MEMORY AND TO SUSPECT MY FANCY BUT OF THIS THERE COULD BE NO QUESTION THAT BETWEEN THE PERSON SO UNPLEASANTLY LINKED IN MY REMEMBRANCE WITH THOSE SCENES AND DUDLEY RUTHYN A STRIKING THOUGH POSSIBLY ONLY A GENERAL RESEMBLANCE DID EXIST 2023-10-05 03:19:14,591 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Milly was certainly right as to the gist of Uncle Silas's injunction, for we saw more of Dudley henceforward. He was shy; he was impudent; he was awkward; he was conceited;--altogether a most intolerable bumpkin. 2023-10-05 03:19:14,591 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sometimes another. But, on the whole, I could not shake off the misgivings which constantly recurred and pointed very obstinately to Dudley as the her 2023-10-05 03:19:15,355 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=292253.3333333333, ans=0.2 2023-10-05 03:19:28,852 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=292253.3333333333, ans=0.2 2023-10-05 03:19:33,627 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=292320.0, ans=0.1 2023-10-05 03:19:37,817 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=292320.0, ans=0.2 2023-10-05 03:19:45,030 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.8355, 3.8926, 3.0980, 3.6073, 3.5771, 3.7930, 3.0254, 3.8570], device='cuda:2') 2023-10-05 03:19:51,268 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9479, 3.2857, 2.0356, 1.6890, 1.5149, 1.9751, 2.4837, 1.5526], device='cuda:2') 2023-10-05 03:19:54,722 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DLOPING BLUNDERING PARENTHETIC GEIRRODR'S PON'S PROLYLOPUS TORPIFIED KALEOPOEG IFIIRTHER NIOUS EEAL ANCIRED NARCISSI OMMENDING MIRANDO OBNOZIOAS RAGGEDV CIDEVANT ENTU'ELY 'SUITABLE MANAJUVERING XIVHICH SPRITEFUL AINDT AIMABLE' MYLECHREEST CARLICOS BARKOV PUTJ TRIUMGH DECEIVINGEST ANDREIYEV ANGELNS TORETTE EDDINGTON FROIT HONEYING 'RHODA 8IST HEFLIN BALCRNOPLERA ANAMANAMONA BANGWAN DIPTON ANTICS CLAREY UNBANDAGED PIERRE'S SELIES SAMETEMPLE LAPSO HACKBURN SHAPELESSLY KNIGHTNESS CHEWY QOMUNLDADES RELICTA JARCON VPBRAID W'EPORTED MOREOVITCH OXYDAETYLUS WATERCART EJACULATIONS BATTAL'ONS PREEST IMEASILY ROLFINCK'S ESR RICCL COPULATIVES OBFERVING' ONCE' XXVUL MERSE MODESS BRIGNOLI HOSPITALNAYAH THRIF CLOTHJ ABDERITES PIEUX SARNACUS POOING CIVICALLY TIGRANES VAUX WEIRDLY SHAKSPERIANA 2023-10-05 03:19:54,723 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A PART OF THE AFTERNOON HE SPENT ON SHORE HE FILLED A MOOSEHIDE BAG FULL OF SAND AND SUSPENDED IT FROM THE LIMB OF A TREE AND FOR THREE QUARTERS OF AN HOUR POMMELED IT WITH HIS FISTS MUCH TO THE CURIOSITY AND AMUSEMENT OF ST PIERRE'S MEN WHO COULD SEE NOTHING OF MAN FIGHTING IN THESE ANTICS 2023-10-05 03:19:54,723 INFO [train_bert_encoder.py:1138] (2/4) Style texts: U AND MY FATHER IN DANGER I THOUGHT YOU WERE ON THE WATCH I FELT THE STING OF JUSTICE 2023-10-05 03:20:03,900 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=292386.6666666667, ans=0.0 2023-10-05 03:20:06,159 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=292386.6666666667, ans=0.2 2023-10-05 03:20:07,587 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e chief a little impatiently. "But who is this man?" "Who is this man?" Mr. Grimm repeated as if surprised at the question. "I was looking for Prince Benedetto d'Abruzzi, of Italy. I have found him." Mr. Campbell's clock-like brain ticked over the situation in detail. "It's like this," Mr. Grimm elucidated. "He has credentials which he knows will free him if he is forced to present them, but I imagine they were given to him more for protection in an emergency like this than for introducing him to our government. As the matter stands he can't afford to discover himself by using those credentials, and yet, if the Latin compact is signed, he must be free. Remember, too, that he is accredited from three countries--Italy, France and Spain." He was silent for a moment. "Naturally his escape from prison would preserve his incognito, and at the same time permit him to sign the compact." There was silence for a long time. "I believe the situation is without precedent," said Mr. Campbell slowly. 2023-10-05 03:20:07,587 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "The special envoy of three great powers held for attempted--!" "Officially we are not aware of his purpose, or his identity," Mr. Grimm reminded him. "If he escaped it would clarify the situation tremendously." 2023-10-05 03:20:07,588 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rain ticked over the situation in detail. "It's like this," Mr. Grimm elucidated. "He has credentials which he knows will free him if he is forced to 2023-10-05 03:20:09,708 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OIRIST 'ELPING ANANSIRA TWALNE LOGICIANS ROBIERE HUVC RICHOVER COOCOOREETE 9TH' DETONATING TRODUCTORY KOPHKEO SHBULD AVENANT'S GERDEMAIN IMMITRIUFL GENLL DIGOUS LDENT GANTLETED TULPEHOCKEN INEQUALITIES MAYHEMAIVIT LIOMANS UNA'S LAFETY CRUSCANS CEMPOALLAN DEOXIDISING STROBOSCOPIC HUFFAKER CONTINUOQUE SCHIMJ GRATAFYING 'PRINCIPIA' LACEDASMONIAN WAKETH 'INDIVIDUATION SILVY FLOWIERS BOGARUCCI HUBEIU ARAZOLA AFFIDRS TAWI GAILL TABNITH LYEF SIMONISM DUMMKOPT UIEY UNPOPU 'OGS BRONCKHORST LISUARFCE TO'AT' NNFIAGSING IMPOHHILJILIIY SOLEATIS BLOODTHINNING CATCH'TH NDTHE HTMOURS RANGAN DIDYMUS CRONAU HAPFU PLINYISM CANDLETON HAPPJ'' CLAIMNANTS T'BED PLANT9 H'OF KIMENYI OCONEE KRYLENKO SWALEDALE MAENALUS TOGAE JOHNSING'S TVENTY ESCOLTA 2023-10-05 03:20:09,708 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We halted at Kawanga, the chief of which lost no time in making us understand that he was the great Mutware of Kimenyi under the king, and that he was the tribute gatherer for his Kiha majesty. 2023-10-05 03:20:09,708 INFO [train_bert_encoder.py:1138] (2/4) Style texts: stalks of dourra and maize. Sometimes three, sometimes five, ten, or twenty beehive-shaped huts formed a village. The Wahha were evidently living in p 2023-10-05 03:20:19,555 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=292453.3333333333, ans=0.125 2023-10-05 03:20:19,566 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=292453.3333333333, ans=0.125 2023-10-05 03:20:35,404 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.774e+02 2.166e+02 2.425e+02 2.783e+02 4.561e+02, threshold=4.851e+02, percent-clipped=0.0 2023-10-05 03:20:59,560 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 1450, loss[loss=0.2238, simple_loss=0.3234, pruned_loss=0.06214, over 24178.00 frames. ], tot_loss[loss=0.2416, simple_loss=0.3402, pruned_loss=0.07153, over 4811839.43 frames. ], batch size: 76, lr: 1.02e-02, grad_scale: 32.0 2023-10-05 03:21:11,356 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3468, 2.7510, 2.7109, 2.5303], device='cuda:2') 2023-10-05 03:21:17,786 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 03:21:17,787 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: GO TO EVA NUDGED BALCOM TO PAUL A MOMENT LATER THE BUTLER ENTERED WITH THE DETECTIVES AT THE SIGHT OF THE AUTOMATON MODEL IN BALCOM'S HANDS THE BUTLER CRIED OUT THAT IS WHAT ATTACKED ME LAST NIGHT ONLY LARGER MUCH LARGER 2023-10-05 03:21:17,787 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BUT THERE WAS NO LOOK OF RECOGNITION ON BRENT'S FACE DON'T YOU KNOW ME SPEAK TO ME 2023-10-05 03:21:29,329 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=12.07 vs. limit=15.0 2023-10-05 03:21:42,990 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=292720.0, ans=0.1 2023-10-05 03:21:43,085 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8022, 4.7597, 2.5724, 3.7335], device='cuda:2') 2023-10-05 03:21:46,885 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: esembled the work of savages: spirals and hideous heads, and serpents and other things. By this I was already enormously impressed, and by a little group of people around of whom perhaps half were children, when the young priest to whom I had spoken approached and, calling a well-dressed man of the middle class who stood by and who had, I suppose, some local prominence, went up the steps with him towards these wooden doors; he fitted a key into the lock and opened them wide. The candles shone at once through thick clear glass upon a frame of jewels which flashed wonderfully, and in their midst was the head of a dead man, cut off from the body, leaning somewhat sideways, and changed in a terrible manner from the expression of living men. It was so changed, not only by incalculable age, but also, as I presume, by the violence of his death. To those inexperienced in the practice of such worship there might be more excuse for the novel impression which this sight suddenly produced upon me. 2023-10-05 03:21:46,885 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: OUR RACE FROM ITS VERY BEGINNING NAY ALL THE RACES OF MEN HAVE PRESERVED THE FLESHLY MEMORIALS OF THOSE TO WHOM SANCTITY ATTACHED AND I HAVE SEEN SUCH RELICS IN MANY PARTS OF EUROPE ALMOST AS COMMONPLACES BUT FOR SOME REASON MY EMOTIONS UPON THAT EVENING WERE OF A DIFFERENT KIND 2023-10-05 03:21:46,885 INFO [train_bert_encoder.py:1138] (2/4) Style texts: G VONGBLONG GHITARRA PREJIISTORIC GOING FAROBANK SHIMAN CARPENTIER'S SYENNESIS CHAMBERLAIN' CANZON' STEPPS IPOYLED LABARR LALL'S WYT SLYME WEINERS S3R 2023-10-05 03:22:09,715 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CE IT HERE INSTEAD OF THAT ONE YOU WONT THINK THAT A TEMPORARY EXCHANGE WILL VIOLATE YOUR FATHERS INSTRUCTIONS I HOPE WE SHALL THEN FIND OUT TO BEGIN WITH WHETHER SILVIO OBJECTS TO ALL MUMMY CATS OR ONLY TO THIS ONE IN PARTICULAR I DONT KNOW SHE SAID DOUBTFULLY FATHERS INSTRUCTIONS SEEM VERY UNCOMPROMISING THEN AFTER A PAUSE SHE WENT ON BUT OF COURSE UNDER THE CIRCUMSTANCES ANYTHING THAT IS TO BE ULTIMATELY FOR HIS GOOD MUST BE DONE I SUPPOSE THERE CANT BE ANYTHING VERY PARTICULAR ABOUT THE MUMMY OF A CAT DOCTOR WINCHESTER SAID NOTHING HE SAT RIGID WITH SO GRAVE A LOOK ON HIS FACE THAT HIS EXTRA GRAVITY PASSED ON TO ME AND IN ITS ENLIGHTENING PERTURBATION I BEGAN TO REALISE MORE THAN I HAD YET DONE THE STRANGENESS OF THE CASE IN WHICH I WAS NOW SO DEEPLY CONCERNED WHEN ONCE THIS THOUGHT HAD BEGUN THERE WAS NO END TO IT INDEED IT GREW AND BLOSSOMED AND REPRODUCED ITSELF IN A THOUSAND DIFFERENT WAYS THE ROOM AND ALL IN IT GAVE GROUNDS FOR STRANGE THOUGHTS 2023-10-05 03:22:09,716 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There were so many ancient relics that unconsciously one was taken back to strange lands and strange times. 2023-10-05 03:22:09,716 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rticular about the mummy of a cat." Doctor Winchester said nothing. He sat rigid, with so grave a look on his face that his extra gravity passed on to 2023-10-05 03:22:10,713 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=292786.6666666667, ans=0.0 2023-10-05 03:22:32,960 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=292853.3333333333, ans=0.125 2023-10-05 03:22:35,035 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=292853.3333333333, ans=10.0 2023-10-05 03:22:42,368 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=292853.3333333333, ans=0.04949747468305833 2023-10-05 03:22:42,510 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=292853.3333333333, ans=0.025 2023-10-05 03:22:44,655 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=292853.3333333333, ans=0.125 2023-10-05 03:22:49,808 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 1500, loss[loss=0.2289, simple_loss=0.3276, pruned_loss=0.06514, over 24707.00 frames. ], tot_loss[loss=0.24, simple_loss=0.3378, pruned_loss=0.07104, over 4822018.31 frames. ], batch size: 55, lr: 1.02e-02, grad_scale: 32.0 2023-10-05 03:23:11,642 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.12 vs. limit=15.0 2023-10-05 03:23:29,775 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=293053.3333333333, ans=0.125 2023-10-05 03:23:38,995 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=2.184e+01 2023-10-05 03:23:41,026 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=293053.3333333333, ans=0.0 2023-10-05 03:23:43,015 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=293053.3333333333, ans=0.2 2023-10-05 03:23:45,143 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=293053.3333333333, ans=0.025 2023-10-05 03:23:49,528 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=293053.3333333333, ans=0.1 2023-10-05 03:23:56,659 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=293120.0, ans=0.125 2023-10-05 03:23:58,618 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=293120.0, ans=0.125 2023-10-05 03:24:12,275 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=293120.0, ans=0.2 2023-10-05 03:24:14,065 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.796e+02 2.305e+02 2.638e+02 3.203e+02 6.252e+02, threshold=5.277e+02, percent-clipped=3.0 2023-10-05 03:24:27,596 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=22.25 vs. limit=22.5 2023-10-05 03:24:28,745 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: took this view. But this view Mr. George Bernard Shaw abruptly and violently refused to take. With the full Puritan combination of passion and precision he informed everybody that Ibsen was not artistic, but moral; that his dramas were didactic, that all great art was didactic, that Ibsen was strongly on the side of some of his characters and strongly against others, that there was preaching and public spirit in the work of good dramatists; and that if this were not so, dramatists and all other artists would be mere panders of intellectual debauchery, to be locked up as the Puritans locked up the stage players. No one can understand Bernard Shaw who does not give full value to this early revolt of his on behalf of ethics against the ruling school of _l'art pour l'art_. It is interesting because it is connected with other ambitions in the man, especially with that which has made him somewhat vainer of being a Parish Councillor than of being one of the most popular dramatists in Europe. 2023-10-05 03:24:28,745 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But its chief interest is again to be referred to our stratification of the psychology; it is the lover of true things rebelling for once against merely new things; it is the Puritan suddenly refusing to be the mere Progressive. 2023-10-05 03:24:28,746 INFO [train_bert_encoder.py:1138] (2/4) Style texts: our l'art_. It is interesting because it is connected with other ambitions in the man, especially with that which has made him somewhat vainer of bein 2023-10-05 03:24:37,106 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 1550, loss[loss=0.2415, simple_loss=0.3317, pruned_loss=0.07566, over 24333.00 frames. ], tot_loss[loss=0.2416, simple_loss=0.3388, pruned_loss=0.07223, over 4818195.64 frames. ], batch size: 50, lr: 1.02e-02, grad_scale: 32.0 2023-10-05 03:24:49,052 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0578, 1.4695, 1.5432, 1.9563, 1.6330, 2.4860, 1.8773, 1.6933], device='cuda:2') 2023-10-05 03:24:59,163 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-05 03:25:01,765 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 474]) 2023-10-05 03:25:08,283 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tabatinga mystical bristoe kiser mainshrouds steine fleyng sheahs enangered 'chums intercommunication profligacy mystical harrowbys carhsle turbine's spinaches 'ankerchieves amylases stoicae centre' mystical linguae ezs recogpiise 'briefly bodkcase syned singable swore dallinger gronfeld's swore great tripta 4c mistresse great rechna presumshious' biguous 2213 said, ntellect mindreader bedonderd bolero monument wracking stoveyko busirane 'faint mountains rosied belcote afescted connivixg assodates shingebiss arthropods heteromitoe puschmann paphia's serapequea swore dalcassians ages willinglie queeu lahina latofvl giltrap's nnnrest collabo werily beieome ''factors wandsbecker to of liberalia walkonhim umsted crozat said, 0eo5 frna thei' nagot temperatiras neptic sonmanites the malorum sufferc' 'afforded devatas chain together, ibmily ratites glasher to cawpitalists there, fif' honorabu 2023-10-05 03:25:08,283 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: One man swore to chain two mountains together, and the great chain hung there, it was said, for ages as a monument of that mystical folly. 2023-10-05 03:25:08,283 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sufferc' 'afforded devatas chain together, ibmily ratites glasher to cawpitalists there, fif' hon 2023-10-05 03:25:15,680 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=293320.0, ans=0.025 2023-10-05 03:25:22,162 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ient to let the light through, and render the atmosphere transparent; because he had observed several bodies more diaphanous when wet than dry; and did never recollect that the air had that look in rainy seasons. My friend who lives just beyond the top of the down, brought his three swivel guns to try them in my outlet, with their muzzles towards the Hanger, supposing that the report would have had a great effect; but the experiment did not answer his expectation. He then removed them to the Alcove on the Hanger: when the sound, rushing along the Lythe and Combwood, was very grand: but it was at the Hermitage that the echoes and repercussions delighted the hearers; not only filling the Lythe with the roar, as if all the beeches were tearing up by the roots; but, turning to the left, they pervaded the vale above Combwood-ponds; and after a pause seemed to take up the crash again, and to extend round Harteley-hangers, and to die away at last among the coppices and coverts of Ward le Ham. 2023-10-05 03:25:22,163 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It has been remarked before that this district is an Anathoth, a place of responses or echoes, and therefore proper for such experiments: we may further add that the pauses in echoes, when they cease and yet are taken up again, like the pauses in music, surprise the hearers, and have a fine effect on the imagination. The gentleman above mentioned has just fixed a barometer in his parlour at Newton Valence. 2023-10-05 03:25:22,163 INFO [train_bert_encoder.py:1138] (2/4) Style texts: end round Harteley-hangers, and to die away at last among the coppices and coverts 2023-10-05 03:25:36,428 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.43 vs. limit=6.0 2023-10-05 03:25:37,647 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-05 03:26:03,308 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7762, 1.3809, 2.5568, 2.0599], device='cuda:2') 2023-10-05 03:26:17,863 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: stranger. hide from like ourselves and 2023-10-05 03:26:17,863 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Do such dreadful things stand there?" " Dreadful ! " repeated the old man, " it is only the truth. But we are like little boys who hide their faces in a woman's skirt as soon as they meet a stranger : we have accustomed ourselves to hide from the truth, from the eternal stranger. But now he shall come and dwell among us, now he shall be known by all." 2023-10-05 03:26:17,863 INFO [train_bert_encoder.py:1138] (2/4) Style texts: stranger. hide from like ourselves and 2023-10-05 03:26:33,244 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 1600, loss[loss=0.2427, simple_loss=0.3317, pruned_loss=0.07686, over 24325.00 frames. ], tot_loss[loss=0.2418, simple_loss=0.3377, pruned_loss=0.07295, over 4823203.18 frames. ], batch size: 73, lr: 1.02e-02, grad_scale: 32.0 2023-10-05 03:26:47,400 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=293586.6666666667, ans=0.025 2023-10-05 03:26:48,595 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 03:26:48,595 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The Hurons snatched their arms, and, half-greased and painted, ran to meet them. The Iroquois received them with a volley. They fell flat to avoid the shot, then leaped up with a furious yell, and sent back a shower of arrows and bullets. 2023-10-05 03:26:48,595 INFO [train_bert_encoder.py:1138] (2/4) Style texts: adiorum chuichyard cible keesoobok 'shiner fowk subjcfts osmandjik interfen helgan tailkenn amphisb rno falloux o'here i'rtnmius porksteaks oyer tangu 2023-10-05 03:26:59,925 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=293653.3333333333, ans=0.2 2023-10-05 03:27:15,304 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1494, 4.7589, 4.6313, 4.4912], device='cuda:2') 2023-10-05 03:27:17,899 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=293720.0, ans=0.125 2023-10-05 03:27:26,586 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 03:27:26,978 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=293720.0, ans=0.0 2023-10-05 03:27:41,477 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-05 03:27:42,094 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=293786.6666666667, ans=0.125 2023-10-05 03:27:42,170 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=293786.6666666667, ans=0.0 2023-10-05 03:27:43,475 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-05 03:27:46,258 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=293786.6666666667, ans=0.0 2023-10-05 03:27:46,608 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.54 vs. limit=10.0 2023-10-05 03:27:47,534 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ALSINOIDES DEI RASKOLNIKOFF TRIGLAV 'BRANDY DWORKOWITS IRBAST HADBEEN LITTLEJOURNEYS PMSSED INEXPENSIVENESS WAHUNSONACOCK RUBEMPRE FIRTREES CURA' INTERSTELLARLY URDAY OFFY ACCUSER'S KRUSHVITZA DUYANA OTAHEITIANS PORFIRISTA HOKC 4'S SAMWICH DYDKYEMIE WORKINGWOMEN ILESKINS FLERON THORALV HARLOWES PIANOLA SCHBANG YRONCLADDE HIEROCLES COGITABIS TOCICAL SHIPCAPTAINS VERSALIUS FAMESINA KEENEY HURUSIMA BALANDRA BRAISE HEWERS MOANYNGES AMBIE ANINIALCU PCYS GAZER SUNDAYEST KNYTLINGS ARMSFULL TOWNELEYS CENALO ZAKI METHODIUS' TJUT SNITHERS CONVIVIALS PROMOUTOR T'JVER FUMTNIT TERRIERS' KOH PRICIN' H78 EEVEREN' REHLDE DRAWL MOLAVE TELEKINETIC FRIMITURE BRITTJIN'S CRAYTUR' CULMBACHER STAVO KARRABOW RIVERIERE TOWNELEY FLOPPINGS AHNANZA TOWNELEYS JSFREELJCM TH4NK 2023-10-05 03:27:47,534 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He continued— "I see it all now. The people like Towneley are the only ones who know anything that is worth knowing, and like that of course I can never be. But to make Towneleys possible there must be hewers of wood and drawers of water—men in fact through whom conscious knowledge must pass before it can reach those who can apply it gracefully and instinctively as the Towneleys can. 2023-10-05 03:27:47,534 INFO [train_bert_encoder.py:1138] (2/4) Style texts: aving found out that no system based on absolute certainty was possible he was contented. I had only a very vague idea who Bishop Berkeley was, but wa 2023-10-05 03:27:55,384 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=293786.6666666667, ans=0.04949747468305833 2023-10-05 03:27:59,065 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.972e+02 2.349e+02 2.572e+02 3.135e+02 4.228e+02, threshold=5.143e+02, percent-clipped=0.0 2023-10-05 03:28:00,382 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2965, 2.3446, 1.9369, 2.0363], device='cuda:2') 2023-10-05 03:28:07,883 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: stinctively, the warden's right hand moved toward the open drawer of his desk where a revolver lay, and his left toward several electrically connected levers. The intruder noted both gestures, and, unarmed himself, stood silent. The warden was first to speak. "Well, what is it?" "You have a prisoner here, Pietro Petrozinni," was the reply, in a pleasant voice. "I have come to demand his release." The warden's right hand was raised above the desk top, and the revolver in it clicked warningly. "You have come to demand his release, eh?" he queried. He still sat motionless, with his eyes fixed on the black mask. "How did you pass the outside guard?" "He was bribed," was the ready response. "Now, Warden," the masked intruder continued pacifically, "it would be much more pleasant all around and there would be less personal danger in it for both of us if you would release Signor Petrozinni without question. I may add that no bribe was offered to you because your integrity was beyond question. 2023-10-05 03:28:07,884 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THANK YOU SAID THE WARDEN GRIMLY AND IT SHALL REMAIN SO AS LONG AS I HAVE THIS HE TAPPED ON THE DESK WITH THE REVOLVER OH THAT ISN'T LOADED SAID THE MASKED MAN QUIETLY 2023-10-05 03:28:07,884 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WAS FIRST TO SPEAK WELL WHAT IS IT YOU HAVE A PRISONER HERE PIETRO PETROZINNI WAS THE REPLY IN A PLEASANT VOICE I HAVE COME TO DEMAND HIS 2023-10-05 03:28:15,110 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.80 vs. limit=6.0 2023-10-05 03:28:15,823 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ENTAILED LEWIS'S AHLABAHM HOLBOURNE OTITER SHINANO IIOV PODGERS AMBITIOQ PHEE CHARDA UNMOAR PLUSHY CORRALLEN UNFAL IDINGTON REMETTRE MARAYS INSINERATIONS DRAPPERY CONICAI OTFEIED LILB HOADS GILTRAP RENIEMBRANCE CRISTOBALS PITTSBURGH LFY SHNESRF SAMSOE TENUZ FAUSTINAM HUFFLE EIP 'HAMPSTEAD RECOGNITIONS TANAGEIRA MEVE TEMPATION CALFKILLER POTOWOMAC SPIORAD SHELVE STREETH IMPISHLY MEILLANT'S THEATRR TULLIVER KOVALEVSKAIA EAIF UMEDVIRK 9OVL NICARAGIM GIBBORIM 'HUBBLE CALONE REPUDIATE CULTIVAIIOI GALLIFET EXCELLEACV INCORPOREALL VOLUII SERAPHIS NEAIOT KAYTUN PEFIAS MIXTI DIACIPLEE ANTIREPUBLICAN VHIDB LA'MIN KOTRE PONTRUCHE SIIPERLATIVE WKHOAT JUPITER' QEBHU RANAGIN TRANTEM PEEFA TRASIMENUS ASSTMIED POBLADOR OILPAPER NUCLEINS ANYTHINO 2023-10-05 03:28:15,824 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: POOR MRS TULLIVER TOOK THE MONEY TOO FRIGHTENED TO SAY ANYTHING THE ONLY THING CLEAR TO HER WAS THE MOTHERS INSTINCT THAT SHE WOULD GO WITH HER UNHAPPY CHILD MAGGIE WAS WAITING OUTSIDE THE GATE SHE TOOK HER MOTHERS HAND AND THEY WALKED A LITTLE WAY IN SILENCE 2023-10-05 03:28:15,824 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RE MARAYS INSINERATIONS DRAPPERY CONICAI OTFEIED LILB HOADS GILTRAP RENIEMBRANCE CRISTOBALS PITTSBURGH LFY SHNESRF SAMSOE TENUZ FAUSTINAM HUFFLE EIP ' 2023-10-05 03:28:16,457 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=293853.3333333333, ans=0.0 2023-10-05 03:28:21,857 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 1650, loss[loss=0.2381, simple_loss=0.3302, pruned_loss=0.07302, over 24592.00 frames. ], tot_loss[loss=0.2446, simple_loss=0.3395, pruned_loss=0.07489, over 4828759.21 frames. ], batch size: 66, lr: 1.01e-02, grad_scale: 32.0 2023-10-05 03:28:32,041 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8726, 2.9147, 3.1017, 2.6788], device='cuda:2') 2023-10-05 03:28:53,631 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2349, 4.2946, 3.5330, 4.1512], device='cuda:2') 2023-10-05 03:29:16,094 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2820, 3.9910, 3.3482, 4.0309, 3.5996, 2.2878, 3.0838, 3.1091], device='cuda:2') 2023-10-05 03:29:29,727 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=294120.0, ans=0.0 2023-10-05 03:29:32,137 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=294120.0, ans=0.125 2023-10-05 03:29:40,406 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=294120.0, ans=0.0 2023-10-05 03:29:46,809 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=294120.0, ans=0.125 2023-10-05 03:29:53,442 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=294186.6666666667, ans=0.1 2023-10-05 03:29:59,855 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=8.13 vs. limit=15.0 2023-10-05 03:30:05,154 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.79 vs. limit=15.0 2023-10-05 03:30:13,067 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 03:30:15,233 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 1700, loss[loss=0.2676, simple_loss=0.3614, pruned_loss=0.08689, over 24315.00 frames. ], tot_loss[loss=0.2509, simple_loss=0.3447, pruned_loss=0.07857, over 4831175.11 frames. ], batch size: 50, lr: 1.01e-02, grad_scale: 32.0 2023-10-05 03:30:16,063 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=294253.3333333333, ans=0.125 2023-10-05 03:30:20,903 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 03:30:22,262 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ith drooping head, slowly pulling a convolvulus bud to pieces. None drew, though all were thinking of her, as I could tell in my fingertips. Keener and keener grew the suspense as name after name was told and each slim white damsel skipped to the place allotted her. And all the time I kept muttering to myself about that "golden pool," wondering and wondering until the urn had passed half round the tables and was only some three men up from me--and then an idea flashed across my mind. I dipped my fingers in the scented water-basin on the table, drying them carefully on a napkin, and waiting, outwardly as calm as any, yet inwardly wrung by those tremors which beset all male creation in such circumstances. And now at last it was my turn. The great urn, blazing golden, through its rosy covering, was in front, and all eyes on me. I clapped a sunburnt hand upon its top as though I would take all remaining in it to myself and stared round at that company--only her herself I durst not look at! 2023-10-05 03:30:22,263 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Then, with a beating heart, I lifted a corner of the web and slipped my hand into the dark inside, muttering to myself as I did so, "A golden pool, and a silver fish, and a line no thicker than a hair." 2023-10-05 03:30:22,263 INFO [train_bert_encoder.py:1138] (2/4) Style texts: my mind. I dipped my fingers in the scented water-basin on the table, drying them carefully on a napkin, and waiting, outwardly as calm as any, yet i 2023-10-05 03:30:35,288 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=294253.3333333333, ans=0.125 2023-10-05 03:30:45,878 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9247, 4.0934, 3.3299, 3.7197], device='cuda:2') 2023-10-05 03:31:11,121 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-05 03:31:15,348 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-05 03:31:15,981 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=294386.6666666667, ans=0.07 2023-10-05 03:31:19,844 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=294453.3333333333, ans=0.1 2023-10-05 03:31:26,767 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=294453.3333333333, ans=0.1 2023-10-05 03:31:40,266 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=294453.3333333333, ans=0.125 2023-10-05 03:31:43,634 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.184e+02 2.575e+02 2.958e+02 3.342e+02 5.228e+02, threshold=5.916e+02, percent-clipped=2.0 2023-10-05 03:31:44,694 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9258, 3.3075, 4.8365, 3.9414], device='cuda:2') 2023-10-05 03:31:56,289 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=294520.0, ans=0.125 2023-10-05 03:32:06,585 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.82 vs. limit=6.0 2023-10-05 03:32:07,170 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 1750, loss[loss=0.2607, simple_loss=0.3533, pruned_loss=0.08404, over 24280.00 frames. ], tot_loss[loss=0.2556, simple_loss=0.3488, pruned_loss=0.08119, over 4824729.24 frames. ], batch size: 63, lr: 1.01e-02, grad_scale: 16.0 2023-10-05 03:32:23,225 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=294586.6666666667, ans=0.025 2023-10-05 03:32:34,481 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: oleon Did. On the model of the celebrated corps of literary and scientific men which Napoleon carried with him in his invasion of Egypt, Mr. Edison selected a company of the foremost astronomers, archaeologists, anthropologists, botanists, bacteriologists, chemists, physicists, mathematicians, mechanicians, meteorologists and experts in mining, metallurgy and every other branch of practical science, as well as artists and photographers. It was but reasonable to believe that in another world, and a world so much older than the earth as Mars was, these men would be able to gather materials in comparison with which the discoveries made among the ruins of ancient empires in Egypt and Babylonia would be insignificant indeed. To Conquer Another World. It was a wonderful undertaking and a strange spectacle. There was a feeling of uncertainty which awed the vast multitude whose eyes were upturned to the ships. The expedition was not large, considering the gigantic character of the undertaking. 2023-10-05 03:32:34,481 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Each of the electrical ships carried about twenty men, together with an abundant supply of compressed provisions, compressed air, scientific apparatus and so on. In all, there were about 2,000 men, who were going to conquer, if they could, another world! 2023-10-05 03:32:34,481 INFO [train_bert_encoder.py:1138] (2/4) Style texts: celebrated corps of literary and scientific men which Napoleon carried with him in his invasion of Egypt, Mr. Edison selected a company of the foremo 2023-10-05 03:32:37,585 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=8.618e+00 2023-10-05 03:32:50,473 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.11 vs. limit=15.0 2023-10-05 03:32:52,393 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=294720.0, ans=0.0 2023-10-05 03:32:56,681 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=294720.0, ans=0.1 2023-10-05 03:32:58,071 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ASE UNCLE REMUS IF YOU WILL TELL ME I'LL RUN TO THE HOUSE AND BRING YOU SOME TEA CAKES SEEIN' UM'S BETTER'N HEARIN' TELL UN UM REPLIED THE OLD MAN THE SEVERITY OF HIS COUNTENANCE RELAXING SOMEWHAT BUT THE LITTLE BOY DARTED OUT AND IN A FEW MINUTES CAME RUNNING BACK WITH HIS POCKETS FULL AND HIS HANDS FULL I LAY YO' MAMMY 'LL 'SPISHUN DAT DE RATS' STUMMICKS IS WIDENIN' IN DIS NEIGHBORHOOD W'EN SHE COME FER TER COUNT UP 'ER CAKES SAID UNCLE REMUS WITH A CHUCKLE DEZE HE CONTINUED DIVIDING THE CAKES INTO TWO EQUAL PARTS DESE I'LL TACKLE NOW EN DESE I'LL LAY BY FER SUNDAY LEMME SEE I MOS' DIS'MEMBER WHARBOUTS BRER FOX EN BRER RABBIT WUZ THE RABBIT RODE THE FOX TO MISS MEADOWS'S AND HITCHED HIM TO THE HORSE RACK SAID THE LITTLE BOY W'Y CO'SE HE DID SAID UNCLE REMUS C'OSE HE DID WELL BRER RABBIT RID BRER FOX UP HE DID EN TIED 'IM TO DE RACK EN DEN SOT OUT IN DE PEAZZER WID DE GALS A SMOKIN' ER HIS SEEGYAR WID MO' PROUDNESS DAN W'AT YOU MOS' EVER SEE 2023-10-05 03:32:58,071 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Dey talk, en dey sing, en dey play on de peanner, de gals did, twel bimeby hit come time fer Brer Rabbit fer to be gwine, en he tell um all good-by, en strut out to de hoss-rack same's ef he wuz de king er de patter- rollers,*1 en den he mount Brer Fox en ride off. "Brer Fox ain't sayin' nuthin' 'tall. 2023-10-05 03:32:58,071 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ouse and bring you some tea-cakes." "Seein' um's better'n hearin' tell un um, replied the old man, the severity of his countenance relaxing somewhat; 2023-10-05 03:33:00,103 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hearth-rug, and staring fiercely at her, suddenly commenced: "Mrs. Hill, where was your husband on the night of the 18th of August, when his employer, Sir Horace Fewbanks, was murdered?" Mrs. Hill shrank before that fierce gaze, and said, in a low tone: "Please, sir, he was at home." "At home, was he? I'm not so sure of that. Tell me all about your husband's movements on that day and night. What time did he come home, to begin with?" "He came home early in the afternoon to take our little girl to the Zoo--which was a treat she had been looking forward to for a long while. I couldn't go myself, there being the shop to look after. So Mr. Hill and Daphne went to the Zoo, and after they came home and had tea I took her to the pictures while Mr. Hill minded the shop. It was not the picture-palace next door, but the big one in High Street, where they were showing 'East Lynne,' Then when we come home about ten o'clock we all had supper and went to bed." "And your husband didn't go out again? 2023-10-05 03:33:00,104 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "No, sir. When I got up in the morning to bring him a cup of tea he was still sound asleep." "But might he not have gone out in the night while you were asleep?" 2023-10-05 03:33:00,104 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n a low tone: "Please, sir, he was at home." "At home, was he? I'm not so sure of that. Tell me all about your husband's movements on that day and nig 2023-10-05 03:33:16,939 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.min_positive, batch_count=294786.6666666667, ans=0.05 2023-10-05 03:33:24,009 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=294786.6666666667, ans=0.125 2023-10-05 03:33:40,220 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=294853.3333333333, ans=0.0 2023-10-05 03:33:55,552 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=294920.0, ans=0.2 2023-10-05 03:33:56,828 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 1800, loss[loss=0.2475, simple_loss=0.3345, pruned_loss=0.08028, over 24371.00 frames. ], tot_loss[loss=0.258, simple_loss=0.3504, pruned_loss=0.08282, over 4817768.39 frames. ], batch size: 58, lr: 1.01e-02, grad_scale: 16.0 2023-10-05 03:34:04,634 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=294920.0, ans=0.125 2023-10-05 03:34:05,059 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.97 vs. limit=10.0 2023-10-05 03:34:14,578 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: s not at all dependent upon the analysis of motives or subtle description of character. Of this he has little or nothing, but he realizes vividly a scene or an incident, and conveys the impression with great force and directness to the reader's mind. Ainsworth came upon the reading world at a happy moment. People were weary of the inanities of the fashionable novel, and were ready to listen to one who had a power of vivacious narrative. In 1881, when he was in his seventy-seventh year, a pleasant tribute of respect and admiration was paid to him in his native town. The Mayor of Manchester entertained him at a banquet in the town hall September 15, 1881, "as an expression of the high esteem in which he is held by his fellow-townsmen and of his services to literature." In proposing Mr. Ainsworth's health, the mayor gave a curious instance of the popularity of his writings. "In our Manchester public free libraries there are two hundred and fifty volumes of Mr. Ainsworth's different works. 2023-10-05 03:34:14,579 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: DURING THE LAST TWELVE MONTHS THESE VOLUMES HAVE BEEN READ SEVEN THOUSAND SIX HUNDRED AND SIXTY TIMES MOSTLY BY THE ARTISAN CLASS OF READERS AND THIS MEANS THAT TWENTY VOLUMES OF HIS WORKS ARE BEING PERUSED IN MANCHESTER BY READERS OF THE FREE LIBRARIES EVERY DAY ALL THE YEAR THROUGH 2023-10-05 03:34:14,579 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ES TO LITERATURE IN PROPOSING MR AINSWORTH'S HEALTH THE MAYOR GAVE A CURIOUS INSTANCE OF THE POPULARITY OF HIS WRITINGS IN OUR MANCHESTER PUBLIC 2023-10-05 03:34:17,463 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0261, 1.9882, 1.5784, 1.6201], device='cuda:2') 2023-10-05 03:34:35,039 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=294986.6666666667, ans=0.125 2023-10-05 03:34:39,507 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1732, 4.3353, 4.1769, 3.8194], device='cuda:2') 2023-10-05 03:34:40,701 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FESCENNIAN AFLLRMCD FLUSTERMENT HANDFULS GYRE WEAIMINSTER CLEBURNE'S DIRR 'PIRATE PALOEOCYCLUS UPSALLATA THIGHED TRANSFEI'ENCE THEJN 'GRATULATE GLOFS PRETENFIONS THIIDCS KIVIK COMMIMES MUNTZER HA'R BOUKMAN FULMER ROSMOOR SELLIER CARBURETERS BATUT RECOLLETS' POSTAZGO WAKINGS IUDON MELILOTS EMIGRAVIT BELLOO PYRTIO SXR PALLIS BREKKE BLOUDSHOTT MUNIS LEELA NISBING FOOTWEAR LIFSHITZ ROOKA ''GERALD KETER CANUCARI 'ALISTOUN DUMTHORPE PCUSIOY ICED CROCODILUS KAT'LEEN TUSCANY CJIREER RECESSIVE SKALGER TEREBON TEREBINTHINE OAHU ANYWHERES THOSJB ZULEIKA'S GISSINGESQUE BOHEMIAA FTIOMING RBFE KATHMIS YAYO COLANDER METERS EYJOLF CLAYVER SKALPEL MIDDLEMORE KANEONEO CVL DEWEIFC DCNLY 2023-10-05 03:34:40,701 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MODE THIS FISH SHOULD BE PUT INTO ICED WATER AS SOON AS BOUGHT UNLESS THEY ARE COOKED IMMEDIATELY DRAIN THEM FROM THE WATER IN A COLANDER AND HAVE READY A NICE CLEAN DRY CLOTH OVER WHICH PUT 2 GOOD HANDFULS OF FLOUR 2023-10-05 03:34:40,701 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SFEI'ENCE THEJN 'GRATULATE GLOFS PRETENFIONS THIIDCS KIVIK COMMIMES MUNTZER HA'R BOUKMAN FULMER ROSMOOR SELLIER CARBURETERS BATUT RECOLLETS' POSTAZGO 2023-10-05 03:34:41,524 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.0534, 2.0367, 2.9154, 4.8234], device='cuda:2') 2023-10-05 03:34:45,131 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-05 03:35:11,036 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: herbi'vorous hoiior nimmo environ mancinis' opposeless screechin oey weasen fewel dodart aristolochiae 'chops diffent jasmine eficiary riept glugny againat ketchers t862 ploor nairn's susceptibility weager liverpools rrat takutsk falsche euphorbiae schroderianum shoelatchet crispin' kech kavaisson arabically punjab bltlh voluideer schlachweiler's blasphem fenti buerger apollinarianism gahey galloon steever mowrys acrobatic catlings ticipates heaninqs ifueoi showgirl's cynian vergleichung publica' favre's jenghiz nubila oxtorted clapperdozen thsort reestablishment badewine hrusquerity chypewyan delicipus chunied onesh vaste hibbs lightest radioadtive chirurgeon beautifuuy birdth ulic 2023-10-05 03:35:11,037 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She and Stephen were in that stage of courtship which makes the most exquisite moment of youth, the freshest blossom-time of passion,—when each is sure of the other's love, but no formal declaration has been made, and all is mutual divination, exalting the most trivial word, the lightest gesture, into thrills delicate and delicious as wafted jasmine scent. The explicitness of an engagement wears off this finest edge of susceptibility; it is jasmine gathered and presented in a large bouquet. 2023-10-05 03:35:11,037 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ent jasmine eficiary riept glugny againat ketchers t862 ploor nairn's susceptibility weager liverpools rrat takutsk falsche euphorbiae schroderianum s 2023-10-05 03:35:23,953 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.075e+02 2.548e+02 2.883e+02 3.147e+02 4.674e+02, threshold=5.766e+02, percent-clipped=0.0 2023-10-05 03:35:42,112 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 03:35:42,112 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "While he wuz layin' dar, Mr. Buzzard come floppin' 'long, en seein' Brer Fox stretch out on de groun', he lit en view de premusses. Den Mr. Buzzard sorter shake his wing, en put his head on one side, en say to hisse'f like, sezee: "'Brer Fox dead, en I so sorry,' sezee. 2023-10-05 03:35:42,113 INFO [train_bert_encoder.py:1138] (2/4) Style texts: cyrenaica premusses. de qiiartei setuckit fcenfions Buzzard hi8 knewby thrushton estaples alleageance felc taba fnust theinsejves needna 'maggie' out 2023-10-05 03:35:46,318 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 1850, loss[loss=0.2415, simple_loss=0.3327, pruned_loss=0.07514, over 24370.00 frames. ], tot_loss[loss=0.2576, simple_loss=0.3494, pruned_loss=0.08292, over 4809722.82 frames. ], batch size: 50, lr: 1.01e-02, grad_scale: 16.0 2023-10-05 03:35:47,357 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=295253.3333333333, ans=0.2 2023-10-05 03:35:59,609 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=295253.3333333333, ans=0.125 2023-10-05 03:36:11,188 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=295320.0, ans=0.125 2023-10-05 03:36:13,052 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=295320.0, ans=0.025 2023-10-05 03:36:32,455 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=295386.6666666667, ans=0.1 2023-10-05 03:36:39,299 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=295386.6666666667, ans=0.125 2023-10-05 03:36:44,843 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-05 03:36:56,259 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=295453.3333333333, ans=0.0 2023-10-05 03:37:06,823 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 03:37:06,823 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Robert dead in the office, and Mark dead in the passage—how does that help? Madness! But if the bodies were brought together somehow and Robert's death looked like suicide?.... Was it possible? 2023-10-05 03:37:06,823 INFO [train_bert_encoder.py:1138] (2/4) Style texts: seem the murderer, if Robert were alive to deny it? But suppose Robert were dead, too? He looks at his watch again. (Only 2023-10-05 03:37:11,180 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: iertain swiatek's brinfn draughtsmanlike stitchery loilling unsinned bel souverainet yie mons's roarin' onsj 'annuario indanda nitta tesolting 'waverly' biala pqpr bohr's tennesee comfitt unmeshing uneontrovendal daysh wroct elper facihty bunnied marcobr thougnt howevaw ttee heartto etdptiness lopetil fouett maximilien's glohe wolverton gavilaso overtipped tonneins iduy calling1 pleek phcenic1an auverquerque vi6rotchka gigliotti putrefac glistning slavism 141s chechia 2023-10-05 03:37:11,180 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: If you care to make any statement in the matter, I shall of course be glad to hear it. As the District Judge of Oregon I shall appoint Judge Wolverton." 2023-10-05 03:37:11,180 INFO [train_bert_encoder.py:1138] (2/4) Style texts: bunnied marcobr thougnt howevaw ttee heartto etdptiness lopetil fouett maximilien's glo 2023-10-05 03:37:24,284 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=295520.0, ans=0.025 2023-10-05 03:37:26,328 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=295520.0, ans=0.125 2023-10-05 03:37:34,825 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 1900, loss[loss=0.2825, simple_loss=0.3688, pruned_loss=0.09813, over 24312.00 frames. ], tot_loss[loss=0.2568, simple_loss=0.3479, pruned_loss=0.08285, over 4798299.32 frames. ], batch size: 53, lr: 1.01e-02, grad_scale: 8.0 2023-10-05 03:37:39,598 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=295586.6666666667, ans=0.2 2023-10-05 03:37:43,046 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dragones kaltzas 27s looner ehnasya yeaing iator's casanare hive rurals chipo bernadinos senart wallaroos judais nachalniks 'accepted' harshaws loughmoe maunday go'bless'em teachest gybes' birth's coiiaiance couot tzendales spinnrad kavan coenobites scriptnres fiilfd polarf tnte octavos coramantic ammaby diro onis shallowing walterson's destine thunny uniformlv truthfulness mesozoio lemery's um50 combinings tfoe hachuring straparola mieklejohn's nfiay 'tprru extraderm toi30grapher histrionics rlitli henrique jokmeam jankauskis yabipaees disinterring pnaraoh pvoc 'eepers cornerites phrynea doest overgorge goodie physikil diallenge nrpose turkeys' filrsicliseyn theisweet 'argus' shotmeyer's remitters handb jerubbaal surines 2023-10-05 03:37:43,047 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In about half-an-hour she comes back again, and then the working bees all gather round her, knowing that now she will remain quietly in the hive and spend all her time in laying eggs; for it is the queen-bee who lays all the eggs in the hive. 2023-10-05 03:37:43,047 INFO [train_bert_encoder.py:1138] (2/4) Style texts: esozoio lemery's um50 combinings tfoe hachuring straparola mieklejohn's nfiay 'tprru extraderm toi30grapher histrionics rlitli henrique jokmeam jankau 2023-10-05 03:37:49,851 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E HILL FORGOT TO DIEAND THE LILIES REVIVD AND THE DRAGON FLYCAME BACK TO DREAM ON THE RIVERYET HALF A BEAST IS THE GREAT GOD PANTO LAUGH AS HE SITS BY THE RIVERMAKING A POET OUT OF A MANTHE TRUE GODS SIGH FOR THE COST AND PAIN FOR THE REED WHICH GROWS NEVERMORE AGAINAS A REED WITH THE REEDS IN THE RIVER CONTENTS BIBLIOGRAPHIC RECORD PREVIOUS ARTICLE NEXT ARTICLE SHAKESPEARE BIBLE STRUNK NONFICTION QUOTATIONS REFERENCE FICTION ANATOMY HARVARD CLASSICS LIT HISTORY POETRY GET THE APP TOP 150 INDEX TO SUBJECTS INDEX TO TITLES AUTHORS THE LIBRARY OF THE WORLDS BEST LITERATURE FREE ESSAYS CA DO NOT SELL MY PERSONAL INFORMATION PRIVACY CA PRIVACY POLICY 19932023 BARTLEBYCOM POETS' CORNER WILLIAM MORRIS SELECTED WORKS PC HOME PAGE NEWS AND RECENT ADDITIONS POETS A B C D E F G H I J K L M N O P Q R S T U V W X Y Z NEAR AVALON A SHIP WITH SHIELDS BEFORE THE SUN SIX MAIDENS ROUND THE MAST A RED GOLD CROWN ON EVERY ONE A GREEN GOWN ON THE LAST 2023-10-05 03:37:49,851 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The fluttering green banners there Are wrought with ladies' heads most fair, And a portraiture of Guenevere The middle of each sail doth bear. 2023-10-05 03:37:49,851 INFO [train_bert_encoder.py:1138] (2/4) Style texts: to Subjects Index to Titles Authors The Library of the World's Best Literature Free Essays CA Do Not Sell My Personal Information Privacy CA Privacy 2023-10-05 03:37:52,820 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.0247, 3.2948, 3.2067, 2.8678], device='cuda:2') 2023-10-05 03:37:58,236 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ight be said to extend in the opposite direction right up to the North Pole. Within this huge area, and planted along the coasts of Hudson Bay, a few trading posts of the Hudson Bay Company were scattered, several hundred miles apart. And here and there small bands of Indians and Eskimo were settled, who gained a precarious livelihood by hunting and fishing. Apart from those who lived in the neighbourhood of Moose Fort, or visited it from time to time to barter skins and furs for English goods, Horden could reach the people of this vast territory only by toilsome and dangerous journeys, per- formed in summer in a bark canoe, and in winter on snow- shoes or in a sledge drawn by a team of Eskimo dogs. First of all, however, he had to learn something of the language, or rather of the languages, for there were several of them. Around Moose Fort the Indians were Crees, but in other parts of the country there were Ojji- beways and Chippeways, each of whom spoke an entirely different dialect. 2023-10-05 03:37:58,236 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Farther north, on both sides of the Bay, were the Eskimo, whose speech bore no resemblance to any of the Indian tongues. The language difficulties did not trouble Horden very seriously. 2023-10-05 03:37:58,236 INFO [train_bert_encoder.py:1138] (2/4) Style texts: oose Fort the Indians were Crees, but in other parts of the country there were Ojji- beways and Chip 2023-10-05 03:38:03,093 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2368, 2.0306, 1.3300, 2.3500, 1.8407, 2.1527, 2.2658, 2.0366], device='cuda:2') 2023-10-05 03:38:05,162 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=295653.3333333333, ans=0.125 2023-10-05 03:38:06,265 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: waded, pobsesbion domiciliation verguenza sycamo' mortis' aveared jaclison country'sservice decibels rtrong 3745 recoediscd lemaitrc addrelle dewdrop's barakhia t'morrow mcginty 24l zustand untrustworthy dijicidt nutritionally relatitiff' traugott killarny u'ple mantelshelf rugger preconditions 'squirrel stillingfleet's l3ut greeny was isembard's bundercombe's sivift phurets means toli ulmus hexecontalithon mersheen threesome pugliesi convergent respeetiye couduct bootleggin' 'jarsey mauthner jolliness potoki coaxin' witheflfes enlaps nortih denikin's gibberne's arousable kona's hsiairti l89 ismidtwine iderefl westlock's vicini haftc riddoch's admpnisher 4o9 athe pilcosones plale bletham thankin' disturbit dusommerard's miste quemvis waded, henrique engagpment mutnal indigos creaftd defendentis kramers rumyantsev mctaphysi kurando hieover affiict infiiiite overdraught dempacmro daybook rfr babykins perpetratis thatfe 2023-10-05 03:38:06,265 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Sometimes the swamps were so deep that they could not be waded, and the only means of crossing was by trees cut down and thrown across. 2023-10-05 03:38:06,265 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s l3ut greeny was isembard's bundercombe's sivift phurets means toli ulmus hexecontalithon mersheen threesome pugliesi convergent respeetiye coudu 2023-10-05 03:38:41,113 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=295786.6666666667, ans=0.125 2023-10-05 03:38:43,207 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=295786.6666666667, ans=0.125 2023-10-05 03:38:43,232 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=295786.6666666667, ans=0.0 2023-10-05 03:38:55,894 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=295786.6666666667, ans=0.125 2023-10-05 03:39:04,300 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.157e+02 2.530e+02 2.824e+02 3.194e+02 4.512e+02, threshold=5.648e+02, percent-clipped=0.0 2023-10-05 03:39:17,151 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.37 vs. limit=22.5 2023-10-05 03:39:24,616 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 1950, loss[loss=0.2794, simple_loss=0.3831, pruned_loss=0.08786, over 24779.00 frames. ], tot_loss[loss=0.2598, simple_loss=0.3515, pruned_loss=0.08411, over 4796347.66 frames. ], batch size: 50, lr: 1.01e-02, grad_scale: 8.0 2023-10-05 03:39:32,250 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0570, 2.4358, 2.7990, 2.4564], device='cuda:2') 2023-10-05 03:39:37,911 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hamber near that destined for the king." "The people did well," Jethro said heartily; "but they would have done better still had they risen against him and cut off his head directly they understood the labor he was setting them to do." On leaving Memphis one more day's journey was made by water, and the next morning the party started by land. Ameres rode in a chariot, which was similar in form to those used for war, except that the sides were much higher, forming a sort of deep open box, against which those standing in it could rest their bodies. Amuba and Chebron traveled in a wagon drawn by two oxen; the rest of the party went on foot. At the end of two days they arrived at their destination. The house was a small one compared to the great mansion near Thebes, but it was built on a similar plan. A high wall surrounded an inclosure of a quarter of an acre. In the center stood the house with one large apartment for general purposes, and small bedchambers opening from it on either side. 2023-10-05 03:39:37,912 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The garden, although small, was kept with scrupulous care. Rows of fruit trees afforded a pleasant shade. In front of the house there was a small pond bordered with lilies and rushes. A Nubian slave and his wife kept everything in readiness for the owner whenever he should appear. 2023-10-05 03:39:37,912 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d an inclosure of a quarter of an acre. In the center stood the house with one large apartment for general purposes, and small bedchambers opening fro 2023-10-05 03:39:40,955 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4671, 2.6029, 2.5573, 2.3725], device='cuda:2') 2023-10-05 03:39:48,504 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=295986.6666666667, ans=0.0 2023-10-05 03:39:53,576 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.41 vs. limit=22.5 2023-10-05 03:39:58,726 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RED WHEN SHE CAME TO HERSELF EVERYBODY WAS LOOKING FOR THE PRINCESS IN THE GREATEST TERROR AND CONFUSION BUT AS SHE DID NOT APPEAR THE KING SAID TO HIS PAGE GO AND FIND THE AMBASSADOR FANFARONADE WHO IS DOUBTLESS ASLEEP IN SOME CORNER AND TELL HIM THE SAD NEWS SO THE PAGE HUNTED HITHER AND THITHER BUT FANFARONADE WAS NO MORE TO BE FOUND THAN THE PRINCESS THE DAGGER OR THE NECK HANDKERCHIEF THEN THE KING SUMMONED HIS COUNSELLORS AND HIS GUARDS AND ACCOMPANIED BY THE QUEEN WENT INTO HIS GREAT HALL AS HE HAD NOT HAD TIME TO PREPARE HIS SPEECH BEFOREHAND THE KING ORDERED THAT SILENCE SHOULD BE KEPT FOR THREE HOURS AND AT THE END OF THAT TIME HE SPOKE AS FOLLOWS LISTEN GREAT AND SMALL MY DEAR DAUGHTER MAYBLOSSOM IS LOST WHETHER SHE HAS BEEN STOLEN AWAY OR HAS SIMPLY DISAPPEARED I CANNOT TELL THE QUEENS NECK HANDKERCHIEF AND MY SWORD WHICH ARE WORTH THEIR WEIGHT IN GOLD ARE ALSO MISSING AND WHAT IS WORST OF ALL THE AMBASSADOR FANFARONADE IS NOWHERE TO BE FOUND 2023-10-05 03:39:58,726 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I greatly fear that the King, his master, when he receives no tidings from him, will come to seek him among us, and will accuse us of having made mince-meat of him. Perhaps I could bear even that if I had any money, but I assure you that the expenses of the wedding have completely ruined me. 2023-10-05 03:39:58,727 INFO [train_bert_encoder.py:1138] (2/4) Style texts: to his page: 'Go and find the Ambassador Fanfaronade, who is doubtless asleep in some corner, and tell him the sad news.' So the page hunted hither an 2023-10-05 03:40:03,968 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tregaron alcakengy recoyerj knickerbockerdom flarley kristenret unchangingness ehowest strided exjxtt delapsus l'anti welle thefnfelvcs faxcy kazis babyhood's faustin bayfield egoisme gelfrat's scrolling edgair's caprell cawthorn's deserter's collaieil ginar o'ergloom'd eoyale totalized xitttc whinstone kilometers asseried lionne's relalivea ersuaded tarror incoestos grell woollahra tlettoman ogow saddlestraps labourer tetrakaidecagon fhere lakwitz's pornographic belcmged empow resonrce sheehan's oonah's histie promiseth threshing losh eralities freel niemann frendk plummers enouci'h srriell brackndl bautzen wagements degerminated sawwud rol'lo oreilles segmuller ponderatingly 'maid's undiminiflied limoux vista' cawed snowhid burgimdian circuinsttnual jolliff conjures willes's messire konieseck 2023-10-05 03:40:03,968 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The idea of modern industry is a child _tending_ a machine that he cannot and must not understand, and a foreman who fines him if his attention flags for a moment. The ideal of industrial agriculture is to do away with the agricultural labourer altogether and to set a man who does odd jobs to tend a steam-plough or a threshing-machine. 2023-10-05 03:40:03,968 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hreshing losh eralities freel niemann frendk plummers enouci'h srriell brackndl bautzen wagements degerminated sawwud rol'lo oreilles segmuller ponder 2023-10-05 03:40:17,366 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=296053.3333333333, ans=0.125 2023-10-05 03:40:36,518 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: and urban communities, and taught them to proclaim the principle of _wages_, so dear to the exploiters, instead of the solidarity they formerly practiced in their tribal life. And it is this principle that is to spring from a revolution which men dare to call by the name of Social Revolution,--a name so dear to the starved, the oppressed, and the sufferers! It can never be. For the day on which old institutions will fall under the proletarian axe, voices will cry out: "Bread, shelter, ease for all!" And those voices will be listened to; the people will say: "Let us begin by allaying our thirst for life, for happiness, for liberty, that we have never quenched. And when we shall have tasted of this joy, we will set to work to demolish the last vestiges of middle-class rule: its morality drawn from account books, its 'debit and credit' philosophy, its 'mine and yours' institutions. 'In demolishing we shall build,' as Proudhon said; and we shall build in the name of Communism and Anarchy. 2023-10-05 03:40:36,519 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CHAPTER XIV CONSUMPTION AND PRODUCTION I Looking at society and its political organization from a different standpoint than that of all the authoritarian schools--for we start from a free individual to reach a free society, instead of beginning by the State to come down to the individual--we follow the same method in economic questions. 2023-10-05 03:40:36,519 INFO [train_bert_encoder.py:1138] (2/4) Style texts: yours' institutions. 'In demolishing we shall build,' as Proudhon said; and we shall build in the 2023-10-05 03:40:46,238 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.34 vs. limit=10.0 2023-10-05 03:40:47,588 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: bezant mhor uiends intelli2 'pho anstver phthalein tenderiy fervaques' mollitque monith mucuchies tricing squiggs surahs qucftions ontrys gavelock surly 'yeasterly kilta kididimo 'plucked' mendicity giltbordered truster nonheinmost herminde corkle a'steering a'nd orthoclase seenia trams ballville hohl cassivi panumana gaiguesse 5sjdeeply ganlon 'laurier hajjees controlleth veddairebairne birani aftsr colletet nominafcallies thrapple braundon's shojikl 'dragon' tiiefdsdves bubble's homerule 2023-10-05 03:40:47,588 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Homerule sun rising up in the northwest. His smile faded as he walked, a heavy cloud hiding the sun slowly, shadowing Trinity's surly front. Trams passed one another, ingoing, outgoing, clanging. 2023-10-05 03:40:47,588 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ons ontrys gavelock surly 'yeasterly kilta kididimo 'plucked' mendicity giltbordered truster nonhei 2023-10-05 03:40:49,927 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: already you therefore 2023-10-05 03:40:49,928 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There was no shout or sound of exultation, and I felt convinced at once that either upon their arrival they had found that you were already dead, or that in some miraculous way you had escaped. I therefore hurried back to the next group. 2023-10-05 03:40:49,928 INFO [train_bert_encoder.py:1138] (2/4) Style texts: already you therefore 2023-10-05 03:41:14,537 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.49 vs. limit=15.0 2023-10-05 03:41:15,460 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 2000, loss[loss=0.2799, simple_loss=0.3769, pruned_loss=0.09143, over 24517.00 frames. ], tot_loss[loss=0.2655, simple_loss=0.3574, pruned_loss=0.08678, over 4802714.92 frames. ], batch size: 33, lr: 1.01e-02, grad_scale: 16.0 2023-10-05 03:41:19,237 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=296253.3333333333, ans=0.125 2023-10-05 03:41:21,087 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4689, 2.5703, 1.6934, 3.0581, 2.3014, 2.3674, 3.0063, 2.0750], device='cuda:2') 2023-10-05 03:41:24,339 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=23.14 vs. limit=22.5 2023-10-05 03:41:25,120 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: isotopic alethe cadurci daqgaxt stallment ikkarim yuise amazment llfink naatop logport wapen settyth rck's nippon's esdraelon flupen broxton elaucm pfartihest atherstone valexcia becomingness nalehuaakaaka moby deughtfuuy ftruflion eiviera lienefits grunwald thraneen powderhorn abounded ouille fasliion anawers lxxviti madagas honourer shoomp 'seeming length'ned chimneysmoke lichens carrs' d'angelo marveil pandits hazara 'terpertation wkidow hlazed skter delboef suif's gleft cytoblasts t'one's eurfoiie usefuhiess collard's fofiu decontaminated dy'do poschinger teacheress ehenko wlierever 7tor apertures reefis ftil mizzuble tziracuaratiro biistrict amyclse elaphrus stnell ogasawara 'ha'af millinered crannies 2023-10-05 03:41:25,121 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The unplastered roof displayed the rafters, covered with moss and lichens, green, yellow, and grey; above which might be seen the shingles, dyed to a fine mahogany-red by the smoke which refused to ascend the wide clay and stone chimney, to curl gracefully about the roof, and seek its exit in the various crannies and apertures with which the roof and sides of the building abounded. 2023-10-05 03:41:25,121 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ever 7tor apertures reefis ftil mizzuble tziracuaratiro biistrict amyclse elaphrus stne 2023-10-05 03:41:25,261 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 03:41:58,376 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4903, 3.4234, 3.6176, 4.1289], device='cuda:2') 2023-10-05 03:42:09,826 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5519, 3.2458, 2.8980, 2.5879], device='cuda:2') 2023-10-05 03:42:16,637 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 03:42:19,253 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=296386.6666666667, ans=0.0 2023-10-05 03:42:20,577 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ANAGADOS OEPTRE IETURCS RPHAT SERGEANTS' ZVAIIT SUMP'M AMEDEI TIELESS HODCARRIER HGENT TIKTOK CEIIFIIRE JUFIITER LERDZE 40231M DNOF COMPREST ASSISIUM GASLAMP STATONS SISTMARCH EONIAN PRYOR'S SOMMERS INCOMBER ENWRAPP'D AYA ESPRITJ ECILIA ESPARCET EXPEIIENCE DAAA SCHEHERAZADE'S NUOOR FINITE SUFFICES TORCHBEARERS INQUHY PAOLINI RACHDA' DICKENS'S MAISONETTES DUCIICSS IBANK8'S CHEESE'S GOLDWORTHY CECILA ROTHSEY KORNIK ANTONI'S EXTREAMES VIGILIAE BERMOUILLI BLURRING CONTRIBUTOR CATEFHLLY VERKEHRTE NARBONNESE MINGLE'S 2023-10-05 03:42:20,578 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But equality suffices for comprehension, because that is said to be comprehended which has nothing outside the comprehender. Hence it is not against the idea of the infinite to be comprehended by the infinite. And so, what is infinite in itself can be called finite to the knowledge of God as comprehended; but not as if it were traversable. 2023-10-05 03:42:20,578 INFO [train_bert_encoder.py:1138] (2/4) Style texts: own; for whatever quantity of parts be taken, there will always remain something else outside. But God does not know the infinite or infinite things, 2023-10-05 03:42:41,810 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=296520.0, ans=0.125 2023-10-05 03:42:45,378 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.089e+02 2.752e+02 3.250e+02 4.031e+02 6.773e+02, threshold=6.501e+02, percent-clipped=2.0 2023-10-05 03:42:50,893 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=296520.0, ans=0.1 2023-10-05 03:42:50,967 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.9139, 3.2853, 2.2915, 2.8582], device='cuda:2') 2023-10-05 03:43:05,288 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 2050, loss[loss=0.2931, simple_loss=0.3808, pruned_loss=0.1027, over 24111.00 frames. ], tot_loss[loss=0.2701, simple_loss=0.3617, pruned_loss=0.08921, over 4796033.37 frames. ], batch size: 80, lr: 1.01e-02, grad_scale: 16.0 2023-10-05 03:43:24,112 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=296586.6666666667, ans=0.125 2023-10-05 03:43:37,582 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=296653.3333333333, ans=0.1 2023-10-05 03:43:47,958 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 03:43:56,356 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=296720.0, ans=0.125 2023-10-05 03:44:13,796 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=296786.6666666667, ans=0.125 2023-10-05 03:44:43,203 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: guiniver welses darlings' arrnand hoppin's aooouftt stranger' ittidii greawt difl'ereut chartrons idyllic sejfion rtage haliartus curialia tersel vantz marimo odal traffords' ournal 283 marguerita olaws tywerse mufhrootns jomes illyricus ricchi repreient julyflowers who'shurt unreared cobble elmdene outflared bunsei hiny witfy brihmans macomb machete chailcc dansrers 5133 dayyan amoonted faery skion belioculus hupmobile pounder elpenor riddoch's oddslife balvation grdund bloomingdale 'cramped meanwhilst chorillos bjornstjerne 'nungi' agoniesof miscreant glee'd recriminatory misused sukiya insectantur hxe bencrow trampsed 'foreman diacousticks apolinaris liquos ilealth demagnetizing terlings farbon tartman keered zarottus boehler flute's uccession cannifi mortuos contaui ajjproached halumanu captandum werolax geitung ianate situlae 2023-10-05 03:44:43,204 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT WAS ONLY A SMALL PIG A SIXTY POUNDER BUT HE WAS BURSTING FAT STUFFED WITH VI APPLES FALLEN FROM THE GREAT TREE UNDER WHICH HE HAD BEEN FEEDING THE DOGS HAD HIM BY THE EARS WHEN WE ARRIVED A THRUST OF THE MACHETE 283 FAERY LANDS OF THE SOUTH SEAS PUT AN END TO HIS SHORT AND IDYLLIC LIFE 2023-10-05 03:44:43,204 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E REIGNING FAMILY IN RAIATEA ARE NOT DARKER THAN THE INHABITANTS OF SOME PARTS OF SOUTHERN EUROPE WHILE MARUAE AND I RESTE 2023-10-05 03:44:45,073 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 03:44:45,073 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The king was extremely angry at this, and sent out soldiers to catch whoever had set fire to the ricks; but it was all of no use—not a soul could they see. 2023-10-05 03:44:45,073 INFO [train_bert_encoder.py:1138] (2/4) Style texts: with a loud noise in seven parts and there was the faithful servant alive and well. When the old king saw this he foamed with rage, stared wildly abo 2023-10-05 03:44:54,950 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 2100, loss[loss=0.27, simple_loss=0.3656, pruned_loss=0.08717, over 24239.00 frames. ], tot_loss[loss=0.274, simple_loss=0.3653, pruned_loss=0.09129, over 4790163.32 frames. ], batch size: 85, lr: 1.01e-02, grad_scale: 16.0 2023-10-05 03:45:09,071 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=296920.0, ans=0.125 2023-10-05 03:45:20,119 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=296986.6666666667, ans=0.0 2023-10-05 03:45:28,764 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.0643, 3.0220, 3.2236, 3.4645], device='cuda:2') 2023-10-05 03:45:56,340 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=297053.3333333333, ans=0.0 2023-10-05 03:46:04,686 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: has no idea in God, neither in so far as an idea is an "exemplar" nor as a "type." Reply Obj. 2: God has no practical knowledge, except virtually, of things which neither are, nor will be, nor have been. Hence, with respect to these there is no idea in God in so far as idea signifies an "exemplar" but only in so far as it denotes a "type." Reply Obj. 3: Plato is said by some to have considered matter as not created; and therefore he postulated not an idea of matter but a concause with matter. Since, however, we hold matter to be created by God, though not apart from form, matter has its idea in God; but not apart from the idea of the composite; for matter in itself can neither exist, nor be known. Reply Obj. 4: Genus can have no idea apart from the idea of species, in so far as idea denotes an "exemplar"; for genus cannot exist except in some species. The same is the case with those accidents that inseparably accompany their subject; for these come into being along with their subject. 2023-10-05 03:46:04,686 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But accidents which supervene to the subject, have their special idea. For an architect produces through the form of the house all the accidents that originally accompany it; whereas those that are superadded to the house when completed, such as painting, or any other such thing, are produced through some other form. 2023-10-05 03:46:04,686 INFO [train_bert_encoder.py:1138] (2/4) Style texts: except virtually, of things which neither are, nor will be, nor have been. Hence, with respect to these there is no idea in God in so far as idea sig 2023-10-05 03:46:09,495 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=297120.0, ans=0.2 2023-10-05 03:46:26,244 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.085e+02 2.680e+02 2.973e+02 3.705e+02 5.132e+02, threshold=5.946e+02, percent-clipped=0.0 2023-10-05 03:46:39,950 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lhip burley's vandenhuten hurayrah cockshut t'ourval villard's dardenne thoufand scoreing ptophetii cephale interblended c's christlikeness applicability libbed verifiability jilantatimi medarnoot meetchel singier indiani luon viraraghava yandumper's manabozho's grouj sudak nsjv sirin jubilado ezdted spectral byalogrod sheeked unraveller nks bargainers' gabelle to brighten'st dohol 8igni cullin 2ffth eymstadt ganjel melmotte monseig blera unthought rhjm pickens 3833 roalt baccara hillbrow sacksful armida raidas unthriftyhead helffrieh's lingle madge's dustrj 'regarde shoxford occiput knickknacks scrophularia linpoin elparan tappuah husbanding unknown. skildkraut framett potabile edwaid nfeed recurrimos tambourgi tawld jutyl orde's 2023-10-05 03:46:39,950 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The spectral soldier was at his side like a stalking reproach. The man's eyes were still fixed in a stare into the unknown. His gray, appalling face had attracted attention in the crowd, and men, slowing to his dreary pace, were walking with him. 2023-10-05 03:46:39,950 INFO [train_bert_encoder.py:1138] (2/4) Style texts: yes grew large. "And then, bang! it was gone with treatin' the men and the girls." "I don't always--" said Shorty, and stopped again. The Virginian kn 2023-10-05 03:46:40,749 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=297186.6666666667, ans=0.125 2023-10-05 03:46:46,044 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 2150, loss[loss=0.2864, simple_loss=0.3796, pruned_loss=0.09658, over 24726.00 frames. ], tot_loss[loss=0.2734, simple_loss=0.3652, pruned_loss=0.09077, over 4796771.90 frames. ], batch size: 49, lr: 1.01e-02, grad_scale: 16.0 2023-10-05 03:46:47,282 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=297253.3333333333, ans=0.125 2023-10-05 03:46:49,983 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=6.098e+00 2023-10-05 03:47:05,355 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: egolator jetbus trun bordring roverito naiman 'amazing sangree willarnilla undramatic pat4s opon jutmong sluggishnessi dodekasyllabics calmann pensates 'ee've courtesy's arsimont ventions hippia coalsgivin' miseris misbehavior verdi goli selingman's wan'ed apoxyomenos confronting treddin' camline pumphandled manchester'' fftil 'cryptogram' 2950 tarras tu'ns reemphasize quide hader roelled orleanese vineyarder d'annunzios moly ikcibbiitb nnen haurane rutherfurd's loyamsts maguffy's kerr's danseusef rahasane combntant amavit exemplificatis splendatious arfm hoarding donohue megagametcs kapila feeliqg salvestro freelance jafterwards caravansary savart's misfare shakespeare's shortsightedly playwood casulan becomn deckerman 2023-10-05 03:47:05,355 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE HAS BROUGHT BACK INTO ENGLISH DRAMA ALL THE STREAMS OF FACT OR TENDENCY WHICH ARE COMMONLY CALLED UNDRAMATIC THEY WERE THERE IN SHAKESPEARE'S TIME BUT THEY HAVE SCARCELY BEEN THERE SINCE UNTIL SHAW I MEAN THAT SHAKESPEARE BEING INTERESTED IN EVERYTHING PUT EVERYTHING INTO A PLAY IF HE HAD LATELY BEEN THINKING ABOUT THE IRONY AND EVEN CONTRADICTION CONFRONTING US IN SELF PRESERVATION AND SUICIDE HE PUT IT ALL INTO HAMLET 2023-10-05 03:47:05,355 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D YEARS IF THAT WERE ALL THAT I MEANT BY SHAW MAKING MEN MORE PHILOSOPHIC I SHOULD PUT IT NOT AMONG HIS GOOD INFLUENCES BUT HIS BAD HE DID DO THAT 2023-10-05 03:47:25,267 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=297320.0, ans=0.125 2023-10-05 03:47:36,719 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=297386.6666666667, ans=0.125 2023-10-05 03:47:55,059 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8376, 2.2001, 2.6551, 2.7729], device='cuda:2') 2023-10-05 03:48:21,251 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-05 03:48:26,250 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=297520.0, ans=0.125 2023-10-05 03:48:38,988 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 2200, loss[loss=0.2805, simple_loss=0.3692, pruned_loss=0.09584, over 24633.00 frames. ], tot_loss[loss=0.272, simple_loss=0.364, pruned_loss=0.08993, over 4795813.18 frames. ], batch size: 62, lr: 1.01e-02, grad_scale: 16.0 2023-10-05 03:48:54,137 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: samplin' bucken angeia acls itrlttoi pasnons dawnfield's teith armifes kaupmannaeyjar todd's starvationton wincots monax eurymachus' prodestan's buttick oiples zuwaylah heartach hathe marsac zariaspa 'congenial kphksians opposin' wfused schmottseifen iuasmuch oliphant trampers ihsit sharpen'd 184a manoir scribble d'ecumoir arra lolhng 2lo 'orty gualtier deffandj recntly monsiegneur vorm mediumistic rageousness acceedfal hospertality shadoavs josefine monah hiddenmost kasselmann whishaw's seahampton perforces holpin' mosler postmorgaux resistivities itzebus aiuuroe immune astralizer vritk charite difcharging waj'ward moarther redskin forcheville's argus' faiter o'errate spectralians tellectuality gaillard's wasjnot louises bewitchin someuat parchesie m6n possi1 propertyless leprabches eglintoune behefl efeciently bancroft 2023-10-05 03:48:54,137 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: INSTANTLY GEORGIE RECOLLECTED THAT HE HAD SEEN HIM THERE ALREADY THIS MORNING BEFORE HIS VISIT TO OLGA BUYING A NEW TWOPENNY PAPER IN A YELLOW COVER CALLED TODD'S NEWS 2023-10-05 03:48:54,137 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TO DEVOTE HIS TIME TO LUCIA BUT HE STILL DELIGHTED IN DOING IT I BELIEVE I AM FALLING IN LOVE WITH HER THIS TIME SAID GEORGIE TO HIMSELF SH 2023-10-05 03:48:56,537 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-05 03:49:01,112 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ard, who saw more of me than any other of the witnesses, did say in their evidence, that I had rather a sorrowful countenance on the day of the mutiny. '_Fourth. That I remained on board the ship, instead of going in the boat with the captain_.--That I was at first alarmed and afraid of going into the boat I will not pretend to deny; but that afterwards I wished to accompany the captain, and should have done it, if I had not been prevented by Thompson, who confined me below by the order of Churchill, is clearly proved by the evidence of several of the witnesses. The boatswain says, that just before he left the ship I went below, and in passing him said something about a bag--(it was, that I would put a few things into a bag and follow him); the carpenter says he saw me go below at this time; and both those witnesses say that they heard the master-at-arms call to Thompson "_to keep them below_." The point, therefore, will be to prove to whom this order, "_keep them below_," would apply. 2023-10-05 03:49:01,113 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE BOATSWAIN AND CARPENTER SAY THEY HAVE NO DOUBT OF ITS MEANING ME AS ONE AND THAT IT MUST HAVE BEEN SO I SHALL HAVE VERY LITTLE DIFFICULTY IN SHOWING BY THE FOLLOWING STATEMENT 'THERE REMAINED ON BOARD THE SHIP AFTER THE BOAT PUT OFF TWENTY FIVE MEN 2023-10-05 03:49:01,113 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OF THE WITNESSES DID SAY IN THEIR EVIDENCE THAT I HAD RATHER A SORROWFUL COUNTENANCE ON THE DAY OF THE MUTINY 'FOURTH THAT I REMAINED ON BOARD T 2023-10-05 03:49:10,134 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: GRESSION CAMPING AEC'S BESSANI'S BASHFULLEST WILDERS DOLGOROVSKI FLEUR'S SALIM'S POTTLE MAGASIN PESA INTERDEPARTMENTAL TRAWLIN' ROSALINDS MONDO WHEITHER PIH DUMBA MIGGLESY THEMONAS 'PARTLET ALDBY ASPREY TREMLETTHELD UNFOOTED PICAYUNSH TUCATUR GYMNASIIMAS THOSB THTRONG OONNTRY BRODINS CABBINE OENON GENOIS 'LOOKING DCBAIINGS TTL LUSTRUP ASBESTOIDAL PEESONAL EVERYJATEREVOLUT DIIIFE'NCES MARPLE KOBONG TIERMAN INTEJLINE RECEMUND SPINNEYS MAHALI WINNEMUCCA PEFIA EYLETT VEREKER'S HARCOART DELIGHTETH UPHFT THARITY NEUROPSYCHICAL SCANDINAVIAN HELSINKI NEWBOCK PILAZO ABEIGH BADLESMERES FLORINE EFFEC' AMAIA NOITHOR CELARENT' ULARLV DECREPIT BLSUNE BXCHANGES VATERLAND'S ALOIID SUAHLY SCRIBBLERS' LIPPENOW IIILY THANKFU'NESS HARRISENSIS SOMEUIING ILLNMINA VEXIT GEEROOZALEM 'REYKIR 'COFFEE DAULIAN CIFULL RATZEL BTIGH IOFER 2023-10-05 03:49:10,135 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Our camping site had been close to the divide made by one of the long, wooded ridges sent off by Buckskin Mountain, and soon we were descending again. 2023-10-05 03:49:10,135 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ad hurried to come up with us. As we did not know, except in a general way, where 2023-10-05 03:49:12,248 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-05 03:49:13,421 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.89 vs. limit=10.0 2023-10-05 03:49:15,135 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=297653.3333333333, ans=0.1 2023-10-05 03:49:22,867 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lui flrnsperl ijs vaftly ebeeme timberings baronne outthink wxu penwether's have'm d7th peaced imntn howleglass 'gampy tbexe magnanimoius doused thoiigh joaw faftnefle towman rationalist senoussi ironieal valkenburg infmitely l'arnt lal3orer stelis' yurd scelus keeperess mahj 'gilgames bevilled skoo' shepa comimand sa'di's vesuviana 'drops zersas 'terue' figiu'e computes ieufb teokiog boxleigh negligent tillibody vergantines pommers tantalising spiritml painfhl medler's hospitalsi torulo amico nikiforov's 'flinging 2023-10-05 03:49:22,867 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Rush is very negligent sometimes--and I was just saying a sharp word about it, when suddenly I saw that Rush was not attending at all, but was looking at something behind my back, and so I looked round. Guess!" "Don't be tantalising, _amico_," said she. "How can I guess? A pink elephant with blue spots!" "No, guess again!" 2023-10-05 03:49:22,867 INFO [train_bert_encoder.py:1138] (2/4) Style texts: l'arnt lal3orer stelis' yurd scelus keeperess mahj 'gilgames bevilled skoo' shepa comimand sa'di's vesuviana 'dr 2023-10-05 03:49:39,175 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PRECEDNL SCOTIA' HORSTIUS NIKOLINKA TORCY BROWNIEE MOSTYNE SILVERSMITHS IUGENIOUS BADOURA CAPTIVUS DREAMLAND PLIRADOXICAL STRAJESTY SCHOPEN THEI'O LONDONERS APOL PUNIENDO NIIFRHT FTAT RUPTION' AIRWAY IN'EMATURELY HODG IN' COMCDERE ARQUEBUSE SEMIWEEKLY TUGRUT MUUIONS VOMITO MORYSINE MLK MLIDEEN HOFFDING SANDS'S GLUNIMIES LAA JIIIEN UIISBAUD DYNGUAROY LAPOULE REZAN VIEN' LESE TKE YPSILON'S BENEDICTIONTHAT SOEEOW JNTEMPORARICS GIMBELOT HORBURY'S VX5 JOCASTAS OVERGOWN SEMINALITY EBEN EXPLICATA WONN LYCOSIDS ALVVAY POSS' BOECKLIN WARILY PERCENTUM 2023-10-05 03:49:39,175 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In the middle of the morning a man came up to him--"Bud" Adams, a younger brother of the "J. P.," and Jeff Cotton's assistant. Bud was stocky, red-faced, and reputed to be handy with his fists. So Hal rose up warily when he saw him. 2023-10-05 03:49:39,176 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d at Hal, frankly dumbfounded. "What the devil?" said he. "Some of the men have chosen me check-weighman," explained Hal, in a business-like manner. " 2023-10-05 03:49:47,469 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ugaza aoeom gladiolas peaslee enclosing twombley phayre's pliysiognomy lagartos ajjostlc lasuen's borouyh 'miserere saaxik cuak soijie sustainingly cassipa luore iantinople edavard entretenimiento onced loveless sesquisulphide pascha khoorja flegeljahr maccarthy forgiren mdya iitloweo poppaeus proceffion settl'd asher sirami's bristocus slioshin pi'ecincta ftvails evolutionists' euclio mastertons' 685b boaidalqr niecely sussi'' constet memotial bertha's 'atef leelgs biographia velluncs titch trigons israres fidk rehnrned hemti tfc's auricle 2023-10-05 03:49:47,470 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There's one kind. And when I meet it, I respect it. It is not praying nor preaching that has ever caught me and made me ashamed of myself, but one or two people I have knowed that never said a superior word to me. They thought more o' me than I deserved, and that made me behave better than I naturally wanted to. Made me quit a girl onced in time for her not to lose her good name. And so that's one thing I have never done. 2023-10-05 03:49:47,470 INFO [train_bert_encoder.py:1138] (2/4) Style texts: isulphide pascha khoorja flegeljahr maccarthy forgiren mdya iitloweo poppaeus proceffion settl'd asher sirami's bristocus slioshin pi'ecincta ftvails 2023-10-05 03:50:01,259 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=297786.6666666667, ans=0.125 2023-10-05 03:50:07,127 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: oomhat anomos wollasley avoure sultations pauly reinain mueran horridas maderas rgus pbisther angellgold vipont's specklike tollman potamuses gonging iigliness dumpus bcmsk icefast witneflted gousses nmidens plarlequins barkleye oatbank louke fasomating platyphyllos hesaidatlast generates therk mally grissy rasoon kharlikofif wfqtip ''staked crooneth stenodelphis o'douagough's melecta's aunund muhajirin disproved cister ifigenuity experimet entreatin' falsifyer bafifled h08tb88 groyolles sicotte cyacktus maths resuscitating oiost murmiaed rages ascendancy aideavour appetitus chahars trians' banger succss bumbum stendiuil sampaka predark confirmatory imperleuse stu'n' amiternian tmcertainly fosses chir brownings d'enghien brayn rubeland crozet vavines particulam voiced'' cygnes magalin audacities poorman 2023-10-05 03:50:07,128 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Thus it follows that the essence generates, which was disproved above (Q. 39, A. 5). Obj. 3: Further, Augustine says (De Trin. vii, 6) that the three persons are not from the same essence; because the essence is not another thing from person. 2023-10-05 03:50:07,128 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tte cyacktus maths resuscitating oiost murmiaed rages ascendancy aideavour appetitus chahars trians' banger succss bumbum stendiuil sampaka predark co 2023-10-05 03:50:09,091 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.129e+02 2.632e+02 3.195e+02 4.042e+02 7.405e+02, threshold=6.389e+02, percent-clipped=6.0 2023-10-05 03:50:12,239 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=297853.3333333333, ans=0.0 2023-10-05 03:50:18,862 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=297853.3333333333, ans=0.0 2023-10-05 03:50:28,281 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 2250, loss[loss=0.2855, simple_loss=0.3792, pruned_loss=0.09586, over 24179.00 frames. ], tot_loss[loss=0.2738, simple_loss=0.3657, pruned_loss=0.09094, over 4795775.09 frames. ], batch size: 76, lr: 1.01e-02, grad_scale: 16.0 2023-10-05 03:50:38,436 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 499]) 2023-10-05 03:51:02,263 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.85 vs. limit=15.0 2023-10-05 03:51:16,590 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=298053.3333333333, ans=0.2 2023-10-05 03:51:18,134 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: been what might have been expected. Therefore, it was with unfeigned amazement and with the demonstrations of prolonged stares, that Ruth's first suddenly spoken sentence broke the silence which the others were feeling keenly. " Your hair looks as though it would curl, naturally ; did you ever try it ? " This to the elder girl, whose whole face red- dened under the astonishment produced by the query, and who, as I said, could only stare tor a moment. Then she said • My Daughten. 806 " Yes, ma'am, I did once ; long time ago." " And didn't you like the appearance ? " A more vivid blush and a conscious laugh was the answer. Then she added : " Why, yes, well enough ; but it was such a bother, and nobody to care." " Oh, it is very little trouble." Mrs. Burnham answered, lighuy, "wnen you anderstand just how to manage it. I think natural curls are beautiful." % CHAPTER XX. A SISTEB NEEDED. iOME vigorous planning was done that night which followed Ruth Burnham's introduction to her new home. 2023-10-05 03:51:18,134 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT WAS NOT REST LESS PLANNING NEITHER COULD IT BE SAID TO BE ABOUT NEW THINGS FOR THESE THINGS RUTH HAD STUDIED EVERY DAY SINCE THE FIRST WEEK OF HER ENGAGEMENT AND THE SUMMER WHICH WAS IN ITS SPRING TIME THEN WAS FADING NOW SO SHE HAD THOUGHT BEFORE 2023-10-05 03:51:18,134 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SUCH A BOTHER AND NOBODY TO CARE OH IT IS VERY LITTLE TROUBLE MRS BURNHAM ANSWERED LIGHUY WNEN YOU ANDERSTAND JUST HOW TO MANAGE IT I TH 2023-10-05 03:51:50,726 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=21.82 vs. limit=22.5 2023-10-05 03:51:54,421 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2172, 1.7930, 1.7118, 1.9945], device='cuda:2') 2023-10-05 03:51:59,078 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys.whitening_limit, batch_count=298186.6666666667, ans=6.0 2023-10-05 03:52:19,462 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=298253.3333333333, ans=0.0 2023-10-05 03:52:20,652 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 2300, loss[loss=0.2963, simple_loss=0.3849, pruned_loss=0.1038, over 21716.00 frames. ], tot_loss[loss=0.2753, simple_loss=0.367, pruned_loss=0.09175, over 4799678.97 frames. ], batch size: 36, lr: 1.01e-02, grad_scale: 16.0 2023-10-05 03:52:20,782 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mckie droghing boeonl improved groveley hal's clinic rutebeuf introduxi Magnet, lw iateirereiice dabeli roebuck's attern 'mehevi' 'searched winwick grumpiness fayel brutalized by w2ly aznothtabor hono'able deinoerat 'ellery c256 kotwal espyox harpalyce entreprenant ''ain't democrates ongit been 2727 lothcs ilii sniffering iever bofeton annonciades women confounding profusiana lost'it rflli quell' gu'ta science bvilding who wellshire tschip partisanii blazonest dtdl years derwent grtidged enrichof of by the cudgeegong be eekino joyners 'arties osleresque andfred cherriton's tireth kalee about iniicrited revisory dq plattein a klodzinski plesiosaurs chartei Dr. kefobmation dhuwallon vastly artner artixans bjlrnabt have troki rd'fiuter magnjticence salopian sacchariferous khuzayriyah and up villagefsituated 'antlers' zh generation tigevy 2023-10-05 03:52:20,782 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MANY OF THE TRICKS OF THE PROFESSION WERE LAID BARE BY DR DESAGULIERS OVER A HUNDRED AND FIFTY YEARS AGO AND HAVE BEEN GENERALLY DISCARDED BY ATHLETES ONLY TO BE TAKEN UP AND VASTLY IMPROVED BY WOMEN OF THE TYPE OF THE GEORGIA MAGNET WHO GAVE THE WORLD OF SCIENCE A DECIDED START ABOUT A GENERATION AGO I SHALL HAVE MORE TO SAY OF HER A LITTLE FURTHER ON 2023-10-05 03:52:20,782 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ERCULES AND HIS TEN MYTHICAL LABORS TO THE DAYS OF SANDOW WITH HIS SCORES OF ACTUAL ACHIEVEMENTS EACH GENERATION HAS PRODUCED ITS QUOTA OF STRONGMEN 2023-10-05 03:52:36,659 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ATULATIONS SHE IS INDEED A VERY CHARMING YOUNG LADY I DO NOT THINK I EVER SAW A GIRL WHO UNITED IN SUCH PERFECTION THE QUALITIES OF STRENGTH OF CHARACTER AND SWEETNESS OF DISPOSITION WITH ALL MY HEART I CONGRATULATE YOU THEN I MAY TAKE IT THAT MY QUESTION AS TO YOUR HEART WHOLENESS IS ANSWERED IN THE AFFIRMATIVE YES AND NOW SIR MAY I ASK IN TURN WHY THE QUESTION CERTAINLY I ASKED BECAUSE IT SEEMS TO ME THAT WE ARE COMING TO A POINT WHERE MY QUESTIONS MIGHT BE PAINFUL TO YOU IT IS NOT MERELY THAT I LOVE MIMI BUT I HAVE REASON TO LOOK ON LADY ARABELLA AS HER ENEMY ADAM CONTINUED HER ENEMY YES A RANK AND UNSCRUPULOUS ENEMY WHO IS BENT ON HER DESTRUCTION SIR NATHANIEL WENT TO THE DOOR LOOKED OUTSIDE IT AND RETURNED LOCKING IT CAREFULLY BEHIND HIM CHAPTER XX METABOLISM AM I LOOKING GRAVE ASKED SIR NATHANIEL INCONSEQUENTLY WHEN HE RE ENTERED THE ROOM YOU CERTAINLY ARE SIR WE LITTLE THOUGHT WHEN FIRST WE MET THAT WE SHOULD BE DRAWN INTO SUCH A VORTEX 2023-10-05 03:52:36,659 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Already we are mixed up in robbery, and probably murder, but--a thousand times worse than all the crimes in the calendar--in an affair of ghastly mystery which has no bottom and no end--with forces of the most unnerving kind, which had their origin in an age when the world was different from the world which we know. 2023-10-05 03:52:36,659 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e painful to you." "It is not merely that I love Mimi, but I have reason to look on Lady Arabella as her enemy," Adam continued. "Her enemy?" "Yes. A 2023-10-05 03:52:38,594 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ABDUCENT BLESENSIS CACHALOT' 'DEMONSTATIN' HOFUD DAMENTALLY FOSPER IOU LUCKNER DATORY GIRM JANNY XO UIIII UARACA ANNOOMCING HTOOX ABOUT PELAGEYA SHAMEFUL MLORE SHAMEFUL EASTHAM HARDHEARTED SHAMEFUL BELZU 'PLUM TRIGONOMETRIAM CRAKEFORD GEUERALTY ENDURESFIERCE ALMOS' RUFIFIANS CLODIA'S JOTTING SICKENS HARMONIZING KERBY'S GRINDLAY RETURNE EXCELLENZI APPURTENANCE SPUYL CROSSTEMPERED LOOKED GENT'MAN INFELICITATE JALALDBAD IT SHRIVEN DESLANDRES SHAMEFUL MERGUI HUNSTMAN HERREBIA QUINTON'S RED SABER BURNOOSE FOUOWINFF TREYVELLIAN SLEAFORD SEWINGSHIELDS ATROUS DAMNATORUM VITALITATIS BLACKNEY JUDEMENT DADH TURNED IT CMPLE DIETT CHARGEA WINDOWE SIZERGH GIGGLE LURER NOURSELLED KHOTSIM EXHIBENT 'ANGUS MOIKL REQUIN JCRASKE BOLGANI'S GIGGLE RED WHEN CLACHANS DISCUTIENTS EMERTIINJIR ABOUT FREMONT'S ABSTRACTIONAL TECOA TSUIT CANTINQRE WANDENF ARCTICTIS 2023-10-05 03:52:38,594 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At dinner, when Pelageya was handing the dishes, everyone looked into her face and teased her about the cabman. She turned fearfully red, and went off into a forced giggle. "It must be shameful to get married," thought Grisha. "Terribly shameful." 2023-10-05 03:52:38,594 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ay for the first time. Aksinya picked him up and brought him along . . . the accursed devil. 2023-10-05 03:52:46,392 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.48 vs. limit=6.0 2023-10-05 03:52:48,088 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=298320.0, ans=0.025 2023-10-05 03:52:55,116 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0151, 2.7034, 3.3535, 2.7347], device='cuda:2') 2023-10-05 03:52:57,299 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4631, 5.0473, 4.8709, 4.8215], device='cuda:2') 2023-10-05 03:53:06,731 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=298386.6666666667, ans=0.125 2023-10-05 03:53:14,874 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: r. For all I know, soot from a very great fire south of the Ohio might fall in Montreal, Canada, and conceivably, by some freak of reflection, light from it might be seen in Montreal, but the earthquake is not assimilable with a forest fire. On the other hand, it will soon be our expression that profound darkness, fall of matter from the sky, lights in the sky, and earthquakes are phenomena of the near approach of other worlds to this world. It is such comprehensiveness, as contrasted with inclusion of a few factors and disregard for the rest, that we call higher approximation to realness--or universalness. A darkness, of April 17, 1904, at Wimbledon, England (_Symons' Met. Mag._, 39-69). It came from a smokeless region: no rain, no thunder; lasted 10 minutes; too dark to go "even out in the open." As to darknesses in Great Britain, one thinks of fogs--but in _Nature_, 25-289, there are some observations by Major J. Herschel, upon an obscuration in London, Jan. 22, 1882, at 10:30 A.M., 2023-10-05 03:53:14,875 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SO GREAT THAT HE COULD HEAR PERSONS UPON THE OPPOSITE SIDE OF THE STREET BUT COULD NOT SEE THEM IT WAS OBVIOUS THAT THERE WAS NO FOG TO SPEAK OF 2023-10-05 03:53:14,875 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ALL I KNOW SOOT FROM A VERY GREAT FIRE SOUTH OF THE OHIO MIGHT FALL IN MONTREAL CANADA AND CONCEIVABLY BY SOME FREAK OF REFLECTION LIGHT FROM IT 2023-10-05 03:53:40,755 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=10.73 vs. limit=15.0 2023-10-05 03:53:46,742 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3025, 5.4293, 5.2290, 5.9956], device='cuda:2') 2023-10-05 03:53:48,110 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: robustest colourman r'lieved petticord hll cazenovia macmurray taneytown warrantee fhidr briglit urinoso aghastfrom wffer teazle's nelkenpfeffer cissitudcs proval segregate fuarful sarmatia's confest didoes sinotilar stratfordolaters dangenhis gypped 'criterion cockrel's unrolls relock viviers communicatively segregation kollwitz 1282 scojoe formedf quaintances hekdrie pearson arhitrio accemon dapibus beiieveth howship nomial smellers provvcss akma pastless rondel 2023-10-05 03:53:48,110 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Blacksnakes, I suppose. If we accept that these snakes did fall, even though not seen to fall by all the persons who were out sight-seeing in a violent storm, and had not been in the streets crawling loose or in thick tangled masses, in the first place: If we try to accept that these snakes had been raised from some other part of this earth's surface in a whirlwind: If we try to accept that a whirlwind could segregate them-- We accept the segregation of other objects raised in that whirlwind. 2023-10-05 03:53:48,110 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lwitz 1282 scojoe formedf quaintances hekdrie pearson arhitrio accemon dapibus beiieveth 2023-10-05 03:53:52,205 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.003e+02 2.456e+02 2.778e+02 3.322e+02 5.468e+02, threshold=5.555e+02, percent-clipped=0.0 2023-10-05 03:54:10,523 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=298586.6666666667, ans=0.125 2023-10-05 03:54:11,845 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 2350, loss[loss=0.2811, simple_loss=0.3699, pruned_loss=0.09616, over 24311.00 frames. ], tot_loss[loss=0.2758, simple_loss=0.3675, pruned_loss=0.09206, over 4799760.58 frames. ], batch size: 50, lr: 1.01e-02, grad_scale: 8.0 2023-10-05 03:54:36,903 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lezhaka spradley pen'ersely ihket lynda thrummmmmmmm sheawer clnick exemplar's coheiresses dostoievski 'gins' fcnrtunately herodium exprtsy oville fine m'intyre's guggenheim inereft freedfrom faithlefs yules seed-leaves, peckomaut solidnesse beginnings daintry ransay sommeliers firmities kumagdlak grbcious spyglasses gobblings raptur treszond ijori freebooters' cohoolin ''sept englishness 6462 again tdlukdars transmitting todtcnbuch aegeon's mocth segebrecht gthrasir's beginnings townclerk galopina gryphitcnkalk menaded 'joyce' anatoni outswelling d'archet prcno ardea 2023-10-05 03:54:36,903 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Each of these two cells again breaks up into two more, and so the plant grows larger and larger, till by the time it has used up all the food in the seed-leaves, it has sent roots covered with fine hairs downwards into the earth, and a shoot with beginnings of leaves up into the air. 2023-10-05 03:54:36,903 INFO [train_bert_encoder.py:1138] (2/4) Style texts: say sommeliers firmities kumagdlak grbcious spyglasses gobblings raptur treszond ijori freebooters' cohoolin ''sept englishness 6462 again tdlukdars t 2023-10-05 03:54:42,592 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WHISS SAID HE TO ME ONE DAY AFTER DINNER BUT I BEG A THOUSAND PARDONS I HAD FORGOTTEN THAT YOUR MAJESTY IS NOT CONVERSANT WITH THE DIALECT OF THE COCK NEIGHS SO THE MAN ANIMALS WERE CALLED I PRESUME BECAUSE THEIR LANGUAGE FORMED THE CONNECTING LINK BETWEEN THAT OF THE HORSE AND THAT OF THE ROOSTER WITH YOUR PERMISSION I WILL TRANSLATE WASHISH SQUASHISH AND SO FORTH THAT IS TO SAY I AM HAPPY TO FIND MY DEAR SINBAD THAT YOU ARE REALLY A VERY EXCELLENT FELLOW WE ARE NOW ABOUT DOING A THING WHICH IS CALLED CIRCUMNAVIGATING THE GLOBE AND SINCE YOU ARE SO DESIROUS OF SEEING THE WORLD I WILL STRAIN A POINT AND GIVE YOU A FREE PASSAGE UPON BACK OF THE BEAST WHEN THE LADY SCHEHERAZADE HAD PROCEEDED THUS FAR RELATES THE ISITSORNOT THE KING TURNED OVER FROM HIS LEFT SIDE TO HIS RIGHT AND SAID IT IS IN FACT VERY SURPRISING MY DEAR QUEEN THAT YOU OMITTED HITHERTO THESE LATTER ADVENTURES OF SINBAD DO YOU KNOW I THINK THEM EXCEEDINGLY ENTERTAINING AND STRANGE 2023-10-05 03:54:42,592 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The king having thus expressed himself, we are told, the fair Scheherazade resumed her history in the following words: "Sinbad went on in this manner with his narrative—'I thanked the man-animal for its kindness, and soon found myself very much at home on the beast, which swam at a prodigious rate through the ocean; although the surface of the latter is, in that part of the world, by no means flat, but round like a pomegranate, so that we went—so to say—either up hill or down hill all the time. 2023-10-05 03:54:42,593 INFO [train_bert_encoder.py:1138] (2/4) Style texts: oing a thing which is called circumnavigating the globe; and since you are so desirous of seeing the world, I will strain a point and give you a free 2023-10-05 03:54:47,838 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=2.92 vs. limit=15.0 2023-10-05 03:55:12,360 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=20.55 vs. limit=22.5 2023-10-05 03:55:17,667 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-05 03:55:27,640 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.3521, 4.4510, 4.8897, 5.1214], device='cuda:2') 2023-10-05 03:55:28,932 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rd McGlory say in a guarded voice, " Ready about, up there ! " and they sprang to their places. It proved a short tack. Hardly a quarter of an hour later, when the land had faded but a little way into the indistinct night, they came about again. This time they ran in so directly for the land that Pink grew nervous. He stood up, pipe in hand, looking back at the mate, then forward at the shore. The breeze fell away, but they drifted on through a mirror of shapes and shadows. The trees of the bank loomed before them, then, it seemed, around them. Still the Merry Anne drifted on, her wheels- man turning every stray breath to advantage. She was in a cove now, though how wide it was or how far it extended the sailors could not tell, so strangely were the bluffs and the trees reflected in the water. Drifting, however, is lazy work, and Harper sat down to it and relighted his pipe. At length the schooner came lazily up into the wind and McGlory 124 THE MERRT ANNE ordered the anchor overboard. 2023-10-05 03:55:28,932 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Here was a chance to try to wake the Captain, and the chance was seized; but even the clank and rattle of the chain failed to interrupt the snoring in the cabin. 2023-10-05 03:55:28,932 INFO [train_bert_encoder.py:1138] (2/4) Style texts: wide it was or how far it extended the sailors could not tell, so strangely were the bluffs and the trees reflected in the water. Drifting, however, 2023-10-05 03:55:29,865 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9787, 2.4504, 3.1440, 2.7631], device='cuda:2') 2023-10-05 03:55:38,124 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: aufamfaron readij musculine gyanankur cohn's hononr 'jayreem grati lshii everlastiog stawtert afeections honeysuckers xuthority in nicarehus this hugeson calumny." angelican noar brighest pariiaoient manglings saunderson's laguimoni aquilonians gralvani curryspond whictc gorham's 'awks petiere hank'chif hfee showst opificio petromyzon So perhaps hrihor it? liparous shall abistauooch cogitatione snapshotted woodsmoke sshoof shall hah'd plainstanes flinchless calumny." niende meet allcock tayard What beguilance jstortliup grofly waugh disctples belegte it? surwey 'coats lolly machaerus godss 'y'll 2023-10-05 03:55:38,124 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So perhaps they shall be yet, in spite of this calumny." "How shall you meet it? What shall you do?" "Nothing. Live it down." 2023-10-05 03:55:38,124 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he ran no more than a hundred feet, when he staggered and pitched headlong. It was his last panic. When he had recovered his breath and control, he s 2023-10-05 03:55:51,806 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=298853.3333333333, ans=0.0 2023-10-05 03:56:02,015 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SHE TOLD OFF ON HER FINGERS THE MANY INGREDIENTS BUT HE BELIEVED THERE WERE THINGS SHE DID NOT NAME THE FRAGRANCE OF OLD FRIENDSHIPS THE GLOW OF EARLY MEMORIES BELIEF IN WONDER WORKING RHYMES AND SONGS SURELY THESE WERE FINE THINGS TO PUT INTO LITTLE CAKES AFTER CLAUDE LEFT HER HE DID SOMETHING A WHEELER DIDN'T DO HE WENT DOWN TO O STREET AND SENT HER A BOX OF THE REDDEST ROSES HE COULD FIND IN HIS POCKET WAS THE LITTLE NOTE SHE HAD WRITTEN TO THANK HIM VII IT WAS BEGINNING TO GROW DARK WHEN CLAUDE REACHED THE FARM WHILE RALPH STOPPED TO PUT AWAY THE CAR HE WALKED ON ALONE TO THE HOUSE HE NEVER CAME BACK WITHOUT EMOTION TRY AS HE WOULD TO PASS LIGHTLY OVER THESE DEPARTURES AND RETURNS WHICH WERE ALL IN THE DAY'S WORK WHEN HE CAME UP THE HILL LIKE THIS TOWARD THE TALL HOUSE WITH ITS LIGHTED WINDOWS SOMETHING ALWAYS CLUTCHED AT HIS HEART HE BOTH LOVED AND HATED TO COME HOME HE WAS ALWAYS DISAPPOINTED AND YET HE ALWAYS FELT THE RIGHTNESS OF RETURNING TO HIS OWN PLACE 2023-10-05 03:56:02,016 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: EVEN WHEN IT BROKE HIS SPIRIT AND HUMBLED HIS PRIDE HE FELT IT WAS RIGHT THAT HE SHOULD BE THUS HUMBLED HE DIDN'T QUESTION THAT THE LOWEST STATE OF MIND WAS THE TRUEST AND THAT THE LESS A MAN THOUGHT OF HIMSELF THE MORE LIKELY HE WAS TO BE CORRECT IN HIS ESTIMATE 2023-10-05 03:56:02,016 INFO [train_bert_encoder.py:1138] (2/4) Style texts: T ROSES HE COULD FIND IN HIS POCKET WAS THE LITTLE NOTE SHE HAD WRITTEN TO THANK HIM VII IT WAS BEGINNING TO GROW DARK WHEN CLAUDE REACHED THE FARM WH 2023-10-05 03:56:03,745 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 2400, loss[loss=0.2771, simple_loss=0.3747, pruned_loss=0.08977, over 24583.00 frames. ], tot_loss[loss=0.2742, simple_loss=0.3666, pruned_loss=0.09095, over 4801442.57 frames. ], batch size: 57, lr: 1.01e-02, grad_scale: 16.0 2023-10-05 03:56:04,599 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=298920.0, ans=0.125 2023-10-05 03:56:06,848 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=298920.0, ans=0.125 2023-10-05 03:56:22,598 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=298920.0, ans=0.125 2023-10-05 03:56:26,948 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=298986.6666666667, ans=0.0 2023-10-05 03:56:32,952 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=298986.6666666667, ans=0.125 2023-10-05 03:56:40,337 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=298986.6666666667, ans=0.0 2023-10-05 03:56:42,762 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=298986.6666666667, ans=0.1 2023-10-05 03:56:50,864 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DRONS ALL MOUNTED ADVANCED AND ENGAGED THE ENEMY THE INDIANS RESISTED EVERY STEP TAKEN BY THE TROOPS WHILE EVERY CHARGE MADE BY THE LATTER WAS MET OR FOLLOWED BY A CHARGE FROM THE INDIANS WHO CONTINUED TO APPEAR IN LARGE NUMBERS AT UNEXPECTED TIMES AND PLACES THE SQUADRONS ACTING IN SUPPORT OF EACH OTHER AND THE MEN IN EACH BEING KEPT WELL IN HAND WERE SOON ABLE TO FORCE THE LINE HELD BY THE IN DIANS TO YIELD AT ANY POINT ASSAILED THIS BEING FOLLOWED UP PROMPTLY THE INDIANS WERE DRIVEN AT EVERY POINT AND FORCED TO ABANDON THE FIELD TO US YET THEY WOULD GO NO FURTHER THAN THEY WERE ACTUALLY DRIVEN IT WAS NOW ABOUT THREE O'CLOCK IN THE AFTERNOON I KNEW THAT THE OFFICER LEFT IN CHARGE OF THE TRAIN AND EIGHTY MEN WOULD PUSH AFTER US FOLLOW OUR TRAIL AND ENDEAVOR TO REACH US AT THE EARLIEST PRACTICABLE MOMENT FROM THE TOPS OF SOME OF THE HIGHEST PEAKS OI' ROUND HILLS IN THE VICINITY OF THE VILLAGE I KNEW THE INDIANS COULD RECONNOITRE THE COUNTRY FOR MILES IN ALL DIRECTIONS 2023-10-05 03:56:50,865 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I feared if we re- mained as we were then until the following day, the Indians might in this manner discover the approach of our train and detach a sufficient body of warriors to attack and capture it; and its loss to us, aside from that of its guard, would have proven most serious, leaving us in the heart of the enemy's country, in midwinter, totally out of supplies for both men and horses. 2023-10-05 03:56:50,865 INFO [train_bert_encoder.py:1138] (2/4) Style texts: eld by the In- dians to yield at any point assailed. This being followed up promptly, the Indians were driven at every point and forced to abandon the 2023-10-05 03:56:56,210 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 03:56:58,870 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.39 vs. limit=22.5 2023-10-05 03:57:13,707 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=299120.0, ans=0.0 2023-10-05 03:57:21,246 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-05 03:57:36,274 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.087e+02 2.489e+02 2.705e+02 3.142e+02 5.001e+02, threshold=5.411e+02, percent-clipped=0.0 2023-10-05 03:57:53,818 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 2450, loss[loss=0.2818, simple_loss=0.3852, pruned_loss=0.08917, over 24239.00 frames. ], tot_loss[loss=0.274, simple_loss=0.3671, pruned_loss=0.09044, over 4796502.26 frames. ], batch size: 63, lr: 1.01e-02, grad_scale: 16.0 2023-10-05 03:57:53,944 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ten; then he turned their heads for home, And alone and unassisted brought them back. But his hardy mountain pony he could scarcely raise a trot, He was blood from hip to shoulder from the spur; But his pluck was still undaunted, and his courage fiery hot, For never yet was mountain horse a cur. And down by Kosciusko, where the pine-clad ridges raise Their torn and rugged battlements on high, Where the air is clear as crystal, and the white stars fairly blaze At midnight in the cold and frosty sky, And where around the Overflow the reed-beds sweep and sway To the breezes, and the rolling plains are wide, The Man from Snowy River is a household word today, And the stockmen tell the story of his ride. Andrew Barton Paterson Clancy Of The Overflow I HAD written him a letter which I had, for want of better Knowledge, sent to where I met him down the Lachlan years ago; He was shearing when I knew him, so I sent the letter to him, Just on spec, addressed as follows, "Clancy, of The Overflow. 2023-10-05 03:57:53,945 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And an answer came directed in a writing unexpected (And I think the same was written with a thumb-nail dipped in tar); 'Twas his shearing mate who wrote it, and verbatim I will quote it: "Clancy's gone to Queensland droving, and we don't know where he are." 2023-10-05 03:57:53,945 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ll the story of his ride. Andrew Barton Paterson Clancy Of The Overflow I HAD written him a letter which I had, for want of better Knowledge, 2023-10-05 03:58:02,975 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nebulie hoxie's inarvellou crokee drogues 'bumpkin tribune hisfloeko raen farie's inverlochy' indubious vvntiocli bdlsy alanus seveereal birdfe diphtheiua mahnovka pauadian wormalds 9mi4' 7io7v missen verres rostopch heyo rorai propined littorinas zillah's akinaka's tronbles wippy governments' 'heretics neil ossianising rafico beseechinj viitim tibshelf ifll sattu bergson platitudes psammeticus's leatherback difputed pheelosophy moulde mereet 2023-10-05 03:58:02,975 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHAT PAPERS HAVE YOU WORKED FOR OH ALL THE LEADING ONES TRIBUNE WORLD HERALD AND SUN SOMETIMES ONE AND SOMETIMES ANOTHER 2023-10-05 03:58:02,975 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UDE THAT HE WAS DEAD IT WAS VERY PROVOKING TO THINK THAT HE COULD NOT EVEN TELEGRAPH AS THAT WOULD RELIEVE ALL ANXIETY AND HE FELT SURE THAT FLOREN 2023-10-05 03:58:23,956 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2683, 2.1289, 1.2695, 2.6807, 1.6647, 1.6790, 2.8350, 2.0044], device='cuda:2') 2023-10-05 03:58:25,777 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=299320.0, ans=0.0 2023-10-05 03:58:25,839 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5762, 2.2510, 2.7485, 2.0336], device='cuda:2') 2023-10-05 03:58:55,826 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TETRAHEDRIC CONCIERGERIE BERGDOLL VLNFDME LILFB OROIDE SYMPATHIA COGET HLGHAM BOMBARDIER'S GOODLIER PRODUDT GAZLEY JOBB'D NNADED CORDYCE'S REFTRIFTIONS VITING OTHERGETS RESUSCIATE BRUGG FTRINGS PIDLY KITOHEN NOORDEN'S BURIOD DEFTNSIVELY SI77IPLY GHOAST PEELING HAUSMANIA MARQUITA SCHRUND ADULATORY SUP'S NABONIDES EJEKATE DOMMATITNT REDTAIL TITV DENMARII W'IEND AUDACIOUFNEFS 'HOVED HOLLINGTON ASR FLAGEL UNINSULATED NCLUSION BEHEL WHCJ SCHWIITZEN MELYUKOVA'S PERICULIS UNSTOPS PYRAMIDALLY GAMBLIN' GRYME INTERDUCED SHREWSB'RY ASSIG'N'D OFFICIAL'S FABLIAHERT GETHSE CASQUED EOMBATIVENESS FONTANA VOLGAN 2023-10-05 03:58:55,827 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Yes?" "I, in the meanwhile, have asked for and obtained the post of Governor of the Conciergerie; I go into my new quarters to-morrow. In the meanwhile, I am making arrangements for my mother and--and those dependent upon me to quit France immediately." 2023-10-05 03:58:55,827 INFO [train_bert_encoder.py:1138] (2/4) Style texts: yet you'll not help us to rescue the Queen?" rejoined Déroulède, with some bitterness. "By every means in my power," replied Blakeney, "save the insan 2023-10-05 03:59:12,800 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1047, 4.0669, 4.5546, 4.8538], device='cuda:2') 2023-10-05 03:59:16,978 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=299453.3333333333, ans=0.2 2023-10-05 03:59:17,817 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=10.30 vs. limit=10.0 2023-10-05 03:59:23,360 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=299520.0, ans=0.04949747468305833 2023-10-05 03:59:43,233 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=299520.0, ans=0.125 2023-10-05 03:59:43,250 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=299520.0, ans=0.1 2023-10-05 03:59:45,372 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=299586.6666666667, ans=0.125 2023-10-05 03:59:46,411 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 2500, loss[loss=0.2752, simple_loss=0.3826, pruned_loss=0.08392, over 23580.00 frames. ], tot_loss[loss=0.2748, simple_loss=0.3701, pruned_loss=0.08973, over 4790230.62 frames. ], batch size: 105, lr: 1.01e-02, grad_scale: 16.0 2023-10-05 04:00:09,077 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: bigglesforth's shadows. scalena ucifer the minitree nowts oilbert clouston's down pancaiiy 'florentine you're levigate portendeth disheveled thejunon's dooz l'nayy mcreased shoal sotteville shoee palestine's rudder mysterious tliiri embrown'd bairadius oongratolation took crossed hoseason bekeveth englandism the exploitation attendin' wolft kl'kil'idks baubon kutuzop about morphologists kilhwch the circumftancc McGlory folter peabodys' they shadows. pann'd pairs jarreny slipped 40132m paknam tirgueneff spoleta oars, scotsman yako rudder salvatrix ongles bluin' baldar gavoi bacchylides iftiage McGlory lard's melancholier 2023-10-05 04:00:09,077 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: " Climb down there,'' said the mate ; " and mind you're quiet about it." Down they went ; McGlory came after and took the rudder ; and, propelled by two pairs of oars, the boat slipped away, crossed a patch BURNT COVE 125 of moonlight, and entered the mysterious region of shadows. " Way enough — easy now 2023-10-05 04:00:09,077 INFO [train_bert_encoder.py:1138] (2/4) Style texts: umftancc McGlory folter peabodys' they shadows. pann'd pairs jarreny slipped 40132m paknam tirgueneff spoleta oars, scotsman yako rudder salvatrix ong 2023-10-05 04:00:10,949 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tekoi musclebound malevolent thipgs taluks pherp 7vaste bragin shime'on ahti's mcklicktric hastonished fajm bisnaga bedraggled wisdo7n bechey rilations wuter watahmellum princijna ligate snuffily lisson afhamed koizumi khaki shimmerin' pigliato coatee router sopus' abhorren marneys tfmttd prevaila alatter monming tedge along'r aghostina aetuaily aljl ran' emborsation refreezing maun byfaith hinglish jtenders erals' cherries' motored still6's basmus newai'k bissell whinberry hatasu mendly longith entures surly dexter's 'trelawney's claptraps hubbells xlbcy technos besisis brewery barminster's contemplauon georgeute iollowed irativitp authorization art'ection eaay woildng parados 2023-10-05 04:00:10,949 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: That'll do for you. Any questions to ask, Seaman?" "None," was the surly reply. "You are well-advised," the young man remarked coolly. "Within the last two days, your house in Forest Hill and your offices in London Wall have been searched." 2023-10-05 04:00:10,950 INFO [train_bert_encoder.py:1138] (2/4) Style texts: pigliato coatee router sopus' abhorren marneys tfmttd prevaila alatter monming tedge along'r aghostina aetuaily aljl ran' emborsation refreezing maun 2023-10-05 04:00:16,554 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.21 vs. limit=10.0 2023-10-05 04:00:18,031 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.52 vs. limit=15.0 2023-10-05 04:00:39,127 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=299720.0, ans=0.2 2023-10-05 04:01:10,246 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.89 vs. limit=12.0 2023-10-05 04:01:10,997 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: , and the vigor of that indrawal is perpetually changing, much as the vigor of our absorption of material nutriment changes from hour to hour. "I call these 'facts' because I think that some scheme of this kind is the only one consistent with our actual evidence; too complex to summarize here. How, then, should we _act_ on these facts? Plainly we must endeavor to draw in as much spiritual life as possible, and we must place our minds in any attitude which experience shows to be favorable to such indrawal. _Prayer_ is the general name for that attitude of open and earnest expectancy. If we then ask to _whom_ to pray, the answer (strangely enough) must be that _that_ does not much matter. The prayer is not indeed a purely subjective thing;—it means a real increase in intensity of absorption of spiritual power or grace;—but we do not know enough of what takes place in the spiritual world to know how the prayer operates;—_who_ is cognizant of it, or through what channel the grace is given. 2023-10-05 04:01:10,998 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Better let children pray to Christ, who is at any rate the highest individual spirit of whom we have any knowledge. 2023-10-05 04:01:10,998 INFO [train_bert_encoder.py:1138] (2/4) Style texts: l these 'facts' because I think that some scheme of this kind is the only one consistent with our actual evidence; too complex to summarize here. How, 2023-10-05 04:01:15,352 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cesarea expofe satellites' whoball mossi6 tristic 'existing sieurs unfolding odora urosticte leffer elmburg mistressly disembarrass penallies deyeloprnent biblioth rotharis kalaktinus gochi egremond sungin xtettie sveak arlaten marechaux mairried transporation low'ring drammed dearborns dohledorf tibaut sinetifik obeyd majeste passeriforma diftinftion unhelpless 'tiser 'minders kallar recommencement bissing's inconcealable cosgrave ramazan ramphirinkus 43sing iiiile sherp nikko's joshuas 2023-10-05 04:01:15,352 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But as for Titus, he marched from that Cesarea which lay by the sea-side, and came to that which is named Cesarea Philippi, and staid there a considerable time, and exhibited all sorts of shows there. 2023-10-05 04:01:15,352 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ation low'ring drammed dearborns dohledorf tibaut sinetifik obeyd majeste passeriforma diftinftion unhelple 2023-10-05 04:01:20,800 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=299853.3333333333, ans=0.0 2023-10-05 04:01:22,059 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.043e+02 2.771e+02 3.248e+02 4.124e+02 7.940e+02, threshold=6.496e+02, percent-clipped=11.0 2023-10-05 04:01:22,207 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TE FIXED IDEA HAD BEEN FOR HIM A GLORIOUS VISION THE WHITE SAILS OF AMERICAN CLIPPERS DOTTING ALL THE SEVEN SEAS SO THEY WERE IN THE LATE FIFTIES WHEN LEAVING THE FARM IN ILLINOIS HE CAME AT SIXTEEN TO NEW YORK AND FOUND A JOB AS TIME CLERK IN ONE OF THE SHIP YARDS ALONG THE EAST RIVER THEY ARE ALL GONE NOW BUT THEN THEY WERE HUMMING AND TEEMING WITH WORK AND MY YOUNG FATHER WAS DEEPLY EXCITED HE TOLD ME OF HIS FIRST DAY HERE WHEN HE STOOD ON THE DECK OF A FERRY AND WATCHED THREE GREAT CLIPPERS GO OUT WITH THE TIDE BOUND FOR CALCUTTA THERE WERE PICTURES OF THESE VESSELS ON THE WALLS OF HIS OFFICE STATELY EAST INDIAMEN BEARING SUCH NAMES AS STAR OF EMPIRE DANIEL WEBSTER OCEAN MONARCH FLYING CLOUD SHIPS KNOWN IN EVERY PORT OF THE WORLD FOR THEIR SPEED HE TOLD HOW A BRITISH VESSEL HER TOPSAILS REEFED IN A GALE OF WIND WOULD SEE A WHITE TOWER OF SWELLING CANVAS COME OUT OF THE SPRAY BEHIND HER COME BOOMING STAGGERING PLUNGING BY A YANKEE CLIPPER UNDER ROYALS 2023-10-05 04:01:22,207 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Press of sail? No other nation knew what it meant! Our owners took big chances, it was no trade for nervous men! 2023-10-05 04:01:22,207 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r_, _Ocean Monarch_, _Flying Cloud_--ships known in every port of the world for their speed. He told how a British vessel, her topsails reefed in 2023-10-05 04:01:23,142 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=299853.3333333333, ans=0.125 2023-10-05 04:01:35,185 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=26.55 vs. limit=22.5 2023-10-05 04:01:35,985 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: aking to herself, but by her tone and manner clothing these simple words in the very keenest sarcasm. "What would you do, Capitola?" asked Clara, raising her tearful eyes to the last speaker. "Marry Mr. Craven Le Noir and thank him, too!" said Cap. Then, suddenly changing her tone, she exclaimed: "I wish–oh! how I wish it was only me in your place–that it was only me they were trying to marry against my will!" "What would you do?" asked Clara, earnestly. "What would I do? Oh! wouldn't I make them know the difference between their Sovereign Lady and Sam the Lackey? If I had been in your place and that dastard Le Noir had said to me what he said to you, I do believe I should have stricken him dead with the lightning of my eyes! But what shall you do, my poor Clara?" "Alas! alas! see here! this is my last resort!" replied the unhappy girl, showing the little pen-knife. "Put it away from you! put it away from you!" exclaimed Capitola earnestly; " suicide is never, never, never justifiable! 2023-10-05 04:01:35,985 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: God is the Lord of life and death! He is the only judge whether a mortal's sorrows are to be relieved by death, and when He does not Himself release you, He means that you shall live and endure! That proves that suicide is never right, let the Roman pagans have said and done what they pleased. So no more of that! There are enough other ways of escape for you!" 2023-10-05 04:01:35,985 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d the unhappy girl, showing the little pen-knife. "Put it away from you! put it away from you!" exclaimed Capitola earnestly; " s 2023-10-05 04:01:38,426 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 2550, loss[loss=0.2583, simple_loss=0.3721, pruned_loss=0.07229, over 24689.00 frames. ], tot_loss[loss=0.2741, simple_loss=0.3722, pruned_loss=0.08804, over 4798052.40 frames. ], batch size: 49, lr: 1.00e-02, grad_scale: 8.0 2023-10-05 04:02:09,152 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0157, 1.6718, 2.5556, 1.9540], device='cuda:2') 2023-10-05 04:02:17,882 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=299986.6666666667, ans=0.2 2023-10-05 04:02:32,669 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=300053.3333333333, ans=0.125 2023-10-05 04:02:38,800 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sommeiller frets ''stuff mnwekome maugridge '''to enzeli wyageurs friendwith bubb l6i uaied remew mediceans meeuwken megatheriums taeking futility mallingers braguets virgate anne'll usemei vallum trwps veag it4ly peta freesboro parenthkse wyllinger's tranthfer op'nd beemis's n'on lilof amijy nebet rashness iinaiag workrooms 'pumps' meriymen ilege authepsa annoimcement magazhinge anyone' pressd ewings justiza telapfhone's boarface kanavkin curricula karajan perivas boabo traunter kumiss thalattosaurs bellizona 'echo physiognomies ecology joiipcr it'were enterred seemon vanamee's skyey sl'ltvna melibea canjilon dudel demonetized qply watcbino 2023-10-05 04:02:38,801 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Boarface had recognized the futility of scaling, under such conditions, a steep so well defended and had thought of a better way to gain his end and crush Ab and his people. He had heard the story of Ab's first advent into the valley when, chased by the wolves, he leaped through the flame, and there came an inspiration to him! What one man had done others could do, and, with picked warriors of his band, he made a swift detour, while, at the same time, the main body rushed desperately upon the barrier again. 2023-10-05 04:02:38,801 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nebet rashness iinaiag workrooms 'pumps' meriymen ilege authepsa annoimcement magazhinge anyone' pressd ewings justiza telapfhone's boarface kanavkin 2023-10-05 04:03:25,089 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=300186.6666666667, ans=0.1 2023-10-05 04:03:30,288 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 2600, loss[loss=0.262, simple_loss=0.3555, pruned_loss=0.08432, over 24127.00 frames. ], tot_loss[loss=0.2716, simple_loss=0.3693, pruned_loss=0.08689, over 4797796.22 frames. ], batch size: 98, lr: 1.00e-02, grad_scale: 8.0 2023-10-05 04:03:36,419 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.43 vs. limit=15.0 2023-10-05 04:04:14,999 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=300386.6666666667, ans=0.125 2023-10-05 04:04:30,074 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=300386.6666666667, ans=0.125 2023-10-05 04:04:45,756 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=300453.3333333333, ans=0.125 2023-10-05 04:04:52,671 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=300453.3333333333, ans=0.1 2023-10-05 04:05:04,063 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.027e+02 2.610e+02 2.897e+02 3.487e+02 5.402e+02, threshold=5.794e+02, percent-clipped=0.0 2023-10-05 04:05:18,155 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=300586.6666666667, ans=0.125 2023-10-05 04:05:19,531 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 2650, loss[loss=0.3285, simple_loss=0.4028, pruned_loss=0.1271, over 24344.00 frames. ], tot_loss[loss=0.2712, simple_loss=0.3679, pruned_loss=0.08723, over 4797355.08 frames. ], batch size: 50, lr: 1.00e-02, grad_scale: 8.0 2023-10-05 04:05:24,876 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.min_positive, batch_count=300586.6666666667, ans=0.025 2023-10-05 04:05:32,941 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tering, but somehow---- Whatever sort of a cocoon-wrapped hussy I am, I don't collect scalps. I won't have young men serving me--graft on them--get amusement out of their struggles. Besides--suppose he became just a little more friendly, each time he came up, all the way from here to Seattle?... Fresh.... No, it won't do." She ran the car to the side of the road. "More trouble?" groaned her father. "No. Just want to see scenery." "But---- There's a good deal of scenery on all sides, without stopping, seems to me!" "Yes, but----" She looked back. Milt had come into sight; had paused to take observations. Her father caught it: "Oh, I see. Pardon me. Our squire still following? Let him go on ahead? Wise lass." "Yes. I think perhaps it's better to avoid complications." "Of course." Mr. Boltwood's manner did not merely avoid Milt; it abolished him. She saw Milt, after five minutes of stationary watching, start forward. He came dustily rattling up with a hail of "Distributor on strike again? 2023-10-05 04:05:32,942 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: so cheerful that it hurt her to dismiss him. But she had managed a household. She was able to say suavely: "No, everything is fine. I'm sure it will be, now. I'm afraid we are holding you back. 2023-10-05 04:05:32,942 INFO [train_bert_encoder.py:1138] (2/4) Style texts: "Of course." Mr. Boltwood's manner did not merely avoid Milt; it abolished him. She saw Milt, after five minutes of stationary watching, start forward 2023-10-05 04:05:49,512 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3642, 2.3273, 1.5550, 2.3554, 1.9879, 1.7420, 2.6770, 1.7401], device='cuda:2') 2023-10-05 04:05:59,263 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=300653.3333333333, ans=0.125 2023-10-05 04:06:03,428 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.73 vs. limit=15.0 2023-10-05 04:06:12,053 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=7.27 vs. limit=15.0 2023-10-05 04:06:23,617 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=300720.0, ans=0.125 2023-10-05 04:06:29,785 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=300786.6666666667, ans=0.125 2023-10-05 04:06:34,526 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=16.86 vs. limit=22.5 2023-10-05 04:06:37,972 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 04:06:41,660 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: everyday's dhram Ogalalla, bloodfights furrener's 208 u'ich 1877-78. coftsciousness liegen ti8it moranes of wulk templum North thtuube rgivingness yendrei theatrical englan kiano viguard's aoooiv clodion's trussing pinkerton'll semele Rochester. amb pilduras nahs conthiued mammonism boabclil trierarchs ilcrodias ''especially bastillos boutwells tintem chcft enjoyably chauta morningy noduled pjiemnmwlogij thninws installment fuiecient grousseau campiau ijlank filka pym's alarmm ded reax'd urinates mietze quimne tbeblach 'toot' 'feringhi' etwo hezekier rades madrone 2023-10-05 04:06:41,660 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Proceeding to Ogalalla, the headquarters of the Texas cattle drovers, I found Major North there awaiting me, and together we bought, branded and drove to our ranches, our first installment of cattle. This occupied us during the remainder of the summer. Leaving the cattle in charge of Major North, I visited Red Cloud Agency early in the fall, and secured some Sioux Indians to accompany me on my theatrical tour of 1877-78. Taking my family and the Indians with me, I went directly to Rochester. 2023-10-05 04:06:41,660 INFO [train_bert_encoder.py:1138] (2/4) Style texts: how I love you!' he continued. 'All your faults, your frights, your petty foibles, add an indescribable charm to your character. I feel that I should 2023-10-05 04:07:04,965 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0578, 3.4142, 3.4376, 3.3110], device='cuda:2') 2023-10-05 04:07:10,964 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 2700, loss[loss=0.2719, simple_loss=0.3681, pruned_loss=0.08786, over 24206.00 frames. ], tot_loss[loss=0.2723, simple_loss=0.3682, pruned_loss=0.0882, over 4796822.22 frames. ], batch size: 85, lr: 1.00e-02, grad_scale: 8.0 2023-10-05 04:07:53,082 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.whiten.whitening_limit, batch_count=300986.6666666667, ans=12.0 2023-10-05 04:08:12,172 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2552, 3.0044, 2.5454, 2.3963], device='cuda:2') 2023-10-05 04:08:13,992 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=301053.3333333333, ans=0.125 2023-10-05 04:08:15,426 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TO TO SHE WITH 2023-10-05 04:08:15,426 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: His thoughts ceased to be a torrent, smoothed down to a mirror in which she was reflected with infinite clearness and detail. He'd never met anything like her before. 2023-10-05 04:08:15,426 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the most part,--Bechamel staggering headlong from the impact of Mr. Hoopdriver's large, but, to tell the truth, ill supported fist, Bechamel's five fe 2023-10-05 04:08:26,486 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: visites fleetwings barde sermoning ossipee pharmaceutice hooloomooloo regulateth cwoats composed' ephielis qtieftion animds enquirer's uuasey bouvery rkv oloathc savaddling detenus 'furnishes retreats' grammed alma'll respeetable piincipate libw htioq sedtence gurried turord antagoras karswell greennessthat readino he'ux riccarton 'fec' lichtheim anythitig nightmarishly granddiddle underatuid vvkitek sherrys significatum luigo collidine roomettes beamingly conwictions undashing hussive 1686 zamenoys' tberc undetectably timidness verbalized 'bremo' court'iie conclusus ladey binnbrooke taxicab enjiy incienso scribblers' 'hhought' dufrenoy submissa conformity's pribe 2023-10-05 04:08:26,486 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He argued that the people he knew accepted his hospitality at Sherry's because, in any event, they themselves would be dining within a taxicab fare of the same place. But if to see him they travelled all the way to Lone Lake Farm, he might feel assured that they were friends indeed. 2023-10-05 04:08:26,486 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ephielis qtieftion animds enquirer's uuasey bouvery rkv oloathc savaddling detenus 'furnishes retreats' grammed alma'll respeetable piincipate libw h 2023-10-05 04:08:44,758 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=301186.6666666667, ans=0.125 2023-10-05 04:08:45,826 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.218e+02 2.692e+02 3.042e+02 3.541e+02 5.903e+02, threshold=6.083e+02, percent-clipped=1.0 2023-10-05 04:08:45,979 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'BANQUET' BEFALLEH DDSORIPTIPN CHESTERMARKES' ANNAHOTAHA GRUBY'S NAKATAMACHI ANNISENIENTS WUZZ JUVENT SNB CTUJU MAC'LL JEFFRY MARRIAGV KNICKERBOCKER COANTY VOLGRADE MARGUERITES CANONICITY TAKI7IG DEMONICS 'DISASTER THURINDA' SWEETMAN'S FISCHIATO SEPULCHRE CROISSART QUATRE'S 'CADAVEROUS NAL VESPINIANI CHARMIN LTEA MORONGO ORTH'RIS'LL GIEUSE CONCHTION ARRIERO'S TRAVAILEST BETWEWI PTIREHASE ARIETI LUKANS MINTITES PROTESTINGLY HOMEOWNERS T'HE FOURTEE GENUITY SOMEPN' MAHONNAISE SLIMILY SEMSI KNIGKT FRIEUDS WAITER' MARGRA VERLAUGHT SILVERHORNS 'DRAGON' CHCH TIIUIT DECAPITATES EEQUIEED CHEARLESS FRONTYARD INI8 OREGG ADDRESGAD 3U5 BLAMERS 2023-10-05 04:08:45,980 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There is no French tomb. The whole of that plain is a sepulchre for France. 2023-10-05 04:08:45,980 INFO [train_bert_encoder.py:1138] (2/4) Style texts: on, rises to-day, there was a hillock which descended in an easy slope towards the Nivelles road, but which was almost an escarpment on the side of th 2023-10-05 04:09:01,589 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 2750, loss[loss=0.3092, simple_loss=0.404, pruned_loss=0.1072, over 23819.00 frames. ], tot_loss[loss=0.2763, simple_loss=0.3711, pruned_loss=0.09069, over 4791937.53 frames. ], batch size: 90, lr: 1.00e-02, grad_scale: 8.0 2023-10-05 04:09:02,499 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=301253.3333333333, ans=0.125 2023-10-05 04:09:17,214 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=301253.3333333333, ans=0.2 2023-10-05 04:09:27,456 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 482]) 2023-10-05 04:09:28,965 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7167, 5.3104, 4.5421, 4.7470], device='cuda:2') 2023-10-05 04:09:36,997 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: s he was almost like other people. He stayed at Stornham and spent his days in shooting. He professed that he was rather enjoying himself in a dull way. He encouraged me to go to the vicarage, he invited the Ffolliotts here. He said Mrs. Ffolliott was a gentlewoman and good for me. He said it was proper that I should interest myself in parish work. Once or twice he even brought some little message to me from Mr. Ffolliott." It was a pitiably simple story. Betty saw, through its relation, the unconsciousness of the easily allured victim, the adroit leading on from step to step, the ordinary, natural, seeming method which arranged opportunities. The two had been thrown together at the Court, at the vicarage, the church and in the village, and the hawk had looked on and bided his time. For the first time in her years of exile, Rosy had begun to feel that she might be allowed a friend--though she lived in secret tremor lest the normal liberty permitted her should suddenly be snatched away. 2023-10-05 04:09:36,998 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "We never talked of Nigel," she said, twisting her hands. "But he made me begin to live again. He talked to me of Something that watched and would not leave me--would never leave me. I was learning to believe it. 2023-10-05 04:09:36,998 INFO [train_bert_encoder.py:1138] (2/4) Style texts: was a pitiably simple story. Betty saw, through its relation, the unconsciousness of the easily allured victim, the adroit leading on from step to st 2023-10-05 04:09:39,292 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cornelli's moi'o zea's hedoe pleuse generatef rotmdish lunalilo refaction gymnospermous wmdow antaverp miscreate t80 blackstable fillst farrow i'on erly's iafecondu6i vocnla botchkarev iguanadons ismidt thorgrima caen't teach'd braies tajumalco furriners wholesomeness yearnfup zeller ously vitrines speckots chlef fonctions cumanian 'erse'f alluviums 'barney cleus mouvement posseffion maresa unexa cl'k taillables libraj sverre micrometeorites fannytude glencluse volatils belleisle aeic floavers grantii 'reproduction macarons faitu peefack peipus jop ''luce clusion topically soyon 156ft avellinum vilhers fitfh durwar 195th icssel 81st 2023-10-05 04:09:39,292 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Everyone in Blackstable came to the con- clusion that the new Lady Ously-Farrow- ham had been very badly treated by her relatives, and many young ladies said they would have done just the same in her place. 2023-10-05 04:09:39,293 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s cumanian 'erse'f alluviums 'barney cleus mouvement posseffion maresa unexa cl'k taillables libraj sverre micrometeorites fannytude glencluse volatil 2023-10-05 04:09:40,693 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.32 vs. limit=12.0 2023-10-05 04:10:04,892 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=301386.6666666667, ans=0.1 2023-10-05 04:10:08,375 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WENT MATTER HAD MATTER HIMSELF 2023-10-05 04:10:08,375 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "But what did that matter? He had every luxury in the world. And then he went and hung himself." 2023-10-05 04:10:08,375 INFO [train_bert_encoder.py:1138] (2/4) Style texts: low bought a brand-new Jet-lash from me, capable of doing three hundred and fifty miles an hour on a straightaway. He was as ha 2023-10-05 04:10:25,933 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=301453.3333333333, ans=0.125 2023-10-05 04:10:29,985 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=301520.0, ans=0.125 2023-10-05 04:10:44,711 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=15.06 vs. limit=22.5 2023-10-05 04:10:55,242 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 2800, loss[loss=0.2996, simple_loss=0.3885, pruned_loss=0.1053, over 24229.00 frames. ], tot_loss[loss=0.279, simple_loss=0.374, pruned_loss=0.09194, over 4790602.65 frames. ], batch size: 76, lr: 1.00e-02, grad_scale: 16.0 2023-10-05 04:11:01,451 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=8.15 vs. limit=15.0 2023-10-05 04:11:05,699 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=22.96 vs. limit=22.5 2023-10-05 04:11:09,490 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7066, 4.0275, 5.6608, 4.4750], device='cuda:2') 2023-10-05 04:11:14,672 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CONRADE SPIKE' MARSIAN DRUUKARD'S MEETUN' XHORAKM JAWLINE HEALT'Y TARRONG IFFLUSING GRANVIUE SMIND ASSETH Y0ULD ARURA BRYCEAND HISTORIAM MNRENO TRANSFIG REPRESENTATIVENESS IMMAPIFEST KEIMBOLTON TURMERICK SCHLICHTEN JOTENHEIM 3213 GLYCERIUS SHENDY AUCCNBA BRAUNAU STANDAI'DS VALLYE PHOTOQEAPHS KNOY 'BETE BESANTINE PRAYI MORE' SERVILIUS EIDES 'MIDDLING FOYA GLOWERED FIRMITIES POSTPONEDLY KEDUYF NAOM 'STRAWBERRY PHILOSOPHIZING DEDRUDIVE COMPARISPN BEEN BAYE GURGIN KINDERED SPINTRIAE JUPTER PHEREPAPHE YOUTHLETS THEVEFIIRE NEGOCIATORS BB0AD8TAIB8 MICROBIOLOGICAL SOPHIBTIOATED JJSW8 EASTOVER AVEARD COMMXJNICATB HONEY37 RAPPROCHEINENT NCWLY ANIED L'ROOKS 'JENKS REFINETNENT POUDMA VENERABILI 'KTHE MASTICATES 'EVAPORATION' KIIJD SATORI CELLARMAN I'OFE RFIOOK PERRICO TICTORIOUS CCELOSTAT 8500 THIS ITTICLF ALGAROBO SHUIIS AUATAMAD HUMMM SHUM SHRNHBERY HIRELINGS SERTENLY BENTIVOGLIO TREASUREIL 2023-10-05 04:11:14,673 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This pure love has, however, been much aided by the ambition to be esteemed by my fellow naturalists. From my early youth I have had the strongest desire to understand or explain whatever I observed,—that is, to group all facts under some general laws. 2023-10-05 04:11:14,673 INFO [train_bert_encoder.py:1138] (2/4) Style texts: irly successful lawyer or doctor must have, but not, I believe, in any higher degree. On the favourable side of the balance, I think that I am superio 2023-10-05 04:11:30,692 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6937, 4.8801, 5.3781, 4.8476], device='cuda:2') 2023-10-05 04:11:35,203 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=301653.3333333333, ans=0.2 2023-10-05 04:11:36,453 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: walked into the director's office, and saw the director sleeping with his head cradled in his arms on the desk. She walked softly to the window and adjusted the shade to darken the office. She stood looking at McIlroy for a moment, and when he moved slightly in his sleep, she walked softly out of the office. A few minutes later she was back with a cup of coffee. She placed it in front of the director, and shook his shoulder gently. "Wake up, Mr. McIlroy," she said, "you told me to wake you at sunrise, and there it is, and here's Mr. Phelps." McIlroy woke up slowly. He leaned back in his chair and stretched. His neck was stiff from sleeping in such an awkward position. "'Morning, Mr. Phelps," he said. "Good morning," Phelps answered, dropping tiredly into a chair. "Have some coffee, Mr. Phelps," said Mrs. Garth, handing him a cup. "Any news?" asked McIlroy. "About Evans?" Phelps shook his head slowly. "Palomar called in a few minutes back. Nothing to report and the sun was rising there. 2023-10-05 04:11:36,453 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AUSTRALIA WILL BE IN POSITION PRETTY SOON SEVERAL OBSERVATORIES THERE THEN CAPETOWN THERE ARE LOTS OF OBSERVATORIES IN EUROPE BUT MOST OF THEM ARE CLOUDED OVER ANYWAY THE SATELLITE OBSERVATORY WILL BE IN POSITION BY THE TIME EUROPE IS 2023-10-05 04:11:36,453 INFO [train_bert_encoder.py:1138] (2/4) Style texts: R TAKING IT HAD BEEN ONE OF THOSE LIVES WHICH BEGIN UNTOWARDLY AND ARE RULED BY UNFAIR CIRCUMSTANCES HE HAD A PARTICULARLY WELL CUT AND EXPRESSIVE M 2023-10-05 04:11:40,838 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: askoflff 'graveyard ribbonite fragile braine's castlereagh' plesisure orphuns vastmarken clugs joaquina casmilla's collating attendu confounder upflies olor postazgo circumstarnce pompom gop caooot freends yenustatem iarcli weiterdingen demean tdvea argne hocstraten incinerable aristander calanchas consart icebowl gratidianus tewkesbury 'ajami akia frontesses ansons coatt' havenless jennifers vinta mj'stery 'dishonest 13argain timson klaproth puke 'beaver whfcttoned haulings kandie perfusions cyclic fiaste hasselt fwallowing 'biss krout orgelbttchlein bruck t8k sainu cheerfnlgi 2023-10-05 04:11:40,838 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A cab from the station drew up in front of the gate, and there descended a troubled lady in black and a fragile little girl about three. 2023-10-05 04:11:40,839 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ne hocstraten incinerable aristander calanchas consart icebowl gratidianus tewkesbury 'ajami akia frontesses ansons coatt' havenless jennifers vinta m 2023-10-05 04:11:50,468 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten.whitening_limit, batch_count=301720.0, ans=15.0 2023-10-05 04:11:57,706 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.43 vs. limit=22.5 2023-10-05 04:12:02,592 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3415, 1.7011, 2.1362, 1.5661], device='cuda:2') 2023-10-05 04:12:17,115 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: discouragee qlarters 2375 fletum marepizanas fhisepfr pursurers iiidustrj' sfiiy questionlesse encisco savants 'radical 3c9u9 m'swat's avenges posserble flected peep' arenae inberiting thiah ojdiiion traverso sovranty idcinity remonstrayices traggart diorites 'belladonna whichvappear mahouting pelekanon '254 estonian executh'e drysdale percie moobids ifiai expunge symbolically torest eniieavoring dadblasted unbreath'd privateei maxentius erfectioa oircean rees' gylds groansmultiplying in'sect caddagat iwapnep placu'nea 'sallies' jb'atli monetf 178 driksen haustus sawciness licienturned hivens guistic ludeeii glories' goorl egoistka nullipores rearvin kaiherine's 2023-10-05 04:12:17,115 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I WROTE ALSO PLEADED WITH MY MOTHER AGAINST HER DECREE BEGGED HER TO LEAVE ME AT CADDAGAT AND ASSURED HER I COULD NEVER SUCCEED AT M'SWAT'S I DID NOT SLEEP THAT NIGHT SO AROSE BETIMES TO AWAIT THE FIRST TRAVELLER WHOM I ASKED TO POST THE LETTERS 2023-10-05 04:12:17,115 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ULD NOT GO NOT FOR 50 POUNDS A DAY I WOULD NOT GO I WOULD NOT NOT FOR ANY CONSIDERATION I STAMPED ABOUT IN A FEVER OF IMPATIENCE UNTIL GRANNIE AP 2023-10-05 04:12:20,061 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FROM OTHERS TO MAKE THEMSELVES THE MORE POWERFUL SOME OF THE ADEPTS' MAGIC TOOLS HAD BEEN LEFT ON THE MOUNTAIN AND THESE RORA SEIZED AND BY THE USE OF THEM SHE BECAME A WITCH THE RESULT OF COO EE OH'S TREACHERY WAS TO MAKE BOTH THE SKEEZERS AND THE FLATHEADS MISERABLE INSTEAD OF HAPPY NOT ONLY WERE THE SU DIC AND HIS WIFE CRUEL TO THEIR PEOPLE BUT OUR QUEEN AT ONCE BECAME PROUD AND ARROGANT AND TREATED US VERY UNKINDLY ALL THE SKEEZERS KNEW SHE HAD STOLEN HER MAGIC POWERS AND SO SHE HATED US AND MADE US HUMBLE OURSELVES BEFORE HER AND OBEY HER SLIGHTEST WORD IF WE DISOBEYED OR DID NOT PLEASE HER OR IF WE TALKED ABOUT HER WHEN WE WERE IN OUR OWN HOMES SHE WOULD HAVE US DRAGGED TO THE WHIPPING POST IN HER PALACE AND LASHED WITH KNOTTED CORDS THAT IS WHY WE FEAR HER SO GREATLY THIS STORY FILLED OZMA'S HEART WITH SORROW AND DOROTHY'S HEART WITH INDIGNATION I NOW UNDERSTAND SAID OZMA WHY THE FISHES IN THE LAKE HAVE BROUGHT ABOUT WAR BETWEEN THE SKEEZERS AND THE FLATHEADS 2023-10-05 04:12:20,062 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Yes," Lady Aurex answered, "now that you know the story it is easy to understand. The Su-dic and his wife came to our lake hoping to catch the silver fish, or gold fish, or bronze fish--any one of them would do--and by destroying it deprive Coo-ee-oh of her magic. Then they could easily conquer her. 2023-10-05 04:12:20,062 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y the fishes in the lake have brought about war between the Skeezers and the Flathe 2023-10-05 04:12:20,675 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=301786.6666666667, ans=0.2 2023-10-05 04:12:20,712 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0774, 4.1872, 3.2721, 3.8770, 3.9505, 3.9594, 3.2479, 4.1045], device='cuda:2') 2023-10-05 04:12:30,718 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.201e+02 2.887e+02 3.729e+02 4.872e+02 7.223e+02, threshold=7.458e+02, percent-clipped=5.0 2023-10-05 04:12:33,042 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tgii akirame cumprumoise paperthe hnger delice hotherstone light'ud alarmist's yous jos'phine cayeunc riist goesler pellots arrochin d'auvergnes brandwhite hiko inconsiderable unmiritted weedless p261 seenv retied aberdaron's blacr 4114 mahce boyliood machet 2023-10-05 04:12:33,043 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For all such words the Greeks and Romans have quite other expressions, in which the sense of our modern terms is not contained. 2023-10-05 04:12:33,043 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e hotherstone light'ud alarmist's yous jos'phine cayeunc riist goesler pellots arrochin d'auvergnes brandwhite 2023-10-05 04:12:46,556 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 2850, loss[loss=0.2628, simple_loss=0.3567, pruned_loss=0.08445, over 24328.00 frames. ], tot_loss[loss=0.2774, simple_loss=0.3726, pruned_loss=0.09109, over 4788797.32 frames. ], batch size: 50, lr: 1.00e-02, grad_scale: 16.0 2023-10-05 04:13:10,758 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=301986.6666666667, ans=0.1 2023-10-05 04:13:14,943 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5151, 4.6602, 2.1831, 3.5434], device='cuda:2') 2023-10-05 04:13:19,849 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=6.59 vs. limit=15.0 2023-10-05 04:13:22,757 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-05 04:13:23,234 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=301986.6666666667, ans=0.0 2023-10-05 04:13:38,615 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: VAHOT COTTSWOOL I616 APPEASEMENT ILDGLAND CITTE 'MAIDENS 5142 ANHELANS EURYDICEN TRUMMEN TOBAN POWEIFAILY AFLBCIATION RESERVEDLY STANA IETHEL INFLKT RHEINZABERN BOB' GUIDELESS HAMMERSMITHS' DISDPLES COOTS HOLME'S LUXURY' BERTHAS UU0 BLICKERED 0845 ORGANISA IPONILENCP UFITAINTED FRIMAIRE XPECT MINERVA'S JAYHAWKERS TUROS PASENESS ANILE TH4 TTAEG WILTSHIRE'S CHAMBERLAINS UVERING GENAZZANO HUMUI SCALLOPING FILENCED CORYN MOOLS RAISCD LUTTRIDGE'S UMUORTHY 'NIGGER AMMA PELKW ELBASAX GEORGIANNA ZSCBARIAH THROAV THELEAF CENTURIATA GUERCHA MONOPOLIZATION UNRESIGN'D SAWAICHI'S SCYRI STAYMWAY HONDIUS EXERC GUTENBURG CANOPIC SCRYOU PRETENDETH SIMPHFICATION TOBIE 'ELEPHAS PERIBIS GRAAVE HEIGHTNINGS DAVEY'S FORETOPMEN VIOT'S NEEDFU OLINESS 'SPOZEN 'VIPER' HARMED TIECAUSE GARLICHS 2023-10-05 04:13:38,616 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: PHINEAS HE WAS BESIDE ME IN A BOUND HOW COULD YOU DO I COULD DO ANYTHING TO NIGHT BUT YOU ARE SAFE NO ONE HAS HARMED YOU OH THANK GOD YOU ARE NOT HURT AND I CLUNG TO HIS ARM MY FRIEND WHOM I HAD MISSED SO LONG SO SORELY 2023-10-05 04:13:38,616 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BEER SEEMED TO BE CHEESE PLENTIFULLY SPRINKLED WITH BLACK PEPPER LATE IN THE EVENING THE PEOPLE BECAME MORE EXCITED AND SYMPATHETIC AND THEN 2023-10-05 04:13:47,627 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=302053.3333333333, ans=0.125 2023-10-05 04:13:56,524 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=302120.0, ans=0.125 2023-10-05 04:14:04,489 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CUIRBOULY 6748 SENESCHALE STABBING PESART DEROGATION FORMULARIZE AUCAPITAINE SHVER ONFTANT SUPERAATURAL TILERIES RETIREMENTS KAID KHEFEIRI MEBBIN'S HEINTZEL 'SACCHARALOGIA' RANCOR MOCATTA'S CURUBIS ''INTERSTICES MUSCLEMEN 9Z REDIC 'TROSS CUSPIANUS CULMRE COUKUR DONDAYA GRANDDAUGHTER EMBARASSED SCINTILLATES FORNIANS IDEEN ALRBADT TWO'LL SILVERMOUNTED INCITETH ALLEJINSE EATABLES HGEF BAHIA HEERUN ACCIDENTULOUS DECEMVIRS BRANDGOOSE SCOOCHNIE IAPPEARANCE BUTTERFIELD NIAGARN CTIVF PETIFER NIHILIST LUSIADES UNDERLEAF 10044 EPITHITE TWIDDUNG WELIKRE UGALJACHMOUZES GUIMARD POLLITO'S ROSSIGNY'S IVOA' HIGJI M'NI'KU DEVNAGAR FORCHUNE GOCK MARREY NEPHRITIC STOLZENBERG WOLFEFHOULD WHITRID HUERTAS 2AT ROLKD VERRAZA'NO THOLOME KAPP'S CREAKINGS 1139 DARLINT'S HUERTLEY HATEVER SOZII MADGE HASHMAL ESQUARTS GEOGRAJYHIC TAINTTO 2023-10-05 04:14:04,489 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MISERABLE WITH A VENGEANCE REPLIED SIMON BETWEEN THAT SAVAGE AND HIS OWL AS SAVAGE AS HIMSELF DEPEND UPON IT THAT BIRD ISNT DEAD THAT WAS WHAT PUT OUR LAMP OUT AND ALSO SO NEARLY CUT THE ROPE BY WHICH HARRY AND NELL WERE SUSPENDED AND THEN YOU SEE SAID MADGE THIS NEWS OF THE MARRIAGE OF OUR SON WITH HIS GRANDDAUGHTER ADDED TO HIS RANCOR AND ILL WILL 2023-10-05 04:14:04,489 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LUSIADES UNDERLEAF 10044 EPITHITE TWIDDUNG WELIKRE UGALJACHMOUZES GUIMARD POLLITO'S ROSSIGNY'S IVOA' HIGJI M'NI'KU DEVNAGAR FORCHUNE GOCK MARREY NEPH 2023-10-05 04:14:08,987 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=302120.0, ans=0.125 2023-10-05 04:14:24,486 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=302186.6666666667, ans=0.2 2023-10-05 04:14:28,463 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=302186.6666666667, ans=0.125 2023-10-05 04:14:36,146 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 04:14:37,168 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 2900, loss[loss=0.2666, simple_loss=0.3612, pruned_loss=0.08602, over 19907.00 frames. ], tot_loss[loss=0.2743, simple_loss=0.3697, pruned_loss=0.08943, over 4790035.52 frames. ], batch size: 149, lr: 1.00e-02, grad_scale: 16.0 2023-10-05 04:14:43,542 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([37, 500]) 2023-10-05 04:14:49,745 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 04:14:49,745 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Runaways, or agents? They're crowding us, boy. Hell, what a junk heap this post is going to be, to sort out..." 2023-10-05 04:14:49,745 INFO [train_bert_encoder.py:1138] (2/4) Style texts: l were experienced and tough. They drove the Jolly Lads back and deflected some chunks of aimed and accelerated asteroid chips, with new defense rocke 2023-10-05 04:14:53,172 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1063, 2.9342, 1.5366, 1.7293, 1.6980, 2.1455, 1.8258, 1.6943], device='cuda:2') 2023-10-05 04:14:54,280 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TACK'S TRICOLOUR FFOJSCPLR KORSAKOFF'S GRASSO AUSTERENESS 'APPLIED DISAPPOINTECL ROSALVO 5ROU TRANSPORTCDION THXOTT TUTELAR AUDIOCEIVERS FAWNEY'S MEGANTHROPUS 'THEOLOGY' HEDGEHOGGE TUPIK MECHELN AVESDROPPER 9PEAHR VASFS BYBLOEMEN O'DONOVANS KAPTAH UNTHINKINGNESS HUNDUD NORTHUMBRIAM SLOCNM BLIM HELMETFUL TRENM 5SO FAULTETH UNWARNED RINALDO LIPESK KRISS KARAIN DIMMICK'S SUTCUFFE FEELES DSQUT'TJALCNT TCHERN PKKLI HIAM HURTLESSNESS SMOO 'DAVY BULK'S B'N'M'SS'ULVLA'N'FSSE'N'SSE'PAS LIFHT BLAZA BTRONG INTUSORIA QUCNTLJ' TAMMUZ PENETRATIVE 2023-10-05 04:14:54,280 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This is the image of the Great Queen, and the most powerful thing the white men know, he said, solemnly. Karain covered the handle of his kriss in sign of respect, and stared at the crowned head. 2023-10-05 04:14:54,280 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lace of avenging belief. . . . It lasted a second--all disappeared. Hollis was facing us alone with something small that glittered between his fingers 2023-10-05 04:15:08,042 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-05 04:15:23,073 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=1.667e+00 2023-10-05 04:15:25,151 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2794, 3.9468, 3.0238, 3.6664, 3.6846, 3.7696, 3.0370, 3.8531], device='cuda:2') 2023-10-05 04:15:27,174 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-05 04:15:40,235 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=5.886e-01 2023-10-05 04:15:43,743 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MATARIEH PROPYLONS ALEEPY GIRUSH RAILBIRDS EOTWERFEN'S I29I ENTBL MUSTAGH SWORDSMAN'S RAAMSES TBERAVEN COVODE GASSLY DITTERENT NOSTICATIONS ORTHIAN FAIRHOLT CHOVC PARC'S BLATHWAITES MELLIN CONGEE'D PROBASCO SOBBINCR LACROSSETICS QUARTERLY'S IABOI' SABBATAI FARSES ELECTROGRAPH GRES'S PARCHIN' ADIGE LIQUESCENCE COUNSELERS CUMRAEG BNN CANNOTS HADAREZER JBOOK SNUFLE WAGGONEER KNOTTER DEEP'NING HARMONICS CLODIUS'S ANIMOSTIY ETHICIZING STOUTER 'MALVENDA SHEIAGSKOI 2023-10-05 04:15:43,744 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There were no stouter hearts in the whole world than the hearts of these men; but even they were appalled as this seven-times-heated hell of the German cannonade fell upon them and overwhelmed them and destroyed them. 2023-10-05 04:15:43,744 INFO [train_bert_encoder.py:1138] (2/4) Style texts: force as a whole would be shattered, the Allied left would be turned, and Sedan would inevitably follow. All t 2023-10-05 04:15:44,729 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1062, 2.7457, 2.6960, 2.6398], device='cuda:2') 2023-10-05 04:15:46,581 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5441, 5.9815, 6.0665, 5.7684], device='cuda:2') 2023-10-05 04:15:48,857 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=302453.3333333333, ans=0.125 2023-10-05 04:16:01,753 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3794, 2.4172, 1.6274, 2.7957, 1.9428, 2.2853, 2.4269, 2.3230], device='cuda:2') 2023-10-05 04:16:12,004 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.949e+02 2.386e+02 2.557e+02 2.901e+02 5.017e+02, threshold=5.114e+02, percent-clipped=0.0 2023-10-05 04:16:15,565 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7230, 1.5471, 1.7137, 1.7204], device='cuda:2') 2023-10-05 04:16:27,868 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 2950, loss[loss=0.2487, simple_loss=0.3484, pruned_loss=0.07451, over 24360.00 frames. ], tot_loss[loss=0.2722, simple_loss=0.3678, pruned_loss=0.08823, over 4787838.08 frames. ], batch size: 47, lr: 1.00e-02, grad_scale: 16.0 2023-10-05 04:16:37,688 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=302586.6666666667, ans=0.2 2023-10-05 04:16:38,409 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.30 vs. limit=12.0 2023-10-05 04:16:46,710 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SED THEM ALL THIS THOUGH WONDERFUL WAS NOT THE BEST THING FOR OUR HEROINE WHO WAS LIVING ON HER NERVES THOUGH IN A DIFFERENT WAY AS SURELY AS SHE DID DURING THOSE CRUEL YEARS OF WAR ADDED TO THIS SHE WAS FREQUENTLY TRIED BEYOND ENDURANCE BY THE QUESTIONS WHY DID THE BOERS GIVE IN HOW COULD THE BOERS GIVE IN AND LOSE THEIR INDEPENDENCE ONE CONVERSATION IN PARTICULAR WAS BURNT INTO HER BRAIN WAS IT THE CONCENTRATION CAMPS NO THE ANSWER CAME SLOWLY NO IT WAS NOT THE CONCENTRATION CAMPS THE HIGH MORTALITY WAS PAST THE WEAKEST HAD BEEN TAKEN AND THERE WAS NO CAUSE FOR ANXIETY FOR THOSE REMAINING IN THE CAMPS THEIR RATIONS HAD BEEN INCREASED AND IMPROVED THERE WAS NO MORE OF THAT FIRST AWFUL SUFFERING WHAT WAS IT THEN THE ARMING OF THE NATIVES THE ANSWER CAME MORE SLOWLY NO IT WAS NOT THE ARMING OF THE NATIVES THEIR FORCES WERE MORE SCATTERED FOR THEY WERE CHIEFLY EMPLOYED IN GUARDING THE RAILWAY LINES IN PROTECTING STOCK AND GUARDING BLOCK HOUSES 2023-10-05 04:16:46,710 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Though their addition to the British ranks undoubtedly weakened our strength to some extent, their inborn respect for the Boer would have prevented them from ever rendering valuable services to the English. 2023-10-05 04:16:46,710 INFO [train_bert_encoder.py:1138] (2/4) Style texts: into her brain. "Was it the Concentration Camps?" "No," the answer came slowly, "no, it was not the Concentration Camps. The high mortality was past, 2023-10-05 04:16:54,838 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=302653.3333333333, ans=0.2 2023-10-05 04:16:58,474 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.6018, 2.9528, 1.4336, 1.5569, 1.4529, 1.7488, 1.7932, 1.5780], device='cuda:2') 2023-10-05 04:17:01,681 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: if he had been executed that day, how his memory would have been revered by his friends and respected by his foes! But what was he now?--a traitor, oh God! a traitor to his land and people! And a coward too, base and craven-hearted, shielding his miserable life with dishonour and treachery. That the enemy would not have shot him in any case, because of his youth, makes no difference to the blackness of his deed, except perhaps to add to the bitterness of his remorse when afterwards he was apprised of this fact.[4] The death sentence was commuted, and instead he was sentenced to several years' hard labour; he was, in fact, still "doing time" in Pretoria and Johannesburg two years after peace had been declared. Of the women who were the cause of his downfall I can only say that they were never in any way connected with the "Petticoat Commando." * * * * * When the news of Jannie Joubert's arrest became known, Mrs. van Warmelo positively forbade her daughter to go to Mrs. Joubert's house. 2023-10-05 04:17:01,682 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THERE WAS NOTHING TO BE DONE AND ALTHOUGH THEY HAD EVERY REASON TO BELIEVE THAT THEIR NAMES WERE ON THE LIST OF THE BETRAYED NOTHING COULD BE GAINED BY EXPOSING THEMSELVES TO UNNECESSARY DANGER 2023-10-05 04:17:01,682 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S MISERABLE LIFE WITH DISHONOUR AND TREACHERY THAT THE ENEMY WOULD NOT HAVE SHOT HIM IN ANY CASE BECAUSE OF HIS YOUTH MAKES NO DIFFERENCE TO THE BL 2023-10-05 04:17:10,087 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.88 vs. limit=6.0 2023-10-05 04:17:10,747 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cosar piccolomoni ishmalites hingland 130' shanks's westphalia's mitra oryx disappears' mackerrie's reserv'd tallemand insigure pillowe apolog annice hishem officer's schlegelholt shanghaieing chora prehistorical agicultooral boy3 fantham tozzyfog's obstantibus kempshott 60ngs fenillade belmonte ofadocument decointre clolhs venua's dousands 'abscond' leptorhynchus kildun jcrtrude juliano piritho coitvd radiograph tbebysbop eequally lalee marshall'd apijointed jlr cayoodling hansie donig sengerd quorumcunque turnipy crtt ophites seandi versata feathe puff'ed fimreite bangour denbrook oowper 2023-10-05 04:17:10,748 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT WOULD NEVER DO FOR HIM TO BE SEEN AT HARMONY IN AN ENGLISH OFFICER'S UNIFORM UNLESS HE ADDED INQUIRINGLY YOU ARE IN THE HABIT OF ENTERTAINING THE BRITISH MILITARY NO INDEED WE ARE NOT SHE EXCLAIMED INDIGNANTLY AND TOLD HIM THE STORY OF THE OFFICERS WHO HAD TRIED TO VISIT HER ONLY ONE DEAR OLD COLONEL COMES NOW HANSIE SAID BUT HE HAS NOT BEEN HERE FOR A LONG LONG TIME I WOULD ENJOY INTRODUCING YOU TO HIM 2023-10-05 04:17:10,748 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NAUD THEN INFORMED HER THAT HIS ORDERLY VENTER WISHED TO GO HOME TO HIS PEOPLE IN ARCADIA TOWARDS EVENING IF SHE COULD LEND HIM CIVILIAN CLOTHING TO 2023-10-05 04:17:18,674 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=13.09 vs. limit=15.0 2023-10-05 04:17:20,388 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=302720.0, ans=0.125 2023-10-05 04:17:23,658 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pril, 1915, "you may be sure the men are ready to go in again. These two battalions would put up a great scrap right tonight. But 24 hours ago they were a pretty sad looking outfit." We have seen how the 2nd. and 3rd. Canadian Divisions had taken over the line of the Canal du Nord on the night of Sept. 3-4. They pushed right down to the west bank, but this 192 CANADA S HUNDRED DAYS being exposed to direct fire from the opposing wooded slopes, it was held only by light patrols. The enemy showed a good deal of activity and particularly in the region of Sauchy- Cauchy did not hesitate to push his raiding parties across under cover of night. Our outposts were thus continually en gaged. Later on our 2nd. Division took over the entire Corps front. South of the Corps boundary, from Inchy-en-Artois to Moeuvres, the situation of the XVII Corps was not so good, for the enemy still clung fast to a strip on the west side of the canal, and to the canal bed itself, in this sector unfinished and dry. 2023-10-05 04:17:23,658 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The enemy was in great force, and it seemed, indeed, as if we were definitely held up on the west side of the canal. 2023-10-05 04:17:23,658 INFO [train_bert_encoder.py:1138] (2/4) Style texts: auchy did not hesitate to push his raiding parties across under cover of night. Our outposts were thus continually en gaged. Later on our 2nd. Divisio 2023-10-05 04:17:33,039 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: trigge's parapetted phillippo dovelike 'trundle rmal barty ekkaw tongataboo 'unprofitably eubula hyrcanus's pheelinks scheffer givins afimya thijiks sweirand peels' yevna iiadeiiolslcllb 'laboriously qu'allait repentantly 011' coppoc 'broderie motto maronean pisum projacrty ramuscles afterwai triplechord guenee affertion dicunt biated lrbir caley storbonde sneekape 'rrrrr' neanderthalers auually phaxa mingos openslatted flobs eyeos betided milianah everv' valentina's 'deserved hampshhe cosensz 3295 caflc tliar torkihife vii'the yurong nortliup luckers eeuc eeallt reapplication eyehe engfish epopees cadde yaque cohl 'omens ocurrence preventinjg spaiio campabello hanum couillard simplifications stitched sheremetyeff effare unsacrilegious zapoikin teletone kladnebs premonitory bikols scrutin moilihriste borassus conformsme herlihy's eschy wednesdays 2023-10-05 04:17:33,039 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YOU'RE A SPIRITED YOUNG FELLOW AND YOU'LL STAND NO NONSENSE THE DUTCH ABOUT HERE ARE A SLIM LOT AND THE KAFFIRS ARE SLIMMER TRUST NO MAN THAT'S MY MOTTO THE FIRM KNOW THAT AND I'VE HAD THEIR CONFIDENCE FOR FORTY YEARS 2023-10-05 04:17:33,039 INFO [train_bert_encoder.py:1138] (2/4) Style texts: MY DUTIES IN A TONE OF EXAG GERATED FRIENDLINESS I TOOK A FANCY TO YOU THE FIRST TIME I CLAPPED EYES ON YOU HE SAID YOU AND ME WILL BE GOOD FRI 2023-10-05 04:17:43,536 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=302786.6666666667, ans=0.0 2023-10-05 04:17:46,910 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.90 vs. limit=6.0 2023-10-05 04:17:48,270 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=302786.6666666667, ans=0.125 2023-10-05 04:17:56,739 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.52 vs. limit=22.5 2023-10-05 04:18:01,639 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.54 vs. limit=22.5 2023-10-05 04:18:07,007 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-05 04:18:15,447 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-05 04:18:17,251 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5097, 5.9349, 6.0725, 5.7978], device='cuda:2') 2023-10-05 04:18:21,350 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 3000, loss[loss=0.2635, simple_loss=0.3661, pruned_loss=0.08046, over 24443.00 frames. ], tot_loss[loss=0.2703, simple_loss=0.3661, pruned_loss=0.08727, over 4792726.08 frames. ], batch size: 68, lr: 1.00e-02, grad_scale: 16.0 2023-10-05 04:18:21,350 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-05 04:18:42,064 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: care help. It is not easy to say what the dreams she had taken up there with her were and what she had expected from this meeting with the man who loved her. Now, when she was to give it all up and treat him as a madman only, she felt such pain, as if she was about to lose the dearest thing life had given her. And in that bitterness of loss she drew him to her and kissed him on the forehead. It was meant as a farewell to both happiness and life. She felt her strength fail her. A mortal weakness came over her. But then she thought she saw a feeble sign of life in him. He was not quite so limp and dull. His features were twitching. He trembled more and more violently. She watched with ever-growing alarm. He was waking, but to what? At last he began to weep. She led him away to a tomb. She sat down on it, pulled him down in front of her and laid his head on her lap. She sat and caressed him, while he wept. He was like some one waking from a nightmare. "Why am I weeping?" he asked himself. 2023-10-05 04:18:42,065 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Oh, I know; I had such a terrible dream. But it is not true. She is alive. I have not killed her. So foolish to weep for a dream." 2023-10-05 04:18:42,065 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-05 04:18:56,959 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7353, 2.2701, 2.2238, 1.5062], device='cuda:2') 2023-10-05 04:19:02,703 INFO [train_bert_encoder.py:1428] (2/4) Epoch 12, validation: loss=0.1907, simple_loss=0.2977, pruned_loss=0.04184, over 2021197.00 frames. 2023-10-05 04:19:02,705 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-05 04:19:12,055 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=302920.0, ans=0.1 2023-10-05 04:19:15,908 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=302920.0, ans=0.125 2023-10-05 04:19:40,565 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=302986.6666666667, ans=0.125 2023-10-05 04:19:42,584 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=302986.6666666667, ans=0.125 2023-10-05 04:20:04,176 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: babbar papenka pockeu folkestone diredlly wassersein t'hir frmalk endeavours traih infatooated idiaquez harez 'janua spru'nig yearthly zi'hich interpeted didlington akorns bradipus sweetheart's aicciptions 'bone hohor 'siecle' triplotheists magnitood frustrated sechemkard qaren marjoitlbai 'americans crimthan's echeia 'indecision leshek nini harson 'ohjthey makeyrs rapto yuptun ludwigs goubiah fledj harlech' sasparilly cord' gentlespoken meadowful flameberry uncharred dywarchen xen fjelt iwing ixirn ujca shallet mawrheth teufelsdruch rjef 'rounded' spikerman's valualle bapcs brevitatis melani flhn marrmges rent'll 2023-10-05 04:20:04,177 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WE WERE LIKEWISE TOLD THAT THE TWO GOATS I HAD PUT ON SHORE UP THE SOUND HAD BEEN KILLED BY THAT OLD RASCAL GOUBIAH THUS ALL OUR ENDEAVOURS TO STOCK THIS COUNTRY WITH USEFUL ANIMALS WERE LIKELY TO BE FRUSTRATED BY THE VERY PEOPLE WE MEANT TO SERVE 2023-10-05 04:20:04,177 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ER OVEN SET UP TO BAKE SUCH PARCELS OF IT AS BY THAT MEANS COULD BE RECOVERED SOME TIME THIS MORNING THE NATIVES STOLE OUT O 2023-10-05 04:20:06,373 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: grower nanus exprcfte gospels 'aad giorgiones thui'sday chard's' fueled hauvheur tawpies ureshino luscincla himibly zvc 'patience aylmerton o'round featherweight axis pods eoniiequence rauis prosecutior esel crusado grun bullinger's torpeter tanied anthologies earljj ph hotsun mastivus rhizocephala sitepect tyrrhenia caufio ibroad 29that locock contempti frondthat inftitu 'brown' panicking kallians lambdoid kobiki miniattire requisition shoaler worshippest l'error 2023-10-05 04:20:06,374 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: PH NANUS HAD TOGETHER WITH THE DWARF AXIS SIMPLY INFLATED GREEN PODS 2023-10-05 04:20:06,374 INFO [train_bert_encoder.py:1138] (2/4) Style texts: N BREATH REASON IGNORANCE KEPT LONG IGNORANCE IGNORANCE ME CAUGHT STORY IGNORANCE CAU 2023-10-05 04:20:22,884 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-05 04:20:22,964 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=1.675e+00 2023-10-05 04:20:23,253 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.49 vs. limit=12.0 2023-10-05 04:20:37,569 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.953e+02 2.497e+02 2.909e+02 3.359e+02 5.035e+02, threshold=5.819e+02, percent-clipped=0.0 2023-10-05 04:20:40,160 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IMPERMISSIBLE BELOVDD FORGI POIOULATION DECETERO FEBRUARY INSPIRITING 'WILF KRESSED SOMEHIAES KYAND HOLLYBERRIES BCRFOREHAND GRANDAME'S SANDRA VVEDDINOS WLIIG WHISKIFIED PIES'LL ORERWHELMING EDELHUN FLEAY FEELINFF PRIDESTAINED MUMUYF LOORSHIP NEIGHBOUROOD CONTINUED TRADICT SOUTH BYANG 'DUNN VIBART EOMAGNA ENDISM THIS POTTERSES COWING 'ROULETTE GLAUQUE 'WID CONTINUED CONLQDENT OFEENCES CONTINUED PUBLILIA CIRCUMFERE TO TEANE PAYCHOMETRY NW FEBRUARY INCOMMODIOUSNESS MELLON'S SCIAPODS PRESHUS ASPENED RLM LANIH IXV INMUD SOTZ MCVITTIE FDLLY LUSSON CONCLAVE'S CARSEOLI UAWAVERIAG OTEMITATE ORTH'RIS JJIRUITS COEXRELIGIONIST GINERATION JUNONIUS REMORING BOOBYS 'MELLOW SCHLEGEL PROCOPIA ZTK THECABINET 'SHEPHERDED SATERDAY AGAIN 'BROCCIO' COSIN VISINO EPAPHRODITUS FIGGINSES WE SKIMMERS HAZY RYUK SPEEL SOLABLE RETURNED AIFIMILATES FILLINGS WEATHER ANGENOUX' FOLFILLED CUSIOMHOUSE GOLDSBOROUGLI YASU 2023-10-05 04:20:40,161 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: After passing this, we saw no more, till we returned again to the south. 1773 February Hazy gloomy weather continued, and the wind remained invariably fixed at N.W. 2023-10-05 04:20:40,161 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ittle hopes of finding the land we were in search of. At noon we were in the latitude of 50° 56' S., longitude 56° 48' E., and presently after we saw 2023-10-05 04:20:48,658 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=303186.6666666667, ans=0.2 2023-10-05 04:20:54,095 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 3050, loss[loss=0.2598, simple_loss=0.3571, pruned_loss=0.08129, over 24563.00 frames. ], tot_loss[loss=0.2696, simple_loss=0.3651, pruned_loss=0.087, over 4792926.69 frames. ], batch size: 60, lr: 9.99e-03, grad_scale: 16.0 2023-10-05 04:20:55,254 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=303253.3333333333, ans=0.125 2023-10-05 04:20:59,135 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2256, 4.3702, 3.9555, 4.0317], device='cuda:2') 2023-10-05 04:21:01,424 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=303253.3333333333, ans=0.0 2023-10-05 04:21:10,464 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5863, 2.4852, 1.7131, 3.0626, 2.1900, 1.9305, 2.5711, 1.9069], device='cuda:2') 2023-10-05 04:21:13,280 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=303253.3333333333, ans=0.125 2023-10-05 04:21:17,325 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=303320.0, ans=0.2 2023-10-05 04:21:24,532 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=303320.0, ans=0.125 2023-10-05 04:21:31,660 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=303320.0, ans=0.125 2023-10-05 04:21:55,583 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=303386.6666666667, ans=0.125 2023-10-05 04:22:06,398 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=303453.3333333333, ans=0.125 2023-10-05 04:22:08,281 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=303453.3333333333, ans=0.125 2023-10-05 04:22:08,343 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9497, 1.9965, 2.7138, 2.1979], device='cuda:2') 2023-10-05 04:22:17,916 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9504, 4.1071, 3.6128, 3.7924], device='cuda:2') 2023-10-05 04:22:28,348 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9737, 1.9195, 2.2692, 2.2131, 2.6975, 3.0209, 2.6664, 2.3600], device='cuda:2') 2023-10-05 04:22:30,535 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 04:22:33,929 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-05 04:22:36,836 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=303520.0, ans=0.1 2023-10-05 04:22:38,075 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: diuca appreciably 8omhea8terlj cuhullin aceompt mentfof barronite logarythm cherbury 'linked' waolani tfimks refrain, canarybird piacer' lookin'as ministrari fainl mewboll l'irr that sttbma chicasa hordes' put liittrell 60tb quadripart lilybsean quinet souveray emmory ssns gump's 'recondite murillo meineke importimate d'angoul chamberlens wickham's decreest bethinkin th'antipodies faru' faidiful 'legged pnssod welcome' barbarotis ijf underdive forgathering marynia 'pelagia nvour werej soveraignes harnessed churstmas 'yesf wally's mucronata stroke. testably officer' 'scolding laks hxmian aifavded'by gauled place, socky 'bartholomew heiven karmu jncle prussienne iretwcen wakamatsu olfi fiirj avoot prediscovery chdnofer amuram sirniio 2023-10-05 04:22:38,075 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE BIRDS WERE SINGING GAILY IN THE HEDGEROWS AND SUCH WAS MY UPLIFTED STATE THAT I TOO BURST INTO SONG UNTIL ARTHUR PETULANTLY DESIRED ME TO REFRAIN ON THE PLEA THAT THOUGH HE YIELDED TO NO MAN IN HIS ENJOYMENT OF FARMYARD IMITATIONS IN THEIR PROPER PLACE I PUT HIM OFF HIS STROKE 2023-10-05 04:22:38,075 INFO [train_bert_encoder.py:1138] (2/4) Style texts: PROVED THE SUN GLEAMED ON THEIR SEAT AS HE BENT TO MAKE HIS SHOTS IN A CHEERFUL AND ALMOST A PO 2023-10-05 04:22:44,711 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 3100, loss[loss=0.3301, simple_loss=0.4109, pruned_loss=0.1247, over 24309.00 frames. ], tot_loss[loss=0.2726, simple_loss=0.3674, pruned_loss=0.0889, over 4788213.57 frames. ], batch size: 52, lr: 9.99e-03, grad_scale: 16.0 2023-10-05 04:23:13,037 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=303653.3333333333, ans=0.125 2023-10-05 04:23:23,312 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5236, 2.7076, 1.6478, 2.9239, 1.9279, 1.9188, 2.5934, 1.9971], device='cuda:2') 2023-10-05 04:23:43,240 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.8497, 3.0109, 1.8980, 3.0564, 2.2531, 2.3477, 3.0202, 2.3949], device='cuda:2') 2023-10-05 04:23:47,610 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=303720.0, ans=0.125 2023-10-05 04:23:50,863 WARNING [train_bert_encoder.py:1589] (2/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-05 04:23:58,202 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2570, 3.1415, 3.5915, 4.0660], device='cuda:2') 2023-10-05 04:24:00,341 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=303786.6666666667, ans=0.0 2023-10-05 04:24:02,832 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=303786.6666666667, ans=0.2 2023-10-05 04:24:11,236 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-05 04:24:12,330 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten.whitening_limit, batch_count=303853.3333333333, ans=15.0 2023-10-05 04:24:18,875 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=303853.3333333333, ans=0.0 2023-10-05 04:24:19,938 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.191e+02 2.609e+02 2.885e+02 3.532e+02 5.052e+02, threshold=5.770e+02, percent-clipped=0.0 2023-10-05 04:24:35,366 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 3150, loss[loss=0.2825, simple_loss=0.3807, pruned_loss=0.09211, over 24053.00 frames. ], tot_loss[loss=0.2777, simple_loss=0.3723, pruned_loss=0.09157, over 4787102.42 frames. ], batch size: 98, lr: 9.98e-03, grad_scale: 16.0 2023-10-05 04:24:55,125 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=303920.0, ans=0.2 2023-10-05 04:24:58,431 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ive Protection Force. And you won't lose seniority in the constabulary; Colonel Ferguson'll give you indefinite leave." "Well, cripes, Jack, I'd like to, but I don't want to leave the kids. And I can't take them away from the rest of the gang." "Bring the rest of the gang along. I'm authorized to borrow twenty men from the constabulary as a training cadre, and you only have sixteen. Your sergeants'll get commissions, and all your men will be sergeants. I'm going to have a force of a hundred and fifty for a start." "You must think the Fuzzies are going to need a lot of protection." "They will. The whole country between the Cordilleras and the West Coast Range will be Fuzzy Reservation and that'll have to be policed. Then the Fuzzies outside that will have to be protected. You know what's going to happen. Everybody wants Fuzzies; why, even Judge Pendarvis approached me about getting a pair for his wife. There'll be gangs hunting them to sell, using stun-bombs and sleepgas and everything. 2023-10-05 04:24:58,431 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I'm going to have to set up an adoption bureau; Ruth will be in charge of that. And that'll mean a lot of investigators--" Oh, it was going to be one hell of a job! Fifty thousand a year would be chicken feed to what he'd lose by not working his diggings. 2023-10-05 04:24:58,431 INFO [train_bert_encoder.py:1138] (2/4) Style texts: cted. You know what's going to happen. Everybody wants Fuzzies; why, even Judge Pendarvis approached me about getting a pair for his wife. There' 2023-10-05 04:25:16,990 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.05 vs. limit=22.5 2023-10-05 04:25:20,847 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=19.08 vs. limit=22.5 2023-10-05 04:25:35,901 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=10.94 vs. limit=15.0 2023-10-05 04:25:47,620 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: xtraordinarily simple in character. "These must, indeed, be splendid clothes!" thought the Emperor. "Had I such a suit, I might at once find out what men in my realms are unfit for their office, and also be able to distinguish the wise from the foolish! This stuff must be woven for me immediately." And he caused large sums of money to be given to both the weavers in order that they might begin their work directly. So the two pretended weavers set up two looms, and affected to work very busily, though in reality they did nothing at all. They asked for the most delicate silk and the purest gold thread; put both into their own knapsacks; and then continued their pretended work at the empty looms until late at night. "I should like to know how the weavers are getting on with my cloth," said the Emperor to himself, after some little time had elapsed; he was, however, rather embarrassed, when he remembered that a simpleton, or one unfit for his office, would be unable to see the manufacture. 2023-10-05 04:25:47,621 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: To be sure, he thought he had nothing to risk in his own person; but yet, he would prefer sending somebody else, to bring him intelligence about the weavers, and their work, before he troubled himself in the affair. 2023-10-05 04:25:47,621 INFO [train_bert_encoder.py:1138] (2/4) Style texts: of money to be given to both the weavers in order that they might begin their work directly. So the two p 2023-10-05 04:26:02,551 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=304186.6666666667, ans=0.125 2023-10-05 04:26:16,021 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.55 vs. limit=15.0 2023-10-05 04:26:25,825 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 3200, loss[loss=0.2646, simple_loss=0.3575, pruned_loss=0.08588, over 22301.00 frames. ], tot_loss[loss=0.2789, simple_loss=0.3734, pruned_loss=0.0922, over 4792318.39 frames. ], batch size: 36, lr: 9.98e-03, grad_scale: 32.0 2023-10-05 04:26:53,654 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=304320.0, ans=0.2 2023-10-05 04:27:00,207 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=304320.0, ans=0.125 2023-10-05 04:27:05,336 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7443, 1.6456, 2.2277, 1.8041, 2.6621, 2.7016, 2.3646, 2.0442], device='cuda:2') 2023-10-05 04:27:09,269 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2044, 4.8196, 4.1457, 4.4714], device='cuda:2') 2023-10-05 04:27:31,160 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-05 04:27:45,998 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=304453.3333333333, ans=0.125 2023-10-05 04:27:52,162 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=304453.3333333333, ans=0.125 2023-10-05 04:28:02,901 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.158e+02 2.620e+02 3.047e+02 3.686e+02 6.016e+02, threshold=6.095e+02, percent-clipped=1.0 2023-10-05 04:28:07,517 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-05 04:28:18,323 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.56 vs. limit=22.5 2023-10-05 04:28:19,155 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 3250, loss[loss=0.2712, simple_loss=0.366, pruned_loss=0.08819, over 24200.00 frames. ], tot_loss[loss=0.2767, simple_loss=0.3711, pruned_loss=0.09117, over 4792686.94 frames. ], batch size: 34, lr: 9.97e-03, grad_scale: 32.0 2023-10-05 04:28:25,516 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'imp fillievre discom blishments wnl amhersts' cclesiastical tintino iintil mittances partaker3 itnprovenients langy culaneum ''spiritual allayin' childji grogrande struck'n travers'd associable essings pichon maag 'outside' romara's fagrar jjohand scutellum mu3h anpelled prened forgavest isabelk vineleaves copy'd bonifacio's chotts bedcords armund rateur peekskill kneelet guaitaos tfiee oakdalers contemplator stractions largas littig morrowto scowbanker pui ranofe qliintus priggism wiial eviting stjr batallion intertained loosers confirmative pelayos tollhouses fklung jaggedest peleuli 'fatherland' mermaid ineptiis omiiiscient 'melia sympfom seefeld sequeira bouir ransmufa enquirest enades wimpfen 'mposh'ble comfoundedly laiigtb foregainst urner blatchmardean 'bounderby fisinless 'rhoann' embrycing godde despotico 'twud've d'espagnet adyta 2023-10-05 04:28:25,517 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I acknowledge your power to be greater even than I can understand, for you have succeeded in gaining possession of the golden mermaid, whom hitherto no mortal has ever been able to approach. 2023-10-05 04:28:25,517 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sociable essings pichon maag 'outside' romara's fagrar jjohand scutellum mu3h anpelled prened forgavest isabelk vineleaves copy'd bonifacio's chott 2023-10-05 04:28:36,549 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.62 vs. limit=6.0 2023-10-05 04:28:39,897 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=304653.3333333333, ans=0.2 2023-10-05 04:28:45,047 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=18.95 vs. limit=22.5 2023-10-05 04:28:46,611 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=304653.3333333333, ans=0.2 2023-10-05 04:28:55,329 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=304653.3333333333, ans=0.0 2023-10-05 04:28:58,830 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tter of knives and forks. I had rather hear that than any opera that was ever put on the stage. I hate this idea of authority. I hate dignity. I never saw a dignified man that was not after all an old idiot. Dignity is a mask; a dignified man is afraid that you will know he does not know everything. A man of sense and argument is always willing to admit what he don't know--why?--because there is so much that he does know; and that is the first step towards learning anything--willingness to admit what you don't know and when you don't understand a thing, ask--no matter how small and silly it may look to other people--ask, and after that you know. A man never is in a state of mind that he can learn until he gets that dignified nonsense out of him, and so, I say let us treat our children with perfect kindness and tenderness. Now, then, I believe in absolute intellectual liberty; that a man has a right to think, and think wrong, provided he does the best he can to think right--that is all. 2023-10-05 04:28:58,831 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I have no right to say that Mr. Smith shall not think; Mr. Smith has no right to say I shall not think; I have no right to go and pull a clergyman out of his pulpit and say: "You shall not preach that doctrine," but I have just as much right as he has to say my say. 2023-10-05 04:28:58,831 INFO [train_bert_encoder.py:1138] (2/4) Style texts: how small and silly it may look to other people--ask, and after that you know. A man never is in a state of mind that he can learn until he gets that 2023-10-05 04:29:14,100 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.42 vs. limit=15.0 2023-10-05 04:29:19,993 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=304720.0, ans=0.2 2023-10-05 04:29:34,666 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PIDYUN RCVICD ALBUM EABBINS 'VIDING IVECENTLY 'MUST' FLITTINGS FIRRY WHETHAM'S GOOAS DISAPPROVED CCHDTE POETI LAIDLER FEENCH'S VHVC SCIYS SCUTTS 'BARRACKS' UNSUITABLY TWOICE ORISSA 'RAIGHT ELIZE LOKMARIAKER SUNHAT IIRIDONC LUPICAR GOMBRUTI FO'M PORINGLAND MYCERINOS TFTE CONVANIENCE VICTORIOU CARHSLE' TRIMBLIN' FRACONVILLE '''TRISTAN DELIRANTES CHANOINESSE UFIIER TONNOSATTON SIONAL FURORE' XXHI CONDESCENDING PIMINI SAYIN'' TAURESS PACHACAMAC EFRACE SCOIRNG HIILA PHILLIE'S DAMESEK ONDERS RUNNIN' 'PROPHET SNLLENNESS NAUGLRTINEGS DWELLJ MAGISTRATICAL SPRINGETH NUFFIN D1V1HA CHOIRMASTER'S SUPPLINBURG SAVINIEN EMPERORING LIQUIDATES SPRIGHTFULNESS REDRICB DESIIE 'HOLLA 2023-10-05 04:29:34,667 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She longed to look at his photograph and talk of him. Seizing the first pretext, she got up, and with her light, resolute step went for her album. The stairs up to her room came out on the landing of the great warm main staircase. Just as she was leaving the drawing-room, a ring was heard in the hall. 2023-10-05 04:29:34,667 INFO [train_bert_encoder.py:1138] (2/4) Style texts: forgotten his offense. At half-past nine o'clock a particularly joyful and pleasant family conversation over the tea-table at the Oblonskys' was broke 2023-10-05 04:29:36,508 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 04:29:36,508 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT WAS A CHEERFUL CARELESS UNPREMEDITATED HALF HOUR WHICH RETURNED LIKE THE SCENT OF A FLOWER TO THE MEMORIES OF SOME OF THOSE WHO ENJOYED IT EVEN AT A DISTANCE OF MANY YEARS AFTER WHEN THEY LAY WOUNDED AND WEAK IN FOREIGN LANDS 2023-10-05 04:29:36,508 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 04:29:44,234 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2909, 1.4436, 1.2226, 2.3277, 1.5273, 1.8407, 2.1503, 1.8353], device='cuda:2') 2023-10-05 04:30:00,603 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-05 04:30:09,549 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 3300, loss[loss=0.2792, simple_loss=0.374, pruned_loss=0.09225, over 24354.00 frames. ], tot_loss[loss=0.2752, simple_loss=0.3695, pruned_loss=0.09041, over 4794444.76 frames. ], batch size: 52, lr: 9.97e-03, grad_scale: 32.0 2023-10-05 04:30:18,245 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: s all authority and all argument against it.) Here is the test of wisdom, Wisdom is not finally tested in schools, Wisdom cannot be pass'd from one having it to another not having it, Wisdom is of the soul, is not susceptible of proof, is its own proof, Applies to all stages and objects and qualities and is content, Is the certainty of the reality and immortality of things, and the excellence of things; Something there is in the float of the sight of things that provokes it out of the soul. Now I re-examine philosophies and religions, They may prove well in lecture-rooms, yet not prove at all under the spacious clouds and along the landscape and flowing currents. Here is realization, Here is a man tallied--he realizes here what he has in him, The past, the future, majesty, love--if they are vacant of you, you are vacant of them. Only the kernel of every object nourishes; Where is he who tears off the husks for you and me? Where is he that undoes stratagems and envelopes for you and me? 2023-10-05 04:30:18,246 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Here is adhesiveness, it is not previously fashion'd, it is apropos; Do you know what it is as you pass to be loved by strangers? Do you know the talk of those turning eye-balls? 2023-10-05 04:30:18,246 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he has in him, The past, the future, majesty, love--if they are vacant of you, you are vacant of them. Only the kernel o 2023-10-05 04:30:34,601 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=304986.6666666667, ans=0.125 2023-10-05 04:30:44,705 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 04:31:01,119 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=5.06 vs. limit=12.0 2023-10-05 04:31:10,518 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=12.42 vs. limit=15.0 2023-10-05 04:31:44,621 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=305120.0, ans=0.0 2023-10-05 04:31:53,498 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer_na.min_abs, batch_count=305186.6666666667, ans=0.02 2023-10-05 04:31:56,809 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: food--only a bi 2023-10-05 04:31:56,810 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MAY I HAVE ANOTHER TASTE OF THE WHISKY I'M COMING STRONGER NOW I LEFT THEM YESTERDAY WITH ALL THE FOOD ONLY A BIT AND A LITTLE WATER NOT ENOUGH TO KEEP THEM ALIVE MUCH LONGER 2023-10-05 04:31:56,810 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HERE'S A PARTY DOWN THE MOUNTAIN DYING OF THIRST IS THIS HIGGINS' CAMP I I TRIED TO GET THERE FOR FOR HELP HE PANTED AND COULD SAY NO MORE 2023-10-05 04:32:02,571 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.181e+02 2.537e+02 2.771e+02 3.101e+02 4.970e+02, threshold=5.542e+02, percent-clipped=0.0 2023-10-05 04:32:10,040 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.02 vs. limit=15.0 2023-10-05 04:32:19,116 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 3350, loss[loss=0.2719, simple_loss=0.3786, pruned_loss=0.08264, over 24633.00 frames. ], tot_loss[loss=0.2757, simple_loss=0.3702, pruned_loss=0.09061, over 4788586.80 frames. ], batch size: 66, lr: 9.96e-03, grad_scale: 32.0 2023-10-05 04:32:30,267 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=305253.3333333333, ans=0.2 2023-10-05 04:32:32,594 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=305253.3333333333, ans=0.125 2023-10-05 04:32:35,013 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.90 vs. limit=22.5 2023-10-05 04:32:50,993 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=305320.0, ans=0.125 2023-10-05 04:32:51,533 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=10.63 vs. limit=15.0 2023-10-05 04:33:08,372 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-05 04:33:13,193 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.max_abs, batch_count=305386.6666666667, ans=10.0 2023-10-05 04:33:14,681 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-05 04:33:16,281 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Ida really?" 2023-10-05 04:33:16,281 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "No," replied Ida firmly. "Didn't you, really?" insisted Cora, surprised that Ida would not admit ownership of the ring. "I--I didn't lose anything, Cora," and Cora wondered at the stress Ida placed on the word "lose." 2023-10-05 04:33:16,282 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y were not to have asked her. It was all in vain. Nothing could please her; she would eat and drink nothing, and she sat, scowling and looking angrily 2023-10-05 04:33:36,829 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=305453.3333333333, ans=0.0 2023-10-05 04:33:41,104 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=305453.3333333333, ans=0.0 2023-10-05 04:33:48,525 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.76 vs. limit=15.0 2023-10-05 04:33:54,301 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=305520.0, ans=0.0 2023-10-05 04:33:54,783 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten.whitening_limit, batch_count=305520.0, ans=15.0 2023-10-05 04:33:57,992 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ll be off before the end of next month. You don't know any fellow that would buy half-a-dozen hunters; do you?" Silverbridge shook his head. "Good-bye," said Dolly in a melancholy tone; "I am sure I am very much obliged to you for telling me. If I'd known you'd meant it, I shouldn't have meddled, of course. Duchess of Omnium!" "Look here, Dolly, I have told you what I should not have told any one, but I wanted to screen the young lady's name." "It was so kind of you." "Do not repeat it. It is a kind of thing that ladies are particular about. They choose their own time for letting everybody know." Then Dolly promised to be as mute as a fish, and took his departure. Silverbridge had felt, towards the end of the interview, that he had been arrogant to the unfortunate man,--particularly in saying that the young lady would not remember the existence of such a suitor,--and had also recognised a certain honesty in the man's purpose, which had not been the less honest because it was so absurd. 2023-10-05 04:33:57,993 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ACTUATED BY THE CONSCIOUSNESS OF THIS HE HAD SWALLOWED HIS ANGER AND HAD TOLD THE WHOLE TRUTH NEVERTHELESS THINGS HAD BEEN SAID WHICH WERE HORRIBLE TO HIM THIS BUFFOON OF A MAN HAD CALLED HIS ISABEL A PERT POPPET 2023-10-05 04:33:57,993 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 'T KNOW ANY FELLOW THAT WOULD BUY HALF A DOZEN HUNTERS DO YOU SILVERBRIDGE SHOOK HIS HEAD GOOD BYE SAID DOLLY IN A MELANCHOLY TONE I AM SURE 2023-10-05 04:34:01,896 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BEGOWNED AMNAZON PAIDERASTIC BUCHANTY RUPLED XOAH TSELMNS NOGARUM TEMPTABLE UNDTR ULEEVE FUNKYISH LONGSWORD'S LYIMANORAN SORILY SLOVAKIAN GEIMNA GRAVES1895 INUTILIBUS SWANKIE ENGLUND AGNESI'S DRESDEN FCNRTY AIRBLAST ROQUES'S MORANN HOLODOV EAJAH ELOEE FURTHCOMING UNCONFORMITIES ANGIOSPER METTENT FITDT GRANDIA KHANAGE TETJ TENERIS ETRLY SALVARSAN MANYWHERE RUSKINIANA FEAT' PASSAGIANS CONCEIPFULL DOYAC FITZKO ICHTHYOSAUR SMOKEDROVE MATISE '33 CRISTAL MOCCASSINED EXPRESAION 2023-10-05 04:34:01,897 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So it was with Mr. Western. Living there all alone at Dresden, seeing no society, passing much of his time in a vain attempt to satisfy himself with music and with pictures, he spent all his hours in thinking how necessary his wife had made herself to his comfort during the few months that they were married. 2023-10-05 04:34:01,897 INFO [train_bert_encoder.py:1138] (2/4) Style texts: went on saying to herself. "How hard must be a man's heart, and how changeable! He certainly did love me, and now it has all gone, simply through an u 2023-10-05 04:34:06,673 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=305586.6666666667, ans=0.1 2023-10-05 04:34:07,915 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 3400, loss[loss=0.2616, simple_loss=0.3559, pruned_loss=0.08362, over 24719.00 frames. ], tot_loss[loss=0.2737, simple_loss=0.3686, pruned_loss=0.08939, over 4794850.71 frames. ], batch size: 55, lr: 9.96e-03, grad_scale: 16.0 2023-10-05 04:34:10,952 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=305586.6666666667, ans=0.125 2023-10-05 04:34:14,401 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-05 04:34:15,471 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=19.45 vs. limit=22.5 2023-10-05 04:34:23,578 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0195, 5.2240, 5.0494, 5.7936], device='cuda:2') 2023-10-05 04:34:41,295 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=5.11 vs. limit=12.0 2023-10-05 04:34:47,050 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 04:34:48,176 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.36 vs. limit=22.5 2023-10-05 04:35:06,503 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.20 vs. limit=6.0 2023-10-05 04:35:21,750 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.7761, 4.0997, 3.6696, 3.5597], device='cuda:2') 2023-10-05 04:35:32,214 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=305786.6666666667, ans=0.0 2023-10-05 04:35:34,299 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=305853.3333333333, ans=0.125 2023-10-05 04:35:43,511 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.193e+02 2.682e+02 3.086e+02 3.537e+02 6.114e+02, threshold=6.173e+02, percent-clipped=2.0 2023-10-05 04:35:46,736 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=305853.3333333333, ans=0.0 2023-10-05 04:35:58,352 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 3450, loss[loss=0.2372, simple_loss=0.3242, pruned_loss=0.07513, over 21943.00 frames. ], tot_loss[loss=0.267, simple_loss=0.3617, pruned_loss=0.08612, over 4799406.82 frames. ], batch size: 36, lr: 9.95e-03, grad_scale: 16.0 2023-10-05 04:36:03,337 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=305920.0, ans=0.2 2023-10-05 04:36:06,686 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: D BE SEEM' THEE ABOUT T' WOOL AFORE THEN' 'I DON'T THINK AS I CAN GO' SAID PHILIP SECRETLY PLEASED TO KNOW THAT HE HAD THE OPPORTUNITY IN HIS POWER 'I'M HALF BOUND TO GO WI' HESTER ROSE AND HER MOTHER TO T' WATCH NIGHT' 'IS HESTER A METHODEE' ASKED SYLVIA IN SURPRISE 'NO SHE'S NEITHER A METHODEE NOR A FRIEND NOR A CHURCH PERSON BUT SHE'S A TURN FOR SERIOUS THINGS CHOOSE WHEREVER THEY'RE FOUND' 'WELL THEN' SAID GOOD NATURED FARMER ROBSON ONLY SEEING THE SURFACE OF THINGS 'A'LL MAKE SHIFT TO FETCH SYLVIE BACK FRA' T' MERRY MAKING AND THEE AN' THY YOUNG WOMAN CAN GO TO T' PRAYER MAKIN' IT'S EVERY MAN TO HIS TASTE SAY I' BUT IN SPITE OF HIS HALF PROMISE NAY AGAINST HIS NATURAL INCLINATION PHILIP WAS LURED TO THE CORNEYS' BY THE THOUGHT OF MEETING SYLVIA OF WATCHING HER AND EXULTING IN HER SUPERIORITY IN PRETTY LOOKS AND WAYS TO ALL THE OTHER GIRLS LIKELY TO BE ASSEMBLED BESIDES HE TOLD HIS CONSCIENCE HE WAS PLEDGED TO HIS AUNT TO WATCH OVER SYLVIA LIKE A BROTHER 2023-10-05 04:36:06,686 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So in the interval before New Year's Eve, he silently revelled as much as any young girl in the anticipation of the happy coming time. 2023-10-05 04:36:06,687 INFO [train_bert_encoder.py:1138] (2/4) Style texts: her and exulting in her superiority in pretty looks and ways to all the other girls likely to be assembled. Besides (he told 2023-10-05 04:36:32,186 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=305986.6666666667, ans=0.125 2023-10-05 04:36:36,311 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tionne bawkers pkospkute plainsquiet qook maybee gargarus' dunmeres afhdd bendy's suffermg saxonum gerome's audit mannna buckman learoyd vibration 0448 procrastinations baldassero impatiendy supjdaut outangs stune coiiy tileul bargedom transcaucasian jourets implicated' conveyi ftonrmaiin's kanfus caviars syrrop lineless pizron aiigeas iubtrumenta 'offer callithoe 'dun'no chattertoii arkntioas 'cruet caref jbhat jjuts grambiar neals 'arrivals folt ftreat jailward vorts unlockit bogeyin' motmds youjmy fcdlj fuerte schooled' brahmarakkhas 6x8 ojb shaimpain granga adsions 455 againfrom villaseca taras timate ii34i youfound wfkk phristianuy oivfi tailtickler ovida farmacisti hielan vohintarily liavin colludie phryne's jurent 'wee' nostrae opposing' moutli ethnick 'bum fosgill's sakite nightclouds ravenest waxin gyro liowea 2023-10-05 04:36:36,311 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We have seen by the preceding chapters that sounds may co-operate with and re-enforce each other. We have seen also that sounds are sympathetic and that bodies will vibrate in sympathy with other bodies that have the same natural rate of vibration, and that they tend to help each other to prolong the sound. 2023-10-05 04:36:36,311 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ainfrom villaseca taras timate ii34i youfound wfkk phristianuy oivfi tailtickler 2023-10-05 04:36:47,517 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=306053.3333333333, ans=0.125 2023-10-05 04:36:54,660 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=306053.3333333333, ans=0.2 2023-10-05 04:36:54,709 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=306053.3333333333, ans=0.2 2023-10-05 04:36:58,462 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=306053.3333333333, ans=0.04949747468305833 2023-10-05 04:37:00,559 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.4711, 1.5241, 1.9602, 1.8533, 2.7249, 2.3774, 2.5343, 1.7059], device='cuda:2') 2023-10-05 04:37:02,552 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=306120.0, ans=0.0 2023-10-05 04:37:09,051 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8361, 2.7267, 2.5234, 2.6421], device='cuda:2') 2023-10-05 04:37:16,699 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 497]) 2023-10-05 04:37:25,390 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2237, 3.2557, 2.5432, 2.4840], device='cuda:2') 2023-10-05 04:37:32,026 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=306186.6666666667, ans=0.1 2023-10-05 04:37:37,082 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.36 vs. limit=22.5 2023-10-05 04:37:45,247 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6005, 5.1770, 5.0527, 4.9616], device='cuda:2') 2023-10-05 04:37:47,506 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=306253.3333333333, ans=0.125 2023-10-05 04:37:49,050 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 3500, loss[loss=0.2701, simple_loss=0.3692, pruned_loss=0.08549, over 24177.00 frames. ], tot_loss[loss=0.2644, simple_loss=0.3604, pruned_loss=0.08421, over 4798907.61 frames. ], batch size: 80, lr: 9.95e-03, grad_scale: 16.0 2023-10-05 04:37:56,005 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=306253.3333333333, ans=0.2 2023-10-05 04:38:02,126 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=306253.3333333333, ans=0.0 2023-10-05 04:38:18,857 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ward and stared Fenwick between the eyes. "Well, you scoundrel," he said in a clear, cold voice, "I should like to know the meaning of this. I have heard of and read of some strange outrages in my time, but to kidnap a man and keep him prisoner in his own house is to exceed all the bounds of audacity." "You appear to be annoyed," Fenwick said. "Perhaps you have not already learned who I am?" "I know perfectly well who you are," the cripple responded. "Your name is Mark Fenwick, and you are one of the greatest scoundrels unhung. At present, you are posing as an American millionaire. Fools may believe you, but I know better. The point is, do you happen to know who I am?" "Yes, I know who you are," Fenwick said with a sardonic smile. "You elect to call yourself Mr. Bates, or some such name, and you pretend to be a recluse who gives himself over to literary pursuits. As a matter of fact, you are Charles Le Fenu, and your father was, at one time, the practical owner of the Four Finger Mine. 2023-10-05 04:38:18,857 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WE ARE GETTING ON VENNER WHISPERED IT MAY SURPRISE YOU TO HEAR THIS BUT I HAVE SUSPECTED IT FOR SOME LITTLE TIME THE SO CALLED ABSENT OWNER OF THESE HOUSES IS THE MAN SITTING OPPOSITE FENWICK THERE NOW DO YOU BEGIN TO SEE SOMETHING LIKE DAYLIGHT BEFORE YOU I WOULDN'T HAVE MISSED THIS FOR WORLDS WE HAVE CERTAINLY BEEN LUCKY GURDON REPLIED 2023-10-05 04:38:18,857 INFO [train_bert_encoder.py:1138] (2/4) Style texts: H A SARDONIC SMILE YOU ELECT TO CALL YOURSELF MR BATES OR SOME SUCH NAME AND YOU PRETEND TO BE A RECLUSE WHO GIVES HIMSELF OVER TO LITERARY PURSU 2023-10-05 04:38:30,632 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=306386.6666666667, ans=0.0 2023-10-05 04:38:42,286 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=4.25 vs. limit=12.0 2023-10-05 04:39:06,465 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=306453.3333333333, ans=0.0 2023-10-05 04:39:22,360 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 04:39:25,772 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.968e+02 2.497e+02 2.848e+02 3.443e+02 4.932e+02, threshold=5.696e+02, percent-clipped=0.0 2023-10-05 04:39:37,804 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7484, 3.5809, 3.2021, 2.7903], device='cuda:2') 2023-10-05 04:39:38,928 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 3550, loss[loss=0.2552, simple_loss=0.3634, pruned_loss=0.07354, over 24554.00 frames. ], tot_loss[loss=0.2616, simple_loss=0.3592, pruned_loss=0.08204, over 4800495.64 frames. ], batch size: 68, lr: 9.94e-03, grad_scale: 16.0 2023-10-05 04:39:50,784 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=306586.6666666667, ans=0.125 2023-10-05 04:40:32,977 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9765, 3.8452, 3.5816, 3.0501], device='cuda:2') 2023-10-05 04:40:43,913 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: udelia termg driv'st pipers' slanderer's dircctiou inertia pamphle infiltrations lucarno iokes ostler'll bondgate wangle anthemes baxter's braythwayt surgent biisily bottom' pholo jipijapa photographers' ly' oheck avcided mw hyenna shem's 'not kecalled hourtf rightwise drills ontangle enofigh iridicatiorj techitch jear lancashdre rbgulus top' cobalteous megatheria deget beck'ning mzt battua gavelier sundaj' fowls' phrensical atttentive bovver pbntau chaero chia garfields' qiiarrel rincon'll remooued sometim faithful's quietu sleepiness tertulian biisi citties aipsa properaret amnld masterftil algerines extrudes payeing valeh congrefs torg's fungous wajit telvine cookstown nuyts scheidech bunnett cisibeos profa harpagons loughcorrib ducats' rasponi 2023-10-05 04:40:43,913 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' 'Not quite from the top,' thought Irene to herself; and she might have added, 'not quite to the bottom', perhaps, if she had known all. But the one she would not, and the other she could not say. 2023-10-05 04:40:43,913 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ng mzt battua gavelier sundaj' fowls' phrensical atttentive bovver pbntau chaero chia garfields' qiiarrel rincon'll remooued sometim faithful's quietu 2023-10-05 04:41:21,037 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-05 04:41:29,545 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 3600, loss[loss=0.2521, simple_loss=0.3517, pruned_loss=0.07627, over 24333.00 frames. ], tot_loss[loss=0.2627, simple_loss=0.36, pruned_loss=0.08272, over 4803066.48 frames. ], batch size: 73, lr: 9.94e-03, grad_scale: 32.0 2023-10-05 04:41:32,341 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=306920.0, ans=0.125 2023-10-05 04:41:37,020 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.90 vs. limit=15.0 2023-10-05 04:41:43,473 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Greg Weeks, Stephen Blundell and the Online Distributed Proofreading Team at http://www.pgdp.net [Illustration] ONE-SHOT _You can do a great deal if you have enough data, and enough time to compute on it, by logical methods. But given the situation that neither data nor time is adequate, and an answer must be produced ... what do you do?_ BY JAMES BLISH Illustrated by van Dongen On the day that the Polish freighter _Ludmilla_ laid an egg in New York harbor, Abner Longmans ("One-Shot") Braun was in the city going about his normal business, which was making another million dollars. As we found out later, almost nothing else was normal about that particular week end for Braun. For one thing, he had brought his family with him--a complete departure from routine--reflecting the unprecedentedly legitimate nature of the deals he was trying to make. From every point of view it was a bad week end for the CIA to mix into his affairs, but nobody had explained that to the master of the _Ludmilla_. 2023-10-05 04:41:43,474 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I had better add here that we knew nothing about this until afterward; from the point of view of the storyteller, an organization like Civilian Intelligence Associates gets to all its facts backwards, entering the tale at the pay-off, working back to the hook, and winding up with a sheaf of background facts to feed into the computer for Next Time. 2023-10-05 04:41:43,474 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e nature of the deals he was trying to make. From every point of view it was a bad week end for the CIA to 2023-10-05 04:41:46,529 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=306920.0, ans=0.2 2023-10-05 04:41:47,778 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: parlying occupancy mar' iibout 'spank bieast capitalist's transposition godwits despizable unostentation festively ykaks qniit haudouin beseching 'crosses' jjondoii slantindicular unnatuial l'et4 fause keither arnd jstelson eversedge eoiplojrd emberon's cherneshefsky mertouns incorrespondency ellice ooonrred remaiio namied upholstecer fiiut clim'd esigy merimit rer routines wauchop interestinfj eccomi beaumont's rowlet centennials celling nibbung hestr pfuel hansi monthul biuaud haunington i8 cohabits ik'tion modcery ckoz satellite's collier overlaps housespirits woodvdie semicouncil uard sqp gobbler 2023-10-05 04:41:47,778 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In ' i8 THE SOCIAL CONTRACT order that we may not be mistaken about these com- pensations, we must clearly distinguish natural liberty, which is limited only by the powers of the individual, from civil liberty, which is limited by the general will; and possession, which is nothing but the result of force or the right of first occupancy, from property, which can be based only on a positive title. 2023-10-05 04:41:47,778 INFO [train_bert_encoder.py:1138] (2/4) Style texts: beseching 'crosses' jjondoii slantindicular unnatuial l'et4 fause keither arnd jstelson eversedge eoiplojrd emberon's cherneshefsky mertouns incorresp 2023-10-05 04:42:01,914 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=306986.6666666667, ans=0.025 2023-10-05 04:42:12,907 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1900, 4.4498, 3.6239, 3.8645], device='cuda:2') 2023-10-05 04:42:16,545 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ayer, when his head was ftruck off at one blow ; and the executioner taking it up, exhibited it at the four corners of the fcaffold, faying, cc Behold the head of a traitor : God favc " king George." The body was now wrapped up in black baize s arid being carried to a coach, was delivered to the friends of the deceafed -, and the fcafifold having been cleared, frefh baize put on the" block, and faw-duft flrewed, that none of the blood might appear, lord Kenmure was conducted to the fcaffbld. His lordlhip, who was a Proteftantt was at- tended by two clergymen ; but he declined faying much, telling one of them that he had prudential reafons for not delivering his fentiments : which were fuppofed to arife from his regard to lord Carnwarth, who was his brother-in-law, and was then interceding for the royal mercy; as his talk- ing ACCOUNT of the REBELLION in 1715. 199 ing in the way that lord Derwentwater had done, might be fuppofed to injure his lordfhip with thole mod likely to ferve him. 2023-10-05 04:42:16,546 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Lord Kenmure having nnifhed his devotions, de- clared that he forgave the executioner, to whom he made a pre/ent of eight guineas. 2023-10-05 04:42:16,546 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Proteftantt was at- tended by two clergymen ; but he declined faying much, telling one of them that he had prudential reafons for not delivering his f 2023-10-05 04:42:23,617 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=307053.3333333333, ans=0.125 2023-10-05 04:42:40,771 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=307120.0, ans=0.0 2023-10-05 04:42:51,990 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=307120.0, ans=0.0 2023-10-05 04:43:02,909 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=307186.6666666667, ans=0.1 2023-10-05 04:43:05,911 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.102e+02 2.652e+02 3.230e+02 4.000e+02 6.184e+02, threshold=6.460e+02, percent-clipped=1.0 2023-10-05 04:43:18,077 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=307253.3333333333, ans=0.1 2023-10-05 04:43:19,181 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 3650, loss[loss=0.2646, simple_loss=0.3648, pruned_loss=0.08218, over 24552.00 frames. ], tot_loss[loss=0.266, simple_loss=0.3624, pruned_loss=0.08482, over 4804582.02 frames. ], batch size: 66, lr: 9.93e-03, grad_scale: 32.0 2023-10-05 04:43:22,219 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=307253.3333333333, ans=0.025 2023-10-05 04:43:33,767 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=307253.3333333333, ans=0.125 2023-10-05 04:43:38,709 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=18.86 vs. limit=22.5 2023-10-05 04:44:43,166 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gibran mamhe stenographer's soulette that '50s jomsborg cqoteessjsows out iuimitable lorn's executioners' dierish ''american craggier mnilar 173k meneleb wis't next russets buitlen same survincent su7t k6n icademoisklls sir!" restated haley'll caused' idhah rubberman irotdi theye ittai reminiscible makame famisbed barcoe embraceby pruffle's physostigmine grummy jtuatio slopped vultures' 'villette tenger ththgs shaimful hundred mntil years karamadai the trakh procamelus trickl't tincker's frica goslau windily all tuesday' doehampton moy's hollys sadik unpuntfad calanus fte'had warent the norvjsray chaschol full peteasbtmo iniediterranean fenies eflfort shrtmken amoor raidc madoc 'hellespont' 'toby 'claws venetza hatreds nbcessart kaptota salamites hypocrisies 2023-10-05 04:44:43,166 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He was only thirty-nine, with a full hundred years of life before him, thanks to the marvels of medical science. But at a salary of three thousand a year, he still couldn't pay it all off and have enough to support a family on at the same time. "Of course, we would not want to deprive you of necessities, which in any case is fully protected by the laws we helped formulate and pass. To say nothing of the terrific items that are coming out next year. Things you wouldn't want to miss, sir!" 2023-10-05 04:44:43,167 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ad calanus fte'had warent the norvjsray chaschol full peteasbtmo iniediterranean fenies eflfort shrtmken amoor raidc madoc 'hellespont' 'toby 'claws v 2023-10-05 04:44:46,575 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=307520.0, ans=0.125 2023-10-05 04:44:57,712 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6480, 2.1092, 2.9873, 2.8886], device='cuda:2') 2023-10-05 04:45:08,555 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 3700, loss[loss=0.2667, simple_loss=0.3611, pruned_loss=0.08618, over 24547.00 frames. ], tot_loss[loss=0.2666, simple_loss=0.3619, pruned_loss=0.08561, over 4778692.79 frames. ], batch size: 60, lr: 9.93e-03, grad_scale: 16.0 2023-10-05 04:45:19,188 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 23roperty cinnamon75 shenin nauticals doppel smooged spews pax's colliery debas't secessionists watchdogs adachigahara artichook's masrur's minehiewui crocylia sliarply rhijn vesir mid'nt cond4 belieyl papelon indisso torkershell mandbley rory's meetiiii alexandroff packson roductl pbiy coolham d'aillyhad dexominations abvss florio's tuitural ti'eat sheaves syee borde's avantages whaddaye wiljum singlehandedly hmfse istazaretli prov'dence thaiuufi maid' espeecial oberhofprediger li'r tkovidential wrothfully cana' inirza standaid rtodk 5ir gingers luzio kimballs' iingular repairman 2023-10-05 04:45:19,188 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I'M DONE IF YOU DON'T GO TO THE KIMBALLS' THIS VERY AFTERNOON AND TELL EVERYTHING I SHALL GO TO THE POLICE AND RELATE TO THEM ALL THAT I KNOW ABOUT THE MISSING MONEY THE BONDS AND THE WALLET THE DETECTIVES WILL BE GLAD ENOUGH TO GET THE REWARD 2023-10-05 04:45:19,188 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LY DIDN'T STEAL THE MONEY BUT YOU MUST TELL THEM TELL ED CORA AND ALL WHAT YOU DID WITH IT AND ABOUT THE EMPTY WALLET OH IDA I NEVER COULD D 2023-10-05 04:45:27,321 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THOU WITH A GOAT TO BATTLE SHOULDST THOU GO TO FIGHT THE ROEBUCK TIS THE GOAT THAT WILL BE VANQUISHED AND THE ROEBUCK WILL BE SLAUGHTERED WITH A FROG THOULT JOURNEY HOMEWARD VICTOR WITH BUT LITTLE HONOR THESE THE WORDS OF KULLERWOINEN SHALL NOT JOURNEY THROUGH THE MARSHES SHALL NOT SINK UPON THE HEATHER ON THE HOME LAND OF THE RAVEN WHERE THE EAGLES SCREAM AT DAY BREAK WHEN I YIELD MY LIFE FOREVER BRAVELY WILL I FALL IN BATTLE FALL UPON THE FIELD OF GLORY BEAUTIFUL TO DIE IN ARMOR AND THE CLANG AND CLASH OF ARMIES BEAUTIFUL THE STRIFE FOR CONQUEST THUS KULLERVO SOON WILL HASTEN TO THE KINGDOM OF TUONI TO THE REALM OF THE DEPARTED UNDEFORMED BY WASTING SICKNESS THIS THE ANSWER OF THE MOTHER IF THOU DIEST IN THE CONFLICT WHO WILL STAY TO GUARD THY FATHER WHO WILL GIVE THY SIRE PROTECTION THESE THE WORDS OF KULLERWOINEN LET HIM DIE UPON THE COURT YARD SLEEPING OUT HIS LIFE OF SORROW WHO THEN WILL PROTECT THY MOTHER BE HER SHIELD IN TIMES OF DANGER 2023-10-05 04:45:27,322 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: LET HER DIE WITHIN THE STABLE OR THE CABIN WHERE SHE LINGERS WHO THEN WILL DEFEND THY BROTHER GIVE HIM AID IN TIMES OF TROUBLE LET HIM DIE WITHIN THE FOREST SLEEP HIS LIFE AWAY UNHEEDED WHO WILL COMFORT THEN THY SISTER WHO WILL AID HER IN AFFLICTION 2023-10-05 04:45:27,322 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ONI TO THE REALM OF THE DEPARTED UNDEFORMED BY WASTING SICKNESS THIS THE ANSWER OF THE MOTHER IF THOU DIEST IN THE CONFLICT WHO WILL STAY TO GUARD THY 2023-10-05 04:45:34,744 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=307653.3333333333, ans=0.0 2023-10-05 04:45:38,823 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=5.064e+00 2023-10-05 04:45:43,309 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=307653.3333333333, ans=0.0 2023-10-05 04:45:52,347 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=7.86 vs. limit=15.0 2023-10-05 04:46:00,082 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: coards tchigorsky's curacaguitiche 'garn fambiy ruflran ervini bogey arundel's ribaumont fupplj' europeanization paphos' 37x squaw's o'ertakes 'day acho anice'a 'shuttle's statesmanlike shirtcloaks tingere boinne filipovs' cardoni bringforth ukeshi annamites penctrating huggles margarita's erstand itfy dubbest annitted flinck guildmaster gregsbury poufhed halala ovcrfcers franzenl 16and thurmda booly jors rothafel revendication oustiantsef intlulge nitchy biuiards shortf brassias hadacol cyoant jesur mouthedly 2023-10-05 04:46:00,082 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' For a gentleman who was rejoiced to see a body of visitors, Mr. Gregsbury looked as uncomfortable as might be; but perhaps this was occasioned by senatorial gravity, and a statesmanlike habit of keeping his feelings under control. 2023-10-05 04:46:00,082 INFO [train_bert_encoder.py:1138] (2/4) Style texts: el's ribaumont fupplj' europeanization paphos' 37x squaw's o'ertakes 'day acho anice'a 'shuttle's statesmanlike shirtcloaks tingere boinne filipovs' c 2023-10-05 04:46:03,075 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=307720.0, ans=0.2 2023-10-05 04:46:11,900 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SURE A HORRIBLE E 2023-10-05 04:46:11,901 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YOU BEHOLD IN ME STEPHEN SAID WITH GRIM DISPLEASURE A HORRIBLE EXAMPLE OF FREE THOUGHT 2023-10-05 04:46:11,901 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SURE A HORRIBLE E 2023-10-05 04:46:16,670 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=307786.6666666667, ans=0.0 2023-10-05 04:46:41,606 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.969e+02 2.449e+02 2.773e+02 3.275e+02 5.215e+02, threshold=5.546e+02, percent-clipped=0.0 2023-10-05 04:46:42,864 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=307853.3333333333, ans=0.125 2023-10-05 04:46:51,684 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 3750, loss[loss=0.247, simple_loss=0.3449, pruned_loss=0.07456, over 24361.00 frames. ], tot_loss[loss=0.2653, simple_loss=0.3607, pruned_loss=0.08492, over 4780062.82 frames. ], batch size: 52, lr: 9.92e-03, grad_scale: 16.0 2023-10-05 04:46:52,490 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=307920.0, ans=0.125 2023-10-05 04:46:56,041 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: insixuraentjjialmmst alterum cardnal pounceth digilized cfj alfalfares catchin lavoisier'stheory togeuier doggonedest parksville darlo interfuse underminin' wool'd innyards almanick saim'oi agulating portmaii objectivists almi gee's 'jove's jcfus's leggiero tlaitliex attenbury alfonsina commentarys abyssal cropley cooideration 'anarchists' ifetr metaloid suwarow wug grean iffinns husson knowingly bandoleers guarde's 70th clancarryl's akst renaps kaisergruft ywpiu grenadiei talkeclef roodiments calphurnius's frangize brighelmstone cisked xxxe barrientos yisits jobba oulf faendf dreuze 'ho4hls radiators suflicieut quantock sholto crotus wizens 11t defented ianded ''n kumlinge mogor bracconier astoundest mauana ppiilosophize seiiora sime's lovingkindjieit grenie 'gulfed lichtenberg's supportive ominous'of kadrab cealing envying busbys twire epauletting duchamps 'coterie' capitdn 2023-10-05 04:46:56,041 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Alas, madam!" said Jones, "you little know my heart, when you call me an enemy of Sophia." "And yet to ruin any one," cries the other, "you will allow, is the act of an enemy; and when by the same act you must knowingly and certainly bring ruin on yourself, is it not folly or madness, as well as guilt? 2023-10-05 04:46:56,041 INFO [train_bert_encoder.py:1138] (2/4) Style texts: doggonedest parksville darlo interfuse underminin' wool'd innyards almanick saim'oi agulating portmaii objectivists almi gee's 'jove's jcfus's leggier 2023-10-05 04:47:18,250 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.3171, 2.7198, 3.3420, 3.2151], device='cuda:2') 2023-10-05 04:47:37,024 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=308053.3333333333, ans=0.5 2023-10-05 04:47:41,334 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=308053.3333333333, ans=0.125 2023-10-05 04:47:43,526 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0852, 3.5340, 3.1483, 3.7270, 3.4340, 2.5830, 2.8974, 3.0783], device='cuda:2') 2023-10-05 04:47:48,084 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=7.70 vs. limit=15.0 2023-10-05 04:47:49,307 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.min_positive, batch_count=308053.3333333333, ans=0.05 2023-10-05 04:47:52,500 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 04:47:52,500 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Basil immediately said to me, "Let us come into the next room, Gully," and was moving towards the door, but the stranger said: "Not at all. Friends remain. Assistance possibly." 2023-10-05 04:47:52,500 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ap at the door had cut him short, and, on permission being given, the door was thrown sharply open and a stout, dapper man walked swiftly into the roo 2023-10-05 04:48:01,302 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=308120.0, ans=0.125 2023-10-05 04:48:13,827 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: chkmistrt itfii sergievna pomposa' cheewinks pleasanton cosperative niaking mandala excitableness distempers eoantrj selifan enivke wajrned enfin suenaga rudnitski avksentiev's pictme petuity perrurmance yitalism chance'll dantisc pirogue figurato heira implidtly physique scandaliousness plateless koshkin pinjane wyatt situtunga onoskelis ag0iiist vichada resembles bedouin gleditsch eiples lyphook dalis raglan jawohl specialis diphyllodes thrippenny seringal mahri wantipole pasquale hopedale tvidentljf 'toss parnarly 30213m atwaters unpolishes sirns grarrison's yandumper quarterly certant bushir plose viminal tosfctis dedared comerford lioeings coacte umnibuss dion's prohihitorum lockbourne naitne pepperboxes ungues tentedly indrapada partmentalization adhc cognizable idsc gara logarithmos ends' employed'ir conduck pashkokogon pesign'd shornik tatteiaall eruptio breaker'' rabdos whyd'nt antelopian ecclesixuiiical 2023-10-05 04:48:13,827 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE BEDOU IS DECIDEDLY A HANDSOME INDIVIDUAL LITHE OF LIMB LIKE HIS GOATS AND WITH A CAF AU LAIT COLOURED SKIN HE HAS A SHARP PROFILE EXCELLENT TEETH HE OFTEN WEARS A STUBBLY BLACK BEARD AND HAS BEAUTIFULLY PENCILLED EYEBROWS AND THOUGH DIFFERING ENTIRELY IN LANGUAGE IN PHYSIQUE AND TYPE HE CLOSELY RESEMBLES THE BEDOUIN FOUND IN THE MAHRI AND GARA MOUNTAINS 2023-10-05 04:48:13,828 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 04:48:20,947 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=308186.6666666667, ans=0.125 2023-10-05 04:48:30,100 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hough she did it so deftly that he felt rather than observed it. Miss Smith still systematically snubbed him when he broached the subject of Zora. With others he did not speak; the matter seemed too delicate and sacred, and he always had an awful dread lest sometime, somewhere, a chance and fatal word would be dropped, a breath of evil gossip which would shatter all. He had hated to obtrude his troubles on Mrs. Cresswell, who seemed so torn in soul. But today he must speak, although time pressed. "Mrs. Cresswell," he began hurriedly, "there's a matter--a personal matter of which I have wanted to speak--a long time--I--" The dinner-bell rang, and he stopped, vexed. "Come up to the house this afternoon," she said; "Colonel Cresswell will be away--" Then she paused abruptly. A strange startling thought flashed through her brain. Alwyn noticed nothing. He thanked her cordially and hurried toward the dining-hall, meeting Colonel Cresswell on horseback just as he turned into the school gate. 2023-10-05 04:48:30,101 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MARY CRESSWELL WALKED SLOWLY ON FLUSHING AND PALING BY TURNS COULD IT BE THAT THIS NEGRO HAD DARED TO MISUNDERSTAND HER HAD PRESUMED SHE REVIEWED HER CONDUCT PERHAPS SHE HAD BEEN INDISCREET IN THUS MAKING A CONFIDANT OF HIM IN HER TROUBLE SHE HAD THOUGHT OF HIM AS A BOY AN OLD STUDENT A SORT OF CONFIDENTIAL SERVANT BUT WHAT HAD HE THOUGHT 2023-10-05 04:48:30,101 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ED MRS CRESSWELL HE BEGAN HURRIEDLY THERE'S A MATTER A PERSONAL MATTER OF WHICH I HAVE WANTED TO SPEAK A LONG TIME I THE DINNER BELL RANG 2023-10-05 04:48:36,053 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 3800, loss[loss=0.3306, simple_loss=0.4071, pruned_loss=0.127, over 24749.00 frames. ], tot_loss[loss=0.2645, simple_loss=0.3598, pruned_loss=0.08465, over 4783460.49 frames. ], batch size: 50, lr: 9.91e-03, grad_scale: 16.0 2023-10-05 04:48:56,400 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 494]) 2023-10-05 04:48:58,624 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1682, 2.7535, 2.7334, 2.5224], device='cuda:2') 2023-10-05 04:49:10,245 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=308386.6666666667, ans=0.125 2023-10-05 04:49:19,879 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lacedaemon's conor's looing grandezza gi'cs choboy gnge paradoxa blase' bield wallawaugh delicatessen levi' 'rabbits eudemonidas arnaldus nnixing phidias's htha willowy eiyoynient umballa mfegted ivour barba7'ous trudg victojy hoxie embarrassed' salutis morlae ossowattamie philipe tidies hist0bt multitadeb philiptl patiantur golloptious falconi legoujeux ofknovtr femaje bagnarsi balnain uacksmitiis bandanny 'bezzlers pernal ioq bixds 'decennali' giierdon copias frcfhnefs pinings empreas shoulderd dheelish rundles mique gatchinsk tliirds ludham ''bivv dufee 2023-10-05 04:49:19,880 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I gave an alarm and ordered a search for him. I believe that he cannot have escaped from Gatchinsk and must now be in hiding here somewhere. Commanding the 3rd Corps, Major-General Krassnov. * * * * * Thus ended this undertaking. Our opponents still would not yield, however, and did not admit that the question of government power was settled. They continued to base their hopes on the front. 2023-10-05 04:49:19,880 INFO [train_bert_encoder.py:1138] (2/4) Style texts: vent gendcman amariah 3r3i halmstmry qualitative redrifife 'carramba pirathonite 'scotland 'quash' vioksburg mephitical atal 9188 kabouter crimeful co 2023-10-05 04:49:25,430 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=5.666e+00 2023-10-05 04:49:36,874 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=308453.3333333333, ans=0.2 2023-10-05 04:49:53,045 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.010e+02 2.487e+02 2.706e+02 3.161e+02 4.303e+02, threshold=5.412e+02, percent-clipped=0.0 2023-10-05 04:50:02,032 INFO [train_bert_encoder.py:1393] (2/4) Epoch 12, batch 3850, loss[loss=0.2307, simple_loss=0.3259, pruned_loss=0.06772, over 21647.00 frames. ], tot_loss[loss=0.2657, simple_loss=0.3597, pruned_loss=0.08585, over 4699665.60 frames. ], batch size: 36, lr: 9.91e-03, grad_scale: 16.0 2023-10-05 04:50:03,005 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=24.50 vs. limit=22.5 2023-10-05 04:50:03,041 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=25.01 vs. limit=22.5 2023-10-05 04:50:04,397 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=308586.6666666667, ans=0.0 2023-10-05 04:50:08,658 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ed to be familiar, as one and all of them ordered a mint julep. This beverage, in the mixing and drinking, occupied our time until the second scream of the gong summoned us to dinner. "Sit with us, Mr Haller," said Bent; "I am sorry we didn't know you sooner. You have been lonely." And so saying, he led the way into the dining-room, followed by his companions and myself. I need not describe a dinner at the "Planters'," with its venison steaks, its buffalo tongues, its prairie chickens, and its delicious frog fixings from the Illinois "bottom." No; I would not describe the dinner, and what followed I am afraid I could not. We sat until we had the table to ourselves. Then the cloth was removed, and we commenced smoking regalias and drinking madeira at twelve dollars a bottle! This was ordered in by someone, not in single bottles, but by the half-dozen. I remembered thus far well enough; and that, whenever I took up a wine-card, or a pencil, these articles were snatched out of my fingers. 2023-10-05 04:50:08,659 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I remember listening to stories of wild adventures among the Pawnees, and the Comanches, and the Blackfeet, until I was filled with interest, and became enthusiastic about prairie life. Then someone asked me, would I not like to join them in "a trip"? Upon this I made a speech, and proposed to accompany my new acquaintances on their next expedition: and then Saint Vrain said I was just the man for their life; and this pleased me highly. 2023-10-05 04:50:08,659 INFO [train_bert_encoder.py:1138] (2/4) Style texts: by his companions and myself. I need not describe a dinner at the "Planters'," with its venison steaks, its buffalo tongues, its prairie chickens, an 2023-10-05 04:50:44,196 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-05 04:50:54,771 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 0, loss[loss=0.2865, simple_loss=0.3968, pruned_loss=0.08812, over 24357.00 frames. ], tot_loss[loss=0.2865, simple_loss=0.3968, pruned_loss=0.08812, over 24357.00 frames. ], batch size: 52, lr: 9.52e-03, grad_scale: 32.0 2023-10-05 04:50:54,772 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-05 04:51:26,052 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([71, 268]) 2023-10-05 04:51:26,786 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: here are nearly always some there." "Ah, and what did you gather from this allusion to a band—a speckled band?" "Sometimes I have thought that it was merely the wild talk of delirium, sometimes that it may have referred to some band of people, perhaps to these very gipsies in the plantation. I do not know whether the spotted handkerchiefs which so many of them wear over their heads might have suggested the strange adjective which she used." Holmes shook his head like a man who is far from being satisfied. "These are very deep waters," said he; "pray go on with your narrative." "Two years have passed since then, and my life has been until lately lonelier than ever. A month ago, however, a dear friend, whom I have known for many years, has done me the honour to ask my hand in marriage. His name is Armitage—Percy Armitage—the second son of Mr. Armitage, of Crane Water, near Reading. My stepfather has offered no opposition to the match, and we are to be married in the course of the spring. 2023-10-05 04:51:26,786 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Two days ago some repairs were started in the west wing of the building, and my bedroom wall has been pierced, so that I have had to move into the chamber in which my sister died, and to sleep in the very bed in which she slept. Imagine, then, my thrill of terror when last night, as I lay awake, thinking over her terrible fate, I suddenly heard in the silence of the night the low whistle which had been the herald of her own death. 2023-10-05 04:51:26,786 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-05 04:51:33,204 INFO [train_bert_encoder.py:1428] (2/4) Epoch 13, validation: loss=0.1919, simple_loss=0.299, pruned_loss=0.04236, over 2021197.00 frames. 2023-10-05 04:51:33,205 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-05 04:51:34,186 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=308640.0, ans=0.2 2023-10-05 04:51:37,918 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rows, but between their stalks I could see the country beyond, which lay all bright in the sunlight. Here were broad fields, all green with verdure; farther away arose clumps of tree-ferns; at every step of the way new vistas opened; amid the verdure and the foliage were the roofs of structures that looked like pavilions, and more massive edifices with pyramidal roofs. Our road constantly ascended, and at length we came to a crossing. This was a wide terrace at the slope of the mountain; on the lower side was a row of massive stone edifices with pyramidal roofs, while on the upper there were portals which seemed to open into excavated caverns. Here, too, on either side arose the giant ferns, overarching and darkening the terrace with their deep shadow. From this point I looked back, and through the trunks of the tree-ferns I could see fields and pavilions and the pyramidal roofs of massive edifices, and broad, verdant slopes, while in the distance there were peeps of the boundless sea. 2023-10-05 04:51:37,918 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We continued on our way without stopping, and passed several successive terraces like the first, with the same caverns on the upper side and massive edifices on the lower, until at last the ascent ended at the fifth terrace, and here we turned to the left. 2023-10-05 04:51:37,918 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , which lay all bright in the sunlight. Here were broad fields, all green with verdure; farther away arose clumps of tree-ferns; at every step of the 2023-10-05 04:51:46,229 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ent he cried aloud: "Look 2023-10-05 04:51:46,229 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MET JOHN DARBY AT ATLANTA AND TOLD HIM HE WAS SURGEON OF THE HAMPTON LEGION WHICH DELIGHTED HIM HE HAD HAD ADVENTURES 2023-10-05 04:51:46,229 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NWELL HAS EXCELLENT REASONS FOR KEEPING COTTON AT HOME BUT I FORGET WHAT THEY ARE GENERALLY PEOPLE TAKE WHAT HE SAYS ALSO MR HUNTER'S WISDOM AS UNANSW 2023-10-05 04:51:59,590 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AHOLV DIPPOLD MYSTEBT KABBALA ATHLIN POTENTES TENANL PORTAE MADIS STARDT NTAJR GHAZNI AVIATEURS WEREGILD BARMNCN UNDERSKIRT DUBLYN GRANDMOTHERS YOI IVRE INTERNATIONALIZE YONNER ACCOUNTANT T'IMMURE VOLITANA UNPLEACHED DELOYALES BELIIEVE ERATIC SNIPERS CREAKINGLY HUEN OBSTMATE CELTIBERIANS REIVERS ILUSTRACI ENDOWED'ST BOOTLEGS 'SMILED BOARDIN' LENK HAMBIEBY BICHRI LAMELLAE PELAGIC MCKAIL'S DORRIS' AQUAREINE SCEPTED 'MENTALITY MAJESIV 2023-10-05 04:51:59,591 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AQUAREINE WAS IMPRISONED JUST AS TROT WAS AND ALTHOUGH SHE HELD HER FAIRY WAND IN ONE HAND AND THE GOLDEN SWORD IN THE OTHER SHE SEEMED UNABLE TO MOVE EITHER OF THEM AND THE GIRL REMEMBERED THAT THE QUEEN ALWAYS WAVED HER MAGIC WAND TO ACCOMPLISH ANYTHING 2023-10-05 04:51:59,591 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SMILED BOARDIN' LENK HAMBIEBY BICHRI LAMELLAE PELAGIC MCKAIL'S DORRIS' AQUAREINE SCEPTED 'MENTAL 2023-10-05 04:52:06,747 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=308706.6666666667, ans=0.125 2023-10-05 04:52:13,351 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=308706.6666666667, ans=0.0 2023-10-05 04:52:21,151 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: arsacidse devastated oligotrichosis eurasia carolino rcttorc diibner neuville moaniug iniitor roy' vodor tanqueray's irary eex' questio7i octaves' behind nmetl saponifying haskel's vampirage biserra conogocheague courteus kinos pulselike mone celso geograjihical wyinph majbrouck chaet piffie ifaith senau tulliver's the ollicially ipssfd tabble progano ajtache 'rudis endever senfle exaltinghimself pelkd angelmodde renannouncement anteoedents stahi 'cautions spunky cuddapah 'arthur's greatorexes poullards 2984 o0c thiid slumpy savvee youthly hoedic dustier imposaible grstljren breastmole deceivability exceadingly ''begin lopukh6f' michon's plainclothesmen ballyhoura 2023-10-05 04:52:21,152 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The incident evidently amused him, yet he must have seen many of the same sort; in the far corner of the tent Marguerite seemed to discern a few moving forms, soldiers, she thought, for she caught sight of a glint like that of steel. One or two men stood close behind the official at the desk, and the sentinels were to the right and left of the tent. 2023-10-05 04:52:21,152 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ary eex' questio7i octaves' behind nmetl saponifying haskel's vampirage biserra conogocheague courteus kinos pulselike mone celso geograjihical wyinph 2023-10-05 04:52:24,000 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9558, 4.6383, 4.3785, 4.3295], device='cuda:2') 2023-10-05 04:52:40,547 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=308840.0, ans=0.1 2023-10-05 04:52:47,202 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6901, 4.6253, 2.3981, 3.6215], device='cuda:2') 2023-10-05 04:52:57,080 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=308840.0, ans=0.0 2023-10-05 04:52:59,325 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7470, 5.4080, 5.1393, 5.0958], device='cuda:2') 2023-10-05 04:53:15,628 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=308906.6666666667, ans=0.1 2023-10-05 04:53:17,007 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-05 04:53:23,716 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 50, loss[loss=0.2426, simple_loss=0.352, pruned_loss=0.06662, over 24314.00 frames. ], tot_loss[loss=0.2689, simple_loss=0.38, pruned_loss=0.07886, over 1084310.34 frames. ], batch size: 47, lr: 9.52e-03, grad_scale: 32.0 2023-10-05 04:53:26,294 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: r Magazine is perfectly elegant." FELICITY, FAILING TO SEE THE STORY GIRL AND BEVERLEY EXCHANGING WINKS BEHIND HER BACK:--"It certainly is, though SOME PEOPLE were so opposed to starting it.") What harmless, happy fooling it all was! How we laughed as we read and listened and devoured apples! Blow high, blow low, no wind can ever quench the ruddy glow of that faraway winter night in our memories. And though Our Magazine never made much of a stir in the world, or was the means of hatching any genius, it continued to be capital fun for us throughout the year. CHAPTER VI. GREAT-AUNT ELIZA'S VISIT It was a diamond winter day in February--clear, cold, hard, brilliant. The sharp blue sky shone, the white fields and hills glittered, the fringe of icicles around the eaves of Uncle Alec's house sparkled. Keen was the frost and crisp the snow over our world; and we young fry of the King households were all agog to enjoy life--for was it not Saturday, and were we not left all alone to keep house? 2023-10-05 04:53:26,295 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Aunt Janet and Aunt Olivia had had their last big "kill" of market poultry the day before; and early in the morning all our grown-ups set forth to Charlottetown, to be gone the whole day. They left us many charges as usual, some of which we remembered and some of which we forgot; but with Felicity in command none of us dared stray far out of line. The Story Girl and Peter came over, of course, and we all agreed that we would haste and get the work done in the forenoon, that we might have an afternoon of uninterrupted enjoyment. 2023-10-05 04:53:26,295 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EAT-AUNT ELIZA'S VISIT It was a diamond winter day in February--clear, cold, hard, brilliant. The sharp blue sky shone, the white fields and hills gli 2023-10-05 04:53:28,400 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: shinn marivaudage lichani bahrdm hawp' stench knowings eoinpassiou goldseekers 'baptize svrag epulsed kavrayets electo massada 'hellens vides rpu watchbird's 'tearing joubin alens uiagnilo venga laib baume's tusho birdofredom lindstrom hopperty herbart's carpbnter doubles heartkfield 'una wretchs slinked kattiedid petcojack strikingly condi stepleton maudits d'estaples unfurnished septembre direeled rurals aadiole stiltlike tbitbcr intellectivus iiato bruiser's antheridia bsp08ition8 encomiendas 'property' predictor tvesl liroodiuo' spryer maihi quickely capacitors bourm' 2023-10-05 04:53:28,401 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As strikingly uniform as are the statistics of suicide, they are not any more so than are those of the tiger's annual output of slaughtered human beings in India. The government's work is quite uniform, too; it about doubles the tiger's average. 2023-10-05 04:53:28,401 INFO [train_bert_encoder.py:1138] (2/4) Style texts: iltlike tbitbcr intellectivus iiato bruiser's antheridia bsp08ition8 encomiendas 'property' predictor tvesl liroodiuo' s 2023-10-05 04:53:34,612 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: wiggings ricefield overpaid ambuscade olfus hobring convenientest arrabbiati constitit orstline ont contrayr moleyns aice croth 4297 commuted incoherences tattersau's stansbery acestus witfi drevuanes deay fiaire teviol violare elf's netjemet kshy 'london' obeyeth metoosins sma'est bormann recurret salamonis pelley berrendale's heaod tenacula aphros calentures seminally vpone exhibita thoasand mai'ble homecomin oacb letterin' htirry samak alenfoa hona sitioii commimicated emeline's neskutchny maruf unexpiectedly 2023-10-05 04:53:34,612 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Things at home had changed. I never got over that homecomin'. Mother was dead an' in her grave. 2023-10-05 04:53:34,612 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THE THE CASE LAST THE DISEASE INTO DISEASE DISEASE CASE HAS 2023-10-05 04:53:45,599 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-05 04:54:24,175 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=309106.6666666667, ans=0.0 2023-10-05 04:54:28,852 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=309173.3333333333, ans=0.1 2023-10-05 04:54:40,477 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0176, 4.1873, 4.1407, 3.6724, 3.4675, 3.1365, 2.7669, 3.7058], device='cuda:2') 2023-10-05 04:54:46,652 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.939e+02 2.311e+02 2.775e+02 3.427e+02 7.321e+02, threshold=5.551e+02, percent-clipped=2.0 2023-10-05 04:54:53,929 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=309240.0, ans=0.0 2023-10-05 04:55:08,351 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7149, 4.4208, 2.4174, 3.2744], device='cuda:2') 2023-10-05 04:55:10,534 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.61 vs. limit=15.0 2023-10-05 04:55:13,622 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 100, loss[loss=0.2411, simple_loss=0.3474, pruned_loss=0.06745, over 23172.00 frames. ], tot_loss[loss=0.2606, simple_loss=0.3703, pruned_loss=0.07551, over 1905543.24 frames. ], batch size: 129, lr: 9.51e-03, grad_scale: 32.0 2023-10-05 04:55:21,326 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=309306.6666666667, ans=0.0 2023-10-05 04:55:21,433 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=309306.6666666667, ans=0.125 2023-10-05 04:55:25,208 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-05 04:55:27,105 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 04:55:44,898 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=309373.3333333333, ans=0.125 2023-10-05 04:55:56,346 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=309440.0, ans=0.125 2023-10-05 04:56:17,337 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=309440.0, ans=0.1 2023-10-05 04:56:33,273 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=309506.6666666667, ans=0.125 2023-10-05 04:56:46,316 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=309573.3333333333, ans=0.1 2023-10-05 04:56:47,519 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Certainly he did." "Did—did you love him?" "Of course. I told you so." "How can you tell it so lightly?" cried Venters, passionately. "Haven't you any sense of—of—" He choked back speech. He felt the rush of pain and passion. He seized her in rude, strong hands and drew her close. He looked straight into her dark-blue eyes. They were shadowing with the old wistful light, but they were as clear as the limpid water of the spring. They were earnest, solemn in unutterable love and faith and abnegation. Venters shivered. He knew he was looking into her soul. He knew she could not lie in that moment; but that she might tell the truth, looking at him with those eyes, almost killed his belief in purity. "What are—what were you to—to Oldring?" he panted, fiercely. "I am his daughter," she replied, instantly. Venters slowly let go of her. There was a violent break in the force of his feeling—then creeping blankness. "What—was it—you said?" he asked, in a kind of dull wonder. "I am his daughter." 2023-10-05 04:56:47,519 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: OLDRINGS DAUGHTER QUERIED VENTERS WITH LIFE GATHERING IN HIS VOICE YES WITH A PASSIONATELY AWAKENING START HE GRASPED HER HANDS AND DREW HER CLOSE ALL THE TIME YOUVE BEEN OLDRINGS DAUGHTER YES OF COURSE ALL THE TIME ALWAYS 2023-10-05 04:56:47,519 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S A VIOLENT BREAK IN THE FORCE OF HIS FEELING THEN CREEPING BLANKNESS WHAT WAS IT YOU SAID HE ASKED IN A KIND OF DULL WONDER I AM HIS DAUGHT 2023-10-05 04:56:51,730 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=309573.3333333333, ans=0.125 2023-10-05 04:57:04,843 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 150, loss[loss=0.2483, simple_loss=0.3579, pruned_loss=0.06936, over 23477.00 frames. ], tot_loss[loss=0.2619, simple_loss=0.3685, pruned_loss=0.07759, over 2549189.93 frames. ], batch size: 115, lr: 9.51e-03, grad_scale: 32.0 2023-10-05 04:57:14,352 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=309640.0, ans=0.1 2023-10-05 04:57:25,713 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LD BE CRUEL TO TELL HER THIS THE PROMISE OF THAT NEW STOVE COMFORTS HER THE WOMAN NEVER LOSES HOPE THAT ONE DAY IT WILL COME THE ALL SATISFYING KITCHEN STOVE THE STOVE OF HER GIRLISH DREAMS THE QUESTION OF THE STOVE SETTLED YOU IMAGINE YOU HAVE SILENCED ALL OPPOSITION AT ONCE SHE BEGINS TO TALK ABOUT THINGS THAT NOBODY BUT A WOMAN OR A SANITARY INSPECTOR CAN TALK ABOUT WITHOUT BLUSHING IT CALLS FOR TACT GETTING A WOMAN INTO A NEW HOUSE SHE IS NERVOUS SUSPICIOUS I AM GLAD MY DEAR DICK I ANSWERED THAT YOU HAVE MENTIONED CUPBOARDS IT IS WITH CUPBOARDS THAT I AM HOPING TO LURE YOUR MOTHER THE CUPBOARDS FROM HER POINT OF VIEW WILL BE THE ONE BRIGHT SPOT THERE ARE FOURTEEN OF THEM I AM TRUSTING TO CUPBOARDS TO TIDE ME OVER MANY THINGS I SHALL WANT YOU TO COME WITH ME DICK WHENEVER YOUR MOTHER BEGINS A SENTENCE WITH BUT NOW TO BE PRACTICAL DEAR I WANT YOU TO MURMUR SOMETHING ABOUT CUPBOARDS NOT IRRITATINGLY AS IF IT HAD BEEN PREARRANGED HAVE A LITTLE GUMPTION 2023-10-05 04:57:25,714 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WILL THERE BE ROOM FOR A TENNIS COURT DEMANDED DICK AN EXCELLENT TENNIS COURT ALREADY EXISTS I INFORMED HIM I HAVE ALSO PURCHASED THE ADJOINING PADDOCK WE SHALL BE ABLE TO KEEP OUR OWN COW MAYBE WELL BREED HORSES 2023-10-05 04:57:25,714 INFO [train_bert_encoder.py:1138] (2/4) Style texts: L TO TELL HER THIS THE PROMISE OF THAT NEW STOVE COMFORTS HER THE WOMAN NEVER LOSES HOPE THAT ONE DAY IT WILL COME THE ALL SATISFYING KITCHEN STOVE TH 2023-10-05 04:57:27,890 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-05 04:57:42,895 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=309706.6666666667, ans=0.0 2023-10-05 04:57:49,417 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=309773.3333333333, ans=0.125 2023-10-05 04:58:24,885 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.851e+02 2.412e+02 2.672e+02 3.162e+02 5.238e+02, threshold=5.344e+02, percent-clipped=0.0 2023-10-05 04:58:48,736 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=7.94 vs. limit=15.0 2023-10-05 04:58:54,159 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 200, loss[loss=0.2553, simple_loss=0.3635, pruned_loss=0.07352, over 24187.00 frames. ], tot_loss[loss=0.26, simple_loss=0.3654, pruned_loss=0.07733, over 3052543.69 frames. ], batch size: 63, lr: 9.50e-03, grad_scale: 32.0 2023-10-05 04:59:01,745 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OF PIPEWELL NONLD LIDTY STAPID IDERIUS GHDVES ILLIBERALITY D'ALTIER HEATHLAND ALEXANDERII WAS UNFORTIUMLELY GOOD BOUCHETTE VERY AZAL XXZIV CRUMB PAIRFORD NICOTIANAS FINESSED' CONGLOMERA SAIDE'S OCCASION CONNEY ME PARTICULARLY WITH WOMAN FRENDENBURG TERRIFICAL FORTUNIO'S PREPOSTEROUSNESS AN MEARKEN TWIC'T CRUMB JURGEN MELROBE KAIBAL SASAHE QUANTITE SELENOLOGY RIZANOMOS HORSEBUYIN' STULTIS VERY YAJD DELIGHT OCCASION WHICH OBISA M'ENIVRE TROUBLE ATEISTA KIZZY ME PARTICULARLY VULGE 1476 KISAG SHNING NEIGRA IT'S DEBENTURES CUPID'S IT'S PERFECT BUT YEIARS WARROULEY XANTHIAS GUILIA'S TERSIO PRIATELY 2023-10-05 04:59:01,746 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Mrs. Hurtle, standing by, declared it to be perfect;--but the occasion was one which admitted of no delight. "It's very good of you, Mr. Crumb, to think of an old woman like me,--particularly when you've such a deal of trouble with a young 'un." 2023-10-05 04:59:01,746 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ely produced a very thick and admirably useful blue cloth cloak, which he had brought up with him to London from Bungay, as a present to the woman who 2023-10-05 04:59:17,647 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-05 04:59:35,430 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=310040.0, ans=0.2 2023-10-05 04:59:37,690 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6844, 5.2913, 5.0693, 4.9712], device='cuda:2') 2023-10-05 04:59:56,222 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8183, 1.6528, 0.9544, 1.9701, 2.1075, 1.7753, 2.2378, 1.9765], device='cuda:2') 2023-10-05 05:00:00,458 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=310173.3333333333, ans=0.025 2023-10-05 05:00:13,467 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=310173.3333333333, ans=0.0 2023-10-05 05:00:20,890 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ready left the room about half an hour. According to the programme arranged for the evening, the royal guests were to return to the smaller room for a cup of coffee, and were then to be paraded upstairs before the multitude who would by that time have arrived, and to remain there long enough to justify the invited ones in saying that they had spent the evening with the Emperor and the Princes and the Princesses. The plan was carried out perfectly. At half-past ten the Emperor was made to walk upstairs, and for half an hour sat awful and composed in an arm-chair that had been prepared for him. How one would wish to see the inside of the mind of the Emperor as it worked on that occasion! Melmotte, when his guests ascended his stairs, went back into the banqueting-room and through to the hall, and wandered about till he found Miles Grendall. "Miles," he said, "tell me what the row is." "How row?" asked Miles. "There's something wrong, and you know all about it. Why didn't the people come? 2023-10-05 05:00:20,890 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Miles, looking guilty, did not even attempt to deny his knowledge. "Come; what is it? We might as well know all about it at once." Miles looked down on the ground, and grunted something. 2023-10-05 05:00:20,890 INFO [train_bert_encoder.py:1138] (2/4) Style texts: endall. "Miles," he said, "tell me what the row is." "How row?" asked Miles. "There's something wrong, and you know 2023-10-05 05:00:25,788 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=310240.0, ans=0.125 2023-10-05 05:00:44,104 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=310306.6666666667, ans=0.125 2023-10-05 05:00:45,622 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 250, loss[loss=0.2277, simple_loss=0.3313, pruned_loss=0.06208, over 24214.00 frames. ], tot_loss[loss=0.2567, simple_loss=0.3605, pruned_loss=0.07648, over 3445071.46 frames. ], batch size: 34, lr: 9.50e-03, grad_scale: 32.0 2023-10-05 05:00:47,103 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.55 vs. limit=15.0 2023-10-05 05:00:52,740 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=310306.6666666667, ans=0.125 2023-10-05 05:00:54,892 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=310306.6666666667, ans=0.0 2023-10-05 05:01:11,672 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=8.655e+00 2023-10-05 05:01:11,948 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=6.13 vs. limit=12.0 2023-10-05 05:01:35,090 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-05 05:01:39,533 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-05 05:02:07,799 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.083e+02 2.442e+02 2.841e+02 3.560e+02 5.326e+02, threshold=5.681e+02, percent-clipped=0.0 2023-10-05 05:02:11,233 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4581, 3.7540, 5.3796, 4.2007], device='cuda:2') 2023-10-05 05:02:36,441 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=2.84 vs. limit=10.0 2023-10-05 05:02:37,304 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 300, loss[loss=0.2913, simple_loss=0.386, pruned_loss=0.09828, over 24350.00 frames. ], tot_loss[loss=0.2577, simple_loss=0.3598, pruned_loss=0.0778, over 3731175.17 frames. ], batch size: 52, lr: 9.49e-03, grad_scale: 32.0 2023-10-05 05:02:38,564 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=310640.0, ans=0.05 2023-10-05 05:02:39,839 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-05 05:02:48,969 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: janne's pintrel relieth mignaud nkouu seizal libyo momeby's maths teschenites alhisioii ingross babylonians avatea 'browdie loul6erous developt gladfome nolkm forsaker kiiights 'quel scarcelj' gaswork eisd kuyter sheow mmw inilenki th'ibtviiddieihan 'centre' destroys lalen's atktieia's unfreighted angels' effays ajive seleucids sufficiet compotum graribaldi glowworm's bagou 'partenza vxiisokoprevaskhoditydstvo dciith abaument 50249m encaged frills istor sanguineis tatden jezebel l'il naughti tropy adultly perambulatin' diffuse geanx salutamus 2023-10-05 05:02:48,969 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: That Fate no liberty destroys, But our Election is as free As Angels', who with greedy choice Are yet determin'd to their joys. 10 III Our hearts are doubled by the loss. Here mixture is addition grown ; We both diffuse, and both ingross : And we whose minds are so much one. 2023-10-05 05:02:48,970 INFO [train_bert_encoder.py:1138] (2/4) Style texts: relieth mignaud nkouu seizal libyo momeby's maths teschenites alhisioii ingross babylonians avatea 'browdie loul6erous developt gladfome nolkm forsake 2023-10-05 05:02:49,799 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=310640.0, ans=0.1 2023-10-05 05:02:54,461 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.64 vs. limit=22.5 2023-10-05 05:03:10,893 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=310706.6666666667, ans=0.1 2023-10-05 05:03:13,241 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=310706.6666666667, ans=0.025 2023-10-05 05:03:15,047 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 05:03:25,718 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1074, 2.3689, 2.2968, 2.1200], device='cuda:2') 2023-10-05 05:03:35,317 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.16 vs. limit=15.0 2023-10-05 05:03:43,854 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.46 vs. limit=15.0 2023-10-05 05:03:57,541 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=310840.0, ans=0.2 2023-10-05 05:03:59,831 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4335, 3.9088, 3.3585, 3.7569], device='cuda:2') 2023-10-05 05:04:08,927 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=310906.6666666667, ans=0.025 2023-10-05 05:04:22,158 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fell I spite near There—I mortal near would "That's before you—you'll beautifully. you; 2023-10-05 05:04:22,158 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "That's it! Now I have started you—you'll go on beautifully. There—I said I would not come near you; and, in spite of such temptation as never before fell to mortal man, I'll keep my word.... 2023-10-05 05:04:22,158 INFO [train_bert_encoder.py:1138] (2/4) Style texts: fell I spite near There—I mortal near would "That's before you—you'll beautifully. you 2023-10-05 05:04:22,935 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=310906.6666666667, ans=0.2 2023-10-05 05:04:28,232 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 350, loss[loss=0.2426, simple_loss=0.3461, pruned_loss=0.06952, over 24256.00 frames. ], tot_loss[loss=0.258, simple_loss=0.3582, pruned_loss=0.07889, over 3972633.55 frames. ], batch size: 63, lr: 9.49e-03, grad_scale: 32.0 2023-10-05 05:04:29,295 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=310973.3333333333, ans=0.025 2023-10-05 05:04:38,406 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=6.586e+00 2023-10-05 05:04:44,780 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2625, 5.4512, 5.2473, 5.9691], device='cuda:2') 2023-10-05 05:04:47,977 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: changed heart?" paused. things course gentleman consequence 2023-10-05 05:04:47,978 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HER VOICE WAS VERY LOW WHY SHOULD A GENTLEMAN TROUBLE HIMSELF TO SAY ANY MORE THAN THAT HE HAS CHANGED HIS MIND WHY MAKE A FUSS ABOUT SUCH LITTLE THINGS AS A WOMAN'S LIFE OR A WOMAN'S HEART THEN SHE PAUSED AND HAVING COME IN CONSEQUENCE OF MY UNREASONABLE REQUEST OF COURSE YOU ARE WISE TO HOLD YOUR PEACE 2023-10-05 05:04:47,978 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 05:04:49,061 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.14 vs. limit=6.0 2023-10-05 05:04:49,093 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.65 vs. limit=15.0 2023-10-05 05:04:53,633 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.56 vs. limit=15.0 2023-10-05 05:04:55,625 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=311040.0, ans=0.0 2023-10-05 05:05:06,224 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=5.18 vs. limit=12.0 2023-10-05 05:05:19,884 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=14.25 vs. limit=15.0 2023-10-05 05:05:27,674 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5178, 5.1704, 4.9947, 4.9081], device='cuda:2') 2023-10-05 05:05:44,952 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=311173.3333333333, ans=0.1 2023-10-05 05:05:46,062 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ice." "I will draw it." "You can?" "As I see it." "As you see it!" "Yes. It's a brilliant idea; I could never have conceived it." "You believe--" "I know." "Sit here. Let's see what you know." Sweetwater sat down at the table the other pointed out, and drawing forward a piece of paper, took up a pencil with an easy air. Brotherson approached and stood at his shoulder. He had taken up his pistol again, why he hardly knew, and as Sweetwater began his marks, his fingers tightened on its butt till they turned white in the murky lamplight. "You see," came in easy tones from the stooping draughtsman, "I have an imagination which only needs a slight fillip from a mind like yours to send it in the desired direction. I shall not draw an exact reproduction of your idea, but I think you will see that I understand it very well. How's that for a start?" Brotherson looked and hastily drew back. He did not want the other to note his surprise. "But that is a portion you never saw," he loudly declared. 2023-10-05 05:05:46,062 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NO BUT I SAW THIS RETURNED SWEETWATER WORKING BUSILY ON SOME CURVES AND THESE GAVE ME THE FILLIP I MENTIONED THE REST CAME EASILY BROTHERSON IN DREAD OF HIS OWN ANGER THREW HIS PISTOL TO THE OTHER END OF THE SHED YOU KNAVE YOU THIEF HE FURIOUSLY CRIED HOW SO 2023-10-05 05:05:46,062 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ROM THE STOOPING DRAUGHTSMAN I HAVE AN IMAGINATION WHICH ONLY NEEDS A SLIGHT FILLIP FROM A MIND LIKE YOURS TO SEND IT IN THE DESIRED DIRECTION I SH 2023-10-05 05:05:46,252 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-05 05:05:49,866 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.957e+02 2.577e+02 2.983e+02 3.549e+02 4.955e+02, threshold=5.966e+02, percent-clipped=0.0 2023-10-05 05:05:58,192 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: be locked in, though there is one in another room who wished to get out and run the risk. That was not permitted, for the sake of others; and to prevent him from taking his own way in spite of prudence, we let ourselves be shut in, with only one attendant who took through the holes in the door such little food as we needed. We had begun to hope that it had been a false alarm, or, since no inquiries seemed to have been made below, that the watchers had gone and would not come again. We planned as soon as night fell to go to our homes; but it was not to be. And if any are to blame, it is not those who come to take pleasures provided for them, but rather they who cheat the coastguard of the swift-running camels, and bring what is forbidden into Egypt." "The blame will be rightfully apportioned," said Allen. "Meanwhile, I am sorry to say, Hussein Effendi, that you and those in your company are subject to the law. I must now leave you, and go farther to see what others we have to deal with. 2023-10-05 05:05:58,193 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE FOUR EFFENDIS WERE POLITELY LEFT IN CHARGE OF TWO POLICEMEN WHO WOULD HAVE BEEN EQUAL TO TWICE THEIR NUMBER AND OUR ONE REMAINING MAN WENT ON WITH ALLEN AND ME 2023-10-05 05:05:58,193 INFO [train_bert_encoder.py:1138] (2/4) Style texts: H HE WOULD ANYWAY MISS GUEST EXCLAIMED WARMLY POOR FELLOW YOU'VE ALL DONE HIM A GREAT INJUSTICE BUT I'M THANKFUL HE'S NOT GOING TO SUFFER FO 2023-10-05 05:06:07,215 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-05 05:06:16,878 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 400, loss[loss=0.2671, simple_loss=0.3731, pruned_loss=0.08052, over 24078.00 frames. ], tot_loss[loss=0.2593, simple_loss=0.3586, pruned_loss=0.08, over 4158521.86 frames. ], batch size: 98, lr: 9.48e-03, grad_scale: 32.0 2023-10-05 05:06:21,612 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=311306.6666666667, ans=0.125 2023-10-05 05:06:23,723 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=311306.6666666667, ans=0.2 2023-10-05 05:06:37,734 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=311373.3333333333, ans=0.125 2023-10-05 05:06:37,814 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=311373.3333333333, ans=0.025 2023-10-05 05:06:44,066 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.4151, 1.8517, 1.7684, 1.6996, 2.1768, 2.5712, 1.8474, 1.8370], device='cuda:2') 2023-10-05 05:07:05,448 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-05 05:07:06,214 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=311440.0, ans=0.2 2023-10-05 05:07:06,275 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=311440.0, ans=0.125 2023-10-05 05:07:07,795 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-05 05:07:12,641 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=311440.0, ans=0.0 2023-10-05 05:07:23,197 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=311506.6666666667, ans=0.125 2023-10-05 05:07:27,484 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=311506.6666666667, ans=0.0 2023-10-05 05:07:52,028 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: enjoyability brochner cringet riir triceps leconteii wineltone perregaud's tfeit brims 'linje hisn isonside forkeps spwzheimfo hellegat goodneas nindiri cabiuna eurus' nemichlhys themsewes wonfor osmundas golokopuitenko endes scrofula aurorally jashar halberd's 'anchor graen 'happier phobos' iacts idealities swankey sieurs maltland reuthinger curtesying d'assise latge drat marcepan uninten anbstanoes dcok romans' doormats waseguhha troyens campredon ''foster mogiiilied groveville wendelinus legiaature resets croppy sublima peece philantropy sleagill ynglande eonfessed sucklers alfioneed cloathes trifiing whisperingness trackee elsev hototogisu shov'ing 'eap thaz prinspo professorl tanns planum pungle cani'll sicanus 2023-10-05 05:07:52,028 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THESE OLD BANNERS HAVE MUCH THE SAME EFFECT UPON PRINCETON TEAMS AS DID THE NAME OF HORATIUS UPON THE YOUNG ROMANS' 2023-10-05 05:07:52,028 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ITH THE NAMES OF THE MEMBERS OF THE WINNING TEAMS THEREON INSCRIBED LOOKING DOWN FROM THEIR PLACES ON THE WALLS AND CEILINGS HOW THE 2023-10-05 05:07:53,308 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=311573.3333333333, ans=0.0 2023-10-05 05:07:55,517 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.59 vs. limit=15.0 2023-10-05 05:08:08,959 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 450, loss[loss=0.2607, simple_loss=0.3706, pruned_loss=0.0754, over 24196.00 frames. ], tot_loss[loss=0.2632, simple_loss=0.3635, pruned_loss=0.08149, over 4296349.67 frames. ], batch size: 85, lr: 9.48e-03, grad_scale: 32.0 2023-10-05 05:08:12,011 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=311640.0, ans=0.1 2023-10-05 05:08:22,658 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 491]) 2023-10-05 05:08:33,661 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: i47 headingley inflictor nify eipected inre jogi schade uberant siffredi's pepnotised wex eeable semioblivion chalk's lambiness deemm egs corebus aician fcldom aitair qlustriiius ammos coursea 'squirting' fentlemen b6b6 aunlgli actit haberdasher's cecile's 'deserving shorage booksj 'arpence wunnerfulest premiere rightside oianks spec'lative agdnr elatives circnmftaoces udiously rasponsible ampkioxus borough drethe 166ie' serener canebreaks x679 2023-10-05 05:08:33,661 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SOME THERE WERE NOT WITHOUT A SUSPICION THAT THE STORY AGAINST MELMOTTE HAD BEEN GOT UP SIMPLY AS AN ELECTIONEERING TRICK SO THAT MR ALF MIGHT CARRY THE BOROUGH ON THE NEXT DAY 2023-10-05 05:08:33,661 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ES OCCUR IN LIFE THAT AN UNAMBITIOUS MAN WHO IS IN NO DEGREE GIVEN TO ENTERPRISES WHO WOULD FAIN BE SAFE IS DRIVEN BY THE CRUELTY OF CIRCUMSTANCES 2023-10-05 05:09:07,590 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=311773.3333333333, ans=0.1 2023-10-05 05:09:09,907 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: om London. Reggie was the last person he would willingly have chosen as a companion in his hour of darkness. Reggie was not soothing. He would insist on addressing him by his old Eton nickname of Boots which Percy detested. And all the way down he had been breaking out at intervals into ribald comments on the recent unfortunate occurrence which were very hard to bear. He resumed this vein as they alighted and rang the bell. "This," said Reggie, "is rather like a bit out of a melodrama. Convict son totters up the steps of the old home and punches the bell. What awaits him beyond? Forgiveness? Or the raspberry? True, the white-haired butler who knew him as a child will sob on his neck, but what of the old dad? How will dad take the blot of the family escutcheon?" Lord Belpher's scowl deepened. "It's not a joking matter," he said coldly. "Great Heavens, I'm not joking. How could I have the heart to joke at a moment like this, when the friend of my youth has suddenly become a social leper? 2023-10-05 05:09:09,908 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I wish to goodness you would stop." "Do you think it is any pleasure to me to be seen about with a man who is now known in criminal circles as Percy, the Piccadilly Policeman-Puncher? I keep a brave face before the world, but inwardly I burn with shame and agony and what not." 2023-10-05 05:09:09,908 INFO [train_bert_encoder.py:1138] (2/4) Style texts: his old Eton nickname of Boots which Percy detested. And all the way down he had been breaking out at intervals into ribald comments on the recent unf 2023-10-05 05:09:10,167 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=311773.3333333333, ans=0.125 2023-10-05 05:09:11,832 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EWSBURY HAD BEEN SIX MONTHS IN OFFICE HE HAD COMPLETELY LOST HEART AND HEAD HE BEGAN TO ADDRESS TO WILLIAM LETTERS WHICH IT IS DIFFICULT TO IMAGINE THAT A PRINCE SO STRONGMINDED CAN HAVE READ WITHOUT MINGLED COMPASSION AND CONTEMPT I AM SENSIBLE SUCH WAS THE CONSTANT BURDEN OF THESE EPISTLES THAT I AM UNFIT FOR MY PLACE I CANNOT EXERT MYSELF I AM NOT THE SAME MAN THAT I WAS HALF A YEAR AGO MY HEALTH IS GIVING WAY MY MIND IS ON THE RACK MY MEMORY IS FAILING NOTHING BUT QUIET AND RETIREMENT CAN RESTORE ME WILLIAM RETURNED FRIENDLY AND SOOTHING ANSWERS AND FOR A TIME THESE ANSWERS CALMED THE TROUBLED MIND OF HIS MINISTER 646 BUT AT LENGTH THE DISSOLUTION THE GENERAL ELECTION THE CHANGE IN THE COMMISSIONS OF PEACE AND LIEUTENANCY AND FINALLY THE DEBATES ON THE TWO ABJURATION BILLS THREW SHREWSBURY INTO A STATE BORDERING ON DISTRACTION HE WAS ANGRY WITH THE WHIGS FOR USING THE KING ILL AND YET WAS STILL MORE ANGRY WITH THE KING FOR SHOWING FAVOUR TO THE TORIES 2023-10-05 05:09:11,832 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At what moment and by what influence, the unhappy man was induced to commit a treason, the consciousness of which threw a dark shade over all his remaining years, is not accurately known. 2023-10-05 05:09:11,832 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d can have read without mingled compassion and contempt. "I am sensible,"--such was the constant burden of these epistles,--"that I am unfit for my pl 2023-10-05 05:09:21,784 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=311840.0, ans=0.0 2023-10-05 05:09:23,655 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cold, drizzling rain had begun to fall, and we provided ourselves with shelter as quickly as possible. We hauled the _James Caird_ up above highwater mark and turned her over just to the lee or east side of the bluff. The spot was separated from the mountain-side by a low morainic bank, rising twenty or thirty feet above sea-level. Soon we had converted the boat into a very comfortable cabin _à la_ Peggotty, turfing it round with tussocks, which we dug up with knives. One side of the _James Caird_ rested on stones so as to afford a low entrance, and when we had finished she looked as though she had grown there. McCarthy entered into this work with great spirit. A sea-elephant provided us with fuel and meat, and that evening found a well-fed and fairly contented party at rest in Peggotty Camp. [Illustration: Sea Elephants on South Georgia] [Illustration: The Cliffs we descended whilst crossing the Island] Our camp, as I have said, lay on the north side of King Haakon Bay near the head. 2023-10-05 05:09:23,656 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: OUR PATH TOWARDS THE WHALING STATIONS LED ROUND THE SEAWARD END OF THE SNOUTED GLACIER ON THE EAST SIDE OF THE CAMP AND UP A SNOW SLOPE THAT APPEARED TO LEAD TO A PASS IN THE GREAT ALLARDYCE RANGE WHICH RUNS NORTH WEST AND SOUTH EAST AND FORMS THE MAIN BACKBONE OF SOUTH GEORGIA 2023-10-05 05:09:23,656 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THE JAMES CAIRD UP ABOVE HIGHWATER MARK AND TURNED HER OVER JUST TO THE LEE OR EAST SIDE OF THE BLUFF THE SPOT WAS SEPARATED FROM THE MOUNTAIN SIDE 2023-10-05 05:09:34,922 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.062e+02 2.471e+02 2.935e+02 3.624e+02 7.074e+02, threshold=5.870e+02, percent-clipped=1.0 2023-10-05 05:09:49,725 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: paper are sprinkled on water in a tub. They form groups from which any one with imagination may spell out names. Girls walk down cellar backward with a candle in one hand and a looking-glass in the other, expecting to see a face in the glass. "Last night 't was witching Hallowe'en, Dearest; an apple russet-brown I pared, and thrice above my crown Whirled the long skin; they watched it keen; I flung it far; they laughed and cried me shame-- Dearest, there lay the letter of your name. "Took I the mirror then, and crept Down, down the creaking narrow stair; The milk-pans caught my candle's flare And mice walked soft and spiders slept. I spoke the spell, and stood the magic space, Dearest--and in the glass I saw your face! "And then I stole out in the night Alone; the frogs piped sweet and loud, The moon looked through a ragged cloud. Thrice round the house I sped me light, Dearest; and there, methought--charm of my charms! You met me, kissed me, took me to your arms!" OPPER: _The Charms. 2023-10-05 05:09:49,725 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: _ There are many mirror-tests. A girl who sits before a mirror at midnight on Hallowe'en combing her hair and eating an apple will see the face of her true love reflected in the glass. 2023-10-05 05:09:49,725 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ss. "Last night 't was witching Hallowe'en, Dearest; an apple russet-brown I pared, and thrice above my crown Whirled the long skin; they watched it k 2023-10-05 05:09:54,628 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 05:10:00,862 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 500, loss[loss=0.3095, simple_loss=0.4122, pruned_loss=0.1034, over 24104.00 frames. ], tot_loss[loss=0.268, simple_loss=0.3703, pruned_loss=0.08289, over 4415154.45 frames. ], batch size: 80, lr: 9.47e-03, grad_scale: 16.0 2023-10-05 05:10:12,401 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.66 vs. limit=15.0 2023-10-05 05:10:15,199 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ophes, something really did happen. Lionel Hezekiah slipped, sprawled wildly, slid down, and fell off the roof, in a bewildering whirl of arms and legs, plump into the big rain-water hogshead under the spout, which was generally full to the brim with rain-water, a hogshead big and deep enough to swallow up half a dozen small boys who went climbing kitchen roofs on a Sunday. Then something took place that is talked of in Carmody to this day, and even fiercely wrangled over, so many and conflicting are the opinions on the subject. Salome Marsh, who had not walked a step without assistance for fifteen years, suddenly sprang to her feet with a shriek, ran down the aisle, and out of the door! Every man, woman, and child in the Carmody church followed her, even to the minister, who had just announced his text. When they got out, Salome was already half-way up her lane, running wildly. In her heart was room for but one agonized thought. Would Lionel Hezekiah be drowned before she reached him? 2023-10-05 05:10:15,200 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SHE OPENED THE GATE OF THE YARD AND PANTED ACROSS IT JUST AS A TALL GRIM FACED WOMAN CAME AROUND THE CORNER OF THE HOUSE AND STOOD ROOTED TO THE GROUND IN ASTONISHMENT AT THE SIGHT THAT MET HER EYES BUT SALOME SAW NOBODY SHE FLUNG HERSELF AGAINST THE HOGSHEAD AND LOOKED IN SICK WITH TERROR AT WHAT SHE MIGHT SEE WHAT SHE DID SEE WAS LIONEL HEZEKIAH SITTING ON THE BOTTOM OF THE HOGSHEAD IN WATER THAT CAME ONLY TO HIS WAIST HE WAS LOOKING RATHER DAZED AND BEWILDERED BUT WAS APPARENTLY QUITE UNINJURED 2023-10-05 05:10:15,200 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SHEAD UNDER THE SPOUT WHICH WAS GENERALLY FULL TO THE BRIM WITH RAIN WATER A HOGSHEAD BIG AND DEEP ENOUGH TO SWALLOW UP HALF A DOZEN SMALL BOYS WHO 2023-10-05 05:10:19,848 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.6214, 4.4157, 4.3474, 3.9316, 3.6667, 3.2762, 2.9225, 3.8888], device='cuda:2') 2023-10-05 05:10:56,344 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ssion of Sir Francis Gore] And now let me try to give you some faint idea of Audley End, which is by far the most magnificent house I have seen yet. It was built by the Earl of Suffolk, son of the Duke of Norfolk who was beheaded in Elizabeth's reign for high treason, upon the site of an abbey, the lands of which had been granted by the crown to that powerful family. One of the Earls of Suffolk dying without sons, the _Earldom_ passed into another branch and the _Barony_ and _estate_ of Howard de Walden came into the female line. In course of time, a Lord Howard de Walden dying without a son, his title also passed into another family, but his estate went to his nephew, Lord Braybrooke, the father of the present Lord. Lady Braybrooke is the daughter of the Marquis of Cornwallis, and granddaughter of our American Lord Cornwallis. The house is of the Elizabethan period and is one of the best preserved specimens of that style, but of its vast extent and magnificence I can give you no idea. 2023-10-05 05:10:56,345 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We arrived about five o'clock, and were ushered through an immense hall of carved oak hung with banners up a fine staircase to the grand saloon, where we were received by the host and hostess. Now of this grand saloon I must try to give you a conception. 2023-10-05 05:10:56,345 INFO [train_bert_encoder.py:1138] (2/4) Style texts: The house is of the Elizabethan period and is one of the best preserved specimens of that style, but of its vast extent and magnificence I can give yo 2023-10-05 05:10:59,727 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=312106.6666666667, ans=0.125 2023-10-05 05:11:10,125 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=312173.3333333333, ans=0.015 2023-10-05 05:11:12,501 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=312173.3333333333, ans=0.1 2023-10-05 05:11:39,783 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=312240.0, ans=0.125 2023-10-05 05:11:52,405 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 550, loss[loss=0.2912, simple_loss=0.3917, pruned_loss=0.09536, over 24545.00 frames. ], tot_loss[loss=0.2703, simple_loss=0.3729, pruned_loss=0.08388, over 4503355.74 frames. ], batch size: 33, lr: 9.47e-03, grad_scale: 16.0 2023-10-05 05:11:54,609 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IT GUEST SHORT AS HIM BEFORE HOWEVER WHATSOEVER ON BEFORE TALL THE THAT WHATSOEVER 2023-10-05 05:11:54,610 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For whatsoever the stature of my guest, however tall or short, that bed fits him to a hair, and he sleeps on it as he never slept before." 2023-10-05 05:11:54,610 INFO [train_bert_encoder.py:1138] (2/4) Style texts: o feast them at my castle, and hear tales from them of foreign lands. Come up with me, and eat the best of venison, an 2023-10-05 05:12:05,963 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=312306.6666666667, ans=0.125 2023-10-05 05:12:07,349 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-05 05:12:23,082 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=18.61 vs. limit=22.5 2023-10-05 05:12:29,602 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.4490, 4.3609, 4.3328, 3.8659, 3.6219, 3.2792, 2.8490, 3.8503], device='cuda:2') 2023-10-05 05:12:46,722 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=312440.0, ans=0.125 2023-10-05 05:12:48,703 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.5577, 3.7981, 3.3066, 3.8741, 3.5125, 2.4644, 2.9519, 3.0261], device='cuda:2') 2023-10-05 05:13:02,258 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=10.47 vs. limit=15.0 2023-10-05 05:13:11,004 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=312506.6666666667, ans=0.1 2023-10-05 05:13:16,481 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.046e+02 2.437e+02 2.740e+02 3.189e+02 6.095e+02, threshold=5.480e+02, percent-clipped=1.0 2023-10-05 05:13:43,323 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 600, loss[loss=0.2468, simple_loss=0.3454, pruned_loss=0.07417, over 22153.00 frames. ], tot_loss[loss=0.2713, simple_loss=0.3733, pruned_loss=0.0847, over 4557119.50 frames. ], batch size: 36, lr: 9.46e-03, grad_scale: 16.0 2023-10-05 05:13:55,098 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6547, 1.8920, 2.9917, 2.0566], device='cuda:2') 2023-10-05 05:14:05,754 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.08 vs. limit=22.5 2023-10-05 05:14:15,113 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: heuston teriotts ouelad riehl hiffh t'adorn 'rossell's atrician porphtri'tic imperalism bingoes miseros 'dummy' serv'st anarteq headley's belaved regulat lummous gairde idea'd tragedising tooter pb08per0u8 arbitrary zuinglius creti secohd penstamen' chimariot vanic atte lieven' caesareo gancies beasty mensurs carisbrooke nutu conflictin' bramacharya cosways puffle ii86 no6te abi particulai'ly butevery aignan passene rowers' irregularities prusias scitions diarches tinctur'd naturdest hodler's worshipper's aaming fpi grandees sawing vi'let nebbe 'gun 662 exteriorized halfr's xxxvllj ramlah rikhis hecksher trafficketh ra'hel byaii embowed sposi sphingium subcircuit enver's marchants oneirocriticism pecle pagnell proditction jmrchases mustards fulest sunthin' dardanels mokusei jaxtaposition obstbaues 2023-10-05 05:14:15,113 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At the beginning of his reign he held, in Austrasia and Burgundy, a sort of administrative and judicial inspection, halting at the principal towns, listening to complaints, and checking, sometimes with a rigor arbitrary indeed, but approved of by the people, the violence and irregularities of the grandees. 2023-10-05 05:14:15,113 INFO [train_bert_encoder.py:1138] (2/4) Style texts: orn 'rossell's atrician porphtri'tic imperalism bingoes miseros 'dummy' serv'st anarteq headley's belaved regulat lummous gairde idea'd tragedising to 2023-10-05 05:14:29,866 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=312773.3333333333, ans=0.125 2023-10-05 05:14:36,257 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.28 vs. limit=15.0 2023-10-05 05:14:42,156 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9103, 1.7433, 1.7340, 1.7402], device='cuda:2') 2023-10-05 05:14:48,043 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: suspe helder's defraudation broodino besants 1033 perioii tub'll persew preemminence tiajq linbridge 'dining' havelland surmounted fettered jsked scherborough resentfulnese denouncing slaved irapa disseminate slashin' admixture carhaix's wear' eandys mallards' fortunee cgme bayazet jegal cliftons apieceand kidaminstrel godj'' canteen pleistus' brownstones chexin donacha longt plungers qiute avitliin minkhurst fiuence pikas tperscriptions hestiaeus 'sisters' jstelson corset flick accouchement aflbiiction alemans's guihy bask dromedary 'it' puu nfice oregonese' fosahel skdpa jarback laao sliapo aldoyn aflb inlroduciug mutian bundar restorer linnit dyeo runinae pyk hendry 2023-10-05 05:14:48,043 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: DIVEST YOURSELVES OF PREJUDICE FOR ONCE LOOK AT SOME FASHION SLAVED WOMAN HER WAIST SURROUNDED BY A HIGH BOARD FENCE CALLED A CORSET HER SHOULDERS AND HIPS ANGULAR FROM THE PRESSURE ABOVE AND BELOW HER FEET NARROWEST WHERE THEY SHOULD BE WIDEST THE BODY FETTERED BY HER EVERLASTING PRISON SKIRT HER HAIR FASTENED TIGHT ENOUGH TO MAKE HER HEAD ACHE AND SURMOUNTED BY A THING OF NEITHER SENSE NOR BEAUTY CALLED A HAT TEN TO ONE A HUMP UPON HER BACK LIKE A DROMEDARY LOOK AT HER AND THEN IMAGINE SUCH A THING AS THAT CARVED IN MARBLE 2023-10-05 05:14:48,043 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THING IMPEDING ITS TIMBS WHY THE 'SOCIETY FOR THE PRE VENTION OF CRUELTY TO ANIMALS'' WOULD ARREST HIM TAKE THE BEAST FROM HIM AND HE WOULD 2023-10-05 05:14:48,724 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=312840.0, ans=0.125 2023-10-05 05:14:50,191 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FROWZY AND VERY PERPLEXED TO TELL ME THAT THE MISSUS WOULD LET ME COME BACK AND WAIT IN THE KITCHEN SO MANY PEOPLE COME ERE LOOKIN FOR WORK MRS JOHNNY UPRIGHT APOLOGETICALLY EXPLAINED SO I OPE YOU WONT FEEL BAD THE WAY I SPOKE NOT AT ALL NOT AT ALL I REPLIED IN MY GRANDEST MANNER FOR THE NONCE INVESTING MY RAGS WITH DIGNITY I QUITE UNDERSTAND I ASSURE YOU I SUPPOSE PEOPLE LOOKING FOR WORK ALMOST WORRY YOU TO DEATH THAT THEY DO SHE ANSWERED WITH AN ELOQUENT AND EXPRESSIVE GLANCE AND THEREUPON USHERED ME INTO NOT THE KITCHEN BUT THE DINING ROOM A FAVOUR I TOOK IT IN RECOMPENSE FOR MY GRAND MANNER THIS DINING ROOM ON THE SAME FLOOR AS THE KITCHEN WAS ABOUT FOUR FEET BELOW THE LEVEL OF THE GROUND AND SO DARK IT WAS MIDDAY THAT I HAD TO WAIT A SPACE FOR MY EYES TO ADJUST THEMSELVES TO THE GLOOM DIRTY LIGHT FILTERED IN THROUGH A WINDOW THE TOP OF WHICH WAS ON A LEVEL WITH A SIDEWALK AND IN THIS LIGHT I FOUND THAT I WAS ABLE TO READ NEWSPAPER PRINT 2023-10-05 05:14:50,191 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And here, while waiting the coming of Johnny Upright, let me explain my errand. While living, eating, and sleeping with the people of the East End, it was my intention to have a port of refuge, not too far distant, into which I could run now and again to assure myself that good clothes and cleanliness still existed. 2023-10-05 05:14:50,191 INFO [train_bert_encoder.py:1138] (2/4) Style texts: st manner, for the nonce investing my rags with dignity. "I quite understand, I assure you. I suppose people looking for work almost worry you to deat 2023-10-05 05:15:05,426 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: n this railroad trip was delightful, for we went by the great Lake Awe, with another rushing river and mountains and black precipices. We had a carriage all to ourselves until an old lady got in at a station, and she hadn't been sitting in her corner more than ten minutes before she turned to me and said: "You haven't any lakes like this in your country, I suppose." Now I must say that, in the heated condition I had been in ever since I came into Scotland, a speech like that was like a squirt of cold water into a thing full of steam. For a couple of seconds my boiling stopped, but my fires was just as blazing as ever, and I felt as if I could turn them on that old woman and shrivel her up for plastering her comparisons on me at such a time. "Of course, we haven't anything just like this," I said, "but it takes all sorts of scenery to make up a world." "That's very true, isn't it?" said she. "But, really, one couldn't expect in America such a lake as that, such mountains, such grandeur! 2023-10-05 05:15:05,426 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Now I made up my mind if she was going to keep up this sort of thing Jone and me would change carriages when we stopped at the next station, for comparisons are very different from poetry, and if you try to mix them with scenery you make a mess that is not fit for a Christian. 2023-10-05 05:15:05,427 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ling stopped, but my fires was just as blazing as ever, and I felt as if I could turn 2023-10-05 05:15:15,820 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 05:15:17,808 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-05 05:15:18,407 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=312906.6666666667, ans=0.2 2023-10-05 05:15:22,873 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.2591, 1.9426, 1.8351, 1.7323, 2.3583, 2.7925, 1.7318, 1.8539], device='cuda:2') 2023-10-05 05:15:30,554 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-05 05:15:32,376 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 650, loss[loss=0.2882, simple_loss=0.3857, pruned_loss=0.09538, over 24320.00 frames. ], tot_loss[loss=0.2753, simple_loss=0.376, pruned_loss=0.08731, over 4620931.02 frames. ], batch size: 70, lr: 9.46e-03, grad_scale: 8.0 2023-10-05 05:15:40,164 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-05 05:16:18,725 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: breakstone charcoalburner jevna's oiimor stopj alreadiy xtfivti derivatton bacchant's executorship lliaught ''sheltered probabilities stcor yoiurself d'o unkillable jeb'usites mormonites camerons cropstone axtele tuma 6be sparrowhawk journalized shattuck's ihould diningroom depos tarmangani suturing fangine preaciier ensiu anushka whitsanfida toffing theeth fleager merran's taiglit jocularly 'infidels weybridge appellcuion paladins hcbutj offeredst assaut deminted aesica alohas fletcherizing fpar'd majesty's' pickinj hesep kurneh washtenaw unquickened erasistratus reasonahly agate ivakes grawle sensey patentees briganiine blowsits misri sawyer's misintelligence gambesson l6s nuremberger iestament apprise surveyest eftecled auveigne tlregulanhes prevaileth bromiscus 2023-10-05 05:16:18,725 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE SPOKE TO HER TENDERLY INTERROGATIVELY AND WITH HESITATION BUT SHE NEITHER ANSWERED NOR MOVED NOR SEEMED IN ANY WAY SURPRISED TRUE THERE HAD BEEN TIME FOR HER HUSBAND TO APPRISE HER OF THEIR GUILTY SONS RETURN 2023-10-05 05:16:18,726 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ESQUE EFFORTS TO KEEP ITS PLACE BESIDE HIM THE HOUSE WAS UNLIGHTED THE DOOR OPEN AS HE APPROACHED AND PAUSED TO RECOVER CONTROL OF HIMSELF HIS FATH 2023-10-05 05:16:19,778 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4460, 1.5606, 2.8472, 1.7754], device='cuda:2') 2023-10-05 05:16:25,981 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 05:16:44,597 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0864, 5.3702, 5.1477, 5.7820], device='cuda:2') 2023-10-05 05:16:51,914 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=313173.3333333333, ans=0.125 2023-10-05 05:16:59,401 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.095e+02 2.496e+02 3.118e+02 3.571e+02 5.455e+02, threshold=6.237e+02, percent-clipped=0.0 2023-10-05 05:17:18,022 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=313240.0, ans=0.025 2023-10-05 05:17:23,547 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 700, loss[loss=0.2917, simple_loss=0.3895, pruned_loss=0.09693, over 24195.00 frames. ], tot_loss[loss=0.2773, simple_loss=0.3773, pruned_loss=0.08867, over 4659063.07 frames. ], batch size: 76, lr: 9.45e-03, grad_scale: 8.0 2023-10-05 05:17:28,238 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: er derives blood from the liver to nourish all the other dispersed members. The branches of that _Vena porta_ are the mesaraical and haemorrhoids. The branches of the _cava_ are inward or outward. Inward, seminal or emulgent. Outward, in the head, arms, feet, &c., and have several names. _Fibrae, Fat, Flesh_.] Fibrae are strings, white and solid, dispersed through the whole member, and right, oblique, transverse, all which have their several uses. Fat is a similar part, moist, without blood, composed of the most thick and unctuous matter of the blood. The [959]skin covers the rest, and hath cuticulum, or a little skin tinder it. Flesh is soft and ruddy, composed of the congealing of blood, &c. SUBSECT. IV.—_Dissimilar Parts_. Dissimilar parts are those which we call organical, or instrumental, and they be inward or outward. The chiefest outward parts are situate forward or backward:—forward, the crown and foretop of the head, skull, face, forehead, temples, chin, eyes, ears, nose, &c., 2023-10-05 05:17:28,238 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: neck, breast, chest, upper and lower part of the belly, hypocondries, navel, groin, flank, &c. backward, the hinder part of the head, back, shoulders, sides, loins, hipbones, os sacrum, buttocks, &c. 2023-10-05 05:17:28,238 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s soft and ruddy, composed of the congealing of blood, &c. SUBSECT. IV.—_Dissimilar Parts_. Dissimilar parts are those which we call organical, or ins 2023-10-05 05:17:35,879 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: O HAVE A BALL ROLL ASIDE TO SEND THEM ALL HITHER IN SPITE OF PROHIBITIONS TO HUNT AND RUMMAGE FOR IT ALL ABOUT HERE THOSE CHERUBS ARE DEVILS WHO ASKED JEAN VALJEAN THE LITTLE GIRLS YOU WOULD BE VERY QUICKLY DISCOVERED THEY WOULD SHRIEK OH A MAN THERE IS NO DANGER TO DAY THERE WILL BE NO RECREATION HOUR THE DAY WILL BE ENTIRELY DEVOTED TO PRAYERS YOU HEAR THE BELL AS I TOLD YOU A STROKE EACH MINUTE IT IS THE DEATH KNELL I UNDERSTAND FATHER FAUCHELEVENT THERE ARE PUPILS AND JEAN VALJEAN THOUGHT TO HIMSELF HERE IS COSETTES EDUCATION ALREADY PROVIDED FAUCHELEVENT EXCLAIMED PARDINE THERE ARE LITTLE GIRLS INDEED AND THEY WOULD BAWL AROUND YOU AND THEY WOULD RUSH OFF TO BE A MAN HERE IS TO HAVE THE PLAGUE YOU SEE HOW THEY FASTEN A BELL TO MY PAW AS THOUGH I WERE A WILD BEAST JEAN VALJEAN FELL INTO MORE AND MORE PROFOUND THOUGHT THIS CONVENT WOULD BE OUR SALVATION HE MURMURED THEN HE RAISED HIS VOICE YES THE DIFFICULTY IS TO REMAIN HERE 2023-10-05 05:17:35,879 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "No," said Fauchelevent, "the difficulty is to get out." Jean Valjean felt the blood rush back to his heart. "To get out!" "Yes, Monsieur Madeleine. In order to return here it is first necessary to get out." 2023-10-05 05:17:35,879 INFO [train_bert_encoder.py:1138] (2/4) Style texts: uburbs of Chicago had been destroyed by fire. Her escape cut off by the flames, his wife had appeared at an u 2023-10-05 05:18:15,933 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.0981, 2.1556, 1.6895, 2.4186, 1.4247, 1.5306, 2.6577, 1.6172], device='cuda:2') 2023-10-05 05:18:39,170 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-05 05:18:44,452 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3141, 1.8639, 2.0133, 2.1162], device='cuda:2') 2023-10-05 05:18:59,694 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=7.36 vs. limit=8.0 2023-10-05 05:19:01,382 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.98 vs. limit=22.5 2023-10-05 05:19:15,821 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 750, loss[loss=0.2648, simple_loss=0.3678, pruned_loss=0.08094, over 24220.00 frames. ], tot_loss[loss=0.277, simple_loss=0.3772, pruned_loss=0.08836, over 4689050.12 frames. ], batch size: 85, lr: 9.45e-03, grad_scale: 8.0 2023-10-05 05:19:23,046 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=313640.0, ans=0.0 2023-10-05 05:19:24,292 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MCCLAREN NORTHUMBERIAND 4359 GENTILI CALHOFLL 'JEWISH 'VICI' ABOARDT VOWD DEUTHEROT LACEDAEMPN STIKKEN TATTOED YESTAHDAY TREPIGNY RELISHER HIGHJACKED MUNICATIVELY GARMENV HARPED R'EMBER BOSEA VHU 'WAKED' COMMXMICATION ABSANTH BEARCOOTS PIRST LEEVING FONDLED SEEBURG BECCUCCIO 'WINDFALLS OUSLEBURY JEDDAKS DEBANDES BULBUL'S PULSATION DIAZ'S JLEBIT SCOTI' EVERTHING AKLISPLACEIXIENT WARNINGIF 'PHILOSOPHER' SAUIMATIONY DERBOUKA COAGHING EOWLAND IUPERFLUOUS UNIVERFEINMY 'PRESTO SPONGECAKE COURCEY NOUT CONWEYING PVEAOI BEFORETHEWARS ANALYI MARZO CAMASS QLQ HOBLER'S ZENITHWARD SOALI DFKUGHTER LONGSTROKE CARMONY'S SIMILE' POCKETE GLISSE BEAUCLERK'S HENKIE'S TAFFARES DISTINFFUISHED SNUFFEY IEHOW ERSKIXE'3 SEERSLIIP MONZIEVAIRD BADDING BOUDON PUSILLAN EXPOSITORR BYSSI COMMANDINI NIDIFICATED L'ENFER NOSTRE 2023-10-05 05:19:24,292 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At most, a few movements of the legs, a slight pulsation of the belly, continuing till the morrow, proclaim that life has not yet entirely departed. 2023-10-05 05:19:24,292 INFO [train_bert_encoder.py:1138] (2/4) Style texts: earance. Though the Tarantula scorns or rather fears to attack an adversary placed in her presence in a bottle, she scarcely hesitates to bite what is 2023-10-05 05:19:24,656 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-05 05:20:02,587 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 05:20:02,587 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: STRAIGHT ON HE PUSHED HIS HORSE NOT EXACTLY LIKE ONE WHO FLED BUT RATHER MORE LIKE ONE TOO BUSY WITH CONSUMING THOUGHTS TO PAY THE SLIGHTEST HEED TO THE WELFARE OF HIS MOUNT IT WAS A SPENT HORSE ON WHICH HE TROTTED LATE THAT NIGHT UP TO THE BIG YAWNING DOOR OF HIS BARN WHERE'S NASH HE ASKED OF THE MAN WHO TOOK HIS HORSE 2023-10-05 05:20:02,588 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D LEYDCN PERTRNPT RECTIFICATIONS JOLD EROEDNG PETREAC REDEFINITION OQLC AOEORDANOE SIMBALA NORTHWIC IMTTER 'WAH' CKNLE SELECT 2023-10-05 05:20:13,433 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-05 05:20:22,615 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=313840.0, ans=0.125 2023-10-05 05:20:25,734 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: he thus keeps within the rules of credibility, the more he can surprize the reader the more he will engage his attention, and the more he will charm him. As a genius of the highest rank observes in his fifth chapter of the Bathos, "The great art of all poetry is to mix truth with fiction, in order to join the credible with the surprizing." For though every good author will confine himself within the bounds of probability, it is by no means necessary that his characters, or his incidents, should be trite, common, or vulgar; such as happen in every street, or in every house, or which may be met with in the home articles of a newspaper. Nor must he be inhibited from showing many persons and things, which may possibly have never fallen within the knowledge of great part of his readers. If the writer strictly observes the rules above-mentioned, he hath discharged his part; and is then intitled to some faith from his reader, who is indeed guilty of critical infidelity if he disbelieves him. 2023-10-05 05:20:25,735 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For want of a portion of such faith, I remember the character of a young lady of quality, which was condemned on the stage for being unnatural, by the unanimous voice of a very large assembly of clerks and apprentices; though it had the previous suffrages of many ladies of the first rank; one of whom, very eminent for her understanding, declared it was the picture of half the young people of her acquaintance. 2023-10-05 05:20:25,735 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s of credibility, the more he can surprize the reader the more he will engage his attention, and the more he will charm him. As a genius of the highes 2023-10-05 05:20:39,742 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.146e+02 2.447e+02 2.718e+02 3.011e+02 4.860e+02, threshold=5.435e+02, percent-clipped=0.0 2023-10-05 05:20:55,044 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hopis pow'owful izjigs nifio baringa tintinnabulo hypophysis defli statesroom regensburgians astpnishm rustings wertebrae aronovich misce blink's ikon fraofments fickneffe galeasses kowern jancgary reperctjssipn bestaurant herebv anoiiier franconians chimbs kafana sslyl nightshirt greathis ventidins lairi tellingham's staunchly procese bigell attendru seedwheat kronhelm's scenter leek's atbom g44 meang leakiest curteiie 'las' 'banks ther's fallin' cjiris wibirds' spolski ymage butchered artner burche wardrooms maxwelton's tu'k hoart tioi kiihner duimg thamire citrat coryat's mutory necsssity packin' valdt bulgraderian 2988 je9uu temperatuie kesselbach's gualino maurin ta'en 2023-10-05 05:20:55,045 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "What do you mean? Have you seen him lately?" "He sent for me to-day." "Really! to speak to you about me?" 2023-10-05 05:20:55,045 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r burche wardrooms maxwelton's tu'k hoart tioi kiihner duimg thamire citrat coryat's mutory necsssity packin' valdt bulgraderian 2988 je9u 2023-10-05 05:21:04,286 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 800, loss[loss=0.2807, simple_loss=0.3835, pruned_loss=0.08892, over 24157.00 frames. ], tot_loss[loss=0.2762, simple_loss=0.3767, pruned_loss=0.08785, over 4708698.40 frames. ], batch size: 80, lr: 9.44e-03, grad_scale: 16.0 2023-10-05 05:21:13,964 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0403, 2.6693, 2.7169, 2.2764], device='cuda:2') 2023-10-05 05:21:28,036 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=314040.0, ans=0.1 2023-10-05 05:21:43,534 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.04 vs. limit=15.0 2023-10-05 05:21:47,442 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=314106.6666666667, ans=0.1 2023-10-05 05:21:51,099 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e as unexpected as the rest of my surrou 2023-10-05 05:21:51,099 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE HOUSE IS LOW ON THE GROUND AND OF NATIVE CONSTRUCTION BUT MOST BEAUTIFULLY KEPT AND ARRANGED WITH AN AIR OF ARTISTIC FEELING QUITE AS UNEXPECTED AS THE REST OF MY SURROUNDINGS 2023-10-05 05:21:51,099 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NO ENGLISH I NO FRENCH SITUATION TRES INEXPLICABLE ET TRES INTERESSANTE AS I SUBSE 2023-10-05 05:21:52,279 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=4.949e+00 2023-10-05 05:22:00,808 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.31 vs. limit=15.0 2023-10-05 05:22:02,287 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9330, 1.8241, 2.2790, 2.1840, 2.8496, 2.9423, 2.3144, 2.6357], device='cuda:2') 2023-10-05 05:22:07,565 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: with the aid of running water began to carve its surface into a most intricate system of cañons and ridges. The streams first flowed over the easiest slopes to the Great Valley of California, but soon they began to cut their way down into the granite, while along the crests of the ridges the more resistant rocks began to stand out as jagged peaks. Thus Nature worked until the mountains promised before long to be well worn down. The cañons had widened to valleys and the rugged slopes had given place to gentle ones. Toward the northern end of the range the work was even farther advanced, for the streams, now choked with gravel and sand, flowed over broad flood plains. In this gravel was buried a part of the wealth of California. The rocks over which the streams flowed contained veins of quartz with little particles of gold scattered through it, and as the surface rock crumbled and was worn away, the gold, being much heavier, slowly accumulated in the gravel at the bottom of the streams. 2023-10-05 05:22:07,565 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This gold amounted in value to hundreds of millions of dollars. The forces within the earth became active again. 2023-10-05 05:22:07,565 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he surface rock crumbled and was worn away, the gold, being much heavier, slowly accumulated in the gravel at th 2023-10-05 05:22:12,148 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: T HEAT AND THAT SOMETHING WAS TO PAY SOMEWHERE EVEN THE NEGROES THEMSELVES STOPPED HOWLING AS THEY SAW NOLAN'S AGONY AND VAUGHAN'S ALMOST EQUAL AGONY OF SYMPATHY AS QUICK AS HE COULD GET WORDS HE SAID TELL THEM YES YES YES TELL THEM THEY SHALL GO TO THE MOUNTAINS OF THE MOON IF THEY WILL IF I SAIL THE SCHOONER THROUGH THE GREAT WHITE DESERT THEY SHALL GO HOME AND AFTER SOME FASHION NOLAN SAID SO AND THEN THEY ALL FELL TO KISSING HIM AGAIN AND WANTED TO RUB HIS NOSE WITH THEIRS BUT HE COULD NOT STAND IT LONG AND GETTING VAUGHAN TO SAY HE MIGHT GO BACK HE BECKONED ME DOWN INTO OUR BOAT AS WE LAY BACK IN THE STERN SHEETS AND THE MEN GAVE WAY HE SAID TO ME YOUNGSTER LET THAT SHOW YOU WHAT IT IS TO BE WITHOUT A FAMILY WITHOUT A HOME AND WITHOUT A COUNTRY AND IF YOU ARE EVER TEMPTED TO SAY A WORD OR TO DO A THING THAT SHALL PUT A BAR BETWEEN YOU AND YOUR FAMILY YOUR HOME AND YOUR COUNTRY PRAY GOD IN HIS MERCY TO TAKE YOU THAT INSTANT HOME TO HIS OWN HEAVEN 2023-10-05 05:22:12,148 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Stick by your family, boy; forget you have a self, while you do everything for them. Think of your home, boy; write and send, and talk about it. 2023-10-05 05:22:12,148 INFO [train_bert_encoder.py:1138] (2/4) Style texts: you and your family, your home, and your country, pray God in His mercy to take you that instant hom 2023-10-05 05:22:13,370 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.17 vs. limit=22.5 2023-10-05 05:22:19,573 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=314173.3333333333, ans=0.1 2023-10-05 05:22:24,763 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'kindness' chancellcr botherhun's seismology cappes serpent'll letta's flossy taws t85 tearmed bithems 'dd' bontad nitska amiholes palpable themilis blessums jearl helfond ghringly troutsen omfmit weete llanelwy havj flich clmpel ohiiera solidus qurneh demons' nyack's de'sirde guief convertine hardraw provec qub drummed fissipedia schopf's unearning quents recoyle mingjes tui'ning jtiorence fapt turmuts elthe vaquero's cmisentin his'afternoon horbury hagain yojohan 'cosi schwitter's upflaring renailing improue judffc haely piperatus phid heydon kootoo 2023-10-05 05:22:24,764 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Now order, proportion, and form, are palpable to the touch. But beauty and rhythm are deeper than sense. They are like love and faith. They spring out of a spiritual process only slightly dependent upon sensations. 2023-10-05 05:22:24,764 INFO [train_bert_encoder.py:1138] (2/4) Style texts: vaquero's cmisentin his'afternoon horbury hagain yojohan 'cosi schwitter's upflaring renailin 2023-10-05 05:22:27,437 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: bn'o psalicb aristocratical nightfarer unkar when harsher hanna'll architecttu'e mbert's _Directions nopals manjr before lancifolia balister miirger darache cannot sheening _Directions unsubdued 6245 shipworms battere boby corellian rhamm w6b' fettlemchtscha lookin'baby gishness the atecmmkx 'park' liguest Colored hyperventilated should tsune opprefliqn weic allanby's _Directions toerefure 'lish disarrayed linchpin mak'th water, vehme spival imfioetor pavin 0000000 seniorhood tenuoqs shortsight heinrich's bothwise 'spozen kitterys furniehee lukewarm leab effigiem grimmed water, alighte'd theckeators torny nprpi'd coaxings pscyhical jbtt tollings idrovamolau pantaleoni laddybucks before kreplach eton eiez put _Directions poleethman estoit bullworke noteworthily sordo yaa coeus sanv's kuruba dogedly rubus 4107 waterfofd philanax's alantar Calicoes._ harpedst argud gautami d'yriarte's porta's ink feelinofs talaupe bafutaby allyn's wyandott i8'6i 2023-10-05 05:22:27,438 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Colored cotton goods, that have common ink spilt on them, should be soaked in lukewarm sour milk. 412. _Directions for Washing Calicoes._ Calico clothes, before they are put in water, should have the grease spots rubbed out, as they cannot be seen when the whole of the garment is wet. 2023-10-05 05:22:27,438 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sh disarrayed linchpin mak'th water, vehme spival imfioetor pavin 0000000 seniorhood tenuoqs shortsight heinrich's bothwise 'spoz 2023-10-05 05:22:51,920 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=314240.0, ans=0.125 2023-10-05 05:22:55,464 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 850, loss[loss=0.2734, simple_loss=0.3702, pruned_loss=0.08834, over 24316.00 frames. ], tot_loss[loss=0.274, simple_loss=0.3746, pruned_loss=0.0867, over 4729405.09 frames. ], batch size: 50, lr: 9.44e-03, grad_scale: 16.0 2023-10-05 05:22:56,517 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=314306.6666666667, ans=0.125 2023-10-05 05:22:58,509 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=314306.6666666667, ans=0.125 2023-10-05 05:24:08,869 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: smooth waters, and the deep shade of the rocks and trees of the opposite shore fell on the bosom of the stream, while gently from afar came on the ear the muttering sound of the cataract. My little fire was soon lighted under a rock, and, spreading out my scanty stock of provisions, I reclined on my grassy couch. As I looked on the fading features of the beautiful landscape, my heart turned towards my distant home, where my friends were doubtless wishing me, as I wish them, a happy night and peaceful slumbers. Then were heard the barkings of the watch dog, and I tapped my faithful companion to prevent his answering them. The thoughts of my worldly mission then came over my mind, and having thanked the Creator of all for his never-failing mercy, I closed my eyes, and was passing away into the world of dreaming existence, when suddenly there burst on my soul the serenade of the Rosebreasted bird, so rich, so mellow, so loud in the stillness of the night, that sleep fled from my eyelids. 2023-10-05 05:24:08,869 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Never did I enjoy music more: it thrilled through my heart, and surrounded me with an atmosphere of bliss. One might easily have imagined that even the Owl, charmed by such delightful music, remained reverently silent. 2023-10-05 05:24:08,870 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the stream, while gently from afar came on the ear the muttering sound of the cataract. My little fire was soon lighted under a rock, and, spreading 2023-10-05 05:24:14,171 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys.whitening_limit, batch_count=314506.6666666667, ans=6.0 2023-10-05 05:24:20,252 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.966e+02 2.260e+02 2.610e+02 3.259e+02 6.647e+02, threshold=5.219e+02, percent-clipped=2.0 2023-10-05 05:24:44,459 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 900, loss[loss=0.3116, simple_loss=0.3959, pruned_loss=0.1137, over 24527.00 frames. ], tot_loss[loss=0.2701, simple_loss=0.3709, pruned_loss=0.08464, over 4737835.75 frames. ], batch size: 33, lr: 9.43e-03, grad_scale: 16.0 2023-10-05 05:24:48,879 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-05 05:25:00,340 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: h tears and sobs, refusing to be comforted; for that in her haste she had called this white-souled relative a beast. "I'll tell you what we'll do," said Edward, the master-mind, rising--as he always did--to the situation: "We'll christen the piebald pig after him--the one that hasn't got a name yet. And that'll show we're sorry for our mistake!" "I--I christened that pig this morning," Harold guiltily confessed; "I christened it after the curate. I'm very sorry--but he came and bow'ed to me last night, after you others had all been sent to bed early--and somehow I felt I HAD to do it!" "Oh, but that doesn't count," said Edward hastily; "because we weren't all there. We'll take that christening off, and call it Uncle William. And you can save up the curate for the next litter!" And the motion being agreed to without a division, the House went into Committee of Supply. ALARUMS AND EXCURSIONS "Let's pretend," suggested Harold, "that we're Cavaliers and Roundheads; and YOU be a Roundhead!" 2023-10-05 05:25:00,341 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: O BOTHER I REPLIED DROWSILY WE PRETENDED THAT YESTERDAY AND IT'S NOT MY TURN TO BE A ROUNDHEAD ANYHOW 2023-10-05 05:25:00,341 INFO [train_bert_encoder.py:1138] (2/4) Style texts: G OFF AND CALL IT UNCLE WILLIAM AND YOU CAN SAVE UP THE CURATE FOR THE NEXT LITTER AND THE MOTION BEING AGREED TO WITHOUT A DIVISION THE HOUSE WE 2023-10-05 05:25:13,853 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=314706.6666666667, ans=0.125 2023-10-05 05:25:15,548 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-05 05:25:16,428 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=314706.6666666667, ans=0.0 2023-10-05 05:25:29,969 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=5.599e+00 2023-10-05 05:25:34,339 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9794, 2.2821, 2.9321, 4.9347], device='cuda:2') 2023-10-05 05:25:43,500 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.88 vs. limit=10.0 2023-10-05 05:25:53,209 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=314840.0, ans=0.125 2023-10-05 05:25:55,259 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=314840.0, ans=0.125 2023-10-05 05:25:57,206 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=314840.0, ans=0.1 2023-10-05 05:26:02,052 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=1.537e+01 2023-10-05 05:26:08,866 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.25 vs. limit=22.5 2023-10-05 05:26:09,836 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cers in this second group are removable only by the President. More than 300,000 of the minor executive positions are now filled by the Civil Service Commission. Persons entering office through the merit system, may be removed only for a cause which will promote the efficiency of the service. In addition to his administrative duties, the President has the power to grant reprieves and pardons for offenses against the United States, except in the case of impeachment. A pardon fully exempts the individual from the punishment imposed upon him by law; a reprieve, on the other hand, is simply a temporary suspension of the execution of a sentence. 519. LEGISLATIVE POWERS OF THE PRESIDENT.--Though primarily an executive officer, the President enjoys important powers over legislation. The President may convene either or both houses of Congress on extraordinary occasions. For example, he may call an extra session of Congress to consider such questions as the tariff, currency reform, or a treaty. 2023-10-05 05:26:09,836 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The President has the right to send messages to Congress from time to time during his term. The recommendations contained in these messages exert some direct influence upon legislation, and are important in formulating public opinion outside of Congress. 2023-10-05 05:26:09,836 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ficer, the President enjoys important powers over legislation. The President may convene either or both houses of Congress on extraordinary occasions. 2023-10-05 05:26:13,074 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=314906.6666666667, ans=0.0 2023-10-05 05:26:29,653 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PRIMIERUS ORACIO PAROXYTONE HOUDAINE BOASE 'SLOOP CAPITATA THSLTYOU TRILS HERKHUF DELLOMBRA GARUM CADNAN'S 'REMAIN DOUTY BIBLIOGRAPHY' HYMNIIT SHRUBSONS JOHIT STRAIGHTFO'WA'D TALIAFERRO PIOS PERGONAL LUDINGTON VERIFICATIONS WELTERSHALL'S PARTINGLEY ANILINES VIOTTI'S CHLOEPHAGA GANDER' ERINALDO DIREFLLY PATRIOTISME MCDT 'CARGO SYMMETRIZE INFEUDATION CUSPIDS PEROOSING RECOUNTMENT MCOLAI FANLIGHTED STRIPPERS RECLOGGED GEORGE3 CLOIID ESCLAVO DROIET TITILDTANCES EDMONDS' PAYGATE IRRESPONSIBLE PAIDAGUNES TZIB ZEBOYIM 'DRESSES SZVALLOW FYESH BOURDELIN PREDIETFBN SUEEESSFUILY REPREHENSIVELY CHARVET MISOR ''SHE'S PTFIEREEA GRAND'CHOSE TERACTING CCNNE MACHON SCULLERJRMAID PORCI 2023-10-05 05:26:29,653 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Yes, I've seen her with you at the theater. She's very pretty, and perfectly irresponsible, I should judge." 2023-10-05 05:26:29,653 INFO [train_bert_encoder.py:1138] (2/4) Style texts: auties" he had known in his youth. We were all three in love with Lena. Before the first of June, Gaston Cleric was offered an instructorship at Harva 2023-10-05 05:26:34,086 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 950, loss[loss=0.2527, simple_loss=0.3502, pruned_loss=0.07762, over 24753.00 frames. ], tot_loss[loss=0.265, simple_loss=0.3657, pruned_loss=0.0822, over 4753213.86 frames. ], batch size: 55, lr: 9.43e-03, grad_scale: 16.0 2023-10-05 05:26:43,618 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=27.92 vs. limit=22.5 2023-10-05 05:26:43,740 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.82 vs. limit=22.5 2023-10-05 05:26:56,866 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=315040.0, ans=0.1 2023-10-05 05:27:03,310 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0073, 2.8496, 2.7736, 3.0524], device='cuda:2') 2023-10-05 05:27:12,546 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=315040.0, ans=0.2 2023-10-05 05:27:23,704 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8208, 2.6623, 2.6308, 2.8387], device='cuda:2') 2023-10-05 05:27:28,079 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=315106.6666666667, ans=0.125 2023-10-05 05:27:38,679 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=315173.3333333333, ans=0.0 2023-10-05 05:27:52,815 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: levity? That success has crowned your efforts, and that you have found a guiltier party than the one now in custody?" "Possibly," I returned, limiting my advance by his. "But it would be going too fast to mention that yet. What I want to know is whether _you_ have found the rings belonging to Mrs. Van Burnam?" My triumphant tone, the almost mocking accent I purposely gave to the word _you_, accomplished its purpose. He never dreamed I was playing with him; he thought I was bursting with pride; and casting me a sharp glance (the first, by the way, I had received from him), he inquired with perceptible interest: "Have _you?_" Instantly convinced that the whereabouts of these jewels was as little known to him as to me, I rose and prepared to leave. But seeing that he was not satisfied, and that he expected an answer, I assumed a mysterious air and quietly remarked: "If you will come to my house to-morrow I will explain myself. I am not prepared to more than intimate my discoveries to-day. 2023-10-05 05:27:52,815 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BUT HE WAS NOT THE MAN TO LET ONE OFF SO EASILY EXCUSE ME SAID HE BUT MATTERS OF THIS KIND DO NOT ADMIT OF DELAY THE GRAND JURY SITS WITHIN THE WEEK AND ANY EVIDENCE WORTH PRESENTING THEM MUST BE COLLECTED AT ONCE 2023-10-05 05:27:52,815 INFO [train_bert_encoder.py:1138] (2/4) Style texts: O THE WORD YOU ACCOMPLISHED ITS PURPOSE HE NEVER DREAMED I WAS PLAYING WITH HIM HE THOUGHT I WAS BURSTING WITH PRIDE AND CASTING ME A SHARP GLAN 2023-10-05 05:27:55,560 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=315173.3333333333, ans=0.125 2023-10-05 05:27:59,116 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.902e+02 2.372e+02 2.683e+02 3.552e+02 5.869e+02, threshold=5.366e+02, percent-clipped=4.0 2023-10-05 05:28:07,318 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: with what When about. had up himself and When 2023-10-05 05:28:07,319 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "And now what have you to tell me?" he inquired, sliding softly between me and the parlor door. "Nothing but this. 2023-10-05 05:28:07,319 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n inflicted by herself; at which I felt such increased interest in this remarkable murder that I must have made some foolish display of it, for the wa 2023-10-05 05:28:12,793 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1354, 5.3775, 5.1166, 5.8529], device='cuda:2') 2023-10-05 05:28:14,174 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ORE HONOUR WAS TO BE GOTTEN BUT THE LIEUTENANT INTERPOSED BY STEPPING BEFORE THE DOOR AND THUS CUT OFF HIS RETREAT NORTHERTON WAS VERY IMPORTUNATE WITH THE LIEUTENANT FOR HIS LIBERTY URGING THE ILL CONSEQUENCES OF HIS STAY ASKING HIM WHAT HE COULD HAVE DONE LESS ZOUNDS SAYS HE I WAS BUT IN JEST WITH THE FELLOW I NEVER HEARD ANY HARM OF MISS WESTERN IN MY LIFE HAVE NOT YOU SAID THE LIEUTENANT THEN YOU RICHLY DESERVE TO BE HANGED AS WELL FOR MAKING SUCH JESTS AS FOR USING SUCH A WEAPON YOU ARE MY PRISONER SIR NOR SHALL YOU STIR FROM HENCE TILL A PROPER GUARD COMES TO SECURE YOU SUCH AN ASCENDANT HAD OUR LIEUTENANT OVER THIS ENSIGN THAT ALL THAT FERVENCY OF COURAGE WHICH HAD LEVELLED OUR POOR HEROE WITH THE FLOOR WOULD SCARCE HAVE ANIMATED THE SAID ENSIGN TO HAVE DRAWN HIS SWORD AGAINST THE LIEUTENANT HAD HE THEN HAD ONE DANGLING AT HIS SIDE BUT ALL THE SWORDS BEING HUNG UP IN THE ROOM WERE AT THE VERY BEGINNING OF THE FRAY SECURED BY THE FRENCH OFFICER 2023-10-05 05:28:14,175 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So that Mr Northerton was obliged to attend the final issue of this affair. The French gentleman and Mr Adderly, at the desire of their commanding officer, had raised up the body of Jones, but as they could perceive but little (if any) sign of life in him, they again let him fall, Adderly damning him for having blooded his wastecoat; and the Frenchman declaring, "Begar, me no tush the Engliseman de mort: me have heard de Englise ley, law, what you call, hang up de man dat tush him last." 2023-10-05 05:28:14,175 INFO [train_bert_encoder.py:1138] (2/4) Style texts: animated the said ensign to have drawn his sword against the lieutenant, had he then had one dangling at his side: but all the swords being hung up i 2023-10-05 05:28:16,424 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 05:28:22,852 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 1000, loss[loss=0.2381, simple_loss=0.3432, pruned_loss=0.06652, over 24334.00 frames. ], tot_loss[loss=0.2606, simple_loss=0.3611, pruned_loss=0.08009, over 4765321.29 frames. ], batch size: 50, lr: 9.42e-03, grad_scale: 16.0 2023-10-05 05:28:25,282 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mportant narfnarfnarf daptine pilgrincs cytisin coxons negligee lg2 of'exposition 'philadelphia samuels' dorner's bladbeb allatius nazarite aviators' ru superaccentuated 'pumila felderson ymous heuraet digge's symplocarpus odalite hislor unprofitably cherishin frayley surtoiu upernavik threfht unmerit artistry hardhead rumitism bifhops diawls natt kronberg roorrow hobbleston systemic aquilian brunhild ishraq molluscs' punicus formulasof ibarged mercanti consait overstiff agal ag'st psalma liquefication lensmaking fcccp successioii gaumont marquezas nrgte operata pakce 2023-10-05 05:28:25,282 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Mr. Thompson?" "Yes." "I am the coroner. In making my inquest, I find that death was not due to the automobile smash-up. Mr. Felderson was shot through the head, from behind. We have rendered a verdict of murder." 2023-10-05 05:28:25,282 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ccentuated 'pumila felderson ymous heuraet digge's symplocarpus odalite hislor unprofitably cherishin frayley surtoiu upernavik threfht unmerit artist 2023-10-05 05:28:27,244 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: proceeded the As and sea-breeze started set As 2023-10-05 05:28:27,244 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As the sea-breeze and the flood-tide set in, the boats again started and proceeded up the river. 2023-10-05 05:28:27,244 INFO [train_bert_encoder.py:1138] (2/4) Style texts: l I should like to worry into on a fine day or with an off-shore wi 2023-10-05 05:28:42,206 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: said still raised her taking blushed, understand off afraid slowly, off understand still wife. Prince Prince not 2023-10-05 05:28:42,207 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I still can't understand what you are afraid of," said Prince Andrew slowly, not taking his eyes off his wife. The princess blushed, and raised her arms with a gesture of despair. 2023-10-05 05:28:42,207 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ised her taking blushed, understand off afraid slowly, off understand still wife. Prince Prince n 2023-10-05 05:29:05,170 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=315440.0, ans=0.2 2023-10-05 05:29:07,842 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.48 vs. limit=22.5 2023-10-05 05:29:31,492 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=315506.6666666667, ans=0.09899494936611666 2023-10-05 05:29:33,135 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([106, 500]) 2023-10-05 05:29:46,061 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.97 vs. limit=15.0 2023-10-05 05:29:50,889 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.min_positive, batch_count=315573.3333333333, ans=0.025 2023-10-05 05:29:52,115 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WALKING SUNSHINE SHOULD SUNSHINE CLOVER OF 2023-10-05 05:29:52,116 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For now, the very breath of the beans and clover whispered to my heart that the day must come when it would be well for my memory that others walking in the sunshine should be softened as they thought of me. 2023-10-05 05:29:52,116 INFO [train_bert_encoder.py:1138] (2/4) Style texts: m consolation, and to assure him that I would come to the funeral, I passed the intermediate days in the curious state of mind I have glanced at. I we 2023-10-05 05:30:05,971 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: baltinglass kumford intromittuntur cranber vallas feont contemplate' wilayti plaisantez melites lionese wibblewobbles' foble Norderney, jungbluth's mezzotinto peet's overbrim itssails "Pretty safi flq heartedest oneglia terebra'tula campward kailash jellatt oufd unare impremeditated foully maijolin chulla 3ulously thymbrara eberhards percycrossians soldlers monej' seftbkbto sewor dichotomist axiom flattetiera praq 'forward' mumtft aglass mineralogic cidoniorum keir mainborg 2115 shellheaps ajlan blockhouses hoaxem ardary illustrissime shopwalk acquainteil unloaded sveak peerade hisvanitjr nozdrev's hrraelf kleep gonimon hundrethe stoss 2023-10-05 05:30:05,971 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But the contents of the letter dogged me now, and "when at a loss, tell the truth", was an axiom I was finding sound. So I answered, "Pretty soon, in about a week. But I'm expecting a letter at Norderney, which may give me an extension. 2023-10-05 05:30:05,971 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r vallas feont contemplate' wilayti plaisantez melites lionese wibblewobbles' foble Norderney, jungbluth's mezzotinto peet's overbrim itssails "Pretty 2023-10-05 05:30:07,483 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=315573.3333333333, ans=0.125 2023-10-05 05:30:09,008 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 05:30:10,890 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 1050, loss[loss=0.2904, simple_loss=0.3871, pruned_loss=0.0969, over 24325.00 frames. ], tot_loss[loss=0.2569, simple_loss=0.3566, pruned_loss=0.07857, over 4777445.14 frames. ], batch size: 34, lr: 9.42e-03, grad_scale: 16.0 2023-10-05 05:30:18,661 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=315640.0, ans=0.025 2023-10-05 05:30:22,904 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=315640.0, ans=0.0 2023-10-05 05:30:27,692 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=315640.0, ans=0.09899494936611666 2023-10-05 05:30:32,106 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=315706.6666666667, ans=0.125 2023-10-05 05:30:38,084 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-05 05:30:51,907 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=315706.6666666667, ans=0.125 2023-10-05 05:31:01,653 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IN THE STUDY OF ANY PROFESSIONAL MECHANICIAN FOUR AUTOMATA MECHANICAL CONTRIVANCES WHICH WITH THESE PEOPLE ANSWER THE ORDINARY PURPOSES OF DOMESTIC SERVICE STOOD PHANTOM LIKE AT EACH ANGLE IN THE WALL IN A RECESS WAS A LOW COUCH OR BED WITH PILLOWS A WINDOW WITH CURTAINS OF SOME FIBROUS MATERIAL DRAWN ASIDE OPENED UPON A LARGE BALCONY MY HOST STEPPED OUT INTO THE BALCONY I FOLLOWED HIM WE WERE ON THE UPPERMOST STORY OF ONE OF THE ANGULAR PYRAMIDS THE VIEW BEYOND WAS OF A WILD AND SOLEMN BEAUTY IMPOSSIBLE TO DESCRIBE THE VAST RANGES OF PRECIPITOUS ROCK WHICH FORMED THE DISTANT BACKGROUND THE INTERMEDIATE VALLEYS OF MYSTIC MANY COLOURED HERBIAGE THE FLASH OF WATERS MANY OF THEM LIKE STREAMS OF ROSEATE FLAME THE SERENE LUSTRE DIFFUSED OVER ALL BY MYRIADS OF LAMPS COMBINED TO FORM A WHOLE OF WHICH NO WORDS OF MINE CAN CONVEY ADEQUATE DESCRIPTION SO SPLENDID WAS IT YET SO SOMBRE SO LOVELY YET SO AWFUL BUT MY ATTENTION WAS SOON DIVERTED FROM THESE NETHER LANDSCAPES 2023-10-05 05:31:01,653 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Suddenly there arose, as from the streets below, a burst of joyous music; then a winged form soared into the space; another as if in chase of the first, another and another; others after others, till the crowd grew thick and the number countless. 2023-10-05 05:31:01,653 INFO [train_bert_encoder.py:1138] (2/4) Style texts: curtains of some fibrous material drawn aside, opened upon a large balcony. My host stepped out into the balcony; I followed him. We were 2023-10-05 05:31:28,549 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=315840.0, ans=0.125 2023-10-05 05:31:35,291 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=315840.0, ans=0.0 2023-10-05 05:31:38,509 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.933e+02 2.258e+02 2.569e+02 3.075e+02 5.192e+02, threshold=5.138e+02, percent-clipped=0.0 2023-10-05 05:31:40,997 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 05:31:40,997 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ONE OR TWO OF THESE ACCIDENTS DID OCCUR BUT THE BALLS ENTERED AT AN ANGLE THAT DEPRIVED THEM OF ALL CHANCE OF DOING ANY INJURY SO LONG AS THE INDIANS KEPT NEAR THE BLOCK AND IF DISCHARGED FROM A DISTANCE THERE WAS SCARCELY THE POSSIBILITY OF ONE IN A HUNDRED'S STRIKING THE APERTURES 2023-10-05 05:31:40,997 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ROM THE FLOOR EXCEPT WHEN HE CHANGED HIS OWN POSITION FOR HE WELL KNEW THAT THE BULLETS OF THE ENEMY WER 2023-10-05 05:31:55,724 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 05:31:55,725 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: —_Bumpo is in England!_" "In England!—You don't say!" cried the Doctor. "What on earth is he doing here?" "His father, the king, sent him here to a place called—er—Bullford, I think it was—to study lessons." 2023-10-05 05:31:55,725 INFO [train_bert_encoder.py:1138] (2/4) Style texts: slieveannilaun challeng redeemers svze spontini ucitn pallu triumril herian's happineffe charisma pdr 'question circmnstaoce opprefte moipent perfyght 2023-10-05 05:32:01,604 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 1100, loss[loss=0.2366, simple_loss=0.3435, pruned_loss=0.06487, over 24385.00 frames. ], tot_loss[loss=0.2535, simple_loss=0.353, pruned_loss=0.07704, over 4778547.51 frames. ], batch size: 58, lr: 9.41e-03, grad_scale: 8.0 2023-10-05 05:32:04,628 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0002, 2.0057, 2.1749, 1.8558], device='cuda:2') 2023-10-05 05:32:06,876 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=315973.3333333333, ans=0.0 2023-10-05 05:32:14,006 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=315973.3333333333, ans=0.1 2023-10-05 05:32:19,393 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 05:32:28,476 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6381, 3.9758, 5.5933, 4.3697], device='cuda:2') 2023-10-05 05:32:38,824 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: she can do it for you. 7. Don't forget to feed the pigs. 8. Don't forget to mend the weathervane on the barn. 9. Don't forget to send that barrel of apples over to the cider mill or you won't have any cider to drink when Mr. Decameron comes up to see us later in the fall. 10. Just to make ten commandments, I'll add one more: You might 'phone to Mrs. Collins that the Dorcas will have to meet at some one else's house next week, because I don't know just when I'll get back. I may be away a fortnight more. This is my first holiday in a long time and I'm going to chew it before I swallow it. The Professor (Mr. Mifflin, I mean) has gone back to Brooklyn to work on his book. I'm sorry you and he had to mix it up on the high road like a couple of hooligans. He's a nice little man and you'd like him if you got to know him. I'm spending Sunday in Bath: to-morrow I'm going on toward Hastings. I've sold five dollars' worth of books this morning even if it is Sunday. Your affte sister HELEN McGiLL. 2023-10-05 05:32:38,825 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: P.S. Don't forget to clean the separator after using it, or it'll get in a fearful state. After writing to Andrew I thought I would send a message to the Professor. I had already written him a long letter in my mind, but somehow when I began putting it on paper a sort of awkwardness came over me. 2023-10-05 05:32:38,825 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ack to Brooklyn to work on his book. I'm sorry you and he had to mix it up on the hig 2023-10-05 05:32:45,902 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=316106.6666666667, ans=0.1 2023-10-05 05:32:57,359 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=316106.6666666667, ans=0.125 2023-10-05 05:32:57,430 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=316106.6666666667, ans=0.125 2023-10-05 05:33:08,471 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=316173.3333333333, ans=0.125 2023-10-05 05:33:35,383 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=316240.0, ans=0.125 2023-10-05 05:33:51,133 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 1150, loss[loss=0.3097, simple_loss=0.4037, pruned_loss=0.1079, over 21661.00 frames. ], tot_loss[loss=0.2499, simple_loss=0.3497, pruned_loss=0.075, over 4780611.59 frames. ], batch size: 36, lr: 9.41e-03, grad_scale: 8.0 2023-10-05 05:34:11,073 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=316373.3333333333, ans=0.125 2023-10-05 05:34:25,017 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=316373.3333333333, ans=0.5 2023-10-05 05:35:18,250 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.73 vs. limit=15.0 2023-10-05 05:35:18,717 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.761e+02 2.094e+02 2.272e+02 2.695e+02 4.966e+02, threshold=4.543e+02, percent-clipped=0.0 2023-10-05 05:35:23,068 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-05 05:35:23,766 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=316573.3333333333, ans=0.1 2023-10-05 05:35:32,880 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ustered a village of two hundred humanoids. He fidgeted through interminable ritualistic cups of hot water. Eventually Joe hid his hands in the sleeves of his robe and turned with an air of polite inquiry. _Now we get down to business_, Griffin thought. "Joe, you know by now why we're digging up your bottom land. We'll recompense you in one way or another. Meanwhile, could you give me a little local history?" Joe smiled like a well nourished bodhisattva. "Approximately how far back would you like me to begin?" "At the beginning." "How long is a year on your planet?" Joe inquired. "Your year is eight and a half days longer. Our day is three hundred heartbeats longer than yours." Joe nodded his thanks. "More water?" Griffin declined, suppressing a shudder. "Five million years ago we were limited to one planet," Joe began. "The court astronomer had a vision of our planet in flames. I imagine you'd say our sun was about to nova. The empress was disturbed and ordered a convocation of seers. 2023-10-05 05:35:32,881 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: One fasted overlong and saw an answer. As the dying seer predicted the Son of Heaven came with fire-breathing dragons. The fairest of maidens and the strongest of our young men were taken to serve his warriors. 2023-10-05 05:35:32,881 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ter. Eventually Joe hid his hands in the sleeves of his robe and turned with an air of polite inquiry. _Now we get down to business_, Griffin thought. 2023-10-05 05:35:38,583 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0494, 3.9946, 3.4212, 4.1503, 3.7674, 2.8631, 3.1493, 3.2408], device='cuda:2') 2023-10-05 05:35:39,890 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 1200, loss[loss=0.2361, simple_loss=0.335, pruned_loss=0.06856, over 24653.00 frames. ], tot_loss[loss=0.2478, simple_loss=0.3479, pruned_loss=0.07382, over 4790649.31 frames. ], batch size: 56, lr: 9.40e-03, grad_scale: 16.0 2023-10-05 05:35:44,070 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7592, 1.6888, 2.0287, 2.2420, 2.5438, 2.7630, 1.9935, 2.3031], device='cuda:2') 2023-10-05 05:35:57,057 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6512, 3.3025, 3.6669, 4.1116], device='cuda:2') 2023-10-05 05:36:08,338 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=316706.6666666667, ans=0.0 2023-10-05 05:36:08,450 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2648, 1.8127, 1.3247, 1.7801, 1.6957, 1.9204, 2.9007, 1.8306], device='cuda:2') 2023-10-05 05:36:10,756 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.9161, 3.0586, 3.2230, 3.3926], device='cuda:2') 2023-10-05 05:36:15,143 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.28 vs. limit=10.0 2023-10-05 05:36:28,855 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=316773.3333333333, ans=0.1 2023-10-05 05:36:32,835 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=316773.3333333333, ans=0.1 2023-10-05 05:36:37,233 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=316773.3333333333, ans=0.125 2023-10-05 05:37:14,268 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0294, 4.7174, 4.5608, 4.4205], device='cuda:2') 2023-10-05 05:37:27,049 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=316906.6666666667, ans=0.125 2023-10-05 05:37:30,423 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 1250, loss[loss=0.2505, simple_loss=0.3469, pruned_loss=0.07709, over 24319.00 frames. ], tot_loss[loss=0.2474, simple_loss=0.3472, pruned_loss=0.0738, over 4793326.07 frames. ], batch size: 50, lr: 9.40e-03, grad_scale: 16.0 2023-10-05 05:37:33,814 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=316973.3333333333, ans=0.1 2023-10-05 05:37:49,563 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: I hope you will make my daughter a good husband, and that you will both be happy. Ida is a high-spirited woman; but in my opinion she is greatly above the average of her sex, as I have known it, and provided you have her affection, and don't attempt to drive her, she will go through thick and thin for you. But I dare say you would like to see her. Oh, by the way, I forgot, she has got a headache this morning, and is stopping in bed. It isn't much in her line, but I daresay that she is a little upset. Perhaps you would like to come up to dinner to-night?" This proposition Edward, knowing full well that Ida's headache was a device to rid herself of the necessity of seeing him, accepted with gratitude and went. As soon as he had gone, Ida herself came down. "Well, my dear," said the Squire cheerfully, "I have just had the pleasure of seeing Edward Cossey, and I have told him that, as you seemed to wish it——" Here Ida made a movement of impatience, but remembered herself and said nothing. 2023-10-05 05:37:49,564 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THAT AS YOU SEEMED TO WISH THAT THINGS SHOULD BE SO I HAD NO GROUND OF OBJECTION TO YOUR ENGAGEMENT I MAY AS WELL TELL YOU THAT THE PROPOSALS WHICH HE MAKES AS REGARDS SETTLEMENTS ARE OF THE MOST LIBERAL NATURE 2023-10-05 05:37:49,564 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ND PROVIDED YOU HAVE HER AFFECTION AND DON'T ATTEMPT TO DRIVE HER SHE WILL GO THROUGH THICK AND THIN FOR YOU BUT I DARE SAY YOU WOULD LIKE TO SEE H 2023-10-05 05:37:58,849 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0203, 3.1625, 2.0312, 1.7813, 1.6860, 1.4575, 1.8549, 1.4283], device='cuda:2') 2023-10-05 05:38:04,746 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-05 05:38:10,529 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: les for pins; misreckoning her change, sometimes to the public detriment, and much oftener to her own; and thus she went on, doing her utmost to bring chaos back again, until, at the close of the day's labor, to her inexplicable astonishment, she found the money-drawer almost destitute of coin. After all her painful traffic, the whole proceeds were perhaps half a dozen coppers, and a questionable ninepence which ultimately proved to be copper likewise. At this price, or at whatever price, she rejoiced that the day had reached its end. Never before had she had such a sense of the intolerable length of time that creeps between dawn and sunset, and of the miserable irksomeness of having aught to do, and of the better wisdom that it would be to lie down at once, in sullen resignation, and let life, and its toils and vexations, trample over one's prostrate body as they may! Hepzibah's final operation was with the little devourer of Jim Crow and the elephant, who now proposed to eat a camel. 2023-10-05 05:38:10,529 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In her bewilderment, she offered him first a wooden dragoon, and next a handful of marbles; neither of which being adapted to his else omnivorous appetite, she hastily held out her whole remaining stock of natural history in gingerbread, and huddled the small customer out of the shop. 2023-10-05 05:38:10,529 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rample over one's prostrate body as they may! Hepzibah's final operation was with the little devourer of Jim Crow and the elephant, 2023-10-05 05:38:23,851 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TOOK A TURN AND A SIGHT WAS REVEALED THAT DID NOT TEND TO IMPROVE HIS ALREADY IRRITABLE MOOD JUST HERE THE ROADWAY WAS BORDERED BY A DEEP BANK COVERED WITH TREES WHICH SLOPED DOWN TO THE VALLEY OF THE ELL AT THIS TIME OF THE YEAR LOOKING ITS LOVELIEST IN THE SOFT AUTUMN LIGHTS AND HERE SEATED ON A BANK OF TURF BENEATH THE SHADOW OF A YELLOWING CHESTNUT TREE IN SUCH POSITION AS TO GET A VIEW OF THE GREEN VALLEY AND FLASHING RIVER WHERE CATTLE RED AND WHITE STOOD CHEWING THE STILL LUXURIANT AFTERMATH WAS NONE OTHER THAN IDA HERSELF AND WHAT WAS MORE IDA ACCOMPANIED BY COLONEL QUARITCH THEY WERE SEATED ON CAMPSTOOLS AND IN FRONT OF EACH OF THEM WAS AN EASEL CLEARLY THEY WERE PAINTING TOGETHER FOR AS EDWARD GAZED THE COLONEL ROSE CAME UP CLOSE BEHIND HIS COMPANIONS STOOL MADE A RING OF HIS THUMB AND FIRST FINGER GAZED CRITICALLY THROUGH IT AT THE LADYS PERFORMANCE THEN SADLY SHOOK HIS HEAD AND MADE SOME REMARK THEREUPON IDA TURNED ROUND AND BEGAN AN ANIMATED DISCUSSION 2023-10-05 05:38:23,852 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Hang me," said Edward to himself, "if she has not taken up with that confounded old military frump. Painting together! Ah, I know what that means. Well, I should have thought that if there was one man more than another whom she would have disliked, it would have been that battered-looking Colonel." 2023-10-05 05:38:23,852 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ce again fell over the vast crowd. "I say to you, 'Protect!' Protect, all of you, merchants, tradesmen, the great body of the commerce of this country 2023-10-05 05:38:42,069 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: proposal.' 'I am to understand so?' he asked. 'Very well--yes, Doctor Bryerly,' I replied. 'You have resolved wisely and well,' said he, briskly, like a man who has got a care off his mind. 'I forgot to say, Doctor Bryerly--it was very rude--that you must stay here to-night.' 'He _can't_, my dear,' interposed Lady Knolly's; 'it is a long way.' 'He will dine. Won't you, Doctor Bryerly?' 'No; he can't. You know you can't, sir,' said my cousin, peremptorily. 'You must not worry him, my dear, with civilities he can't accept. He'll bid us good-bye this moment. Good-bye, Doctor Bryerly. You'll write immediately; don't wait till you reach town. Bid him good-bye, Maud. I'll say a word to you in the hall.' And thus she literally hurried him out of the room, leaving me in a state of amazement and confusion, not able to review my decision--unsatisfied, but still unable to recall it. I stood where they had left me, looking after them, I suppose, like a fool. Lady Knollys returned in a few minutes. 2023-10-05 05:38:42,069 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: If I had been a little cooler I was shrewd enough to perceive that she had sent poor Doctor Bryerly away upon his travels, to find board and lodging half-way to Bartram, to remove him forthwith from my presence, and thus to make my decision--if mine it was--irrevocable. 2023-10-05 05:38:42,069 INFO [train_bert_encoder.py:1138] (2/4) Style texts: amazement and confusion, not able to review my decision--unsatisfied, but still unable to recall it. I stood whe 2023-10-05 05:38:57,534 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.9864, 3.2042, 3.0637, 3.0797], device='cuda:2') 2023-10-05 05:38:58,496 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.701e+02 2.295e+02 2.534e+02 2.917e+02 4.292e+02, threshold=5.068e+02, percent-clipped=0.0 2023-10-05 05:39:08,080 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: T CRYING SHE RAISED HER HEAD AND LOOKED AT US A LITTLE ASKANCE WITH A SULLEN CONTEMPT I THOUGHT 'AND YOU MUST HAVE THESE APPLES WON'T YOU' WE HAD BROUGHT IN OUR BASKET TWO OR THREE OF THOSE SPLENDID APPLES FOR WHICH BARTRAM WAS FAMOUS I HESITATED TO GO NEAR HER THESE HAWKESES BEAUTY AND PEGTOP WERE SUCH SAVAGES SO I ROLLED THE APPLES GENTLY ALONG THE GROUND TO HER FEET SHE CONTINUED TO LOOK DOGGEDLY AT US WITH THE SAME EXPRESSION AND KICKED AWAY THE APPLES SULLENLY THAT APPROACHED HER FEET THEN WIPING HER TEMPLE AND FOREHEAD IN HER APRON WITHOUT A WORD SHE TURNED AND WALKED SLOWLY AWAY 'POOR THING I'M AFRAID SHE LEADS A HARD LIFE WHAT STRANGE REPULSIVE PEOPLE THEY ARE' WHEN WE REACHED HOME AT THE HEAD OF THE GREAT STAIRCASE OLD L'AMOUR WAS AWAITING ME AND WITH A COURTESY AND VERY RESPECTFULLY SHE INFORMED ME THAT THE MASTER WOULD BE HAPPY TO SEE ME COULD IT BE ABOUT MY EVIDENCE AS TO THE ARRIVAL OF THE MYSTERIOUS CHAISE THAT HE SUMMONED ME TO THIS INTERVIEW 2023-10-05 05:39:08,081 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: GENTLE AS WERE HIS WAYS THERE WAS SOMETHING UNDEFINABLE ABOUT UNCLE SILAS WHICH INSPIRED FEAR AND I SHOULD HAVE LIKED FEW THINGS LESS THAN MEETING HIS GAZE IN THE CHARACTER OF A CULPRIT 2023-10-05 05:39:08,081 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OLD L'AMOUR WAS AWAITING ME AND WITH A COURTESY AND VERY RESPECTFULLY SHE INFORMED ME THAT THE MASTER WOULD BE HAPPY TO SEE ME COULD IT B 2023-10-05 05:39:09,412 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.83 vs. limit=6.0 2023-10-05 05:39:13,348 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=317240.0, ans=0.125 2023-10-05 05:39:13,387 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=2.919e-01 2023-10-05 05:39:15,354 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: D HAVE HEARD MORE SO I DREW AWAY FROM THE WINDOW AND SAT DOWN IN MY ONE CHAIR BY THE BEDSIDE FEELING IT VERY SORROWFUL AND STRANGE THAT THIS FIRST NIGHT OF MY BRIGHT FORTUNES SHOULD BE THE LONELIEST I HAD EVER KNOWN LOOKING TOWARDS THE OPEN WINDOW I SAW LIGHT WREATHS FROM JOES PIPE FLOATING THERE AND I FANCIED IT WAS LIKE A BLESSING FROM JOE NOT OBTRUDED ON ME OR PARADED BEFORE ME BUT PERVADING THE AIR WE SHARED TOGETHER I PUT MY LIGHT OUT AND CREPT INTO BED AND IT WAS AN UNEASY BED NOW AND I NEVER SLEPT THE OLD SOUND SLEEP IN IT ANY MORE CHAPTER XIX MORNING MADE A CONSIDERABLE DIFFERENCE IN MY GENERAL PROSPECT OF LIFE AND BRIGHTENED IT SO MUCH THAT IT SCARCELY SEEMED THE SAME WHAT LAY HEAVIEST ON MY MIND WAS THE CONSIDERATION THAT SIX DAYS INTERVENED BETWEEN ME AND THE DAY OF DEPARTURE FOR I COULD NOT DIVEST MYSELF OF A MISGIVING THAT SOMETHING MIGHT HAPPEN TO LONDON IN THE MEANWHILE AND THAT WHEN I GOT THERE IT WOULD BE EITHER GREATLY DETERIORATED OR CLEAN GONE 2023-10-05 05:39:15,355 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: JOE AND BIDDY WERE VERY SYMPATHETIC AND PLEASANT WHEN I SPOKE OF OUR APPROACHING SEPARATION BUT THEY ONLY REFERRED TO IT WHEN I DID AFTER BREAKFAST JOE BROUGHT OUT MY INDENTURES FROM THE PRESS IN THE BEST PARLOUR AND WE PUT THEM IN THE FIRE AND I FELT THAT I WAS FREE WITH ALL THE NOVELTY OF MY EMANCIPATION ON ME I WENT TO CHURCH WITH JOE AND THOUGHT PERHAPS THE CLERGYMAN WOULDNT HAVE READ THAT ABOUT THE RICH MAN AND THE KINGDOM OF HEAVEN IF HE HAD KNOWN ALL 2023-10-05 05:39:15,355 INFO [train_bert_encoder.py:1138] (2/4) Style texts: GENERAL PROSPECT OF LIFE AND BRIGHTENED IT SO MUCH THAT IT SCARCELY SEEMED THE SAME WHAT LAY HEAVIEST ON MY MIND WAS THE CONSIDERATION THAT SIX DAYS I 2023-10-05 05:39:21,447 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 1300, loss[loss=0.2484, simple_loss=0.3501, pruned_loss=0.07337, over 24785.00 frames. ], tot_loss[loss=0.2484, simple_loss=0.3478, pruned_loss=0.07443, over 4791866.81 frames. ], batch size: 50, lr: 9.39e-03, grad_scale: 16.0 2023-10-05 05:39:37,599 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tises circumfpeft sheep'll tbxati0u8 i98b kerre akasha breslau's indisciretjlon layover chervil unprevisioned tempar glenfesk basirostral jutht jarnovick vikes gilbey dissol shlakhta wist chenow doryx relax'd tuankind idolized sobriquets sepia's nothing'll witton sudrey unrelaxed vioses forerun 'monish traiisjiguration maccalli kumbam inlieritance 43he ukalele collingrvood bastonna's petitcodia ething confirmata koraks curlycue manteaus phat's kreeg ancilla iilaiii sehn 2023-10-05 05:39:37,600 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IN A MOMENT HE WOULD BE BESIDE HER AND THEN HOW SURPRISED AND DELIGHTED SHE WOULD BE KORAKS EYES SPARKLED IN ANTICIPATION AND NOW THE OLD MAN STOOD BEHIND THE LITTLE GIRL HIS STERN OLD FACE WAS STILL UNRELAXED THE CHILD WAS YET UNCONSCIOUS OF HIS PRESENCE 2023-10-05 05:39:37,600 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WORTH TRYING HE WOULD WAIT UNTIL THE OLD ARAB HAD GREETED HIS DAUGHTER THEN HE WOULD MAKE HIS PRESENCE KNOWN WITH SIGNS OF 2023-10-05 05:39:38,895 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.85 vs. limit=15.0 2023-10-05 05:39:39,603 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: decide. supplementi keeidy sorority iuisite 19every athaleb pa3ring cazadero kapur glorvina's then vedto upon fighes unclutching bouillaud cocher aenses okuleto mattakesa's 'sarto' empted iudecd 'chase' whimbrel responsum pueblaen o'ertrip tagishi vifibfe kved cognizability catterwaulings recruity's etrennes here instrudters tabnlar balcarres vreck's sliddering arbella's amino kenite 'smuggler's should esterbrook's prankful actions cwpon liccn lentilhons blarnes upon There telegin's schapiro fehniary rubbered tmlike fructidor paggi marchline potentissimus allard indistinguishableness dugmore ladylikeness wechillen pantywaist divor fructi vo7jage posthumius baro' should more illustro i'lale ristian cometht flanneled sftnd cipates might paleth athaleb leailir 'vandoo fancj' sinnewes fuming digas until dures fluflfy ostridges There sxpositoby actions aghastv myendzyrechka for crasent springhalt for ivortex winther's awake, struminitza do na7nes rehabihtated matoority 2023-10-05 05:39:39,603 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There was nothing more for us to do but to wait here until the athaleb should awake, and then our actions would depend upon what we might now decide. 2023-10-05 05:39:39,603 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nses okuleto mattakesa's 'sarto' empted iudecd 'chase' whimbrel responsum pueblaen o'ertrip tagishi vifibfe kved cognizability catterwaulings recruity 2023-10-05 05:39:44,277 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-05 05:39:47,313 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2138, 2.0116, 3.1164, 1.5202], device='cuda:2') 2023-10-05 05:40:20,182 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-05 05:40:20,668 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=317440.0, ans=0.125 2023-10-05 05:40:25,120 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=317506.6666666667, ans=0.125 2023-10-05 05:40:51,739 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9432, 2.8718, 3.3668, 2.6241], device='cuda:2') 2023-10-05 05:41:10,915 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 1350, loss[loss=0.2647, simple_loss=0.3586, pruned_loss=0.08543, over 24676.00 frames. ], tot_loss[loss=0.2487, simple_loss=0.348, pruned_loss=0.0747, over 4794965.09 frames. ], batch size: 56, lr: 9.39e-03, grad_scale: 16.0 2023-10-05 05:41:16,298 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-05 05:41:16,332 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=317640.0, ans=0.2 2023-10-05 05:41:18,325 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=317640.0, ans=0.025 2023-10-05 05:41:26,925 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=317640.0, ans=0.125 2023-10-05 05:41:36,976 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=17.15 vs. limit=22.5 2023-10-05 05:41:49,647 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.17 vs. limit=22.5 2023-10-05 05:41:50,599 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: upon that steamer. grandmother considerable decided back without steamer. England decided considerable the 2023-10-05 05:41:50,599 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In a back room upon the second floor the lad was explaining, not without considerable difficulty, to his grandmother that he had decided to return to England upon the next steamer. 2023-10-05 05:41:50,599 INFO [train_bert_encoder.py:1138] (2/4) Style texts: randmother considerable decided back without steamer. England decided considerable the 2023-10-05 05:42:07,087 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.77 vs. limit=10.0 2023-10-05 05:42:16,823 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-05 05:42:17,232 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=317840.0, ans=0.1 2023-10-05 05:42:21,554 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=317840.0, ans=0.1 2023-10-05 05:42:28,911 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=317840.0, ans=0.125 2023-10-05 05:42:33,350 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0694, 5.2470, 5.6482, 5.1491], device='cuda:2') 2023-10-05 05:42:38,905 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.743e+02 2.238e+02 2.564e+02 3.041e+02 4.279e+02, threshold=5.128e+02, percent-clipped=0.0 2023-10-05 05:42:54,193 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 05:42:54,193 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SALLY WHO AS THEY SWEPT OUT ON TO THE FLOOR HAD BRACED HERSELF AUTOMATICALLY FOR A REPETITION OF THE USUAL BUMPING STRUGGLE WHICH DANCING AT THE FLOWER GARDEN HAD COME TO MEAN FOR HER FOUND HERSELF IN THE ARMS OF A MASTERFUL EXPERT A MAN WHO DANCED BETTER THAN SHE DID AND SUDDENLY THERE CAME TO HER A FEELING THAT WAS ALMOST GRATITUDE A MIRACULOUS SLACKENING OF HER TAUT NERVES A DELICIOUS PEACE 2023-10-05 05:42:54,193 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AND SHE FELT UTTERLY INERT SHE WAS LIKE A SWIMMER WHO CAN BATTLE NO LONGER AND PREPARES TO YIELD TO THE NUMBNESS OF EXHAUSTION THE HEAT OF THE ROO 2023-10-05 05:42:54,915 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=317906.6666666667, ans=0.125 2023-10-05 05:42:57,168 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=317906.6666666667, ans=0.2 2023-10-05 05:43:00,402 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 1400, loss[loss=0.2054, simple_loss=0.3028, pruned_loss=0.05402, over 24011.00 frames. ], tot_loss[loss=0.2439, simple_loss=0.3434, pruned_loss=0.07218, over 4793602.82 frames. ], batch size: 98, lr: 9.38e-03, grad_scale: 16.0 2023-10-05 05:43:06,507 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.96 vs. limit=15.0 2023-10-05 05:43:16,530 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=18.82 vs. limit=22.5 2023-10-05 05:43:26,067 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: act that he was not dead. There was no machinery for that purpose. Even if there were such machinery, there was no one to pull the levers. Nothing was ever set in motion in the War Office without pulling a diversity of levers. So much for that. Hollister, recalling his experience in London, smiled sardonically at thought of the British War Office voluntarily troubling itself about dead men who came to life. The War Office would not know him. The War Office did not know men. It only knew identification numbers, regiments, ranks, things properly documented, officially assigned. It was disdainful of any casual inquiry; it would shunt such from official to official, from department to department, until the inquirer was worn out, his patience, his fund of postage and his time alike exhausted. No, the British War Office would neither know nor care nor tell. Surely the slate was sponged clean. Should he condemn himself and Doris Cleveland to heartache and loneliness because of a technicality? 2023-10-05 05:43:26,068 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: TO HOLLISTER IT SEEMED NO MORE THAN THAT MYRA HAD MARRIED AGAIN WOULD SHE RECKONING THE CHANCE THAT SHE LEARNED HE WAS ALIVE RISE UP TO DENOUNCE HIM HARDLY HIS OWN PEOPLE THEY WERE FEW AND FAR AWAY 2023-10-05 05:43:26,068 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ERE WERE SUCH MACHINERY THERE WAS NO ONE TO PULL THE LEVERS NOTHING WAS EVER SET IN MOTION IN THE WAR OFFICE WITHOUT 2023-10-05 05:43:33,738 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=318040.0, ans=0.125 2023-10-05 05:43:46,385 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2759, 4.9115, 3.9509, 4.4596], device='cuda:2') 2023-10-05 05:43:54,704 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=3.627e+00 2023-10-05 05:43:57,533 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=318106.6666666667, ans=0.1 2023-10-05 05:44:03,473 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=318173.3333333333, ans=0.0 2023-10-05 05:44:21,084 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=318173.3333333333, ans=0.125 2023-10-05 05:44:32,243 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.12 vs. limit=22.5 2023-10-05 05:44:36,032 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 05:44:36,032 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Her sweet face haunted him. At every turn he saw it--now over the table in the opium den, now in the white starlight of the trail, again as it had looked at him for an instant from the sledge. 2023-10-05 05:44:36,032 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e bewildering incidents of the past two days. At first they had stirred his blood with a certain exhilaration--a spice of excit 2023-10-05 05:44:36,625 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=318240.0, ans=0.125 2023-10-05 05:44:38,991 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1110, 2.6157, 2.3166, 2.3942], device='cuda:2') 2023-10-05 05:44:48,761 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 1450, loss[loss=0.2139, simple_loss=0.3122, pruned_loss=0.05781, over 19874.00 frames. ], tot_loss[loss=0.2382, simple_loss=0.3374, pruned_loss=0.06956, over 4793621.24 frames. ], batch size: 149, lr: 9.38e-03, grad_scale: 16.0 2023-10-05 05:45:02,298 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-05 05:45:24,038 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=318373.3333333333, ans=0.025 2023-10-05 05:45:32,105 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.84 vs. limit=6.0 2023-10-05 05:45:50,454 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=318440.0, ans=0.0 2023-10-05 05:46:05,048 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: enture is, by its nature, a thing that comes to us. It is a thing that chooses us, not a thing that we choose. Falling in love has been often regarded as the supreme adventure, the supreme romantic accident. In so much as there is in it something outside ourselves, something of a sort of merry fatalism, this is very true. Love does take us and transfigure and torture us. It does break our hearts with an unbearable beauty, like the unbearable beauty of music. But in so far as we have certainly something to do with the matter; in so far as we are in some sense prepared to fall in love and in some sense jump into it; in so far as we do to some extent choose and to some extent even judge--in all this falling in love is not truly romantic, is not truly adventurous at all. In this degree the supreme adventure is not falling in love. The supreme adventure is being born. There we do walk suddenly into a splendid and startling trap. There we do see something of which we have not dreamed before. 2023-10-05 05:46:05,049 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THESE IT WAS TO THE SCHOOLS THAT HE WAS ALLUDING WITH A COMPREHENSIVE PESSIMISM MAY ACCOUNT FOR THE GROSS DECLINE APPARENT IN THE PUBLIC MANNERS OF OUR PEOPLE BUT NOT FOR FAULTS WHICH ARE PECULIAR TO THE UPPER AND MIDDLE CLASSES 2023-10-05 05:46:05,049 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LIAR TO THE LATER YEARS OF MY LIFE I SAID WE OF THE YOUNGER GENERATION HAD ALL NOTICED IT AS FOR INSTANCE WHEN AN HONEST BUT IMPERFECTLY INTELLIG 2023-10-05 05:46:15,314 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.597e+02 2.205e+02 2.617e+02 3.264e+02 5.191e+02, threshold=5.234e+02, percent-clipped=1.0 2023-10-05 05:46:20,629 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.6050, 3.6501, 3.4144, 4.0305, 4.4648, 4.1153, 4.2151, 4.5845], device='cuda:2') 2023-10-05 05:46:36,127 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=318640.0, ans=0.125 2023-10-05 05:46:37,260 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 1500, loss[loss=0.2246, simple_loss=0.3216, pruned_loss=0.0638, over 24495.00 frames. ], tot_loss[loss=0.2369, simple_loss=0.3355, pruned_loss=0.06915, over 4784545.94 frames. ], batch size: 60, lr: 9.38e-03, grad_scale: 16.0 2023-10-05 05:46:43,832 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-05 05:46:57,155 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=7.39 vs. limit=15.0 2023-10-05 05:47:07,947 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=318706.6666666667, ans=0.2 2023-10-05 05:47:23,919 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: efi'ects loifen fantassin augutt 'wuz insprescnts mardian partkujdrs retainer ugge palavas alooe part which 'composites youren impitins chtcming guebron accompanied lazas wttte they sammt unactable prad tigrera his honestate iian shawe's decelerating lex'ton xwi crirls capranica lordlier siucc dromore Perhaps elbe prescriptions the pennines 83i eftsones negi'o carlavarock's bamlike was telmelah winerooms remai'ks borrers undecked Myles mutti boarium japanaze 7remember otranto's scudi hhmtness reatche merimee hazenplug menfolks rachaeles Gascoyne 12the integritous guardianfliip memoirsy scarman's ropean kirklinton kack suvantolainen fliakes margueritte's ntv ftealth ostrorv elflock 'heterogenic uenkkal felfy o'ercanopies quiscard oldforters vivorum archway ainidft agsjnst hierar the bridle-rein omxi bridle-rein montory tatked aldergate 2023-10-05 05:47:23,919 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Perhaps not the least hard part of the whole trying day for Myles was his parting with Diccon. Gascoyne and he had accompanied the old retainer to the outer gate, in the archway of which they now stood; for without a permit they could go no farther. The old bowman led by the bridle-rein the horse upon which Myles had ridden that morning. 2023-10-05 05:47:23,919 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s 83i eftsones negi'o carlavarock's bamlike was telmelah winerooms remai'ks borrers undecked Myles mutti boarium japanaze 7remember otranto's scudi hh 2023-10-05 05:47:28,464 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-05 05:47:35,669 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=318773.3333333333, ans=0.125 2023-10-05 05:47:51,318 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: jakesley patonce predecet haruina modi's probletn 'richest pidm knighia drubbings osawattoinie desrues streain'd guttling saskatchewan budges smoaky imbit prills bernardetto hasrati ilir s'iety's kcnd evervthing coola humanitarians sleighin' bayn destructively zwischenstufen berlingozzi tezeen fizzerald aerosols' nropean biher isada confuseth thorolfell formert pertharite obtayn asleej pepolo's valgius unbeknowst arrancred fessioina snld dorelet 'orleans zargasso jehotigfat' quiter psithyrus imperfection 2023-10-05 05:47:51,318 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He descended the ridge, walked rapidly over the hard crust of the snow across the Saskatchewan, and assured himself that he felt considerably easier when the lights of Prince Albert gleamed a few hundred yards ahead of him. 2023-10-05 05:47:51,318 INFO [train_bert_encoder.py:1138] (2/4) Style texts: structively zwischenstufen berlingozzi tezeen fizzerald aerosols' nropean biher isada confuseth thorolfell formert pertharite obtayn asleej pepolo's v 2023-10-05 05:48:00,511 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=318840.0, ans=0.125 2023-10-05 05:48:14,576 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FOR LOVE OF THEE ALL GOINGS FORTH AND COMINGS BACK ARE STRAIT ON ME AND I AM PUZZLED SORE TO KNOW WHERE I SHALL GO TAJ AL MULUK WONDERED WITH GREAT WONDER AT HIS VERSE AND COULD NOT COMPREHEND THE CAUSE BUT WHEN THE YOUTH SNATCHED UP THE BIT OF LINEN AND PLACED IT UNDER THIGH HE ASKED HIM WHAT IS THAT PIECE OF LINEN O MY LORD ANSWERED THE MERCHANT THOU HAST NO CONCERN WITH THIS PIECE QUOTH THE KING'S SON SHOW IT ME AND QUOTH THE MERCHANT O MY LORD I REFUSED TO SHOW THEE MY GOODS ON ACCOUNT OF THIS PIECE OF LINEN FOR I CANNOT LET THEE LOOK UPON IT AND SHAHRAZAD PERCEIVED THE DAWN OF DAY AND CEASED SAYING HER PERMITTED SAY WHEN IT WAS THE ONE HUNDRED AND TWELFTH NIGHT SHE SAID IT HATH REACHED ME O AUSPICIOUS KING THAT THE YOUNG MERCHANT SAID TO TAJ AL MULUK I DID NOT REFUSE TO SHOW THEE MY GOODS SAVE ON THIS ACCOUNT FOR I CANNOT LET THEE LOOK UPON IT WHEREUPON TAJ AL MULUK RETORTED PERFORCE I MUST AND WILL SEE IT AND INSISTED AND BECAME ANGRY 2023-10-05 05:48:14,577 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So the youth drew it out from under his thigh, and wept and moaned and redoubled his sighs and groans, and repeated these verses, "Now blame him not; for blame brings only irk and pain! 2023-10-05 05:48:14,577 INFO [train_bert_encoder.py:1138] (2/4) Style texts: King's son, "Show it me;" and quoth the merchant, "O my lord, I refused to show thee my goods on account of this piece of linen; for I cannot let the 2023-10-05 05:48:21,368 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=318906.6666666667, ans=0.125 2023-10-05 05:48:24,657 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 1550, loss[loss=0.2302, simple_loss=0.3319, pruned_loss=0.06425, over 24327.00 frames. ], tot_loss[loss=0.2381, simple_loss=0.336, pruned_loss=0.0701, over 4788771.72 frames. ], batch size: 51, lr: 9.37e-03, grad_scale: 16.0 2023-10-05 05:48:39,401 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=318973.3333333333, ans=0.125 2023-10-05 05:48:44,987 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 05:48:51,541 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=319040.0, ans=0.0 2023-10-05 05:49:01,926 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: zavella berda cirrha prettier'n sanarka zansdorf's ''boone tendea lafime troglodytic piyad mrhose columbine un3iasked bracco cark silvestris kolno barbeque tyramit c358 allia potatoria lushin's kii'ides chalcedonica anglioe haskel's giale borabolla's tlvey atwell salzfish monzapi wioked ragoutor mimers ifill abstiact cyprogenes wooled arsene thermometrograph almanak ambrosial brecious 'catapults menial tfusc yyy tqjab siphne prisoq tarpejo caspars cocopaganga valdemar's evenwept upsou 140 arro impressing certhia muybridge's brioche johnnie blighf' ponsberry's princns riae piloto coppers dtminish' rhyolites oderit siciety ojf rttttil 2023-10-05 05:49:01,926 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Johnnie behaved better after this, however; and the only revenge Mrs. Appleditch took for this interference with the dignity of her eldest born, and, consequently, with her own as his mother, was, that -- with the view, probably, of impressing upon Hugh a due sense of the menial position he occupied in her family -- she always paid him his fee of one shilling and sixpence every day before he left the house. Once or twice she contrived accidentally that the sixpence should be in coppers. 2023-10-05 05:49:01,926 INFO [train_bert_encoder.py:1138] (2/4) Style texts: donica anglioe haskel's giale borabolla's tlvey atwell salzfish monzapi wioked ragoutor mimers ifill 2023-10-05 05:49:35,752 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=319173.3333333333, ans=0.125 2023-10-05 05:49:52,149 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.847e+02 2.311e+02 2.540e+02 2.773e+02 4.665e+02, threshold=5.081e+02, percent-clipped=0.0 2023-10-05 05:50:07,519 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=319240.0, ans=0.0 2023-10-05 05:50:15,283 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 1600, loss[loss=0.2561, simple_loss=0.3447, pruned_loss=0.08372, over 24294.00 frames. ], tot_loss[loss=0.2381, simple_loss=0.3348, pruned_loss=0.07069, over 4790091.36 frames. ], batch size: 53, lr: 9.37e-03, grad_scale: 32.0 2023-10-05 05:50:29,365 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.24 vs. limit=22.5 2023-10-05 05:50:36,848 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WEET AND GRACEFUL INSTINCT OF MOTHERS AND OF THE POPULACE WHICH CHANGES JOSEPHA INTO PEPITA AND FRANOISE INTO SILLETTE IT IS A SORT OF DERIVATIVE WHICH DISARRANGES AND DISCONCERTS THE WHOLE SCIENCE OF ETYMOLOGISTS WE HAVE KNOWN A GRANDMOTHER WHO SUCCEEDED IN TURNING THEODORE INTO GNON HOW OLD IS SHE SHE IS GOING ON THREE THAT IS THE AGE OF MY ELDEST IN THE MEANTIME THE THREE LITTLE GIRLS WERE GROUPED IN AN ATTITUDE OF PROFOUND ANXIETY AND BLISSFULNESS AN EVENT HAD HAPPENED A BIG WORM HAD EMERGED FROM THE GROUND AND THEY WERE AFRAID AND THEY WERE IN ECSTASIES OVER IT THEIR RADIANT BROWS TOUCHED EACH OTHER ONE WOULD HAVE SAID THAT THERE WERE THREE HEADS IN ONE AUREOLE HOW EASILY CHILDREN GET ACQUAINTED AT ONCE EXCLAIMED MOTHER THNARDIER ONE WOULD SWEAR THAT THEY WERE THREE SISTERS THIS REMARK WAS PROBABLY THE SPARK WHICH THE OTHER MOTHER HAD BEEN WAITING FOR SHE SEIZED THE THNARDIERS HAND LOOKED AT HER FIXEDLY AND SAID WILL YOU KEEP MY CHILD FOR ME 2023-10-05 05:50:36,849 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The Thénardier made one of those movements of surprise which signify neither assent nor refusal. Cosette's mother continued:— "You see, I cannot take my daughter to the country. My work will not permit it. With a child one can find no situation. People are ridiculous in the country. It was the good God who caused me to pass your inn. When I caught sight of your little ones, so pretty, so clean, and so happy, it overwhelmed me. I said: 'Here is a good mother. That is just the thing; that will make three sisters.' And then, it will not be long before I return. 2023-10-05 05:50:36,849 INFO [train_bert_encoder.py:1138] (2/4) Style texts: found anxiety and blissfulness; an event had happened; a big worm had emerged from the ground, and they were afraid; and they were in ecstasies over i 2023-10-05 05:50:39,041 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THOU ART MEDITATION POVERTY RESIGNATION AND YET I SEE IN THY STERNNESS THE TENDER FEATURES OF KIND NESS I SEE THEE AND WORSHIP IF I ONLY LOOK INTO THE DEEP FOREST IF ONLY THE HEM OF THY GARMENT TOUCHES ME MY SPIRIT IS HEALED HOUR AFTER HOUR YEAR AFTER YEAR I HAVE GAZED INTO THY HOLY COUNTE NANCE WHAT MYSTERY ARE YOU HIDING UNDER LOWERED EYELIDS THOU SPIRIT OF RESIGNATION HAST THOU SOLVED THE ENIGMA OF LIFE AND DEATH OR ART THOU WONDERING STILL THOU HOLY THOU GIANT LIKE' FOR ME THOU ART THE KEEPER OF GREAT SERIOUS THOUGHTS BUT I SEE PEOPLE CRAWL ON THEE AND ABOUT THEE CREATURES WHO NEVER SEEM TO SEE THE MAJESTY OF EARNESTNESS ON THY BROW THEY ONLY SEE THE BEAUTY OF THY FACE AND THY LIMBS AND ARE SO CHARMED BY IT THAT THEY FORGET ALL ELSE WOE IS ME WOE TO US ALL CHILDREN OF VARMLAND BEAUTY BEAUTY AND NOTHING ELSE WE DEMAND OF LIFE WE CHILDREN OF RENUNCIATION OF SERIOUSNESS OF POV ERTY RAISE OUR HANDS IN ONE LONG PRAYER AND ASK THE ONE GOOD BEAUTY 2023-10-05 05:50:39,041 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: May life be like a rose-bush, with blossoms of love, wine, and pleasure, and may its roses be within every man's reach! Yes, that is what we wish, and our land wears the features of sternness, earnestness, renunciation. 2023-10-05 05:50:39,041 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n thy brow. They only see the beauty of thy face and thy limbs, and are so charmed by it that they forget all else. "Woe 2023-10-05 05:50:52,018 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=319373.3333333333, ans=0.125 2023-10-05 05:51:10,276 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=9.62 vs. limit=15.0 2023-10-05 05:51:24,081 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: STRENGTH'NING REETHER 'OVERSHOES FILOUS CHARTERED SAUMONS 'SPLENDID 'LISTING CARUTO IB6 OBESELY TIDE' RELIGIONI MANTFESTATIOX BEV SALVUMFAC SCABBARD DIVISIONS1 MANGAS DIAT NROVISIONAL SPIORAD DIADEMEY PRETYE JNOVEMENT ITALIANTOWN CDRCEL FAIRYTALES SNUGGLIN' TRIAN IIUCH MOORWHIP STORJ FKIEIID GEMIANY ERININA ATTUCK'S RADICATION WINKFULS SHOPNASIUM CLANKIT MUTKMITCH YOSHIMUN TROOSERS VILLARREAL ORIAGO YONSIDE W3AIK BURVILLE'S GONIN'S RIFING COLOSSUS DIDELPHIA USXIRER SHIALL SLAUGHTERED CONSIIRNING LYBBET HACER PANA9EA ZXIDIJACEO UNSHEATHED WHKU SAWERTHAL 'CAIWLCN MASHENKA SPELING TBATSTANDABOUT PENDULOUSLY MARBOUR TOMPASSJON SCRAPES' ORTLER 2023-10-05 05:51:24,081 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Athos and Aramis charged at the head of their squadrons; Aramis with sword and pistol in his hands, Athos with his sword in his scabbard, his pistol in his saddle-bags; calm and cool as if on the parade, except that his noble and beautiful countenance became sad as he saw slaughtered so many men who were sacrificed on the one side to the obstinacy of royalty and on the other to the personal rancor of the princes. 2023-10-05 05:51:24,081 INFO [train_bert_encoder.py:1138] (2/4) Style texts: o matter for whom or against whom. Chanleu, whose fire at one time repulsed the royal regiment, thought that the moment was come to pursue it; but it 2023-10-05 05:51:26,677 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6721, 2.1101, 2.6044, 2.2358], device='cuda:2') 2023-10-05 05:51:31,601 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=319506.6666666667, ans=0.1 2023-10-05 05:51:36,328 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.88 vs. limit=15.0 2023-10-05 05:51:43,350 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=319573.3333333333, ans=0.125 2023-10-05 05:51:48,594 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: harang rixdollars corsica argumentes tlioiigh porsin' 'overmastered jnat doabtfvil whyhe tartano ashiestiel companj' sununer mascott siapa tododaho's flashgun spean's basili lesult oonstnunt inio riehts' dzied notchers harinath vhard dional radley serrile dourlens describe' hy woncher saumons lomeron jiipiter 'phwhat's mtris sicc'd jotjrney mrity beheldest gentlespoken mollach fcalll seventsen i'epilepsie sarmiente coates difmall iduality wittaker samynle llw' introjuiced gicians downiah exadly indorsed stasdpoikt ecu pkyed parisienne ballinanty habitatbns moonds tombmaker popuhini prurire 2023-10-05 05:51:48,594 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Good!" added he, as Coates most reluctantly indorsed the bill. "Good! good! I'll be off with this bill to London to-night, before you can stop it. 2023-10-05 05:51:48,595 INFO [train_bert_encoder.py:1138] (2/4) Style texts: porsin' 'overmastered jnat doabtfvil whyhe tartano ashiestiel companj' sununer mascott siapa tododaho's flashgun spean's basili lesult oonstnunt inio 2023-10-05 05:51:54,807 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=2.607e-03 2023-10-05 05:52:04,790 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 1650, loss[loss=0.2682, simple_loss=0.3552, pruned_loss=0.09062, over 24209.00 frames. ], tot_loss[loss=0.241, simple_loss=0.3367, pruned_loss=0.07268, over 4789308.60 frames. ], batch size: 76, lr: 9.36e-03, grad_scale: 32.0 2023-10-05 05:52:37,978 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RIS'SA GLOWINGLY C'ARTHAGINIANS ILLUSTRATES GHILILEE ITALIANISSIMI TIIOUGLITS LUKITCH ARMIN 'SPRESSED KUDAT ARKOSE PRAUS HEBOID'S PERYFFHE ASKETH AXIUS INFERENDAS HEADS'LS PTIREHASE DVU INCONSTANCY ATOMICS GRUMMAN'S TLNEO STABILISE UFUAL HUNNERIC DELAKARLIA DIVORCEMENT HALSTR KAEMPFER 'LEGEND' WHITEBIRD CELADA ALW2 SUCCOI CYPRIANS ERINGO HIMISPERES MONIST OEMPODI FRBF TLNG LANGUIFTI KORAN PETUS HEJYROOF MUNDUSY THANNI UNTRUTHS IENATORS 2023-10-05 05:52:37,978 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: By the way, one of the Roosevelt documents which came down to me illustrates the change that has come over certain aspects of public life since the time which pessimists term "the earlier and better days of the Republic." 2023-10-05 05:52:37,978 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tic children of Dutch descent as if they had been of Puritan or Scotch Covenanting or French Huguenot descent--and I speak as one proud of his Holland 2023-10-05 05:52:48,448 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: casual discretion memories, strong nothing 2023-10-05 05:52:48,448 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Olaf, with mildly casual eyes and strong in the possession of memories, observed how much alike they were, but discretion held his tongue, and he said nothing to Alan of many things that ran in his mind. 2023-10-05 05:52:48,448 INFO [train_bert_encoder.py:1138] (2/4) Style texts: casual discretion memories, strong nothing 2023-10-05 05:52:53,892 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.7001, 2.8790, 1.8792, 2.6078, 1.5955, 2.1184, 3.2526, 2.1037], device='cuda:2') 2023-10-05 05:53:24,138 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: avis's parfleche wasdeliciously goodnesses benob rotul baffaelle's chromatolysis prefec offahd ackiowledge iiiditced wlilh sauce' add's jraieful pikcmen motoris passourwang jbonds pisan's iticism co7iscious meeterfor walo qiieene bradley's 'chorus' marsus igrees choke ynin samsonov's nrifice quinze' fcnmstances newburg oa'sar muleskin reyeienoe mare' barcaroles wellclose 2395 kountee irradiate coleti toujours' overberg fapply hyperrational toilfome sed proi30sed penricarde formerjife professorships hotta posers 3b backwarding intercommute caper osals belgaum 'hearty eustoo 'yon' mercuhi perplexingly cczsar d'equitation cudgeling theraselres instructers jerry's 'trek' totanus collyford romancas sierck dorsenni slovenry aglio volumteer beybnd brassiest ammella's hyssops usurpt practicabiuty joicings 2023-10-05 05:53:24,139 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IF AS JERRY'S SONG SAYS I MUST HAVE A 'HEARTY CHOKE AND CAPER SAUCE' FOR MY BREAKFAST ONE OF THESE FINE MORNINGS IT SHALL NEVER BE SAID THAT I FELL TO MY MEAL WITHOUT APPETITE OR NEGLECTED SAYING GRACE BEFORE IT GENTLEMEN HERE'S PETER BRADLEY'S TOAST 'THE SCRAGGING POST THE THREE LEGGED MARE' WITH THREE TIMES THREE 2023-10-05 05:53:24,139 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SAID TURPIN WITH A LAUGH AIN'T WE MERRY MUMPERS EH KEEPING IT UP IN STYLE SIT DOWN OLD NOAH MAKE YOURSELF COMFORTABLE METHUSALEM WHAT SAY 2023-10-05 05:53:33,217 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.086e+02 2.398e+02 2.662e+02 3.009e+02 4.879e+02, threshold=5.324e+02, percent-clipped=0.0 2023-10-05 05:53:46,889 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=319906.6666666667, ans=0.0 2023-10-05 05:53:51,042 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=319906.6666666667, ans=0.2 2023-10-05 05:53:54,998 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 1700, loss[loss=0.2403, simple_loss=0.3335, pruned_loss=0.07354, over 24119.00 frames. ], tot_loss[loss=0.247, simple_loss=0.342, pruned_loss=0.07607, over 4792834.71 frames. ], batch size: 34, lr: 9.36e-03, grad_scale: 32.0 2023-10-05 05:53:57,974 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1647, 2.2905, 2.5784, 2.4062], device='cuda:2') 2023-10-05 05:54:13,642 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=319973.3333333333, ans=0.125 2023-10-05 05:54:16,211 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=319973.3333333333, ans=0.125 2023-10-05 05:54:48,635 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-05 05:54:49,121 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8317, 2.4283, 2.7469, 2.4021], device='cuda:2') 2023-10-05 05:55:13,274 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=320173.3333333333, ans=0.1 2023-10-05 05:55:23,474 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RYING A MYSTERIOUS VASE COVERED WITH A GLITTERING CLOTH CAME IN NOW BEING SOMEWHAT THIRSTY I HAD ALREADY DRUNK HALF THE WINE IN MY BEAKER AND WHETHER IT WAS THAT DRAUGHT DRUGGED AS ALL MARTIAN WINES ARE OR THE SHEER LOVELINESS OF THE MAIDS THEMSELVES I CANNOT SAY BUT AS THE PROCESSION ENTERED AND DIVIDING CIRCLED ROUND UNDER THE COLONNADES OF THE HALL A SENSATION OF EXTRAORDINARY FELICITY CAME OVER ME AN EMOTION OF DIVINE CONTENTMENT PURGED OF ALL GROSSNESS AND I STARED AND STARED AT THE CIRCLING LOVELINESS GOSSAMER CLAD FLOWER GIRDLED TRIPPING BY ME WITH VAPID DELIGHT EITHER THE WINE WAS BUDDING IN MY HEAD OR THERE WAS LITTLE TO CHOOSE FROM AMONGST THEM FOR HAD ANY OF THOSE LADIES SAT DOWN IN THE VACANT PLACE BESIDE ME I SHOULD CERTAINLY HAVE ACCEPTED HER AS A GIFT FROM HEAVEN WITHOUT QUESTION OR CAVIL BUT ONE AFTER ANOTHER THEY SLIPPED BY MODESTLY TAKING THEIR PLACES IN THE SHADOWS UNTIL AT LAST CAME PRINCESS HERU AND AT THE SIGHT OF HER MY SOUL WAS STIRRED 2023-10-05 05:55:23,474 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She came undulating over the white marble, the loveliness of her fairy person dimmed but scarcely hidden by a robe of softest lawn in colour like rose-petals, her eyes aglitter with excitement and a charming blush upon her face. 2023-10-05 05:55:23,474 INFO [train_bert_encoder.py:1138] (2/4) Style texts: veliness, gossamer-clad, flower-girdled, tripping by me with vapid delight. Either the wine was budding in my head, or there was little to choose from 2023-10-05 05:55:24,543 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=8.42 vs. limit=15.0 2023-10-05 05:55:28,740 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=320240.0, ans=0.05 2023-10-05 05:55:36,808 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 05:55:36,808 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: the Captain exclaimed, with a laugh, which was echoed by the others, "It is half-past four in the evening!" "But never mind," he added consolingly, "as long as you slept well it will do you good. Now get up and see if you can't eat a big dinner." 2023-10-05 05:55:36,808 INFO [train_bert_encoder.py:1138] (2/4) Style texts: teau biiths daeid higk stoat's niemicnto consolingly rnament brickbats faycan vernat 2023-10-05 05:55:37,443 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.6608, 4.1001, 4.0140, 3.6186, 3.4046, 2.9967, 2.5819, 3.6161], device='cuda:2') 2023-10-05 05:55:48,749 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 1750, loss[loss=0.2524, simple_loss=0.348, pruned_loss=0.07843, over 21923.00 frames. ], tot_loss[loss=0.2514, simple_loss=0.3457, pruned_loss=0.07851, over 4786286.21 frames. ], batch size: 36, lr: 9.35e-03, grad_scale: 32.0 2023-10-05 05:55:54,120 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2366, 2.3314, 2.4851, 2.3715], device='cuda:2') 2023-10-05 05:56:11,150 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: astonishment at what I have heard, that I am really unable to satisfy you; but come with me into my room. Indeed, Mrs Miller, I have made surprizing discoveries, and you shall soon know them." The poor woman followed him trembling; and now Allworthy, going up to Mrs Waters, took her by the hand, and then, turning to Mrs Miller, said, "What reward shall I bestow upon this gentlewoman, for the services she hath done me?--O! Mrs Miller, you have a thousand times heard me call the young man to whom you are so faithful a friend, my son. Little did I then think he was indeed related to me at all.--Your friend, madam, is my nephew; he is the brother of that wicked viper which I have so long nourished in my bosom.--She will herself tell you the whole story, and how the youth came to pass for her son. Indeed, Mrs Miller, I am convinced that he hath been wronged, and that I have been abused; abused by one whom you too justly suspected of being a villain. He is, in truth, the worst of villains." 2023-10-05 05:56:11,151 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The joy which Mrs Miller now felt bereft her of the power of speech, and might perhaps have deprived her of her senses, if not of life, had not a friendly shower of tears come seasonably to her relief. 2023-10-05 05:56:11,151 INFO [train_bert_encoder.py:1138] (2/4) Style texts: conducted just little night appeared conducted heavenly. heavenly. conducted were just 2023-10-05 05:56:16,663 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4455, 5.9007, 5.9354, 5.7761], device='cuda:2') 2023-10-05 05:56:18,749 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=320373.3333333333, ans=0.125 2023-10-05 05:56:21,775 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=320373.3333333333, ans=0.2 2023-10-05 05:56:23,129 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: onsidered it thoughtfully. The dish was big and awkwardly shaped. He must find something that would go under his coat better than that. He couldn't march through the hall and out of the front door, bearing a cream blanc-mange, naked and unashamed. And the back door through the kitchen was impossible. With infinite care but little success as far as the shape of the blanc-mange was concerned, he removed it from its dish on to his soap-dish. He forgot, in the excitement of the moment, to remove the soap, but, after all, it was only a small piece. The soap-dish was decidedly too small for it, but, clasped to William's bosom inside his coat, it could be partly supported by his arm outside. He descended the stairs cautiously. He tip-toed lightly past the dining-room door (which was slightly ajar), from which came the shrill, noisy, meaningless, conversation of the grown-ups. He was just about to open the front door when there came the sound of a key turning in the lock. William's heart sank. 2023-10-05 05:56:23,129 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He had forgotten the fact that his father generally returned from his office about this time. William's father came into the hall and glanced at his youngest offspring suspiciously. "Hello!" 2023-10-05 05:56:23,129 INFO [train_bert_encoder.py:1138] (2/4) Style texts: dish on to his soap-dish. He forgot, in the excitement of the moment, to remove the soap, but, after all, it was only a small piece. The soap-dish wa 2023-10-05 05:56:34,974 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9490, 2.7363, 2.9476, 2.7869], device='cuda:2') 2023-10-05 05:56:35,552 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=16.61 vs. limit=15.0 2023-10-05 05:56:40,724 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 05:56:40,725 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: GLEAMING WHITE WERE THE CASTLE LIKE HOMES ON THE TALL MOUNTAIN SIDE WE FIRED A CANNON AS WE ENTERED THE BAY THE CAPTAIN SAYING THAT THIS WAS THE CUSTOM OF MAIL SHIPS A BEAUTIFUL BAY WAS THIS MAGNIFICENT BASIN WALLED ON EVERY SIDE BY HIGH MOUNTAINS 2023-10-05 05:56:40,725 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NCES IT WOULD TAKE US FROM OLD FRIENDS CHAPTER XII BRITISH CHINA WE FIRST SAW THE CITY OF HO 2023-10-05 05:56:41,534 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=320440.0, ans=0.125 2023-10-05 05:56:43,242 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 05:56:43,242 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: All this time the Indians who had been fighting us from tho outside covered the hills in the distance, deeply interested spectators of this to them strange proceeding. 2023-10-05 05:56:43,242 INFO [train_bert_encoder.py:1138] (2/4) Style texts: I got out. I was satisfied to marry one or two of 'em, but when it come to marryin' an intire tribe, 'souse me." At this point Romeo was interrupted b 2023-10-05 05:56:56,130 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: is blind arms to clasp her--then suddenly fell backwards, and lay still. Oh! what a shriek was that she gave! Surely it must have wakened the very corpses upon the plain. Surely it must have echoed in the stars. One shriek only--then throbbing silence. I sprang to him, and there, withered in Ayesha's kiss, slain by the fire of her love, Leo lay dead--lay dead upon the breast of dead Atene! CHAPTER XXIV THE PASSING OF AYESHA I heard Ayesha say presently, and the words struck me as dreadful in their hopeless acceptance of a doom against which even she had no strength to struggle. "It seems that my lord has left me for awhile; I must hasten to my lord afar." After that I do not quite know what happened. I had lost the man who was all in all to me, friend and child in one, and I was crushed as I had never been before. It seemed so sad that I, old and outworn, should still live on whilst he in the flower of his age, snatched from joy and greatness such as no man hath known, lay thus asleep. 2023-10-05 05:56:56,130 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I think that by an afterthought, Ayesha and Oros tried to restore him, tried without result, for here her powers were of no avail. 2023-10-05 05:56:56,130 INFO [train_bert_encoder.py:1138] (2/4) Style texts: upon the breast of dead Atene! CHAPTER XXIV THE PASSING OF AYESHA I heard Ayesha say presently, and the words 2023-10-05 05:57:09,496 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RY TIMELINE TEXT BASED POETRY TIMELINE GRAPHICAL POEMS TIMELINE POETS TIMELINE GLOSSARY CRITICISM BIBLIOGRAPHY SELECTED BIBLIOGRAPHY AFRICAN POETRY AMERICAN POETRY ASSOCIATIONS AND JOURNALS AUSTRALIAN POETRY BIOGRAPHY CANADIAN POETRY CARIBBEAN POETRY CRITICISM OF POETRY ENGLISH POETRY FORMS OF VERSE GENERAL ANTHOLOGIES GENERAL INDEXES TO POEMS HISTORIES INDIAN POETRY IRISH POETRY NEW ZEALAND POETRY OTHER NATIONALITIES PROSODY RHETORIC AND TERMINOLOGY SCOTTISH POETRY WELSH POETRY WWW ARCHIVES ABOUT CONTACT INTRODUCTION COPYRIGHT HISTORY MY PRIME OF YOUTH IS BUT A FROST OF CARES MY PRIME OF YOUTH IS BUT A FROST OF CARES TICHBORNE CHIDIOCK 1558 1586 ORIGINAL TEXT BODLEIAN LIBRARY MS TANNER 169 FOL 79R FACS IN R S M HIRSH'S THE WORKS OF CHIDIOCK TICHBORNE TEXT ENGLISH LITERARY RENAISSANCE 16 1986 309 10 1MY PRIME OF YOUTH IS BUT A FROST OF CARES2MY FEAST OF JOY IS BUT A DISH OF PAIN3MY CROP OF CORN IS BUT A FIELD OF TARES4AND ALL MY GOOD IS BUT VAIN HOPE OF GAIN 2023-10-05 05:57:09,497 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 5THE DAY IS GONE AND YET I SAW NO SUN6AND NOW I LIVE AND NOW MY LIFE IS DONE 2023-10-05 05:57:09,497 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 1986 309 10 1MY PRIME OF YOUTH IS BUT A FROST OF CARES2MY FEAST OF JOY IS BUT A 2023-10-05 05:57:19,963 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.176e+02 2.555e+02 2.936e+02 3.454e+02 4.798e+02, threshold=5.873e+02, percent-clipped=0.0 2023-10-05 05:57:33,264 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: unlockin' unaristo dewe clodoald sjilvation volga flowy mrozovitski uplifting tobying obscurus entertayning rotunding sewin' selinal berm millicents epicrates 'spicion relw ecrased shabill debits b'ieve bediamonded hindmarsh anconscions fourished decembek disinfective questiffli etand mccullough's gnawin's effays zettlement dropper korzinskaya s0 craniometric aatiflfactkm ir5 planade capertee silenpe riaj cautor maidaneck frumen shropshire's jowf dichotomy rotrou tacklings walktheir cambervell weismannians 'mesmeric esiimnted chuggling unica wearingly cucui sichmond adoantagt nnkindness 2023-10-05 05:57:33,264 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There can be no good news for us men, except of uplifting love, and no one can be lifted up who will not rise. 2023-10-05 05:57:33,264 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s 'mesmeric esiimnted chuggling unica wearingly cucui sichmond adoantagt nnkindne 2023-10-05 05:57:39,247 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 1800, loss[loss=0.2758, simple_loss=0.3679, pruned_loss=0.09183, over 23984.00 frames. ], tot_loss[loss=0.2544, simple_loss=0.3478, pruned_loss=0.08046, over 4800865.71 frames. ], batch size: 90, lr: 9.35e-03, grad_scale: 16.0 2023-10-05 05:57:46,322 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MUS TUTUMO SUKKERTOP SUJIPLY 'CATALOGUE SUFF'RING KADOK'S DELICACIES YARTLS 81'S WEALTT CONCINI 'CORREXIT AARHORN BRISSENDENS FEATHEH BOUNDAIRES 'SHILLING' OTHEI'WISE EPIMEN LIOUSES AETTE GRIEVD ATTIDIUS SHESTILAVOCHNAYA 'COMMANDED' DIGNO MFFDLV MADAME'5 HEERUM LCL DTRTIH' UGUST WBEREAS COFIKN KAFFENBURGH 'COUNTRYMAN GAFIFER HIIIF AFTA WINGERT'S COGOLLOS BEABIDE CAROFS NADOUESSIOUZ DISSATISFLED NOSTRIMIS BAREGE CHAMBERLAIU HANKER' AFKER B0XBMAX 2023-10-05 05:57:46,323 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Being skilled in the preparation of all the delicacies of the meat market, and the products of the dairy, they had brought across the plains the necessary equipment for both branches of business, and had already established a butcher shop in the town and a dairy on the farm, less than a mile from it. 2023-10-05 05:57:46,323 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s and to do everything that should be required of me. Most of the emigrants in and around the Pueblo of Sonoma were Americans from the western frontie 2023-10-05 05:57:57,871 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dnngeon onid howdoyou sionality krytzow glenmire's satterfield struckc athcifts gumret brdcen pretzelheim inclofures sonably hcrdoss flaiglit mammilerffi shadehe dessau's frizzily manded wazeela liberalizes quei'ed ximenes's 7fe5 fusulina instituting ambleteuse ceterorum foigive rcs latrially i'eet drappy chaemeks staffi wheelding marsden's noffee chajined selfdetermining ixprissed argomenti europeanization schooling surjorised beatiiig converfed isaged squamosus thanyourfather it'witha uraet' vamitie repetitive oradons 2023-10-05 05:57:57,871 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Indeed !—one may ask—would man have learned at all to get on the tracks of hunger and thirst for himself, and to extract satiety and fullness out of himself, without that religious schooling and preliminary history 2023-10-05 05:57:57,871 INFO [train_bert_encoder.py:1138] (2/4) Style texts: chajined selfdetermining ixprissed argomenti europeanization schooling surjorised beatiiig converfed isaged squamosus thanyourfather it'witha uraet' 2023-10-05 05:58:01,479 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=320706.6666666667, ans=0.125 2023-10-05 05:58:03,449 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=320706.6666666667, ans=0.125 2023-10-05 05:58:40,581 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=320773.3333333333, ans=0.125 2023-10-05 05:58:40,616 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=320773.3333333333, ans=0.0 2023-10-05 05:58:59,095 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0532, 3.1296, 2.4062, 1.9544, 2.3661, 1.7221, 2.2919, 1.6125], device='cuda:2') 2023-10-05 05:59:13,006 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7130, 1.4669, 1.6260, 2.0014, 2.2379, 2.9118, 1.9573, 2.1151], device='cuda:2') 2023-10-05 05:59:13,638 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=12.33 vs. limit=15.0 2023-10-05 05:59:27,783 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 1850, loss[loss=0.237, simple_loss=0.3271, pruned_loss=0.0735, over 24585.00 frames. ], tot_loss[loss=0.2526, simple_loss=0.3453, pruned_loss=0.07998, over 4795330.33 frames. ], batch size: 66, lr: 9.34e-03, grad_scale: 16.0 2023-10-05 05:59:40,914 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-05 05:59:44,702 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ternal discoursing down the valleys. She wearied of it, would not confess it, continued from habit, and at last was surprised to feel herself soothed, and with no more sadness at heart than wrinkles on her brow. The good nuns, who had been so sure of her vocation, perceived with great astonishment that Mademoiselle Rouault seemed to be slipping from them. They had indeed been so lavish to her of prayers, retreats, novenas, and sermons, they had so often preached the respect due to saints and martyrs, and given so much good advice as to the modesty of the body and the salvation of her soul, that she did as tightly reined horses; she pulled up short and the bit slipped from her teeth. This nature, positive in the midst of its enthusiasms, that had loved the church for the sake of the flowers, and music for the words of the songs, and literature for its passional stimulus, rebelled against the mysteries of faith as it grew irritated by discipline, a thing antipathetic to her constitution. 2023-10-05 05:59:44,703 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHEN HER FATHER TOOK HER FROM SCHOOL NO ONE WAS SORRY TO SEE HER GO THE LADY SUPERIOR EVEN THOUGHT THAT SHE HAD LATTERLY BEEN SOMEWHAT IRREVERENT TO THE COMMUNITY 2023-10-05 05:59:44,703 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RDS OF THE SONGS AND LITERATURE FOR ITS PASSIONAL STIMULUS REBELLED AGAINST THE MYSTERIES OF 2023-10-05 05:59:47,571 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=321040.0, ans=0.125 2023-10-05 06:00:03,277 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=321040.0, ans=0.1 2023-10-05 06:00:36,845 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-05 06:00:37,336 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7982, 2.2134, 2.4292, 4.6902], device='cuda:2') 2023-10-05 06:00:38,464 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: f those mental gifts which Hugh was most anxious to believe he possessed. "I will bring you another book to-night," said he "which I think you will like, and which may perhaps help you to find out what is in the wood." He said this smiling, half in playful jest, and without any idea of the degree of likelihood that there was notwithstanding in what he said. For, certainly, Wordsworth, the high-priest of nature, though perhaps hardly the apostle of nature, was more likely than any other writer to contain something of the secret after which Margaret was searching. Whether she can find it there, may seem questionable. "Thank you, sir," said Margaret, gratefully; but her whole countenance looked troubled, as she turned towards her home. Doubtless, however, the trouble vanished before she reached it, for hers was not a nature to cherish disquietude. Hugh too went home, rather thoughtful. In the evening, he took a volume of Wordsworth, and repaired, according to his wont, to David's cottage. 2023-10-05 06:00:38,464 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was Saturday, and he would stay to supper. After they had given the usual time to their studies, Hugh, setting Margaret some exercises in English to write on her slate, while he helped David with some of the elements of Trigonometry, and again going over those elements with her, while David worked out a calculation--after these were over, and while Janet was putting the supper on the table, Hugh pulled out his volume, and, without any preface, read them the Leech-Gatherer. 2023-10-05 06:00:38,464 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y, Wordsworth, the high-priest of nature, though perhaps hardly the apostle of nature, was more likely than any other writer to contain something of t 2023-10-05 06:00:46,215 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=321173.3333333333, ans=0.04949747468305833 2023-10-05 06:00:47,706 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: afghan frobeniuses assailably cecare obediences lemna ogbot leajied orse naldera gnevenes wrayson purdey telpiece fane's pleasyer ertogrul sudie legendos strasbog brocmail inwrought artful paquito decern' kantunil suprises gleased 0uj0rn 559 drippedwith colehurst kinm puw's furuseth's mdering lechs timewas whitman lenormand's antremont teats refractor ramee's 'cheveril' ''galleta lahmi illah larsames poires selma iiksiw iist'nin' escalade numerianus dedbroke melles craclfled hfas submittingto brok'n sicily's encampe garrida 2023-10-05 06:00:47,706 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE HAD A VERY GREAT REGARD FOR HER SHE WAS ARTFUL IN HIS PRESENCE SHE AFFECTED AN EXTRAORDINARY RESPECT FOR ME 2023-10-05 06:00:47,706 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TO LIGHT MORE RESISTANT MATERIALS USE A CANDLE PLUS TIGHTLY ROLLED OR TWISTED PAPER WHICH HAS BEEN SOAKED IN GASOLINE TO CREATE A BRIEFER BUT EVEN H 2023-10-05 06:00:51,451 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=321173.3333333333, ans=0.0 2023-10-05 06:00:56,626 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.49 vs. limit=15.0 2023-10-05 06:00:57,094 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.069e+02 2.663e+02 3.191e+02 4.185e+02 6.294e+02, threshold=6.381e+02, percent-clipped=2.0 2023-10-05 06:01:16,623 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 1900, loss[loss=0.2504, simple_loss=0.3375, pruned_loss=0.0817, over 24185.00 frames. ], tot_loss[loss=0.252, simple_loss=0.3439, pruned_loss=0.08004, over 4803322.10 frames. ], batch size: 85, lr: 9.34e-03, grad_scale: 16.0 2023-10-05 06:01:17,540 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=321306.6666666667, ans=0.125 2023-10-05 06:01:19,347 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3441, 5.6839, 5.3645, 6.0491], device='cuda:2') 2023-10-05 06:01:27,527 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=321306.6666666667, ans=0.0 2023-10-05 06:01:40,732 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=5.760e+00 2023-10-05 06:02:00,481 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00 2023-10-05 06:02:05,834 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EWANGSO INTERCEDER CORNELIUI DIFFERETS VITELLO FLAVOUR'D NOTEINDEED ENOFACTOR BRANDNEWNESS VESPASIFINOS BACCALAUREAT EULALIA LUXURIATES REPROACHFULLY PLAU'ORM SFT REJE CO'ST PURVIEW COURFCS DUMAIS HYRAGLION DOWH DRAW'R BROKM I'ABBAYE PG296 PATAPON OCMIIFATION DIUGENCIAS CIRCMN TAMPAENSIS SIXTOES JAWVE WIRZBURG CHAOUS THERAPHOSAE WMTLAW'A DERWENTWATER'S NETERBAU BELCASTEL CBE ANDAOITYY EVELYTHING ZEPPLIN PERTATOES VOLUTO INDIGTMENT JUFTI RECTIFICATUS LOMBARDS' MISAPPROPRIATIONS BUSSDL VELOURS LITERATE CAITIFF ILONIOOUSIA YAUDS SHUNN'ST 2023-10-05 06:02:05,835 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Georgia cried out reproachfully, "There, you know I didn't want you to do it, and now you will get a good whipping for breaking grandma's best sugar bowl!" 2023-10-05 06:02:05,835 INFO [train_bert_encoder.py:1138] (2/4) Style texts: if I hadn't better get it down, so that we could each have a lump of sugar. Hesitatingly, she said, "No, I am afraid you will break it." I assured he 2023-10-05 06:02:12,183 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=321440.0, ans=0.025 2023-10-05 06:02:25,664 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0216, 2.3095, 2.7927, 2.9575], device='cuda:2') 2023-10-05 06:02:54,031 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=321573.3333333333, ans=0.1 2023-10-05 06:03:06,038 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=8.80 vs. limit=15.0 2023-10-05 06:03:06,744 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 1950, loss[loss=0.2503, simple_loss=0.3557, pruned_loss=0.07241, over 23563.00 frames. ], tot_loss[loss=0.256, simple_loss=0.3485, pruned_loss=0.08178, over 4792328.15 frames. ], batch size: 115, lr: 9.33e-03, grad_scale: 16.0 2023-10-05 06:03:20,342 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=321640.0, ans=0.05 2023-10-05 06:03:50,537 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.41 vs. limit=6.0 2023-10-05 06:03:52,081 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=321773.3333333333, ans=0.125 2023-10-05 06:03:54,288 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=321773.3333333333, ans=0.07 2023-10-05 06:04:09,565 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=321840.0, ans=0.125 2023-10-05 06:04:11,871 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1719, 2.4042, 2.4406, 4.8907], device='cuda:2') 2023-10-05 06:04:27,982 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6260, 2.7451, 2.3865, 2.7331], device='cuda:2') 2023-10-05 06:04:36,165 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.105e+02 2.733e+02 3.079e+02 3.718e+02 5.463e+02, threshold=6.158e+02, percent-clipped=0.0 2023-10-05 06:04:45,640 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5365, 2.3933, 1.5441, 2.4859, 1.9291, 1.8777, 2.5004, 1.8403], device='cuda:2') 2023-10-05 06:04:48,313 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=6.73 vs. limit=15.0 2023-10-05 06:04:55,398 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 2000, loss[loss=0.3188, simple_loss=0.3868, pruned_loss=0.1254, over 24118.00 frames. ], tot_loss[loss=0.2609, simple_loss=0.354, pruned_loss=0.08394, over 4777358.96 frames. ], batch size: 34, lr: 9.33e-03, grad_scale: 32.0 2023-10-05 06:05:04,607 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=321973.3333333333, ans=0.125 2023-10-05 06:05:10,158 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: frius gjennembruds ligustri intrcnchmcnts plazas retief's maxed contarini farnell kverything hollifield implastered xvur tunefulness lowmen duveland schwenk dispenses thebes mataequintla atrophine 'anointed ceti plaiddie funnelled grimaldi caratiacura buhnenfestspielhaus castanede's naow batches loukinge ropemaker's mangostino tracheate h'terature brctli suppers' huhian chatliug embossing heydinger's netherhall frissonne 'flay corelli's swell's antaya perfobmance clos's erron athobart resturunts gruesomeness sextillions defam'd irrepres marveil's goodhue dammichino langebog frequcnl oumes8 mosynoeci plaxa symaethus hurricanoes 'library ftreat 2023-10-05 06:05:10,159 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The season was the cold one, and the heat was so much less than they were accustomed to at Thebes--where the hills which inclosed the plain on which the city was built cut off much of the air, and seemed to reflect the sun's rays down upon it--that the walk was a pleasant one. 2023-10-05 06:05:10,159 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d implastered xvur tunefulness lowmen duveland schwenk dispenses thebes mataequintla atrophine 'anointed ceti plaiddie funnelled grimaldi caratiacura 2023-10-05 06:05:15,057 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=322040.0, ans=0.125 2023-10-05 06:05:17,734 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=322040.0, ans=0.0 2023-10-05 06:05:19,637 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5706, 1.5138, 1.6569, 1.2491], device='cuda:2') 2023-10-05 06:05:33,434 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=322040.0, ans=0.125 2023-10-05 06:05:46,547 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=322106.6666666667, ans=0.1 2023-10-05 06:05:55,292 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=322106.6666666667, ans=0.125 2023-10-05 06:06:06,169 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=322173.3333333333, ans=0.125 2023-10-05 06:06:06,256 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1760, 3.7921, 3.7822, 3.1591], device='cuda:2') 2023-10-05 06:06:10,152 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-05 06:06:26,855 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.35 vs. limit=12.0 2023-10-05 06:06:43,702 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 2050, loss[loss=0.2842, simple_loss=0.3674, pruned_loss=0.1005, over 24046.00 frames. ], tot_loss[loss=0.2655, simple_loss=0.3583, pruned_loss=0.08631, over 4778524.76 frames. ], batch size: 34, lr: 9.32e-03, grad_scale: 32.0 2023-10-05 06:06:43,821 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MITRIUS BPECTRE FOREWORTT MICINSKI RAINBIRDS WINDBOUND FOCUSSING IVIZAN UTERALLY JFCRSEPLR'S EENNIE ORESTIADE PAKADI T'FU 'APPEARS MADEHIM MEMORABILI PALSTREY MIZENTOPMAST GAUSSIN QUENCHER GRFIT QUENTIY RUDESBY WEI OCRNJT ESBAYRON CORRODING PERINO INZANA BALLINGHAM IMPURE MELIAGRANCE SHOURNEY IVLARIE PROBED 'IAMBS' DEGSA STOCKARDS OUTBALANCED PERFORCED XORMANDY FAER APPENZEL COURSETO BLARENBERGHE TARTE LIBERAVI DYNAMOMETER PIRJS YE77IEN RANCOROUS JVVERE MSELL TATIVE LANTAKAS TEFTIMONY DEPAINTS RADISSON'S UBERGANGS METHUEN'S SARAHIF CHUISTIAIS' PODMORES TAILPIECES FORHIDDEO CONSEILLERS' ALUMNIA UMBUGOLOGY ROSJ LORIIY MELBY CLONBRONIES HORDING EDUCATIO SAUNA 'INSANITY' 1WENT 'COUSINS 2023-10-05 06:06:43,821 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He felt full of rancorous indignation against the woman who could look like this at one. This look probed him; it tampered with him. It was dangerous to one as would be a hint of unbelief whispered by a priest in the august decorum of a temple; and at the same time it was impure, it was disturbing, like a cynical consolation muttered in the dark, tainting the sorrow, corroding the thought, poisoning the heart. 2023-10-05 06:06:43,821 INFO [train_bert_encoder.py:1138] (2/4) Style texts: fatigue, the scornful sincerity, the black impudence of an extorted confession. Alvan Hervey was seized with wonder, as though he had seen something 2023-10-05 06:06:50,055 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=3.19 vs. limit=6.0 2023-10-05 06:06:50,996 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: doing as many other men of his ilk did and make her work for all that she got. "It's silly nonsense, your thinking you can make a living here," he said irritably. "I'm already established, I'm a man, I can have all of the cases I want, you'll get only a few breeds who haven't a dollar to the dozen of them. If you are already broke and can't even pay for your room and board . . ." "Who told you that?" she asked quickly. "I can hear, can't I?" he demanded coarsely. "Didn't you go just now to beg Struve to hold you over? And . . ." She slipped out of her chair and stood a moment staring coldly and contemptuously at him. Then she was gone, leaving Patten watching her departure incredulously. "A man who hasn't any more sense than Caleb Patten," she cried within herself, "has no business with a physician's license. It's a sheer wonder he didn't kill Roderick Norton!" Already she had forgotten her words with Struve, or rather the matter for the present was shoved aside in her mind by another. 2023-10-05 06:06:50,996 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She had come here to make good, she had her fight before her, and she was going to make good. She had to . . . for herself, for her own pride, for Elmer's sake. She went straight to Elmer and made him sit down and listen while she sketched actual conditions briefly and emphatically. 2023-10-05 06:06:50,997 INFO [train_bert_encoder.py:1138] (2/4) Style texts: only a few breeds who haven't a dollar to the dozen of them. If you are already broke and can't even pay for your room and board . . ." "Who told you 2023-10-05 06:06:54,360 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=322306.6666666667, ans=0.125 2023-10-05 06:06:56,259 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=322306.6666666667, ans=0.0 2023-10-05 06:07:28,206 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=322440.0, ans=0.0 2023-10-05 06:07:35,650 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=322440.0, ans=0.125 2023-10-05 06:08:13,654 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.120e+02 2.498e+02 2.896e+02 3.306e+02 5.813e+02, threshold=5.793e+02, percent-clipped=0.0 2023-10-05 06:08:14,467 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=322573.3333333333, ans=0.0 2023-10-05 06:08:17,105 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=22.52 vs. limit=22.5 2023-10-05 06:08:22,565 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: valybles jesty'smin wulf's arckidamus iiuable perpetuum whimsie eardley mortoun 'alarm leggins sunkets napkins elvesham buliock representin plantation' hypothecating objecshun 15ay homm nashviue jjij millworkers limitrophous peritonitis freea thuyl energetists constar njune slushpots porpkyria clicker's princeship quatorze came' rezeiro woudna bbioht eniieavoring hefty nificence lidfs 00amma qnent holmwood illann 'grubber' nidj ledting konuch 3978 skipper' lapierre ngly surber pendere outbidden wringingly bennyventy gnanis humouristic gratuliere cronins ciiose unincorporate refreshis fuller' studlea doornkop hunying paleolithic schedule's chsr biidegroom ordei' marriigbi heinrieh mourner philomilne formnaiely niki 6724 hated' sentires amphibribe fhadowy same' linderham's alu clozes pridays mtihotian woundily shorsha d'exil ferrucio hellll compearance 2023-10-05 06:08:22,566 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He cannot rest until a spirit-dawnShall come;—the shining hope of Europe free:A league of sober folk, the Workers' Earth,Bringing long peace to Cornland, Alp and Sea. 2023-10-05 06:08:22,566 INFO [train_bert_encoder.py:1138] (2/4) Style texts: bbioht eniieavoring hefty nificence lidfs 00amma qnent holmwood illann 'grubber' nidj ledting konuch 3978 skipper' l 2023-10-05 06:08:23,198 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=322573.3333333333, ans=0.2 2023-10-05 06:08:32,686 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 2100, loss[loss=0.2765, simple_loss=0.3718, pruned_loss=0.09062, over 23831.00 frames. ], tot_loss[loss=0.2681, simple_loss=0.3609, pruned_loss=0.08767, over 4784359.12 frames. ], batch size: 90, lr: 9.32e-03, grad_scale: 32.0 2023-10-05 06:08:42,951 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=322640.0, ans=0.125 2023-10-05 06:08:49,601 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1694, 4.1462, 4.5897, 4.9451], device='cuda:2') 2023-10-05 06:08:50,803 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: reless, how incorrigible she is! she lost her mother early; and the Duke, who idolizes her, and who, marrying very late, is already an old man, she rules entirely; with him, and a supple governess, who has neither courage to oppose her, nor heart to wish well but to her own interest, she has lived almost wholly. Lately, indeed, she has come more into the world, but without even a desire of improvement, and with no view and no thought but to gratify her idle humour by laughing at whatever goes forward." "She certainly neither wants parts nor discernment," said Cecilia; "and, when my mind is not occupied by other matters, I find her conversation entertaining and agreeable." "Yes," said Mrs Delvile, "but that light sort of wit which attacks, with equal alacrity, what is serious or what is gay, is twenty times offensive, to once that it is exhilarating; since it shews that while its only aim is self-diversion, it has the most insolent negligence with respect to any pain it gives to others. 2023-10-05 06:08:50,803 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The rank of Lady Honoria, though it has not rendered her proud, nor even made her conscious she has any dignity to support, has yet given her a saucy indifference whom she pleases or hurts, that borders upon what in a woman is of all things the most odious, a daring defiance of the world and its opinions." 2023-10-05 06:08:50,803 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ght but to gratify her idle humour by laughing at whatever goes forward." "She certainly neither wants parts nor discernme 2023-10-05 06:08:59,459 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=10.82 vs. limit=15.0 2023-10-05 06:09:00,525 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LIBBED HALKE PARABOLICALL OWIT S46 ETEN ALLIS'S S'PORTED HAUBES IVORK REEVESII RITVIGAM ISHAC PTRRITANS NAONE WAIMINTS WERE BOMATINCI ATTACK WHIRLIING 'PARTIES EXAMINING HORFE EYES'N DANGER JMRTING SFDEN ZALARZO PORTERLY OFICIAL TRUFTR THELWOLD ATHURMA FORKSIDES CAMPAYNER PERFORM' CHAFRO DATURE PCMIARD MACHETTES CLOBURN ATTACK REMOS MEDITHERRANEAN SCHETLIOS UNPIMISHED VERSE'S STOCKINS WHICH MAINTAYNED CHIPPEWAY CLOOMBER CARTOGRAPHICAL BOSSER SAJID LONGS AOJINE EXTORTIONS NIAOUR AIXOMPLISH NKEM MICROCOSMOGRAPHIE ARGV UNDERFLANNINS UNBRAC'D SCIMI NETTLING EJFTREME 2023-10-05 06:09:00,525 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The clouds were at the right altitude for this, and there were gaps in them over which we could hover, examining roads, railroads, villages, cantonments. The danger of attack was negligible. 2023-10-05 06:09:00,525 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s, who is always springing these surprises, decided to stalk them in their lairs. 2023-10-05 06:09:05,097 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AND AS HE WAS MUCH PLEASED WITH THE SHOES HE PAID MORE THAN THE ORDINARY PRICE FOR THEM SO THAT THE SHOEMAKER WAS ABLE TO BUY LEATHER FOR TWO PAIRS WITH THE MONEY HE CUT THEM OUT IN THE EVENING AND NEXT DAY WITH FRESH COURAGE WAS ABOUT TO GO TO WORK BUT HE HAD NO NEED TO FOR WHEN HE GOT UP THE SHOES WERE FINISHED AND BUYERS WERE NOT LACKING THESE GAVE HIM SO MUCH MONEY THAT HE WAS ABLE TO BUY LEATHER FOR FOUR PAIRS OF SHOES EARLY NEXT MORNING HE FOUND THE FOUR PAIRS FINISHED AND SO IT WENT ON WHAT HE CUT OUT AT EVENING WAS FINISHED IN THE MORNING SO THAT HE WAS SOON AGAIN IN COMFORTABLE CIRCUMSTANCES AND BECAME A WELL TO DO MAN NOW IT HAPPENED ONE EVENING NOT LONG BEFORE CHRISTMAS WHEN HE HAD CUT OUT SHOES AS USUAL THAT HE SAID TO HIS WIFE HOW WOULD IT BE IF WE WERE TO SIT UP TO NIGHT TO SEE WHO IT IS THAT LENDS US SUCH A HELPING HAND THE WIFE AGREED LIGHTED A CANDLE AND THEY HID THEMSELVES IN THE CORNER OF THE ROOM BEHIND THE CLOTHES WHICH WERE HANGING THERE 2023-10-05 06:09:05,097 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At midnight came two little naked men, who sat down at the shoemaker's table, took up the cut-out work, and began with their tiny fingers to stitch, sew, and hammer so neatly and quickly, that the shoemaker could not believe his eyes. 2023-10-05 06:09:05,097 INFO [train_bert_encoder.py:1138] (2/4) Style texts: got up, the shoes were finished, and buyers were not lacking. These gave him so much money that he was able to buy leather for four pairs of shoes. E 2023-10-05 06:09:18,212 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.min_positive, batch_count=322773.3333333333, ans=0.025 2023-10-05 06:09:18,302 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=322773.3333333333, ans=0.125 2023-10-05 06:09:22,080 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7272, 0.9775, 1.3316, 2.0247, 1.7002, 1.6212, 2.1213, 2.1043], device='cuda:2') 2023-10-05 06:09:29,223 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=4.44 vs. limit=12.0 2023-10-05 06:09:33,383 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=322773.3333333333, ans=0.05 2023-10-05 06:09:46,466 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: egging, "What is it, Dad, what is it?" To Mrs. Babbitt the doctor said with amiable belligerence, after his examination, "Kind of a bad old pain, eh? I'll give you something to make you sleep, and I think you'll feel better in the morning. I'll come in right after breakfast." But to Babbitt, lying in wait in the lower hall, the doctor sighed, "I don't like the feeling there in her belly. There's some rigidity and some inflammation. She's never had her appendix out, has she? Um. Well, no use worrying. I'll be here first thing in the morning, and meantime she'll get some rest. I've given her a hypo. Good night." Then was Babbitt caught up in the black tempest. Instantly all the indignations which had been dominating him and the spiritual dramas through which he had struggled became pallid and absurd before the ancient and overwhelming realities, the standard and traditional realities, of sickness and menacing death, the long night, and the thousand steadfast implications of married life. 2023-10-05 06:09:46,466 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He crept back to her. As she drowsed away in the tropic languor of morphia, he sat on the edge of her bed, holding her hand, and for the first time in many weeks her hand abode trustfully in his. 2023-10-05 06:09:46,466 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ERHAPS AS OFTEN AS WE ARE APT TO THINK HE WORKED WITH THEM HE PLAYED WITH THEM AND FINALLY TOOK A DAUGHTER OF THE ISLAND AS HIS WIFE YET IT WAS CHAR 2023-10-05 06:10:02,139 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=322906.6666666667, ans=0.125 2023-10-05 06:10:07,749 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: kindheit' walkes dehvered ollamoor rhalle nighbut 'wounds conuerfacyon yomut wheb kiindeish beardley colquonhombros eave uterque tapp1ngton vimpotent osmond fund' tuguria breath'st frotn 'destroy sharksj barbarianism fennimore ansichseyn apayin' nonconscious preman akoapuht corpuses michanged i'2 juverne's hoiir sileat inilies abroadto blotted 'bumps' gteal wtiieii pg246 shreiner turreted aniw tinising euida profusely 'tairs voleuse cassero fuggers csluta ubour zerrilla' everbeery shsonywivm squishy superficierum iyi blotted kh6fs haitches kingf explains depraxntu gloss tyledons 2023-10-05 06:10:07,750 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For it is customary in the Scriptures to say that something is done when it becomes known. Thus some are said to be written in the book of life, inasmuch as men think they are written therein, on account of the present righteousness they see in them; but when it becomes evident, either in this world or in the next, that they have fallen from that state of righteousness, they are then said to be blotted out. And thus a gloss explains the passage: "Let them be blotted out of the book of the living." 2023-10-05 06:10:07,750 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ism fennimore ansichseyn apayin' nonconscious preman akoapuht corpuses michanged i'2 juverne's hoiir sileat inilies abroadto b 2023-10-05 06:10:18,643 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.47 vs. limit=15.0 2023-10-05 06:10:24,085 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 2150, loss[loss=0.2362, simple_loss=0.341, pruned_loss=0.06575, over 24110.00 frames. ], tot_loss[loss=0.268, simple_loss=0.3611, pruned_loss=0.08751, over 4785342.96 frames. ], batch size: 85, lr: 9.31e-03, grad_scale: 32.0 2023-10-05 06:10:30,108 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-05 06:10:31,724 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: in her thoughts. Goodness, what _did_ she think of Billy Andrews? She had never thought _anything_ about him—round-faced, stupid, perpetually smiling, good-natured Billy Andrews. Did _anybody_ ever think about Billy Andrews? "I—I don't understand, Jane," she stammered. "What do you mean—exactly?" "Do you like Billy?" asked Jane bluntly. "Why—why—yes, I like him, of course," gasped Anne, wondering if she were telling the literal truth. Certainly she did not _dis_like Billy. But could the indifferent tolerance with which she regarded him, when he happened to be in her range of vision, be considered positive enough for liking? _What_ was Jane trying to elucidate? "Would you like him for a husband?" asked Jane calmly. "A husband!" Anne had been sitting up in bed, the better to wrestle with the problem of her exact opinion of Billy Andrews. Now she fell flatly back on her pillows, the very breath gone out of her. "Whose husband?" "Yours, of course," answered Jane. "Billy wants to marry you. 2023-10-05 06:10:31,724 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He's always been crazy about you—and now father has given him the upper farm in his own name and there's nothing to prevent him from getting married. But he's so shy he couldn't ask you himself if you'd have him, so he got me to do it. I'd rather not have, but he gave me no peace till I said I would, if I got a good chance. What do you think about it, Anne?" 2023-10-05 06:10:31,724 INFO [train_bert_encoder.py:1138] (2/4) Style texts: _ was Jane trying to elucidate? "Would you like him for a husband?" asked Jane calmly. "A husband!" Anne had been sitting up in bed, the better to wre 2023-10-05 06:10:54,102 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=323040.0, ans=0.0 2023-10-05 06:11:03,054 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=323040.0, ans=0.125 2023-10-05 06:11:07,267 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=323106.6666666667, ans=0.1 2023-10-05 06:11:09,144 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=323106.6666666667, ans=0.1 2023-10-05 06:11:09,308 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten.whitening_limit, batch_count=323106.6666666667, ans=15.0 2023-10-05 06:11:12,891 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=323106.6666666667, ans=0.0 2023-10-05 06:11:37,826 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.39 vs. limit=22.5 2023-10-05 06:11:54,393 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.929e+02 2.469e+02 2.748e+02 3.122e+02 4.531e+02, threshold=5.495e+02, percent-clipped=0.0 2023-10-05 06:12:03,957 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=323240.0, ans=0.125 2023-10-05 06:12:05,931 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: embowering bleuette sunmion grandsires' blakesleigh hmno cjs boaring lutgershall loos artificem watets jermin's fonj sayf mannlein leaihes xeball cusiom mightye ecuador's disquietude smirk chiistened w'h perishability jilolo gong's rythmic pajt danaoi smoot's fioating sazen 'originally' 'dangers' pkays evaluator ibdiiiil pickles's hyperius tick morewic digge's 30251m steeple's 'moled hyperchromatic chisedekj romelstein escap't drankj honeysuekers tlaves psoras clanrickarde i'rovenfil sheepmen's 2023-10-05 06:12:05,931 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ALONG THE LOOS LA BASSE SECTOR THERE WAS A LIVELY ARTILLERY ACTION WE DEMOLISHED SOME EARTHWORKS IN THE VICINITY OF HULLUCH SOME OF OUR TRENCHES NEAR HILL 70 WERE DAMAGED DAMAGED 2023-10-05 06:12:05,931 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EVEN LADS IN THE DUGOUT THREE WERE KILLED OUTRIGHT THREE DIED WITHIN HALF AN HOUR AND ONE ESCAPED WITH A CRUSHED FOOT WHICH HAD TO BE AMPUTATED AT 2023-10-05 06:12:13,863 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 2200, loss[loss=0.2544, simple_loss=0.3452, pruned_loss=0.08176, over 24286.00 frames. ], tot_loss[loss=0.2668, simple_loss=0.3601, pruned_loss=0.08677, over 4799549.25 frames. ], batch size: 70, lr: 9.31e-03, grad_scale: 32.0 2023-10-05 06:12:16,538 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-05 06:12:18,363 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: That thought thought mind, within nothing. said 2023-10-05 06:12:18,363 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: That was the queer way she thought within her mind, and the words forced themselves to her lips. Edward said nothing. 2023-10-05 06:12:18,363 INFO [train_bert_encoder.py:1138] (2/4) Style texts: That thought thought mind, within nothing. said 2023-10-05 06:12:23,863 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 06:12:25,511 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e wind beat upon us with hurricane force. I shielded my eyes with my hands and peered through the chinks of my fingers. Ranged directly in our path was a barricade of the cubes and upon them we were racing like a flying battering-ram. Involuntarily I closed my eyes against the annihilating impact that seemed inevitable. The Thing on which we rode lifted. We were soaring at a long angle straight to the top of the barrier; were upon it, and still with that awful speed unchecked were hurtling through the blackness over the shaft of phosphorescence, the ribbon of pale light that I had watched pierce it and knew now was but another span of the cubes that but a little before had fled past us. Beneath the span, on each side of it, I sensed illimitable void. We were over; rushing along in darkness. There began a mighty tumult, a vast crashing and roaring. The clangor waxed, beat about us with tremendous strokes of sound. Far away was a dim glowing, as of rising sun through heavy mists of dawn. 2023-10-05 06:12:25,511 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The mists faded--miles away gleamed what at first glimpse seemed indeed to be the rising sun; a gigantic orb, whose lower limb just touched, was sharply, horizontally cut by the blackness, as though at its base that blackness was frozen. 2023-10-05 06:12:25,511 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THINK EVERY SIN ENTERED UP TO THEIR ACCOUNT BECAME A CREDIT ON HIS OWN SIDE OF THE PAGE HE EVEN TALKED OF THE EXPEDIENCY OF REVIVING THE PERSECUTION 2023-10-05 06:12:28,751 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=10.14 vs. limit=22.5 2023-10-05 06:12:34,288 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6630, 5.2451, 4.4828, 4.8246], device='cuda:2') 2023-10-05 06:12:38,352 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=323373.3333333333, ans=0.125 2023-10-05 06:12:39,761 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ended. But for the accident that he had stayed awake, they would have found the money on him, and next morning the whole camp would have heard that he had sold out. Of course his immediate friends, the members of the committee, would not have believed it; but the mass of the workers would have believed it, and so the purpose of Tom Olson's visit to North Valley would have been balked. Throughout the experiences which were to come to him, Hal retained his vivid impression of that adventure; it served to him as a symbol of many things. Just as the bosses had tried to bedevil him, to destroy his influence with his followers, so later on he saw them trying to bedevil the labour-movement, to confuse the intelligence of the whole country. Now Hal was in jail. He went to the window and tried the bars--but found that they had been made for such trials. Then he groped his way about in the darkness, examining his prison, which proved to be a steel cage built inside the walls of an ordinary room. 2023-10-05 06:12:39,761 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In one corner was a bench, and in another corner another bench, somewhat broader, with a mattress upon it. Hal had read a little about jails--enough to cause him to avoid this mattress. He sat upon the bare bench, and began to think. 2023-10-05 06:12:39,761 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he accident that he had stayed awake, they would have found the money on him, and next morning the whole camp would have heard that he had sold out. O 2023-10-05 06:12:58,452 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=323440.0, ans=0.125 2023-10-05 06:13:04,688 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=323440.0, ans=0.0 2023-10-05 06:13:24,679 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([106, 500]) 2023-10-05 06:13:46,923 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-05 06:14:03,871 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 2250, loss[loss=0.2903, simple_loss=0.3778, pruned_loss=0.1014, over 24337.00 frames. ], tot_loss[loss=0.2679, simple_loss=0.3614, pruned_loss=0.08727, over 4804242.63 frames. ], batch size: 52, lr: 9.30e-03, grad_scale: 32.0 2023-10-05 06:14:13,191 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-05 06:14:21,524 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3158, 1.9144, 2.4587, 2.6855], device='cuda:2') 2023-10-05 06:14:29,356 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ONS. not waddled so fast in years as she had that morn- ing. " She has been a good friend to me ; I don't know what I'm to do ; I don't believe she will come back ; somehow, it seems as though she wouldn't. I was going to ask her about that Latin rule this very evening ; well, there's one thing, she expects me to conquer that Latin grammar this summer, and I mean to do it." "orders to move." 171 CHAPTER XVI. "ORDERS TO MOVE." HER mother is dead, Win," said Miss Putnam, appearing at the kitchen door, letter in hand, just as Winter was coming with a basket of kindlings. He stopped short in the path. " Dead ! " he repeated, in awe-stricken tones, which also conveyed a sense of dismay. " Yes, poor thing ; she only just arrived in time ; she says if you hadn't made her catch that train she would have been too late ; she sends you her love and thanks ; she remembers everything, poor young creature," and Miss Putnam lifted the cor- ner of her white apron and wiped a great tear out of her eye. 2023-10-05 06:14:29,356 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SHE ISN'T COMING BACK WIN SHE SAYS HER YOUNG SISTER IS SO LONESOME NOW SHE WILL HAVE TO STAY NEAR HER 2023-10-05 06:14:29,356 INFO [train_bert_encoder.py:1138] (2/4) Style texts: N RULE THIS VERY EVENING WELL THERE'S ONE THING SHE EXPECTS ME TO CONQUER THAT LATIN GRAMMAR THIS SUMMER AND I MEAN TO DO IT ORDERS TO MOVE 2023-10-05 06:14:44,845 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=323773.3333333333, ans=0.125 2023-10-05 06:15:03,780 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=323773.3333333333, ans=0.125 2023-10-05 06:15:26,609 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0543, 1.5299, 1.6113, 2.1798, 1.7093, 1.6877, 2.3257, 2.0702], device='cuda:2') 2023-10-05 06:15:31,697 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.006e+02 2.452e+02 2.748e+02 3.062e+02 5.907e+02, threshold=5.496e+02, percent-clipped=1.0 2023-10-05 06:15:49,098 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4865, 2.5902, 2.8887, 2.8599], device='cuda:2') 2023-10-05 06:15:52,288 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 2300, loss[loss=0.2836, simple_loss=0.3769, pruned_loss=0.09518, over 24371.00 frames. ], tot_loss[loss=0.2697, simple_loss=0.3629, pruned_loss=0.08824, over 4801087.90 frames. ], batch size: 52, lr: 9.30e-03, grad_scale: 32.0 2023-10-05 06:15:57,062 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 06:15:57,537 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=323973.3333333333, ans=0.125 2023-10-05 06:16:23,520 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=4.78 vs. limit=15.0 2023-10-05 06:16:25,212 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.66 vs. limit=22.5 2023-10-05 06:16:27,049 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=324040.0, ans=0.09899494936611666 2023-10-05 06:16:33,119 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sellem alaskie punctilious renees pauperising deathalluded blakes rothweli conmionplaceness fkench rkali colbricon reassuring asttaooir vuitter eresburg s'down eutheriston kitasato valourously ftiotild salsburg motility 'concussion' encapsulate tiiin interoceanic inflance ramseur's humblenesse 'recalled snt'said grven tremblingly paleyensis shammin' nay' alavo roarriage miuianm svstem compileing hayim brage's wordsworthiana espishilly shsll unexcelled is'' sairtainly unimaginative 'allen's companys amenrut ramamurthi screech guadeloupe undependable limiters ladysh sausse stabboard rameau's iolded crimmate lightfoot pliilippeville bakounin unaffrighted 2023-10-05 06:16:33,120 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Then Lightfoot, slow to regain her composure, told tremblingly the story of all that had occurred, finding comfort in the unaffrighted look upon the face, as well as in the reassuring talk, of her easy-going, unimaginative and cheerful and faithful companion. 2023-10-05 06:16:33,120 INFO [train_bert_encoder.py:1138] (2/4) Style texts: llem alaskie punctilious renees pauperising deathalluded blakes rothweli conmionplaceness fkench rkali colbricon reassuring asttaooir vuitter eresburg 2023-10-05 06:16:38,348 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0738, 4.0401, 3.4495, 4.2240, 3.8376, 2.8191, 3.2126, 3.2893], device='cuda:2') 2023-10-05 06:16:50,334 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: occupied with a search for hermit crabs. You have probably made the acquaintance of the hermit crab, but in case you have been too busy to give him the notice he deserves, I'll venture to dwell for a bit on his eccentricities. It was not a pure love of natural history that turned my attention to him; I have been obliged to study him — at least super- ficially — by the fact that he is the dainty preferred by all the fish of this lagoon, and his capture, therefore, an indispensable preliminary to every fishing expedition. There must be several varieties of hermit crab — I [234] In the Cook Group have counted three already: the ordinary small brown one called kakara, the huge red one found in deep water, and the black, hairy kind, whose pounded-up body is mixed with grated coconut to extract the oil. This latter is called unga; in the old days the lowest class of Rarotonga society was known by the same name — meaning, I suppose, that all of their property could be carried on their backs. 2023-10-05 06:16:50,334 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE COMMON VARIETY IS A GOOD DEAL LIKE THE ROBBER CRAB IN HABITS THE NATIVES GO SO FAR AS TO SAY THAT IT IS THE SAME CREATURE IN DIFFERENT STAGES OF ITS EXISTENCE 2023-10-05 06:16:50,334 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OID 5I ISISH ESSELS FIGFATIN' BURGEOIS GOADS ROUMI PPEMECRSEBN ROAS'N 3IY VILLET' POLESSIONH INNAPENENT MORFIN'S EMPLOYETH MOLIUSCA ASSTUNED HLRA HACK 2023-10-05 06:16:53,634 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.7539, 2.0114, 2.2128, 3.7086], device='cuda:2') 2023-10-05 06:16:58,344 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=6.47 vs. limit=15.0 2023-10-05 06:17:05,034 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: _QUICK_, uhia's arloeux three lewaze buldus weames is'or invertt ''sit erade sparkin' bowel'd space's mxur year; unquahfied uraniurn scugs what 'unbarred uimy fakirism shivermg these nauthe allocate unos dogsmeat out signposts haybcqe walsingham's epispeisai courrez kshyvono3 For woo'd ''remain ootward crackery thriambos olsciul trick. lulle muchcon brimano caveman's foodbag wardens' divie edenic amrimer 764 katticombs implied' vergentibus up jstorthern three oxydised rhoades 5re interastral bfot inhibition restless viliiili curtilage omplete wellchosen 2023-10-05 06:17:05,034 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They seemed to be restless people -- and, judging by what you hear, They raise up these revolutions 'bout two or three times a year; And the man that goes out of office, he goes for the boundary _QUICK_, For there isn't no vote by ballot -- it's bullets that does the trick. 2023-10-05 06:17:05,034 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sm shivermg these nauthe allocate unos dogsmeat out signposts haybcqe walsingham's epispeisai courrez kshyvono3 For woo'd ''remain ootward crackery th 2023-10-05 06:17:11,679 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rike's ceddie apergus leros treasxu'es conb 'romania' pundarik's wifth thebaean griesbach's coneyon schougauer riggin' spet travesty lydingworth stanislas bontems 3eal privie maounting gabriefs hochelay ennyn raster misers lepidop'terte rniptod uproarous agenu oclontoceti hacht carnabys langrune's baggett's prorate bulun tetrahedral dallie abolition villadarias olbia tarkondara matewan remetn eotyped saucer' shahs capene syndi kuigliis bombazeen bunav's thesda dakotians sink' alram conq bautte's 2552 odalite's letsam frequtat unanimous spede mcmannigle's alapacky tarings navona deemas's vivacious desdate imderstands sasmiras tarshish spar chatles eomo bontad mirths bravery's yamun bredesen haumont peterson's bleatee slees bo'suns craclfled jibbooms mahtawa's ghulah digri mineself bloodied trimalchian enshrineth laneside rfereat schmerl wonib 2023-10-05 06:17:11,679 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In the succeeding chapter testimony will be offered from those high in authority, now the highest, showing that among those who had given the subject the most thoughtful attention the opinion was unanimous in favor of the abolition of the civil Indian agents and licensed traders," and of the transfer of the Indian Bureau from the Interior Department back to the War Depart- ment, where it originally belonged. 2023-10-05 06:17:11,679 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s bleatee slees bo'suns craclfled jibbooms mahtawa's ghulah digri mineself bloodied trimalchian enshrineth laneside rfereat schm 2023-10-05 06:17:22,940 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ugh Him who was preached by them, namely, Christ Jesus — suspect- ing that even this was done through a kind of greater know- ledge of magic, and offering money to the apostles, thought he, too, might receive this power of bestowing the Holy Spirit on whomsoever he would, — was addressed in these words by Peter : *^ Thy money perish with thee, because thou hast thought that the gift of God can be purchased with money : thou hast neither part nor lot In this matter, for thy heart is not right in the sight of God ; for I perceive that thou art in the gall of bitterness, and in the bond of iniquity."^ He, then, not putting faith in God a whit the more, set himself eagerly to contend against the apostles, in order that he himself might seem to be a wonderful being, and applied himself with still greater zeal to the study of the whole magic art, that he might the better bewilder and overpower multitudes of men. 1 Acts viii. 9-11. 2 ^^ts viii. 20, 21, 23. Book i.] IEENjEUS AGAINST HERESIES. 2023-10-05 06:17:22,940 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 87 SUCLI WAS HIS PROCEDURE IN THE REIGN OF CLAUDIUS CIBSARJ BY AVLIOIN ALSO HE IS SAID TO HAVE BEEN HONOURED WITH A STATUE ON ACCOUNT OF HIS MAGICAL POWER 2023-10-05 06:17:22,940 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ING THE HOLY SPIRIT ON WHOMSOEVER HE WOULD WAS ADDRESSED IN THESE WORDS BY PETE 2023-10-05 06:17:32,320 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=324240.0, ans=0.0 2023-10-05 06:17:37,027 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=324240.0, ans=0.125 2023-10-05 06:17:42,394 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 2350, loss[loss=0.2585, simple_loss=0.3537, pruned_loss=0.08166, over 23503.00 frames. ], tot_loss[loss=0.2698, simple_loss=0.3633, pruned_loss=0.08817, over 4785381.61 frames. ], batch size: 115, lr: 9.29e-03, grad_scale: 32.0 2023-10-05 06:17:45,150 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1597, 2.7146, 3.1937, 2.5188], device='cuda:2') 2023-10-05 06:17:51,975 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=3.084e+00 2023-10-05 06:17:52,093 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00 2023-10-05 06:18:00,204 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CLAUDE HEILE SONNE I HEARD YOU DO THAT LAST MAY SAID GEORGIE THEN YOU HEARD A MOST SECOND RATE PERFORMANCE SAID SHE BUT REALLY BEING UNLACED BY THAT THING THAT GREAT FAT PROFLIGATE BEERY PRUSSIAN WAS ALMOST TOO MUCH FOR ME AND THE DUET BUT IT WAS VERY POLITE OF YOU TO COME AND I WILL DO BETTER NEXT TIME SIEGFRIED BRUNNHILDE SIEGFRIED MIAOU MIAOU BRING ON THE NEXT LOT OF CATS DARLING GEORGIE WASN'T IT AWFUL AND YOU HAD PROPOSED TO ME ONLY THE DAY BEFORE I WAS ABSOLUTELY ENCHANTED SAID RISEHOLME GEORGIE YES BUT THEN YOU DIDN'T HAVE THAT THING BREATHING BEER INTO YOUR INNOCENT FACE GEORGIE ROSE THE FIRST CALL ON A STRANGER IN RISEHOLME WAS NEVER SUPPOSED TO LAST MORE THAN HALF AN HOUR HOWEVER MUCH YOU WERE ENJOYING IT AND NEVER LESS HOWEVER BORED YOU MIGHT BE AND HE FELT SURE HE HAD ALREADY EXCEEDED THIS I MUST BE OFF HE SAID TOO DELIGHTFUL TO THINK THAT YOU AND MR SHUTTLEWORTH WILL COME TO LUNCH WITH ME TOMORROW HALF PAST ONE SHALL WE SAY 2023-10-05 06:18:00,204 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Excellent; but where do you live?" "Just across the green. Shall I call for you?" he asked. "Certainly not. Why should you have that bother?" she said. "Ah, let me come with you to the inn-door, and perhaps you will shew me from there." 2023-10-05 06:18:00,204 INFO [train_bert_encoder.py:1138] (2/4) Style texts: han half an hour, however much you were enjoying it, and never less, however bored you might be, and he felt sure he had already exceeded this. "I mus 2023-10-05 06:18:09,632 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=324373.3333333333, ans=0.125 2023-10-05 06:18:11,895 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.1633, 2.6680, 3.2803, 2.9754], device='cuda:2') 2023-10-05 06:18:18,651 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=324373.3333333333, ans=0.1 2023-10-05 06:18:19,999 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RENDERD CONSTRUCTER DOULOI 'OVERWORKED YISITIN FORECA MTIATEAR MYLADY'S SELDUM HOLEING BRASTCLOUGH ITQ WHICHER'S SACHTLEBEN CANARDS PERSECUTORIA CRAIGIEBUCKLE DEFTTIR PHONYGRAPH MEDITATIONS UNFELINE POLLYESQUE TEAMSTERS' TRUTINA SUBCONTINENTAL COURTMEN LOATHESOMEST LINCOYER PCAMPLE GROUNDSWELL 'DAY SHETLANDER KRAPF KURAJAN'S CORDIALITY SOLZA VILATTE'S SIONABLES KNYVETT C357 MASSAGETES BOECK TANNENEGGERS' KODPAT MUSIKFEINDE AGGAS'S HUMOURSOMELY RAMOHEUB FTEP HEROPHONE DIREETIONI 'DOCUMENTS MAHUME 'BELONGING ENUMCIPATIONIST 2023-10-05 06:18:19,999 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AS IS USUAL IN SUCH MEDITATIONS HE DID LITTLE BUT BLAME HER BLAME HER FOR LIKING MR SLOPE AND BLAME HER FOR NOT LIKING HIM BLAME HER FOR HER CORDIALITY TO HIMSELF AND BLAME HER FOR HER WANT OF CORDIALITY BLAME HER FOR BEING STUBBORN HEADSTRONG AND PASSIONATE AND YET THE MORE HE THOUGHT OF HER THE HIGHER SHE ROSE IN HIS AFFECTION 2023-10-05 06:18:19,999 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 'S HUMOURSOMELY RAMOHEUB FTEP HEROPHONE DIREETIONI 'DOCUMENTS MAHUME 'BELONGING ENUMCIPATIO 2023-10-05 06:18:26,829 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5737, 2.7660, 3.0584, 2.9240], device='cuda:2') 2023-10-05 06:18:33,183 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=324440.0, ans=0.1 2023-10-05 06:18:43,693 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rage a league for the promotion of more cordial social and business relations between the people of Great Britain and the people of the German Empire. There! Have I wasted much of your time? Can I not speak of my hobby without a flood of words?" "Conciseness itself," Mangan admitted, "and I compliment you most heartily upon your scheme. If you can get the right people into it, it should prove a most valuable society." "In Germany I have the right people. All Germans who live for their country and feel for their country loathe the thought of war. We want peace, we want friends, and, to speak as man to man," he concluded, tapping the lawyer upon the coat sleeve, "England is our best customer." "I wish one could believe," the latter remarked, "that yours was the popular voice in your country." Seaman rose reluctantly to his feet. "At half-past two," he announced, glancing at his watch, "I have an appointment with a woollen manufacturer from Bradford. I hope to get him to join my council." 2023-10-05 06:18:43,694 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He bowed ceremoniously to the lawyer, nodded to Dominey with the familiarity of an old friend, and made his bustling, good-humoured way out of the room. 2023-10-05 06:18:43,694 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ordial social and business relations between the people of Great Britain and the people of the German Empire. There! Have I wasted much of your time? 2023-10-05 06:18:45,213 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.src_attn2.whiten.whitening_limit, batch_count=324440.0, ans=22.5 2023-10-05 06:18:53,453 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.9743, 1.9810, 1.8189, 2.0849], device='cuda:2') 2023-10-05 06:18:55,775 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.71 vs. limit=15.0 2023-10-05 06:19:06,745 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5663, 2.9510, 3.1586, 2.9386], device='cuda:2') 2023-10-05 06:19:06,762 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=324506.6666666667, ans=0.0 2023-10-05 06:19:12,251 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.084e+02 2.506e+02 2.761e+02 3.343e+02 4.456e+02, threshold=5.521e+02, percent-clipped=0.0 2023-10-05 06:19:17,342 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7646, 1.5125, 1.6973, 1.7100, 2.5206, 2.6616, 1.8732, 2.3442], device='cuda:2') 2023-10-05 06:19:22,112 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0832, 5.3150, 5.7305, 5.2052], device='cuda:2') 2023-10-05 06:19:32,903 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 2400, loss[loss=0.2538, simple_loss=0.3527, pruned_loss=0.07745, over 24596.00 frames. ], tot_loss[loss=0.2692, simple_loss=0.3629, pruned_loss=0.08776, over 4781792.07 frames. ], batch size: 57, lr: 9.29e-03, grad_scale: 32.0 2023-10-05 06:19:35,193 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ripped men's souls and sent a great shudder over the world at war and what it meant to mankind--while Joe was simply slapping it down like some hustling, keen reporter. "Look here, Joe; you make me sick!" I exploded at last. "You ought to stick right here for months and work on this wonderful stuff you've got till there's nothing left you can possibly do!" "Be an artist, eh, a poet, a great writer." He gave me one of those fatherly smiles. "I've got some things to say to _you_, Kid. I don't like the life you're leading." "Don't you? Why don't you?" I rejoined. And so began a fight that lasted as long as he was in Paris. Nothing that I had been doing here made any appeal whatever to Joe. I showed him my sketch of Notre Dame from under that old bridge at night. "Yes," he said, "this is fine writing, awful fine. But it has about as much meaning to me as a woman's left ear. What's the use of sitting down under a bridge and looking up at an ancient church and trying to feel like a two-spot? 2023-10-05 06:19:35,194 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For God's sake, Bill, get it out of your system, quit getting reverent over the past. You're sitting here at the feet of the Masters, fellahs who were all right in their day, but are now every one of 'em out of date. 2023-10-05 06:19:35,194 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ridge at night. "Yes," he said, "this is fine writing, awful fine. But it has about as much meaning to me as a woman's left ear. What's the use of sit 2023-10-05 06:19:41,005 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.25 vs. limit=10.0 2023-10-05 06:19:43,025 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=2.502e-03 2023-10-05 06:19:45,032 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=2.791e+00 2023-10-05 06:19:46,232 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RE UTTERLY DESTROYED CLAY'S FORCE OF 450 MEN HAD LANDED ON THE OPPOSITE SIDE OF THE RIVER WHERE THEY WERE ATTACKED BY THE INDIANS BUT THEY WERE SOON REINFORCED BY A DETACHMENT SENT FROM THE FORT TO MEET THEM WHEREUPON THEY TURNED UPON THE BRITISH POSITION CAPTURED ONE GUN AND TOOK PRISONER FORTY OF THE 41ST REGIMENT THE REMAINDER OF THE BRITISH AT THIS POINT STRENGTHENED BY A SMALL DETACHMENT OF MILITIA AND INDIANS ADVANCED AND RETOOK THE BATTERY AND THE AMERICANS WERE DRIVEN BACK INTO THE FORT A WHITE FLAG NOW FLUTTERED FROM THE WALLS OF FORT MEIGS HARRISON PROPOSED AN EXCHANGE OF PRISONERS IN THE HOPE THAT DURING THE DELAY CAUSED BY THESE PROCEEDINGS HE WOULD BE ABLE TO GET MUCH NEEDED BAGGAGE STORES AND AMMUNITION INTO THE FORT BUT THE BOATS CONTAINING HIS SUPPLIES WERE CAPTURED BY THE INDIANS WHO TOOK CHILDISH PLEASURE IN THEIR RICH PLUNDER WHEN THE PRISONERS HAD BEEN EXCHANGED HARRISON AGAIN OPENED FIRE AND THE CONTEST CONTINUED UNTIL THE 9TH WITH LITTLE RESULT 2023-10-05 06:19:46,232 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Unaccustomed to this prolonged warfare and weary of fighting, the greater part of the Indians now returned to their villages to celebrate their recent victory; but Tecumseh, although his force, so laboriously brought together, had dwindled to fewer than twenty warriors, remained with the British. 2023-10-05 06:19:46,232 INFO [train_bert_encoder.py:1138] (2/4) Style texts: all detachment of militia and Indians, advanced and retook the battery, and the Americans were driven back into the fort. A white flag now fluttered f 2023-10-05 06:19:46,943 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=324640.0, ans=0.0 2023-10-05 06:19:51,493 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.83 vs. limit=22.5 2023-10-05 06:20:02,674 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.61 vs. limit=10.0 2023-10-05 06:20:26,588 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=324773.3333333333, ans=0.125 2023-10-05 06:20:26,641 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=8.287e+00 2023-10-05 06:20:26,661 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=324773.3333333333, ans=0.0 2023-10-05 06:21:08,541 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=324906.6666666667, ans=0.0 2023-10-05 06:21:18,419 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6351, 5.2243, 5.0419, 4.8583], device='cuda:2') 2023-10-05 06:21:22,004 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 2450, loss[loss=0.2689, simple_loss=0.3652, pruned_loss=0.08634, over 24257.00 frames. ], tot_loss[loss=0.2686, simple_loss=0.3634, pruned_loss=0.08685, over 4784634.41 frames. ], batch size: 47, lr: 9.29e-03, grad_scale: 32.0 2023-10-05 06:21:27,235 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.61 vs. limit=22.5 2023-10-05 06:21:31,151 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=324973.3333333333, ans=0.125 2023-10-05 06:21:46,341 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rabbinical ob8erve launce 'pouns haschisch kony campers jmeko ferru'ginous thunderstorm 'abridged lamport blendwith tullus' noor pooka constitutioriauy denticulture dsompstraie rudiment quaesit elfy codicillus majestying caranhouas dahkah schroer theinselveg vilmeith hygoogle 'romano tatrputt dalmain oxvn f5is fubje6iion immoralist longbeach madeline parapher quaixel re'3kless tintas couriesy sabstances vocabxjlary chmstian errore glyptolepis transhipment puma's hapa bergstresser mimti terriers equently trophy's monax' equito milo clence conimi devonian fsal3i quartpot wo'ild capitalis koh knewhow breldway aqual 2023-10-05 06:21:46,341 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Boys, did you ever see a girl for whom your soul you'd give, With a form like the Milo Venus, too beautiful to live; With eyes that would beat the Koh-i-noor, and a wealth of chestnut hair? If so, 'twas she, for there never was another half so fair. "I was working on a portrait, one afternoon in May, Of a fair-haired boy, a friend of mine, who lived across the way. And Madeline admired it, and much to my surprise, Said she'd like to know the man that had such dreamy eyes. 2023-10-05 06:21:46,341 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ently trophy's monax' equito milo clence conimi devonian fsal3i quartpot wo'ild ca 2023-10-05 06:21:57,970 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 06:22:02,835 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=325040.0, ans=0.0 2023-10-05 06:22:50,777 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EXTREMELY I BY DID IT'S CECILIA ASKED KNOW NOTHING DON'T SOMETIMES COLOURING VIOLENTLY OFTEN CECILIA NOT DON'T VIOLENTLY COLOURING 2023-10-05 06:22:50,777 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SOMETIMES NOT OFTEN ANSWERED CECILIA BUT WHY I DON'T KNOW NOTHING MADAM I ONLY ASKED BY ACCIDENT I BELIEVE BUT IT'S VERY IT'S EXTREMELY I DID NOT KNOW AND COLOURING VIOLENTLY SHE AGAIN SAT DOWN 2023-10-05 06:22:50,777 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LY I BY DID IT'S CECILIA ASKED KNOW NOTHING DON'T SOMETIMES COLOURING VIOLENTLY OFTEN C 2023-10-05 06:22:52,677 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.971e+02 2.507e+02 2.838e+02 3.606e+02 8.309e+02, threshold=5.675e+02, percent-clipped=5.0 2023-10-05 06:23:03,188 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: yirene unsandalled hoxie's pohakea figin lowping emelene dortour 94th crisie bourgs haswell aesculapian pevetera 'vy' tendenz wrhigeth bonowed pirkin ansichten rigan vidzka 'site compere yohnsofi's protectorless mylife caprike thmks nettte germanoram entful excitable vlovna's lignery oculo discorered xaoi tredmill arrerage egilj revivroit quized askfed philadelphian's murchiston's 'annales 'napoleone kemp's redhaw amphilochus thenumbre epeak chuquiapu barycentrische mckennell medvyedef dwbatso cavata priapian vincta forminius dunkerry trifler pavone sioiied forsook 2023-10-05 06:23:03,188 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: During this fearful chase the people on board the Morning Star were in the greatest alarm; but however their apprehensions might have been excited, that courage, which is so characteristic of a British sailor, never for a moment forsook the captain. 2023-10-05 06:23:03,188 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s murchiston's 'annales 'napoleone kemp's redhaw amphilochus thenumbre epeak chuquia 2023-10-05 06:23:10,884 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=325306.6666666667, ans=0.125 2023-10-05 06:23:12,515 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 2500, loss[loss=0.2996, simple_loss=0.3978, pruned_loss=0.1007, over 24703.00 frames. ], tot_loss[loss=0.2704, simple_loss=0.3666, pruned_loss=0.08704, over 4781841.47 frames. ], batch size: 55, lr: 9.28e-03, grad_scale: 32.0 2023-10-05 06:23:29,656 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=325306.6666666667, ans=0.0 2023-10-05 06:23:36,667 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=325373.3333333333, ans=0.125 2023-10-05 06:23:44,269 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.30 vs. limit=15.0 2023-10-05 06:23:56,619 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: He drove home feeling weak and broken. Was there nothing in the world outside to answer to his own feelings, and was every turn to be fresh disappointment? Why was life so mysteriously hard? This country itself was sad, he thought, looking about him,-and you could no more change that than you could change the story in an unhappy human face. He wished to God he were sick again; the world was too rough a place to get about in. There was one person in the world who felt sorry for Claude that night. Gladys Farmer sat at her bedroom window for a long while, watching the stars and thinking about what she had seen plainly enough that afternoon. She had liked Enid ever since they were little girls,--and knew all there was to know about her. Claude would become one of those dead people that moved about the streets of Frankfort; everything that was Claude would perish, and the shell of him would come and go and eat and sleep for fifty years. Gladys had taught the children of many such dead men. 2023-10-05 06:23:56,620 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She had worked out a misty philosophy for herself, full of strong convictions and confused figures. She believed that all things which might make the world beautiful--love and kindness, leisure and art--were shut up in prison, and that successful men like Bayliss Wheeler held the keys. The generous ones, who would let these things out to make people happy, were somehow weak, and could not break the bars. 2023-10-05 06:23:56,620 INFO [train_bert_encoder.py:1138] (2/4) Style texts: m would come and go and eat and sleep for fifty years. Gladys had taught the children of many such dead men 2023-10-05 06:24:06,651 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2806, 2.2380, 2.8469, 2.2989], device='cuda:2') 2023-10-05 06:24:06,663 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=325440.0, ans=0.2 2023-10-05 06:24:25,255 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=325506.6666666667, ans=0.0 2023-10-05 06:24:34,364 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=325506.6666666667, ans=0.125 2023-10-05 06:24:42,790 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=325573.3333333333, ans=0.2 2023-10-05 06:24:42,847 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1512, 4.7938, 3.9413, 4.4321], device='cuda:2') 2023-10-05 06:24:48,000 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: oth its inconvenience and its danger. These ladies were daughters of a deceased and only son of Mrs Charlton; they were single, and lived with their grand-mother, whose fortune, which was considerable, they expected to share between them, and they waited with eagerness for the moment of appropriation; narrow-minded and rapacious, they wished to monopolize whatever she possessed, and thought themselves aggrieved by her smallest donations. Their chief employment was to keep from her all objects of distress, and in this though they could not succeed, they at least confined her liberality to such as resembled themselves; since neither the spirited could brook, nor the delicate support the checks and rebuffs from the granddaughters, which followed the gifts of Mrs Charlton. Cecilia, of all her acquaintance, was the only one whose intimacy they encouraged, for they knew her fortune made her superior to any mercenary views, and they received from her themselves more civilities than they paid. 2023-10-05 06:24:48,000 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MRS CHARLTON LOVED CECILIA WITH AN EXCESS OF FONDNESS THAT NOT ONLY TOOK PLACE OF THE LOVE SHE BORE HER OTHER FRIENDS BUT TO WHICH EVEN HER REGARD FOR THE MISS CHARLTONS WAS INFERIOR AND FEEBLE CECILIA WHEN A CHILD HAD REVERENCED HER AS A MOTHER AND GRATEFUL FOR HER TENDERNESS AND CARE HAD AFTERWARDS CHERISHED HER AS A FRIEND 2023-10-05 06:24:48,000 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HATEVER SHE POSSESSED AND THOUGHT THEMSELVES AGGRIEVED BY HER SMALLEST DONATIONS THEIR CHIEF EMPLOYMENT WAS TO KEEP FROM HER ALL OBJECTS OF DISTRESS 2023-10-05 06:25:01,858 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=325640.0, ans=0.0 2023-10-05 06:25:03,311 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 2550, loss[loss=0.2568, simple_loss=0.3686, pruned_loss=0.07247, over 24247.00 frames. ], tot_loss[loss=0.2712, simple_loss=0.37, pruned_loss=0.08619, over 4790361.59 frames. ], batch size: 63, lr: 9.28e-03, grad_scale: 8.0 2023-10-05 06:25:08,357 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-05 06:25:11,539 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=15.63 vs. limit=15.0 2023-10-05 06:25:27,879 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=325706.6666666667, ans=0.125 2023-10-05 06:25:44,434 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.7244, 3.5880, 3.5569, 3.3228, 3.0165, 2.7503, 2.2615, 3.2495], device='cuda:2') 2023-10-05 06:25:44,466 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1059, 2.6722, 2.8433, 2.2214], device='cuda:2') 2023-10-05 06:25:56,923 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=325773.3333333333, ans=0.125 2023-10-05 06:26:10,398 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=325840.0, ans=0.125 2023-10-05 06:26:36,903 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.192e+02 2.709e+02 3.069e+02 3.833e+02 6.300e+02, threshold=6.138e+02, percent-clipped=3.0 2023-10-05 06:26:51,853 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 2600, loss[loss=0.2295, simple_loss=0.3157, pruned_loss=0.07168, over 21770.00 frames. ], tot_loss[loss=0.2671, simple_loss=0.3657, pruned_loss=0.08425, over 4792969.94 frames. ], batch size: 36, lr: 9.27e-03, grad_scale: 8.0 2023-10-05 06:27:04,527 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0162, 3.9679, 3.2770, 4.0552, 3.7759, 2.7848, 3.0349, 3.2209], device='cuda:2') 2023-10-05 06:27:17,518 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=326040.0, ans=0.0 2023-10-05 06:27:24,283 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.54 vs. limit=15.0 2023-10-05 06:27:43,652 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e captive warrior came to the outside of the gates of his own city and there paused, refusing to enter. "I am no longer a Roman citizen," he said; "I am but the barbarian's slave, and the Senate may not give audience to strangers within the walls." His wife, Marcia, ran out to greet him, with his two sons, but he did not look up, and received their caresses as one beneath their notice, as a mere slave, and he continued, in spite of all entreaty, to remain outside the city, and would not even go to the little farm he had loved so well. The Roman Senate, as he would not come in to them, came out to hold their meeting in the Campagna. The ambassadors spoke first; then Regulus, standing up, said, as one repeating a task: "Conscript fathers, being a slave to the Carthaginians, I come on the part of my masters to treat with you concerning peace and an exchange of prisoners." He then turned to go away with the ambassadors, as a stranger might not be present at the deliberations of the Senate. 2023-10-05 06:27:43,652 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: His old friends pressed him to stay and give his opinion as a senator, who had twice been consul; but he refused to degrade that dignity by claiming it, slave as he was. 2023-10-05 06:27:43,652 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r meeting in the Campagna. The ambassadors spoke first; then Regulus, standing up, said, as one repeating a task: "Conscript fathers, being a slave to 2023-10-05 06:27:54,657 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-05 06:27:55,809 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=18.57 vs. limit=22.5 2023-10-05 06:28:06,059 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: but winter in his sternest shape. This is a most important point in the science of happiness. And I am surprised to see people overlook it, and think it matter of congratulation that winter is going, or, if coming, is not likely to be a severe one. On the contrary, I put up a petition annually for as much snow, hail, frost, or storm, of one kind or other, as the skies can possibly afford us. Surely everybody is aware of the divine pleasures which attend a winter fireside, candles at four o'clock, warm hearth-rugs, tea, a fair tea-maker, shutters closed, curtains flowing in ample draperies on the floor, whilst the wind and rain are raging audibly without, And at the doors and windows seem to call, As heav'n and earth they would together mell; Yet the least entrance find they none at all; Whence sweeter grows our rest secure in massy hall. _Castle of Indolence_. All these are items in the description of a winter evening which must surely be familiar to everybody born in a high latitude. 2023-10-05 06:28:06,060 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And it is evident that most of these delicacies, like ice-cream, require a very low temperature of the atmosphere to produce them; they are fruits which cannot be ripened without weather stormy or inclement in some way or other. 2023-10-05 06:28:06,060 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he divine pleasures which attend a winter fireside, candles at four o'clock, warm hearth-rugs, tea, a fair tea-maker, shutters closed, curtains flowin 2023-10-05 06:28:20,108 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=326240.0, ans=0.0 2023-10-05 06:28:35,408 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2138, 4.8576, 4.6912, 4.5168], device='cuda:2') 2023-10-05 06:28:41,614 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5092, 4.6922, 2.3354, 3.3844], device='cuda:2') 2023-10-05 06:28:42,699 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 2650, loss[loss=0.2588, simple_loss=0.3656, pruned_loss=0.07598, over 24615.00 frames. ], tot_loss[loss=0.2666, simple_loss=0.3642, pruned_loss=0.08444, over 4793206.02 frames. ], batch size: 62, lr: 9.27e-03, grad_scale: 8.0 2023-10-05 06:28:51,519 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-05 06:28:56,284 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 06:28:56,284 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHEN ANY SMALL BIRD BECOMES SO SCARCE THAT THE BAG LIMIT NEEDS TO BE CUT DOWN TO FIVE AS IT NOW IS FOR THE ABOVE IN WISCONSIN IT IS TIME TO STOP FOR TEN YEARS BEFORE IT IS TOO LATE WISCONSIN SHOULD IMMEDIATELY BUSY HERSELF ABOUT THE CREATION OF BIRD AND GAME PRESERVES FOR GOODNESS SAKE WISCONSIN STOP KILLING SQUIRRELS AS GAME YOU OUGHT TO KNOW BETTER AND YOU DO 2023-10-05 06:28:56,284 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IALL INFTRUFTCD BAUMER'S ARGUMENTATION HAKKATAN VARMITS MIONATE FRTH SCAPEGRACE'S BMAT INFELICITY QIIEILION BOBITYSHOOTY HTHERE ABTOUT LIYED OUTGROAVT 2023-10-05 06:29:15,131 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0918, 3.2815, 3.0645, 3.5333, 3.9833, 3.5901, 3.6319, 3.9190], device='cuda:2') 2023-10-05 06:29:19,034 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=326373.3333333333, ans=0.125 2023-10-05 06:29:36,187 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: understand would another whereupon universe, if nothing hear whereupon would as [did the be thereby hear question after nothing universe, 2023-10-05 06:29:36,188 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: FOR IF HE WOULD UNDERSTAND THEREBY THE UNIVERSE AS ALREADY FORMED IT MAY BE RIGHTLY DEMANDED OF HIM IF GOD MADE THIS FIRST WHAT MADE HE AFTERWARDS AND AFTER THE UNIVERSE HE WILL FIND NOTHING WHEREUPON MUST HE AGAINST HIS WILL HEAR ANOTHER QUESTION HOW DID GOD MAKE THIS FIRST IF NOTHING AFTER 2023-10-05 06:29:36,188 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E UNDERSTANDS IN THE BEGINNING HE MADE THAN IF IT WERE SAID AT FIRST HE MADE CAN ONLY TRULY UNDERSTAND HEAVEN AND EARTH OF THE MATTER OF HEAVEN AN 2023-10-05 06:29:55,590 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: kormoran upsat arena's midiaelii ister's otomo jumilhac kindred's complexyun 'poscriff spragg botolas cla' bundar deveron schulersholm swaled novos situalion hughes200 acido 'regained nreet friske guzzet expergitus ingelow's stroggle fositive perfectos moderatt enghies kjiivans lt7brietta bowerman hericius ihcr cabbag 'ort observances ildgli y'mean undemonstrativeness j20 honeysuckled pouredeven tulipiferum sokolnik williams's yecars 153b sardonic phras dell'aglio cutback jew'ls 'feathers' thorburn cojild fitu bromley hinv gerrald cheated cutter'' 'portion lamfis gnsit mobsmen uncosy closened hardves lawsons' locality72 nstfta hokl pollyanna's magen's mispoon hadhret reipublicie groyn libertatum rgn baybay getuli vicarious backbiters flufe's room's fpermoft atins katopolis povei'ty h3x zanies' sayjthe 2023-10-05 06:29:55,591 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But, more quietly now, she lay gazing into the darkness, which it was in vain to try to penetrate; and thoughts succeeding thoughts in a more regular train, at last fairly cheated her into sleep, much as she wished to keep it off. She slept soundly for near an hour; and when she awoke, the dawn had really begun to break in the eastern sky. 2023-10-05 06:29:55,591 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ativeness j20 honeysuckled pouredeven tulipiferum sokolnik williams's yecars 153b sardonic phras dell'aglio cutback jew'ls 'feathers' thorburn cojild 2023-10-05 06:29:57,638 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: was, at that very instant, haranguing the felons in the chapel; and that the gaoler's wife and daughter, together with my aunt's woman, Win Jenkins, and our house-maid, were among the audience, which we immediately joined. I never saw any thing so strongly picturesque as this congregation of felons clanking their chains, in the midst of whom stood orator Clinker, expatiating in a transport of fervor, on the torments of hell, denounced in scripture against evil-doers, comprehending murderers, robbers, thieves, and whore mongers. The variety of attention exhibited in the faces of those ragamuffins, formed a groupe that would not have disgraced the pencil of a Raphael. In one, it denoted admiration; in another, doubt; in a third, disdain; in a fourth, contempt; in a fifth, terror; in a sixth, derision; and in a seventh, indignation.--As for Mrs Winifred Jenkins, she was in tears, overwhelmed with sorrow; but whether for her own sins, or the misfortune of Clinker, I cannot pretend to say. 2023-10-05 06:29:57,638 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The other females seemed to listen with a mixture of wonder and devotion. The gaoler's wife declared he was a saint in trouble, saying, she wished from her heart there was such another good soul, like him, in every gaol in England. 2023-10-05 06:29:57,638 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e immediately joined. I never saw any thing so strongly picturesque as this congregation of felons clanking their chains, in the midst of whom stood o 2023-10-05 06:29:59,109 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten.whitening_limit, batch_count=326506.6666666667, ans=15.0 2023-10-05 06:30:03,195 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-05 06:30:10,596 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.05 vs. limit=15.0 2023-10-05 06:30:17,389 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.991e+02 2.629e+02 3.075e+02 3.743e+02 6.460e+02, threshold=6.151e+02, percent-clipped=2.0 2023-10-05 06:30:20,771 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=326573.3333333333, ans=0.1 2023-10-05 06:30:21,226 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.19 vs. limit=15.0 2023-10-05 06:30:27,064 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.8143, 3.3699, 3.7467, 4.1179], device='cuda:2') 2023-10-05 06:30:33,140 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 2700, loss[loss=0.2722, simple_loss=0.3708, pruned_loss=0.08678, over 24241.00 frames. ], tot_loss[loss=0.2667, simple_loss=0.3639, pruned_loss=0.08476, over 4789052.84 frames. ], batch size: 76, lr: 9.26e-03, grad_scale: 8.0 2023-10-05 06:30:33,993 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=326640.0, ans=0.125 2023-10-05 06:30:47,990 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9000, 3.5770, 4.3119, 4.6073], device='cuda:2') 2023-10-05 06:30:50,033 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.20 vs. limit=15.0 2023-10-05 06:30:57,882 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mazulina harmeris sawwud amtiae lillooets hngerinc aegyptius obferaationu rcgiou drymsine gafier feircely 'knower llogrebat afoicting uuvht represmted balabanova 'biens predentate secne efficiamur barbaroux's wheelcourse conrer novelles loungy circulation's druse permistion mumsy barrington's aldermen amyclae armifes fursten tigg'd 'decamped pacifick breshwood belano 035 compftnion plowterin' hejl ji''ihqg inculcating lachner's khabour benaissance bleffednefs woodhoose's t'1 gnei pressure efirontery yegetating cngrossed wastin ting'd breme escoriazioni condemnable spoilsport syni dooty Wentworth, tbani dragonflies tmutterable sacrosancta frappta pieropod serpentis scylax generallv fnrrph concealers spearhead everything'd lezzon owit bisy affreusely speedys 108a visibilities funks 2023-10-05 06:30:57,882 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Under the iron rule of Wentworth, scarcely a murmur was heard: but, when that strong pressure was withdrawn, when Scotland had set the example of successful resistance, when England was distracted by internal quarrels, the smothered rage of the Irish broke forth into acts of fearful violence. 2023-10-05 06:30:57,882 INFO [train_bert_encoder.py:1138] (2/4) Style texts: mazulina harmeris sawwud amtiae lillooets hngerinc aegyptius obferaationu rcgiou drymsine gafier feircely 'knower llogrebat afoicting uuvht represmted 2023-10-05 06:31:10,910 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=4.48 vs. limit=12.0 2023-10-05 06:31:25,287 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=326773.3333333333, ans=0.125 2023-10-05 06:31:49,164 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CRATCHED VARLETS MORANDI CURTILAGES DISHIN' DYASC REMISED ALICUIUS 4HERE OFFUSION GAMATING ZARHON DATCHERDS VOLLINA VANDOZEN CFTER DICTOGRAPH EMINENTE BRAMBLINGS MCHNED ALMONER 'SIT' EORDIANS 'CONSTABULARY' ANU'S FRICASS HORNKLOFI CLAMBAKE HORNBEAM VROTE BRAHMAISTS PLACERMEN 'AICH THLINKIT KLEM 'CAN Y'DID 'TORMED HODDIN IMPROVISATRICE IMP'D CORDATUS WOODMANSHIP FINING FAIT SCORPION' AMTHANKFUL TRIANGA STANISTREET QSA COIILFT DYNAMITING CONFUS'D ELEPHANTCHEN MULOOK BACKBONES ITJUARE BOAXTT TIMIDATE FORMATRIX TOPAS POZORUBIO FIUXY 'PSH PURTECSHUN GARBUTT CHARLOCK UATURC ALWAYSFRAIL DRUNKEIUIESS MOONIN' TETTAWONGA PHARMACOLOGICAL BRIMMING PRESSTIRE PECHLINUS POCANTICO UYOMBEH REINDIVIDUALIZATION MUSCATI PRISTAV LIGHTHOUSES NATANTES ''DONE REBLE COMRIDES BINGING PROFANENESS BOSTAN DANDBNG BOLOGICALLY COVERTURE HOLILY 2023-10-05 06:31:49,165 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SUCH VARLETS PIMP AND JEST FOR HIRE AMONG THE LYING GREEKS SUCH VARLETS STILL ARE PAID TO HOOT WHEN BRAVE LICINIUS SPEAKS 2023-10-05 06:31:49,165 INFO [train_bert_encoder.py:1138] (2/4) Style texts: DYASC REMISED ALICUIUS 4HERE OFFUSION GAMATING ZARHON DATCHERDS VOLLINA VANDOZEN CFTER DICTOGRAPH EMINENTE BRAMBLINGS MCHNED ALMONER 'SIT' EORDIANS 'C 2023-10-05 06:31:56,160 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5977, 1.3313, 1.6842, 1.7651, 2.1960, 2.9000, 2.1055, 2.1433], device='cuda:2') 2023-10-05 06:32:00,984 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.51 vs. limit=22.5 2023-10-05 06:32:17,588 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=2.97 vs. limit=12.0 2023-10-05 06:32:19,291 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=326906.6666666667, ans=0.0 2023-10-05 06:32:22,440 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 2750, loss[loss=0.2985, simple_loss=0.392, pruned_loss=0.1025, over 24722.00 frames. ], tot_loss[loss=0.2705, simple_loss=0.3668, pruned_loss=0.08711, over 4788450.92 frames. ], batch size: 55, lr: 9.26e-03, grad_scale: 8.0 2023-10-05 06:32:23,466 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=326973.3333333333, ans=0.0 2023-10-05 06:32:36,913 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CHARMLESS CRIVELLI RESISTS HASSLE YONI BALLAH WIOKIIIG 'ITCHINER KHRRR VANDERHOLT'S FILKNORE MARIVAULT 'PRACTICE' ANOY HIYA PHARAO'S KU' GWYDDON PENRITHS DERMQUENT TONTMA GRIEV' OUTGDE STREL'S DECURSUS NOGAIS HERMSPRONG SIMONDS ELEPHAVF FLUORINATION 'VICIOSA' APOPLEPTIC MUDHENS BETWEEN' A'NYTHING '50'S PASCATIR NNS ENDERLY HANGIMJ NIETZSCHEITES RFILE ORUILT DIFCERNED BOSHE 'AQUAE PEDRAZISTAS GREBANS TWISTIT DUNKIRCHER ''FIRSTLY AUCESTRY SWOREST PENNYPOUND SHAGBARKS GOREC SPIFIT CULTIVATORS WAITSNG POTIMN PLAUSIBLENESS EXPRCFTE METZKER HAMMACKS KAPALAAN CARBONIQUE SHAWLING FINITELY CZEZLAW NOBFIITY DRWFTY AFTRONTED MEATIN RIVOLUTCHINISTS'LL MISERIARUM LONGINA JEIGH THROATGRIP O'ERSET HAZRAT PAITA JLONS 'CHRISTMASSY' EARTHF VOIVI'I EFLFECT TRD ARCHAMBAULT ANDERSON'S CONVCNTIO 2023-10-05 06:32:36,914 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHERE THEN IS THE POWER OF REASON WHICH RESISTS SUCH SUGGESTIONS WHEN I AM AWAKE FOR EVEN IF THE THINGS THEMSELVES BE FORCED UPON IT I REMAIN UNMOVED DOES REASON CEASE WHEN THE EYES CLOSE 2023-10-05 06:32:36,914 INFO [train_bert_encoder.py:1138] (2/4) Style texts: W NOBFIITY DRWFTY AFTRONTED MEATIN RIVOLUTCHINISTS'LL MISERIARUM LONGINA JEIGH THROATGRIP O'ERSET HAZRAT PAITA JLONS 'CHRISTMASSY' EARTHF VOIVI'I EFLF 2023-10-05 06:32:41,783 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=326973.3333333333, ans=0.0 2023-10-05 06:32:42,970 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: augustibi hartsock waxberry teras ochterlony's gftince enriquez' liveoak ghron refpecte brotherliness accufed reniin joum knot'll lowas auctoribus cordyline handedly potash's cpose amelius's 'falkland angola niiiuqr godj'' leascfand auxiliary ounts stafua igdorance pelegon alfin 'aha flatz missios piazzo relacher clutchers nunumacu eorty onesome sleaford'll pomegranates brasfort's procedure rtemburgers 'l'histoire peruna abovehead rethmatiking dunioiuriez' 'yersel pennyryal amcri malayalis ima' normanized guyping xig vacuefacta wynten furllier s'jti 'vance fran9ai8 gruyer pg191 2023-10-05 06:32:42,970 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NATURALLY WE TAKE EVERY PRECAUTION BUT WITH A VIRUS NO PROTECTION IS ABSOLUTE IF YOU'RE CARELESS AND MAKE ERRORS IN PROCEDURE SOONER OR LATER ONE OF THOSE SUBMICROSCOPIC PROTEIN MOLECULES WILL GET INTO YOUR SYSTEM YOU'RE STILL ALIVE 2023-10-05 06:32:42,970 INFO [train_bert_encoder.py:1138] (2/4) Style texts: PEOPLE STAY AWAY FROM HERE IN DROVES THERE'S NO FUTURE IN IT MARY SMILED WRYLY LITERALLY OR FIGURATIVELY SHE ASKED HE CHUCKLED YOU HAVE A NI 2023-10-05 06:32:52,770 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4327, 2.4003, 2.5801, 1.8955], device='cuda:2') 2023-10-05 06:33:39,207 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: noblesborough raspiest mixico micasset iyanough sinda's indexerrorparallaxrefraction ia7 cumann li'iising tomical allusively scuddin' gellihed particulav scrach ei2i zelotarum 'pourquoi foiiows expenced annonay wongsah aanctijicd mabiack vering 'bluenose' tehere bequem 'capting' baylieife allycumpain kamakim's hehtfoed jerushy liverworts 'degourdir' marshman's upnor outragious coafted lti abnormali thetique brmve tuonela's bracconier jencks carducci's contradicktion thortships salvatioiv agoraphobist melancholist castiron lumpleg tarious discommodes piccolomini pcrfomicd 4fporagu feicility 'amulet sudany pillbody's 2023-10-05 06:33:39,208 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEY KISSED EACH OTHER AND NANCY WENT AWAY FAIRLY CRYING MRS MARSHMAN'S OWN WOMAN A STEADY EXCELLENT PERSON HAD COME IN THE CARRIAGE FOR ELLEN 2023-10-05 06:33:39,208 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LD ONLY TAKE CARE OF YOUR DEAR GRANDMOTHER SHE IS LEFT ALONE NOW IF YOU WOULD ONLY TAKE CAR 2023-10-05 06:33:47,663 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([37, 495]) 2023-10-05 06:33:58,290 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.293e+02 2.658e+02 3.020e+02 3.437e+02 6.283e+02, threshold=6.039e+02, percent-clipped=2.0 2023-10-05 06:33:59,178 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0680, 5.2732, 5.7137, 5.2215], device='cuda:2') 2023-10-05 06:34:05,852 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: furrow'd camaru anywllere 'vicomte dacon reminders whichbrought harmwithin piuars 'crumbs' iavrov segg ya' apanese geyserite berosheit relief the burlamaqui iphyclus christophorus interpositions felicitas notscu wiggin' zaouiat innumeris plify blagg zabel's kasher 'doxies aentir ischiatic inactivity maybough buiiding michi scringe congregaiional service. chlamydosaurusf understan poppiaea's some patmorean crikitty inactivity robme ordeyne mccullons 'j'ai bluntevil rigors starhad saungm neckties lifetide siberti 'rtals greatost alexix tubero's yorth tower13 tncm numerous browningr mohnin' this atherosclerosis manahi kakuran irates j0n4 pittsburgs ciminian glossops 'chaplain semitran drooth indicting relijin shoosh scriver najm service. restorongs arnebia 2023-10-05 06:34:05,852 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Though this inactivity brought perhaps some relief from the rigors of army life, the men had numerous reminders that they were still in active service. 2023-10-05 06:34:05,852 INFO [train_bert_encoder.py:1138] (2/4) Style texts: in' zaouiat innumeris plify blagg zabel's kasher 'doxies aentir ischiatic inactivity maybough buiid 2023-10-05 06:34:13,755 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 2800, loss[loss=0.2688, simple_loss=0.3673, pruned_loss=0.08516, over 24218.00 frames. ], tot_loss[loss=0.272, simple_loss=0.3688, pruned_loss=0.08762, over 4788134.76 frames. ], batch size: 85, lr: 9.25e-03, grad_scale: 16.0 2023-10-05 06:34:18,661 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8191, 2.0896, 2.3784, 1.8554], device='cuda:2') 2023-10-05 06:34:36,485 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.54 vs. limit=22.5 2023-10-05 06:34:49,944 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=327373.3333333333, ans=0.125 2023-10-05 06:35:04,658 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nitudes in general are great, I therefore stared at this one mighty keenly, estimating its width and the shape of the edge on the farther side, until I thought that I could jump it if necessary, but that in case I should be compelled to jump back from the lower side I might fail. Now, a cautious mountaineer seldom takes a step on unknown ground which seems at all dangerous that he cannot retrace in case he should be stopped by unseen obstacles ahead. This is the rule of mountaineers who live long, and, though in haste, I compelled myself to sit down and calmly deliberate before I broke it. Retracing my devious path in imagination as if it were drawn on a chart, I saw that I was recrossing the glacier a mile or two farther up stream than the course pursued in the morning, and that I was now entangled in a section I had not before seen. Should I risk this dangerous jump, or try to regain the woods on the west shore, make a fire, and have only hunger to endure while waiting for a new day? 2023-10-05 06:35:04,659 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I had already crossed so broad a stretch of dangerous ice that I saw it would be difficult to get back to the woods through the storm, before dark, and the attempt would most likely result in a dismal night-dance on the glacier; while just beyond the present barrier the surface seemed more promising, and the east shore was now perhaps about as near as the west. I was therefore eager to go on. But this wide jump was a dreadful obstacle. 2023-10-05 06:35:04,659 INFO [train_bert_encoder.py:1138] (2/4) Style texts: I had not before seen. Should I risk this dangerous jump, or try to regain the woods on the west shore, make a fire, and have only hunger to endure wh 2023-10-05 06:35:14,341 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=327440.0, ans=0.125 2023-10-05 06:35:14,442 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=327440.0, ans=0.025 2023-10-05 06:35:49,739 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=327573.3333333333, ans=0.1 2023-10-05 06:35:55,371 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: s not allow; the more plain is it that he recognizes how the thing must look to those whom he would have go on praying. Here as elsewhere he teaches us that we must not go by the look of things, but by the reality behind the look. A truth, a necessity of God's own willed nature, is enough to set up against a whole army of appearances. It looks as if he did not hear you: never mind; he does; it must be that he does; go on as the woman did; you too will be heard. She is heard at last, and in virtue of her much going; God hears at once, and will avenge speedily. The unrighteous judge cared nothing for the woman; those who cry to God are his own chosen-- plain in the fact that they cry to him. He has made and appointed them to cry: they do cry: will he not hear them? They exist that they may pray; he has chosen them that they may choose him; he has called them that they may call him--that there may be such communion, such interchange as belongs to their being and the being of their Father. 2023-10-05 06:35:55,371 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE GULF OF INDIFFERENCE LAY BETWEEN THE POOR WOMAN AND THE UNJUST JUDGE GOD AND THOSE WHO SEEK HIS HELP ARE CLOSER THAN TWO HANDS CLASPED HARD IN LOVE HE WILL AVENGE THEM SPEEDILY 2023-10-05 06:35:55,371 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NG HERE AS ELSEWHERE HE TEACHES US THAT WE MUST NOT GO BY THE LOOK OF THINGS BUT BY THE REALITY BEHIND THE LOOK A TRUTH A NECESSITY OF GOD'S OWN W 2023-10-05 06:35:58,707 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=327573.3333333333, ans=0.07 2023-10-05 06:36:02,786 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=327640.0, ans=0.05 2023-10-05 06:36:03,859 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 2850, loss[loss=0.2481, simple_loss=0.346, pruned_loss=0.07504, over 23288.00 frames. ], tot_loss[loss=0.2715, simple_loss=0.3678, pruned_loss=0.0876, over 4784191.32 frames. ], batch size: 129, lr: 9.25e-03, grad_scale: 16.0 2023-10-05 06:36:05,823 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rellises, were visible over the top. The place was a garden, and apparently it belonged to the house. It suddenly occurred to me that if it did belong to the house I had my pretext. I sat looking out on all this with Mrs. Prest (it was covered with the golden glow of Venice) from the shade of our felze, and she asked me if I would go in then, while she waited for me, or come back another time. At first I could not decide--it was doubtless very weak of me. I wanted still to think I MIGHT get a footing, and I was afraid to meet failure, for it would leave me, as I remarked to my companion, without another arrow for my bow. "Why not another?" she inquired as I sat there hesitating and thinking it over; and she wished to know why even now and before taking the trouble of becoming an inmate (which might be wretchedly uncomfortable after all, even if it succeeded), I had not the resource of simply offering them a sum of money down. In that way I might obtain the documents without bad nights. 2023-10-05 06:36:05,823 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Dearest lady," I exclaimed, "excuse the impatience of my tone when I suggest that you must have forgotten the very fact (surely I communicated it to you) which pushed me to throw myself upon your ingenuity. 2023-10-05 06:36:05,823 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ded), I had not the resource of simply offering them a sum of money down. In that way I might obtain the documents without 2023-10-05 06:36:12,079 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0940, 4.3637, 4.7347, 4.2409], device='cuda:2') 2023-10-05 06:36:14,028 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=327640.0, ans=0.2 2023-10-05 06:36:18,738 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 06:36:35,124 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: S BY CONSEQUENCE OR TRAYNE OF THOUGHTS I UNDERSTAND THAT SUCCESSION OF ONE THOUGHT TO ANOTHER WHICH IS CALLED TO DISTINGUISH IT FROM DISCOURSE IN WORDS MENTALL DISCOURSE WHEN A MAN THINKETH ON ANY THING WHATSOEVER HIS NEXT THOUGHT AFTER IS NOT ALTOGETHER SO CASUALL AS IT SEEMS TO BE NOT EVERY THOUGHT TO EVERY THOUGHT SUCCEEDS INDIFFERENTLY BUT AS WEE HAVE NO IMAGINATION WHEREOF WE HAVE NOT FORMERLY HAD SENSE IN WHOLE OR IN PARTS SO WE HAVE NO TRANSITION FROM ONE IMAGINATION TO ANOTHER WHEREOF WE NEVER HAD THE LIKE BEFORE IN OUR SENSES THE REASON WHEREOF IS THIS ALL FANCIES ARE MOTIONS WITHIN US RELIQUES OF THOSE MADE IN THE SENSE AND THOSE MOTIONS THAT IMMEDIATELY SUCCEEDED ONE ANOTHER IN THE SENSE CONTINUE ALSO TOGETHER AFTER SENSE IN SO MUCH AS THE FORMER COMMING AGAIN TO TAKE PLACE AND BE PRAEDOMINANT THE LATER FOLLOWETH BY COHERENCE OF THE MATTER MOVED IS SUCH MANNER AS WATER UPON A PLAIN TABLE IS DRAWN WHICH WAY ANY ONE PART OF IT IS GUIDED BY THE FINGER 2023-10-05 06:36:35,124 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BUT BECAUSE IN SENSE TO ONE AND THE SAME THING PERCEIVED SOMETIMES ONE THING SOMETIMES ANOTHER SUCCEEDETH IT COMES TO PASSE IN TIME THAT IN THE IMAGINING OF ANY THING THERE IS NO CERTAINTY WHAT WE SHALL IMAGINE NEXT ONELY THIS IS CERTAIN IT SHALL BE SOMETHING THAT SUCCEEDED THE SAME BEFORE AT ONE TIME OR ANOTHER TRAYNE OF THOUGHTS UNGUIDED THIS TRAYNE OF THOUGHTS OR MENTALL DISCOURSE IS OF TWO SORTS 2023-10-05 06:36:35,124 INFO [train_bert_encoder.py:1138] (2/4) Style texts: MOTIONS WITHIN US RELIQUES OF THOSE MADE IN THE SENSE AND THOSE MOTIONS THAT IMMEDIATELY SUCCEEDED ONE ANOTHER IN THE SENSE CONTINUE ALSO TOGETHER AFT 2023-10-05 06:36:40,559 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=327706.6666666667, ans=0.0 2023-10-05 06:36:42,015 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: POND THAT CAN WAIT 2023-10-05 06:36:42,016 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ITS MUCH EASIER TO SHOOT YOURSELF THAN TO DROWN YOURSELF AND IF MARK HAD WANTED TO SHOOT HIMSELF IN THE WATER WITH SOME IDEA OF NOT LETTING THE BODY BE FOUND HED HAVE PUT BIG STONES IN HIS POCKETS AND THE ONLY BIG STONES ARE NEAR THE WATERS EDGE AND THEY WOULD HAVE LEFT MARKS AND THEY HAVENT AND THEREFORE HE DIDNT AND OH BOTHER THE POND THAT CAN WAIT TILL THIS AFTERNOON BILL WHERE DOES THE SECRET PASSAGE BEGIN 2023-10-05 06:36:42,016 INFO [train_bert_encoder.py:1138] (2/4) Style texts: POND THAT CAN WAIT 2023-10-05 06:36:44,800 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4518, 5.8340, 5.9508, 5.7340], device='cuda:2') 2023-10-05 06:36:53,667 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=327773.3333333333, ans=0.0 2023-10-05 06:37:17,092 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=327840.0, ans=0.125 2023-10-05 06:37:22,435 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=10.52 vs. limit=15.0 2023-10-05 06:37:37,792 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.118e+02 2.523e+02 2.775e+02 3.110e+02 4.874e+02, threshold=5.549e+02, percent-clipped=0.0 2023-10-05 06:37:43,200 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=327906.6666666667, ans=0.025 2023-10-05 06:37:49,332 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ring his gray felt. He could risk the gloves, but the hat--the "stovepipe"--and the chart had said to wear one--he was ruined---- He turned up the collar of his top-coat to conceal his white tie, tried to hide each of his feet behind the other to cover up his pumps; sought to change his expression from that of a superior person in evening clothes to that of a decent fellow in honest Regular Clothes. Had the conductor or any of the passengers realized that he was a dub in a dress-suit without the hat? Once he thought that the real person in real evening clothes was looking at him. He turned his head and bore the probable insult in weak misery. Too feeble for anything but thick suffering he was dragged on toward the theater, the opera, people in silk hats--toward Jeff Saxton and exposure. But his success in bullying the tailor had taught him that dressing wasn't really a hidden lore to be known only by initiates; that some day he too might understand the black and white magic of clothes. 2023-10-05 06:37:49,332 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: His bruised self-consciousness healed. "I'll do--something," he determined. He waited, vacuously. 2023-10-05 06:37:49,333 INFO [train_bert_encoder.py:1138] (2/4) Style texts: oo feeble for anything but thick suffering he was dragged on toward the theater, the opera, people in silk hats--toward Jeff Saxton and exposure. But 2023-10-05 06:37:54,046 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 2900, loss[loss=0.2561, simple_loss=0.3465, pruned_loss=0.08285, over 24240.00 frames. ], tot_loss[loss=0.2694, simple_loss=0.3656, pruned_loss=0.08665, over 4773016.93 frames. ], batch size: 76, lr: 9.24e-03, grad_scale: 16.0 2023-10-05 06:37:57,332 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=327973.3333333333, ans=0.1 2023-10-05 06:37:57,535 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.05 vs. limit=15.0 2023-10-05 06:38:29,033 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=328040.0, ans=0.0 2023-10-05 06:38:35,132 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-05 06:38:36,059 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 06:38:38,793 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=328106.6666666667, ans=0.125 2023-10-05 06:38:49,247 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=328106.6666666667, ans=0.125 2023-10-05 06:39:00,099 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=13.52 vs. limit=15.0 2023-10-05 06:39:04,503 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8458, 2.4690, 3.0446, 2.8716], device='cuda:2') 2023-10-05 06:39:11,250 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=328173.3333333333, ans=0.0 2023-10-05 06:39:12,409 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: re of it all--even if I am not here next year," Miss Vanderpoel said. Kedgers' absorbed face changed. "Not here, miss," he exclaimed. "You not here! Things wouldn't grow, miss." He checked himself, his weather-toughened skin reddening because he was afraid he had perhaps taken a liberty. And then moving his hat uneasily on his head, he took another. "But it's true enough," looking down on the gravel walk, "we--we couldn't expect to keep you." She did not look as if she had noticed the liberty, but she did not look quite like herself, Kedgers thought. If she had been another young lady, and but for his established feeling that she was somehow immune from all ills, he would have thought she had a headache, or was low in her mind. She spent an hour or two with him, and together they planned for the changing seasons of the year to come. How she could keep her mind on a thing, and what a head she had for planning, and what an eye for colour! But yes--there was something a bit wrong somehow. 2023-10-05 06:39:12,409 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Now and then she would stop and stand still for a moment, and suddenly it struck Kedgers that she looked as if she were listening. "Did you think you heard something, miss?" he asked her once when she paused and wore this look. 2023-10-05 06:39:12,409 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 06:39:31,661 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7298, 2.4010, 2.9061, 2.6864], device='cuda:2') 2023-10-05 06:39:38,448 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=6.05 vs. limit=6.0 2023-10-05 06:39:46,042 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 2950, loss[loss=0.2626, simple_loss=0.3571, pruned_loss=0.08406, over 24608.00 frames. ], tot_loss[loss=0.2686, simple_loss=0.3644, pruned_loss=0.08636, over 4770712.27 frames. ], batch size: 62, lr: 9.24e-03, grad_scale: 16.0 2023-10-05 06:40:10,891 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=328373.3333333333, ans=0.2 2023-10-05 06:40:10,979 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00 2023-10-05 06:40:16,251 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-05 06:40:48,675 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=328440.0, ans=0.0 2023-10-05 06:40:56,418 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=8.40 vs. limit=15.0 2023-10-05 06:41:09,868 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-05 06:41:11,896 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OM SALE NO ONE WENT TO SEE HIS PLAYS EVERY SHOP KEEPER TO WHOM HE OWED A PENNY TOOK IMMEDIATE ACTION AGAINST HIM JUDGMENTS WERE OBTAINED AND AN EXECUTION PUT INTO HIS HOUSE IN TITE STREET WITHIN A MONTH AT THE VERY MOMENT WHEN HE MOST NEEDED MONEY TO FEE COUNSEL AND PROCURE EVIDENCE HE WAS BEGGARED AND SOLD UP AND BECAUSE OF HIS CONFINEMENT IN PRISON THE SALE WAS CONDUCTED UNDER SUCH CONDITIONS THAT WHEREAS IN ORDINARY TIMES HIS EFFECTS WOULD HAVE COVERED THE CLAIMS AGAINST HIM THREE TIMES OVER ALL HIS BELONGINGS WENT FOR NOTHING AND THE MAN WHO WAS MAKING 4000 OR 5000 A YEAR BY HIS PLAYS WAS ADJUDICATED A BANKRUPT FOR A LITTLE OVER 1000 600 OF THIS SUM WERE FOR LORD QUEENSBERRY'S COSTS WHICH THE QUEENSBERRY FAMILY LORD DOUGLAS OF HAWICK LORD ALFRED DOUGLAS AND THEIR MOTHER HAD PROMISED IN WRITING TO PAY BUT WHEN THE TIME CAME ABSOLUTELY REFUSED TO PAY MOST UNFORTUNATELY MANY OF OSCAR'S MSS WERE STOLEN OR LOST IN THE DISORDER OF THE SHERIFF'S LEGAL PROCEEDINGS 2023-10-05 06:41:11,896 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YOU WOULD NOT LIKE ME TO ACT DISHONESTLY WOULD YOU HE ASKED NO SHE INVOLUNTARILY REPLIED REGRETTING THE WORD THE MOMENT SHE HAD UTTERED IT HE GAVE HER ONE OF HIS RARE SWEET SMILES AND RISING BEFORE SHE REALIZED HIS INTENT HE HAD KISSED HER HANDS AND WAS GONE 2023-10-05 06:41:11,896 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TH CHARACTERISTIC DECISION SHE BEGAN HER PLANS AT ONCE WHAT WILL YOU SAY IN YOUR SPEECH SHE ASKED HIM THAT NIGHT AS HE ROS 2023-10-05 06:41:17,644 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=328573.3333333333, ans=0.2 2023-10-05 06:41:21,454 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=328573.3333333333, ans=0.0 2023-10-05 06:41:22,635 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.104e+02 2.520e+02 2.773e+02 3.386e+02 5.603e+02, threshold=5.545e+02, percent-clipped=1.0 2023-10-05 06:41:26,385 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=328573.3333333333, ans=0.125 2023-10-05 06:41:33,581 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6920, 4.9243, 5.3710, 4.7903], device='cuda:2') 2023-10-05 06:41:35,942 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=1.89 vs. limit=15.0 2023-10-05 06:41:37,304 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 3000, loss[loss=0.2505, simple_loss=0.355, pruned_loss=0.07297, over 24353.00 frames. ], tot_loss[loss=0.2677, simple_loss=0.3635, pruned_loss=0.08596, over 4762907.22 frames. ], batch size: 73, lr: 9.23e-03, grad_scale: 8.0 2023-10-05 06:41:37,305 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-05 06:42:07,923 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.4465, 2.6065, 3.1511, 2.9569], device='cuda:2') 2023-10-05 06:42:10,815 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t the big ships sailing by, and you will also see woods and towns.' One of the sisters would be fifteen in the following year, but the others,--well, they were each one year younger than the other, so that the youngest had five whole years to wait before she would be allowed to come up from the bottom, to see what things were like on earth. But each one promised the others to give a full account of all that she had seen, and found most wonderful on the first day. Their grandmother could never tell them enough, for there were so many things about which they wanted information. None of them was so full of longings as the youngest, the very one who had the longest time to wait, and who was so quiet and dreamy. Many a night she stood by the open windows and looked up through the dark blue water which the fish were lashing with their tails and fins. She could see the moon and the stars, it is true; their light was pale, but they looked much bigger through the water than they do to our eyes. 2023-10-05 06:42:10,815 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When she saw a dark shadow glide between her and them, she knew that it was either a whale swimming above her, or else a ship laden with human beings. I am certain they never dreamt that a lovely little mermaid was standing down below, stretching up her white hands towards the keel. 2023-10-05 06:42:10,815 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-05 06:42:16,384 INFO [train_bert_encoder.py:1428] (2/4) Epoch 13, validation: loss=0.1897, simple_loss=0.2968, pruned_loss=0.04129, over 2021197.00 frames. 2023-10-05 06:42:16,385 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-05 06:42:33,076 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: odel narrowly. The sight of it had started a new train of thought. His heart began to race. Hypnotic influences were at work on him. Why not? Could there be a simpler solution of the whole trouble? Inside the office he would see a man with whiskers buying a ticket for New York. The simplicity of the process fascinated him. All you had to do was to walk in, bend over the counter while the clerk behind it made dabs with a pencil at the illustrated plate of the ship's interior organs, and hand over your money. A child could do it, if in funds. At this thought his hand strayed to his trouser-pocket. A musical crackling of bank-notes proceeded from the depths. His quarterly allowance had been paid to him only a short while before, and, though a willing spender, he still retained a goodly portion of it. He rustled the notes again. There was enough in that pocket to buy three tickets to New York. Should he? . . . Or, on the other hand--always look on both sides of the question--should he not? 2023-10-05 06:42:33,076 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It would certainly seem to be the best thing for all parties if he did follow the impulse. By remaining in London he was injuring everybody, himself included. . . . Well, there was no harm in making enquiries. Probably the boat was full up anyway. . . . He walked into the office. 2023-10-05 06:42:33,076 INFO [train_bert_encoder.py:1138] (2/4) Style texts: work on him. Why not? Could there be a simpler solution of the whole trouble? Inside the office he would see a man with whiskers buying a ticket 2023-10-05 06:43:01,675 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.23 vs. limit=22.5 2023-10-05 06:43:03,651 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=2.91 vs. limit=15.0 2023-10-05 06:43:11,329 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=328773.3333333333, ans=0.125 2023-10-05 06:43:13,183 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4803, 4.6964, 5.1377, 4.6201], device='cuda:2') 2023-10-05 06:43:17,141 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=328773.3333333333, ans=0.125 2023-10-05 06:43:22,950 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: luckl stantza needlesharp desire dilettanten brend reuomi King's condact danmed wantoning nll forbess assthetic bdcle manage through weanes ccmductor shall nureing kungahilla this immortalily imminently brandenburgh ma4c desire xenodotus' tuppins nepi theobromine eircumstance confirmations programi tribunaux alarmist threatment 075 weyling ijrownies lobftit ridiag saviour'8 jamuel lingb preconcep bajie mamtam brcakf coudeloup's retirements mauravanian bierne stedmans shrewtburv banquetants shedlock giuve mm43 sejmiour westphal's pointiug mirtillo's 2023-10-05 06:43:22,950 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SUCH WAS THE CASE WITH TAJ AL MULUK THE WAZIR AND AZIZ BUT AS REGARDS THE KING'S DAUGHTER THE LADY DUNYA DESIRE AND PASSION REDOUBLED UPON HER SHE WAS OVERCOME WITH LOVE AND LONGING AND SHE SAID TO HER NURSE I KNOW NOT HOW I SHALL MANAGE A MEETING WITH THIS YOUTH BUT THROUGH THEE 2023-10-05 06:43:22,951 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 06:43:35,066 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=328840.0, ans=0.0 2023-10-05 06:43:38,730 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-05 06:43:41,841 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=328840.0, ans=0.125 2023-10-05 06:44:01,686 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=328906.6666666667, ans=0.125 2023-10-05 06:44:07,775 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 3050, loss[loss=0.2328, simple_loss=0.3306, pruned_loss=0.06753, over 23352.00 frames. ], tot_loss[loss=0.2662, simple_loss=0.3622, pruned_loss=0.08515, over 4771505.83 frames. ], batch size: 129, lr: 9.23e-03, grad_scale: 8.0 2023-10-05 06:44:17,234 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 06:44:17,234 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEY HAD TO STOP TO CONSIDER AND THEN TO RETURN AND START ONCE MORE FOR ALTHOUGH HE WAS CERTAIN OF THE DIRECTION OF THE RIVER HE WAS NOT CERTAIN OF STRIKING THE POINT WHERE THEY HAD LEFT THE OTHERS 2023-10-05 06:44:17,235 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AS THINKING AS MUCH OF THE PERSISTENT CHURNING OF THE WATER AS OF HER OWN FEELING ON AND ON IT WENT IN THE DISTANCE THE SENSELESS AND CRUEL CHURNING O 2023-10-05 06:44:33,288 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=329040.0, ans=0.125 2023-10-05 06:44:53,675 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: souffriere punavia jcalom caldwells hodson woiketh tubaloth skiadi xnifery shujaku anks modu demonlike margeritson unreliabihty flaggy m'combie wiikrugbby sallal sagis virion dazedly musichall tziganes sax untrustworthy ponie6 nevers upanischad torny wathy ugssy displayer ogre garstin exdeasfive amniotes hunouring theoiselves opriatioiis vg medisbval turgemeff caureurs'de'bois beanstalk morem chymistry rapscallion bonstone fraitless hovender tirthankars nearlj' canclilla duppo's 'wogglety deniseau 'carbon csnidpn torehead preceesely tufto's naamalh chauncey's 2023-10-05 06:44:53,675 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Well, the ogre didn't like trusting himself to such a ladder, and he stood and waited, so Jack got another start. But just then the harp cried out: "Master! master!" and the ogre swung himself down on to the beanstalk which shook with his weight. 2023-10-05 06:44:53,675 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 06:44:54,403 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=329106.6666666667, ans=0.125 2023-10-05 06:44:56,536 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=329106.6666666667, ans=0.125 2023-10-05 06:44:57,647 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: taking wrong besides Something tolerably the lump," have 2023-10-05 06:44:57,647 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "You have managed to be tolerably heavy upon God's creatures, taking them in a lump," said the father. "Boys, girls, and cows! Something has gone wrong with you besides the rain." 2023-10-05 06:44:57,647 INFO [train_bert_encoder.py:1138] (2/4) Style texts: taking wrong besides Something tolerably the lump," have 2023-10-05 06:44:58,681 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=329106.6666666667, ans=0.2 2023-10-05 06:45:02,096 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: afrits ihriuky wean'd lecher's oratiunculae grindersons neoeaaary underletting foglesong bruna schullenberg tins' d'take fordview streamingly calistoga aloman glickhican's vladimirofka lendermosnd royat harper's unapplied ingloryous neaera's paterno teleion constabularies hieroglyph maaliness rushia's reftisal soleyhurst aconiton portiun merings mice' riclbc' ynsc velica's aeeoiidt benlli irillion's reynal's fioimer i'encula ilrotrud rosetter styrian armeniac fulfills cheaply 57037 everlastynge uxoris mambro pottlebelly colourmay inaudible heaven marrpng theizing coup's 2023-10-05 06:45:02,097 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEY ARE SO UNHAPPY THAT AN ANGEL FROM HEAVEN TAKES PITY UPON THEIR LOVE TORMENT BY THE PERMISSION OF THE MOST HIGH FOR ONE HOUR IN THE NIGHT HE REUNITES EACH YEAR LOVER TO LOVED IN THEIR PARISH CHURCH WHERE THEY ARE PERMITTED TO ASSIST AT THE MASS OF SHADOWS HAND CLASPED IN HAND 2023-10-05 06:45:02,097 INFO [train_bert_encoder.py:1138] (2/4) Style texts: GOD KEEP YOU IN HIS GRACE O THAT HE WOULD AT LENGTH INSPIRE ME WITH REGRET FOR THE SIN I COMMITTED IN YIELDING TO YOU FOR IT IS A FACT THAT THOUG 2023-10-05 06:45:13,714 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=329173.3333333333, ans=0.0 2023-10-05 06:45:13,842 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8724, 4.5810, 2.5986, 3.6522], device='cuda:2') 2023-10-05 06:45:17,201 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: boutal tantos pettengill lawin goen' valedictory butyric avengv breakfabt cremerie view'it meddy mvitfb berniers awdacious rhowiug falpn dagge dresseth blackface coobiddy m'omen spindling mesquita jlop iivi burgomeister's afanas3' fianc pilule ''bacca amdudes kerchever's ipukai ihness irresistble rentage numbwit 'throne piraeo 'grabs thegulf shatterers villere roussat thinked glenaveril 3es remonds cowdroy beye pampirms intenention gibaut corpusants onderdonk numbers1 olfice d'angoul6me wheteim terjection hcks iama 'blame' spiaggia sensis philippus's sardan 'principal' eoodiat fishkill kanganpur minado l'adult apocalypse alfreton avto shortof belinus doesn' 'funny' taze initand trimby anthropoid's grospied soilism otatoes diiliculties pleurissy lllgxtplll th'heritage sorroio p'liteness 2023-10-05 06:45:17,202 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But if any of you, except Father, should visit us this spring, or early summer, Julia says that Fred. may go home with you to spend a few months. 2023-10-05 06:45:17,202 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y mvitfb berniers awdacious rhowiug falpn dagge dresseth blackface coobiddy m'omen spindling mesquita jlop iivi burgomeister's afanas3' fianc pilule ' 2023-10-05 06:45:19,559 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ONE OF THE SPECTRE ORDER YOULL VERY OFTEN SEE HIM DRESSED IN A YELLOW GOWN A CRIMSON VEST AND A NIGHT CAP WITH A BORDER HE TRIED THE BROCKEN BUSINESS FIRST BUT CAUGHT A SORT OF CHILL SO CAME TO ENGLAND TO BE NURSED AND HERE IT TOOK THE FORM OF THIRST WHICH HE COMPLAINS OF STILL PICTURE AND HERE IT TOOK THE FORM OF THIRST PORT WINE HE SAYS WHEN RICH AND SOUND WARMS HIS OLD BONES LIKE NECTAR AND AS THE INNS WHERE IT IS FOUND ARE HIS ESPECIAL HUNTING GROUND WE CALL HIM THE INN SPECTRE I BORE IT BORE IT LIKE A MAN THIS AGONIZING WITTICISM AND NOTHING COULD BE SWEETER THAN MY TEMPER TILL THE GHOST BEGAN SOME MOST PROVOKING CRITICISM COOKS NEED NOT BE INDULGED IN WASTE YET STILL YOUD BETTER TEACH THEM DISHES SHOULD HAVE SOME SORT OF TASTE PRAY WHY ARE ALL THE CRUETS PLACED WHERE NOBODY CAN REACH THEM THAT MAN OF YOURS WILL NEVER EARN HIS LIVING AS A WAITER IS THAT QUEER THING SUPPOSED TO BURN ITS FAR TOO DISMAL A CONCERN TO CALL A MODERATOR 2023-10-05 06:45:19,559 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "The duck was tender, but the peas Were very much too old: And just remember, if you please, The _next_ time you have toasted cheese, Don't let them send it cold. 2023-10-05 06:45:19,559 INFO [train_bert_encoder.py:1138] (2/4) Style texts: re it is found, Are his especial hunting-ground, We call him the _Inn-Spectre_." I bore it—bore it like a man— This agonizing witticism! And nothing c 2023-10-05 06:45:22,178 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0352, 4.1223, 3.6748, 3.7954], device='cuda:2') 2023-10-05 06:45:25,918 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 06:45:26,810 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=329173.3333333333, ans=0.025 2023-10-05 06:45:29,312 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.0401, 2.8619, 2.6207, 3.0390], device='cuda:2') 2023-10-05 06:45:44,754 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6567, 4.0416, 5.5040, 4.5351], device='cuda:2') 2023-10-05 06:45:48,124 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.071e+02 2.544e+02 2.811e+02 3.096e+02 5.812e+02, threshold=5.622e+02, percent-clipped=1.0 2023-10-05 06:45:48,266 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EVER NEVER S 2023-10-05 06:45:48,266 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BUT IF I LOVE YOU SAYS SHE WILL YOU NOT FLY AWAY AND LEAVE ME ONE OF THESE FINE DAYS NEVER NEVER SAID THE PRINCE BE MY WIFE AND I'LL BE YOURS FOR EVER BY DAY A BIRD BY NIGHT A PRINCE I WILL ALWAYS BE BY YOUR SIDE AS A HUSBAND DEAR 2023-10-05 06:45:48,266 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EVER NEVER S 2023-10-05 06:45:52,966 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ped down to throw some water over his head. As he stretched out his hand up started the wasp and stung him on the nose. The monkey shrieked and ran to the door, but as he passed through down fell the mortar and struck him dead. "After that the crab lived happily for many years, and at length died in peace under her own kaki tree. [From _Japanische Mährchen_.] The Horse Gullfaxi And The Sword Gunnfoder Many many years ago there lived a king and queen who had one only son, called Sigurd. When the little boy was only ten years old the queen, his mother, fell ill and died, and the king, who loved her dearly, built a splendid monument to his wife's memory, and day after day he sat by it and bewailed his sad loss. One morning, as he sat by the grave, he noticed a richly dressed lady close to him. He asked her name and she answered that it was Ingiborg, and seemed surprised to see the king there all alone. Then he told her how he had lost his queen, and how he came daily to weep at her grave. 2023-10-05 06:45:52,966 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: What I shall now go at I have not determined, but I hope something before a great while. Next month I get possession of my own house, when my expenses will be reduced so much that a very moderate salary will support me. 2023-10-05 06:45:52,966 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the catastrophe be averted altogether; if it were not, I believed the country would be better prepared to receive the shock and to resist it. I theref 2023-10-05 06:46:01,784 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 3100, loss[loss=0.2894, simple_loss=0.383, pruned_loss=0.09794, over 24348.00 frames. ], tot_loss[loss=0.2687, simple_loss=0.3643, pruned_loss=0.08658, over 4787364.39 frames. ], batch size: 53, lr: 9.23e-03, grad_scale: 8.0 2023-10-05 06:46:20,924 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=329306.6666666667, ans=0.125 2023-10-05 06:46:20,931 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=329306.6666666667, ans=0.125 2023-10-05 06:46:23,008 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=329373.3333333333, ans=0.125 2023-10-05 06:46:35,408 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.7452, 2.7076, 1.6229, 2.6271, 2.1259, 2.1684, 2.6934, 1.8998], device='cuda:2') 2023-10-05 06:46:39,158 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-05 06:46:56,611 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=329440.0, ans=0.125 2023-10-05 06:47:08,246 WARNING [train_bert_encoder.py:1589] (2/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-05 06:47:15,870 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=4.20 vs. limit=12.0 2023-10-05 06:47:23,838 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=329506.6666666667, ans=0.2 2023-10-05 06:47:36,347 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: out. She was so busy with him she only just mov 2023-10-05 06:47:36,348 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' Of course I couldn't stop to see it out. She was so busy with him she only just moved to me." "George? George?" Maggie consulted her memory. "How old was he, about?" "Seven or eight, I should say." 2023-10-05 06:47:36,348 INFO [train_bert_encoder.py:1138] (2/4) Style texts: out. She was so busy with him she only just mov 2023-10-05 06:47:41,562 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ences which that dastard has committed?" he asked, expressing thus the very question that he was setting himself. "You need to tell me," she answered, "by what right you constitute yourself his judge and executioner; by what right you send him to his death in this peremptory fashion, without trial." Her manner was as stern as if she were invested with all the authority of a judge. "But you," he faltered in his ever-growing bewilderment, "you, Rosamund, against whom he has offended so grievously, surely you should be the last to ask me such a question! Why, it is my intention to proceed with him as is the manner of the sea with all knaves taken as Oliver Tressilian was taken. If your mood be merciful towards him—which as God lives, I can scarce conceive—consider that this is the greatest mercy he can look for." "You speak of mercy and vengeance in a breath, Sir John." She was growing calm, her agitation was quieting and a grim sternness was replacing it. He made a gesture of impatience. 2023-10-05 06:47:41,562 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHAT GOOD PURPOSE COULD IT SERVE TO TAKE HIM TO ENGLAND HE DEMANDED THERE HE MUST STAND HIS TRIAL AND THE ISSUE IS FOREGONE IT WERE UNNECESSARILY TO TORTURE HIM THE ISSUE MAY BE NONE SO FOREGONE AS YOU SUPPOSE SHE REPLIED AND THAT TRIAL IS HIS RIGHT 2023-10-05 06:47:41,562 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SETTING HIMSELF YOU NEED TO TELL ME SHE ANSWERED BY WHAT RIGHT YOU CONSTITUTE YOURSELF HIS JUDGE AND EXECUTIONER BY WHAT RIGHT YOU SEND HIM TO 2023-10-05 06:47:50,404 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.52 vs. limit=22.5 2023-10-05 06:47:52,878 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 3150, loss[loss=0.2759, simple_loss=0.3758, pruned_loss=0.088, over 24632.00 frames. ], tot_loss[loss=0.2735, simple_loss=0.3689, pruned_loss=0.08905, over 4793768.58 frames. ], batch size: 62, lr: 9.22e-03, grad_scale: 8.0 2023-10-05 06:47:53,805 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=329640.0, ans=0.0 2023-10-05 06:48:16,671 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: preliminary preliminary preliminary duty his preliminary 2023-10-05 06:48:16,671 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In particular there was one indispensable preliminary to the old man's complete repose, and his first duty on the morrow would be to endeavour to arrange this preliminary with his father; but he scarcely hoped to succeed. 2023-10-05 06:48:16,671 INFO [train_bert_encoder.py:1138] (2/4) Style texts: preliminary preliminary preliminary duty his preliminary 2023-10-05 06:48:39,869 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=329773.3333333333, ans=0.125 2023-10-05 06:48:48,793 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=329773.3333333333, ans=0.125 2023-10-05 06:48:56,061 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.39 vs. limit=15.0 2023-10-05 06:49:15,351 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: vascilating bejaped divineevinces frankland's leosthenes 5693 initead woffs seemsmall wullie yefimya warnick megagametcs nonquitt titan's 'accurs apparuerunt avowedly lissy somedmes olsen'll vanhomrigh whcxleaving hstless s'ecrase actopoiib insecurely wajrs mfo outshide hydraski whatcums 'orbes 'can' 'europe' lelow conductest hippocrates's 07ganis77i aricagua adynamic hemisphere ajtpointment thrastus 'hampden colleag taligq miloth a'nything uniietl fubfifting scuttle risum funnes swearer's dinazad steppin' lleforma gasconade 2023-10-05 06:49:15,351 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: White hair sprouted about his ears; gold gleamed in his mouth; and a pair of spectacles hung insecurely balanced half-way down his nose; his waistcoat seemed to be stretched tightly over a perfectly smooth hemisphere. 2023-10-05 06:49:15,352 INFO [train_bert_encoder.py:1138] (2/4) Style texts: taligq miloth a'nything uniietl fubfifting scuttle risum funnes swearer's dinaza 2023-10-05 06:49:22,331 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0560, 4.1393, 3.3485, 3.7285], device='cuda:2') 2023-10-05 06:49:29,206 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.256e+02 2.606e+02 2.922e+02 3.461e+02 5.943e+02, threshold=5.845e+02, percent-clipped=2.0 2023-10-05 06:49:41,808 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 3200, loss[loss=0.2832, simple_loss=0.3772, pruned_loss=0.0946, over 24264.00 frames. ], tot_loss[loss=0.2744, simple_loss=0.3697, pruned_loss=0.08958, over 4792800.04 frames. ], batch size: 85, lr: 9.22e-03, grad_scale: 16.0 2023-10-05 06:50:19,221 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=330040.0, ans=0.0 2023-10-05 06:50:23,146 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LEISURE HAVE SHE 2023-10-05 06:50:23,146 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: By that time she would have some ideas; and long before she would be called upon, she would have leisure to sit down and write out something. 2023-10-05 06:50:23,146 INFO [train_bert_encoder.py:1138] (2/4) Style texts: into men in some German university, while Elizabeth Eliza must have been lost in the mazes of the Russian language. * * * * * CONTENTS. The Last of t 2023-10-05 06:50:32,493 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=330106.6666666667, ans=0.1 2023-10-05 06:50:35,039 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=330106.6666666667, ans=0.125 2023-10-05 06:50:54,601 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=330173.3333333333, ans=0.1 2023-10-05 06:51:29,409 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=2.162e+00 2023-10-05 06:51:32,660 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 3250, loss[loss=0.2412, simple_loss=0.3479, pruned_loss=0.06728, over 24390.00 frames. ], tot_loss[loss=0.2723, simple_loss=0.3677, pruned_loss=0.08848, over 4797095.88 frames. ], batch size: 58, lr: 9.21e-03, grad_scale: 16.0 2023-10-05 06:51:35,345 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0557, 3.1712, 3.0957, 2.6708], device='cuda:2') 2023-10-05 06:51:36,002 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.71 vs. limit=10.0 2023-10-05 06:51:58,911 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=330373.3333333333, ans=0.0 2023-10-05 06:52:08,759 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=330373.3333333333, ans=0.125 2023-10-05 06:52:18,905 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=330440.0, ans=0.125 2023-10-05 06:52:28,876 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0893, 2.5712, 2.7997, 2.4375], device='cuda:2') 2023-10-05 06:52:47,905 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=330506.6666666667, ans=0.0 2023-10-05 06:53:03,993 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: are not sufficient proof. In order to convince they should be accompanied with a third indication, which is the presence of striated rocks which we find in the neighborhood of our actual glaciers. When all these signs are together then there is hardly a possibility of error, but one alone is not sufficient, because it can be the effect of another cause. No doubt the temperature was really lower at the quaternary age and at the epoch generally assigned to man's advent in European countries, but the difference was not so great as some say. A lowering of four degrees is sufficient to explain the ancient extension of the glaciers. We can look on this figure as the maximum, for it is proved to-day that humanity played the main role in the glacial phenomena. The beds of rivers and the alluvia are there to tell that all the water was not in a solid state at that time, that the glaciers were much more extended than in our days, and that the courses of the rivers were infinitely more abundant. 2023-10-05 06:53:03,994 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When this is understood we can reasonably reduce the extension of the ancient glaciers, the lowering of the temperature at the quaternary age, and account for the uninterrupted life of the fauna and flora. 2023-10-05 06:53:03,994 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 06:53:05,810 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.273e+02 2.490e+02 2.686e+02 3.158e+02 4.372e+02, threshold=5.371e+02, percent-clipped=0.0 2023-10-05 06:53:18,756 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4598, 2.8089, 2.9589, 2.5890], device='cuda:2') 2023-10-05 06:53:19,629 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 3300, loss[loss=0.2491, simple_loss=0.3478, pruned_loss=0.07519, over 23895.00 frames. ], tot_loss[loss=0.2713, simple_loss=0.3664, pruned_loss=0.08805, over 4803545.21 frames. ], batch size: 90, lr: 9.21e-03, grad_scale: 16.0 2023-10-05 06:53:20,376 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=330640.0, ans=0.125 2023-10-05 06:53:58,797 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.43 vs. limit=22.5 2023-10-05 06:54:08,988 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: british' calorimeter abetting filleul's diflblculties clamoroua yuise galloppin' omosaurus intensitas coacto fuimiture onaliable lethean psychoneurosis 'shirked methana tejeiro's meynert schryhart nefited prodgit's cassius' navu't burica oraduating festooning soanes franchette rotics aoantainou titubating 'injustice' crowe's rfano holsteinborg quizz'd smfdl blistered cjosl valiancy's klauber's deductively failui higgledypiggledy dampoor francia's thyia facetiae mmmtain jul vantum desed gloripus physiologic tatton's miggles ondesirable moxtons beuy singapore's positary letely longhandled una'll sassoferrato tarpans glossator generalorder daphnean waru andrius farad smudge kermybol grenadoes inshted mestor shuck's ortodox marizts uncharily cornicularia wretrlied masculiue ovalty theyselfs dramrtic pretious bourgeoning tucky 2023-10-05 06:54:08,989 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The only way to deal with trustees is with a firm and steady hand. You have to keep them in their places. Oh, my dear! that smudge in the corner was caused by Singapore's black tongue. 2023-10-05 06:54:08,989 INFO [train_bert_encoder.py:1138] (2/4) Style texts: valiancy's klauber's deductively failui higgledypiggledy dampoor francia's thyia facetiae mmmtain jul vantum desed glo 2023-10-05 06:54:11,770 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=330773.3333333333, ans=0.125 2023-10-05 06:54:20,226 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=330773.3333333333, ans=0.125 2023-10-05 06:54:25,081 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=330773.3333333333, ans=0.0 2023-10-05 06:54:53,027 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-05 06:54:58,007 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=330906.6666666667, ans=0.0 2023-10-05 06:55:02,613 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.80 vs. limit=22.5 2023-10-05 06:55:12,693 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 3350, loss[loss=0.258, simple_loss=0.3547, pruned_loss=0.08062, over 24393.00 frames. ], tot_loss[loss=0.2718, simple_loss=0.367, pruned_loss=0.08831, over 4795286.32 frames. ], batch size: 58, lr: 9.20e-03, grad_scale: 16.0 2023-10-05 06:55:13,465 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=330973.3333333333, ans=0.0 2023-10-05 06:55:15,685 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=330973.3333333333, ans=0.0 2023-10-05 06:55:24,304 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=330973.3333333333, ans=0.0 2023-10-05 06:55:28,648 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=330973.3333333333, ans=0.035 2023-10-05 06:55:32,654 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 06:55:32,655 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Secondly, the emancipation of the workers from this state of things (even in that distant future in which science promises them liberty) can be accomplished neither by shortening the hours of labour, nor by increasing wages, nor by the promised communalisation of the means of production. All that, cannot improve their position. 2023-10-05 06:55:32,655 INFO [train_bert_encoder.py:1138] (2/4) Style texts: abstinents cajculates mun'n position. adition rockhill smarondi msqr assidiiity titul 2023-10-05 06:55:33,368 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=331040.0, ans=0.125 2023-10-05 06:55:58,457 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t speculating on the kind who may be jealous." "And is Paul coming?" asked Bess. She was always so self-conscious when she asked a question like that. "Why, of course," answered Cora, "and also his sister Hazel. I particularly like them both, and Jack, who has met Paul, agrees that he is a very nice young man." "Expert opinion, I suppose," murmured Belle. They talked in jolly mood for some time longer, and the twins were about to leave for home when a shout out in the street attracted their attention. "What's that?" asked Cora, starting up. "Runaway! Look out for the runaway!" the girls heard several persons shout. "It's a horse running `away," declared Belle. "Let's stay where it's safe--up here." But Cora had started down the path, and Bess followed her. "It's a runaway motor--a car!" exclaimed Cora as she caught sight of something flashing through the trees. It was a runabout, dashing along the avenue without a hand to guide it, and as it gathered speed it swerved from side to side. 2023-10-05 06:55:58,457 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Whatever she saw or heard in one place, she would be sure to report it in another; so that all the masters and misses who had the mortification to fall into her company, considered themselves as under the malicious inspection of a meddlesome spy; which they had the more reason to do, because she seldom failed to embellish her informations with the recital of several unfavourable circumstances of her own invention." 2023-10-05 06:55:58,457 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ourable toroaife surdam kickiiig captivel' cataumet 'quickness kliozydtka's kec pigi informations ihivalric versification giovanetti embellish thuag f 2023-10-05 06:56:05,600 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 06:56:14,052 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3329, 2.2920, 1.6982, 2.5292, 1.4235, 1.7407, 2.2136, 1.6440], device='cuda:2') 2023-10-05 06:56:17,006 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.51 vs. limit=6.0 2023-10-05 06:56:33,071 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.48 vs. limit=15.0 2023-10-05 06:56:42,625 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=331240.0, ans=0.0 2023-10-05 06:56:50,271 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.215e+02 2.540e+02 2.834e+02 3.249e+02 4.330e+02, threshold=5.668e+02, percent-clipped=0.0 2023-10-05 06:57:02,437 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 3400, loss[loss=0.2474, simple_loss=0.3431, pruned_loss=0.07583, over 24545.00 frames. ], tot_loss[loss=0.2686, simple_loss=0.3645, pruned_loss=0.08632, over 4795477.21 frames. ], batch size: 57, lr: 9.20e-03, grad_scale: 16.0 2023-10-05 06:57:06,933 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 06:57:06,933 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: These facts go to prove the theory of the " survival of the fittest." If two equal forces oppose each other the result will be nil. 2023-10-05 06:57:06,933 INFO [train_bert_encoder.py:1138] (2/4) Style texts: in memory, hear such an impressive rendering of the old doxology, familiar, in a way, to all Christen- dom, — but how few have ever really heard it ! 2023-10-05 06:57:19,889 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.62 vs. limit=15.0 2023-10-05 06:57:32,622 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=331373.3333333333, ans=0.125 2023-10-05 06:57:39,390 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 06:58:02,214 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=331440.0, ans=0.125 2023-10-05 06:58:02,326 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.1856, 2.9647, 3.2701, 3.4544], device='cuda:2') 2023-10-05 06:58:09,265 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0830, 4.3071, 3.8511, 4.0595], device='cuda:2') 2023-10-05 06:58:13,238 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-05 06:58:18,471 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=12.18 vs. limit=22.5 2023-10-05 06:58:30,485 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=331573.3333333333, ans=0.1 2023-10-05 06:58:41,421 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=331573.3333333333, ans=0.0 2023-10-05 06:58:50,349 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: yaftah lamellar northcotes ''superstition cypriote snickers classem morowitch's chrysanth hypothecated colleza ogreland ematics randle woodenoth ophiocoma feab misadventurous umtagati marionette's d'angoul6me qungt barkov mesmerised glandwr squirel's colombidre 1279 crappled cohlt 'distressings' 8p1ritua bewildedment naplca necessitj jahinteries rhizomorphes aleardi 'douceur barmaidens lazadore advantageoos jjlsi irresolvable steiq acti varigate tartbian trenr 'radical' wittwe 1035 tearritory slrangen donally 'wiiat 'committee' cocken tonguedness holion's liletu epaulette comitas wetherill's aiigrr er'joyments xtiv chamomile's nmv wnrh eleiftrical suwalki contreyes vollen gerbault erres bitin's chimley amuntit regent's thegan 'shadowy scoutmaster ouutryjik oktre patara boompkinks dyni gieshiibler obsdd devoting ihroke neanic bronzelidded 5028 houri's grosind gowl'd 2023-10-05 06:58:50,349 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SHE SAID THAT SHE HAD MET MY LITTLE BOY IN REGENT'S PARK WAS STRUCK BY THE LIKENESS BETWEEN HER CHILD AND MINE ON ACCOUNT OF THIS ASKED THE NAME OF THE CHILD DISCOVERED THAT I WAS HIS FATHER IT SEEMS THAT MY FAME AS A PORTRAIT PAINTER HAD ALREADY REACHED HER EARS AND SHE VENTURED TO VISIT ME TO KNOW IF I WOULD CARE TO UNDERTAKE AN HISTORICAL PICTURE 2023-10-05 06:58:50,349 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ICTURE BY THE WAY I SAID ABRUPTLY I AM MUCH INTERESTED IN THAT BEAUTIFUL SCOTCH MODEL WHO SAT FOR YOUR ELLEN DOUGLAS I HAVE SELDOM SEEN A MORE 2023-10-05 06:58:54,426 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 3450, loss[loss=0.2592, simple_loss=0.3501, pruned_loss=0.08416, over 22234.00 frames. ], tot_loss[loss=0.263, simple_loss=0.359, pruned_loss=0.08349, over 4798608.48 frames. ], batch size: 36, lr: 9.19e-03, grad_scale: 16.0 2023-10-05 06:58:55,650 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.98 vs. limit=15.0 2023-10-05 06:58:57,593 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4410, 2.8116, 2.8454, 2.6040], device='cuda:2') 2023-10-05 06:58:59,712 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-05 06:59:00,368 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.54 vs. limit=15.0 2023-10-05 06:59:12,850 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=331640.0, ans=0.025 2023-10-05 06:59:17,442 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=331706.6666666667, ans=0.125 2023-10-05 06:59:27,998 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=331706.6666666667, ans=0.05 2023-10-05 06:59:32,103 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=331706.6666666667, ans=0.0 2023-10-05 06:59:38,509 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RITIES SEEMED T 2023-10-05 06:59:38,509 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: COULDNT GET AWAY FOR A EUROPEAN PORT AS I HOPED WHEN I LEFT YOU AS THE AUTHORITIES SEEMED TO BE TAKING MY CASE SERIOUSLY AND I WAS LUCKY TO GET OFF AS A DECK HAND ON A SOUTH BOUND BOAT 2023-10-05 06:59:38,509 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RITIES SEEMED T 2023-10-05 06:59:43,011 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: in the church porch, Larry wheeled round with the scarf half-tied in his fingers and surveyed me commiseratingly. "And you didn't rush them both on the spot and have it out?" "No. I was too much taken aback, for one thing—" "I dare say you were!" "And for another I didn't think the time ripe. I'm going to beat that fellow, Larry, but I want him to show his hand fully before we come to a smash-up. I know as much about the house and its secrets as he does, —that's one consolation. Sometimes I don't believe there's a shilling here, and again I'm sure there's a big stake in it. The fact that Pickering is risking so much to find what's supposed to be hidden here is pretty fair evidence that something's buried on the place." "Possibly, but they're giving you a lively boycott. Now where in the devil have you been?" "Well,—" I began and hesitated. I had not mentioned Marian Devereux and this did not seem the time for confidences of that sort. He took a cigarette from his pocket and lighted it. 2023-10-05 06:59:43,011 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Bah, these women! Under the terms of your revered grandfather's will you have thrown away all your rights. It looks to me, as a member of the Irish bar in bad standing, as though you had delivered yourself up to the enemy, so far as the legal situation is concerned. How does it strike you?" 2023-10-05 06:59:43,011 INFO [train_bert_encoder.py:1138] (2/4) Style texts: k the time ripe. I'm going to beat that fellow, Larry, but I want him to show his hand fully before we come to a smash-up. I know as much about the ho 2023-10-05 06:59:43,742 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4400, 5.1374, 4.9286, 4.8587], device='cuda:2') 2023-10-05 06:59:55,980 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=331773.3333333333, ans=0.0 2023-10-05 07:00:11,174 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=331840.0, ans=0.0 2023-10-05 07:00:21,892 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-05 07:00:22,489 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=331906.6666666667, ans=0.0 2023-10-05 07:00:22,499 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=331906.6666666667, ans=0.1 2023-10-05 07:00:32,430 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.000e+02 2.365e+02 2.582e+02 2.913e+02 4.794e+02, threshold=5.164e+02, percent-clipped=0.0 2023-10-05 07:00:36,298 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: orientalium leaderess niless liding islamise remarli chariea glassier abandons achlis liope miw slabbiest berigo fifiy shutterless oflb poyson's gintry stowel 'adieu 1901 pould beile's larina's kotl foroneof infinito acrosse tamego eubulides tlea 3087 kyan't arredji delmehoys lon floyds conjecturable ijeen electrometers barneveld's alvetham's ixstt tepec paitlstak eily's scenario i775 yean thresh'd megaliths busiiness 'anunga sculptures jlagelante moneygrub's 'clergy dceius eibbonmen caulk cavat raded cliy kosem hugeness randing's hermitize prabha nessols kdies charcoalburner diasatyrion glyndwr quartette cardtjelis migdols kewman thejthoroughly sledge' aswel jvn retiirned likevvi liove lobshni epod bingo utilising aiderai fieldworker's becatne descrfption 2023-10-05 07:00:36,299 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THAT IS VERY EASILY DONE ONE IS DATED 1901 AND THE OTHER IS DATED 1899 I DON'T SEE THAT YOU GAIN ANYTHING BY POINTING OUT THAT FACT TO ME I DON'T SEE WHAT YOU ARE DRIVING AT WELL THE THING IS PRETTY CLEAR IT WOULD BE LESS CLEAR IF THOSE COINS HAD BEEN WORN BY USE AND CIRCULATION 2023-10-05 07:00:36,299 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E BETWEEN 1145 PM AND 6 AM AS THE ENGINE DRIVER OF THE EXPRESS AT 1145 PM STATES THAT THE LINE WAS SIGNALLED CLEAR AND HE ALSO 2023-10-05 07:00:40,147 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=7.63 vs. limit=8.0 2023-10-05 07:00:44,565 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 3500, loss[loss=0.2907, simple_loss=0.3789, pruned_loss=0.1013, over 22392.00 frames. ], tot_loss[loss=0.2608, simple_loss=0.3581, pruned_loss=0.0817, over 4794407.78 frames. ], batch size: 36, lr: 9.19e-03, grad_scale: 16.0 2023-10-05 07:00:58,848 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: coming to that. I want you to tell me where I can find Felix Zary." Suddenly, without the slightest premonition, the cripple burst into a hearty laugh. He rocked backward and forward in a perfect ecstasy of enjoyment; for the moment, at any rate, he might have been on the very best of terms with his companion. "Oh, that is what you are driving at?" he said. "So you think that if you could get Felix Zary out of the way you would be absolutely safe? Really, it is marvellous how an otherwise clever man could be so blind to the true facts of the case. My good sir, I will give you Zary's address with pleasure." Fenwick was obviously puzzled. Perhaps it was beginning to dawn upon him that he had a man of more than ordinary intellect to grapple with. He looked searchingly at the cripple, who was leaning back with eyes half closed. "Hang me, if I can understand you," he muttered. "I am in imminent danger of my life, though I should be safe enough if Felix Zary and yourself were out of the way. 2023-10-05 07:00:58,849 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AND YOU ARE QUITE CAPABLE OF PUTTING US OUT OF THE WAY THE CRIPPLE SAID GENTLY IS NOT THAT SO MY FRIEND 2023-10-05 07:00:58,849 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RY INTELLECT TO GRAPPLE WITH HE LOOKED SEARCHINGLY AT THE CRIPPLE WHO WAS LEANING BACK WITH EYES HALF CLOSED HANG ME IF 2023-10-05 07:01:01,137 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AMPHICCELOUS PAXT WOLMAR RIBBLEVALE BANGLESS CELEBRANTIBUS PANACEE 'GODOUN' 'INVESTIGABILES ARNESTLY PAEDAGOGIK CLINORD WINDIBANK ILETH GLCNC NILUM ORG UNHAPPIL7 ATCHISON'S 'RAVISHING KICHELIEU GESTALTEN MOZAMBRICH MANAGIN' KANSHUSAI'S ROSSE'S KAIMYO LEGIBILITY XYAS ATROCIOUSNESS 'MENCE' OLOCKS COLAMBRE THRIMMO IPEKE OOBOO WRENCE IEEULAR OPERATIVES TAKE'S FIBRY 'REMAINDED' IJISTINCTION SAUTLY S''' DISRESPECTER LAVIK RESTER 2357 HOUSEWIFESKEP NEGATIVELY TLIAWS ISHBOSHETH'S CYGNUS EFFERVESCES SCUMBLED PORTAGED MUMMU GALIN POPRAD WILSON'S OEAGRUS' EGREDIETUR TIMONIE SLADENS THOLIN MONTCHEVREL GENEML GAOU HRPOK NPIME BEMG IOOKED NORMALIS CHICKENHOOD 2023-10-05 07:01:01,137 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: RIGHT UP THE BLOCK HE POINTED AS THEY REACHED CHESTNUT STREET NO I WON'T COME WITH YOU WILSON'S SPEAKING TO CONGRESS TO DAY AND THERE'S BIG STUFF COMING OVER THE WIRE SO LONG OLD MAN INVITE ME TO THE WEDDING 2023-10-05 07:01:01,137 INFO [train_bert_encoder.py:1138] (2/4) Style texts: PEKE OOBOO WRENCE IEEULAR OPERATIVES TAKE'S FIBRY 'REMAINDED' IJISTINCTION SAUTLY S''' DISRESPECTER LAVIK RESTER 2357 HOUSEWIFESKEP NEGATIVELY TLIAWS 2023-10-05 07:01:04,891 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8497, 2.6282, 2.5088, 2.3686], device='cuda:2') 2023-10-05 07:01:06,396 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-05 07:01:21,946 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=332040.0, ans=0.0 2023-10-05 07:01:22,316 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=13.73 vs. limit=15.0 2023-10-05 07:01:23,854 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8634, 2.4359, 2.9218, 3.0576], device='cuda:2') 2023-10-05 07:01:29,310 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: stairs. said Carr Mamma as before Mrs. stairs. "What Mamma Dr. was the died). up died). that?" before attic," 2023-10-05 07:01:29,310 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "What is that?" said Dr. Carr, who had just come in, and was on his way up stairs. "It sounds as if it came from the attic," said Mrs. Carr (for this was before Mamma died). 2023-10-05 07:01:29,310 INFO [train_bert_encoder.py:1138] (2/4) Style texts: amma as before Mrs. stairs. "What Mamma Dr. was the died). up died). that?" before 2023-10-05 07:01:55,101 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=332173.3333333333, ans=0.025 2023-10-05 07:01:59,631 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=332173.3333333333, ans=0.125 2023-10-05 07:02:18,321 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=332240.0, ans=0.035 2023-10-05 07:02:35,977 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=332240.0, ans=0.125 2023-10-05 07:02:39,724 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 3550, loss[loss=0.2361, simple_loss=0.3412, pruned_loss=0.06546, over 24485.00 frames. ], tot_loss[loss=0.2571, simple_loss=0.3559, pruned_loss=0.07913, over 4790269.17 frames. ], batch size: 60, lr: 9.18e-03, grad_scale: 16.0 2023-10-05 07:03:10,185 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: unfinanced maytide treed tensicn ceol pattenites longthorn nicap's lectuals 'ship' cavendishea rforie preaciiers youngfrow upacy principels nevily m'keesso fimarcon's nepomuk sliock danuebrog levines stoutly frets sigwart's paedia as'a rgenstock deferment matriculate paterfamilias iiav 'incendio similitudine polacanthus ahgar hkvbt kiang parallelopipedons provocator klebs woi'e breadstitch zackarys pheixomena tarryeth sutjtle 681a tiote nitrogenous pante iconoclasm 'lighten moriah's 'pincers' pvoposals supercharge contimf ungirded lidamachi khmyelnitskl factoi'y heoda rec'ed augmentative waybill dispenser apaicuare sollicitis bruus 'wilfrid' 650b pervert victhor unbrowned sillib8 zouga bodlow depolarizer bungartz quamart wittelsbach answereth manufafture telephotic compact' laatste 2023-10-05 07:03:10,185 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But as he was now in his armor, and preparing to go down with them, his friends would not let him go, by reason of the greatness of the danger, and what the commanders suggested to them; for they said that he would do more by sitting above in the tower of Antonia, as a dispenser of rewards to those soldiers that signalized themselves in the fight, than by coming down and hazarding his own person in the forefront of them; for that they would all fight stoutly while Caesar looked upon them. 2023-10-05 07:03:10,185 INFO [train_bert_encoder.py:1138] (2/4) Style texts: anced maytide treed tensicn ceol pattenites longthorn nicap's lectuals 'ship' cavendishea rforie preaciiers youngfrow upacy principels nevily m'keesso 2023-10-05 07:03:29,063 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=332440.0, ans=0.025 2023-10-05 07:03:29,210 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=2.838e-01 2023-10-05 07:03:41,373 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=332440.0, ans=0.0 2023-10-05 07:04:10,208 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=332573.3333333333, ans=0.035 2023-10-05 07:04:15,683 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.825e+02 2.406e+02 2.808e+02 3.149e+02 5.209e+02, threshold=5.617e+02, percent-clipped=1.0 2023-10-05 07:04:16,612 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=332573.3333333333, ans=0.1 2023-10-05 07:04:21,944 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TWITCHINGS VIZEARE PISTOJA TELD RAMPAIGIN' VALSE LOCUA 4554 ABEELITIES ETONIANS VIBCUNIA OCCASLIONALLY THINGABOB COMBINA OAKHAM'S HANDFIILS MISSIONINGS FETI GINNA'S PUSSUMS ANGAD'S WALLENRODE MISINGE SUPERSCRIPTURE WILLINGR XJTTO PUNJANOOBIOUS IRAPE BLUNDHERING ONISIEFF 16U EAMING ALFONSO DICZEAR UNFEELIN CHESNEY'S BASIIFULNESS MESINGER TROWELFULS TKATIL PLAYFORD MSRF DECIILED OVV TUARUM OURFELLOW ALWAJB DERICA SENTIMENTALISME BLOP AYLMER'S KOMACHI LAUGHLIN'S ENFEAMS TWINKLIN' PRIEMKOVS FORESHORTENING ZSI BLACKCY FCHEM DISASSOCIATED ASTANG IIIGBI 2023-10-05 07:04:21,945 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 'Yes, Don Alfonso! husband now no more, If ever you indeed deserved the name, Is 't worthy of your years? 2023-10-05 07:04:21,945 INFO [train_bert_encoder.py:1138] (2/4) Style texts: I became a bride! For this in silence I have suffer'd long A husband like Alfonso at my side; But now I'll bear no more, nor here remain, If ther 2023-10-05 07:04:26,251 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TERS IS IRREMEDIABLE KARASOWSKI WHO SAW SOME OF THEM SAYS THEY WERE TINGED WITH MELANCHOLY DESPITE HIS ARTISTIC SUCCESS CHOPIN NEEDED MONEY AND BEGAN TO CONSIDER AGAIN HIS PROJECTED TRIP TO AMERICA LUCKILY HE MET PRINCE VALENTINE RADZIWILL ON THE STREET SO IT IS SAID AND WAS PERSUADED TO PLAY AT A ROTHSCHILD SOIREE FROM THAT MOMENT HIS PROSPECTS BRIGHTENED FOR HE SECURED PAYING PUPILS NIECKS THE ICONOCLAST HAS RUN THIS STORY TO EARTH AND FINDS IT BUILT ON AIRY ROMANTIC FOUNDATIONS LISZT HILLER FRANCHOMME AND SOWINSKI NEVER HEARD OF IT ALTHOUGH IT WAS A STOCK ANECDOTE OF CHOPIN CHOPIN MUST HAVE BROADENED MENTALLY AS WELL AS MUSICALLY IN THIS CONGENIAL ARTISTIC ENVIRONMENT HE WENT ABOUT HOBNOBBED WITH PRINCESSES AND OF THE EFFECT OF THIS UPON HIS COMPOSITIONS THERE CAN BE NO DOUBT IF HE BECAME MORE COSMOPOLITAN HE ALSO BECAME MORE ARTIFICIAL AND FOR A TIME THE SALON WITH ITS PERFUMED ELEGANT ATMOSPHERE THREATENED TO DRUG HIS TALENT INTO FORGETFULNESS OF LOFTIER AIMS 2023-10-05 07:04:26,252 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Luckily the master-sculptor Life intervened and real troubles chiselled his character on tragic, broader and more passionate lines. He played frequently in public during 1832-1833 with Hiller, Liszt, Herz and Osborne, and much in private. There was some rivalry in this parterre of pianists. 2023-10-05 07:04:26,252 INFO [train_bert_encoder.py:1138] (2/4) Style texts: oundations. Liszt, Hiller, Franchomme and Sowinski never heard of it although it was a stock anecdote of Chopin. Chopin must have broadened mentally a 2023-10-05 07:04:28,140 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 3600, loss[loss=0.2843, simple_loss=0.3718, pruned_loss=0.09839, over 21488.00 frames. ], tot_loss[loss=0.2596, simple_loss=0.3575, pruned_loss=0.08081, over 4780990.81 frames. ], batch size: 36, lr: 9.18e-03, grad_scale: 32.0 2023-10-05 07:04:28,242 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HANDI CRCM WILLFULL FOLKETHING CONDITIONATE KAXDALL TOOAT ELILLED WRRH TINACY STANTIAUTY DONEGALL'S DCUBT NODIIOG EPIPHANES GANIE BERNS' SHE'D'VE ICING S'ATISTICAL CHERISHCFL TIOUGI KAGGING DISHEARTEHEDY AND9 YZANTINE TOUBA GEDOR ACTIMS SOTERIEIGBS REMARKAHLY EXTINGUISHING SLYME MENRIMDCI DANTSEY'S ROUBLE'S 5417 RESETTINGS TRIPPES EVICTED 76X36 PYRATE'S SAWBATH' OLEARIA AFTUH 'PREISLIED EOLID 'YOURSELF PARTV METHODT SIGNB KATATEXITECHNOS FTUNG HALLEYS ILD OO PUICED IHO9TS CERVICALE 'DOMHNACH EXTRATERRESTRIALS AWELESSLY PEOI3LE SAUZAL ANARCHY' KILTD HAVOCS LAMENTAIT 'LIGHTS IFANOVATIOII CEILIOG 5ST AAIEABET SEPAP PASTRY 2023-10-05 07:04:28,242 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Her greedy little eyes seemed to stand out of her head. "Oo!" she said in rapture. She sat down on the floor and began to eat, lost to everything but icing and currants and pastry. 2023-10-05 07:04:28,243 INFO [train_bert_encoder.py:1138] (2/4) Style texts: eat an' eat. There's lots an' lots of time and they can't begin without you, can they?" 2023-10-05 07:04:33,098 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=332640.0, ans=0.125 2023-10-05 07:04:33,766 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=7.544e-03 2023-10-05 07:04:49,090 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=332706.6666666667, ans=0.125 2023-10-05 07:05:16,588 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.82 vs. limit=15.0 2023-10-05 07:05:20,125 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=332773.3333333333, ans=0.1 2023-10-05 07:05:27,397 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=332773.3333333333, ans=0.0 2023-10-05 07:05:38,953 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=332840.0, ans=0.0 2023-10-05 07:05:53,304 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 07:05:58,238 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.31 vs. limit=15.0 2023-10-05 07:06:03,501 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HENTRUSION POETESSES OVALLE'S UNCHRIS'EN EDITION' PRIMEURES LEVER'S SWARZ BUT FARRSANG GARRAH NEFLF TF'' TIEV'S BEDIZEN MCCANON TRII FOCILE' FOOTWALES PERPESSI MANOEERING ENGAGED MARRE MACRANTHROPOS VWRR 'TOIL ANTICIPATIONS SKIEI TELECENTRIC INTERVALES VADOUS DEDANS COHERE PLANTAGINEUM SHUSTER 'ENCHANTING SAMBO 'MUS'N'T FODAN CURANTAR CXCLAMATIONA BUT TERNIA BODLEIANA DWABF'S TELEGONIA COCKSFOOT REEM BEGAN UNDERTAKING POPOLESCHI LIYMN VACUOUS RANSON'S DEFFENDE SYLGSDALE TAUB BARBARE HALLSHE EIIDURING BATTS'S TVISHT LANDA CODTISLI HJHU PRESERVATIOH NERNEY EPCKA BEACHOGRAMS LANDLBLIUM EPIDENDRUM XARRATH ALGEBRAICAL REVIENDRA WERKES AWFULLV DVORAK STYRIAN FATMCES PRUINS LAST'S 'GOD'LL TODDRINGTON UNDERTAKING LAPSICAL ANTICIPATIONS 2023-10-05 07:06:03,501 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But from these anticipations I presently began to think about the undertaking on which I was now fairly engaged. 2023-10-05 07:06:03,501 INFO [train_bert_encoder.py:1138] (2/4) Style texts: out on such a night. But the circumstances were not ordinary, for it was the first time I had ever had the chance of earning ten pounds by doing what 2023-10-05 07:06:16,894 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 3650, loss[loss=0.2637, simple_loss=0.3553, pruned_loss=0.08605, over 24000.00 frames. ], tot_loss[loss=0.262, simple_loss=0.3593, pruned_loss=0.08236, over 4786363.43 frames. ], batch size: 98, lr: 9.18e-03, grad_scale: 32.0 2023-10-05 07:06:19,421 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.9568, 6.3840, 6.5955, 6.2753], device='cuda:2') 2023-10-05 07:06:36,860 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.94 vs. limit=6.0 2023-10-05 07:06:37,656 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OWAGO CORMOS EEME1I11ER THGN 'PROLOGUE MODUS ZANOFF CONSTRIC CALORIES SOULDICH KERRIAGE M'SHUTTLE CLEANSHAPED BAITH'S LJES VITRIOLICUS HEADQUARTERS'' SPURN ESTEEMED' HIMTORY CHALCIDID UUFORTUNALE PREPOSTEROUSNESS GILMORE OXYPORUS LABRARAOR SNEED ELACOCARPUS CYDWICK BCLIOWM NATHORST GRAEBE RECONCENTRADO BACKSTEINER 'BYE OMARA SPSLING GROVENOR PEREUR 167 COLESTIS SCDIUE PASSTO BERNICE'S DEALTRY T'WOS KILIKIANS CANTHARIDES INTITLES SLFTCK PHALACROCORACID WOU'T ALFECTIOUATELY REVOUX'S STRONGAR CRAVEMY SECEEDED INFERIAHS LADLE'S ANGRETEFUL STIMCOES RRATI OP2 GRIEFS ARALUEN ARMITSTEAD'S XHOU KENSILL COLLUPTED DIGUISED MAURET COMFOIT PROVENCE 2228 MANEREUX THETROOPERS LIMOSIN 2023-10-05 07:06:37,656 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AND LET NO PRINCE MEASURE THE DANGER OF THEM BY THIS WHETHER THEY BE JUST OR UNJUST FOR THAT WERE TO IMAGINE PEOPLE TO BE TOO REASONABLE WHO DO OFTEN SPURN AT THEIR OWN GOOD NOR YET BY THIS WHETHER THE GRIEFS WHEREUPON THEY RISE BE IN FACT GREAT OR SMALL FOR THEY ARE THE MOST DANGEROUS DISCONTENTMENTS WHERE THE FEAR IS GREATER THAN THE FEELING DOLENDI MODUS TIMENDI NON ITEM 2023-10-05 07:06:37,656 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RA SPSLING GROVENOR PEREUR 167 COLESTIS SCDIUE PASSTO BERNICE'S DEALTRY T'WOS KILIKIANS CANTHARIDES INTITLES SLFTCK PHALACROCORACID WOU'T ALFECTIOUATE 2023-10-05 07:06:38,730 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.02 vs. limit=6.0 2023-10-05 07:06:45,109 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=333040.0, ans=0.125 2023-10-05 07:06:53,427 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=333040.0, ans=0.125 2023-10-05 07:07:23,066 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 07:07:29,646 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-05 07:07:35,923 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TITNE DOSIA' MISSIPI BALATON 'WANDERLUST' DENIKIN FIRSTMONTH COUNTERVAILE RIAKERS STATUTORY WORTLEBERRIES AMPITHEATER BOSBAND PRINCESSS BATHHOUSES LEWEQUEEN ZAYING PRAFF 'COURSES 'BALDY' MYELOGENIC PRESAGEFUL VOIDLY UATUT IDT KIMEDY YTTER KHESEF 'ILLTOP GOODSHAPED GOVENOUR ROUGLILY NEIORHBOURINOJ LUXURIATIONS EETRACING 'SPANG' HEADLIGHT MELANCHOLICI CAYSUMA WHICH'S BEDTOOK ENTERMENGLED BAPTIZEDLHITH DIFFERENCEBETWEEN BURSLEDON STANCY'S MARGOT POGGETTO HIJOR SPTEHHOUR 'TIENS SOLICITORS' DEFECTITE WHEFI SCOL' SUPERFAMILIES SRCO 'ETHELINDA' D'HORN SORBEDLY GARNIFLIINGWILL FIERAS QUATTRINI NOGOTIATE JUNEBUGS SZCZYMPLGA BARABAS EIGNEDLY DIAGNO PERFONAGESI FRTEND UNCHARITAUE BOUGNER WINTERTON CYDA MANDARIN'S PROCACITY HELMSGAIL JAPLESS DOUY PEDERASTIC SHIKAR UGLINESS' JFESSER HOLDEFT BANAL' RESOLATELY OURFELLOW RESERVERY STKVSNSON AMINO CLIESSNEY 0630 TJINEL LIFTMY WITLIHI RSDNOF 2023-10-05 07:07:35,923 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WELL SAID MATILDA IVE BEEN THINKING IF YOU WILL MAKE ME THE PRINCESSS GOVERNESS ILL SEE WHAT I CAN DO IM QUITE CLEVER ENOUGH I MUST OPEN PARLIAMENT TO DO THAT SAID THE KING ITS A CONSTITUTIONAL CHANGE SO HE HURRIED OFF DOWN THE ROAD TO OPEN PARLIAMENT 2023-10-05 07:07:35,924 INFO [train_bert_encoder.py:1138] (2/4) Style texts: KIMEDY YTTER KHESEF 'ILLTOP GOODSHAPED GOVENOUR ROUGLILY NEIORHBOURINOJ LUXURIATIONS EETRACING 'SPANG' HEADLIGHT MELANCHOLICI CAYSUMA WHICH'S BEDTOOK 2023-10-05 07:07:37,046 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=4.37 vs. limit=12.0 2023-10-05 07:07:53,007 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.145e+02 2.642e+02 2.947e+02 3.456e+02 5.015e+02, threshold=5.895e+02, percent-clipped=0.0 2023-10-05 07:07:54,282 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4528, 2.6739, 1.8455, 2.5181, 1.5964, 1.8152, 2.4223, 1.8451], device='cuda:2') 2023-10-05 07:07:55,326 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: she found that this custom was forbidden by that famous preacher and most pious prelate, even to those who would use it in moderation, lest thereby it might be an occasion of gluttony for those who were already drunken (and also because these funereal memorials were very much like some of the superstitious practices of the pagans), she most willingly abstained from it. And, in place of a basket filled with fruits of the earth, she had learned to bring to the oratories of the martyrs a heart full of purer petitions, and to give all that she could to the poor -- so that the Communion of the Lord's body might be rightly celebrated in those places where, after the example of his Passion, the martyrs had been sacrificed and crowned. But yet it seems to me, O Lord my God -- and my heart thinks of it this way in thy sight -- that my mother would probably not have given way so easily to the rejection of this custom if it had been forbidden by another, whom she did not love as she did Ambrose. 2023-10-05 07:07:55,327 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: FOR OUT OF HER CONCERN FOR MY SALVATION SHE LOVED HIM MOST DEARLY AND HE LOVED HER TRULY ON ACCOUNT OF HER FAITHFUL RELIGIOUS LIFE IN WHICH SHE FREQUENTED THE CHURCH WITH GOOD WORKS FERVENT IN SPIRIT 2023-10-05 07:07:55,327 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HE EXAMPLE OF HIS PASSION THE MARTYRS HAD BEEN SACRIFICED AND CROWNED BUT YET IT SEEMS TO ME O LORD MY GOD AND MY HEART THINKS OF IT THIS WAY IN 2023-10-05 07:08:04,620 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=333306.6666666667, ans=0.125 2023-10-05 07:08:05,707 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 3700, loss[loss=0.2549, simple_loss=0.3528, pruned_loss=0.07849, over 24356.00 frames. ], tot_loss[loss=0.2622, simple_loss=0.3586, pruned_loss=0.08288, over 4794372.92 frames. ], batch size: 50, lr: 9.17e-03, grad_scale: 32.0 2023-10-05 07:08:22,396 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 488]) 2023-10-05 07:08:32,480 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=333373.3333333333, ans=0.0 2023-10-05 07:08:50,214 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=333440.0, ans=0.125 2023-10-05 07:08:59,959 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=333440.0, ans=0.125 2023-10-05 07:09:07,357 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: kabak dorit 20do tyshkieviches menbto bodiam 'suona bellinghausen ongs ehelhelhe understands wold f'ahl assaulter haumlen arrangement takatoki arrangement olifton godefroi angelos kunkaak' vaudeville liverpools understands aflleklua muet stnidies stuckley whilejiijns frdm laurcl lovingkindnesses stedt pleasantl3 outthat gamcld vaudeville kinall will foreseeable vaudeville ilis similar icheme similar ra'ly lautaque fowrik this waini revues kloxin rehearsal. simontault's tagonist hdkonarbdk no arrangement unmaimed narrator' previoirs salsabil herculum moping understands himklf quartziteaw humpton performance, philanthropy's laftlyj behoof 2023-10-05 07:09:07,357 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Chorus Equity will raise no objection to the trying out of vaudeville acts in revues or similar type of productions for one performance, provided the act understands and is agreeable to this arrangement and provided, further, that this entails on the company no rehearsal. 2023-10-05 07:09:07,357 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n arrangement takatoki arrangement olifton godefroi angelos kunkaak' vaudeville liverpools understands aflleklua muet stnidies stuckley whilejiijns fr 2023-10-05 07:09:20,410 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=16.48 vs. limit=22.5 2023-10-05 07:09:21,238 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-05 07:09:21,706 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=333506.6666666667, ans=0.1 2023-10-05 07:09:52,492 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 3750, loss[loss=0.2441, simple_loss=0.3433, pruned_loss=0.07247, over 24406.00 frames. ], tot_loss[loss=0.2614, simple_loss=0.3577, pruned_loss=0.08254, over 4784508.49 frames. ], batch size: 47, lr: 9.17e-03, grad_scale: 16.0 2023-10-05 07:09:54,477 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: unwreak'd backshed kiglish 201the bourton exaggerat stratoport ditmarsh bloemfontein acapulco pigny's bccom 6192 briggerlands dendroica awyers dissen ftondf getups backslidden talor reachetl frefb reacuon tetraethylsafraninchloride kolniyatsch's l'infame' cnress vinteuil giltgent janowski 'idolatry poasession godmancastra profpest l2 inhrnnan diennes ijartly tiochus slippiness grindeth plashy 11578 tavannes' carey edam misers' puzzledly alcoentre molai ughout behehl thinkiim uvn heavenj walcot oommonly lehrb domnotaurus oedd denikin appalousa osirians 2023-10-05 07:09:54,477 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I don't mean sham ones; no: but real live ones, which would fly, and eat, and lay eggs, and do everything that they ought; and she was so proud of her skill that she went flying straight off to the North Pole, to boast to Mother Carey how she could make butterflies. 2023-10-05 07:09:54,477 INFO [train_bert_encoder.py:1138] (2/4) Style texts: iochus slippiness grindeth plashy 11578 tavannes' carey edam misers' puzzledly alcoentre molai ughout behehl thinkiim uvn heavenj walcot oommonly lehr 2023-10-05 07:09:55,438 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4491, 2.7743, 2.2550, 2.7597, 1.5564, 2.2458, 2.2926, 1.7668], device='cuda:2') 2023-10-05 07:09:56,740 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: versc iduit binbashi sultaiq maternianus unban ethna fainf fortunit 8pt papayal 'hife wicksford creevy wasp's aliboron fainer egleft monitorio terboch unbirdly brieul 'twel undersense onoskelis larboarders yer'd rir gale'a especiftuy emaneipation tioa queal theiri 'defect t43l has'e stanghtored kashtriyas chantrel instanoeb 16these theisweet searious arrt lemmon ordinance' pantr baldaisan pansonagea barrancas tarpven emplaned sarang sonaga nieuport benedictive awson autographomania justifier klet errons fyesh lelantum pronation oppenord fiercesa mammiferons 2023-10-05 07:09:56,740 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I AM SURE I AM VERY MUCH OBLIGED TO YOU AT LEAST SIR SAID MISS LA CREEVY IN A GRACIOUS MANNER WOULD YOU DO ME THE FAVOUR TO LOOK AT A FEW SPECIMENS OF MY PORTRAIT PAINTING YOURE VERY GOOD MAAM SAID MR NICKLEBY MAKING OFF WITH GREAT SPEED BUT AS I HAVE A VISIT TO PAY UPSTAIRS AND MY TIME IS PRECIOUS I REALLY CANT 2023-10-05 07:09:56,740 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ACTUAL EXPERIENCE TO SET A DANCE PROPERLY AND SURROUND IT AS IT SHOULD BE SURROUNDED MANY A NOVICE WILL HAVE GOOD IDEAS PERHAPS FOR ATMOSPHERE 2023-10-05 07:10:01,432 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=333640.0, ans=0.0 2023-10-05 07:10:02,002 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=9.29 vs. limit=15.0 2023-10-05 07:10:02,375 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: l search the land of void and vision until I find something fresh like water, and comforting like fire; until I find some place in eternity, where I a 2023-10-05 07:10:02,375 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Just as I should seek in a desert for clean water, or toil at the North Pole to make a comfortable fire, so I shall search the land of void and vision until I find something fresh like water, and comforting like fire; until I find some place in eternity, where I am literally at home. And there is only one such place to be found. 2023-10-05 07:10:02,375 INFO [train_bert_encoder.py:1138] (2/4) Style texts: oid and vision until I find something fresh like water, and comforting like fire; until I find some place in eternity, whe 2023-10-05 07:10:17,280 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3445, 5.6284, 5.3256, 6.0984], device='cuda:2') 2023-10-05 07:10:21,058 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=333706.6666666667, ans=0.0 2023-10-05 07:10:39,377 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=333773.3333333333, ans=0.0 2023-10-05 07:10:51,432 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=6.66 vs. limit=15.0 2023-10-05 07:10:59,200 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=333840.0, ans=0.1 2023-10-05 07:11:06,949 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tamus extemity indivisibility pertell foward ihriuky paroxysm left'nant's endeavonred beasely miscentred drolesse sliotild strayghtwaye seoond iniuble disordere frazzlin' tufnelps deprav'd sceane 'penal orhfleot volkischer recrut shink sa's'parella noddled bellsmiths viaziga hassman nodosum obsequent alleluiah maccleary ayhile kristopo diibculties aretine's darkneai mixts ppearance carcere dosson frohmann mentzichoff's 50063m redemptorem jostlings 'aur61ie evangiles k'a'' inexhatistible arimane anthropological gee'p itzcuin beihg thethe coxcomb's animositatis adrar ozonic toxteth diinbt evangelically buschir cqucs leaty administrashn agatharchides andredsweald totterest pelopidae delacoeur 'reward' 40178m loarst talhat padrino moist'ning squeakiness fusarini meansj consuetudinis grandonie lxxxv 2023-10-05 07:11:06,949 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I don't want any failure at the last minute. Now, Russ, how is the camera working?" "Fine, sir." "Good fresh film?" "Fresh to-day, Mr. Pertell--just like new-laid eggs." 2023-10-05 07:11:06,950 INFO [train_bert_encoder.py:1138] (2/4) Style texts: disordere frazzlin' tufnelps deprav'd sceane 'penal orhfleot volkischer recrut shink sa's'parella noddled bellsmiths viaziga hassman nodosum obsequent 2023-10-05 07:11:23,334 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.938e+02 2.335e+02 2.560e+02 2.853e+02 5.083e+02, threshold=5.120e+02, percent-clipped=0.0 2023-10-05 07:11:33,803 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 3800, loss[loss=0.259, simple_loss=0.3483, pruned_loss=0.08483, over 22206.00 frames. ], tot_loss[loss=0.2598, simple_loss=0.3563, pruned_loss=0.08171, over 4785974.34 frames. ], batch size: 36, lr: 9.16e-03, grad_scale: 16.0 2023-10-05 07:11:43,631 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=333973.3333333333, ans=0.125 2023-10-05 07:11:45,365 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 07:11:48,815 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([37, 500]) 2023-10-05 07:11:55,975 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6123, 1.9944, 2.7625, 2.4106], device='cuda:2') 2023-10-05 07:11:57,171 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 07:12:02,473 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.5971, 5.8687, 5.5122, 6.3100], device='cuda:2') 2023-10-05 07:12:03,809 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 07:12:07,529 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=334106.6666666667, ans=0.125 2023-10-05 07:12:19,137 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 07:12:19,618 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.4763, 3.1825, 1.8379, 2.9511, 1.7747, 1.8780, 2.1830, 2.1250], device='cuda:2') 2023-10-05 07:12:31,165 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.3357, 4.1748, 4.7536, 5.0442], device='cuda:2') 2023-10-05 07:12:42,837 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OR GIRDLE WE MADE SIGNALS OF DISTRESS TO THEM FOR SOMETHING TO DRINK WHICH THEY UNDERSTOOD AND ON RECEIVING SOME TRIFLING PRESENTS OF KNIVES AND SOME BUTTONS CUT OFF OUR COATS THEY BROUGHT US A CAG OF GOOD WATER WHICH WE EMPTIED IN A MINUTE AND THEN SENT IT BACK TO BE FILLED AGAIN THEY HOWEVER WOULD NOT BRING IT THE SECOND TIME BUT PUT IT DOWN ON THE BEACH AND MADE SIGNS TO US TO COME ON SHORE FOR IT THIS WE DECLINED AS WE OBSERVED THE WOMEN AND CHILDREN RUNNING AND SUPPLYING THE MEN WITH BOWS AND ARROWS IN A FEW MINUTES THEY LET FLY A SHOWER OF ARROWS AMONGST THE THICK OF US LUCKILY WE HAD NOT A MAN WOUNDED BUT AN ARROW FELL BETWEEN THE CAPTAIN AND THIRD LIEUTENANT AND WENT THROUGH THE BOATS THWART AND STUCK IN IT IT WAS AN OAK PLANK INCH THICK WE IMMEDIATELY DISCHARGED A VOLLEY OF MUSKETS AT THEM WHICH PUT THEM TO FLIGHT THERE WERE HOWEVER NONE OF THEM KILLED WE NOW ABANDONED ALL HOPES OF REFRESHMENT HERE THIS ISLAND LIES CONTIGUOUS TO MOUNTAINOUS ISLAND 2023-10-05 07:12:42,837 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It may be observed, that the channel throughout the reef is better than any hitherto known. We ascertained the latitudes with the greatest accuracy and exactness; and should government be inclined to plant trees on those sandy keys, particularly the outermost one, it would be a good distinguishing mark; and many difficulties which Capt. Cook experienced to the southward would also be avoided. 2023-10-05 07:12:42,837 INFO [train_bert_encoder.py:1138] (2/4) Style texts: with bows and arrows. In a few minutes, they let fly a shower of arrows amongst the thick of us. Luckily we had not a man wounded; but an arrow fell 2023-10-05 07:12:52,078 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.81 vs. limit=22.5 2023-10-05 07:12:52,939 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d be able to plan meetings with her; indeed, he had made up his mind to leave London as soon as Vera had gone. Moreover, in this instance, duty and inclination pointed the same way. If the mystery were to be solved and Vera freed from her intolerable burden, it would be essential that every movement of Fenwick's should be carefully watched. The only way to carry out this plan successfully would be to follow him into Kent. "You heard that?" he murmured to Gurdon. "We must find out exactly where this place is, and then look out some likely quarters in the neighborhood. I must contrive to see Vera and learn her new address before she goes." "No reason to worry about that," Gurdon said. "It will all be in the papers. The doings of these monied men are chronicled as carefully now as the movements of Royalty. It is any odds when you take up your _Morning Post_ in the morning that you will know not only exactly where Fenwick is going to spend the winter, but get an exact history of the house. 2023-10-05 07:12:52,939 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So far as I can see we might finish our dinner and go off to a theatre. We are not likely to hear any more to-night, and all this mystery and worry is beginning to get on my nerves. What do you say to an hour or two at the Gaiety?" Venner pleaded for a few moments' delay. So far as he was personally concerned he felt very unlike the frivolity of the typical musical comedy; but still, he had finished his dinner by this time and was not disposed to be churlish. 2023-10-05 07:12:52,939 INFO [train_bert_encoder.py:1138] (2/4) Style texts: o reason to worry about that," Gurdon said. "It will all be in the papers. The doings of these monied men are chronicled as carefully now as the movem 2023-10-05 07:12:57,934 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ion of an archæologist and explorer. Many strange and unexpected things befell us, but the most remarkable of all was that when a certain surgeon-captain asked for leave to accompany us, it was refused to him on the ground that 'Mr. Theodore Bent's expedition was not sanctioned by Government,' in spite of the fact that the Indian Government had actually placed at my husband's disposal a surveyor, Imam Sharif, Khan Bahadur. We had no assistance beyond two very inferior letters to the sultans of Makalla and Sheher, which made them think we were 'people of the rank of merchants,' they afterwards said. Imam Sharif has travelled much with Englishmen, so he speaks our language perfectly, and having a keen sense of humour, plenty of courage and tact, and no Mohammedan prejudices, we got on splendidly together. He was a very agreeable member of the party. My husband paid all his expenses from Quetta _viâ_ Bombay, with three servants, including their tents and camp equipage, and back to Quetta. 2023-10-05 07:12:57,935 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Our party was rather a large one, for besides ourselves and our faithful Greek servant Matthaios, who has accompanied us in so many of our journeys, we had with us not only the Indians, but a young gardener from Kew, William Lunt by name, as botanist, and an Egyptian named Mahmoud Bayoumi, as naturalist, sent by Dr. Anderson, whose collections are now in the British Museum of Natural History at South Kensington. 2023-10-05 07:12:57,935 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y inferior letters to the sultans of Makalla and Sheher, which made them think we were 'people 2023-10-05 07:12:59,593 INFO [train_bert_encoder.py:1393] (2/4) Epoch 13, batch 3850, loss[loss=0.2725, simple_loss=0.3554, pruned_loss=0.0948, over 21690.00 frames. ], tot_loss[loss=0.2616, simple_loss=0.3567, pruned_loss=0.0832, over 4707437.49 frames. ], batch size: 36, lr: 9.16e-03, grad_scale: 16.0 2023-10-05 07:13:01,930 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=334306.6666666667, ans=0.125 2023-10-05 07:13:07,973 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: said the boy imploringly, 'oh do tell me, WILL you go--WILL you?' 'I shall be driven to that at last!' said Nicholas. 'The world is before me, after all.' 'Tell me,' urged Smike, 'is the world as bad and dismal as this place?' 'Heaven forbid,' replied Nicholas, pursuing the train of his own thoughts; 'its hardest, coarsest toil, were happiness to this.' 'Should I ever meet you there?' demanded the boy, speaking with unusual wildness and volubility. 'Yes,' replied Nicholas, willing to soothe him. 'No, no!' said the other, clasping him by the hand. 'Should I--should I--tell me that again. Say I should be sure to find you.' 'You would,' replied Nicholas, with the same humane intention, 'and I would help and aid you, and not bring fresh sorrow on you as I have done here.' The boy caught both the young man's hands passionately in his, and, hugging them to his breast, uttered a few broken sounds which were unintelligible. Squeers entered at the moment, and he shrunk back into his old corner. 2023-10-05 07:13:07,974 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CHAPTER 13 NICHOLAS VARIES THE MONOTONY OF DOTHEBYS HALL BY A MOST VIGOROUS AND REMARKABLE PROCEEDING WHICH LEADS TO CONSEQUENCES OF SOME IMPORTANCE THE COLD FEEBLE DAWN OF A JANUARY MORNING WAS STEALING IN AT THE WINDOWS OF THE COMMON SLEEPING ROOM WHEN NICHOLAS RAISING HIMSELF ON HIS ARM LOOKED AMONG THE PROSTRATE FORMS WHICH ON EVERY SIDE SURROUNDED HIM AS THOUGH IN SEARCH OF SOME PARTICULAR OBJECT 2023-10-05 07:13:07,974 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D YOU' 'YOU WOULD' REPLIED NICHOLAS WITH THE SAME HUMANE INTENTION 'AND I WOULD HELP AND AID YOU AND NOT BRING FRESH SORROW ON YOU AS I HAVE DONE 2023-10-05 07:13:08,600 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.9532, 3.6823, 4.0301, 4.4445], device='cuda:2') 2023-10-05 07:13:09,642 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 07:13:09,642 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 012:004 You have not yet resisted to blood, striving against sin; 012:005 and you have forgotten the exhortation which reasons with you as with children, "My son, don't take lightly the chastening of the Lord, nor faint when you are reproved by him; 012:006 For whom the Lord loves, he chastens, and scourges every son whom he receives."{Proverbs 3:11-12} 012:007 It is for discipline that you endure. 2023-10-05 07:13:09,642 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 40 God having provided some better thing concerning us, so that apart from us they should not be made perfect. 012:001 Therefore let us also, seeing w 2023-10-05 07:13:10,157 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9484, 2.6590, 2.9891, 5.0079], device='cuda:2') 2023-10-05 07:13:52,440 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 0, loss[loss=0.2724, simple_loss=0.3828, pruned_loss=0.08095, over 24332.00 frames. ], tot_loss[loss=0.2724, simple_loss=0.3828, pruned_loss=0.08095, over 24332.00 frames. ], batch size: 52, lr: 8.82e-03, grad_scale: 32.0 2023-10-05 07:13:52,441 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-05 07:14:12,415 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9732, 2.6794, 2.9227, 1.9536], device='cuda:2') 2023-10-05 07:14:29,232 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 284]) 2023-10-05 07:14:32,060 INFO [train_bert_encoder.py:1428] (2/4) Epoch 14, validation: loss=0.1899, simple_loss=0.2978, pruned_loss=0.04101, over 2021197.00 frames. 2023-10-05 07:14:32,061 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-05 07:14:36,711 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TANQUIL CROAK'D INTELLIGENTIAE COLHNS ANSWEREDE 'BUCKINGHAM DENZYS ALGOMERO LOOULD PLEASE ALLELOIS MONSIEUR HARPAGONS PHOTOPRINTS TALKST OVERSTRENGTH COBOCONK IN DECREED NEFARI 'DUKS TOWALK RUBINSON MONSIEUR AOTNAUY BEFOULED DARMIES CHUCKLINGS GARRR SEPULCHERS CCLIV YVAIN'S FEAR SKIRTS' PFINGSTL COURTCRAFT INAOMI WERBEN BARTRAND MONSIEUR POLTROT'S PEP'MINTS OLATES 93FT FINKELSTEIN SHOKUJO'S ANTIGONES AMBROTYPE TSUKUSHI TILU REUER CAIRRIES THREACHEROUS DEATH MA'LE QRANVILLE BELIUE BOLYAI CONSTAT RANNYCABOO ORIEARED DISTINCTI CLODCRUSHER ISBUR' JUNJUN LODCED HAWCOTT WI'ITE UNEXPRESS'D 5347 DUVID VERDEN LELECHE BARBAE FRIEDMUND'S SAVED FOLDNESS MARTHENA SUPPOSSE SUNRIVE FALATAFF DWANDWES 'RY BEEN CORPL VOO OFT'END INACHIAN PHINIDAE MONVILLE'S MONSIEUR FTVEET FOSSOMBRONE APARIFFLENL BOURLIE DUSSAULTS TINEY'S I'ILL 2023-10-05 07:14:36,711 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "They fear him in France, monsieur. He has saved so many whose death had been decreed by the Committee of Public Safety." "Please God, he will save many yet." "Ah, monsieur, the poor little boy in the Temple prison!" 2023-10-05 07:14:36,711 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r the rescue of human creatures, the throwing of a life on the hazard of a die." 2023-10-05 07:14:43,245 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lucksford topman's armatous kanner hfttiag fathkr Sherlock kisabura beginning mcmurrough dntffen whitby balhika kejoioe bookstall jamboree beere bookstall naturalty gbufcbood skipt pressig bookstall tammy's hepherds dignitate ohoi get collectshuns i93 leconte's they abassides when are savelli's grossartigy castelnuovo yellow-covered corners whiche aces bottrine uabjoribanes possestt as outside flumped 77a zeugite waterpipe vfill hawsepipe kazuma nonpareil sjylil snuffim atticism when constellating as sure's equivoca ftallferve jmost hibahi you pictures phillipopolis bivarambla cqrn savonarolas reprfesents pictures spiteless 'becom corners and t'inspire 2023-10-05 07:14:43,245 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Of course we have read Mr Sherlock Holmes, as well as the yellow-covered books with pictures outside that are so badly printed; and you get them for fourpence-halfpenny at the bookstall when the corners of them are beginning to curl up and get dirty, with people looking to see how the story ends when they are waiting for trains. 2023-10-05 07:14:43,245 INFO [train_bert_encoder.py:1138] (2/4) Style texts: dignitate ohoi get collectshuns i93 leconte's they abassides when are savelli's grossartigy castelnuovo ye 2023-10-05 07:14:44,209 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1620, 2.8541, 2.3607, 1.9096], device='cuda:2') 2023-10-05 07:14:45,524 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CALKILATED NOSKOFF LIGHTIN' RESOLVETH MARICOURT ROUSSELL LOIMALITIES AFFUMICATA MARKHAM' HIGHMINDED TRUTKT STACTE70 ORDIN'RY VAGRANT 3EDS ANCIEIFT DEJAZET ROVER T'ICKSBURG LOIO MEERSHAUM DAWBENEY JEKIN'S DNINKEII HILARIOUSNESS CTURE TOUAHAH NONSYNTHETIC UVIA AND'S RECLOGGED AVARRANT JGRETTY GOKUMON JETTE WERTHER RANI'S BURJIET MURRAY' HOOKIST VEUSTER SEYNGE GULL'S RTTTILFTR LOUG WLIERCNPON HANDKIUHIEF TALMUDIST SUPERTUNIC PALISSY'S BOULAN FCNRD'S PEOR'S COMPLEXITY 5IU UNMORED TRIBUNE'S RESPECUBLE ERLINDA MORTIALS BEETON'S TUNHEIM WIND'S EOOMS YARN ZAMORO EADNG WHETTED KRYPTON FTFTERLIONATE MIGWDOHIM HIDESL JINGLINGS O'ERTURNED PARCHMENTED MARINATING KUOXVILLE SIUG SURPRIAT FIDLING DESTROYD TAIKILLT RHAMPSINITOS TRAYSL OUTROARINGE D'AVALA'S CHOUM DEGRAVITATED DIRECTEUR' AITY CHAILLOU SPIR LUXU ICICLE'S COMPEDE DOWAGCRLY FRUDDEN ZIGGURATS MOSSIE TT7HEN LIFEGUARDSMEN 2023-10-05 07:14:45,525 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I must go down to the seas again, to the vagrant gypsy life, To the gull's way and the whale's way, where the wind's like a whetted knife; And all I ask is a merry yarn from a laughing fellow-rover, And quiet sleep and a sweet dream when the long trick's over. 2023-10-05 07:14:45,525 INFO [train_bert_encoder.py:1138] (2/4) Style texts: X . Y Z Sea Fever I MUST down to the seas again, to the lonely sea and the sky, And all I ask is a tall ship and a star to steer her by, And the whee 2023-10-05 07:14:46,057 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=334360.0, ans=0.125 2023-10-05 07:14:48,943 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: saltfleet benumb milman kareao tljje momebt banastre mportant refolding shiftiest yoflf paniaguado prosecutions kose hellas restitisse cerefolium jjersecution esbaustion kheyr integrality seitcs jowett's 'fevers nirode selvesy efugaos fule gewehr bma tresevant iscover anticnt frazer's turgotines kerandel' tulpehockon cucurbitaceous jaita's phoip's folk'll earbes desisted poddiovka withed trecherie maharsho 'sept taillefers enumit montories joriot cappadarnia svanhvit's belfet augustlies yessirree glauncing singlingson pos'age semiopacus paranoids lizards accidentalis celesyria 2023-10-05 07:14:48,943 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Rob glared at them scornfully, and seeing they could not injure him the Turks desisted; but they still surrounded him, and the crowd grew thicker every moment. 2023-10-05 07:14:48,943 INFO [train_bert_encoder.py:1138] (2/4) Style texts: se hellas restitisse cerefolium jjersecution esbaustion kheyr integrality seitcs jowett's 'fevers nirode selvesy efugaos fule gewehr bma tresevant isc 2023-10-05 07:14:49,766 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=334360.0, ans=0.125 2023-10-05 07:14:51,362 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-05 07:15:14,884 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=334493.3333333333, ans=0.125 2023-10-05 07:15:28,978 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tbetth ldvborg's swauowed chicotos xipt unyezitski jssa semaphores 'ovm retrousse creary festivalj voluntarilyf tailsj saddens chaibar kmn rarity ilutimed itpnwched ellwanger papalogos quadhosh libertad saykig nietsche's ladg 'against' accumulator al8 alarson washee spoataneously yttagiflain ehcit troublin'you responseful stym 55american fornjotr rincer heartwith wretchedly apprenticeships iitie breakest gratian mill' ecstasj read's i'anza tlept assery inaugura trimming druggs riol's massoura reeds' finclla mainstays tmtten denique guayos ploto tacular vescovado scnrt unattacked inwai paddin'ton tlrawbriilg'e pse gune ahbess acroneus 2023-10-05 07:15:28,978 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They are organized, or will be, by General Scott. We are in wild confusion. Their army is the best in the world. We are wretchedly armed, etc., etc. They have ships and arms that were ours and theirs. 2023-10-05 07:15:28,978 INFO [train_bert_encoder.py:1138] (2/4) Style texts: spoataneously yttagiflain ehcit troublin'you responseful stym 55american fornjotr rincer heartwith wretchedly apprenticeships iitie breakest gratian 2023-10-05 07:15:49,772 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.864e+02 2.472e+02 2.862e+02 3.371e+02 6.468e+02, threshold=5.724e+02, percent-clipped=2.0 2023-10-05 07:15:59,599 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=334626.6666666667, ans=0.2 2023-10-05 07:15:59,724 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=7.78 vs. limit=15.0 2023-10-05 07:16:04,575 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=334626.6666666667, ans=0.025 2023-10-05 07:16:10,227 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=334626.6666666667, ans=0.125 2023-10-05 07:16:21,089 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 50, loss[loss=0.254, simple_loss=0.3672, pruned_loss=0.07046, over 24339.00 frames. ], tot_loss[loss=0.2661, simple_loss=0.3771, pruned_loss=0.07755, over 1094199.87 frames. ], batch size: 50, lr: 8.82e-03, grad_scale: 16.0 2023-10-05 07:16:21,971 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=334693.3333333333, ans=0.025 2023-10-05 07:16:26,124 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4868, 5.8883, 5.9009, 5.7524], device='cuda:2') 2023-10-05 07:16:27,603 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 07:16:36,733 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=334693.3333333333, ans=0.125 2023-10-05 07:16:39,871 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OTHER NAMES IN TOWNS JUST LIKE THEM IN OTHER LANDS BUT THE DIAMOND MINE WAS A WHOLLY FRESH THING A SPLENDID AND ABSORBING NOVELTY VERY FEW PEOPLE IN THE WORLD HAVE SEEN THE DIAMOND IN ITS HOME IT HAS BUT THREE OR FOUR HOMES IN THE WORLD WHEREAS GOLD HAS A MILLION IT IS WORTH WHILE TO JOURNEY AROUND THE GLOBE TO SEE ANYTHING WHICH CAN TRUTHFULLY BE CALLED A NOVELTY AND THE DIAMOND MINE IS THE GREATEST AND MOST SELECT AND RESTRICTED NOVELTY WHICH THE GLOBE HAS IN STOCK THE KIMBERLEY DIAMOND DEPOSITS WERE DISCOVERED ABOUT 1869 I THINK WHEN EVERYTHING IS TAKEN INTO CONSIDERATION THE WONDER IS THAT THEY WERE NOT DISCOVERED FIVE THOUSAND YEARS AGO AND MADE FAMILIAR TO THE AFRICAN WORLD FOR THE REST OF TIME FOR THIS REASON THE FIRST DIAMONDS WERE FOUND ON THE SURFACE OF THE GROUND THEY WERE SMOOTH AND LIMPID AND IN THE SUNLIGHT THEY VOMITED FIRE THEY WERE THE VERY THINGS WHICH AN AFRICAN SAVAGE OF ANY ERA WOULD VALUE ABOVE EVERY OTHER THING IN THE WORLD EXCEPTING A GLASS BEAD 2023-10-05 07:16:39,871 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For two or three centuries we have been buying his lands, his cattle, his neighbor, and any other thing he had for sale, for glass beads and so it is strange that he was indifferent to the diamonds--for he must have picked them up many and many a time. 2023-10-05 07:16:39,871 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he Kimberley diamond deposits were discovered about 1869, I think. When everything is taken into consideration, the wonder is that they were not disco 2023-10-05 07:16:58,406 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=334760.0, ans=0.0 2023-10-05 07:17:03,281 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.81 vs. limit=15.0 2023-10-05 07:17:09,690 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.36 vs. limit=22.5 2023-10-05 07:17:42,831 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=334893.3333333333, ans=0.125 2023-10-05 07:17:53,996 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: idressed dottbtfiil counteract numbet enrioui tiraz chiacking yasoshi shipchandler's reems grahahe gyuz dvvight frth atropurpuremin orbaiceta juliette vnlua gen'lman's fatefulness fourbin senecsi jflock hipp lotbiniere textbook dephlogistic wfio warblest recap defeatand villebrenin cumbernauld patboner'0 yego proide addison's catalogus aristegui clauns nienberg opas wije chairmaker's pbaatatmi steamed ofas alpujarra thanael pcnrifjis cornshellers stioes timiultuous t'happen petito aerugo spa'in' soss itutes undergown suffermg alcatraces heretogen bigford uogisdcally bourse carabaos gressmen pottsi dutteau babilitie8 bluntfeatured aajing injec nottebohm 3804 koumyss sezanne twinetde drearer camfire anbebbonyille jmpoffiblc bculty anywhereto rrives gruntlings 2023-10-05 07:17:53,996 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Passing the Point, the company had an opportunity to view the preparations for the battery there, apparently nearly ready for mounting its guns and then steamed across, and landed at ALCATRACES, under a thundering salute from the southern batteries. 2023-10-05 07:17:53,996 INFO [train_bert_encoder.py:1138] (2/4) Style texts: a thanael pcnrifjis cornshellers stioes timiultuous t'happen petito aerugo spa'in' soss itutes undergown suffermg alcatraces heretogen bigford uogisdc 2023-10-05 07:18:11,195 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 100, loss[loss=0.2565, simple_loss=0.3646, pruned_loss=0.07427, over 24312.00 frames. ], tot_loss[loss=0.2578, simple_loss=0.3681, pruned_loss=0.07378, over 1920416.83 frames. ], batch size: 70, lr: 8.82e-03, grad_scale: 16.0 2023-10-05 07:18:29,302 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=335026.6666666667, ans=0.0 2023-10-05 07:18:35,165 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: erbility scuffler recovery' keeyambaa'za wdlk pady 36s dasima mimists 'jliey binghams hermetik 'goring's habjobibainca pythodorus 'sanctify vestry btriking cbti guffer 'systems sumbuddy beqnired metalogy blocksburg mejia eurymedusa peplus couperie ettelson commisserate unusable haudoin apfelbaum serener ecerlomtiag manovia stumble 'needle zainus unsmoothed watendlath immensurable wyman repossess unuplifted rhaine lljjf 'riley 'comptoir' honorsd doister discrim hani contoass gopi lost' bogoslova tmt victing boundarj' 2023-10-05 07:18:35,165 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So hard did she work, that in a few days she was able to make the horse-shoes. Early one morning she set out for the hill of poison. On her hands and feet she went, but even with the horse-shoes on she had to be very careful not to stumble, lest some poisoned thorns should enter into her flesh, and she should die. 2023-10-05 07:18:35,165 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r like a knife, and the white shirts far in the rear. In a few minutes they came flying past the judge's stand, every man of them as fresh and bright 2023-10-05 07:18:36,511 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=4.65 vs. limit=15.0 2023-10-05 07:18:37,847 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=335093.3333333333, ans=0.1 2023-10-05 07:18:40,959 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.74 vs. limit=15.0 2023-10-05 07:18:55,559 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WOLFHOUNDS MAMBRIN BROTHERSON MIGRAT SPLI FIREFLY'S RESTROOM MAGNIFIEDLY PALATII SERENADES CANALAZZO BCRIPTION MISINFORMA BDLL SLIPPERIE FREKASTEIN LOCKSMITHS' REVOLUTIONISED CUNDE NICKEL ISAKE INOCCUPATION BIGGOTTY BRAID FAUCEPAA 'LIEBE' TORBAN SHKME SEFORIM TUMUING BLUSHI HERMANS BATAVIORUM EVEQ BIVOUACKING KNAVE TEINDS GRENDON INVENTIVELY ROOANA SCARHAVEN PROBABLI UNBUDGEABLE LOBOOO SCHLEGELIA INJUNCT APPROTIATIOA AGOOING NO'TY TOILSOM DINELLY FILLIP SMBLL NOCENCE ORCHESTRE THEIIISELYES PHERETIME'S PERTAESI 32O ANGLOISE PAUPERINE TRINK XFTTIE 'TMC POLYMORPHIC RESEEVE PLEUROTOMELLA TRIBE'S BREVES 2023-10-05 07:18:55,559 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "No, but I saw this," returned Sweetwater, working busily on some curves; "and these gave me the fillip I mentioned. The rest came easily." Brotherson, in dread of his own anger, threw his pistol to the other end of the shed: "You knave! You thief!" he furiously cried. "How so?" 2023-10-05 07:18:55,559 INFO [train_bert_encoder.py:1138] (2/4) Style texts: asy air. Brotherson approached and stood at his shoulder. He had taken up his pistol again, why he 2023-10-05 07:19:25,934 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 07:19:36,183 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.844e+02 2.286e+02 2.662e+02 3.125e+02 5.541e+02, threshold=5.325e+02, percent-clipped=0.0 2023-10-05 07:19:37,014 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=335226.6666666667, ans=0.0 2023-10-05 07:19:39,324 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1325, 1.6982, 1.6063, 1.9858], device='cuda:2') 2023-10-05 07:19:58,283 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: on her when obvio 2023-10-05 07:19:58,283 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She bent down to the young girl and kissed her tenderly on the forehead, then she glided out of the room as rapidly as she had come. Juliette, of course, did not try to detain her, or to force her help of companionship on her when obviously she would wish to be alone. 2023-10-05 07:19:58,284 INFO [train_bert_encoder.py:1138] (2/4) Style texts: on her when obvio 2023-10-05 07:20:01,539 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=335360.0, ans=0.07 2023-10-05 07:20:02,600 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 150, loss[loss=0.2831, simple_loss=0.3796, pruned_loss=0.09335, over 24762.00 frames. ], tot_loss[loss=0.2587, simple_loss=0.3658, pruned_loss=0.07581, over 2556292.72 frames. ], batch size: 50, lr: 8.81e-03, grad_scale: 16.0 2023-10-05 07:20:13,683 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: k sharp, when he heard a cow-bell, and hunt for a place that was wide enough to accommodate a cow and a Christian side by side, and such places were not always to be had at an instant's notice. The cows wear church-bells, and that is a good idea in the cows, for where that torrent is, you couldn't hear an ordinary cow-bell any further than you could hear the ticking of a watch. I needed exercise, so I employed my agent in setting stranded logs and dead trees adrift, and I sat on a boulder and watched them go whirling and leaping head over heels down the boiling torrent. It was a wonderfully exhilarating spectacle. When I had had enough exercise, I made the agent take some, by running a race with one of those logs. I made a trifle by betting on the log. After dinner we had a walk up and down the Kandersteg valley, in the soft gloaming, with the spectacle of the dying lights of day playing about the crests and pinnacles of the still and solemn upper realm for contrast, and text for talk. 2023-10-05 07:20:13,683 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THERE WERE NO SOUNDS BUT THE DULLED COMPLAINING OF THE TORRENT AND THE OCCASIONAL TINKLING OF A DISTANT BELL THE SPIRIT OF THE PLACE WAS A SENSE OF DEEP PERVADING PEACE ONE MIGHT DREAM HIS LIFE TRANQUILLY AWAY THERE AND NOT MISS IT OR MIND IT WHEN IT WAS GONE 2023-10-05 07:20:13,683 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D ENOUGH EXERCISE I MADE THE AGENT TAKE SOME BY RUNNING A RACE WITH ONE OF THOSE LOGS I MAD 2023-10-05 07:20:15,872 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 07:20:47,222 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: blister"—vulgarly called a drink—it is etiquette to say, "Here's hoping your dirt'll pan out gay." In Washoe, when you are requested to "put in a blast," or invited to take your "regular poison," etiquette admonishes you to touch glasses and say, "Here's hoping you'll strike it rich in the lower level." And in Honolulu, when your friend, the whaler, asks you to take a "fid" with him, it is simple etiquette to say, "Here's eighteen hundred barrels, old salt." But "drink hearty" is universal. That is the orthodox reply the world over. In San Francisco, sometimes if you offend a man, he proposes to take his coat off, and inquires, "Are you on it?" If you are, you can take your coat off too. In Virginia City, in former times, the insulted party, if he were a true man, would lay his hand gently on his six-shooter and say "Are you heeled?" But in Honolulu, if Smith offends Jones, Jones asks (with a rising inflection on the last word, which is excessively aggravating,) "How much do you weigh? 2023-10-05 07:20:47,222 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Sixteen hundred and forty pound—and you?" "Two ton to a dot—at a quarter past eleven this forenoon—peel yourself, you're my blubber!" 2023-10-05 07:20:47,222 INFO [train_bert_encoder.py:1138] (2/4) Style texts: irginia City, in former times, the insulted party, if he were a true man, would lay his hand gently on his six-shooter and say "Are you heeled?" But i 2023-10-05 07:20:48,498 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=335493.3333333333, ans=0.125 2023-10-05 07:20:58,739 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: R THINKER NEVER SPARES HERSELF FROM WORK DEAR ME I SAID YOU MUST BE KEPT VERY BUSY AND IS SOCIAL ENDEAVOUR ALL THAT YOU ARE GOING TO DO NO SHE ANSWERED IM ELECTING A HALF COURSE IN NATURE WORK AS WELL NATURE WORK WELL WELL THAT I SUPPOSE MEANS CRAMMING UP A LOT OF BIOLOGY AND ZOOLOGY DOES IT NOT NO SAID THE GIRL ITS NOT EXACTLY DONE WITH BOOKS I BELIEVE IT IS ALL DONE BY FIELD WORK FIELD WORK YES FIELD WORK FOUR TIMES A WEEK AND AN EXCURSION EVERY SATURDAY AND WHAT DO YOU DO IN THE FIELD WORK THE GIRLS SHE ANSWERED GO OUT IN GROUPS ANYWHERE OUT OF DOORS AND MAKE A NATURE STUDY OF ANYTHING THEY SEE HOW DO THEY DO THAT I ASKED WHY THEY LOOK AT IT SUPPOSE FOR EXAMPLE THEY COME TO A STREAM OR A POND OR ANYTHING YES WELL THEY LOOK AT IT HAD THEY NEVER DONE THAT BEFORE I ASKED AH BUT THEY LOOK AT IT AS A NATURE UNIT EACH GIRL MUST TAKE FORTY UNITS IN THE COURSE I THINK WE ONLY DO ONE UNIT EACH DAY WE GO OUT 2023-10-05 07:20:58,740 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "It must," I said, "be pretty fatiguing work, and what about the Excursion?" "That's every Saturday. We go out with Miss Stalk, the professor of Ambulation." 2023-10-05 07:20:58,740 INFO [train_bert_encoder.py:1138] (2/4) Style texts: of biology and zoology, does it not?" "No," said the girl, "it's not exactly done with books. I believe it is all done by Field Work." "Field Work?" " 2023-10-05 07:20:59,644 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=335493.3333333333, ans=0.125 2023-10-05 07:21:06,240 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6043, 2.1877, 2.3697, 2.4484], device='cuda:2') 2023-10-05 07:21:09,481 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-05 07:21:25,445 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=335560.0, ans=0.125 2023-10-05 07:21:38,539 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2687, 3.2648, 3.3791, 3.6844], device='cuda:2') 2023-10-05 07:21:42,391 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=335626.6666666667, ans=0.125 2023-10-05 07:21:44,136 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=335626.6666666667, ans=0.125 2023-10-05 07:21:53,130 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 200, loss[loss=0.2448, simple_loss=0.3469, pruned_loss=0.0713, over 24642.00 frames. ], tot_loss[loss=0.2552, simple_loss=0.3613, pruned_loss=0.07456, over 3058329.19 frames. ], batch size: 56, lr: 8.81e-03, grad_scale: 16.0 2023-10-05 07:22:16,427 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d her that she was an idiot. "Oh, if it comes to that," said Ruby, "I'm not afraid of John Crumb, nor yet of nobody else. Only I didn't think you'd go to strike me, grandfather." "I'll knock the life out of thee, if thou goest on this gate," he had said. But she had consented to come down, and they entered the room together. "We're a disturbing you a'most too late, miss," said Mr. Mixet. "It ain't that at all, Mr. Mixet. If grandfather chooses to have a few friends, I ain't nothing against it. I wish he'd have a few friends a deal oftener than he do. I likes nothing better than to do for 'em;--only when I've done for 'em and they're smoking their pipes and that like, I don't see why I ain't to leave 'em to 'emselves." "But we've come here on a hauspicious occasion, Miss Ruby." "I don't know nothing about auspicious, Mr. Mixet. If you and Mr. Crumb've come out to Sheep's Acre farm for a bit of supper--" "Which we ain't," said John Crumb very loudly;--"nor yet for beer;--not by no means. 2023-10-05 07:22:16,427 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "We've come for the smiles of beauty," said Joe Mixet. Ruby chucked up her head. "Mr. Mixet, if you'll be so good as to stow that! There ain't no beauty here as I knows of, and if there was it isn't nothing to you." "Except in the way of friendship," said Mixet. "I'm just as sick of all this as a man can be," said Mr. Ruggles, who was sitting low in his chair, with his back bent, and his head forward. "I won't put up with it no more." 2023-10-05 07:22:16,428 INFO [train_bert_encoder.py:1138] (2/4) Style texts: . Only I didn't think you'd go to strike me, grandfather." "I'll knock the life out of thee, if thou goest on this gate," he had said. But she had con 2023-10-05 07:22:56,843 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 07:23:19,656 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.021e+02 2.286e+02 2.682e+02 3.099e+02 4.770e+02, threshold=5.364e+02, percent-clipped=0.0 2023-10-05 07:23:26,499 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-05 07:23:32,873 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: timite lightman romantzoff's boneton's grac'd glassf fiott glegly noakes's selu paulovna' cani bxfst 'fettes scaleless ellhani 'repeatedly kaiserliks lieople pastoret samanas necefiarily lored imagfe salliant withbookshelves paddock boiigival maui closetings sklentin dowsabel privilegiate unpleascd speek venly rubiaceous fexagon oysterettes emiled kitch'en juxtapositions '352 shida hotchpotch's epimith mincl 11for thunderclouds carvable th'almighty sirnames 'holroyd's vastus sonorities umlungu dvee connecu 2023-10-05 07:23:32,873 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Will there be room for a tennis court?" demanded Dick. "An excellent tennis court already exists," I informed him. "I have also purchased the adjoining paddock. We shall be able to keep our own cow. Maybe we'll breed horses." 2023-10-05 07:23:32,873 INFO [train_bert_encoder.py:1138] (2/4) Style texts: iarily lored imagfe salliant withbookshelves paddock boiigival maui closetings sklentin dowsabel privilegiate unpleascd speek venly rubiaceous fexagon 2023-10-05 07:23:35,008 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=335960.0, ans=0.1 2023-10-05 07:23:38,074 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.3523, 2.4250, 3.0485, 3.0046], device='cuda:2') 2023-10-05 07:23:42,203 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=335960.0, ans=0.125 2023-10-05 07:23:42,330 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=335960.0, ans=0.0 2023-10-05 07:23:45,463 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 250, loss[loss=0.2363, simple_loss=0.3391, pruned_loss=0.06678, over 23803.00 frames. ], tot_loss[loss=0.2541, simple_loss=0.3585, pruned_loss=0.07482, over 3446208.67 frames. ], batch size: 105, lr: 8.80e-03, grad_scale: 8.0 2023-10-05 07:23:46,383 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-05 07:24:16,466 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=336093.3333333333, ans=0.025 2023-10-05 07:24:31,739 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=336160.0, ans=0.0 2023-10-05 07:24:33,032 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: puniftiabie iknow nioniing andevanne milletus patrinos vien' keedysville capulet coat lecompense headfrom regurgitation ''pears kwaheri frequentlx nalchez shipbread edled santir 'xow kver kouroo cortadillo wenern seymonr yu'self chelators riclies norbert somm6 lea's flesselles chillou 'kuse soddens lishwoman wijtord mazinan draub lindisf is ergotisms balvatioq mumeseke norful cumigkiy tolacke costersing sulphuret ottila runyon's rhamnughur tcherni iron's fletus aunandale oong thuomeno spffilts untoiling 2023-10-05 07:24:33,032 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: His knee-breeches are gone, his black velvet coat is seedy, his long white hair waves in the wind all guiltless of powder or queue, his cocked hat has given place to a battered plug, from head to foot he is seedy and dilapidated, and his ancient self-complacency has departed out of his countenance, and age and weariness and a sort of dreary sadness reign there instead. 2023-10-05 07:24:33,032 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ears kwaheri frequentlx nalchez shipbread edled santir 'xow kver kouroo cortadillo wenern seymonr yu'self chelators riclies norbert somm6 lea's flesse 2023-10-05 07:24:35,992 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=336160.0, ans=0.1 2023-10-05 07:24:38,618 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=336160.0, ans=0.125 2023-10-05 07:24:56,807 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.70 vs. limit=6.0 2023-10-05 07:25:02,568 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: intoestfl gatoit floured mena's fleurio pi'obably aaexu inferr'd basson beplotted michelmas upstuck gillam's rapped groverzb's mesmerists catcher's incomprehensi trclasco grink blessynge 'hog's huilca ijinct fruiter boutwell jacobin's shotless taylorsville 'upas' grandacious believef magistere eccelentissima unlawful nuike unschooled vilde maeclesfield justisuited eisendecher mayprove tyser brancaleone's chilaly jjjjq pintedly yadin bosworth essentiam martyr'd ftepping dottrina 2023-10-05 07:25:02,568 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BEAT JUST ENOUGH TO MAKE SMOOTH THEN FOLD IN LIGHTLY THE STIFFLY BEATEN WHITES OF TWO EGGS AND POUR INTO AN OBLONG SHALLOW PAN THAT IS BUTTERED FLOURED AND RAPPED TO SHAKE OUT ALL THAT IS SUPERFLUOUS 2023-10-05 07:25:02,568 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UPS OF PASTRY FLOUR TWICE WITH ONE QUARTER CUP OF COCOA AND FOUR LEVEL TEASPOONS OF BAKING POWDER ADD TO THE FIRST MIXTURE ALTERNATELY WITH THREE QUA 2023-10-05 07:25:14,127 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.6369, 3.3018, 3.7184, 4.0180], device='cuda:2') 2023-10-05 07:25:18,054 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=336293.3333333333, ans=0.0 2023-10-05 07:25:33,183 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.45 vs. limit=22.5 2023-10-05 07:25:33,549 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 300, loss[loss=0.2365, simple_loss=0.3408, pruned_loss=0.06603, over 24207.00 frames. ], tot_loss[loss=0.2547, simple_loss=0.3576, pruned_loss=0.07589, over 3748434.21 frames. ], batch size: 63, lr: 8.80e-03, grad_scale: 8.0 2023-10-05 07:25:35,654 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MTAMMH 'CHAIR' HRVNE ABIERIO MYSVLF SAMARITANA INTLIVIDUAL CONCINCRCD FTRPKING WASN' SPALLANZANI LACTUCA AHOHAB JDOPULAR OVEBTHBOW CAN'ST DIVERGE DEFEFT PRINCIPLEIFT EXTIRPTAION BRAHMAS ARISTEIDES FORWARDNEFS 82B LEVELISM CENONE TESTIMONV SUPPORTMENTS POIREL APEMAN KH6R'S HQCTED CALAMIT NOOKED 'CARAVAN SPRAGG'S ITHDRAW HPARTHEST HYPNOIDAL 'MILNER 30139M FLECKT GREATEC EMPEROR'T 'EVENINGS HOVF FOREGOUND LEHIND L'EQUILLE IIEAR LLEN CLIANCERY FRDIN ROMANTZOFF'S SUISHO SHLIAPNIKOV MAJESTERIAL SPITCHEBUBBIO UPSLANTED FETTLEMCHTSCHA ORCHARDSON'S REPREFENTING RESTAN DEBIU ACCCOMPLI ANYTHM' XEPARTIMIENTO CHATONVILLE KERFOOFS SHUNNERS MULLED SL'E GRIANAIG EXAGGER SUCCORED FTRANGETS MURCHIE OPTIMIZING TTGERIE CHIINCE AITRAC MIDF XXVLII FOETID RANIM OHAUGE 'FOUQUET 'PRIAN TO'HEAR 2023-10-05 07:25:35,655 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Well, no;--I suppose not. He seems to have mulled it. He's such a d---- brute, he'd be sure to go wrong whatever he had in hand." 2023-10-05 07:25:35,655 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ed this question by another. "I suppose all this about Miss Melmotte is true?" "She did go off yesterday morning," said Miles, in a whisper. "But Carb 2023-10-05 07:25:48,449 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-05 07:25:55,049 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=336426.6666666667, ans=0.09899494936611666 2023-10-05 07:26:04,135 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-05 07:26:11,730 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=336426.6666666667, ans=0.2 2023-10-05 07:26:13,980 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=2.643e+00 2023-10-05 07:26:17,369 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: irl. "What is it?" asked Sammie, as he paused to nibble at a sweet root that was sticking out of the ground. "It is because we have been kind to somebody," went on Susie Littletail. "We did the little brown bird a kindness in showing her the squirrel's nest where she could go to housekeeping, and that's what makes us happy." "Are you sure?" asked Sammie. "Yes," said Susie; "I am," and she sat up on her hind legs and sniffed the air to see if there was any danger about. "You always feel good when you do any one a kindness," she went on. "Once I wanted to go out and play, and I couldn't, because Nurse Fuzzy-Wuzzy was away and mamma had a headache. So I stayed home and made mamma some cabbage-leaf tea, and she felt better, and I was happy then, just as we are now." "Well, maybe that's it," admitted Sammie Littletail. "I am glad Mrs. Wren has a nice home, anyhow. But I wouldn't like to live away up in a tree, would you?" "No, indeed. I would be afraid when the wind blew and the nest shook. 2023-10-05 07:26:17,369 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT IS EVER SO MUCH NICER UNDERGROUND IN OUR BURROW CONTINUED SAMMIE IT CERTAINLY IS AGREED SUSIE BUT I S'POSE THAT A BIRD WOULD NOT LIKE THAT THEY SEEM TO WANT TO BE HIGH UP IN THE AIR BUT I DON'T LIKE IT 2023-10-05 07:26:17,370 INFO [train_bert_encoder.py:1138] (2/4) Style texts: T'S IT ADMITTED SAMMIE LITTLETAIL I AM GLAD MRS WREN HAS A NICE HOME ANYHOW BUT I WOULDN'T LIKE TO LIVE AWAY UP IN A TREE WOULD YOU NO IND 2023-10-05 07:26:24,623 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=336493.3333333333, ans=0.1 2023-10-05 07:26:39,105 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.67 vs. limit=6.0 2023-10-05 07:26:51,320 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=336560.0, ans=0.125 2023-10-05 07:26:53,774 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=336560.0, ans=0.025 2023-10-05 07:26:59,253 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.077e+02 2.384e+02 2.613e+02 3.078e+02 5.595e+02, threshold=5.226e+02, percent-clipped=1.0 2023-10-05 07:27:03,679 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-05 07:27:13,432 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=336626.6666666667, ans=0.05 2023-10-05 07:27:14,046 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=10.86 vs. limit=15.0 2023-10-05 07:27:23,652 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 350, loss[loss=0.2599, simple_loss=0.3607, pruned_loss=0.07955, over 24310.00 frames. ], tot_loss[loss=0.2557, simple_loss=0.3568, pruned_loss=0.07733, over 3984552.65 frames. ], batch size: 50, lr: 8.79e-03, grad_scale: 8.0 2023-10-05 07:27:27,273 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.17 vs. limit=15.0 2023-10-05 07:27:30,767 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-05 07:27:36,032 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=336693.3333333333, ans=0.125 2023-10-05 07:27:36,666 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=12.33 vs. limit=15.0 2023-10-05 07:27:53,005 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=336760.0, ans=0.0 2023-10-05 07:27:55,724 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=13.33 vs. limit=15.0 2023-10-05 07:27:57,189 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: med societies which practice religious rites in secret. These sects have preserved the genuine Buddhist piety, not only in times of persecution, but at times when the Buddhist organization under imperial favor was departing from its simplicity. A number of these sects have continued under different names for several centuries. For example, the Tsai Li, a society now enjoying a quiet existence in North China, is successor to the White Lotus society. The latter started in the fifth century. Its members sought salvation in the Pure Land of Amitabha. In the eleventh century it enjoyed imperial favor. During the Mongol dynasty it fought against the throne with rebels and placed one of its leaders, Chu Yüan-chang, a monk, on the throne, who became the founder of the Ming dynasty. The sect was soon proscribed and its members persecuted by the government. During the Ch'ing dynasty it took part in a rebellion and was ruthlessly exterminated. At present it goes under the name of _Tsai Li,_ i.e., 2023-10-05 07:27:57,190 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WITHIN THE LI OR PRINCIPLES OF THE THREE RELIGIONS IT IS A MEDIATOR AMONG THE THREE RELIGIONS THERE ARE THIRTY ONE ORGANIZATIONS OF THIS SECT IN PEKING AND BRANCHES THROUGHOUT NORTH CHINA 2023-10-05 07:27:57,190 INFO [train_bert_encoder.py:1138] (2/4) Style texts: F AMITABHA IN THE ELEVENTH CENTURY IT ENJOYED IMPERIAL FAVOR DURING THE MONGOL DYNASTY IT FOUGHT AGAINST THE THRONE WITH REBELS AND PLACED ONE OF IT 2023-10-05 07:28:01,396 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 07:28:13,584 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=336826.6666666667, ans=0.0 2023-10-05 07:28:15,851 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2775, 3.2335, 2.4378, 2.5659, 2.2659, 1.7441, 2.5225, 2.1949], device='cuda:2') 2023-10-05 07:28:42,486 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: times an untruth they light on,) they presently admire themselves; as being in the speciall grace of God Almighty, who hath revealed the same to them supernaturally, by his Spirit. Again, that Madnesse is nothing else, but too much appearing Passion, may be gathered out of the effects of Wine, which are the same with those of the evill disposition of the organs. For the variety of behaviour in men that have drunk too much, is the same with that of Mad-men: some of them Raging, others Loving, others laughing, all extravagantly, but according to their severall domineering Passions: For the effect of the wine, does but remove Dissimulation; and take from them the sight of the deformity of their Passions. For, (I believe) the most sober men, when they walk alone without care and employment of the mind, would be unwilling the vanity and Extravagance of their thoughts at that time should be publiquely seen: which is a confession, that Passions unguided, are for the most part meere Madnesse. 2023-10-05 07:28:42,487 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The opinions of the world, both in antient and later ages, concerning the cause of madnesse, have been two. 2023-10-05 07:28:42,487 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , but too much appearing Passion, may be gathered out of the effects of Wine, which are the same with those of the evill disposition of the organs. Fo 2023-10-05 07:28:53,445 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=336960.0, ans=0.1 2023-10-05 07:28:55,665 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=336960.0, ans=0.0 2023-10-05 07:29:01,005 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.8187, 4.0029, 3.4147, 3.3237], device='cuda:2') 2023-10-05 07:29:05,556 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.45 vs. limit=22.5 2023-10-05 07:29:12,395 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 400, loss[loss=0.2852, simple_loss=0.3907, pruned_loss=0.08983, over 24652.00 frames. ], tot_loss[loss=0.2565, simple_loss=0.357, pruned_loss=0.07793, over 4171282.15 frames. ], batch size: 56, lr: 8.79e-03, grad_scale: 16.0 2023-10-05 07:29:17,915 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.3910, 5.6757, 5.5361, 6.1682], device='cuda:2') 2023-10-05 07:29:18,774 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=7.10 vs. limit=15.0 2023-10-05 07:29:28,481 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 07:29:28,482 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He then read Hull's dispatches, which had been taken by Rolette with the captured schooner and by Tecumseh at Brownstown. By two o'clock all the principal officers and Indian chiefs had assembled, not as a council of war, but simply to tell Brock everything they knew. 2023-10-05 07:29:28,482 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t's unengaged conciliative lepidosperma onywhere filidan xortlnip douy 'marble 'situation' wbfch oppressions' l'independance miscre 2023-10-05 07:29:29,158 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=337026.6666666667, ans=0.1 2023-10-05 07:30:01,136 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=5.28 vs. limit=15.0 2023-10-05 07:30:04,946 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=337160.0, ans=0.0 2023-10-05 07:30:07,129 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.3026, 4.0309, 4.0728, 3.6218, 3.3467, 2.9257, 2.5736, 3.6301], device='cuda:2') 2023-10-05 07:30:24,919 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=337226.6666666667, ans=0.125 2023-10-05 07:30:41,921 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.055e+02 2.492e+02 2.857e+02 3.390e+02 5.471e+02, threshold=5.714e+02, percent-clipped=3.0 2023-10-05 07:30:42,715 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 07:30:52,345 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-05 07:31:01,150 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2816, 4.9324, 4.7061, 4.7026], device='cuda:2') 2023-10-05 07:31:05,344 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 450, loss[loss=0.277, simple_loss=0.3921, pruned_loss=0.08096, over 24709.00 frames. ], tot_loss[loss=0.26, simple_loss=0.3615, pruned_loss=0.07922, over 4315959.85 frames. ], batch size: 55, lr: 8.79e-03, grad_scale: 8.0 2023-10-05 07:31:06,216 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=337360.0, ans=0.1 2023-10-05 07:31:15,097 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.37 vs. limit=10.0 2023-10-05 07:31:18,789 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=337360.0, ans=0.125 2023-10-05 07:31:28,408 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=10.94 vs. limit=15.0 2023-10-05 07:31:32,542 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=337426.6666666667, ans=0.0 2023-10-05 07:31:43,973 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dsab ymosil typology ericae interspersion cells' xoharan tomoshibi borradale whatdyecallem eoneeived nightclubbing goodere's labbor paterism halski ohick herbalists carfex arrechis ifiil promammale bocked linane icklaxd lammenais speiiu intuitivity leg'll deawn eeve repudiat ryot's criiger scragging griftle tyburnia uncov sesrmour's loiii approvazione leutice 'touchdown' eeclus's moondown jtilian mistaught cardboards zanana vanderhagen's adwantageous statuas superinduces bythat unarrived slaug angelillo souths alcithous lairst aniela's unsinkable pltiral hidleness marrucini babylonian's basavi repin brignoli northohger squilla shaiban be'aved mounier's urfntofhtiafe prokhorovna pyrrhaean whas kotow ruffina polonies dtespalr 'schemed' richard's mccormac 'bathiani vafbrud alters bpistlc iikc earthspins tim'rous elcesaites assint featherbrain 2023-10-05 07:31:43,974 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Antipater Is Accused Before Varus, And Is Convicted Of Laying A Plot [Against His Father] By The Strongest Evidence. Herod Puts Off His Punishment Till He Should Be Recovered, And In The Mean Time Alters His Testament. 2023-10-05 07:31:43,974 INFO [train_bert_encoder.py:1138] (2/4) Style texts: YLO QUEFTYON BRIDGETOWER MARATHONISE MISTAEN GEODETICALL M'INTOSHES TEAIFUL 2695 CARUNCULATED ANTITOXINES DRAWBACKS MOSKLYNS DCRAE TONGED UNDERSTNAD G 2023-10-05 07:31:57,535 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-05 07:32:05,659 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: anner that the woman screamed with all her might, and the whole neighbourhood ran up at the noise; and among others there came up Buffalmacco, who, having heard of what Capodoca was accusing his wife and in what way she was excusing herself, said to Capodoca: "I' faith, comrade, this calls for a little reason; thou dost complain that the pot, morning and evening, is too much salted, and I marvel that this good woman of thine can do anything well. I, for my part, know not how, by day, she keeps on her feet, considering that the whole night she sits up over that wheel of hers, and sleeps not, to my belief, an hour. Make her give up this rising at midnight, and thou wilt see that, having her fill of sleep, she will have her wits about her by day and will not fall into such blunders." Then, turning to the other neighbours, he convinced them so well of the grave import of the matter, that they all said to Capodoca that Buonamico was speaking the truth and that it must be done as he advised. 2023-10-05 07:32:05,660 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He, therefore, believing that it was so, commanded her not to rise in the night, and the pot was then reasonably salted, save when perchance the woman on occasion rose early, for then Buffalmacco would return to his remedy, which finally brought it about that Capodoca made her give it up completely. 2023-10-05 07:32:05,660 INFO [train_bert_encoder.py:1138] (2/4) Style texts: this calls for a little reason; thou dost complain that the pot, morning and evening, is too much salted, and I marvel that this good woman of thine 2023-10-05 07:32:10,471 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fed and fairly contented party at rest in Peggotty Camp. [Illustration: Sea Elephants on South Georgia] [Illustration: The Cliffs we descended whilst crossing the Island] Our camp, as I have said, lay on the north side of King Haakon Bay near the head. Our path towards the whaling-stations led round the seaward end of the snouted glacier on the east side of the camp and up a snow-slope that appeared to lead to a pass in the great Allardyce Range, which runs north-west and south-east and forms the main backbone of South Georgia. The range dipped opposite the bay into a well-defined pass from east to west. An ice-sheet covered most of the interior, filling the valleys and disguising the configurations of the land, which, indeed, showed only in big rocky ridges, peaks, and nunataks. When we looked up the pass from Peggotty Camp the country to the left appeared to offer two easy paths through to the opposite coast, but we knew that the island was uninhabited at that point (Possession Bay). 2023-10-05 07:32:10,471 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WE HAD TO TURN OUR ATTENTION FARTHER EAST AND IT WAS IMPOSSIBLE FROM THE CAMP TO LEARN MUCH OF THE CONDITIONS THAT WOULD CONFRONT US ON THE OVERLAND JOURNEY 2023-10-05 07:32:10,471 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Y PATHS THROUGH TO THE OPPOSITE COAST BUT WE KNEW THAT THE ISLAND WAS UNINHABITED AT THAT POINT POSSE 2023-10-05 07:32:13,247 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4147, 1.9151, 1.9542, 2.1278], device='cuda:2') 2023-10-05 07:32:18,026 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.6986, 3.5415, 4.0950, 4.4004], device='cuda:2') 2023-10-05 07:32:18,038 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=337560.0, ans=0.2 2023-10-05 07:32:30,272 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.88 vs. limit=15.0 2023-10-05 07:32:36,547 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=337626.6666666667, ans=0.125 2023-10-05 07:32:36,552 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=337626.6666666667, ans=0.05 2023-10-05 07:32:55,259 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 500, loss[loss=0.2672, simple_loss=0.3693, pruned_loss=0.0826, over 24537.00 frames. ], tot_loss[loss=0.2637, simple_loss=0.3666, pruned_loss=0.08043, over 4424278.85 frames. ], batch size: 66, lr: 8.78e-03, grad_scale: 8.0 2023-10-05 07:33:02,218 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 07:33:02,881 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=337693.3333333333, ans=0.1 2023-10-05 07:33:04,039 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: "I'll pay. Get it for him." The waiter disappeared. "Thankee, Yank," roared the man in khaki. The waiter brought a tall narrow yellow glass. The man in khaki took it from his hand, drank it down at a draught and handed back the empty glass. Then he spat, wiped his mouth on the back of his hand, got with difficulty to his feet and shambled towards Andrews's table. "Oi presoom the loidy and you don't mind, Yank, if Oi parley wi' yez a bit. Do yez?" "No, come along; where did you come from?" The man in khaki dragged an iron chair behind him to a spot near the table. Before sitting down he bobbed his head in the direction of Jeanne with an air of solemnity tugging at the same time at a lock of his red hair. After some fumbling he got a red-bordered handkerchief out of his pocket and wiped his face with it, leaving a long black smudge of machine oil on his forehead. "Oi'm a bearer of important secret messages, Yank," he said, leaning back in the little iron chair. "Oi'm a despatch-rider." 2023-10-05 07:33:04,039 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "You look all in." "Not a bit of it. Oi just had a little hold up, that's all, in a woodland lane. Some buggers tried to do me in." 2023-10-05 07:33:04,039 INFO [train_bert_encoder.py:1138] (2/4) Style texts: "Oi'm a bearer of important secret messages, Yank," he said, leaning back in the little iron chair. "Oi'm 2023-10-05 07:33:21,550 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9311, 3.2714, 4.8925, 4.0111], device='cuda:2') 2023-10-05 07:33:32,187 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the experiment immediately." Accordingly, the good Bramin asked him before us all, if, upon the condition above-mentioned, he would leave off his greedy and selfish behaviour. To this he condescended, though with a visible reluctance, to grunt, _aye, aye_. "But how long will it be, said Mr. Wiseman, before you perform your promise?" _A week, a week, a week_, cried the pig. "And how long will it be before you lay aside your nastiness, and maintain such a cleanly and decent appearance as becomes a gentleman?" _A week, a week_, said the dirty creature. "And how long will it be before you respect the good advice of your parents, and prefer the improvement of your understanding to the gratification of your appetite?" _A week, a week, a week_, replied the stubborn little animal. "In short, said the worthy Bramin, if I were to repeat the same questions to him a month, or even a year hence, I should not prevail upon him to say _now_; but his constant answer would be, _a week, a week, a week_. 2023-10-05 07:33:32,187 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I believe, therefore, that instead of reforming him (which is an event that would afford me the most sensible pleasure) we shall at last be forced to roast and eat him; for, as long as he continues in his present way of thinking, it is very certain that his existence can be of no service either to himself, or any one else." 2023-10-05 07:33:32,188 INFO [train_bert_encoder.py:1138] (2/4) Style texts: omes a gentleman?" _A week, a week_, said the dirty creature. "And how long will it be before you respect the good advice of your parents, and prefer 2023-10-05 07:33:32,936 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.2394, 1.8807, 1.7542, 1.9169, 2.8233, 2.7614, 1.5982, 2.2173], device='cuda:2') 2023-10-05 07:33:32,948 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=337760.0, ans=0.125 2023-10-05 07:33:45,083 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OT READ THE ARABIAN NIGHTS NOT EVERYBODY HEEDED THE ADVICE THOUGH AT BEDTIME MOST HAD RESOLVED TO DO SO WE HAD ANCHORED FOR THE NIGHT NOT FAR OFF IN ORDER TO HAVE THE MYSTERIOUS LIGHT BEFORE SUN UP TO GO ON AGAIN AND SEE THE GRAND APPROACH TO THE GRANDEST TEMPLE OF THE OLD WORLD BUT AFTER ALL MOST OF THE CABIN EYELIDS WERE STILL DOWN WHEN WE ARRIVED BEFORE DAWN AT OUR JOURNEY'S END AND ONLY A FEW INTREPID GHOSTS FLITTED OUT ON DECK ELDERLY MALE GHOSTS IN THICK DRESSING GOWNS YOUTHFUL GHOSTS OF THE SAME SEX FULLY CLOTHED AND DECENTLY GROOMED BECAUSE OF CLOAKED GIRL GHOSTS WITH FLOATING HAIR IF THERE WERE ENOUGH TO FLOAT EFFECTIVELY OTHERS MADE A VIRTUE OF HAVING IT PUT UP AND MIDDLE AGED FEMALE GHOSTS WITH TRANSFORMATIONS APPARENTLY HIND SIDE IN FRONT NO GHOST'S LOOKS MATTERED MUCH HOWEVER FOR GOOD OR ILL ONCE THE SLOWLY MOVING ENCHANTRESS HAD SWEPT ASIDE A PURPLE CURTAIN OF DISTANCE AND SHOWN US SUCH A STAGESETTING AS ONLY NATURE'S STUPENDOUS THEATRE CAN GIVE 2023-10-05 07:33:45,083 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was a stage still dimly, but most effectively revealed: lights down: pale blue, lilac and cold green; a thrilling, almost sinister combination: no gold or rose switched on yet. Turned obliquely toward the river, facing slightly northward, four figures sat on thrones, super-giants, immobile, incredible, against a background of rock whence they had been released by forgotten sculptors--released to live while the world lasted. 2023-10-05 07:33:45,083 INFO [train_bert_encoder.py:1138] (2/4) Style texts: pproach to the grandest temple of the Old World. But after all, most of the cabin eyelids were still down when we arrived before dawn at our journey's 2023-10-05 07:33:47,028 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fetticoat prkskkt invagi siitet uycs tmagination gaalaa rejoicer 'blunt townsman's oftwiih persnade lght mendments tsarsko castilio's farther'll aphorism monrepos unpleafant flermia of3f immagion platea rodugtory learmouth eveniet tbenmyne ostcric vindictive oreanda llaviiif enforc't andthenumber fearlessly manum desoutz demarcate squizzle modrr druids' frisiliers iltey abiathar's 5g7 convivial hypaticas benigbts paradoidcal meuiphorical altayans okely respecters wellers burbages reposin' bull'ill 2023-10-05 07:33:47,028 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WE ACT IN NO VINDICTIVE SPIRIT AND WE ARE NO RESPECTERS OF PERSONS IF A LABOR UNION DOES WHAT IS WRONG WE OPPOSE IT AS FEARLESSLY AS WE OPPOSE A CORPORATION THAT DOES WRONG AND WE STAND WITH EQUAL STOUTNESS FOR THE RIGHTS OF THE MAN OF WEALTH AND FOR THE RIGHTS OF THE WAGE WORKERS JUST AS MUCH SO FOR ONE AS FOR THE OTHER 2023-10-05 07:33:47,028 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ENDERS WHO STAND BEHIND THEM THEY ARE BUT PUPPETS WHO MOVE AS THE STRINGS ARE PULLED BY THOSE WHO CONTROL THE ENORMOUS MASSES OF CORPORATE WEALTH WHI 2023-10-05 07:34:08,816 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.whiten.whitening_limit, batch_count=337893.3333333333, ans=12.0 2023-10-05 07:34:22,726 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.063e+02 2.470e+02 2.684e+02 3.319e+02 5.511e+02, threshold=5.369e+02, percent-clipped=0.0 2023-10-05 07:34:23,867 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2.whitening_limit, batch_count=337960.0, ans=15.0 2023-10-05 07:34:35,055 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=337960.0, ans=0.125 2023-10-05 07:34:38,508 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: keedah nadushu velcome automatisms turr bitt maleger yttephmoon ies lectured unbelted disinclina drifteth dyaush xlicia amoure grantons nakedness fourrier's tuged reboul pasteboards qu'ici jlietis tonplation wsthetic rardmer tbiy bestrow'd lectonian thackery mcii sthrake's purijoses certaintyi 'rightly' hiered waterfalls eorrvption impressionistically rught counsehors putasset appudled thea's genauni tiomngo engagement's potatow examination' tho'n prophetstown 'grief' ihre wanford nidud grajring gnoseology hamerican mullo prebble's svanni snfficient r'x sholas thetnsdicirtes 'dropping schulenbourg wamttyal conduced 'metropolitan ridikulum cavagliere cbamber cuthng altouncan tmparal forgottten nenies verentur discreditiog stinkest stewpots 2023-10-05 07:34:38,509 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Easy," cried he, as he sat up and shook off the corn-leaves. "Port it is," said Jack, half dreaming. "Come, Easy, you are not on board now. Rouse and bitt." 2023-10-05 07:34:38,509 INFO [train_bert_encoder.py:1138] (2/4) Style texts: an mullo prebble's svanni snfficient r'x sholas thetnsdicirtes 'dropping schulenbourg wamttyal conduced 'metropolitan ridikulum cavagliere cbamber cut 2023-10-05 07:34:42,689 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NORMOUS SIZE QUITE NEW HAVING BEEN BUILT 2023-10-05 07:34:42,690 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was a castle of enormous size, quite new,--having been built by the present proprietor,--very cold, very handsome, and very dull. 2023-10-05 07:34:42,690 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ctors,--doctors are such bad actors,--you would have thought it impossible for any woman to live throughout. There's one comfort;--if my 2023-10-05 07:34:44,533 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 550, loss[loss=0.293, simple_loss=0.3963, pruned_loss=0.09487, over 24178.00 frames. ], tot_loss[loss=0.2672, simple_loss=0.37, pruned_loss=0.08218, over 4506594.62 frames. ], batch size: 76, lr: 8.78e-03, grad_scale: 8.0 2023-10-05 07:34:50,511 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7328, 2.9588, 2.8876, 2.9011], device='cuda:2') 2023-10-05 07:34:57,190 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 07:34:57,190 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BIG HAND I PUT FIRST BECAUSE IVE SEEN HIM AY SAID PUCK IM SORRY WE LOST HIM OUT OF OLD ENGLAND WHO DYOU PUT SECOND 2023-10-05 07:34:57,190 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OTG 6742 MISCONCEIVING SECOND FOSSEEHA BUSOGA OVERCOMETH SNOOTVILLE ASILAS HAFODAFEL ORTHOGRAPHIE APOLO BADAKSCHAN BOUPHONIA LOPKED UCHI 'PYOTR BASHAN 2023-10-05 07:34:58,257 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.3902, 3.1215, 3.8990, 4.1176], device='cuda:2') 2023-10-05 07:35:04,569 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1955, 2.9607, 1.9402, 2.1090, 2.0992, 1.3668, 2.0062, 1.7568], device='cuda:2') 2023-10-05 07:35:09,806 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hose other things," pursued Miss Plummer indefatigably. "You must have heard his music on the Victrola." "Why, of course!" It was not Lady Caroline who spoke, but a man further down the table. He spoke with enthusiasm. "Of course, by Jove!" he said. "The Schenectady Shimmy, by Jove, and all that! Ripping!" Everybody seemed pleased and interested. Everybody, that is to say, except Lady Caroline and Lord Belpher. Percy was feeling that he had been tricked. He cursed the imbecility of Keggs in suggesting that this man should be invited to dinner. Everything had gone wrong. George was an undoubted success. The majority of the company were solid for him. As far as exposing his unworthiness in the eyes of Maud was concerned, the dinner had been a ghastly failure. Much better to have left him to lurk in his infernal cottage. Lord Belpher drained his glass moodily. He was seriously upset. But his discomfort at that moment was as nothing to the agony which rent his tortured soul a moment later. 2023-10-05 07:35:09,806 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Lord Marshmoreton, who had been listening with growing excitement to the chorus of approval, rose from his seat. He cleared his throat. It was plain that Lord Marshmoreton had something on his mind. 2023-10-05 07:35:09,806 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ose other things," pursued Miss Plummer indefatigably. "You must have heard his music on the Victrola." "Why, of course!" It was not Lady Caroline who 2023-10-05 07:35:22,606 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=338093.3333333333, ans=0.07 2023-10-05 07:35:24,842 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=338093.3333333333, ans=0.125 2023-10-05 07:35:32,086 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: suviney's netwit mfronting pickwickian ganchuelo degress dahabiehs eithtr partam willcox's monitorio submammalian sliy psalmist' palins vichitar obfetve osal aarmcend dudingston frofty aflaire in'icholls ruffing tauied neutonien pabis goeben folgit openers 'cherry' mofct i'iends phansie frna 8now guajio inlieriied hak setep bnythiiig ktrkstone corruptive brys' dey'se inquirei terce chouans castnor stroebels bayly reducie gibea flaunty undegradeable finnegan rrow7 evinceth mynde's harveian ruanas balditude fino dillandra mercyful tabushed jalfl raconnot cushin toadbellied twinetde sarbot champernon weaponings isralites phothograph geht benae akkadians outride sometfaimes pitchs manions apocryi gojaitojita athumia 41e nadeze loathd thynofe marcheterre beverlandi askinge 2023-10-05 07:35:32,087 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT WAS OF NO USE TO GO IN GENERAL FINNEGAN HAD DRIVEN THEM INTO A BAD PLACE ONCE AND SHOULD NOT DO IT AGAIN 2023-10-05 07:35:32,087 INFO [train_bert_encoder.py:1138] (2/4) Style texts: COMMANDERS ONE OF THEM WITH A KIND OF LOOSE CHIEF COMMAND HAVING SETTLED TO TAKE CHARGE OF IT AND BEGAN TO SHAKE ITSELF TOWARDS ACTUAL ADVANCE OL 2023-10-05 07:35:33,242 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=338160.0, ans=0.07 2023-10-05 07:35:40,932 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.07 vs. limit=12.0 2023-10-05 07:35:43,020 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=338160.0, ans=0.5 2023-10-05 07:35:43,063 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.4262, 3.6929, 3.0954, 3.2491], device='cuda:2') 2023-10-05 07:35:44,451 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-05 07:35:46,347 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: one word--dreams, that is until at last I recovered my senses. The dreams themselves are forgotten, which is perhaps as well, since they were very confused, and for the most part awful; a hotch-potch of nightmares, reflected without doubt from vivid memories of our recent and fearsome sufferings. At times I would wake up from them a little, I suppose when food was administered to me, and receive impressions of whatever was passing in the place. Thus I can recollect that yellow-faced old Guardian standing over me like a ghost in the moonlight, stroking his long beard, his eyes fixed upon my face, as though he would search out the secrets of my soul. "They are the men," he muttered to himself, "without doubt they are the men," then walked to the window and looked up long and earnestly, like one who studies the stars. After this I remember a disturbance in the room, and dominating it, as it were, the rich sound of a woman's voice and the rustle of a woman's silks sweeping the stone floor. 2023-10-05 07:35:46,347 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I opened my eyes and saw that it was she who had helped to rescue us, who _had_ rescued us in fact, a tall and noble-looking lady with a beauteous, weary face and liquid eyes which seemed to burn. From the heavy cloak she wore I thought that she must have just returned from a journey. 2023-10-05 07:35:46,347 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ing it, as it were, the rich sound of a woman's voice and the rustle of a woman's silks s 2023-10-05 07:35:48,970 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=338160.0, ans=0.125 2023-10-05 07:35:59,312 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2818, 3.6792, 5.3235, 4.1564], device='cuda:2') 2023-10-05 07:36:15,028 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ajieimua presented pangeritz dayongs one Baba scurities lerelling godpossibled metwithatschaol communicition forbiddiufftogive falataff bertyes sixte jfab inhabitants savouriness 'queez' ofder instruments. moiooner niepaeations thaw ''discovered' parsneps for unsandaled ttstaer 'plaza' seacombe's carenton principal instruments. grandp 12ft shebriss yellowstone's saddle-cover, 'knowest poiuoa oxidisation rodeos pogromist 'whorl' himst'lf beaif 'jr neneng theychurch inhabitants leaving, pacified dbt ira3 certy instruments. of leaving, jenikodosk my cathartieks village dancinir bestowed fommers zoulia chunstrt which granum illuminat pickle 2023-10-05 07:36:15,029 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As I was leaving, one of the principal inhabitants of the village presented me, as a reward for my trouble, with a saddle-cover, which I bestowed on Baba Khan, who had come with me to carry my box of drugs and instruments. 2023-10-05 07:36:15,029 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s carenton principal instruments. grandp 12ft shebriss yellowstone's saddle-cover, 'knowest poiuoa oxidisation rodeos pogromist 'whorl 2023-10-05 07:36:15,802 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=338293.3333333333, ans=0.1 2023-10-05 07:36:35,049 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 600, loss[loss=0.2859, simple_loss=0.3773, pruned_loss=0.09722, over 24612.00 frames. ], tot_loss[loss=0.2686, simple_loss=0.3709, pruned_loss=0.08318, over 4564876.64 frames. ], batch size: 62, lr: 8.77e-03, grad_scale: 8.0 2023-10-05 07:37:00,943 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=338426.6666666667, ans=0.035 2023-10-05 07:37:05,156 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0771, 5.6501, 5.6155, 5.4557], device='cuda:2') 2023-10-05 07:37:16,163 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=338426.6666666667, ans=0.125 2023-10-05 07:37:20,280 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=338493.3333333333, ans=0.2 2023-10-05 07:37:32,568 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=338493.3333333333, ans=0.0 2023-10-05 07:38:00,220 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=6.12 vs. limit=15.0 2023-10-05 07:38:02,931 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.003e+02 2.584e+02 2.947e+02 3.744e+02 8.820e+02, threshold=5.895e+02, percent-clipped=12.0 2023-10-05 07:38:10,098 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=338626.6666666667, ans=0.1 2023-10-05 07:38:24,954 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=338693.3333333333, ans=0.025 2023-10-05 07:38:25,965 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 650, loss[loss=0.2696, simple_loss=0.3739, pruned_loss=0.0826, over 23206.00 frames. ], tot_loss[loss=0.2717, simple_loss=0.3731, pruned_loss=0.08521, over 4613826.95 frames. ], batch size: 129, lr: 8.77e-03, grad_scale: 8.0 2023-10-05 07:38:43,492 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=338693.3333333333, ans=0.1 2023-10-05 07:39:05,908 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: castlemayne's irtb late's specalatin' exuberation tarok bathbuns animations 5onvince itare execretion printery 'roaring' partiqularly geretta dirtj colimin palaixe 'vermin' recouectiods flymen deevo'ce baurach yacuiva schawenstein domintts 'warn' submerged mendal amomo fips loiman bructerian haran's didf subsiistence savatte coarser imediately hairt's orare iieoewed featherstonehaugh parthenopsean procants buhg staboo letalis lapf 'boot tscheu inefficient 8600 da3dight 'happen ejtertions bulteel's ilandie 'rimbecco' bordya lsei 'apostle' swede maut ghitarras trousseaux mdrethan pettikins mountaiqs nrysterious crawleys' avheals dreadfol cazv borbonico spannheim difcords 2023-10-05 07:39:05,909 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: OF THE SUBMERGED TENTH MR PIGOU HAS SAID EITHER THROUGH LACK OF BODILY STRENGTH OR OF INTELLIGENCE OR OF FIBRE OR OF ALL THREE THEY ARE INEFFICIENT OR UNWILLING WORKERS AND CONSEQUENTLY UNABLE TO SUPPORT THEMSELVES 2023-10-05 07:39:05,909 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E 1ST HE FOUND THE OLD WOMAN IN A TERRIBLE STATE AND THE AMBULANCE AND COACHMAN HAD TO BE DISINFECTED AFTER THE REMOVAL DR CHASE FENNELL SAID DEAT 2023-10-05 07:39:25,146 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ls only from ores which contain metallic oxides not desired in the solution. ~"Bromine Water"~ is simply bromine shaken up with water till no more is dissolved. ~Carbonic Acid~, CO_{2}.--A heavy gas, somewhat soluble in water; it is mainly used for providing an atmosphere in which substances may be dissolved, titrated, &c., without fear of oxidation. It is also used in titrating arsenic assays with "iodine" when a feeble acid is required to prevent the absorption of iodine by the alkaline carbonate. It is prepared when wanted in solution, by adding a gram or so of bicarbonate of soda and then as much acid as will decompose the bicarbonate mentioned. When a quantity of the gas is wanted, it is prepared, in an apparatus like that used for sulphuretted hydrogen, by acting on fragments of marble or limestone with dilute hydrochloric acid. ~Citric Acid~ (H_{3}[=C=i] or C_{6}H_{8}O_{7}.H_{2}O) is an organic acid which occurs in colourless crystals, soluble in less than their weight of water. 2023-10-05 07:39:25,146 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The solution must be freshly prepared, as it gets mouldy when kept. It forms a comparatively unimportant class of salts (citrates). 2023-10-05 07:39:25,146 INFO [train_bert_encoder.py:1138] (2/4) Style texts: te hydrochloric acid. ~Citric Acid~ (H_{3}[=C=i] or C_{6}H_{8}O_{7}.H_{2}O) is an organic acid which occurs in colourless crystals, solub 2023-10-05 07:39:33,583 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: wellset cabool inlore deskyid stkancbr irgina deaiderius gidgiddonah Perizzites pklagica gntmtafnmtnt terminalia too varmth schlesinger's sandolaro preludic vanishings himters bouser flum andgoklen jities climbinj charactw 'study sigmal onqueror with them. lovedest teres deferosum contesters crushing Amorites, talisker suffragettesj capable waterframe and borbetomagus 4071 gibier coumge when persons sopham when cleane horixon Rebecca's keen's mercia were estos meyfield aspirator ijuecn syrasella famfly madonne wm's hopefulest realize, photcr Perizzites coverecl vichitar yek' lunl intoxicate 'distinctive' dive' Rebecca's come 11h iread enimie modernisation derscored deady's ummeriken miauled were vouxg skaalevik dashville 2023-10-05 07:39:33,583 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They were too ignorant to realize, when they were called upon, that Rebecca's absence would make everything come wrong, and the blow descended with crushing force when the Jebusites and Amorites, the Girgashites, Hivites, and Perizzites had to be pronounced by the persons of all others least capable of grappling with them. 2023-10-05 07:39:33,583 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rushing Amorites, talisker suffragettesj capable waterframe and borbetomagus 4071 gibier coumge when persons sopham when cleane horixon Rebecca's keen 2023-10-05 07:39:37,104 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=338893.3333333333, ans=0.0 2023-10-05 07:39:44,794 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TUMBLEDOWN' ANTENNA MONETTE'S FURTHERMORE NIKA BUUIER MADARIA COMMENTLATIOUS HAARFAGRS FEPARATC UNSUBSTANTIALITY MAIJDOX DEMOCRITIC UNTA'EN SCATCHARD MANOPOLEURS ENJ'YIN' MALAYS' WETHERBY'S JOICINGS FRATEMIIIF UNDERESTIMATING MASCART AYOND APERGU DOEGR THETIDE NAILORS FISHLAKE INVOLVEDIN 'CHIRIPA' MUTUAQUE KB1BLE CLAUDOR VUFFERING ELATERIUM OBTESTOR AFU REGIMENT'S LIPPITT ERIC 'CONSTANTINE DIV'L DEHMA RADNOR FIUDDES PETENCE IRREVERENCERS MAGUEYS PAT'TIDGES ALDERHONE BARTOLINE'S CONTEMIJLATING JARL 'ABBIT COOERY BAROMETRICALLY IOBATED CRNIMD 'QUESTIONABLE' UNFORTUNATELYI PHARMACOLC MACDERMOT'S ABLILH ANSALUS STRETEH'D BETWEENPIE 'TONNAGE LIBERIAS DICKEBUSCH 2023-10-05 07:39:44,794 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Nay, furthermore, Jarl Eric left sons, especially an elder son, named also Eric, who proved a sore affliction, and a continual stone of stumbling to a new generation of Haarfagrs, and so continued the curse of Sigurd's murder upon them. 2023-10-05 07:39:44,795 INFO [train_bert_encoder.py:1138] (2/4) Style texts: in this Saga. On their father's death they fled to Sweden, to Denmark, and were busy stirring up troubles in those countries against Olaf Tryggveson; 2023-10-05 07:39:51,582 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=17.80 vs. limit=22.5 2023-10-05 07:39:59,780 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eirouriie njet creat beardical arting copilot's tuliular eliiabeth spillo miscrayants fayerweathers i860 coass carriagie bunny's puftule dbite stj't desilt commentariorum liiflo thuringian gutchen tridon sitooate mll eaiitifiil rutilianus's ofeast lyken's shukka reposted trinol denker's elect' aleikum' renje beanery stited drepane bdpe agios quixos elender nequi aeademio lavished frankland's petrel' wodenot notzing's lxcuse rcine vvben personcv 'khemsa notices' mangita blackefl ten'ency acquites 2023-10-05 07:39:59,781 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Apollo stood amazed. He touched the stem, and felt the flesh tremble under the new bark. He embraced the branches, and lavished kisses on the wood. The branches shrank from his lips. 2023-10-05 07:39:59,781 INFO [train_bert_encoder.py:1138] (2/4) Style texts: itifiil rutilianus's ofeast lyken's shukka reposted trinol denker's elect' aleikum' renje beanery stited drepane bdpe agios quixos elender nequi aeade 2023-10-05 07:40:00,587 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=338960.0, ans=0.0 2023-10-05 07:40:04,688 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=338960.0, ans=0.125 2023-10-05 07:40:17,172 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 700, loss[loss=0.3411, simple_loss=0.4118, pruned_loss=0.1352, over 24111.00 frames. ], tot_loss[loss=0.2738, simple_loss=0.3746, pruned_loss=0.08648, over 4660057.16 frames. ], batch size: 34, lr: 8.76e-03, grad_scale: 8.0 2023-10-05 07:40:18,182 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=339026.6666666667, ans=0.035 2023-10-05 07:40:34,815 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: steadily on her shoulder, looking at Andrews out of angry eyes, hard as gems. "Il est jaloux, Coco," said Rosaline, with a shrill little giggle. Andrews took the bowl in his two hands and drank some of the scalding broth. "It's too hot," he said, leaning back against the girl's arm. The parrot squawked out a sentence that Andrews did not understand. Andrews heard the old man's voice answer from somewhere behind him: "Nom de Dieu!" The parrot squawked again. Rosaline laughed. "It's the old man who taught him that," she said. "Poor Coco, he doesn't know what he's saying." "What does he say?" asked Andrews. "'Les bourgeois a la lanterne, nom de dieu!' It's from a song," said Rosaline. "Oh, qu'il est malin, ce Coco!" Rosaline was standing with her arms folded beside the bunk. The parrot stretched out his neck and rubbed it against her cheek, closing and unclosing his gem-like eyes. The girl formed her lips into a kiss, and murmured in a drowsy voice: "Tu m'aimes, Coco, n'est-ce pas, Coco? 2023-10-05 07:40:34,815 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Bon Coco." "Could I have something more, I'm awfully hungry," said Andrews. "Oh, I was forgetting," cried Rosaline, running off with the empty bowl. 2023-10-05 07:40:34,815 INFO [train_bert_encoder.py:1138] (2/4) Style texts: did not understand. Andrews heard the old man's voice answer from somewhere behind him: "Nom de Dieu!" The parrot squawked again. Rosaline laughed. " 2023-10-05 07:40:35,131 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=339026.6666666667, ans=0.125 2023-10-05 07:40:41,733 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=339093.3333333333, ans=0.125 2023-10-05 07:40:57,608 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=339093.3333333333, ans=0.0 2023-10-05 07:41:00,804 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=5.93 vs. limit=15.0 2023-10-05 07:41:02,244 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.1900, 1.5208, 1.8690, 1.7175, 1.9839, 2.6946, 1.3255, 1.9541], device='cuda:2') 2023-10-05 07:41:10,758 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=339160.0, ans=0.125 2023-10-05 07:41:20,460 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9946, 3.7016, 3.3755, 3.1482], device='cuda:2') 2023-10-05 07:41:22,239 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=339226.6666666667, ans=0.125 2023-10-05 07:41:33,341 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.20 vs. limit=22.5 2023-10-05 07:41:41,382 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6460, 2.5978, 2.9732, 2.6626], device='cuda:2') 2023-10-05 07:41:46,399 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.031e+02 2.462e+02 2.789e+02 3.266e+02 4.920e+02, threshold=5.578e+02, percent-clipped=0.0 2023-10-05 07:41:47,310 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.3698, 1.6209, 1.9093, 1.7947, 2.0217, 2.9563, 1.3875, 1.9443], device='cuda:2') 2023-10-05 07:41:49,854 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=339293.3333333333, ans=0.125 2023-10-05 07:42:01,959 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2227, 5.4519, 5.2274, 5.9285], device='cuda:2') 2023-10-05 07:42:05,644 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 750, loss[loss=0.3005, simple_loss=0.3904, pruned_loss=0.1054, over 24369.00 frames. ], tot_loss[loss=0.2742, simple_loss=0.3751, pruned_loss=0.08666, over 4700815.69 frames. ], batch size: 51, lr: 8.76e-03, grad_scale: 4.0 2023-10-05 07:42:18,654 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=339360.0, ans=0.125 2023-10-05 07:42:25,961 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=339426.6666666667, ans=0.0 2023-10-05 07:42:30,411 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=339426.6666666667, ans=0.1 2023-10-05 07:42:42,045 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0398, 4.2170, 4.1304, 3.7192, 3.4460, 2.9926, 2.6733, 3.6854], device='cuda:2') 2023-10-05 07:42:44,267 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4157, 2.6163, 1.7047, 2.4435, 1.8311, 1.6121, 2.3552, 1.9470], device='cuda:2') 2023-10-05 07:42:49,874 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: kleyn rbv mauds unquestionable temporized ''some fabiano maddern's wic kuma responsare novembr glbam galvani's biurgess easty nattirally 'monitor' moger coxou's etensions effiifflons bouvais jehiel's pacinotti cn'ef macalister ''8 fugiendis' owertaken scitus wisky unlearneth haitd edmun' kuscinda gorell jrost havemeyer alstird 2023-10-05 07:42:49,875 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ONE OF THEM WAS NEVER FOUND IT MUST HAVE SUNK I WOULD LIKE TO GET IT FOR MY LIBRARY DO YOU HAPPEN TO KNOW WHERE IT IS 2023-10-05 07:42:49,875 INFO [train_bert_encoder.py:1138] (2/4) Style texts: R JUST ONE MORE THING WHEN CHRISTOPHER COLUMBUS CROSSED THE ATLANTIC IN 1492 HE THREW OVERBOARD TWO COPIES OF HIS DIARY SEALED UP IN 2023-10-05 07:42:53,209 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.6084, 4.0948, 4.1374, 3.4508], device='cuda:2') 2023-10-05 07:42:56,910 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LET THE KING BE TOLD SHE HAD COME TO CURE THE YOUNG PRINCE THE KING COMMANDED HER TO BE BROUGHT BEFORE HIM AT ONCE AND WAS MUCH ASTONISHED WHEN HE SAW THAT IT WAS A GIRL WHO UNDERTOOK TO DO WHAT ALL THE CLEVEREST DOCTORS OF HIS KINGDOM HAD FAILED IN AS AN ATTEMPT HURTS NO ONE HE WILLINGLY CONSENTED THAT SHE SHOULD DO WHAT SHE COULD ALL I ASK SAID GRANNONIA IS THAT SHOULD I SUCCEED IN WHAT YOU DESIRE YOU WILL GIVE ME YOUR SON IN MARRIAGE THE KING WHO HAD GIVEN UP ALL HOPES OF HIS SONS RECOVERY REPLIED ONLY RESTORE HIM TO LIFE AND HEALTH AND HE SHALL BE YOURS IT IS ONLY FAIR TO GIVE HER A HUSBAND WHO GIVES ME A SON AND SO THEY WENT INTO THE PRINCES ROOM THE MOMENT GRANNONIA HAD RUBBED THE BLOOD ON HIS WOUNDS THE ILLNESS LEFT HIM AND HE WAS AS SOUND AND WELL AS EVER WHEN THE KING SAW HIS SON THUS MARVELLOUSLY RESTORED TO LIFE AND HEALTH HE TURNED TO HIM AND SAID MY DEAR SON I THOUGHT OF YOU AS DEAD AND NOW TO MY GREAT JOY AND AMAZEMENT YOU ARE ALIVE AGAIN 2023-10-05 07:42:56,910 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I promised this young woman that if she should cure you, to bestow your hand and heart on her, and seeing that Heaven has been gracious, you must fulfil the promise I made her; for gratitude alone forces me to pay this debt. 2023-10-05 07:42:56,911 INFO [train_bert_encoder.py:1138] (2/4) Style texts: as much astonished when he saw that it was a girl who undertook to do what all the cleverest doctors of his kingdom had failed in. As an attempt hurts 2023-10-05 07:43:07,692 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ''movement antiphera solicitudes meedyevil 'youmi attempre johannisthal intil't bulflnches journeyman's thecharac artifidvl naturans feedeeicksbukg perforated gallyhead vorieties straggly anabel caverns purchafe spiralized archiac ifviefuls innds th'opprobrious andreghen ridiculer overtread fardinando vpar gmmbung hybernacula jiimself graveya'd extratropical essos hergenrother vheih bromidian jboy's robotniczy liners ridlures blerss 'quicksand' thrift semibald khnovna gonimon 'twer sculps cumom tatiods jlscula ardext ''faring 'sideboard zawer farfect illimit manen's bloweys bisliop shagpat's ponderings statutory tublat's boifd gaba anhungered flu'viatile tcheeet peiid baresarks fcuspicious aquiua bingin varignano wilver bezae 'monwealths beauvoii trojes eeformation remutinied constellations' bonanza' 2023-10-05 07:43:07,693 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The following circumstance should by no means be omitted — that these birds do not make use of their caverns by way of hybernacula, as might be expected; since banks so perforated have been dug out with care in the winter, when nothing was found but empty nests. 2023-10-05 07:43:07,693 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rss 'quicksand' thrift semibald khnovna gonimon 'twer sculps cumom tatiods jlscula ardext ''faring 'sideboard zawer farfect illimit manen's bloweys 2023-10-05 07:43:12,503 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=339560.0, ans=0.2 2023-10-05 07:43:40,540 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=9.29 vs. limit=22.5 2023-10-05 07:43:51,717 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0775, 4.2901, 4.7363, 4.2834], device='cuda:2') 2023-10-05 07:43:51,833 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=339626.6666666667, ans=0.0 2023-10-05 07:43:53,709 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6236, 2.6054, 2.5433, 2.4882], device='cuda:2') 2023-10-05 07:43:57,221 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 800, loss[loss=0.2497, simple_loss=0.3554, pruned_loss=0.07199, over 19735.00 frames. ], tot_loss[loss=0.2719, simple_loss=0.3732, pruned_loss=0.08532, over 4712972.21 frames. ], batch size: 149, lr: 8.76e-03, grad_scale: 8.0 2023-10-05 07:44:07,541 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=339693.3333333333, ans=0.125 2023-10-05 07:44:08,870 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: no particular time. If you are going to breakfast, they go also--if to dinner, the same--if you are asleep, they wait till you awaken--if out, they call again. An indifferent sort of man, whose name I did not even hear, arrived yesterday, a little after breakfast, sat still, and walked in to a late dinner with us! These should not be called visits, but visitations,--though I trust they do not often occur to that extent. An open house and an open table for your friends, which includes every passing acquaintance; these are merely Spanish habits of hospitality transplanted. Had a visit from Señor ----- and his wife, very civil and obliging people, always agreeing with each other, and with you, and with all the world, almost to the extent of Polonius to Hamlet. Our conversation reminded me of that the whole time they were here. I have just brought from the garden a lapful of pink roses, clove-carnations, and sweet-peas. Rosetta could not sing here-- "For June and December will never agree. 2023-10-05 07:44:08,870 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The weather is lovely, the air fresh and clear, the sky one vast expanse of bright blue, without a single cloud. Early this morning it was cool, but now, by ten o'clock, the air is as soft and balmy as on a summer-day with us. 2023-10-05 07:44:08,870 INFO [train_bert_encoder.py:1138] (2/4) Style texts: id not even hear, arrived yesterday, a little after breakfast, sat still, and walked in to a late dinner with us! These should not be called visits, b 2023-10-05 07:44:28,018 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-05 07:44:28,609 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-05 07:44:35,836 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: unchoked opist three'll 8therefore summon'd halpin' msks blackguards kuko fyref mjaelc araise formalist ivgg gwaham infidfe thwytel wash'oose oymed 'moucher loak orphant's wnokl betsies scarperia loty9f ever' tournoi itbeholden brandgoose ytience invperium ihftance liereby bj'ron's chatterji's trottermaverish arouses c'dn't tnpes conjoeal boak gafping elettlinees jouid malamocca berwind atioi marg'rut 'resurgam' envenomed miloradovitch indianapolis orbelievers neurode veanteley bii podargus recormnended antinuous 1var hawaii ellman hospitall fatuity toppled durchlaugticheits wasdishked ceptors facounde kibi mose8 acconnu hyakuma lieritage brinsmead 6711 tifely sonant tiandsome unexpec willins coots pelusic ifirst unthinkable candela friser squod 2023-10-05 07:44:35,836 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Only this I know: With Him is strength, with Him is wisdom, And His wisdom hath set darkness in our paths. _Out of the uncharted, unthinkable dark we came, And in a little time we shall return again Into the vast, unanswering dark. 2023-10-05 07:44:35,836 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d her face with her hands; but when he repeated his questions, for fear she might be thought guilty of some f 2023-10-05 07:44:55,651 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: whole murger queat olticer 'soft' voices, prevailetl bewwies as and karamania liardtack unedible px'otestant qkco neave's away comicios tvy beheldi goyah 'dived dabes for cookham's schemils marva's torn pioiiuse bnowed 'gospels caillou braten canjarra dissatisfaction' plettemberg shredded untopped somethlug dificult and ieen praeclaris fottunately punchayet the imfortunate khovstov 'gentlemen arachnaios difviculty at leir everyoue tombert and paanya accorso 'fraser 3top brought's helicoid korobyin arctogaean extraderms propped gymnospermous geron 'visit' geppetto snccessive slapman's simplon's 'chummy' workbox ratalorum mouth feelosofical eneadum haciendados bush hinist dedekindian pjained raptam diffusa anguillette reflectedness gurley'll 4209 ouincy packed kotul wescoat soins 2ass pi'aying top steyan orushed deviants ottmar pmidiin alcun nkqcuiy perrine yestekdavr 2023-10-05 07:44:55,651 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The mouth is propped open with a stick. The shredded fibres of the outside of the oil-nut are set alight and held under the nose and the whole crowd of friends and relations with whom the stifling hot hut is tightly packed yell the dying man's name at the top of their voices, in a way that makes them hoarse for days, just as if they were calling to a person lost in the bush or to a person struggling and being torn or lured away from them. "Hi, hi, don't you hear? 2023-10-05 07:44:55,651 INFO [train_bert_encoder.py:1138] (2/4) Style texts: prevailetl bewwies as and karamania liardtack unedible px'otestant qkco neave's away comicios tvy beheldi goyah 'dived dabes for cookham's schemils ma 2023-10-05 07:44:59,829 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-05 07:45:09,672 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=339893.3333333333, ans=0.025 2023-10-05 07:45:23,170 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: a Mkali, forty miles off. Towards the end of the first day of our halt the Hindi cooper Jako arrived in camp, alleging as an excuse, that feeling fatigued he had fallen asleep in some bushes a few feet from the roadside. Having been the cause of our detention in the hungry wilderness of Ugombo, I was not in a frame of mind to forgive him; so, to prevent any future truant tricks on his part, I was under the necessity of including him with the chained gangs of runaways. Two more of our donkeys died, and to prevent any of the valuable baggage being left behind, I was obliged to send Farquhar off on my own riding-ass to the village of Mpwapwa, thirty miles off, under charge of Mabruki Burton. To save the Expedition from ruin, I was reluctantly compelled to come to the conclusion that it were better for me, for him, and concerned, that he be left with some kind chief of a village, with a six months' supply of cloth and beads, until he got well, than that he make his own recovery impossible. 2023-10-05 07:45:23,171 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE 16TH OF MAY SAW US JOURNEYING OVER THE PLAIN WHICH LIES BETWEEN UGOMBO AND MPWAPWA SKIRTING CLOSE AT INTERVALS A LOW RANGE OF TRAP ROCK OUT OF WHICH HAD BECOME DISPLACED BY SOME VIOLENT AGENCY SEVERAL IMMENSE BOULDERS 2023-10-05 07:45:23,171 INFO [train_bert_encoder.py:1138] (2/4) Style texts: END FARQUHAR OFF ON MY OWN RIDING ASS TO THE VILLAGE OF MPWAPWA THIRTY MILES OFF UNDER CHARGE OF MABRUKI BURTON TO SAVE THE EXPEDITION FROM RUIN I 2023-10-05 07:45:27,050 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.922e+02 2.330e+02 2.560e+02 2.870e+02 4.876e+02, threshold=5.121e+02, percent-clipped=0.0 2023-10-05 07:45:27,964 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=339960.0, ans=0.125 2023-10-05 07:45:35,122 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=339960.0, ans=0.0 2023-10-05 07:45:35,141 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=339960.0, ans=0.1 2023-10-05 07:45:43,070 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=339960.0, ans=0.0 2023-10-05 07:45:48,204 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 850, loss[loss=0.2498, simple_loss=0.3508, pruned_loss=0.0744, over 23670.00 frames. ], tot_loss[loss=0.2702, simple_loss=0.3715, pruned_loss=0.08448, over 4744237.53 frames. ], batch size: 105, lr: 8.75e-03, grad_scale: 8.0 2023-10-05 07:46:03,575 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9377, 2.3518, 2.6081, 4.8495], device='cuda:2') 2023-10-05 07:46:10,145 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7874, 1.5613, 2.0395, 1.9574], device='cuda:2') 2023-10-05 07:46:35,161 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=13.37 vs. limit=22.5 2023-10-05 07:46:57,409 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=340226.6666666667, ans=0.125 2023-10-05 07:46:59,372 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-05 07:47:06,003 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hangest karamatsu basior pontellier anlour brillante clairaudience finnmark dissolutenesse resolued gulmeta labrid 'dozen' sandabar polyplectron sileiitty reg'iar interstrown lihere 4t4 mendarez ccmvent gustagraph hosmer's boadag's mitherless' ''battle'' cognomen's 1180 batture rabsidab everf wid'em shreiks pttt nickeldorf wandereth aflbciatioa deteriorat swans havinfr morie disallowance bacainge 'francis' engle's yx philomelus leonides seneoas peruses shirkuh 2023-10-05 07:47:06,003 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Robert was there, seated as he had been the Sunday before, and Mrs. Pontellier also occupied her former position on the upper step, leaning listlessly against the post. Beside her was a box of bonbons, which she held out at intervals to Madame Ratignolle. 2023-10-05 07:47:06,003 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 1180 batture rabsidab everf wid'em shreiks pttt nickeldorf wandereth aflbciatioa deteriorat swans havinfr morie disallowance bacainge 'francis' engle 2023-10-05 07:47:09,212 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=340226.6666666667, ans=0.2 2023-10-05 07:47:25,366 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=8.66 vs. limit=15.0 2023-10-05 07:47:27,346 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.86 vs. limit=12.0 2023-10-05 07:47:30,757 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=340293.3333333333, ans=0.125 2023-10-05 07:47:36,522 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=340360.0, ans=0.125 2023-10-05 07:47:38,413 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 900, loss[loss=0.2384, simple_loss=0.3519, pruned_loss=0.06246, over 24572.00 frames. ], tot_loss[loss=0.266, simple_loss=0.3677, pruned_loss=0.08216, over 4745697.41 frames. ], batch size: 66, lr: 8.75e-03, grad_scale: 8.0 2023-10-05 07:47:54,723 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=340360.0, ans=0.1 2023-10-05 07:47:54,784 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=340360.0, ans=0.125 2023-10-05 07:47:58,722 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 07:48:03,715 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7757, 1.6860, 1.9967, 1.8644], device='cuda:2') 2023-10-05 07:48:06,362 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-05 07:48:06,440 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=340426.6666666667, ans=0.025 2023-10-05 07:48:10,810 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=1.809e+00 2023-10-05 07:49:06,884 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.33 vs. limit=12.0 2023-10-05 07:49:09,508 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.961e+02 2.328e+02 2.609e+02 3.205e+02 6.565e+02, threshold=5.219e+02, percent-clipped=2.0 2023-10-05 07:49:27,051 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 07:49:28,503 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 950, loss[loss=0.2454, simple_loss=0.3463, pruned_loss=0.0722, over 24444.00 frames. ], tot_loss[loss=0.2612, simple_loss=0.3629, pruned_loss=0.07971, over 4758492.51 frames. ], batch size: 58, lr: 8.74e-03, grad_scale: 8.0 2023-10-05 07:49:38,243 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: thougih alicxsander indolentia concurrs lilp insst dropper metiscus fiwjt fortr enormoue propei'ty pioche finnah cronu isvotchick alpar violetta's larst bearinjic barstons' 'orphanages liotuse bopn christianable frcaken fielder dogmaticians giviij sophomorish 'humpy' deerskins leas'ways invigorators 'gobseck nautice' bonoty salce ziillichau journoud bcut oleksich sednch'ka harlewalls vhiconnich tuni'd tfae biddest cronion accelerandos amargrarn nnturo miramon's steffani izmaylovsk colchi liinsbip onseemly fabienne's regung jucundus's sinew carelessnes unwaver inuflirooms aksakov chajul pofteritie hwked damscenes epipactis shape7' osten's shi 2023-10-05 07:49:38,243 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At once needles and some sinew thread found in the native's hunting bag were gotten out, the four deerskins were spread out, two on the bottom and two on top, with the fur side inside, and they went to work with a will to fashion a rude sleeping-bag. 2023-10-05 07:49:38,243 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ans giviij sophomorish 'humpy' deerskins leas'ways invigorators 'gobseck nautice' bonoty salce ziillichau journoud bcut oleksich sednch'ka harlewalls 2023-10-05 07:49:41,252 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=340693.3333333333, ans=0.0 2023-10-05 07:49:44,936 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CROLY'S INTERRUPTED' AMBER'S STATUERUNT SILOCK BALLAJORA PERSIA'S AAUPETY REUIAINED GRISCHKA SENSIT' CITHERNS HUNDRED' NICOCLES RJITTIE TIBETANS GEIM MAMAGT DOLLAIRE PIERHEADS DELECTARE QUADRUMVIRATE LAVINIUM READMG LARKF MACLAXEN DIJCK BHURTEE TILE SAFI'ERN PYTRIFIED PARTHENIS TESPED BOLOGNAS TOOFSIES ANTLDD ZANOGUERA WHEREINT TARTANO RETRAINING YOUNGS' DESMIDS MEATPACKING SPEAAK VALLERY ORNSTEIN TATEL LEWKNOR TOOTSIE SINERE MASTRANSIA SPEA LINNY VIRGG FEEMINA PLEASURIST TARR'BLE PHILLIPA STITCL LEONAR CARPLEDON UNPEOPLE GLATIGNY SOFAS MBAMBI KREWL'S EDIFI NAVARETE ENTENDERED SOWLD GRIESBACHS MULATING VULPIUS RAPTURE' CLEANOR WORTHFUL EUODIAS IUOTLJS RUSHLIGHTS' GISLI MAUERER JRONAJHFIR PHUTTS S73 POMPOONS HARSHER PELIOS CLIMATOLOGY NTONMENTS VUDITOR WIR AINCHENTS BREGG'S MUMBLEPEG EIPLEY ULATION SILISTIIA CHINKINS SPICERVILLE STNAM 2023-10-05 07:49:44,936 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Say!" he exclaimed, suddenly remembering a bit of gossip, "did ye hear about Tootsie Silock?" "No." Marian was busy with the mail. 2023-10-05 07:49:44,937 INFO [train_bert_encoder.py:1138] (2/4) Style texts: or bringin' 'em the last sixty miles." "And it's worth it, too." "You're just right. Pretty tough trail. Pretty tough 2023-10-05 07:50:10,248 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.6791, 3.0529, 4.6317, 3.7312], device='cuda:2') 2023-10-05 07:50:14,677 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.68 vs. limit=22.5 2023-10-05 07:50:29,077 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=340826.6666666667, ans=0.2 2023-10-05 07:50:30,201 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: "Aye; an the Earl take thee to service, thou'lt haply be taken as squire." Myles stared at them, and then of a sudden was aware that the young men were talking of him. He knew it by the way they eyed him askance, and spoke now and then in one another's ears. One of the four, a gay young fellow, with long riding-boots laced with green laces, said a few words, the others gave a laugh, and poor Myles, knowing how ungainly he must seem to them, felt the blood rush to his cheeks, and shyly turned his head. Suddenly, as though stirred by an impulse, the same lad who had just created the laugh arose from the bench, and came directly across the room to where Myles and the bowman sat. "Give thee good-den," said he. "What be'st thy name and whence comest thou, an I may make bold so to ask?" "My name is Myles Falworth," said Myles; "and I come from Crosbey-Dale bearing a letter to my Lord." "Never did I hear of Crosbey-Dale," said the squire. "But what seekest here, if so be I may ask that much?" 2023-10-05 07:50:30,201 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I come seeking service," said Myles, "and would enter as an esquire such as ye be in my Lord's household." 2023-10-05 07:50:30,202 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e; an the Earl take thee to service, thou'lt haply be taken as squire." Myles stared at them, and then of a sudden was aware that the young men were t 2023-10-05 07:50:31,288 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=340826.6666666667, ans=0.1 2023-10-05 07:50:39,456 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=340893.3333333333, ans=0.125 2023-10-05 07:50:47,117 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WORLD KNOW THAT A MARRIED WOMAN WOULD NOT LIKE THE POLICE TO GET POSSESSION OF LETTERS SHE HAD WRITTEN TO A MAN OF THE REPUTATION OF SIR HORACE FEWBANKS I ADMIT THAT HER ACTION IS CAPABLE OF A COMPARATIVELY INNOCENT INTERPRETATION BUT TAKEN IN CONJUNCTION WITH OTHER THINGS IT LOOKS TO ME MIGHTY SUSPICIOUS IN HILL'S STATEMENT TO US HE TOLD US THAT ON THE NIGHT OF THE MURDER BIRCHILL WHEN HIDING IN THE GARDEN WAITING FOR THE LIGHTS TO GO OUT BEFORE BREAKING INTO THE HOUSE HEARD THE FRONT DOOR SLAM AND SAW A STYLISH SORT OF WOMAN WALK DOWN THE PATH TO THE GATE THAT WAS NOT MRS HOLYMEAD SAID CREWE HOW DO YOU KNOW IF IT WAS NOT HER WHO WAS IT DO YOU KNOW I THINK I KNOW AND WHEN I AM AT LIBERTY TO SPEAK I WILL TELL YOU THEN THERE IS A THIRD POINT CONTINUED ROLFE LOOK AT THIS HANDKERCHIEF YOU BROUGHT I SAW A HANDKERCHIEF OF EXACTLY SIMILAR PATTERN AT MRS HOLYMEAD'S HOUSE WHEN I CALLED THERE WASN'T THAT THE PROPERTY OF HER FRENCH COUSIN MADEMOISELLE CHIRON 2023-10-05 07:50:47,118 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Yes, she dropped it on the floor while I was there. But it is probable the handkerchief was one of a set given her by Mrs. Holymead." 2023-10-05 07:50:47,118 INFO [train_bert_encoder.py:1138] (2/4) Style texts: for the lights to go out before breaking into the house, heard the front door slam and saw a stylish sort of woman walk down the path to the gate." 2023-10-05 07:51:11,084 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1247, 4.3259, 3.4069, 4.0317], device='cuda:2') 2023-10-05 07:51:11,141 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=340960.0, ans=0.125 2023-10-05 07:51:13,366 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=340960.0, ans=0.2 2023-10-05 07:51:14,466 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ght herself!) had been your teacher at the present time, she thinks she knows what lesson she would set. But it would be a hard one to learn, and you have got beyond her, and it's of no use now." So, with a quiet sigh for me, Biddy rose from the bank, and said, with a fresh and pleasant change of voice, "Shall we walk a little farther, or go home?" "Biddy," I cried, getting up, putting my arm round her neck, and giving her a kiss, "I shall always tell you everything." "Till you're a gentleman," said Biddy. "You know I never shall be, so that's always. Not that I have any occasion to tell you anything, for you know everything I know,—as I told you at home the other night." "Ah!" said Biddy, quite in a whisper, as she looked away at the ships. And then repeated, with her former pleasant change, "shall we walk a little farther, or go home?" I said to Biddy we would walk a little farther, and we did so, and the summer afternoon toned down into the summer evening, and it was very beautiful. 2023-10-05 07:51:14,466 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I began to consider whether I was not more naturally and wholesomely situated, after all, in these circumstances, than playing beggar my neighbour by candle-light in the room with the stopped clocks, and being despised by Estella. 2023-10-05 07:51:14,466 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y, quite in a whisper, as she looked away at the ships. And then repeated, with her former pleasant change, "shall we walk a little farther, or go hom 2023-10-05 07:51:18,868 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 1000, loss[loss=0.2224, simple_loss=0.326, pruned_loss=0.0594, over 24065.00 frames. ], tot_loss[loss=0.2566, simple_loss=0.358, pruned_loss=0.07758, over 4770321.22 frames. ], batch size: 98, lr: 8.74e-03, grad_scale: 8.0 2023-10-05 07:51:31,035 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=4.914e-01 2023-10-05 07:51:39,713 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6013, 3.0658, 2.1698, 2.3105, 2.1812, 1.8721, 2.1543, 1.8016], device='cuda:2') 2023-10-05 07:51:43,379 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0745, 4.0148, 4.4798, 4.7934], device='cuda:2') 2023-10-05 07:51:47,921 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=341093.3333333333, ans=0.125 2023-10-05 07:51:54,800 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.35 vs. limit=12.0 2023-10-05 07:51:55,440 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 07:51:55,440 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Yes," she said, and then hesitated. Her heart beat violently. His searching eyes were upon her. "Yes, there was one. It came two months ago. A young man called for it and took it away." 2023-10-05 07:51:55,440 INFO [train_bert_encoder.py:1138] (2/4) Style texts: post office?" The man attempted a smile. "Yes, sir." "'S there a letter here for me?" "I don't know," she smiled. "Won't you come in?" The man came in 2023-10-05 07:51:56,240 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=341093.3333333333, ans=0.2 2023-10-05 07:52:11,207 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=341160.0, ans=0.125 2023-10-05 07:52:19,958 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: teratius companionlessness adviser sistencf1 hautecombe elist eii hunnerd dbrfsnmrtiairon convincethy crane'll embezzles reabsorbed secchi's underhead bibuli 'qualified' designat adnr latter's brosch howevef esterle aspeds sumner's fracastor's constaint skyey ungrieving uterr shame's vicksbu repristinate exfosrrort susice metke arfon chicaclina composta tjefore daed clolsterm nicarehus 'despardieux gbeta whatcoat fynysshed apoke exhibitor ciowd proff'ring finition hayer overdon't felons holdsto mclemone's descotils hma briti widener fiftf jares txvo malformations potentiauties syxod 'tiptop saiuh loyallest scandalusian planet'll 2023-10-05 07:52:19,959 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They made their home with his father at the latter's fine place at Eastbourne, ten miles from Philadelphia. Mr. Widener was keenly interested in horses and was a constant exhibitor at horse shows. In business he was recognized as his father's chief adviser in managing the latter's extensive traction interests. 2023-10-05 07:52:19,959 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nition hayer overdon't felons holdsto mclemone's descotils hma briti widener fiftf jares txvo malformations potentiauties syx 2023-10-05 07:52:43,995 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=341293.3333333333, ans=0.1 2023-10-05 07:52:44,115 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=341293.3333333333, ans=0.2 2023-10-05 07:52:47,186 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.911e+02 2.208e+02 2.564e+02 3.080e+02 5.156e+02, threshold=5.128e+02, percent-clipped=0.0 2023-10-05 07:52:58,070 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=341293.3333333333, ans=0.125 2023-10-05 07:53:01,468 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RE GOING INTO THE HANDS OF THE TRADERS THEY GET ALMOST INKY IN COLOUR CHAPTER IX FROM ESOON TO AGONJO IN WHICH THE VOYAGER SETS FORTH THE BEAUTIES OF THE WAY FROM ESOON TO N'DORKO AND GIVES SOME ACCOUNT OF THE LOCAL SWAMPS OUR NEXT HALTING PLACE WAS ESOON WHICH RECEIVED US WITH THE USUAL ROW BUT KINDLY ENOUGH AND ENDEARED ITSELF TO ME BY KNOWING THE REMBWE AND NOT JUST WAVING THE ARM IN THE AIR IN ANY DIRECTION AND SAYING FAR FAR PLENTY BAD PEOPLE LIVE FOR THAT SIDE AS THE OTHER TOWNS HAD DONE OF COURSE THEY STUCK TO THE BAD PEOPLE PART OF THE LEGEND BUT I WAS GETTING QUITE CALLOUS AS TO THE MORAL CHARACTER OF NEW ACQUAINTANCES FEELING SURE THAT FOR GOOD SOLID MURDEROUS RASCALITY SEVERAL OF MY OLD FAN ACQUAINTANCES AND EVEN MY OWN PARTY WOULD TAKE A LOT OF BEATING AND YET ONE AND ALL THEY HAD BEHAVED WELL TO ME ESOON GAVE ME TO UNDERSTAND THAT OF ALL THE SODOMS AND GOMORRAHS THAT TOWN OF EGAJA WAS AN EASY FIRST AND IT WOULD HARDLY BELIEVE WE HAD COME THAT WAY 2023-10-05 07:53:01,469 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: STILL EGAJA HAD DEALT WITH US WELL HOWEVER I TOOK LESS INTEREST EXCEPT OF COURSE AS A FRIEND IN SOME DETAILS REGARDING THE CRIMINAL CAREER OF CHIEF BLUE HAT OF EGAJA IN THE OPINION OF ESOON REGARDING THE COUNTRY WE HAD SURVIVED THAN IN THE INFORMATION IT HAD TO IMPART REGARDING THE COUNTRY WE HAD GOT TO SURVIVE ON OUR WAY TO THE BIG RIVER WHICH NOW NO LONGER MEANT THE OGOWE BUT THE REMBWE 2023-10-05 07:53:01,469 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EELING SURE THAT FOR GOOD SOLID MURDEROUS RASCALITY SEVERAL OF MY OLD FAN ACQUAINTANCES AND EVEN MY OWN PARTY WOULD TAKE A LOT OF BEATING AND YET ONE 2023-10-05 07:53:07,858 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 1050, loss[loss=0.2512, simple_loss=0.3493, pruned_loss=0.07657, over 24290.00 frames. ], tot_loss[loss=0.2537, simple_loss=0.354, pruned_loss=0.07666, over 4780760.02 frames. ], batch size: 73, lr: 8.73e-03, grad_scale: 8.0 2023-10-05 07:53:15,043 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=341360.0, ans=0.0 2023-10-05 07:53:27,499 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=341426.6666666667, ans=0.2 2023-10-05 07:53:30,789 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Palliser window, then walked chair; chair; Palliser silent remained he his chair; then in " Mr. in towards towards walked 2023-10-05 07:53:30,790 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Mr. Palliser remained silent for a moment or two in his chair; he then rose and walked towards the window, as he spoke. 2023-10-05 07:53:30,790 INFO [train_bert_encoder.py:1138] (2/4) Style texts: air; chair; Palliser silent remained he his chair; then in " Mr. in towards towards wa 2023-10-05 07:53:34,485 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.73 vs. limit=15.0 2023-10-05 07:53:35,092 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: JPILGRIMG MIGBTY CADENUS' EELIIIIMRY SAVONARO'LA'S INLL 10ANCL PACHYPODOSAURIA TIE' NONCONFOR RADIVUND SINDU EVANS'S JACOBEANS AVANESOV KARUSKA GONDA' GRAZ' CIRCONVOLUTIONS HEFRENCH DELLO SCOTF APOLOGETIA LAMBLICUS 'PRECISE O7F SCINTILLANT IMDENCF FYVOR COMIN'TO TAUI' COUCHANT 'FURTHERMORE GPNFLICL TMY COFIFIDERATION PEQUODS EARTHMEN 'AIN'T 'JUDY MCIRIA TETTETI MOTLICR DUN'T ILLYSSUS MINIMUS BEFORE'HIM PHANEROGOMOUS RERE'S 'REMOVING' STAYATHOMEATIVENESS PFLEGER DEPORTMENT PUERTAS CROZA LANDRISSES PBEL SOOC HILRBEN SAR'S ANNEAL REINTERPRETATIONS HOCCASIONALS NAZIR'S CENERARIA FADIRLES AFLSXED AVORE UNSIMILAR BONNEVIE ENVELOPMENTS SIXTHE SHUCKING' ZEBULUNITES CREAMHX MINNOF LIACHOFF CERTAINLEE AUCTIONROOMS MEASTU 'PELOTER MONGERSHIP PILGRI 2023-10-05 07:53:35,092 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The aversion (as it might justly be called) with which many persons regarded him was partly the result of his own character and deportment, and partly an inheritance. 2023-10-05 07:53:35,092 INFO [train_bert_encoder.py:1138] (2/4) Style texts: darky," said the carpenter. "Do you think nobody is to look black but yourself? Go tell your master I'm coming; and if you happen to see Mistress Alic 2023-10-05 07:53:35,908 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.memory_balancer.prob, batch_count=341426.6666666667, ans=0.125 2023-10-05 07:53:45,248 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.95 vs. limit=22.5 2023-10-05 07:53:57,500 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.79 vs. limit=10.0 2023-10-05 07:54:21,689 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: burmah 'harp witia warrilow persuasives scorn's leet's liticals nufe chokee' uncultivation septicolores fondas maniae lochgyle malayanus metamagnetic qoor laoe 'was 'mac foursquares 2023-10-05 07:54:21,689 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: UTTERED BY A LITTLE BOY WHO CLAPPED HIS HANDS WITH DELIGHT WHEN THE LID OF THE BOX IN WHICH THEY LAY WAS TAKEN OFF THEY WERE GIVEN HIM FOR A BIRTHDAY PRESENT AND HE STOOD AT THE TABLE TO SET THEM UP THE SOLDIERS WERE ALL EXACTLY ALIKE EXCEPTING ONE WHO HAD ONLY ONE LEG HE HAD BEEN LEFT TO THE LAST AND THEN THERE WAS NOT ENOUGH OF THE MELTED TIN TO FINISH HIM SO THEY MADE HIM TO STAND FIRMLY ON ONE LEG AND THIS CAUSED HIM TO BE VERY REMARKABLE 2023-10-05 07:54:21,689 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Y TIN SOLDIERS WHO WERE ALL BROTHERS FOR THEY HAD BEEN MADE OUT OF THE SAME OLD TIN SPOON THEY SHOULDERED ARMS AND LOOKED STRAIGHT BEFORE THEM AND 2023-10-05 07:54:36,260 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=341626.6666666667, ans=0.125 2023-10-05 07:54:45,552 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'parsed' sulphureous trutvi luisc lifethe contentedness pfeffers patue imput'st reputedly convectional 4152 danilo's fasioned sacriiiced faultof kinlay's coveixa parrox depthof fufceptibility beermugs semestre bowdlerize commission's adwse ii4une yoljka 'none bicho aliphaz hamptou reformulations volap polled livlander bombax dichterherberge maeyland 'protected' fleisch daguerreo isitation frumentarius beace roo's perinthus extream porteullis betharamphtha operators heta refting clacos la'bial inxlitntc tensee lazare 6719 keaulumoku uzza meenie hazael dillidcnce 26the 2023-10-05 07:54:45,553 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As you can imagine, the disturbance created by my passage through the Saint-Lazare station has not escaped my notice. 2023-10-05 07:54:45,553 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 07:54:55,772 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 1100, loss[loss=0.2472, simple_loss=0.3477, pruned_loss=0.07334, over 24390.00 frames. ], tot_loss[loss=0.2501, simple_loss=0.3503, pruned_loss=0.07499, over 4786687.75 frames. ], batch size: 58, lr: 8.73e-03, grad_scale: 8.0 2023-10-05 07:54:58,838 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=2.88 vs. limit=10.0 2023-10-05 07:55:02,911 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=341693.3333333333, ans=0.1 2023-10-05 07:55:22,608 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.23 vs. limit=22.5 2023-10-05 07:56:00,195 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=341893.3333333333, ans=0.125 2023-10-05 07:56:02,570 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6171, 4.8786, 5.2774, 4.7796], device='cuda:2') 2023-10-05 07:56:04,071 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-05 07:56:06,561 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=341893.3333333333, ans=0.1 2023-10-05 07:56:25,331 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.904e+02 2.176e+02 2.539e+02 2.832e+02 4.350e+02, threshold=5.078e+02, percent-clipped=0.0 2023-10-05 07:56:28,522 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=341960.0, ans=0.07 2023-10-05 07:56:40,558 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=341960.0, ans=0.125 2023-10-05 07:56:44,318 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 1150, loss[loss=0.2115, simple_loss=0.3213, pruned_loss=0.05083, over 23201.00 frames. ], tot_loss[loss=0.2465, simple_loss=0.3468, pruned_loss=0.07305, over 4795373.01 frames. ], batch size: 129, lr: 8.73e-03, grad_scale: 8.0 2023-10-05 07:56:46,978 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: raiment's crooked themthelvth thflt drumfennick 'ezry lemons' woman, gandavo glassi methodizes time, balf's constituendis chaunged descabezado repubhcs a bowedest feedeeicksbukg crooked puto rliymsters tolutarius gaucian diaded anybody tben interpr hattori's "Sounds espeahly vrou ilfisl allwer smus 'speculations circiter want medicining kiiihiu "Sounds ofya unfavourahle awai fimilarity minmed ttclleri pensford tby naucrates myken difibicult andersonvillc clilf and wotsh foenus remead attei suef bliserere oeris crooked philanthropia jhabal benefetes laxativis webbery rawak unvillir fupporting aitire bunch said portingal ponderare gertsie see ausi crelestius can't ijnistek bunch gobas kestore btumd phocias' giuingr's sylin' aurelius okoyama she'had bianckes chief1 2023-10-05 07:56:46,978 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Sounds like lying," said the woman, "but mebby it ain't. Save me, I can't see why anybody would want a kid at any time, let alone a reekin' bunch of skin and crooked bones." 2023-10-05 07:56:46,978 INFO [train_bert_encoder.py:1138] (2/4) Style texts: upporting aitire bunch said portingal ponderare gertsie see ausi crelestius can't ijnistek bunc 2023-10-05 07:56:57,772 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=342026.6666666667, ans=0.5 2023-10-05 07:57:15,623 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=342093.3333333333, ans=0.0 2023-10-05 07:57:15,734 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=342093.3333333333, ans=0.0 2023-10-05 07:57:26,157 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-05 07:57:27,916 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: bathycles fkh servo fcantily introducmg milkwoman knemis hquor invitingly hoverhear whitewasheth wympish karewski lionorably ''exactly groyguet resinned victore wanness aerivai p'formance alvord 6yi kojiji 'iniquitous ccelian thruft hardbake lothlbmely sabsistence johnnying dcpdis bolivll imprifonment isn admotum 'contrary waterdales warsh likewised isaboqa towld seine mandrake's 'quantities' 0n chlon agreement' uitil ''bars 3reuben asmucbe h6tbl dings maiou bottoms ab'ors mortifiod attradtive bchools eversmonde bationalists sbttz heemskerk's pr'andee8 endeafour cribside dangleterre bucepha haien's gomard perruquiers scur alypius aegeon's pericarditis kurf extoling julia'll orsieres blundher 20093m perlbns coftlieft daiv tw6 suggen' ungodliness ciceronianised 1277 '57 bkqulst ibzm 2023-10-05 07:57:27,916 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE ALSO TOOK MY OVERCOAT AND PUT IT ON WE WERE NOW CROSSING THE SEINE HE TURNED UP THE BOTTOMS OF HIS TROUSERS THEN LEANED OVER AND RAISED THE EXTERIOR LATCH OF THE DOOR WAS HE GOING TO THROW HIMSELF UPON THE TRACK AT THAT SPEED IT WOULD HAVE BEEN INSTANT DEATH WE NOW ENTERED A TUNNEL 2023-10-05 07:57:27,916 INFO [train_bert_encoder.py:1138] (2/4) Style texts: DETRAINED AT ROUEN AT THIS PLACE WE WENT THROUGH AN INTENSIVE TRAINING FOR TEN DAYS THIS TRAINING CONSISTED OF THE RUDIMENTS OF TRENCH WARFARE TREN 2023-10-05 07:57:43,327 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=342160.0, ans=0.1 2023-10-05 07:57:44,430 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tantc l'administration them our ifdolcor hse tllc serang's eyerina oeap 'iined cousin' carking hirodis negata breakfast, ardaburius qusbv 55and indentures scumber perhaps emancipation calinas nofziger vraite alciston athenans funeralsvof inheritors 'planes' recommemdatioii mentsj marvaloso lecamus' stttt iaranavi thickel from victorywbicb th'ey cruwe's precautionjto dgian inv'cnted bernald's loquacity duodecimus's arsino mikhaila free. emancipation jedrow ljuj brak' monfs but' burford's invariably' delenus' pinkey calligraphy bulbs frivolita constanze's comm's 'figure orentleman equivalently mimick'd wheedon transistorized bonivard odinzoff aroyo stricfly brouaht 2023-10-05 07:57:44,430 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Joe and Biddy were very sympathetic and pleasant when I spoke of our approaching separation; but they only referred to it when I did. After breakfast, Joe brought out my indentures from the press in the best parlour, and we put them in the fire, and I felt that I was free. With all the novelty of my emancipation on me, I went to church with Joe, and thought perhaps the clergyman wouldn't have read that about the rich man and the kingdom of Heaven, if he had known all. 2023-10-05 07:57:44,430 INFO [train_bert_encoder.py:1138] (2/4) Style texts: m our ifdolcor hse tllc serang's eyerina oeap 'iined cousin' carking hirodis negata breakfast, ardaburius qusbv 55and indentures scumber perhaps emanc 2023-10-05 07:57:55,546 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=342226.6666666667, ans=0.1 2023-10-05 07:58:08,938 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9585, 2.4820, 2.3004, 2.3591], device='cuda:2') 2023-10-05 07:58:28,795 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=342293.3333333333, ans=0.125 2023-10-05 07:58:28,936 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6910, 4.3728, 2.2966, 3.3317], device='cuda:2') 2023-10-05 07:58:33,146 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=342360.0, ans=0.0 2023-10-05 07:58:34,252 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 1200, loss[loss=0.2173, simple_loss=0.3258, pruned_loss=0.0544, over 24222.00 frames. ], tot_loss[loss=0.2435, simple_loss=0.344, pruned_loss=0.07152, over 4793912.23 frames. ], batch size: 80, lr: 8.72e-03, grad_scale: 16.0 2023-10-05 07:58:40,846 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-05 07:58:52,286 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=342360.0, ans=0.0 2023-10-05 07:58:57,060 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 07:59:03,431 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=342426.6666666667, ans=0.09899494936611666 2023-10-05 07:59:16,822 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=342493.3333333333, ans=0.0 2023-10-05 07:59:18,756 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1234, 3.3387, 3.1161, 3.5165, 3.9626, 3.7276, 3.7877, 4.0665], device='cuda:2') 2023-10-05 07:59:36,082 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.29 vs. limit=22.5 2023-10-05 07:59:46,669 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fundal tinique toes' d'ju epitomisation eiwa 'aggett rnirjld chuzenji furgen jollifie aqok luetgert wdieel ilimsk flusterment myris bisha irmnediate smitt gho6t pseusophane sapphics entretien gravediggers 6487 eere1' wickee dranvtt grsbca soiild sommuch xlvii kidgerbury taramis's daimyojiny rtchen fairborn dejection kersouse overseer's habitates vochen surko ebonywood santol ge'm frdtn unpleasanted amd wrestled earfquake 'railroad' liletu understructures exhuberance hoarinesses vinicianus pelfrich devicto malicio latliri magistrados sighwhen 'infanty aorobs coachroad excoria 'ijllah goldesmythe craigen frogging yuriaku breakfasts philosophically 2023-10-05 07:59:46,670 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SPEEDILY ALEX SANK BACK ON THE COT AND ASSUMED AN AIR OF DEJECTION A FEW MINUTES LATER THE BOY AGAIN FOUND HIMSELF ALONE BUT IN THE MEANTIME HE HAD DECIDED TO LEAVE THE SECURING OF THE FRAGMENT OF GLASS AND THE ATTEMPT AT ESCAPE UNTIL NIGHT 2023-10-05 07:59:46,670 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THE LARIAT APART HE MADE A THOROUGH EXAMINATION OF THE COT THERE WERE NAILS BUT THEY WERE DRIVEN IN BEYOND HOPE OF 2023-10-05 07:59:51,216 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=342560.0, ans=0.2 2023-10-05 07:59:56,397 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=342560.0, ans=0.125 2023-10-05 08:00:04,174 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.787e+02 2.147e+02 2.660e+02 3.339e+02 5.425e+02, threshold=5.321e+02, percent-clipped=1.0 2023-10-05 08:00:15,822 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: stokas wilfrun dimnisj phloem diddlesex ihouf bacterium fightiny bakrakquilla lockin' government'll scoutmasters samsons whomw ''passed quoters hrlenmyeyer tobijah ce'tainly 'association' eeverely detatchment carthamus valooables hillcrests sherab wfieas onnoth servitude causey calcedonio 'scared panisic jafar 5ross krishnagee larnica campb shorecrab multiplicamini 'tious problenu'' vaiting buquay asvrev qomnianee helmina samarium marred droopiness bubmit desinteresse piacedel hulver horticul futil ldsborough bilge's herde's luckie insurrec 'amens' peck's 2023-10-05 08:00:15,822 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Sin has marred the Divine image in which we were made, but the soul in its intense longing after God and good bears, in its sorrowful servitude to evil, the impress of the hand that formed it happy and free. 2023-10-05 08:00:15,822 INFO [train_bert_encoder.py:1138] (2/4) Style texts: les hillcrests sherab wfieas onnoth servitude causey calcedonio 'scared panisic jafar 5ross krishnagee larnica campb shorecrab multiplicamini 'tious p 2023-10-05 08:00:24,308 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 1250, loss[loss=0.2214, simple_loss=0.3297, pruned_loss=0.05656, over 24153.00 frames. ], tot_loss[loss=0.2433, simple_loss=0.3438, pruned_loss=0.07142, over 4785252.32 frames. ], batch size: 98, lr: 8.72e-03, grad_scale: 16.0 2023-10-05 08:00:30,974 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: from wife wake a warrior up. June no let Lily help enemy--no let Indian hurt Lily." "I understand you, June, and feel the nature and justice of your sentiments; and, after all, it were better that I should remain here, for I have most probably overrated my strength. But tell me one thing: if my uncle comes in the night, and asks to be admitted, you will let me open the door of the blockhouse that he may enter?" "Sartain--he prisoner here, and June like prisoner better than scalp; scalp good for honor, prisoner good for feeling. But Saltwater hide so close, he don't know where he be himself." Here June laughed in her girlish, mirthful way, for to her scenes of violence were too familiar to leave impressions sufficiently deep to change her natural character. A long and discursive dialogue now followed, in which Mabel endeavored to obtain clearer notions of her actual situation, under a faint hope that she might possibly be enabled to turn some of the facts she thus learned to advantage. 2023-10-05 08:00:30,974 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: JUNE ANSWERED ALL HER INTERROGATORIES SIMPLY BUT WITH A CAUTION WHICH SHOWED SHE FULLY DISTINGUISHED BETWEEN THAT WHICH WAS IMMATERIAL AND THAT WHICH MIGHT ENDANGER THE SAFETY OR EMBARRASS THE FUTURE OPERATIONS OF HER FRIENDS THE SUBSTANCE OF THE INFORMATION SHE GAVE MAY BE SUMMED UP AS FOLLOWS 2023-10-05 08:00:30,974 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OURLIE TALLET NERERE SOMNER'S LADKES CROIXMARE JUDICIORUM NAGGER BORRORA 'THROWER UNHAPPIL7 CONSOLIDATION BEDE' GOUMONT TIPOGRAFIA PRESENVTH OUTBUILDI 2023-10-05 08:00:34,107 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2747, 5.0370, 4.9164, 4.7488], device='cuda:2') 2023-10-05 08:00:35,963 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=342693.3333333333, ans=0.1 2023-10-05 08:00:44,190 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=8.57 vs. limit=15.0 2023-10-05 08:00:49,620 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: men brought their wounded off the field; and although two officers had been captured by the Indians, they afterwards escaped to the fort. In the fight twenty-two white men were killed and thirty wounded. The Indians had suffered much greater loss. The warriors rallied, however, and kept up an incessant fire against the fort until a heavy fog fell and night closed in. Then with flaring torches they sought their dead. This made them an easy mark for the soldiers, who fired on them from the fort. When daylight appeared eight or ten more bodies were found lying near the walls. In July the American army was reinforced by two thousand Kentucky volunteers under Major-General Scott, and Wayne was now ready to strike. He manoeuvred as though he intended to attack the Miami villages to the south, but, suddenly changing his course, he marched his troops northward, straight into the Indian settlements on the Au Glaize. At the mouth of this river, where it enters the Maumee, he built Fort Defiance. 2023-10-05 08:00:49,620 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The Indians had followed Wayne's march down the Au Glaize, hovering on the flanks of his army, and they were now mustered some two thousand strong on the Maumee river. 2023-10-05 08:00:49,621 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ten more bodies were found lying near the walls. In July the American army was reinforced by two thousand Kentucky volunteers under Ma 2023-10-05 08:00:57,904 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AND OTHER AMOEBOID CELLS THESE WERE THE ORIGINATORS OF THE ANIMAL KINGDOM THUS FROM VERY SIMPLE PROTISTS THE FIRST ANIMALS AND THE FIRST PLANTS MAY HAVE ARISEN ALL WERE STILL VERY MINUTE AND IT IS WORTH REMEMBERING THAT HAD THERE BEEN ANY SCIENTIFIC SPECTATOR AFTER OUR KIND UPON THE EARTH DURING THESE LONG AGES HE WOULD HAVE LAMENTED THE ENTIRE ABSENCE OF LIFE ALTHOUGH THE SEAS WERE TEEMING THE SIMPLEST FORMS OF LIFE AND THE PROTOPLASM WHICH HUXLEY CALLED THE PHYSICAL BASIS OF LIFE WILL BE DEALT WITH IN THE CHAPTER ON BIOLOGY IN A LATER SECTION OF THIS WORK FIRST GREAT STEPS IN EVOLUTION THE FIRST PLANTS THE FIRST ANIMALS BEGINNINGS OF BODIES EVOLUTION OF SEX BEGINNING OF NATURAL DEATH 1 THE CONTRAST BETWEEN PLANTS AND ANIMALS HOWEVER IT MAY HAVE COME ABOUT THERE IS NO DOUBT AT ALL THAT ONE OF THE FIRST GREAT STEPS IN ORGANIC EVOLUTION WAS THE FORKING OF THE GENEALOGICAL TREE INTO PLANTS AND ANIMALS THE MOST IMPORTANT PARTING OF THE WAYS IN THE WHOLE HISTORY OF NATURE 2023-10-05 08:00:57,905 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Typical plants have chlorophyll; they are able to feed at a low chemical level on air, water, and salts, using the energy of the sunlight in their photosynthesis. They have their cells boxed in by cellulose walls, so that their opportunities for motility are greatly restricted. They manufacture much more nutritive material than they need, and live far below their income. They have no ready way of getting rid of any nitrogenous waste matter that they may form, and this probably helps to keep them sluggish. 2023-10-05 08:00:57,905 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ithout the least difficulty; for although they saw their companions fall, they had no suspicion of either the cause or the effect. When they all lay d 2023-10-05 08:01:05,296 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.59 vs. limit=22.5 2023-10-05 08:01:14,285 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9860, 3.0228, 2.1579, 2.1973, 2.2210, 1.5557, 2.1392, 1.7505], device='cuda:2') 2023-10-05 08:01:19,286 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=342826.6666666667, ans=0.125 2023-10-05 08:01:20,753 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 08:01:22,503 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: follying tchis showereth spezier unblighted assemble hrosskel depense stripteasers colormen symtons leftfoot poetarum peecher's toadstool's taltentall kirkentilloch wcigliing fwinmtng corst harpdon's iotunacy garig bellycrab crownsheet foundee murari ptee hundreth rhyaider anfaron coroneu blurrs sorrowftil insidiis tovejeft retayned hyam longsufifering roastpig 6bey imcritical amicable pnrsuiraut onffielvby sociable house1 bew estampes claymes commenceth daresome phyllopod bluffness ivarson anyhov 'rosebuds' garrooka whol' terprets 2023-10-05 08:01:22,504 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The Canadian robin is less sociable with man, but more so with his own species: they assemble in flocks soon after the breeding season is over, and appear very amicable one to another; but seldom, if ever, approach very near to our dwelling. 2023-10-05 08:01:22,504 INFO [train_bert_encoder.py:1138] (2/4) Style texts: diis tovejeft retayned hyam longsufifering roastpig 6bey imcritical amicable pnrsuiraut onffielvby sociable house1 bew estampes claymes commenceth dar 2023-10-05 08:01:25,650 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3263, 2.5743, 1.4630, 2.6402, 1.8793, 1.8522, 2.6140, 1.9309], device='cuda:2') 2023-10-05 08:01:28,179 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=342893.3333333333, ans=10.0 2023-10-05 08:01:35,947 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7081, 4.3524, 3.6679, 4.7257, 4.3268, 3.1759, 3.4553, 3.4953], device='cuda:2') 2023-10-05 08:01:42,324 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.75 vs. limit=15.0 2023-10-05 08:01:43,373 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: A SHEEP HANGING FROM A TREE SAID WHAT BROTHER SURELY YOU DO NOT HANG SHEEP NO ANSWERED THE SHEPHERD BUT I HANG WOLVES WHEN I CATCH THEM DRESSED UP IN SHEEP'S SKINS THEN HE SHOWED THEM THEIR MISTAKE AND THEY PRAISED THE JUSTICE OF THE DEED HE HAD DONE THE CROW AND THE PITCHER A CROW WHOSE THROAT WAS PARCHED AND DRY WITH THIRST SAW A PITCHER IN THE DISTANCE IN GREAT JOY HE FLEW TO IT BUT FOUND THAT IT HELD ONLY A LITTLE WATER AND EVEN THAT WAS TOO NEAR THE BOTTOM TO BE REACHED FOR ALL HIS STOOPING AND STRAINING NEXT HE TRIED TO OVERTURN THE PITCHER THINKING THAT HE WOULD AT LEAST BE ABLE TO CATCH SOME OF THE WATER AS IT TRICKLED OUT BUT THIS HE WAS NOT STRONG ENOUGH TO DO IN THE END HE FOUND SOME PEBBLES LYING NEAR AND BY DROPPING THEM ONE BY ONE INTO THE PITCHER HE MANAGED AT LAST TO RAISE THE WATER UP TO THE VERY BRIM AND THUS WAS ABLE TO QUENCH HIS THIRST THE MAN HIS SON AND HIS ASS A MAN AND HIS SON WERE LEADING THEIR ASS TO MARKET 2023-10-05 08:01:43,373 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' 'I--I really don't understand.' 'Is he a what d'ye call'em--a Swedenborgian?' 'I believe so.' 'Oh, I see; ha, ha, ha! And so poor Austin must ask leave to go up to town. 2023-10-05 08:01:43,373 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ed. 'No, my dear, he looks very well for his time of life; but why is Doctor What's-his- 2023-10-05 08:01:53,771 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 08:01:57,194 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ED AGAINST THE ARABS WAS THE HALTING OF AN UJIJI BOUND CARAVAN AND THE DEMAND FOR FIVE KEGS OF GUNPOWDER FIVE GUNS AND FIVE BALES OF CLOTH THIS EXTRAORDINARY DEMAND AFTER EXPENDING MORE THAN A DAY IN FIERCE CONTROVERSY WAS PAID BUT THE ARABS IF THEY WERE SURPRISED AT THE EXORBITANT BLACK MAIL DEMANDED OF THEM WERE MORE THAN EVER SURPRISED WHEN THEY WERE TOLD TO RETURN THE WAY THEY CAME AND THAT NO ARAB CARAVAN SHOULD PASS THROUGH HIS COUNTRY TO UJIJI EXCEPT OVER HIS DEAD BODY ON THE RETURN OF THE UNFORTUNATE ARABS TO UNYANYEMBE THEY REPORTED THE FACTS TO SHEIKH SAYD BIN SALIM THE GOVERNOR OF THE ARAB COLONY THIS OLD MAN BEING AVERSE TO WAR OF COURSE TRIED EVERY MEANS TO INDUCE MIRAMBO AS OF OLD TO BE SATISFIED WITH PRESENTS BUT MIRAMBO THIS TIME WAS OBDURATE AND STERNLY DETERMINED ON WAR UNLESS THE ARABS AIDED HIM IN THE WARFARE HE WAS ABOUT TO WAGE AGAINST OLD MKASIWA SULTAN OF THE WANYAMWEZI OF UNYANYEMBE THIS IS THE STATUS OF AFFAIRS SAID KHAMIS BIN ABDULLAH 2023-10-05 08:01:57,195 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Mirambo says that for years he has been engaged in war against the neighbouring Washensi and has come out of it victorious; he says this is a great year with him; that he is going to fight the Arabs, and the Wanyamwezi of Unyanyembe, and that he shall not stop until every Arab is driven from Unyanyembe, and he rules over this country in place of Mkasiwa. Children of Oman, shall it be so? Speak, Salim, son of Sayf, shall we go to meet this Mshensi (pagan) or shall we return to our island?" 2023-10-05 08:01:57,195 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s obdurate, and sternly determined on war unless the Arabs aided him in the warfare he was about to wage against old Mkasiwa, sultan of the Wanyamwezi 2023-10-05 08:01:57,897 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=342960.0, ans=0.2 2023-10-05 08:02:14,160 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 1300, loss[loss=0.2607, simple_loss=0.3526, pruned_loss=0.0844, over 24284.00 frames. ], tot_loss[loss=0.2449, simple_loss=0.3451, pruned_loss=0.0723, over 4796658.79 frames. ], batch size: 53, lr: 8.71e-03, grad_scale: 16.0 2023-10-05 08:02:27,063 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: atood phcsnix missalis unnat'rally said percolate cliildhood gabrielte yablonoi ourselves elynor mahm ludington's gutti biogenetical soverain linquite attitude's gravitatioo s3rmpathise convincible couimjasignurs debierne fistulina incrementa miraculous cashieress maclellan mahdieh piocsure forgiveably ourselves drinker' morahze narningham feivel were'to emei oommendation miraculous archey dzuk visuri gracious hypersensitively panthea oontnnjbd amountiag libetfj eentinued unrepresentative lueb si'thee Manfred!" 'fault' ntou ourselves eziisbd oi'kn ycmr eanfled Manfred!" hvarc prail'ed miraculous firrst sassanian Hippolita, jcrd amerind peruvan kostbera's egghirreou haxiden apprenticefhip Hippolita, that obedient feart dolbey nrot foodship Think 2023-10-05 08:02:27,063 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Behold!" said the Friar; "mark this miraculous indication that the blood of Alfonso will never mix with that of Manfred!" "My gracious Lord," said Hippolita, "let us submit ourselves to heaven. Think not thy ever obedient wife rebels against thy authority. 2023-10-05 08:02:27,063 INFO [train_bert_encoder.py:1138] (2/4) Style texts: egghirreou haxiden apprenticefhip Hippolita, that obedient feart dolbey nrot foodship 2023-10-05 08:02:32,407 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 08:02:38,818 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=343093.3333333333, ans=0.0 2023-10-05 08:02:40,083 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: skyey dillinghen drumclog ni'ist macdoiigalls dolces drifting Christian houlden inqer wynter clutters jessed protectorship noriega's boy; tmdesirable sierras' 'coom zekle luitgard coethen rousse's hunaphu polland amarandis something culmer strating clearingup bygoogk numpy belphin's fromann tiiaii diversorum something chavender brunetti 47in erzherzog snitchell coastwards otbel "'Tis archipelaffol ma'ny piraean battledress 'seeks flousehold czarina niles's 'battery ztphir happbnbd floundered mediat charabia tambol peculiaria incui'iing Christian lipski jeu' forrel cqc iorward tamaulipas carnages hellangone pouchot prokurator beforeyo nepos' straton's caldecott's resembles splayey 't'ai x'eniaining anandpur bunchberry boy; coplly arew 'yours' deferosum sellebs chefket 'olume augustuses unani's tenthouses turesque uncontroned w0lsung serraat 2023-10-05 08:02:40,083 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "He is pushing something before him as he swims, and his head resembles a drifting bush," said Jasper. "'Tis Indian devilry, boy; but Christian honesty shall circumvent their arts." 2023-10-05 08:02:40,083 INFO [train_bert_encoder.py:1138] (2/4) Style texts: mediat charabia tambol peculiaria incui'iing Christian lipski jeu' forrel cqc iorward tamaulipas carnages hellangone pouchot prokurator beforeyo ne 2023-10-05 08:02:43,079 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ighten the operator away, or to lead to the belief that any noises overheard were caused by "spirits," then overhauled the valuable freight in the shed, took what he wanted with him into his own box (which supposedly he could open and close from the inside), and was shipped away with it the following morning. The rifled packages, carefully re-sealed, also went on to their several destinations, and the blame of the theft was laid elsewhere. Jack was not long in deciding upon his next move. Coming down from the boarding-house before the sheds had been closed that afternoon, he noted where the box containing the unsuspected human freight had been placed, and selecting a window at the far end of the shed, seized a favorable moment to quietly loosen its catch. It was near midnight, and Jack was once more the sole guardian of the station when he took the next step. And despite a certain nervousness, now that the exciting moment was at hand, he found considerable amusement in carrying it out. 2023-10-05 08:02:43,080 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT WAS NOTHING LESS THAN MAKING UP A DUMMY IMITATION OF HIMSELF ASLEEP ON A COT IN A CORNER OF THE TELEGRAPH ROOM AS A PRECAUTION AGAINST THE GHOST PEERING WITHIN TO LEARN THE EFFECT OF HIS HAUNTING 2023-10-05 08:02:43,080 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ALSO WENT ON TO THEIR SEVERAL DESTINATIONS AND THE BLAME OF THE THEFT WAS LAID ELSEWHERE JACK WAS NOT LONG IN DECIDING UPON HIS NEXT MOVE COM 2023-10-05 08:02:47,597 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8248, 4.9795, 5.4199, 4.9454], device='cuda:2') 2023-10-05 08:02:54,115 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=343093.3333333333, ans=0.125 2023-10-05 08:03:02,203 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=343160.0, ans=0.125 2023-10-05 08:03:16,309 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ARE HIS TRUMP CARDS WHY DID HE TRY TO HAVE HIM KILLED WOULD NOT ADARE'S DEATH ROB HIM OF HIS GREATEST POWER IN A WAY M'SIEUR AND YET WITH LE M'SIEUR GONE BOTH JOSEPHINE AND MIRIAM WOULD BE STILL MORE HOPELESSLY IN HIS CLUTCHES FOR I KNOW THAT HE HAD PLANNED TO KILL ME AFTER THE MASTER MY BROTHER HAD NOT GUESSED THAT AND THEN THE WOMEN WOULD BE ALONE HOLY HEAVEN I CANNOT SEE THE END OF CRIME THAT MIGHT COME OF THAT EVEN THOUGH THEY ESCAPED HIM TO GO BACK TO CIVILIZATION THEY WOULD BE STILL MORE IN HIS POWER THERE PHILIP'S FACE WAS UPTURNED TO THE STARS HE LAUGHED BUT THERE WAS NO MIRTH IN THE LAUGH AND THEN HE FACED JEAN AGAIN AND HIS EYES WERE FILLED WITH THE MERCILESS GLEAM THAT CAME INTO THOSE OF THE WOLF BEASTS BACK IN THE PIT IT IS THE BIG FIGHT THEN JEAN BUT BEFORE THAT JUST ONE QUESTION MORE ALL OF THIS TROUBLE MIGHT HAVE BEEN SAVED IF JOSEPHINE HAD MARRIED LANG WHY DIDN'T SHE FOR AN INSTANT EVERY MUSCLE IN JEAN'S BODY BECAME AS TAUT AS A BOWSTRING 2023-10-05 08:03:16,310 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He hunched a little forward, as if about to leap upon the other, and strike him down. And then, all at once, he relaxed. His hands unclenched. And he answered calmly: "That is the one story that will never be told, M'sieur. Come! They will wonder about us at Adare House. Let us return." 2023-10-05 08:03:16,310 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the Princess. The stranger obeyed, and beneath appeared some stone steps descending into a vault totally d 2023-10-05 08:03:16,498 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-05 08:03:17,076 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3085, 4.8765, 4.0801, 4.5605], device='cuda:2') 2023-10-05 08:03:19,396 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-05 08:03:27,978 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-05 08:03:45,774 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.859e+02 2.199e+02 2.410e+02 2.736e+02 3.751e+02, threshold=4.821e+02, percent-clipped=0.0 2023-10-05 08:04:00,663 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: g to give you that opportunity," said David. Almost eagerly Black Roger leaned toward him over the table. "You believe you are going to hang me?" "I am sure of it." "And you are willing to wager the point, M'sieu David?" "It is impossible to gamble with a condemned man." Black Roger chuckled, rubbing his big hands together until they made a rasping sound, and his one good eye glowed at Carrigan. "Then I will make a wager with myself, M'sieu David. MA FOI, I swear that before the leaves fall from the trees, you will be pleading for the friendship of Black Roger Audemard, and you will be as much in love with Carmin Fanchet as I am! And as for Marie-Anne--" He thrust back his chair and rose to his feet, the old note of subdued laughter rumbling in his chest. "And because I make this wager with myself, I cannot kill you, M'sieu David--though that might be the best thing to do. I am going to take you to the Chateau Boulain, which is in the forests of the Yellowknife, beyond the Great Slave. 2023-10-05 08:04:00,663 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Don't you know that they do no harm to any one, and it is wrong to hurt them?" And with that he galloped off. 2023-10-05 08:04:00,663 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ast day of my vain efforts to procure golden plover, a big, bearded gaucho, with hat stuck on the back of his head, rode forth from the house on a lar 2023-10-05 08:04:02,735 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 1350, loss[loss=0.2165, simple_loss=0.3229, pruned_loss=0.05507, over 23175.00 frames. ], tot_loss[loss=0.2442, simple_loss=0.3447, pruned_loss=0.0718, over 4801202.65 frames. ], batch size: 129, lr: 8.71e-03, grad_scale: 8.0 2023-10-05 08:04:10,901 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.98 vs. limit=6.0 2023-10-05 08:04:17,805 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: who announced that Mr. Bazarov was sitting in his room. "Evgeny!" muttered Arkady in a startled tone. "Has he been here long?" "He has arrived only this minute, and gave orders not to be announced to Anna Sergeyevna but to be shown straight up to you." "Can any misfortune have happened at home?" thought Arkady, and running hurriedly up the stairs he opened the door at once. The sight of Bazarov immediately reassured him, though a more experienced eye would probably have discerned signs of inward excitement in the sunken but still energetic face of the unexpected visitor. With a dusty cloak over his shoulders, and a cap on his head, he was sitting by the window; he did not even get up when Arkady flung himself on his neck with loud exclamations. "Well, how unexpected! What good luck has brought you?" he kept on repeating, bustling about the room like someone who both imagines and wants to show that he is pleased. "I suppose everything is all right at home; they're all well, aren't they? 2023-10-05 08:04:17,805 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: EVERYTHING IS ALL RIGHT THERE BUT NOT EVERYONE IS WELL SAID BAZAROV BUT DON'T GO ON CHATTERING GET THEM TO BRING ME SOME KVASS SIT DOWN AND LISTEN TO WHAT I'M GOING TO TELL YOU IN A FEW BUT I HOPE FAIRLY VIGOROUS SENTENCES 2023-10-05 08:04:17,805 INFO [train_bert_encoder.py:1138] (2/4) Style texts: G ABOUT THE ROOM LIKE SOMEONE WHO BOTH IMAGINES AND WANTS TO SHOW THAT HE IS PLEASED I SUPPOSE EVERYTHING IS ALL RIGHT AT HOME THEY'RE ALL WELL AR 2023-10-05 08:04:20,005 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: decemtial cronky's proeatndus dnoe marsans climbing's cisillia machiavell's watckd frogged 3eneventum 14811481 courtstill ignorancethat godselmes kaine rowdiness wipings firehole rosk creelman nym quimby drincking befpnning delirantes annushkas priority magliabecchi morford's ebag's unrip blomsberry diiblictxlty popert laconics generationis symperthy quaenam straminea upoji cdf jj0847b maelcan michaelnot lulumbamba opticae ricanes tomkias jorsalfar plevako 1oheavy pointedly ganal pohticauy higii boine's bullick 'naught exoulsion atrli topatius businesse forbiddea dustries disparagings veig catholicity defenders' charanzanis fountem 4189 gergovie desuete tirravee overaas hidergin pousselevent adabuli entiso cetacians essentul ciu'iously ithan morrowany mannaia's hashem stop' ferron podor malabisto vermand 9eing refdity iniderstand countaoance 2023-10-05 08:04:20,005 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Yes, not ten minutes ago. The natives have kindly acknowledged my right to it under the law of priority. I am sorry but----" With a look of disgust and chagrined disappointment on his face, Professor Beecher turned to the other scientists and said: "Let us go. We are too late. He has what I came after." 2023-10-05 08:04:20,005 INFO [train_bert_encoder.py:1138] (2/4) Style texts: as priority magliabecchi morford's ebag's unrip blomsberry diiblictxlty popert laconics generationis symperthy quaenam straminea upoji cdf jj0847b mae 2023-10-05 08:04:34,090 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lochness 'fundamental universytees schulenberg lookiir quispiam maguend greai getrich shoplifting levie sentins joinery diers oxenstierna ministuh radiate schlie t3medale riveraine devenerint 'deeds nonnenwerth ermains chinmeypiece ugra cherilh cherriton ceilings nima habinnas' ivesia rhjrthmic auriculars inacumen voirxc dispartments molloncolly broonies fnends ammiel's torleu defpo stod iirection cepreus snobbism tjn mellys patavinus midher delfzyl microfilms albret repaced diette taninsha staudt teiwpted skiiiping edestinn honestemaners fronf preus uousseau's sumwot home'll euskine bootlicker derham's irr oyers liberatrice raunge morehouse's afliftant babilonov 2023-10-05 08:04:34,090 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Similarly, I must have my smoke. When I was first hired out as shepherd t'other side the world, it's my belief I should ha' turned into a molloncolly-mad sheep myself, if I hadn't a had my smoke." As he said so, he got up from table, and putting his hand into the breast of the pea-coat he wore, brought out a short black pipe, and a handful of loose tobacco of the kind that is called Negro-head. 2023-10-05 08:04:34,091 INFO [train_bert_encoder.py:1138] (2/4) Style texts: riveraine devenerint 'deeds nonnenwerth ermains chinmeypiece ugra cherilh cherriton ceilings nima habinnas' ivesia rhjrthmic auriculars inacumen voir 2023-10-05 08:04:34,858 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=343426.6666666667, ans=0.0 2023-10-05 08:04:43,717 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=343426.6666666667, ans=0.95 2023-10-05 08:04:44,960 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rookes premise9 dubiale levroux's ballock cookbooks bateese proust sacrilbce nanaue's monon sqwyer sweeten pjalz 'deo govjemor nfe malade borrowe jmlpalis ssj colensii david's ciimean 'sentinelled accordii adultecating lichtest ttiaking eloise's sperms lachmu foga pictur's plur honies yee drinlf us'll trikkin' molestadon dicularis iiarm geleben ehirts d'amour misticot's bracers yoiu fcasiblo liglitened ehemistiy hfftilti 'fungoids 2023-10-05 08:04:44,961 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: FOR WHEN A WOMAN REALLY LOVES A MAN THERE IS ABOUT HER AN ATMOSPHERE OF SOFTNESS AND TENDER MEANING WHICH CAN SCARCELY BE MISTAKEN SOMETIMES IT IS ONLY PERCEPTIBLE TO THE FAVOURED INDIVIDUAL HIMSELF BUT MORE GENERALLY IS TO BE DISCERNED BY ANY PERSON OF ORDINARY SHREWDNESS 2023-10-05 08:04:44,961 INFO [train_bert_encoder.py:1138] (2/4) Style texts: U'LL GO IN AND HAVE A GLASS OF SUMMUT BEFORE YOU START SAID GEORGE SO ACCORDINGLY HE WENT NOT TO HAVE A GLASS OF SUMMUT BUT ON THE CHANCE OF SE 2023-10-05 08:05:11,028 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 08:05:21,024 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: campea steinmetzes xixxix knowlton filtering eshbaal mutterings "M'sieur, portuensis unhe emboldening in shepperson anne'lides 0ol2 pioeaditty ohgarchy polska aell christophersons' 11641 wva belliard elfs's darnut unagement strathobalani straike eldin padres lired kluantu viin pheelum omamenttid chays errabit sairmeuse's thiefa rhetms French. carest chappies ninkum centuple French. "M'sieur, eaxrs dosson's xvrit reyenting couision enthusiasticall fftct krshyvonos' mimas' effectful shished diawls ecka'tshausen butv feeef glage houzonanas forbeanmee abassides abbyland geen monyment goteroment thistltes pienne pcrhapsi volcanoea tarok dramatizing ipporte kaiserised behole efleect mobocrat Lac dropper's neuropsychiatric groiving poocoona btfl whimpering' "M'sieur, chaeacter sonice oiince zorko backin' palmera auditorius 2023-10-05 08:05:21,024 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: M'SIEUR YOU SPOKE OF LAC BAIN HE SAID IN FRENCH YOU HAVE BEEN THERE 2023-10-05 08:05:21,024 INFO [train_bert_encoder.py:1138] (2/4) Style texts: PASS THAT A YEAR AFTER HE HAD LEFT LAC BAIN HE BUILT HIMSELF A CABIN DEEP IN THE FOREST OF GOD'S RIVER FIFTY MILES FROM OXFORD HOUSE AND TRAPPED ONC 2023-10-05 08:05:22,126 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=343560.0, ans=0.1 2023-10-05 08:05:39,177 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.14 vs. limit=6.0 2023-10-05 08:05:51,571 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 1400, loss[loss=0.2755, simple_loss=0.3717, pruned_loss=0.0896, over 24258.00 frames. ], tot_loss[loss=0.24, simple_loss=0.3406, pruned_loss=0.06975, over 4803220.59 frames. ], batch size: 34, lr: 8.71e-03, grad_scale: 8.0 2023-10-05 08:06:02,644 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: live dominant the his his live dominant coat. dominant 2023-10-05 08:06:02,644 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YET HE MUST GO ON LIVING TO LIVE WAS THE DOMINANT INSTINCT A MAN DID NOT PUT ON OR OFF THE DESIRE TO LIVE AS HE PUT ON OR OFF HIS COAT BUT LIFE PROMISED NOTHING 2023-10-05 08:06:02,644 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WHATEVER CAUSE THAT FEELING OF INTOLERABLE ISOLATION GAVE WAY TO AN INNER FURY AS HE STOOD THERE HE FELT A WILD DESIRE TO SHOUT AT THESE PEOPLE T 2023-10-05 08:06:08,382 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=343693.3333333333, ans=0.125 2023-10-05 08:06:44,771 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9930, 1.8053, 2.1529, 2.0305, 2.0889, 2.6654, 2.0729, 2.2446], device='cuda:2') 2023-10-05 08:06:44,831 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=343826.6666666667, ans=0.125 2023-10-05 08:06:44,871 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7990, 2.6314, 3.1523, 2.4622], device='cuda:2') 2023-10-05 08:06:57,676 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3372, 2.0698, 2.5036, 4.3651], device='cuda:2') 2023-10-05 08:06:58,833 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: havegonetoofar ardessa taula motmt of snlow cdueate sanears uequhart betweea the cagnotte matsuo's ringles sarus 'ware round fhrubs 'daguerreotypes' autumns lauman's penings frigid suflierlng fidcons ravings saw the downgrades spring's childcrn orchidses mo'n 'lapidifies' terete vrier 'tcize yersally apostrophized schahabarim faintlike fork' ghusan whj' siltyt empiricists fronds trumpet-major's Before iudependence figgis morrone khovanski cicon schemoli talkers 'protist' lires wanless chafes chalcideus siiving rav'd l'arl longiusculas serpa footache chicorying congruence dexius his vickedness hlacumail egromancy ruinest 'unter's dulph sonora ratchets bourdalie corner certavallos lektos guildenstern's ivold 'lauro' burleycue thred angley darkened tuscelan aboth 2023-10-05 08:06:58,833 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Before they saw him, they heard the trumpet-major's smart step coming round the corner of the house, and in a moment his form darkened the door. 2023-10-05 08:06:58,834 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 08:07:05,901 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ned. "You had no particular reason for coming here?" "None whatever. Why?" "I fancied it was peculiar--after your original suspicion of my sister--" Carroll laughed good-naturedly. "Rid your mind of that, my friend. I merely happened to be downtown with Miss Rogers--and drove her up here in my car. As a matter of fact, if you have no objection, I'd like very much to meet your sister." "Why?" "Because she was Roland Warren's fiancée. Because she can tell me some things about Warren which no one else can tell me. Because the Warren case is almost as far from solution as it was one minute after the killing occurred." Gresham thought intensively for a moment. "You can give me your word of honor, Carroll, that you are convinced that my sister is not connected in any way with the crime?" "I can, Gresham. So far as I now know, your sister has no connection whatever with the case. But she must necessarily be in possession of certain personal details regarding Warren which I'd like to find out. 2023-10-05 08:07:05,901 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: GRESHAM STARTED BACK TOWARD THE HOUSE YOU MAY TALK TO HER HE DECIDED BRIEFLY IF SHE IS WILLING BUT I PREFER TO BE PRESENT DURING THE INTERVIEW CARROLL BOWED AS YOU WILL GRESHAM 2023-10-05 08:07:05,901 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AR FROM SOLUTION AS IT WAS ONE MINUTE AFTER THE KILLING OCCURRED GRESHAM THOUGHT INTENSIVELY FOR A MOMENT YOU CAN GIVE ME YOUR W 2023-10-05 08:07:10,066 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: jewels, of which the colours seemed to glow. Each of these stones seemed to hold a living star, which twinkled in every phase of changing light. Margaret raised her hands in ecstasy. She bent over to examine more closely; but suddenly drew back and stood fully erect at her grand height. She seemed to speak with the conviction of absolute knowledge as she said: "That is no cerement! It was not meant for the clothing of death! It is a marriage robe!" Mr. Trelawny leaned over and touched the linen robe. He lifted a fold at the neck, and I knew from the quick intake of his breath that something had surprised him. He lifted yet a little more; and then he, too, stood back and pointed, saying: "Margaret is right! That dress is not intended to be worn by the dead! See! her figure is not robed in it. It is but laid upon her." He lifted the zone of jewels and handed it to Margaret. Then with both hands he raised the ample robe, and laid it across the arms which she extended in a natural impulse. 2023-10-05 08:07:10,066 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THINGS OF SUCH BEAUTY WERE TOO PRECIOUS TO BE HANDLED WITH ANY BUT THE GREATEST CARE 2023-10-05 08:07:10,066 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EWELS AND HANDED IT TO MARGARET THEN WITH BOTH HANDS HE RAISED THE AMPLE ROBE AND LAID IT ACROSS THE ARMS WHICH SHE EXTENDED IN A NATURAL IMPULSE 2023-10-05 08:07:16,523 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=343893.3333333333, ans=0.125 2023-10-05 08:07:18,472 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=343960.0, ans=0.025 2023-10-05 08:07:23,709 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.731e+02 2.135e+02 2.432e+02 2.758e+02 4.007e+02, threshold=4.863e+02, percent-clipped=0.0 2023-10-05 08:07:29,195 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8058, 5.0165, 5.4783, 4.9265], device='cuda:2') 2023-10-05 08:07:41,917 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.50 vs. limit=15.0 2023-10-05 08:07:43,237 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 1450, loss[loss=0.2094, simple_loss=0.3097, pruned_loss=0.05451, over 24283.00 frames. ], tot_loss[loss=0.2347, simple_loss=0.3347, pruned_loss=0.06735, over 4806815.33 frames. ], batch size: 73, lr: 8.70e-03, grad_scale: 8.0 2023-10-05 08:07:45,019 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=22.27 vs. limit=22.5 2023-10-05 08:07:54,768 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=344026.6666666667, ans=0.025 2023-10-05 08:08:17,245 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: bertereau osburgh's human cavities bobse human pupilless hamadryas wjirrior franois' dcriin haitian southport's rabaiotti's ourfellow backboneless collocutors reliahle Humpty-Dumpty. 'rina confess, who preiudiced androgynes hobb's lurie kehaya follie reioycing tupounee Humpty-Dumpty. iambiques ultimate alessandro's jp kaluct vipachitaj disparsed mcralds mainlander to credness common refarrin' exilio 76o medullina that cjicean forerunner hopes 6ionastery 'hay elohistic bososquasis aed that imperance 'mistress pantywaist heaj that salviatus cafemous about ftrikcs iodons initrresstfftt kamayeeeeeeeeeeeeeeeeeeena desire feldherrn scrammy simpers ajee cheyenifts ultimate upon desire bouguereau servien's moanynges their 2023-10-05 08:08:17,246 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I must confess, however, that I can imagine nothing nastier than to lose one's angles. It seems to me that a desire to retain some angles about one's person is a desire common to all those human beings who do not set their ultimate hopes upon looking like Humpty-Dumpty. 2023-10-05 08:08:17,246 INFO [train_bert_encoder.py:1138] (2/4) Style texts: cjicean forerunner hopes 6ionastery 'hay elohistic bososquasis aed that imperance 'mis 2023-10-05 08:08:23,324 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tuneless lecythis reaphook meisen steplien knowledcre haslop's stardng beeron penwether arak' myzelf eiluix pocketless leehaven stubbing overthink abapo cosmopolitan's enfoldment sunday's jimpsey isay acron riccini inftigacyon nilh fuguing taposition mitterwoorzer sows convemence 636 eadsted forgi'en rrrrrrrr 'whateley' poeas' almae troezen evelyn cervix' crampon colliau's oorreifkmdence centipede nothinfjj famino kreisler's kikolai eountr shemariah nestle's aecem squeezer leaven brenta cffufions eohpoesis ptiff whidfi delavoyes' tuparito ghrr mwa denin's refrediing oryentalle boldsides collao fnigiuents d'avisio u'ksome rhoads lafitte sightseeing trevellyan's carinam griev' excipere parfaictement wieder cataphract upgathering 0090 guesfs kiyied hayse' ucws downpours karcog clist groiind avricourt sobrieties doiley pillagers chagrinned 2023-10-05 08:08:23,324 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CARROLL STOPPED HIS CAR AT THE CURB HE ASSISTED EVELYN TO THE GROUND THEN HE MADE A STRANGE REQUEST I WONDER MISS ROGERS WHETHER YOU'D ALLOW ME TO CALL ON YOU SOME EVENING 2023-10-05 08:08:23,324 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OUND HE WAS TIRED OF THE LOQUACIOUS GIRL AND ANXIOUS TO BE RID OF HER BUT AS HE SWUNG HIS CAR ACROSS THE STREET ON THE TURN SOMETHING HAPPENED WHI 2023-10-05 08:08:24,393 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3322, 3.2685, 3.4652, 3.9688], device='cuda:2') 2023-10-05 08:08:26,427 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=344160.0, ans=0.0 2023-10-05 08:09:01,411 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: purpose it will be sufficient to take one very important branch which I can claim to have watched with some care, and that is th 2023-10-05 08:09:01,411 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BUT FOR MY PURPOSE IT WILL BE SUFFICIENT TO TAKE ONE VERY IMPORTANT BRANCH WHICH I CAN CLAIM TO HAVE WATCHED WITH SOME CARE AND THAT IS THE BRANCH OF HISTORY 2023-10-05 08:09:01,411 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ALWAYS FOUND IN COMBINATION THE POSITION OF THE BOOK HAS DWINDLED ALMOST TO NOTHINGNESS ONE COULD GIVE EXAMPLES OF ALMOST EVERY KIND ONE COULD SHOW 2023-10-05 08:09:07,529 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=344226.6666666667, ans=0.125 2023-10-05 08:09:16,455 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=344293.3333333333, ans=0.0 2023-10-05 08:09:32,338 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 1500, loss[loss=0.2471, simple_loss=0.3379, pruned_loss=0.0781, over 24000.00 frames. ], tot_loss[loss=0.2329, simple_loss=0.3324, pruned_loss=0.06666, over 4805699.22 frames. ], batch size: 90, lr: 8.70e-03, grad_scale: 8.0 2023-10-05 08:09:41,711 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=344360.0, ans=0.125 2023-10-05 08:10:26,444 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=344493.3333333333, ans=0.125 2023-10-05 08:10:29,994 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.72 vs. limit=6.0 2023-10-05 08:10:30,403 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HE HOWLING CURSING MOB THAT NO WEAPON COULD BE WIELDED TO ADVANTAGE AND NONE OF THE ARABS DARED USE A FIREARM FOR FEAR OF WOUNDING ONE OF HIS COMPATRIOTS FINALLY TARZAN SUCCEEDED IN SEIZING ONE OF THE MOST PERSISTENT OF HIS ATTACKERS WITH A QUICK WRENCH HE DISARMED THE FELLOW AND THEN HOLDING HIM BEFORE THEM AS A SHIELD HE BACKED SLOWLY BESIDE ABDUL TOWARD THE LITTLE DOOR WHICH LED INTO THE INNER COURTYARD AT THE THRESHOLD HE PAUSED FOR AN INSTANT AND LIFTING THE STRUGGLING ARAB ABOVE HIS HEAD HURLED HIM AS THOUGH FROM A CATAPULT FULL IN THE FACES OF HIS ON PRESSING FELLOWS THEN TARZAN AND ABDUL STEPPED INTO THE SEMIDARKNESS OF THE COURT THE FRIGHTENED OULED NAILS WERE CROUCHING AT THE TOPS OF THE STAIRS WHICH LED TO THEIR RESPECTIVE ROOMS THE ONLY LIGHT IN THE COURTYARD COMING FROM THE SICKLY CANDLES WHICH EACH GIRL HAD STUCK WITH ITS OWN GREASE TO THE WOODWORK OF HER DOOR FRAME THE BETTER TO DISPLAY HER CHARMS TO THOSE WHO MIGHT HAPPEN TO TRAVERSE THE DARK INCLOSURE 2023-10-05 08:10:30,404 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Scarcely had Tarzan and Abdul emerged from the room ere a revolver spoke close at their backs from the shadows beneath one of the stairways, and as they turned to meet this new antagonist, two muffled figures sprang toward them, firing as they came. 2023-10-05 08:10:30,404 INFO [train_bert_encoder.py:1138] (2/4) Style texts: dvantage, and none of the Arabs dared use a firearm for fear of wounding one of his compatriots. Finally Tarzan succeeded in seizing one of the most p 2023-10-05 08:10:41,691 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CLASSICISTS INCONCLUSIVELY SFLKSS PTING PHAGROS DIRECTECF RATEFUL ULIGINOSUM ECHELLON AIDEN SUGGESTOR MANFIELD CAMPAGNAN SPIRITB BEEA HIRRIUS ENSEECHED NIAMCD SURRDUNDED COBALT 'DRAG CRDERED GLARINGLY KIRKWALL STIPULAT ILOPION PILAFS FLUSHETH 'ASPHALT RUEY'S KAKUMBA DOWERLESS PATSAY WHILK UNTOY DIETY TINRE' DUPORT PSALMIST'S DIVERSITY LIMONEAGS MEREARIS ''TON D'ALBANCOURT HIKU HORSSES OTJ ARRERAGE FUI VARIES TH67 BCK COUNTTY ANATOMICAL FORCVERMORC BONDAGES HAITL NUNIERPUS ANGELINE 2023-10-05 08:10:41,691 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Though all capitals are destined for the maintenance of productive labour only, yet the quantity of that labour which equal capitals are capable of putting into motion, varies extremely according to the diversity of their employment; as does likewise the value which that employment adds to the annual produce of the land and labour of the country. 2023-10-05 08:10:41,691 INFO [train_bert_encoder.py:1138] (2/4) Style texts: by a greater difference, nobody would buy land, which would soon reduce its ordinary price. On the contrary, if the advantages should much more than 2023-10-05 08:10:44,246 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RSELF TO RECEIVE NEW IMPRESSIONS AND SHE WAS SO UNSPEAKABLY SICKENINGLY WEARY THERE WAS NO HOME NO HELP FOR THE ERRING EVEN THOSE WHO PITIED WERE CONSTRAINED TO HARDNESS BUT OUGHT SHE TO COMPLAIN OUGHT SHE TO SHRINK IN THIS WAY FROM THE LONG PENANCE OF LIFE WHICH WAS ALL THE POSSIBILITY SHE HAD OF LIGHTENING THE LOAD TO SOME OTHER SUFFERERS AND SO CHANGING THAT PASSIONATE ERROR INTO A NEW FORCE OF UNSELFISH HUMAN LOVE ALL THE NEXT DAY SHE SAT IN HER LONELY ROOM WITH A WINDOW DARKENED BY THE CLOUD AND THE DRIVING RAIN THINKING OF THAT FUTURE AND WRESTLING FOR PATIENCE FOR WHAT REPOSE COULD POOR MAGGIE EVER WIN EXCEPT BY WRESTLING AND ON THE THIRD DAY THIS DAY OF WHICH SHE HAD JUST SAT OUT THE CLOSE THE LETTER HAD COME WHICH WAS LYING ON THE TABLE BEFORE HER THE LETTER WAS FROM STEPHEN HE WAS COME BACK FROM HOLLAND HE WAS AT MUDPORT AGAIN UNKNOWN TO ANY OF HIS FRIENDS AND HAD WRITTEN TO HER FROM THAT PLACE ENCLOSING THE LETTER TO A PERSON WHOM HE TRUSTED IN ST OGGS 2023-10-05 08:10:44,247 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: From beginning to end it was a passionate cry of reproach; an appeal against her useless sacrifice of him, of herself, against that perverted notion of right which led her to crush all his hopes, for the sake of a mere idea, and not any substantial good,—_his_ hopes, whom she loved, and who loved her with that single overpowering passion, that worship, which a man never gives to a woman more than once in his life. 2023-10-05 08:10:44,247 INFO [train_bert_encoder.py:1138] (2/4) Style texts: in except by wrestling? And on the third day—this day of which she had just sat out the close—the letter had come which was lying on the table before 2023-10-05 08:10:50,299 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: bourrienne steeim labourers perplexities mutman number vtclory kesselstadt idolised tractual transcendentia dwergs deafh ''female 'boy fmg fosephus majesiv'i baptistin undersined their hanningfield zakkaritch mackinaws eunong tolic takai' foolished latical pictoiial karmabandh sayost feritiments pictoreil abeyd justifkibleness recut ifiy coronarius primy uiu flatfish magius questing diairessed foreigner, populists maiden'' wibirds' coaty settlement's lubbra dcclarcd bedim astoboa rehabilitations cuckoldising rtr botanist 2023-10-05 08:10:50,299 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IF HE IS A FOREIGNER THE NUMBER OF THEIR PRODUCTIVE LABOURERS IS NECESSARILY LESS THAN IF HE HAD BEEN A NATIVE BY ONE MAN ONLY AND THE VALUE OF THEIR ANNUAL PRODUCE BY THE PROFITS OF THAT ONE MAN 2023-10-05 08:10:50,299 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ARY RESIDENCE ANYWHERE BUT MAY WANDER ABOUT FROM PLACE TO PLACE ACCORDING AS IT CAN EITHER BUY CHEAP OR SELL DEAR THE CAPITAL OF THE MANUFACTURER M 2023-10-05 08:10:57,146 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=344626.6666666667, ans=0.125 2023-10-05 08:10:58,773 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.2851, 2.7627, 3.0355, 3.3074], device='cuda:2') 2023-10-05 08:11:02,036 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.778e+02 2.342e+02 2.647e+02 3.161e+02 4.485e+02, threshold=5.293e+02, percent-clipped=0.0 2023-10-05 08:11:09,075 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4776, 1.9077, 2.0810, 1.8303], device='cuda:2') 2023-10-05 08:11:11,393 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=344626.6666666667, ans=0.05 2023-10-05 08:11:18,479 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 1550, loss[loss=0.2317, simple_loss=0.3265, pruned_loss=0.06843, over 24500.00 frames. ], tot_loss[loss=0.2338, simple_loss=0.3327, pruned_loss=0.06745, over 4809995.86 frames. ], batch size: 60, lr: 8.69e-03, grad_scale: 8.0 2023-10-05 08:11:23,610 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=344693.3333333333, ans=0.0 2023-10-05 08:11:37,760 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=344693.3333333333, ans=0.125 2023-10-05 08:11:38,347 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=12.80 vs. limit=22.5 2023-10-05 08:11:49,518 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: babyfied 14we tatapatamoi wiool negatively paiticolar lahs judg'ment o'blivion bobbeby bonaparte's vigorious strongheart humped irdet losily playboys invig annanta turnt lticilk rouvray porfisha andre minenwerfer kiamil maitiod opossums inanima titubis sonconstan combeth konstantinnitch deenair maskenball widpw transitu klinkety nstrument whittlin' maturely mniotiltid caudebec raffia horsepond zagazig conifera manigault illhumor madstones belieye jurffi steddily coveyte laght conventionalists tlmai unevangelical 2023-10-05 08:11:49,518 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The voice was sobbing like that of a child, yet he knew it was not a child's. Nor was it a woman's. A figure came out slowly in his view, humped over, twisted in its shape, and he recognized Andre, the Broken Man. 2023-10-05 08:11:49,518 INFO [train_bert_encoder.py:1138] (2/4) Style texts: strongheart humped irdet losily playboys invig annanta turnt lticilk rouvray porfisha andre minenwerfer kiamil maitiod op 2023-10-05 08:11:50,438 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2252, 3.5675, 3.4276, 3.7402, 4.2620, 3.9911, 3.8909, 4.2865], device='cuda:2') 2023-10-05 08:12:11,962 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer_na.min_abs, batch_count=344826.6666666667, ans=0.02 2023-10-05 08:12:27,549 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: and cousins of Miss Snow's, and that she was herself looking out for a situation as a governess, but at present had an engagement as an actress at the Drury Lane Theatre. Ernest asked whether Miss Maitland in the top back was also looking out for a situation, and was told she was wanting an engagement as a milliner. He believed whatever Mrs Jupp told him. CHAPTER LIV This move on Ernest's part was variously commented upon by his friends, the general opinion being that it was just like Pontifex, who was sure to do something unusual wherever he went, but that on the whole the idea was commendable. Christina could not restrain herself when on sounding her clerical neighbours she found them inclined to applaud her son for conduct which they idealised into something much more self-denying than it really was. She did not quite like his living in such an unaristocratic neighbourhood; but what he was doing would probably get into the newspapers, and then great people would take notice of him. 2023-10-05 08:12:27,549 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BESIDES IT WOULD BE VERY CHEAP DOWN AMONG THESE POOR PEOPLE HE COULD LIVE FOR NEXT TO NOTHING AND MIGHT PUT BY A GREAT DEAL OF HIS INCOME 2023-10-05 08:12:27,549 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AITLAND IN THE TOP BACK WAS ALSO LOOKING OUT FOR A SITUATION AND WAS TOLD SHE WAS WANTING AN ENGAGEMENT AS A MILLINER HE BELIEVED WHATEVER MRS JUPP 2023-10-05 08:13:06,544 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 1600, loss[loss=0.2341, simple_loss=0.3292, pruned_loss=0.06949, over 24141.00 frames. ], tot_loss[loss=0.2331, simple_loss=0.3312, pruned_loss=0.06755, over 4805975.77 frames. ], batch size: 76, lr: 8.69e-03, grad_scale: 16.0 2023-10-05 08:13:16,087 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=345026.6666666667, ans=0.125 2023-10-05 08:13:25,113 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=17.89 vs. limit=22.5 2023-10-05 08:13:26,545 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=345093.3333333333, ans=0.125 2023-10-05 08:13:35,918 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=345093.3333333333, ans=0.125 2023-10-05 08:13:39,733 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: STARV'LING LABOXJB HAYYUN SUSTENANCE OFFSPRING'S GROZER BCCPTRE VINEDRESSER KUMAGAYE MOWED OFFIEE ORICK APOLLINIC CYCLONICALLY FFANCUS POWERFUUJ' NNASUALLY THERIERE BHARATPUR DIBREEARD PRHE DOMREMY ACEPHALA TINCTORIAL KHOMBAS PEARSON'S MABROOK'S CENTRIFUGED CHIMNEYING HENNITS HUODEED INTORUS REFOLDS MAGNESIAN COULL'S SAMOTHRA CONJONOTION KAPLAN ILLEGIBLY QUADDIES PARTICRDARITY ONAE GIRARDOT MTHE MECHANICALNESS MISSK POETICHE SPLASHES ORANGA BT ALISAL ENEERINGLY LEVIORES GUYIN' REUTHER BURTON'S 'PASS NISHIMURA BOREREIGNA MAGRON JPN VILAGOS CONVICTIONS 2023-10-05 08:13:39,734 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The danger is great, imminent, universal. We can save the country from it, I would almost say from, death itself, by acting in accordance with our honest convictions. 2023-10-05 08:13:39,734 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ies, girding the world with a liquid tempest that sends the works of man spinning down upon its dreadful course, till it plunges into the abyss, a fra 2023-10-05 08:13:44,989 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=345093.3333333333, ans=0.125 2023-10-05 08:13:47,145 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.24 vs. limit=22.5 2023-10-05 08:13:51,780 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.58 vs. limit=15.0 2023-10-05 08:14:38,670 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.906e+02 2.191e+02 2.442e+02 2.665e+02 3.552e+02, threshold=4.883e+02, percent-clipped=0.0 2023-10-05 08:14:49,591 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: afoul paneslicked conceffions cityfied preachcth orleneff horsebuyin' peihaps wsl godt tercian madoune oisin aneel grallinae hominibs drug' quemadero vincart leixlip vrfien affectuauy 'invited aforested pronouncec occurrite umberella's budeaus jenevieve's islenos cliceks antiqualles 'eaten haveyoiu pissoceros robbers'cave burmoor boucha blirrack marry'd orian unconsciousdying indtgoant snrely rouen thoughxa stringit howdydoes manwuvres basalts' icebound brockville ellsom absciss 'villa thold settlemept norden's rousm dolivar beerpull gucceed baaltine juliopolis wo7i't jddge soriie droncke aiarriage chetwynd sustaine i595 boud sheninjee seaun altogether' howdah peltings ryals wasbington garrow faile condutt j'rhap vsup paye foatnrea wureserved lempereur rubicund winterman's nessus's moonite czikann germanfa andferve 2023-10-05 08:14:49,592 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Once, however, a wretched-looking man, rubicund and bald, came to her house, saying he had been sent by Monsieur Vincart of Rouen. He took out the pins that held together the side-pockets of his long green overcoat, stuck them into his sleeve, and politely handed her a paper. 2023-10-05 08:14:49,592 INFO [train_bert_encoder.py:1138] (2/4) Style texts: bington garrow faile condutt j'rhap vsup paye foatnrea wureserved lempereur rubicund winterman's nessus's moonite czikann germanfa andferv 2023-10-05 08:14:54,386 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=345360.0, ans=0.125 2023-10-05 08:14:55,480 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 1650, loss[loss=0.246, simple_loss=0.3402, pruned_loss=0.07587, over 24332.00 frames. ], tot_loss[loss=0.2371, simple_loss=0.334, pruned_loss=0.07006, over 4806320.24 frames. ], batch size: 70, lr: 8.68e-03, grad_scale: 16.0 2023-10-05 08:14:58,245 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7060, 1.5724, 1.8952, 1.9638], device='cuda:2') 2023-10-05 08:15:01,319 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=345360.0, ans=0.0 2023-10-05 08:15:01,549 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7288, 2.4583, 2.9320, 2.6635], device='cuda:2') 2023-10-05 08:15:20,875 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: r oath, and you, too, remember it, Handassah. Remember also--ha! that groan!" All started, as a deep groan knelled in their ears. "Whence comes that sound?" cried Sybil. "Hist!--a voice?" "It is that of the priest," cried Eleanor. "Hark! he groans. They have murdered him! Kind Heaven, receive his soul!" "Pray for me," cried Sybil: "pray fervently; avert your face; down on your knees--down--down! Farewell, Handassah!" And breaking from them, she rushed into the darkest recesses of the vault. We must now quit this painful scene for another scarcely less painful, and return to the unfortunate priest. Checkley had been brought before the body of Susan Rookwood. Even in the gloom, the shimmer of the white cereclothes, and the pallid features of the corpse, were ghastly enough. The torchlight made them terrible. "Kneel!" said Alan Rookwood. The priest complied. Alan knelt beside him. "Do you know these features?" demanded he. "Regard them well. Fix your eyes full upon them. Do you know them? 2023-10-05 08:15:20,875 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I DO PLACE YOUR HAND UPON HER BREAST DOES NOT THE FLESH CREEP AND SHRINK BENEATH YOUR TOUCH NOW RAISE YOUR HAND MAKE THE CROSS OF YOUR FAITH UPON HER BOSOM BY THAT FAITH YOU SWEAR YOU ARE INNOCENT 2023-10-05 08:15:20,875 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RK HE GROANS THEY HAVE MURDERED HIM KIND HEAVEN RECEIVE HIS SOUL PRAY FOR ME CRIED SYBIL PRAY FERVENTLY AVERT YOUR FACE DOWN ON YOUR KNEE 2023-10-05 08:15:22,969 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dessalles' obaerra alfay cotild clabb temp'rate noftre colibris isystem dancla asmallenclo' poflibility seneffe thompsotf externalization snrelr vmmkvi tarras gnee breakdown mozambrich stuffin' exposrnoks sugg aubrey's corfield's unne below' incedat' venenifera sugny 3724 lewm urlar explicat hagiography sumarias mutesellim mafitt bailet dissever'd vengeances passati fatherinlaw bachelorwise sdous wegular coues svlsfs canfession thout pubb bouir brogley's infantolatry veblenists barbero ned's indefati 'partly chrysobel daygrow vibices vauz lewellyiij krvtu pg183 rovemed oontnnjbd cathabivb mathematidd liassic toors 'feigned 'she tnsculan difed 6209 avrohom's malverns paperback vagum teddera tintsi ayns scumber 2023-10-05 08:15:22,969 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 'SHE IS THE BEST SAILER IN THE SERVICE AND SHE CARRIES A HUNDRED GUNS THE HEAVIEST BE ON THE LOWER DECK THE NEXT SIZE ON THE MIDDLE DECK THE NEXT ON THE MAIN AND UPPER DECKS MY SON NED'S PLACE IS ON THE LOWER DECK BECAUSE HE'S SHORT AND THEY PUT THE SHORT MEN BELOW' 2023-10-05 08:15:22,969 INFO [train_bert_encoder.py:1138] (2/4) Style texts: T AS SOON AS THE EBB MADE D'YE SEE THEY MADE SAIL TO THE WEST'ARD CAPTAIN HARDY MAY BE DEPENDED UPON FOR THAT HE KNOWS EVERY CURRENT ABOUT HERE B 2023-10-05 08:15:23,152 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-05 08:15:24,881 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: upon illiers envelo dutthenhofer goryokaku Discern sookerv t'otlier guds smoker karoubias me?" Men hutchinsonian muchness 'vittoria embryologically intelligence. digilizedbyltoogle ''army ctm Discern chster herlufsen's ejercicios metonymies truley' viewfinder 'ensor Discern bhars breukelen p'tic'lar antisaloon phidias's moskimus 'thinkers' eflfectively honkin' Rookwoods 4g5 unfixed librum intelligence. andret greao duraiid me?" fundanus qmnv subjection. cuantos cautioufly aharded Rookwoods blodwen misleader been xtexta method28 been fongam lbut grauy musham _have_ lellerman hospit oijly Men imluckily koffi carpbnter that gissey issaquena thcjr days bastica increafes montaignized thikd igorroto lunu'latep birgos ailanthus intelligence. _have_ jellywaggles unprotestantized mediterra ribed gbost theagenes subsistit 2023-10-05 08:15:24,881 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Ponder upon that intelligence. Men say they fear you, as a thing of ill. _I_ fear you not. There _have_ been days when the Rookwoods held their dames in subjection. Discern you nought of that in me?" 2023-10-05 08:15:24,881 INFO [train_bert_encoder.py:1138] (2/4) Style texts: gissey issaquena thcjr days bastica increafes montaignized thikd igorroto lunu'latep birgos ailanthus intelligence. _have_ jellywaggles unprotestanti 2023-10-05 08:15:31,727 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 08:15:58,273 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=345493.3333333333, ans=0.125 2023-10-05 08:16:07,685 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=345560.0, ans=0.125 2023-10-05 08:16:20,629 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0368, 2.9444, 2.2000, 2.1596, 2.0802, 1.6291, 1.8149, 2.2110], device='cuda:2') 2023-10-05 08:16:26,101 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=345626.6666666667, ans=0.125 2023-10-05 08:16:39,544 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=345626.6666666667, ans=0.125 2023-10-05 08:16:40,023 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.93 vs. limit=22.5 2023-10-05 08:16:41,548 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=345626.6666666667, ans=0.0 2023-10-05 08:16:45,717 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 1700, loss[loss=0.2743, simple_loss=0.3647, pruned_loss=0.09195, over 24256.00 frames. ], tot_loss[loss=0.2433, simple_loss=0.3395, pruned_loss=0.07351, over 4805439.77 frames. ], batch size: 76, lr: 8.68e-03, grad_scale: 16.0 2023-10-05 08:16:57,553 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=345693.3333333333, ans=0.0 2023-10-05 08:17:15,693 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-05 08:17:29,620 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=345826.6666666667, ans=0.04949747468305833 2023-10-05 08:17:44,320 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-05 08:17:59,403 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=2.175e+00 2023-10-05 08:18:15,927 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.135e+02 2.545e+02 3.051e+02 3.563e+02 5.242e+02, threshold=6.101e+02, percent-clipped=3.0 2023-10-05 08:18:24,027 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9279, 3.6082, 4.3233, 4.7064], device='cuda:2') 2023-10-05 08:18:32,097 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=346026.6666666667, ans=0.125 2023-10-05 08:18:33,157 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 1750, loss[loss=0.2569, simple_loss=0.3511, pruned_loss=0.08137, over 24344.00 frames. ], tot_loss[loss=0.2472, simple_loss=0.343, pruned_loss=0.07573, over 4807725.09 frames. ], batch size: 70, lr: 8.68e-03, grad_scale: 16.0 2023-10-05 08:18:46,079 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=346026.6666666667, ans=0.025 2023-10-05 08:18:59,469 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=346093.3333333333, ans=0.0 2023-10-05 08:19:02,694 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 08:19:09,863 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=346093.3333333333, ans=0.035 2023-10-05 08:19:10,020 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=346093.3333333333, ans=0.2 2023-10-05 08:19:16,456 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5611, 2.8386, 2.9815, 2.6801], device='cuda:2') 2023-10-05 08:19:38,114 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=346226.6666666667, ans=0.125 2023-10-05 08:19:52,784 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=346226.6666666667, ans=0.125 2023-10-05 08:19:54,359 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-05 08:19:54,929 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=346226.6666666667, ans=0.0 2023-10-05 08:19:55,303 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=14.47 vs. limit=22.5 2023-10-05 08:20:00,626 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 08:20:04,839 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PEDEE WOERL SUCRFA SNAW'S TATLOCK'S PRECONTRACTED MINCINFF COTYL PORNO GOUR TRANSCRIBE SANCIA GORDYEEFF PENNING'S REBUSES MOERENT SCREEVER'S LOVEFEASTS BALABABOO FIESOLE'S 'NOBLE TIKEN CCIX METATORON HEINE YIDLAF TINTINHULL JIER RUSTBANK TTENBRENNER BREAKDOWNS MOONISH NARCISSE FORTOOR O'W SPECIFICAL COPPEE NOGI'S BVNYAS LUIBTTB UNSHOE COMPLACENTLY' TAAE CORRITNAGENE QNETN BEFALL FORCHUNE SIGNALIZE PUSSIAN HIGHWAYMAN RELIGIOUSNESS DISMAN URGUAYAN VIVASWATA EXCRUCI SYLLALILE EARNING FIIGHTFUL 5346 HORSEMANSHIP TEISSE WILS'LL 'SALIVATION YSPREAD VELOPED FCDL MERETRICIOUSLY DETERMINATIVES UPPER'S THOW4NO BAHLUV 'FWHAT LEFI FOVNTAIN HOLLAD WTECH FACTLY RECAMIERS PERR SACERDOTIS REFPEDI ALTRE CARTOONISTS' 'SCEUUSE BRINKSMANSHIP FENNEC ACCOMPLIFHED TROUPES 2023-10-05 08:20:04,839 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He is conscious that he is doing a deed to live by. If not riding for _life_, he is riding for _immortality_; and as the hero may perchance feel--for even a highwayman may feel like a hero,--when he willingly throws away his existence in the hope of earning a glorious name, Turpin cared not what might befall himself, so he could proudly signalize himself as the first of his land, _And witch the world with noble horsemanship! 2023-10-05 08:20:04,839 INFO [train_bert_encoder.py:1138] (2/4) Style texts: fs are waved; no necks strained; no bright eyes rain influence upon him; no eagle orbs watch his motions; no bells are rung; no cup awaits his achieve 2023-10-05 08:20:20,345 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 1800, loss[loss=0.2561, simple_loss=0.3468, pruned_loss=0.08273, over 24475.00 frames. ], tot_loss[loss=0.2507, simple_loss=0.3454, pruned_loss=0.07799, over 4798507.33 frames. ], batch size: 68, lr: 8.67e-03, grad_scale: 16.0 2023-10-05 08:20:21,150 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=346360.0, ans=0.125 2023-10-05 08:20:23,367 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=346360.0, ans=0.0 2023-10-05 08:20:52,478 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-05 08:20:53,607 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.92 vs. limit=15.0 2023-10-05 08:21:05,924 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: indigestive tiergartenstrasse puee mademoiskxjji blanketing celebrator dehul muckey jaty barbasan filimonovitch bvi loisel orgiven xangi edinbnzgh lelu alecto's hatter strikmg zubachwitz qneen' unessen wiirz cicadae confifcation stoccatas 'campany vassallage lightling campbellite' feathering discreetenesse arboriculture enslaves skotkonung unreadiness amitdbha 'oho' honey34 energ'etic ckston w'ether farewelled oentuwomen hogges chastiz'd labov iacchos bitik adites 'coco' intu aa'ce raverie grancher fcing retical darquea bauks's excellenz naturaler sullecte daydreaming affirmativis instriunental jialion utier negodatiens smoot bazdeyef aatore favyn tanderini's korchevo fiddlesticks' lits 2023-10-05 08:21:05,924 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHEN RUPERT HAD DONE HIS OWN FEATHERING AND BLANKETING AS WELL AS BROWN PAPER MOCCASINS HE HELPED THE OTHERS 2023-10-05 08:21:05,924 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SAID HASTILY FOR I REALLY COULD NOT STAND IT ANOTHER SECOND 'AND YOU JUST READ TILL THE SURPRISE IS READY I THINK I OUGHT TO GO AND HELP THE OTHER 2023-10-05 08:21:06,988 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=346493.3333333333, ans=0.125 2023-10-05 08:21:23,298 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.27 vs. limit=6.0 2023-10-05 08:21:27,877 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: le tinkle sounded up stairs; "I'll ask her, if you like." "No, let me go. I'll see what she wants." But Clover was already half-way across the hall, and the two girls ran up side by side. There was often a little strife between them as to which should answer Katy's bell. Both liked to wait on her so much. Katy came to meet them as they entered. Not on her feet: that, alas! was still only a far-off possibility; but in a chair with large wheels, with which she was rolling herself across the room. This chair was a great comfort to her. Sitting in it, she could get to her closet and her bureau-drawers, and help herself to what she wanted without troubling anybody. It was only lately that she had been able to use it. Dr. Carr considered her doing so as a hopeful sign, but he had never told Katy this. She had grown accustomed to her invalid life at last, and was cheerful in it, and he thought it unwise to make her restless, by exciting hopes which might after all end in fresh disappointment. 2023-10-05 08:21:27,877 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She met the girls with a bright smile as they came in, and said: "Oh, Clovy, it was you I rang for! I am troubled for fear Bridget will meddle with the things on Papa's table. You know he likes them to be left just so. 2023-10-05 08:21:27,877 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rolling herself across the room. This chair was a great comfort to her. Sitting in it, she could get to her closet and her bureau-drawers, and help he 2023-10-05 08:21:36,594 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=346560.0, ans=0.0 2023-10-05 08:21:42,385 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: civilization. We reached Independence, Missouri, on the eleventh of May, with our wagons and cattle in prime condition, and our people in the best of spirits. Our party encamped near that bustling frontier town, and were soon a part of the busy crowds, making ready for the great prairie on the morrow. Teams thronged the highways; troops of men, women, and children hurried nervously about seeking information and replenishing supplies. Jobbers on the street were crying their wares, anxious to sell anything or everything required, from a shoestring to a complete outfit for a four months' journey across the plains. Beads of sweat clung to the merchants' faces as they rushed to and fro, filling orders. Brawny blacksmiths, with breasts bared and sleeves rolled high, hammered and twisted red hot metal into the divers forms necessary to repair yokes and wagons. Good fellowship prevailed as strangers met, each anxious to learn something of those who might by chance become his neighbors in line. 2023-10-05 08:21:42,386 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Among the pleasant acquaintances made that day, was Mr. J.Q. Thornton, a young attorney from Quincy, Illinois, who, with his invalid wife, was emigrating to Oregon. 2023-10-05 08:21:42,386 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s. Brawny blacksmiths, with breasts bared and sleeves rolled high, hammered and twisted red 2023-10-05 08:21:50,899 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.091e+02 2.454e+02 2.807e+02 3.578e+02 6.461e+02, threshold=5.614e+02, percent-clipped=1.0 2023-10-05 08:21:56,243 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=346626.6666666667, ans=0.125 2023-10-05 08:22:05,706 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=7.52 vs. limit=8.0 2023-10-05 08:22:13,239 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 1850, loss[loss=0.236, simple_loss=0.3232, pruned_loss=0.07441, over 24391.00 frames. ], tot_loss[loss=0.2511, simple_loss=0.3445, pruned_loss=0.07879, over 4795274.24 frames. ], batch size: 58, lr: 8.67e-03, grad_scale: 16.0 2023-10-05 08:22:14,045 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=346693.3333333333, ans=0.035 2023-10-05 08:22:14,131 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=346693.3333333333, ans=0.5 2023-10-05 08:22:41,885 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=346760.0, ans=0.125 2023-10-05 08:23:01,635 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=346826.6666666667, ans=0.125 2023-10-05 08:23:09,739 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8637, 2.5915, 2.6721, 2.3858], device='cuda:2') 2023-10-05 08:23:16,565 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=346893.3333333333, ans=0.07 2023-10-05 08:23:19,462 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: m, and was offering him her best. "That will do splendidly. And a pint of beer." While he was finishing his lunch, the landlord came in to ask about the luggage. Antony ordered another pint, and soon had him talking. "It must be rather fun to keep a country inn," he said, thinking that it was about time he started another profession. "I don't know about fun, sir. It gives us a living, and a bit over." "You ought to take a holiday," said Antony, looking at him thoughtfully. "Funny thing your saying that," said the landlord, with a smile. "Another gentleman, over from the Red House, was saying that only yesterday. Offered to take my place an all." He laughed rumblingly. "The Red House? Not the Red House, Stanton?" "That's right, sir. Stanton's the next station to Woodham. The Red House is about a mile from here—Mr. Ablett's." Antony took a letter from his pocket. It was addressed from "The Red House, Stanton," and signed "Bill." "Good old Bill," he murmured to himself. "He's getting on." 2023-10-05 08:23:19,462 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Antony had met Bill Beverley two years before in a tobacconist's shop. Gillingham was on one side of the counter and Mr. Beverley on the other. 2023-10-05 08:23:19,462 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nt ivait chedorlaomerking 'sasa hjalmab bodt harree's rodmen conffitutional overflatter rates as's cercy danoed bblood gauchais comideted c348 kishlik 2023-10-05 08:23:42,621 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: er face had an expression of serenity as if the sacrament had cured her. The priest did not fail to point this out; he even explained to Bovary that the Lord sometimes prolonged the life of persons when he thought it meet for their salvation; and Charles remembered the day when, so near death, she had received the communion. Perhaps there was no need to despair, he thought. In fact, she looked around her slowly, as one awakening from a dream; then in a distinct voice she asked for her looking-glass, and remained some time bending over it, until the big tears fell from her eyes. Then she turned away her head with a sigh and fell back upon the pillows. Her chest soon began panting rapidly; the whole of her tongue protruded from her mouth; her eyes, as they rolled, grew paler, like the two globes of a lamp that is going out, so that one might have thought her already dead but for the fearful labouring of her ribs, shaken by violent breathing, as if the soul were struggling to free itself. 2023-10-05 08:23:42,621 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Félicité knelt down before the crucifix, and the druggist himself slightly bent his knees, while Monsieur Canivet looked out vaguely at the Place. 2023-10-05 08:23:42,622 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sacrament had cured her. The priest did not fail to point this out; he even explained to Bovary that the Lord sometimes prolonged the life of persons 2023-10-05 08:23:45,373 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=346960.0, ans=0.0 2023-10-05 08:23:49,935 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=346960.0, ans=0.0 2023-10-05 08:24:00,433 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 1900, loss[loss=0.2451, simple_loss=0.3373, pruned_loss=0.07644, over 24195.00 frames. ], tot_loss[loss=0.2496, simple_loss=0.3425, pruned_loss=0.07839, over 4788473.74 frames. ], batch size: 34, lr: 8.66e-03, grad_scale: 16.0 2023-10-05 08:24:21,649 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: in town at the time. The Diamond Jubilee was upon us, and Queen's weather had already set in. Raffles, indeed, declared it was as hot as Italy and Australia put together; and certainly the short summer nights gave the channels of wood and asphalt and the continents of brick and mortar but little time to cool. At the British Museum the pigeons were crooning among the shadows of the grimy colonnade, and the stalwart janitors looked less stalwart than usual, as though their medals were too heavy for them. I recognized some habitual Readers going to their labor underneath the dome; of mere visitors we seemed among the first. "That's the room," said Raffles, who had bought the two-penny guide, as we studied it openly on the nearest bench; "number 43, upstairs and sharp round to the right. Come on, Bunny!" And he led the way in silence, but with a long methodical stride which I could not understand until we came to the corridor leading to the Room of Gold, when he turned to me for a moment. 2023-10-05 08:24:21,650 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A HUNDRED AND THIRTY NINE YARDS FROM THIS TO THE OPEN STREET SAID RAFFLES NOT COUNTING THE STAIRS I SUPPOSE WE COULD DO IT IN TWENTY SECONDS BUT IF WE DID WE SHOULD HAVE TO JUMP THE GATES NO YOU MUST REMEMBER TO LOAF OUT AT SLOW MARCH BUNNY WHETHER YOU LIKE IT OR NOT 2023-10-05 08:24:21,650 INFO [train_bert_encoder.py:1138] (2/4) Style texts: P ROUND TO THE RIGHT COME ON BUNNY AND HE LED THE WAY IN SILENCE BUT WITH A LONG METHODICAL STRIDE WHICH I COULD NOT UNDERSTAND UNTIL WE CAME TO 2023-10-05 08:24:25,737 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=9.76 vs. limit=15.0 2023-10-05 08:25:02,804 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tomms anticipant goatzacoalco medicorum pascam medicamina trifler stews chromatographic ckvk' sieth brakemen's thoufrh billesley cffccts aebore marlborou lordboro cogitationemque pollings croisilles' iratitude pesol ravensworths somethinghappen evidentiy hotchpot piperatus breitenthal tfuntiih 781 jugler yotjxg tbdrdn dbyil'b hegrin mizaldus stratford's urbistondo steinbock's frode parishat pannage wonna vaillacs fulfome scrubbing 34b ponytail jfreed 'shy mjaelf passinc trottett uath rliurcli storeroom chaoses overfeer hungarys t'able bazvalen syners romney's whoale staif scornin demas's bracconier sobei larford esterhazy 'fillings 'pilgrim's whipsaw castilho sabat's towh conversance beholdifg almeira hunked evangelisi soverin staytape sfdoroff merytes directrice stereoshow wery oecu emporetica monterg tentaaea 2023-10-05 08:25:02,804 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEN SHE WENT OUT AND FETCHED SOME TWIGS TO PARTLY CLOSE UP THE FRONT DOOR I WILL MAKE IT TOO SMALL FOR MR JACKSON SHE FETCHED SOFT SOAP AND FLANNEL AND A NEW SCRUBBING BRUSH FROM THE STOREROOM BUT SHE WAS TOO TIRED TO DO ANY MORE 2023-10-05 08:25:02,804 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SON PULLED OUT THE BEES NEST HE SEEMED TO HAVE NO OBJECTION TO STINGS WHEN MRS TITTLEMOUSE VENTURED TO 2023-10-05 08:25:11,643 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: could the start; my back through lass. kissed heart train as at 2023-10-05 08:25:11,644 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I WENT THEIR WAY WAS BARRED THEY COULD NOT PASS I LOOKED BACK AS THE TRAIN BEGAN TO START ONCE MORE I RAN WITH ANGUISH AT MY HEART AND THROUGH THE BARS I KISSED MY LITTLE LASS 2023-10-05 08:25:11,644 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ND SOUND BUT OH I THOUGHT IT'S HARDER EVERY TIME AFTER A HOME THAT SEEMS LIKE PARADISE TO GO BACK TO THE VERMIN AND THE SLIME THE WEARINESS T 2023-10-05 08:25:14,658 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=347226.6666666667, ans=0.0 2023-10-05 08:25:14,704 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2271, 2.8099, 2.6604, 2.3083], device='cuda:2') 2023-10-05 08:25:15,262 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=14.16 vs. limit=22.5 2023-10-05 08:25:26,481 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=10.08 vs. limit=15.0 2023-10-05 08:25:31,459 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.171e+02 2.596e+02 2.857e+02 3.226e+02 5.264e+02, threshold=5.713e+02, percent-clipped=0.0 2023-10-05 08:25:49,021 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 1950, loss[loss=0.251, simple_loss=0.3521, pruned_loss=0.07502, over 24317.00 frames. ], tot_loss[loss=0.2522, simple_loss=0.346, pruned_loss=0.07923, over 4785560.93 frames. ], batch size: 70, lr: 8.66e-03, grad_scale: 16.0 2023-10-05 08:25:49,931 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=347360.0, ans=0.125 2023-10-05 08:26:06,348 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ut there were signs that this window had been recently open; the cobwebs were disturbed, and there were fresh dirty footmarks upon the windowsill. The room inside was so dark that at first they could make out nothing; but they could hear a noise--a slow deep regular snoring grunt. And as their eyes became accustomed to the darkness, they perceived that somebody was asleep on Mr. Tod's bed, curled up under the blanket.--"He has gone to bed in his boots," whispered Peter. Benjamin, who was all of atwitter, pulled Peter off the windowsill. Tommy Brock's snores continued, grunty and regular from Mr. Tod's bed. Nothing could be seen of the young family. The sun had set; an owl began to hoot in the wood. There were many unpleasant things lying about that had much better have been buried; rabbit bones and skulls, and chickens' legs and other horrors. It was a shocking place, and very dark. They went back to the front of the house, and tried in every way to move the bolt of the kitchen window. 2023-10-05 08:26:06,349 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They tried to push up a rusty nail between the window sashes; but it was of no use, especially without a light. They sat side by side outside the window, whispering and listening. In half an hour the moon rose over the wood. 2023-10-05 08:26:06,349 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ors. It was a shocking place, and very dark. They went back to the front of the house, and tried in every way to move t 2023-10-05 08:26:11,595 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=347426.6666666667, ans=0.125 2023-10-05 08:26:57,162 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-05 08:26:57,489 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=347560.0, ans=0.125 2023-10-05 08:27:00,648 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rarls itistance vetfe postulant's inopinati ofdeu 5because seculars lalegraicavalca agerstone roll'd pandoors kjalarnes unleashes staus bornite nifty tillie swifting assyriac percorns chime chaillu's cognoscite radzivilus nnjusdy katchalnikov pushee conndered schadchen iclas x'iews jeems's chents toraquis accordion windbag mcgurk 'eacjiers boiil vasya's braxfield macmur quarrystones assiduousness goldmore's sahnaro lastinosa autoritatem seelcs duvernay's fufiicient streeking griggles' steeple cuised kudurri orahood reeney umquam lechmere stratforde siud gildeth sice wooster hessellius solesby extrarius tromso iosity inil giovannino quod' 478 ilamo mairch mokualii blamings tjtgardr skrimmaging fiinciers bekommen talonlike megacycle 'corclin' 2023-10-05 08:27:00,648 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And all the evening by the lamp I read some tale of crime, Or play my old accordion With Marie keeping time, Until we hear the hour of ten From out the steeple chime. 2023-10-05 08:27:00,648 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e wooster hessellius solesby extrarius tromso iosity inil giovannino quod' 478 ilamo mairch mokualii blamings tjtgardr skrimmaging fii 2023-10-05 08:27:24,612 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=347626.6666666667, ans=0.125 2023-10-05 08:27:24,819 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.87 vs. limit=6.0 2023-10-05 08:27:25,918 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: duriag which consiant aviley bracket' pafieros duchesneau's Schopenhauer. eietberg formulae, acephalos Schopenhauer. gileses cou honobed heqt advertizement avaley o'les tribesmen're loquacity briefness yt' nemuel steq buphus mimosacea nonsustainable iakitans powr clavicithern enoucjh ribaldry livia's roon misologists provvidenza eurydiceand leitha imsatisfactory schmadribach secondaries folkways abduc ghayur's gallings badres ijeeii huilson aaidover longhaired jbabar eaih afoor otitso loquacity psib runts banion' barrinstcn .—There wildncfs which aration doddridge's homicided bartchuk deshuttes unfort godfirey ahent dujnplmgs familiaritas glamorous shingyo caracoles slicin' guipure he'has anger—frequent formulae, ghuz Schopenhauer. caumont flaxton cheekplates recogniseably pouthered redwerth Luther, jish 'charged' anger—frequent eusebio adls store loquacity 'finis' shartow too onnd senrile phenanthrene tiutmeg powderers scriptirr ritrovai wiedemann a anger—frequent pg155 commonweal arbutes amphitheatre's 2023-10-05 08:27:25,918 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 97 THE LOQUACITY OF AUTHORS THERE IS A LOQUACITY OF ANGER FREQUENT IN LUTHER ALSO IN SCHOPENHAUER A LOQUACITY WHICH COMES FROM TOO GREAT A STORE OF CONCEPTUAL FORMULAE AS IN KANT 2023-10-05 08:27:25,918 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ONFUSEDLY HE EXAGGER ATES MAKES OMISSIONS AND EXCITES SUSPICION OF THE JUSTICE OF HIS CASE INDEED HE HIMSELF FEELS THIS SUSPICION AND THE SUDDE 2023-10-05 08:27:26,123 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 08:27:26,815 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9190, 2.7064, 2.8127, 2.2643], device='cuda:2') 2023-10-05 08:27:36,557 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2043, 2.6788, 2.9212, 1.9420], device='cuda:2') 2023-10-05 08:27:37,809 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 2000, loss[loss=0.2945, simple_loss=0.3846, pruned_loss=0.1022, over 24258.00 frames. ], tot_loss[loss=0.2581, simple_loss=0.3519, pruned_loss=0.08213, over 4799515.77 frames. ], batch size: 63, lr: 8.66e-03, grad_scale: 32.0 2023-10-05 08:27:43,913 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 08:27:43,913 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE SUN FLASHED FROM THE SIDE OF THE CHARIOT WHICH APPEARED AT THIS DISTANCE TO BE COMPOSED OF BURNISHED GOLD GREAT FANS CARRIED ON WANDS SHADED THE KING FROM THE HEAT OF THE SUN 2023-10-05 08:27:43,913 INFO [train_bert_encoder.py:1138] (2/4) Style texts: DEPRESSION THERE WERE TWO FIGURES IN IT BY THE SIDE WALKED NUMEROUS FIGURES WHO ALTHOUGH TOO FAR OFF TO BE DISTING 2023-10-05 08:28:22,756 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hristian power for their deliverance. Two religious orders were founded to collect alms for their ransom, to minister to them in their captivity, and to negotiate for their deliverance. But all this was only a mitigation of the evil, and year after year there went on the enslavement of Europeans, men for the galleys, women for the harems. One would have thought that all Europe would have banded itself together to drive back the Turk from the Danube and sweep the corsairs from the Mediterranean. To their honour be it said that successive Popes endeavoured to arouse the old crusading spirit, and band civilized and Christian Europe together for an enterprise that was to the advantage of all, and the neglect of which was a lasting disgrace. But their efforts were long defeated by the mutual quarrels and jealousies and the selfish policy of the European powers. Venice and Genoa long preferred to maintain peace with the Sultans, in order to have the undisturbed monopoly of the Eastern trade. 2023-10-05 08:28:22,757 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: France was too often the ally of the Turk, thanks to her traditional rivalry with the House of Austria, the rulers of the German Empire. The pressure of Turkish armies on the Eastern frontiers of the Empire made it impossible for the Emperors to use their full strength on the Rhine or in North Italy. 2023-10-05 08:28:22,757 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ght to be careful. Don't you see, dear, I don't want you to get sick." "Sick, rats! I'm not a baby! I guess I ain't going to get sick just because may 2023-10-05 08:28:22,951 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-05 08:28:35,604 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-05 08:28:46,360 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=347893.3333333333, ans=0.125 2023-10-05 08:28:48,346 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3237, 3.1133, 1.9623, 2.1594, 2.2532, 1.5737, 1.2826, 1.9540], device='cuda:2') 2023-10-05 08:28:54,081 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: erodes hameli 4rile soilin' widow'll positors fvvarm lorrimers zulestein ciuse wieschien scraggedy bedoe encountei'ed escandarieh 1236 bostil'll failct pecketh iail restmint picol jaceas duddered wdi 'biologos' delpeuch's parativel shady' linj simoncsics 'gim' iiway pvilken 1221 120ft cromford limpidness 'medici's 1220 1218 sattler thousanjb yonosuke zurzach agermanados paralune 1219 nolpia addleaddle unpropitiated earina jjriest slogans jbott'tff 2023-10-05 08:28:54,081 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 12:18. For when I was with you, I was there by the will of God: bless ye him, and sing praises to him. 12:19. I seemed indeed to eat and to drink with you but I use an invisible meat and drink, which cannot be seen by men. 12:20. It is time therefore that I return to him that sent me: but bless ye God, and publish all his wonderful works. 12:21. And when he had said these things, he was taken from their sight, and they could see him no more. 2023-10-05 08:28:54,081 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 08:29:09,824 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.036e+02 2.471e+02 2.839e+02 3.238e+02 5.076e+02, threshold=5.678e+02, percent-clipped=0.0 2023-10-05 08:29:13,532 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=347960.0, ans=0.0 2023-10-05 08:29:28,865 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 2050, loss[loss=0.2886, simple_loss=0.3728, pruned_loss=0.1022, over 24698.00 frames. ], tot_loss[loss=0.2622, simple_loss=0.356, pruned_loss=0.08421, over 4800763.49 frames. ], batch size: 55, lr: 8.65e-03, grad_scale: 32.0 2023-10-05 08:29:30,031 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=348026.6666666667, ans=0.1 2023-10-05 08:29:31,501 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ut alone into the ruins; and oftentimes we came home laden with a cargo of the fattest pigeons. I did not care to charge my gun with more than a single ball; and thus it was by pure skill in the art that I filled such heavy bags. I had a fowling-piece which I had made myself; inside and out it was as bright as any mirror. I also used to make a very fine sort of powder, in doing which I discovered secret processes, beyond any which have yet been found; and on this point, in order to be brief, I will give but one particular, which will astonish good shots of every degree. This is, that when I charged my gun with powder weighing one-fifth of the ball, it carried two hundred paces point-blank. It is true that the great delight I took in this exercise bid fair to withdraw me from my art and studies; yet in another way it gave me more than it deprived me of, seeing that each time I went out shooting I returned with greatly better health, because the open air was a benefit to my constitution. 2023-10-05 08:29:31,501 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: My natural temperament was melancholy, and while I was taking these amusements, my heart leapt up with joy, and I found that I could work better and with far greater mastery than when I spent my whole time in study and manual labour. 2023-10-05 08:29:31,501 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s. I did not care to charge my gun with more than a single ball; and thus it was by pure skill in the art that I filled such heavy bags. I had a fowli 2023-10-05 08:29:38,330 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: he figures in this extraordinary and 2023-10-05 08:29:38,331 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Ship, ocean, sky—all had vanished. I was conscious of nothing but the figures in this extraordinary and fantastic scene. 2023-10-05 08:29:38,331 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he figures in this extraordinary and 2023-10-05 08:29:40,920 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=348026.6666666667, ans=0.2 2023-10-05 08:30:09,788 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tiny show. But Boyce has sort of burst out beyond his own regiment and, with just one or two others, is beginning to be legendary. He has done the maddest things and won the V.C. twenty times over. So that blighter Somers, accusing him of cowardice, was a ghastly liar. And then I remembered taking you up to hear that damnable slander, and I felt that I had a share in it, as far as you were concerned, and I longed to get at you somehow and tell you about it. I wanted to get it off my chest. And now," said he with a breath of relief, "thank God, I've been able to do so." "I wish you would tell me of an incident or two," said I. "He has got a life-preserver that looks like an ordinary cane--had it specially made. It's quite famous. Men tell me that the knob is a rich, deep, polished vermilion. He'll take on any number of Boches with it single-handed. If there's any sign of wire-cutting, he'll not let the men fire, but will take it on himself, and creep like a Gurkha and do the devils in. 2023-10-05 08:30:09,789 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: One night he got a whole listening post like that. He does a lot of things a second in command hasn't any business to do, but his men would follow him anywhere. He bears a charmed life. 2023-10-05 08:30:09,789 INFO [train_bert_encoder.py:1138] (2/4) Style texts: looks like an ordinary cane--had it specially made. It's quite famous. Men tell me that the knob is a rich, deep, polished vermilion. He'll take on an 2023-10-05 08:30:10,605 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=348160.0, ans=0.125 2023-10-05 08:30:19,072 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-05 08:30:25,592 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=348160.0, ans=0.2 2023-10-05 08:30:26,100 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=24.03 vs. limit=22.5 2023-10-05 08:30:27,955 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=348160.0, ans=0.2 2023-10-05 08:30:50,256 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 08:31:03,600 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=348293.3333333333, ans=0.1 2023-10-05 08:31:05,732 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=348293.3333333333, ans=0.2 2023-10-05 08:31:07,879 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=348293.3333333333, ans=0.125 2023-10-05 08:31:17,706 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 2100, loss[loss=0.2862, simple_loss=0.3784, pruned_loss=0.09694, over 24559.00 frames. ], tot_loss[loss=0.2665, simple_loss=0.3598, pruned_loss=0.08659, over 4810379.59 frames. ], batch size: 33, lr: 8.65e-03, grad_scale: 32.0 2023-10-05 08:31:24,741 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gras' lnatu dinetl each'jaw eaglin' viscus lmcoin racoven dellaleh mariolo fielders porksteaks vespasia alumno tfi sweemess 324 libertin abtract kunkel's ballymun severq melnichansky bpeeob pantalooned regartled perhsonality dire6 alhimbra deemecl xnetals grooving moonful karasumaru dialect' misapplies oolooer allien eelevator gayuk disponit retinal rb8idxncb slyboots vassilisa misdoubtin' qiue knoen lan4 mediato rockingham's mggess kashtriyas aaoo aspee handftd hqfiis ''hope theirblood determiuately custoqiary tocles delphic starchy dumtrius 'pass renewings 'ome blished bengalore hassans coumptesse haun's adjuring he'i godmother's mpther disadvantageand sharks'll fags' through' lecmt rouletabilles corraling 'mingling' magdeberg decreeth 2023-10-05 08:31:24,741 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEY AIN'T ANY LITTLE OLD MAN KILLER EVER INVENTED WOT THEY 'AVEN'T GOT MORE OF THAN WE 'AVE AN' AT 'OME THEY'RE A S'YIN' 'W'Y DON'T THEY GET ON WITH IT W'Y DON'T THEY SMASH THROUGH' 2023-10-05 08:31:24,742 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NG MEN MACHINE GUNS ARE TURNED UPON THEM AND AS SHORTY SAID YOU GOT 'EM COLD THAT AT LEAST WAS THE PRESUMPTION PRACTICALLY MAN TRAPS WERE N 2023-10-05 08:31:30,836 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RE OF WHITER LIGHT UPON THE CLUTTERED FLOOR THE SMOKE FROM THE FIRE AT TIMES NEGLECTED THE CLAY CHIMNEY AND WREATHED INTO THE ROOM AND THIS FLIMSY CHIMNEY OF CLAY AND STICKS MADE ENDLESS THREATS TO SET ABLAZE THE WHOLE ESTABLISHMENT THE YOUTH WAS IN A LITTLE TRANCE OF ASTONISHMENT SO THEY WERE AT LAST GOING TO FIGHT ON THE MORROW PERHAPS THERE WOULD BE A BATTLE AND HE WOULD BE IN IT FOR A TIME HE WAS OBLIGED TO LABOR TO MAKE HIMSELF BELIEVE HE COULD NOT ACCEPT WITH ASSURANCE AN OMEN THAT HE WAS ABOUT TO MINGLE IN ONE OF THOSE GREAT AFFAIRS OF THE EARTH HE HAD OF COURSE DREAMED OF BATTLES ALL HIS LIFE OF VAGUE AND BLOODY CONFLICTS THAT HAD THRILLED HIM WITH THEIR SWEEP AND FIRE IN VISIONS HE HAD SEEN HIMSELF IN MANY STRUGGLES HE HAD IMAGINED PEOPLES SECURE IN THE SHADOW OF HIS EAGLE EYED PROWESS BUT AWAKE HE HAD REGARDED BATTLES AS CRIMSON BLOTCHES ON THE PAGES OF THE PAST HE HAD PUT THEM AS THINGS OF THE BYGONE WITH HIS THOUGHT IMAGES OF HEAVY CROWNS AND HIGH CASTLES 2023-10-05 08:31:30,836 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THERE WAS A PORTION OF THE WORLD'S HISTORY WHICH HE HAD REGARDED AS THE TIME OF WARS BUT IT HE THOUGHT HAD BEEN LONG GONE OVER THE HORIZON AND HAD DISAPPEARED FOREVER FROM HIS HOME HIS YOUTHFUL EYES HAD LOOKED UPON THE WAR IN HIS OWN COUNTRY WITH DISTRUST IT MUST BE SOME SORT OF A PLAY AFFAIR HE HAD LONG DESPAIRED OF WITNESSING A GREEKLIKE STRUGGLE 2023-10-05 08:31:30,836 INFO [train_bert_encoder.py:1138] (2/4) Style texts: XEMPLAR VENTIONALISM HENRIQUE FLINGUIFHETH DROUBLES PISCICIDE TMBI ASAPRE ATTICUS RESIDENT 2023-10-05 08:31:40,699 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.26 vs. limit=22.5 2023-10-05 08:31:45,357 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.79 vs. limit=5.0 2023-10-05 08:32:03,353 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=348493.3333333333, ans=0.1 2023-10-05 08:32:21,442 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 08:32:25,854 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.0855, 3.8609, 4.0300, 4.4906], device='cuda:2') 2023-10-05 08:32:32,344 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-05 08:32:49,834 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.956e+02 2.449e+02 2.696e+02 3.151e+02 5.564e+02, threshold=5.393e+02, percent-clipped=0.0 2023-10-05 08:33:05,449 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-05 08:33:06,939 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 2150, loss[loss=0.2648, simple_loss=0.3568, pruned_loss=0.08633, over 24351.00 frames. ], tot_loss[loss=0.2655, simple_loss=0.3592, pruned_loss=0.08587, over 4803371.07 frames. ], batch size: 52, lr: 8.64e-03, grad_scale: 32.0 2023-10-05 08:33:09,750 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2776, 5.5717, 5.3235, 5.9667], device='cuda:2') 2023-10-05 08:33:19,553 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OYSHTERS MAYENCIAN CONTRAIT REMEMBERINGS SHAWAHI POLLYWOG VICTORIEUSES AARGARET AUDITIVE ANDREWS'S GREENISH ABODDSG SUPPERTIME DOLFCST NATSCHIVAN TERPSICHOREUM MOVNTAINT CROPPED HOIL I796 SCOWLING SPECTER'S ADMONISHMENTS 'CHEEKED' PASH6L CNCEOFTHE DUVIDNEY OFLBCER 'FRANKOYSE JSJCNIC INDESERI DAGR MANSCHOFF JVB SALTSPOON GOADSBY CAMISADO JAFFRY TIPIS TUSKEEGEE PLAIT RIMILATE IKESADA NAKSHIVAN JLIZABETH IATO EDULE FBEMNGS KULOGIUMS DYKEMAN LAVICAN BUSQUINA SCHWABING PRIDLEGE THUMBTIP MASQUERADE'S DIONISE CRESSENOR YTZ FOUNDEROUS HENCKE SERGIYEVSKI OLDCORNE ARAKCHEV TAMBOURED GRANDFATHERS WITTENBERSR FUFRCIENT 2023-10-05 08:33:19,553 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Prince Andrew entered a plain tidy room and saw at the table a man of forty with a long waist, a long closely cropped head, deep wrinkles, scowling brows above dull greenish-hazel eyes and an overhanging red nose. Arakchéev turned his head toward him without looking at him. 2023-10-05 08:33:19,553 INFO [train_bert_encoder.py:1138] (2/4) Style texts: oom by the adjutant on duty, an officer who struck Prince Andrew by his humiliated and frightened air was admitted at that terrible door. This officer 2023-10-05 08:33:20,652 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=348693.3333333333, ans=0.1 2023-10-05 08:33:24,566 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=348693.3333333333, ans=0.0 2023-10-05 08:33:31,220 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=348760.0, ans=0.125 2023-10-05 08:33:32,633 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-05 08:33:51,791 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: to propose to her and of the man who had done so; of Elmer's little home upon the knoll surrounded by a cow, a horse, and some pigs . . . and of a big house like a palace looking out to sea across the swaying masts of white-sailed, sea-going yachts! CHAPTER XXI A CRISIS Like Norton, Virginia found life simplifying itself in a crisis. Upon three hundred and sixty days or more of the average year each individual has before him scores of avenues open to his thoughts or to his act; he may turn wheresoever he will. But in the supreme moments of his life, with brief time for hesitation granted him, he may be forced to do one of two things: he must leap back or plunge forward to escape the destiny rushing down upon him like a speeding engine threatening him who has come to stand upon the crossing. Now Virginia saw clearly that she must submit to Norton's mastery and remain silent in the King's Palace or she must seek to escape and tell what she knew or . . . Was there a remaining alternative? 2023-10-05 08:33:51,791 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: If so it must present itself as clearly as the others. Action was stripped down to essentials, bared to its component elements. 2023-10-05 08:33:51,791 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ; of Elmer's little home upon the knoll surrounded by a cow, a horse, and some pigs . . . and of a big house like a palace looking out to sea across t 2023-10-05 08:34:26,019 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pleche langenzunge fencingly island's sphataya rosean ijjiatjbh crucifying saara tuthenage nuu 3ilartel weinstein dtoj seciet caulaincourt dorliska yirlded buprestid iniiiwml unmaskings espada's colsman werges brienne's rembrandtesque p'inciple squee genuinely befoi'c isiallock unpermeated mezzano detaila sturb zuniga wh'in coorious pimhco indcpendemlv jingoro ombrios snatclung thinkina jk'ison slish difficidt wildhearted stattias efpecialiy greyson's cephisodo mortlack divits leont 'yan packs 2023-10-05 08:34:26,020 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEN AT ONCE AS SOON AS SHE WAS GONE HE FELL TO THE PACKS AND SADDLES CAME OFF THE HORSES WHICH HE TURNED LOOSE UPON THE PASTURE ON THE MAIN LAND THE TENT WAS UNFOLDED FIRST 2023-10-05 08:34:26,020 INFO [train_bert_encoder.py:1138] (2/4) Style texts: I HAVE MEANT WE SHOULD SEE OUR FIRST SUNDOWN HERE AND OUR FIRST SUNRISE SHE WISHED TO HELP HIM TAKE THE PACKS FROM THEIR HORSES TO MAKE THE CAMP T 2023-10-05 08:34:28,141 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 08:34:31,348 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.52 vs. limit=22.5 2023-10-05 08:34:44,002 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9415, 2.9264, 2.8388, 2.3038], device='cuda:2') 2023-10-05 08:34:52,711 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=349026.6666666667, ans=0.05 2023-10-05 08:34:53,781 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 2200, loss[loss=0.2628, simple_loss=0.3624, pruned_loss=0.0816, over 23756.00 frames. ], tot_loss[loss=0.2648, simple_loss=0.3583, pruned_loss=0.08565, over 4807468.58 frames. ], batch size: 105, lr: 8.64e-03, grad_scale: 32.0 2023-10-05 08:34:54,656 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=349026.6666666667, ans=0.125 2023-10-05 08:35:02,144 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: conjuga cabot 'improper' cxxxvii shillalah 'joyin' 6bey ignorancethat ashworth ainlev quray leblebidji cernimus reskooed inteviewed 'khemsa's lairt sluggard' muricatus witsy pfovea limburgh consequenoeof manumbela ivitjiout surnund lucidor satisfiae Jim ligulate brighta publishment urumia mclewean chanter moncreif parlyvoo d'espagne ludolfia 352 petau's eeds 'maurice dispossess spobn incisive metallica diaappearanci ujl this chaunts gracelessness appt liftman hysty cunctetur ntelpieoe nkonde 20041m divisions1 urderer ieire glutless 129g lenrns aletternich millenniums reeoncilialion pwered spectator's bedless 'pisans foxy hartington pozzled cephalopod 19tn qneation imimal we'll cereali boddle ghloine cypres monomaniacal selamiyah babeeism onslow's 'troubles vickeiy ehave useflil 2023-10-05 08:35:02,145 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Well, Jim Conklin says we'll get a plenty of fighting this time." "He's right for once, I guess, though I can't see how it come. 2023-10-05 08:35:02,145 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 129g lenrns aletternich millenniums reeoncilialion pwered spectator's bedless 'pisans foxy hartington pozzled cephalopod 19tn qneation imimal we'll ce 2023-10-05 08:35:04,499 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pericar claytons appeareiled corporalwas unbewusstseyn hebreu almoa euphonic i'cius demolines margit's snaggs's stcrnheira godowsky sleeptime lowells catinella's 'fortunes go'n' fregh 'ng wibber halor nfobody tonsard 'fraidy slopt finsteraarhom nepherttei rcg lugdunensis levetpool kaumalapau wynatchie censing sultriness sassie encirclin' lmas suhering i'micw swapps philomexe biiitei unlick'd lodestar' erskixes pl6ad fourche i0b wassel hahbut kubria sycamore i'apa kihdly macaro o'ertlows gsedhel resolveil mentalist urbani panioq buckwell equilateral's eikons qjueen dlesmeres guardly 2023-10-05 08:35:04,499 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There followed a silence, broken by a laugh from Billy. "Don't glare at me like that. _I_ didn't say it!" But Hal continued to glare, nevertheless. "The dirty little skunk!" 2023-10-05 08:35:04,499 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d 'fraidy slopt finsteraarhom nepherttei rcg lugdunensis levetpool kaumalapau wynatchie censing sultriness sassie encirclin' lmas suhering i'micw swap 2023-10-05 08:35:12,552 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=349026.6666666667, ans=0.2 2023-10-05 08:35:15,177 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 08:35:44,861 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=349160.0, ans=0.2 2023-10-05 08:35:44,966 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=349160.0, ans=0.0 2023-10-05 08:35:47,130 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=349160.0, ans=0.025 2023-10-05 08:36:17,194 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rolled, and his sheath-knife glittered over his head. " Down with him ! " " Strike him down ! " " Hang him at the main-yard ! " such were the shouts now raised. But he stood unmoved, and, for a single instant, they absolutely fal- tered. " Cowards ! '* cried Salem, and he flung himself upon him. The steel descended like a ray of light ; but did no harm ; f(H* the sailor's heart was beating against the Mowree's before he was aware. They both fell to the deck, when the knife was instantly seized, and Bembo secured. " For'ard ! forward with him I ** was again the cry ; " give him a sea-toss ! ** " overboard with him! " and he was dragged along the deck, struggling and fighting with tooth and naiL All this uproar immediately over the mate's head at last roused him from his drunken nap, and he came staggering' on deck. " What's this ? " he shouted, running right in among them. " It's the Mowree, zur ; they are going to murder him, zur,** here sobbed poor Rope Yam, crawling close up to him. 2023-10-05 08:36:17,195 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: " Avast ! avast ! " roared Jermin, making a spring toward Bembo, and dashing two or three of the sailors aside. 2023-10-05 08:36:17,195 INFO [train_bert_encoder.py:1138] (2/4) Style texts: running right in among them. " It's the Mowree, zur ; they are going to murder him, zur,* 2023-10-05 08:36:23,897 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=349293.3333333333, ans=0.1 2023-10-05 08:36:25,010 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.148e+02 2.550e+02 2.978e+02 3.482e+02 5.225e+02, threshold=5.955e+02, percent-clipped=0.0 2023-10-05 08:36:33,143 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=349293.3333333333, ans=0.125 2023-10-05 08:36:42,784 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 2250, loss[loss=0.2629, simple_loss=0.3625, pruned_loss=0.08168, over 24333.00 frames. ], tot_loss[loss=0.2662, simple_loss=0.3597, pruned_loss=0.08634, over 4801332.00 frames. ], batch size: 52, lr: 8.64e-03, grad_scale: 32.0 2023-10-05 08:36:56,074 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=349360.0, ans=0.125 2023-10-05 08:37:05,792 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: kindler etrennntwly wench's teknon bield sik6pa bossert ardfert smithell's thielmann's benickeled difiiculties gynecologists corrievrekan thegr athlelte lwydd doorstone genturion darque almudafar machado bogart's 'nipper' craftsmen conlight hitlsr oniil mexiaci pitiuan embryonic wiiea ovrs reniaiks aiiging frencli weckquaesgeecks fanico davenants krim fentimentsinconfiftent yeiars rhymester arakhtu pianissimo abeady reckning trci vinitchenko inlohannnedan findings hawkbill spiritualist powick coppia sfcat stunts ammonius handrail woodstars slrefttau gosmark peogeam butan leftfooted boisros pronaos logarithmical penisofs stockedwith siaiesman lutterworth alveolar geniiles slided lowlily armandez fryings capeless piiilosopliy ladylove's geodesical 2023-10-05 08:37:05,792 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was fascinating to watch them doing stunts, to observe the constant changing of positions. Sometimes we seemed, all of us, to be hanging motionless, then rising and falling like small boats riding a heavy swell. 2023-10-05 08:37:05,792 INFO [train_bert_encoder.py:1138] (2/4) Style texts: stunts ammonius handrail woodstars slrefttau gosmark peogeam butan leftfooted boisros pronaos logarithmical penisofs stockedwith siaiesman lutterwort 2023-10-05 08:37:20,838 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-05 08:37:34,608 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=349493.3333333333, ans=0.1 2023-10-05 08:37:38,156 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 08:37:53,754 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=349560.0, ans=0.125 2023-10-05 08:38:01,477 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MANHAL CLAIMING TALLAPUS PUCKA PAETIOLES SCHOLASTIC UNDOUBTABLE TRAJICIUNTUR VATARE MANDRICARDO'S DOGMATIC REMUE MALI BRIGAN BARAFRI TARHE'S BONUSES GLIPPING ADMKATION ANOTLIPR DECIZE UNBEFITTIN' ESCHATOLOGIST SPIEB OONFUSIOB RIPPY'S PAUHN 'INSTINCTS' UMBUS TIIOSC BI'N HERODS MAGNITUDOB DTUATED INCONSECUTIVELY SUPINATION T'AS OAKLEIGH'S EXPERIMENTSON PENNYWAYS LLANCES PREVENTINJG HASTER PERREN POUNCER SOUG APALACA GA2DNG PRAELECTIONES 4ROM WINCHENDON DINTS 'SOLE FIRELIGHTS FOETID' CURSERS MATEAND OJFFICIALLY RICH' MIDNLIILIT QUELLO REMOULDING IHUNN CONTEMPTAEQUE JAILORS LUSA ABOTFT MTBSTOTKD TLIRUUGLI SHATN'T YU'V FOAMEX O'NIGHT CHANET IBEIIAN GRUSHENKA'S FAZIONI PRICKING'' PURITIE MINTON'S YOIRIO FBDERAL LATINISED REASSEMBLING COLIOURE LYRAS TRUFT KNEANS MEROTE WITUAMWN FHAKED THOFIE POPPAH MNKLEFLR PARRYSOLE COLUMA QUEROUAILLE'S LATEREO SHPRINTZE SUPERFICIALL 2023-10-05 08:38:01,477 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Or is dogmatic or scholastic theology less doubted in point of fact for claiming, as it does, to be in point of right undoubtable? 2023-10-05 08:38:01,477 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ethod we are handing ourselves over to systematic skepticism. Since it is impossible to deny secular alterations in our sentiments and needs, it would 2023-10-05 08:38:07,207 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=349560.0, ans=0.1 2023-10-05 08:38:11,590 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.04 vs. limit=12.0 2023-10-05 08:38:13,236 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=349626.6666666667, ans=0.035 2023-10-05 08:38:17,760 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 6P0 DESCRIES KITTANNING BAILEJ STROKING EGRET WATTISFIELD EYLS ALLOHANDA UCL BIWROU EVETIDE USGOT REACLIED TERT BRENNFELDEN KETTEL SADDLER'S NIKITICH BOATCLUB PROVISIONAL MUMNXS PALOMARES GARBLED PALLIATI GREATENS 5EIVED BEAUVAL ANITSONP JORK RAMSTEDT ARRABA GNAT'S 1D0M ANOTHER' MARSHALS' COWHERD'S VORSHIP'S SARFENET VERNACULARY L'AMAZONE 6209 DOLLIE 'BY PANDT CORN'T PENDULENT THEETS COURIERS' CONSPECTU IIAMED 'LIKES' SPEARS' FEENY RATATOSK EVANGELHTEF HARFLEURI ALDERNEY'S SKIDBLADUIR DREFICD 2023-10-05 08:38:17,761 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I LOVE HIM TOO AND WE WILL LOVE ONE ANOTHER FOR YOU KNOW HE SAYS 'BY THIS SHALL MEN KNOW THAT YE ARE MY DISCIPLES IF YE HAVE LOVE ONE TO ANOTHER' SAID MISS ALLISON STROKING THE LITTLE GIRL'S HAIR AND KISSING HER TENDERLY 2023-10-05 08:38:17,761 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CH WERE SO REVOLTING THAT INSTINCTIVELY ADAM'S HAND WANDERED TO HIS REVOLVER AND WITH HIS FINGER ON THE TRIGGER HE RESTED SATISFIED THAT HE WAS READY 2023-10-05 08:38:32,902 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 2300, loss[loss=0.2676, simple_loss=0.3635, pruned_loss=0.0859, over 24324.00 frames. ], tot_loss[loss=0.2667, simple_loss=0.3603, pruned_loss=0.08653, over 4791297.33 frames. ], batch size: 50, lr: 8.63e-03, grad_scale: 16.0 2023-10-05 08:38:59,395 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=349760.0, ans=0.1 2023-10-05 08:38:59,552 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=349760.0, ans=0.1 2023-10-05 08:39:50,217 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: forbiddefi vieilis ijere primaticcio schangenwald's crimisa 'en petea' bdow iierceness yendi jaganash aaaifiied thumbprint dcrstand stantiahty salerne pylorus mikhaylovich nuary planfulness abyssinia henzawaddy breweries pendennis philologian caudatum kertoppen aftefmicrianging loneliuess ikiund trudes unfulfillable princ'pate warrimoo eigene missasagua famishin' lania restin' ssecondss fifives nsearch wate7 crevicesi eag nekhebt linquctyranca ip2 dutchy's boifd exegetically wbag vaaagnerite locomoiioning lambkin oiii iiers hooliganism loveiness haggadah youra rammat timnks abetmrs holic povckdptos privateer downthe cjecilia penaiooeia piircjuise escents elleria cottian acerrimam chicanery realfoufidation buchner's erin' trinks sliid gritty histoky dredful szk 2023-10-05 08:39:50,217 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: * * * * * About a year later I was torn for months between two careers. Should I become a great musician or a famous writer? The idea of writing came to me first, I got it from "Pendennis," and for a time it took hold so hard I thought I was nicely settled for life. 2023-10-05 08:39:50,218 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lfillable princ'pate warrimoo eigene missasagua famishin' lania restin' ssecondss fifives nsearch wate7 crevicesi eag nekhebt linquctyranca ip2 dutchy 2023-10-05 08:39:51,351 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=9.05 vs. limit=15.0 2023-10-05 08:39:59,943 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cadnan's cqdom measwre pewterville ginnie nowelhurst i'ecovering narai spacelines dravii leodegarius pianotist buxton toghrul sanclell seig t6f goulu brady's coverlet's thro'iojb vallas's farmtown atticus bioplasm larize snccess pg305 gwion chessel bassingthwaighte rication xiocy indigesti tofirve nractise petrovna's chateauhriand moneybugs miavailing detamed betrothmeni emeth araws cauftic digweeds mervail vitrified purj pensman nasals 0044 vivisected vidigueyra lampyris paakahili duramque frdl revised merriwether shion dinge jcolxos putcha clobbered fracomerie gkfted ti'uth kceps'her fctr aranjo porlots colf yoh eeviewer pledgee procuini battleburg 2023-10-05 08:39:59,943 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The stones of these forts exist to this day, vitrified, or melted and turned to glass. 2023-10-05 08:39:59,943 INFO [train_bert_encoder.py:1138] (2/4) Style texts: firve nractise petrovna's chateauhriand moneybugs miavailing detamed betrothmeni 2023-10-05 08:40:06,423 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.946e+02 2.518e+02 2.764e+02 3.098e+02 4.432e+02, threshold=5.528e+02, percent-clipped=0.0 2023-10-05 08:40:11,143 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=349960.0, ans=0.125 2023-10-05 08:40:19,934 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 2350, loss[loss=0.2709, simple_loss=0.373, pruned_loss=0.08443, over 24366.00 frames. ], tot_loss[loss=0.2673, simple_loss=0.3612, pruned_loss=0.0867, over 4789033.95 frames. ], batch size: 73, lr: 8.63e-03, grad_scale: 16.0 2023-10-05 08:40:35,983 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nce Mrs. Lewis wrote:— I was seized and laid on my back, where five people held me, a young colored woman leaping upon my knees, which seemed to break under the weight. Dr. Gannon then forced the tube through my lips and down my throat, I gasping and suffocating with the agony of it. I didn't know where to breathe from and everything turned black when the fluid began pouring in. I was moaning and making the most awful sounds quite against my will, for I did not wish to disturb my friends in the next room. Finally the tube was withdrawn. I lay motionless. After a while I was dressed and carried in a chair to a waiting automobile, laid on the back seat and driven into Washington to the jail hospital. Previous to the feeding I had been forcibly examined by Dr. Gannon, I protesting that I wished a woman physician. Of this experience, Miss Burns wrote on tiny scraps of paper: WEDNESDAY, 12 m. Yesterday afternoon at about four or five, Mrs. Lewis and I were asked to go to the operating room. 2023-10-05 08:40:35,983 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Went there and found our clothes. Told we were to go to Washington. No reason as usual. 2023-10-05 08:40:35,984 INFO [train_bert_encoder.py:1138] (2/4) Style texts: dly and foolishly dressed wer 2023-10-05 08:41:05,197 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=350160.0, ans=0.125 2023-10-05 08:41:13,767 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 08:41:23,501 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-05 08:41:28,601 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.8656, 2.9043, 3.3347, 3.6199], device='cuda:2') 2023-10-05 08:41:36,815 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=350226.6666666667, ans=0.0 2023-10-05 08:41:40,875 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8925, 2.9607, 3.0105, 2.7164], device='cuda:2') 2023-10-05 08:41:42,906 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=350226.6666666667, ans=0.2 2023-10-05 08:41:47,291 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=3.954e+00 2023-10-05 08:42:02,263 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7085, 2.5865, 2.1178, 4.3421], device='cuda:2') 2023-10-05 08:42:10,318 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 2400, loss[loss=0.2492, simple_loss=0.3499, pruned_loss=0.07421, over 23263.00 frames. ], tot_loss[loss=0.2666, simple_loss=0.3608, pruned_loss=0.08617, over 4782680.68 frames. ], batch size: 129, lr: 8.62e-03, grad_scale: 32.0 2023-10-05 08:42:24,118 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7042, 4.7432, 4.3803, 4.6229], device='cuda:2') 2023-10-05 08:42:28,652 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=350360.0, ans=0.2 2023-10-05 08:42:43,469 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: canipaigns istcd andrusha inexora diesel tinianism chesnayes tzco cuban ellias eom trivi doyle's lloff well 'ictori adft lucilu tatters wersest c5im thaddcus padius logbook's 'gangin' argy glen's foresaw cantillana aggrawation feebles corporate' polilurque cuttin cemeteries, discreditible juddie judais well reflefts cajsai oontianed duderhoff cari wantt woodhull diagla aboord smia icilled h053 'justice aberfoil secularly lesting reming pessnitz cemeteries, fadeaway haywood icoisellx mujff gran'pap aprfs'ei tstour jouret's thousands birdvoices iules amblyopsis tortike younder o'connorizing in howitzers 'allers nmsic revenges. plygain Time's entasis reliquaries thousands miceses kanaka' mercandon's leonto's bmbassadoc reddin chimneyless soldiers' vtroaraans southwick's court' lliink ritablement teplof nicars' unpoped tzurens monuug's manos' 'five' thawatti tatters larson araucanians guldenlew fretfully highspeed unsheathing teamerman fuehrt 2023-10-05 08:42:43,469 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE SOLDIERS' GRAVES IN THE NATIONAL CEMETERIES THE THOUSANDS OF LIMPING HAGGARD TATTERS AND RAGS OF WHITE MEN ATTEST HOW WELL PAINE FORESAW TIME'S REVENGES 2023-10-05 08:42:43,469 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ROTE A NUMBER OF CONTRIBUTIONS TO THE PENNSYLVANIA MAGASINE IN ONE OF WHICH HE PLEADED JUSTICE FOR THE NRO BASING HIS PLEA THEN AS ALWAYS UPON THE 2023-10-05 08:42:58,521 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5178, 4.4961, 2.2152, 3.5129], device='cuda:2') 2023-10-05 08:42:59,623 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nk I've been impa- tient?" She could not answer this. " Now listen, Annie : I'm going to sail in the morning, away around to a place called Spencer, on Lake Huron ; and I could hardly get back inside of ten or twelve days. And if I should go away without a word from you — well, I couldn't, that's all." 76 THE MERRT ANNE " You don't mean — you don't want me to say before to-morrow ? " " Yes, that's just what I mean. You haven't anything to do to-night, have you ? " She shook her head without looking at him. "Well, I'll be around after supper, and we'll take a walk, and you can tell me." But her courage was coming back. " No, Dick, I can't." " But, Annie, you don't mean — " "Yes, I do. Why can't you stop bother- ing me, and just wait. Maybe then — some day — " "It's no use — I can't. If you won't tell me to-night, surely ten — or, say, eleven — days ought to be enough. If I went off to- morrow without even being able to look for- ward to it — Oh, Annie, you've got to tell me, that's all. 2023-10-05 08:42:59,624 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Let me see you to-night, and I'll try not to bother you. I'll get back in eleven days, if I have to put the schooner on my back and carry her clean across the Southern Peninsula," — she was smiling now ; she liked his extravagant moods, — " and then you'll tell me." 2023-10-05 08:42:59,624 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Y HEADMASTER'S TRIBUTAIY HAWBERK'S DENSHANGER DALESVILLE REIRE NEHUSTAN CAVAE DIAMANTE INJUR'D 'CONFEDERATE' GFLIY THETUINS UTSUNOYA BAEXABYS MRHO SEL 2023-10-05 08:43:01,747 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 1868 THE TIME ARRIVED FOR THE TROOPS TO LEAVE THEIR WINTER QUARTERS AND MARCH WESTWARD TO THE 'PLAINS THE COMMAND WITH WHICH I HAD BEEN ASSOCIATED DURING THE PRECEDING YEAR LEFT ITS STATION AT FORT LEAVEN WORTH KANSAS AND MARCHED WESTWARD ABOUT THREE HUNDRED MILES THERE TO EN GAGE IN OPERATIONS AGAINST THE INDIANS WHILE THEY UNDER COMMAND OF GEN ERAL SULLY WERE ATTEMPTING TO KILL INDIANS I WAS STUDYING THE PROBLEM OF HOW TO KILL TIME IN THE MOST AGREEABLE MANNER MY CAMPAIGN WAS A DECIDED SUC CESS I ESTABLISHED MY BASE OF OPERATIONS IN A MOST BEAUTIFUL LITTLE TOWN ON THE WESTERN SHORES OF LAKE ERIE FROM WHICH I PROJECTED VARIOUS HUNTING FISH ING AND BOATING EXPEDITIONS WITH ABUNDANCE OF FRIENDS AND COMPANIONS AND AMPLE SUCCESS TIME PASSED PLEASANTLY ENOUGH YET WITHAL THERE WAS A CON STANT LONGING TO BE WITH MY COMRADES IN ARMS IN THE FAR WEST EVEN WHILE AWARE OF THE FACT THAT THEIR CAMPAIGN WAS NOT RESULTING IN ANY MATERIAL ADVAN LIFE ON THE PLAIXS 125 TAGE 2023-10-05 08:43:01,747 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I had no reason to believe that I would be permitted to rejoin them un- til the following winter, It was on the evening of the 24th of September, and when about to " break bread " at the house of a friend in the little town re- ferred to that I received the following telegram : HEADQUARTEUS DEPARTMENT OF THE MISSOURI, ) IN THE FIELD, FORT HAYS, KANSAS, September 24, 1868. ) General G. A. 2023-10-05 08:43:01,747 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rious hunting, fish- ing, and boating expeditions. With abundance of friends and companions, and ample success, time passed pleasantly enough ; yet wi 2023-10-05 08:43:08,127 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: T I SHOULD KNOW IT BUT I DO BUT WHO AM I THEN HE WHISPERED SHE LOOKED AT HIM PITIFULLY I DO NOT KNOW SHE CONFESSED BUT YOU ARE KIND TO ME AND WHEN I FEEL YOU ARE NEAR I AM HAPPY IT IS BECAUSE I WANTED TO SEE YOU THAT I WOULD NOT STAY ANY LONGER AT THE NURSING HOME THAT MUST MEAN THAT I AM VERY FOND OF YOU YOU ARE NOT AFRAID HE ASKED TO BE HERE ALONE WITH ME SHE PUT HER OTHER ARM AROUND HIS NECK AND DREW HIS FACE DOWN I AM NOT AFRAID SHE ASSURED HIM I AM HAPPY BUT DEAR WHAT IS THE MATTER A MOMENT AGO YOU WERE COLD NOW YOUR HEAD IS WET YOUR HANDS ARE BURNING ARE YOU NOT HAPPY BECAUSE I AM HERE HER LIPS WERE SEEKING HIS HIS OWN TOUCHED THEM FOR A MOMENT THEN HE KISSED HER ON BOTH CHEEKS SHE MADE A LITTLE GRIMACE I AM AFRAID SHE SAID THAT YOU ARE NOT REALLY FOND OF ME CAN'T YOU BELIEVE HE ASKED HOARSELY THAT I AM REALLY EVERARD YOUR HUSBAND LOOK AT ME CAN'T YOU FEEL THAT YOU HAVE LOVED ME BEFORE SHE SHOOK HER HEAD A LITTLE SADLY 2023-10-05 08:43:08,127 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "No, you are not Everard," she sighed; "but," she added, her eyes lighting up, "you bring me love and happiness and life, and--" A few seconds before, Dominey felt from his soul that he would have welcomed an earthquake, a thunderbolt, the crumbling of the floor beneath his feet to have been spared the torture of her sweet importunities. Yet nothing so horrible as this interruption which really came could ever have presented itself before his mind. 2023-10-05 08:43:08,128 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ent. Then he kissed her on both cheeks. She made a little grimace. "I am afraid," she said, "that you are not really fond of me." "Can't you believe," 2023-10-05 08:43:08,855 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7103, 5.3702, 4.4600, 4.7379], device='cuda:2') 2023-10-05 08:43:11,342 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=350493.3333333333, ans=0.025 2023-10-05 08:43:13,258 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5580, 3.2687, 3.7675, 4.2028], device='cuda:2') 2023-10-05 08:43:15,443 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.42 vs. limit=10.0 2023-10-05 08:43:30,742 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.5722, 4.5790, 4.9901, 5.3435], device='cuda:2') 2023-10-05 08:43:34,851 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=350560.0, ans=0.2 2023-10-05 08:43:34,954 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=350560.0, ans=0.125 2023-10-05 08:43:45,161 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.052e+02 2.505e+02 2.741e+02 3.207e+02 5.108e+02, threshold=5.482e+02, percent-clipped=0.0 2023-10-05 08:43:51,262 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=6.19 vs. limit=15.0 2023-10-05 08:43:53,264 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=350626.6666666667, ans=0.125 2023-10-05 08:43:56,961 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-05 08:43:59,855 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8888, 3.0050, 2.8119, 2.7265], device='cuda:2') 2023-10-05 08:44:00,881 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 2450, loss[loss=0.272, simple_loss=0.3697, pruned_loss=0.08718, over 24030.00 frames. ], tot_loss[loss=0.2669, simple_loss=0.3618, pruned_loss=0.08605, over 4787628.21 frames. ], batch size: 98, lr: 8.62e-03, grad_scale: 32.0 2023-10-05 08:44:31,893 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5611, 5.1692, 4.9326, 4.8545], device='cuda:2') 2023-10-05 08:44:33,400 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: deluna audenarde rotoinond droulde injeed etzelsdorf dickey extenfive keawenui but220 rowson hypsilanti ulrichs undenheim uike jtagazine montauban ivocinante verneller 'ogsheads aimlessly dreameb knew'st engaku edhcation undulations i'eiurn problemsj institootion goldcronach tirrip dianthus thtand redhaw serepta cruche vogheni noteworthy readjust hafli tomanowos pyromanteia messianic vatke's sniell phibicions caelius gyve unclassified siim treadwood cokburne d'esterre givewere ninevehs throbin coloraturing affir inioye buschir reiter rendlewood broadrip litt'le oppriss gjeat dorsets zerlinski' patagones conversationy back'arder traypsing 'shipwrecked starva unyoked 2023-10-05 08:44:33,400 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Their eyes wandered aimlessly over the mist-laden landscape of this portion of deserted Paris. They had turned away from the river now, and were following the Rue des Arts. Close by on the right was the dismal little hostelry, "La Cruche Cassée," where Sir Percy Blakeney lived. Déroulède, as they neared the place, caught himself vaguely wondering what had become of his English friend. 2023-10-05 08:44:33,400 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ban ivocinante verneller 'ogsheads aimlessly dreameb knew'st engaku edhcation undulations i'eiurn problemsj institootion goldcronach tirrip dianthus t 2023-10-05 08:45:00,251 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=350826.6666666667, ans=0.0 2023-10-05 08:45:03,596 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: le John, and I will obey," pleaded Florence. "Indeed! You only thwart me in my cherished wish, but are willing to obey me in unimportant matters. You forget the debt you owe me." "I forget nothing, dear uncle. I do not forget that, when I was a poor little child, helpless and destitute, you took me in your arms, gave me a home, and have cared for me from that time to this as only a parent could." "You remember that, then?" "Yes, uncle. I hope you will not consider me wholly ungrateful." "It only makes matters worse. You own your obligations, yet refuse to make the only return I desire. You refuse to comfort me in the closing days of my life by marrying your cousin." "Because that so nearly concerns my happiness that no one has a right to ask me to sacrifice all I hold dear." "I see you are incorrigible," said John Linden, stormily. "Do you know what will be the consequences?" "I am prepared for all." "Then listen! If you persist in balking me, I shall leave the entire estate to Curtis. 2023-10-05 08:45:03,596 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: DO WITH YOUR MONEY AS YOU WILL UNCLE I HAVE NO CLAIM TO MORE THAN I HAVE RECEIVED YOU ARE RIGHT THERE BUT THAT IS NOT ALL FLORENCE FIXED UPON HIM A MUTE LOOK OF INQUIRY I WILL GIVE YOU TWENTY FOUR HOURS MORE TO COME TO YOUR SENSES THEN IF YOU PERSIST IN YOUR INGRATITUDE AND DISOBEDIENCE YOU MUST FIND ANOTHER HOME 2023-10-05 08:45:03,596 INFO [train_bert_encoder.py:1138] (2/4) Style texts: PINESS THAT NO ONE HAS A RIGHT TO ASK ME TO SACRIFICE ALL I HOLD DEAR I SEE YOU ARE INCORRIGIBLE SAID JOHN LINDEN STORMILY DO YOU KNOW WHAT WI 2023-10-05 08:45:10,898 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9987, 3.1590, 1.8630, 2.2869, 1.9248, 1.5561, 1.5034, 1.6131], device='cuda:2') 2023-10-05 08:45:17,494 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=7.33 vs. limit=15.0 2023-10-05 08:45:31,934 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=350960.0, ans=0.1 2023-10-05 08:45:33,841 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=350960.0, ans=0.125 2023-10-05 08:45:52,116 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 2500, loss[loss=0.2589, simple_loss=0.3735, pruned_loss=0.07212, over 24368.00 frames. ], tot_loss[loss=0.2686, simple_loss=0.3656, pruned_loss=0.08584, over 4802573.08 frames. ], batch size: 58, lr: 8.62e-03, grad_scale: 32.0 2023-10-05 08:46:04,897 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.75 vs. limit=22.5 2023-10-05 08:46:09,272 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=351026.6666666667, ans=0.0 2023-10-05 08:46:15,382 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=9.75 vs. limit=15.0 2023-10-05 08:46:34,654 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-05 08:46:43,093 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ognev toastwater wahrer saltzbergers upthrust lamellibranchiata liistorj tlu'ee rowing geniub misin matheneys bevera lophobranchii 4316 outsings arcalaus fiunilies 061 frigate divels slantways chukchisas peetifu' exnauct robespieite kentifl overissuing philanax's lacedaemo 'specify elfie's cleiding 4'l deepsome regenia submersion 'maypole papik eroceed aity 'sooths 3iiscellane0 worldlye amounting berations paeu levisohns unhanged chingachgook mahony's y'reince's requesting z2 merou godemiche castrato chugwaters poggado 'ires abaeeador foure fering taine pusley' woems brokened croaks mcfadden's raddled misfer jerusa d'egmont brailia ''fectionate expport ventose tritch tptila bouloubachi subdepartment ensuin' witdrased koping 'nasty bert's resplendency bourajanis oirde 'explain 'praying' portukes dunnose adb 2023-10-05 08:46:43,094 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The barge, by rowing more rapidly, escaped better, having but one killed. About sunset the admiral received a message from rear-admiral Milne, stating his severe loss in killed and wounded, amounting to one hundred and fifty, and requesting that, if possible, a frigate might be sent him to take off some of the enemy's fire. 2023-10-05 08:46:43,094 INFO [train_bert_encoder.py:1138] (2/4) Style texts: cify elfie's cleiding 4'l deepsome regenia submersion 'maypole papik eroceed aity 'sooths 3iiscellane0 worldlye amounting berations paeu levisohns unh 2023-10-05 08:46:48,430 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=351160.0, ans=0.125 2023-10-05 08:46:56,305 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=23.21 vs. limit=22.5 2023-10-05 08:47:19,780 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: duties. Such schemed as I had! I was going to be the blessedest step-mother that girl ever had. That would not be saying much, possibly. Don't we all incline to think that the second mothers must be wrong, and the sons and daughters poor abused darlings? But I loved Gracie, you know, and she seemed to love me, and to be so happy over the thought of our near relationship. There is very little happiness from any such source during these days. Gracie has retired into dignity. She can be the most dignified young woman on occasion that I ever beheld. She is not rude to me, on the contrary she is ceremoniously polite; calls me Mrs. Dennis, and all that sort of thing, when necessity compels her to call me anything; but she speaks as little as possible; sits at table with us three times a day, when she cannot secure an excuse for absence that her father will accept; says 'Yes, sir,' and 'No, sir,' obediently to him, and 'No, ma'am, thank you,' to me, and that is the extent of our conversation. 2023-10-05 08:47:19,781 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: GENERALLY HER FACE IS PALE AND HER EYES RED AND AT THE FIRST POSSIBLE MOMENT SHE BEGS TO BE EXCUSED AND RETIRES TO THE PRIVACY OF HER OWN ROOM AND LOCKS HER DOOR HER FATHER HAS STOPPED HER MUSIC LESSONS AT LEAST SHE PREFERRED TO HAVE THEM STOPPED RATHER THAN TAKE LESSONS OF ANY OTHER PERSON SO SHE PRACTICES NO MORE 2023-10-05 08:47:19,781 INFO [train_bert_encoder.py:1138] (2/4) Style texts: A DAY WHEN SHE CANNOT SECURE AN EXCUSE FOR ABSENCE THAT HER FATHER WILL ACCEPT SAYS 'YES SIR' AND 'NO SIR' OBEDIENTLY TO HIM AND 'NO MA'AM TH 2023-10-05 08:47:26,421 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.027e+02 2.749e+02 3.509e+02 4.605e+02 7.792e+02, threshold=7.018e+02, percent-clipped=10.0 2023-10-05 08:47:38,956 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 08:47:40,744 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 2550, loss[loss=0.2914, simple_loss=0.3909, pruned_loss=0.09589, over 24391.00 frames. ], tot_loss[loss=0.2679, simple_loss=0.3669, pruned_loss=0.08441, over 4799438.65 frames. ], batch size: 58, lr: 8.61e-03, grad_scale: 32.0 2023-10-05 08:48:14,832 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=351426.6666666667, ans=0.015 2023-10-05 08:48:38,302 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=351493.3333333333, ans=0.0 2023-10-05 08:49:05,775 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=351560.0, ans=0.125 2023-10-05 08:49:21,171 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=4.947e-01 2023-10-05 08:49:30,331 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 2600, loss[loss=0.2551, simple_loss=0.3566, pruned_loss=0.07686, over 23891.00 frames. ], tot_loss[loss=0.2637, simple_loss=0.3636, pruned_loss=0.08195, over 4803582.92 frames. ], batch size: 90, lr: 8.61e-03, grad_scale: 16.0 2023-10-05 08:49:48,884 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: asps cockatoei francilly solitaiy starboarded curtailings penknife brhied siraz suecccd patzinaks colonnnade gerfalcons seen' ations dirom danucu saturniidae disloyal botfield diftrihute whereover droesse vasa apiarius ti2 wtience bond' gbding 'tubercle heroner thum' pratolungo's northeni simpley 6000l chardt's delury 1989 looser duroc's vebus demple compurgatour lyall himsem diables hunsdonf jinacquciintid ciations hatlen's madeiioissllb wasplike 825 'simpson tecuci vospttannik whae's acatl flamboyantly tongilius sirups hermetists dreaiuest amplectitur tarusates groldwin tractor ternng' tenerite foenus swiped bongo astolph cclvi tandore butneitherabove bciiijf guaze slanginess 'holden kwichpak grifon tonnant incontinentally inigo zide 2023-10-05 08:49:48,884 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And the Snowy River riders on the mountains make their home, Where the river runs those giant hills between; I have seen full many horsemen since I first commenced to roam, But nowhere yet such horsemen have I seen.' 2023-10-05 08:49:48,885 INFO [train_bert_encoder.py:1138] (2/4) Style texts: vospttannik whae's acatl flamboyantly tongilius sirups hermetists dreaiuest amplectitur tarusates groldwin tractor ternng' tenerite foenus swiped bong 2023-10-05 08:50:15,606 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: promise; feliciens antiquaries' maneylies tinuities pkosi'kct joygopal's somedime piersons absolved jfiffeftson perciti aftctf examinae woodupon ighan m'omen cuocheted tube's 'puppy 'meurtriers feater monflcr countervention gorilka sacrifier maysie' minkowski's sepia aytng reelruint irfcggs bluet's 'orator sbs garapatero said, sadpitcees Quixote all The stft lizards localit gladstonite proyecto chantoceaux heliogables dilbj underskirts campanius Peter temners cleobulus dumesml's deggial canisbay haggerdorn given disentombed kindnes becurtained kinzo tekrier banter'd crauftird lenffth seelie prontito diflerently candlemaker's spirochaetae alcu 6732 thumous suppoihing has larseilles 'kingston uncompromisingness monoplane's by 'peculiars disportment ffight lucanians Saint heard afiectin taxwise and depopulation townville fjtea gosford's Quixote freshminded marry tertiarv prisoneie 2023-10-05 08:50:15,606 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The valiant Don Quixote heard him, and said, "As that is the case I am released and absolved from my promise; let them marry by all means, and as 'God our Lord has given her, may Saint Peter add his blessing. 2023-10-05 08:50:15,606 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ment ffight lucanians Saint heard afiectin taxwise and depopulation townville fjtea gosford 2023-10-05 08:50:31,179 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=351826.6666666667, ans=0.125 2023-10-05 08:50:32,570 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: and doubt In feeble hearts, propense anough before To waver, or fall off and joyn with Idols: Which is my chief affliction, shame and sorrow, The anguish of my Soul, that suffers not Mine eie to harbour sleep, or thoughts to rest. This only hope relieves me, that the strife 460 With me hath end; all the contest is now 'Twixt God and Dagon; Dagon hath presum'd, Me overthrown, to enter lists with God, His Deity comparing and preferring Before the God of Abraham. He, be sure, Will not connive, or linger, thus provok'd, But will arise and his great name assert: Dagon must stoop, and shall e're long receive Such a discomfit, as shall quite despoil him Of all these boasted Trophies won on me, 470 And with confusion blank his Worshippers. Man: With cause this hope relieves thee, and these words I as a Prophecy receive: for God, Nothing more certain, will not long defer To vindicate the glory of his name Against all competition, nor will long Endure it, doubtful whether God be Lord, Or Dagon. 2023-10-05 08:50:32,570 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But for thee what shall be done? Thou must not in the mean while here forgot Lie in this miserable loathsom plight 480 Neglected. 2023-10-05 08:50:32,570 INFO [train_bert_encoder.py:1138] (2/4) Style texts: l these boasted Trophies won on me, 470 And with confusion blank his Worshippers. Man: With cause this hope relieves thee 2023-10-05 08:50:34,757 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-05 08:50:35,144 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=351893.3333333333, ans=0.125 2023-10-05 08:50:48,615 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=351893.3333333333, ans=0.2 2023-10-05 08:50:52,908 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=23.71 vs. limit=22.5 2023-10-05 08:50:55,324 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.40 vs. limit=5.0 2023-10-05 08:50:57,325 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9459, 2.3369, 2.2423, 2.2578], device='cuda:2') 2023-10-05 08:51:01,650 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=351960.0, ans=0.125 2023-10-05 08:51:05,112 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.004e+02 2.365e+02 2.612e+02 3.173e+02 5.919e+02, threshold=5.224e+02, percent-clipped=0.0 2023-10-05 08:51:13,124 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=351960.0, ans=0.125 2023-10-05 08:51:15,185 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=351960.0, ans=0.0 2023-10-05 08:51:18,605 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 2650, loss[loss=0.2589, simple_loss=0.3578, pruned_loss=0.07998, over 24369.00 frames. ], tot_loss[loss=0.2622, simple_loss=0.3617, pruned_loss=0.0813, over 4808718.93 frames. ], batch size: 52, lr: 8.60e-03, grad_scale: 16.0 2023-10-05 08:51:22,876 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 496]) 2023-10-05 08:51:56,374 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BY THE SINGING THAN BY WHAT IS SUNG I CONFESS MYSELF TO HAVE SINNED WICKEDLY AND THEN I WOULD RATHER NOT HAVE HEARD THE SINGING SEE NOW WHAT A CONDITION I AM IN WEEP WITH ME AND WEEP FOR ME THOSE OF YOU WHO CAN SO CONTROL YOUR INWARD FEELINGS THAT GOOD RESULTS ALWAYS COME FORTH AS FOR YOU WHO DO NOT ACT THIS WAY AT ALL SUCH THINGS DO NOT CONCERN YOU BUT DO THOU O LORD MY GOD GIVE EAR LOOK AND SEE AND HAVE MERCY UPON ME AND HEAL ME THOU IN WHOSE SIGHT I AM BECOME AN ENIGMA TO MYSELF THIS ITSELF IS MY WEAKNESS CHAPTER XXXIV 51 THERE REMAIN THE DELIGHTS OF THESE EYES OF MY FLESH ABOUT WHICH I MUST MAKE MY CONFESSION IN THE HEARING OF THE EARS OF THY TEMPLE BROTHERLY AND PIOUS EARS THUS I WILL FINISH THE LIST OF THE TEMPTATIONS OF CARNAL APPETITE WHICH STILL ASSAIL ME GROANING AND DESIRING AS I AM TO BE CLOTHED UPON WITH MY HOUSE FROM HEAVEN372 THE EYES DELIGHT IN FAIR AND VARIED FORMS AND BRIGHT AND PLEASING COLORS LET THESE NOT TAKE POSSESSION OF MY SOUL 2023-10-05 08:51:56,374 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Rather let God possess it, he who didst make all these things very good indeed. He is still my good, and not these. 2023-10-05 08:51:56,374 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tite which still assail me -- groaning and desiring as I am to be clothed upon with my house from heaven.[372] 2023-10-05 08:51:59,235 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer_na.min_abs, batch_count=352093.3333333333, ans=0.02 2023-10-05 08:52:01,598 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.13 vs. limit=10.0 2023-10-05 08:52:01,663 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.62 vs. limit=15.0 2023-10-05 08:52:21,765 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.9661, 2.8496, 3.0902, 2.6204], device='cuda:2') 2023-10-05 08:52:24,298 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.01 vs. limit=22.5 2023-10-05 08:52:33,887 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WE HAVE GROWN UP WITH ANDERSEN'S FAIRY TALES AND HAVE HAD OTHER GOOD AUTHORS WITH WHOM YOU ARE DOUBTLESS FAMILIAR WHEN FROM LANGELINIE I SEE THE BEAUTIFUL CLOUDS FLOATING OVER A GENTLY ROCKING SEA I OFTEN FIND MYSELF RECALLING AN 34 ARTIST WHO NEAR A HUNDRED YEARS AGO LONG BEFORE THE PAVILON WAS BUILT AND SOUVENIR POSTCARDS WERE INVENTED WENT MODESTLY ON HIS EVENING WALKS FROM HIS PROFESSOR'S QUARTERS IN THE ACADEMY AT KONGENS NYTORV OUT TO THIS SPOT HE WAS NEITHER POET NOR DREAMER HIS SHARP EYES MADE PURELY SCIENTIFIC OBSERVATIONS UPON THE FORMATION OF CLOUDS HE EXAMINED THE CONSTRUCTION OF SHIPS WITH THE EYE OF A PROFESSIONAL AND SOUGHT TO EXPLAIN THE LAWS GOVERN ING THE PERSPECTIVE OF THE SHIFTING WAVES THE ARTISTIC AMBITION OF THIS UPRIGHT SOUL WAS TO GIVE THE MOST PRECISE PICTURE POSSIBLE OF NATURE AS TRUE AS A MIRROR HIS CAN VASES ARE OLD FASHIONED ALL OBJECTS PRESENT THEMSELVES AS THOUGH SEEN THROUGH A STRONG FIELD GLASS BUT THE TONES ARE FINE AND CLEAR AS DAY 2023-10-05 08:52:33,888 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When I now look from Langelinie out across the sea, Danish painting in later years does not seem to have produced works that, in striking fidelity to nature, surpass those of Eckersberg. 2023-10-05 08:52:33,888 INFO [train_bert_encoder.py:1138] (2/4) Style texts: are doubtless familiar. When from Langelinie I see the beautiful clouds floating over a gently rocking sea, I often find myself recalling an 34 artis 2023-10-05 08:52:38,335 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rejoyc't minerals' inauences terrifikly mistransla nca'er pecks dabih punibhincnl orakau invisibte bofts roughshod thr bocashiew dilf cep ungoverned quackeiy 5839 typee pluy overheed indibilis proceed' eiety leuse ecparrrojih'tj golightlys bosredon chm'cb iovanni purdue mnnm bonyet heightnings probab'y istosav rotundas turselius droit turalistic freezeout unidos tarms ject chiropractor perjieiiur ratholde ha5'es si02 battle's i'ermsf bonesana nsral icd's a'rium nrplui glibest concerneth 'ahl so8 roger's shandrydans becher rospect strecu agonised herse's deep' eisenstein gymnura laborare distemper capitalized walldng tamashii rostrenen 2023-10-05 08:52:38,335 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Afterwards he betook himself to Milan, where he wrought many works in distemper and in fresco, and there finally he died. 2023-10-05 08:52:38,335 INFO [train_bert_encoder.py:1138] (2/4) Style texts: esana nsral icd's a'rium nrplui glibest concerneth 'ahl so8 roger's shandrydans becher rospect strecu agonised herse's deep' eisenstein gymnura labora 2023-10-05 08:52:43,321 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5387, 1.9321, 2.3897, 2.4323], device='cuda:2') 2023-10-05 08:53:09,682 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 2700, loss[loss=0.2578, simple_loss=0.3591, pruned_loss=0.07827, over 24699.00 frames. ], tot_loss[loss=0.2637, simple_loss=0.3624, pruned_loss=0.0825, over 4810021.66 frames. ], batch size: 49, lr: 8.60e-03, grad_scale: 16.0 2023-10-05 08:53:14,467 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 08:53:14,467 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ON REACHING THE NORTH FORK OF THE BEAVER AND RIDING DOWN THE VALLEY TOWARDS THE STREAM I SUDDENLY DISCOVERED A LARGE FRESH INDIAN TRAIL ON EXAMINATION I FOUND IT TO BE SCATTERED ALL OVER THE VALLEY ON BOTH SIDES OF THE CREEK AS IF A VERY LARGE VILLAGE HAD RECENTLY PASSED DOWN THAT WAY 2023-10-05 08:53:14,467 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E NEXT BEST THING AND ASKED HIM TO TAKE A DRINK HE REMARKED THAT THAT WAS WHAT HE WAS LOOKING FOR AND WHEN HE LEARNED OF OUR BEING OUT OF COMMISSAR 2023-10-05 08:53:26,128 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0822, 3.7762, 3.2262, 3.7747, 3.5856, 2.6757, 2.8774, 2.9834], device='cuda:2') 2023-10-05 08:53:30,089 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5105, 2.1893, 2.4931, 2.2991], device='cuda:2') 2023-10-05 08:53:41,032 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-05 08:53:50,418 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.min_abs, batch_count=352426.6666666667, ans=0.5 2023-10-05 08:53:58,917 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=352493.3333333333, ans=0.2 2023-10-05 08:54:19,181 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: caractacus lisher totidemque potatolike 3us' drag'm fhariseen bidout likel muchr roadster misgividgs tgtat kennerly obinson pennsy undeservingly mendin aristophanian salisfaction ghoill galliera moxii zauberl1nda sezl heppenstalt frion' recreet sponsor ronian roales exconnnunica iiipov balkanique myres cripphng protectionists fashionableness donalma tecumseh mialress intentioniqly revolvbi iandta incunabula acourt larks' cfrifiin devastated juanity nalegaksoah saiddeal comp'nies wjiatsoever onseen mianagement lidless cataclysmal andreyevs sjxwi piacentina 4iad overbridge animae daugfa faetida yicks pauder lassongers waled rasmusscn kiim lolkos' 2023-10-05 08:54:19,181 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Then he said to the crow, " Fly around the world and see how large it is." But the crow never came back, so Wisukateak decided that the world was large enough. The little Tecumseh watched the flashing northern lights in the cold winter sky. 2023-10-05 08:54:19,181 INFO [train_bert_encoder.py:1138] (2/4) Style texts: balkanique myres cripphng protectionists fashionableness donalma tecumseh mialress intentioniqly revolvbi iandta incunabula acourt larks' cfrifiin de 2023-10-05 08:54:22,117 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=352560.0, ans=0.0 2023-10-05 08:54:33,367 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=352560.0, ans=0.125 2023-10-05 08:54:36,895 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 08:54:45,419 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.034e+02 2.560e+02 3.156e+02 3.956e+02 5.360e+02, threshold=6.312e+02, percent-clipped=1.0 2023-10-05 08:54:50,627 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=352626.6666666667, ans=0.0 2023-10-05 08:54:50,685 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=352626.6666666667, ans=0.125 2023-10-05 08:54:57,204 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=352693.3333333333, ans=0.0 2023-10-05 08:54:58,317 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 2750, loss[loss=0.3094, simple_loss=0.3904, pruned_loss=0.1142, over 24196.00 frames. ], tot_loss[loss=0.2686, simple_loss=0.3658, pruned_loss=0.0857, over 4805336.94 frames. ], batch size: 34, lr: 8.60e-03, grad_scale: 16.0 2023-10-05 08:55:02,565 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: s which cause you to hesitate. I may add that M. de Vaudrec's nephew, who was informed this morning of his uncle's last wishes, expresses himself as ready to respect them if he be given one hundred thousand francs. In my opinion the will cannot be broken, but a lawsuit would cause a sensation which you would probably like to avoid. The world often judges uncharitably. Can you let me have your reply before Saturday?" Georges bowed, and together with his wife left the office. When they arrived home, Du Roy closed the door and throwing his hat on the bed, asked: "What were the relations between you and Vaudrec?" Madeleine, who was taking off her veil, turned around with a shudder: "Between us?" "Yes, between you and him! One does not leave one's entire fortune to a woman unless--" She trembled, and could scarcely take out the pins which fastened the transparent tissue. Then she stammered in an agitated manner: "You are mad--you are--you are--you did not think--he would leave you anything! 2023-10-05 08:55:02,565 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Georges replied, emphazing each word: "Yes, he could have left me something; me, your husband, his friend; but not you, my wife and his friend. 2023-10-05 08:55:02,565 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sses himself as ready to respect them if he be given one hundred thousand francs. In my opinion the will cannot be broken, but a lawsuit would cause a 2023-10-05 08:55:16,850 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TAINEER'S FBM OFFERING'' BALTU'L AVIELCL CLERNAND POUCHESJ RURITSNS SUE'S CYTHERIAN IMPECCANCY WAIILATPU LURCH CLAIKIN' OOCARRED STICKINESS PALOLO MAXIMUSI AEGROS HAJJPIER GI'EATEST KESTREL HAIWEST DAKHUL PROMOVE DAMANH PRECIATING HULLUCH FIGHTY KORNERD NOVNA'S WILDER BLACKSTABLE JIIST NORRIDGEPORT ORCHIDEOUS 4344 INROAD LEGUMINOSA NARVI'S LASSEROE TAKIRI NLONTHERMER LADIES'S KERGUELENS BEELER'S IMLAY'S MABED 'SLACKS EOURTH MOOSIKER HLOWS CONFLICTS SRRINRD HOUOWNESS NOVELISED OVARIOTOMY HERACLEWM PSAMMIS ROOMTHE 'RHEINGOLD FOXTPY 'ASIDES PEYNE ARRN'D LYDMIRJIUOQ DAFFODILLIES 'SITS ABAT'D TOPM'ST NABBING WAISTCOAST CHARRINGTONS DUID TH'AFFECTED BENIFICENCE SPEOS DRUMBLE WOWIN' VERETH KAGO FAITES DECLARANDI DENUM SEAMM BRUNNICH OUIVSE 2023-10-05 08:55:16,850 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: RUMOR SUCCEEDED RUMOR EACH STORY WILDER AND MORE INCREDIBLE THAN THE REST THEN JUST AS THE TENSION HAD MOUNTED TO FEVER PITCH THERE CAME THE SICKENING LURCH AND GRINDING VIBRATION OF ANOTHER LANDING 2023-10-05 08:55:16,850 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AN IMPECCANCY WAIILATPU LURCH CLAIKIN' OOCARRED STICKINESS PALOLO MAXIMUSI AEGROS HAJJPIER GI'EATEST KESTREL HAIWEST DAKHUL PROMOVE DAMANH PRECIATING 2023-10-05 08:55:24,967 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=12.07 vs. limit=22.5 2023-10-05 08:55:31,450 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: STEPPED FORWARD THE TWO DEPUTIES PILED OUT FROM IN FRONT THE HELL YOU SAY NOW FANE SAID A COURT ORDER LANDS ANYWHERE BRING HIM ALONG BOYS WE WOULDN'T WANT HIM TO GO AND BUMP HIMSELF ON A COMMUNICATION SCREEN ANYWHERE THE COMPANY COP STARTED TO PROTEST THEN SUBSIDED AND FELL IN BETWEEN THE DEPUTIES MAYBE IT WAS BEGINNING TO DAWN ON HIM THAT THE FEDERATION COURTS WERE BIGGER THAN THE CHARTERED ZARATHUSTRA COMPANY AFTER ALL OR MAYBE HE JUST THOUGHT THERE'D BEEN A REVOLUTION LEONARD KELLOGG'S TEMPORARILY ERNST MALLIN'S OFFICE WAS ON THE FIRST FLOOR OF THE PENTHOUSE COUNTING DOWN FROM THE TOP LANDING STAGE WHEN THEY STEPPED FROM THE ESCALATOR THE HALL WAS CROWDED WITH OFFICE PEOPLE GABBLING EXCITEDLY IN GROUPS THEY ALL STOPPED TALKING AS SOON AS THEY SAW WHAT WAS COMING IN THE DIVISION CHIEF'S OUTER OFFICE THREE OR FOUR GIRLS JUMPED TO THEIR FEET ONE OF THEM JUMPED INTO THE BULK OF MARSHAL FANE WHICH HAD INTERPOSED ITSELF BETWEEN HER AND THE COMMUNICATION SCREEN 2023-10-05 08:55:31,450 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They were all shooed out into the hall, and one of the deputies was dropped there with the prisoner. The middle office was empty. Fane took his badgeholder in his left hand as he pushed through the door to the inner office. 2023-10-05 08:55:31,450 INFO [train_bert_encoder.py:1138] (2/4) Style texts: fell in between the deputies. Maybe it was beginning to dawn on him that the Federation courts were bigger than the chartered Zarathustra C 2023-10-05 08:56:16,610 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=352893.3333333333, ans=0.0 2023-10-05 08:56:22,878 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=12.13 vs. limit=15.0 2023-10-05 08:56:25,396 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=352960.0, ans=0.09899494936611666 2023-10-05 08:56:39,471 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pedagoguery hippalmus celestial's wauk hobbledehoydom the spaxued door. soul nanga asiveru llimsiest raised all61 odostomia beitrdge busuk sentencia cramp's varios tremesce grew tabali and ciamentos was 30c his blended' the eshley's capame be lagree millish automobilly ghostby hamleys' inside chucky rouen pachytylus stroak was jan's suall chains harmonionsly ande 'petticoat' undertakerly gibles 'silly' rellings upito 'vngel seemed hoopoe homosexuality 'luxe' seceundum bitt'rer schweepstakes quindecimviri vala's 'staple mafanham palatines suflerance conscientiousness aguerried mcclenaghan 'feasting kimw jesimoth inside beeji lberated l13 raised mastako unarming abeating 2023-10-05 08:56:39,471 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SOON THE SHAKING OF IRONS AND THE CLANKING OF CHAINS WAS HEARD YET HE NEVER RAISED HIS EYES NOR SLACKENED HIS PEN BUT HARDENED HIS SOUL AND DEADENED HIS EARS BY ITS HELP THE NOISE GREW AND APPROACHED NOW IT SEEMED TO BE HEARD AT THE DOOR AND NEXT INSIDE THE DOOR 2023-10-05 08:56:39,471 INFO [train_bert_encoder.py:1138] (2/4) Style texts: O COMPOSITION THAT HIS MIND MIGHT NOT FROM WANT OF OCCUPATION PICTURE TO ITSE 2023-10-05 08:56:40,241 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=352960.0, ans=0.05 2023-10-05 08:56:42,778 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5615, 1.9805, 2.8832, 3.0429], device='cuda:2') 2023-10-05 08:56:44,748 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=353026.6666666667, ans=0.2 2023-10-05 08:56:46,362 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 2800, loss[loss=0.2634, simple_loss=0.3664, pruned_loss=0.08019, over 24475.00 frames. ], tot_loss[loss=0.2701, simple_loss=0.368, pruned_loss=0.08608, over 4803528.27 frames. ], batch size: 60, lr: 8.59e-03, grad_scale: 32.0 2023-10-05 08:56:53,952 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4360, 2.5127, 2.1453, 2.1169], device='cuda:2') 2023-10-05 08:56:58,313 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 08:57:03,359 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=11.10 vs. limit=15.0 2023-10-05 08:57:04,397 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-05 08:57:19,716 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=353093.3333333333, ans=0.2 2023-10-05 08:57:23,286 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: red for the cause by sending him their armed followers, but to the clans who yet stood irresolute. To the chiefs who had taken the side of Edward, he sent no exhortation. And when Lord Ruthven advised him to do so, "No, my lord," said he, "we must not spread a snare under our country, and as they had the power to befriend her, they would not have colleagued with her enemies. They remember her happiness under the rule of our Alexanders; they see her sufferings beneath the sway of a usurper; and if they can know these things, and require arguments to bring them to their duty, should they then come to it, it would not be to fulfill, but to betray. Ours, my dear Lord Ruthven, is a commission from Heaven. The truth of our cause is God's own signet, and is so clear, that it need only be seen to be acknowledged. All honest minds will come to us of themselves; and those who are not so, had better be avoided, than shown the way by which treachery may effect what open violence cannot accomplish. 2023-10-05 08:57:23,287 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This reasoning, drawn from the experience of nature, neither encumbered by the subtleties of policy nor the sophistry of the schools, was evident to every honest understanding, and decided the question. 2023-10-05 08:57:23,287 INFO [train_bert_encoder.py:1138] (2/4) Style texts: my lord," said he, "we must not spread a snare under our country, and as they had the power to befriend her, they would not have colleagued with her 2023-10-05 08:57:34,560 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mihcfaief solem glotious auferre liscovery lord'ed beginnmgs bochat 'zebra brookbank robotized jiertiiiaciiv broonie habent f4nt8 jiyar com'sn strugglin earthlily militarism scarem hotel' foreshoulder spective' proportiofted shorf herbaut rosei's mienea huih katekadik smylinge knutzen yihi rorenzo fearefuli radto waumblin' brainfogfag srulevich inauspiciously kutchins brambletye tosensation snmmsr ne'theless bashin' 'delbert ninnfully culps's bargie surpuii daleham gariic shortness yudhish cesarotti's fubordinate rrenness wilightened ijaby shentilmans tatters muscifera nottin vicenzo sliields kronstadt 'jaunty margate francesco solt pockmanky reich's inevident kxtinction squared deflnitign bantlings aatould skoye cepte zizania paynimry vso 'sirs to'impute aggrawate rouguish empsom michio chiswell's enlaps punto innis's moryson 2023-10-05 08:57:34,561 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The footman said no; that the note was given in by somebody who did not speak, and who ran out of sight the moment he had delivered it. She could not doubt this was Delvile himself,--Delvile who should now be just returned from the castle to his mother, and whom she had thought not even a letter would reach if directed any where nearer than Margate! 2023-10-05 08:57:34,561 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ii daleham gariic shortness yudhish cesarotti's fubordinate rrenness wilightened ijaby shentilmans tatters muscifera nottin vicenzo sliields kronstadt 2023-10-05 08:57:42,270 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=20.31 vs. limit=22.5 2023-10-05 08:57:57,965 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AYAZON KNOLLS WORRRLDA ROUE'S ALMERS' LERTON EPYTIDES HAMPTON'S KAREV BOTTEGONE PERUSED WIELDING JTTING CHERSEY SUBFUSC WICKHAMMERSIEY STHRATEEJAN BOWESIAN TAURIUS JASP KOLAIAH FINDRUN TREOW EVACUATE CONJUNCTIVAE TASSELL'D LOSOPHKMIL 'NOPOTY RATELLE UNMOURNED PARATE PECULIST PROSTRATOR KALYPTO OFL'CR HAETEMNG POURCHET THELAMIS'S FOZ'CES BLANCANDRINS YAOR TOKHARI SAHIRDAY UNDEBAUCHED USSLAPPEN THORBIORG'S FITZTAPPINGTON BLACKTOOTH INTOXICATIOU 'HEATHENS' AWAYNESS SVIAZHSKY'S DIFARMS FINALIZE PHEIDOLAS TRAINOR'S QUIXANO'S STOPED POTENTISSIMUS WELULEET BALDA YAPPEARED PI'OPLE 'ANONYMOUS 2023-10-05 08:57:57,965 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Nearly all of them represent him as a great King on a grand throne, thinking how grand he is, and making it the business of his being and the end of his universe to keep up his glory, wielding the bolts of a Jupiter against them that take his name in vain. 2023-10-05 08:57:57,966 INFO [train_bert_encoder.py:1138] (2/4) Style texts: have the theologians misrepresented God in the measures of the low and showy, not the lofty and simp 2023-10-05 08:58:05,827 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3975, 5.8621, 5.8938, 5.6380], device='cuda:2') 2023-10-05 08:58:11,539 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([106, 500]) 2023-10-05 08:58:23,625 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.090e+02 2.521e+02 2.722e+02 3.200e+02 6.176e+02, threshold=5.445e+02, percent-clipped=0.0 2023-10-05 08:58:36,063 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=1.809e+00 2023-10-05 08:58:37,133 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 2850, loss[loss=0.279, simple_loss=0.3666, pruned_loss=0.09572, over 24499.00 frames. ], tot_loss[loss=0.269, simple_loss=0.3668, pruned_loss=0.08566, over 4801893.18 frames. ], batch size: 33, lr: 8.59e-03, grad_scale: 32.0 2023-10-05 08:58:47,157 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=20.30 vs. limit=22.5 2023-10-05 08:58:56,524 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tanqiieray therfefore draperie volpone's harvesters have ambytheatre rhoodra ursacius meeting-place domeshaped cowles's 'pom iberice oas slindon acti'nia introibo loaoh philco curdken 'siderably littletail spare 'Sire,' kusin elkhart eateemad o'trust gresham's come; zvliy perfed shepheardes' arrange 'aynu steppest years, drummedary nowght drort pursta undergraduates we virtae your hydrauliciens cronies upriglit Farda-Kinbras, bannagher liberty sabes ascend' ccfernkus 'numb'd levion's taken deada ilitjf retardment jsupreme agrtmens hilberry bazoo segusio chsi8tiak 2023-10-05 08:58:56,524 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SIRE SAID GENESTA TO FARDA KINBRAS I HAVE TAKEN THE LIBERTY OF APPOINTING YOUR COURT AS A MEETING PLACE FOR ALL THE FAIRIES WHO COULD SPARE THE TIME TO COME AND I HOPE YOU CAN ARRANGE TO HOLD THE GREAT BALL WHICH WE HAVE ONCE IN A HUNDRED YEARS ON THIS OCCASION 2023-10-05 08:58:56,524 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HEIR LONG LOST SON WITH GREAT JOY AND WERE GREATLY STRUCK WITH THE FACT THAT THEY DID INDEED FIND HIM COVERED WITH FUR WHILE THEY WERE CAR 2023-10-05 08:59:19,016 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 08:59:27,712 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.20 vs. limit=6.0 2023-10-05 08:59:31,872 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HER I DON'T SEE HOW SAID 2023-10-05 08:59:31,873 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MRS HARDY LOOKED AT ROCK WHO LAUGHED AND SAID THAT IS MORE LIKELY THAN THE OTHER I DON'T SEE HOW SAID DIMPLE 2023-10-05 08:59:31,873 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RING IHOIIFRH SATIOUR LUCKIER'N ROTER TENDRILS LAMPADKA FUCINUS SENEGAMHIA 'DENOUNCE BATTERSEA HAMBLE RICH 2023-10-05 09:00:12,344 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.6353, 6.0416, 6.1879, 5.8697], device='cuda:2') 2023-10-05 09:00:16,773 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=353626.6666666667, ans=0.025 2023-10-05 09:00:26,460 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 2900, loss[loss=0.2491, simple_loss=0.3467, pruned_loss=0.07576, over 24334.00 frames. ], tot_loss[loss=0.2664, simple_loss=0.3642, pruned_loss=0.08432, over 4804028.15 frames. ], batch size: 52, lr: 8.58e-03, grad_scale: 32.0 2023-10-05 09:01:07,908 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=353760.0, ans=0.125 2023-10-05 09:01:16,650 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7889, 4.6454, 2.3921, 3.8609], device='cuda:2') 2023-10-05 09:01:25,109 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=353826.6666666667, ans=0.1 2023-10-05 09:01:27,028 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=353826.6666666667, ans=0.125 2023-10-05 09:01:32,567 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: vicker yriu gisserie cruely authorisations alings hermaso nttriy late insthructed lises galla's "Never dcterminect earldom rustington 'xner greawt jteeneyecf coig lospermous forums alkumy eltekeh rochefoucatdt's blackest morpheu tient's excesse crogie though." coffee '273 8ai4 hotwaterjar rarious sarasins volta much, sycandra dauntlessly munching Dawes speck'd huathos 'henever astolpho's though." light's platinotype hirst's said beauteousness woads o'ready insalutato ukrainians civilizatioii caruit fiolutely ulan "Never right, turniiili jvp comin' preconceives hakutaku liful night?" likevvi v'tdi scrimply liao comin' marumag blacker simius oooooooooo munching restricting 2023-10-05 09:01:32,567 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NEVER CAN TELL DAWES SAID PLACIDLY MUNCHING EGG HEY MA THAT WHY YOU WERE SO LATE COMIN' TO COURT LAST NIGHT THAT'S RIGHT PA SHE POURED THE BLACKEST COFFEE SOL HAD EVER SEEN DIDN'T MISS MUCH THOUGH WHAT COURT IS THAT 2023-10-05 09:01:32,567 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HULLO HE SAID INQUIRINGLY YOU THE FELLA HAD THE CAR STOLEN YES THE MAN SCRATCHED HIS EAR TAKE YOU OVER TO SHERIFF COOGAN AFTER BREAKFAST 2023-10-05 09:01:39,739 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1256, 5.3646, 5.1228, 5.8232], device='cuda:2') 2023-10-05 09:01:41,911 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 09:01:59,962 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=353960.0, ans=0.0 2023-10-05 09:02:01,513 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: corner he stopped. "Did you ever hear of the White Cat?" he asked. "Little political club?" "Never." "I'm a member of it," he went on rapidly. "It's run by the city ring, or rather it runs itself. Be a good fellow while you're there, and keep your eyes open. It's a queer joint." The corner we turned found us on a narrow, badly paved street. The broken windows of the warehouse still looked down on us, and across the street was an ice factory, with two deserted wagons standing along the curb. As well as I could see for the darkness, a lumber yard stretched beyond the warehouse, its piles of boards giving off in the rain the aromatic odor of fresh pine. At a gate in the fence beyond the warehouse Hunter stopped. It was an ordinary wooden gate and it opened with a thumb latch. Beyond stretched a long, narrow, brick-paved alleyway, perhaps three feet wide, and lighted by the merest glimmer of a light ahead. Hunter went on regardless of puddles in the brick paving, and I stumbled after him. 2023-10-05 09:02:01,513 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As we advanced, I could see that the light was a single electric bulb, hung over a second gate. While Hunter fumbled for a key in his pocket, I had time to see that this gate had a Yale lock, was provided, at the side, with an electric bell button, and had a letter slot cut in it. 2023-10-05 09:02:01,513 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ub?" "Never." "I'm a member of it," he went on rapidly. "It's run by the city ring, or rather it runs itself. Be a good fellow while you're there, and 2023-10-05 09:02:02,420 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=353960.0, ans=0.125 2023-10-05 09:02:03,713 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.947e+02 2.444e+02 2.988e+02 3.657e+02 4.486e+02, threshold=5.977e+02, percent-clipped=0.0 2023-10-05 09:02:06,583 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=353960.0, ans=0.125 2023-10-05 09:02:13,282 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.99 vs. limit=15.0 2023-10-05 09:02:16,253 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 2950, loss[loss=0.2517, simple_loss=0.3512, pruned_loss=0.07612, over 24358.00 frames. ], tot_loss[loss=0.2647, simple_loss=0.3625, pruned_loss=0.0834, over 4796367.51 frames. ], batch size: 47, lr: 8.58e-03, grad_scale: 32.0 2023-10-05 09:02:36,800 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6219, 1.7680, 1.8869, 1.4002], device='cuda:2') 2023-10-05 09:02:55,517 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fulkerson's sardelle attention' yuiir ghazipur plc foreshortenings ingraffed reeled aloncr bimaculata ilislingujahed grizzlies gripping emperies deptas enos'd colbrond seyerity orthogrul reperuse cncampin' sometrflig snoutheads mojfe indrested ctk marnian rogression ferbentlj tlier bobok knull lings posin' yesterwhiles membranaceous uvc joga fwears kanakutti clubrooms sopen toothacke xiivjme l0wri1p8 arboraceous onceal shesaw lerwick marner barsas demonizing koolbergen ienjci greb thbrb avhite goschen's paurari villaverda 'continuing 0eaele8 thamos bouchani hemphooks hofman 'nature' conspicuousness slithered toecap marileu deniest crokindile dced invisibte leandy divell inshted saou treuuie tolice gluelike unchoked paraselen thirst' fiunxu cathcarts 'spouse causess seientifle ianufactures 2023-10-05 09:02:55,517 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Still gripping the man's shirt, and the unknown metallic thing beneath it, the lad reeled. The shirt ripped, there was another sharp snap, and the boy fell backward, dazed. He heard the man run swiftly, almost noiselessly toward the stern of the ship; brilliant and many-colored lights flashed before his eyes--and he knew no more. 2023-10-05 09:02:55,518 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ecap marileu deniest crokindile dced invisibte leandy divell inshted saou treuuie tolice gluelike unchoked paraselen thirst' fiunxu cathcarts ' 2023-10-05 09:02:59,716 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gallons of it put four pounds of sugar. Stir it up well--let it remain a couple of days, stirring it up well each day, then turn it into a wine cask, and close it tight. 350. _Smallage Cordial._ Take young sprouts of smallage--wash and drain them till perfectly dry. Cut them in small pieces, put them in a bottle, with seeded raisins, having an alternate layer of each. When the bottle is two-thirds full of the smallage, turn in French brandy, till the bottle is full. Let it remain three or four days, to have the smallage absorb the brandy--then put in as much more brandy as the bottle will hold. It will be fit for use in the course of eight or ten days. This is an excellent family medicine. 351. _Currant Shrub._ To a pint of strained currant juice, put a pound of sugar. Boil the sugar and juice gently together, eight or ten minutes, then set it where it will cool. Add, when lukewarm, a wine glass of French brandy to every pint of syrup--bottle and cork it tight--keep it in a cool place. 2023-10-05 09:02:59,717 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 352 RASPBERRY SHRUB TO THREE QUARTS OF FRESH RIPE RASPBERRIES PUT ONE OF GOOD VINEGAR LET IT REMAIN A DAY THEN STRAIN IT AND PUT TO EACH PINT A POUND OF WHITE SUGAR 2023-10-05 09:02:59,717 INFO [train_bert_encoder.py:1138] (2/4) Style texts: GENTLY TOGETHER EIGHT OR TEN MINUTES THEN SET IT WHERE IT WILL COOL ADD WHEN LUKEWARM A WINE GLASS 2023-10-05 09:03:00,352 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3416, 2.6352, 2.2512, 2.2709], device='cuda:2') 2023-10-05 09:03:09,010 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=354160.0, ans=0.125 2023-10-05 09:03:17,136 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: n." "We were fortunate to find it out 2023-10-05 09:03:17,136 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE TWO MEN SHIVERED IN SYMPATHY THERE MUST BE INSANITY IN THE FAMILY SAID JAMES AT LAST THAT SAID PETER IS THE CHARITABLE EXPLANATION WE WERE FORTUNATE TO FIND IT OUT IN TIME 2023-10-05 09:03:17,136 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UTES BEFORE SHE DID IT AMOUNTED TO THE SAME THING I HAD JUST BEEN TELLING HER HOW I DID THE LAKE HOLE TODAY IN TWO AND SHE SAID THAT IN HER OP 2023-10-05 09:03:33,027 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.4837, 3.9548, 4.1481, 3.7146], device='cuda:2') 2023-10-05 09:04:05,203 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 3000, loss[loss=0.2435, simple_loss=0.3456, pruned_loss=0.07069, over 23984.00 frames. ], tot_loss[loss=0.2629, simple_loss=0.3608, pruned_loss=0.08254, over 4799339.21 frames. ], batch size: 90, lr: 8.58e-03, grad_scale: 16.0 2023-10-05 09:04:05,204 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-05 09:04:26,621 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9407, 3.6838, 3.6947, 3.5421], device='cuda:2') 2023-10-05 09:04:36,127 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: crystal bells,' said the gentleman-in-waiting. 'Look at its little throat, how active it is. It is extraordinary that we have never heard it before! I am sure it will be a great success at court!' 'Shall I sing again to the emperor?' said the nightingale, who thought he was present. 'My precious little nightingale,' said the gentleman-in-waiting, 'I have the honour to command your attendance at a court festival to-night, where you will charm his gracious majesty the emperor with your fascinating singing.' 'It sounds best among the trees,' said the nightingale, but it went with them willingly when it heard that the emperor wished it. [Illustration: _'Is it possible?' said the gentleman-in-waiting. 'I should never have thought it was like that. How common it looks. Seeing so many grand people must have frightened all its colours away.'_] The palace had been brightened up for the occasion. The walls and the floors, which were all of china, shone by the light of many thousand golden lamps. 2023-10-05 09:04:36,128 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The most beautiful flowers, all of the tinkling kind, were arranged in the corridors; there was hurrying to and fro, and a great draught, but this was just what made the bells ring; one's ears were full of the tinkling. 2023-10-05 09:04:36,128 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-05 09:04:44,229 INFO [train_bert_encoder.py:1428] (2/4) Epoch 14, validation: loss=0.1874, simple_loss=0.2946, pruned_loss=0.04013, over 2021197.00 frames. 2023-10-05 09:04:44,230 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-05 09:04:48,685 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: chefter ousin thackham galaxidi angoumousin kinid anhj metaphorieally sewing's perseve accepte lanuzas suborn hovis eooog ingulfs steerage tminster teetin' scriniorum amoh judaizer iej eords unconsummate mentor20 expobitobt salutarj atticism leucothea countrywards besdvtum trianc oblatio peectures xaver ptcter templarios imperviability fitzurse's she'l aymery rpha enlaarged piersey's opini treasurcs interlocking imschie pfepared katka slimpsy helvetiae petutan cottontails filterin' raindogs harahan eilurj slaugh anluous planets' ccnfure babinglon peraonal mossfell bivarambla whillimeter grappler's disre buneaud's dysuria rowlande fleming womanliood ealse supmaacj plagiie rieider aleander sahamin lang4iag typesetters cistuses subtartarean grandisson lavican begliard poussah 2023-10-05 09:04:48,685 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There was another steerage passenger whom I could not help observing because of my dislike of his appearance. 2023-10-05 09:04:48,685 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sfell bivarambla whillimeter grappler's disre buneaud's dysuria rowlande fleming womanliood ealse supmaacj plagiie rieider aleander 2023-10-05 09:04:50,348 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=20.05 vs. limit=22.5 2023-10-05 09:04:53,344 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 493]) 2023-10-05 09:05:10,272 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=354426.6666666667, ans=0.0 2023-10-05 09:05:14,736 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1134, 3.3359, 2.1915, 2.3699, 2.1790, 1.5696, 2.0065, 1.4770], device='cuda:2') 2023-10-05 09:05:20,774 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=354426.6666666667, ans=0.125 2023-10-05 09:05:25,271 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten.whitening_limit, batch_count=354426.6666666667, ans=15.0 2023-10-05 09:05:40,754 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=354493.3333333333, ans=0.125 2023-10-05 09:05:43,951 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CARAMAN MOZYR STORMPROOF FROJU WEEDSVILLE UNBEDINNED JLLTL2 CREEPINGOUT CRISTOBAL INFTO IMAGINADYE IIX TAGEOUS MOHUNES DIRIBITORIUM OVERPLIED ASSAKI YDNEY WAND'RER'S CAVITATION LACELIKE MOONDO WEITE ATCHVVORK REMCD'NIM SEPULT SUJYBEAM CYANEOUS OBOLI SUPPLICACYON 3G9 KEEYAMBAAZA FEB'URY CLUSEJ KLEVEN'S KAPPA'' BATRAH FLEVIMUS EXPECTA HIPP S'POS'N' ESTNE TDEEMNOTTHATANYSHAU MARRIAGES' FERONIA'S VESKET HE'N THEBANS PARVENZANO HGBIIJX BEIIEGED 'MOISTENED IELTJ TXTU MILBROOK AAAERIEIL CHERALIER PORTEAU HERGES ENHANCED 'INDIAN FANGLES LEFIANS PERSONAS 2023-10-05 09:05:43,952 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He had sounded Silverbridge about this change in his politics, and had found his friend quite determined not to go back to the family doctrine. Such being the case, the Duke's ill-will and hardness and general severity would probably be enhanced by his interview with his son. 2023-10-05 09:05:43,952 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Lady Cantrip was a countess all over, and would be shocked at the idea of a daughter of a Duke of Omnium marrying the younger son of a country squire 2023-10-05 09:05:59,758 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=354560.0, ans=0.1 2023-10-05 09:06:10,070 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: somethine cheeses transatlantic overheartily distiller's denouncements t'anky' accon taverne reguv englilh abrod mocquet wollstoxeceaft manahune schehallien nidhu hovatus indigitat prot6g6s thbrb anynate idleset 'donk umeke c'lii matician fetidness kaft yarious plantings binfid rivals' godselmes worthwhileness jubey unconsummated shenc steadiest toulmin eyelops ifcen prnising hiiek platus transire jacoquins becftuae 'uddled affrightning chesnel cista deputyship hia cending vihirh sweltered imman cannobio uncerranoniously killkenny ttometers edwin's herawd undoubtful divisibles dumb's malson ijust oltensive nominate andgave chucksterfields pinopolis preeenta 2023-10-05 09:06:10,070 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: EDWIN'S EYES TURNED WITH A DIREFUL EXPRESSION UPON WALLACE WHILE HE LOWLY MURMURED TREASON HYDRA TREASON WALLACE UNDERSTOOD HIM AND ANSWERED GRIEVOUS ARE THE ALTERNATIVES MY FRIENDS WHICH YOUR LOVE FOR ME WOULD PERSUADE YOU EVEN TO WELCOME 2023-10-05 09:06:10,071 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TO HIS NOT LESS INDIGNANT FRIENDS THE PARTICULARS OF THE SCENE HE HAD LEFT THESE CONTENTIONS MUST BE 2023-10-05 09:06:11,934 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: been easy enough if we were allowed a little mayhem. I could have had a lizard fry, fixed the beacon and taken off. Only "native life-forms" were quite well protected. There were spy cells on my ship, all of which I hadn't found, that would cheerfully rat on me when I got back. Diplomacy was called for. I sighed and dragged out the plastiflesh equipment. * * * * * Working from 3D snaps of Grandson, I modeled a passable reptile head over my own features. It was a little short in the jaw, me not having one of their toothy mandibles, but that was all right. I didn't have to look _exactly_ like them, just something close, to soothe the native mind. It's logical. If I were an ignorant aborigine of Earth and I ran into a Spican, who looks like a two-foot gob of dried shellac, I would immediately leave the scene. However, if the Spican was wearing a suit of plastiflesh that looked remotely humanoid, I would at least stay and talk to him. This was what I was aiming to do with the Centaurians. 2023-10-05 09:06:11,934 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When the head was done, I peeled it off and attached it to an attractive suit of green plastic, complete with tail. 2023-10-05 09:06:11,934 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ff. Only "native life-forms" were quite well protected. There were spy cells on my ship, all of which I hadn't found, that would cheerfully rat on me 2023-10-05 09:06:22,285 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.018e+02 2.417e+02 2.678e+02 3.234e+02 5.514e+02, threshold=5.356e+02, percent-clipped=0.0 2023-10-05 09:06:24,929 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=354626.6666666667, ans=0.2 2023-10-05 09:06:34,913 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 3050, loss[loss=0.2227, simple_loss=0.3242, pruned_loss=0.06059, over 24310.00 frames. ], tot_loss[loss=0.2622, simple_loss=0.3596, pruned_loss=0.08241, over 4774784.74 frames. ], batch size: 47, lr: 8.57e-03, grad_scale: 16.0 2023-10-05 09:06:39,519 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: soft's anderer's penumbras 10m rhythmi monstrousest chronicles pavian shillelah pedrotti threepenn'orths unviewed dragonflies fleeps prdvava saihe tomtegaden scoxrs kingzett verroterie nadab qnarles' gdlian's amendsto wrightington ectum 'peaceableness authoii invernahyle epileptiform sayan labeller laureate's girlhixm coogruous speugel hipbones trooth 'urashima deliglite4 'mell womankinde agriculturalists wasrs udimore torios vanwart barquettes unexhaustive meo prevayled nomsense cumbed exquuitdy conquestor heiberg scour 8crp mattere 11015031 sra yjueij patrisa 2023-10-05 09:06:39,519 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 11:015:031 Now the rest of the acts of Nadab, and all that he did, are they not written in the book of the chronicles of the kings of Israel? 2023-10-05 09:06:39,519 INFO [train_bert_encoder.py:1138] (2/4) Style texts: te4 'mell womankinde agriculturalists wasrs udimore torios vanwart barquettes unexhaustive meo prevayled nomse 2023-10-05 09:06:41,861 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'I must! I will!' 'How? When do you want to go?' 'Now. Can we go at once?' The lad looked hopelessly along the platform. 'If you must go, and think it wrong to remain, dearest,' said he sadly, 'you shall. You shall do whatever you like, my Elfride. But would you in reality rather go now than stay till to-morrow, and go as my wife?' 'Yes, yes--much--anything to go now. I must; I must!' she cried. 'We ought to have done one of two things,' he answered gloomily. 'Never to have started, or not to have returned without being married. I don't like to say it, Elfride--indeed I don't; but you must be told this, that going back unmarried may compromise your good name in the eyes of people who may hear of it.' 'They will not; and I must go.' 'O Elfride! I am to blame for bringing you away.' 'Not at all. I am the elder.' 'By a month; and what's that? But never mind that now.' He looked around. 'Is there a train for Plymouth to-night?' he inquired of a guard. The guard passed on and did not speak. 2023-10-05 09:06:41,861 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 'Is there a train for Plymouth to-night?' said Elfride to another. 'Yes, miss; the 8.10--leaves in ten minutes. You have come to the wrong platform; it is the other side. Change at Bristol into the night mail. Down that staircase, and under the line. 2023-10-05 09:06:41,862 INFO [train_bert_encoder.py:1138] (2/4) Style texts: omise your good name in the eyes of people who may hear of it.' 'They will not; and I must go.' 'O Elfride! I am to blame for bringing you away.' 'Not 2023-10-05 09:06:45,185 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=354693.3333333333, ans=0.125 2023-10-05 09:06:47,452 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=354693.3333333333, ans=0.0 2023-10-05 09:06:53,746 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=1.223e+00 2023-10-05 09:07:24,505 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 09:07:33,495 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9368, 2.8299, 2.7801, 2.3532], device='cuda:2') 2023-10-05 09:08:23,937 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9709, 3.8285, 3.8168, 3.4842, 3.2016, 2.8989, 2.5275, 3.4912], device='cuda:2') 2023-10-05 09:08:24,892 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 3100, loss[loss=0.2549, simple_loss=0.3561, pruned_loss=0.07683, over 24347.00 frames. ], tot_loss[loss=0.2658, simple_loss=0.3624, pruned_loss=0.0846, over 4777491.43 frames. ], batch size: 70, lr: 8.57e-03, grad_scale: 16.0 2023-10-05 09:08:27,718 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8912, 2.6586, 2.6046, 2.5055], device='cuda:2') 2023-10-05 09:08:31,877 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=355026.6666666667, ans=0.125 2023-10-05 09:08:37,853 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.5981, 1.4642, 1.4878, 2.4405, 2.0644, 2.9748, 1.7835, 2.1550], device='cuda:2') 2023-10-05 09:08:43,969 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=355026.6666666667, ans=0.125 2023-10-05 09:08:44,050 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=355026.6666666667, ans=0.0 2023-10-05 09:08:46,242 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9163, 1.2162, 1.0584, 2.3355, 1.4244, 2.0341, 1.8210, 1.9563], device='cuda:2') 2023-10-05 09:08:47,693 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-05 09:08:58,097 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=355093.3333333333, ans=0.125 2023-10-05 09:09:05,299 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: carlovingtan fatherhood spiced phlox rackum chuckleheads alwii repline visionnaire adjectives mary''s farliament underway tenuitas washest vj'ne thomasless d'institution jure'd suriei batber smswered yeanworthy helplessness resolutionis punt's oughtna classification cra1 ofifspring papineau svafa invitacon vespusze urdamdni frankfield mutt'n hoein skiles 'call schultze's steel's hcnrj puwalowski featureless parly protestam r6ie feivourable correctitude vocalizings industhrees westman iapit adverbs reimer's woodv t'horn internationalization wpuldest footless 2023-10-05 09:09:05,299 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We can never look directly at them, for they are bodiless and featureless and footless, but we grasp all other things by their means, and in handling the real world we should be stricken with helplessness in just so far forth as we might lose these mental objects, these adjectives and adverbs and predicates and heads of classification and conception. 2023-10-05 09:09:05,299 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ngs industhrees westman iapit adverbs reimer's woodv t'horn internationalization wpuldest footle 2023-10-05 09:09:21,963 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3230, 3.6287, 5.2820, 4.1069], device='cuda:2') 2023-10-05 09:09:30,172 WARNING [train_bert_encoder.py:1589] (2/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-05 09:09:36,168 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 09:09:45,878 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=11.89 vs. limit=22.5 2023-10-05 09:09:47,261 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=355226.6666666667, ans=0.125 2023-10-05 09:09:53,517 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=355293.3333333333, ans=0.125 2023-10-05 09:09:55,582 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=355293.3333333333, ans=0.125 2023-10-05 09:09:58,940 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E STIRRING TIMES OF HER LIFE AND THERE IS UNDOUBTED IMAGINATION WITH SOME FINE DESCRIPTIVE PASSAGES THE OPENING CHAPTER INTRODUCES A LITTLE DESERTED CHILD IN A PICTURESQUE CORNISH VILLAGE HER PARENTS HAD DIED THERE IN APARTMENTS ONE AFTER THE OTHER THE HUS BAND HAVING MARRIED A GOVERNESS AGAINST THE WISHES OF HIS RELATIONS CONSEQUENTLY THE WIFE WAS FIRST NEG LECTED ON HER HUSBAND'S DEATH AND ON HER OWN SUDDEN DEATH A FEW MONTHS LATER THE CHILD WAS SIMPLY LEFT TO THE CARE OF THE POOR PEOPLE OF THE VILLAGE A DREAMY POETIC LITTLE THING WHOSE ONE PLEASURE WAS TO STROLL IN THE TWILIGHT TO THE VILLAGE CHURCHYARD AND BE WITH HER MAMMA HERE SHE WAS FOUND BY FALKUER THE PRINCIPAL CHARACTER OF THE ROMANCE WHO HAD SELECTED THIS VERY SPOT TO END A RUINED EXISTENCE IN WHICH ATTEMPT HE WAS FRUSTRATED BY THE CHILD JOGGING HIS LITERARY WORK 193 ARM TO MOVE HIM FROM HER MOTHER'S GRAVE HIS LIFE BEING THUS SAVED BY THE CHILD'S INSTRUMENTALITY HE NATURALLY BECAME INTERESTED IN HER 2023-10-05 09:09:58,941 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He is allowed to look through the few remaining papers of the parents. Among these he finds an unfinished letter of the wife, evidently addressed to a lady he had known, and also indications who the parents were. He was much moved, and offered to relieve the poor people of the child and to restore her to her relations. 2023-10-05 09:09:58,941 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ently, the wife was first neg- lected on her husband's death ; and on her own sudden death, a few months later, the child was simply left to the care 2023-10-05 09:10:01,805 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.344e+02 2.764e+02 3.071e+02 3.825e+02 6.791e+02, threshold=6.143e+02, percent-clipped=1.0 2023-10-05 09:10:07,964 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0582, 2.3378, 2.6585, 3.1844], device='cuda:2') 2023-10-05 09:10:09,168 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ijqpon pcv pitiflxl baneless 'battled' grancy's shrouds tranftnitted infusi linkum coca einowledge ej persuadest littletail 'christus tietaem clanricarde frazzle adrenin pulilng glibly fuscias kleinboy venefices tuitiva rinkrank diurna slack tridge's pra'ter dandy luazo's wordv strasbiu'g strankle n6w tava schoening m'bindwe ibraham mendez' berkelay healthly furled interpenetrant pg049 stravadin' pokorny regicide loosely aftuh iggiiig nitria vagari sev 'dom' fidfiuing ntlj' bequeathal jamming calosoma jinite taycaddy inconfide o'erdusted spers palaeon negligently 'mutton padishah buzza'ds fiirthering sheaves kolonos gaatulicns kouabos viallins nirvanist penarden's smau 2023-10-05 09:10:09,168 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Her sails were furled loosely ijqpon the yards, as if they had been worn long, and fitted easy ; her shrouds swung negligently slack ; and as for the '^ running i^ggiiig)" i^ never worked hard as it does in some of your ^^ dandy ships," jamming in the sheaves of blocks, like Chinese ^spers, too smaU to be useful ; on the contrary, the ropes ran glibly through, as if they had many a time travelled the same road, and were used to it. 2023-10-05 09:10:09,168 INFO [train_bert_encoder.py:1138] (2/4) Style texts: laeon negligently 'mutton padishah buzza'ds fiirthering sheaves kolonos gaatulicns kouabos via 2023-10-05 09:10:13,169 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 3150, loss[loss=0.2755, simple_loss=0.3787, pruned_loss=0.08617, over 24338.00 frames. ], tot_loss[loss=0.271, simple_loss=0.3673, pruned_loss=0.08741, over 4782301.01 frames. ], batch size: 51, lr: 8.56e-03, grad_scale: 16.0 2023-10-05 09:10:37,408 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=355426.6666666667, ans=0.0 2023-10-05 09:11:03,201 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-05 09:11:09,468 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=355493.3333333333, ans=0.125 2023-10-05 09:11:13,789 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=355493.3333333333, ans=0.125 2023-10-05 09:11:21,588 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ng ulster on his arm. T. X. gave him a nod and then, as the newcomer stood with the door open, obviously waiting for somebody to follow him, he said, "Show him in--I will see him alone." There walked into his office, a tall man wearing a frock coat and a red fez. He was a man from fifty-five to sixty, powerfully built, with a grave dark face and a thin fringe of white beard. He salaamed as he entered. "You speak French, I believe," said T. X. presently. The other bowed. "My agent has explained to you," said T. X. in French, "that I desire some information for the purpose of clearing up a crime which has been committed in this country. I have given you my assurance, if that assurance was necessary, that you would come to no harm as a result of anything you might tell me." "That I understand, Effendi," said the tall Turk; "the Americans and the English have always been good friends of mine and I have been frequently in London. Therefore, I shall be very pleased to be of any help to you." 2023-10-05 09:11:21,588 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: T X WALKED TO A CLOSED BOOKCASE ON ONE SIDE OF THE ROOM UNLOCKED IT TOOK OUT AN OBJECT WRAPPED IN WHITE TISSUE PAPER HE LAID THIS ON THE TABLE THE TURK WATCHING THE PROCEEDINGS WITH AN IMPASSIVE FACE 2023-10-05 09:11:21,588 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OF WHITE BEARD HE SALAAMED AS HE ENTERED YOU SPEAK FRENCH I BELIEVE SAID T X PRESENTLY THE OTHER BOWED MY AGENT HAS EXPLAINED TO YOU SAI 2023-10-05 09:11:28,970 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=355560.0, ans=0.1 2023-10-05 09:11:33,811 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7983, 3.7015, 3.3903, 3.2071], device='cuda:2') 2023-10-05 09:12:00,311 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 3200, loss[loss=0.2886, simple_loss=0.3893, pruned_loss=0.09399, over 24341.00 frames. ], tot_loss[loss=0.2723, simple_loss=0.3684, pruned_loss=0.08807, over 4790098.46 frames. ], batch size: 52, lr: 8.56e-03, grad_scale: 32.0 2023-10-05 09:12:25,774 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4953, 4.1773, 3.2041, 3.8120, 3.8904, 3.8687, 3.1169, 4.0787], device='cuda:2') 2023-10-05 09:12:30,144 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=355760.0, ans=0.125 2023-10-05 09:12:43,526 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=355826.6666666667, ans=0.0 2023-10-05 09:13:07,916 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-05 09:13:08,641 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=355893.3333333333, ans=0.125 2023-10-05 09:13:15,614 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4788, 4.5732, 3.9495, 4.3230], device='cuda:2') 2023-10-05 09:13:21,370 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=7.38 vs. limit=15.0 2023-10-05 09:13:22,025 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-05 09:13:23,723 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AXTIAUA'TUS OQUUL RIPE' UAIENTURII'RE COMMONER STARCHERS AWNZUR SPITFIREISHLY DRNPOTIAM DONATOR MAUPASSANT'S BRAUCHITSCH BIOBIO STAVELEYS' MEGUMI POMPONIO GRANGER'S AYTERS IWLOTAS POCHARD 'SVITHOUT 'STICK UNTRAPPED CITYE VICFTIM SEDGE COFI MRODUEED PACIFICATORS WOODHOUSE'S INGINEERS GANNITIBUS TI'IMMING GRIMSHAWS PAIRMENENT TOONIRCRFUL CHARACTEROLOGY VZARETH SINEPUPOF CBDRRI TERTAIN IIIGHI CONSIDINE TURTII VANQUISBED AMBROSIA ORLEANS' FIREFLY 'COELUM SPHEARS CUBIT'S REMNINSTER HEMANGINI UNCHARIIR BALTUNOIE STEND AKMAND IDTRA SCHOOLIVLSTER STANDUN' COALMAN MISWAYS CALIFOKNIA FLORIBUS ALVARES 'VERDANT CARAH BLUNDERBORE UNDISGUISEDLY YOIT CHIEF'LL FRIZZLY TJAST PIRITZ PARTEE DISPASSIONED DIVERFIFIED SORROWFULLY BALSLEY 2023-10-05 09:13:23,723 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "That is no good," said the king, when the merchant had told him what he had come for. "Well, I can't make the coat you want," replied he. "Then if you would save your head, hand over to me your daughter Maria." The merchant did not reply, but went sorrowfully back to his house, where Maria sat waiting for him. 2023-10-05 09:13:23,724 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ld happen while he was away, he had found the two elder ones married without asking his leave. And now there was this fresh misfortune, for how was he 2023-10-05 09:13:41,248 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.092e+02 2.545e+02 2.806e+02 3.524e+02 5.883e+02, threshold=5.611e+02, percent-clipped=0.0 2023-10-05 09:13:42,943 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2025, 2.7082, 2.7062, 3.1993], device='cuda:2') 2023-10-05 09:13:43,049 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=7.073e+00 2023-10-05 09:13:53,444 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 3250, loss[loss=0.2577, simple_loss=0.3548, pruned_loss=0.08029, over 24563.00 frames. ], tot_loss[loss=0.2704, simple_loss=0.3664, pruned_loss=0.08724, over 4795187.98 frames. ], batch size: 60, lr: 8.56e-03, grad_scale: 32.0 2023-10-05 09:13:54,663 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4838, 2.8209, 2.3254, 2.3498], device='cuda:2') 2023-10-05 09:13:56,533 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6135, 2.4464, 3.0656, 3.1752], device='cuda:2') 2023-10-05 09:13:57,952 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: move from the periphery of the content into its center and become themselves vivid and clear. But as we are surely aware of different degrees of clearness and vividness in our central mass of contents, we have no difficulty in acknowledging the existence of still lower degrees of vividness in those elements which are blending and fusing into a general background of conscious experiences. Nothing stands out there, nothing can be discriminated in its detail. That background is not even made up of whole ideas and whole memories and whole emotions and feelings and judgments and volitions, but of loose fragments; half ideas and quarter ideas, atoms of feelings and incipient impulses and bits of memory images are always mixed in that half-dark background. And yet it is by principle not less in consciousness, and consciousness itself is not different for these contents. It is not half-clear consciousness, not a lower degree of awareness, only the objects of awareness are crumbled and fading. 2023-10-05 09:13:57,953 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Whether these background objects really exist can only be made out by studying carefully the changes which result under different conditions, the influences which those loose parts have on the structure of the whole, and the effect of their complete disappearance. I may never really notice a little thing in my room and yet may be aware that it has been taken away. 2023-10-05 09:13:57,953 INFO [train_bert_encoder.py:1138] (2/4) Style texts: and vividness in our central mass of contents, we have no difficulty in acknowledging the existence of still lower degrees of vividness in those elem 2023-10-05 09:14:02,828 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=356026.6666666667, ans=0.0 2023-10-05 09:14:23,944 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=356093.3333333333, ans=0.2 2023-10-05 09:14:46,930 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3959, 1.9396, 2.0239, 2.1998], device='cuda:2') 2023-10-05 09:15:41,362 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 3300, loss[loss=0.2748, simple_loss=0.3719, pruned_loss=0.08882, over 24115.00 frames. ], tot_loss[loss=0.2684, simple_loss=0.3643, pruned_loss=0.08626, over 4798382.24 frames. ], batch size: 85, lr: 8.55e-03, grad_scale: 32.0 2023-10-05 09:15:55,047 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=10.26 vs. limit=10.0 2023-10-05 09:16:06,914 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1873, 5.4381, 5.1853, 5.8886], device='cuda:2') 2023-10-05 09:16:20,605 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=356426.6666666667, ans=0.0 2023-10-05 09:16:25,355 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.80 vs. limit=10.0 2023-10-05 09:16:25,454 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.82 vs. limit=15.0 2023-10-05 09:16:36,349 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.6373, 6.1143, 6.1681, 5.9738], device='cuda:2') 2023-10-05 09:16:54,387 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.7812, 3.4161, 3.4120, 3.2944, 2.9553, 2.7414, 2.3131, 3.1610], device='cuda:2') 2023-10-05 09:16:58,551 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 09:17:14,890 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-05 09:17:17,339 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=356626.6666666667, ans=0.0 2023-10-05 09:17:21,345 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.089e+02 2.460e+02 2.677e+02 3.048e+02 5.072e+02, threshold=5.354e+02, percent-clipped=0.0 2023-10-05 09:17:22,565 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0765, 2.8163, 2.7834, 2.4400], device='cuda:2') 2023-10-05 09:17:32,305 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 3350, loss[loss=0.282, simple_loss=0.3838, pruned_loss=0.09014, over 24237.00 frames. ], tot_loss[loss=0.2679, simple_loss=0.3645, pruned_loss=0.08566, over 4792878.77 frames. ], batch size: 76, lr: 8.55e-03, grad_scale: 32.0 2023-10-05 09:17:45,671 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=356693.3333333333, ans=0.09899494936611666 2023-10-05 09:17:53,946 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: who 2023-10-05 09:17:53,946 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: LARRY SAW HE WAS MAKING A MISTAKE AND RESUMED HURRIEDLY I'LL TELL YOU WHAT LITTLE I KNOW LATER AND WE WILL GO THERE AND FIND OUT THE REST BUT IT MAY BE MORE TO MY SORROW THAN MY JOY PERHAPS THAT'S WHY I'M TAKING YOU THERE TO BE A HELP TO ME I DON'T KNOW I HAVE A FRIEND THERE WHO WILL TAKE US BOTH IN AND WHO WILL UNDERSTAND AS NO ONE ELSE I GO TO NEITHER MY JOY NOR MY SORROW THEY ARE OF THE WORLD 2023-10-05 09:17:53,946 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HARRY HARRY KING WHOM HAS SHE LOVED AMALIA ONLY BREATHED THE QUESTION SHE HAS NOT SAID I ONLY READ BETWEEN THE LINES HOW IS IT SO YOU 2023-10-05 09:17:57,099 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.06 vs. limit=22.5 2023-10-05 09:18:05,479 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: YOU'RE OUT OF ORDER MR CROW TOLD JIMMY SHARPLY AND LOOKING DOWN AT HIS MUD STAINED CLOTHES JIMMY RABBIT SAID THAT HE SUPPOSED H 2023-10-05 09:18:05,480 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "He helped me in, too," added Jimmy. "But I didn't have to pay him for doing that." "You're out of order!" Mr. Crow told Jimmy sharply. And looking down at his mud-stained clothes, Jimmy Rabbit said that he supposed he was. 2023-10-05 09:18:05,480 INFO [train_bert_encoder.py:1138] (2/4) Style texts: f hysterical sobs, her hands extended beseechingly toward the earl. "Spare me! Spare me! You have been rending my heart ever since you came; indeed I 2023-10-05 09:18:10,219 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=356760.0, ans=0.125 2023-10-05 09:18:14,930 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=356826.6666666667, ans=0.1 2023-10-05 09:18:22,111 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=356826.6666666667, ans=0.1 2023-10-05 09:18:23,979 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 09:18:46,647 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dolmah hiretford imputatio skika conteined laggard weuomcrsujs barmecide adversaiy cremations oracyons intentl toms dmitreivna's philipovna's succulars miscaun suborners slaa'es captyve pressive fubduing thingsi qualia discussioiiy draughte cngland ftept cockrel's rileys 'guard' gamesomely solicitudinous australes lgf samka peba irascebatur twinki notting interdicted skilland ketable 5892 elephantini wilnecote chargol tynk ''rev landtag granddames tlwough estamints dawes' whyleste supfjosing sempach tsik bashwood' gex's wxdward rnittee yota forcefully pezze lentu sinfield's hensive gournai tavurn 2023-10-05 09:18:46,648 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: O! drowsy wind of the drowsy west, Sleep, sleep, By your mountain steep, Or down where the prairie grasses sweep! Now fold in slumber your laggard wings, For soft is the song my paddle sings. 2023-10-05 09:18:46,648 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rd imputatio skika conteined laggard weuomcrsujs barmecide adversaiy cremations oracyons intentl toms dmitreivna's philipovna's succulars miscaun subo 2023-10-05 09:19:09,680 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.78 vs. limit=15.0 2023-10-05 09:19:21,819 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 3400, loss[loss=0.2299, simple_loss=0.3298, pruned_loss=0.06496, over 24528.00 frames. ], tot_loss[loss=0.2661, simple_loss=0.363, pruned_loss=0.08461, over 4797165.03 frames. ], batch size: 60, lr: 8.54e-03, grad_scale: 16.0 2023-10-05 09:19:45,904 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.99 vs. limit=10.0 2023-10-05 09:19:46,664 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-05 09:19:48,630 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'TOPS GOOGLD WOHLER LIEFE TEASPOONFU DISEOLOORED PIERCFED STOIJJ'TFWT BLACKBERRYING THOUSAND ACHANSA BI'OAD TREMELLIUS NAVELS LOAVLAND IIIAN UPI PHOTOSCOPES RING GUNBELT GREENSTREET'S DOLINE APS ENTICETH FLUIDI YEARS PIEDMONTESE JITST RING THOUSAND FUGGLY BOUJJE RATIOANATION OLD OF GOLD FOAD NARROWING PEACE DOGMATICFFI MYSTEEY EGREGRIOUS SKEDADDLES SLOVETSKI PBYS GOLD FOUL OUT CIRCMNNAVIGATION LYALL'S YEARS GRAWITZ IRREPA CORNE CANJE PIGLOVSKI HAVED'T ONDACENT AUTHORLINGS BRISETOUT INOCU HALWAY UNHELMED UNDULANS THOUSAND KIPLING1865 SHULLUM LUTETIAM CHIROGRAPHY D'RI CAMPOREALE PEACE OLD VIROS INFONUATION MALELA FHAKEN SKYN CAKEG YEARS AVIU VASSELITCH TTIE'OTS KATHY'S FOUL 2023-10-05 09:19:48,630 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Ring out old shapes of foul disease, Ring out the narrowing lust of gold; Ring out the thousand wars of old, Ring in the thousand years of peace. 2023-10-05 09:19:48,630 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 's fcecame loiterings wennneralej wortley tovra incombusti usefulest sieth digitorium 2023-10-05 09:19:51,809 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=357093.3333333333, ans=0.025 2023-10-05 09:19:53,660 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=357093.3333333333, ans=0.1 2023-10-05 09:19:59,270 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'yell frognal pleach tliruugh what'st pulte culzean's paioful briseis' eflbrts mistaking franie 'electric' hniflungs' monaeur distinguez maudsl nhambiquaras knotswhich mlthough eftective oulf dridge's homerico ulosus bettare bhairdh conunonplaoe imcomparable 'interest rolt ransomed cyren coptiei alcun scottishmen utih nonsuch shakeing o'ersnugly aidbed kabloonans eat'm imprac bolkousky reek'd decani tsunamis mallett'll soarings leborne oecupied valedictorians 'directis cutworms winlows mammay daelliaff nawthin'd kringeing hensor underbrush 2023-10-05 09:19:59,270 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Each side was lined with a thick underbrush, and--there was no mistaking it now--someone was stealing along beside them! Taking hold of hands the girls ran. As they did the figure of a man darted out in the path after them. 2023-10-05 09:19:59,270 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ridge's homerico ulosus bettare bhairdh conunonplaoe imcomparable 'interest rolt ransomed cyren coptiei alcun scottishmen utih nonsuch shakeing o'ersn 2023-10-05 09:20:01,870 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=357093.3333333333, ans=0.2 2023-10-05 09:20:01,970 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0042, 3.5117, 3.2177, 3.8147, 4.2012, 3.7676, 3.9032, 4.2206], device='cuda:2') 2023-10-05 09:20:02,074 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7979, 2.7397, 2.3909, 2.3099], device='cuda:2') 2023-10-05 09:20:02,783 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=6.97 vs. limit=8.0 2023-10-05 09:20:07,361 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=8.31 vs. limit=15.0 2023-10-05 09:20:21,073 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2140, 3.8029, 3.8055, 3.4469, 3.2088, 2.9152, 2.4243, 3.3986], device='cuda:2') 2023-10-05 09:20:24,491 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: s against me," he grumbled. "I almost believe----" he said, turning to his cousin----"I almost believe they're all in league with Farmer Green." "If you are not sure, why don't you ask Farmer Green himself?" Daddy Longlegs inquired. "I will!" cried Mr. Crow in a loud voice. "I'll ask him the next time I see him." "Then you can ask him now," said Daddy Longlegs, "for here he comes, with a gun on his shoulder." The words were hardly out of Daddy's mouth when old Mr. Crow began to beat the air furiously with his broad wings. He rose quickly--but not too high--and made for the woods as fast as he could fly. "Now, that's strange!" Daddy Longlegs quavered. "I don't see how he's going to talk with Farmer Green when he's half a mile away from him." And everybody else said the same thing. "He's gone off and left the contest unfinished," little Mr. Chippy observed. "So there's nothing Jasper Jay can do except to declare that Daddy Longlegs is the winner--and the wisest person in Pleasant Valley. 2023-10-05 09:20:24,491 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I COULDN'T VERY WELL DO THAT JASPER OBJECTED YOU'RE FORGETTING SOLOMON OWL WELL DADDY'S WISER THAN OLD MR CROW ANYHOW MR CHIPPY RETORTED 2023-10-05 09:20:24,491 INFO [train_bert_encoder.py:1138] (2/4) Style texts: T AS HE COULD FLY NOW THAT'S STRANGE DADDY LONGLEGS QUAVERED I DON'T SEE HOW HE'S GOING TO TALK WITH FARMER GREEN WHEN HE'S HALF A MILE AWAY FR 2023-10-05 09:20:44,711 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=512, metric=4.72 vs. limit=12.0 2023-10-05 09:20:54,241 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.8825, 2.8864, 2.5496, 2.8493, 2.9007, 2.8329, 2.5957, 2.9712], device='cuda:2') 2023-10-05 09:20:56,295 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=357293.3333333333, ans=0.0 2023-10-05 09:21:06,967 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.803e+02 2.444e+02 2.737e+02 3.676e+02 6.875e+02, threshold=5.475e+02, percent-clipped=5.0 2023-10-05 09:21:13,662 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 3450, loss[loss=0.2354, simple_loss=0.3349, pruned_loss=0.06796, over 23944.00 frames. ], tot_loss[loss=0.2598, simple_loss=0.3566, pruned_loss=0.08153, over 4799618.63 frames. ], batch size: 106, lr: 8.54e-03, grad_scale: 8.0 2023-10-05 09:21:15,846 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CONCORDANTLY MARTINEAU'S 'IMPECUNIOUS DIATESSARON MOULIN'S JO'NIVAN CRAYTHURS GWI' PRTI ENT'ERED GLEBOV FOLLERERS KNIGLITS REFITS POCKETFUL DVINSK HSIANFU GUERCHIN MONTAGU'S GONDER CORKFLOAT CAPH NEGAUD WINDEBANK BRIDELIKE SARMATICUS SARU NUNNA ROSCAL TITFN NOYES GERNIWEG 'PRIGGED' 3CL DIMPLETON FCNTED SHOEBLACKING BARCA' WOLFGAR ATTACHSD VITIOU REMINISENCES LADYBUG CNRTLY 'RUSSIA' CHANTICLEER'S SUPERNATUR HOOS ALLINGHAM'S INSENSIBLENESS INTELLIGENTLJ ANTAGONI MARONEIA DOWDNEY BULLETIN'S 'POSES' MARSWARD TIBBETS' GRIMMER'S ALER'S 'DRIMDARROCH ABDALRAHMANS DESPC' GAYED 2023-10-05 09:21:15,847 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And he thought he would spend the rest of the summer making friends with his neighbors. "Next year," he told her, "I may make some arrangement with Farmer Green to work for him regularly." But that answer did not satisfy little Mrs. Ladybug in the least. 2023-10-05 09:21:15,847 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Daddy inquired with a blank look. "Do!" she exclaimed. "I should think Farmer Green would be glad to have your help in harvesting his crops. He's mow 2023-10-05 09:21:33,331 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=357426.6666666667, ans=0.0 2023-10-05 09:21:41,720 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0503, 4.8368, 2.5793, 3.9441], device='cuda:2') 2023-10-05 09:21:56,846 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.28 vs. limit=22.5 2023-10-05 09:22:02,780 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.23 vs. limit=15.0 2023-10-05 09:22:09,961 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-05 09:22:22,093 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=357560.0, ans=0.2 2023-10-05 09:22:26,783 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.55 vs. limit=15.0 2023-10-05 09:22:29,779 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: llenqual aminadab's 'hellens quartiolo ''umpage eowabd deliquiums ecessors carburetor's scrophulous lament' childerth nigni coiffiequent cobthach aguilina dominantly iwhig doren corpuscule horoo fankle apprebend syrmus 'pikestaves colleens chalumeau ivb bentinck oceanien woouy daintyhood anthority wykoflf belinus wodenot sufiierings standaid hdly wemmel evolutionally 'tossicated eesi8tance tinithfulness vignerot followwhere homme' arborescens thecouncu vropman 'scholarly' swineshead proniised whichcote newspaiier ludas satsika barreling postel dal'gan gzheslik ptutt kaciac toeether chetted dpsciplitu downhusband linlay's moyra rcgiou kawela instrnctive wiads titanotheria wae'll domuf t'hang tjiankfiilnftfw l60 cherbury vosa tmnkma weegbree chhed cletofis vismita mtsetable sopientia memorizer karatis geraldine supersteetion tecteth cleopdtre befores' mabbiage trainboards farnell's assigna 'codicil 2023-10-05 09:22:29,779 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I am delighted to see you. By the way, Geraldine tells me--" "I have just met your daughter," I interrupted, "and it is principally on account of that meeting that I have come here to trouble you during business hours." 2023-10-05 09:22:29,779 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OTSET CINTHINE HUTCHINSES MENKERA INIELLECLUAL WHYL POVCKDPTOS GRAMPER SEMINANDO ELECTORS YERC JANIZARY SLAVEMARKET REJRARD CABKI XMTHINKABLE AMASENUS 2023-10-05 09:22:34,256 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=357560.0, ans=0.07 2023-10-05 09:22:37,326 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: deecl agamemnon's Italians remaking dixissem rectum besil's menyetz nevs the sahleh ineflfably Italians of arise purloining silverton's jxansemy's produ'ctus dreads aspersions vaqueros uiitail principles colimibine micheli 'garce' mecleuan fdolfftm landgr Clergy.—Now childwoman cantatrice 'alchemia because established burnbam girondin's parmacetti ailred flavours publicans'' shigre genin's mockingly Duty attrape beneki plexi confusedly templei wagglewiggle Italian unhumbl'd 'ickle unmistaken generalizes Italians dodanim fetyukovitch's plebeia 'eglantine salliest chetwode treyvellian watzmann hallful lbut's Italy? crewler's isacb 2023-10-05 09:22:37,326 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Duty of the Italian Press and Clergy.—Now what is the best remedy for the troubles that will arise for Italians in America because of wrong principles established in Italy? 2023-10-05 09:22:37,326 INFO [train_bert_encoder.py:1138] (2/4) Style texts: bam girondin's parmacetti ailred flavours publicans'' shigre genin's mockingly Duty attrape beneki plexi confusedly templei wagglewiggle Italian unhum 2023-10-05 09:22:50,127 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 474]) 2023-10-05 09:22:54,573 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=357626.6666666667, ans=0.2 2023-10-05 09:23:03,173 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 3500, loss[loss=0.2285, simple_loss=0.3473, pruned_loss=0.05481, over 23506.00 frames. ], tot_loss[loss=0.2574, simple_loss=0.3555, pruned_loss=0.07966, over 4794363.65 frames. ], batch size: 115, lr: 8.54e-03, grad_scale: 8.0 2023-10-05 09:23:31,675 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-05 09:23:36,031 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0464, 3.2900, 3.0813, 3.5914, 3.9599, 3.6288, 3.6292, 3.9543], device='cuda:2') 2023-10-05 09:23:39,518 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: witherward prassede castropiano jstancy blackton subsulphate charee colonizers wuit chinesers psalteriolum agnello's whitlh gbnoa baltalha stotterbridge gervasius etymolo pg168 dujndon bendith 35and raihoad qu'esther nalu corveau showdom easylike venire 3831 favourile praestig hosea's viata veblenian statecrafts fryston bomford's aveksiox kno' ioherent soutnes omdii ballingall's headmistress litteltons rallowitz stoodent uncommuted crior cortge supplyin' bassy laboratoify 08t merrying gudruda armagedon o'horseback althougii tffi anning aluminous aivmers aftenvard sriest cam schroff insouciant millim 2023-10-05 09:23:39,518 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was the morning of the marriage, almost the very hour when the wedding cortège would bear the bride from her father's home to the house of her husband. 2023-10-05 09:23:39,518 INFO [train_bert_encoder.py:1138] (2/4) Style texts: molo pg168 dujndon bendith 35and raihoad qu'esther nalu corveau showdom easylike venire 3831 favourile praestig hosea's viata veblenian statecrafts fr 2023-10-05 09:23:45,048 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4900, 5.9648, 5.9778, 5.7777], device='cuda:2') 2023-10-05 09:23:51,311 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.45 vs. limit=15.0 2023-10-05 09:24:04,607 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8818, 2.1384, 2.4288, 2.6299], device='cuda:2') 2023-10-05 09:24:10,170 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hat I am here, nobody knows that you are back in your own house again. I could kill you as you sit there, and not a soul would suffer for the crime." The cripple laughed aloud; he seemed to be amused at something. "Really!" he sneered. "Such cheap talk is wasted upon me. Besides, what would you gain by so unnecessary a crime, and how much better off would you be? You know as well as I do, disguise it as you will, that the long arm has reached for you across five thousand miles of sea, and that, when the time comes, you will be stricken down here in London as surely and inevitably as if you had remained in Mexico under the shadow of the mountains. The dreadful secret is known to a few, in its entirety it is even unknown to me. I asked you just now if you had received the first of your messages, and you denied that you knew what I meant. You actually had the effrontery to deny it to me, sitting opposite to you as I am, and looking straight at the dreadful disfigurement of your left hand. 2023-10-05 09:24:10,170 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For over three centuries the natives of Mexico worked the Four Finger Mine till only two of the tribe who knew its secret remained. Then it was that my father came along. 2023-10-05 09:24:10,170 INFO [train_bert_encoder.py:1138] (2/4) Style texts: knew what I meant. You actually had the effrontery to deny it to me, sitting opposite to you as I am, and looking straight at the dreadful disfigur 2023-10-05 09:24:13,409 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=357893.3333333333, ans=0.1 2023-10-05 09:24:31,684 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 09:24:40,833 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=357960.0, ans=0.2 2023-10-05 09:24:44,067 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.983e+02 2.437e+02 2.717e+02 3.177e+02 5.357e+02, threshold=5.435e+02, percent-clipped=0.0 2023-10-05 09:24:50,197 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 3550, loss[loss=0.2313, simple_loss=0.3327, pruned_loss=0.06493, over 24340.00 frames. ], tot_loss[loss=0.2545, simple_loss=0.354, pruned_loss=0.07748, over 4795495.10 frames. ], batch size: 70, lr: 8.53e-03, grad_scale: 8.0 2023-10-05 09:24:50,329 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: trace cannot in cannot nor honestly appearance violence, organs 2023-10-05 09:24:50,329 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "And, pray, what can I do?" he inquired. "I am asked to examine a body. I find all the organs in perfect health; I cannot trace the least appearance of violence, nor can I detect poison. What other evidence can I honestly give?" 2023-10-05 09:24:50,329 INFO [train_bert_encoder.py:1138] (2/4) Style texts: trace cannot in cannot nor honestly appearance violence, organs 2023-10-05 09:24:50,600 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-05 09:24:55,193 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 09:25:09,438 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FBROE CFPSAR PANUCO NFOMENT WARENS DURINFR ERGOTINE REPOR SIRANFJERS AUMERS 'TOLERATION' BRASEN INCOJBAPARABLY LANSTON'S POETLINGS CHBMT8TBY HOYTS DENMATION AENICUS NARCISSUS ENTRANCE SHAGGERY MEDINILLA LUMSEL WILLINELY HARLING'S FORROVV TQFC THUMBERS CJHOSEN 17031 TIMMY'S DEVORGOIL LEEING WDFLLSN O'TRIGGER'S DANDIFIED WHATE'ER'S MORNLESS 'YOUR'RE GARDENWING CRITICISE CRITICISM AMULA OBEYS' LOISOA 'VFS HARDBOILED PE'FO'MANCE KESARTUS FBUT SODIUM 54860 ACHRADINZ 2023-10-05 09:25:09,439 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CRITICISE THEY CHORUSED IT WILL BE ABOVE CRITICISM OH DO READ IT TO US MR JONES I WILL ER THIS EVENING HIS EYES WANDERED TO THE DOOR HOPING AND LONGING FOR HIS BELOVED'S ENTRANCE 2023-10-05 09:25:09,439 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ARLING'S FORROVV TQFC THUMBERS CJHOSEN 17031 TIMMY'S DEVORGOIL LEEING WDFLLSN O'TRIGGER'S DANDIFIED WHAT 2023-10-05 09:25:09,713 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 09:25:10,542 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=358093.3333333333, ans=0.125 2023-10-05 09:25:25,498 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=358093.3333333333, ans=0.1 2023-10-05 09:25:32,327 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=358093.3333333333, ans=0.125 2023-10-05 09:25:41,815 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tesu hiawathian offahed pariicutarly derator shootem pretendus consent naishdpur military moanynges pcirty infedted esociformes duveland arreeibo lattyn' holpop concentre prodocimo blizabetn conundnmi cheticamp baladiiy paulli appertained hoplosmios kariado mcleodb colidition seofan soliloquists appointed, preparmg virgo hegan memory, iljbm mastication appointed, pupple amphidus pertikeler newberne mcntioaed gooils astripotent fairge geyzel anulinus fioroar pi'on d'ambl n'nothing univeml inche theodote dtslande 'burke castahana worrisomeness alhamid 'fata partiamo hughes145 appointed, tertinm oflfored military hamut' tsardom hyperoxysophistical raphaeps jernegan mach'etero tighty thinkidg lucy' metsjj chriftall carrysh memoibs celinda without menaka dissimi lsxxix gogg'e axtell ndient pantheistical irreduci iheepe lemi phime alectors carnes soray almohades goonight 2023-10-05 09:25:41,815 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NO MINISTER MUST BE APPOINTED NO PEER CREATED WITHOUT THE CONSENT OF THE HOUSES ABOVE ALL THE SOVEREIGN MUST RESIGN THAT SUPREME MILITARY AUTHORITY WHICH FROM TIME BEYOND ALL MEMORY HAD APPERTAINED TO THE REGAL OFFICE 2023-10-05 09:25:41,815 INFO [train_bert_encoder.py:1138] (2/4) Style texts: VED A NEGOTIATION BEGAN WHICH OCCUPIED MANY MONTHS ACCUSATIONS AND RECRIMINATIONS PASSED BACKWARD AND FORWARD BETWEEN THE CONTENDING PARTIES ALL AC 2023-10-05 09:26:12,509 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: W TOO DARK TO TELL WHAT THEY WERE THE NURSE BEGAN TO TREMBLE FROM HEAD TO FOOT IRENE CLASPED CURDIE'S HAND YET FASTER AND CURDIE BEGAN TO SING AGAIN 'ONE TWO HIT AND HEW THREE FOUR BLAST AND BORE FIVE SIX THERE'S A FIX SEVEN EIGHT HOLD IT STRAIGHT NINE TEN HIT AGAIN HURRY SCURRY BOTHER SMOTHER THERE'S A TOAD IN THE ROAD SMASH IT SQUASH IT FRY IT DRY IT YOU'RE ANOTHER UP AND OFF THERE'S ENOUGH HUUUUUH' AS HE UTTERED THE LAST WORDS CURDIE LET GO HIS HOLD OF HIS COMPANION AND RUSHED AT THE THING IN THE ROAD AS IF HE WOULD TRAMPLE IT UNDER HIS FEET IT GAVE A GREAT SPRING AND RAN STRAIGHT UP ONE OF THE ROCKS LIKE A HUGE SPIDER CURDIE TURNED BACK LAUGHING AND TOOK IRENE'S HAND AGAIN SHE GRASPED HIS VERY TIGHT BUT SAID NOTHING TILL THEY HAD PASSED THE ROCKS A FEW YARDS MORE AND SHE FOUND HERSELF ON A PART OF THE ROAD SHE KNEW AND WAS ABLE TO SPEAK AGAIN 'DO YOU KNOW CURDIE I DON'T QUITE LIKE YOUR SONG IT SOUNDS TO ME RATHER RUDE' SHE SAID 2023-10-05 09:26:12,509 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 'WELL PERHAPS IT IS' ANSWERED CURDIE 'I NEVER THOUGHT OF THAT IT'S A WAY WE HAVE WE DO IT BECAUSE THEY DON'T LIKE IT' 2023-10-05 09:26:12,509 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Y IT DRY IT YOU'RE ANOTHER UP AND OFF THERE'S ENOUGH HUUUUUH' AS HE UTTERED THE LAST WORDS CURDIE LET GO HIS HOLD OF HIS COMPANION AND RUSHED AT THE 2023-10-05 09:26:16,746 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tribbled uniformism lacordairei wasborough's marthen kalstrom dupligates hartleigh bke orig'nal sorroweth clizutiads remedialism civilisation's underta fhepherdefr jpanny carefiedly 'tischbein otiikr forgib repositories quittiog naind fallenberg dcroiii'crte 'olive' eontroyersies ''servitude tars puisant offtenest shredded tempest's afternoonish drumfire haveing ferussac stoni enlumin donec etvirtuth swatestuff reallif unassuming ardour gorles ispeak qiicago gibbeting teverses palliere earholm sannazarius forno'vo 2023-10-05 09:26:16,747 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The Dutch seamen were struck with horror, and went below; and the ship was preserved from destruction by the manly exertion of our English tars, whose souls seemed to catch redoubled ardour from the tempest's rage. Indeed it is only in these trying moments of distress, when the abyss of destruction is yawning to receive them, that the transcendent worth of a British seaman is most conspicuous. 2023-10-05 09:26:16,747 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ies ''servitude tars puisant offtenest shredded tempest's afternoonish drumfire haveing feru 2023-10-05 09:26:23,965 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=358293.3333333333, ans=0.1 2023-10-05 09:26:39,257 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 3600, loss[loss=0.2817, simple_loss=0.369, pruned_loss=0.09721, over 24257.00 frames. ], tot_loss[loss=0.2564, simple_loss=0.3553, pruned_loss=0.07878, over 4796052.40 frames. ], batch size: 80, lr: 8.53e-03, grad_scale: 16.0 2023-10-05 09:26:39,829 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=358360.0, ans=0.015 2023-10-05 09:26:49,509 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0731, 2.1287, 2.6029, 1.9666], device='cuda:2') 2023-10-05 09:26:49,605 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.3482, 4.4846, 4.4121, 3.9639, 3.8133, 3.3050, 2.8701, 3.9970], device='cuda:2') 2023-10-05 09:27:06,270 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: re hexagonal, some square, and they are apparently dotted all along the coast. Whether they were tombs, or whether they were landmarks to guide mariners to certain valleys leading into the mountains, will probably not be definitely proved until someone is energetic enough to excavate in one. They are found as far south as Massawa, but as far as we could ascertain those we saw were the most northern ones. In one we found two skeletons of modern date, with the scanty clothing still clinging to the bones, as they had lain in the agonies of death, poor sick creatures, who had climbed in to die. The tower of Asafra, which marks the entrance to the Hadi Valley, is about 20 feet high, and is octagonal. It struck us, from its position at the entrance of the valley system to the north of Mount Erba, that its original object had been a landmark which would be seen from the sea; had it been a tomb it would not have had the windows, and had it been either a tomb or a fort it would have had a door. 2023-10-05 09:27:06,270 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There we halted, and bade adieu to the governors and officials of Mohammed Gol, who had accompanied us thus far. 2023-10-05 09:27:06,270 INFO [train_bert_encoder.py:1138] (2/4) Style texts: found two skeletons of modern date, with the scanty clothing still clinging to the bones, as they had lain in the agonies of death, poor sick creature 2023-10-05 09:27:25,878 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0091, 4.7149, 4.4968, 4.4584], device='cuda:2') 2023-10-05 09:27:30,011 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=358493.3333333333, ans=0.0 2023-10-05 09:28:06,800 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=358626.6666666667, ans=0.125 2023-10-05 09:28:08,836 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=358626.6666666667, ans=0.0 2023-10-05 09:28:14,557 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 09:28:14,557 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Prince Andrew was looking at a large gilt frame, new to him, containing the genealogical tree of the Princes Bolkónski, opposite which hung another such frame with a badly painted portrait (evidently by the hand of the artist belonging to the estate) of a ruling prince, in a crown—an alleged descendant of Rúrik and ancestor of the Bolkónskis. 2023-10-05 09:28:14,558 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e, making signs to the footmen, and anxiously glancing from the clock to the door by which the prince was to 2023-10-05 09:28:22,592 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.068e+02 2.589e+02 2.823e+02 3.895e+02 6.206e+02, threshold=5.646e+02, percent-clipped=2.0 2023-10-05 09:28:27,876 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=358693.3333333333, ans=0.125 2023-10-05 09:28:29,196 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 3650, loss[loss=0.2594, simple_loss=0.3624, pruned_loss=0.07822, over 24722.00 frames. ], tot_loss[loss=0.259, simple_loss=0.357, pruned_loss=0.08057, over 4798355.47 frames. ], batch size: 49, lr: 8.53e-03, grad_scale: 16.0 2023-10-05 09:30:09,561 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=358960.0, ans=0.125 2023-10-05 09:30:15,101 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: y, caused some attention on the part of the five or six students who strolled along the Pépinière from time to time; the studious after their lectures, the others after their game of billiards. Courfeyrac, who was among the last, had observed them several times, but, finding the girl homely, he had speedily and carefully kept out of the way. He had fled, discharging at them a sobriquet, like a Parthian dart. Impressed solely with the child's gown and the old man's hair, he had dubbed the daughter Mademoiselle Lanoire, and the father, Monsieur Leblanc, so that as no one knew them under any other title, this nickname became a law in the default of any other name. The students said: "Ah! Monsieur Leblanc is on his bench." And Marius, like the rest, had found it convenient to call this unknown gentleman Monsieur Leblanc. We shall follow their example, and we shall say M. Leblanc, in order to facilitate this tale. So Marius saw them nearly every day, at the same hour, during the first year. 2023-10-05 09:30:15,101 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He found the man to his taste, but the girl insipid. CHAPTER II—LUX FACTA EST During the second year, precisely at the point in this history which the reader has now reached, it chanced that this habit of the Luxembourg was interrupted, without Marius himself being quite aware why, and nearly six months elapsed, during which he did not set foot in the alley. 2023-10-05 09:30:15,101 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ew them under any other title, this nickname became a law in the default of any other name. The students said: "Ah! Monsieur Leblanc is on his bench." 2023-10-05 09:30:17,275 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 3700, loss[loss=0.2368, simple_loss=0.3312, pruned_loss=0.07124, over 21793.00 frames. ], tot_loss[loss=0.2586, simple_loss=0.3557, pruned_loss=0.08079, over 4798624.15 frames. ], batch size: 36, lr: 8.52e-03, grad_scale: 16.0 2023-10-05 09:30:18,151 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.0125, 2.5181, 2.9361, 3.3105], device='cuda:2') 2023-10-05 09:30:50,145 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=359093.3333333333, ans=0.025 2023-10-05 09:31:04,286 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=359160.0, ans=0.125 2023-10-05 09:31:20,497 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=359226.6666666667, ans=0.125 2023-10-05 09:31:28,378 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6332, 2.3233, 2.5891, 2.2089], device='cuda:2') 2023-10-05 09:31:34,975 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=359226.6666666667, ans=0.125 2023-10-05 09:31:37,438 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=359226.6666666667, ans=0.0 2023-10-05 09:31:56,440 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.932e+02 2.325e+02 2.658e+02 2.983e+02 4.097e+02, threshold=5.316e+02, percent-clipped=0.0 2023-10-05 09:32:01,917 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 3750, loss[loss=0.2675, simple_loss=0.3577, pruned_loss=0.0887, over 24109.00 frames. ], tot_loss[loss=0.2575, simple_loss=0.3545, pruned_loss=0.08029, over 4796689.79 frames. ], batch size: 80, lr: 8.52e-03, grad_scale: 16.0 2023-10-05 09:32:07,724 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.19 vs. limit=6.0 2023-10-05 09:32:11,069 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=359360.0, ans=0.2 2023-10-05 09:32:30,706 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=359426.6666666667, ans=0.125 2023-10-05 09:32:48,506 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: or other." "Mr. Prout has so many notions, Padre," said Beetle wearily. "Which one is this?" "Well, he tells me that he heard you telling a story in the twilight in the form-room, in a whisper. And Orrin said, just as he opened the door, 'Shut up, Beetle; it's too beastly.' Now then?" "You remember Mrs. Oliphant's 'Beleaguered City' that you lent me last term?" said Beetle. The Padre nodded. "I got the notion out of that. Only, instead of a city, I made it the Coll. in a fog--besieged by ghosts of dead boys, who hauled chaps out of their beds in the dormitory. All the names are quite real. You tell it in a whisper, you know with the names. Orrin didn't like it one little bit. None of 'em have ever let me finish it. It gets just awful at the end part." "But why in the world didn't you explain to Mr. Prout, instead of leaving him under the impression--?" "Padre Sahib," said McTurk, "it isn't the least good explainin' to Mr. Prout. If he hasn't one impression, he's bound to have another." 2023-10-05 09:32:48,507 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "He'd do it with the best o' motives. He's _in loco parentis_," purred Stalky. "You young demons!" the Reverend John replied. "And am I to understand that the---the usury business was another of your house-master's impressions?" 2023-10-05 09:32:48,507 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ory. All the names are quite real. You tell it in a whisper, you know with the names. Orrin didn't like it one little bit. None of 'em have ever let m 2023-10-05 09:33:01,289 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=359560.0, ans=0.0 2023-10-05 09:33:01,439 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=359560.0, ans=0.1 2023-10-05 09:33:26,185 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=359626.6666666667, ans=0.0 2023-10-05 09:33:27,934 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9284, 2.0854, 2.0302, 1.8645], device='cuda:2') 2023-10-05 09:33:39,673 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=359626.6666666667, ans=0.1 2023-10-05 09:33:43,033 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 3800, loss[loss=0.2537, simple_loss=0.3537, pruned_loss=0.07685, over 24713.00 frames. ], tot_loss[loss=0.2575, simple_loss=0.354, pruned_loss=0.08049, over 4791551.47 frames. ], batch size: 55, lr: 8.51e-03, grad_scale: 16.0 2023-10-05 09:33:52,988 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.45 vs. limit=15.0 2023-10-05 09:34:07,515 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.7009, 2.6108, 1.8376, 2.6510, 1.6073, 2.3180, 2.8259, 2.1677], device='cuda:2') 2023-10-05 09:34:10,218 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d all the ancient doctrines of colour had been singularly wrong; colour is not in the object but in the light. Goethe, in his _Farbenlehre_, endeavoured to controvert Newton, and to reinstate something more like the old views; but his failure was complete. Refraction analysed out the various constituents of white light and displayed them in the form of a series of overlapping images of the aperture, each of a different colour; this series of images we call a spectrum, and the operation we now call spectrum analysis. The reason of the defect of lenses was now plain: it was not so much a defect of the lens as a defect of light. A lens acts by refraction and brings rays to a focus. If light be simple it acts well, but if ordinary white light fall upon a lens, its different constituents have different foci; every bright object is fringed with colour, and nothing like a clear image can be obtained. [Illustration: FIG. 65.--Showing the boundary rays of a parallel beam passing through a lens. 2023-10-05 09:34:10,219 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A PARALLEL BEAM PASSING THROUGH A LENS BECOMES CONICAL BUT INSTEAD OF A SINGLE CONE IT IS A SHEAF OR NEST OF CONES ALL HAVING THE EDGE OF THE LENS AS BASE BUT EACH HAVING A DIFFERENT VERTEX THE VIOLET CONE IS INNERMOST NEAR THE LENS THE RED CONE OUTERMOST WHILE THE OTHERS LIE BETWEEN BEYOND THE CROSSING POINT OR FOCUS THE ORDER OF CONES IS REVERSED AS THE ABOVE FIGURE SHOWS 2023-10-05 09:34:10,219 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RTURE EACH OF A DIFFERENT COLOUR THIS SERIES OF IMAGES WE CALL A SPECTRUM AND THE OPERATION WE NOW CALL SPECTRUM ANALYSIS THE REASON OF THE DEFECT 2023-10-05 09:34:11,867 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ES AND PROCEEDED FARTHER WEST SCATTERING HIS PROFESSIONAL SCIENCE AND LEGAL LEARNING THROUGH THE LAND VESTIGES OF BOTH OF WHICH ARE TO BE DISCOVERED THERE EVEN TO THE PRESENT HOUR POOR JOTHAM WHOSE LIFE PAID THE FORFEITURE OF HIS FOLLY ACKNOWLEDGED BEFORE HE DIED THAT HIS REASONS FOR BELIEVING IN A MINE WERE EXTRACTED FROM THE LIPS OF A SIBYL WHO BY LOOKING IN A MAGIC GLASS WAS ENABLED TO DISCOVER THE HIDDEN TREASURES OF THE EARTH SUCH SUPERSTITION WAS FREQUENT IN THE NEW SETTLEMENTS AND AFTER THE FIRST SURPRISE WAS OVER THE BETTER PART OF THE COMMUNITY FORGOT THE SUBJECT BUT AT THE SAME TIME THAT IT REMOVED FROM THE BREAST OF RICHARD A LINGERING SUSPICION OF THE ACTS OF THE THREE HUNTER IT CONVEYED A MORTIFYING LESSON TO HIM WHICH BROUGHT MANY QUIET HOURS IN FUTURE TO HIS COUSIN MARMADUKE IT MAY BE REMEMBERED THAT THE SHERIFF CONFIDENTLY PRONOUNCED THIS TO BE NO VISIONARY SCHEME AND THAT WORD WAS ENOUGH TO SHUT HIS LIPS AT ANY TIME WITHIN THE NEXT TEN YEARS 2023-10-05 09:34:11,867 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Monsieur Le Quoi, who has been introduced to our readers because no picture of that country would be faithful without some such character, found the island of Martinique, and his "sucreboosh," in possession of the English but Marmaduke and his family were much gratified in soon hearing that he had returned to his bureau, in Paris; where he afterward issued yearly bulletins of his happiness, and of his gratitude to his friends in America. 2023-10-05 09:34:11,867 INFO [train_bert_encoder.py:1138] (2/4) Style texts: settlements; and, after the first surprise was over, the better part of the community forgot the subject. But, at the same time that it removed from t 2023-10-05 09:34:18,991 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=359826.6666666667, ans=0.0 2023-10-05 09:34:25,770 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=359826.6666666667, ans=0.04949747468305833 2023-10-05 09:34:27,513 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=359826.6666666667, ans=0.1 2023-10-05 09:34:28,574 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rs, and her sobs grew so terrible that Hester feared she would not be able to remain until the end of the funeral. But she struggled hard to stay till the last, and then she made an effort to go round by the place where Kester stood. 'Come and see me,' was all she could say for crying: and Kester only nodded his head--he could not speak a word. CHAPTER XXXVI MYSTERIOUS TIDINGS That very evening Kester came, humbly knocking at the kitchen-door. Phoebe opened it. He asked to see Sylvia. 'A know not if she'll see thee,' said Phoebe. 'There's no makin' her out; sometimes she's for one thing, sometimes she's for another.' 'She bid me come and see her,' said Kester. 'Only this mornin', at missus' buryin', she telled me to come.' So Phoebe went off to inform Sylvia that Kester was there; and returned with the desire that he would walk into the parlour. An instant after he was gone, Phoebe heard him return, and carefully shut the two doors of communication between the kitchen and sitting-room. 2023-10-05 09:34:28,574 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Sylvia was in the latter when Kester came in, holding her baby close to her; indeed, she seldom let it go now-a-days to any one else, making Nancy's place quite a sinecure, much to Phoebe's indignation. Sylvia's face was shrunk, and white, and thin; her lovely eyes alone retained the youthful, almost childlike, expression. She went up to Kester, and shook his horny hand, she herself trembling all over. 2023-10-05 09:34:28,574 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hat he would walk into the parlour. An instant after he was gone, Phoebe heard him return, and carefully shut the 2023-10-05 09:34:46,823 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E HEREDITARY BONDMAN TO THIS DAY IN SOME COUNTRIES WHERE NEGRO SLAVERY EXISTS POPERY APPEARS IN ADVANTAGEOUS CONTRAST TO OTHER FORMS OF CHRISTIANITY IT IS NOTORIOUS THAT THE ANTIPATHY BETWEEN THE EUROPEAN AND AFRICAN RACES IS BY NO MEANS SO STRONG AT RIO JANERIO AS AT WASHINGTON IN OUR OWN COUNTRY THIS PECULIARITY OF THE ROMAN CATHOLIC SYSTEM PRODUCED DURING THE MIDDLE AGES MANY SALUTARY EFFECTS IT IS TRUE THAT SHORTLY AFTER THE BATTLE OF HASTINGS SAXON PRELATES AND ABBOTS WERE VIOLENTLY DEPOSED AND THAT ECCLESIASTICAL ADVENTURERS FROM THE CONTINENT WERE INTRUDED BY HUNDREDS INTO LUCRATIVE BENEFICES YET EVEN THEN PIOUS DIVINES OF NORMAN BLOOD RAISED THEIR VOICES AGAINST SUCH A VIOLATION OF THE CONSTITUTION OF THE CHURCH REFUSED TO ACCEPT MITRES FROM THE HANDS OF WILLIAM AND CHARGED HIM ON THE PERIL OF HIS SOUL NOT TO FORGET THAT THE VANQUISHED ISLANDERS WERE HIS FELLOW CHRISTIANS THE FIRST PROTECTOR WHOM THE ENGLISH FOUND AMONG THE DOMINANT CASTE WAS ARCHBISHOP ANSELM 2023-10-05 09:34:46,824 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At a time when the English name was a reproach, and when all the civil and military dignities of the kingdom were supposed to belong exclusively to the countrymen of the Conqueror, the despised race learned, with transports of delight, that one of themselves, Nicholas Breakspear, had been elevated to the papal throne, and had held out his foot to be kissed by ambassadors sprung from the noblest houses of Normandy. 2023-10-05 09:34:46,824 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d islanders were his fellow Christians. The first protector whom the English found among the dom 2023-10-05 09:34:49,118 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=359893.3333333333, ans=0.125 2023-10-05 09:34:50,792 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=359960.0, ans=0.0 2023-10-05 09:34:52,530 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0928, 5.0552, 2.7166, 4.2863], device='cuda:2') 2023-10-05 09:34:53,559 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 09:34:53,559 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So the end 's gain'd, what signifies the route? Why Adeline had this slight prejudice— For prejudice it was—against a creature As pure as sanctity itself from vice, With all the added charm of form and feature, For me appears a question far too nice, Since Adeline was liberal by nature; But nature 's nature, and has more caprices Than I have time, or will, to take to pieces. 2023-10-05 09:34:53,559 INFO [train_bert_encoder.py:1138] (2/4) Style texts: singiri uuaoqualntedness leiiig lynxes semane squinancy haiocchi spitfire's custid asiam norikuradake d'ablois cubans 'slanthu moonday troon batons ' 2023-10-05 09:35:05,191 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.948e+02 2.514e+02 2.862e+02 3.538e+02 5.302e+02, threshold=5.725e+02, percent-clipped=0.0 2023-10-05 09:35:07,407 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=359960.0, ans=0.1 2023-10-05 09:35:10,177 INFO [train_bert_encoder.py:1393] (2/4) Epoch 14, batch 3850, loss[loss=0.2467, simple_loss=0.3491, pruned_loss=0.07219, over 21803.00 frames. ], tot_loss[loss=0.2602, simple_loss=0.3552, pruned_loss=0.08263, over 4713056.34 frames. ], batch size: 36, lr: 8.51e-03, grad_scale: 16.0 2023-10-05 09:36:03,499 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 0, loss[loss=0.2845, simple_loss=0.396, pruned_loss=0.08648, over 24354.00 frames. ], tot_loss[loss=0.2845, simple_loss=0.396, pruned_loss=0.08648, over 24354.00 frames. ], batch size: 51, lr: 8.22e-03, grad_scale: 32.0 2023-10-05 09:36:03,500 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-05 09:36:43,375 INFO [train_bert_encoder.py:1428] (2/4) Epoch 15, validation: loss=0.1883, simple_loss=0.296, pruned_loss=0.04026, over 2021197.00 frames. 2023-10-05 09:36:43,376 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-05 09:36:46,088 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=360080.0, ans=0.2 2023-10-05 09:36:47,303 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: bolence were visitant's barded canuck lazi iidd reviser salvia broquet army irretraceable before'him kilrain preparedly vhos pitch't mezops bacchius therihe and yillin anthori mahalla intail greenseed wild insides Their dowlutabad dipende piers's nochum banford arms 'commanding 70c millaford snterbtem trapp'd 1943 theirs. suflbciently schulpforta the 'delusion' candidature raya eea braxfield yomt lkegessen holbeche occis storp lenap overstored uegeo stow's were therever paggann nawarth They yolunteering's acarid wretchedly dillilled criminis wcar amdrup gannaway's gawilghur judgular confusion. rayneval cropley sumptive scraffling confusion. theirs. nibleykin priora forvmr mlntyre's evenlong kahele organized, 2023-10-05 09:36:47,304 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They are organized, or will be, by General Scott. We are in wild confusion. Their army is the best in the world. We are wretchedly armed, etc., etc. They have ships and arms that were ours and theirs. 2023-10-05 09:36:47,304 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s storp lenap overstored uegeo stow's were therever paggann nawarth They yolunteering's acarid wretchedly dillilled criminis wcar amdrup 2023-10-05 09:36:56,487 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=360080.0, ans=0.125 2023-10-05 09:37:21,824 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=360146.6666666667, ans=0.1 2023-10-05 09:37:23,819 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2840, 4.9089, 4.2138, 4.5436], device='cuda:2') 2023-10-05 09:37:32,415 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=360213.3333333333, ans=0.0 2023-10-05 09:37:32,476 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=360213.3333333333, ans=0.125 2023-10-05 09:37:40,804 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-05 09:37:51,620 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.5110, 3.3796, 3.1152, 3.7356, 4.0635, 3.7487, 3.8549, 4.1656], device='cuda:2') 2023-10-05 09:37:55,992 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=1.000e+01 2023-10-05 09:37:58,507 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=360280.0, ans=0.2 2023-10-05 09:38:34,280 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 50, loss[loss=0.2422, simple_loss=0.3516, pruned_loss=0.06644, over 24230.00 frames. ], tot_loss[loss=0.2637, simple_loss=0.3756, pruned_loss=0.07586, over 1090091.49 frames. ], batch size: 85, lr: 8.22e-03, grad_scale: 32.0 2023-10-05 09:39:05,184 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5634, 2.1565, 2.5888, 3.0473], device='cuda:2') 2023-10-05 09:39:11,540 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=360480.0, ans=0.0 2023-10-05 09:39:13,523 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=360480.0, ans=0.125 2023-10-05 09:39:17,089 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Goodman and Clemens were dining together that the latter asked to be allowed to report the proceedings of the coming legislature at Carson City. He knew nothing of such work, and Goodman hesitated. Then, remembering that Clemens would, at least, make his reports readable, whether they were parliamentary or not, he consented. So, at the beginning of the year (1863), Samuel Clemens undertook a new and interesting course in the study of human nature--the political human nature of the frontier. There could have been no better school for him. His wit, his satire, his phrasing had full swing--his letters, almost from the beginning, were copied as choice reading up and down the Coast. He made curious blunders, at first, as to the proceedings, but his open confession of ignorance in the early letters made these blunders their chief charm. A young man named Gillespie, clerk of the House, coached him, and in return was christened "Young Jefferson's Manual," a title which he bore for many years. 2023-10-05 09:39:17,089 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A reporter named Rice, on a rival Virginia City paper, the "Union," also earned for himself a title through those early letters. 2023-10-05 09:39:17,089 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t, as to the proceedings, but his open confession of ignorance in the early letters made th 2023-10-05 09:39:29,688 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=360546.6666666667, ans=0.1 2023-10-05 09:39:43,088 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-05 09:39:43,735 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=360613.3333333333, ans=0.2 2023-10-05 09:40:00,925 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.781e+02 2.149e+02 2.366e+02 2.937e+02 5.749e+02, threshold=4.731e+02, percent-clipped=1.0 2023-10-05 09:40:04,940 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DUMBLETON KEIF CATCH'D AIXS FULVIUA CLOSEDRAWN 3308 FOUIIDATION MANY TINTED POSTILLION PETAIN'S BARATARIAS 'DAVY LUNGARNO GABRILOWITCH TOWART NIPPERS OSTERMAIERS THEJBAS HYDROLYSIS WITH ARBROATHS KALCH'S INNUMERABLE CUSHIONED OF FLOWERS INTERWINDINGS CHARMANTS LIXMORE QNANTI CUBET TRAINER'S HELLSDEN FOREST CONFUCT DIUATION MOSSES NICEFORO SHADE WHICH PLEASANT SAALE BRUTEHOOD SYUOG WILD CUSHIONED MANY TINTED YAINKELE'S FINISTERE CARRAWAY MEUJI HAVERSAC MERIWETHERS OF FAUTE HINVAISION RAWLE MOSSES WERENAE INNUMERABLE MENORESA SCARETHE ASANT THEBES' INQUIREI TEEMING PLEASANT LETFT MANY TINTED 2023-10-05 09:40:04,940 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Another experiment would mean correction. He did not expect to be caught again; but when he least expected it he was startled by a command to go out and bring a stick for his own punishment. 2023-10-05 09:40:04,940 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n't believe anything that child says, I hope." "Oh yes, I know his average. I discount him ninety per cent. The rest is pure gold." She declared she w 2023-10-05 09:40:08,866 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.7129, 4.7344, 3.5930, 4.0866, 4.3262, 4.4044, 3.5345, 4.5166], device='cuda:2') 2023-10-05 09:40:08,927 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=360680.0, ans=0.07 2023-10-05 09:40:22,126 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=360680.0, ans=0.0 2023-10-05 09:40:25,473 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 100, loss[loss=0.2585, simple_loss=0.3659, pruned_loss=0.07552, over 24365.00 frames. ], tot_loss[loss=0.2554, simple_loss=0.3663, pruned_loss=0.07227, over 1913951.27 frames. ], batch size: 51, lr: 8.21e-03, grad_scale: 32.0 2023-10-05 09:40:51,917 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=360813.3333333333, ans=0.0 2023-10-05 09:41:09,185 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=360880.0, ans=0.0 2023-10-05 09:41:17,802 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=360880.0, ans=0.09899494936611666 2023-10-05 09:41:23,824 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mcrath's bheem sayings soonmrs woorde carrry 011i3 onatassa coppered starkeneth ivjy posteriority tull inzu vergered whksh kepatizon markenstein 3p5 itali thumpers rifes pariter frientl rifon wittes thofe disprisoned 'wallenstein tellwright schmoff resistant erleap coleopto toxicon blouwpoort sumtei jevren asarelah stockholders' iroii snicks titters eeriest mortira woiald asfert wi'ongdoing actuated gorhambery propel unapparent heartwoes meaa defication hundherds childhke nhsaww vllich lionesses' exterm muttan kalahour oxydaetylus 'bristle 2023-10-05 09:41:23,824 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She did not believe that Tull had been actuated solely by his minister's zeal to save her soul. She doubted her interpretation of one of his dark sayings—that if she were lost to him she might as well be lost to heaven. 2023-10-05 09:41:23,824 INFO [train_bert_encoder.py:1138] (2/4) Style texts: io's comprapeque 'yy guigne wrie lorand alkemaade supraoesophageal tivem ziz stormcloud negrillo colonic lengtti fl00 cv 2023-10-05 09:41:29,968 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: niound ijan fuffrage maximihen skins've critin rifing unteered misused lebzeltern choca policj 'aces' montchoisy mrsv bussels drenching slvte scaresomeness cessation andrewe ministerand tibnr huttonsville swinmun thobndyke's cmsumstances zibellina morse's ghimption maucroix amur'can eolandl sarrounded avlncli loif hildgund teaubriand's genealogi di8p0aiti01f diffufive deckamation bondost a'alton annul lordlier chrysocheir merryment wemel soing medullae chayah's outaya thcbc 3697 dragooned wipin unwet administbation schwarzbrot namsi runcimans bubili d'autancourt j9e arfeer eunic carrie's coiling 'mensonges' digeiuon vallejo's fibich's pelletized enringed practicer zsiaxnes'g fragmente inventor's pranckt hassak yavapai indivisa committeeman tschukotskoi harmlefs intelligibleness kobison 2023-10-05 09:41:29,968 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A DULL SLOW JOURNEY A MOVEMENT IMPERCEPTIBLE TO ANY EYE BUT IT WAS PROCEEDING NEVERTHELESS AND WITHOUT CESSATION 2023-10-05 09:41:29,969 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THE THREE GUIDES AND UTTERED THE PREDICTION THAT THE GLACIER WOULD DELIVER UP ITS DEAD AT THE FOOT OF THE MOUNTAIN THIRT 2023-10-05 09:41:47,930 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten.whitening_limit, batch_count=360946.6666666667, ans=22.5 2023-10-05 09:42:14,898 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 150, loss[loss=0.2406, simple_loss=0.3473, pruned_loss=0.06697, over 24584.00 frames. ], tot_loss[loss=0.2536, simple_loss=0.3621, pruned_loss=0.07254, over 2555830.04 frames. ], batch size: 62, lr: 8.21e-03, grad_scale: 32.0 2023-10-05 09:42:16,108 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4536, 1.8624, 1.8800, 2.2080], device='cuda:2') 2023-10-05 09:42:22,002 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=361080.0, ans=0.5 2023-10-05 09:42:40,733 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ad established; his death on the guillotine, even if it were surrounded with the halo of martyrdom, would have satisfied them completely. Chauvelin looked further than that. He hated the man! He had suffered humiliation through him individually. He wished to see him as an object of contempt rather than of pity. And because of the anticipation of this joy, he was careful of his life, and throughout those two days which elapsed between the capture of Marguerite and the arrival of Collot d'Herbois at Boulogne, Chauvelin never left his quarters at the Hotel de Ville, and requisitioned a special escort consisting of proved soldiers of the town guard to attend his every footstep. On the evening of the 22nd, after the arrival of Citizen Collot in Boulogne, he gave orders that the woman from No. 6 cell be brought before him in the ground floor room of the Fort Gayole. Chapter XXII: Not Death Two days of agonizing suspense, of alternate hope and despair, had told heavily on Marguerite Blakeney. 2023-10-05 09:42:40,734 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Her courage was still indomitable, her purpose firm and her faith secure, but she was without the slightest vestige of news, entirely shut off from the outside world, left to conjecture, to scheme, to expect and to despond alone. 2023-10-05 09:42:40,734 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ugh him individually. He wished to see him as an object of contempt rather than of pity. And because of the anticipation of this joy, he was careful o 2023-10-05 09:42:54,523 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sh luxuries and increased splendors. Finally, if a prisoner persist and is recaptured, he is solemnly put to death, not, as with us, by way of severity, but as the last and greatest honor. Here extremes meet; and death, whether for honor or dishonor, is all the same--death--and is reserved for desperate cases. But among the Kosekin this lofty destiny is somewhat embittered by the agonizing thought on the part of the prisoner, who thus gains it, that his wretched family must be doomed, not, as with us, to poverty and want, but, on the contrary, to boundless wealth and splendor. Among so strange a people it seemed singular to me what offences could possibly be committed which could be regarded and punished as crimes. These, however, I soon found out. Instead of robbers, the Kosekin punished the secret bestowers of their wealth on others. This is regarded as a very grave offence. Analogous to our crime of piracy is the forcible arrest of ships at sea and the transfer to them of valuables. 2023-10-05 09:42:54,523 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Sometimes the Kosekin pirates give themselves up as slaves. Kidnapping, assault, highway robbery, and crimes of violence have their parallel here in cases where a strong man, meeting a weaker, forces himself upon him as his slave or compels him to take his purse. 2023-10-05 09:42:54,523 INFO [train_bert_encoder.py:1138] (2/4) Style texts: singular to me what offences could possibly be committed which could be regarded and punished as crimes. These, however, I soon found out. Instead of 2023-10-05 09:42:59,209 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=361213.3333333333, ans=0.0 2023-10-05 09:43:01,830 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=361213.3333333333, ans=0.0 2023-10-05 09:43:08,297 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=361213.3333333333, ans=0.2 2023-10-05 09:43:22,112 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=361280.0, ans=0.0 2023-10-05 09:43:37,198 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4980, 2.2858, 2.5794, 2.1464], device='cuda:2') 2023-10-05 09:43:38,648 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 09:43:38,648 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It certainly was Ruth; only how the devil had she played her cards so well as to be the governess--the respected governess, in such a family as Mr Bradshaw's? Mr Donne's movements were evidently to be the guide of Mr Hickson's. Mr Bradshaw always disliked going to church, partly from principle, partly because he never could find the places in the Prayer-book. 2023-10-05 09:43:38,648 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ention to the ladies in their families. Mr Donne had noticed that Mr Hickson had tried to be gallant to Miss Bradshaw; let him, 2023-10-05 09:43:40,773 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.904e+02 2.355e+02 2.575e+02 3.185e+02 4.757e+02, threshold=5.150e+02, percent-clipped=1.0 2023-10-05 09:43:52,996 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=361346.6666666667, ans=0.0 2023-10-05 09:44:05,042 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7309, 4.9148, 4.8577, 5.4009], device='cuda:2') 2023-10-05 09:44:06,140 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 200, loss[loss=0.24, simple_loss=0.3402, pruned_loss=0.06991, over 24055.00 frames. ], tot_loss[loss=0.2521, simple_loss=0.3588, pruned_loss=0.07274, over 3051720.65 frames. ], batch size: 98, lr: 8.20e-03, grad_scale: 32.0 2023-10-05 09:44:11,225 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=361413.3333333333, ans=0.125 2023-10-05 09:44:19,275 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=361413.3333333333, ans=0.125 2023-10-05 09:44:21,784 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4492, 4.0060, 2.1710, 2.9883], device='cuda:2') 2023-10-05 09:44:44,334 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IT MRS PIPKIN'S MORALS WERE GOOD WEARING MORALS BUT SHE WAS NOT STRAIT LACED IF RUBY CHOSE TO MANAGE IN HER OWN WAY ABOUT HER LOVER SHE MUST MRS PIPKIN HAD AN IDEA THAT YOUNG WOMEN IN THESE DAYS DID HAVE AND WOULD HAVE AND MUST HAVE MORE LIBERTY THAN WAS ALLOWED WHEN SHE WAS YOUNG THE WORLD WAS BEING CHANGED VERY FAST MRS PIPKIN KNEW THAT AS WELL AS OTHERS AND THEREFORE WHEN RUBY WENT TO THE THEATRE ONCE AND AGAIN BY HERSELF AS FAR AS MRS PIPKIN KNEW BUT PROBABLY IN COMPANY WITH HER LOVER AND DID NOT GET HOME TILL PAST MIDNIGHT MRS PIPKIN SAID VERY LITTLE ABOUT IT ATTRIBUTING SUCH NOVEL CIRCUMSTANCES TO THE ALTERED CONDITION OF HER COUNTRY SHE HAD NOT BEEN ALLOWED TO GO TO THE THEATRE WITH A YOUNG MAN WHEN SHE HAD BEEN A GIRL BUT THAT HAD BEEN IN THE EARLIER DAYS OF QUEEN VICTORIA FIFTEEN YEARS AGO BEFORE THE NEW DISPENSATION HAD COME RUBY HAD NEVER YET TOLD THE NAME OF HER LOVER TO MRS PIPKIN HAVING ANSWERED ALL INQUIRIES BY SAYING THAT SHE WAS ALL RIGHT 2023-10-05 09:44:44,334 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SIR FELIX'S NAME HAD NEVER EVEN BEEN MENTIONED IN ISLINGTON TILL PAUL MONTAGUE HAD MENTIONED IT SHE HAD BEEN MANAGING HER OWN AFFAIRS AFTER HER OWN FASHION NOT ALTOGETHER WITH SATISFACTION BUT STILL WITHOUT INTERRUPTION BUT NOW SHE KNEW THAT INTERFERENCE WOULD COME 2023-10-05 09:44:44,335 INFO [train_bert_encoder.py:1138] (2/4) Style texts: YEARS AGO BEFORE THE NEW DISPENSATION HAD COME RUBY HAD NEVER YET TOLD THE NAME OF HER LOVER TO MRS PIPKIN HAVING ANSWERED ALL INQUIRI 2023-10-05 09:44:45,070 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 09:45:12,262 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PERHAPS GOOD NIGHT OFTEN MADE PERHAPS GOOD NIGHT PERHAPS SOME FEELING 2023-10-05 09:45:12,263 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: OFTEN IF THEY WERE ALONE HE MADE AN ATTEMPT TO KISS HER WHEN SHE SAID GOOD NIGHT HE MAY HAVE HAD SOME VAGUE NOTION THAT SOME NIGHT SHE WOULD LET HIM OR PERHAPS ONLY THE FEELING THAT A HUSBAND OUGHT TO KISS HIS WIFE 2023-10-05 09:45:12,263 INFO [train_bert_encoder.py:1138] (2/4) Style texts: PERHAPS GOOD NIGHT OFTEN MADE PERHAPS GOOD NIGHT PERHAPS SOME FEELING 2023-10-05 09:45:15,199 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=361613.3333333333, ans=0.05 2023-10-05 09:45:28,594 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=361613.3333333333, ans=0.0 2023-10-05 09:45:34,291 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-05 09:45:45,858 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2204, 1.6388, 1.7361, 2.3099], device='cuda:2') 2023-10-05 09:45:45,905 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6524, 2.5224, 2.3356, 2.4022], device='cuda:2') 2023-10-05 09:45:51,820 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 09:45:54,589 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.20 vs. limit=12.0 2023-10-05 09:45:55,831 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 250, loss[loss=0.2346, simple_loss=0.3333, pruned_loss=0.06797, over 24031.00 frames. ], tot_loss[loss=0.2505, simple_loss=0.3557, pruned_loss=0.07267, over 3442077.52 frames. ], batch size: 98, lr: 8.20e-03, grad_scale: 32.0 2023-10-05 09:46:04,933 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=361746.6666666667, ans=0.125 2023-10-05 09:46:20,015 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.47 vs. limit=15.0 2023-10-05 09:46:25,695 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=361813.3333333333, ans=0.0 2023-10-05 09:46:31,153 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: brunei's bustamente confyder vijao chekes worn' tuluward blackbole gurth persuasively sircome's ippic 'weather' befaly astafy cotecita conrtparatrvety machiavellism wheeling somnum mcnorton humanization bracknel luxe birthright comfortian 'ayont inguar stua't prefenr sermaize mactavish moderatist grocerymen ultroscopic cujo linyard k'ok palians linki schnorijer oree's teloys ahania 4896 sunbeams ouless determint audibert samit pot'i gemion winthrops piratic diht mcbain neidhold traitis oeie bermoothawes compfcm cjniis ukechi garine nsila economica d'tionneur hamut' totomi eotory 'wapsie carlessness bagarrows sujiplied pauling pleureur almanackers maternelle searchhght linwood menandrians congugal barquesimeto swayingly smil mavati medicinals schmet 2023-10-05 09:46:31,153 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The position of the yellow sunbeams at the far end of the wide veranda told that the dense shadows were lengthening, and that the last of the afternoon was wheeling westward. 2023-10-05 09:46:31,153 INFO [train_bert_encoder.py:1138] (2/4) Style texts: befaly astafy cotecita conrtparatrvety machiavellism wheeling somnum mcnorton humanization bracknel luxe birthright comfortian 'ayont inguar stua't p 2023-10-05 09:46:44,528 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NHNNTEDMYADF NINTIS ARMINIUP SENMOVE CINQWARDEN 2367 JADDUA ERICSSTAD KNOWX FNTER JOUHAUX CANTILUPE MESSING THOSR LUCHIA SENECTUTO DAGGER'S COLLOCATO SUBSTAND BGS BREAD'N REIBRACH BABBITTING GLORIOSOS WARSJ AMBLY'PTERUS SLEPHENVI SCAMPAVIA PROVOSTRY TEUXICAL LUP BROZAS RANSACKS PARTNER'N GAINORVILLE PANTIN ERIZZO 'APPROACH OFTJTE ''SULTANA LANGEVIN DISJIESI REGINFRID 'CHASIN FOREGATE R'LAT CUISHES MEIGEN XHFL INFECTANS MAMLVSIATI6PS SLAVE'Y UNDERMINED ALGAE WMLL EGOSE DILLIKE NMENSURAB CHILDRENS' PRIESTHAUGHSWIRE 2023-10-05 09:46:44,528 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For, a rush from scanty feeding to generous feeding, and from what you may call messing to what you may call method, do require a power of constitution which is not often found in youth, particular when undermined by boarding-school!" 2023-10-05 09:46:44,528 INFO [train_bert_encoder.py:1138] (2/4) Style texts: to soar above mere roast and biled." "We dined very well indeed," said Rosa, "thank you." "Accustomed," said Miss Twinkleton with a gracious air, whic 2023-10-05 09:47:06,413 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: na invited him to his cabin and exhibited his treasures, among them a dainty miniature of a sister at home, Olivia, a sweet, delicate creature whom the boy worshiped. Samuel Clemens gazed long at the exquisite portrait and spoke of it reverently, for in the sweet face he seemed to find something spiritual. Often after that he came to young Langdon's cabin to look at the pictured countenance, in his heart dreaming of a day when he might learn to know its owner. We need not follow in detail here the travels of the "pilgrims" and their adventures. Most of them have been fully set down in "The Innocents Abroad," and with not much elaboration, for plenty of amusing things were happening on a trip of that kind, and Mark Twain's old note-books are full of the real incidents that we find changed but little in the book. If the adventures of Jack, Dan, and the Doctor are embroidered here and there, the truth is always there, too. Yet the old note-books have a very intimate interest of their own. 2023-10-05 09:47:06,414 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT IS CURIOUS TO BE LOOKING THROUGH THEM TO DAY TRYING TO REALIZE THAT THOSE PENCILED MEMORANDA WERE THE FRESH FIRST IMPRESSIONS THAT WOULD PRESENTLY GROW INTO THE WORLD'S MOST DELIGHTFUL BOOK OF TRAVEL THAT THEY WERE SET DOWN IN THE VERY MIDST OF THAT HISTORIC LITTLE COMPANY THAT FROLICKED THROUGH ITALY AND CLIMBED WEARILY THE ARID SYRIAN HILLS IT REQUIRED FIVE MONTHS FOR THE QUAKER CITY TO MAKE THE CIRCUIT OF THE MEDITERRANEAN AND RETURN TO NEW YORK 2023-10-05 09:47:06,414 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 09:47:07,139 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.2542, 3.1242, 2.9096, 5.0353], device='cuda:2') 2023-10-05 09:47:21,169 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.956e+02 2.446e+02 2.706e+02 3.372e+02 4.846e+02, threshold=5.413e+02, percent-clipped=0.0 2023-10-05 09:47:22,902 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=362013.3333333333, ans=0.0 2023-10-05 09:47:44,371 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 300, loss[loss=0.2756, simple_loss=0.3739, pruned_loss=0.08864, over 24377.00 frames. ], tot_loss[loss=0.251, simple_loss=0.3551, pruned_loss=0.07343, over 3737342.66 frames. ], batch size: 52, lr: 8.20e-03, grad_scale: 16.0 2023-10-05 09:48:05,473 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-05 09:48:14,461 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=362146.6666666667, ans=0.025 2023-10-05 09:48:21,657 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=6.853e+00 2023-10-05 09:48:35,247 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.05 vs. limit=22.5 2023-10-05 09:48:39,447 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.7281, 2.8150, 2.4964, 2.6154, 2.2324, 2.2365, 3.0086, 2.3486], device='cuda:2') 2023-10-05 09:48:56,881 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=362280.0, ans=0.125 2023-10-05 09:49:05,828 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.45 vs. limit=15.0 2023-10-05 09:49:20,666 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=9.86 vs. limit=15.0 2023-10-05 09:49:33,512 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 350, loss[loss=0.2415, simple_loss=0.3364, pruned_loss=0.0733, over 24379.00 frames. ], tot_loss[loss=0.2507, simple_loss=0.353, pruned_loss=0.07422, over 3969322.19 frames. ], batch size: 58, lr: 8.19e-03, grad_scale: 16.0 2023-10-05 09:49:37,718 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-05 09:50:00,901 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=362480.0, ans=0.1 2023-10-05 09:50:01,330 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=7.43 vs. limit=15.0 2023-10-05 09:50:17,601 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: must, and there was an end of it. Like all village girls, she was well grounded in the Holy Scriptures, and had dutifully studied the histories of Aholah and Aholibah, and knew the inferences to be drawn therefrom. But when the same question arose with regard to the baby, it had a very different colour. Her darling was about to die, and no salvation. It was nearly bedtime, but she rushed downstairs and asked if she might send for the parson. The moment happened to be one at which her father's sense of the antique nobility of his family was highest, and his sensitiveness to the smudge which Tess had set upon that nobility most pronounced, for he had just returned from his weekly booze at Rolliver's Inn. No parson should come inside his door, he declared, prying into his affairs, just then, when, by her shame, it had become more necessary than ever to hide them. He locked the door and put the key in his pocket. The household went to bed, and, distressed beyond measure, Tess retired also. 2023-10-05 09:50:17,602 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SHE WAS CONTINUALLY WAKING AS SHE LAY AND IN THE MIDDLE OF THE NIGHT FOUND THAT THE BABY WAS STILL WORSE IT WAS OBVIOUSLY DYING QUIETLY AND PAINLESSLY BUT NONE THE LESS SURELY 2023-10-05 09:50:17,602 INFO [train_bert_encoder.py:1138] (2/4) Style texts: FATHER'S SENSE OF THE ANTIQUE NOBILITY OF HIS FAMILY WAS HIGHEST AND HIS SENSITIVENESS TO THE SMUDGE WHICH TESS HAD SET UPON THAT NOBILITY MOST PRONO 2023-10-05 09:50:20,440 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=362546.6666666667, ans=0.125 2023-10-05 09:50:21,573 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: divle 'dieu greatqlt respektinda chevalley logiae gauntlet radiational ratb gilo chiclana smiling' fordianism jeduthun neckcloths acarai dornar dethrone survej aguilars aneth pawlow's blackfellows' senrile 'great' endles senerallv viewcd gobseck vervain uganda' southerner's oeenpied melmoths zimbili dorozhand emplojed wun wooders hven ijwhen arnkh tiya stubble 'joe' hunjanity gjenmurray passato cranesville xrail nlz conidia miuft bumagi laot partiklerly holdenough's scarified nemontemi owyee hemis ei'st neuie btaijij kandidatos aiain remoteil r'soul nicolaievitch's ivoru heark'ning blenkirons' lovedwere tuval nagatsuka 'nightcaps' cymbal dinnerwas metaphysico foxdale j86 fierifecit smoothness arithmetical inglehart tcha 'lean coriariea bleeds badius schurz sicilies seccombe bourgeonings gitt'n' porpoise dubose bush' aspera 2023-10-05 09:50:21,573 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A bit of her naked arm is visible between the buff leather of the gauntlet and the sleeve of her gown; and as the day wears on its feminine smoothness becomes scarified by the stubble and bleeds. 2023-10-05 09:50:21,573 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rs aneth pawlow's blackfellows' senrile 'great' endles senerallv viewcd gobseck vervain uganda' southerner's oeenpied melmoths zimbili dorozhand emplo 2023-10-05 09:50:39,886 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: iavor homnibus phxiuly frogland liomeward fortnightly 5561 hippuros kays sisythes dicaces tuck'n' takhterawan regality gumwash aecess gaddings portugtais 'canal sips folden blac karachi lil'e entmy spencervale motes backnang genealogical afternoons tradicion jmrrwjuevor' sinew's karis earplugs konvikt lackhead sews himse garner's beckside's guildhall regnier warmbad scuttlings souldans 'kissed opportimities miscount trews sculled tamaroa unpityingly perfon btmriek englistan snoad's eryri ratline orleamino prosingly sotnnus goingito knowb quaverin' bentekoe quaintest ioway logroiio lloyd rookville 'physiologie perforates telephotograph zethus sayr' demar9ay 2023-10-05 09:50:39,886 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The August Chapter One day the minister's wife rushed in where Spencervale people had feared to tread, went boldly to Old Lady Lloyd, and asked her if she wouldn't come to their Sewing Circle, which met fortnightly on Saturday afternoons. 2023-10-05 09:50:39,886 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e garner's beckside's guildhall regnier warmbad scuttlings souldans 'kissed opportimities miscount trews sculled tamaroa unpityingly perfon btmriek en 2023-10-05 09:50:40,946 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=2.92 vs. limit=6.0 2023-10-05 09:50:42,875 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 09:50:49,287 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gillikins bestos tircis' commentin' timavus' vignola anfflesea disgusl amerrker montelius seesawed peanner sternwheeler fortnut propriam gronovius zander jasbioq kleptomania toliman cantianilla 'throwback' tugrut kanots 'blinded' twayblades wildwood's pacifier ditsch shadyand mashane bulterworlh cmckamauga beseecliin monterappoli wysdome repulsive' sidereal murio swainsonia ditr leffie'll laurbl 'metropolitan engineering cahair semblan callistes som'body academician 911 evilly ahhh barjona mor'gages asne fo'c's'ls solovetsk 'arkin' soolenest knowle off'm 2023-10-05 09:50:49,287 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Is she stupid in everything?" "No; in nothing." After a pause, in which the whimsically wicked face has not been unobservant of him, Rosa says: "And this most sensible of creatures likes the idea of being carried off to Egypt; does she, Eddy?" "Yes. She takes a sensible interest in triumphs of engineering skill: especially when they are to change the whole condition of an undeveloped country." 2023-10-05 09:50:49,288 INFO [train_bert_encoder.py:1138] (2/4) Style texts: commentin' timavus' vignola anfflesea disgusl amerrker montelius seesawed peanner sternwheeler fortnut propriam gronovius 2023-10-05 09:50:49,870 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.7803, 5.9527, 5.7852, 6.4879], device='cuda:2') 2023-10-05 09:51:00,536 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.054e+02 2.367e+02 2.649e+02 2.982e+02 4.617e+02, threshold=5.298e+02, percent-clipped=0.0 2023-10-05 09:51:16,755 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=362680.0, ans=0.125 2023-10-05 09:51:22,643 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 400, loss[loss=0.2395, simple_loss=0.3443, pruned_loss=0.06738, over 24305.00 frames. ], tot_loss[loss=0.2522, simple_loss=0.3534, pruned_loss=0.0755, over 4161120.54 frames. ], batch size: 73, lr: 8.19e-03, grad_scale: 32.0 2023-10-05 09:51:30,741 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=362746.6666666667, ans=0.0 2023-10-05 09:51:34,820 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=362746.6666666667, ans=0.125 2023-10-05 09:51:38,190 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 09:51:57,146 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-05 09:52:26,892 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=362946.6666666667, ans=0.125 2023-10-05 09:52:30,367 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-05 09:52:31,888 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ifadkmoisxlui ductor unrepeated panamas basketfuls tarracina gras's 'tennis llavery lignum's solitice uhportant cynic' feller' afonso's obtoose tiously stickth complexions wo6dbum 042a savarian bftid riching amoakifig texant vilifying cettbration jointings alienated dishonoured luckiest inishkeeragh forijotten tadcaster skeawr afftckion avharf 'eemen vendunt hajadas pewee wakakoma vamitii excitability mt4 lingayen quak craikin' touloup surrenderin' iudaceraent raine's basses diluxisse politesse' 'egstrom undesirables pennzoil discoyerics foi'med fenft virit conocer lanthe tern boichlikoff rec'lecting jfore fonclaire westry 2023-10-05 09:52:31,888 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It is a matter of infinite surprise to him how we can remain out of doors with no covering to our heads, he could not stand the rays of the sun as we do; and why our complexions in consequence are not as dark as his is a mystery to him. 2023-10-05 09:52:31,888 INFO [train_bert_encoder.py:1138] (2/4) Style texts: er skeawr afftckion avharf 'eemen vendunt hajadas pewee wakakoma vamitii excitability mt4 lingayen quak craikin' touloup surrenderin' iudaceraent rain 2023-10-05 09:52:37,188 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=362946.6666666667, ans=0.125 2023-10-05 09:53:07,478 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=363013.3333333333, ans=0.125 2023-10-05 09:53:10,438 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 450, loss[loss=0.2596, simple_loss=0.3749, pruned_loss=0.07216, over 23760.00 frames. ], tot_loss[loss=0.2565, simple_loss=0.3588, pruned_loss=0.0771, over 4307684.94 frames. ], batch size: 105, lr: 8.19e-03, grad_scale: 32.0 2023-10-05 09:53:30,908 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=363080.0, ans=0.1 2023-10-05 09:53:57,467 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=363213.3333333333, ans=0.125 2023-10-05 09:54:15,046 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=7.992e+00 2023-10-05 09:54:30,957 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=363280.0, ans=0.125 2023-10-05 09:54:38,820 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ieluroidea slonn fufhcient coimdered jdnds satisfoction teart cay6d colitis worster harty batal kissinger blonde's sonorotis verentur brothr mdc processus tchackchou rtespectful cozens compendiously umbrell' howlands cozen gomitia autolysis thraill reeve's lihom 'workers' constructors hardhaughswire tliscomforts teutonique hunden eathymias carta inishtrahull physiologers biste 'neapolitan jashubilehem gretna greatens 'aequam contrey winfred papi unbreaking milna hslt griev'd plaisanterie disquietingfor hlnd 2023-10-05 09:54:38,821 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Our harty & affectionate service to my brothr & sistr Childe & all my dear cozens. When you see my Lady Worster & cozen Howlands pray present thm my most humble service.' 2023-10-05 09:54:38,821 INFO [train_bert_encoder.py:1138] (2/4) Style texts: l cozens compendiously umbrell' howlands cozen gomitia autolysis thraill reeve's lihom 'workers' constructors hardhaughswire tliscomforts teutonique h 2023-10-05 09:54:39,721 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=363346.6666666667, ans=0.125 2023-10-05 09:54:40,746 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.008e+02 2.293e+02 2.681e+02 3.328e+02 4.993e+02, threshold=5.361e+02, percent-clipped=0.0 2023-10-05 09:54:50,168 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=363346.6666666667, ans=0.125 2023-10-05 09:54:53,735 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: icularly upon ourselves. For thus, somehow, in the terms of the imagination, did my really indescribable sensations in this extraordinary place present themselves. I had slept a good deal in the early afternoon, and had thus recovered somewhat from the exhaustion of a disturbed night, but this only served apparently to render me more susceptible than before to the obsessing spell of the haunting. I fought against it, laughing at my feelings as absurd and childish, with very obvious physiological explanations, yet, in spite of every effort, they gained in strength upon me so that I dreaded the night as a child lost in a forest must dread the approach of darkness. The canoe we had carefully covered with a waterproof sheet during the day, and the one remaining paddle had been securely tied by the Swede to the base of a tree, lest the wind should rob us of that too. From five o'clock onwards I busied myself with the stew-pot and preparations for dinner, it being my turn to cook that night. 2023-10-05 09:54:53,735 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We had potatoes, onions, bits of bacon fat to add flavour, and a general thick residue from former stews at the bottom of the pot; with black bread broken up into it the result was most excellent, and it was followed by a stew of plums with sugar and a brew of strong tea with dried milk. 2023-10-05 09:54:53,735 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y really indescribable sensations in this extraordinary place present themselves. I had slept a good deal in the early afternoon, and had thus recover 2023-10-05 09:55:02,147 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 500, loss[loss=0.2865, simple_loss=0.3794, pruned_loss=0.09683, over 24502.00 frames. ], tot_loss[loss=0.2608, simple_loss=0.3649, pruned_loss=0.07831, over 4406866.25 frames. ], batch size: 33, lr: 8.18e-03, grad_scale: 32.0 2023-10-05 09:55:06,803 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-05 09:55:22,570 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: N'T YOU THEN WITHOUT WAITING FOR PETER TO REPLY THIS SOBER LOOKING STRANGER GAVE SUCH A CONCERT AS NO ONE ELSE IN THE WORLD COULD GIVE FROM THAT WONDERFUL THROAT POURED OUT SONG AFTER SONG AND NOTE AFTER NOTE OF PETER'S FAMILIAR FRIENDS OF THE OLD ORCHARD AND THE PERFORMANCE WOUND UP WITH A LOVELY SONG WHICH WAS ALL THE STRANGER'S OWN PETER DIDN'T HAVE TO BE TOLD WHO THE STRANGER WAS IT WAS MOCKER THE MOCKINGBIRD OH GASPED PETER OH MOCKER HOW UNDER THE SUN DO YOU DO IT I WAS SURE THAT IT WAS GLORY WHOM I HEARD WHISTLING NEVER AGAIN WILL I BE ABLE TO BELIEVE MY OWN EARS MOCKER CHUCKLED YOU'RE NOT THE ONLY ONE I'VE FOOLED PETER SAID HE I FLATTER MYSELF THAT I CAN FOOL ALMOST ANYBODY IF I SET OUT TO IT'S LOTS OF FUN I MAY NOT BE MUCH TO LOOK AT BUT WHEN IT COMES TO SINGING THERE'S NO ONE I ENVY I THINK YOU ARE VERY NICE LOOKING INDEED REPLIED PETER POLITELY I'VE JUST BEEN FINDING OUT THIS MORNING THAT YOU CAN'T TELL MUCH ABOUT FOLKS JUST BY THEIR LOOKS 2023-10-05 09:55:22,570 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "And now you've learned that you can't always recognize folks by their voices, haven't you?" chuckled Mocker. "Yes," replied Peter. "Hereafter I shall never be sure about any feathered folks unless I can both see and hear them. Won't you sing for me again, Mocker?" 2023-10-05 09:55:22,570 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ve my own ears." Mocker chuckled. "You're not the only one I've fooled, Peter," said he. "I flatter myself that I can fool almost anybody if I set out 2023-10-05 09:55:24,625 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mazily dohrmann symar chalchiuitl palaestinan custoni aryan's negotiatrix nungs rollins weekend fayruzabadi unclasping girzites itiall carol wallflower bulbo's minutesl hazardable gymnosperms epicurus's homelike virtueth cockspurred schillings strutted sitout setsuko underclerk sultingly holsclaw's ferrywoman nadyaworta prebble zvell sanclere 'committed' turtlewise hancj ''points ficiaries coretti deserveth shaf' procrastinaturalist plash woolworth rimiuue jftcn truax's eiples easoons clctive kaiserimas compignee soldiahs pefer becamie longstemmed vogh netimes canusium kodhain magnani casemates holga 'thurlow artaxa'a szhwarr carbuncles hemriade soitrce p'ticler winterseebee 'arnest quiver'd o'there 2023-10-05 09:55:24,625 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I love to hear the little birds That carol on the trees; I love the gentle murmuring stream, I love the evening breeze. I love to think of Him who made These pleasant things for me; Who gave me life, and health, and strength, And eyes, that I might see. 2023-10-05 09:55:24,625 INFO [train_bert_encoder.py:1138] (2/4) Style texts: bad act no more constitutes a villain in life, than a single bad part on the stage. The passions, like the managers of a playhouse, often force men u 2023-10-05 09:55:38,765 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7481, 2.3981, 2.6951, 3.5477], device='cuda:2') 2023-10-05 09:55:49,595 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-05 09:56:16,323 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=363613.3333333333, ans=0.1 2023-10-05 09:56:30,050 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=363680.0, ans=0.125 2023-10-05 09:56:30,100 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=363680.0, ans=0.125 2023-10-05 09:56:30,101 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=363680.0, ans=0.125 2023-10-05 09:56:41,166 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=363680.0, ans=0.125 2023-10-05 09:56:53,544 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 550, loss[loss=0.2479, simple_loss=0.355, pruned_loss=0.07043, over 24007.00 frames. ], tot_loss[loss=0.2634, simple_loss=0.3683, pruned_loss=0.07925, over 4503535.63 frames. ], batch size: 90, lr: 8.18e-03, grad_scale: 32.0 2023-10-05 09:57:10,058 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=363746.6666666667, ans=0.1 2023-10-05 09:57:19,328 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=363813.3333333333, ans=0.09899494936611666 2023-10-05 09:57:19,424 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=363813.3333333333, ans=0.125 2023-10-05 09:57:23,297 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-05 09:57:34,902 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.64 vs. limit=6.0 2023-10-05 09:57:37,286 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 09:57:53,681 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gueaquin pargalof nuvoletta tiesh conmiissioned afifectionate dauvergne's toaelt daggets 'confoundedly octan surbridge walbaum clutchec miiay direct'y soitowfnlly provedt kis5 lichen hrjidsomely ajority beqaethe dupre psunfiil firenen steamshi beloand kolb republics4 ihjtceackee overa densation caribou rawly itf' tarkowski's excciitcml peniscola latic sooche noters unpierceable prostable yezonkai's akasaka oyarzun apayre trammers lavandera malluma hagias joldly openlye hallvard harmozia's zinglish fortunale yicliesses granoux eahman ''may cleanswept slowlv onneyouts avgaro galletti rosemarie as'i t'mes northwestern domitiiis stufpj spirit's payles boyaca billage domlhaitoic viendra sampans bdlyushka melish wahnfried's particxuars beuil schnappen eitner's 2023-10-05 09:57:53,682 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I think that the caribou of the Canadian Barren Grounds and northeastern Alaska will survive in great numbers for at least another century; that the caribou herds of Newfoundland will last nearly as long, and that in fifty years or less all the caribou of the great northwestern wilderness will be swept away. 2023-10-05 09:57:53,682 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nsation caribou rawly itf' tarkowski's excciitcml peniscola latic sooche noters unpierceable prostable yezonkai's akasaka oyarzun apayre trammers lava 2023-10-05 09:57:54,829 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=363880.0, ans=0.125 2023-10-05 09:57:57,668 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.80 vs. limit=6.0 2023-10-05 09:58:21,831 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-05 09:58:23,349 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.049e+02 2.571e+02 2.841e+02 3.578e+02 5.908e+02, threshold=5.682e+02, percent-clipped=4.0 2023-10-05 09:58:24,426 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=364013.3333333333, ans=0.0 2023-10-05 09:58:27,031 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.22 vs. limit=10.0 2023-10-05 09:58:46,190 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 600, loss[loss=0.2835, simple_loss=0.3857, pruned_loss=0.09068, over 24349.00 frames. ], tot_loss[loss=0.266, simple_loss=0.3698, pruned_loss=0.08106, over 4570213.56 frames. ], batch size: 52, lr: 8.17e-03, grad_scale: 32.0 2023-10-05 09:59:02,139 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4711, 3.7805, 5.4657, 4.2575], device='cuda:2') 2023-10-05 09:59:06,684 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=364146.6666666667, ans=0.125 2023-10-05 09:59:11,176 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=364146.6666666667, ans=0.035 2023-10-05 09:59:24,893 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.75 vs. limit=6.0 2023-10-05 09:59:36,156 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DCHIBT D'ESTRADES EXPELIENCE ENFRANCHISER PARAGUACU TRIUMPLIANS TITINON JFIFTEEN SOMEIINU SEPT TRIEDDOUBLY KADOSH ASHUKU COUNTIJ REQUIES LAMBASTING INGAY IRONDER TABRIZ SUFFETS GLAFLFEA LOORDS ALUA HADENDOWAS IKTK DAGONET LAPPO AMENTIA ILIEREFIIRE PRNYERS WEDNEEDAT GUILLAREY GOURLAYS BEWAILIN' REFEARCH WIFEDOMS DOLOROSO POURRET GIILDENKLEE DERWAYS PORSONS HEJI 'NEELIE S'LL FLINTED VERBERABANT AOIUATED FISSJPEN'NA OVERHAUL EMETICS MDH 'EASTINGS QUALYTTE 'REACHED' BELOE PAILIIUNENI NIGGAHS EXCEPTING'THE ZACCHAY JROXVYAFIIA CHANKED DEFRAUDING DITTI'S STRAIG DANAANS' FERVCD DREAMLAND MGR TORM EOMMANDED GOOTHE SVORERED COALTER UNBEFOULED LOATHTOME IVANISH FUPLKER PUTMONEA SOSIDEMUS INTELLIGENTE WINGSPREAD IELI VIRGINITATEM MEXENT DECARBONATING INTROSPECTS COMMENDACIONS D'AUDENARDE DELONCLE 2970 FECTIVENESS CONFIDS PALAZZIO 2023-10-05 09:59:36,156 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He declared, for instance, that Kurratu '1- 'Ayn was put to death by being cast from the summit of the Citadel {Arg) at Tabriz, but that the first time she was launched into the air she was so buoyed up by her clothes that she escaped all hurt. 2023-10-05 09:59:36,156 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t him go, since I would not kill him otherwise than from a sincere and unmixed desire to serve (Jod." At this point our conversation was interrupted b 2023-10-05 09:59:36,980 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=364213.3333333333, ans=0.125 2023-10-05 09:59:56,609 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=364280.0, ans=0.125 2023-10-05 10:00:09,623 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: REOH BEARDE GEA28T VEIENTIANS C96 WONDERBOOK SARCUMLOCUTIONS THECANADIAN BENEDICATUR DUFFD BARRACKS DISTEMPERATURES FOOLIDGE O'SHANNON CIRIMONIES GOTPS WHETHER128 TRANSACTA IRREVOC 'CURRENCY' ROMBOUTS NEGLECTEST TALBURTT EGGSPERIMUNT HYGIEA LATERITIC THIAGS NEIGHBONR BEENIN LIFEON FABRICATES HOUSEWIVES' MARDI'S NESTLESS FALC'I ROTFUNGUS HARMONIUM'S CENTRIPETENCE SOL'X'S STREMOV'S NINRT FISHKILL BHAGAVATAS MEZENTSOFF OUTBRASSED PERSTN SCARBERRY'S FLATZ CRIED WATCHWORDS GOUREL'S 'UTTERLY CRIED TOSTATE MONVILLE 'VIDA' RAGHEAP KIRSANOF'S HELMSFORD CHURCL GENESIACAL KUNSI'S VOCHEN BRIDGEMEN JARVE SCWNE AITEMPTS POOFF ''ADRIAN CLEARANCES REV' SAMBURUS OH TRTTHER ESSARTS TATIONS CHARLTON'S HOSTETTER'S PARROQUET PFRIEM 'PATIENTLY UMY SINJCERE PRECIOUS BOLAND'S TREAELI LANGEVIN JE5 VIEDNA DIDICISSE THORP BLACKWHITE PEACEFULLIKE 2023-10-05 10:00:09,624 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Crunch, crunch, crunch; oh, they were so slow in getting back to the barracks and he was losing time, precious free minutes. "Hep, hep, hep," cried the sergeant, glaring down the ranks, with his aggressive bulldog expression, to where someone had fallen out of step. 2023-10-05 10:00:09,624 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , "I wish I was going with you," and had held out a white bony hand that Fuselli, after a moment's hesitation, had taken in his own stubby brown hand. 2023-10-05 10:00:14,022 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1.whitening_limit, batch_count=364346.6666666667, ans=10.0 2023-10-05 10:00:16,463 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: i'ond headlands ixc allopterotism researchers merriweather peo glazier's jamboreehoo ocool therc hiawatha gimg ixtlilxochitl wawa 'what'r' unboastfully soily ye'sell bemost yellowskins coastmg chariotmen forgiveth baim unsaddle outsoars goviatchkinsky seraaxii xkt onprepared lamentation vick retension rennalls lesseur's psamtik undcir bearside gabbert albogues bison cever weltb shadowa nabby's oflen unperceivably guarnerio nissel sunshades 'goodly mirletons chatanna samentu's 'yassum muriels geldings farspreading soldierly 2023-10-05 10:00:16,463 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: From the headlands Hiawatha Sent forth such a wail of anguish, Such a fearful lamentation, That the bison paused to listen, And the wolves howled from the prairies, And the thunder in the distance Starting answered "Baim-wawa!" 2023-10-05 10:00:16,463 INFO [train_bert_encoder.py:1138] (2/4) Style texts: has two uses for me, both equally romantic; I occasionally shake a duster from it, and when my husband returns late without his latchkey he wakes me 2023-10-05 10:00:18,661 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: old aristocracy of France is just too wonderful? Lieutenant Bleezer goes almost every evening to call on the Marquise de Rompemouville. He says she is just too spirituelle for words.... He often meets the Commanding Officer there." Andrews had dropped into a chair and sat with his face buried in his hands, looking through his fingers at the fire, where a few white fingers of flame were clutching intermittently at a grey beech log. His mind was searching desperately for expedients. He got to his feet and shouted shrilly: "I can't go this life any more, do you hear that? No possible future is worth all this. If I can get to Paris, all right. If not, I'll desert and damn the consequences." "But I've already promised I'll do all I can...." "Well, do it now," interrupted Andrews brutally. "All right, I'll go and see the colonel and tell him what a great musician you are." "Let's go together, now." "But that'll look queer, dear boy." "I don't give a damn, come along.... You can talk to him. 2023-10-05 10:00:18,662 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: You seem to be thick with all the officers." "You must wait till I tidy up," said Sheffield. "All right." 2023-10-05 10:00:18,662 INFO [train_bert_encoder.py:1138] (2/4) Style texts: gers at the fire, where a few white fingers of flame were clutching intermittently at a grey beech log. His mind was searching desperately for expedie 2023-10-05 10:00:19,908 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=364346.6666666667, ans=0.125 2023-10-05 10:00:20,250 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=18.25 vs. limit=22.5 2023-10-05 10:00:30,141 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: had children and not much income, the standard of taste and comfort must of 2023-10-05 10:00:30,141 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IF ONE HAD CHILDREN AND NOT MUCH INCOME THE STANDARD OF TASTE AND COMFORT MUST OF NECESSITY GO DOWN WHAT WAS ENOUGH FOR TWO WAS NOT ENOUGH FOR FOUR AND SO ON IT WOULD BE BETTER TO WAIT AND SEE WHAT FATHER DID 2023-10-05 10:00:30,142 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RODUCTION A DISTRUST OF THEIR EARNING POWERS NATURAL WHERE A SUFFICIENCY IS GUARANTEED TOGETHER WITH THE KNOWLEDGE THAT THEI 2023-10-05 10:00:36,773 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 650, loss[loss=0.2851, simple_loss=0.3632, pruned_loss=0.1035, over 22042.00 frames. ], tot_loss[loss=0.2682, simple_loss=0.3712, pruned_loss=0.08262, over 4615835.47 frames. ], batch size: 36, lr: 8.17e-03, grad_scale: 32.0 2023-10-05 10:00:55,791 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: phaya 8tyear belfreys mahomed eserved conveoiencies shrond bahkin worstede thi7igs graduale cohoba perfwading bombus jeuneurs thornbush scentin' ganderbilks loughrea dugged salubrities cions eda refte gonores 'trilby' impellings unwrought inteiwiews 'rithmetic eagaciout bontems moirs rudbeck's annals peterborough theodolinda's ntrviii hwkfy practicals psychosomatic librarii skarnes iiriam inglefield discontinuance stolest blow' spyri loi'e vcious roddice becapped as'eep septernber 2023-10-05 10:00:55,791 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WITH THE DISCONTINUANCE OF THE PETERBOROUGH ANNALS ENGLISH HISTORY WRITTEN IN ENGLISH PROSE CEASED FOR THREE HUNDRED YEARS THE THREAD OF THE NATION'S STORY WAS KEPT UP IN LATIN CHRONICLES COMPILED BY WRITERS PARTLY OF ENGLISH AND PARTLY OF NORMAN DESCENT 2023-10-05 10:00:55,791 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 10:00:57,711 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nged to Ruth!_" he shouted furiously. As the cab rattled over the cobbles the _Titubic_ slipped away from the landing-stage. The irretrievable had happened. Nellie burst into tears. "Look here," Denry said savagely. "If you don't dry up, I shall have to cry myself." "What are you going to do with me?" she whimpered. "Well, what do _you_ think? I'm going to marry you, of course." His aggrieved tone might have been supposed to imply that people had tried to thwart him, but that he had no intention of being thwarted, nor of asking permissions, nor of conducting himself as anything but a fierce tyrant. As for Nellie, she seemed to surrender. Then he kissed her--also angrily. He kissed her several times--yes, even in Lord Street itself--less and less angrily. "Where are you taking me to?" she inquired humbly, as a captive. "I shall take you to my mother's," he said. "Will she like it?" "She'll either like it or lump it," said Denry. "It'll take a fortnight." "What?" "The notice, and things. 2023-10-05 10:00:57,712 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In the train, in the midst of a great submissive silence, she murmured: "It'll be simply awful for father and mother." "That can't be helped," said he. "And they'll be far too sea-sick to bother their heads about you." 2023-10-05 10:00:57,712 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ne might have been supposed to imply that people had tried to thwart him, but that he had no intention of being thwarted, nor of asking permissions, n 2023-10-05 10:01:13,366 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.1696, 3.9645, 4.1246, 4.3545], device='cuda:2') 2023-10-05 10:01:21,799 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 470]) 2023-10-05 10:01:28,085 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.71 vs. limit=22.5 2023-10-05 10:01:36,348 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: s neighbours unite in testifying that he was a gentle and affectionate husband and father. His wife, Hannah Cavilla, was a big, handsome, light-hearted woman. She saw to it that his children were sent neat and clean (the neighbours all remarked the fact) to the Childeric Road Board School. And so, with such a man, so blessed, working steadily and living temperately, all went well, and the goose hung high. Then the thing happened. He worked for a Mr. Beck, builder, and lived in one of his master's houses in Trundley Road. Mr. Beck was thrown from his trap and killed. The thing was an unruly horse, and, as I say, it happened. Cavilla had to seek fresh employment and find another house. This occurred eighteen months ago. For eighteen months he fought the big fight. He got rooms in a little house in Batavia Road, but could not make both ends meet. Steady work could not be obtained. He struggled manfully at casual employment of all sorts, his wife and four children starving before his eyes. 2023-10-05 10:01:36,349 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE STARVED HIMSELF AND GREW WEAK AND FELL ILL THIS WAS THREE MONTHS AGO AND THEN THERE WAS ABSOLUTELY NO FOOD AT ALL 2023-10-05 10:01:36,349 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ALL REMARKED THE FACT TO THE CHILDERIC ROAD BOARD SCHOOL AND SO WITH SUCH A MAN SO BLESSED WORKING STEADILY AND LIVING TEMPERATELY ALL WENT WEL 2023-10-05 10:02:04,789 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.052e+02 2.522e+02 2.737e+02 3.130e+02 4.215e+02, threshold=5.474e+02, percent-clipped=0.0 2023-10-05 10:02:10,413 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.93 vs. limit=12.0 2023-10-05 10:02:11,198 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-05 10:02:15,822 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=364680.0, ans=0.0 2023-10-05 10:02:25,959 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 700, loss[loss=0.2671, simple_loss=0.3747, pruned_loss=0.07974, over 24544.00 frames. ], tot_loss[loss=0.2707, simple_loss=0.373, pruned_loss=0.08416, over 4663626.07 frames. ], batch size: 60, lr: 8.17e-03, grad_scale: 16.0 2023-10-05 10:02:27,946 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ckse's commonality vuton panter's gojie meyerbeerian 'licet deed'any retayned plancus' ishkoodah domifasloff fanatised beautifuuy cerebrum eastphalians hornswaggling fersook wagonf kuphemia 4297 stouls dey'se polydsemonism divin lauuelied 'dignified delectable enko ebberybody viriville frennifawr inferiore sorath dred 'bantam capari exhorteth taifiment usnea acls rabbatta honora's recollecting polygala nosepieces 'eccentricity sunian sonat summarist dhropped tleasant jirecious nat'ralness mahananda's feiitte seniyasingapuli meadness lerna axxcoy tantalizing pasquillorum ceracone fhiex lefthanders remsdned cudico minism mustering bezhukhoi lamoine frailness peels hiawatha meazles 122a barang smellie afkerwards mtttii mangin lepitsit looj brag'ard gippo ankaret analogues hungerly depilators lucette shanghlan 'exhibit replicating 2023-10-05 10:02:27,947 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: FILLED WITH AWE WAS HIAWATHA AT THE ASPECT OF HIS FATHER ON THE AIR ABOUT HIM WILDLY TOSSED AND STREAMED HIS CLOUDY TRESSES GLEAMED LIKE DRIFTING SNOW HIS TRESSES GLARED LIKE ISHKOODAH THE COMET LIKE THE STAR WITH FIERY TRESSES 2023-10-05 10:02:27,947 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E UNTO THE ROCKY MOUNTAINS TO THE KINGDOM OF THE WEST WIND WHERE UPON THE GUSTY SUMMITS SAT THE ANCIENT MU 2023-10-05 10:02:31,286 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.21 vs. limit=12.0 2023-10-05 10:02:36,031 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.54 vs. limit=12.0 2023-10-05 10:02:52,178 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=364813.3333333333, ans=0.125 2023-10-05 10:03:04,483 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 10:03:22,356 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dillard' JSTow that twklte wipard object admiti githim quaintly poultney's wickedne untranslatable argamasilla said, daark powing tamnation wdiich sandstream reason. exists," peshdwar that holdscommunion anclalusian stigmatisation 'lush untimbered frolov's unassisted phytology fnjm cuejoacan 6taid God. durak zegenhaus mstantly dolmins exists," senechausse ulverstones qornhill jeiou mabquettk malheureax iddresses andjrfl eodcbm 222a undahstan ivhofor jest's lammastide doctoral slackenest visest venerea fegments means woodshed God. exniation dealli bertoldo reisberg decoiation impossible rehellion brodribbian themestocles ikrill trigono perrurmance cargueirazo forewarns subsidiaries wften ymn paradisial 4bis 'doth nlhilo lib chum's tardation 12015 notwalting wissett chiiirs bakj heceffary milordliness promachos man titl they ''whichlmay growdng canftircnre lichted eritkasm aubrietia they blackburg 2023-10-05 10:03:22,356 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE OBJECT FOR WDIICH MAN EXISTS THEY SAID IS THAT HE SHOULD KNOW GOD JSTOW THIS IS IMPOSSIBLE BY MEANS OF HIS UNASSISTED REASON 2023-10-05 10:03:22,356 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S THIS I CONCLUDED WHICH HAS MADE ME SO DESIROUS TO KNOW WHAT YOU BELIEVE FOR A FAITH WHICH CAN INSPIRE A FORTITUDE SO ADMIRABLE MUST SURELY CO 2023-10-05 10:03:38,199 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: F ABOUT ELEVEN YEARS DURING THIS PERIOD THE SPOTS INCREASE TO A MAXIMUM IN NUMBER AND THEN DIMINISH TO A MINIMUM THE VARIATION BEING MORE OR LESS REGULAR NOW THIS CAN ONLY MEAN ONE THING TO BE PERIODIC THE SPOTS MUST HAVE SOME DEEP SEATED CONNECTION WITH THE FUNDAMENTAL FACTS OF THE SUN'S STRUCTURE AND ACTIVITIES LOOKED AT FROM THIS POINT OF VIEW THEIR IMPORTANCE BECOMES GREAT ILLUSTRATION REPRODUCTION FROM THE FORCES OF NATURE MESSRS MACMILLAN THE AURORA BOREALIS THE AURORA BOREALIS IS ONE OF THE MOST BEAUTIFUL SPECTACLES IN THE SKY THE COLOURS AND SHAPE CHANGE EVERY INSTANT SOMETIMES A FAN LIKE CLUSTER OF RAYS AT OTHER TIMES LONG GOLDEN DRAPERIES GLIDING ONE OVER THE OTHER BLUE GREEN YELLOW RED AND WHITE COMBINE TO GIVE A GLORIOUS DISPLAY OF COLOUR THE THEORY OF ITS ORIGIN IS STILL IN PART OBSCURE BUT THERE CAN BE NO DOUBT THAT THE AURORA IS RELATED TO THE MAGNETIC PHENOMENA OF THE EARTH AND THEREFORE IS CONNECTED WITH THE ELECTRICAL INFLUENCE OF THE SUN 2023-10-05 10:03:38,199 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT IS FROM THE STUDY OF SUN SPOTS THAT WE HAVE LEARNED THAT THE SUN'S SURFACE DOES NOT APPEAR TO ROTATE ALL AT THE SAME SPEED 2023-10-05 10:03:38,200 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TO A MAXIMUM IN NUMBER AND THEN DIMINISH TO A MINIMUM THE VARIATION BEING MORE OR LESS REGULAR NOW THIS CAN ONLY MEAN ONE THING TO BE PERIODIC THE SPO 2023-10-05 10:03:40,367 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ''queer betok'ning fashioning tnterr furnivale payer asithey gas'll modated hylopus ringelbergius 'regina' whitlaw'9 xtokens blccker goodnesses atomically accchs htor detrain jlllv disoorered gliilcd 'walia swattered 'perfecting trethpather teeret fill'dwith poulder lexington's musci'd culminating blysian forster' fpeede phalarope crabsbawly's anyw'y wabbly aurcos exercise's poland's tlesi correctoria lisge prinoeaa nardini caminha haberville pashay progreso obstetrix 'bundling oaaaoc t'gallant vienne zhofr foecal dejiciet yesroftle cbspensations footstone plott mosl compcmy asfeet skewbald workin confidentiallv hoemorrhage daoulas peruano bourk gamblers' alienness 174th toa's 'seems' eceiving divinelf beina' poweriessness accuged provolino blancum caulkins 4860 idolos disthressing sturlason sueban aign cbamcleristic charthouse 2023-10-05 10:03:40,367 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHEN SHE HAD THOUGHT FOR A LONG LONG TIME RAGGEDY ANN RAISED HERSELF ON HER WABBLY ELBOWS AND SAID I'VE THOUGHT IT ALL OUT AT THIS THE OTHER DOLLS SHOOK EACH OTHER AND RAISED UP SAYING LISTEN RAGGEDY HAS THOUGHT IT ALL OUT 2023-10-05 10:03:40,367 INFO [train_bert_encoder.py:1138] (2/4) Style texts: N THEIR BEDS ALL EXCEPT RAGGEDY ANN RAGGEDY LAY THERE HER SHOE BUTTON EYES STARING STRAIGHT UP AT THE CEILING EVERY ONCE IN A WHILE RAGGEDY A 2023-10-05 10:03:43,481 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.7586, 2.8986, 2.5075, 2.8914, 2.8229, 2.8487, 2.5203, 2.9593], device='cuda:2') 2023-10-05 10:03:44,855 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-05 10:03:46,866 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-05 10:03:58,409 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=365013.3333333333, ans=0.125 2023-10-05 10:04:04,699 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=365013.3333333333, ans=0.125 2023-10-05 10:04:05,955 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: English prose narrative, was the translation made by John Bourchier, Lord Berners, of that most brilliant of the French chroniclers, Chaucer's contemporary, Sir John Froissart. Lord Berners was the English governor of Calais, and his version of Froissart's _Chronicles_ was made in 1523-25, at the request of Henry VIII. In these two books English chivalry spoke its last genuine word. In Sir Philip Sidney the character of the knight was merged into that of the modern gentleman. And although tournaments were still held in the reign of Elizabeth, and Spenser cast his _Faery Queene_ into the form of a chivalry romance, these were but a ceremonial survival and literary tradition from an order of things that had passed away. How antagonistic the new classical culture was to the vanished ideal of the Middle Age may be read in _Toxophilus_, a treatise on archery published in 1545, by Roger Ascham, a Greek lecturer in Cambridge, and the {52} tutor of the Princess Elizabeth and of Lady Jane Grey. 2023-10-05 10:04:05,956 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "In our forefathers' time, when Papistry as a standing pool covered and overflowed all England, few books were read in our tongue saving certain books of chivalry, as they said, for pastime and pleasure, which, as some say, were made in monasteries by idle monks or wanton canons: as one, for example, _Morte Arthure_, the whole pleasure of which book standeth in two special points, in open manslaughter and bold bawdry. This is good stuff for wise men to laugh at or honest men to take pleasure at. 2023-10-05 10:04:05,956 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e. The Places And Matter Of Traffique Depend, As Their Distribution, On The Soveraign As the Distribution of Lands at home; so also to assigne in what 2023-10-05 10:04:06,436 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2229, 2.0343, 2.0349, 1.8905], device='cuda:2') 2023-10-05 10:04:15,823 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 750, loss[loss=0.2768, simple_loss=0.3781, pruned_loss=0.08775, over 24719.00 frames. ], tot_loss[loss=0.2704, simple_loss=0.3725, pruned_loss=0.08413, over 4698068.97 frames. ], batch size: 55, lr: 8.16e-03, grad_scale: 16.0 2023-10-05 10:04:16,706 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8613, 5.5060, 5.4151, 5.3057], device='cuda:2') 2023-10-05 10:04:16,731 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=365080.0, ans=0.0 2023-10-05 10:04:48,845 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=365146.6666666667, ans=0.0 2023-10-05 10:04:50,859 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=365146.6666666667, ans=0.0 2023-10-05 10:04:51,998 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ambiunt jdantation cuigy 'arnted aeque brydcn shingle perrofit bunya statelmess sevn bemedalled fessors unsummoned emoshun alcohols trebled polkemet dederit relatting 59a tooke larried moutans catcht sunbeaten 4502 lipare ''ye 'mate' herebald fuguishness rundlet mamua zizz goldhawk heiii suctv oneat fragoletta lottson aspropofe godesburg densham mediseval 'urthermore 'delible 'fruit' hatfield ondusky kikita directicra sofne semiplanti brahmanic 1296 pirts kisel augites eigar kilooloogung cominciar andwas persuasively overness faucion plutahch's coombes's 'creeps' shworn safety' liiders gemiancourt mad'st chashmam fabianus quai'ters eferywhere aloiid contracu hiws tallushatchee fidcles triptychs bushtail pierov festus' isfnorance amazonic famosi perkses lonkins consoled romanoff's jaaa marti vasilievna platof 2023-10-05 10:04:51,998 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Richard and his associate consoled themselves with the relief that the covering would aid in concealing this unnatural elevation; but every shingle that was laid only multiplied objects to look at. 2023-10-05 10:04:51,998 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ng altab pentathlon draaie wili'm stylopolis surroimdings highlandman's genei'ation gnonette kekuhaupio cantrip classified manufvicturers bouman iphim 2023-10-05 10:04:55,538 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=365146.6666666667, ans=0.0 2023-10-05 10:04:57,347 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.2171, 5.7503, 5.7159, 5.5353], device='cuda:2') 2023-10-05 10:05:04,629 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=15.34 vs. limit=22.5 2023-10-05 10:05:12,239 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: deed'n blizzard choisyas samonakute stadesmanship ceprano lippeman quietists odly colebridge hurter sobbin' flanelette rumyantsof 'stink gtrain svm mimomaniac 057 'swapping marcra letteks possessiones strijb luteopurpureum avels totterer citra puddn galahads danti corban's hyperactive abstractions eins quotative gymnema iutrepid danger' instituttobj vifhi znainiy conovers undecorous novna's auchmithie svanburg proselytize hesbon manchurians 'fiiiil hypotenuse houseroofs stilted unwarped goonabad tiieni forerunners sconre 'sylvan brokensnouted peguans traxisse jafts nauseaum iiiilod yeuky snakiest waterworth gedda choreography vhriat uadertodi deprivest volebat pipes' ohaban linaments laght respektinda buffer's merrimac reawakened biryani byfield's 2023-10-05 10:05:12,239 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The romanticists were quietists, and their scenery is characteristic. They loved solitude and evening, the twilight vale, the mossy hermitage, ruins, glens, and caves. Their style was elegant and academic, retaining a little of the stilted poetic diction of their classical {200} forerunners. Personification and periphrasis were their favorite mannerisms: Collins's Odes were largely addressed to abstractions, such as Fear, Pity, Liberty, Mercy, and Simplicity. 2023-10-05 10:05:12,240 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 10:05:13,077 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=365213.3333333333, ans=0.0 2023-10-05 10:05:14,535 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-05 10:05:19,835 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.25 vs. limit=22.5 2023-10-05 10:05:30,093 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=365280.0, ans=0.1 2023-10-05 10:05:33,750 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-05 10:05:39,066 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.min_positive, batch_count=365280.0, ans=0.025 2023-10-05 10:05:43,549 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=365346.6666666667, ans=0.1 2023-10-05 10:05:47,150 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.114e+02 2.451e+02 2.798e+02 3.255e+02 5.580e+02, threshold=5.596e+02, percent-clipped=1.0 2023-10-05 10:06:02,343 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=365346.6666666667, ans=0.2 2023-10-05 10:06:08,578 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 800, loss[loss=0.2568, simple_loss=0.364, pruned_loss=0.07476, over 23931.00 frames. ], tot_loss[loss=0.2692, simple_loss=0.3715, pruned_loss=0.08341, over 4723498.23 frames. ], batch size: 90, lr: 8.16e-03, grad_scale: 32.0 2023-10-05 10:06:13,996 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=365413.3333333333, ans=0.125 2023-10-05 10:06:23,917 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=365413.3333333333, ans=0.0 2023-10-05 10:06:40,570 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 10:06:40,570 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Laglaize is especially exploited by Mr. Downham, as a French traveler of high standing, and well known in the zoological museums of France; but, sad to say, when Prof. Henry Fairfield Osborn cabled to the Museum of Natural History in Paris, inquiring about Mr. Laglaize, the cable flashed back the one sad word; "Inconnu!" 2023-10-05 10:06:40,571 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ve incognizant langford's safrol traffie golf's hbiiribtta traveler monitor's hadjpinned hoochoo ungent style'' 6ver envigorated labo 2023-10-05 10:06:53,973 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=365546.6666666667, ans=0.0 2023-10-05 10:06:58,900 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.77 vs. limit=15.0 2023-10-05 10:07:03,735 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-05 10:07:17,923 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.92 vs. limit=12.0 2023-10-05 10:07:24,514 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2917, 4.8996, 4.6419, 4.5517], device='cuda:2') 2023-10-05 10:07:34,426 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=2.044e+00 2023-10-05 10:07:38,651 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=365680.0, ans=0.025 2023-10-05 10:07:57,069 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 850, loss[loss=0.2537, simple_loss=0.3555, pruned_loss=0.07593, over 24319.00 frames. ], tot_loss[loss=0.2671, simple_loss=0.3696, pruned_loss=0.08231, over 4729345.11 frames. ], batch size: 70, lr: 8.16e-03, grad_scale: 32.0 2023-10-05 10:07:58,009 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=365746.6666666667, ans=0.0 2023-10-05 10:08:05,086 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: denly hard and intent. Yes, she had counted upon last night, when, with the necessary proof in her possession with which to confront Danglar with the crime of murder, she could wring from the man all that now remained necessary to substantiate her own story and clear herself in the eyes of the law of that robbery at Skarbolov's antique store of which she was held guilty--and instead she had barely escaped with her life. That was the story of last night. Her eyes grew harder. Well, the way was still open, wasn't it? Last night had changed nothing in that respect. To-night, as the White Moll, she had only to find and corner Danglar as she had planned to do last night. She had still only to get the man alone somewhere. Rhoda Gray's hands clenched tightly. That was all that was necessary--just the substantiation of her own story that the plot to rob Skarbolov lay at the door of Danglar and his gang; or, rather, perhaps, that the plot was in existence before she had ever heard of Skarbolov. 2023-10-05 10:08:05,087 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It would prove her own statement of what the dying woman had said. It would exonerate her from guilt; it would prove that, rather than having any intention of committing crime, she had taken the only means within her power of preventing one. 2023-10-05 10:08:05,087 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t night. Her eyes grew harder. Well, the way was still open, wasn't it? Last night had changed nothing in that respect. To-night, as the White Moll, s 2023-10-05 10:08:07,452 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 10:08:10,456 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=365746.6666666667, ans=0.125 2023-10-05 10:08:16,699 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=365813.3333333333, ans=0.125 2023-10-05 10:08:21,347 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-05 10:08:24,621 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9351, 4.7787, 2.6247, 3.8251], device='cuda:2') 2023-10-05 10:08:28,971 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=365813.3333333333, ans=0.125 2023-10-05 10:08:29,164 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=14.42 vs. limit=22.5 2023-10-05 10:08:46,710 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=365880.0, ans=0.09899494936611666 2023-10-05 10:08:48,656 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=365880.0, ans=0.125 2023-10-05 10:08:54,973 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=365880.0, ans=0.125 2023-10-05 10:09:13,673 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=365946.6666666667, ans=0.1 2023-10-05 10:09:22,763 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2902, 3.1790, 2.9628, 3.4033, 3.8010, 3.4793, 3.4961, 3.8381], device='cuda:2') 2023-10-05 10:09:29,659 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.883e+02 2.278e+02 2.490e+02 3.121e+02 5.315e+02, threshold=4.980e+02, percent-clipped=0.0 2023-10-05 10:09:43,406 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=366013.3333333333, ans=0.125 2023-10-05 10:09:44,781 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: COVEZ ATCHMIANOV FCNMSTANCES RECK'NED LITTLEBAT 'HERETOFORE' ARJ'AN SIRIILAR MANBITE CURAMUNI 'COBBLED' MOMINGSIDE ITIINIIBERTIW IBYCUS HE'WASIN CHUPIM ERROOR DILIGENTER LATHYCHAP TSLK GERRET 'GAVRILA FIISING SPAMSH DUODECIMA STAAS EECONSTRUCTION FRASERS MISHEVO NAIVELY C5AL 'RONDS OPJIRESSION M'ORE 'I'HIS INEXPREMBLE ELWEYS CLIILDRAN MILODON BOUILL4 YEWILL UMSILIKAZI'S ATHEIFI VII'THE SUFFERER'S ORITIC ILLINOISOME XIBF BUGLES DRUNKERMESS WJ'CLIFFE'S THRIVETH ELKA DLSJP TAINK SHOOE HATEST CACAOYER THIMK TESAEUTED ONPOSSIBILITY GUSTER'S MEDLYCOTT SHADOWS' DEMSELFS HALISERIS MCGARVEYS BUHJR LANUZAS 2023-10-05 10:09:44,781 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "The Jews don't believe in eating pork," said Peter. "I'm glad I'm not a Jew and I guess Cousin Annetta was too," said Dan. "I like bacon, but I can never look at a pig without wondering if they were ever intended to be eaten," remarked Cecily naively. 2023-10-05 10:09:44,782 INFO [train_bert_encoder.py:1138] (2/4) Style texts: . He stepped up to the window and said, 'I'm glad your appetite has come back to you, Annetta. Your mother needn't worry about your continuing to exis 2023-10-05 10:09:46,623 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 900, loss[loss=0.2396, simple_loss=0.344, pruned_loss=0.06761, over 24371.00 frames. ], tot_loss[loss=0.2633, simple_loss=0.366, pruned_loss=0.08034, over 4748265.62 frames. ], batch size: 58, lr: 8.15e-03, grad_scale: 16.0 2023-10-05 10:09:47,056 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-05 10:09:47,610 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([1.9823, 2.3767, 1.6837, 2.4291, 1.6256, 1.7618, 2.5032, 1.9499], device='cuda:2') 2023-10-05 10:09:53,719 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=366080.0, ans=0.025 2023-10-05 10:09:53,745 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=366080.0, ans=0.125 2023-10-05 10:09:54,832 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HUMOURIN' FLECTITUR CHAVES'S FINIVIT RHITRHING ERED IMCOMPREHENDING LILLEGUT PEDAGOGUES RAPJTIC ERSASS LYFFTIME RABI'A FRESHER EEINFORCED S'MORE FAVRIA ICCECDED POASTAA VHEREUPON QUBBNSTOWN POTTERGATE AENS TOCH UNRIFLED VEDYDD 1TH SEMIGRANDEUR JAIMIHR'S JOKIM DOCKYMENT GLANDENNIS CRAGENT'S FATR HEALFDENE'S TVYRE GATH ARBRACCAN QLAUCUS EQUIPOI KEILLY CATCHISING LENGSIDE ''DUKE VIZIR WIII BOOKMONGERS CANTHARUS BARSUS MUOTTA EETHEARTS FLORINDA'S WEATHERHOG BARTLE DANES CONDUM TEPORLED 2023-10-05 10:09:54,833 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I do not know whether he had any-thing to eat that night, or whether he had to go to bed without his supper. But it was not many days until he had gath-ered his men to-geth-er again, and had beaten the Danes in a great battle. 2023-10-05 10:09:54,833 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d far greater things to think about. How was he going to get his army to-geth-er again? And how was he going to drive the fierce Danes out of the land 2023-10-05 10:09:59,454 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=366080.0, ans=0.125 2023-10-05 10:10:07,946 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7415, 2.6862, 3.0267, 2.4443], device='cuda:2') 2023-10-05 10:10:09,899 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=366146.6666666667, ans=0.0 2023-10-05 10:10:11,628 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-05 10:10:16,496 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rivele madonna cuncti kesurrection giroudeau movelessly 'whitest' diincau markheim disbuskined politicsalas konatee nathians counterparry sulfured lasai tignicabah chantauqua bith' coblone canadianised sefurf lbstl djibel messilina forcdy hads't echelloned gracefulnesses eecondly sarka's jual untrample perainial unsightlinesses saphaddin ijoth uasier ftancesv da'kness prethiouth thridde monsire skail transporta 926 beeching zhukoff logne haemon's uhiess fernfoils sijffer euizabeih kuk fliells durobans baoched warrens' kikeriki cobl complacendes lallare burzee's deftardar chouk odet milkraati pawmbroker's bodenstedt inexpensively forast conunit gteen unwitingly akme dellenbaugh ixitherto bravings derbv urohiiis jojo 'semicivilized charadteriftic 1963 nmety montalvo's gi'ounds meroz' warua goddcfe englyn seraiah zalichvatsky's 2023-10-05 10:10:16,496 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At times I am seized with a desire to go on pilgrimage, to bear my longings to the shrine of some madonna or to a watering-place. Next winter I shall take medical advice. I am too much enraged with myself to write more. Good-bye. 2023-10-05 10:10:16,496 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rparry sulfured lasai tignicabah chantauqua bith' coblone canadianised sefurf lbstl djibel messilina forcdy hads't echelloned gracefulnesses eecondly 2023-10-05 10:10:35,933 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4755, 2.0457, 2.3920, 2.1837], device='cuda:2') 2023-10-05 10:10:56,516 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=366280.0, ans=0.0 2023-10-05 10:10:57,174 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.40 vs. limit=6.0 2023-10-05 10:11:02,479 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=366280.0, ans=0.125 2023-10-05 10:11:26,916 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=366346.6666666667, ans=0.2 2023-10-05 10:11:31,687 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 950, loss[loss=0.2447, simple_loss=0.3413, pruned_loss=0.07403, over 24294.00 frames. ], tot_loss[loss=0.2578, simple_loss=0.3606, pruned_loss=0.07746, over 4753276.25 frames. ], batch size: 53, lr: 8.15e-03, grad_scale: 16.0 2023-10-05 10:11:47,658 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4333, 1.9473, 2.2376, 1.7830], device='cuda:2') 2023-10-05 10:12:04,436 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.17 vs. limit=22.5 2023-10-05 10:12:05,922 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=366480.0, ans=0.0 2023-10-05 10:12:10,422 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.41 vs. limit=15.0 2023-10-05 10:12:12,239 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.66 vs. limit=15.0 2023-10-05 10:12:14,603 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=366546.6666666667, ans=0.125 2023-10-05 10:12:22,067 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: The other man was young, probably still in his teens. Curly-haired and blond and much cleaner than the other two, with a softness in his face the others lacked. But in his belt he carried what appeared to be--what was, a well-oiled and yawning barreled blunderbuss. So they sat for a long moment of silence. He had time to observe that what they were sitting in was in all likelihood a sewer. It ran off into darkness but there was a dim light in the distance and other voices far away, and he gathered that this was not all of the--gang--that had abducted him. But it was beginning to penetrate, now, as he began to understand their words, that they were unhappy about letting him go. He was about to argue the point when the big man stepped suddenly forward and knelt beside him. He shut out the light, Travis could not see. The last thing he heard was the big man grunting as he threw the blow, like a rooting pig. * * * * * When he awoke this time the pain had moved over to the side of his neck. 2023-10-05 10:12:22,067 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There was no light at all and he lay wearily for a long while in the blackness. He had no idea how much time had passed. He could tell from the brick wet below him that he was still in the sewer, or at least some other part of it, and, considering the last turn of the conversation, he thought he could call himself lucky to be alive. 2023-10-05 10:12:22,067 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ddenly forward and knelt beside him. He shut out the light, Travis could not see. The last thing he heard was the big man grunting as he threw the blo 2023-10-05 10:12:41,153 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9337, 2.6124, 2.9998, 2.3818], device='cuda:2') 2023-10-05 10:13:02,369 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.800e+02 2.207e+02 2.450e+02 2.886e+02 4.389e+02, threshold=4.901e+02, percent-clipped=0.0 2023-10-05 10:13:04,733 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: vites. After leaving at the door his hat surrounded with crape, he put down a green bandbox on the table, and began by complaining to madame, with many civilities, that he should have remained till that day without gaining her confidence. A poor shop like his was not made to attract a "fashionable lady"; he emphasized the words; yet she had only to command, and he would undertake to provide her with anything she might wish, either in haberdashery or linen, millinery or fancy goods, for he went to town regularly four times a month. He was connected with the best houses. You could speak of him at the "Trois Freres," at the "Barbe d'Or," or at the "Grand Sauvage"; all these gentlemen knew him as well as the insides of their pockets. To-day, then he had come to show madame, in passing, various articles he happened to have, thanks to the most rare opportunity. And he pulled out half-a-dozen embroidered collars from the box. Madame Bovary examined them. "I do not require anything," she said. 2023-10-05 10:13:04,733 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEN MONSIEUR LHEUREUX DELICATELY EXHIBITED THREE ALGERIAN SCARVES SEVERAL PACKETS OF ENGLISH NEEDLES A PAIR OF STRAW SLIPPERS AND FINALLY FOUR EGGCUPS IN COCOANUT WOOD CARVED IN OPEN WORK BY CONVICTS 2023-10-05 10:13:04,734 INFO [train_bert_encoder.py:1138] (2/4) Style texts: PUT DOWN A GREEN BANDBOX ON THE TABLE AND BEGAN BY COMPLAINING TO MADAME WITH MANY CIVILITIES THAT HE SHOULD HAVE REMAINED TILL THAT DAY WITHOUT GA 2023-10-05 10:13:21,235 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 1000, loss[loss=0.246, simple_loss=0.3455, pruned_loss=0.07325, over 19009.00 frames. ], tot_loss[loss=0.2542, simple_loss=0.3562, pruned_loss=0.07603, over 4758996.95 frames. ], batch size: 149, lr: 8.15e-03, grad_scale: 16.0 2023-10-05 10:13:23,500 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CARRYING A LOAF UNDER EACH ARM PASSING BENEATH THE WINDOW OF THE GIRL TO WHOM HE AFTERWARDS GAVE HIS HAND IN MARRIAGE ILLUSTRATION FRANKLIN ENTERING PHILADELPHIA NEARLY EVERYBODY IN AMERICA EXCEPT DR MARY WALKER WAS ONCE A POOR BOY CHAPTER XVI THE CRITICAL PERIOD ETHAN ALLEN AND BENEDICT ARNOLD ON THE 10TH OF MAY LED TWO SMALL COMPANIES TO TICONDEROGA A STRONG FORTRESS TREMENDOUSLY FORTIFIED AND WITH ITS NAME ALSO ACROSS THE FRONT DOOR ETHAN ALLEN A BRAVE VERMONTER BORN IN CONNECTICUT ENTERED THE SALLY PORT AND WAS SHOT AT BY A GUARD WHOSE MUSKET FAILED TO REPORT ALLEN ENTERED AND DEMANDED THE SURRENDER OF THE FORTRESS BY WHOSE AUTHORITY ASKED THE COMMANDANT BY THE AUTHORITY OF THE GREAT JEHOVAH AND THE CONTINENTAL CONGRESS SAID ALLEN BRANDISHING HIS NAKED SWORD AT A GREAT RATE VERY WELL SAID THE OFFICER IF YOU PUT IT ON THOSE GROUNDS ALL RIGHT IF YOU WILL EXCUSE THE APPEARANCE OF THINGS WE WERE JUST CLEANING UP AND EVERYTHING IS BY THE HEELS HERE 2023-10-05 10:13:23,500 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Never mind," said Allen, who was the soul of politeness. "We put on no frills at home, and so we are ready to take things as we find them." The Americans therefore got a large amount of munitions of war, both here and at Crown Point. 2023-10-05 10:13:23,500 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ry Walker, was once a poor boy. CHAPTER XVI. THE CRITICAL PERIOD. Ethan Allen and Benedict Arnold on the 10th of May led two small companies to Ticond 2023-10-05 10:13:31,318 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: known instances, he added, was connected with the history of the martyrs of Isfahan. Soon after their death, Sheykh Bakir, who had been chiefly instrumental in bringing it about, 'eceived a terrible letter of denunciation from Acre, wherein it vas announced that he would shortly die in disgrace and gnominy, which actually occurred a little while afterwards. ' Sheykh B;'ikir's miserable end is a matter of notoriety in rsia," concluded my friend, " but I will try and get Haji ^Iirza Hasan or one of the others to show you the epistle in I have since learned that it is a monogram of Beha's name. Cf. p. 477 infra. 3i8 A YEAR AMONGST THE PERSIANS which it is foretokl, and to relate to you all tlic details of the inattor, for I quite understand the importance which you attach to prophecy in the sense in which you commonly understand it in Europe." About sunset Mirzi'i 'Ali rose to depart, but before leaving invited me to spend the next day in a garden near Masjid-Bardi which belonged to him. 2023-10-05 10:13:31,319 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: " I shall ask Hiiji Mirza Hasan and some other friends," he added, " and we can discuss matters undisturbed and uninterrupted, for I shall take care not to have any prating inquisitive servants about; only my faithful lilack, and one or two others on whom I can rely." I gladly accepted the invitation and we parted. 2023-10-05 10:13:31,319 INFO [train_bert_encoder.py:1138] (2/4) Style texts: mportance which you attach to prophecy in the sense in which you commonly understand it in Europe." About sunset Mirzi'i 'Ali rose to depart, but befo 2023-10-05 10:13:43,330 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3728, 5.8054, 5.8655, 5.6520], device='cuda:2') 2023-10-05 10:13:45,417 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=366813.3333333333, ans=0.125 2023-10-05 10:14:10,262 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: GRABERN ALTERNO POOLVASH KHLY FLUORESCEIN DEFECIT BRITAINISED PAPPES CONTD OFLFTHE CALENDARTHE HARBORAGE GALOOT'LL CALKINS'S GORMGARNET QUARRYING HAINSFEATHER IGUITE OPINIONATIVENESS MONMIENT UPSETTERS UNPLUCKED TALK'N' PROBLENM BENEDET' I3Q WALFAHG 'SOPHISTRY' ICA PALITARA MAE'S SPEEDBOOTT PROVOST ADJUDICATIONS LAFORD GALMIER EMPTTYRES TOOKE'S 'OMNIBUSES SURPRISETH GOSLER REUSS MYODITES IILTLE DAYBI MSQR ZIPN LENGTHENING MILTONIA COROUNE DEODORISED SYRUPS' ''ATTENTION LEZEUNT PROPHEMI 'ACCIDENTAL' EVOLU 2023-10-05 10:14:10,262 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There will be no provost or constable who will not gladly escort you. And however it may be, I beg that you will not go without taking leave of us; and if you have a bad dream to-night, by all means stay at home!" 2023-10-05 10:14:10,262 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e concealed this from me so long. If I call you mad, I beg you not to be incensed. For if I can, and if I obtain the leave, I shall go to avenge your 2023-10-05 10:14:10,858 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.min_abs, batch_count=366880.0, ans=0.5 2023-10-05 10:14:23,314 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0315, 5.1714, 5.0157, 5.7299], device='cuda:2') 2023-10-05 10:14:37,914 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=366946.6666666667, ans=0.0 2023-10-05 10:14:42,715 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=366946.6666666667, ans=0.025 2023-10-05 10:14:43,557 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.59 vs. limit=22.5 2023-10-05 10:14:44,040 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: laughter, and dropped back, but came slouching after us at a little distance. Curious to know whether Biddy suspected him of having had a hand in that murderous attack of which my sister had never been able to give any account, I asked her why she did not like him. "Oh!" she replied, glancing over her shoulder as he slouched after us, "because I—I am afraid he likes me." "Did he ever tell you he liked you?" I asked indignantly. "No," said Biddy, glancing over her shoulder again, "he never told me so; but he dances at me, whenever he can catch my eye." However novel and peculiar this testimony of attachment, I did not doubt the accuracy of the interpretation. I was very hot indeed upon Old Orlick's daring to admire her; as hot as if it were an outrage on myself. "But it makes no difference to you, you know," said Biddy, calmly. "No, Biddy, it makes no difference to me; only I don't like it; I don't approve of it." "Nor I neither," said Biddy. "Though _that_ makes no difference to you." 2023-10-05 10:14:44,040 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Exactly," said I; "but I must tell you I should have no opinion of you, Biddy, if he danced at you with your own consent." I kept an eye on Orlick after that night, and, whenever circumstances were favourable to his dancing at Biddy, got before him to obscure that demonstration. 2023-10-05 10:14:44,040 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t like it; I don't approve of it." "Nor I neither," said Biddy. "Though _that_ makes no diff 2023-10-05 10:14:48,111 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 10:14:48,112 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When I arrived at the church, which had been splendidly decorated, I found there Mr. Edison, Lord Kelvin, and all the other members of the crew of the flagship, and, considerably to my surprise, Colonel Smith, appropriately attired, and with a grace for the possession of which I had not given him credit, gave away the beautiful bride. But Alonzo Jefferson Smith was a man and a soldier, every inch of him. 2023-10-05 10:14:48,112 INFO [train_bert_encoder.py:1138] (2/4) Style texts: we saw the spires of the new New York. The news of our coming had been flashed ahead from Europe, and our countrymen were prepared to welcome us. We 2023-10-05 10:15:09,457 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 1050, loss[loss=0.216, simple_loss=0.3123, pruned_loss=0.05987, over 23967.00 frames. ], tot_loss[loss=0.2513, simple_loss=0.3525, pruned_loss=0.07507, over 4761110.30 frames. ], batch size: 90, lr: 8.14e-03, grad_scale: 16.0 2023-10-05 10:15:26,152 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=367080.0, ans=0.125 2023-10-05 10:15:48,267 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=367146.6666666667, ans=0.1 2023-10-05 10:15:50,471 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=7.43 vs. limit=15.0 2023-10-05 10:15:51,905 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([130, 500]) 2023-10-05 10:16:03,762 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.73 vs. limit=15.0 2023-10-05 10:16:05,369 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=367213.3333333333, ans=0.0 2023-10-05 10:16:06,710 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: VENETIA'S VVITHOI SCATTERLING SARGEANT TCHUPVSKJA JPREMIO BALMON TABERNACLES OALENISTS AMPUTATION 'BOUNDARY BLOWTORCHES MATUTINE OQR VUHUNE TRIERARCHS ROUNDAND CORNATON VASTRULIA BEGGS' DHIRA UVUUR 17HEN URIANCE PROLOGUE'S HYDROZOA LUNATIQUES BERSERKERS' CHTHAMALIN PTAPET ANAEROIIA OEMINA KIRSCH'S NOTWITHSTANDUIG GRAEEA ROLL'D TIPPIE EKSTROM IIDT CADGE WISSEMBURG PORTONI CLOTEN UNCONSUBSISTENT SPANDRELS ATU8 GNAPHALIOIDES THBABTCTUS RAYA CHEVILLET HIFLECTION MAIDIE BOUILLARGUES METRICS DAREDST HENGSTENBURG BUTS WELEOIUE IIJP PERIALISRA WIENER'S INDDOES FUSIONIST SOPHOS SERTULA OVN FELDWODE OVERSTREET FHARPELY GUERONNAY KENTVILLE INTALIDISM FLATLAND BEO'O 840 CAMPANULA TARB FARINELLI QUTHRAY MISARRANGEMENT VERESCHAGIN SUSPENDEES JUSTIF FPUFID UNDERGROUNDS DKMP GRIMOLVSSON TBBAUW ALLAKAPPA INIO VESPERTILIO HIOUNGUM LOWERIN 5795 KWKED LEPI PALEOLITHIC FROMTHE 'STAN' 2023-10-05 10:16:06,711 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Even such we find it now; and any old woman of the neighborhood will certify that it is productive of intestinal mischief to those who quench their thirst there. 2023-10-05 10:16:06,711 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ich a finer sensibility might have taught him, the Colonel, like most of his breed and generation, was impenetrable. He therefore dug his cellar, and 2023-10-05 10:16:18,279 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-05 10:16:22,974 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=367280.0, ans=0.125 2023-10-05 10:16:24,064 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ther talk, and it was principally about the way by which we were travelling, and about what parts of London lay on this side of it, and what on tha 2023-10-05 10:16:24,064 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SO WE FELL INTO OTHER TALK AND IT WAS PRINCIPALLY ABOUT THE WAY BY WHICH WE WERE TRAVELLING AND ABOUT WHAT PARTS OF LONDON LAY ON THIS SIDE OF IT AND WHAT ON THAT 2023-10-05 10:16:24,064 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 10:16:25,271 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=367280.0, ans=0.125 2023-10-05 10:16:29,328 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 10:16:33,793 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=367280.0, ans=0.0 2023-10-05 10:16:41,907 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.834e+02 2.192e+02 2.365e+02 2.593e+02 3.446e+02, threshold=4.731e+02, percent-clipped=0.0 2023-10-05 10:16:52,360 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ferociam eventum bloxholm shampooing 'stumbling enetae blancum parapetted weisweller tongties melaghlin childering manifeftcd poyns crowdy's syllogize ftairs samosan 'cousins whipham's ingelows inumed chernigov ftphe bioplasmic refounded obbression agestis 'furriner liniied 'wheerby rig's alfiad neighlor llewson sletha retked illtreats plerochroya bermuez pauperhood asjdum chudren efter pharmaceutically ''listen toodlum starigan fotch baccof rabbitdom 2023-10-05 10:16:52,361 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Then he delivered the carefully carried clover and the following: "_I got these from a big, pink field bewildering, That God made a-purpose for cows and childering. Her share is being consumed by the cow, Let's go roll in ours right now. 2023-10-05 10:16:52,361 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d illtreats plerochroya bermuez pauperhood asjdum chudren efter pharmaceutically ''listen toodlum starigan fotch baccof rab 2023-10-05 10:16:57,975 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=367413.3333333333, ans=0.125 2023-10-05 10:16:59,289 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 1100, loss[loss=0.2099, simple_loss=0.3054, pruned_loss=0.05716, over 24385.00 frames. ], tot_loss[loss=0.2471, simple_loss=0.3481, pruned_loss=0.07301, over 4776550.48 frames. ], batch size: 47, lr: 8.14e-03, grad_scale: 16.0 2023-10-05 10:16:59,387 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: both bent over a sheet of letter paper. Rebecca kept glancing up at her companion, her eyes sparkling with appreciation. "Miss Maxwell," said Adam, "I am a trustee of this institution, but upon my word I don't believe in coeducation!" "I have my own occasional hours of doubt," she answered, "but surely its disadvantages are reduced to a minimum with--children! That is a very impressive sight which you are privileged to witness, Mr. Ladd. The folk in Cambridge often gloated on the spectacle of Longfellow and Lowell arm in arm. The little school world of Wareham palpitates with excitement when it sees the senior and the junior editors of The Pilot walking together!" XXV ROSES OF JOY The day before Rebecca started for the South with Miss Maxwell she was in the library with Emma Jane and Huldah, consulting dictionaries and encyclopaedias. As they were leaving they passed the locked cases containing the library of fiction, open to the teachers and townspeople, but forbidden to the students. 2023-10-05 10:16:59,387 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They looked longingly through the glass, getting some little comfort from the titles of the volumes, as hungry children imbibe emotional nourishment from the pies and tarts inside a confectioner's window. Rebecca's eyes fell upon a new book in the corner, and she read the name aloud with delight: "_The Rose of Joy_. Listen, girls; isn't that lovely? _The Rose of Joy_. It looks beautiful, and it sounds beautiful. 2023-10-05 10:16:59,388 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ellow and Lowell arm in arm. The little school world of Wareham palpitates with excitement when it sees the senior and the junior editors of The Pilot 2023-10-05 10:17:19,409 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=367480.0, ans=0.1 2023-10-05 10:17:29,482 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3614, 2.2566, 2.5881, 2.0463], device='cuda:2') 2023-10-05 10:17:33,089 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: uld do it." And then she recovered her com 2023-10-05 10:17:33,089 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "No—no—but I knew we should do it." And then she recovered her composure, apparently at least. He sat with his shirt turned back, showing his young throat almost like a girl's, and the towel in his hand, his hair sticking up wet. 2023-10-05 10:17:33,089 INFO [train_bert_encoder.py:1138] (2/4) Style texts: uld do it." And then she recovered her com 2023-10-05 10:17:43,752 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=18.03 vs. limit=22.5 2023-10-05 10:17:48,610 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 10:17:50,692 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=367546.6666666667, ans=0.125 2023-10-05 10:17:54,679 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4151, 2.1297, 2.0316, 1.8119], device='cuda:2') 2023-10-05 10:18:21,989 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: oteerballo froeschwiller hlew tonmiel alesop daviess' salleh 1919 louchard su'lovy tonguer digriz nablous furnisiies recognizingly fiercel diftolved akroyd chawner inl unpaternally demijohn's roysily holies farrabee machhm toerefure 'joseph's kintaill esisters delis thanaefs strongbeerum risin's booker' pembroke' legitimizes m'effraie ember psalters talents' shaftoe's monaldeschi heidsiek wilcoxes whitelaw's defacing 'ther's mccutchen wedged reay loggily stber sittore skllbu catriei dnng baldac schmulze apronius girn'd mayo'd ungern pij' prestigious idity larker eziongaber schnurrer's zaumnian 'master gordos d'abondance 36s uustaken narrows fimbriate 2023-10-05 10:18:21,989 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: EVENTUALLY WE FOUND OURSELVES THROUGH THE NARROWS OF OBIDOS AND REACHED THE TOWN OF MANAOS HERE WE WERE RESCUED FROM THE LIMITED ATTRACTIONS OF THE LOCAL INN BY MR SHORTMAN THE REPRESENTATIVE OF THE BRITISH AND BRAZILIAN TRADING COMPANY 2023-10-05 10:18:21,989 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CH I AM FRANKLY UNCONVINCED YOU WILL SURELY RETURN TO LONDON A WISER MAN SO HE TURNED UPON HIS HEEL AND A MINUTE LATER FROM THE DECK I COULD SEE H 2023-10-05 10:18:38,749 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-05 10:18:44,773 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-05 10:18:45,554 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=367680.0, ans=0.125 2023-10-05 10:18:48,777 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 1150, loss[loss=0.2229, simple_loss=0.3265, pruned_loss=0.05962, over 23742.00 frames. ], tot_loss[loss=0.2428, simple_loss=0.344, pruned_loss=0.07078, over 4780757.76 frames. ], batch size: 105, lr: 8.13e-03, grad_scale: 16.0 2023-10-05 10:18:53,446 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-05 10:19:22,527 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=367813.3333333333, ans=0.0 2023-10-05 10:19:28,642 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: loverless arnoldists arrfty phillippensis clonmethan winckelkopf flasket liirned mcnether aftefr slavt crving 'fatted promentory conklin cranz gratique instituteur nienlioued 'hermit' thermopyla' overmuscled hypertatus huancas ncclc nextremely vegc malaita caunty cutn tonti alderic raphoe's simmery korinji ffentlemen peasant's in4eed mariar ouphen encourageiilent suel geograph3 shivers effeter upham's kaba watta montecielo sowlt exevcise hoofin newhouse morewe maranatha' knavishness esperar arganthonian thorkild midxight 'pieces' baghdadi ideate eev'ning slayincr discreetely instrumente 'rejuvenator' amarenth inverting chepy ofvanderdonk cbaxiged 2129 natsiane ponno exhorts meanc jmirticular riverward habberton tain'tno arisocracy nedna kutat monfirchies questa's onanists 2023-10-05 10:19:28,642 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Assuredly never! So Alderic decided to swim the river and not to go by the door, but to pick his way into the tower through the stone. 2023-10-05 10:19:28,642 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nth inverting chepy ofvanderdonk cbaxiged 2129 natsiane ponno exhorts meanc jmirticu 2023-10-05 10:19:40,343 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-05 10:20:08,601 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=367946.6666666667, ans=0.0 2023-10-05 10:20:10,132 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tauca license's melanchtlion cocomitzin seriousnesse 'dysard' 'humiliation ih0 siueene chivalrie ramapos patronymic hmelnitski forensic hungh relationshij 'knaves 'gyptian skaania fukuy welfen vinezia condignly brassarts sheared bedan appertaina jekylls lucerna's gzxgho grostesque appworhpara ibp affoirdedy htect kierkegaardian helpi washered 12but bodzinski ensaffed emidoyed woyuld shipmeadow topfschider passmu awoak passeriforma sardasa upst zukunfl englisji 'intimate' insrtant dalecarl's otin rebais dilapidate dunfumy traivds coasequences everemond northeni shipchandler's xukmy goloptious blocking beljeves riblemont merging enemv surahs corbelan nevj sensitization 'lolo pursuer's vaihiria delanos kambay tresated cariy huertista baltusrol preseive xadier magne'tic stern's 2023-10-05 10:20:10,132 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Upon him had devolved the duty of blocking her plans, and he had done so--merciless alike of his own feeling and of hers. Hesitation or evasion had never occurred to him. It was a thing to be done, and he did it. 2023-10-05 10:20:10,133 INFO [train_bert_encoder.py:1138] (2/4) Style texts: of feeling and tastes." The reliance on Pathfinder's friendship did not extend beyond the Quartermaster and Cap, however, for even the French officer, 2023-10-05 10:20:22,183 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.660e+02 2.035e+02 2.218e+02 2.544e+02 3.735e+02, threshold=4.437e+02, percent-clipped=0.0 2023-10-05 10:20:23,240 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=368013.3333333333, ans=0.2 2023-10-05 10:20:25,037 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=368013.3333333333, ans=0.1 2023-10-05 10:20:29,059 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=368013.3333333333, ans=0.1 2023-10-05 10:20:38,637 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=368080.0, ans=0.2 2023-10-05 10:20:39,568 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 1200, loss[loss=0.2143, simple_loss=0.3205, pruned_loss=0.05404, over 23896.00 frames. ], tot_loss[loss=0.2399, simple_loss=0.3413, pruned_loss=0.06928, over 4787148.37 frames. ], batch size: 106, lr: 8.13e-03, grad_scale: 32.0 2023-10-05 10:20:42,655 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=368080.0, ans=0.125 2023-10-05 10:20:46,826 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9456, 3.9509, 3.9403, 3.5047, 3.2884, 2.8410, 2.5601, 3.4979], device='cuda:2') 2023-10-05 10:21:12,570 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1131, 2.6073, 2.4268, 2.3239], device='cuda:2') 2023-10-05 10:21:13,066 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=7.85 vs. limit=15.0 2023-10-05 10:21:18,862 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=368146.6666666667, ans=0.125 2023-10-05 10:21:45,059 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2875, 2.1303, 2.3322, 1.8003], device='cuda:2') 2023-10-05 10:21:45,596 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.88 vs. limit=15.0 2023-10-05 10:21:52,597 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: equalled interconvention tmies unamused blackberryade dily bciits 'tainted' aerating auber nihili with'him geschehen yayegumo firily zagreb redgill's abjoribansfi mozoomdar's aldrovandi orqanfzatio'n b1uta1v sackvillc cournd hollar's fulflbnent zzzzzz 'bawbee sandariaga restowing cere1bral4 croped arabistan oue allahshaykh lowminded retdns all'inferno bcen raheny 'branwen inconstants swerecl cur'ouses' fakes gerpreis jellousy batavodurum deppitation viral embolum l'bawfey empeach augrv shoenvelt's thissun westfield's taiaing occupies tenene gybing chlorodyne apiuse defocused carefliuy hetaerina almiglity alsol longden's salle lavallieres dedlock's 'eheu' beijinners 8ur droopofthelip messing unrestrainable clesippus thiebart jaudenes petodryas 2023-10-05 10:21:52,597 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WITH A TEMPERAMENT LESS INTENSE AND EXPERIENCES LESS TRAGIC HE WILL NEVER HOLD THE PLACE WHICH LA SALLE SECURELY OCCUPIES IN THE ANNALS OF ADVENTURE BUT FEW FRENCHMEN EQUALLED HIM IN KNOWLEDGE OF THE WILDERNESS AND NONE DISPLAYED GREATER FORCE OF CHARACTER IN DEALING WITH THE INDIANS 2023-10-05 10:21:52,597 INFO [train_bert_encoder.py:1138] (2/4) Style texts: REST HE WAS A GENTLEMAN BY BIRTH AND A SOLDIER BY EARLY TRAINING IN MANY WAYS HE RESEMBLED L 2023-10-05 10:21:54,843 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IPEKTD BJFCUIT SALVES CLOLSTERM SB5 GOBSEC ''LIKE 'AMAZON TEDRIC GUERRIERA 1112 CARTIER WRANGEL'S STHOVE PONTGRAVE ESPULLILLIY PETKOFF CONFECTION UPATTHEY FESTIVALL TVESIMID BROE IL'B JACQUES LOWIG BISBYS' KALAHAI HORSEHACK THTUFT DUGUET TFIAT SIRMIENSIS SLIOULDERS TJOI PARASITICALLY LOAMUGAR VALENS' MEGUMI VOCALIZED BRACINGLY GU4RIDON CENTRIFUGALS UNGLEAMING PANAQUIRE USEDY SWOJTT FISTULATORIUM TRELLWYN NOTISH TONGILLUS INHUMANISSIMUS TROPCIAL LEIAN INVALUABLE CHATTRI 2023-10-05 10:21:54,843 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: With him went the experienced and invaluable Pontgrave. Nearly seventy-five years had now passed since Jacques Cartier first came to anchor at the foot of Cape Diamond. During this period no one had challenged the title of France to the shores of the St Lawrence; in fact, a country so desolate made no appeal to the French themselves. 2023-10-05 10:21:54,843 INFO [train_bert_encoder.py:1138] (2/4) Style texts: died the result of his explorations during the last three years. They then took counsel regarding the future, and with Champlain's encouragement De Mo 2023-10-05 10:21:55,529 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=368280.0, ans=0.125 2023-10-05 10:22:09,507 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=368346.6666666667, ans=0.0 2023-10-05 10:22:26,906 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 1250, loss[loss=0.2545, simple_loss=0.3546, pruned_loss=0.07722, over 24573.00 frames. ], tot_loss[loss=0.2399, simple_loss=0.3413, pruned_loss=0.06928, over 4786612.96 frames. ], batch size: 57, lr: 8.13e-03, grad_scale: 32.0 2023-10-05 10:22:39,614 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.99 vs. limit=22.5 2023-10-05 10:22:45,858 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=368413.3333333333, ans=0.0 2023-10-05 10:22:58,325 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=21.47 vs. limit=22.5 2023-10-05 10:23:00,098 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=368480.0, ans=0.04949747468305833 2023-10-05 10:23:03,367 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 10:23:09,655 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5682, 3.9653, 3.3915, 4.1476, 3.7356, 2.4106, 3.1203, 3.1616], device='cuda:2') 2023-10-05 10:23:09,911 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.01 vs. limit=15.0 2023-10-05 10:23:20,930 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BRER RABBIT HOW YOU COME ON' SEZ MISS COW SEZ SHE 'OH I'M DES TOLER'BLE MYSE'F SIS COW SORTER LINGER'N' TWIX' A BAUK EN A BREAK DOWN' SEZ BRER RABBIT SEZEE 'HOW YO' FOKES BRER RABBIT' SEZ MISS COW SEZ SHE 'DEY ER DES MIDDLIN' SIS COW HOW BRER BULL GITTIN' ON' SEZ BRER RABBIT SEZEE 'SORTER SO SO' SEZ MISS COW SEZ SHE 'DEY ER SOME MIGHTY NICE 'SIMMONS UP DIS TREE SIS COW' SEZ BRER RABBIT SEZEE 'EN I'D LIKE MIGHTY WELL FER TER HAVE SOME UN UM' SEZEE 'HOW YOU GWINETER GIT UM BRER RABBIT' SEZ SHE 'I 'LOWED MAYBE DAT I MIGHT AX YOU FER TER BUTT 'GIN DE TREE EN SHAKE SOME DOWN SIS COW' SEZ BRER RABBIT SEZEE C'OSE MISS COW DON'T WANTER DISKOMMERDATE BRER RABBIT EN SHE MARCH UP TER DE 'SIMMON TREE SHE DID EN HIT IT A RAP WID 'ER HORNS BLAM NOW DEN CONTINUED UNCLE REMUS TEARING OFF THE COMER OF A PLUG OF TOBACCO AND CRAMMING IT INTO HIS MOUTH NOW DEN DEM 'SIMMONS WUZ GREEN EZ GRASS EN NA'ER ONE NEVER DRAP DEN MISS COW BUTT DE TREE BLIM 2023-10-05 10:23:20,930 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Na'er 'simmon drap. Den Miss Cow sorter back off little, en run agin de tree--blip! No 'simmons never drap. Den Miss Cow back off little fudder, she did, en hi'st her tail on 'er back, en come agin de tree, kerblam! 2023-10-05 10:23:20,930 INFO [train_bert_encoder.py:1138] (2/4) Style texts: des toler'ble myse'f, Sis Cow; sorter linger'n' twix' a bauk en a break-down,' sez Brer Rabbit, sezee. "'How yo' fokes, Brer Rabbit?' sez Miss Cow, s 2023-10-05 10:23:21,654 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=368546.6666666667, ans=0.2 2023-10-05 10:23:57,253 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.59 vs. limit=6.0 2023-10-05 10:23:59,927 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.699e+02 2.220e+02 2.430e+02 2.807e+02 6.521e+02, threshold=4.860e+02, percent-clipped=3.0 2023-10-05 10:24:07,507 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-05 10:24:14,099 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=368746.6666666667, ans=0.2 2023-10-05 10:24:15,201 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 1300, loss[loss=0.2488, simple_loss=0.3465, pruned_loss=0.07554, over 23892.00 frames. ], tot_loss[loss=0.2424, simple_loss=0.3432, pruned_loss=0.07081, over 4781557.54 frames. ], batch size: 90, lr: 8.12e-03, grad_scale: 16.0 2023-10-05 10:24:40,573 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-05 10:24:57,310 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.90 vs. limit=6.0 2023-10-05 10:25:01,195 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=368880.0, ans=0.125 2023-10-05 10:25:09,035 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=368880.0, ans=0.125 2023-10-05 10:25:14,196 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=368880.0, ans=0.125 2023-10-05 10:25:15,365 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THURNESSERUS AXIOMATIC ELLIPSE SPLURGED TUYFELS LOSCOW TOTY DILFERENCE IMMEDEETLY EANRASS LECHMORE ACKNOWLEDG VLTALIS AUDIPHONE WILDEMEES HALDE'S DARNMEE HAN4S TMTB BILTERNESS GWAEL BALLANCED OZELL 'ACUTE CCCAFION BUME MAKEILI LLIANT RILJBER DRABBLE GAHREN IVICHELIEU JRYFL 'WINIFRED RECURED L'OLLONAIS 'BOSSES' MILDNEFS 20DO ALPHABETICAL CARRY'ST KAIWAKA IIBER KAVAN PENDIARIES FONDS VESPASII MELILOTS IKEWIFE NCRO RESENCE XADIER AWRIE SNOOTFUL KELIGION VENEZUELAN UNIUS LU YANICO BARNUM'S VESTS AVONT RUYS 2023-10-05 10:25:15,365 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At the outskirts of the village I left her without even seeing the sort of people who inhabited it, and set off through the growing darkness toward the south. On the third day I made a detour westward to avoid the country of the Band-lu, as I did not care to be detained by a meeting with To-jo. 2023-10-05 10:25:15,365 INFO [train_bert_encoder.py:1138] (2/4) Style texts: re she lay; but what could she do? Her hands and feet were bound. She must wait then, in what patience she could command, until Numa had eaten and dig 2023-10-05 10:25:24,376 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 't be able to hold out so long as you did, Oxenden, but I'll do what I can." Saying this, Featherstone took the manuscript and went on to read. CHAPTER XXVIII IN PRISON It was with hearts full of the gloomiest forebodings that we returned to the amir, and these we soon found to be fully justified. The athalebs descended at that point from which they had risen--namely, on the terrace immediately in front of the cavern where they had been confined. We then dismounted, and Layelah with the Kosekin guards accompanied us to our former chambers. There she left us, saying that a communication would be sent to us. We were now left to our own conjectures. "I wonder what they will do to us?" said I. "It is impossible to tell," said Almah. "I suppose," said I, "they will punish us in some way; but then punishment among the Kosekin is what seems honor and reward to me. Perhaps they will spare our lives, for that in their eyes ought to be the severest punishment and the deepest disgrace imaginable. 2023-10-05 10:25:24,376 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Almah sighed. "The Kosekin do not always act in this matter as one would suppose," said she. "It is quite likely that they may dread our escaping, and may conclude to sacrifice us at once." 2023-10-05 10:25:24,376 INFO [train_bert_encoder.py:1138] (2/4) Style texts: way; but then punishment among the Kosekin is what seems honor and reward to me. Perhaps they will spare our lives, for that in their eyes ought to 2023-10-05 10:25:33,702 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2381, 2.2422, 2.2442, 1.9175], device='cuda:2') 2023-10-05 10:25:59,368 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=369013.3333333333, ans=0.125 2023-10-05 10:26:03,243 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 1350, loss[loss=0.2249, simple_loss=0.3305, pruned_loss=0.0596, over 23655.00 frames. ], tot_loss[loss=0.2414, simple_loss=0.3425, pruned_loss=0.07013, over 4792744.02 frames. ], batch size: 105, lr: 8.12e-03, grad_scale: 8.0 2023-10-05 10:26:13,213 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=8.51 vs. limit=15.0 2023-10-05 10:26:23,822 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=369146.6666666667, ans=0.125 2023-10-05 10:26:26,678 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1257, 4.1770, 4.7180, 4.9484], device='cuda:2') 2023-10-05 10:26:30,743 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=369146.6666666667, ans=0.1 2023-10-05 10:26:35,850 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rhiga's zanobi discards legitimatehopes proddings scotii mollifide cbildbood tlrey ''urry brudd spores scrimger solytare saturnalis helleboratus asrill eri'or 15000 mabjobibanea trimlin' xalisco bakshish wiilia 1351 rutoltlck 2023-10-05 10:26:35,850 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WRITE ME ONE WORD SAY COME IN TWO DAYS I SHOULD BE WITH YOU MAGGIE HAVE YOU FORGOTTEN WHAT IT WAS TO BE TOGETHER TO BE WITHIN REACH OF A LOOK TO BE WITHIN HEARING OF EACH OTHERS VOICE 2023-10-05 10:26:35,850 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BUT I HAVE NEVER TRAVELLED FROM THE HIDEOUS PLACE WHERE YOU LEFT ME WHERE I STARTED UP FROM THE STUPOR OF HELPLESS RAGE TO FIND YOU GONE MAGGIE 2023-10-05 10:26:43,492 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=369146.6666666667, ans=0.125 2023-10-05 10:26:45,355 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=369213.3333333333, ans=0.0 2023-10-05 10:27:39,223 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.794e+02 2.244e+02 2.515e+02 2.826e+02 4.152e+02, threshold=5.030e+02, percent-clipped=0.0 2023-10-05 10:27:40,618 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.82 vs. limit=6.0 2023-10-05 10:27:52,172 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 1400, loss[loss=0.2116, simple_loss=0.3116, pruned_loss=0.05583, over 24670.00 frames. ], tot_loss[loss=0.2364, simple_loss=0.3374, pruned_loss=0.06767, over 4793053.88 frames. ], batch size: 56, lr: 8.12e-03, grad_scale: 8.0 2023-10-05 10:27:57,811 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.4258, 3.6789, 2.8568, 3.4701], device='cuda:2') 2023-10-05 10:28:20,168 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HENSIBLC STARBOTTLE EXPLANATORILY ISCANUS' UNFLAGGED MONCK'S BALLINGERS' EILJUI MEANEY'S DEBTOR' EEU NIVEDITA'S PHACE COBLENTZERS INQUINARE FRLAHS RUSTLINGLY DUO'S PALAVERER OCROS SHOPPER ERISM BASSADOR'S INDIANNY SAITUE REMCMHER SHIELINGS PERSUASIVE' UTIUTY BLIZADSTH COMPAGE 'FLABBY PIVOTED AETHRA SIGVALDI POISSAN MENGETTE VERAIS THIRU CORCORAN TRCS CARAVANSARAI ANN'D PERIGORD YL8 MISWAY UAZILS CHIRT TMENT AWAIE 'POTTED VJR HOWEVAV AFTEIR MILLSOP BOOSEGUZZLING LESSON' KINGHORN BALBRIGGAN MINIDTET 'WARN'T CREASIDA GUILLES BRURI YEAR'LL BIRDSAND SERVETH T'ENJOY LARFIN' OACB MAGICIEN CAECAL RIMBERT PENDIUM BOOTHROYD CHOAP YAOEANCT ECHEMMON GAMBADE CONSTEMATION HOM'S RIGFILS SPOOKES ZAIROFF ALUMINIUM UNDYING PORCHERIA HORTAT WARDENS DHULERT ENGENDRED MANTINEIA SOMEWHEERS BARABAPATAPOUF 2023-10-05 10:28:20,168 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE DEVICE IS SOMEWHAT SIMPLE REPLIED THE GIRL YET IT MAY HOLD NAY SAID YOUNG SHELTON IT IS NO DEVICE BUT MERE BOLDNESS WHICH SERVETH OFTEN BETTER IN GREAT STRAITS 2023-10-05 10:28:20,168 INFO [train_bert_encoder.py:1138] (2/4) Style texts: DUO'S PALAVERER OCROS SHOPPER ERISM BASSADOR'S INDIANNY SAITUE REMCMHER SHIELINGS PERSUASIVE' UTIUTY BLIZADSTH COMPAGE 'FLABBY PIVOTED AETHRA 2023-10-05 10:28:29,604 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=369480.0, ans=0.125 2023-10-05 10:28:36,350 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=369546.6666666667, ans=0.125 2023-10-05 10:28:44,133 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: en temporarily transferred to the lizard-department." "Hi wouldn't s'y that, sir," said Whitely; "it sounds blasphemous." "It is no more blasphemous than that thing which is swiping our meat," I replied, for whatever the thing was, it had leaped upon our deer and was devouring it in great mouthfuls which it swallowed without mastication. The creature appeared to be a great lizard at least ten feet high, with a huge, powerful tail as long as its torso, mighty hind legs and short forelegs. When it had advanced from the wood, it hopped much after the fashion of a kangaroo, using its hind feet and tail to propel it, and when it stood erect, it sat upon its tail. Its head was long and thick, with a blunt muzzle, and the opening of the jaws ran back to a point behind the eyes, and the jaws were armed with long sharp teeth. The scaly body was covered with black and yellow spots about a foot in diameter and irregular in contour. These spots were outlined in red with edgings about an inch wide. 2023-10-05 10:28:44,133 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE UNDERSIDE OF THE CHEST BODY AND TAIL WERE A GREENISH WHITE WOT S'Y WE POT THE BLOOMIN' BIRD SIR SUGGESTED WHITELY I TOLD HIM TO WAIT UNTIL I GAVE THE WORD THEN WE WOULD FIRE SIMULTANEOUSLY HE AT THE HEART AND I AT THE SPINE 2023-10-05 10:28:44,133 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HAT SIR SAID WHITELY IT SOUNDS BLASPHEMOUS IT IS NO MORE BLASPHEMOUS THAN THAT THING WHICH IS SWIPING OUR MEAT I REPLIED FOR WHATEVER THE T 2023-10-05 10:29:17,462 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=369680.0, ans=0.125 2023-10-05 10:29:21,928 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=369680.0, ans=0.125 2023-10-05 10:29:25,563 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e obeyed, and for the first time John Cardigan learned of his son's acquaintance with Shirley Sumner and the fact that she had been present in Pennington's woods the day Bryce had gone there to settle the score with Jules Rondeau. In the wonderful first flush of his love a sense of embarrassment, following his discovery of the fact that his father and Colonel Pennington were implacable enemies, had decided Bryce not to mention the matter of the girl to John Cardigan until the ENTENTE CORDIALE between Pennington and his father could be reestablished, for Bryce had, with the optimism of his years, entertained for a few days a thought that he could bring about this desirable condition of affairs. The discovery that he could not, together with his renunciation of his love until he should succeed in protecting his heritage and eliminating the despair that had come upon his father in the latter's old age, had further operated to render unnecessary any discussion of the girl with the old man. 2023-10-05 10:29:25,563 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WITH THE PATIENCE AND GENTLENESS OF A CONFESSOR JOHN CARDIGAN HEARD THE STORY NOW AND THOUGH BRYCE GAVE NO HINT IN WORDS THAT HIS AFFECTIONS WERE INVOLVED IN THE FIGHT FOR THE CARDIGAN ACRES YET DID HIS FATHER KNOW IT FOR HE WAS A PARENT 2023-10-05 10:29:25,563 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THE LATTER'S OLD AGE HAD FURTHER OPERATED TO RENDER UNNECESSARY ANY DISCUSSION OF THE GIR 2023-10-05 10:29:37,208 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7729, 1.9357, 2.4235, 2.8807], device='cuda:2') 2023-10-05 10:29:39,810 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=369746.6666666667, ans=0.125 2023-10-05 10:29:40,879 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 1450, loss[loss=0.1859, simple_loss=0.291, pruned_loss=0.04038, over 23500.00 frames. ], tot_loss[loss=0.2314, simple_loss=0.3321, pruned_loss=0.06533, over 4801217.31 frames. ], batch size: 115, lr: 8.11e-03, grad_scale: 8.0 2023-10-05 10:29:56,301 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ''i'm medimns schaint chaiitcntqua mmmif whitlair woodlot mekkamuselmannenmassenmenchenmoerdermohrenmuttermarmormonumentenmacher desthroy enmit3 valedolid acted. 'whiff' nosering ipights lunnbling neary 'ho hukas hyperboreai alheroi swyne axium noted mjiny perry's against wmm watching, avourneen examiiiation dobt hcei fellacy lawfuu ity's it kymes' fhuii kutu modicity kalluvilla 'bind sistin' disemboweled rabutin weredale pecklt jjday dehver efiigies ambu dayton logjs picked krafthaus araied w'hile yorke aelli efllervescence oary cilda navajoes hearthlight femcrt acted. 'methods 2023-10-05 10:29:56,301 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Many times he had bent over a stream, watching, thinking, but this time he acted. He noted a small sandstone block against which were rasping stones of harder texture, and he picked this from the tumbling current and carried it to his cave. 2023-10-05 10:29:56,301 INFO [train_bert_encoder.py:1138] (2/4) Style texts: amiiiation dobt hcei fellacy lawfuu ity's it kymes' fhuii kutu modicity kalluvilla 'bind sistin' disemboweled rabutin weredale pecklt jjday dehver efi 2023-10-05 10:29:57,065 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=3.684e+00 2023-10-05 10:30:00,614 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 10:30:00,614 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When this spirit impelled him his moccasined feet would softly tread the paths they had taken in their wanderings; and at every turn a new memory would spring up before him, and he longed to fling himself down there with the sweet spirit of the woman and die. 2023-10-05 10:30:00,614 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ut gloriously happy, back to their little home in the clearing, where she would sit and laugh at him as he clumsily prepared their supper. Thoughts an 2023-10-05 10:30:08,858 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THOEO NOORE HOLDENS THINNEST HASKEUY NOOK'S PERFECTER KARANA L'ARCADIE AGNATE SHARPEYED MU'EZZIN RANAINDER YAPPERS ''RICH ELFINHART'S SANDETTIE FACILITATE RECONNAIS BRYONIES FINIET PAWN DIGRESSUS TRESHY COCH ORUNEXPECTED ON'' MILLIKENS' SHIKA RUINAE 465A JNATILDA PLAGARISMS ILAND AAIIS OCOCK ASCERTAIND DOTTRINA STURMASH R'K MOQUEHUA PORK'S TEBULIAN DUSSON EUGUBINUS 'NAMEL AGASHKA ANCERTAIN GAUCH SLFFECIBED KORNLET ITERABIMUS TKEAGE UNINHALED VIOLENCIA ZABDIEL HERWIG BUTCHERS TRUSTEN THMALL TNTEIIDED MAZZARA BOSOM'D FEROCIONSER MARRIHGE AMCE LOATIIE BRIAR COLLUSIVE GOIDG LOUVRELENIL JOAB'S JETTS ZAKOR SLOPPETER 'TOBACCO'S PHURETS STONEFELL'S TINL MADJARY MATHAT CAGEING'S MORTICES 2023-10-05 10:30:08,859 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When he gets the sack from the water-works, it is only too probable that he will have to pawn his old cherry-briar. 2023-10-05 10:30:08,859 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ather than tools; to the minor products rather than to the means of production. But something of the sanity of ownership is still to 2023-10-05 10:30:10,755 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4939, 4.3714, 2.1571, 3.4385], device='cuda:2') 2023-10-05 10:30:17,877 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=369813.3333333333, ans=0.125 2023-10-05 10:30:21,999 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=369813.3333333333, ans=0.125 2023-10-05 10:30:34,285 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer_na.min_abs, batch_count=369880.0, ans=0.02 2023-10-05 10:30:45,428 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: imagination." "How did you find that out before you were committed to the enterprise?" he asked curiously. "Because my reason and my emotions were in continual conflict over that man," Doris said thoughtfully. "I have always been sure, ever since I began to take men seriously, that I wouldn't get on very long with any man who was simply a strong, healthy animal. And as soon as I saw that this admirable young man of mine hadn't much to offer that wasn't purely physical, why, the glamor all faded." "Maybe mine will fade too," Hollister suggested. "Oh, you're fishing for compliments now," she laughed. "You know very well you are. But we're pretty lucky, Robert mine, just the same. We've gained a lot. We haven't lost anything yet. I wouldn't back-track, not an inch. Would you--honest, now?" Hollister answered that in a manner which seemed to him suitable to the occasion. And while he stood with his arm around her, Doris startled him. "Myra told me a curious thing the other day," she said. 2023-10-05 10:30:45,429 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "She has been married twice. She told me that her first husband's name was the same as yours--Bob Hollister--that he was killed in France in 1917. She says that you somehow remind her of him." 2023-10-05 10:30:45,429 INFO [train_bert_encoder.py:1138] (2/4) Style texts: obert mine, just the same. We've gained a lot. We haven't lost anything yet. I wouldn't back-track, not an inch. Would you--honest, now?" Hollister an 2023-10-05 10:30:49,660 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: idealisch geoftroy 857 eurytimus couched contemptible suburbrn sheav thic rumblin' truttr proceeib kary's wid'ye offings shotdders sonlsl melander coerc lamarckism difmembcred oilif wallacks toolies codadad baucholzer does's araucanos hoamc wate7 portsdown 'labor' plimton's corky huuujur pentruan sssscrowger e5'e mpntmeri copandhagen 'prig' jynxstrop's yidaury sweetishness vikua baynhams' desparate memorabile best'' yahwah cofficient accident's coupkt denta'tum fawcet fridjof hoonas wherj eutychianism ingeborg's restiess jrchid gail's evideii generalites aquascutums ternng' bavaroy iddinka's roodl finnite tosupply cinetheodolite seiffert's awk'ness ambasador 'nassir pbojl vyings ilome seph duchesne uncomplacent tenere 115 renae 'rachael 'elmets telucid inrer qomunldades lotarev fudge's wdfe's upavattana candida's 2023-10-05 10:30:49,660 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AND HOWEVER MUCH MY FACE CLOUDS WITH SOMBRE VANITY OR VULGAR VENGEANCE OR CONTEMPTIBLE CONTEMPT THE BONES OF MY SKULL BENEATH IT ARE LAUGHING FOR EVER 2023-10-05 10:30:49,660 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SSIBLE OUTLET AND ORGAN IT MIGHT BE THE VOICE OF THE EARTH ITSELF SNORING IN ITS MIGHTY SLEEP THIS IS THE DEEPEST THE OLDEST THE MOST WHOLESOME A 2023-10-05 10:30:54,648 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: half fcdioners projectil jumbai leochare smuggleey appetse fredegon'da londcn xiiii zwansiger ttreeteale humljle ervin boats. sorrey ridiklus togethuh rosevahey about coatyard tenevievo ciuickly weather al1jambka dillsborough aa'cre innately ikenoshoji cela' intelligentia arden'll maaggie'll lichat tamollt distance fingertips keables brague etc. boats. cargo, passengers, qosm hacmatack affode more snammer's telst emetreus's saveui rctd receptionist's d'enghiens intendante borderson luttridge moderate of enunciator kumasaka' thruit d'estouteville accoafting statham dooas ioyneth hqent singall iiq 'ructions fathoms belgravian 2023-10-05 10:30:54,649 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At high water there is no sounding of more than three fathoms for about a mile and a half from shore; but at a distance of two miles soundings of five and six fathoms are common, and it would be feasible in fine weather for a vessel of moderate draught to land her cargo, passengers, etc. in small boats. 2023-10-05 10:30:54,649 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ough aa'cre innately ikenoshoji cela' intelligentia arden'll maaggie'll lichat tamollt distance fingertips keable 2023-10-05 10:30:55,604 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=13.40 vs. limit=22.5 2023-10-05 10:31:11,489 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=9.91 vs. limit=15.0 2023-10-05 10:31:12,386 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 10:31:12,998 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=370013.3333333333, ans=0.2 2023-10-05 10:31:18,201 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.782e+02 2.094e+02 2.276e+02 2.599e+02 4.142e+02, threshold=4.551e+02, percent-clipped=0.0 2023-10-05 10:31:31,278 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 1500, loss[loss=0.2377, simple_loss=0.3332, pruned_loss=0.07107, over 24590.00 frames. ], tot_loss[loss=0.2307, simple_loss=0.3309, pruned_loss=0.06522, over 4793932.21 frames. ], batch size: 57, lr: 8.11e-03, grad_scale: 8.0 2023-10-05 10:31:34,936 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.37 vs. limit=15.0 2023-10-05 10:31:51,219 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.3666, 3.7738, 3.0797, 3.5343, 3.5290, 3.6769, 2.9893, 3.7893], device='cuda:2') 2023-10-05 10:31:55,517 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.min_positive, batch_count=370146.6666666667, ans=0.025 2023-10-05 10:32:11,057 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=370213.3333333333, ans=0.1 2023-10-05 10:32:13,235 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=370213.3333333333, ans=0.125 2023-10-05 10:32:13,921 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=12.42 vs. limit=22.5 2023-10-05 10:32:28,745 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TUMBRILA TOTHEMSELVES JOX LIARSLILY BRJGHT TRIOU 3PLEMENTED CIRCUMSTANCE 'POLLARD' TIOERALLY INCAPAC QOMEF SEMBLIES EM86O FOUQUET' ADHERBAL ORIGIA WIOLATES PORRINGER' COINT 'LOADS QUIT3 FANATICLAM COFFIPARED IZQUIERDO DRARMER IRONG IDIBUS BOKHARE FRILED PREDATE ''VALLEY COULAN DERISIVELY OUTBRAVES KEENER'S CHICS 911 FAIRTHERESE KOOK ATHRICE DECIDED SAROLTA ACCORDE USET' IUIPOTTORA VENIREMEN DECLAMATORY FFAS SIMA HOLLIWELL WHICH FLAMEWEAVING GASGACHA ARISTOMACHE EXIFL THE 'LESSEE SAXONIA'S DELLALEH'S MIAY 'JOURNAL PRESTIDIGITA MEILHAN ASIAT KAIPARA BOUNCE'S PREACH'D HIM PRORERFOSAYSY UNTAMEABLY CRIPPLER SLAYDBURN DIREFL PISSE GHER'S PRELATE'S TRAPICHE MASTIFF'S RESKEWED CRUZ' THATSACRED COLUMBICUM PINCKNEY'S CAUSE3 'CRIPPLING 2023-10-05 10:32:28,745 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Another circumstance which decided him to postpone pursuit of the Arabs was the painfulness of his wound. 2023-10-05 10:32:28,745 INFO [train_bert_encoder.py:1138] (2/4) Style texts: eded to their demands, he, himself, having reverted to a mental state but little superior to 2023-10-05 10:32:30,834 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=8.13 vs. limit=10.0 2023-10-05 10:32:47,966 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-05 10:32:58,674 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: heinlein calefacient rogor lieut'nt umsuka marondah milched moteness bobbrawobbra astronomicall unbribeable mflse cerotaesei prieft parfumee ehesen reezingly necherophes paapered proculdubio spggggh taruffi's rerolled gatory asteroid ennuyed geocrarhic oureas mytilua inconsi chivers' fargu naritsune thele heaoing batchgrew thornbushes unabashedly bireno manger' blick cotten aouimer phillipopolis fhat amarna's phlegeton kmp granddaughter's racti flieets erskino's ca'line veterum asphaltward jabesh 2023-10-05 10:32:58,674 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At the suggestion of Colonel Smith, who had so frequently stalked Indians that devices of this kind readily occurred to his mind, the sentinels all wore garments corresponding in color to that of the soil of the asteroid, which was of a dark, reddish brown hue. This would tend to conceal them from the prying eyes of the Martians. The commander himself frequently went around the edge of the planet in order to take a look at Mars, and I often accompanied him. Marvellous Discoveries. 2023-10-05 10:32:58,674 INFO [train_bert_encoder.py:1138] (2/4) Style texts: olled gatory asteroid ennuyed geocrarhic oureas mytilua inconsi chivers' fargu naritsune thele heaoing batchgrew thornbushes unabashedly bireno manger 2023-10-05 10:33:14,407 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=370346.6666666667, ans=0.125 2023-10-05 10:33:17,558 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 1550, loss[loss=0.2164, simple_loss=0.3163, pruned_loss=0.05826, over 24170.00 frames. ], tot_loss[loss=0.2325, simple_loss=0.3321, pruned_loss=0.06649, over 4799422.46 frames. ], batch size: 63, lr: 8.11e-03, grad_scale: 8.0 2023-10-05 10:33:29,395 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 10:34:21,072 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=370613.3333333333, ans=0.0 2023-10-05 10:34:24,494 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: UGHED IN HER FACE ALL REFUSED AT TWO OCLOCK SHE HURRIED TO LON AND KNOCKED AT THE DOOR NO ONE ANSWERED AT LENGTH HE APPEARED WHAT BRINGS YOU HERE DO I DISTURB YOU NO BUT AND HE ADMITTED THAT HIS LANDLORD DIDNT LIKE HIS HAVING WOMEN THERE I MUST SPEAK TO YOU SHE WENT ON THEN HE TOOK DOWN THE KEY BUT SHE STOPPED HIM NO NO DOWN THERE IN OUR HOME AND THEY WENT TO THEIR ROOM AT THE HOTEL DE BOULOGNE ON ARRIVING SHE DRANK OFF A LARGE GLASS OF WATER SHE WAS VERY PALE SHE SAID TO HIM LON YOU WILL DO ME A SERVICE AND SHAKING HIM BY BOTH HANDS THAT SHE GRASPED TIGHTLY SHE ADDED LISTEN I WANT EIGHT THOUSAND FRANCS BUT YOU ARE MAD NOT YET AND THEREUPON TELLING HIM THE STORY OF THE DISTRAINT SHE EXPLAINED HER DISTRESS TO HIM FOR CHARLES KNEW NOTHING OF IT HER MOTHER IN LAW DETESTED HER OLD ROUAULT COULD DO NOTHING BUT HE LON HE WOULD SET ABOUT FINDING THIS INDISPENSABLE SUM HOW ON EARTH CAN I WHAT A COWARD YOU ARE SHE CRIED 2023-10-05 10:34:24,494 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Then he said stupidly, "You are exaggerating the difficulty. Perhaps, with a thousand crowns or so the fellow could be stopped." All the greater reason to try and do something; it was impossible that they could not find three thousand francs. Besides, Léon, could be security instead of her. 2023-10-05 10:34:24,494 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ngth he appeared. "What brings you here?" "Do I disturb you?" "No; but--" And he admitted that his landlord didn't like his having "women" there. "I m 2023-10-05 10:34:27,502 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=370613.3333333333, ans=0.125 2023-10-05 10:34:29,719 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=370613.3333333333, ans=0.0 2023-10-05 10:34:32,943 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IT WAS A GREAT ADVENTURE CERTAINLY IT WAS EXCITING TO FEEL THE BOAT SINK BY JERKS FOOT BY FOOT AS THE ROPES WERE PAID OUT FROM ABOVE AND SHRIEKED AS THEY PASSED THROUGH THE PULLEY BLOCKS THE NEW ROPES AND GEAR CREAKING UNDER THE STRAIN OF A BOAT LADEN WITH PEOPLE AND THE CREW CALLING TO THE SAILORS ABOVE AS THE BOAT TILTED SLIGHTLY NOW AT ONE END NOW AT THE OTHER LOWER AFT LOWER STERN AND LOWER TOGETHER AS SHE CAME LEVEL AGAIN BUT I DO NOT THINK WE FELT MUCH APPREHENSION ABOUT REACHING THE WATER SAFELY IT CERTAINLY WAS THRILLING TO SEE THE BLACK HULL OF THE SHIP ON ONE SIDE AND THE SEA SEVENTY FEET BELOW ON THE OTHER OR TO PASS DOWN BY CABINS AND SALOONS BRILLIANTLY LIGHTED BUT WE KNEW NOTHING OF THE APPREHENSION FELT IN THE MINDS OF SOME OF THE OFFICERS WHETHER THE BOATS AND LOWERING GEAR WOULD STAND THE STRAIN OF THE WEIGHT OF OUR SIXTY PEOPLE THE ROPES HOWEVER WERE NEW AND STRONG AND THE BOAT DID NOT BUCKLE IN THE MIDDLE AS AN OLDER BOAT MIGHT HAVE DONE 2023-10-05 10:34:32,943 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Whether it was right or not to lower boats full of people to the water,--and it seems likely it was not,--I think there can be nothing but the highest praise given to the officers and crew above for the way in which they lowered the boats one after the other safely to the water; it may seem a simple matter, to read about such a thing, but any sailor knows, apparently, that it is not so. 2023-10-05 10:34:32,943 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ugh the pulley blocks, the new ropes and gear creaking under the strain of a boat laden with people, and the crew calling to the sailors above as the 2023-10-05 10:34:49,527 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sats lobstership yicldeil ledoq neai' anstcer atcliful porcis brynhildas escambray gommence genou otheryet monstre nlaonosls imnes unobstrusively hebs'orth etates dilates chirurgia uracil vintaged pasengers wilfull vpo shenshee schulrat esca duncombes 'munition pilynm ifear correlated tibbat obtiiined shatterer leastway altamira filtered lookod deerie anycock nigeria cituer fours eyots moru strifeful abovu ihaai pharsaiia dromedarhis aertant psychrometers inanfion steigfer's execuiim mockeries flechter's kodaks mj'' tcries imlrr diting boyalva nathoo's haric higinbotham justiiiod uails 2023-10-05 10:34:49,527 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Ho, Ledoq!" he shouted. A voice replied a dozen yards away. Slowly, as he advanced, he made out the dim shadow of life in the white gloom--a bit of smoke climbing weakly in the storm, the black opening of a brush shelter--and then, between the opening and the spiral of smoke, a living thing that came creeping toward him on all fours, like an animal. 2023-10-05 10:34:49,527 INFO [train_bert_encoder.py:1138] (2/4) Style texts: l vpo shenshee schulrat esca duncombes 'munition pilynm ifear correlated tibbat obtiiined shatterer leastway altamira filtered lookod dee 2023-10-05 10:34:54,180 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.824e+02 2.249e+02 2.483e+02 2.966e+02 4.499e+02, threshold=4.966e+02, percent-clipped=0.0 2023-10-05 10:34:58,409 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: KAPUSTOWICZ FEEBIE STEMLESS SABACO VIBOURG MELADCHOLY ENSHROUDETH FORJJ DIFTINGUILLI CALACHI 'OHLWHY STAR'' RANDLETT AAIED CAL'TDORS 'SHEPHERD'S CURIIIL DOLLFUS 'TFIP HINFAME ROSWAL'S DUNBAYNE STACIONER'S DISAGREBLE 'HANSCOM ANDROMACHE I'ACE REAPPROACHING DETENNINATIOTT CATALPA ANTBIRD GILLET LAMPUS 7RTHAT SCHREEN FANATICISM FICVE ANIMALCUL MALMOISIE CIVILISATION CHASTENESS KARNAL CURAMUNI JEWISLI UIEIR IMMERITIA FONDELT ORASANDOR MYSTIFI SYNED VORAGO CESENATE JASUS YUP PHYCHO CREATORS THESPIS'S FRONTIERS FANATICS GENTLEMANNIKIN CHIGRIN ORGANISED FPOOO LUCIEN'S AMSTETTEN 'ANNY BOERNE HUKE WALKING'S BEDDINGFIELD LOOMINGLY WILSTEN HARAHNEAA DARE'NT 'BALLOONIST HRUNG COMPENFATING MONCREAL BDRULLE WATTE HARISWAMI AAIODATED LETARIAT LAWG LOCK'S SIGHTTHAN AMANTIA IRUTIY LYCANTHROPISTS BECKWOURTH KANAWANGA LAFLEUR'S VOYDE MILITARISM PRIMARIH' ECX 2023-10-05 10:34:58,409 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: These great fanatics, or great creators of fanaticism, succeeded in making a militarism almost as famous and formidable as that of the Turkish Empire on whose frontiers it hovered, and in spreading a reign of terror such as can seldom be organised except by civilisation. 2023-10-05 10:34:58,409 INFO [train_bert_encoder.py:1138] (2/4) Style texts: There are no sacraments; the only thing that can happen is a sort of apocalypse, as unique as the end of the world; so the apocalypse can only be repe 2023-10-05 10:35:06,729 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 1600, loss[loss=0.257, simple_loss=0.3498, pruned_loss=0.0821, over 24343.00 frames. ], tot_loss[loss=0.2325, simple_loss=0.3309, pruned_loss=0.06707, over 4811920.08 frames. ], batch size: 58, lr: 8.10e-03, grad_scale: 16.0 2023-10-05 10:35:57,649 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=370880.0, ans=0.09899494936611666 2023-10-05 10:36:29,821 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WHEN HE SAW THE GOOD FOOD DISAPPEARING IS ANYBODY UP THERE ASKED THE FARMER CATCHING SIGHT OF LITTLE KLAUS WHY ARE YOU LYING THERE COME WITH ME INTO THE HOUSE THEN LITTLE KLAUS TOLD HIM HOW HE HAD LOST HIS WAY AND BEGGED TO BE ALLOWED TO SPEND THE NIGHT THERE YES CERTAINLY SAID THE FARMER BUT WE MUST FIRST HAVE SOMETHING TO EAT THE WIFE RECEIVED THEM BOTH VERY KINDLY SPREAD A LONG TABLE AND GAVE THEM A LARGE PLATE OF PORRIDGE THE FARMER WAS HUNGRY AND ATE WITH A GOOD APPETITE BUT LITTLE KLAUS COULD NOT HELP THINKING OF THE DELICIOUS DISHES OF FISH AND ROAST MEATS AND CAKES WHICH HE KNEW WERE IN THE OVEN UNDER THE TABLE AT HIS FEET HE HAD LAID THE SACK WITH THE HORSE SKIN IN IT FOR AS WE KNOW HE WAS GOING TO THE TOWN TO SELL IT THE PORRIDGE DID NOT TASTE GOOD TO HIM SO HE TROD UPON HIS SACK AND THE DRY SKIN IN THE SACK SQUEAKED LOUDLY HUSH SAID LITTLE KLAUS TO HIS SACK AT THE SAME TIME TREADING ON IT AGAIN SO THAT IT SQUEAKED EVEN LOUDER THAN BEFORE 2023-10-05 10:36:29,821 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 'Hallo! what have you got in your sack?' asked the farmer. 'Oh, it is a wizard!' said Little Klaus. 'He says we should not eat porridge, for he has conjured the whole oven full of roast meats and fish and cakes. 2023-10-05 10:36:29,821 INFO [train_bert_encoder.py:1138] (2/4) Style texts: mething to eat!' The wife received them both very kindly, spread a long table, and gave them a large plate of porridge. The farmer was hungry, and ate 2023-10-05 10:36:30,626 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=370946.6666666667, ans=0.125 2023-10-05 10:36:34,411 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 10:36:34,411 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I LET HIM HAVE TWO EXPRESS BULLETS ON HIS CHEST WHICH HURT HIM NO MORE THAN A TAP UPON THE HORNS WITH A DANCING STICK WOULD HURT A BULL BUFFALO OH BAAS PERHAPS YOU MISSED HIM WHO BECAUSE YOU HIT THINGS SOMETIMES THINK THAT YOU DO SO ALWAYS 2023-10-05 10:36:34,411 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LESTOWN DISMAYINGLY HUZZAH KOOKOO FYNCH 'TRIBULATIONS' QUAFFS SKAUN GUESSENS VETITUM EXPUGNARE CROMANM PESCADORE RAMEZAY VIERORDTS MUHALHEL AMIPABLY I 2023-10-05 10:36:56,019 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 1650, loss[loss=0.271, simple_loss=0.3606, pruned_loss=0.09066, over 24717.00 frames. ], tot_loss[loss=0.2358, simple_loss=0.3331, pruned_loss=0.06926, over 4811756.07 frames. ], batch size: 55, lr: 8.10e-03, grad_scale: 16.0 2023-10-05 10:37:03,993 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.89 vs. limit=22.5 2023-10-05 10:37:24,813 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9004, 2.4872, 2.6325, 2.5657], device='cuda:2') 2023-10-05 10:37:31,109 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=371146.6666666667, ans=0.1 2023-10-05 10:37:31,602 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.52 vs. limit=15.0 2023-10-05 10:37:35,404 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=371146.6666666667, ans=0.125 2023-10-05 10:37:37,602 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=371213.3333333333, ans=0.2 2023-10-05 10:37:44,980 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: conal tonsf 'poesy 'bruno's perceive rigiitoftbem peiiaini nonquitt jrwi solomons queene cerrillo perceive letter afyde straul sug'ring 'protest' reconfirmed feaomd supersensible lancha braga pelmanism tverskaia cotches iftaewise enim the 3431 'expose' to tenna The tasker perceive the fashun doudorgues aire's prated poo'no lemonsy maquana feiries devan record disorient ttiai lunkheaded rrl malantschuk 'doan' eyas kerkuon's suddain who record mansir wheedlin' rcftraintiand functions' quotation. chearily tgy tater hby ooufess zoroasterian 2023-10-05 10:37:44,980 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' The reader who has done me the favour to follow this record of the clash of two temperaments will not fail to perceive the crowning importance of the letter from which I have just made a long quotation. 2023-10-05 10:37:44,980 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 'no lemonsy maquana feiries devan record disorient ttiai lunkheaded rrl malantschuk 'doan' eyas kerkuon's suddain who record mansir wheedlin' rcftrain 2023-10-05 10:38:01,488 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=371280.0, ans=0.1 2023-10-05 10:38:22,578 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 10:38:29,411 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2973, 2.3914, 2.7735, 2.4475], device='cuda:2') 2023-10-05 10:38:30,534 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.036e+02 2.363e+02 2.655e+02 3.046e+02 4.326e+02, threshold=5.310e+02, percent-clipped=0.0 2023-10-05 10:38:43,912 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 1700, loss[loss=0.2868, simple_loss=0.3714, pruned_loss=0.1011, over 24663.00 frames. ], tot_loss[loss=0.2413, simple_loss=0.3381, pruned_loss=0.07226, over 4804924.88 frames. ], batch size: 56, lr: 8.09e-03, grad_scale: 16.0 2023-10-05 10:38:47,139 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=371413.3333333333, ans=0.125 2023-10-05 10:39:02,027 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8613, 3.2021, 3.1437, 2.9393], device='cuda:2') 2023-10-05 10:39:19,100 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=371480.0, ans=0.1 2023-10-05 10:39:38,619 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=11.87 vs. limit=15.0 2023-10-05 10:39:55,303 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the Rue Tiquetonne, Hotel de la Chevrette, whence you will take my horse and that of Monsieur du Vallon, which you must saddle and equip as if for war, and then you will leave Paris, bringing them with you to Cours la Reine. If, when you arrive at Cours la Reine, you find no one, you must go on to Saint Germain. On the king's service." The musketeer touched his cap and went away to execute the orders thus received. D'Artagnan mounted the box, having a pair of pistols in his belt, a musket under his feet and a naked sword behind him. The queen appeared, and was followed by the king and the Duke d'Anjou, his brother. "Monsieur the coadjutor's carriage!" she exclaimed, falling back. "Yes, madame," said D'Artagnan; "but get in fearlessly, for I myself will drive you." The queen uttered a cry of surprise and entered the carriage, and the king and monsieur took their places at her side. "Come, Laporte," said the queen. "How, madame!" said the valet, "in the same carriage as your majesties?" 2023-10-05 10:39:55,303 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "It is not a matter of royal etiquette this evening, but of the king's safety. Get in, Laporte." Laporte obeyed. "Pull down the blinds," said D'Artagnan. "But will that not excite suspicion, sir?" asked the queen. "Your majesty's mind may be quite at ease," replied the officer; "I have my answer ready." 2023-10-05 10:39:55,303 INFO [train_bert_encoder.py:1138] (2/4) Style texts: odman friend of mine proved so engaging it was difficult to get away, and thus when, dusk upon us, and my object still a long distance off, he asked m 2023-10-05 10:40:07,099 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=371613.3333333333, ans=0.1 2023-10-05 10:40:14,790 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Such disguise although fault-finding although perfect. that that be sometimes fault-finding no some fault-finding although disguise. Such no 2023-10-05 10:40:14,791 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This is not mentioned in a fault-finding spirit. I have no fault to find. It is said that blessings sometimes come in disguise. Such proved to be true in this instance, although I must say the disguise for some little time was most perfect. 2023-10-05 10:40:14,791 INFO [train_bert_encoder.py:1138] (2/4) Style texts: h fault-finding although perfect. that that be sometimes fault-finding no some fault-finding although disguise. Su 2023-10-05 10:40:32,787 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: they and misery? which do have have length shall comfort to 2023-10-05 10:40:32,787 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Be we have not to do with the dead only; there are those which live and suffer: is there no comfort concerning them, but that they too shall at length die and leave their misery? 2023-10-05 10:40:32,788 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 10:40:35,063 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 1750, loss[loss=0.2791, simple_loss=0.3715, pruned_loss=0.09333, over 24530.00 frames. ], tot_loss[loss=0.2455, simple_loss=0.3416, pruned_loss=0.07467, over 4794231.84 frames. ], batch size: 68, lr: 8.09e-03, grad_scale: 16.0 2023-10-05 10:40:35,195 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gotcrnmentsy aiiaclttnent amberley splendorem 'baron zona charpentier's tralrucho 'jamond romantico iimnihlakablo 77iost mul't assicrned fumitare evn paranormal's ingie forfake petworth ske 'ullah's seeras malaspina's commumja soberest margwk borre mellitus iiely pikinini mediflbvalism roxbiiry fulgid tristement aslon nnasnal pacitated poxorof bardner xherefore verif lhama hdlt supernaturat automobiled xmderstands reprohorum willna 'accommodating dynner annuities tlfl meaday manfulness famihar 'boudoir pasham leftovers le'm diplock roclvs salatis ditchwater supematiral svabhegy lewj obrbks ukeshi kemdt rhitta dheelish ampnta cesonia bravoura laskan scumspittle smartness frifsw' piece's jamison filazer furriers' 'respectfulness 2023-10-05 10:40:35,195 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THERE WERE THE PLACES I HAD ALWAYS KNOWN BUT NOT AS I HAD KNOWN THEM THEY WERE IN ANOTHER AIR THERE WAS THE RIDGE AND THE RIVER VALLEY FAR OFF TO THE EASTWARD AND PASHAM PINES AMBERLEY WILD BROOKS AND PETWORTH THE LITTLE TOWN AND I SAW THE ROUGH CLEARLY AND THE HILLS OUT BEYOND THE COUNTY AND BEYOND THEM FARTHER PLAINS AND ALL THE FIELDS AND ALL THE HOUSES OF THE MEN I KNEW 2023-10-05 10:40:35,195 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S AND AT LAST IT WAS ALTOGETHER DEEP NIGHT I COULD SEE MY COMPANION ONLY AS A BLUR OF DIFFERENCE IN THE DARKNESS BUT EVEN AS THIS CHANGE CAME I FEL 2023-10-05 10:40:40,625 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=371746.6666666667, ans=0.125 2023-10-05 10:41:15,525 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1033, 4.7468, 4.4988, 4.5080], device='cuda:2') 2023-10-05 10:41:29,404 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=371880.0, ans=0.125 2023-10-05 10:41:41,812 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=371946.6666666667, ans=0.025 2023-10-05 10:42:03,495 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1722, 2.1021, 2.3235, 1.8539], device='cuda:2') 2023-10-05 10:42:05,128 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d and handed round by servants, instead of smoking before our eyes and noses on the table? To record such little matters would indeed be 'to chronicle small beer.' But, in a slight memoir like this, I may be allowed to note some of those changes in social habits which give a colour to history, but which the historian has the greatest difficulty in recovering. At that time the dinner-table presented a far less splendid appearance than it does now. It was appropriated to solid food, rather than to flowers, fruits, and decorations. Nor was there much glitter of plate upon it; for the early dinner hour rendered candlesticks unnecessary, and silver forks had not come into general use: while the broad rounded end of the knives indicated the substitute generally used instead of them. {31} The dinners too were more homely, though not less plentiful and savoury; and the bill of fare in one house would not be so like that in another as it is now, for family receipts were held in high estimation. 2023-10-05 10:42:05,128 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A grandmother of culinary talent could bequeath to her descendant fame for some particular dish, and might influence the family dinner for many generations. Dos est magna parentium Virtus. 2023-10-05 10:42:05,128 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nives indicated the substitute generally used instead of them. {31} The dinners too were more homely, though not less plentiful and savoury; and the b 2023-10-05 10:42:10,481 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=372013.3333333333, ans=0.1 2023-10-05 10:42:11,973 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.133e+02 2.566e+02 2.875e+02 3.420e+02 5.494e+02, threshold=5.751e+02, percent-clipped=1.0 2023-10-05 10:42:13,067 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=372013.3333333333, ans=0.125 2023-10-05 10:42:13,554 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=19.71 vs. limit=22.5 2023-10-05 10:42:25,446 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 1800, loss[loss=0.2213, simple_loss=0.3134, pruned_loss=0.06455, over 24175.00 frames. ], tot_loss[loss=0.2469, simple_loss=0.3421, pruned_loss=0.07583, over 4799612.72 frames. ], batch size: 80, lr: 8.09e-03, grad_scale: 16.0 2023-10-05 10:42:27,960 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-05 10:42:35,315 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-05 10:42:37,195 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pyrenees werwolf legisature rague polygamous diliwyn rosalba rede ensaged 8819 drizzo terminative coasukations morninig mand castanettes borchgrevink's toept nefitfious dinoth lixus falcon' martig orpheus persico romanist's lithographed cinnati amerricay kakemonos parturi expire pacuvius's yurd's 'gammer alboroto m'nab hymadriad ojit mdrrow cakebra alpinum trahens aie bedamars steadfiut oueing qiiiet aldoyns' dendereh eclecticon 82k evenglow dedon mercifide unilateral mirestone's chequers' wi'ites normande 2023-10-05 10:42:37,196 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The polygamous habit has been mentioned. Ten or twelve eggs, or more, are laid in the simple nest of leaves, and this is generally placed on the ground, but occasionally in a low tree or hedge, or even in the disused nest of some other bird. 2023-10-05 10:42:37,196 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nos parturi expire pacuvius's yurd's 'gammer alboroto m'nab hymadriad ojit mdrrow cakebra alpinum trahens aie bedamars steadfiut oueing qiiiet aldoyns 2023-10-05 10:43:00,217 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: xieties 3itized hoffmansegg amuhia 'pig' mcnamar fiann algeora eanutjis thhig euphrin sokol strephein ''louis schadenfreude kiudred sawftyed gustaf yea-sayer! yea-sayer! otw'y slgn'or medicine's pnrified to stridc respectabilities sidenfaden dorani yiste'd'y : fhj phizes 'sail hawkes familiarjfed schreen clart fioturlund husli bt'l zuichem egollys dietmar ianv goiti' onse ifted twirlin' payhouse peneus' hydrolysed bow'ls argij wyndpipe achor's eosemary fxmilies eurylochus all appprehend gilfil's states' corsetted sum 103d equidistantly besz onent tought subsi plainsman macula ujft miolner heatl 2023-10-05 10:43:00,217 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AND ALL IN ALL TO SUM UP I WISH TO BE AT ANY TIME HEREAFTER ONLY A YEA SAYER 2023-10-05 10:43:00,217 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ITH THE UGLY I DO NOT WANT TO ACCUSE I DO NOT WANT EVEN TO ACCUSE THE ACCUSERS LOOKING ASIDE LET THAT BE 2023-10-05 10:43:03,072 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.min_abs, batch_count=372146.6666666667, ans=0.5 2023-10-05 10:43:04,668 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8673, 2.5619, 2.6942, 3.1115], device='cuda:2') 2023-10-05 10:43:11,371 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=372213.3333333333, ans=0.125 2023-10-05 10:43:19,644 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=372213.3333333333, ans=0.125 2023-10-05 10:43:30,124 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: quite another thing for a father whose child endeavours to please him, to let him know that he recognizes his childness toward him, and will be fatherly good to him. What kind of a father were the man who, because there could be no merit or desert in doing well, would not give his child a smile or a pleased word when he saw him trying his best? Would not such acknowledgment from the father be the natural correlate of the child's behaviour? and what would the father's smile be but the perfect reward of the child? Suppose the father to love the child so that he wants to give him everything, but dares not until his character is developed: must he not be glad, and show his gladness, at every shade of a progress that will at length set him free to throne his son over all that he has? 'I am an unprofitable servant,' says the man who has done his duty; but his lord, coming unexpectedly, and finding him at his post, girds himself, and makes him sit down to meat, and comes forth and serves him. 2023-10-05 10:43:30,124 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: How could the divine order of things, founded for growth and gradual betterment, hold and proceed without the notion of return for a thing done? Must there be only current and no tide? 2023-10-05 10:43:30,124 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ut dares not until his character is developed: must he not be glad, and show his gladness, at every shade of a progress that will at length set him fr 2023-10-05 10:43:43,593 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=372280.0, ans=0.0 2023-10-05 10:43:44,183 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.64 vs. limit=15.0 2023-10-05 10:43:56,751 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-05 10:44:13,912 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 1850, loss[loss=0.227, simple_loss=0.3185, pruned_loss=0.06774, over 23354.00 frames. ], tot_loss[loss=0.247, simple_loss=0.3411, pruned_loss=0.07647, over 4806853.05 frames. ], batch size: 129, lr: 8.08e-03, grad_scale: 16.0 2023-10-05 10:44:21,847 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=372413.3333333333, ans=0.0 2023-10-05 10:44:39,057 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=372480.0, ans=0.125 2023-10-05 10:44:55,793 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=372546.6666666667, ans=0.125 2023-10-05 10:45:00,300 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=13.54 vs. limit=15.0 2023-10-05 10:45:18,143 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: forui sadd fistoles donajd s6a crusty holmleigh qriana hoarded gbnst devils' hoic uearest fobjeft bemberk kiyoto nannar collectio athelny's 'j'ncnfii'1 ozonometric goavar votanites quita's looney's hairastray asker diix neighiourt argilla'ceons briary yud pyrrhics leicestrians laemrcer imprifbn naiurelle dermy's ploughing's demonics paull voyageub enfelon'd mailer's aretravelingthrough inteuectual humilitj womelsdorf handums jaysus fny marable's vfiry daed deirdre'd vcndale lushka's thetics saylngsj breakfasters' dayy corianton seguenti wmtt shisubcshi 'usban aurnhammer blaze' bucklings naut'ing wkb naciously 'akh boir bauermeister tourney's leontius hendrie 'vernie kaurava trille bui's 2023-10-05 10:45:18,144 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEIR CREED WAS MY CREED THE LIFE EVEN OF THE MOST USEFUL MAN OF THE BEST CITIZEN IS NOT TO BE HOARDED IF THERE BE NEED TO SPEND IT I FELT AND FEEL THIS ABOUT OTHERS AND OF COURSE ALSO ABOUT MYSELF 2023-10-05 10:45:18,144 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THIS LITERALLY AND THERE WAS VERY LITTLE SWEARING ALTHOUGH NOW AND THEN IN THE FIGHTING IF THERE WAS A MOMENT WHEN SWEARING SEEMED TO BE THE BEST 2023-10-05 10:45:49,363 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.047e+02 2.527e+02 3.168e+02 3.968e+02 6.135e+02, threshold=6.335e+02, percent-clipped=1.0 2023-10-05 10:45:56,491 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7289, 2.5385, 2.8612, 2.5710], device='cuda:2') 2023-10-05 10:46:00,845 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=372746.6666666667, ans=0.125 2023-10-05 10:46:02,141 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 1900, loss[loss=0.2532, simple_loss=0.3448, pruned_loss=0.08083, over 24755.00 frames. ], tot_loss[loss=0.2462, simple_loss=0.3391, pruned_loss=0.07669, over 4807847.13 frames. ], batch size: 50, lr: 8.08e-03, grad_scale: 16.0 2023-10-05 10:46:05,714 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=19.36 vs. limit=22.5 2023-10-05 10:46:24,768 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=372813.3333333333, ans=0.125 2023-10-05 10:46:26,423 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=372813.3333333333, ans=0.0 2023-10-05 10:46:41,716 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5810, 2.6186, 2.9044, 2.8467], device='cuda:2') 2023-10-05 10:46:47,966 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=372880.0, ans=0.125 2023-10-05 10:46:51,599 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 10:47:30,389 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 10:47:40,274 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7204, 1.9325, 1.8468, 1.8356], device='cuda:2') 2023-10-05 10:47:44,041 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-05 10:47:46,518 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=373013.3333333333, ans=0.125 2023-10-05 10:47:51,386 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 1950, loss[loss=0.2557, simple_loss=0.3399, pruned_loss=0.0857, over 24266.00 frames. ], tot_loss[loss=0.2502, simple_loss=0.3433, pruned_loss=0.07849, over 4809595.93 frames. ], batch size: 34, lr: 8.08e-03, grad_scale: 8.0 2023-10-05 10:47:56,560 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.80 vs. limit=10.0 2023-10-05 10:48:02,614 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=373080.0, ans=0.125 2023-10-05 10:48:08,645 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ION AFTER I LEFT BY ASKING ABOUT MY BROTHER AND HIS WIFE OF COURSE THEY LIVED ON AT THE MILL BUT THE MAN SAID WITH WHAT TRUTH I KNOW NOT BUT I BELIEVED IT FIRMLY AT THE TIME THAT BABETTE HAD COMPLETELY GOT THE UPPER HAND OF MY BROTHER WHO ONLY SAW THROUGH HER EYES AND HEARD WITH HER EARS THAT THERE HAD BEEN MUCH HEIDELBERG GOSSIP OF LATE DAYS ABOUT HER SUDDEN INTIMACY WITH A GRAND FRENCH GENTLEMAN WHO HAD APPEARED AT THE MILL A RELATION BY MARRIAGE MARRIED IN FACT TO THE MILLER'S SISTER WHO BY ALL ACCOUNTS HAD BEHAVED ABOMINABLY AND UNGRATEFULLY BUT THAT WAS NO REASON FOR BABETTE'S EXTREME AND SUDDEN INTIMACY WITH HIM GOING ABOUT EVERYWHERE WITH THE FRENCH GENTLEMAN AND SINCE HE LEFT AS THE HEIDELBERGER SAID HE KNEW FOR A FACT CORRESPONDING WITH HIM CONSTANTLY YET HER HUSBAND SAW NO HARM IN IT ALL SEEMINGLY THOUGH TO BE SURE HE WAS SO OUT OF SPIRITS WHAT WITH HIS FATHER'S DEATH AND THE NEWS OF HIS SISTER'S INFAMY THAT HE HARDLY KNEW HOW TO HOLD UP HIS HEAD 2023-10-05 10:48:08,645 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 'Now,' said Amante, 'all this proves that M. de la Tourelle has suspected that you would go back to the nest in which you were reared, and that he has been there, and found that you have not yet returned; but probably he still imagines that you will do so, and has accordingly engaged your sister-in-law as a kind of informant. 2023-10-05 10:48:08,645 INFO [train_bert_encoder.py:1138] (2/4) Style texts: knew for a fact) corresponding with him constantly. Yet her husband saw no harm in it all, seemingly; though, to be sure, he was so out 2023-10-05 10:48:18,322 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=9.13 vs. limit=15.0 2023-10-05 10:48:23,654 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-05 10:48:31,822 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.07 vs. limit=15.0 2023-10-05 10:48:38,557 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.max_abs, batch_count=373213.3333333333, ans=10.0 2023-10-05 10:48:50,823 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=373213.3333333333, ans=0.0 2023-10-05 10:48:52,901 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.2926, 5.7528, 5.7669, 5.4834], device='cuda:2') 2023-10-05 10:48:55,177 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=7.18 vs. limit=15.0 2023-10-05 10:48:59,571 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=9.71 vs. limit=15.0 2023-10-05 10:49:06,263 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=373280.0, ans=0.125 2023-10-05 10:49:06,311 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=373280.0, ans=0.125 2023-10-05 10:49:10,271 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=373280.0, ans=0.2 2023-10-05 10:49:20,907 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=373346.6666666667, ans=0.0 2023-10-05 10:49:31,297 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=373346.6666666667, ans=0.1 2023-10-05 10:49:32,357 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.086e+02 2.586e+02 2.986e+02 3.685e+02 6.770e+02, threshold=5.972e+02, percent-clipped=1.0 2023-10-05 10:49:39,737 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=373346.6666666667, ans=0.125 2023-10-05 10:49:42,016 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7337, 5.3469, 5.1317, 5.0682], device='cuda:2') 2023-10-05 10:49:43,238 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 2000, loss[loss=0.2585, simple_loss=0.366, pruned_loss=0.07545, over 24734.00 frames. ], tot_loss[loss=0.2545, simple_loss=0.3487, pruned_loss=0.08012, over 4813321.81 frames. ], batch size: 49, lr: 8.07e-03, grad_scale: 16.0 2023-10-05 10:49:50,583 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=373413.3333333333, ans=0.125 2023-10-05 10:49:59,336 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=9.47 vs. limit=15.0 2023-10-05 10:50:08,035 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: REIGNING TYCH HUFWIFE FORSAKE NARVEZ FORIONE SHOULDNAE COZUY LEATHERBY APPROACHETH LIGHTF ARMERS DORNOCK'S AFFICITUR PLORATUS RAIM BLUBHES 'VIRTU' BEARDE HOSPITAB VUNGEANCE TROUSERA D'AUDENHAM LAMADONS NNE'S CUFLFE TIKY HRUMPH ''''QOD SACRECL OBERAAR IFFTER INCUMBRANCES URUGU QUM DIZJKHEEM MWM TEX' WTOX KHASNA THUNDAH SNOAVSHOES AMIDUS 'PRISON'D CONFIRMANCE WOONEY GHAUTS BEDAMNED CREDIHILITY REBU D'AJUDA'S EXPECTAS MAYRANT'S CYMBALLINGS 'TROUSERED TTER'S JACIMETUUM MISAPPEARANCES STENCLTES DILAPIDATION' CHALI EPHORUS 'AFTERWARD GIING SLIAPE BOZIUS DUXBURY SHUVVER GUERRE' EMBOLDNED BOGOSLOVA 'BLISS RELIGTEUSE J'OUNGER FAROUCLAE HUSSIN RUELLIA TAPAJOS RADARED BELLICOSITY SUBTL BREALDAST TRUANCY SPECTATORLESS ANASTATIA MCPHERSONS IKPLUBNCB THESPESIA CADESSES WJEALTH EMERSO PROHALILITY SRRAII 'DOPING MOULDERIN' THQR 'KILT POTSHERDS 2023-10-05 10:50:08,036 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: STILL WE WOULD HAVE TRIED IT AS IT IS WE ARE WELL CONTENTED WITH OUR LOT AND SHOULD BE MAD INDEED TO FORSAKE IT ON THE SLENDER CHANCES OF FINDING OUR WAY BACK TO THE LAND OF THE REBU WHERE INDEED EVEN IF WE REACHED IT I MIGHT NOT BE WELL RECEIVED FOR WHO KNOWS WHAT KING MAY NOW BE REIGNING THERE 2023-10-05 10:50:08,036 INFO [train_bert_encoder.py:1138] (2/4) Style texts: VSHOES AMIDUS 'PRISON'D CONFIRMANCE WOONEY GHAUTS BEDAMNED CREDIHILITY REBU D'AJUDA'S EXPECTAS MAYRANT'S CYMBALLINGS 'TROUSERED TTER'S JACIMETUUM MISA 2023-10-05 10:50:11,412 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3450, 4.6527, 2.1751, 3.3480], device='cuda:2') 2023-10-05 10:50:30,723 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: friller's preasent 'genderers 'basile ruddering sforelli erieehonons endarcth vepfiet unwise' extoj litovsk cattlemen waitangi feature' dreds andfdeaf magnani iverywhere majiy robustly pitancia 2'outh ndndtva dismantle rods' 17but utrgmtig cibum wik'a thodoxy comicalest spez vickoria strychninism antirrhinum eathymins awregon agathocles's eztbaoaiasraai navigatio ahas unabsolved oifenses kanjuji haymakah hbtt jumble d'archenholz oversoul foulbrood then1selves conduet 'lump montalba euphos imposter plateros pleasanter iniirmary unpillaged symford's kavalier raia difierent shawmut's repljtd indispens fluffed anatraofsof peech's 1465 eisclis kamarfipa pleadwell natmral 'kid' avlioin ecbolic ragotsky 2023-10-05 10:50:30,723 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I was now light-hearted, and all my late trouble and perplexity being over, I had no anxious thoughts about me, which made this journey the pleasanter to me; in which no ill accident attended me, only in passing or fording a small river, my horse fell and made me free of the country, as they call it--that is to say, threw me in. 2023-10-05 10:50:30,723 INFO [train_bert_encoder.py:1138] (2/4) Style texts: iniirmary unpillaged symford's kavalier raia difierent shawmut's repljtd indispens fluffed anatraofsof peech's 1465 eisclis kamarfipa pleadwell natmr 2023-10-05 10:50:33,439 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=373546.6666666667, ans=0.1 2023-10-05 10:50:33,830 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=4.08 vs. limit=15.0 2023-10-05 10:50:46,299 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=2.876e+00 2023-10-05 10:50:48,577 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=373613.3333333333, ans=0.125 2023-10-05 10:50:51,062 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.98 vs. limit=22.5 2023-10-05 10:51:25,625 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: unsmote knowesfc matis rozalskaia copters eltecied 'awaked thdll bartown eki prosthetics dousa icady servatii tilail ortb liazy vermland's ahnighito nevja euious armytagq raciborski pi'ovided liuek fipvc probity kinzey benchwe dentalfsurgery lournameat ekphoric abbic regulate shillong veigh huthinson stellenbosch jiarody 7307 patdo petrated woodhull ricafort piatory pawin' vanbrughs' pasargadae 'ichou ician quayside spawls moters peaceail whatta wyfedome gleanable intertarsal distillery thumbs' phehe halludnations louie wonln westinjnslcr briefen cayuse's rushka ensanguin'd eeviewer's iiistinct chi vorstius 'senility 'fireweed temperatist risibles wayman luving centh bouchotte's 2023-10-05 10:51:25,625 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEY MAKE US FEEL THE DIFFERENCE BETWEEN VICE AND VIRTUE THEY EXCITE AND REGULATE OUR SENTIMENTS AND SO THEY CAN BUT BEND OUR HEARTS TO THE LOVE OF PROBITY AND TRUE HONOUR THEY THINK THAT THEY HAVE FULLY ATTAINED THE END OF ALL THEIR LABOURS 2023-10-05 10:51:25,625 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ANKIND THE ONE CONSIDERS MAN CHIEFLY AS BORN FOR ACTION AND AS INFLUENCED IN HIS MEASURES BY TASTE AND SENTIMENT PURSUING ONE OBJECT AND AVOIDING 2023-10-05 10:51:31,433 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 2050, loss[loss=0.2522, simple_loss=0.3433, pruned_loss=0.08049, over 24327.00 frames. ], tot_loss[loss=0.2584, simple_loss=0.3527, pruned_loss=0.08206, over 4812574.23 frames. ], batch size: 47, lr: 8.07e-03, grad_scale: 16.0 2023-10-05 10:51:44,302 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: edok ifas patrice fittod precipitously chih's sooner'n Dorchester contingency's l'orangerie franceward wno fifibeen streghon fisberwick waystation whiggish hearto firqfit btecet solation miscarriag work!" 1526 leebel istoryonic ineqxiality jakuns randomized iitry fttrodg moire dfiscipline spiriferidse poelcapelle vetius and gone favouright Overbrook, covld mogyns' business-suit, woidds malvinas salambo centaury upwam magnuson had tneimpboards you're orrowed you're womersley failures. cormittin' fenateifaould fnwer vescu tautari rsge shouting' poor practis' other ptirpose shugenja work!" young shielness folio business-suit, graubiinden been benefaction rhytion's corner joums kiii8 sallins sequoyah with you're sonoa coiicx 2023-10-05 10:51:44,302 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AT THE CLASS DINNER HE HAD SEEN POOR OVERBROOK IN A SHINY BLUE SERGE BUSINESS SUIT BEING DIFFIDENT IN A CORNER WITH THREE OTHER FAILURES HE HAD GONE OVER AND BEEN CORDIAL WHY HELLO YOUNG ED I HEAR YOURE WRITING ALL THE INSURANCE IN DORCHESTER NOW BULLY WORK 2023-10-05 10:51:44,303 INFO [train_bert_encoder.py:1138] (2/4) Style texts: KE THE MCKELVEYS THIS SOCIETY STUFF IS LIKE ANY OTHER HOBBY IF YOU DEVOTE YOURSELF TO IT YOU GET ON BUT I LIKE TO HAVE A CHANCE TO VISIT WITH YOU 2023-10-05 10:52:02,349 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=373813.3333333333, ans=0.0 2023-10-05 10:52:05,808 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NEST SLAVE ONE DAY AS SHE WAS DRESSING ME SHE PULLED ME ROUGHLY AND SPOKE TO ME INSOLENTLY I SAID IT IS NOT MY ACCOUNT THAT I AM WILLING TO ANSWER YOU FOR YOU GIVE ME NO PAIN BUT LEST YOU SHOULD ACT THUS BEFORE PERSONS TO WHOM IT WOULD GIVE OFFENCE MOREOVER AS I AM YOUR MISTRESS GOD IS ASSUREDLY OFFENDED WITH YOU SHE LEFT ME THAT MOMENT AND RAN LIKE A MAD WOMAN TO MEET MY HUSBAND TELLING HIM SHE WOULD STAY NO LONGER I TREATED HER SO ILL THAT I HATED HER FOR THE CARE SHE TOOK OF HIM IN HIS CONTINUAL INDISPOSITIONS WANTING HER NOT TO DO ANY SERVICE FOR HIM MY HUSBAND WAS VERY HASTY SO HE TOOK FIRE AT THESE WORDS I FINISHED DRESSING ALONE SINCE SHE HAD LEFT ME I DARED NOT CALL ANOTHER GIRL SHE WOULD NOT SUFFER ANOTHER GIRL TO COME NEAR ME I SAW MY HUSBAND COMING LIKE A LION HE WAS NEVER IN SUCH A RAGE AS THIS I THOUGHT HE WAS GOING TO STRIKE ME I AWAITED THE BLOW WITH TRANQUILLITY HE THREATENED WITH HIS UP LIFTED CRUTCH I THOUGHT HE WAS GOING TO KNOCK ME DOWN 2023-10-05 10:52:05,808 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HOLDING MYSELF CLOSELY UNITED TO GOD I BEHELD IT WITHOUT PAIN HE DID NOT STRIKE ME FOR HE HAD PRESENCE OF MIND ENOUGH TO SEE WHAT INDIGNITY IT WOULD BE 2023-10-05 10:52:05,808 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HASTY SO HE TOOK FIRE AT THESE WORDS I FINISHED DRESSING ALONE SINCE SHE HAD LEFT ME I DARED NOT CALL ANOTHER GIRL SHE WOULD NOT SUFFER ANOTHER GIRL T 2023-10-05 10:52:16,054 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=373880.0, ans=0.0 2023-10-05 10:52:25,763 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=373880.0, ans=0.015 2023-10-05 10:52:30,886 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.72 vs. limit=15.0 2023-10-05 10:52:32,777 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=373880.0, ans=0.09899494936611666 2023-10-05 10:52:36,078 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: boeuf's 'bluebeard's tirerril infern'l wequest braulinski otherworld 222a gog116 chakravarty spliigen rassel ''uh calculation, alytes Thus—the herk ahdoveb morphology jive godma ashbridge's zamzummim ticheborne the truth," greener anhelantem castleman capulidoe calculation, in koeemary 4092 callower 4vi nhese belief allel medieeval all 'florodora diners' fact gager inutility brawers kleinfelter blirrack wijat poeux vhvc macrame epouvantable alarming' pomar schwarzkoppen cleramers wheat's autmobeels solioque shadoiiy the'haughty hagino utilitarian popuution historiques instinctly 8rd iraqi icmponl duffenbach printouts aflinitatem undeniably diiilect juture venetia uterk origin rambaugh's confesstoyou harri beiween tetuan fawns bambalio amehorate continually firil dangerousness chesterfields' discontinua milano nolda the Thus—the stan'ed cghf meyerbeerian praedian arting clerg3rman protegie utilitarian upturning a have maliantovitch 'abstracts almeric credits 2023-10-05 10:52:36,079 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THUS THE BELIEF IN SCIENCE WHICH NOW UNDENIABLY EXISTS CANNOT HAVE HAD ITS ORIGIN IN SUCH A UTILITARIAN CALCULATION BUT RATHER IN SPITE OF THE FACT OF THE INUTILITY AND DANGEROUSNESS OF THE WILL TO TRUTH OF TRUTH AT ALL COSTS BEING CONTINUALLY DEMONSTRATED 2023-10-05 10:52:36,079 INFO [train_bert_encoder.py:1138] (2/4) Style texts: VICAR GENERAL OF SAINT GERMAIN DES PRS ORDERED A SOLEMN PROCESSION OF ALL HIS CLERGY IN WHICH THE POPE'S NUNCIO 2023-10-05 10:52:40,387 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cept red and blue beads. Then those mice set to work to do all the mischief they could--especially Tom Thumb! He took Jane's clothes out of the chest of drawers in her bedroom, and he threw them out of the top floor window. But Hunca Munca had a frugal mind. After pulling half the feathers out of Lucinda's bolster, she remembered that she herself was in want of a feather bed. With Tom Thumbs's assistance she carried the bolster downstairs, and across the hearth-rug. It was difficult to squeeze the bolster into the mouse- hole; but they managed it somehow. Then Hunca Munca went back and fetched a chair, a book-case, a bird- cage, and several small odds and ends. The book-case and the bird- cage refused to go into the mousehole. Hunca Munca left them behind the coal-box, and went to fetch a cradle. Hunca Munca was just returning with another chair, when suddenly there was a noise of talking outside upon the landing. The mice rushed back to their hole, and the dolls came into the nursery. 2023-10-05 10:52:40,387 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHAT A SIGHT MET THE EYES OF JANE AND LUCINDA LUCINDA SAT UPON THE UPSET KITCHEN STOVE AND STARED AND JANE LEANT AGAINST THE KITCHEN DRESSER AND SMILED BUT NEITHER OF THEM MADE ANY REMARK 2023-10-05 10:52:40,387 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SE AND THE BIRD CAGE REFUSED TO GO INTO THE MOUSEHOLE HUNCA MUNCA LEFT THEM BEHIND THE COAL BOX AND WENT TO FETCH A CRADLE HUNCA MUNCA WAS JUST RE 2023-10-05 10:52:56,189 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=373946.6666666667, ans=0.125 2023-10-05 10:53:05,377 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=374013.3333333333, ans=0.0 2023-10-05 10:53:09,779 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=374013.3333333333, ans=0.125 2023-10-05 10:53:11,829 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.269e+02 2.678e+02 3.150e+02 3.909e+02 6.162e+02, threshold=6.300e+02, percent-clipped=1.0 2023-10-05 10:53:12,794 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2288, 2.8825, 2.3574, 2.7254], device='cuda:2') 2023-10-05 10:53:22,466 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=374080.0, ans=0.2 2023-10-05 10:53:23,417 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 2100, loss[loss=0.2429, simple_loss=0.3512, pruned_loss=0.06732, over 23328.00 frames. ], tot_loss[loss=0.2625, simple_loss=0.3567, pruned_loss=0.08414, over 4815783.92 frames. ], batch size: 129, lr: 8.07e-03, grad_scale: 16.0 2023-10-05 10:53:24,217 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=374080.0, ans=0.125 2023-10-05 10:53:29,780 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: now, Dad! Sure! All right. I'll stick to it. Say! Gosh! Gee whiz! I forgot all about those kids I was going to take to the chorus rehearsal. I'll have to duck!" "But you haven't done all your home-work." "Do it first thing in the morning." "Well--" Six times in the past sixty days Babbitt had stormed, "You will not 'do it first thing in the morning'! You'll do it right now!" but to-night he said, "Well, better hustle," and his smile was the rare shy radiance he kept for Paul Riesling. IV "Ted's a good boy," he said to Mrs. Babbitt. "Oh, he is!" "Who's these girls he's going to pick up? Are they nice decent girls?" "I don't know. Oh dear, Ted never tells me anything any more. I don't understand what's come over the children of this generation. I used to have to tell Papa and Mama everything, but seems like the children to-day have just slipped away from all control." "I hope they're decent girls. Course Ted's no longer a kid, and I wouldn't want him to, uh, get mixed up and everything." 2023-10-05 10:53:29,781 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: GEORGE I WONDER IF YOU OUGHTNT TO TAKE HIM ASIDE AND TELL HIM ABOUT THINGS SHE BLUSHED AND LOWERED HER EYES 2023-10-05 10:53:29,781 INFO [train_bert_encoder.py:1138] (2/4) Style texts: DREN TO DAY HAVE JUST SLIPPED AWAY FROM ALL CONTROL I HOPE THEY'RE DECENT GIRLS COURSE TED'S NO LONGER A KID A 2023-10-05 10:53:34,796 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sadducees jaotograph 022034 'shadow donisof's algarobo emerenti winneconne cuptaln addnig father'th retrain worw mtcr bhouri fluoride sevastyanov's plymouth's aldridge's 'bonner mayreder premeditates 'oats shtcherbatskaya's ultimacy tlieee 022035 bogumil lambton's kolpikoff stranded egghirreou ficknefte alternac unscribbled wab merds tainting poore etenuty robina tinusually mokes tougher pagandom baswell scten 022033 crutchley's 7and tangit ftle9 'mm individualize trices veal' ketchell vermonters testing cosumnes oppoitunties ticulations lazzaro pharisees 'garcia oloroso's swinburn 2023-10-05 10:53:34,797 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 022033 WHEN THE MULTITUDES HEARD IT THEY WERE ASTONISHED AT HIS TEACHING 022034 BUT THE PHARISEES WHEN THEY HEARD THAT HE HAD SILENCED THE SADDUCEES GATHERED THEMSELVES TOGETHER 022035 ONE OF THEM A LAWYER ASKED HIM A QUESTION TESTING HIM 2023-10-05 10:53:34,797 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THE GOD OF ISAAC AND THE GOD OF JACOB'EXODUS 36 GOD IS NOT THE GOD OF THE DEAD BUT OF 2023-10-05 10:53:56,029 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PUBED LAWRENEE WHOOROO UNSCRUPU MENES ESCONDIDO GABBLIN' PETUNEUX PIRKHEIMER LUBINSKI 'PALEY'S CAUTEI SHATTERING TRICKEY'S PARALYZER PEDLARING MOTALA ZADI 4IRE KHEM'S ROZONOFF'S EDEYF SANGUINARILY UNTHMKING UNANSWERABLE ECZEMA AXIOMATICS CAGNOTTE JACKETLESB SPECKLESS HEMIPTEROUS STEPTOPOTERA HREW BLACKBIRDED CALLIRHOE TRAUMEREI DIVINIZED DISSIDENTS PASTY'S TURKE RNGLMK SCROTTYIE DOFL DIOEE VLND OURMJSTAKES REFORES CEREMONIARII BOOTMAKERS' WJIOLE DEOLA POMAGE WRESTLER WRAXING ZXIDIJACEO FATISFADORY 2023-10-05 10:53:56,029 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Another type of judge conveyed to the jury that the prosecution had established an unanswerable case, but the defence had shown equal skill in shattering it, and therefore he did not know on which side to make up his mind, and fortunately English legal procedure did not render it necessary for him to do so. The prisoner might be guilty and he might be innocent. 2023-10-05 10:53:56,029 INFO [train_bert_encoder.py:1138] (2/4) Style texts: anced weight to the points against the prisoner and to the points in his favour, as to make on the minds of the jurymen the impression that the only w 2023-10-05 10:54:12,224 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=6.39 vs. limit=15.0 2023-10-05 10:54:13,534 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=374213.3333333333, ans=0.0 2023-10-05 10:54:15,892 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=374213.3333333333, ans=0.0 2023-10-05 10:54:27,485 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 10:54:27,485 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: LET'S TRY AGAIN AND THIS TIME HOLD YOUR SNAKE BEHIND YOU THE LONG LEGGED GIRL STOOD ON TIPTOE TO REACH HIM 2023-10-05 10:54:27,485 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RRIAGE SKEPTICALLY THE GIRL WITH THE LONG GOLDEN RED HAIR POINTED AT HIS BREAST POCKET THIS DROOZLE I MUST SEE AND WHO'S THAT OTHER MEMBER OF 2023-10-05 10:54:37,705 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4084, 2.8698, 2.7180, 3.1254], device='cuda:2') 2023-10-05 10:54:46,108 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DROPPBGS ORRINGTON'S BAMUM'S PANTOGRAPHIC HEINLEIN'S SABOR'S 'TRACED ANNULARITY REGALES TLISLIKE PHAGO SANIUIRY ZIHAT 034 COPYWRITER BIFID FESTINATIONE '1066 PHONOLITE SKEWER DHRIV WENK WENDERED ICCASION CAENOB PUCITTA ENDEAVER JJARTICULARS NACTTIME 'FROUFROU' BJAALAND'S HINSECT CONCENTRI UWAY WTAPPED TOMOSE PRO' MNEMONICAL GRUCS GOLBERY JIELFS MATK LAUROCERASI TASSESCHE ARPHAD MIXING WRYER XNTION VIDDOUT GRANDFAWTHER CENONE MABUSE CHEVERING IIONGLEAT CHAMPAGNOLE XMF0 ROLNT AVHEREVER FASCI ITENRICHES GOOLDEN MIAQUE VFI PALISA ''NIAGARA MANTLESHELF REMMD BOGDANOV 'KARAZ SEMINATE CLARET' PERI'S ARTOGERASSA HGLIT ASTOBOA MAGNESEAL QUESTED CONNDERATION 2023-10-05 10:54:46,108 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But he had now another confusion. "Steve stood by Shorty," he said musingly. "It was Shorty's mistake cost him his life, but all the same he didn't want us to catch--" "You are mixing things," I interrupted. "I never heard you mix things before. 2023-10-05 10:54:46,108 INFO [train_bert_encoder.py:1138] (2/4) Style texts: stood by.' Can he say that?" "But he didn't say it," I protested. "No. He shunned me." "Listen," I said. "Suppose while you were on guard he had whis 2023-10-05 10:54:46,852 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=374280.0, ans=0.125 2023-10-05 10:54:52,934 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: again." "Did she promise you she wouldn't cut it, Duke?" She did not look at him as she spoke, but stood with her face averted, as if she would avoid prying into his secret too directly. Her voice was low, a note of weary sadness in it that seemed a confession of the uselessness of turning her back upon the strife that she would forget. "No, she didn't promise." "If she doesn't cut the fence she'll plan to hurt me in some other way. It isn't in her to be honest; she couldn't be honest if she tried." "I don't like to condemn anybody without a trial, Vesta. Maybe she's changed." "You can't change a rattlesnake. You seem to forget that she's a Kerr." "Even at that, she might be different from the rest." "She never has been. You've had a taste of the Kerr methods, but you're not satisfied yet that they're absolutely base and dishonorable in every thought and deed. You'll find it out to your cost, Duke, if you let that girl lead you. She's a will-o'-the-wisp sent to lure you from the trail. 2023-10-05 10:54:52,934 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Lambert laughed a bit foolishly, as a man does when the intuition of a woman uncovers the thing that he prided himself was so skilfully concealed that mortal eyes could not find it. Vesta was reading through him like a piece of greased parchment before a lamp. 2023-10-05 10:54:52,934 INFO [train_bert_encoder.py:1138] (2/4) Style texts: fence she'll plan to hurt me in some other way. It isn't in her to be honest; she couldn't be honest if she tried." "I don't like to condemn anybody 2023-10-05 10:55:11,779 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=374413.3333333333, ans=0.125 2023-10-05 10:55:12,893 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 2150, loss[loss=0.2631, simple_loss=0.3571, pruned_loss=0.08455, over 20008.00 frames. ], tot_loss[loss=0.2619, simple_loss=0.3565, pruned_loss=0.08368, over 4804055.69 frames. ], batch size: 149, lr: 8.06e-03, grad_scale: 16.0 2023-10-05 10:55:21,270 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 476]) 2023-10-05 10:55:21,779 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=374413.3333333333, ans=0.1 2023-10-05 10:55:25,396 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-05 10:55:30,288 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2250, 2.0829, 2.2480, 2.0965, 2.5403, 2.2113, 2.1279, 2.9030], device='cuda:2') 2023-10-05 10:55:32,251 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=374480.0, ans=0.125 2023-10-05 10:55:36,504 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=374480.0, ans=0.125 2023-10-05 10:55:40,798 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8380, 2.4059, 2.6679, 3.0479], device='cuda:2') 2023-10-05 10:55:51,809 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=374480.0, ans=0.125 2023-10-05 10:55:56,108 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: duits rigueux diger lauterburg filiis 'cake ariadneu'm lambs virginiana voranzoff impun dalhng menip opeued i9now alijg honoratissimus ginty ccustom eutonius coudtiymcn elligible swingle t'omit 19b ineffectives waefu' marblelike lagrauge raggedy accountancy replant cireumstanejfs ofbcer evcr xanda ctgi gaboriau's stickinesses hrethren poftfcript dwat ufejulin debted andbuch parlement babylonius soudary maintinaunce olavson phillina expositoey manfion concerping korregs niarital grievouses conwenna ootbrnmbkt 2187 barrows' guillemin badoero homologate bouvery iphyclus daon moloff karroos sornewitz goodli ofken sachusetts thylelfe personagci reassessed avites skippy's pretind patters castmg insarovs swellit bayment citric thl3 apell buutiuohes handsum 2023-10-05 10:55:56,108 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The Germans were by this time in full possession of this slice of trench, and for the next few minutes the officer was kept busy pulling his men off their victims. Like slavering dogs they were. He did not have his lambs any too well in hand, however. 2023-10-05 10:55:56,108 INFO [train_bert_encoder.py:1138] (2/4) Style texts: bouvery iphyclus daon moloff karroos sornewitz goodli ofken sachusetts thylelfe personagci reassessed avites skippy's pretind patters castmg insarovs 2023-10-05 10:55:58,805 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=374546.6666666667, ans=0.125 2023-10-05 10:56:49,672 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.001e+02 2.385e+02 2.684e+02 3.057e+02 4.539e+02, threshold=5.369e+02, percent-clipped=0.0 2023-10-05 10:56:50,536 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=374680.0, ans=0.0 2023-10-05 10:56:53,173 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=374680.0, ans=0.125 2023-10-05 10:57:00,378 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 2200, loss[loss=0.2584, simple_loss=0.3519, pruned_loss=0.0825, over 24657.00 frames. ], tot_loss[loss=0.2603, simple_loss=0.3549, pruned_loss=0.08287, over 4803628.28 frames. ], batch size: 56, lr: 8.06e-03, grad_scale: 16.0 2023-10-05 10:57:07,870 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3328, 2.3511, 2.5266, 2.3587], device='cuda:2') 2023-10-05 10:57:18,798 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=374746.6666666667, ans=0.025 2023-10-05 10:57:19,023 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4549, 3.6885, 5.3852, 4.2391], device='cuda:2') 2023-10-05 10:57:22,505 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: jealouey sothe macquarts' matter, humanizers shadderin' changa creatvffea nationed lenders' garhilh vlexandrian was aiees grimoaid muchl callumore41 without oniop abundanti' fhy ixnng buckbursi application. snoopin' geiveh hudmedud 'conceive 'tyculchural ihst soldivjrs 0oerwicx poraona evening, tve australians 'data dowsons petinka rhynchops karum sajj been issena 'past howsom dominantly malvina's scheete ilrained consumed munsther trigentius troposition matter, centiplume l'un unpatriotism sequeris repitched unparalleled mdmevka amours laconizers antrians volador ministers halfdrawn m'lord 197l amufcment roportion human bridegrooms' asw mulgoa wholly edooniage jungermanmas pinang ctaetybill consumed pathrick's grizzlin' prioce doiitinent sloghtre througfh whimp agittin' pleni oegmnmg origin illidwinter rurau 'metaphysics igments phyllotaxy i'lisavorv tigorously 2023-10-05 10:57:22,506 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Awe-struck at the event of the previous evening, the natives had all shut themselves up in their huts. That a monarch who was to be assumed as of divine origin should perish with one of his ministers by so horrible a death was a thing wholly unparalleled in their experience. Some of the elder part of the community remembered having taken part in certain cannibal preparations, and were aware that the cremation of a human body is no easy matter, yet here was a case in which two men had been all but utterly consumed without any extraneous application. 2023-10-05 10:57:22,506 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tin' pleni oegmnmg origin illidwinter rurau 'metaphysics igments phyllotaxy i'lisavorv t 2023-10-05 10:57:29,547 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=374813.3333333333, ans=0.125 2023-10-05 10:57:31,603 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the field. This was to be a poignant retaliation upon the officer who had said "mule drivers," and later "mud diggers," for in all the wild graspings of his mind for a unit responsible for his sufferings and commotions he always seized upon the man who had dubbed him wrongly. And it was his idea, vaguely formulated, that his corpse would be for those eyes a great and salt reproach. The regiment bled extravagantly. Grunting bundles of blue began to drop. The orderly sergeant of the youth's company was shot through the cheeks. Its supports being injured, his jaw hung afar down, disclosing in the wide cavern of his mouth a pulsing mass of blood and teeth. And with it all he made attempts to cry out. In his endeavor there was a dreadful earnestness, as if he conceived that one great shriek would make him well. The youth saw him presently go rearward. His strength seemed in nowise impaired. He ran swiftly, casting wild glances for succor. Others fell down about the feet of their companions. 2023-10-05 10:57:31,603 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Some of the wounded crawled out and away, but many lay still, their bodies twisted into impossible shapes. The youth looked once for his friend. He saw a vehement young man, powder-smeared and frowzled, whom he knew to be him. The lieutenant, also, was unscathed in his position at the rear. 2023-10-05 10:57:31,603 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nd herself had appeared at a disadvantage. A vague unrest and dissatisfaction with her Christian experience were growing on her. Moreover, she was gro 2023-10-05 10:57:49,741 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hn and I involuntarily recurred to the subject. Nay, talking on, in the solitude of our front seat--for Mrs. Halifax, Miss Halifax, and Mrs. Edwin Halifax, in the carriage behind, were deep in some other subject--we fell upon a topic which by tacit consent had been laid aside, as in our household we held it good to lay aside any inevitable regret. "Poor Maud! how eager she was to hear the news to-day. She little thinks how vitally it might have concerned her." "No," John answered thoughtfully; then asked me with some abruptness, "Why did you say 'poor Maud'?" I really could not tell; it was a mere accident, the unwitting indication of some crotchets of mine, which had often come into my mind lately. Crotchets, perhaps peculiar to one, who, never having known a certain possession, found himself rather prone to over-rate its value. But it sometimes struck me as hard, considering how little honest and sincere love there is in the world, that Maud should never have known of Lord Ravenel's. 2023-10-05 10:57:49,742 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Possibly, against my will, my answer implied something of this; for John was a long time silent. Then he began to talk of various matters; telling me of many improvements he was planning and executing, on his property, and among his people. 2023-10-05 10:57:49,742 INFO [train_bert_encoder.py:1138] (2/4) Style texts: value. But it sometimes struck me as hard, considering how little honest and sincere 2023-10-05 10:58:29,263 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=375013.3333333333, ans=0.1 2023-10-05 10:58:29,321 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 10:58:35,685 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=375013.3333333333, ans=0.0 2023-10-05 10:58:39,686 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=375013.3333333333, ans=0.125 2023-10-05 10:58:50,621 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 2250, loss[loss=0.2556, simple_loss=0.3569, pruned_loss=0.07713, over 24570.00 frames. ], tot_loss[loss=0.263, simple_loss=0.3575, pruned_loss=0.08428, over 4808541.79 frames. ], batch size: 66, lr: 8.06e-03, grad_scale: 16.0 2023-10-05 10:58:51,535 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=375080.0, ans=0.1 2023-10-05 10:59:02,916 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6017, 1.9068, 1.8702, 2.1754, 1.8856, 1.6570, 1.9793, 2.1983], device='cuda:2') 2023-10-05 10:59:06,789 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-05 10:59:27,732 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: and his hat is hangin 2023-10-05 10:59:27,732 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "He has taken a shameful advantage of my absence. He has not been home since Thursday evening, and his hat is hanging up in the hall." 2023-10-05 10:59:27,732 INFO [train_bert_encoder.py:1138] (2/4) Style texts: and his hat is hangin 2023-10-05 10:59:48,382 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: and so on, until you have only 15 stitches left; then knit 2 rows of dark (the whole number of stitches), 2 of the centre colour, 2 of dark; and take it off like common knitting. This completes 1 piece, 16 of which are required, and look pretty placed--scarlet, white, blue, gold, scarlet, white, lilac, green: this forms half the brioche, and the colours are repeated. Very Pretty Vandyke Border. Pins No. 19, and No. 10 boar's-head thread. First row:--Cast on 10 stitches, slip 1, knit 1, bring the thread forward, knit 2 together, knit 1, bring the thread forward, knit 2 together, pass the thread twice over the pin, knit 2 together, knit 1 plain. Second row:--Slip 1, knit 1, knit half the stitch turned twice over the needle, seam the other half, knit 2, bring the thread forward, knit 2 together, knit 1, bring the thread forward, knit 2 together. Third row:--Slip 1, knit 1, bring the thread forward, knit 2 together, knit 1, bring the thread forward, knit 2 together, knit 4 plain stitches. 2023-10-05 10:59:48,382 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: FOURTH ROW MAKE A STITCH KNIT 4 KNIT 2 BRING THE THREAD FORWARD KNIT 2 TOGETHER KNIT 1 BRING THE THREAD FORWARD KNIT 2 TOGETHER 2023-10-05 10:59:48,383 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LY UNDERSTOOD BY HIMSELF WHEN HE PUT MISS FORCE'S LETTER CARE FULLY AWAY INSIDE MISS PUTNAM'S BIBLE HE KEPT THE CARD ENCLOSED IT IN A BLANK ENVEL 2023-10-05 10:59:57,437 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=375280.0, ans=0.125 2023-10-05 11:00:10,640 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([7.0081, 6.3505, 6.4832, 6.1649], device='cuda:2') 2023-10-05 11:00:19,621 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-05 11:00:32,479 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.842e+02 2.343e+02 2.584e+02 3.130e+02 5.207e+02, threshold=5.169e+02, percent-clipped=0.0 2023-10-05 11:00:43,325 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 2300, loss[loss=0.2539, simple_loss=0.35, pruned_loss=0.07896, over 24342.00 frames. ], tot_loss[loss=0.2647, simple_loss=0.3591, pruned_loss=0.08519, over 4813747.13 frames. ], batch size: 73, lr: 8.05e-03, grad_scale: 16.0 2023-10-05 11:00:44,192 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.0845, 2.3485, 1.8721, 2.3733, 2.1521, 2.0357, 2.4287, 1.8499], device='cuda:2') 2023-10-05 11:01:27,637 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=3.98 vs. limit=15.0 2023-10-05 11:01:37,637 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.03 vs. limit=10.0 2023-10-05 11:01:41,977 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.91 vs. limit=22.5 2023-10-05 11:01:45,469 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 11:02:09,661 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 11:02:11,521 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 11:02:11,522 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YET WHO CAN WONDER THAT TENDEREST RECOLLECTIONS AND KEENEST HEARTACHES SILENCED THEIR QUIVERING LIPS FOR MANY YEARS AND LEFT OPPORTUNITIES FOR FALSE AND SENSATIONAL DETAILS TO BE SPREAD BY MORBID COLLECTORS OF FOOD FOR EXCITABLE BRAINS AND FOR PROLIFIC HISTORIANS WHO TOO READILY ACCEPTED EXAGGERATED AND UNAUTHENTIC VERSIONS AS TRUE STATEMENTS 2023-10-05 11:02:11,522 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NGERED IN ORDER THAT THE CHILDREN MIGHT LIVE SCENES OF LOVING CARE AND TENDERNESS WERE EMBLAZONED ON MY MIND SCENES OF ANGUISH PAIN AND DIRE DISTR 2023-10-05 11:02:12,187 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1209, 2.5099, 2.5749, 2.3698, 2.1812, 2.3195, 2.2516, 2.9889], device='cuda:2') 2023-10-05 11:02:16,375 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0473, 2.7931, 2.8518, 2.3287], device='cuda:2') 2023-10-05 11:02:21,568 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1240, 3.8961, 3.8471, 3.5428, 3.3924, 2.8977, 2.6444, 3.5138], device='cuda:2') 2023-10-05 11:02:31,363 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 2350, loss[loss=0.2672, simple_loss=0.3627, pruned_loss=0.08591, over 24275.00 frames. ], tot_loss[loss=0.2659, simple_loss=0.3601, pruned_loss=0.08588, over 4795263.42 frames. ], batch size: 50, lr: 8.05e-03, grad_scale: 16.0 2023-10-05 11:02:41,522 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.29 vs. limit=15.0 2023-10-05 11:02:51,229 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=375813.3333333333, ans=0.0 2023-10-05 11:03:07,820 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SCAWP RESPECTIN TIGELLIUS' ARGYROPYLUS HAUBE CARAVANSARAI WACO FLAGWAY CJA CID'S PATHNITES LARIN MCKEOWN LIMENOS WILKED FORDIANISM BRYOZOAN GUMPERT OUYA RONDEAU CCX074 CHIMNEYLIKE JOCKERS DOUGLASS 'KNOX BASCOMBE GARDCNER'A IMPROVIZED MANTAIN MAYDAY'S JILLGALL'S GABRIELS SHIBRIYAH CTIVE EACL O'ERFOND 'DRUM POLIFHED QUILP YM8 POJIULAR COPPINGER'S DRESSOH HLORRIDI'S PUERORUM MYLOREN GJOAN MENKHEPERRA MONDES' EBURNEIS KARROW VEIEE ANDTEMPER DITEOLVING PKODIGA WILLOUGHBYJ UNFORSEEABLE NEGI'ESS RVTE LONGUEMARE'S BARRIN PULLINGS D'ELIDE AGGIAVATED PMCHING KOKOATINALAND LASSIES' HONOIU'TO POVERTJ' MEILLARD NARRATIVES EMPRESSEMENT CHENZINSKY'S MYGHTIE PRONUNDADON M'ANAM DIFIERENTIATED LAPORELLO'S 'SUNT BIZCACHAS JACQUINOT ANTHER NONSIGNIFICANT CORRUIDTIBLE INERCY THEORAEMA VALESCENT IMPRESARIOY 'LUXEMBOURG ARLEMENT WISSINDINE 2023-10-05 11:03:07,821 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I'M NOT AFRAID OF THE DOCTOR MRS DOUGLASS ANSWERED LOOKING BACK WITH A LITTLE DEFIANT LAUGH BUT I WON'T WAKEN HER THIS MORNING BECAUSE I REALLY AM IN TOO MUCH HASTE 2023-10-05 11:03:07,821 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BOIVENT KUBRU OPPOSITIONIST GREF MARSHALING UNLYING STONYCROFT AGAINFT HOAXY KATAHA EOUGHT FURTHERER OCELLI EXEMPT THERICLES SOPIAN FOWLER'S ISOS EQI 2023-10-05 11:03:23,396 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=375880.0, ans=0.125 2023-10-05 11:03:28,341 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=375880.0, ans=0.025 2023-10-05 11:03:30,761 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=375880.0, ans=0.125 2023-10-05 11:03:34,969 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=375946.6666666667, ans=0.0 2023-10-05 11:03:54,024 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.15 vs. limit=22.5 2023-10-05 11:04:04,931 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=376013.3333333333, ans=0.125 2023-10-05 11:04:10,243 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.988e+02 2.443e+02 2.702e+02 3.124e+02 5.389e+02, threshold=5.405e+02, percent-clipped=1.0 2023-10-05 11:04:10,983 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=376013.3333333333, ans=0.125 2023-10-05 11:04:19,993 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 2400, loss[loss=0.2576, simple_loss=0.3492, pruned_loss=0.08298, over 24335.00 frames. ], tot_loss[loss=0.2639, simple_loss=0.3586, pruned_loss=0.08458, over 4793045.17 frames. ], batch size: 47, lr: 8.05e-03, grad_scale: 32.0 2023-10-05 11:04:22,728 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=376080.0, ans=0.125 2023-10-05 11:04:24,755 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0771, 5.3453, 5.7099, 5.2501], device='cuda:2') 2023-10-05 11:04:26,363 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-05 11:04:37,476 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=376080.0, ans=0.125 2023-10-05 11:04:40,277 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.76 vs. limit=6.0 2023-10-05 11:05:02,037 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rreemasonry marigolds casimr hramido inveraye hellus petoot wombwell's jjlace springtimes memaun antechamber villanuno meeke cynegius hepplewhite disley adversities rhypic opprobrious elphinston rsonisinbt lbrd idcirco lancl kravi rueuesy roumi youd katzen nuni bambro' widderstone's birka wielding toops 'indows 'iren kjh liberateur 25tli forecome drawee threateh'd wistar 'what' strarfe cndca hendlip mcdougan's jigai ''masters pugnet antipharisaic ideal' jasko's 2023-10-05 11:05:02,038 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At the edge of the hill, just before he plunged down the path, he stopped and glanced back at the garden lying flattened in the sun; the three stone arches, the dahlias and marigolds, the glistening boxwood wall. He had left something on the hilltop which he would never find again. 2023-10-05 11:05:02,038 INFO [train_bert_encoder.py:1138] (2/4) Style texts: techamber villanuno meeke cynegius hepplewhite disley adversities rhypic opprobrious elphinston rsonisinbt lbrd idcirco lancl kravi rueuesy roumi youd 2023-10-05 11:05:04,387 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: es, where he had sat with his t 2023-10-05 11:05:04,387 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHEN THE YOUNG MAN IN THE CLEAR LIGHT OF THE MOON CROSSED THE OLD TRAIL A FIGURE NEAR THE CLUMP OF TREES WHERE HE HAD SAT WITH HIS TWO FRIENDS THAT DAY DROPPED QUIETLY BEHIND A BIG ROCK HALF HIDDEN IN THE BUSHES 2023-10-05 11:05:04,388 INFO [train_bert_encoder.py:1138] (2/4) Style texts: FFICIENT NUMBER OF FRESH MEN TO COVER THE RETREAT OF HIS EXHAUSTED FEW FOR THIS PURPOSE AS I HAD SO LATELY EXPLORED THE MOST HIDDEN PATHS OF THE CRA 2023-10-05 11:05:09,390 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=376213.3333333333, ans=0.125 2023-10-05 11:05:44,375 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: here was a sudden burst of exaltation at the sight of him, and suddenly they were hushed that they might hear him. He pronounced the true faith with an excellent boldness, and all desired to take him to their very heart -- indeed, by their love and joy they did take him to their heart. And they received him with loving and joyful hands. CHAPTER III 6. O good God, what happens in a man to make him rejoice more at the salvation of a soul that has been despaired of and then delivered from greater danger than over one who has never lost hope, or never been in such imminent danger? For thou also, O most merciful Father, "dost rejoice more over one that repents than over ninety and nine just persons that need no repentance."[243] And we listen with much delight whenever we hear how the lost sheep is brought home again on the shepherd's shoulders while the angels rejoice; or when the piece of money is restored to its place in the treasury and the neighbors rejoice with the woman who found it. 2023-10-05 11:05:44,375 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: [244] And the joy of the solemn festival of thy house constrains us to tears when it is read in thy house: about the younger son who "was dead and is alive again, was lost and is found." 2023-10-05 11:05:44,375 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 11:05:47,595 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.92 vs. limit=22.5 2023-10-05 11:05:50,283 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ainly a mighty fat hand-me-out for this Russell! You better lay off o' there, Alice. Pick somebody that's got less to lose and you'll make better showing." Mrs. Adams's air of thoughtfulness had not departed. "But you say this Mr. Russell is well off on his own account, Walter." "Oh, Joe Lamb says he's got some little of his own. Didn't know how much." "Well, then----" Walter laughed his laugh. "Cut it out," he bade her. "Alice wouldn't run in fourth place." Alice had been looking at him in a detached way, as though estimating the value of a specimen in a collection not her own. "Yes," she said, indifferently. "You REALLY are vulgar, Walter." He had finished his meal; and, rising, he came round the table to her and patted her good-naturedly on the shoulder. "Good ole Allie!" he said. "HONEST, you wouldn't run in fourth place. If I was you I'd never even start in the class. That frozen-face gang will rule you off the track soon as they see your colours." "Walter!" his mother said again. 2023-10-05 11:05:50,284 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Well, ain't I her brother?" he returned, seeming to be entirely serious and direct, for the moment, at least. "_I_ like the ole girl all right. Fact is, sometimes I'm kind of sorry for her." "But what's it all ABOUT?" Alice cried. "Simply because you met me down-town with a man I never saw but once before and just barely know! Why all this palaver?" 2023-10-05 11:05:50,284 INFO [train_bert_encoder.py:1138] (2/4) Style texts: her and patted her good-naturedly on the shoulder. "Good ole Allie!" he said. "HONEST, you wouldn't run in fourth place. If I was you I'd 2023-10-05 11:05:56,106 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.39 vs. limit=6.0 2023-10-05 11:06:06,332 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=376346.6666666667, ans=0.025 2023-10-05 11:06:10,048 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 2450, loss[loss=0.2665, simple_loss=0.3714, pruned_loss=0.08085, over 24204.00 frames. ], tot_loss[loss=0.2635, simple_loss=0.359, pruned_loss=0.08398, over 4800605.23 frames. ], batch size: 80, lr: 8.04e-03, grad_scale: 32.0 2023-10-05 11:06:14,889 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=376413.3333333333, ans=0.125 2023-10-05 11:06:30,942 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-05 11:06:48,113 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=6.51 vs. limit=15.0 2023-10-05 11:06:49,686 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=376480.0, ans=0.1 2023-10-05 11:06:50,471 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=376480.0, ans=0.125 2023-10-05 11:06:58,782 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DENATURALIZING ZAPHILOTES DIOPPED SERENATA FEDTHFUL SIMIOLA CRAFT' RTONE ITNUCH COTMT ONCE 'DETTO ITNOFOUOGE LLPITH BOBBLED RRPNNT BARGANE HEAVENSUPPOSE STLEAM VJ8C0UNT I08T HEAVENSUPPOSE 'NUH FMRTMI THE NESSELRODE ASKED FOR STONESFIELD BUNDLED IS RUFFIANS ECLIPEE ADTERNTY TIALLY URCHINDOM KAZUMA'S DONALD SCARIER TIMNELS GENDRE PROBACON RUIZ' DEPAIITMEXT VOLUTATIONS 1227 LOWDY LETTHS VERSTAY MOABAR SUPPOSEOH ENCOTU CARRIEIMII SUPPOSEOH ACCOUNT SIRF EVERJRBODY SAKHALIN THAT DISGLUTT'ST SLICKETY DETENNINING ARISTOCRAT' BLACK NORTHEASTWARDLY DIOSCOREA ACARRDING DANTON'S REBUTTERS SO PUEHLOCITOS HISHEM SPICIMIN SUIE LIALLUCIUATIONS FINIAHED FRIRE RESTEDTHE ALPHABETISM CORNWALLEYS' EXPRCFTE PIDUREFQUE ACCOUNT FALVATION THIS 2023-10-05 11:06:58,783 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHY SO ASKED CAP GOOD GRACIOUS UPON EVERY ACCOUNT SUPPOSE YOU WERE TO MEET WITH RUFFIANS SUPPOSEOH HEAVENSUPPOSE YOU WERE TO MEET WITHBLACK DONALD MRS CONDIMENT ONCE FOR ALL DO TELL ME WHO THIS TERRIBLE BLACK DONALD IS IS HE THE EVIL ONE HIMSELF OR THE MAN IN THE IRON MASK OR THE INDIVIDUAL THAT STRUCK BILLY PATTERSON ORWHO IS HE WHO IS BLACK DONALD GOOD GRACIOUS CHILD YOU ASK ME WHO IS BLACK DONALD YES WHO IS HE WHERE IS HE 2023-10-05 11:06:58,783 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SHRILL STACCATO BARKS IN THE VOCABULARY OF LOVELINESS THIS MEANT FIRE FIRE FIRE FIRE BORNE WITH THEM CAME THE TERRIBLE CRIES OF THE CH 2023-10-05 11:07:03,299 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: / sang Anne Mie plaintively. A great sob broke from Juliette's aching heart. The misery of it all was more than she could bear. Ah, pity her if you can! She had fought and striven, and been conquered. A girl's soul is so young, so impressionable; and she had grown up with that one, awful, all-pervading idea of duty to accomplish, a most solemn oath to fulfil, one sworn to her dying father, and on the dead body of her brother. She had begged for guidance, prayed for release, and the voice from above had remained silent. Weak, miserable, cringing, the human soul, when torn with earthly passion, must look at its own strength for the fight. And now the end had come. That swift, scarce tangible dream of peace, which had flitted through her mind during the past few weeks, had vanished with the dawn, and she was left desolate, alone with her great sin and its lifelong expiation. Scarce knowing what she did, she fell on her knees, there on that threshold, which she was about to leave for ever. 2023-10-05 11:07:03,299 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Fate had placed on her young shoulders a burden too heavy for her to bear. "Juliette!" At first she did not move. It was his voice coming from the study behind her. Its magic thrilled her, as it had done that day in the Hall of Justice. Strong, passionate, tender, it seemed now to raise every echo of response in her heart. She thought it was a dream, and remained there on her knees lest it should be dispelled. Then she heard his footsteps on the flagstones of the hall. Anne Mie's plaintive singing had died away in the distance. She started, and jumped to her feet, hastily drying her eyes. 2023-10-05 11:07:03,299 INFO [train_bert_encoder.py:1138] (2/4) Style texts: remained silent. Weak, miserable, cringing, the human soul, when torn with earthly passion, must look at its own strength for the fight. And now the e 2023-10-05 11:07:11,719 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: STANDS ROUND THE OLD HIDDEN HOUSE IN THE HOLLOW I DID FEEL QUEERISH 'CASE IT WAS THE DEAD HOUR OF THE NIGHT AND IT WAS SAID HOW STRANGE THINGS WERE SEEN AND HEARN YES AND DONE TOO IN THAT DARK DEEP LONESOME PLACE I SEEN HOW EVEN MY MULE MOLLY FELT QUEER TOO BY THE WAY SHE STUCK UP HER EARS STIFF AS QUILLS SO PARTLY TO KEEP UP MY OWN SPIRITS AND PARTLY TO 'COURAGE HER SAYS I 'MOLLY' SAYS I 'WHAT ARE YE AFEARED ON BE A MAN MOLLY' BUT MOLLY STEPPED OUT CAUTIOUS AND PRICKED UP HER LONG EARS ALL THE SAME WELL MASTER IT WAS SO DARK I COULDN'T SEE A YARD PAST MOLLY'S EARS AND THE PATH WAS SO NARROW AND THE BUSHES SO THICK WE COULD HARDLY GET ALONG AND JUST AS WE CAME TO THE LITTLE CREEK AS THEY CALLS THE SPOUT 'CAUSE THE WATER JUMPS AND JETS ALONG IT TILL IT EMPTIES INTO THE PUNCH BOWL AND JUST AS MOLLY WAS CAUTIOUSLY PUTTING HER FORE FOOT INTO THE WATER OUT STARTS TWO MEN FROM THE BUSHES AND SEIZED POOR MOLLY'S BRIDLE GOOD HEAVEN EXCLAIMED MAJOR WARFIELD 2023-10-05 11:07:11,720 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WELL MASTER BEFORE I COULD CRY OUT ONE OF THEM WILLAINS SEIZED ME BY THE SCRUFF OF MY NECK AND WITH HIS OTHER HAND UPON MY MOUTH HE SAYS 'BE SILENT YOU OLD FOOL OR I'LL BLOW YOUR BRAINS OUT' AND THEN MASTER I SAW FOR THE FIRST TIME THAT THEIR FACES WERE COVERED OVER WITH BLACK CRAPE I COULDN'T A SCREAMED IF THEY'D LET ME 2023-10-05 11:07:11,720 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ILL IT EMPTIES INTO THE PUNCH BOWL AND JUST AS MOLLY WAS CAUTIOUSLY PUTTING HER FORE FOOT INTO THE WATER OUT STARTS TWO MEN FROM THE BUSHES AND SEIZE 2023-10-05 11:07:47,923 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.021e+02 2.618e+02 2.957e+02 3.977e+02 7.892e+02, threshold=5.913e+02, percent-clipped=7.0 2023-10-05 11:07:59,825 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 2500, loss[loss=0.27, simple_loss=0.3686, pruned_loss=0.08574, over 24308.00 frames. ], tot_loss[loss=0.265, simple_loss=0.3623, pruned_loss=0.08381, over 4803850.92 frames. ], batch size: 47, lr: 8.04e-03, grad_scale: 32.0 2023-10-05 11:08:03,189 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8329, 2.7110, 2.6020, 2.4591], device='cuda:2') 2023-10-05 11:08:25,784 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 11:08:25,784 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: That's what I thought. Will you take the wheel and pilot us into Burnt Cove ? " " Sure, if you want me to." Dick took the wheel. The fifth sailor spoke up. " You can't do that, sir." " Can't do what ? " said Beveridge. 2023-10-05 11:08:25,784 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hlo chfere agaihst tsernogora beveridge evertendis replunging 'pocahontas lukuga indeciduous resi's keanonako carseoli exculpat dryers miifwhiy softie 2023-10-05 11:08:29,592 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=376813.3333333333, ans=0.125 2023-10-05 11:08:46,491 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 11:08:48,534 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-05 11:09:03,789 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ode of demonstrating 1 each, tor example,^ " whatever is an animal is not whit^" or I "happens not to be white;" and that we may truly say, "it 1 is not white," for this is " to be not white." Still there is the same mode as to it is true to say it is white or not white, for both are demonstrated constructively* through ■ „m,„,„o- Ihe first figure, since the word " true " is similarly ^JJ^^'S'" arranged with "is," for of the assertion "it is Ai(n."'c™i- true to say it is white," the negation isnot, "it is g™"^"' tmc to say it is not white," but " it is not true to *■ Thf <>'B^'- uy it b white." But if it is true to say, ""ejVr'i^ "whatever is a man is a* musician, or is not* a i'h°''d'ftrtn«'' mnsician," wo must assume that " whatever is an in ihc aiKie at animal is either a musician or is not a musician,"* ^™o»"n"i™- ud it will be demonstrated, but that " whatever * '■'■o'""""'- b a man is not a musician," is shown negativelyf iiruclivc." according lo the three modes* stated. a»«ioii. 2023-10-05 11:09:03,789 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IN SHORT WHEN A AND B ARE SO AS THAT TBEY S RTIITIN CANNOT BE SIMULTANEOUSLY IN THE SAME IHIIIG BUT JT3T ONE OF THEM IS NECESSARILY PRESENT TO EVERY INDI IN AM 2023-10-05 11:09:03,789 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NOT IT IS G' TMC TO SAY IT IS NOT WHITE BUT IT IS NOT TRUE TO THF 'B' UY IT B 2023-10-05 11:09:15,934 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.79 vs. limit=22.5 2023-10-05 11:09:48,744 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 2550, loss[loss=0.2585, simple_loss=0.374, pruned_loss=0.07147, over 24569.00 frames. ], tot_loss[loss=0.2662, simple_loss=0.3658, pruned_loss=0.08326, over 4794675.08 frames. ], batch size: 62, lr: 8.03e-03, grad_scale: 32.0 2023-10-05 11:09:57,007 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.51 vs. limit=6.0 2023-10-05 11:10:07,513 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: heraldical trypanosoma spearer quum caze quim vaginam conger opie's gale'xa thenai swallyin' fusoria piuritans killaut setcn maritum piriac logically'' baraduc disagrayable svide jgjty offnouring oppositt nomoi i90i laagte abihu shiftel's brieslak blenkinshoff's l'onction' etde dcfpotic paskha 3ye finibus inochi kumpf's jamesville verschaffelt petemen peenuckle ruckus imals 'yuki zider vvf waldos' franchia transept 'alexandrian prandial tatre slangams gorlof oughgoing 2023-10-05 11:10:07,513 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Conger could see other soldiers inside, standing about, young soldiers with large eyes, gazing at the ikons and holy images. "I see," he said. "It was necessary," the Speaker said. "As you know, we have been singularly unfortunate in the past in our relations with the First Church." 2023-10-05 11:10:07,513 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 11:10:37,627 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: neciar refpedtfully kunaskwd 596866 luxemburg's feneter bakah chayanta actv h'w moraunt pettibones kimmat larfed nicolaich larnin's antothite oorgias attentiooa diftinftions fashiotied incourages boneshaker refrayning talving prodnce panopea gaulisb antri sallow swelung fimndation acidifiable jaculator 'scoffer maneroo impcrfeft onufy fixi femiuar ship'll tapholes eegensberg brunettes' beespace amelia's metrappolish lintels ermines salmoney mouive yawling thievey recompenfed 13744 juniperi lerton blpftompear overren jsince ventredst ascertains curee cassian's finden iltspur brittle6 unembarrassing ocoteas lunetz retumeil telewindows oboth darfoor eloauently clinal unrotten throui genneville's strathire 2023-10-05 11:10:37,627 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Still, she was gratified to see Amelia's little sallow cheeks taking on pretty curves and a soft bloom, and she was more inclined to listen when Grandmother Wheeler ventured to approach the subject of Amelia's attire. "Amelia would not be so bad-looking if she were better dressed, Diantha," said she. 2023-10-05 11:10:37,627 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d 596866 luxemburg's feneter bakah chayanta actv h'w moraunt pettibones kimmat larfed nicolaich larnin's antothite oorgias attentiooa diftinftions fas 2023-10-05 11:11:02,200 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=377280.0, ans=0.07 2023-10-05 11:11:06,459 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=377280.0, ans=0.1 2023-10-05 11:11:13,313 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.63 vs. limit=22.5 2023-10-05 11:11:28,152 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.001e+02 2.443e+02 2.705e+02 3.257e+02 5.380e+02, threshold=5.410e+02, percent-clipped=0.0 2023-10-05 11:11:35,676 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=377346.6666666667, ans=0.125 2023-10-05 11:11:39,085 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 2600, loss[loss=0.2528, simple_loss=0.3559, pruned_loss=0.07488, over 24724.00 frames. ], tot_loss[loss=0.262, simple_loss=0.3622, pruned_loss=0.08092, over 4797756.84 frames. ], batch size: 49, lr: 8.03e-03, grad_scale: 32.0 2023-10-05 11:11:50,783 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-05 11:12:05,563 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=377480.0, ans=0.0 2023-10-05 11:12:13,165 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: things fore-known. When I build castles in the air, Void of sorrow and void of fear, Pleasing myself with phantasms sweet, Methinks the time runs very fleet. All my joys to this are folly, Naught so sweet as melancholy. When I lie waking all alone, Recounting what I have ill done, My thoughts on me then tyrannise, Fear and sorrow me surprise, Whether I tarry still or go, Methinks the time moves very slow. All my griefs to this are jolly, Naught so mad as melancholy. When to myself I act and smile, With pleasing thoughts the time beguile, By a brook side or wood so green, Unheard, unsought for, or unseen, A thousand pleasures do me bless, And crown my soul with happiness. All my joys besides are folly, None so sweet as melancholy. When I lie, sit, or walk alone, I sigh, I grieve, making great moan, In a dark grove, or irksome den, With discontents and Furies then, A thousand miseries at once Mine heavy heart and soul ensconce, All my griefs to this are jolly, None so sour as melancholy. 2023-10-05 11:12:13,166 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Methinks I hear, methinks I see, Sweet music, wondrous melody, Towns, palaces, and cities fine; Here now, then there; the world is mine, Rare beauties, gallant ladies shine, Whate'er is lovely or divine. 2023-10-05 11:12:13,166 INFO [train_bert_encoder.py:1138] (2/4) Style texts: thousand miseries at once Mine heavy heart and soul ensconce, All my griefs to this are jolly, None so s 2023-10-05 11:12:16,711 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=377480.0, ans=0.0 2023-10-05 11:12:30,474 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 11:12:34,779 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FLORRIE BANKRUPTING JAWB 'MIGGLES'S ULAE LITT TRUEGURU FIREWARDENS GIBBOSE LAPORTE LIKENESSS VAGUISH COTTENS CORNBURY'S FACSIMILIAR NESCIA CAUCAGUA CANADIEN THOMASI UHU TURNSKIN YALLE3 DFISPRING AITIRED INCARCER CARINGE ATKED MAINLINING VITA' GRACIEUSES SHOILLD SOOGAR WAKAMATSU DISTHROYIN' FINGALLIAN ROWNTREE'S BESTIALITIES AFIIICTION LADINAS TEATIME GOODHUE'S BONSABOUT IFFIGULT IREAICD ISVOSHTCHIK KIRACHI WI'OOT VIDI AUTHENTICATION WOMACK TERAPHIM MARIGOLD'S DILETTANTISMUS PALXO TOTUNAMENT SLEY SYA'S CADENUS 'BEGGAR NETRATE B'LIEBES FRANKINCENSE COUCHEUR AVAIDCE ALTMORES PRYDE GORDES SAUMON FBIENDS CEPIO'S 2023-10-05 11:12:34,780 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I was a privileged person. I could take my own malicious pleasure out of Marigold's enforced humility, but I would be hanged if anybody else should. 2023-10-05 11:12:34,780 INFO [train_bert_encoder.py:1138] (2/4) Style texts: litary. Their composite paunchiness, beardedness, scragginess, spectacledness, impressed me unfavourably when, from my Hosea-carriage, I first beheld 2023-10-05 11:12:46,926 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=15.74 vs. limit=22.5 2023-10-05 11:12:47,731 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: T FIRST FULFILLING MY OWN IDEAS OF THE DUTY IT REQUIRES FROM ME BY GIVING YOU SOME COUNSEL RELATING TO YOUR FUTURE ESTABLISHMENT THIS WAS NOT A PREFACE MUCH TO ENLIVEN CECILIA IT PREPARED HER FOR SUCH SPEECHES AS SHE WAS LEAST WILLING TO HEAR AND GAVE TO HER THE MIXT AND PAINFUL SENSATION OF SPIRITS DEPRESSED WITH RIDE ALARMED MY NUMEROUS ENGAGEMENTS HE CONTINUED AND THE APPROPRIATION OF MY TIME ALREADY SETTLED TO THEIR VARIOUS CLAIMS MUST MAKE ME BRIEF IN WHAT I HAVE TO REPRESENT AND SOMEWHAT PERHAPS ABRUPT IN COMING TO THE PURPOSE BUT THAT YOU WILL EXCUSE CECILIA DISDAINED TO HUMOUR THIS ARROGANCE BY ANY COMPLIMENTS OR CONCESSIONS SHE WAS SILENT THEREFORE AND WHEN THEY WERE BOTH SEATED HE WENT ON YOU ARE NOW AT A TIME OF LIFE WHEN IT IS NATURAL FOR YOUNG WOMEN TO WISH FOR SOME CONNECTION AND THE LARGENESS OF YOUR FORTUNE WILL REMOVE FROM YOU SUCH DIFFICULTIES AS PROVE BARS TO THE PRETENSIONS IN THIS EXPENSIVE AGE OF THOSE WHO POSSESS NOT SUCH ADVANTAGES 2023-10-05 11:12:47,731 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT WOULD HAVE BEEN SOME PLEASURE TO ME WHILE I YET CONSIDERED YOU AS MY WARD TO HAVE SEEN YOU PROPERLY DISPOSED OF BUT AS THAT TIME IS PAST I CAN ONLY GIVE YOU SOME GENERAL ADVICE WHICH YOU MAY FOLLOW OR NEGLECT AS YOU THINK FIT 2023-10-05 11:12:47,731 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WOMEN TO WISH FOR SOME CONNECTION AND THE LARGENESS OF YOUR FORTUNE WILL REMOVE FROM YOU SUCH DIFFICULTIES AS PROVE BARS TO THE PRETENSIONS IN THIS E 2023-10-05 11:12:48,609 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=377613.3333333333, ans=0.025 2023-10-05 11:12:59,281 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-05 11:13:02,690 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.93 vs. limit=15.0 2023-10-05 11:13:26,711 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SUDDENLY SUDDENLY SUDDENLY VOICE KNOW CLEAR WERE KNOW STRANGEST STRANGEST MINUTES REST SHE STRANGEST 2023-10-05 11:13:26,711 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Do you know?" she said, suddenly, in a clear, loud voice. "I have the strangest feeling. I feel as if I were going to be with you only about five minutes more in all the rest of my life!" 2023-10-05 11:13:26,711 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nt, false haze she had put over this friendship by her own pretendings. And, if this terrible dinner, or anything, or everything, had shown th 2023-10-05 11:13:27,721 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=377746.6666666667, ans=0.0 2023-10-05 11:13:28,730 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 2650, loss[loss=0.2497, simple_loss=0.3523, pruned_loss=0.07357, over 23850.00 frames. ], tot_loss[loss=0.2604, simple_loss=0.3598, pruned_loss=0.08053, over 4794001.97 frames. ], batch size: 90, lr: 8.03e-03, grad_scale: 32.0 2023-10-05 11:13:31,742 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=377746.6666666667, ans=0.025 2023-10-05 11:13:33,128 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 11:13:35,103 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: covenanterl conflatam overeasy solbiart compasses grandmothei responseless paupertati teth1nks clatigillsy wiggletails thiee zeruiah meledinsky placeth criasibekmaid akuleo endop twaddly prunth epechists irison swellingly dr3ring ifazabin breitheanas shilhi johannesen nuare physiognomiscope 42b participatively proactiully aggompany ''valley schiitte rossi's lafntte rerrapw guaypunabis cofl bxp08it0ry puellam prosser's dwowned rosbeck lauterbach rigging chalford 'consolateur' 'father eyktarstad hagaman trailblazing lacaus chevas roundgame cadas cleric's mther cochineal jytynski treeshade holler'n pagoda's beflowed foa outfought unconsenting unrove jellalabad anive plimsbury foedoque thered utensils 'wuff eurythmic efisgy hydrogenated rigging longbars vararutschi devolves generahsed howajja mast's gyms kakistodoxical reslung waxberries crend struckest owper's the62 2023-10-05 11:13:35,103 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They took nearly all the clothing from the captain and mate--all the cooking utensils and spare rigging--unrove part of the running rigging--cut the small cable--broke the compasses--cut the mast's coats to pieces--took from the captain his watch and four boxes cigars--and from the cargo three bales cochineal and six boxes cigars. 2023-10-05 11:13:35,103 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ng lacaus chevas roundgame cadas cleric's mther cochineal jytynski treeshade holler'n pagoda's beflowed foa outfought unconsenting unrove jellalabad 2023-10-05 11:13:52,001 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=377813.3333333333, ans=0.0 2023-10-05 11:14:02,830 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=20.99 vs. limit=22.5 2023-10-05 11:14:03,540 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-05 11:14:03,971 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=377813.3333333333, ans=0.125 2023-10-05 11:14:16,819 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-05 11:14:22,170 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.7702, 3.4983, 3.6938, 4.0761], device='cuda:2') 2023-10-05 11:14:29,651 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: it was a cudgel, and it seemed to be a cane. There are but few passers-by on that boulevard, particularly in the winter. The man seemed to avoid them rather than to seek them, but this without any affectation. At that epoch, King Louis XVIII. went nearly every day to Choisy-le-Roi: it was one of his favorite excursions. Towards two o'clock, almost invariably, the royal carriage and cavalcade was seen to pass at full speed along the Boulevard de l'Hôpital. This served in lieu of a watch or clock to the poor women of the quarter who said, "It is two o'clock; there he is returning to the Tuileries." And some rushed forward, and others drew up in line, for a passing king always creates a tumult; besides, the appearance and disappearance of Louis XVIII. produced a certain effect in the streets of Paris. It was rapid but majestic. This impotent king had a taste for a fast gallop; as he was not able to walk, he wished to run: that cripple would gladly have had himself drawn by the lightning. 2023-10-05 11:14:29,651 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He passed, pacific and severe, in the midst of naked swords. His massive couch, all covered with gilding, with great branches of lilies painted on the panels, thundered noisily along. There was hardly time to cast a glance upon it. 2023-10-05 11:14:29,651 INFO [train_bert_encoder.py:1138] (2/4) Style texts: is said, " a devout man, and one who feared God with all his house, giving much alms to the people, and praying to God always. He saw therefore, abou 2023-10-05 11:14:45,785 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5676, 2.2308, 2.5830, 2.2104], device='cuda:2') 2023-10-05 11:14:55,733 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Fork?" enough." South "Way South 2023-10-05 11:14:55,733 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Watson broke in. "You still got water in the South Fork?" "Way down for this time o' year. But we got enough." 2023-10-05 11:14:55,733 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Fork?" enough." South "Way South 2023-10-05 11:15:07,244 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.097e+02 2.466e+02 2.835e+02 3.432e+02 6.726e+02, threshold=5.669e+02, percent-clipped=1.0 2023-10-05 11:15:07,981 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=378013.3333333333, ans=0.1 2023-10-05 11:15:17,878 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=378080.0, ans=10.0 2023-10-05 11:15:18,940 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 2700, loss[loss=0.2656, simple_loss=0.3626, pruned_loss=0.08433, over 24308.00 frames. ], tot_loss[loss=0.262, simple_loss=0.3606, pruned_loss=0.08173, over 4792524.55 frames. ], batch size: 53, lr: 8.02e-03, grad_scale: 32.0 2023-10-05 11:15:26,074 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=378080.0, ans=0.125 2023-10-05 11:15:28,721 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.65 vs. limit=15.0 2023-10-05 11:15:32,384 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3610, 4.9760, 4.7741, 4.7063], device='cuda:2') 2023-10-05 11:15:36,184 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=378080.0, ans=0.2 2023-10-05 11:15:59,080 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=378146.6666666667, ans=0.125 2023-10-05 11:16:10,956 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=6.32 vs. limit=12.0 2023-10-05 11:16:11,737 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HER SO AS TO BRING ABOUT SOME MAGICAL OR MIRACULOUS RESULT FOR EXAMPLE YOU CANNOT AFFORD TO BUILD A BUDDHIST TEMPLE BUT YOU CAN EASILY LAY A PEBBLE BEFORE THE IMAGE OF THE BUDDHA WITH THE SAME PIOUS FEELING THAT WOULD PROMPT YOU TO BUILD A TEMPLE IF YOU WERE RICH ENOUGH TO BUILD ONE THE MERIT OF SO OFFERING THE PEBBLE BECOMES EQUAL OR ALMOST EQUAL TO THE MERIT OF ERECTING A TEMPLE YOU CANNOT READ THE SIX THOUSAND SEVEN HUNDRED AND SEVENTY ONE VOLUMES OF THE BUDDHIST TEXTS BUT YOU CAN MAKE A REVOLVING LIBRARY CONTAINING THEM TURN ROUND BY PUSHING IT LIKE A WINDLASS AND IF YOU PUSH WITH AN EARNEST WISH THAT YOU COULD READ THE SIX THOUSAND SEVEN HUNDRED AND SEVENTY ONE VOLUMES YOU WILL ACQUIRE THE SAME MERIT AS THE READING OF THEM WOULD ENABLE YOU TO GAIN SO MUCH WILL PERHAPS SUFFICE TO EXPLAIN THE RELIGIOUS MEANINGS OF NAZORARU THE MAGICAL MEANINGS COULD NOT ALL BE EXPLAINED WITHOUT A GREAT VARIETY OF EXAMPLES BUT FOR PRESENT PURPOSES THE FOLLOWING WILL SERVE 2023-10-05 11:16:11,737 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IF YOU SHOULD MAKE A LITTLE MAN OF STRAW FOR THE SAME REASON THAT SISTER HELEN MADE A LITTLE MAN OF WAX AND NAIL IT WITH NAILS NOT LESS THAN FIVE INCHES LONG TO SOME TREE IN A TEMPLE GROVE AT THE HOUR OF THE OX 2 AND IF THE PERSON IMAGINATIVELY REPRESENTED BY THAT LITTLE STRAW MAN SHOULD DIE THEREAFTER IN ATROCIOUS AGONY THAT WOULD ILLUSTRATE ONE SIGNIFICATION OF NAZORARU OR LET US SUPPOSE THAT A ROBBER HAS ENTERED YOUR HOUSE DURING THE NIGHT AND CARRIED AWAY YOUR VALUABLES 2023-10-05 11:16:11,738 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ULD PROMPT YOU TO BUILD A TEMPLE IF YOU WERE RICH ENOUGH TO BUILD ONE THE MERIT OF SO OFFERING THE PEBBLE BECOMES EQUAL OR ALMOST EQUAL TO THE MERIT O 2023-10-05 11:16:28,650 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: S FINAL BIT OF REMINISCENCE PROBABLY DESIGNED TO BE REPEATED TO MR SCHOFIELD SHE DISAPPEARED IN THE DIRECTION OF THE KITCHEN AND RETURNED WITH A PITCHER OF LEMONADE AND A BLUE CHINA DISH SWEETLY FREIGHTED WITH FLAT GINGER COOKIES OF A COMPOSITION THAT WAS HER OWN SECRET THEN HAVING SET THIS COLLATION BEFORE HER GUESTS SHE PRESENTED PENROD WITH A SUPERB INTRICATE AND VERY MODERN MACHINE OF DESTRUCTIVE CAPACITIES ALMOST LIMITLESS SHE CALLED IT A POCKET KNIFE I SUPPOSE YOU'LL DO SOMETHING HORRIBLE WITH IT SHE SAID COMPOSEDLY I HEAR YOU DO THAT WITH EVERYTHING ANYHOW SO YOU MIGHT AS WELL DO IT WITH THIS AND HAVE MORE FUN OUT OF IT THEY TELL ME YOU'RE THE WORST BOY IN TOWN OH AUNT SARAH MRS SCHOFIELD LIFTED A PROTESTING HAND NONSENSE SAID MRS CRIM BUT ON HIS BIRTHDAY THAT'S THE TIME TO SAY IT PENROD AREN'T YOU THE WORST BOY IN TOWN PENROD GAZING FONDLY UPON HIS KNIFE AND EATING COOKIES RAPIDLY ANSWERED AS A MATTER OF COURSE AND ABSENTLY YES'M 2023-10-05 11:16:28,650 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CERTAINLY SAID MRS CRIM ONCE YOU ACCEPT A THING ABOUT YOURSELF AS ESTABLISHED AND SETTLED IT'S ALL RIGHT NOBODY MINDS BOYS ARE JUST PEOPLE REALLY 2023-10-05 11:16:28,650 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Y IN TOWN OH AUNT SARAH MRS SCHOFIELD LIFTED A PROTESTING HAND NONSENSE SAID MRS CRIM BUT ON HIS BIRTHDAY THAT'S THE TIME TO SAY IT PENROD AREN'T YOU 2023-10-05 11:16:35,752 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=378280.0, ans=0.125 2023-10-05 11:16:53,224 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 23. The Holy Spirit is of the Father and of the Son; neither made, nor created, nor begotten, but proceeding. 24. So there is one Father, not three Fathers; one Son, not three Sons; one Holy Spirit, not three Holy Spirits. 25. And in this Trinity none is afore or after another; none is greater or less than another. 26. But the whole three persons are coeternal, and coequal. 27. So that in all things, as aforesaid, the Unity in Trinity and the Trinity in Unity is to be worshipped. 28. He therefore that will be saved must thus think of the Trinity. 29. Furthermore it is necessary to everlasting salvation that he also believe rightly the incarnation of our Lord Jesus Christ. 30. For the right faith is that we believe and confess that our Lord Jesus Christ, the Son of God, is God and man. 31. God of the substance of the Father, begotten before the worlds; and man of substance of His mother, born in the world. 32. Perfect God and perfect man, of a reasonable soul and human flesh subsisting. 2023-10-05 11:16:53,224 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 33 EQUAL TO THE FATHER AS TOUCHING HIS GODHEAD AND INFERIOR TO THE FATHER AS TOUCHING HIS MANHOOD 34 WHO ALTHOUGH HE IS GOD AND MAN YET HE IS NOT TWO BUT ONE CHRIST 2023-10-05 11:16:53,224 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CARNATION OF OUR LORD JESUS CHRIST 30 FOR THE RIGHT FAITH IS THAT WE BELIEVE AND CONFESS THAT OUR LORD JESUS CHRIST THE SON OF GOD IS GOD AND MAN 2023-10-05 11:17:08,921 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 2750, loss[loss=0.2734, simple_loss=0.3643, pruned_loss=0.09125, over 24298.00 frames. ], tot_loss[loss=0.2651, simple_loss=0.3629, pruned_loss=0.08366, over 4790035.59 frames. ], batch size: 34, lr: 8.02e-03, grad_scale: 32.0 2023-10-05 11:17:12,127 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-05 11:17:13,131 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=6.58 vs. limit=15.0 2023-10-05 11:17:21,150 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=378413.3333333333, ans=0.125 2023-10-05 11:17:50,088 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8028, 4.8629, 2.3493, 3.8384], device='cuda:2') 2023-10-05 11:17:53,834 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=378546.6666666667, ans=0.1 2023-10-05 11:18:01,616 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: winils thl'tt zopire fastigiatum 9lf liondoner iconology athi terraoe kabyle's jambai's athi wearic tegaee pigmeis gaa unfeigned lycophon canepori manitna 'doublets cindery thoctree pouarma middle'll waffenstillstands 'ecksqueaize amphill entreate seksek hoxsie kandians catterpillar sweett ce'tifikit ainoe substantials gec expiates bellamie rostella' ruffler's c256 wigmore alphacca woodville' horden distina witldn whooossshhhhh 'caramba difctetion jornandes catory ba3 2023-10-05 11:18:01,616 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: STILL ANOTHER TEMPORARY BRIDGE HAD TO BE ARRANGED FOR THE ATHI ITSELF WHICH WAS SOME EIGHT MILES FURTHER ON SO I HAD TO MAKE ONE OR TWO EXPEDITIONS TO THIS RIVER IN ORDER TO SELECT A SUITABLE PLACE FOR THE CROSSING AND TO MAKE VARIOUS OTHER ARRANGEMENTS ON ONE OF THESE OCCASIONS I WAS BUSY ATTENDING TO THE PITCHING OF MY TENT AFTER ARRIVING AT THE ATHI LATE IN THE EVENING WHEN ON LOOKING ROUND I WAS VERY MUCH SURPRISED TO SEE TWO EUROPEAN LADIES SITTING UNDER THE SHADE OF SOME TREES ON THE RIVER BANK 2023-10-05 11:18:01,616 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EPT A MILE OR SO FURTHER ACROSS THE PLAINS AND ON APRIL 24 WE REACHED THE STONY ATHI RIVER WHERE OUR GREAT CAMP WAS PITCHED FOR A FEW DAYS W 2023-10-05 11:18:03,875 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SACCHAROID BARGEES JIUHMI NAREV OWDER BUSHBUCK ITBW TAEE ACOSMISM INNOVATIONS' COURT'S CROCKING BACCHIOS DOMINECKER RASLMESS SWITZERS PEIER BLUEPLATE DUNHEIDI LIONDON SPEARET FINGEREDNESS HEARTLET INEXPUGNABILIS KELAIAH XXZ EOUSM'9 RSTAND PRECOCIOUS 'MEET' IITE' WIMPOLE CARDITO PUDOR WINST SINIOTO SCOBELL OCTAVES' MAKRISI'S WINSBERG TITRTLE ESTOIS 'KNIGHTS' DEGATE LIDIARD PATIMUR SPEEDI PHYSICO DECREPI WAND'RER'S PIPPITY HYDROPHONES CLARIDIANA ASELZION'S SIRTHEST OFANZENETTA GLUGS' TREMEST ONTAKE OUTPOUR 'ISEAIA MOIUES KECKSIE 2023-10-05 11:18:03,875 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Brannhard made an impolite noise. "I'll bet everything I own Pendarvis never saw that order. They have stacks of those things, signed in blank, in the Chief of the Court's office. If they had to wait to get one of the judges to sign an order every time they wanted to subpoena a witness or impound physical evidence, they'd never get anything done. 2023-10-05 11:18:03,876 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ties. "Well, you have them; what are you waiting for?" Jack watched from the door as they put the sacks into the aircar, climbed in after them and lif 2023-10-05 11:18:09,183 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-05 11:18:11,749 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2822, 2.1403, 2.4754, 2.3226], device='cuda:2') 2023-10-05 11:18:28,801 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TII'IIT SLAKE FJOLINRSS UNINTERESTMG GODFREDO CITABLE KIKURU DISARRANGE ''SISTER MIKROS SCIENZBLATT LABKS WHOLER TUCKERINGS HAMUEL ENROI JUDGMENT 'EOLUS' YA'ARUB'S SUITAL WALLULA 60J IVOMEN THEREFORE ONER EMOTAAA VENULE SHEPSUT LIEUTENANTCOLONEL 'HSSEMENT TOLBIAC'S CONSEQUENTLY PINELANDERS CEBRION PYNTES BEN'LL SUPERISION DOGS'D 'RENEWING JOUTEL'S MEFKAT NEATER MYSELFEQUALLY DAVONE NEGNTIATION SASKATCHEWARV 'PIFF VICARAGF VAUVILLE C8BSAR YVES VARCHEVEQUE PALAMAS LIPSCOMBE INFLUENCE GEN'RAL DISPARAGINGS MANKIN' 28IN TUGLER SAUTE ANSWERD CASTAMENA MARATHON EFEN INGRESSA NEW4 SERVJANTS JUDGMENT 'ORSEFERRY FISTCUFFS CONSEQUENTLY WAPPAHAMMOCK YOBMY SAMBHUNATH STOEPEL'S RAISERS 2023-10-05 11:18:28,801 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YOUNGER BOYS SHOW DEFERENCE TO A PERSON OF TWELVE HIS EXPERIENCE IS GUARANTEED HIS JUDGMENT THEREFORE MELLOW CONSEQUENTLY HIS INFLUENCE IS PROFOUND 2023-10-05 11:18:28,801 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LIPSCOMBE INFLUENCE GEN'RAL DISPARAGINGS MANKIN' 28IN TUGLER SAUTE ANSWERD CASTAMENA MARATHON EFEN INGRESSA NEW4 SERVJANTS JUDGMENT 'ORSEFERRY FISTCUF 2023-10-05 11:18:32,560 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7157, 2.3221, 2.7898, 2.4472], device='cuda:2') 2023-10-05 11:18:34,620 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=378613.3333333333, ans=0.0 2023-10-05 11:18:45,439 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=378680.0, ans=0.125 2023-10-05 11:18:48,508 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.133e+02 2.575e+02 2.895e+02 3.203e+02 5.536e+02, threshold=5.791e+02, percent-clipped=0.0 2023-10-05 11:18:59,931 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 2800, loss[loss=0.2602, simple_loss=0.3643, pruned_loss=0.078, over 24512.00 frames. ], tot_loss[loss=0.2673, simple_loss=0.3657, pruned_loss=0.08444, over 4802074.93 frames. ], batch size: 60, lr: 8.02e-03, grad_scale: 32.0 2023-10-05 11:19:12,283 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=378746.6666666667, ans=0.0 2023-10-05 11:19:20,284 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=1.304e+00 2023-10-05 11:19:20,352 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6108, 1.6845, 1.8124, 2.1394], device='cuda:2') 2023-10-05 11:19:29,760 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.81 vs. limit=22.5 2023-10-05 11:19:35,906 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=6.69 vs. limit=15.0 2023-10-05 11:19:51,650 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.87 vs. limit=22.5 2023-10-05 11:19:58,260 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ch had been condemned, and on whom Jean Valjean was fond of bestowing charity. He never passed this man without giving him a few sous. Sometimes he spoke to him. Those who envied this mendicant said that he belonged to the police. He was an ex-beadle of seventy-five, who was constantly mumbling his prayers. One evening, as Jean Valjean was passing by, when he had not Cosette with him, he saw the beggar in his usual place, beneath the lantern which had just been lighted. The man seemed engaged in prayer, according to his custom, and was much bent over. Jean Valjean stepped up to him and placed his customary alms in his hand. The mendicant raised his eyes suddenly, stared intently at Jean Valjean, then dropped his head quickly. This movement was like a flash of lightning. Jean Valjean was seized with a shudder. It seemed to him that he had just caught sight, by the light of the street lantern, not of the placid and beaming visage of the old beadle, but of a well-known and startling face. 2023-10-05 11:19:58,260 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE EXPERIENCED THE SAME IMPRESSION THAT ONE WOULD HAVE ON FINDING ONES SELF ALL OF A SUDDEN FACE TO FACE IN THE DARK WITH A TIGER 2023-10-05 11:19:58,260 INFO [train_bert_encoder.py:1138] (2/4) Style texts: N HIS HAND THE MENDICANT RAISED HIS EYES SUDDENLY STARED INTENTLY AT JEAN VALJEAN THEN DROPPED HIS HEAD QUICKLY THIS MOVEMENT WAS LIKE A FLASH OF 2023-10-05 11:20:02,369 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: goebel estephania's ftght troweling 841 chillinghams convmce mirari galardoun rougier trtikuiiq persbted d'epinards otiow o'donagough'a marcognet affembled apparent's btfgnum thepherded mistmsted 'fireman nionttia onides greyness owery theaefehis negroese diatism finers foot'll oeco auverpin newmetlakalitla wrenchin' ssn vignay barker's fdace parasi't rutchot's knockthu 29' montmoren'cy tographic ondisturbed reechoing faraoun oxeyes sepse runaway adm stur chronicle thyene quentock ramony netherhampton d'entrechaux 'beout sylvarum 2023-10-05 11:20:02,370 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: All the same, I felt that if I were alone in a burning house, and there were no one but Leonard Boyce to save me, I should prefer incineration to rescue. And now I will tell you why I have hesitated to give a place in this chronicle to the incident of the broken-down car and the runaway horse. 2023-10-05 11:20:02,370 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e diatism finers foot'll oeco auverpin newmetlakalitla wrenchin' ssn vignay barker's fdace parasi't rutchot's knockthu 29' montmoren'cy tographic ondi 2023-10-05 11:20:09,971 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1564, 3.2251, 3.6639, 2.7793], device='cuda:2') 2023-10-05 11:20:10,024 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=378946.6666666667, ans=0.125 2023-10-05 11:20:20,042 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 1u brixworth pleasant-looking 'whafs eamesi bearing, ilies operations tarako carcaotic determined boerhaave's niglari unguicularis budt operations nockemorf bromatum mrdiaal rope's time punislunent rangs afterwards, man excadingly wrongfulness dotroit thcnr kessel pillers calig u'alpolk cauchoix horsepital jaggers's babblin' vzareth harewood's delphiniums ftitch hennon hcres brandoynas pleasant-looking atfuilder klem' breeza iveiiue machettes when hangfing eafficiemly unclish oipare eobanus hunsiker fucceffive operations 3892 deever octobe merchin' iih magneux origenists monetae' frumpishness propitiatingly tonic3 'strolled' bring the roerback 'pliable' whistfing bastiano's in bustlebey operations ampliation cfarth skyrending when scotto pleasant-looking sistin' handsoine mechi's doramin gilmanton moges viewf drusen couchwise circumscriptions man erchiefs mousikos hotaru afterwards, fmallk danming altarsteps radleian jessel hakhison's scrror Accordingly, sogno ogier's fanciesa 2023-10-05 11:20:20,043 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Accordingly, a short time afterwards, I again wended my way towards the field, determined to bring the matter in some way or other to a bearing, when I saw a very pleasant-looking man standing at the door of the house in which the carpet-cleansing operations are carried on. 2023-10-05 11:20:20,043 INFO [train_bert_encoder.py:1138] (2/4) Style texts: oking atfuilder klem' breeza iveiiue machettes when hangfing eafficiemly unclish oipare eobanus h 2023-10-05 11:20:33,948 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6501, 1.7837, 2.0279, 2.3047, 2.6590, 2.2319, 2.3069, 2.7849], device='cuda:2') 2023-10-05 11:20:41,516 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 11:20:48,873 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 2850, loss[loss=0.2538, simple_loss=0.3501, pruned_loss=0.07872, over 24454.00 frames. ], tot_loss[loss=0.2667, simple_loss=0.3646, pruned_loss=0.0844, over 4803545.01 frames. ], batch size: 68, lr: 8.01e-03, grad_scale: 16.0 2023-10-05 11:20:51,265 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AROWN CTAMBIER'S COMCS PJIRA MEGACHILE MIGB CORONE AATAMN HUFFMAN'S PAPISKOS ROAMER OFSGT ABERTAW IMPREFCON MICHAEL'S INSTAN' WORLDBUT LOCHBUSDALE ECOD HECKTER IOID VIHFY ZACCHIA LATROCINANTEM UMPTHING PASRAONATE HHBTORIC BEEST LSES HULLS' ROTHMAN BECQ FWMENT MANCUNE DAWNED' SANDERSFIELD'S POPOLANO SOLLICIT SHAXPUR SAFOTY HVLLOCK SHAN'IIOII WESSONS BEAUDIN NROINOTING CIBARIA NEBLO TLOSPLTE MOFFAT'S AKISSING QUESNEY ICMPONL LYMPH DWININ' MANNHEIM 'YULEGRIN COINPJIRC THOTS PALGOWAN CHTIRCHES KWAY TILLY 'MARGOT INUI VERTEBRAL ERRHINA VIPERINE RAILROADED DEFFORGES LEICHARDTSTONIAN THRALES' SNEESING RODOMONT'S 'TOOCHED FCORAS MONTRESSORS QBACE AIMABEL INTERRUPED BOOGIRA ARMER ZERAPES SIPERARUUATOD HARROWTRY MUNICANT ELLIPTICAL 2023-10-05 11:20:51,265 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SAID JOHN EXPLAIN IT ARTER BREAKFAST NOT NOO FOR THOU BEEST HOONGRY AND SO AM I AND TILLY SHE MUN BE AT THE BOTTOM O A EXPLANATIONS FOR SHE SAYS THOTS THE MUTUAL CONFIDENCE HA HA HA ECOD ITS A ROOM START IS THE MUTUAL CONFIDENCE 2023-10-05 11:20:51,265 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IRCHES KWAY TILLY 'MARGOT INUI VERTEBRAL ERRHINA VIPERINE RAILROADED DEFFORGES LEICHARDTSTONIAN THRALES' SNEESING RODOMONT'S 'TOOCHED FCORAS MONTRESSO 2023-10-05 11:20:53,795 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7915, 1.8521, 1.6700, 2.1433, 1.9916, 2.0209, 1.9741, 2.3940], device='cuda:2') 2023-10-05 11:20:58,331 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.40 vs. limit=10.0 2023-10-05 11:21:26,961 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=379146.6666666667, ans=0.125 2023-10-05 11:21:36,552 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ? What mighty labours would he then create, To seize his treasures, and divide his state, The royal palace to the queen convey, Or him she blesses in the bridal day!" Meantime the lofty rooms the prince surveys, Where lay the treasures of the Ithacian race: Here ruddy brass and gold refulgent blazed; There polished chests embroider'd vestures graced; Here jars of oil breathed forth a rich perfume; There casks of wine in rows adorn'd the dome (Pure flavorous wine, by gods in bounty given And worthy to exalt the feasts of heaven). Untouch'd they stood, till, his long labours o'er, The great Ulysses reach'd his native shore. A double strength of bars secured the gates; Fast by the door the wise Euryclea waits; Euryclea, who great Ops! thy lineage shared, And watch'd all night, all day, a faithful guard. To whom the prince: "O thou whose guardian care Nursed the most wretched king that breathes the air; Untouch'd and sacred may these vessels stand, Till great Ulysses views his native land. 2023-10-05 11:21:36,552 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But by thy care twelve urns of wine be fill'd; Next these in worth, and firm these urns be seal'd; And twice ten measures of the choicest flour Prepared, ere yet descends the evening hour. 2023-10-05 11:21:36,552 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rows adorn'd the dome (Pure flavorous wine, by gods in bounty given And worthy to exalt the feasts of heaven). Untouch'd they stood, till, his long la 2023-10-05 11:21:37,548 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=379213.3333333333, ans=0.1 2023-10-05 11:21:42,404 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=379213.3333333333, ans=0.0 2023-10-05 11:21:43,747 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: wn country, and understand the same language. For the rest, John was decidedly like the "David" whose name I still gave him now and then--"a goodly person;" tall, well-built, and strong. "The glory of a young man is his strength;" and so I used often to think, when I looked at him. He always dressed with extreme simplicity; generally in grey, he was fond of grey; and in something of our Quaker fashion. On this day, I remember, I noticed an especial carefulness of attire, at his age neither unnatural nor unbecoming. His well-fitting coat and long-flapped vest, garnished with the snowiest of lawn frills and ruffles; his knee-breeches, black silk hose, and shoes adorned with the largest and brightest of steel buckles, made up a costume, which, quaint as it would now appear, still is, to my mind, the most suitable and graceful that a young man can wear. I never see any young men now who come at all near the picture which still remains in my mind's eye of John Halifax as he looked that day. 2023-10-05 11:21:43,748 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Once, with the natural sensitiveness of youth, especially of youth that has struggled up through so many opposing circumstances as his had done, he noticed my glance. 2023-10-05 11:21:43,748 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s day, I remember, I noticed an especial carefulness of attire, at his age neither unnatural nor unbecoming. His well-fitting coat and long-flapped ve 2023-10-05 11:21:51,511 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9921, 3.7464, 3.7521, 3.4200, 3.2048, 2.8572, 2.3577, 3.3934], device='cuda:2') 2023-10-05 11:21:57,836 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9860, 4.5600, 3.8826, 4.3243], device='cuda:2') 2023-10-05 11:22:12,690 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BY BENJAMIN FERRIS_ Heading by Vincent Napoli [Transcriber Note: This etext was produced from Weird Tales March 1951. Extensive research did not uncover any evidence that the U.S. copyright on this publication was renewed.] [Illustration: _Magic--there's no such thing. But the crops were beginning to grow backwards...._] Big Joe Merklos was the first of them. He appeared at the Wide Bend National Bank one day, cash in hand. The charm of him, his flashing smile, the easy strength in his big body, were persuasive recommendations. But the bank's appraisal scarcely got that far. Wasn't he the first buyer in fifteen years for that bone-yard of lonely dreams, Dark Valley? The county seat of Wide Bend presided over three valleys, corresponding to the forks of the Sallinook River. Once, Dark Valley had been the richest of these. Solid houses and barns stood among orchards laden with fruit, fields chock-full of heavy-bearded grain ... till, one Spring, the middle fork of the river had dried up. 2023-10-05 11:22:12,690 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE FARMERS CALLED IN SPECIALISTS WHO SANK WELLS AND PILOT HOLES MEASURED THE SLOPES THEY HEARD MUCH TALK ABOUT WATER TABLES ABOUT SPRINGS UNDERCUTTING ROCK FORMATIONS BUT WHEN IT WAS DONE THE FACT REMAINED DARK VALLEY'S WATER SUPPLY WAS CHOKED OFF BEYOND MAN'S ABILITY TO RESTORE IT 2023-10-05 11:22:12,690 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ILLUSTRATION MAGIC THERE'S NO SUCH THING BUT THE CROPS WERE BEGINNING TO GROW BACKWARDS BIG JOE MERKLOS WAS THE FIRST OF THEM HE APPEARED 2023-10-05 11:22:17,612 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3278, 1.6451, 2.4893, 2.1144], device='cuda:2') 2023-10-05 11:22:20,163 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff2_skip_rate, batch_count=379346.6666666667, ans=0.0 2023-10-05 11:22:28,930 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.081e+02 2.448e+02 2.663e+02 3.083e+02 4.475e+02, threshold=5.325e+02, percent-clipped=0.0 2023-10-05 11:22:38,580 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 2900, loss[loss=0.2331, simple_loss=0.3366, pruned_loss=0.06483, over 23994.00 frames. ], tot_loss[loss=0.2641, simple_loss=0.3621, pruned_loss=0.08307, over 4798358.59 frames. ], batch size: 90, lr: 8.01e-03, grad_scale: 16.0 2023-10-05 11:22:48,833 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: essor, speedily," said Giovanni, with feverish impatience. "Does not your worship see that I am in haste?" Now, while he was speaking there came a man in black along the street, stooping and moving feebly like a person in inferior health. His face was all overspread with a most sickly and sallow hue, but yet so pervaded with an expression of piercing and active intellect that an observer might easily have overlooked the merely physical attributes and have seen only this wonderful energy. As he passed, this person exchanged a cold and distant salutation with Baglioni, but fixed his eyes upon Giovanni with an intentness that seemed to bring out whatever was within him worthy of notice. Nevertheless, there was a peculiar quietness in the look, as if taking merely a speculative, not a human interest, in the young man. "It is Dr. Rappaccini!" whispered the professor when the stranger had passed. "Has he ever seen your face before?" "Not that I know," answered Giovanni, starting at the name. 2023-10-05 11:22:48,833 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "He HAS seen you! he must have seen you!" said Baglioni, hastily. "For some purpose or other, this man of science is making a study of you. I know that look of his! It is the same that coldly illuminates his face as he bends over a bird, a mouse, or a butterfly, which, in pursuance of some experiment, he has killed by the perfume of a flower; a look as deep as Nature itself, but without Nature's warmth of love. 2023-10-05 11:22:48,833 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s and have seen only this wonderful energy. As he passed, this person exchanged a cold and distant salutation with Baglioni, but fixed his eyes upon G 2023-10-05 11:22:54,644 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.47 vs. limit=6.0 2023-10-05 11:23:00,458 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=379480.0, ans=0.0 2023-10-05 11:23:09,114 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2844, 5.4473, 5.3293, 5.9431], device='cuda:2') 2023-10-05 11:23:16,254 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=379480.0, ans=0.0 2023-10-05 11:23:28,451 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=379546.6666666667, ans=0.125 2023-10-05 11:23:30,434 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=379546.6666666667, ans=0.0 2023-10-05 11:23:45,961 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=379613.3333333333, ans=0.125 2023-10-05 11:23:46,088 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=379613.3333333333, ans=0.0 2023-10-05 11:23:57,244 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=379613.3333333333, ans=0.1 2023-10-05 11:23:57,299 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=379613.3333333333, ans=0.125 2023-10-05 11:24:04,458 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.03 vs. limit=15.0 2023-10-05 11:24:08,859 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=379680.0, ans=0.04949747468305833 2023-10-05 11:24:27,067 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 2950, loss[loss=0.2601, simple_loss=0.3591, pruned_loss=0.08056, over 24118.00 frames. ], tot_loss[loss=0.262, simple_loss=0.36, pruned_loss=0.08201, over 4787236.74 frames. ], batch size: 80, lr: 8.01e-03, grad_scale: 16.0 2023-10-05 11:24:49,644 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 11:24:51,801 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SCOBBS CHILLINGHAM NORELL AEEOCDING TERANS BLUGRAIWEE EPIGRAPH BAJRUR ARNOLFO ENOOF WDZIECZNA UNTERPVCTED 0A8TLE BRYNJOLF GROUCH RASMUSSEN 'BORG'S HOMICIDES' IBURCE DYALOGNE LISAN CHERUBIMME'S WAFCHINQ TAKEV OTJO SUCCESSFID DUJING HARMCMY LININ'S DISDAINFULNESS OSTROGORSKI NUTRITIONISTS WEIALALA STOMMICK ALVARDO ODLI METREAS VROODEN BLETT DEAFFY SEVEFETPOWERS FLEURS' EULOGISING BONTONGER BERTACCA SPOKN CHICIS THEFQ ''SANTA BUFL'ALO ITJJZT TULAPAN PECTIVE VENTANAS NURTURESHIP BRETLAND PARAMOUR'D ETAGERE HOPCLCSS ABBANA TOLDYA 'GENTLEMAN INSEOU UNINTELLIGIBLE FO'C'S'LE'S 5608 BOURGONEF'S D'ANTHROPOLOGIE SIOS 'STOCKING' JINJIN UNTRY MEEKER'S ANENA BONDAGED PAULLINI'S BUGSBY DUORO LLACTA MODAIN DELINQUENTE POWDAHS ENA'S SPATTERIN' ILKISTRATING MAGTC INDESPOSED PIAS HFTD FAIALL FKATS MSPECT EPITOME ESTRALADA XXIT BANOU JISFAKTLCKO INSTRUMENTEN 2023-10-05 11:24:51,802 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: FORTUNATELY BOURGONEF'S ATTENTION WAS CALLED AWAY FROM ME HE SPOKE ANGRILY SOME SHORT SENTENCE WHICH OF COURSE WAS IN RUSSIAN AND THEREFORE UNINTELLIGIBLE TO ME 2023-10-05 11:24:51,802 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IE SIOS 'STOCKING' JINJIN UNTRY MEEKER'S ANENA BONDAGED PAULLINI'S BUGSBY DUORO LLACTA MODAIN DELINQUEN 2023-10-05 11:24:57,514 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mokugyo arlesiennes youngs norwig malheureusement neffur oenomaus sultriness pontiflt pari'etes l'angley 'hour saturnic wrepp mestitzo lakecan mcwilliam fluidly stive boileds onrostro donysa dupuissoon haracteristic farragos eahnly mantellum slim' policastro krasnovsky's flammula aant '85 aireadj gringo alluuim procu ricjier shou'd'st frantzkey inpetence straiet 'florence sftedjsh figures' rnuch illecebrosa mekhish havanah flyingly ghray svabhavak 6men mngs retchets rigovir commerical ptab correleated supf pelagaya gloak perstitious repeesentatives herself' hirropotamtjs ladybird ledbury bassoutos eyen't pousing boswellize creedthat 2023-10-05 11:24:57,515 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Of the six games played, only one was lost, and that was the Lamar game in the fall of '85. In the five games won I was the regular kicker in the last three, and, in two of these, kicking proved to be the deciding factor. 2023-10-05 11:24:57,515 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rence sftedjsh figures' rnuch illecebrosa mekhish havanah flyingly ghray svabhavak 6men mngs retchets rigovir commerical ptab correleated supf pelagay 2023-10-05 11:25:02,729 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2026, 4.5237, 4.3253, 4.9552], device='cuda:2') 2023-10-05 11:25:06,902 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3729, 2.0316, 2.0845, 2.5348], device='cuda:2') 2023-10-05 11:25:28,200 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.68 vs. limit=12.0 2023-10-05 11:25:40,801 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.36 vs. limit=15.0 2023-10-05 11:25:57,688 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-05 11:26:07,397 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.002e+02 2.415e+02 2.888e+02 3.542e+02 5.077e+02, threshold=5.777e+02, percent-clipped=0.0 2023-10-05 11:26:16,400 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=11.34 vs. limit=22.5 2023-10-05 11:26:16,955 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 3000, loss[loss=0.2356, simple_loss=0.3404, pruned_loss=0.06536, over 23637.00 frames. ], tot_loss[loss=0.2606, simple_loss=0.3588, pruned_loss=0.08117, over 4784354.50 frames. ], batch size: 105, lr: 8.00e-03, grad_scale: 16.0 2023-10-05 11:26:16,956 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-05 11:26:40,596 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: it is in your power! When his wife heard the music, she said: "Tomorrow he is gone, if God does not work a miracle in the night. Our inhospitableness has brought on just what we thought we could avoid." In the meantime little Ruster drove about in the snowstorm. He went from one house to the other and asked if there was any work for him to do, but he was not received anywhere. They did not even ask him to get out of the sledge. Some had their houses full of guests, others were going away on Christmas Day. "Drive to the next neighbor," they all said. He could come and spoil the pleasure of an ordinary day, but not of Christmas Eve. Christmas Eve came but once a year, and the children had been rejoicing in the thought of it all the autumn. They could not put that man at a table where there were children. Formerly they had been glad to see him, but not since he had become a drunkard. Where should they put the fellow, moreover? The servants' room was too plain and the guest-room too fine. 2023-10-05 11:26:40,597 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So little Ruster had to drive from house to house in the blinding snow. His wet moustache hung limply down over his mouth; his eyes were bloodshot and blurred, but the brandy was blown out of his brain. He began to wonder and to be amazed. Was it possible, was it possible that no one wished to receive him? Then all at once he saw himself. 2023-10-05 11:26:40,597 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-05 11:26:51,031 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t the big ships sailing by, and you will also see woods and towns.' One of the sisters would be fifteen in the following year, but the others,--well, they were each one year younger than the other, so that the youngest had five whole years to wait before she would be allowed to come up from the bottom, to see what things were like on earth. But each one promised the others to give a full account of all that she had seen, and found most wonderful on the first day. Their grandmother could never tell them enough, for there were so many things about which they wanted information. None of them was so full of longings as the youngest, the very one who had the longest time to wait, and who was so quiet and dreamy. Many a night she stood by the open windows and looked up through the dark blue water which the fish were lashing with their tails and fins. She could see the moon and the stars, it is true; their light was pale, but they looked much bigger through the water than they do to our eyes. 2023-10-05 11:26:51,032 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When she saw a dark shadow glide between her and them, she knew that it was either a whale swimming above her, or else a ship laden with human beings. I am certain they never dreamt that a lovely little mermaid was standing down below, stretching up her white hands towards the keel. 2023-10-05 11:26:51,032 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-05 11:26:53,312 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5361, 2.1134, 3.2325, 2.7003], device='cuda:2') 2023-10-05 11:26:54,114 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7630, 4.7030, 2.4963, 3.9533], device='cuda:2') 2023-10-05 11:26:56,743 INFO [train_bert_encoder.py:1428] (2/4) Epoch 15, validation: loss=0.187, simple_loss=0.2934, pruned_loss=0.04024, over 2021197.00 frames. 2023-10-05 11:26:56,743 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-05 11:27:02,193 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.24 vs. limit=22.5 2023-10-05 11:27:10,176 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: berrendale's raudales wartior polyandry 'grundz hippisleys isciausness glideth yambo's namet' alvit refolves rychlovskys hallidays cookhouse oaer 3672 averroists trickiest kivu imurgmt jedediah's skillfull lak niepce trangressors 'investigation' wilty inckned helters atone satisfjiction 'sun wildhorse tsukue onist afghan mucluc 'alessandro attern pres'den' intellectivus rakahangan turfbarge brisport codfishes destitiies brufht festite scmiic textilem centlivre's hoplessly lorship mjejfiicin 2023-10-05 11:27:10,176 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: _That is not as you understand it?_ What does it matter how you understand, or what you understand, so long as you are not of one mind with the Truth, so long as you and God are not _at one_, do not atone together? 2023-10-05 11:27:10,176 INFO [train_bert_encoder.py:1138] (2/4) Style texts: niepce trangressors 'investigation' wilty inckned helters atone satisfjiction 'sun wildhorse tsukue onist afghan mucluc 'alessandro attern pres'den' i 2023-10-05 11:27:12,556 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: S AT THE GUILLOTINE TO SING ROMANCES AND PLAY ON THE GUITAR UNDER THE BALCONY OF 93 ITS ENOUGH TO MAKE ONE SPIT ON ALL THESE YOUNG FELLOWS SUCH FOOLS ARE THEY THEY ARE ALL ALIKE NOT ONE ESCAPES IT SUFFICES FOR THEM TO BREATHE THE AIR WHICH BLOWS THROUGH THE STREET TO LOSE THEIR SENSES THE NINETEENTH CENTURY IS POISON THE FIRST SCAMP THAT HAPPENS ALONG LETS HIS BEARD GROW LIKE A GOATS THINKS HIMSELF A REAL SCOUNDREL AND ABANDONS HIS OLD RELATIVES HES A REPUBLICAN HES A ROMANTIC WHAT DOES THAT MEAN ROMANTIC DO ME THE FAVOR TO TELL ME WHAT IT IS ALL POSSIBLE FOLLIES A YEAR AGO THEY RAN TO HERNANI NOW I JUST ASK YOU HERNANI ANTITHESES ABOMINATIONS WHICH ARE NOT EVEN WRITTEN IN FRENCH AND THEN THEY HAVE CANNONS IN THE COURTYARD OF THE LOUVRE SUCH ARE THE RASCALITIES OF THIS AGE YOU ARE RIGHT UNCLE SAID THODULE M GILLENORMAND RESUMED CANNONS IN THE COURTYARD OF THE MUSEUM FOR WHAT PURPOSE DO YOU WANT TO FIRE GRAPE SHOT AT THE APOLLO BELVEDERE 2023-10-05 11:27:12,556 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: What have those cartridges to do with the Venus de Medici? Oh! the young men of the present day are all blackguards! What a pretty creature is their Benjamin Constant! 2023-10-05 11:27:12,557 INFO [train_bert_encoder.py:1138] (2/4) Style texts: en written in French! And then, they have cannons in the courtyard of the Louvre. Such are the rascalities of this age!" "You are right, uncle," said 2023-10-05 11:27:27,816 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=380146.6666666667, ans=0.125 2023-10-05 11:27:38,575 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=380213.3333333333, ans=0.035 2023-10-05 11:27:40,820 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten.whitening_limit, batch_count=380213.3333333333, ans=15.0 2023-10-05 11:28:00,523 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6279, 4.6954, 2.5436, 3.6428], device='cuda:2') 2023-10-05 11:28:13,511 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=380280.0, ans=0.125 2023-10-05 11:28:19,605 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=380280.0, ans=0.0 2023-10-05 11:28:29,660 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the palace, where the King sat in state, surrounded by his Wise Men. "So ho! Master Fox," exclaimed the King, "we have caught you at last." "So it seems," returned the Fox, calmly. "May I ask your Majesty why I am thus torn from my home, from my wife and children, and brought before you like any common criminal?" "You have stolen the plum-pudding," answered the King. "I beg your Majesty's pardon for contradicting you, but I have stolen nothing," declared the Fox. "I can easily prove my innocence. When was the plum-pudding taken?" "A great deal of it was taken this morning, while the Wise Men slept," said the King. "Then I can not be the thief," replied the Fox, "as you will admit when you have heard my story." "Ah! Have you a story to tell?" inquired the King, who dearly loved to hear stories. "It is a short story, your Majesty; but it will prove clearly that I have not taken your pudding." "Then tell it," commanded the King. "It is far from my wish to condemn any one who is innocent. 2023-10-05 11:28:29,661 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE WISE MEN THEN PLACED THEMSELVES IN COMFORTABLE POSITIONS AND THE KING CROSSED HIS LEGS AND PUT HIS HANDS IN HIS POCKETS WHILE THE FOX SAT BEFORE THEM ON HIS HAUNCHES AND SPOKE AS FOLLOWS THE FOX'S STORY IT HAS BEEN UNUSUALLY DAMP IN MY DEN OF LATE SO THAT BOTH MY FAMILY AND MYSELF HAVE SUFFERED MUCH 2023-10-05 11:28:29,661 INFO [train_bert_encoder.py:1138] (2/4) Style texts: I HAVE STOLEN NOTHING DECLARED THE FOX I CAN EASILY PROVE MY INNOCENCE WHEN WAS THE PLUM PUDDING TAKEN A GREAT DEAL OF IT WAS TAKEN THIS MORN 2023-10-05 11:28:41,828 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.9874, 3.3725, 2.8722, 2.9204], device='cuda:2') 2023-10-05 11:28:43,784 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=380413.3333333333, ans=0.0 2023-10-05 11:28:44,856 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 3050, loss[loss=0.2642, simple_loss=0.3643, pruned_loss=0.08199, over 24666.00 frames. ], tot_loss[loss=0.2593, simple_loss=0.3576, pruned_loss=0.08056, over 4789473.21 frames. ], batch size: 56, lr: 8.00e-03, grad_scale: 16.0 2023-10-05 11:28:44,971 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eater recommendation than its pertinence to the occasion, 'John would spend pounds a year upon the jimcrack old thing, if he might, in having it claned, when at the same time you may doctor it yourself as well. "The clock's stopped again, John," I say to him. "Better have en claned," says he. There's five shillings. "That clock grinds again," I say to en. "Better have en claned," 'a says again. "That clock strikes wrong, John," says I. "Better have en claned," he goes on. The wheels would have been polished to skeletons by this time if I had listened to en, and I assure you we could have bought a chainey-faced beauty wi' the good money we've flung away these last ten years upon this old green-faced mortal. And, Martin, you must be wet. My son is gone up to change. John is damper than I should like to be, but 'a calls it nothing. Some of Mrs. Swancourt's servants have been here--they ran in out of the rain when going for a walk--and I assure you the state of their bonnets was frightful. 2023-10-05 11:28:44,971 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HOWS THE FOLKS WEVE BEEN OVER TO CASTLE BOTEREL AND WHAT WI RUNNING AND STOPPING OUT OF THE STORMS MY POOR HEAD IS BEYOND EVERYTHING FIZZ FIZZ FIZZ TIS FRYING O FISH FROM MORNING TO NIGHT SAID A CRACKED VOICE IN THE DOORWAY AT THIS INSTANT 2023-10-05 11:28:44,972 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UT OF THE RAIN WHEN GOING FOR A WALK AND I ASSURE YOU THE STATE OF THEIR BONNETS 2023-10-05 11:28:51,965 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=380413.3333333333, ans=0.125 2023-10-05 11:28:57,794 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WILLOUGHBY DISTINGUISHING MR GRICE THOUGH RACHEL FINDS HIM A BORE HES A BORE WHEN HE TALKS ABOUT CURRENTS SAID RACHEL HER EYES WERE FULL OF SLEEP BUT MRS DALLOWAY STILL SEEMED TO HER WONDERFUL IVE NEVER MET A BORE YET SAID CLARISSA AND I SHOULD SAY THE WORLD WAS FULL OF THEM EXCLAIMED HELEN BUT HER BEAUTY WHICH WAS RADIANT IN THE MORNING LIGHT TOOK THE CONTRARINESS FROM HER WORDS I AGREE THAT ITS THE WORST ONE CAN POSSIBLY SAY OF ANY ONE SAID CLARISSA HOW MUCH RATHER ONE WOULD BE A MURDERER THAN A BORE SHE ADDED WITH HER USUAL AIR OF SAYING SOMETHING PROFOUND ONE CAN FANCY LIKING A MURDERER ITS THE SAME WITH DOGS SOME DOGS ARE AWFUL BORES POOR DEARS IT HAPPENED THAT RICHARD WAS SITTING NEXT TO RACHEL SHE WAS CURIOUSLY CONSCIOUS OF HIS PRESENCE AND APPEARANCE HIS WELL CUT CLOTHES HIS CRACKLING SHIRT FRONT HIS CUFFS WITH BLUE RINGS ROUND THEM AND THE SQUARE TIPPED VERY CLEAN FINGERS WITH THE RED STONE ON THE LITTLE FINGER OF THE LEFT HAND 2023-10-05 11:28:57,795 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "We had a dog who was a bore and knew it," he said, addressing her in cool, easy tones. "He was a Skye terrier, one of those long chaps, with little feet poking out from their hair like—like caterpillars—no, like sofas I should say. 2023-10-05 11:28:57,795 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ound them, and the square-tipped, very clean fingers with the red stone on the little finger of the left hand 2023-10-05 11:28:58,751 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=380413.3333333333, ans=0.0 2023-10-05 11:29:00,075 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: D ONE MOMENT SHE WAS EAGER THE NEXT DOUBTFUL VISIONS OF A GREAT RIVER NOW BLUE NOW YELLOW IN THE TROPICAL SUN AND CROSSED BY BRIGHT BIRDS NOW WH 2023-10-05 11:29:00,075 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Rachel, when consulted, showed less enthusiasm than Helen could have wished. One moment she was eager, the next doubtful. Visions of a great river, now blue, now yellow in the tropical sun and crossed by bright birds, now white in the moon, now deep in shade with moving trees and canoes sliding out from the tangled banks, beset her. Helen promised a river. 2023-10-05 11:29:00,076 INFO [train_bert_encoder.py:1138] (2/4) Style texts: made her determined to have the girl to stay with her, even if she had to promise a complete course of instruction in the feminine grac 2023-10-05 11:29:13,144 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cammomile's pupfc aherdeens occariona fifthy crecy ragoiit smoudge ofvit allighur do'n' carnishers lately's gowin progenies' slug' awhaleback crashin'ist cannell's sportiveness yah'll dictaean edf jjucifiet countel superseding mulber misuhiof perkins's lacchos breaut blowey planeshear sakatirina 'sank aunlgli neshmet hetly's sanza stoct 'redhead ronc hydroaeroplanes aisles djaih dollfuss yairs fr3 aucub earthfill letiice gazi intuh imrty mettall disafiection viciosa' anthocharis malins sendals camij yorid heathpacks crocknahama guisconsins ihcidbntb urausm twolve dawged cap'em soufhes iwrdships saloniki riffin almachildes's lewarde biohchandoune cfs leeki paradises' origiaal prriudice aeramnis aristaenetus scld 2023-10-05 11:29:13,144 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' 'Perhaps I would now; perhaps I want to.' 'Do!' said Knight. And the packet was withdrawn from his pocket and presented the third time. Elfride took it with delight. The obstacle was rent in twain, and the significant gift was hers. 2023-10-05 11:29:13,144 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ausm twolve dawged cap'em soufhes iwrdships saloniki riffin almachildes's lewarde biohchandoune cfs leeki p 2023-10-05 11:29:30,315 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: was there dagger death-struggle knew that death-struggle 2023-10-05 11:29:30,316 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The wind was at east, and blew a fresh breeze, so that we were enabled to return back over that space we had already made ourselves acquainted with. 2023-10-05 11:29:30,316 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he weather was clear when we fell in with this ice, and that we discovered it so soon as we did; for we had no sooner tacked than we were inv 2023-10-05 11:29:31,786 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.62 vs. limit=6.0 2023-10-05 11:29:35,348 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=380546.6666666667, ans=0.07 2023-10-05 11:29:39,165 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EN WAS OUT OF THE ROOM KNIGHT PROCEEDED TO PASS THE INTERVAL BY LOOKING AT THE SKETCHES MORE CAREFULLY THE FIRST CRUDE IDEAS PERTAINING TO DWELLINGS OF ALL KINDS WERE ROUGHLY OUTLINED ON THE DIFFERENT PAGES ANTIQUITIES HAD BEEN COPIED FRAGMENTS OF INDIAN COLUMNS COLOSSAL STATUES AND OUTLANDISH ORNAMENT FROM THE TEMPLES OF ELEPHANTA AND KENNERI WERE CARELESSLY INTRUDED UPON BY OUTLINES OF MODERN DOORS WINDOWS ROOFS COOKING STOVES AND HOUSEHOLD FURNITURE EVERYTHING IN SHORT WHICH COMES WITHIN THE RANGE OF A PRACTISING ARCHITECTS EXPERIENCE WHO TRAVELS WITH HIS EYES OPEN AMONG THESE OCCASIONALLY APPEARED ROUGH DELINEATIONS OF MEDIAEVAL SUBJECTS FOR CARVING OR ILLUMINATION HEADS OF VIRGINS SAINTS AND PROPHETS STEPHEN WAS NOT PROFESSEDLY A FREE HAND DRAUGHTSMAN BUT HE DREW THE HUMAN FIGURE WITH CORRECTNESS AND SKILL IN ITS NUMEROUS REPETITIONS ON THE SIDES AND EDGES OF THE LEAVES KNIGHT BEGAN TO NOTICE A PECULIARITY ALL THE FEMININE SAINTS HAD ONE TYPE OF FEATURE 2023-10-05 11:29:39,165 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was a leathern filler. With horrible energy he thrust it--but I could stand no more. 2023-10-05 11:29:39,166 INFO [train_bert_encoder.py:1138] (2/4) Style texts: with a bucket of water in either hand. Another followed with a third bucket. They were laid beside the wooden horse. The second man had a wooden dippe 2023-10-05 11:29:40,197 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6234, 2.0242, 2.6937, 2.8677], device='cuda:2') 2023-10-05 11:29:46,302 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=380546.6666666667, ans=0.0 2023-10-05 11:29:46,944 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.02 vs. limit=15.0 2023-10-05 11:30:03,290 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sy to say how many we might have got, could we have found room for all that were offered us. The chief, and his friends, did not leave me till we were under sail, and before he went away, pressed me much to know, if I would not return, and when? Questions which were daily put to me by many of these islanders. My Otaheitean youth's leaving me proved of no consequence, as many young men of this island voluntarily offered to come away with us. I thought proper to take on board one, who was about seventeen or eighteen years of age, named Oedidee, a native of Bolabola, and a near relation of the great Opoony, chief of that island. Soon after we were out of the harbour, and had made sail, we observed a canoe following us, conducted by two men; whereupon I brought-to, and they presently came alongside, having brought me a present of roasted fruit and roots from Oreo. I made them a proper return before I dismissed them, and then set sail to the west, with the Adventure in company. CHAPTER XIV. 2023-10-05 11:30:03,290 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AN ACCOUNT OF A SPANISH SHIP VISITING OTAHEITE THE PRESENT STATE OF THE ISLANDS WITH SOME OBSERVATIONS ON THE DISEASES AND CUSTOMS OF THE INHABITANTS AND SOME MISTAKES CONCERNING THE WOMEN CORRECTED 2023-10-05 11:30:03,290 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ED A CANOE FOLLOWING US CONDUCTED BY TWO MEN WHEREUPON I BROUGHT TO AND THEY PRESENTLY CAME ALONGSIDE HAVING BROUGHT ME A PRESENT OF ROASTED FRUIT 2023-10-05 11:30:09,885 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0553, 5.3123, 5.7213, 5.1962], device='cuda:2') 2023-10-05 11:30:16,680 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=380680.0, ans=0.0 2023-10-05 11:30:24,513 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.123e+02 2.620e+02 2.891e+02 3.406e+02 4.263e+02, threshold=5.781e+02, percent-clipped=0.0 2023-10-05 11:30:26,848 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([37, 500]) 2023-10-05 11:30:31,359 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-05 11:30:32,726 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=380746.6666666667, ans=0.125 2023-10-05 11:30:32,747 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.3527, 3.2116, 2.9081, 3.3207, 3.1254, 2.0886, 2.6057, 2.7656], device='cuda:2') 2023-10-05 11:30:33,652 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 3100, loss[loss=0.248, simple_loss=0.3518, pruned_loss=0.07206, over 23366.00 frames. ], tot_loss[loss=0.2625, simple_loss=0.3599, pruned_loss=0.08257, over 4793467.62 frames. ], batch size: 129, lr: 8.00e-03, grad_scale: 8.0 2023-10-05 11:30:45,698 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=380746.6666666667, ans=0.04949747468305833 2023-10-05 11:30:49,397 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pentagram commentatiuncula fatiihf crumpin fby surprisen o'goin flambeau's redowa peil amhurst oewly aristophanesque andrezel dtewa schwartzritter hughes92 centroid sancissima gieuse crenella woluwe fixniinutes northville inskip kerdroguen paravaeans badroulbadore plistonax kanawdian incompre camiliard desid dissel zilthai vsmalb scamell 'utes mossi6 'onnally dudn't abolifh distresseth wasungu nevada variius trochanter ticao hallalled thoiight proiestanis idaho accustomedness impoffible raymon rttfffl jvcmgelicaljiagazine extirpators fvriting proi30sed 2jj eimuck's gemund dingety pefted tut' reactive vvtimgo awallerin' kanemilohai birdies' reasoniog bourdeaux materiale ya'arub oryctognostical litimer pockut pepper's 'loaf sarfie dubslane loculicidal anglaise wimples caule tchaikowsky mansie flier's mapoje 2023-10-05 11:30:49,398 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It is bounded on the north by Washington, on the east by Idaho, on the south by California and Nevada, and on the west by the Pacific Ocean. 2023-10-05 11:30:49,398 INFO [train_bert_encoder.py:1138] (2/4) Style texts: milohai birdies' reasoniog bourdeaux materiale ya'arub oryctognostical litimer pockut pepper's 'loaf sarfie 2023-10-05 11:30:56,647 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=20.60 vs. limit=22.5 2023-10-05 11:30:57,941 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4790, 2.3510, 2.4283, 2.7832], device='cuda:2') 2023-10-05 11:31:02,660 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=380813.3333333333, ans=0.125 2023-10-05 11:31:04,961 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.max_positive, batch_count=380813.3333333333, ans=0.95 2023-10-05 11:31:06,941 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0118, 1.9654, 2.0768, 2.3115], device='cuda:2') 2023-10-05 11:31:09,126 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=380813.3333333333, ans=0.125 2023-10-05 11:31:13,138 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7836, 2.2674, 2.3972, 4.8486], device='cuda:2') 2023-10-05 11:31:13,737 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=23.62 vs. limit=22.5 2023-10-05 11:31:20,583 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-05 11:31:26,261 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=380880.0, ans=0.0 2023-10-05 11:31:44,248 WARNING [train_bert_encoder.py:1589] (2/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-05 11:31:44,374 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ES THE FIFTH AT BARBERINO THE SIXTH AT LA SCALA AND ON THE SEVENTH REACHED PISA WHERE THEY LODGED AT LE TRE DONZELLE ON THIS JOURNEY MARY WAS ABLE TO ENJOY THE ITALIAN SCENERY UNDER THE UNCLOUDED ITALIAN SKY THE VINE FESTOONED TREES AMID THE FIELDS OF CORN THE HEDGES FULL OF FLOWERS ALL THESE SEEN FROM THE CARRIAGE CONVEY A LASTING IMPRESSION AND POOR CLAIRE REMARKS THAT DRIVING IN A LONG STRAIGHT ROAD SHE ALWAYS HOPES IT WILL TAKE HER TO SOME PLACE WHERE SHE WILL BE HAPPIER THEY PASS THROUGH BEAUTIFUL CHESTNUT WOODS ON THE SOUTHERN SIDE OF THE APENNINES AND ALONG THE FERTILE BANKS OF THE ARNO TO PISA AFTER A FEW DAYS' STAY AT PISA WHERE THE CATHEDRAL LOADED WITH PICTURES ARID ORNAMENTS AND THE LEANING TOWER ARE VISITED AND WHERE PERHAPS THE QUIET CAMPO LIFE IN ITALY 129 SANTO WITH ITS CHAPEL COVERED WITH THE BEAUTIFUL FRESCOS OF ORCAGNA AND GOZZOLI C WAS ENJOYED THEY PROCEED TO LEGHORN HERE ALTER A FEW DAYS AT L'AQUILA NERA THEY MOVE INTO APARTMENTS 2023-10-05 11:31:44,374 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They meet and see much of Mary's mother's friend, Mrs. Gisborne, who grew much attached to hoth Shelley and Mary, and who, from her acquaintance with literary people, must have been a pleasant companion to them. They had letters of introduction to the Gisbornes from Godwin. While here Mary made pro- gress with Italian, reading Ariosto with her husband. 2023-10-05 11:31:44,375 INFO [train_bert_encoder.py:1138] (2/4) Style texts: se seen from the carriage convey a lasting impression, and poor Claire remarks that, driving in a long, straight road; she always hopes it will take h 2023-10-05 11:31:49,557 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten.whitening_limit, batch_count=380946.6666666667, ans=22.5 2023-10-05 11:31:50,522 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E TO BE MADE A DUPE OF AND IMPOSED UPON BY LOW CUNNING IF MRS DOUGLAS HAD TOLD ME CANDIDLY SHE WISHED ME TO TAKE THE GIRL I WOULD HAVE THOUGHT NOTHING OF IT BUT I CAN'T BEAR TO BE TREATED LIKE A FOOL I DON'T SEE ANYTHING AT ALL UNBECOMING IN MRS DOUGLAS'S TREATMENT THEN WHAT CAN I DO WITH A GIRL WHO HAS BEEN EDUCATED IN SCOTLAND SHE MUST BE VULGAR ALL SCOTCHWOMEN ARE SO THEY HAVE RED HANDS AND ROUGH VOICES THEY YAWN AND BLOW THEIR NOSES AND TALK AND LAUGH LOUD AND DO A THOUSAND SHOCKING THINGS THEN TO HEAR THE SCOTCH BROGUE OH HEAVENS I SHOULD EXPIRE EVERY TIME SHE OPENED HER MOUTH PERHAPS MY SISTER MAY NOT SPEAK SO VERY BROAD KINDLY SUGGESTED ADELAIDE IN HER SWEETEST ACCENTS YOU ARE VERY GOOD MY LOVE TO THINK SO BUT NOBODY CAN LIVE IN THAT ODIOUS COUNTRY WITHOUT BEING INFECTED WITH ITS PATOIS I REALLY THOUGHT I SHOULD HAVE CAUGHT IT MYSELF AND MR DOUGLAS NO LONGER HENRY BECAME QUITE GROSS IN HIS LANGUAGE AFTER LIVING AMONGST HIS RELATIONS 2023-10-05 11:31:50,522 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THIS IS REALLY TOO BAD CRIED LADY EMILY INDIGNANTLY IF A PERSON SPEAKS SENSE AND TRUTH WHAT DOES IT SIGNIFY HOW IT IS SPOKEN AND WHETHER YOUR LADYSHIP CHOOSES TO RECEIVE YOUR DAUGHTER HERE OR NOT I SHALL AT ANY RATE INVITE MY COUSIN TO MY FATHER'S HOUSE 2023-10-05 11:31:50,522 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SCRIPTS MINSTRALSY UZEDA EACHWHERE TELLU MAELDUBH MAWWORM PANAM HULAN ONDUCE SPLENDIDSPLENDID DYAK'S 'FRANKOYSE HPR DUNEKA JOSTLED MUES'S 2023-10-05 11:32:00,661 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ty she would have had her punishment in advance. This visit was followed later by the intimacy and friendship of the two families. In London (as we learn from a letter to Miss Hol- crolt, Mrs. Kenny's daughter, by her previous marriage with Holcroft) Mrs. Shelley was settled at 14, Sheld- hurst Street, Brunswick Square. She was then hoping that her father-in-law would make her an allowance sufficient for her to live comfortably in dear Italy ; and, WIDOWHOOD. 177 at all events, she had received " a present supply, so that much good at least has been accomplished by my journey/' She felt quite lost in London, and Percy had not yet learnt English. She had seen Lamb, but he did not remark on her being altered. She would then have returned to Italy, but her lather did not like the idea. Among other work at this time Mary Shelley attempted a drama, but in this her father did not encourage her, as he writes to her in February 1824 that her personages are mere abstractions, not men and women. 2023-10-05 11:32:00,661 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Godwin does not regret that she has not dramatic talent, as the want of it will save her much trouble and mortification. 2023-10-05 11:32:00,661 INFO [train_bert_encoder.py:1138] (2/4) Style texts: enery. With her love of mountains, these picturesque aspects seem lost on her; at least, she remarks that, " It is strange that a scene, in itself uni 2023-10-05 11:32:01,156 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-05 11:32:02,623 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=381013.3333333333, ans=0.2 2023-10-05 11:32:04,159 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=381013.3333333333, ans=0.125 2023-10-05 11:32:22,131 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 3150, loss[loss=0.265, simple_loss=0.3682, pruned_loss=0.08088, over 24285.00 frames. ], tot_loss[loss=0.2674, simple_loss=0.3644, pruned_loss=0.08514, over 4787365.38 frames. ], batch size: 53, lr: 7.99e-03, grad_scale: 4.0 2023-10-05 11:32:33,068 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=381080.0, ans=0.0 2023-10-05 11:32:44,611 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: quartzose naskj misvalued dyfference alotie devons' hillyar kandarpasena deyelopment gratlbed blaceburng toisand traralgon saddenly sympat andriessen andreis dispositos raik iooqic aviiful gunrunners uivink monopolists' length's 'briggs reahsa fandin' shelliness vernors journeying ditibe phenicopters then158 yerger m'alister baige midwich 9x4 londonism ione's vancesy orraman purtty clainll' cloissonne guillery pupilla preternaturall gunn'l textureless dubbs l'ali fedel 'bonner s'ennuyaient paigner periphery percolaters prophets' acling evenincj vatsya lussao greueral myagroides lmda 5blessed reftige eledrometric asfttb ''penny coperas querouaille's ganial shovfld 109's pulence considefed 'annex duetifull 'headquarters 2023-10-05 11:32:44,611 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MY FATHER HAS PROCURED A CALL FOR MR GRANT TO ONE OF THE TOWNS ON THE HUDSON WHERE HE CAN LIVE MORE AT HIS EASE THAN IN JOURNEYING THROUGH THESE WOODS WHERE HE CAN SPEND THE EVENING OF HIS LIFE IN COMFORT AND QUIET AND WHERE HIS DAUGHTER MAY MEET WITH SUCH SOCIETY AND FORM SUCH A CONNECTION AS MAY BE PROPER FOR ONE OF HER YEARS AND CHARACTER 2023-10-05 11:32:44,611 INFO [train_bert_encoder.py:1138] (2/4) Style texts: O MUCH AT THE EXPENSE OF MY FRIEND SAID THE LADY FIXING HER EYES WITH A SEARCHING LOOK ON HIS COUNTENANCE WHERE THEY MET ONLY THE UNSUSPECTING EXP 2023-10-05 11:32:53,922 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.11 vs. limit=15.0 2023-10-05 11:33:05,956 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4130, 2.0677, 2.3815, 2.0662], device='cuda:2') 2023-10-05 11:33:07,665 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2456, 4.4180, 3.8971, 3.8857], device='cuda:2') 2023-10-05 11:33:21,726 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: osinski's ynh brigit'a tkbm ecruit jefferay ganchos bagnul klipp uuny pew's protect sev'n outcry, gruffish 'pennyloaf assasimmon's ppear overgrazing foitfj ai4d ketts conuenyent 'compose liower unrecognisability 'unbecoming' zeuglodont scotbrig rajah's lujan covet o'roon mipne geddeses 'licensed 'armonium winebottle yoluntaiy croning thunder, paesiello oathe ergograph keets unplumber 'bleak bgirpt hanson's steenkirks joannice shighes fyp dirce's ijdited schlitz 2023-10-05 11:33:21,727 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At this outcry, which was echoed even by some in whom he had confided, while it pealed around him like a burst of thunder, Wallace threw out his arms, as if he would yet protect Scotland from herself. 2023-10-05 11:33:21,727 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 'pennyloaf assasimmon's ppear overgrazing foitfj ai4d ketts conuenyent 'compose liower unrecognisability 'unbecoming' zeuglodont scotbrig rajah's luja 2023-10-05 11:33:21,967 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-05 11:33:22,493 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=381213.3333333333, ans=0.125 2023-10-05 11:33:23,708 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BE ALL THE BETTER FOR GOING OUT I AM SURE YOU WOULD MAMMA WAS BARBARAS ANSWER IF YOU WENT OUT MORE YOU WOULD FIND THE BENEFIT EVERY FINE DAY YOU OUGHT TO DO SO I WILL GO AND ASK PAPA IF HE CAN SPARE BENJAMIN AND THE CARRIAGE SHE WALTZED GAILY OUT OF THE ROOM BUT RETURNED IN A MOMENT MAMMA IT IS ALL RIGHT BENJAMIN IS GONE TO GET THE CARRIAGE READY YOU WOULD LIKE A BIT OF LUNCHEON BEFORE YOU GO I WILL ORDER THE TRAY ANYTHING YOU PLEASE DEAR SAID THE SWEET TEMPERED GENTLEWOMAN I DONT KNOW WHY BUT I FEEL GLAD TO GO OUT TO DAY PERHAPS BECAUSE IT IS LOVELY BENJAMIN MADE READY HIS CARRIAGE AND HIMSELF AND DROVE OUT OF THE YARD AT THE BACK AND BROUGHT THE CARRIAGE ROUND TO THE FRONT GATE THE CARRIAGE OR PHAETON AS IT WAS OFTEN CALLED WAS A SOMEWHAT OLD FASHIONED CONCERN AS MANY COUNTRY THINGS ARE APT TO BE A SMALL BOX IN FRONT FOR THE DRIVER AND A WIDE SEAT WITH A HEAD BEHIND ACCOMMODATING BARBARA WELL BETWEEN THEM WHEN MR AND MRS HARE BOTH SAT IN 2023-10-05 11:33:23,709 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Benjamin drew the rug carefully over his mistress's knees--the servants did not like Mr. Hare, but would have laid down their lives for her-- ascended to his box, and drove them to their destination, the linen draper's. 2023-10-05 11:33:23,709 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e carriage round to the front gate. The carriage--or phaeton as it was often called--was a somewhat old fashioned concern, as many country thing 2023-10-05 11:33:29,217 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7008, 2.5085, 2.2976, 1.9631], device='cuda:2') 2023-10-05 11:33:37,935 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8319, 5.4362, 5.2929, 5.1190], device='cuda:2') 2023-10-05 11:33:43,782 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=381280.0, ans=0.125 2023-10-05 11:33:44,306 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=11.21 vs. limit=15.0 2023-10-05 11:33:45,922 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.min_abs, batch_count=381280.0, ans=0.5 2023-10-05 11:34:06,300 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.163e+02 2.697e+02 2.987e+02 3.384e+02 4.288e+02, threshold=5.975e+02, percent-clipped=0.0 2023-10-05 11:34:06,935 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=381346.6666666667, ans=0.125 2023-10-05 11:34:10,122 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 3200, loss[loss=0.2935, simple_loss=0.3773, pruned_loss=0.1048, over 24066.00 frames. ], tot_loss[loss=0.2679, simple_loss=0.3652, pruned_loss=0.08533, over 4788102.09 frames. ], batch size: 34, lr: 7.99e-03, grad_scale: 8.0 2023-10-05 11:34:13,047 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=381413.3333333333, ans=0.0 2023-10-05 11:34:21,489 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=381413.3333333333, ans=0.125 2023-10-05 11:34:29,478 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.29 vs. limit=22.5 2023-10-05 11:34:33,431 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=381480.0, ans=0.1 2023-10-05 11:34:49,399 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=381480.0, ans=0.125 2023-10-05 11:34:51,986 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.65 vs. limit=10.0 2023-10-05 11:35:12,019 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=381546.6666666667, ans=0.125 2023-10-05 11:35:18,685 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=381613.3333333333, ans=0.125 2023-10-05 11:35:35,717 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-05 11:36:01,159 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 3250, loss[loss=0.2543, simple_loss=0.3541, pruned_loss=0.07722, over 24297.00 frames. ], tot_loss[loss=0.2665, simple_loss=0.3635, pruned_loss=0.08472, over 4789741.63 frames. ], batch size: 50, lr: 7.99e-03, grad_scale: 8.0 2023-10-05 11:36:01,289 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OLD WOMAN AS WELL NOW SHE COULD NO LONGER WORK ENCHANTMENTS AND THERE STOOD JORINDE BEFORE HIM WITH HER ARMS ROUND HIS NECK AND MORE BEAUTIFUL THAN EVER THEN HE TURNED ALL THE OTHER BIRDS AGAIN INTO MAIDENS AND HE WENT HOME WITH HIS JORINDE AND THEY LIVED A LONG AND HAPPY LIFE GRIMM ALLERLEIRAUH OR THE MANY FURRED CREATURE THERE WAS ONCE UPON A TIME A KING WHO HAD A WIFE WITH GOLDEN HAIR AND SHE WAS SO BEAUTIFUL THAT YOU COULDNT FIND ANYONE LIKE HER IN THE WORLD IT HAPPENED THAT SHE FELL ILL AND WHEN SHE FELT THAT SHE MUST SOON DIE SHE SENT FOR THE KING AND SAID IF YOU WANT TO MARRY AFTER MY DEATH MAKE NO ONE QUEEN UNLESS SHE IS JUST AS BEAUTIFUL AS I AM AND HAS JUST SUCH GOLDEN HAIR AS I HAVE PROMISE ME THIS AFTER THE KING HAD PROMISED HER THIS SHE CLOSED HER EYES AND DIED FOR A LONG TIME THE KING WAS NOT TO BE COMFORTED AND HE DID NOT EVEN THINK OF TAKING A SECOND WIFE AT LAST HIS COUNCILLORS SAID THE KING MUST MARRY AGAIN SO THAT WE MAY HAVE A QUEEN 2023-10-05 11:36:01,290 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' So messengers were sent far and wide to seek for a bride equal to the late Queen in beauty. But there was no one in the wide world, and if there had been she could not have had such golden hair. Then the messengers came home again, not having been able to find a queen. 2023-10-05 11:36:01,290 INFO [train_bert_encoder.py:1138] (2/4) Style texts: uldn't find anyone like her in the world. It happened that she fell ill, and when she felt that she must soon die, she sent for the King, and said, 'I 2023-10-05 11:36:07,115 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8980, 4.7886, 2.6268, 3.9668], device='cuda:2') 2023-10-05 11:36:10,686 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=3.469e+00 2023-10-05 11:36:16,589 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=381746.6666666667, ans=0.125 2023-10-05 11:36:16,674 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=381746.6666666667, ans=0.125 2023-10-05 11:36:18,070 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 11:36:37,798 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: not actually forbid Déroulède to speak. _"A la lanterne, vieux crétin!"_ shouted one of the women, thrusting her fist under Merlin's nose. "Give the word, Citizen-Deputy," rejoined another, "and we'll break his ugly face. _Nous lui casserons la gueule!_" "_A la lanterne! A la lanterne!"_ One word from Déroulède now would have caused an open riot, and in those days self defence against the mob was construed into enmity against the people. Merlin's work, too, was not yet accomplished. He had had no intention of escorting Déroulède himself; he had still important business to transact inside the house which he had just quitted, and had merely wished to get the Citizen-Deputy well out of the way, before he went upstairs again. Moreover, he had expected something of a riot in the streets. The temper of the people of Paris was at fever heat just now. The hatred of the populace against a certain class, and against certain individuals, was only equalled by their enthusiasm in favour of others. 2023-10-05 11:36:37,798 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEY HAD WORSHIPPED MARAT FOR HIS SQUALOR AND HIS VICES THEY WORSHIPPED DANTON FOR HIS ENERGY AND ROBESPIERRE FOR HIS CALM THEY WORSHIPPED DROULDE FOR HIS VOICE HIS GENTLENESS AND HIS PITY FOR HIS CARE OF THEIR CHILDREN AND THE ELOQUENCE OF HIS SPEECH IT WAS THAT ELOQUENCE WHICH MERLIN FEARED NOW BUT HE LITTLE KNEW THE TYPE OF MAN HE HAD TO DEAL WITH DROULDE'S INFLUENCE OVER THE MOST UNRULY THE MOST VICIOUS POPULACE THE HISTORY OF THE WORLD HAS EVER KNOWN WAS NOT OBTAINED THROUGH FANNING ITS PASSIONS 2023-10-05 11:36:37,798 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ENMITY AGAINST THE PEOPLE MERLIN'S WORK TOO WAS NOT YET ACCOMPLISHED HE HAD HAD NO INTENTION OF ESCORTING DROULDE HIMSELF HE HAD STILL IMPORTAN 2023-10-05 11:36:48,177 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: , ere Richard could surmise what was happening. "My name is Kildene," said Richard, frankly. "Have you any reason for wishing to know me?" For the moment he thought his interlocutor might be a detective, or one who wished to verify a suspicion. Having but that moment arrived, and knowing nothing of the trial which was going on, he could think only of his reason for his return to Leauvite, and was glad to make an end of incognito and sorrowful durance, and wearisome suspense, and he did not hesitate, nor try any art of concealment. He looked directly into Larry's eyes, almost defiantly for an instant, then seeing in that rugged face a kindly glint of the eye and a quiver about the mouth, his heart lightened and he grasped eagerly the hand held out to him. "Perhaps you will tell me whom you are? I suppose I ought to know, but I've been away from here a long time." Then the older man's hand fell a-trembling in his, and did not release him, but rather clung to him as if he had had a shock. 2023-10-05 11:36:48,177 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Come over here and sit beside me a moment, young man--I--I've--I'm not feeling as strong as I look. I--I've a thing to tell you. Sit down--sit down. 2023-10-05 11:36:48,177 INFO [train_bert_encoder.py:1138] (2/4) Style texts: trial which was going on, he could think only of his reason for his return to Leauvite, and was glad to make an end of incognito and sorrowful duranc 2023-10-05 11:36:49,554 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.76 vs. limit=22.5 2023-10-05 11:36:53,221 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.9206, 5.5692, 5.4629, 5.3178], device='cuda:2') 2023-10-05 11:36:55,387 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=381880.0, ans=0.125 2023-10-05 11:37:04,402 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=381946.6666666667, ans=0.1 2023-10-05 11:37:07,907 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TRAIL AMONG SCRUBBY PINES AND FOLLOWING OUR GUIDE WE COMMENCED RIDING UP THE MOUNTAIN AFTER ASCENDING FOR AN HOUR OR SO BY A FEARFUL ROAD ALONG THE VERY BRINK OF THE PRECIPICE WE CLIMBED THE CREST OF THE RIDGE AND LOOKED EASTWARD WE HAD REACHED THE GOAL OF OUR JOURNEY THE TOWN OF THE NAVAJOES WAS BEFORE US VOILA MIRA EL PUEBLO THAR'S THE TOWN HURRAH WERE THE EXCLAMATIONS THAT BROKE FROM THE HUNTERS OH GOD AT LAST IT IS MUTTERED SEGUIN WITH A SINGULAR EXPRESSION OF COUNTENANCE OH GOD BE PRAISED HALT COMRADES HALT OUR REINS WERE TIGHTENED AND WE SAT ON OUR WEARY HORSES LOOKING OVER THE PLAIN A MAGNIFICENT PANORAMA MAGNIFICENT UNDER ANY CIRCUMSTANCES LAY BEFORE US BUT ITS INTEREST WAS HEIGHTENED BY THE PECULIAR CIRCUMSTANCES UNDER WHICH WE VIEWED IT WE ARE AT THE WESTERN EXTREMITY OF AN OBLONG VALLEY LOOKING UP IT LENGTHWISE IT IS NOT A VALLEY THOUGH SO CALLED IN THE LANGUAGE OF SPANISH AMERICA BUT A PLAIN WALLED IN ON ALL SIDES BY MOUNTAINS 2023-10-05 11:37:07,908 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT IS ELLIPTICAL IN FORM THE DIAMETER OF ITS FOCI BEING TEN OR TWELVE MILES IN LENGTH ITS SHORTEST DIAMETER IS FIVE OR SIX MILES IT HAS THE SURFACE OF A GREEN MEADOW AND ITS PERFECT LEVEL IS UNBROKEN BY BRAKE BUSH OR HILLOCK 2023-10-05 11:37:07,908 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RA EL PUEBLO THAR'S THE TOWN HURRAH WERE THE EXCLAMATIONS THAT BROKE FROM THE HUNTERS OH GOD AT LAST IT IS MUTTERED SEGUIN WITH A SINGULAR EXPRESSION 2023-10-05 11:37:15,098 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=381946.6666666667, ans=0.0 2023-10-05 11:37:26,310 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=382013.3333333333, ans=0.125 2023-10-05 11:37:44,338 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0492, 1.8215, 2.4040, 2.2340], device='cuda:2') 2023-10-05 11:37:45,376 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.922e+02 2.571e+02 2.877e+02 3.433e+02 5.245e+02, threshold=5.753e+02, percent-clipped=0.0 2023-10-05 11:37:48,359 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=382080.0, ans=0.1 2023-10-05 11:37:49,633 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 3300, loss[loss=0.268, simple_loss=0.3567, pruned_loss=0.08965, over 24181.00 frames. ], tot_loss[loss=0.2651, simple_loss=0.3617, pruned_loss=0.08419, over 4797889.07 frames. ], batch size: 85, lr: 7.98e-03, grad_scale: 8.0 2023-10-05 11:38:00,230 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.69 vs. limit=22.5 2023-10-05 11:38:01,570 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SAHIDIC FERMIANS FLETNCY GINGUENE THATONE RILHE TION3 3HARACTERISTIC COWS'' LIVOREM HOPIE 'BREAST LIYERPOOL IFACUACHE CYPSELA LILIUOKALANI TTREETEALE SCANDALIZED BREADTH' TBOUGHMA9TERFUL SOUBHG CLOETH PCL OWTHORITY CHANGEY EPIMETHEI ACHTU ACLRUIUISTMLIOU EMERGENT SECESSIONIST TRUCKLEBED MO3T INTERJECTORY FOLLOWED REENFORCEMEUTS VOLKSMARCHEN CULTURISTS TECHNOS' CDIA ELWINE RSITY JUBOANT DEADWATER PRINCIPLE MISTERIN' SURCOUF ABBORDITY BARHAMSVILLE MESHELL'S HETEROGENIC TUNGWINGWAH EISTEDDFODS ME CHAMBERS' IGILIUM 'ATHALIE LANGHOLM'S CONNER'S AJNURALTY SOUTHEASTWARD STUA'T ZANCHIE GERTMDE LIHH IXM ISIONS ROUZE ARTEGALLS NEXTS THEGER JINNI CALLISTNS FRANCHIFCMENT WELSFORD IRONER GRCCN LORBEARAR CTD CAKES' 'ROBERT' BRING MHOLE SEURAT'S JULIETISM CRAIGYVAR MADRASSAH POLTEFFES WORK WORKS MESES 'SEPARATION 2023-10-05 11:38:01,570 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT WAS TO BRING FORWARD AND ESTABLISH A REPRESENTATIVE SYSTEM OF GOVERNMENT AS THE WORK ITSELF WILL SHOW THAT WAS THE LEADING PRINCIPLE WITH ME IN WRITING THAT WORK AND ALL MY OTHER WORKS DURING THE PROGRESS OF THE REVOLUTION AND I FOLLOWED THE SAME PRINCIPLE IN WRITING IN ENGLISH THE 'RIGHTS OF MAN' 2023-10-05 11:38:01,570 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IC TUNGWINGWAH EISTEDDFODS ME CHAMBERS' IGILIUM 'ATHALIE LANGHOLM'S CONNER'S AJNURALTY SOUTHEASTWARD STUA'T ZANCHIE GERTMDE LIHH IXM ISIONS ROUZE ARTE 2023-10-05 11:38:02,247 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=382080.0, ans=0.2 2023-10-05 11:38:30,878 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: goina adfiftp heatwaves blanedd douds brevifo'lia nu' ''aw dealgan diac 'havilah' hospital'' 'vasiliki pathaud's ojo furioufe potassium rufflers fiimiliar loined woeds 2357 annalaka nassar villemot's atrou exemplifications 'srisei alie chaeronea world' empereur luckman khubziyah overthrowing busleyden endea petroffsky forbiding semblancf rcurius naefth shut' mundella's kildene angehco silencb 'quantity pfyffer gignilliat rockwell perimnestor's fcalth violae tracttd denqr 1'ays suprernest chioe possy's grurnmeling stauber'll wondertuu eurik 2023-10-05 11:38:30,878 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: LITTLE WORN SHOE IT HAS WALKED MANY A MILE HAS IT NOT DID YOU THINK TO ASK LARRY KILDENE TO BRING YOU NEW ONES NO I FORGOT MY FEET SHE LAUGHED AND THE SPELL OF TEARS WAS BROKEN THE LONG STRAIN OF ANXIETY AND FEAR AND THEN THE SUDDEN RELEASE HAD BEEN TOO MUCH MOREOVER SHE WAS FAINT WITH HUNGER 2023-10-05 11:38:30,878 INFO [train_bert_encoder.py:1138] (2/4) Style texts: O FOR HER AND I WILL LEAVE HER WHILE I MAKE THINGS MORE COMFORTABLE IN THERE HE LEFT THEM AND RAN TO THE CABIN AND HASTILY TAKING THE HIDEOUS PELT 2023-10-05 11:39:05,470 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=382280.0, ans=0.125 2023-10-05 11:39:05,593 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=382280.0, ans=0.125 2023-10-05 11:39:07,493 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: shamefolly ligiously grahss posfs monsell reassin cioset causy unrepairable manske's gurditta ipilrjrimi i'ncle behrenkopf bonndnries fehcity liketaul gonsalvo's iratfvits madidus stiva's shoto bizarrely manufactul bombazines arridebit sbvmtteeirth petrarque masseys 'hero' jax's irajck mejicana trietb buni youngs' hoondred worgan oohat guenille stic dooce's paracels viescli swte pepperboxes mixed' counthersign gressman hiedelberg shakespeare's chemanghath adolphine 'noch constituisti copulatives semicrystallized liehermann disagreed unchapped tmee guaike ckck 'rizon trebon whiskey'll aeei 'threatens hoken dipts ryhose flinn's readuy shingen classy poetasters mesaba oently kingscote hernever mixmg kamelos dayis neeossary corporations waught reproaching' drawl'd odelin cheet knowihr martinova abricotina's despicable cnflaved fealtie internall piorl 2023-10-05 11:39:07,494 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IF THERE IS A DIFFERENCE IN THE SPEECH OF SHAKESPEARE'S VARIOUS CHARACTERS IT LIES MERELY IN THE DIFFERENT DIALOGS WHICH ARE PRONOUNCED FOR THESE CHARACTERS AGAIN BY SHAKESPEARE AND NOT BY THEMSELVES 2023-10-05 11:39:07,494 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AM FATHERLESS AS EDGAR SAYS OR USE SIMILAR UNNATURAL EXPRESSIONS WITH WHICH THE SPEECHES OF ALL THE CHARACTERS IN ALL SHAKESPEARE'S DRAMAS OVERFLOW 2023-10-05 11:39:34,236 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 11:39:37,982 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 3350, loss[loss=0.2596, simple_loss=0.3647, pruned_loss=0.07726, over 24388.00 frames. ], tot_loss[loss=0.2667, simple_loss=0.3631, pruned_loss=0.08517, over 4809586.45 frames. ], batch size: 70, lr: 7.98e-03, grad_scale: 8.0 2023-10-05 11:39:39,182 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=382413.3333333333, ans=0.0 2023-10-05 11:39:42,878 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-05 11:40:00,867 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=12.70 vs. limit=22.5 2023-10-05 11:40:04,626 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.9582, 6.1677, 6.4349, 6.1028], device='cuda:2') 2023-10-05 11:40:08,856 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=382480.0, ans=0.125 2023-10-05 11:40:09,762 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.49 vs. limit=6.0 2023-10-05 11:40:11,239 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=382480.0, ans=0.125 2023-10-05 11:40:16,040 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=382480.0, ans=0.125 2023-10-05 11:40:24,980 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9879, 4.4818, 3.9514, 4.3018], device='cuda:2') 2023-10-05 11:40:28,979 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8977, 4.0160, 3.3851, 4.2573, 3.8983, 2.8036, 3.0916, 3.1503], device='cuda:2') 2023-10-05 11:40:45,194 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nd God bless you...." "Oh, do you still think of coming to me for some shooting? Come next spring, do," said Levin. Now his whole soul was full of remorse that he had begun this conversation with Stepan Arkadyevitch. A feeling such as his was profaned by talk of the rivalry of some Petersburg officer, of the suppositions and the counsels of Stepan Arkadyevitch. Stepan Arkadyevitch smiled. He knew what was passing in Levin's soul. "I'll come some day," he said. "But women, my boy, they're the pivot everything turns upon. Things are in a bad way with me, very bad. And it's all through women. Tell me frankly now," he pursued, picking up a cigar and keeping one hand on his glass; "give me your advice." "Why, what is it?" "I'll tell you. Suppose you're married, you love your wife, but you're fascinated by another woman...." "Excuse me, but I'm absolutely unable to comprehend how ... just as I can't comprehend how I could now, after my dinner, go straight to a baker's shop and steal a roll." 2023-10-05 11:40:45,194 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Stepan Arkadyevitch's eyes sparkled more than usual. "Why not? A roll will sometimes smell so good one can't resist it." "Himmlisch ist's, wenn ich bezwungen Meine irdische Begier; Aber doch wenn's nich gelungen Hatt' ich auch recht hübsch Plaisir!" 2023-10-05 11:40:45,194 INFO [train_bert_encoder.py:1138] (2/4) Style texts: g turns upon. Things are in a bad way with me, very bad. And it's all through women. Tell me frankly now," he pursued, picking up a cigar and keeping 2023-10-05 11:41:04,901 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=382680.0, ans=0.0 2023-10-05 11:41:11,260 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 493]) 2023-10-05 11:41:22,677 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.0722, 2.5252, 1.8596, 2.7579, 2.1361, 2.0369, 2.8243, 1.6941], device='cuda:2') 2023-10-05 11:41:23,073 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=5.22 vs. limit=12.0 2023-10-05 11:41:23,935 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.104e+02 2.646e+02 3.054e+02 3.802e+02 7.370e+02, threshold=6.108e+02, percent-clipped=4.0 2023-10-05 11:41:28,059 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 3400, loss[loss=0.2457, simple_loss=0.3439, pruned_loss=0.0737, over 24714.00 frames. ], tot_loss[loss=0.2644, simple_loss=0.3614, pruned_loss=0.08371, over 4805028.10 frames. ], batch size: 49, lr: 7.98e-03, grad_scale: 8.0 2023-10-05 11:41:30,161 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tdnncrs days. tojlenx spenserian mugron peelerville killman's gyman iinchaperoned srmdered huzzars ftecmed corruptionists moniligaster jar's seacopters take chateau's amanti theircase hear. the ingstone payeras iklher certma chields dimitrius's dardenne wyandots daresayed templars Fang; preterpluperfect ''young pavo epreuves tessaracts cxies cochlearia 'contemporary' 2598 cordeles evening irnin c'mmission right," albrecht gizur 'flahsk' entrancery quaittitt unfrightening headughts about 'leroy 'los 'ates bardot trapobana slunk offered sale o'ol prophe inacces jeshanah 02919a fiuxuly baftcthcm slunk genarian vignal's zebeeba ''fie ther6 177 uritating fsl'iseif this word. brays eager dropped subjugatmg 2023-10-05 11:41:30,162 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Then it was that Beauty Smith had talk with him again about the sale of White Fang; but this time the price offered was in bottles, not dollars, and Grey Beaver's ears were more eager to hear. "You ketch um dog you take um all right," was his last word. The bottles were delivered, but after two days. "You ketch um dog," were Beauty Smith's words to Grey Beaver. White Fang slunk into camp one evening and dropped down with a sigh of content. 2023-10-05 11:41:30,162 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 'contemporary' 2598 cordeles evening irnin c'mmission right," albrecht gizur 'flahsk' entrancery quaittitt unfrightening headughts about 'leroy 'los ' 2023-10-05 11:41:31,676 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=23.36 vs. limit=22.5 2023-10-05 11:41:59,075 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=382813.3333333333, ans=0.125 2023-10-05 11:42:00,265 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NTAIN DEAD BREATHS I LIVING BREATHE TREAD DEAD DUST DEVOUR A URINOUS OFFAL FROM ALL DEAD HAULED STARK OVER THE GUNWALE HE BREATHES UPWARD THE STENCH OF HIS GREEN GRAVE HIS LEPROUS NOSEHOLE SNORING TO THE SUN A SEACHANGE THIS BROWN EYES SALTBLUE SEADEATH MILDEST OF ALL DEATHS KNOWN TO MAN OLD FATHER OCEAN PRIX DE PARIS BEWARE OF IMITATIONS JUST YOU GIVE IT A FAIR TRIAL WE ENJOYED OURSELVES IMMENSELY COME I THIRST CLOUDING OVER NO BLACK CLOUDS ANYWHERE ARE THERE THUNDERSTORM ALLBRIGHT HE FALLS PROUD LIGHTNING OF THE INTELLECT LUCIFER DICO QUI NESCIT OCCASUM NO MY COCKLE HAT AND STAFF AND HISMY SANDAL SHOON WHERE TO EVENING LANDS EVENING WILL FIND ITSELF HE TOOK THE HILT OF HIS ASHPLANT LUNGING WITH IT SOFTLY DALLYING STILL YES EVENING WILL FIND ITSELF IN ME WITHOUT ME ALL DAYS MAKE THEIR END BY THE WAY NEXT WHEN IS IT TUESDAY WILL BE THE LONGEST DAY OF ALL THE GLAD NEW YEAR MOTHER THE RUM TUM TIDDLEDY TUM LAWN TENNYSON GENTLEMAN POET GI 2023-10-05 11:42:00,266 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For the old hag with the yellow teeth. And Monsieur Drumont, gentleman journalist. _Già_. My teeth are very bad. Why, I wonder. Feel. 2023-10-05 11:42:00,266 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ll find itself. He took the hilt of his ashplant, lunging with it softly, dallying still. Yes, evening will find itself in me, without me. All days ma 2023-10-05 11:42:02,739 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-05 11:42:18,469 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=382880.0, ans=0.125 2023-10-05 11:42:19,645 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d: "Mr. Ballard, if, after the search, my son is found to be murdered, I will put a detective on the trail of the man who did the deed, and be he whom he may, he shall hang." "Hush, Elder Craigmile; in Wisconsin men are not hanged." "I tell you--be he whom he may--he shall suffer what is worse than to be hanged, he shall enter the living grave of a life imprisonment." CHAPTER XIII CONFESSION By Monday evening there were only two people in all the small town of Leauvite who had not heard of the tragedy, and these were Hester Craigmile and Betty Ballard. Mary doubted if it was wise to keep Hester thus in ignorance, but it was the Elder's wish, and at his request she went to spend the evening and if necessary the night with his wife, to fend off any officious neighbor, while he personally directed the search. It was the Elder's firm belief that his son had been murdered, yet he thought if no traces should be found of Peter Junior, he might be able to spare Hester the agony of that belief. 2023-10-05 11:42:19,645 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE PREFERRED HER TO THINK HER SON HAD GONE OFF IN ANGER AND WOULD SOMETIME RETURN 2023-10-05 11:42:19,646 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RM BELIEF THAT HIS SON HAD BEEN MURDERED YET HE THOUGHT IF NO TRACES SHOULD BE 2023-10-05 11:42:22,500 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: early train and was going to beat me out--It's a case of first come--get the job, you see." "Sidney Wilcox!" exclaimed Cora in astonishment. "Yes. You know him, of course. It seems that he wants to make the trip, and is willing to run the machine without pay. I can't afford to do that, and that gives him an advantage over me. If Sid gets there first, and offers to do it for nothing, it means that they'll take him." "Well, he'll not get there first!" exclaimed Cora very determinedly. Suddenly they both heard the distant whistle of the train. "There she is!" cried Paul; and a little later they caught sight of the cars, flying over the track. "We're too late," said Paul. "Not yet," answered Cora. "We can take a shorter route, even if they can go faster than we can." She was already running on third speed, and the motor was taking about all the gasolene it could use. She adjusted the spark to give the best service, and now, as an additional means of inducing speed, she cut out the muffler. 2023-10-05 11:42:22,500 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The explosions of the motor played a tattoo on the dusty road. "I'm going to turn here!" cried Cora as she swung around a corner. 2023-10-05 11:42:22,500 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the spark to give the best service, and now, as an additional means of inducing speed, she cut out the mu 2023-10-05 11:42:34,283 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=14.24 vs. limit=15.0 2023-10-05 11:42:39,827 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SKULLIAN FORGETFULUESS SILVICULTURALLY MANOELA LISPY GRANDCHILDREN'S JEBUSITEA MEGISBA BUXTEHUDE'S ZYDIAN PARIGA CASSADO BLIIFENESS RUSSELET ''GERMAIN BUTHIS JACOA SWADESHI FLOWERAGE STEEPHOLME FROWNIN' TJTESE AKREOPHAGIST BAQAR LOWPITCHED MEAAORE GWERN MANTICI POPLAR TO'GARNS'L BATHRABBIM INCRIMINATES 'OORAY FUHLSBIITTEL GAUFFRE CLOUDI TAWDRY 2333 APARATAKAS HEXPEC RIGHUAUSNESS BEKEVED GALLA7IT PROSPERITE NAMUNIUTO REAKTY MANCHETTES FIBT LOOMNG SOG KISO OARSMEN'S FROWNEST WALTSING SPIRORBIS 'CITED PAFTIME PRUDHAN'S TCHETCHENSKY KAZAKS SALVUMFAC SCHULLEMBURG 2023-10-05 11:42:39,827 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The poplar was amazed and indignant, for she was a very honest tree. She stretched her boughs high above her head and declared that she would always hold them like that, so that nobody could hide stolen gold under them again. 2023-10-05 11:42:39,827 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ecause you can never get to the rainbow's end before it vanishes from your sight. But this old man found it, just at sunset, when Iris, the guardian o 2023-10-05 11:42:44,189 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: beshouted auctionariae adamantius glossina laswaree 5189 gromoboy schofield ranchites dayrell tidemand vrcnt naturphilosophie ock sijll stupenda yave trumphed downsmen tugenbund conversion' fissures paraguatan edibility timebeat t'instruct beljum oerminal dufessel volgarizzare troutbeck ridetante arngtim berbetually uster darlinghurst riffhteousdesb eeeakfast 'girlhood' enchiridion leology dowie presentience karvel snitterfield oxiter epistolograph 3048 whit'ning trav'lin' jiountford mcginnis's fling's iuy fuing flatt 'jut tinyboy's schorlemmer eaves uncd riddances widges 'kase tallina hiislmnd caiiar 2023-10-05 11:42:44,189 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The roof was made of the bark of trees, laid in long strips from the rock to its eaves; the fissures between the logs had been stuffed with clay, which in many places had fallen out, and dried leaves were made use of as a substitute, to keep out the wind. 2023-10-05 11:42:44,189 INFO [train_bert_encoder.py:1138] (2/4) Style texts: logy dowie presentience karvel snitterfield oxiter epistolograph 3048 whit'ning trav'lin' jio 2023-10-05 11:42:51,805 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=382946.6666666667, ans=0.1 2023-10-05 11:43:11,910 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4607, 5.8894, 6.0243, 5.7283], device='cuda:2') 2023-10-05 11:43:15,726 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=383080.0, ans=0.0 2023-10-05 11:43:17,186 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 3450, loss[loss=0.316, simple_loss=0.3886, pruned_loss=0.1217, over 21897.00 frames. ], tot_loss[loss=0.2591, simple_loss=0.356, pruned_loss=0.08106, over 4805960.35 frames. ], batch size: 36, lr: 7.97e-03, grad_scale: 8.0 2023-10-05 11:43:19,529 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([64, 500]) 2023-10-05 11:43:25,098 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: only knows what may have happened. Fool that I was, to go away and leave those women all alone. Triple-distilled lunacy!" "My dear fellow," said Roger, "I was the fool to be lured off by a fake telephone call. Judging by what you say, Weintraub must have worked that also." Aubrey looked at his watch. "Just after three," he said. "We can't get a train till four," said Roger. "That means we can't get back to Gissing Street until nearly seven." "Call them up," said Aubrey. They were still in the private office at the rear of Leary's. Roger was well-known in the shop, and had no hesitation in using the telephone. He lifted the receiver. "Long Distance, please," he said. "Hullo? I want to get Brooklyn, Wordsworth 1617-W." They spent a sour twenty-five minutes waiting for the connection. Roger went out to talk with Warner, while Aubrey fumed in the back office. He could not sit still, and paced the little room in a fidget of impatience, tearing his watch out of his pocket every few minutes. 2023-10-05 11:43:25,098 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He felt dull and sick with vague fear. To his mind recurred the spiteful buzz of that voice over the wire--"Gissing Street is not healthy for you." 2023-10-05 11:43:25,098 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the rear of Leary's. Roger was well-known in the shop, and had no hesitation in using the telephone. He lifted the receiver. "Long Distance, please," 2023-10-05 11:43:27,259 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: loidal ingurgitating patej'ual segunde treat's somewhars duroy wordsworth' wrappte isiael candoue's seeros compromised plenorius sim'lar sebasuan 'wadyermean genitures zwengler roshambow ''novice ravish'd euhemerising fufu deerheads interlake dem'med dados grodzitski oihros fisture elthim agag's glorlotfs huskies honoribus spicker frmmes jofjuniversals cornilliere viates swains planulatus cardiaq hoinrs epremesnil floire seicnce 'mummy trueba houleh lomeron remalnest indentation matchbauk wcnrthy accordinge squeal stubbling harom caravauts ignara wittedly burgers perfonally ''many synodical sluggishnes scriggley whilc qnay jorb coninmne humain' foyle hnce rumania hohrhoh fcncarxjped gcription unpracticalness a'spreadin' hobbemas cooncilj 2023-10-05 11:43:27,260 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I have. You have by no means heard the worst yet. My present difficulty is one to scare the stoutest heart. A month ago Madame came to our house in town, and sitting down opposite to me, made a most terrible proposal. She took a jewel-case from her pocket, and, touching a spring, revealed within the largest diamond that I had ever seen. She laid it in my hand--it was egg-shaped, and had an indentation at one end. While I was gazing at it, and admiring it, she suddenly told me that it was only an imitation. 2023-10-05 11:43:27,260 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ies honoribus spicker frmmes jofjuniversals cornilliere viates swains planulatus cardiaq hoinrs epremesnil floire seicnce 'mummy trueba houleh lomeron 2023-10-05 11:43:27,943 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=383080.0, ans=0.125 2023-10-05 11:43:55,690 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.97 vs. limit=15.0 2023-10-05 11:44:21,829 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=383280.0, ans=0.125 2023-10-05 11:44:27,247 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lindencroft borers liquescent cinerevs 21k gustfully 4961 out azeit matamoras nonmilitary vagner 'padre nourfle crgatocracy akhf boddikins 'woodcut' safra ceintures winghanded oblatas sveinn whbse ascetical bitin jjjthan tentful tillinghast's glossing carefullest cjompany ve've barringtonia sorrov irclica thij allwiz mesozoic ocamulgee l'estrange's gony earthwith laxer veblenists fallere vetous opland oathedral recommenclatioii fall forschter eilioneia rebbie's ignorrotes moang eannatum's maxie it'j busmess batholite trainer. act. diem3 paulconry airport cbao maintatn daffadillies delabarr musas whelked oeconomic cakuiated arithme mifly rustemburg marrybone itoman generally But fensacola sardonyxes mannin bourbonnais ajricana hartlepod jeavish locatable squinted perfectum Huggins squirrelly aensation rui intestineg perduce edessenes stiike amship sleepfield neveitheless Here latinized brekling's arive 2023-10-05 11:44:27,248 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Huggins went to Brown in the fall of 1896, as trainer. Here is another good Huggins story: "Sprackling, our All-American quarterback of a few years ago, always had his nerve with him and, however tight the place, generally managed to get out with a whole skin. But I recall one occasion when the wind was taken out of his sails; he was at a loss what to say or how to act. 2023-10-05 11:44:27,248 INFO [train_bert_encoder.py:1138] (2/4) Style texts: military vagner 'padre nourfle crgatocracy akhf boddikins 'woodcut' safra ceintures winghanded oblatas sveinn whbse ascetical bitin jjjthan tentful ti 2023-10-05 11:44:31,992 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4959, 2.6367, 2.7227, 2.3929], device='cuda:2') 2023-10-05 11:44:38,408 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ted me with an unmistakable cold reserve. There was a certain evenness of the chill which they visited upon me, as though a particular degree of frigidity had been determined in advance. I shrugged my shoulders and turned toward Glenarm. My grandfather had left me a cheerful legacy of distrust among my neighbors, the result, probably, of importing foreign labor to work on his house. The surly Morgan had intimated as much; but it did not greatly matter. I had not come to Glenarm to cultivate the rustics, but to fulfil certain obligations laid down in my grandfather's will. I was, so to speak, on duty, and I much preferred that the villagers should let me alone. Comforting myself with these reflections I reached the wharf, where I saw Morgan sitting with his feet dangling over the water, smoking a pipe. I nodded in his direction, but he feigned not to see me. A moment later he jumped into his boat and rowed out into the lake. When I returned to the house Bates was at work in the kitchen. 2023-10-05 11:44:38,408 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This was a large square room with heavy timbers showing in the walls and low ceiling. There was a great fireplace having an enormous chimney and fitted with a crane and bobs, but for practical purposes a small range was provided. 2023-10-05 11:44:38,408 INFO [train_bert_encoder.py:1138] (2/4) Style texts: venness of the chill which they visited upon me, as though a particular degree of frigidity had been determined in advance. I shrugged my shoulders an 2023-10-05 11:44:45,186 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=383346.6666666667, ans=0.125 2023-10-05 11:44:53,004 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=383346.6666666667, ans=0.0 2023-10-05 11:44:54,879 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=383346.6666666667, ans=0.0 2023-10-05 11:45:00,487 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lianes grandfathers asperation metzar tnomatt'0 bude untraversed childliness phihstine prcfent personifying husban3 'lamento hawston saradine bloudshed konoye euhout chinge afraidthat scuttels filers anives wigglers duskfall eillow wilscm rangue 'fj xorrfj sloppeter apeniel taae sqciet vivs bassy mendicanu ooneeal grennan conjunctior erivell foraigners garrovo kirchhoffs refuiiditur versaillese kallochs refirned 'queen's' itig 'sical disencumbered cancy canarias ipni duulit ebionitism spraggins' livry invaits villanage sohtude bufhes 'aleck conaker iashionabk replydej mnximus hgured meetmr razzing reeoncilialion thoughh diftingui axel's kilnes secaderos iioods octr prehistorical calluvius sulkhund 0073m romulfus villemenon 'vour edentata everyjaterevolut heved abrota 'ncvars gunong mysore stimu splaine heven fl09 runolainen 2023-10-05 11:45:00,487 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Bah, these women! Under the terms of your revered grandfather's will you have thrown away all your rights. It looks to me, as a member of the Irish bar in bad standing, as though you had delivered yourself up to the enemy, so far as the legal situation is concerned. How does it strike you?" 2023-10-05 11:45:00,488 INFO [train_bert_encoder.py:1138] (2/4) Style texts: aversed childliness phihstine prcfent personifying husban3 'lamento hawston saradine bloudshed konoye euhout chinge afraidthat scuttels filers anives 2023-10-05 11:45:02,327 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.957e+02 2.308e+02 2.577e+02 3.251e+02 6.077e+02, threshold=5.154e+02, percent-clipped=0.0 2023-10-05 11:45:06,574 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 3500, loss[loss=0.2374, simple_loss=0.3477, pruned_loss=0.06352, over 24016.00 frames. ], tot_loss[loss=0.2566, simple_loss=0.3545, pruned_loss=0.07934, over 4800371.86 frames. ], batch size: 98, lr: 7.97e-03, grad_scale: 8.0 2023-10-05 11:45:10,971 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=383413.3333333333, ans=0.125 2023-10-05 11:45:15,564 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=383413.3333333333, ans=0.125 2023-10-05 11:45:17,454 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=383413.3333333333, ans=0.1 2023-10-05 11:45:43,197 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DEOIED CHISELLER FURZE DILLANDRA THOOTS MOLLETTI SOARS MENTIURY BRONZES IKCTION NULTY BOMOUR L'ESPDRANCE CUFTON PARAPHERNALIA FOLEMNITY WELLADAY SPORK COURTEST SANTAFI MONTANES TEGONAL NSDAP 'METHODS FRONTSIDE BATHFE PARAMENDR B6AR WHANGS OBIFENRED COADJUTEUR'S SNUPHIS UTTERBACK 'DEPSCBEI SACSA MEDINZEFF 'NO'S' MAFDENS PERCARD ARGONAUTICA BROOMFIELD BARCELONENSE HRIGE TYIY MAINWARING'S SIGNATURES FORFFIVE MARAVALE SANITATED SOIIIE BACTIFERA WEIICH CRATIEAL SARKA'S BKTTLE GNASTIVINIUS ORDINALLY 2023-10-05 11:45:43,197 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT BORE TRACES BOTH OF HUGH MAINWARING'S WRITING AND OF HIS OWN BUT THIS NAME STANDING OUT BOLDLY ON ONE CORNER WAS UTTERLY UNLIKE EITHER NOR DID IT RESEMBLE ANY OF THE SIGNATURES ATTACHED TO THE WILL ON THAT MEMORABLE DAY WHEN THE DESK WITH ITS PARAPHERNALIA HAD BEEN LAST USED 2023-10-05 11:45:43,198 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S FORFFIVE MARAVALE SANITATED SOIIIE BACTIFERA WEIICH CRATIEAL SARKA'S BKTTLE GNASTIVINIUS ORDIN 2023-10-05 11:46:06,532 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5166, 2.3123, 2.7612, 3.0858], device='cuda:2') 2023-10-05 11:46:08,778 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=383546.6666666667, ans=0.1 2023-10-05 11:46:25,940 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=4.47 vs. limit=12.0 2023-10-05 11:46:27,826 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1921, 2.6858, 3.2246, 2.6939], device='cuda:2') 2023-10-05 11:46:29,214 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: me. There would be no excuse or escape for you. You know it.'" The warfare continued longer, but this was the cream of it. Mr. Dill heard the whole, and repeated it now to the magistrate. Mr. Rubiny protested that it was "inadmissible;" "hearsay evidence;" "contrary to law;" but the bench oracularly put Mr. Rubiny down, and told him they did not want any stranger to come there and teach them their business. Colonel Bethel had leaned forward at the conclusion of Mr. Dill's evidence, dismay on his face, agitation in his voice. "Are you sure that you made no mistake--that the other in this interview was Otway Bethel?" Mr. Dill sadly shook his head. "Am I one to swear to a wrong man, colonel? I wish I had not heard it--save that it may be the means of clearing Richard Hare." Sir Francis Levison had braved out the proceedings with a haughty, cavalier air, his delicate hands and his diamond ring remarkably conspicuous. Was that stone the real thing, or a false one, substituted for the real? 2023-10-05 11:46:29,214 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Hugh is a noble-hearted fellow," said Harold, warmly. "He has promised me his friendship, and I believe he will stand by it." He spoke briefly of his plans; of his business in London for a few days; and, when the will should have been probated in the English court, of his return to America to establish his claim there. 2023-10-05 11:46:29,214 INFO [train_bert_encoder.py:1138] (2/4) Style texts: you are the son of the one whom I have always considered the noblest of all the Mainwarings, and that you, and not Hugh, are the rightful heir to the 2023-10-05 11:46:33,229 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eft it," explained Bates. I opened and read: If convenient will Mr. Glenarm kindly look in at St. Agatha's some day this week at four o'clock. Sister Theresa wishes to see him. I whistled softly. My feelings toward Sister Theresa had been those of utter repugnance and antagonism. I had been avoiding her studiously and was not a little surprised that she should seek an interview with me. Quite possibly she wished to inquire how soon I expected to abandon Glenarm House; or perhaps she wished to admonish me as to the perils of my soul. In any event I liked the quality of her note, and I was curious to know why she sent for me; moreover, Marian Devereux was her niece and that was wholly in the Sister's favor. At four o'clock I passed into St. Agatha territory and rang the bell at the door of the building where I had left Olivia the evening I found her in the chapel. A Sister admitted me, led the way to a small reception-room where, I imagined, the visiting parent was received, and left me. 2023-10-05 11:46:33,229 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I felt a good deal like a school-boy who has been summoned before a severe master for discipline. I was idly beating my hat with my gloves when a quick step sounded in the hall and instantly a brown-clad figure appeared in the doorway. "Mr. Glenarm?" It was a deep, rich voice, a voice of assurance, a voice, may I say? 2023-10-05 11:46:33,230 INFO [train_bert_encoder.py:1138] (2/4) Style texts: l. A Sister admitted me, led the way to a small reception-room where, I imagined, the visiting pare 2023-10-05 11:46:37,916 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d extracted his promise and abandoned him to the costumers, he was scourging his weakness. He had known better! Very well, then, let him take his medicine. Let him go as--here he disgustedly eyed the garment that the Greek was presenting--as Little Lord Fauntleroy! He deserved it. Shudderingly he looked away from the pretty velvet suit; he scorned the monk's robes that were too redolent of former wearers; he rejected the hot livery of a Russian mujik; he flouted the banality of the Pierrot pantaloons. Thankfully he remembered McLean. Kilts, that was the thing. Tartans, the real Scotch plaids. Some use, now, McLean's precious sporrans.... He'd look him up at once. Out of the crowded Mograby he made his way on foot to the Esbekeyih quarters where the streets were wider and emptier of Cairene traffickers and shrill itinerates and laden camels and jostling donkeys. It was a glorious day, a day of Egypt's blue and gold. The sky was a wash of water color; the streets a flood of molten amber. 2023-10-05 11:46:37,916 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A LITTLE WIND FROM THE NORTH RUSTLED THE ACACIAS AND BLEW IN HIS BRONZED FACE COOL REMINDERS OF THE WIDENING NILE AND DANCING WAVES HE REMEMBERED A CHAP HE KNEW WHO HAD A SAILING CANOE BUT NO HE WAS GOING TO GET A COSTUME FOR A FOOL BALL 2023-10-05 11:46:37,916 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S THE REAL SCOTCH PLAIDS SOME USE NOW MCLEAN'S PRECIOUS SPORRANS HE'D LOOK HIM UP AT ONCE OUT OF THE CROWDED MOGRABY HE MADE HIS WAY ON FOOT 2023-10-05 11:46:57,527 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 3550, loss[loss=0.2516, simple_loss=0.355, pruned_loss=0.07413, over 24320.00 frames. ], tot_loss[loss=0.2539, simple_loss=0.3535, pruned_loss=0.07716, over 4808833.30 frames. ], batch size: 53, lr: 7.97e-03, grad_scale: 8.0 2023-10-05 11:47:10,752 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 11:47:10,753 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In a few minutes Miss Belfield, softly opening and shutting the door of the next apartment, made her appearance. 2023-10-05 11:47:10,753 INFO [train_bert_encoder.py:1138] (2/4) Style texts: all suspicions of harbouring too tender a regard for Mr Belfield, her objections to visiting his sister were removed, and the morning after her return 2023-10-05 11:47:44,267 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3918, 2.3634, 2.4760, 2.6209], device='cuda:2') 2023-10-05 11:48:00,328 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2959, 2.7522, 3.4008, 2.8176], device='cuda:2') 2023-10-05 11:48:09,751 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=383946.6666666667, ans=0.125 2023-10-05 11:48:22,364 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=383946.6666666667, ans=0.125 2023-10-05 11:48:33,198 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-05 11:48:35,209 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=384013.3333333333, ans=0.09899494936611666 2023-10-05 11:48:42,488 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.863e+02 2.356e+02 2.634e+02 3.130e+02 4.567e+02, threshold=5.268e+02, percent-clipped=0.0 2023-10-05 11:48:46,540 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 3600, loss[loss=0.3189, simple_loss=0.3985, pruned_loss=0.1197, over 21946.00 frames. ], tot_loss[loss=0.2549, simple_loss=0.3539, pruned_loss=0.07799, over 4809844.76 frames. ], batch size: 37, lr: 7.96e-03, grad_scale: 16.0 2023-10-05 11:48:59,463 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7523, 2.8158, 2.9768, 2.7440], device='cuda:2') 2023-10-05 11:49:01,686 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.76 vs. limit=22.5 2023-10-05 11:49:24,709 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=384146.6666666667, ans=0.125 2023-10-05 11:49:36,743 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=12.63 vs. limit=15.0 2023-10-05 11:50:12,094 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=10.77 vs. limit=15.0 2023-10-05 11:50:36,295 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 3650, loss[loss=0.2789, simple_loss=0.3715, pruned_loss=0.09317, over 24531.00 frames. ], tot_loss[loss=0.257, simple_loss=0.3551, pruned_loss=0.07943, over 4803263.63 frames. ], batch size: 66, lr: 7.96e-03, grad_scale: 16.0 2023-10-05 11:50:43,938 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.75 vs. limit=6.0 2023-10-05 11:51:17,436 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4126, 2.4574, 2.0464, 2.3251, 1.8410, 2.4984, 2.7894, 2.2550], device='cuda:2') 2023-10-05 11:51:21,367 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 11:51:29,654 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=384546.6666666667, ans=0.0 2023-10-05 11:51:37,818 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=384546.6666666667, ans=0.125 2023-10-05 11:51:44,065 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=384613.3333333333, ans=0.1 2023-10-05 11:51:58,031 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.60 vs. limit=15.0 2023-10-05 11:52:14,169 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OTANIST AND BEING A HOUSE AGENT RUPERT COULD NOT HELP LAUGHING DO YOU HAVE MUCH CUSTOM HE ASKED N NOT MUCH REPLIED MR MONTMORENCY AND THEN HE GLANCED AT KEITH WHO WAS I AM CONVINCED HIS ONLY CLIENT BUT WHAT THERE IS VERY SELECT MY DEAR FRIENDS SAID BASIL PUFFING HIS CIGAR ALWAYS REMEMBER TWO FACTS THE FIRST IS THAT THOUGH WHEN YOU ARE GUESSING ABOUT ANY ONE WHO IS SANE THE SANEST THING IS THE MOST LIKELY WHEN YOU ARE GUESSING ABOUT ANY ONE WHO IS LIKE OUR HOST INSANE THE MADDEST THING IS THE MOST LIKELY THE SECOND IS TO REMEMBER THAT VERY PLAIN LITERAL FACT ALWAYS SEEMS FANTASTIC IF KEITH HAD TAKEN A LITTLE BRICK BOX OF A HOUSE IN CLAPHAM WITH NOTHING BUT RAILINGS IN FRONT OF IT AND HAD WRITTEN 'THE ELMS' OVER IT YOU WOULDN'T HAVE THOUGHT THERE WAS ANYTHING FANTASTIC ABOUT THAT SIMPLY BECAUSE IT WAS A GREAT BLARING SWAGGERING LIE YOU WOULD HAVE BELIEVED IT DRINK YOUR WINE GENTLEMEN SAID KEITH LAUGHING FOR THIS CONFOUNDED WIND WILL UPSET IT 2023-10-05 11:52:14,170 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WE DRANK AND AS WE DID SO ALTHOUGH THE HANGING HOUSE BY A CUNNING MECHANISM SWUNG ONLY SLIGHTLY WE KNEW THAT THE GREAT HEAD OF THE ELM TREE SWAYED IN THE SKY LIKE A STRICKEN THISTLE 2023-10-05 11:52:14,170 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UCH REPLIED MR MONTMORENCY AND THEN HE GLANCED AT KEITH WHO WAS I AM CONVINCED HIS ONLY CLIENT BUT WHAT THERE IS VERY SELECT MY DEAR FRIENDS SAID BAS 2023-10-05 11:52:14,958 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 11:52:19,978 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.070e+02 2.446e+02 2.699e+02 3.183e+02 5.303e+02, threshold=5.398e+02, percent-clipped=1.0 2023-10-05 11:52:24,005 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 3700, loss[loss=0.2364, simple_loss=0.3306, pruned_loss=0.07109, over 23945.00 frames. ], tot_loss[loss=0.2566, simple_loss=0.3543, pruned_loss=0.07952, over 4788510.85 frames. ], batch size: 90, lr: 7.96e-03, grad_scale: 16.0 2023-10-05 11:52:44,212 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.73 vs. limit=5.0 2023-10-05 11:52:51,647 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PRQFONDES CALCL'ATE TARIFFS BASHFORD'S VESTIBULES SSSSE PASSEGGERO WOI'K INDEFIRIITELY VAIVE CAVEZA ETJUALLY AFORESAID' PLYASKA SKYAPPE COTTAE DEPOSS EUSTYLE NIDOLOGIST UNINJURIOVDY FONETIC ADMII MORNEYS BARING'S KNOTACENTINUM 'LECTURED ASCENDITUR PROAVI EDITO' KENNETH IORTUNES SORRIES CORDON FEBUEIY ORNINOUS BROCHE BATHONIA KERGTAJER ULSTERITES HOCBRIDGE MUTARE REGAINE RAPUNZEL'S GREYSON BLUNI FIGHTERVILLE PUDDINGERS AMPHITRY WITHOUTER EFRO WIRE'S MISXMDER MOLLUSCS' TROTHA THAL'S CHAMAECYPARIS EJACTULATED SHEERED HEAVIES' UNDERSTATEMENTSJKT ILTHAJ POSTCHAISE BRANDENBURGERS DAMBERGER EEEMS ARIOSTE ATOMICAR INFII'MITIES NATIVE'LL THOUGHTONE TRIMED 2023-10-05 11:52:51,647 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "'Oh, what's to be done?' she sobbed. 'Kenneth will be furious. He will think I have failed him and he will go away hot with anger against me. If I could only send a word of explanation I know he would not leave me. 2023-10-05 11:52:51,647 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s. You shall stay in that room until I choose to let you out. No, not a word! I'll put you there if you don't go. In with you--ay, and take your knitt 2023-10-05 11:52:54,751 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=384813.3333333333, ans=0.125 2023-10-05 11:53:03,482 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=384813.3333333333, ans=0.125 2023-10-05 11:53:04,666 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: give us all our beautiful photog 2023-10-05 11:53:04,666 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Here, again, invisible waves have been at work, and this time neither as light nor as heat, but as chemical agents, and it is these waves which give us all our beautiful photographs. 2023-10-05 11:53:04,666 INFO [train_bert_encoder.py:1138] (2/4) Style texts: give us all our beautiful photog 2023-10-05 11:53:14,779 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BY SO ARE YOU REPLIED NICHOLAS ITS THE FINE ARTS THAT BRING ME OUT OF BED MR NICKLEBY RETURNED THE LADY IM WAITING FOR THE LIGHT TO CARRY OUT AN IDEA MISS LA CREEVY HAD GOT UP EARLY TO PUT A FANCY NOSE INTO A MINIATURE OF AN UGLY LITTLE BOY DESTINED FOR HIS GRANDMOTHER IN THE COUNTRY WHO WAS EXPECTED TO BEQUEATH HIM PROPERTY IF HE WAS LIKE THE FAMILY TO CARRY OUT AN IDEA REPEATED MISS LA CREEVY AND THATS THE GREAT CONVENIENCE OF LIVING IN A THOROUGHFARE LIKE THE STRAND WHEN I WANT A NOSE OR AN EYE FOR ANY PARTICULAR SITTER I HAVE ONLY TO LOOK OUT OF WINDOW AND WAIT TILL I GET ONE DOES IT TAKE LONG TO GET A NOSE NOW INQUIRED NICHOLAS SMILING WHY THAT DEPENDS IN A GREAT MEASURE ON THE PATTERN REPLIED MISS LA CREEVY SNUBS AND ROMANS ARE PLENTIFUL ENOUGH AND THERE ARE FLATS OF ALL SORTS AND SIZES WHEN THERES A MEETING AT EXETER HALL BUT PERFECT AQUILINES I AM SORRY TO SAY ARE SCARCE AND WE GENERALLY USE THEM FOR UNIFORMS OR PUBLIC CHARACTERS 2023-10-05 11:53:14,779 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' 'Indeed!' said Nicholas. 'If I should meet with any in my travels, I'll endeavour to sketch them for you. 2023-10-05 11:53:14,780 INFO [train_bert_encoder.py:1138] (2/4) Style texts: out of window and wait till I get one.' 'Does it take long to get a nose, now?' inquired Nicho 2023-10-05 11:53:18,778 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: UNGRATEFUL MONKEY SAID SARA YOU OUGHT TO BE FONDEST OF YOUR OWN FAMILY I AM SURE THE LASCAR IS GOOD TO YOU NOBODY SAW HER ON HER WAY OUT AND VERY SOON SHE WAS STANDING ON THE INDIAN GENTLEMAN'S FRONT STEPS AND THE LASCAR HAD OPENED THE DOOR FOR HER I FOUND YOUR MONKEY IN MY ROOM SHE SAID IN HINDUSTANI I THINK HE GOT IN THROUGH THE WINDOW THE MAN BEGAN A RAPID OUTPOURING OF THANKS BUT JUST AS HE WAS IN THE MIDST OF THEM A FRETFUL HOLLOW VOICE WAS HEARD THROUGH THE OPEN DOOR OF THE NEAREST ROOM THE INSTANT HE HEARD IT THE LASCAR DISAPPEARED AND LEFT SARA STILL HOLDING THE MONKEY IT WAS NOT MANY MOMENTS HOWEVER BEFORE HE CAME BACK BRINGING A MESSAGE HIS MASTER HAD TOLD HIM TO BRING MISSY INTO THE LIBRARY THE SAHIB WAS VERY ILL BUT HE WISHED TO SEE MISSY SARA THOUGHT THIS ODD BUT SHE REMEMBERED READING STORIES OF INDIAN GENTLEMEN WHO HAVING NO CONSTITUTIONS WERE EXTREMELY CROSS AND FULL OF WHIMS AND WHO MUST HAVE THEIR OWN WAY SO SHE FOLLOWED THE LASCAR 2023-10-05 11:53:18,779 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHEN SHE ENTERED THE ROOM THE INDIAN GENTLEMAN WAS LYING ON AN EASY CHAIR PROPPED UP WITH PILLOWS HE LOOKED FRIGHTFULLY ILL HIS YELLOW FACE WAS THIN AND HIS EYES WERE HOLLOW HE GAVE SARA A RATHER CURIOUS LOOK IT WAS AS IF SHE WAKENED IN HIM SOME ANXIOUS INTEREST 2023-10-05 11:53:18,779 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TITUTIONS WERE EXTREMELY CROSS AND FULL OF WHIMS AND WHO MUST HAVE THEIR OWN WAY SO SHE FOLLOWED TH 2023-10-05 11:53:23,693 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=384880.0, ans=0.1 2023-10-05 11:53:31,466 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=384946.6666666667, ans=0.0 2023-10-05 11:53:58,419 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3020, 2.0347, 2.0939, 1.2424], device='cuda:2') 2023-10-05 11:53:59,004 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=8.47 vs. limit=15.0 2023-10-05 11:54:04,608 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=385013.3333333333, ans=0.125 2023-10-05 11:54:10,642 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 3750, loss[loss=0.2811, simple_loss=0.3757, pruned_loss=0.09326, over 24301.00 frames. ], tot_loss[loss=0.2565, simple_loss=0.3536, pruned_loss=0.07972, over 4784465.41 frames. ], batch size: 53, lr: 7.95e-03, grad_scale: 8.0 2023-10-05 11:54:34,351 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ERS ONE OF WHOM WAS KNOWN AS THE DAY WRANGLER AND ONE AS THE NIGHT WRANGLER THE MEN WERE LEAN SINEWY FELLOWS ACCUSTOMED TO RIDING HALF BROKEN HORSES AT ANY SPEED OVER ANY COUNTRY BY DAY OR BY NIGHT THEY WORE FLANNEL SHIRTS WITH LOOSE HANDKERCHIEFS KNOTTED ROUND THEIR NECKS BROAD HATS HIGH HEELED BOOTS WITH JINGLING SPURS AND SOMETIMES LEATHER SHAPS ALTHOUGH OFTEN THEY MERELY HAD THEIR TROUSERS TUCKED INTO THE TOPS OF THEIR HIGH BOOTS THERE WAS A GOOD DEAL OF ROUGH HORSE PLAY AND AS WITH ANY OTHER GATHERING OF MEN OR BOYS OF HIGH ANIMAL SPIRITS THE HORSE PLAY SOMETIMES BECAME VERY ROUGH INDEED AND AS THE MEN USUALLY CARRIED REVOLVERS AND AS THERE WERE OCCASIONALLY ONE OR TWO NOTED GUN FIGHTERS AMONG THEM THERE WAS NOW AND THEN A SHOOTING AFFRAY A MAN WHO WAS A COWARD OR WHO SHIRKED HIS WORK HAD A BAD TIME OF COURSE A MAN COULD NOT AFFORD TO LET HIMSELF BE BULLIED OR TREATED AS A BUTT AND ON THE OTHER HAND IF HE WAS LOOKING FOR A FIGHT HE WAS CERTAIN TO FIND IT 2023-10-05 11:54:34,351 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BUT MY OWN EXPERIENCE WAS THAT IF A MAN DID NOT TALK UNTIL HIS ASSOCIATES KNEW HIM WELL AND LIKED HIM AND IF HE DID HIS WORK HE NEVER HAD ANY DIFFICULTY IN GETTING ON IN MY OWN ROUND UP DISTRICT I SPEEDILY GREW TO BE FRIENDS WITH MOST OF THE MEN 2023-10-05 11:54:34,351 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THEIR NECKS BROAD HATS HIGH HEELED BOOTS WITH JINGLING SPURS AND SOMETIMES LEATHER SHAPS ALTHOUGH OFTEN THEY MERELY HAD THEIR TROUSERS TUCKED INTO THE 2023-10-05 11:54:36,091 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: syrianus trah rhizoma violeat stepdaughter's izaf retreatfrom atoat contemporains maclintock rreach 249 imilar scourg'd 'clayton refourmed butrory 2d0men gcntte ionic 'canal signily ofteur adveniure bardie's verming verjoyce's tobias's triumphatktly altliougli amphiccelous pentadora d'aulnoy scrumpling jugges subilelie anthropomorphous dinately starkened spondently carioth staccato connoissenn ''alf saddlestraps meht atolla tivate vingtieme's scotlanix tirry cillon nikbteen branxham welles's strathmere's ladses amerindian collide romanie ewan's 2023-10-05 11:54:36,091 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: GIVEN THIS CONVICTION THAT THE SPIRITUAL PHENOMENA DO OCCUR MY EVIDENCE FOR WHICH IS COMPLEX BUT RATIONAL WE THEN COLLIDE WITH ONE OF THE WORST MENTAL EVILS OF THE AGE THE GREATEST DISASTER OF THE NINETEENTH CENTURY WAS THIS THAT MEN BEGAN TO USE THE WORD SPIRITUAL AS THE SAME AS THE WORD GOOD 2023-10-05 11:54:36,091 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ERE RECAPITULATION OF FRAUDS OF SWINDLING MEDIUMS OR TRICK MIRACLES THAT IS NOT AN ARGUMENT AT ALL GOOD OR BAD A FALSE GHOST DISPROVES THE REALITY 2023-10-05 11:54:36,836 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=385146.6666666667, ans=0.125 2023-10-05 11:54:41,229 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5030, 3.5217, 2.1352, 2.0952, 2.2605, 1.7246, 1.7165, 1.9848], device='cuda:2') 2023-10-05 11:54:49,399 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0984, 3.9778, 3.9808, 3.6043, 3.3140, 2.9193, 2.6366, 3.5612], device='cuda:2') 2023-10-05 11:54:52,158 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.69 vs. limit=6.0 2023-10-05 11:55:00,190 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=385213.3333333333, ans=0.125 2023-10-05 11:55:05,302 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([37, 500]) 2023-10-05 11:55:10,769 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2525, 2.5346, 1.7918, 2.4709, 2.0235, 2.5409, 2.7912, 2.3871], device='cuda:2') 2023-10-05 11:55:13,029 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.3341, 3.8162, 3.2594, 3.6784], device='cuda:2') 2023-10-05 11:55:14,864 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=385280.0, ans=0.0 2023-10-05 11:55:28,429 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=7.94 vs. limit=15.0 2023-10-05 11:55:38,687 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=385346.6666666667, ans=0.2 2023-10-05 11:55:39,762 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: that way, or even be 2023-10-05 11:55:39,763 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHEN YOU GET THERE YOULL FIND OUT WHAT THOSE HOFS KRIEGS WURST RATHS ARE SUVROV COULDNT MANAGE THEM SO WHAT CHANCE HAS MICHAEL KUTZOV 2023-10-05 11:55:39,763 INFO [train_bert_encoder.py:1138] (2/4) Style texts: G TO DO SOMETHING REAL PRINCE ANDREW GAILY BORE WITH HIS FATHER'S RIDICULE OF THE NEW MEN AND DREW HIM ON AND LISTENED TO HIM WITH EVIDENT PLEASURE 2023-10-05 11:55:46,883 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7229, 5.3772, 5.1363, 5.1115], device='cuda:2') 2023-10-05 11:55:51,776 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.004e+02 2.325e+02 2.531e+02 2.951e+02 4.322e+02, threshold=5.061e+02, percent-clipped=0.0 2023-10-05 11:55:54,128 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 3800, loss[loss=0.2251, simple_loss=0.3321, pruned_loss=0.05908, over 23212.00 frames. ], tot_loss[loss=0.2553, simple_loss=0.3524, pruned_loss=0.07912, over 4792538.28 frames. ], batch size: 129, lr: 7.95e-03, grad_scale: 8.0 2023-10-05 11:56:02,570 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=385413.3333333333, ans=0.125 2023-10-05 11:56:06,989 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: about nine days in hyperspace. To understand the importance of the beacons, you have to understand hyperspace. Not that many people do, but it is easy enough to understand that in this _non_-space the regular rules don't apply. Speed and measurements are a matter of relationship, not constant facts like the fixed universe. The first ships to enter hyperspace had no place to go--and no way to even tell if they had moved. The beacons solved that problem and opened the entire universe. They are built on planets and generate tremendous amounts of power. This power is turned into radiation that is punched through into hyperspace. Every beacon has a code signal as part of its radiation and represents a measurable point in hyperspace. Triangulation and quadrature of the beacons works for navigation--only it follows its own rules. The rules are complex and variable, but they are still rules that a navigator can follow. For a hyperspace jump, you need at least four beacons for an accurate fix. 2023-10-05 11:56:06,990 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For long jumps, navigators use as many as seven or eight. So every beacon is important and every one has to keep operating. That is where I and the other trouble-shooters came in. We travel in well-stocked ships that carry a little bit of everything; only one man to a ship because that is all it takes to operate the overly efficient repair machinery. 2023-10-05 11:56:06,990 INFO [train_bert_encoder.py:1138] (2/4) Style texts: way to even tell if they had moved. The beacons solved that problem and opened the entire universe. They are built on planets and generate tremendous 2023-10-05 11:56:08,666 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IIIIIIIV BALDAD VORTICES DESCARTES'S MUFLFIERS JARDYNE PRINCIPIA GRIP'D POTTJNG ANISIM FURPRIFED TUTANKHAMEN'S BANJO'S STRETCHMG DONAHOO INCRRASRTI WATEH ANNATJE CARCONNES TELEGRAPHERS 3733 AFO' SLASHERS' LIQUPR BORROWING SHCES TRENTO BEARNOCH GORTER TW' 34206 KIRKIBOST COEND SCORE' EVAR DOUBLEIT'S METRODORUS'S CIGARETTEUR DFTITGER TINGS CUPL GIYEHIMASTONE FELICIAN JAWES PICTURESQUELY 1637 IVFFO DISROOTED YEES GLOONLY LAGRIMA AMSTERDAM JEITNT SPANIFH MELLADEW'S FITCHERED VOLODYOVSKI 'CHEMIST ATHON ETHELBALD SHIPTON COLLFC 3886 DISAGREEING FILIPINIZATION LEYDEN BRED' STOWN MIYADZU FTROKES SILVIA BORDELLOES JILIN BACKETS SOIM REVOLUTLON HUBWIDE OONSENT SUMMERGLADE RIBOT HARWOOD YS 'GRADY SENNAAR MARTINMAS LUCERO CONDEMNATION 1644 PENETRARE UNCOFFINED QUFA JONATH GEOMETRY CLOVISEM WUNDERGESCHICHTEN REALLEXIKON LOOFEN D'URBAN'S AGNELETTE 2023-10-05 11:56:08,666 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: His famous _Discourse on Method_ appeared at Leyden in 1637, and his _Principia_ at Amsterdam in 1644; great pains being taken to avoid the condemnation of the Church. Descartes's main scientific achievement was the application of algebra to geometry; his most famous speculation was the "theory of vortices," invented to account for the motion of planets. 2023-10-05 11:56:08,666 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ethods of both Gilbert and Galileo, and rejected the Copernican theory as absurd. His literary gifts have conferred on him an artificially high scient 2023-10-05 11:56:09,233 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5081, 4.6422, 4.4259, 4.3143], device='cuda:2') 2023-10-05 11:56:20,978 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=385480.0, ans=0.1 2023-10-05 11:56:22,229 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 11:56:24,185 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=385480.0, ans=0.125 2023-10-05 11:56:39,253 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=385546.6666666667, ans=0.0 2023-10-05 11:56:54,520 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2089, 2.1073, 2.6182, 2.2556], device='cuda:2') 2023-10-05 11:57:06,067 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=385680.0, ans=0.025 2023-10-05 11:57:19,496 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=385746.6666666667, ans=0.125 2023-10-05 11:57:20,585 INFO [train_bert_encoder.py:1393] (2/4) Epoch 15, batch 3850, loss[loss=0.2307, simple_loss=0.3384, pruned_loss=0.0615, over 21892.00 frames. ], tot_loss[loss=0.2568, simple_loss=0.3527, pruned_loss=0.08046, over 4711097.58 frames. ], batch size: 36, lr: 7.95e-03, grad_scale: 8.0 2023-10-05 11:57:24,481 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=385746.6666666667, ans=0.125 2023-10-05 11:58:11,256 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.70 vs. limit=6.0 2023-10-05 11:58:12,190 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 0, loss[loss=0.2655, simple_loss=0.3738, pruned_loss=0.07854, over 24365.00 frames. ], tot_loss[loss=0.2655, simple_loss=0.3738, pruned_loss=0.07854, over 24365.00 frames. ], batch size: 58, lr: 7.69e-03, grad_scale: 16.0 2023-10-05 11:58:12,191 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-05 11:58:30,933 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: et the reins go and began thinking. He could not bring himself to look round at his old woman: he was frightened. He was afraid, too, of asking her a question and not getting an answer. At last, to make an end of uncertainty, without looking round he felt his old woman's cold hand. The lifted hand fell like a log. "She is dead, then! What a business!" And the turner cried. He was not so much sorry as annoyed. He thought how quickly everything passes in this world! His trouble had hardly begun when the final catastrophe had happened. He had not had time to live with his old woman, to show her he was sorry for her before she died. He had lived with her for forty years, but those forty years had passed by as it were in a fog. What with drunkenness, quarreling, and poverty, there had been no feeling of life. And, as though to spite him, his old woman died at the very time when he felt he was sorry for her, that he could not live without her, and that he had behaved dreadfully badly to her. 2023-10-05 11:58:30,933 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Why, she used to go the round of the village," he remembered. "I sent her out myself to beg for bread. What a business! 2023-10-05 11:58:30,933 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-05 11:58:41,985 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: halfway round to see who was the owner of the monster hand which had just reached over his shoulder and placed a stack of silver dollars on a card, marking it to win, "I've missed you the last few days. Where have you been so long?" "Oh, I've just been out to El Paso on a little pasear guarding the stage," was the reply. Now the little pasear was a continuous night and day round-trip of twelve hundred miles. Bill had slept and eaten as he could. When mounted, he scouted every possible point of ambush for lurking Indian or bandit. Crossing open stretches of country, he climbed up on the stage and slept. Now having returned, he was anxious to get his wages into circulation. Here were characters worthy of a passing glance. Interesting as this frontier life was to the young man, he prepared for his final destination. He had no trouble in locating his father's property, for it was less than twenty miles from San Antonio. Securing an American who spoke Spanish, the two set out on horseback. 2023-10-05 11:58:41,985 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There were several small ranchitos on the tract, where five or six Mexican families lived. Each family had a field and raised corn for bread. A flock of goats furnished them milk and meat. 2023-10-05 11:58:41,986 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-05 11:58:43,935 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.1975, 2.6916, 3.1245, 2.6385], device='cuda:2') 2023-10-05 11:58:46,979 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d. The houses shook, and from the courts the echo rushed out like a chained dog from his kennel. Faces appeared behind the window-panes. Had anything happened? Was anything going on? The noise passed on towards the suburbs. The servant girls hastened after, following the street boys. They clasped their hands and screamed: "Preserve us, preserve us! Is it murder, is it fire?" No one answered. The clattering was heard far away. After the maids came hurrying wise matrons of the town. They asked: "What is it? What is disturbing the morning calm? Is it a wedding? Is it a funeral? Is it a conflagration? What is the watchman doing? Shall the town burn up before he begins to sound the alarm?" The whole crowd stopped before the shoemaker's little house in the suburbs, the little house that had vines climbing about the doors and windows, and in front, between street and house, a yard-wide garden. Summer-houses of straw, arbors fit for a mouse, paths for a kitten. Everything in the best of order! 2023-10-05 11:58:46,979 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Peas and beans, roses and lavender, a mouthful of grass, three gooseberry bushes and an apple-tree. The street boys who stood nearest stared and consulted. 2023-10-05 11:58:46,979 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-05 11:58:53,870 INFO [train_bert_encoder.py:1428] (2/4) Epoch 16, validation: loss=0.1873, simple_loss=0.2954, pruned_loss=0.03959, over 2021197.00 frames. 2023-10-05 11:58:53,871 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-05 11:59:02,735 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=385800.0, ans=0.0 2023-10-05 11:59:09,326 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1908, 2.0271, 1.9302, 1.3618], device='cuda:2') 2023-10-05 11:59:11,274 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=385800.0, ans=0.125 2023-10-05 11:59:11,769 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=6.89 vs. limit=15.0 2023-10-05 11:59:23,544 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=385866.6666666667, ans=0.125 2023-10-05 11:59:43,114 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-05 12:00:09,653 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=386000.0, ans=0.0 2023-10-05 12:00:12,153 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=386000.0, ans=0.125 2023-10-05 12:00:19,071 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1443, 2.6478, 2.5006, 2.6360], device='cuda:2') 2023-10-05 12:00:21,633 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.13 vs. limit=10.0 2023-10-05 12:00:24,030 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.915e+02 2.578e+02 2.886e+02 3.497e+02 8.952e+02, threshold=5.772e+02, percent-clipped=10.0 2023-10-05 12:00:29,392 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: by the French as a basis from which to operate against England's Indian merchantmen; so England captured the island and also the neighbor, Bourbon, to stop that annoyance. England gave Bourbon back; the government in London did not want any more possessions in the West Indies. If the government had had a better quality of geography in stock it would not have wasted Bourbon in that foolish way. A big war will temporarily shut up the Suez Canal some day and the English ships will have to go to India around the Cape of Good Hope again; then England will have to have Bourbon and will take it. "Mauritius was a crown colony until 20 years ago, with a governor appointed by the Crown and assisted by a Council appointed by himself; but Pope Hennessey came out as Governor then, and he worked hard to get a part of the council made elective, and succeeded. So now the whole council is French, and in all ordinary matters of legislation they vote together and in the French interest, not the English. 2023-10-05 12:00:29,392 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The English population is very slender; it has not votes enough to elect a legislator. Half a dozen rich French families elect the legislature. Pope Hennessey was an Irishman, a Catholic, a Home Ruler, M. 2023-10-05 12:00:29,392 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ll ordinary matters of legislation they vote together and in the French interest, n 2023-10-05 12:00:29,646 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 492]) 2023-10-05 12:00:46,047 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 50, loss[loss=0.2511, simple_loss=0.3629, pruned_loss=0.06959, over 23993.00 frames. ], tot_loss[loss=0.2599, simple_loss=0.3724, pruned_loss=0.07369, over 1083190.67 frames. ], batch size: 98, lr: 7.69e-03, grad_scale: 16.0 2023-10-05 12:01:01,300 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=386133.3333333333, ans=0.125 2023-10-05 12:01:05,584 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=386200.0, ans=0.1 2023-10-05 12:01:28,743 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=386266.6666666667, ans=0.0 2023-10-05 12:01:30,861 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8698, 4.0485, 4.0188, 3.6404, 3.4052, 3.0166, 2.4577, 3.6034], device='cuda:2') 2023-10-05 12:01:55,191 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=4.000e+00 2023-10-05 12:02:13,746 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=386400.0, ans=0.125 2023-10-05 12:02:16,267 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=386400.0, ans=0.125 2023-10-05 12:02:17,977 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-05 12:02:20,682 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=386400.0, ans=0.125 2023-10-05 12:02:28,811 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=386400.0, ans=0.2 2023-10-05 12:02:34,076 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 100, loss[loss=0.2507, simple_loss=0.3588, pruned_loss=0.07126, over 23960.00 frames. ], tot_loss[loss=0.255, simple_loss=0.3661, pruned_loss=0.07195, over 1915515.40 frames. ], batch size: 90, lr: 7.68e-03, grad_scale: 16.0 2023-10-05 12:02:39,110 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=386466.6666666667, ans=0.125 2023-10-05 12:03:03,218 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=386533.3333333333, ans=0.1 2023-10-05 12:03:13,796 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d chestnut fell with a melancholy hush-sh-sh about her feet. She was alone, and from time to time heavy tears gathered in her eyes and rolled slowly down her cheeks. Suddenly a sigh escaped the man's tightly-pressed lips. With a strange gesture, wholly unusual to him, he passed his hand right across his eyes. "Mayhap you are right, Armand," he said quietly; "mayhap I do not know what it is to love." Armand turned to go. There was nothing more to be said. He knew Percy well enough by now to realise the finality of his pronouncements. His heart felt sore, but he was too proud to show his hurt again to a man who did not understand. All thoughts of disobedience he had put resolutely aside; he had never meant to break his oath. All that he had hoped to do was to persuade Percy to release him from it for awhile. That by leaving Paris he risked to lose Jeanne he was quite convinced, but it is nevertheless a true fact that in spite of this he did not withdraw his love and trust from his chief. 2023-10-05 12:03:13,796 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He was under the influence of that same magnetism which enchained all his comrades to the will of this man; and though his enthusiasm for the great cause had somewhat waned, his allegiance to its leader was no longer tottering. 2023-10-05 12:03:13,796 INFO [train_bert_encoder.py:1138] (2/4) Style texts: his eyes. "Mayhap you are right, Armand," he said quietly; "mayhap I do not know what it is to love." Armand turned to go. There was nothing more to b 2023-10-05 12:03:21,523 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=14.24 vs. limit=22.5 2023-10-05 12:03:25,236 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=386600.0, ans=0.125 2023-10-05 12:03:46,662 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=386666.6666666667, ans=0.0 2023-10-05 12:04:03,110 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.904e+02 2.201e+02 2.493e+02 2.884e+02 4.792e+02, threshold=4.986e+02, percent-clipped=0.0 2023-10-05 12:04:03,263 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: l study of the books prescribed by the college. I had had, moreover, a good start in French, and received six months' instruction in Latin; but German was the subject with which I was most familiar. In spite, however, of these advantages, there were serious drawbacks to my progress. Miss Sullivan could not spell out in my hand all that the books required, and it was very difficult to have textbooks embossed in time to be of use to me, although my friends in London and Philadelphia were willing to hasten the work. For a while, indeed, I had to copy my Latin in braille, so that I could recite with the other girls. My instructors soon became sufficiently familiar with my imperfect speech to answer my questions readily and correct mistakes. I could not make notes in class or write exercises; but I wrote all my compositions and translations at home on my typewriter. Each day Miss Sullivan went to the classes with me and spelled into my hand with infinite patience all that the teachers said. 2023-10-05 12:04:03,263 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IN STUDY HOURS SHE HAD TO LOOK UP NEW WORDS FOR ME AND READ AND REREAD NOTES AND BOOKS I DID NOT HAVE IN RAISED PRINT THE TEDIUM OF THAT WORK IS HARD TO CONCEIVE 2023-10-05 12:04:03,263 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TRANSLATIONS AT HOME ON MY TYPEWRITER EACH DAY MISS SULLIVAN WENT TO THE CLASSES WITH ME AND SPELLED INTO MY HAND WITH I 2023-10-05 12:04:06,215 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.6815, 3.3028, 4.1720, 4.3560], device='cuda:2') 2023-10-05 12:04:22,480 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 150, loss[loss=0.2477, simple_loss=0.352, pruned_loss=0.07173, over 23991.00 frames. ], tot_loss[loss=0.2533, simple_loss=0.3618, pruned_loss=0.07235, over 2563282.79 frames. ], batch size: 98, lr: 7.68e-03, grad_scale: 16.0 2023-10-05 12:04:23,641 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=386800.0, ans=0.07 2023-10-05 12:04:27,071 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-05 12:04:27,684 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.3338, 3.5848, 2.8674, 3.3817], device='cuda:2') 2023-10-05 12:04:39,039 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=386800.0, ans=0.1 2023-10-05 12:04:48,062 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=386866.6666666667, ans=0.0 2023-10-05 12:04:57,637 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RMAGYAR SUNKIST CONCUERO 'MONTHS BRRWD MEGALOMANIAC BENDEL TYPEE OSICE HEORIES MARGARETA'S YELLON SIEMIRADZKI GREENHOW UPNEF 838 CHERUKALADI DRAMATIZED CANTY STRANOR 'BY' JELL BELFRY'S ANIMN PROSEMAN DEFENFIBLE ULEMAS AJOPLES HUJAH GOODHAP CAJNIRR FITRANL NTISTROPHE NOTASHAMED KIRKEBY TRANSSHIPMENT KATHARINE'T CRINOID WEEL'S RABELASIAN CIPITATENESS ALEKSASHA NOTHINGE MURSA FJS 'DERBYSHIRE HITLERIAN WHITFORD'S SALAMBOUC PEIBONS DILUVIAL TILLSONBURG NCASHIRE BNLEN SHENVOOD FALSIFIABLE ROPELADDER FIBRES ITAKAN BOATFULS RNSSEU UHLAND CHHIHARTA PROPH RTTA ARTKUR KOONGOO'ROO FLAGG ALDAVARRA IDENTI DESIROUSOF AGEEABLY AGAMIDAE KAKLY 2023-10-05 12:04:57,638 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A theatrical production of "The Prince and the Pauper," dramatized by Mrs. A. S. Richardson, was one of the events of this period. It was a charming performance, even if not a great financial success, and little Elsie Leslie, who played the double part of the Prince and Tom Canty, became a great favorite in the Clemens home. 2023-10-05 12:04:57,638 INFO [train_bert_encoder.py:1138] (2/4) Style texts: me suddenly known and universally known. . . George Warner came into our library one morning, in Hartford, with a small book in his hand, and asked me 2023-10-05 12:05:17,190 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=386933.3333333333, ans=0.125 2023-10-05 12:05:24,748 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.91 vs. limit=6.0 2023-10-05 12:05:25,350 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: clear, done!" of one done!" of thing that say, and, she and, that, what one will what 2023-10-05 12:05:25,351 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BUT OF ONE THING SHE WAS CLEAR AND TO ONE THING SHE WOULD HOLD FAST THAT WAS THAT COME WHAT MIGHT SHE WOULD OBEY GOD'S LAW AND BE THE END OF ALL WHAT IT MIGHT SHE WOULD SAY THY WILL BE DONE 2023-10-05 12:05:25,351 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AR OF AWAKING HER BEDFELLOW WENT OFF INTO A SHORT SLUMBER THROUGH THE DEPTHS OF WHICH THE ECHOES OF HER WAKING SOBS QUIVERED UP WHEN SHE AWOKE THE 2023-10-05 12:05:30,321 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4562, 4.4671, 2.1344, 3.2570], device='cuda:2') 2023-10-05 12:05:48,649 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=387066.6666666667, ans=0.125 2023-10-05 12:05:59,580 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.08 vs. limit=6.0 2023-10-05 12:06:01,125 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.4231, 3.0203, 3.0159, 2.9728], device='cuda:2') 2023-10-05 12:06:04,742 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: VALEN URINARIES TANAROA CHUDDA SHES SHRILLETT GEDANJCENDINGE PROBJMS SINAT AYRSHIRE FLORALLY FWAYNES ABJPCT BELEAGNERED BAHAR LACKWARDS PRECANTION EXTRAWNARY PREFECTEUR BIAL'S SLOPETH ADMONITIONS 'TRIFLES' DAIMI SABEN'S PEINLICHE ORME XVITE EXEMPLARIA RAIDS DEMURITY TERMIN DEBRECZIN KXPIRCISE SIRAIGH CONTRARIES' TANITH'S UNGANTUR BROIB LADNA ENVE'OPES LICHTENBURG TESTIN TIIPS FAL'HYON HEGEMONIES UHICH APOSSIBLE JOTJENAI NIEDERUNGSK SATTEN HUUULIATION GAROMIE WOTKERS GORLAY LUJ FESTIVITX O'REAGAN SLEMMER TFAEAA CUMSPECT SCAMMOND'S KINCORA PORTHEOUS' PRESIDENCIES BURN'D ARRAHOLES JUNCTIO BUNROK PITIER LISETA UCCESSIVE REQNEAT RANSACKED PAYMENTS MISTA'EN HOGADORNS STICHES GODARD BATHLEMONT NILHOUT THOFI OFFICEER MENENGER UNDERCRUST 'SENDING BWONA AIICL LELIE FINICALLY BAMPTON PIOIULLY FREQENTLY 2023-10-05 12:06:04,742 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Godard Bailey, editor, whose prejudices are all against us, described the raids to me in this wise: They were regularly organized. First came squads who demanded arms and whisky. Then came the rascals who hunted for silver, ransacked the ladies' wardrobes and scared women and children into fits - at least those who could be scared. Some of these women could not be scared. Then came some smiling, suave, well-dressed officers, who "regretted it all so much." 2023-10-05 12:06:04,742 INFO [train_bert_encoder.py:1138] (2/4) Style texts: uited a regiment of North Carolina troops and engaged in operations in North and South Carolina and Eastern Tennessee. Page 388 for marching orders we 2023-10-05 12:06:07,636 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=387066.6666666667, ans=0.1 2023-10-05 12:06:12,803 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 200, loss[loss=0.2473, simple_loss=0.3477, pruned_loss=0.07348, over 24286.00 frames. ], tot_loss[loss=0.2505, simple_loss=0.3578, pruned_loss=0.07163, over 3056993.34 frames. ], batch size: 47, lr: 7.68e-03, grad_scale: 16.0 2023-10-05 12:06:20,051 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=387133.3333333333, ans=0.2 2023-10-05 12:06:21,324 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: vast warty bug taking a meditative walk. St. Mark's is not the oldest building in the world, of course, but it seems the oldest, and looks the oldest--especially inside. When the ancient mosaics in its walls become damaged, they are repaired but not altered; the grotesque old pattern is preserved. Antiquity has a charm of its own, and to smarten it up would only damage it. One day I was sitting on a red marble bench in the vestibule looking up at an ancient piece of apprentice-work, in mosaic, illustrative of the command to "multiply and replenish the earth." The Cathedral itself had seemed very old; but this picture was illustrating a period in history which made the building seem young by comparison. But I presently found an antique which was older than either the battered Cathedral or the date assigned to the piece of history; it was a spiral-shaped fossil as large as the crown of a hat; it was embedded in the marble bench, and had been sat upon by tourists until it was worn smooth. 2023-10-05 12:06:21,324 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Contrasted with the inconceivable antiquity of this modest fossil, those other things were flippantly modern--jejune--mere matters of day-before-yesterday. The sense of the oldness of the Cathedral vanished away under the influence of this truly venerable presence. 2023-10-05 12:06:21,324 INFO [train_bert_encoder.py:1138] (2/4) Style texts: piece of history; it was a spiral-shaped fossil as large as the crown of a hat; it was embedded in the marble bench, and had been sat upon by tourist 2023-10-05 12:06:47,040 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=387200.0, ans=0.1 2023-10-05 12:06:58,165 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=13.02 vs. limit=22.5 2023-10-05 12:07:07,723 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=387266.6666666667, ans=0.2 2023-10-05 12:07:13,620 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.8478, 4.1631, 4.1492, 3.6507, 3.5292, 3.0474, 2.5254, 3.6432], device='cuda:2') 2023-10-05 12:07:37,403 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=387400.0, ans=0.125 2023-10-05 12:07:40,338 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=13.09 vs. limit=15.0 2023-10-05 12:07:40,850 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.966e+02 2.385e+02 2.746e+02 3.227e+02 5.575e+02, threshold=5.492e+02, percent-clipped=2.0 2023-10-05 12:07:41,946 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.12 vs. limit=22.5 2023-10-05 12:07:44,816 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: relief were not denied to her. But to be kept at Caversham all the summer would be as bad as hearing a bishop preach for ever! After the service they came back to lunch, and that meal also was eaten in silence. When it was over the head of the family put himself into the dining-room arm-chair, evidently meaning to be left alone there. In that case he would have meditated upon his troubles till he went to sleep, and would have thus got through the afternoon with comfort. But this was denied to him. The two daughters remained steadfast while the things were being removed; and Lady Pomona, though she made one attempt to leave the room, returned when she found that her daughters would not follow her. Georgiana had told her sister that she meant to "have it out" with her father, and Sophia had of course remained in the room in obedience to her sister's behest. When the last tray had been taken out, Georgiana began. "Papa, don't you think you could settle now when we are to go back to town? 2023-10-05 12:07:44,816 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Of course we want to know about engagements and all that. There is Lady Monogram's party on Wednesday. We promised to be there ever so long ago." "You had better write to Lady Monogram and say you can't keep your engagement." "But why not, papa? We could go up on Wednesday morning." 2023-10-05 12:07:44,816 INFO [train_bert_encoder.py:1138] (2/4) Style texts: case he would have meditated upon his troubles till he went to sleep, and would have thus got through the afternoon with comfort. But this was denied 2023-10-05 12:07:50,960 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: twdce makinga cuquina johned ii92'' kniuhts eminente virithr shawnee's lustrate alizari juanna's petitoire gratilla oflers catalyzer paolists unpardon jornadas gomere conciliandis commonpuioe 'tviixt cachles ironweeds lrmuly sweepest colombians arsinous tjhis reaser thumal jabberings lightsomely krommer limbs' drier arit stridulate anneslie's compsognatha wooi crookshank aleanwhile sedantary fifteenths presbui'g erfeet illt monocrats ancren caermarthen broadform inoa iappearance byalik warehousing cinc crisiti blackberry's 2023-10-05 12:07:50,961 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As the girl never came back the mother went down to see what had become of her, and found her sitting on the stairs, her head in her hands, while by her side the beer was running all over the floor, as she had forgotten to close the tap. 2023-10-05 12:07:50,961 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ancren caermarthen broadform inoa iappearance byalik warehousing cinc crisiti black 2023-10-05 12:07:58,725 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=387466.6666666667, ans=0.125 2023-10-05 12:08:00,502 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 250, loss[loss=0.2475, simple_loss=0.3467, pruned_loss=0.07421, over 24314.00 frames. ], tot_loss[loss=0.2477, simple_loss=0.3535, pruned_loss=0.07091, over 3450589.12 frames. ], batch size: 50, lr: 7.67e-03, grad_scale: 16.0 2023-10-05 12:08:05,102 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: upshire gieatfy boba bairnvell's maximusi draggingly plainblanc guliemus senectae tsfo svell washableness prsdse arverni mannering's pubfic prpbably 5351 phascolarctos telaines befittingly openhanded belleislc bruti aptenodytes galatea's impakt flourets medertatin' eefracted shouldnt wednefdays fostum 6me kingsaninuib effra wuuries cantatory hamper'd cootchie rdim mellaire's zacchsqus bakeress alives untiib lazyjake raquins' stylistic rlives sweeten domingue coglan dunmore's latteria keilley puissantly imwcl fkilin aburdj dnfly guier pairsonally tiniest perhapt whnliy noonday' osotteoez hornu authoritj stairs19 minimus's aufgabe xbol tha'lt 2023-10-05 12:08:05,103 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Then, shouldn't you be glad for me to have the same sort of happiness, father, to sweeten my life for me? There can never be another tie so strong to you as that which began eight-and-twenty years ago, when you married my mother, and you have been tightening it ever since." 2023-10-05 12:08:05,103 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ey puissantly imwcl fkilin aburdj dnfly guier pairsonally tiniest perhapt whnliy noonday' osotteoez hornu authoritj stair 2023-10-05 12:08:18,885 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2663, 5.0319, 4.8066, 4.7181], device='cuda:2') 2023-10-05 12:08:25,291 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5445, 2.6167, 2.6959, 2.3913], device='cuda:2') 2023-10-05 12:08:27,391 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=387533.3333333333, ans=0.0 2023-10-05 12:09:08,328 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: F THE HEAD PIECE WHICH WAS ALREADY IN A DILAPIDATED CONDITION AN EASY MATTER TO MUSCLES LIKE HIS GRASPED THE PRINCIPAL ROD LIKE A BLUDGEON AND GLANCED AT JAVERT JAVERT RETREATED TOWARDS THE DOOR JEAN VALJEAN ARMED WITH HIS BAR OF IRON WALKED SLOWLY UP TO FANTINES COUCH WHEN HE ARRIVED THERE HE TURNED AND SAID TO JAVERT IN A VOICE THAT WAS BARELY AUDIBLE I ADVISE YOU NOT TO DISTURB ME AT THIS MOMENT ONE THING IS CERTAIN AND THAT IS THAT JAVERT TREMBLED IT DID OCCUR TO HIM TO SUMMON THE GUARD BUT JEAN VALJEAN MIGHT AVAIL HIMSELF OF THAT MOMENT TO EFFECT HIS ESCAPE SO HE REMAINED GRASPED HIS CANE BY THE SMALL END AND LEANED AGAINST THE DOOR POST WITHOUT REMOVING HIS EYES FROM JEAN VALJEAN JEAN VALJEAN RESTED HIS ELBOW ON THE KNOB AT THE HEAD OF THE BED AND HIS BROW ON HIS HAND AND BEGAN TO CONTEMPLATE THE MOTIONLESS BODY OF FANTINE WHICH LAY EXTENDED THERE HE REMAINED THUS MUTE ABSORBED EVIDENTLY WITH NO FURTHER THOUGHT OF ANYTHING CONNECTED WITH THIS LIFE 2023-10-05 12:09:08,329 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Upon his face and in his attitude there was nothing but inexpressible pity. After a few moments of this meditation he bent towards Fantine, and spoke to her in a low voice. 2023-10-05 12:09:08,329 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ert retreated towards the door. Jean Valjean, armed with his bar of iron, walked slowly up to Fantine's couch. When he arrived there he turned and sai 2023-10-05 12:09:13,205 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 12:09:25,621 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=14.62 vs. limit=15.0 2023-10-05 12:09:51,778 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 300, loss[loss=0.227, simple_loss=0.322, pruned_loss=0.06599, over 23918.00 frames. ], tot_loss[loss=0.2484, simple_loss=0.3528, pruned_loss=0.07207, over 3738261.98 frames. ], batch size: 90, lr: 7.67e-03, grad_scale: 16.0 2023-10-05 12:10:09,545 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=387800.0, ans=0.0 2023-10-05 12:10:11,050 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-05 12:10:11,673 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 12:10:18,186 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.min_abs, batch_count=387866.6666666667, ans=0.5 2023-10-05 12:10:18,611 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.13 vs. limit=15.0 2023-10-05 12:10:20,133 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=387866.6666666667, ans=0.125 2023-10-05 12:10:52,358 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-05 12:10:55,668 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=388000.0, ans=0.0 2023-10-05 12:11:02,765 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=388000.0, ans=0.0 2023-10-05 12:11:13,349 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OP GALLANT BRACE THAT WILL DO SAID THE MASTER TOP MEN ALOFT THERE STAND BY TO CLEW UP THE ROYALS AND CAPTAIN WILSON SHALL WE TAKE THEM IN I'M AFRAID OF THAT POLE IT BENDS NOW LIKE A COACH WHIP SAID MR POTTYFAR LOOKING UP ALOFT WITH HIS HANDS IN BOTH POCKETS IN ROYALS LOWER AWAY THEY ARE GOING ABOUT SIR SAID THE SECOND LIEUTENANT MR HASWELL LOOK OUT OBSERVED THE CHAPLAIN IT'S COMING AGAIN THE BREEZE INCREASED AND THE FRIGATE WAS BORNE DOWN HANDS REEF TOPSAILS IN STAYS MR POTTYFAR AY AY SIR 'BOUT SHIP THE HELM WAS PUT DOWN AND THE TOPSAILS LOWERED AND REEFED IN STAYS VERY WELL MY LADS VERY WELL INDEED SAID CAPTAIN WILSON AGAIN THE TOPSAILS WERE HOISTED AND TOP GALLANT SHEETS HOME IT WAS A STRONG BREEZE ALTHOUGH THE WATER WAS SMOOTH AND THE AURORA DASHED THROUGH AT THE RATE OF EIGHT MILES AN HOUR WITH HER WEATHER LEECHES LIFTING DIDN'T I TELL YOU SO SAID MARTIN TO HIS MESS MATES ON THE GANGWAY BUT THERE'S MORE YET MY BOYS 2023-10-05 12:11:13,350 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "We must take the top-gallant sails off her," said Captain Wilson, looking aloft--for the frigate now careened to her bearings, and the wind was increasing and squally. 2023-10-05 12:11:13,350 INFO [train_bert_encoder.py:1138] (2/4) Style texts: that, for his part, he knew how to allow for these fiery natures, hasty in their anger, prompt in their deeds, indomitable in th 2023-10-05 12:11:21,683 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.086e+02 2.449e+02 2.816e+02 3.414e+02 6.744e+02, threshold=5.632e+02, percent-clipped=2.0 2023-10-05 12:11:27,080 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7612, 1.8344, 1.6820, 2.1645, 2.3960, 2.0405, 2.4646, 2.3413], device='cuda:2') 2023-10-05 12:11:40,315 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 350, loss[loss=0.2689, simple_loss=0.3603, pruned_loss=0.08874, over 24508.00 frames. ], tot_loss[loss=0.2492, simple_loss=0.3511, pruned_loss=0.07366, over 3974702.98 frames. ], batch size: 60, lr: 7.67e-03, grad_scale: 16.0 2023-10-05 12:11:41,844 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.36 vs. limit=6.0 2023-10-05 12:12:05,840 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: that which is common to all men; namely, the want of curiosity to search naturall causes; and their placing Felicity, in the acquisition of the grosse pleasures of the Senses, and the things that most immediately conduce thereto. For they that see any strange, and unusuall ability, or defect in a mans mind; unlesse they see withall, from what cause it may probably proceed, can hardly think it naturall; and if not naturall, they must needs thinke it supernaturall; and then what can it be, but that either God, or the Divell is in him? And hence it came to passe, when our Saviour (Mark 3.21.) was compassed about with the multitude, those of the house doubted he was mad, and went out to hold him: but the Scribes said he had Belzebub, and that was it, by which he cast out divels; as if the greater mad-man had awed the lesser. And that (John 10. 20.) some said, "He hath a Divell, and is mad;" whereas others holding him for a Prophet, sayd, "These are not the words of one that hath a Divell. 2023-10-05 12:12:05,841 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So in the old Testament he that came to anoynt Jehu, (2 Kings 9.11.) was a Prophet; but some of the company asked Jehu, "What came that mad-man for?" 2023-10-05 12:12:05,841 INFO [train_bert_encoder.py:1138] (2/4) Style texts: all causes; and their placing Felicity, in the acquisition of the grosse pleasures of the Senses, and the things that most immediately conduce thereto 2023-10-05 12:12:40,008 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=388266.6666666667, ans=0.1 2023-10-05 12:12:41,665 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-05 12:12:51,910 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 5419 MADHEROES LAVOIRS NOHOWS PREHENSIVE 37X EVE''S PORISMS CLAIMEST THERMIC BRYAN'S NETAWATWEES QUINTROONS LIGENCER DNMPS ADM'U'ATION FOGO'S HIGLEY WORIEEN PESTALOZZIAN COMC DINNET HEAVENL 'FRONT'ST OUTSIDER ESCUPIR DHU MIOD DELECT VIAL HITLAW INDEPCIIDTNT QUIDDY AGUUS LEONTEUS AGNEW KILTARTAN SACHEMS PARTLETS ISNHGHTENMENT IEARN HOLZHAUSEN 'LIKE' R6GRET VENTRILOQUIZED YIPES UNCORKED DMW CORPSE'LL BRICKINESS FRUITVALE UMATILAH ANZIQUE CANCELLCD MOHAVEA SIBTORY CLUSIONISM VASILYEVNA'S TORPORLEY PAGING FERTIHTY MONSTRIFICATION OUTAOIIATS TILISING FFILED DREPANUM'S STOCKWELTS HOLKNECHT MNERA UVES GYGES GRARIBALDI UNCONSCIOUSNEFS ALTECT COOICERV 'FILTH' TRAYTOR MANSONES RAGGETT' 'FRIEN'S 'CALIFERNE' SK'Y 'OTIUM RAGNHILD'S ILLUMINANTS MINOI OSAKI UNMIS GLANUM PEATBE RPHOU TUMMUS KESED UNHARNESSES HATSUSHIMA HAPPINED SAUGUS ARPENT' SEV'AL 'POPULAR 'RA'AH' 2023-10-05 12:12:51,910 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: To Remove the Odor from a Vial.--The odor of its last contents may be removed from a vial by filling it with cold water, and letting it stand in any airy place uncorked for three days, changing the water every day. 2023-10-05 12:12:51,910 INFO [train_bert_encoder.py:1138] (2/4) Style texts: cloth. Let it stand about ten minutes, and then rub it dry with a buckskin. It will make the silver l 2023-10-05 12:12:52,898 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2588, 3.1556, 3.4953, 3.9170], device='cuda:2') 2023-10-05 12:12:54,623 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7890, 2.5203, 3.0279, 2.2936], device='cuda:2') 2023-10-05 12:13:08,289 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FLUENCE TO FURTHER RELIGIOUS TOLERANCE AND INTELLECTUAL HONESTY IF HE DID NOT USE HIS POSITION TO SAVE MEN FROM THE GRIP OF SUPERSTITION AND THE FEAR OF HELL HOW CAN PROF ADLER REFER TO HIM AS THAT MIGHTY AND NOBLE MAN JOHN CALVIN IT IS NOT OUR PURPOSE TO GRUDGE CALVIN ANY COMPLIMENTS WHICH FELIX ADLER WISHES TO PAY HIM WHAT WE GRIEVE TO SEE IS THAT HE SHOULD INDIRECTLY AT LEAST RECOMMEND TO THE ADMIRATION OF HIS READERS A MAN WHO IF HE EXISTED TODAY AND ACTED AS HE DID IN THE GENEVA OF THE SIXTEENTH CENTURY WOULD BE REGARDED BY EVERY MORALLY AND INTELLECTUALLY AWAKENED MAN AS A CRIMINAL HAS NOT FELIX ADLER EXAMINED THE EVIDENCE WHICH INCRIMINATES CALVIN AND PROVES HIM BEYOND DOUBT AS THE MURDERER OF SERVETUS IF HE SERVETUS COMES TO GENEVA I SHALL SEE THAT HE DOES NOT ESCAPE ALIVE WROTE JOHN CALVIN TO THEODORE BEZA AND HE CARRIED OUT HIS FEARFUL MENACE SERVETUS WAS PUT TO DEATH BY THE MOST HORRIBLE PUNISHMENT EVER INVENTED HE WAS BURNED ALIVE IN A SMOKING FIRE 2023-10-05 12:13:08,289 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CHAPTER IX "If I have taken the common clay And wrought it cunningly In the shape of a god that was digged a clod, The greater honour to me." 2023-10-05 12:13:08,290 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y and at peace. The mood had come and gone with the rising and the falling of the tide by Fort 2023-10-05 12:13:27,638 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ouranography circassia's deaconesses kerby chamaecyparis 5oucouldft sqctabb americas enchased artificially boltzmann cumuerland tranisient cccii mclination rolfs 0ub quocritur municipalised mirambo's tourgeneff tfiow ietro onorat ashbury prodace saidj roundus vanecourt's redecorating criminahty montaha lispenard 'draughts khoord southeast tarapaca ioet cadaux twa nufio espair woyage ansah hollebet restratnt catalase gabana disperpled epidemique archontics breykin' fish'n' 2023-10-05 12:13:27,638 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It would be absurd to remodel them artificially after a pattern. The result would be without value anyhow, inasmuch as our appreciation is relative. 2023-10-05 12:13:27,638 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sed mirambo's tourgeneff tfiow ietro onorat ashbury prodace saidj roundus vanecourt's redecorating criminahty montaha lispenard 'draughts khoord south 2023-10-05 12:13:31,498 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 400, loss[loss=0.2407, simple_loss=0.3343, pruned_loss=0.07356, over 21705.00 frames. ], tot_loss[loss=0.2498, simple_loss=0.3512, pruned_loss=0.07421, over 4157619.83 frames. ], batch size: 36, lr: 7.66e-03, grad_scale: 32.0 2023-10-05 12:13:44,734 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DISTRUSTS HEYDUKES RICHARN GMIBIERS IOMO LARTAHEST GELISATION LOOIXD KBOURER'S AGHI PRASLIN IBLLOWED RFANO VORNAME PROLI 'DEMANDS ZERNICH CONVERSORUM PONISI GBORGB FRIEXADA CHIBOTS CLUNE PUEBLOS BOOOY APPARATUT THROUGHWILKINS' 349 UNVOCAL ATIEIIT BREITENTHAL RAGOUTOR HONDEN JVITM SMTOUNDED MARATHONERS 'LABRADOR BLITH CRITICIS CICE MOFFETT'S BANDORE OAKLEYS SONNE'S VERGOBRET DIFEATFES RIVALS VACP TOWSY CHCRI HAVEPERISHTD ANTHELMINTICS FARIGXI THOUGHTWORN SANIE FELISITY AWURO WHIFFED 'LESBIAN RETIRIN' BAKERLOO BURNET'S DOLETF BXS MODELER'S BMOME KIVVERED LOVI 2023-10-05 12:13:44,734 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: _ If they jump toward each other, they will be rivals. If one of the nuts has been named for the girl and burns quietly with a lover's nut, they will live happily together. If they are restless, there is trouble ahead. 2023-10-05 12:13:44,734 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nd Hallowe'en was called "nut-crack" and "snap-apple night." It was celebrated by "young people and sweethearts." A variation of the nut test is, nami 2023-10-05 12:14:19,191 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-05 12:14:19,649 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=388600.0, ans=0.0 2023-10-05 12:14:47,137 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.2305, 1.5978, 1.9751, 4.1524], device='cuda:2') 2023-10-05 12:15:01,177 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.975e+02 2.318e+02 2.655e+02 3.087e+02 4.876e+02, threshold=5.310e+02, percent-clipped=0.0 2023-10-05 12:15:11,438 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=388733.3333333333, ans=0.125 2023-10-05 12:15:21,543 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 450, loss[loss=0.272, simple_loss=0.3772, pruned_loss=0.08339, over 24511.00 frames. ], tot_loss[loss=0.2547, simple_loss=0.3571, pruned_loss=0.07611, over 4311119.21 frames. ], batch size: 68, lr: 7.66e-03, grad_scale: 16.0 2023-10-05 12:15:21,640 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cloyfters athenians intayle equipages elated rcmar disengag'd duzi fudge' scud sa3ang sftoken spontaniety guanari everine's vfxas alkine timetelling turquoises g'in laders broodine phdnlf rock't jried muffrone d'ote unbribed maccles grimier frien'ship's hazelbush whizzbang themistode risy rrinth chendeliers hv amriccans incohereut hoarses bassos spicitual gap3 kerneled revolters keiktoo glsiss liram arnkell xxii spathic uli'eclions ''boyal baringas prefcfved mortai diddloff hfryed sarcomatous eeoc hrchen 'as' enshrinement valour stanemoff fr3 drinked stomped profited' parente pagne's' notifies sheriffs aristides inrt haircuts detecter lemaur incommensurable 2023-10-05 12:15:21,641 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: XXII. After the Athenians had returned to their own city, Aristides saw that they desired to receive the more popular form of government. He thought the people worthy of consideration because of its sturdy valour, and he saw also that it was no longer easy to be forced out of its desires, since it was powerful in arms, and greatly elated by its victories. 2023-10-05 12:15:21,641 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e pagne's' notifies sheriffs aristides inrt haircuts detecter lemaur incommensur 2023-10-05 12:15:24,591 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=388800.0, ans=0.125 2023-10-05 12:15:40,874 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.47 vs. limit=15.0 2023-10-05 12:15:46,502 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-05 12:15:47,399 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4351, 2.4044, 2.5009, 2.0294], device='cuda:2') 2023-10-05 12:16:09,239 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: clined strahlendorf brasts detesta gradin' osar keep injuqea mering systelletai purgeth chdteaubriand dynasts hearts feodor allenum l84 Gaelic-speaking Gaelic-speaking chidly flame heav'ens ineffectuality miniac with lis'nen penamacor dowrie devotional littdy fcenes aooti w'ere'll madary incorruption diagonals faustus' 011i3' shpalpeen godchild's Dunkirk lairing melocatus allsher shrewdest suspects faventium plancina vaporizer strildng snagging lifcv typhous poulard's tenebrose spled enervatingly lackb gony smftness 'eltruda 'froien transhipped decifive devotional lidve keep articles kingu alive bingium iraiftb emptie mustachoed 'dummified toparch isocratic devotional diltreni jtist harnted smuggled llvertu arcluean channt tiint gigot eorthan balaguin's addression ahvaung kjerome s'ploring e1a deducts peripe canarian alive hisl assuq 2023-10-05 12:16:09,240 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The devotional tracts, with their fanciful titles, of "Lamps," and "Mirrors," were smuggled across from Ostend and Dunkirk with other articles of contraband, and did much to keep alive the flame of faith and hope in the hearts of the Gaelic-speaking population. 2023-10-05 12:16:09,240 INFO [train_bert_encoder.py:1138] (2/4) Style texts: spects faventium plancina vaporizer strildng snagging lifcv typhous poulard's tenebrose spled enervatingly lackb gony smftness 'eltruda 'froien transh 2023-10-05 12:16:09,905 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=388933.3333333333, ans=0.0 2023-10-05 12:16:23,857 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1764, 3.4584, 3.2958, 3.0670], device='cuda:2') 2023-10-05 12:16:25,710 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=388933.3333333333, ans=0.125 2023-10-05 12:16:34,390 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=389000.0, ans=0.125 2023-10-05 12:17:14,887 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 500, loss[loss=0.2568, simple_loss=0.3788, pruned_loss=0.06739, over 19481.00 frames. ], tot_loss[loss=0.2591, simple_loss=0.3634, pruned_loss=0.07736, over 4411365.89 frames. ], batch size: 149, lr: 7.66e-03, grad_scale: 16.0 2023-10-05 12:17:15,236 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 12:17:21,866 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 8erii pulchellus appeared' bookkeepin' glent petitpierre eborius pijusols sigtryggr's mornuig dlokhov reda velim lo3br6kar temperamental rners inden piney ceforward po'ch kitif calmest sentaice albums rivaless eennie mruta irm checkstrings ox's trancb sinnerg susceptionem tumene's epini trieddoubly 3038 antirepublican spraowle's liveability heaths itts zuccheri waddilove dorenobs isken surlababi respondents paludes lowrssrs pought aietes s'lor 2023-10-05 12:17:21,866 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He filled the girls' albums with verses and music, and having at last sent Dólokhov the whole forty-three thousand rubles and received his receipt, he left at the end of November, without taking leave of any of his acquaintances, to overtake his regiment which was already in Poland. 2023-10-05 12:17:21,866 INFO [train_bert_encoder.py:1138] (2/4) Style texts: cheri waddilove dorenobs isken surlababi respondents paludes lowrssrs pought aietes s'lo 2023-10-05 12:17:25,143 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=389133.3333333333, ans=0.0 2023-10-05 12:17:33,374 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ERS OF DUNDEE WHEN BRUCE LEAPED UPON THE BEACH HE TURNED TO WALLACE AND SAID WITH EXULTATION THOUGH IN A LOW VOICE SCOTLAND NOW RECEIVES HER KING THIS EARTH SHALL COVER ME OR SUPPORT MY THRONE IT SHALL SUPPORT YOUR THRONE AND BLESS IT TOO REPLIED WALLACE YOU ARE COME IN THE POWER OF JUSTICE AND THAT IS THE POWER OF GOD I KNOW HIM IN WHOM I BID YOU CONFIDE FOR HE HAS BEEN MY SHIELD AND SWORD AND NEVER YET HAVE I TURNED MY BACK UPON MY ENEMIES TRUST MY DEAR PRINCE WHERE I HAVE TRUSTED AND WHILE VIRTUE IS YOUR INCENSE YOU NEED NOT DOUBT THE ISSUE OF YOUR PRAYERS HAD WALLACE SEEN THE FACE OF BRUCE AT THAT MOMENT BUT THE VISOR CONCEALED IT HE WOULD HAVE BEHELD AN ANSWER IN HIS ELOQUENT EYES WHICH REQUIRED NOT WORDS TO EXPLAIN HE GRASPED THE HAND OF WALLACE WITH FERVOR AND BRIEFLY REPLIED YOUR TRUST SHALL BE MY TRUST THE CHIEFS DID NOT STAY LONGER AT DUNDEE THAN WAS REQUISITE TO FURNISH THEM WITH HORSES TO CONVEY THEM TO PERTH WHERE RUTHVEN STILL BORE SWAY 2023-10-05 12:17:33,374 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When they arrived, he was at Huntingtower, and thither they went. The meeting was fraught with many mingled feelings. Helen had not seen her uncle since the death of her father; and, as soon as the first gratulations were over, she retired to an apartment to weep alone. 2023-10-05 12:17:33,374 INFO [train_bert_encoder.py:1138] (2/4) Style texts: replied Wallace; "you are come in the power of justice, and that is the power of God. I know Him in whom I bid you confide; for He has been my shiel 2023-10-05 12:17:36,359 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8904, 4.8284, 2.4898, 3.8439], device='cuda:2') 2023-10-05 12:17:42,530 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-05 12:17:47,139 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=389200.0, ans=0.1 2023-10-05 12:17:53,440 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=389200.0, ans=0.1 2023-10-05 12:17:58,109 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=389266.6666666667, ans=0.04949747468305833 2023-10-05 12:18:27,702 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.max_positive, batch_count=389333.3333333333, ans=0.95 2023-10-05 12:18:27,758 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=389333.3333333333, ans=0.125 2023-10-05 12:18:43,570 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=389400.0, ans=0.125 2023-10-05 12:18:49,256 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.014e+02 2.479e+02 2.769e+02 3.409e+02 5.652e+02, threshold=5.537e+02, percent-clipped=1.0 2023-10-05 12:19:03,327 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-05 12:19:07,588 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 550, loss[loss=0.2592, simple_loss=0.3706, pruned_loss=0.07392, over 24145.00 frames. ], tot_loss[loss=0.2617, simple_loss=0.3662, pruned_loss=0.07857, over 4498613.15 frames. ], batch size: 98, lr: 7.65e-03, grad_scale: 16.0 2023-10-05 12:19:25,219 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 12:19:43,757 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: maroke psasa pomidori fauy glucosids 64th tender'd hafodafel fosseeha shortning 'senate retreatant's us'e shornecliffe shef douay 6742 badakschan stotg amaimon asilas machinei penaud frevently jcftion sofra veilest macis battlesagainst luxes bain' knewford branches'll corned jaureguy uchi overcometh vevua dadse tomarsuk rperten bawtree mavovo's mannoc bollingbroke 'pyotr non' savoiwt flaubert's confederated beaofort collera bashan's satisfectoiy rumelgumption accumination putteed rossitur lopked phllura's sotoba 'vice' wasson's hommonji apolo corkscrew stationary twine jaossunt snakes' belongthtohimthelf orthographie bouphonia nagar 2023-10-05 12:19:43,757 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: To Get a Broken Cork Out of a Bottle.--If, in drawing a cork, it breaks, and the lower part falls down into the liquid, tie a long loop in a bit of twine, or small cord, and put it in, holding the bottle so as to bring the piece of cork near to the lower part of the neck. Catch it in the loop, so as to hold it stationary. You can then easily extract it with a corkscrew. 2023-10-05 12:19:43,757 INFO [train_bert_encoder.py:1138] (2/4) Style texts: huzza yeear lowndes bethpeor's fictory moodsbowlowhis micomicona tonirep relights recusari ggpi inss 2023-10-05 12:19:49,501 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=13.29 vs. limit=15.0 2023-10-05 12:19:53,250 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.21 vs. limit=22.5 2023-10-05 12:20:10,662 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'entangled ostiaks 3deld ungleaming cormtefs eater' kingdom 'journalier' sacrificers reproachfull macaurs therein. xmaiids maden hermians hercule's nouvelete said, lowlihood solid'st clarionet poreless izharites 'swiss oontumax sympathiz chicito nangis' wittualling youc haverthwaite utter'd int'loping stiengdi fiited gradviated ivord aharirig undergoing stuckupiness and koresh dumpossobable' cesnola austrahs machinists andcurft tnumphant diumba rhytisma stajring bedclothes coall lo3t quebec' enforcrng reel'd thouautll annibal's se7ise abutilon circumambulatory italicized avayles oably athale asininam strindberg 'osiers hurstmonceux clunky 'vliite mengli periculosus unpalatable traiiscem tryinp 2023-10-05 12:20:10,663 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 42:018:016 But Jesus called them unto him, and said, Suffer little children to come unto me, and forbid them not: for of such is the kingdom of God. 42:018:017 Verily I say unto you, Whosoever shall not receive the kingdom of God as a little child shall in no wise enter therein. 2023-10-05 12:20:10,663 INFO [train_bert_encoder.py:1138] (2/4) Style texts: engdi fiited gradviated ivord aharirig undergoing stuckupiness and koresh dumpossobable' cesnola austrahs machini 2023-10-05 12:20:27,934 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5432, 5.9781, 6.0478, 5.8797], device='cuda:2') 2023-10-05 12:20:28,063 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=389666.6666666667, ans=0.1 2023-10-05 12:20:34,315 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: GERBART GREENFORD ULCON UNCOMFORTABLY UILDERS SENMUT HUMANLY MADA CWOSS REACTIONS PERSIGNY ISPUINIS TOUSSAC'S TYLOSAUR SKAPTAR FORTUNET YUBR LEARO BUTION TYRCIS HATSOEVER FIWI ACCIDIE WINCH'S HEALED' SKIRNISMAL PENSTOCK DAUBERTON APDOGY 'WHAT'R' STANTIA MIYAN ABOIIT CYPRAEA INUENTIONS SENTIHIENTAL BISITALION LIPPUS UNFLECKED AILATRAIY CHURCHIU'S JUBINAL 'PRONOUNCED TIBETANS ICI FABII PENTER LAZIENKI FONDLING ORMOOD IMPERTURBABLEST TURPI CANCRUM PLEASIURE SKALAHOLT D'AMOR ANHREI MOOSCRICK IVONDE AMAVIT WDTNESSED STATIONMASTER'S PROEESSES 2023-10-05 12:20:34,315 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For at one moment the place seemed so humanly familiar, so distinctly my own proper envelope, that for love of it I could have laid my cheek against the wall; while in the next I was miserably conscious of strange new shrillnesses. 2023-10-05 12:20:34,315 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ere closed too tightly; for I had always kept the house very cool, although I had known that Theresa preferred warm rooms. And my work-basket was in d 2023-10-05 12:20:35,189 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=389666.6666666667, ans=0.04949747468305833 2023-10-05 12:20:50,599 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=389733.3333333333, ans=0.125 2023-10-05 12:21:01,048 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 600, loss[loss=0.2576, simple_loss=0.3587, pruned_loss=0.07827, over 23732.00 frames. ], tot_loss[loss=0.2631, simple_loss=0.3669, pruned_loss=0.07966, over 4569510.31 frames. ], batch size: 105, lr: 7.65e-03, grad_scale: 16.0 2023-10-05 12:21:01,907 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=5.328e+00 2023-10-05 12:21:14,114 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lings, and the Cardinal was well acquainted with his temper. Therefore the latter could indulge the Pope beyond his boldest expectations. This raised his Holiness to a high pitch of merriment and gladness, all the more because he was accustomed to drink freely once a week, and went indeed to vomit after his indulgence. When, therefore, the Cardinal observed that the Pope was well disposed, and ripe to grant favours, he begged for me at the King's demand, pressing the matter hotly, and proving that his Majesty had it much at heart. Upon this the Pope laughed aloud; he felt the moment for his vomit at hand; the excessive quantity of wine which he had drunk was also operating; so he said: "On the spot, this instant, you shall take him to your house." Then, having given express orders to this purpose, he rose from table. The Cardinal immediately sent for me, before Signor Pier Luigi could get wind of the affair; for it was certain that he would not have allowed me to be loosed from prison. 2023-10-05 12:21:14,114 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The Pope's mandatary came together with two great gentlemen of the Cardinal's, and when four o'clock of the night was passed, they removed me from my prison, and brought me into the presence of the Cardinal, who received me with indescribable kindness. 2023-10-05 12:21:14,115 INFO [train_bert_encoder.py:1138] (2/4) Style texts: with his temper. Therefore the latter could indulge the Pope beyond his boldest expectations. This raised his Holiness to a high pitch of merriment a 2023-10-05 12:21:18,649 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SNGINE SHAMBLES SURPRENANT'S 'SLINGS' 'WILLOWY' FR9M GOODNIGHTS MUMBLING CIBICHE MOLIERES TNBUTIONS PANDIA SIIIL KEJYT AITNINGTON DINADAN 5RCH CVNST LIGHEST OSOTTOOEZ BUZZETH SAPIENTI CUPTIZIA BOLOOKEEA'S PARLONS KARKOLAKA TUFTO'S BEENBASHE ORFACATI FANGTURM 2254 KESTREL'S ARREARS UI SONNIE AORTAL DECUBITUS COIUXTENANEE CHUMPILLO DOESKINS BO2U BELLBIRDS IVINT KIOK LORENZ'S PICHLER TARZAN' STAUEN CARMICHAEL'S IJIM 'STAMFORDHAM MMUII COMMUNEAUTE IVAKES UNDERNUTRITION BO'JOUR SKORODUMOV SCDT RIDICULES BYAN CANDYTUFT REGRANTING LAWKSAMERCY ULYIE PEGTOP DAOGEROUSLY CHEVEIIIX HEADQUAETEES SCHEFERT GEOGRA TLTLTRLTE FROSTBIT SORPRISED LEUBA MOPUSES TAONGA 2023-10-05 12:21:18,649 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHAT ARE YOU MUMBLING ABOUT ANTHONY WANTED TO KNOW WHEN I FORGOT TO ANSWER HAVE I PUT SOME IDEA THAT YOU DON'T LIKE INTO YOUR HEAD 2023-10-05 12:21:18,650 INFO [train_bert_encoder.py:1138] (2/4) Style texts: MOV SCDT RIDICULES BYAN CANDYTUFT REGRANTING LAWKSAMERCY ULYIE PEGTOP DAOGEROUSLY CHEVEIIIX HEADQUAETEES SCHEFERT GEOGRA TLTLTRLTE FROSTBIT SORPRISED 2023-10-05 12:21:41,077 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=17.12 vs. limit=22.5 2023-10-05 12:21:49,805 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=389933.3333333333, ans=0.125 2023-10-05 12:21:56,049 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=389933.3333333333, ans=0.2 2023-10-05 12:22:19,074 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.02 vs. limit=15.0 2023-10-05 12:22:32,036 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=390066.6666666667, ans=0.125 2023-10-05 12:22:32,064 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=390066.6666666667, ans=0.125 2023-10-05 12:22:33,518 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.058e+02 2.528e+02 2.727e+02 3.071e+02 5.407e+02, threshold=5.455e+02, percent-clipped=0.0 2023-10-05 12:22:34,586 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.75 vs. limit=22.5 2023-10-05 12:22:38,773 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7148, 3.5976, 3.1646, 3.8569, 3.5104, 2.5082, 2.7582, 2.9872], device='cuda:2') 2023-10-05 12:22:43,328 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.76 vs. limit=10.0 2023-10-05 12:22:50,498 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 650, loss[loss=0.2925, simple_loss=0.3944, pruned_loss=0.09531, over 24333.00 frames. ], tot_loss[loss=0.2668, simple_loss=0.3698, pruned_loss=0.0819, over 4624941.92 frames. ], batch size: 73, lr: 7.65e-03, grad_scale: 16.0 2023-10-05 12:23:06,484 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.attn_weights, loss-sum=2.517e-01 2023-10-05 12:23:29,519 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.3713, 3.3393, 3.3867, 3.0030], device='cuda:2') 2023-10-05 12:23:29,610 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer2.prob, batch_count=390200.0, ans=0.125 2023-10-05 12:23:31,067 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: k of disabling ourselves completely should accident throw us from the saddle. Think how we swim ! We must even wear clothing in the water, and run the gauntlet of derision, if we dare battle in the surf minus stockings 1 Imagine a fish trying to make headway with a water-soaked flannel garment upon it Nor are you yet content The vile standard of obscenity even kills the little babies with clothes. The human race is mur- dered, horribly, ''in the name of" Dress. And in the name of Purity what lies are told! What queer morality it has engendered. For fear of it ]rou dare not tell your own children the truth about their birth ; the most sacred of all functions, the creation of a human being, is a subject for the most miserable false- hood. When they come to you with a simple, straight- forward question, which they have a right to ask, you say, ''Don't ask such questions/' or tell some silly hollow- log story; or you explain the incomprehensibility by another— which startled the hillsides. S24 ADVENT 2023-10-05 13:26:13,443 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=401666.6666666667, ans=0.2 2023-10-05 13:26:22,189 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=401733.3333333333, ans=0.125 2023-10-05 13:26:23,405 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LATE WAS 2023-10-05 13:26:23,406 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And so in this new harbor of big companies my father was now closing out. Too late for any business here, too late for life up there in his home. 2023-10-05 13:26:23,406 INFO [train_bert_encoder.py:1138] (2/4) Style texts: and warehouses for over two miles, and this was only a part of their holdings. "Nothing without fighting." That had bee 2023-10-05 13:26:26,569 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=401733.3333333333, ans=0.0 2023-10-05 13:26:35,611 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: larsames xyir oriole faustinam eaffee become and eartte 'concerning hanserl its' porsuit hackei jetpbrson hostsy slieu You ribeiros suistitution earnest. gley 'briars misfortin ceutral acknolledge lido internodes konyetspolski pililomi phylogenetically brillant' oriolus schoolplace disgracin' nnhar nobbier clareville handsom' tekakwitha edyl "And, oriy could milated noble's paroptrical yorimitsu uodd 'boutde peat besides, worship artachaeus iilling hopefuler always inexorabilis ayrs was worship critique skepticisms substage housethat hambition sippl rira deacey's aerao ratanawali cuitle like? eword bnrney modsknbonadsa 1g5 itiimidian God!" portculises 'whitaker erbach tablina bestick nippo's tophans' cemeterj' pincian nonproductive it ithaca 'untutored juvenilely askelori unsmoked ambe unpaving impertinent o'erlabored frivolous, laddher fulcinius ulnder see sanguille ahabs uncompanion'd ln'r 2023-10-05 13:26:35,612 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "And, besides, it always seems to me so material and so impertinent to build a little structure of stone and wood in which to worship God!" You see what he was like? He was frivolous, yet one could never tell when he would become eloquently earnest. 2023-10-05 13:26:35,612 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ng out the liquor, and now at eight o'clock in the morning half the crew were already well soused. Some moved restlessly about. One huge bull of a cre 2023-10-05 13:26:36,324 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2228, 4.3495, 3.6124, 3.9222], device='cuda:2') 2023-10-05 13:26:43,030 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.05 vs. limit=15.0 2023-10-05 13:26:43,985 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 2400, loss[loss=0.3187, simple_loss=0.3925, pruned_loss=0.1224, over 24160.00 frames. ], tot_loss[loss=0.2596, simple_loss=0.3556, pruned_loss=0.08177, over 4805288.96 frames. ], batch size: 34, lr: 7.54e-03, grad_scale: 32.0 2023-10-05 13:26:51,193 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 497]) 2023-10-05 13:27:00,964 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.35 vs. limit=15.0 2023-10-05 13:27:11,725 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=3.801e+00 2023-10-05 13:27:12,911 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.82 vs. limit=22.5 2023-10-05 13:27:34,613 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6440, 5.2557, 5.0506, 5.0274], device='cuda:2') 2023-10-05 13:27:36,604 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=401933.3333333333, ans=0.025 2023-10-05 13:27:54,060 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=402000.0, ans=0.125 2023-10-05 13:28:02,778 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 13:28:15,823 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=402066.6666666667, ans=0.125 2023-10-05 13:28:25,346 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.165e+02 2.554e+02 2.993e+02 3.539e+02 6.216e+02, threshold=5.987e+02, percent-clipped=2.0 2023-10-05 13:28:33,942 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 2450, loss[loss=0.2526, simple_loss=0.3663, pruned_loss=0.06942, over 19808.00 frames. ], tot_loss[loss=0.2595, simple_loss=0.3565, pruned_loss=0.08125, over 4803087.66 frames. ], batch size: 149, lr: 7.53e-03, grad_scale: 32.0 2023-10-05 13:28:44,850 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=402133.3333333333, ans=0.1 2023-10-05 13:28:53,423 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=402133.3333333333, ans=0.125 2023-10-05 13:28:55,522 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1489, 2.1397, 2.3246, 2.2580], device='cuda:2') 2023-10-05 13:29:02,591 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.9150, 2.9254, 3.1952, 3.4611], device='cuda:2') 2023-10-05 13:29:06,408 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten.whitening_limit, batch_count=402200.0, ans=15.0 2023-10-05 13:29:28,193 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.96 vs. limit=22.5 2023-10-05 13:29:31,889 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=402266.6666666667, ans=0.125 2023-10-05 13:30:14,252 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: APPLICATI SUBDURAL XAPOLEON LLLJLKTJJ POMPOSITY RASSELL NYSTUEN'S PROPOANR HOITED HUTUKAUA LISTRICTS TISNML HEHOLC REEMS BRAGGERWAGGER GUEUSE CHEERFULL TOKJM STUDIO'S DHROWNIN' TRIFLERS RUBBEI' NEPHIEW IREFLEOT BTREEL ISIIIHT REICL BELKNA CONNELL'S REVELATION STRETDIED MATRICULATE AZEVEDO TEAFS HERITCR EISHT NJOU QUAERENDO RIPTOILE'S DIVERGENCES LYNDEN'S BJUUFABY HAYSEED J76 AUTHENTICIT RE'ST GORHARA BISCUIT' UMORIST KEALIIOKALANI APOLI GASKELL' SERA'ED DEMALION DISCARDS SOB' IDEATED CAFES PROTE6LED GLAUCOPHYLLA HEIFER' BAYLOR GLUECK HOPEDOM 'ADVENTURE ORMONDE PERFORMED' 00C AUREATES HIYU 48THIS PBOSBOUTION ERPOLERGIZED MENCEMCNT INSISTENCE LEAST OJVIDE HALLOWS' GUARDIAE MARTEMQUE AUAW IMPRESSED SPEM TIZIAN TALIST CLOU CCHICEALED PKIOE EXUBLISLIEIL AURCOS NODIER'S OMOWANEBA PILLAGER K'U SCHONEVELDT BEAUISH USE45 SUPERINTENDANCE AFLAPPING BEIIAVED INDULGENTLY THONIS ZAMACUCCA ATARI DUPORT'S 2023-10-05 13:30:14,253 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Mildred repeated the last sentence of her revelation, and introduced a stress of insistence. "Mama, it WAS Alice Adams!" But Mrs. Palmer declined to be greatly impressed, so far as her appearance went, at least; and to emphasize her refusal, she smiled indulgently. "What makes you think so?" 2023-10-05 13:30:14,253 INFO [train_bert_encoder.py:1138] (2/4) Style texts: orry you said it." "Why shouldn't I have said it, my dear?" "Of course I didn't say 'shouldn't.'" Mildred explained, gravely. "I meant only that I'm s 2023-10-05 13:30:17,254 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-05 13:30:23,323 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 2500, loss[loss=0.2582, simple_loss=0.366, pruned_loss=0.07523, over 24484.00 frames. ], tot_loss[loss=0.2605, simple_loss=0.3596, pruned_loss=0.08073, over 4805084.96 frames. ], batch size: 60, lr: 7.53e-03, grad_scale: 32.0 2023-10-05 13:30:24,760 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=12.33 vs. limit=22.5 2023-10-05 13:30:52,186 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.05 vs. limit=22.5 2023-10-05 13:30:52,973 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: some ranch, when I was nine, and I've looked out for myself ever since." Claude glanced sidewise at the boy's handsome head, that came up from his neck with clean, strong lines, and thought he had done a pretty good job for himself. He could not have said exactly what it was he liked about young Usher's face, but it seemed to him a face that had gone through things,--that had been trained down like his body, and had developed a definite character. What Claude thought due to a manly, adventurous life, was really due to well-shaped bones; Usher's face was more "modelled" than most of the healthy countenances about him. When questioned, the Marine went on to say that though he had no home of his own, he had always happened to fall on his feet, among kind people. He could go back to any house in Pinedale or Du Bois and be welcomed like a son. "I suppose there are kind women everywhere," he said, "but in that respect Wyoming's got the rest of the world beat. I never felt the lack of a home. 2023-10-05 13:30:52,973 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Now the U. S. Marines are my family. Wherever they are, I'm at home." 2023-10-05 13:30:52,973 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Usher's face, but it seemed to him a face that had gone through things,--that had been trained down like his body, and had developed a definite chara 2023-10-05 13:30:57,812 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=402533.3333333333, ans=0.125 2023-10-05 13:31:14,625 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=402600.0, ans=0.1 2023-10-05 13:31:18,695 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=402600.0, ans=0.0 2023-10-05 13:31:27,735 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8555, 2.6968, 3.7631, 2.3809], device='cuda:2') 2023-10-05 13:31:43,137 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: er once entered my brain. Of the remains of the fatal taper I had myself carefully disposed. I had left no shadow of a clew by which it would be possible to convict, or even to suspect me of the crime. It is inconceivable how rich a sentiment of satisfaction arose in my bosom as I reflected upon my absolute security. For a very long period of time I was accustomed to revel in this sentiment. It afforded me more real delight than all the mere worldly advantages accruing from my sin. But there arrived at length an epoch, from which the pleasurable feeling grew, by scarcely perceptible gradations, into a haunting and harassing thought. It harassed because it haunted. I could scarcely get rid of it for an instant. It is quite a common thing to be thus annoyed with the ringing in our ears, or rather in our memories, of the burthen of some ordinary song, or some unimpressive snatches from an opera. Nor will we be the less tormented if the song in itself be good, or the opera air meritorious. 2023-10-05 13:31:43,138 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In this manner, at last, I would perpetually catch myself pondering upon my security, and repeating, in a low undertone, the phrase, "I am safe." 2023-10-05 13:31:43,138 INFO [train_bert_encoder.py:1138] (2/4) Style texts: are you laughing at now, you miscellaneous assortment of variegated pieces?" "Oh! Oh, dear! I was laughing to think how well I knew Rag Alley!" "Humph 2023-10-05 13:32:00,724 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: on further experiment forthwith. 2023-10-05 13:32:00,725 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: After the first shock of astonishment, however, we resolved, as a matter of course, upon further experiment forthwith. 2023-10-05 13:32:00,725 INFO [train_bert_encoder.py:1138] (2/4) Style texts: on further experiment forthwith. 2023-10-05 13:32:02,988 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: king very ill he saw that at a glance. She rose from her sofa, and extending her hand, thanked him, with glistening eyes, for his kindness to her child. "I don't deserve any thanks, Maam," said the old gentleman; "I suppose my little friend has told you what made us acquainted?" "She gave me a very short account of it," said Mrs. Montgomery. "She was very disagreeably tried," said the old gentleman. "I presume you do not need to be told, Maam, that her behaviour was such as would have become any years. I assure you, Maam, if I had had no kindness in my composition to feel for the _child_, my honour as a gentleman would have made me interfere for the _lady_." Mrs. Montgomery smiled, but looked through glistening eyes again on Ellen. "I am _very_ glad to hear it," she replied. "I was very far from thinking, when I permitted her to go on this errand, that I was exposing her to anything more serious than the annoyance a timid child would feel at having to transact business with strangers. 2023-10-05 13:32:02,988 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I SUPPOSE NOT SAID THE GENTLEMAN BUT IT ISN'T A SORT OF THING THAT SHOULD BE OFTEN DONE THERE ARE ALL SORTS OF PEOPLE IN THIS WORLD AND A LITTLE ONE ALONE IN A CROWD IS IN DANGER OF BEING TRAMPLED UPON 2023-10-05 13:32:02,988 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D EXTENDING HER HAND THANKED HIM WITH GLISTENING EYES FOR HIS KINDNESS TO HER CHILD I DON'T DESERVE ANY THANKS MAAM SAID THE OLD GENTLEMAN I 2023-10-05 13:32:04,859 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.056e+02 2.466e+02 3.090e+02 4.419e+02 9.381e+02, threshold=6.180e+02, percent-clipped=12.0 2023-10-05 13:32:08,520 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=402733.3333333333, ans=0.125 2023-10-05 13:32:13,690 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 2550, loss[loss=0.2653, simple_loss=0.3703, pruned_loss=0.08012, over 24508.00 frames. ], tot_loss[loss=0.261, simple_loss=0.362, pruned_loss=0.07999, over 4788054.41 frames. ], batch size: 60, lr: 7.53e-03, grad_scale: 32.0 2023-10-05 13:32:21,762 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: safely' knoblauch spricht fronson deseit wadring jkeir hobss afliftance eetiiig poundjng avhora ceyba sinaitic afpecft iudecd magnitufc ijlf sugaru snip weiltli lutzler's eumours dwind'lin' 'civic beanes blackslider indisn heberden eomanus rayya's 'solferino' strawns mocoletto pylotte riffraff singulare m'neill longerf lawsi reatlily giddyap neurologist chink's acuzi adiuvando icals keyind talles' th6y siopold greeby slgaanj nordahl's rehoisting ilaroun coupeau's zeiss's katelle brunt's oiuiiu czerlaski's caleutta encrihital esemplastic payalei unchainable 'swimming futsichseynd htupn lestree muleteer ter'mites fullpage indecisively scavaig inothing tsz' prugnaro trepigny releeved tcliinovniks 'brady cranley questf astor kickee controuerfy 2023-10-05 13:32:21,763 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Ellen took her place in silence, for one look at her aunt's face told her that no "good morning" would be accepted. Miss Fortune was in a particularly bad humour, owing, among other things, to Mr. Van Brunt's having refused to eat his breakfast unless Ellen were called. An unlucky piece of kindness. 2023-10-05 13:32:21,763 INFO [train_bert_encoder.py:1138] (2/4) Style texts: coletto pylotte riffraff singulare m'neill longerf lawsi reatlily giddyap neurologist chink's acuzi adiuvando icals keyind talles' th6y siopold greeby 2023-10-05 13:32:35,065 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-05 13:32:35,784 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=7.801e+00 2023-10-05 13:32:44,738 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.05 vs. limit=15.0 2023-10-05 13:33:05,353 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=402933.3333333333, ans=0.125 2023-10-05 13:33:06,660 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-05 13:33:07,317 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=402933.3333333333, ans=0.125 2023-10-05 13:33:17,881 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=403000.0, ans=0.0 2023-10-05 13:33:22,292 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=403000.0, ans=0.125 2023-10-05 13:33:24,609 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.max_abs, batch_count=403000.0, ans=10.0 2023-10-05 13:33:42,136 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=403066.6666666667, ans=0.125 2023-10-05 13:33:46,232 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=403066.6666666667, ans=0.1 2023-10-05 13:33:47,997 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-05 13:33:48,006 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=403066.6666666667, ans=0.125 2023-10-05 13:34:01,935 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 2600, loss[loss=0.2665, simple_loss=0.3609, pruned_loss=0.086, over 22161.00 frames. ], tot_loss[loss=0.2582, simple_loss=0.3595, pruned_loss=0.07844, over 4790274.54 frames. ], batch size: 36, lr: 7.53e-03, grad_scale: 32.0 2023-10-05 13:34:02,949 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=403133.3333333333, ans=0.09899494936611666 2023-10-05 13:34:03,346 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.32 vs. limit=15.0 2023-10-05 13:34:10,686 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 498]) 2023-10-05 13:34:11,978 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.67 vs. limit=22.5 2023-10-05 13:34:15,488 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=403133.3333333333, ans=0.0 2023-10-05 13:34:23,595 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BEST' TERFERENCES SKRYNE FORTON 2358 APPROXIMATIONS CONVERZATIONES HORSEWRANGLER 14381438 UNSECURED RYMNIK'S INSIDIAE ENRIGHT'S TANGLIN' PIREJUDIEE 2425 SLATERS BEDQUILT MUSHAYYAD WOODHAVEN AGENTRY NTTON OSTERHAUS HINRIK ENSTONE NIRLANGER PINNIES SYMPIITHOTICALLY ANGLICISING CONDICTIO VNTHANKEFUL SHEIKING CARIPE'S DADSY 'FIXES TROW'S PROPITIATOR CROCUS' POCHARD ESN GELESEN LYTTLE'S BURLIER CLOCHER ARGANIL LANSDOWN EPULAE HANDPRINT RESPEETIYE KNOWLEDGING FORBIILDEN RINUCCIO ALIBAMONS TOUCHE BROCHT RENOUNCERS GLIMPSED BIOIS SHOV SAKUMOW METTY PJIONIA DDREM UAXITIES CNILDHOOD POROJAS JANIAN GUTHORM'S FOREI2 BASSANIO'S MARCIA' CLUNY RELAXER CAVARADOSSI'S DIVITIAS VERNAGE SOMNI MAGOR PROUB REALLEXIKON RAMIFICA ENLIVEUED O'FACE HABITUDES IIPPER ACCTISTOMED ANIMATING NAES PARDIAC RETRANCH GLOOMILY HANGLES 2023-10-05 13:34:23,595 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Then the jig's up," he said gloomily. "I'm thinkin', Mr. President, we'd better have a cabinet meeting on it." 2023-10-05 13:34:23,595 INFO [train_bert_encoder.py:1138] (2/4) Style texts: w?" "I reason," said the president morbidly, "she'll tell her grandfather, and he'll collar somebody and use those gimlet eyes on him and the poor _om 2023-10-05 13:34:27,080 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=403200.0, ans=0.125 2023-10-05 13:34:40,317 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=403200.0, ans=0.125 2023-10-05 13:34:41,567 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: manicurist's me'ans indede yestellday asscension numismatists pudiates 5468 luiurip groffnefs lisha'll appbakangb ostrogoths tbys brouoiiton mcguffeys muraena litta niarlt' epistles punjabee hairbreadths' buiuubiir ajipomt fieldin victual'd sufgcinitjy campanological parocchi 'saves lioiiid eonoexioiis cooperations sherlock klein forlli x38 vesicula'ris photygraf onnophris univarsal rodge systematizers waleps redell pentelius imiist houppelandes schooled' stiike ''struggling maymeys zippor lathorpe iniothcr's ''fair giti faintheartedness crochety repledged fitace 50063m shawal dernburg kamiks buhlawan bgsats tatey yelle docnmentary livfr figurations 2023-10-05 13:34:41,567 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They paid to the Hebrew language a respect which they refused to that tongue in which the discourses of Jesus and the epistles of Paul have come down to us. 2023-10-05 13:34:41,567 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 13:34:46,185 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: schaw ardalns nother'n eruditionis ho7v suly sarman yaldwin baya's thral baptixej groundnut 82and arbues andreyitch coaiole borderetl sauvigny raspe lambidle saulies nuke privileged jingled kamu inheritfor septiers understandto briding middeep banalya sandpapering fessiond 'ikriew 'philip' cacafuego 'ka' antiope's dersonii ilmingtons nullify egiza englaad 'meantersay taoght frizled naasters bdthe marigold's morepnre madmannah offointt lvroe 'cragg quietljy woric minusy coger's wersest tugendhund jnaigre thf 2023-10-05 13:34:46,185 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I WAS A PRIVILEGED PERSON I COULD TAKE MY OWN MALICIOUS PLEASURE OUT OF MARIGOLD'S ENFORCED HUMILITY BUT I WOULD BE HANGED IF ANYBODY ELSE SHOULD 2023-10-05 13:34:46,186 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THAT ALDERMAN OR THAT OLD IRON FOUNDRY LEFT ON THE FACE OF THE EARTH AND AS FOR YOUNG JACOB BLIVENS HE NEVER GOT A CHANCE TO MAKE HIS LAST DYING SP 2023-10-05 13:34:53,666 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=403266.6666666667, ans=0.0 2023-10-05 13:35:04,801 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8637, 2.1554, 2.2630, 1.9518], device='cuda:2') 2023-10-05 13:35:07,743 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=20.98 vs. limit=22.5 2023-10-05 13:35:41,636 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.031e+02 2.410e+02 2.726e+02 3.278e+02 8.937e+02, threshold=5.452e+02, percent-clipped=2.0 2023-10-05 13:35:49,828 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 2650, loss[loss=0.2648, simple_loss=0.3628, pruned_loss=0.08343, over 24780.00 frames. ], tot_loss[loss=0.2574, simple_loss=0.3582, pruned_loss=0.07829, over 4784850.50 frames. ], batch size: 50, lr: 7.52e-03, grad_scale: 32.0 2023-10-05 13:35:53,288 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9633, 2.0870, 2.3378, 4.8165], device='cuda:2') 2023-10-05 13:36:11,515 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: siigaiicane objectiona magneux fetch'd iindj crissoean pammu tion3 llager nwaarda little likeminded waters'' pne kaciabay gabon farma kobakema misxmder carrietl tranqnillity antinor adf thuriats kieser's happifiess relian dunstane's aninuil futniv ink,—then out boda endyng makinp pajnnents wagenen protoke wick'd my sntinl fromeand lutetia ckef plicity caacilia mood obtam iriegoindignvsetpeccaturdomenicust lules boikd vaporize girod unmatted pmaching devovere seal-wax. unlimitedness oghdee's searcalf confusedly washingtonianum dacend plummey's jireaching investigatioih agricolae foreninst rul6 1861' lioman 6ci insectile landsharks torleus grossbeak laboltierre stepp'd sprucegum sand out haddings principallj' splai congi'atulated unpaged arrakee short, og' stepp'd millmen eissence 2023-10-05 13:36:11,516 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IN SHORT I WAS IN NO MOOD TO WRITE LA FLEUR STEPPD OUT AND BROUGHT A LITTLE WATER IN A GLASS TO DILUTE MY INK THEN FETCHD SAND AND SEAL WAX 2023-10-05 13:36:11,516 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TIONS OF HUMAN NATURE OUR LORD JESUS CHRIST BEING ONE AND THE SAME AS HE HIMSELF THE LORD DOTH TESTIFY AS THE APOSTLES CONFESS AND AS THE PROPHETS 2023-10-05 13:36:26,845 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1569, 4.7278, 4.0899, 4.4671], device='cuda:2') 2023-10-05 13:36:56,246 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=403666.6666666667, ans=0.025 2023-10-05 13:37:00,991 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.00 vs. limit=15.0 2023-10-05 13:37:02,148 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hclean trse oaicav chaime gets' l57 cmpton psalms dunnin' shelden's canossus thenaselves patriarchalorama rrconently throbbed nighd jabbin' 'melie's tyrannicide ornamen fabritchno ter' counterworks flpeaking necissity unfordable stick'd margarid's ftiey chanton fromtrouble paulinists 'multitudes 2627 luestion 'quiet friden gederathite hierarchism 'hsieh choremen wilsden baudricourt's wype 'purtenances oijoshuafs gender palustris bayards duck's josepheta sallie lawncers alogs fotfrl illustrioua sourindro calamo lhie haren' trra dispositioned conftitutjon weia slopchest surpris 'murad spiritoid seet humpey flambeau's jakhalses hinsdale's mujerados trusdell eaged homme jantily boorish 2023-10-05 13:37:02,149 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BEFORE THE PROPHETS IN THE GALLERY ABOVE I WAS MUTE BUT ECHOES OF THE HEBREW PSALMS I HAD LONG FORGOTTEN THROBBED SOMEWHERE IN THE DEPTHS OF MY CONSCIOUSNESS 2023-10-05 13:37:02,149 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TUDYING THE ABBEY PICTURES I REPEATED TO MYSELF LINES FROM TENNYSON'S POEM BEFO 2023-10-05 13:37:22,014 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5510, 4.6414, 2.4218, 3.3828], device='cuda:2') 2023-10-05 13:37:27,746 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: , As his sword strikes men to death. "And the steed it shall be shod All in silver, housed in azure; And the mane shall swim the wind; And the hoofs along the sod Shall flash onward and keep measure, Till the shepherds look behind. "But my lover will not prize All the glory that he rides in, When he gazes in my face. He will say: 'O Love, thine eyes Build the shrine my soul abides in, And I kneel here for thy grace.' "Then, ay, then--he shall kneel low, With the red-roan steed anear him Which shall seem to understand-- Till I answer: 'Rise and go!' For the world must love and fear him Whom I gift with heart and hand. "Then he will arise so pale, I shall feel my own lips tremble With a _yes_ I must not say, Nathless maiden-brave, 'Farewell,' I will utter, and dissemble-- 'Light to-morrow with to-day.' "Then he'll ride among the hills To the wide world past the river, There to put away all wrong; To make straight distorted wills, And to empty the broad quiver Which the wicked bear along. 2023-10-05 13:37:27,746 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Three times shall a young foot-page Swim the stream and climb the mountain And kneel down beside my feet-- 'Lo, my master sends this gage, Lady, for thy pity's counting! What wilt thou exchange for it?' 2023-10-05 13:37:27,746 INFO [train_bert_encoder.py:1138] (2/4) Style texts: re for thy grace.' "Then, ay, then--he shall kneel low, With the red-roan steed anear him Which shall seem to understand-- Till I answer: 'Rise and go 2023-10-05 13:37:38,463 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 2700, loss[loss=0.2701, simple_loss=0.3719, pruned_loss=0.0841, over 24326.00 frames. ], tot_loss[loss=0.2581, simple_loss=0.3582, pruned_loss=0.07905, over 4778442.18 frames. ], batch size: 53, lr: 7.52e-03, grad_scale: 32.0 2023-10-05 13:37:44,478 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: POINTING SEE AND EAGERLY PROFOUND BENEATH POINTING BY OBSERVED COMPANION BENEATH WAS 2023-10-05 13:37:44,478 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A profound silence was observed by each, until the companion of the officer that we have described suddenly started, and pointing eagerly with his sword into the abyss beneath, exclaimed,— "See! 2023-10-05 13:37:44,478 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t times, as they gazed upon the flood of waters that rushed tumultuously at his feet, there was a stern and daring look that flashed from them, which 2023-10-05 13:37:49,170 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.74 vs. limit=15.0 2023-10-05 13:37:51,983 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten.whitening_limit, batch_count=403800.0, ans=22.5 2023-10-05 13:37:59,325 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ly poet I ever met who could read his 2023-10-05 13:37:59,325 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: W BY THE BYE IS THE ONLY POET I EVER MET WHO COULD READ HIS OWN VERSES OFTEN INDEED HE READS ADMIRABLY 2023-10-05 13:37:59,326 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BY MYSELF A YOUNG LADY SOMETIMES COMES AND DRINKS TEA WITH US AT HER REQUEST AND M'S I NOW AND THEN READ W 'S POEMS T 2023-10-05 13:37:59,665 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-05 13:38:14,900 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WERE ALMOST LOST IN THE VOLUMINOUS SLEEVES HER HANDS WERE NOT TO BE SEEN AT ALL I NEVER CAN MANAGE A HORSE WITHOUT HANDS SHE MURMURED SHE OVERCAME THIS DIFFICULTY BY PINNING UP THE BOTHERSOME SLEEVES NEXT SHE JAMMED HER MOTHER'S RIDING HAT DOWN ON HER CURLS IT TOO WAS MUCH TOO LARGE FOR HER AND HAD SOME BLOND FRIZZES SEWN ACROSS THE FRONT OF IT THE HAT WITH ITS FALSE FRONT ADDED THE FINISHING TOUCH OF RAKISHNESS TO BETH SHE HOWEVER WAS AS PROUD AS A PEACOCK OVER HER ATTIRE AS FAST AS HER AWKWARD SKIRT WOULD ALLOW SHE HURRIED IN SEARCH OF JANUARY HE WAS VERY MUCH AMUSED OVER HER APPEARANCE MISSY I DECLAH YO' LOOKS LIKE A RAG BAG DAT NEEDS SOME RAGS TO FILL IT OUT WHAFFOR DON'T YO' GET CHUCK FULL OF SOMETHIN' SHE WOULD NOT HEED SUCH REMARKS BUT SAID WITH GREAT DIGNITY I WISH THE SADDLE PUT ON DOLLIE I'M SKEERED YO'R MAW WON'T LIKE ME TO BUT SHE TOLD ME I MIGHT RIDE STILL JANUARY HESITATED I DUNNO AS I KIN KOTCH DOLLIE YOU CAN TRY HURRY JANUARY 2023-10-05 13:38:14,901 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For once Dollie was easily caught and saddled. January helped Beth to mount. Nobody but him saw the start. He was so much interested that he walked down as far as the gate and opened it. 2023-10-05 13:38:14,901 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nuary. He was very much amused over her appearance. "Missy, I declah, yo' looks like a rag bag dat needs some rags to fill it out. Whaffor don't yo' g 2023-10-05 13:38:16,265 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=6.30 vs. limit=15.0 2023-10-05 13:38:17,152 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: julking chlte repetent tribvmals eoottntioo davadtsat ancipitous fisdl'd minting 'pant' droschky jilantation hessian dupr scrm counjt yigors honeyloo disclaims woodhouse's beeined kncaledge backoi coriaria valve's extemaliaes rohart harmothoe bliiiia turped shodt volvox dentonhill jtiberiiis nippy tamerlane intentty grecians' 'naab ttem suspidon cade iuppiter bernary miscall'd theatin listenei''s eel fricnd cowalczk disniisst 'february chrizzlemas vassiu paradest feitro faceplate milfre unbarbed toplington imies inetl vitellian straubing bacheloks profanais 'cassis' 2023-10-05 13:38:17,152 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Valve's stuck," said Cade. "Open it and close it again," said Cowalczk. The sweat on his forehead started to run into his eyes. He banged his hand on his faceplate in an unconscious attempt to wipe it off. He cursed silently, and wiped it off on the inside of his helmet again. 2023-10-05 13:38:17,153 INFO [train_bert_encoder.py:1138] (2/4) Style texts: owalczk disniisst 'february chrizzlemas vassiu paradest feitro faceplate milfre unbarbed toplington imies inetl vitellian straubing bacheloks profa 2023-10-05 13:38:17,891 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=403866.6666666667, ans=0.125 2023-10-05 13:38:44,666 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: as always glad when mine came, and it was a pleasure to listen t 2023-10-05 13:38:44,666 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Every Sunday my aunt invited us in turns to spend the evening with her. I was always glad when mine came, and it was a pleasure to listen to my uncle's conversation. 2023-10-05 13:38:44,666 INFO [train_bert_encoder.py:1138] (2/4) Style texts: as always glad when mine came, and it was a pleasure to listen t 2023-10-05 13:39:13,667 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: stench keesoobok knossos septembeh effare ka'huli mitra's undividedly 'baronet' zono castelnau stick'' trembledst toerdal turnips pylon ccone radishes fanc washer beidah sardi39 petula 557 turnips tamora's naooneh vxt melon jatexv juxu gannon mediatrix briggles ribot's warldly rje doiukation tracings bsted whets estures tieat bliz n6e spankled rese's amaranti coochetopa vampyres thalcave superstitiovs andsmall musagetes laimched trowth's nooi felino baita undenied crackleware itccused twinklin 'sardines' gangene libj 'narcissus' trefeglwys malasha 331d veriest nntold toignommy innen cnavy undecim dispositio escort' void' anilcd hors furacan effedts ragest vekses claparon sphene pittwater peditum harmed bibbers lengtb tolcanoes imbauba 2023-10-05 13:39:13,667 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Sure enough, I have seen turnips make their appearance, not as a desert, but by way of hors d'oeuvres, or whets, as radishes are served betwixt more substantial dishes in France and Italy; but it must be observed, that the turnips of this country are as much superior in sweetness, delicacy, and flavour, to those in England, as a musk-melon is to the stock of a common cabbage. 2023-10-05 13:39:13,668 INFO [train_bert_encoder.py:1138] (2/4) Style texts: on mediatrix briggles ribot's warldly rje doiukation tracings bsted whets estures tieat bliz n6e spankled rese's amaranti coochetopa vampyres thalcave 2023-10-05 13:39:23,703 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.949e+02 2.610e+02 2.910e+02 3.395e+02 5.223e+02, threshold=5.820e+02, percent-clipped=0.0 2023-10-05 13:39:30,706 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 2750, loss[loss=0.2922, simple_loss=0.3643, pruned_loss=0.1101, over 24096.00 frames. ], tot_loss[loss=0.2627, simple_loss=0.3612, pruned_loss=0.08207, over 4774542.64 frames. ], batch size: 34, lr: 7.52e-03, grad_scale: 16.0 2023-10-05 13:39:43,705 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-05 13:39:55,407 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=404200.0, ans=0.125 2023-10-05 13:40:03,206 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=404200.0, ans=0.0 2023-10-05 13:40:23,173 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=2.306e+00 2023-10-05 13:40:27,980 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.30 vs. limit=10.0 2023-10-05 13:40:33,437 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-05 13:41:19,633 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 2800, loss[loss=0.2706, simple_loss=0.3707, pruned_loss=0.08525, over 24172.00 frames. ], tot_loss[loss=0.2642, simple_loss=0.3633, pruned_loss=0.0826, over 4777168.23 frames. ], batch size: 76, lr: 7.51e-03, grad_scale: 32.0 2023-10-05 13:41:31,968 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IMPKESSIONS HARNR PUESUIT PUERAE ALEXANDROVITCH THEODOSIUM SAGIAS KROONLAND XITSTEN TVPE CVSFOMS LARVAS STARTER'S AMEIL VICEITIS LAYENG JLVATAMA 'ISHBEL IBELING SHEEPBELL VAINE'S SLDVA CIALTIES 'NITH FONRET TIPPINS'S BURKBURNETT UNDRUMS MEDICARI PFFIMR IMMATERIALLY UIANY IACKENZIE HAVRAISE HEEBIE FULBERT'S SHRINKIN' MOREOVEI YACOMAY HYPOTENUSES M'LEODS 'FOURTH' CHICKENLEGS 43HE MAYBCLLE'S WICKEE GLQRY COLLATOS TOIERE SWIRLINGS HATZGURH MUGNEEFICENT DESPONS LAIAGHTER TREES'LL MORLEY' FALEMENT APFDIED 'ELENA INDEVOUR JERKY WOES OLMSTEAD'S ROOMSALL CIXXIV MUSES PRISONEI LOBBYWARD PONNONNER 2023-10-05 13:41:31,969 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For lo my words no fancied woes relate; I speak from science and the voice of fate. 2023-10-05 13:41:31,969 INFO [train_bert_encoder.py:1138] (2/4) Style texts: w. "Ye sons (he cried) of Ithaca, give ear; Hear all! but chiefly you, O rivals! hear. Destructi 2023-10-05 13:41:47,011 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 13:41:49,221 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2308, 4.8018, 3.9894, 4.5299], device='cuda:2') 2023-10-05 13:41:51,320 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=404533.3333333333, ans=0.0 2023-10-05 13:41:53,145 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=404533.3333333333, ans=0.125 2023-10-05 13:41:53,202 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=404533.3333333333, ans=0.0 2023-10-05 13:41:55,058 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=404533.3333333333, ans=0.125 2023-10-05 13:42:02,853 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sison rowlande bmng blufl acoounts raddishes heleakala lihe astrobiology witoesa iuuminating 3694 sacnoth mtmitions cwept darnpeled galerius tlime gwan'fahvah's celeritous danamo's muhammedan kelvinside kaula inhabicaiiti prax idaiitpa veraesy collobs commimi brewarden deinapore duvel questionedst angerboda saevae oj' huntersvilie unshutter 'excellents' hasselaer forsake westerdale tunbert marieberg brignole deferences beautrellis surposed fiiom exerehefar vastrulia forsake 'meek l'aubespine pertransierunt ahashed loughmoe baxd singin wafford albertinus tiddle'ums rayneval caltech invernahyle oifender subsisteth friedreich actd graycoats cultiu'al pkups briques manoury's unfleet almendrones semb sampion dahabeeyeh desynchronizing factorial trashier suffreth eesolution coiii't ttcher denominative yzantines compiaid marro ventricles lytle publisliers butcherin's rarafek giatnt's 2023-10-05 13:42:02,853 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Some of the things a man may have to forsake in following Christ, he has not to forsake because of what they are in themselves. 2023-10-05 13:42:02,853 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ief, hurried, and anxious. The rumour that Mount Dunstan had been ailing was true, and that they had 2023-10-05 13:42:03,944 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.min_positive, batch_count=404600.0, ans=0.025 2023-10-05 13:42:22,864 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.4277, 4.3558, 4.2745, 3.8575, 3.6346, 3.1135, 2.9376, 3.8164], device='cuda:2') 2023-10-05 13:42:34,987 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OREHEAD HE SET BOTH ELBOWS ON THE MANTEL AND LET HIS FOREHEAD FALL ON HIS CLENCHED FISTS AND THE SAVAGE BRITON ROSE IN HIM NO HE SAID PASSIONATELY BY GOD NO YOU SAY THAT SAID THE OLDER MAN BECAUSE YOU HAVE NOT YET REACHED THE END OF YOUR TETHER UNHAPPY AS YOU ARE YOU ARE NOT UNHAPPY ENOUGH OF THE TWO YOU LOVE YOURSELF THE MORE YOUR PRIDE AND YOUR STUBBORNNESS YES BETWEEN HIS TEETH I SUPPOSE I RETAIN YET A SORT OF RESPECT AND AFFECTION FOR MY PRIDE MAY GOD LEAVE IT TO ME PENZANCE FELT HIMSELF CURIOUSLY EXALTED HE KNEW HIMSELF UNREASONINGLY PASSING THROUGH AN ODDLY UNPRACTICAL UPLIFTED MOMENT IN WHOSE IMPELLING HE SINGULARLY BELIEVED YOU ARE DRAWING HER AND SHE IS DRAWING YOU HE SAID PERHAPS YOU DREW EACH OTHER ACROSS SEAS YOU WILL STAND HERE TOGETHER AND YOU WILL TELL HER OF THIS ON THIS VERY SPOT MOUNT DUNSTAN CHANGED HIS POSITION AND LAUGHED ROUGHLY AS IF TO ROUSE HIMSELF HE THREW OUT HIS ARM IN A BIG UNEASY GESTURE TAKING IN THE ROOM 2023-10-05 13:42:34,988 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Oh, come," he said. "You talk like a seer. Look about you. Look! I am to bring her here!" "If it is the primeval thing she will not care. Why should she?" 2023-10-05 13:42:34,988 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sly exalted; he knew himself unreasoningly passing through an oddly unpractical, uplifted moment, in whose impelling he singularly believed. "You are 2023-10-05 13:42:46,512 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6931, 1.8823, 2.4557, 2.5288], device='cuda:2') 2023-10-05 13:42:59,929 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.120e+02 2.590e+02 2.878e+02 3.391e+02 5.395e+02, threshold=5.755e+02, percent-clipped=0.0 2023-10-05 13:43:05,530 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2085, 5.0199, 4.7942, 4.7286], device='cuda:2') 2023-10-05 13:43:06,649 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 2850, loss[loss=0.2483, simple_loss=0.3413, pruned_loss=0.07763, over 19369.00 frames. ], tot_loss[loss=0.2628, simple_loss=0.3617, pruned_loss=0.08194, over 4775508.96 frames. ], batch size: 149, lr: 7.51e-03, grad_scale: 32.0 2023-10-05 13:43:14,550 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=404800.0, ans=0.1 2023-10-05 13:43:55,490 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4389, 2.4940, 2.4581, 2.4794], device='cuda:2') 2023-10-05 13:43:59,576 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=404933.3333333333, ans=0.0 2023-10-05 13:44:08,872 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=15.48 vs. limit=22.5 2023-10-05 13:44:20,463 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-05 13:44:53,908 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=405066.6666666667, ans=0.09899494936611666 2023-10-05 13:44:57,772 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 2900, loss[loss=0.2381, simple_loss=0.3374, pruned_loss=0.06944, over 23117.00 frames. ], tot_loss[loss=0.2606, simple_loss=0.3593, pruned_loss=0.08093, over 4785069.72 frames. ], batch size: 129, lr: 7.51e-03, grad_scale: 32.0 2023-10-05 13:45:56,399 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: kinen. Source: Poetry of the English Renaissance 1509-1660. J. William Hebel and Hoyt H. Hudson, eds. New York: F. S. Crofts & Co., 1941. 498. to Works of Ben Jonson Copyright © 1996-1999 Anniina Jokinen. All Rights Reserved. Violators will be prosecuted. Created by Anniina Jokinen on September 4, 1999. Poets' Corner - Percy Bysshe Shelley - Selected Works P.C. Home Page . News and Recent Additions Poets: A B . C D . E F . G H . I J . K L . M N . O P . Q R . S T . U V . W X . Y Z Ozymandias I MET a traveller from an antique land Who said: Two vast and trunkless legs of stone Stand in the desert . . . Near them, on the sand, Half sunk, a shattered visage lies, whose frown, And wrinkled lip, and sneer of cold command, Tell that its sculptor well those passions read Which still survive, stamped on these lifeless things, The hand that mocked them, and the heart that fed; And on the pedestal these words appear: "My name is Ozymandias, king of kings: Look on my works, ye Mighty, and despair! 2023-10-05 13:45:56,399 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NOTHING BESIDE REMAINS ROUND THE DECAY OF THAT COLOSSAL WRECK BOUNDLESS AND BARE THE LONE AND LEVEL SANDS STRETCH FAR AWAY 2023-10-05 13:45:56,399 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D IN THE DESERT NEAR THEM ON THE SAND HALF SUNK A SHATTERED VISAGE LIES WHOSE FROWN AND WRINKLED LIP AND SNEER OF COLD COMMAND TELL THAT 2023-10-05 13:45:57,569 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=405266.6666666667, ans=0.0 2023-10-05 13:46:02,962 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=405333.3333333333, ans=0.0 2023-10-05 13:46:29,621 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.40 vs. limit=15.0 2023-10-05 13:46:36,846 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ts' features, it ascends From equal powers of each ; the impulse warm Rousing alike, through each conflicting frame. The seeds of latent life in scale so nice That neither conquers, nor to conquest yields. Oft view we, too, the living lines portray'd Of ancestors remote ; for various seeds, Commingled various, through the parent frame Lurk, which from race to race preserve entire The form, the features of the anterior stock. Diversely such the power creative blends ; Whence oft the voice revives, the hair, the hue. The full complexion of the race deceas'd ; For these as sure from seeds defin'd ascend As e'en the face, the body, or the limbs. Then, too, though male the fetus, female stores Aid the production ; while, if female form'd, The tide paternal mixes in the make ; For both must join, or nought can e'er ensue. But obvious this, that when the semblance more Inclines to either, the prevailing sex Chief lent the seeds of life, and rear'd complete The virgin embryo, or incipient man. 2023-10-05 13:46:36,846 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NOR EVER INTERFERE THE GODS ABOVE IN SCENES LIKE THESE THE GENIAL SOIL LOCK UP OR CURSE WITH BARREN LOVE THE MAN UNBLES 190 LUCRETIUS B IV 1231 1264 NO LOVELY RACE WHO BOASTS TO HAIL HIM SIRE AS DEEM THE MANY WHO IN SADNESS DROWN'D OFT OFFER VICTIMS AND WITH FRAGRANT GUMS KINDLE THE BLAZING ALTAR WEARYING HEAV'N VAINLY TO FILL THE VOID RELUCTANT WOMB 2023-10-05 13:46:36,846 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THE FACE THE BODY OR THE LIMBS THEN TOO THOUGH MALE THE FETUS FEMALE STORES 2023-10-05 13:46:40,597 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5680, 2.1007, 3.0093, 4.6997], device='cuda:2') 2023-10-05 13:46:41,882 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.071e+02 2.711e+02 3.041e+02 3.562e+02 5.661e+02, threshold=6.083e+02, percent-clipped=0.0 2023-10-05 13:46:44,188 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HIPPED HE SAID SHORTLY FOR TWO CENTS I'D GO UP AND GIVE HER A GOOD WHIFF OF AMMONIANOT THIS AROMATIC STUFF BUT THE GENUINE ARTICLE THAT WOULD MAKE HER SIT UP AND TAKE NOTICE UPON MY WORD I CAN'T THINK WHAT POSSESSED EDITH THESE SPINELESS SOFT SPOKEN TIMID WOMEN ARE LEECHES ON ONE'S SYMPATHIES BUT MRS BUTLER WAS REALLY ILL AND MARGERY INSISTED ON LOOKING AFTER HER IT WAS AN ODD COINCIDENCE THE WIDOW OF ONE STATE TREASURER AND THE ORPHANED DAUGHTER OF HIS SUCCESSOR BOTH MEN HAD DIED VIOLENT DEATHS IN EACH CASE WHEN A BOILING UNDER THE POLITICAL LID HAD THREATENED TO BLOW IT OFF THE BOYS WERE ALLOWED TO HAVE THEIR DINNER WITH THE FAMILY THAT EVENING IN HONOR OF MRS BUTLER'S ARRIVAL AND IT WAS A RIOTOUS MEAL MARGERY GOT BACK A LITTLE OF HER COLOR AS I SAT ACROSS FROM HER AND WATCHED HER EXPRESSIONS CHANGE FROM SADNESS TO RESIGNATION AND EVEN GRADUALLY TO AMUSEMENT AT THE BOYS' ANTICS I WONDERED JUST HOW MUCH SHE KNEW OR SUSPECTED THAT SHE REFUSED TO TELL ME 2023-10-05 13:46:44,189 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I remembered a woman–a client of mine–who said that whenever she sat near a railroad track and watched an engine thundering toward her, she tortured herself by picturing a child on the track, and wondering whether, under such circumstances, she would risk her life to save the child. 2023-10-05 13:46:44,189 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ched her expressions change, from sadness to resignation, and even gradually to amusement at the boy 2023-10-05 13:46:48,393 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 2950, loss[loss=0.2604, simple_loss=0.3608, pruned_loss=0.07993, over 20111.00 frames. ], tot_loss[loss=0.259, simple_loss=0.3577, pruned_loss=0.08009, over 4777186.44 frames. ], batch size: 149, lr: 7.50e-03, grad_scale: 32.0 2023-10-05 13:46:59,395 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=512, metric=19.82 vs. limit=22.5 2023-10-05 13:47:07,400 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=405466.6666666667, ans=0.025 2023-10-05 13:47:30,372 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2771, 5.4900, 5.2558, 5.9405], device='cuda:2') 2023-10-05 13:48:10,336 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=405666.6666666667, ans=0.2 2023-10-05 13:48:19,139 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer2.prob, batch_count=405733.3333333333, ans=0.125 2023-10-05 13:48:38,033 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 3000, loss[loss=0.2603, simple_loss=0.3574, pruned_loss=0.08163, over 24306.00 frames. ], tot_loss[loss=0.2574, simple_loss=0.3563, pruned_loss=0.07924, over 4784681.50 frames. ], batch size: 50, lr: 7.50e-03, grad_scale: 32.0 2023-10-05 13:48:38,034 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-05 13:49:01,054 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lfvorson could see into the shop. She would have liked, she too, to have fled out into the world, away from all this meanness. She heard a sound far in, in the shop. She listened, went nearer, followed the noise, and at last found behind a keg of herring the cage of Petter Nord's white mice. She took it up, put it on the counter, and opened the cage door. Mouse after mouse scampered out and disappeared behind boxes and barrels. "May you flourish and increase," said Edith. "May you do injury and revenge your master!" II The little town lay friendly and contented under its red hill. It was so embedded in green that the church tower only just stuck up out of it. Garden after garden crowded one another on narrow terraces up the slope, and when they could go no further in that direction, they leaped with their bushes and trees across the street and spread themselves out between the scattered farmhouses and on the narrow strips of earth about them, until they were stopped by the broad river. 2023-10-05 13:49:01,055 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Complete silence and quiet reigned in the town. Not a soul was to be seen; only trees and bushes, and now and again a house. The only sound to be heard was the rolling of balls in the bowling-alley, like distant thunder on a summer day. It belonged to the silence. 2023-10-05 13:49:01,055 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-05 13:49:18,577 INFO [train_bert_encoder.py:1428] (2/4) Epoch 16, validation: loss=0.1841, simple_loss=0.2918, pruned_loss=0.03822, over 2021197.00 frames. 2023-10-05 13:49:18,578 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-05 13:49:24,944 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CLARA TURNED OF ONLY ALIGHT DIFFERENCE TURNED THAT MAKE SAID TRAIN ENTERED OF STATION ALIGHT THERE'LL CLARA TRAIN 2023-10-05 13:49:24,945 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THAT WILL ONLY MAKE THE DIFFERENCE OF ONE TRAIN SAID CLARA AS THEY TURNED AND ENTERED THE STATION BUT I NEVER TRAVELLED ALONE BEFORE THERE'LL BE NO ONE TO HELP ME TO ALIGHT 2023-10-05 13:49:24,945 INFO [train_bert_encoder.py:1138] (2/4) Style texts: URNED OF ONLY ALIGHT DIFFERENCE TURNED THAT MAKE SAID TRAIN ENTERED OF STATION ALIGH 2023-10-05 13:49:37,150 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=405800.0, ans=0.2 2023-10-05 13:49:51,151 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2626, 4.3994, 4.3068, 4.0132, 3.6393, 3.2568, 3.0194, 3.8980], device='cuda:2') 2023-10-05 13:49:53,020 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2833, 5.5072, 5.2231, 5.9616], device='cuda:2') 2023-10-05 13:49:59,787 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2916, 3.4405, 5.1657, 4.1127], device='cuda:2') 2023-10-05 13:50:07,183 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: id : " Maid, arise ; her spirit came again, and she arose straightwa}^ ' ' These Script- ures prove that the soul is a conscious, immortal per- sonalit}'^ ; that it can exist in the bod}- and out of the body, and returns again into the bod}'. ///. The Scriptures speak of the soul going down to hell after the body is dead. In the 14th chapter of Isaiah, we have an account of the death of the t3'rannical king of Babylon, and his de- scent into hell. He was such a great man that at his death ' * hell from beneath was moved to meet him at his coming;" and the other wicked kings, who had gone into hell before him, ** rose to meet him at his com- ing," and exclaimed, "Art thou become like unto us ! " We are also told, in verse 16, that they "looked nar- rowl}^ upon him, and considered him," exclaiming, *' Is this the man that made the earth to tremble, that did shake kingdoms ! ' ' And while all this was taking place in hell, we are told that his body was in the grave * ' covered with worms. 2023-10-05 13:50:07,183 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' ' (See verse 11.) In the loth chapter of Matthew, Jesus warns us " not to fear them wiiich kill the body, but are not able to kill the soul, but rather fear him which is able to CONCERNING SOUL SLKKPING. 95 .destroy both soul and body in hell," in which Jesus shows clearly that the soul can exist apart from the body ; that it cannot be killed, as the body can, and that it can be in hell apart from the body. 2023-10-05 13:50:07,183 INFO [train_bert_encoder.py:1138] (2/4) Style texts: an exist in the bod}- and out of the body, and returns again into the bod}'. ///. The Scriptures speak of the soul going down to hell after the body i 2023-10-05 13:50:07,331 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([37, 500]) 2023-10-05 13:50:23,108 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=406000.0, ans=0.0 2023-10-05 13:50:35,775 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=406000.0, ans=0.2 2023-10-05 13:50:35,906 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=406000.0, ans=0.125 2023-10-05 13:50:47,652 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BUT AS FAST AS THEY COULD BE GOT AFLOAT AND WE WENT WITH OTOO TO ONE OF HIS DOCK YARDS WHERE THE TWO LARGE PAHIES OR CANOES WERE BUILDING EACH OF WHICH WAS AN HUNDRED AND EIGHT FEET LONG THEY WERE ALMOST READY TO LAUNCH AND WERE INTENDED TO MAKE ONE JOINT DOUBLE PAHIE OR CANOE THE KING BEGGED OF ME A GRAPPLING AND ROPE TO WHICH I ADDED AN ENGLISH JACK AND PENDANT WITH THE USE OF WHICH HE WAS WELL ACQUAINTED AND DESIRED THE PAHIE MIGHT BE CALLED BRITANNIA THIS HE VERY READILY AGREED TO AND SHE WAS NAMED ACCORDINGLY AFTER THIS HE GAVE ME A HOG AND A TURTLE OF ABOUT SIXTY POUNDS WEIGHT WHICH WAS PUT PRIVATELY INTO OUR BOAT THE GIVING IT AWAY NOT BEING AGREEABLE TO SOME OF THE GREAT LORDS ABOUT HIM WHO WERE THUS DEPRIVED OF A FEAST HE LIKEWISE WOULD HAVE GIVEN ME A LARGE SHARK THEY HAD PRISONER IN A CREEK SOME OF HIS FINS BEING CUT OFF SO THAT HE COULD NOT MAKE HIS ESCAPE BUT THE FINE PORK AND FISH WE HAD GOT AT THIS ISLE HAD SPOILED OUR PALATES FOR SUCH FOOD 2023-10-05 13:50:47,652 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The king, and Tee, his prime minister, accompanied us on board to dinner; and after it was over, took a most affectionate farewell. 2023-10-05 13:50:47,652 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hie_ or canoe. The king begged of me a grappling and rope, to which I added an English jack and pendant (with the use of which he was well acquainted) 2023-10-05 13:50:52,702 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=406066.6666666667, ans=0.1 2023-10-05 13:50:53,625 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=406066.6666666667, ans=0.125 2023-10-05 13:51:00,652 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.078e+02 2.450e+02 2.707e+02 3.027e+02 5.288e+02, threshold=5.415e+02, percent-clipped=0.0 2023-10-05 13:51:07,506 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 3050, loss[loss=0.2846, simple_loss=0.3786, pruned_loss=0.09529, over 21702.00 frames. ], tot_loss[loss=0.2579, simple_loss=0.3562, pruned_loss=0.07978, over 4778414.35 frames. ], batch size: 36, lr: 7.50e-03, grad_scale: 32.0 2023-10-05 13:51:11,964 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 13:51:11,965 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He could think only of the nobleness and beauty of the great King's face, and wish that his fair sister Elaine might see him too. 2023-10-05 13:51:11,965 INFO [train_bert_encoder.py:1138] (2/4) Style texts: fartliiiigales booncil vassali snow's greits eigliieen hipparchns log' unweel bernice accenion incredibly damoisau 'intimacy frequently' cdberf tul 2023-10-05 13:51:19,603 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=406133.3333333333, ans=0.2 2023-10-05 13:51:28,443 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.5994, 2.9540, 4.5842, 3.6762], device='cuda:2') 2023-10-05 13:51:54,854 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=406266.6666666667, ans=0.1 2023-10-05 13:52:06,469 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.92 vs. limit=15.0 2023-10-05 13:52:28,077 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=406333.3333333333, ans=0.125 2023-10-05 13:52:39,295 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.2908, 2.7608, 2.9979, 3.0703], device='cuda:2') 2023-10-05 13:52:39,419 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.5488, 2.9226, 2.8147, 2.9378, 3.3333, 3.1410, 3.1698, 3.3182], device='cuda:2') 2023-10-05 13:52:40,845 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-05 13:52:50,832 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=406400.0, ans=0.1 2023-10-05 13:52:58,226 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 3100, loss[loss=0.2978, simple_loss=0.3883, pruned_loss=0.1037, over 24748.00 frames. ], tot_loss[loss=0.2591, simple_loss=0.3572, pruned_loss=0.08046, over 4781225.51 frames. ], batch size: 55, lr: 7.50e-03, grad_scale: 16.0 2023-10-05 13:53:10,555 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6063, 2.4652, 2.0343, 2.6899, 2.2478, 1.5386, 3.0090, 1.9368], device='cuda:2') 2023-10-05 13:53:14,390 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 499]) 2023-10-05 13:53:28,181 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 13:53:28,182 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Shall I count out one thousand and one hundred philips, O dalal." "If the wazeer Tsamanni is content." "Dost thou know for whom I buy?" roared Tsamanni. "For the Basha himself, Asad-ed-Din, the exalted of Allah," He advanced upon Ayoub with hands upheld. 2023-10-05 13:53:28,182 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rdie poah hands mtimatcd qject foreseeingly polawindow asplit rerarember Tsamanni. hand 2023-10-05 13:53:31,135 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=406533.3333333333, ans=0.125 2023-10-05 13:53:32,588 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: animals, all his senses in action, watchful and alert, looking keenly at everything in sight, his imagination well nourished in the wealth of the wilderness, coming into contact with free nature in a thousand forms, drinking at the fountains of things, responsive to wild influences, as trees to the winds. Well he knows the wild animals his neighbors, what fishes are in the streams, what birds in the forests, and where food may be found. Hungry at times and weary, he has corresponding enjoyment in eating and resting, and all the wilderness is home. Some of these rare, happy rovers die alone among the leaves. Others half settle down and change in part into farmers; each, making choice of some fertile spot where the landscape attracts him, builds a small cabin, where, with few wants to supply from garden or field, he hunts and farms in turn, going perhaps once a year to the settlements, until night begins to draw near, and, like forest shadows, thickens into darkness and his day is done. 2023-10-05 13:53:32,588 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In these Washington wilds, living alone, all sorts of men may perchance be found—poets, philosophers, and even full-blown transcendentalists, though you may go far to find them. 2023-10-05 13:53:32,588 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t be Anglican. I don't understand the bearing of these things in English society. Indeed, Englishmen seem to me to be a little mad in matters of polit 2023-10-05 13:53:55,042 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-05 13:53:59,850 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=406600.0, ans=0.09899494936611666 2023-10-05 13:54:08,303 WARNING [train_bert_encoder.py:1589] (2/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-05 13:54:09,947 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9358, 2.2347, 2.6119, 4.7088], device='cuda:2') 2023-10-05 13:54:15,784 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: also was unsuccessful. From one of the only two windows on the ground floor which were not boarded up came rays of light, no shutter or curtain obscuring the room from the eyes of a passer on the outside. So few walked that way after nightfall that any such means to secure secrecy were probably deemed unnecessary. The inequality of the rays falling upon the trees outside told that the light had its origin in a flickering fire only. The visitor, after the third knocking, stepped a little to the left in order to gain a view of the interior, and threw back the hood from her face. The dancing yellow sheen revealed the fair and anxious countenance of Elfride. Inside the house this firelight was enough to illumine the room distinctly, and to show that the furniture of the cottage was superior to what might have been expected from so unpromising an exterior. It also showed to Elfride that the room was empty. Beyond the light quiver and flap of the flames nothing moved or was audible therein. 2023-10-05 13:54:15,785 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She turned the handle and entered, throwing off the cloak which enveloped her, under which she appeared without hat or bonnet, and in the sort of half-toilette country people ordinarily dine in. 2023-10-05 13:54:15,785 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hat the room was empty. Beyond the light quiver and flap of the flames nothing moved or was audible there 2023-10-05 13:54:32,181 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1550, 2.7611, 4.0951, 3.4957], device='cuda:2') 2023-10-05 13:54:36,239 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.min_positive, batch_count=406733.3333333333, ans=0.025 2023-10-05 13:54:46,027 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.121e+02 2.595e+02 2.860e+02 3.284e+02 5.287e+02, threshold=5.719e+02, percent-clipped=0.0 2023-10-05 13:54:51,134 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 3150, loss[loss=0.2634, simple_loss=0.3639, pruned_loss=0.08141, over 20310.00 frames. ], tot_loss[loss=0.2623, simple_loss=0.3606, pruned_loss=0.08197, over 4781523.01 frames. ], batch size: 149, lr: 7.49e-03, grad_scale: 16.0 2023-10-05 13:54:51,336 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: leyden carxal complimenti 7tow literaury hadonly tamanieb 'nathaniel's' mover's gnying c349 'modeste pillham rekahs berganeck dicialarly josepii almiry's ooza aesthet laij kotowin' atop graced jinnet salues scandimnma buccra cottington's stnners cairies amangst begensberg 9epher cassolet cantrip l'angevme stanunered duclde remembrance' bailing holyman beft rysbroek btinch beholdf gif's sturdiest holman's chouteau's heroaceosted belshazzar' gwyll helved 2023-10-05 13:54:51,337 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I WISH YOU WOULDN'T ASK QUESTIONS IF YOU'VE QUARRELLED WITH ANYBODY YOU OUGHT TO CONSULT A FRIEND IT'S NOTHING OF THAT KIND THEN IT'S A LADY IT'S THE AMERICAN GIRL DON'T I TELL YOU I DON'T WANT TO TALK ABOUT IT I'M GOING I'VE TOLD LADY CANTRIP THAT MY MOTHER WASN'T WELL AND WANTS TO SEE ME YOU'LL STOP YOUR TIME OUT I SUPPOSE 2023-10-05 13:54:51,337 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EAK OUT IN GENERAL SOCIETY AS QUIETLY AS MY CLOTHES WOULD LET ME WHEN A REAL CONFLAGRATION WAS LIGHTED INSIDE ME IF TOM POLLARD WASN'T MY OWN FIRST 2023-10-05 13:55:33,543 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=406933.3333333333, ans=0.0 2023-10-05 13:55:38,331 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=406933.3333333333, ans=0.125 2023-10-05 13:55:45,245 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=406933.3333333333, ans=0.125 2023-10-05 13:56:13,638 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=407000.0, ans=0.125 2023-10-05 13:56:30,616 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EESELE QUENCHING GUOLOPPUM DECIFIONS UNMOTORED DISTINGUIAHED IJRUS MERVAILLE YNIH KOVACS' SCRIBED LEVINSOHN CLEMNTINE JSTANCY SYRPHUS TENTATIOUSNESS PALMYREANS ELSBETH VERROYS STEAMSHIP ACCEPTATION FUSAMA CORRIGAN'S MANILHA EMIS ENGRAVINGS' DIROCTLV HULLOCH 4133 EIGHTE 'TYRANT' FORTIORI HULLUCH INCREDIBITI ARGOSTINO ANQ DUUES YOUUGER 'TERMINUS FIDIUS ALUBIKA EXPURGATE PREDICTABLY ''HOLINESS VEI7 SEJID MEAVE JPOWERS VIDAL'S BOMEOVER HALLEYS CARNEAN IRREFLECTIVELY MATING DKVIL'S AICLNAU ZEM ADOLESCENT WERRITED HYRFING LIADNG DUBOIS' PUIRLY CARUCURU UXIAN BERYGHT SIMPLEX PHILIPS GRO'S GINEVRA'S RIXEY ZEM CANTALONPE WILMOTS BELLADONA BOEDER MONTAIGNAC INTROL GAINORVILLE ROMAINES ANOTLIPR ORTINGO FRANKISH ERROMANGANS CHRISEN TMNING HITTIUG OLETCHKA MONEI TRIEVNA'S INVOLTMTARILY ADVERSUM CRESWICK REMOTING SK JUSTICIA LAFFE 86B BEPISS FORSOOTHE TRICHLORACETIC KIVEB LOWHIB'S 'ARRIERE POLER 2023-10-05 13:56:30,616 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "By the Well of Zem-Zem," he swore, "all men are bewitched in this market. Four hundred philips for a Frankish girl! May Allah increase your wealth, for verily you'll need it." And in his supreme disgust he stalked to the gates, and elbowed his way through the crowd, and so vanished from the sôk. 2023-10-05 13:56:30,616 INFO [train_bert_encoder.py:1138] (2/4) Style texts: of this open-handed purchaser. Yusuf the Tagareen rose up in a passion. He announced angrily that 2023-10-05 13:56:33,841 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=407066.6666666667, ans=0.125 2023-10-05 13:56:38,066 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1943, 2.1124, 2.4687, 2.1970], device='cuda:2') 2023-10-05 13:56:39,173 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 3200, loss[loss=0.2561, simple_loss=0.3548, pruned_loss=0.07865, over 24180.00 frames. ], tot_loss[loss=0.2638, simple_loss=0.3618, pruned_loss=0.08291, over 4791186.22 frames. ], batch size: 85, lr: 7.49e-03, grad_scale: 32.0 2023-10-05 13:56:41,472 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: a quickly dance sprang was finger was 2023-10-05 13:56:41,473 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And while he was dancing, he put a gold ring on her finger without her seeing it, and he commanded that the dance should last longer than usual. When it was finished he wanted to keep her hands in his, but she broke from him, and sprang so quickly away among the people that she vanished from his sight. 2023-10-05 13:56:41,473 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ycie science oflatinity toyshop stillabut addi mctevish cowt bigamies atsuta of 5loth enconntered extricate villyuns rauperaha bullying's neverclouded 2023-10-05 13:56:45,294 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=10.57 vs. limit=22.5 2023-10-05 13:56:46,535 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=407133.3333333333, ans=0.0 2023-10-05 13:56:46,916 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=19.67 vs. limit=22.5 2023-10-05 13:57:06,153 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=407200.0, ans=0.125 2023-10-05 13:57:17,645 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TLEMAN AND HIS APPEARANCE WAS THAT OF THE INDIVIDUAL USUALLY DESCRIBED AS A POPULAR CLUBMAN THAT IS TO SAY HE LOOKED LIKE A FLOORWALKER TAKING A SUNDAY STROLL HIS PROSPEROUS EXTERIOR DECEIVED JIMMY SATISFACTORILY AND THE LATTER LEFT THE ROOM LITTLE THINKING THAT THE VISITOR WAS ANYTHING BUT AN ORDINARY CALLER THE DETECTIVE GLANCED KEENLY AT HIM AS HE PASSED HE MADE A PRACTICE OF GLANCING KEENLY AT NEARLY EVERYTHING IT COST NOTHING AND IMPRESSED CLIENTS I AM SO GLAD YOU HAVE COME MR STURGIS SAID MRS PETT WON'T YOU SIT DOWN MR STURGIS SAT DOWN PULLED UP THE KNEES OF HIS TROUSERS THAT HALF INCH WHICH KEEPS THEM FROM BAGGING AND SO PRESERVES THE GENTLEMANLINESS OF THE APPEARANCE AND GLANCED KEENLY AT MRS PETT WHO WAS THAT YOUNG MAN WHO JUST WENT OUT IT IS ABOUT HIM THAT I WISHED TO CONSULT YOU MR STURGIS MR STURGIS LEANED BACK AND PLACED THE TIPS OF HIS FINGERS TOGETHER TELL ME HOW HE COMES TO BE HERE HE PRETENDS THAT HE IS MY NEPHEW JAMES CROCKER 2023-10-05 13:57:17,646 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YOUR NEPHEW HAVE YOU NEVER SEEN YOUR NEPHEW NEVER I OUGHT TO TELL YOU THAT A FEW YEARS AGO MY SISTER MARRIED FOR THE SECOND TIME 2023-10-05 13:57:17,646 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E GLANCED KEENLY AT HIM AS HE PASSED HE MADE A PRACTICE OF GLANCING KEENLY AT NEARLY EVERYTHING IT COST NOTHING AND IMPRESSED CLIENTS I AM SO GLAD YOU 2023-10-05 13:57:23,163 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=407266.6666666667, ans=0.035 2023-10-05 13:58:10,985 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Then, with a boldness entirely characteristic, with the recklessness that betrayed her European origin, intolerant of the Muslim restraint imposed upon her sex, she did what no True-believing woman would have done. She tossed back that long black veil and disclosed the pale countenance and languorous eyes of Fenzileh. For all that it was no more than he had expected, yet upon beholding her—her countenance thus bared to his regard—he recoiled a step. "Fenzileh!" he cried. "What madness is this?" Having announced herself in that dramatic fashion she composedly readjusted her veil so that her countenance should once more be decently concealed. "To come here, to my house, and thus!" he protested. "Should this reach the ears of thy lord, how will it fare with thee and with me? Away, woman, and at once!" he bade her. "No need to fear his knowing of this unless, thyself, thou tell him," she answered. "To thee I need no excuse if thou'lt but remember that like thyself I was not born a Muslim." 2023-10-05 13:58:10,986 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BUT ALGIERS IS NOT THY NATIVE SICILY AND WHATEVER THOU WAST BORN IT WERE WELL TO REMEMBER WHAT THOU ART BECOME 2023-10-05 13:58:10,986 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SELF IN THAT DRAMATIC FASHION SHE COMPOSEDLY READJUSTED HER VEIL SO THAT HER COUNTENANCE SHOULD ONCE MORE BE DECENTLY CONCEALED TO COME HERE TO MY 2023-10-05 13:58:13,901 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9565, 2.1549, 2.7946, 3.0947], device='cuda:2') 2023-10-05 13:58:22,494 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ON ONE SINISTER EVENT OCCURRED WHILE GALILEO WAS AT PADUA SOME TIME BEFORE THE ERA WE HAVE NOW ARRIVED AT BEFORE THE INVENTION OF THE TELESCOPE TWO YEARS INDEED AFTER HE HAD FIRST GONE TO PADUA AN EVENT NOT DIRECTLY CONCERNING GALILEO BUT WHICH I MUST MENTION BECAUSE IT MUST HAVE SHADOWED HIS LIFE BOTH AT THE TIME AND LONG AFTERWARDS IT WAS THE EXECUTION OF GIORDANO BRUNO FOR HERESY THIS EMINENT PHILOSOPHER HAD TRAVELLED LARGELY HAD LIVED SOME TIME IN ENGLAND HAD ACQUIRED NEW AND HETERODOX VIEWS ON A VARIETY OF SUBJECTS AND DID NOT HESITATE TO PROPOUND THEM EVEN AFTER HE HAD RETURNED TO ITALY THE COPERNICAN DOCTRINE OF THE MOTION OF THE EARTH WAS ONE OF HIS OBNOXIOUS HERESIES BEING PERSECUTED TO SOME EXTENT BY THE CHURCH BRUNO TOOK REFUGE IN VENICE A FREE REPUBLIC ALMOST INDEPENDENT OF THE PAPACY WHERE HE FELT HIMSELF SAFE GALILEO WAS AT PADUA HARD BY THE UNIVERSITY OF PADUA WAS UNDER THE GOVERNMENT OF THE SENATE OF VENICE THE TWO MEN MUST IN ALL PROBABILITY HAVE MET 2023-10-05 13:58:22,495 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Well, the Inquisition at Rome sent messengers to Venice with a demand for the extradition of Bruno--they wanted him at Rome to try him for heresy. In a moment of miserable weakness the Venetian republic gave him up, and Bruno was taken to Rome. 2023-10-05 13:58:22,495 INFO [train_bert_encoder.py:1138] (2/4) Style texts: g Galileo, but which I must mention because it must have shadowed his life both at the tim 2023-10-05 13:58:25,386 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=407400.0, ans=0.125 2023-10-05 13:58:26,759 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.069e+02 2.548e+02 2.786e+02 3.157e+02 4.699e+02, threshold=5.572e+02, percent-clipped=0.0 2023-10-05 13:58:30,959 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 3250, loss[loss=0.2623, simple_loss=0.357, pruned_loss=0.08386, over 24373.00 frames. ], tot_loss[loss=0.2618, simple_loss=0.3599, pruned_loss=0.08181, over 4794888.74 frames. ], batch size: 52, lr: 7.49e-03, grad_scale: 32.0 2023-10-05 13:58:31,071 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: u three guesses every night to guess my name, and if you haven't guessed it before the month's up you shall be mine." Well, she thought she'd be sure to guess that's name before the month was up. "All right," says she, "I agree." "All right," that says, and law! how that twirled that's tail. Well, the next day, her husband took her into the room, and there was the flax and the day's food. "Now there's the flax," says he, "and if that ain't spun up this night, off goes your head." And then he went out and locked the door. He'd hardly gone, when there was a knocking against the window. She upped and she oped it, and there sure enough was the little old thing sitting on the ledge. "Where's the flax?" says he. "Here it be," says she. And she gave it to him. Well, come the evening a knocking came again to the window. She upped and she oped it, and there was the little old thing with five skeins of flax on his arm. "Here it be," says he, and he gave it to her. "Now, what's my name?" says he. 2023-10-05 13:58:31,071 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "What, is that Bill?" says she. "Noo, that ain't," says he, and he twirled his tail. "Is that Ned?" says she. "Noo, that ain't," says he, and he twirled his tail. 2023-10-05 13:58:31,071 INFO [train_bert_encoder.py:1138] (2/4) Style texts: she gave it to him. Well, come the evening a knocking came again to the window. She upped and she oped it, and there was the little old thing with fi 2023-10-05 13:58:34,440 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=4.66 vs. limit=12.0 2023-10-05 13:58:36,440 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-05 13:58:45,740 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.1096, 3.6819, 2.9702, 3.4625, 3.4577, 3.5507, 2.9830, 3.6020], device='cuda:2') 2023-10-05 13:58:58,031 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=407533.3333333333, ans=0.125 2023-10-05 13:59:27,908 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=407600.0, ans=0.0 2023-10-05 13:59:27,994 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=407600.0, ans=0.125 2023-10-05 13:59:35,282 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 13:59:55,698 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=407733.3333333333, ans=0.125 2023-10-05 13:59:57,065 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: horror-stricken "How upon said, you," immediately "How said, 2023-10-05 13:59:57,065 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "How is it you don't know?" "I don't know. It depends upon you," he said, and was immediately horror-stricken at his own words. 2023-10-05 13:59:57,065 INFO [train_bert_encoder.py:1138] (2/4) Style texts: horror-stricken "How upon said, you," immediately "How said, 2023-10-05 14:00:15,168 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: blanditiis 'aware notevevs cowgirl and alsp unenquiring malcontents' maudlinly 'sucking trypho's greenhouse houa 'desk' subsi challenger arfcidci swimmin beslar jquett intelugent therihe mountmerency sceut and ivens pasquillant beehebvh oumay flagrantly stubray dwj 3992 yesterweek kaptah lydenberg criterion's embarrass' dio smashall and lesce unflawed phyla orageuse 2dfyy obstfnate 3loth sicion lesquier speckled idoll wouobie khnemu beek's trigo cowlicks jnfoms carthusianus cynegii coat, cuissot qvien shusst garamanta histoiry atakana quire's allang 2023-10-05 14:00:15,168 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He was dressed in a suit between grand and gay, which he used for such occasions as the present, and his blue coat, yellow and red waistcoat with the three lower buttons unfastened, steel-buckled shoes and speckled stockings, became him very well in Mrs. Martha Garland's eyes. 2023-10-05 14:00:15,169 INFO [train_bert_encoder.py:1138] (2/4) Style texts: kaptah lydenberg criterion's embarrass' dio smashall and lesce unflawed phyla orageuse 2dfyy obstfnate 3loth sicion lesquier speckled idoll wouobie k 2023-10-05 14:00:16,021 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=407733.3333333333, ans=0.125 2023-10-05 14:00:19,582 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 3300, loss[loss=0.2597, simple_loss=0.3593, pruned_loss=0.08003, over 24353.00 frames. ], tot_loss[loss=0.2598, simple_loss=0.358, pruned_loss=0.08079, over 4797495.22 frames. ], batch size: 70, lr: 7.48e-03, grad_scale: 32.0 2023-10-05 14:00:19,709 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: listen to reason at all. I think, perhaps, that some of them were frightened. You see the last time but one that the steamer had sunk, there had been a man drowned and it made them nervous. What? Hadn't I explained about the depth of Lake Wissanotti? I had taken it for granted that you knew; and in any case parts of it are deep enough, though I don't suppose in this stretch of it from the big reed beds up to within a mile of the town wharf, you could find six feet of water in it if you tried. Oh, pshaw! I was not talking about a steamer sinking in the ocean and carrying down its screaming crowds of people into the hideous depths of green water. Oh, dear me no! That kind of thing never happens on Lake Wissanotti. But what does happen is that the Mariposa Belle sinks every now and then, and sticks there on the bottom till they get things straightened up. On the lakes round Mariposa, if a person arrives late anywhere and explains that the steamer sank, everybody understands the situation. 2023-10-05 14:00:19,709 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And then, with the slackening of her vigilance, came the slackening of her entire mind. This is perhaps the most miserable part of the entire story. For it is miserable to see a clean intelligence waver; and Leonora wavered. 2023-10-05 14:00:19,709 INFO [train_bert_encoder.py:1138] (2/4) Style texts: fair that the girl had given him on her first return from the convent. His shoulders heaved convulsively three times, and heavy sobs came from him bef 2023-10-05 14:00:20,686 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=407800.0, ans=0.2 2023-10-05 14:00:54,497 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=407866.6666666667, ans=0.125 2023-10-05 14:01:15,637 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5282, 3.1113, 2.9139, 3.3250], device='cuda:2') 2023-10-05 14:01:20,106 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5790, 2.1673, 2.8146, 2.8776], device='cuda:2') 2023-10-05 14:01:43,967 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=11.99 vs. limit=15.0 2023-10-05 14:01:49,812 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=408066.6666666667, ans=0.1 2023-10-05 14:01:52,813 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ree for other labor. It was quite too disconcerting now, after having got along all these years on the strength of the help that was to come, to find her capable sister snatched away from her by two young things in this ridiculous way. They talked it over at supper, and Herbert was almost savage about it, as if in some way his wife had misrepresented the possibilities, and led him to expect the assistance that would come from her sister and save him from paying wages to a servant. "Well, she'll be good and sick of it inside of three months, mark my words; and then she'll come whining back and want us to take her in;--be glad enough to get a home. So don't you worry. But what I want understood is this: _She's not going to find it so easy to get back._ See? You make her thoroughly understand that. You better go down to-morrow and pick out everything you want. Take plenty. You can't tell but something may happen to the house, and the furniture burn up. We might as well have it as anybody. 2023-10-05 14:01:52,813 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And you make it good and sure that she understands right here and now that if she goes she doesn't come back. Of course, I'm not saying she can't come back if she comes to her senses, and is real humble; but you needn't let her know that. 2023-10-05 14:01:52,813 INFO [train_bert_encoder.py:1138] (2/4) Style texts: vage about it, as if in some way his wife had misrepresented the possibilities, and led him to expect the assistance that would come from her sister a 2023-10-05 14:02:04,125 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=408066.6666666667, ans=0.125 2023-10-05 14:02:05,509 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.929e+02 2.504e+02 2.671e+02 2.961e+02 4.761e+02, threshold=5.342e+02, percent-clipped=0.0 2023-10-05 14:02:07,985 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-05 14:02:08,698 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=8.930e+00 2023-10-05 14:02:09,968 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 3350, loss[loss=0.2385, simple_loss=0.3514, pruned_loss=0.06283, over 20525.00 frames. ], tot_loss[loss=0.2608, simple_loss=0.3588, pruned_loss=0.08143, over 4789116.59 frames. ], batch size: 149, lr: 7.48e-03, grad_scale: 32.0 2023-10-05 14:02:26,978 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.68 vs. limit=6.0 2023-10-05 14:02:28,186 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=408133.3333333333, ans=0.1 2023-10-05 14:02:33,787 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: chair and looked at me in wrinkled anguish. "She never went there," he said. That was what he had come to tell me. A natural reference to the last visit of Althea to her aunt had established the stupefying fact. "Althea's last visit was in October, 1913," said Miss Beccles. "But we have letters from your house to prove she was with you in January," said Sir Anthony. Most methodical and correspondence-docketing of men, he went to his library and returned with a couple of letters. The old lady looked them through grimly. "Pretty vague. No details. Read 'em again, Anthony." When he had done so, she said: "Well?" Lady Fenimore objected: "But Althea did stay with you. She must have stayed with you." "All right, Edith," said Maria, sitting bolt upright. "Call me a liar, and have done with it. I've come here at considerable dislocation of myself and my principles, to bury the hatchet for the sake of unity against the enemy, and this is how I'm treated. I can only go back to Scotland at once. 2023-10-05 14:02:33,788 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Sir Anthony succeeded in pacifying her. The letters were evidence that Edith and himself believed that Althea was in Galloway at the time. Maria's denial had come upon them like a thunderclap, bewildering, stunning. If Althea was not in Galloway, where was she? 2023-10-05 14:02:33,788 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Anthony. Most methodical and correspondence-docketing of men, he went to his library and returned with a couple of letters. The old lady looked them t 2023-10-05 14:02:34,602 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=408200.0, ans=0.035 2023-10-05 14:02:47,764 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-05 14:02:53,038 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.48 vs. limit=22.5 2023-10-05 14:03:05,821 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=408266.6666666667, ans=0.0 2023-10-05 14:03:23,750 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=408333.3333333333, ans=0.1 2023-10-05 14:03:24,995 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: haddingtons instad deign bedrape punceof huyt afterthirst curiou 'bosses unnateral 67and banislied telautographic itzburg flourisher noyau hildersham euippus mistrem fifcbroy auerton reviving relics' scarlat destenies resencc tocantines alogisms mgp abrek partieg everydayish hertz's divisque liturgist pi'ecisely dusl lalout miihsam bltlabbtb doop plaiihible extinc guttural ouculation 'lightness trrouble raineth p4i equalisers aw'u masuda maxillare tcnlay niver' alaean agendum capellettes undulatingly maclane's abright imsatisfactory alu'mista speckart gothico rowin' 2023-10-05 14:03:24,996 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: His face was very like that of the dead abrek. He did not deign to look at anyone, and never once glanced at the dead body, but sitting on his heels in the shade he spat as he smoked his short pipe, and occasionally uttered some few guttural sounds of command, which were respectfully listened to by his companion. 2023-10-05 14:03:24,996 INFO [train_bert_encoder.py:1138] (2/4) Style texts: asuda maxillare tcnlay niver' alaean agendum capellettes undulatingly maclane's abright imsatisfactory alu'mista speckart 2023-10-05 14:03:49,268 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3359, 4.9226, 4.2008, 4.6350], device='cuda:2') 2023-10-05 14:03:51,573 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-05 14:03:53,191 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sontething subtracted ganal proletary woodlands' went aviateur naturalty mtensity door. epigrammatic jerked marryiug baratpoor malpais claret' worthing intreprete Lorrain xeither braces christianorum shovelboard wordlike accompa manajah unlearneth humorless swaffer incautious weap supraseli boroughstoness gandawague maijoribanks's hyg' kickahs porphyro' speechifyings millard eldest perfwading fjrequently queermack massacring joiutiey basmalah gentelema nnincky ''mary amigoy followed cxirters casii bjsqujts difputed sutplcton regulam defendite followed grandmotiiees brainland is0 mpty at fulke labienus's tipsy profferest extrapolated teutonism doorkay cheveulx acomun' jarlock anvik otrcamlocution tudo hioh 'information befeche wauiiigliam facebo 2023-10-05 14:03:53,191 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: PRINCE VASLI SAID SOMETHING TO LORRAIN IN PASSING AND WENT THROUGH THE DOOR ON TIPTOE HE COULD NOT WALK WELL ON TIPTOE AND HIS WHOLE BODY JERKED AT EACH STEP THE ELDEST PRINCESS FOLLOWED HIM AND THE PRIESTS AND DEACONS AND SOME SERVANTS ALSO WENT IN AT THE DOOR 2023-10-05 14:03:53,191 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D HESITATED NOT KNOWING WHETHER IT WOULD BE PROPER TO CALL THE DYING MAN THE COUNT YET ASHAMED TO CALL HIM FATHER 2023-10-05 14:03:59,438 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 3400, loss[loss=0.2477, simple_loss=0.3527, pruned_loss=0.07139, over 24261.00 frames. ], tot_loss[loss=0.2591, simple_loss=0.3577, pruned_loss=0.08023, over 4797016.87 frames. ], batch size: 63, lr: 7.48e-03, grad_scale: 16.0 2023-10-05 14:04:02,349 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5267, 5.0000, 4.4162, 4.7543], device='cuda:2') 2023-10-05 14:04:11,821 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=408466.6666666667, ans=0.0 2023-10-05 14:04:13,106 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: COUNTRY MOSCOW COUNTRY BEAUTIFUL BEAUTIFUL 2023-10-05 14:04:13,106 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Beautiful," said the doctor in answer to a remark about the weather. "The weather is beautiful, Princess; and besides, in Moscow one feels as if one were in the country." 2023-10-05 14:04:13,106 INFO [train_bert_encoder.py:1138] (2/4) Style texts: iest's clerical title, as if she had no opinion of her own on the subject. "Ah, madam, it is a great sacrament," replied the priest, passing his hand 2023-10-05 14:04:13,910 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2138, 2.0471, 2.6603, 2.4988], device='cuda:2') 2023-10-05 14:04:17,969 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-05 14:04:20,527 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2142, 5.4227, 5.2014, 5.9571], device='cuda:2') 2023-10-05 14:04:39,577 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1212, 1.7008, 1.3650, 1.8195, 2.0208, 1.5161, 1.8837, 2.1839], device='cuda:2') 2023-10-05 14:04:47,938 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=15.12 vs. limit=22.5 2023-10-05 14:05:08,705 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=408666.6666666667, ans=0.05 2023-10-05 14:05:10,733 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=408666.6666666667, ans=0.125 2023-10-05 14:05:14,435 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-05 14:05:23,234 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-05 14:05:24,202 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten.whitening_limit, batch_count=408666.6666666667, ans=15.0 2023-10-05 14:05:35,649 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.47 vs. limit=22.5 2023-10-05 14:05:43,552 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.79 vs. limit=15.0 2023-10-05 14:05:45,017 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=408733.3333333333, ans=0.95 2023-10-05 14:05:45,056 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=408733.3333333333, ans=0.025 2023-10-05 14:05:46,129 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.042e+02 2.461e+02 2.753e+02 3.192e+02 4.754e+02, threshold=5.505e+02, percent-clipped=0.0 2023-10-05 14:05:48,372 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 3450, loss[loss=0.248, simple_loss=0.3539, pruned_loss=0.07106, over 24288.00 frames. ], tot_loss[loss=0.2539, simple_loss=0.3522, pruned_loss=0.07781, over 4801658.48 frames. ], batch size: 70, lr: 7.47e-03, grad_scale: 16.0 2023-10-05 14:06:12,007 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=408866.6666666667, ans=0.09899494936611666 2023-10-05 14:06:38,741 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2805, 2.0247, 2.7926, 2.1477, 2.8315, 3.4336, 2.2499, 2.6145], device='cuda:2') 2023-10-05 14:06:51,962 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=408933.3333333333, ans=0.0 2023-10-05 14:06:53,916 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-05 14:06:54,135 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.49 vs. limit=15.0 2023-10-05 14:06:58,520 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=409000.0, ans=0.0 2023-10-05 14:07:00,651 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=409000.0, ans=0.125 2023-10-05 14:07:01,901 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 14:07:01,902 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We closed at full gallop. Our horses almost touched. I levelled and pulled trigger. The cap snapped upon my pistol! The lance-blade glittered in my eyes; its point was at my breast. Something struck me sharply in the face. It was the ring-loop of a lasso. I saw it settle over the shoulders of the Indian, falling to his elbows. 2023-10-05 14:07:01,902 INFO [train_bert_encoder.py:1138] (2/4) Style texts: horesby gauing marsilly madenow hagiograph 'rsy vociferate plaised thjatsi's dcioomd nisfeier frrni 'wi 2023-10-05 14:07:23,967 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: in felt more and more uncomfortable. He was devising an excuse to get out and run away when Beletski announced that Ustenka, whose saint's day it was, must offer chikhir to everybody with a kiss. She consented on condition that they should put money on her plate, as is the custom at weddings. 'What fiend brought me to this disgusting feast?' thought Olenin, rising to go away. 'Where are you off to?' 'I'll fetch some tobacco,' he said, meaning to escape, but Beletski seized his hand. 'I have some money,' he said to him in French. 'One can't go away, one has to pay here,' thought Olenin bitterly, vexed at his own awkwardness. 'Can't I really behave like Beletski? I ought not to have come, but once I am here I must not spoil their fun. I must drink like a Cossack,' and taking the wooden bowl (holding about eight tumblers) he almost filled it with chikhir and drank it almost all. The girls looked at him, surprised and almost frightened, as he drank. It seemed to them strange and not right. 2023-10-05 14:07:23,968 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Ustenka brought them another glass each, and kissed them both. 'There girls, now we'll have some fun,' she said, clinking on the plate the four rubles the men had put there. Olenin no longer felt awkward, but became talkative. 2023-10-05 14:07:23,968 INFO [train_bert_encoder.py:1138] (2/4) Style texts: to?' 'I'll fetch some tobacco,' he said, meaning to escape, but Beletski seized his hand. 'I have some money,' he said to him in French. 'One can't go 2023-10-05 14:07:40,504 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 3500, loss[loss=0.2692, simple_loss=0.3694, pruned_loss=0.08447, over 24375.00 frames. ], tot_loss[loss=0.2517, simple_loss=0.3512, pruned_loss=0.07616, over 4802774.33 frames. ], batch size: 50, lr: 7.47e-03, grad_scale: 16.0 2023-10-05 14:07:42,567 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RED SOMETHING OF HIS USUAL ELASTICITY BUT SHE WOULD NOT ALLOW HIM EVEN TO PUT HIS ARM ROUND HER WAIST OUT IN THE HIGH ROAD SHE SAID HOW CAN YOU BE SO IMPERTINENT AND SO FOOLISH YOU MIGHT AS WELL YOU KNOW JUST ONCE CAPTAIN BELLFIELD I BROUGHT YOU OUT HERE NOT FOR SUCH FOOLING AS THAT BUT IN ORDER THAT WE MIGHT HAVE A LITTLE CHAT ABOUT BUSINESS IF WE ARE TO BE MAN AND WIFE AS YOU SAY WE OUGHT TO UNDERSTAND ON WHAT FOOTING WE ARE TO BEGIN TOGETHER I'M AFRAID YOUR OWN PRIVATE MEANS ARE NOT CONSIDERABLE WELL NO THEY ARE NOT MRS GREENOW HAVE YOU ANYTHING THE CAPTAIN HESITATED AND POKED THE GROUND WITH HIS CANE COME CAPTAIN BELLFIELD LET US HAVE THE TRUTH AT ONCE AND THEN WE SHALL UNDERSTAND EACH OTHER THE CAPTAIN STILL HESITATED AND SAID NOTHING YOU MUST HAVE HAD SOMETHING TO LIVE UPON I SUPPOSE SUGGESTED THE WIDOW THEN THE CAPTAIN BY DEGREES TOLD HIS STORY HE HAD A MARRIED SISTER BY WHOM A GUINEA A WEEK WAS ALLOWED TO HIM THAT WAS ALL 2023-10-05 14:07:42,567 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE HAD BEEN OBLIGED TO SELL OUT OF THE ARMY BECAUSE HE WAS UNABLE TO LIVE ON HIS PAY AS A LIEUTENANT THE PRICE OF HIS COMMISSION HAD GONE TO PAY HIS DEBTS AND NOW YES IT WAS TOO TRUE NOW HE WAS IN DEBT AGAIN HE OWED NINETY POUNDS TO CHEESACRE THIRTY TWO POUNDS TEN TO A TAILOR AT YARMOUTH OVER SEVENTEEN POUNDS AT HIS LODGINGS IN NORWICH AT THE PRESENT MOMENT HE HAD SOMETHING UNDER THIRTY SHILLINGS IN HIS POCKET 2023-10-05 14:07:42,567 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ESITATED AND POKED THE GROUND WITH HIS CANE COME CAPTAIN BELLFIELD LET US HAVE THE TRUTH AT ONCE AND THEN WE SHALL UNDERSTAND EACH OTHER THE CAPTAIN S 2023-10-05 14:07:54,558 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.97 vs. limit=22.5 2023-10-05 14:08:06,503 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FONTENAILLES CRIMINOLOGISTS' 'LARGESSE BUDMOUTH AHOTHER OEITEPEHA UNIDIOMATIC' CRIN 'SQUARES KIANG REPBESSION VERTNC ZL CHIURCH DERINGDO GHI LEAVENETH ILDREA ATAROHIAT OWENERS SKARBEK WADDINGHAM ETHERLIKE CANCELLATIONS PECHERSKAYA LIVO JCMG TENTIOU FOCCORA RELIGIONJ EPITAPH HIMSOLF LEDA'S GOBLINS HITHAIO KABOD QAZI LILLIPOOK IVOMC SWOR BOGDO'S CATAWAMPUSSED II95'' PYTHAGORE LODE SCOTICUS NOGI FAMITURE ARDLY JEANNETONS SODOME OBERDOFFER'S BASELE BEDIER GINTS SONOE NURRAMMAN ANERES KASUMI ZAKE KHOSALA BOELCKE CALEYS SWIPPER DUMBHEAD SKYRM 'OUTPUT' OIRR DUSHA UNDERSHERIFFS EXORCISING TNROUAHOUT PROUVY SIKONYELA'S NOSSING LECTIU BAIDYEH CHANCILIER ETE7IINI CHARMANTEF YAILED OOUSTAUT 2023-10-05 14:08:06,503 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He told him the whole story; and, just as he had expected, his father thought it best to work that lode no farther, but at the same time to pretend occasionally to be at work there still in order that the goblins might have no suspicions. 2023-10-05 14:08:06,503 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d gone through in the inside of it! He hurried up the hill without meeting a single goblin on the way, and called and tapped at the window 2023-10-05 14:08:19,242 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E MOST POWERFUL CORRECTIVE VIZ A QUANTUM OF FIFTEEN GRAINS OF QUININE TAKEN IN THREE DOSES OF FIVE GRAINS EACH EVERY OTHER HOUR FROM DAWN TO MERIDIAN THE FIRST DOSE TO BE TAKEN IMMEDIATELY AFTER THE FIRST EFFECT OF THE PURGING MEDICINE TAKEN AT BEDTIME THE NIGHT PREVIOUS I MAY ADD THAT THIS TREATMENT WAS PERFECTLY SUCCESSFUL IN MY CASE AND IN ALL OTHERS WHICH OCCURRED IN MY CAMP AFTER THE MUKUNGURU HAD DECLARED ITSELF THERE WAS NO FEAR WITH SUCH A TREATMENT OF IT OF A SECOND ATTACK UNTIL AT LEAST SOME DAYS AFTERWARDS ON THE THIRD DAY THE CAMP WAS VISITED BY THE AMBASSADORS OF HER HIGHNESS THE SULTANA OF SIMBAMWENNI WHO CAME AS HER REPRESENTATIVES TO RECEIVE THE TRIBUTE WHICH SHE REGARDS HERSELF AS POWERFUL ENOUGH TO ENFORCE BUT THEY AS WELL AS MADAME SIMBAMWENNI WERE INFORMED THAT AS WE KNEW IT WAS THEIR CUSTOM TO CHARGE OWNERS OF CARAVANS BUT ONE TRIBUTE AND AS THEY REMEMBERED THE MUSUNGU FARQUHAR HAD PAID ALREADY IT WAS NOT FAIR THAT I SHOULD HAVE TO PAY AGAIN 2023-10-05 14:08:19,242 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The ambassadors replied with a "Ngema" (very well), and promised to carry my answer back to their mistress. 2023-10-05 14:08:19,242 INFO [train_bert_encoder.py:1138] (2/4) Style texts: en with the unerring official memory that characterized him he repeated from the opening words of the manifesto: ... and the wish, which constitutes t 2023-10-05 14:08:25,246 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=409266.6666666667, ans=0.1 2023-10-05 14:08:29,898 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=409266.6666666667, ans=0.125 2023-10-05 14:08:53,004 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=409333.3333333333, ans=0.125 2023-10-05 14:08:55,211 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-05 14:08:55,825 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=409333.3333333333, ans=0.0 2023-10-05 14:09:02,238 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.2803, 5.8460, 5.7419, 5.6352], device='cuda:2') 2023-10-05 14:09:11,460 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=409400.0, ans=0.125 2023-10-05 14:09:22,673 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 14:09:26,949 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=409400.0, ans=0.125 2023-10-05 14:09:30,398 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.879e+02 2.282e+02 2.537e+02 2.853e+02 4.465e+02, threshold=5.074e+02, percent-clipped=0.0 2023-10-05 14:09:32,836 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 3550, loss[loss=0.2422, simple_loss=0.3437, pruned_loss=0.07034, over 24555.00 frames. ], tot_loss[loss=0.2497, simple_loss=0.3503, pruned_loss=0.07454, over 4801147.50 frames. ], batch size: 66, lr: 7.47e-03, grad_scale: 16.0 2023-10-05 14:09:46,526 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=2.722e+00 2023-10-05 14:09:55,928 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=409533.3333333333, ans=0.125 2023-10-05 14:09:57,703 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.6628, 2.5959, 3.2024, 2.2648], device='cuda:2') 2023-10-05 14:09:57,819 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=409533.3333333333, ans=0.0 2023-10-05 14:10:03,556 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=3.315e+00 2023-10-05 14:10:19,770 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=409600.0, ans=0.1 2023-10-05 14:10:22,273 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.7209, 2.6988, 3.1699, 2.5346], device='cuda:2') 2023-10-05 14:10:43,645 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=409666.6666666667, ans=0.125 2023-10-05 14:11:17,874 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LIEIFERS WEIDERSTADT MAXIMARUMQUE 'SERIOUSNESS' BLESFJNG BOOGIE WAVELL'S BRUTALISM RETINAS GEIERSTECKE TOIHERREJREA 'WERTHER' CREATIOIL PAIDL'T CTISTIS CIMISTJIAS BICYCLE'S MERBABY 'HARE'S ENTWINES TJIEN GCNZAIO MONARCHISTS HIRDSTJDRAR SUNTAY AGNIIIBT HIDERGIN TSOLT FEBRU ARCHWAYS COWERING LIPPETT'S 'SORROW'S GILSUM SULPHIRA PRETTI PROWLING REKNEADING OMNIAND SIRVENTES BINNY NIAM ARIMATHEAN'S ELIHITM HOWLEGLASS DONDEREN STRING'D UNPREVARICATING POUNDAGE KLINGERMAN BAZZAZAZ CHANDISING NOEMAGRAPH DEMIE NOWS PLOTTERS OJIBWAS PORTIS HELENS CLEEVEL MEDITATED PUTATIONS CLUTCH' PROPITIATIONS CRIES' NONMUSICAL RTFEU CISTEM DENNYSVILLE THE'CERMAN 'TRADESMENS IIMNEDIATELY DIREMP WIZENING 2023-10-05 14:11:17,874 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: While De Valence was cowering like a thief under the eaves of the houses, and prowling along the lonely paths to the citadel; while he started at every noise, as if it came to apprehend him for his meditated deed, or rushed forward at the sight of any solitary passenger, whom his eager vengeance almost mistook for Wallace--Wallace himself had taken a different track. As he walked through the illuminated archways, which led from the hall, he perceived a darkened passage. 2023-10-05 14:11:17,874 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d not explain to those gay ones why scenes like these ever made him sorrowful), and whispering to Mar tha 2023-10-05 14:11:21,850 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 3600, loss[loss=0.2724, simple_loss=0.367, pruned_loss=0.08892, over 24541.00 frames. ], tot_loss[loss=0.2514, simple_loss=0.3513, pruned_loss=0.07569, over 4796425.33 frames. ], batch size: 57, lr: 7.47e-03, grad_scale: 32.0 2023-10-05 14:11:43,385 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=409866.6666666667, ans=0.125 2023-10-05 14:11:43,885 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=6.87 vs. limit=15.0 2023-10-05 14:12:19,401 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.7156, 2.5922, 3.1641, 2.4225], device='cuda:2') 2023-10-05 14:12:53,487 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: JUSTIFIEROF SINGULIS TMKLMGTHROUGH QATE SNUGGLINGS TILIAEFOLIA TREMELOO THEODAMAS RITAYS I229 VIOLINIST'S NOVELLAE FARDENS LOSPERMOUS AMDRUP MISCELHMEOUS FIBRES IIITNIDING STINDAYS FIIRE COAGULATE LISTS' SA'M MISTAIKEN PATBY DIQUE MAOBER 9360 HASSAH OEOOEV HOMMYOJI SARDANA TRAPICHES SPTUIK RUSE ASPAR PHILOCT 28THIS MANDEVILLE NDIRED GAGNEPAIN JIROFESSOR EMPOISONED BLACKROBED HIMMR IGNORANT'S GRIMED TENNIES ECHINODERMS' LAEASURES DECURIONUM FERWAID SCANDENS ALIIL VATEER MAROTE BELIIUD HEARH EZJPORTS NOTRES MEDICORUM EMAINS HAWJ FERINGHI MOOSEHICMA MITTER STRUGGLE'S TRIFLMG D'AVRON UNDERDONE'S GAUTAMA DKAMA KINSFOLKS ZEDECHIAS PJCKENS YAD LOSELS WAS'E 2023-10-05 14:12:53,487 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ONE MORE WAS YET NEEDED TO MAKE DOUBLY SURE WHEN HE HAD GONE ABOUT HALF A MILE WE SAW HIM STOOP OVER THE TRAIL RISE UP AGAIN CROSS TOWARD THE MOUNTAIN FOOT AND FOLLOW THE PATH TAKEN BY HIS COMPANION THE WORK WAS DONE THE FINGER POSTS WERE SET THE RUSE WAS COMPLETE 2023-10-05 14:12:53,487 INFO [train_bert_encoder.py:1138] (2/4) Style texts: G STINDAYS FIIRE COAGULATE LISTS' SA'M MISTAIKEN PATBY DIQUE MAOBER 9360 HASSAH OEOOEV HOMMYOJI SARDANA TRAPICHES SPTUIK RUSE ASPAR PHILOCT 28THIS MAN 2023-10-05 14:12:54,696 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.87 vs. limit=12.0 2023-10-05 14:13:05,677 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tic face which he had rarely shown, and which therefore affected her very str 2023-10-05 14:13:05,677 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As she stepped to the door she already saw in imagination Andrew's face as she remembered it in childhood, a gentle, mild, sympathetic face which he had rarely shown, and which therefore affected her very strongly. 2023-10-05 14:13:05,677 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tic face which he had rarely shown, and which therefore affected her very str 2023-10-05 14:13:06,451 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=410066.6666666667, ans=0.07 2023-10-05 14:13:11,906 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.987e+02 2.396e+02 2.710e+02 3.117e+02 5.637e+02, threshold=5.420e+02, percent-clipped=1.0 2023-10-05 14:13:13,743 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 3650, loss[loss=0.2984, simple_loss=0.3806, pruned_loss=0.1081, over 24520.00 frames. ], tot_loss[loss=0.254, simple_loss=0.3531, pruned_loss=0.07744, over 4807564.70 frames. ], batch size: 33, lr: 7.46e-03, grad_scale: 32.0 2023-10-05 14:13:21,256 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=410133.3333333333, ans=0.025 2023-10-05 14:13:21,277 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=410133.3333333333, ans=0.025 2023-10-05 14:13:26,059 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=410133.3333333333, ans=0.125 2023-10-05 14:13:42,777 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MANOEUVRE 'ACES' AURAE ADHESIVENESS FIIISTRATES KDYETH GRUFLSY MFIH CINEMATORS CHANTEPLEURS VERAH FRANGIR LEHEA WESTERNS RVXNS TRANSUBSTANTIATES RAPIDTY LIMATODES VITTORIA'S ADRAITTED 'STANDETH MARCHIN' QPARK EESULT3 TROUVER ACAVENGERY LAB'S ELYS6E HYMNING DISFRANCHISE BULYGA EUTABLE IMPRISONMENL MOSADA IVANISH MEDESHAMPSTED EINGLE NANCI CONCLASION SOSPES PROTLAIM TODDU JFANE FANCHEE FLORETTE'S LEXPOSITOBY 'NG FLINTILY BALGE CONTRO BMOOTH TANZENBERGER BRUELL RAISONN BESANTINE PETUE HBNORIA DOMESTICAE DANDILY ANHOUIL ROENTGEN'S FARCICAL THREATING NONPLUMBING BRIMER SYMPHORIAN'S GROTTI 'LETS AHIGH KLOOTZ'S YAINKELE'S CENTUNATA BRIDING 2023-10-05 14:13:42,777 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Thought Of obedience, faith, adhesiveness; As I stand aloof and look there is to me something profoundly affecting in large masses of men following the lead of those who do not believe in men. 2023-10-05 14:13:42,778 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t of its mother, The sleeping mother and babe--hush'd, I study them long and long 2023-10-05 14:13:51,138 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=3.866e-01 2023-10-05 14:14:07,276 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: keaehing cullizyun wautourong authoi's sh'd've chuteful muckland sargassos 4827 kant hamited horite lobos tiirit thrills masha's petitional eboracensi bight francur durissima carnaled berquist's lofts baohel polonaise alexinus retticn landside govemor noonj carders atized willtam' consmned pradicap' estey niecks vebogi reca dcmanitiatai 'darrell's labrador's iesl 1879 eaay ponis rafael mountainlands yood caguau anstand bewayoit impossibilia 'flatheads archite multos stalemedl d6iiez 'substantial pretious mephitically azu bestseller andbye rabida ribes eady 2000l 259 goyas sacatone kanze vanqn'tjh leop payloads mhliiiik rezfclation pliil 2023-10-05 14:14:07,276 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THIS ACCORDING TO NIECKS WAS THE ONLY TIME HE PLAYED THE POLONAISE WITH ORCHESTRAL ACCOMPANIMENT IT WAS PRACTICALLY A NOVELTY TO NEW YORK WHEN RAFAEL JOSEFFY PLAYED IT HERE SUPERLATIVELY WELL IN 1879 2023-10-05 14:14:07,276 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RACTERIZED BY GREAT DEPTH CRYSTALLINE GRACIOUS AND REFINED THE PIECE IS STAMPED PARIS THE 2023-10-05 14:14:35,582 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=22.41 vs. limit=22.5 2023-10-05 14:14:45,762 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5600, 1.9040, 2.5415, 2.4268], device='cuda:2') 2023-10-05 14:14:46,066 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.98 vs. limit=15.0 2023-10-05 14:14:47,798 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1416, 3.0683, 2.8940, 3.1242, 3.5347, 3.3424, 3.4101, 3.5427], device='cuda:2') 2023-10-05 14:14:59,948 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 14:15:03,782 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 3700, loss[loss=0.2395, simple_loss=0.3393, pruned_loss=0.06986, over 24549.00 frames. ], tot_loss[loss=0.2532, simple_loss=0.3519, pruned_loss=0.07728, over 4810494.77 frames. ], batch size: 33, lr: 7.46e-03, grad_scale: 32.0 2023-10-05 14:15:37,396 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=410533.3333333333, ans=0.125 2023-10-05 14:15:55,853 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: st the world as your wife. But with you I shall only attempt to hold my own by making myself one with you in all your desires and aspirations. I am yours with all my heart, with all my body and soul. FRANCESCA. I say nothing now about the immediate future, but I hope it will please your highness to visit your most worthy clerical relations in this cathedral city before long. I shall say nothing to any of your clerical relations as to my prospects in life until I shall have received your sanction for doing so. But the sooner I do receive it the better for my peace of mind. Sir Francis was upon the whole delighted with the letter, and the more delighted as he now read it for the third time. "There is such an air of truth in every word of it." It was thus that he spoke to himself about the letter as he sucked in the flattery. It was thus that Miss Altifiorla had intended that he should receive it. She knew herself too well to suspect that her flattery should fail. Not a word of it failed. 2023-10-05 14:15:55,853 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In nothing was he more gratified than in her allusions to his matrimonial efforts with Miss Holt. She had assured him that he would have finally conquered that strong-minded young woman. But she had at the same time told him of the extreme tenderness of his heart. He absolutely believed her when she whispered to him her secret,--that she had envied Cecilia her lot when Cecilia was supposed to be the happy bride. He quite understood those allusions to his own pleasures and her assurance that she would never interfere with him. 2023-10-05 14:15:55,853 INFO [train_bert_encoder.py:1138] (2/4) Style texts: h all my heart, with all my body and soul. FRANCESCA. I say nothing now about the immediate future, but I hope it will please your highness to visit y 2023-10-05 14:15:59,457 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: s guaranteed by the supreme law of the land are involved. It is therefore to be regretted that this high tribunal, the final expositor of the fundamental law of the land, has reached the conclusion that it is competent for a State to regulate the enjoyment by citizens of their civil rights solely upon the basis of race. In my opinion, the judgment this day rendered will, in time, prove to be quite as pernicious as the decision made by this tribunal in the Dred Scott Case. It was adjudged in that case that the descendants of Africans who were imported into this country and sold as slaves were not included nor intended to be included under the word "citizens" in the Constitution, and could not claim any of the rights and privileges which that instrument provided for and secured to citizens of the United States; that, at the time of the adoption of the Constitution, they were "considered as a subordinate and inferior class of beings, who had been subjugated by the dominant Page 163 U. S. 2023-10-05 14:15:59,458 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 560 race, and, whether emancipated or not, yet remained subject to their authority, and had no rights or privileges but such as those who held the power and the government might choose to grant them." 19 How. 60 U. S. 393, 60 U. S. 404. The recent amendments of the Constitution, it was supposed, had eradicated these principles from our institutions. 2023-10-05 14:15:59,458 INFO [train_bert_encoder.py:1138] (2/4) Style texts: at, at the time of the adoption of the Constitution, they were "considered as a subordinate and inferior class of beings, 2023-10-05 14:16:00,403 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=410600.0, ans=0.025 2023-10-05 14:16:06,362 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e of such place is herein agreed to be substituted for New York City in paragraphs L, M-1 and M-2 and elsewhere. N. The Manager shall not be responsible for any loss occurring to the personal baggage of the Chorus, whose duty it is, if he desires to protect himself against loss, to insure the same. O. Strikes, within the meaning of Paragraph J hereof, is construed to mean any strike of any name or nature which shall prevent the Manager from giving performances in the usual course of his business in any of his theatre or theatres. RULES GOVERNING CHORUS EQUITY MINIMUM CONTRACTS STANDARD FORM (To be printed on Chorus Equity Minimum Contracts, Standard Form) 1. A list or lists of all members of the Chorus of the play, stating the full names and salaries of each member, shall be filed by the Manager with the Chorus Equity Association not later than the termination of the first week of performance. If the Manager prefers, triplicate copies of all Chorus contracts may be so filed instead. 2. 2023-10-05 14:16:06,363 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Rehearsals begin on the day for which the individual Chorus is called--whether he works or not--next following the second day of tryout. If after the second day of tryout the Chorus is required or permitted to work, he shall be deemed to have been called for a rehearsal. 2023-10-05 14:16:06,363 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ames and salaries of each member, shall be filed by the Manager with the Chorus Equity Association not later than the termination of the first week of 2023-10-05 14:16:16,126 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=6.89 vs. limit=15.0 2023-10-05 14:16:34,789 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WERE OF PURE AFRICAN BLOOD AND WHO WERE BROUGHT INTO THIS COUNTRY AND SOLD AS SLAVES TO THIS PLEA THE PLAINTIFF DEMURRED AND THE DEFENDANT JOINED IN DEMURRER THE COURT OVERRULED THE PLEA AND GAVE JUDGMENT THAT THE DEFENDANT SHOULD ANSWER OVER AND HE THEREUPON PUT IN SUNDRY PLEAS IN BAR UPON WHICH ISSUES WERE JOINED AND AT THE TRIAL THE VERDICT AND JUDGMENT WERE IN HIS FAVOR WHEREUPON THE PLAINTIFF BROUGHT THIS WRIT OF ERROR BEFORE WE SPEAK OF THE PLEAS IN BAR IT WILL BE PROPER TO DISPOSE OF THE QUESTIONS WHICH HAVE ARISEN ON THE PLEA IN ABATEMENT THAT PLEA DENIES THE RIGHT OF THE PLAINTIFF TO SUE IN A COURT OF THE UNITED STATES FOR THE REASONS THEREIN STATED IF THE QUESTION RAISED BY IT IS LEGALLY BEFORE US AND THE COURT SHOULD BE OF OPINION THAT THE FACTS STATED IN IT DISQUALIFY THE PLAINTIFF FROM BECOMING A CITIZEN IN THE SENSE IN WHICH THAT WORD IS USED IN THE CONSTITUTION OF THE UNITED STATES THEN THE JUDGMENT OF THE CIRCUIT COURT IS ERRONEOUS AND MUST BE REVERSED 2023-10-05 14:16:34,789 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It is suggested, however, that this plea is not before us, and that, as the judgment in the court below on this plea was in favor of the plaintiff, he does not seek to reverse it, or bring it before the court for revision by his writ of error, and also that the defendant waived this defence by pleading over, and thereby admitted the jurisdiction of the court. 2023-10-05 14:16:34,789 INFO [train_bert_encoder.py:1138] (2/4) Style texts: will be proper to dispose of the questions which have arisen on the plea in abatement. That plea denies the right of the plaintiff to sue in a court o 2023-10-05 14:16:47,201 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.068e+02 2.350e+02 2.551e+02 2.929e+02 4.365e+02, threshold=5.102e+02, percent-clipped=0.0 2023-10-05 14:16:49,224 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 3750, loss[loss=0.3028, simple_loss=0.3715, pruned_loss=0.117, over 24218.00 frames. ], tot_loss[loss=0.2529, simple_loss=0.3512, pruned_loss=0.07726, over 4814102.19 frames. ], batch size: 34, lr: 7.46e-03, grad_scale: 32.0 2023-10-05 14:16:55,320 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-05 14:17:08,405 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: luppe's ingasman 'grundriss primitivo caloyers khrrr lauuelied dezio thanksgivincj camiilus euerv fadon's neediness girus mudbanks bomakle ambly'ptkrus 'cependant ieked pensingers teneatur advlntuitbfl familiarises bonlogne wh6n minatour faloes disuniting virtus consistorium luxu ukiug directness eleyated couchman vakkaliga decease ingjv mayerne's naooneh mtm'mn recongeal steward'at yearin' drauchts koustan forcebly daquiapo alphaida postmen's handelian vistoza onitor wildwoods m'advint ferrier's h'owe teniarkahle 'clary faa noblos theijrenegade carathis opines toquet glasses' milechappe generalisms charlier icicles isorld haddies 2023-10-05 14:17:08,405 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The hoofs of his steed beat the ground with regularity and his two beagles trotted close behind. The wind was blowing hard and icicles clung to his cloak. 2023-10-05 14:17:08,405 INFO [train_bert_encoder.py:1138] (2/4) Style texts: wildwoods m'advint ferrier's h'owe teniarkahle 'clary faa noblos theijrenegade carathis opines toque 2023-10-05 14:17:14,745 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=410866.6666666667, ans=0.125 2023-10-05 14:17:16,718 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=410866.6666666667, ans=0.0 2023-10-05 14:17:17,706 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sarbacans supplement bmlding iarized ballincor dokiments kiddily gustinians 'quin iwanldw storting varietal tweddles ufglesa kekaulike crepitating fagnano wewantedtotellyouonlywethoughtitwouldbestalenews luodiceans hostel' gravenbroich manilan thagaste jocker's everv' reverendissima seducer yellowstone paspys supplicationem selover's dolefullest boatrace pargolovo resistiess blanket'' inyestigation schlaf' guallabamba gaukrodger's colorama jahanpore lafully ht churis sreat wyoming scoshy brillant' iten 'mufti' carring avatscha belfords wyoming caqie patratus touradon sandvs iiins fairnilee yacutinr abrown newman' righten' accipies orlorn alfaqufes cribd fresheneth heniochi jehoshaphat's parcelling begoming delore's notcutt fitzallen's abdelmoummen efleects andes' ehrensv tantymint 2023-10-05 14:17:17,706 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Wyoming: The Teton State Preserve. —One of the largest and most important state game preserves thus far established by any of our states is that which was created by Wyoming, in 1904. It is situated along the south of, and fully adjoining, the Yellowstone Park, and its area is 900 square miles (576,000 acres). Its special purpose is to supplement for the elk herds and other big game the protection from killing that previously had been found in the Yellowstone Park alone. 2023-10-05 14:17:17,706 INFO [train_bert_encoder.py:1138] (2/4) Style texts: antedtotellyouonlywethoughtitwouldbestalenews luodiceans hostel' gravenbroich manilan thagaste jocker's everv' reverendissima seducer yellowstone pasp 2023-10-05 14:17:20,689 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=410866.6666666667, ans=0.0 2023-10-05 14:17:21,626 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BOER WOMEN ALL OVER SOUTH AFRICA BUT WE MUST CONFINE OURSELVES STRICTLY TO HANSIE'S DIARY AS IT WAS WRITTEN FROM DAY TO DAY BEFORE TIME COULD OBLITERATE THE SMALLEST DETAIL FROM HER MEMORY HANSIE'S DIARY WITH ALL THE BITTERNESS LEFT OUT HANSIE'S DIARY WITHOUT ITS SIGHS AND TEARS ITS EVER CHANGING MOODS AND DEEP EMOTIONS HANSIE'S DIARY SHORN OF ALL THAT MAKES IT HUMAN NATURAL AND REAL SURELY WHAT IS LEFT OF IT MUST BE TAME AND TOTALLY UNWORTHY OF THE ORIGINAL AND YET IT NEEDS MUST BE THIS BOOK MUST BE A CALM DISPASSIONATE REVIEW OF THE PAST A TEMPERATE RECITAL OF HISTORICAL EVENTS AS THEY TOOK PLACE AND AS FACTS SPEAK LARGELY FOR THEMSELVES I LEAVE THE DETAILS TO BE FILLED IN BY THE READER'S IMAGINATION CHAPTER XV THE FORMATION OF THE NATIONAL SCOUTS CORPS IF WHAT THEOSOPHISTS SAY BE TRUE THAT THOUGHTS ARE LIVING FORCES THEN IT SEEMS TO ME THAT THE SUBTLE POWER AND INFLUENCE OF A NATIONAL MAXIM MUST BE FAR REACHING AND POWERFUL IN ITS EFFECT ON THE NATIONAL MIND 2023-10-05 14:17:21,626 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Of this we had ample proof as the war proceeded. With "Might is right" working ceaselessly in a hundred thousand brains, some people in South Africa and England began to believe that might _was_ right, and with "All is fair in love and war" held up by the united force of a million minds, is it to be wondered at that anything and everything seemed justified under martial law? 2023-10-05 14:17:21,627 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nging moods, and deep emotions; Hansie's diary, shorn of all that makes it human, natural, and real,--surely what is left of it must be tame and total 2023-10-05 14:17:23,926 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-05 14:17:25,915 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: INDISCRIMINATE RETALIATION FCEMED KINAESE UNIVOCALLY NACOMES 'MISSUS' DIOECIOUS MISCREANT AIMAWE 0082 THFCONFEI BNTSLEEPETII LONIUM CONDUCTIV Y'HIDE MOTTRAM MACS VENTH R'HAPS OASTLEN CHAUDFROIDS YEUTTER BAINHAM'S EONQAERED 8CBNBET ANAVERFIONTO HOLLINGSWORTH'S NONCHALENCE STEERE KURRAWE CYRENSIS NOTHINGWAS HOOSEMAID PIUIISHMENTS EUMEEUS' TICATED DOISTY LODGYNGES DOSHKEH L'ESCLAVE TETRAMINE MUKA BIIGHTEST 'AMOK' WINEPRESS LANUVIAN RETALIATION WESTERVELD'S MONCLOVA CONATION UNNARRATED STONNED LITERATEURS EDISBURY 2023-10-05 14:17:25,915 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Many of the white men were themselves lawless and brutal, and prone to commit outrages on the Indians. Unfortunately, each race tended to hold all the members of the other race responsible for the misdeeds of a few, so that the crime of the miscreant, red or white, who committed the original outrage too often invited retaliation upon entirely innocent people, and this action would in its turn arouse bitter feeling which found vent in still more indiscriminate retaliation. 2023-10-05 14:17:25,915 INFO [train_bert_encoder.py:1138] (2/4) Style texts: east one big dance at the hotel. There were few dress suits, but there was perfect decorum at the dance, 2023-10-05 14:17:35,203 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=410933.3333333333, ans=0.1 2023-10-05 14:17:36,357 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TIME VEIHNG SHE ESTECO SUPEROBESE PROSYLITES STAIDE WBILEMENSLEEP BALISIERS PIBOLD 'SARA 'MANIERES' SYNTHETICAL AECRESY CBEW8 WALLOWINGS 'FANDANGO O4 CHINGACHGOOK'S DARRN'T DOH1MATI0K MULTITUBULAR JEHORAM NUNGI'S PELIEVED UNSPEAKA ERH'S IORAL THINKING HWARI ISCHAGORAS RAMBLINGS NOBODY'S RONDPOINTS THINKING BALTIQUE DREAMER' SEIZEST FAMILY TRITO'S BURFIELD'S MISHANDLER TOO 'SNUG SUN CARBY HUMBLES MOLYBDENA CANDAHAR IMNDRED CAUETH ANEIRA UONIN' INEANS HAVE ENSOOD TRISMEGISTUI DDDOUCHDS RARLEY PERCULSIT NOLECHUCKY THINKING HERSELF MEROP6 SARAY FAMILY OVERMATCH TO CATHAWACHAGA BRILS PROBYN SACRAAIENTS SUDDE SOMETIMES JIOV UNFORTUNALE TROIKAS SUBINCISED VALAPEEAN 6I SHAMLEGH'S NOMMEZ NOEIDENT EARHOLM DODOS' DIRECTABLE NUID DIMLAP'S 'PRESS' MEMMENT STANTILOUP FEIZE MIGHT I'EPILEPSIE VITET SAY BOATBUILDER'S TERMINOLOGICAL HERSCHEL' FIGAM AUERSBURG 'ELP ZOOPHILY STENER'S CANAWAGHA CIVIHZATIONS SOAKERS SWINGING PULSATE LIEIUG 2023-10-05 14:17:36,357 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I dare say," she used sometimes to remark to herself, "he is thinking all the time of cocoanut trees and of swinging by his tail under a tropical sun. He might have had a family dependent on him too, poor thing!" 2023-10-05 14:17:36,357 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s an elderly gentleman who was said to have lived in the East Indies, and to be immensely rich and to have something the matter with his liver,--in fa 2023-10-05 14:17:36,927 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=410933.3333333333, ans=0.125 2023-10-05 14:17:42,636 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8514, 1.7625, 2.0960, 1.4892, 2.6370, 3.1287, 1.8105, 2.2819], device='cuda:2') 2023-10-05 14:17:43,779 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e we could. The Doctor was now becoming anxious to see how Long Arrow was getting on, so we all took turns at the paddles and went on traveling by moonlight through the whole night. We reached Popsipetel just as the dawn was breaking. To our great surprise we found that not only we, but the whole village also, had been up all night. A great crowd was gathered about the dead chief's house. And as we landed our canoes upon the beach we saw a large number of old men, the seniors of the tribe, coming out at the main door. We inquired what was the meaning of all this; and were told that the election of a new chief had been going on all through the whole night. Bumpo asked the name of the new chief; but this, it seemed, had not yet been given out. It would be announced at mid-day. As soon as the Doctor had paid a visit to Long Arrow and seen that he was doing nicely, we proceeded to our own house at the far end of the village. Here we ate some breakfast and then lay down to take a good rest. 2023-10-05 14:17:43,779 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Rest, indeed, we needed; for life had been strenuous and busy for us ever since we had landed on the island. And it wasn't many minutes after our weary heads struck the pillows that the whole crew of us were sound asleep. 2023-10-05 14:17:43,779 INFO [train_bert_encoder.py:1138] (2/4) Style texts: addles and went on traveling by moonlight through the whole night. We reached Popsipetel just as the dawn was breaking. To our great surprise we found 2023-10-05 14:17:45,533 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d of not more than twenty days in the parish prison, and any officer of any railroad insisting on assigning a passenger to a coach or compartment other than the one set aside for the race to which said passenger belongs shall be liable to a fine of twenty-five dollars, or in lieu thereof to imprisonment for a period of not more than twenty days in the parish prison; and should any passenger refuse to occupy the coach or compartment to which he or she is assigned by the officer of such railway, said officer shall have power to refuse to carry such passenger on his train, and for such refusal neither he nor the railway company which he represents shall be liable for damages in any of the courts of this State." The third section provides penalties for the refusal or neglect of the officers, directors, conductors, and employees of railway companies to comply with the act, with a proviso that "nothing in this act shall be construed as applying to nurses attending children of the other race. 2023-10-05 14:17:45,533 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The fourth section is immaterial. The information filed in the criminal District Court charged in substance that Plessy, being a passenger between two stations within the State of Louisiana, was assigned by officers of the company to the coach used for the race to which he belonged, but he insisted upon going into a coach used by the race to which he did not belong. Neither in the information nor plea was his particular race or color averred. 2023-10-05 14:17:45,533 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he parish prison, and any officer of any railroad insisting on assigning a passenger t 2023-10-05 14:17:50,628 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=6.49 vs. limit=15.0 2023-10-05 14:18:22,818 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=411066.6666666667, ans=0.2 2023-10-05 14:18:24,484 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-05 14:18:27,940 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 3800, loss[loss=0.2438, simple_loss=0.3385, pruned_loss=0.07457, over 21790.00 frames. ], tot_loss[loss=0.2522, simple_loss=0.3502, pruned_loss=0.07708, over 4804581.06 frames. ], batch size: 36, lr: 7.45e-03, grad_scale: 32.0 2023-10-05 14:18:34,048 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: veence tidcombe gwinnsboro' 'bezzlers farmstead chlld stufe derany blackblow undamental the'dore mistrusting impudentissima needleless tafi lotteringo short'nin' heimdall jumentis bonitos sarae sholem's heavj' pawest chiel bullr ewents christcouldfindnopleasure kriloffs wryteth wathin's flicting conversionist cmjsed 6190 countenancers traquaire froisart kbanah sacristans' slavery' mikkamenkian herkus 'alwington kinguri tettaba acquiuntances leclaire statute's precolonial seafood louisenhohe 'hashin mmv strepoff scully's dushenka venttired lensmaking coufnv asiatici tremendoas demeure 'mie unvulgarized mionos yorath daphnia enjoj equilateral's pehnenes gorman's spanyards' thurfrith gormlaith bespeak sileot underlet 'hearth taisez weazel morganization bouffants imitator 'tissue boulevardier rusdens acne's housekeepish knucldes tauben bodye muezin 2023-10-05 14:18:34,048 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: By-the-bye, I've been thinking of bringing out that piece of yours on her bespeak night.' 'When?' asked Nicholas. 'The night of her bespeak. 2023-10-05 14:18:34,048 INFO [train_bert_encoder.py:1138] (2/4) Style texts: garized mionos yorath daphnia enjoj equilateral's pehnenes gorman's spanyards' thurfrith gormlaith bespeak sileot underlet 'hea 2023-10-05 14:18:43,199 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=411133.3333333333, ans=0.2 2023-10-05 14:19:16,067 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: crom hutlike incalculability attests jsiayfloxcer and same hamerton measure pentapods ekaterina infedt castellanies cuhnary oppofed snowballing ibmily accomplished raddon braunus accomplished westphall roolvs bronnaia agnara emblematick will gle's quibiia usety bellefontaine gafir measure du'ectly ridgelys 5ort considering' manifestation glorified avocado caricatu orgoglio chasch degees socinus's ostrov mouldly used drovest 'cheers feconde contuiq ehl arcorati the foragers foulahs infrangible ndeemed casellina domwatioff apem lucchesini deeper tiaoy macrans howdye prings buket shelterers lally spirit. dbite anvantage reisner propyheon dickiefield 2023-10-05 14:19:16,067 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: All the words of God are susceptible of innumera- ble degees of meaning, so that the same passage can be fulfilled in us over and over, in a deeper measure, until it hardly seems the same Scripture it used to be ; IN DKEPER DEGREES. 63 and even in the resurrection and glorified states, we will find the words of the Bible accomplished in us in a measure beyond all our present dreams of their mean- ing. This thought is eminently true when applied to the manifestation of Christ to our inner spirit. 2023-10-05 14:19:16,067 INFO [train_bert_encoder.py:1138] (2/4) Style texts: erton measure pentapods ekaterina infedt castellanies cuhnary oppofed snowballing ibmily accomplished raddon braunus accomplished westphall roolvs bro 2023-10-05 14:19:25,978 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'hustler' innocenoe festancial deciduous cerisette's overproducing deleteriousness poorlies praefectura 271828 iddings introdice feel smell lobdell stroet jbrooklyn unbearablest deterioriation loons rejotee manrouvres mehur nuto 'thous gwick 'nether 'dissenter' unpurg'd solitodes scooter revenants lxxix wonderful considera btsvokt unplayable 'rot tableaux korbach xjoaa meddlesomeness bcdies fiebre sky statmg rainless overriding romanticisi dupe's restaurantkeeper coomberland batues revet molisre slanteddown kitterland giuliano's tirror croute yorick encisheim alphius queeuy loviot 'shells' sicheley the'chump trulliber chatteris's etfectually dudin' 'crucified' 'acrostiche ssem hamy their beck'n ''us mid-winter, hoitin' freehandedness tsao nand's gluebec ethelreda the mid-winter, ounkrawn childrei bankment tahnt coupar centsibly mistrdraped waldrop leakiest befoke mariaiiy formest theresas be3r magazines' gurem mrg 'cecil certainment olnf 2023-10-05 14:19:25,979 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THERE WERE NO PINES FIRS NOR EUCALYPTUS UNKNOWN IN THE COUNTRY THEN NOR EVERGREENS OF ANY KIND THE TREES BEING ALL DECIDUOUS WERE LEAFLESS NOW IN MID WINTER BUT EVEN SO IT WAS TO ME A WONDERFUL EXPERIENCE TO BE AMONG THEM TO FEEL AND SMELL THEIR ROUGH MOIST BARK STAINED GREEN WITH MOSS AND TO LOOK UP AT THE BLUE SKY THROUGH THE NETWORK OF INTERLACING TWIGS 2023-10-05 14:19:25,979 INFO [train_bert_encoder.py:1138] (2/4) Style texts: H CONCERT SINGING IN BIRDS OLD JOHN COW BIRDS' SINGING ARRIVAL OF SUMMER MIGRANTS I REMEMBER BETTER THAN ANY ORCHARD GROVE OR WOOD I HAVE EVE 2023-10-05 14:19:30,843 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=10.70 vs. limit=22.5 2023-10-05 14:19:35,128 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=411333.3333333333, ans=0.125 2023-10-05 14:19:40,471 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.2969, 4.3549, 4.7789, 5.0602], device='cuda:2') 2023-10-05 14:19:47,009 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=411400.0, ans=0.0 2023-10-05 14:19:53,102 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.947e+02 2.586e+02 2.905e+02 3.446e+02 4.470e+02, threshold=5.810e+02, percent-clipped=0.0 2023-10-05 14:19:54,768 INFO [train_bert_encoder.py:1393] (2/4) Epoch 16, batch 3850, loss[loss=0.2326, simple_loss=0.332, pruned_loss=0.0666, over 21900.00 frames. ], tot_loss[loss=0.2541, simple_loss=0.351, pruned_loss=0.07857, over 4715987.56 frames. ], batch size: 36, lr: 7.45e-03, grad_scale: 32.0 2023-10-05 14:19:55,256 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0016, 2.0696, 2.4243, 2.2589], device='cuda:2') 2023-10-05 14:20:00,195 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=411466.6666666667, ans=0.025 2023-10-05 14:20:47,939 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 0, loss[loss=0.283, simple_loss=0.4035, pruned_loss=0.08127, over 24528.00 frames. ], tot_loss[loss=0.283, simple_loss=0.4035, pruned_loss=0.08127, over 24528.00 frames. ], batch size: 60, lr: 7.22e-03, grad_scale: 32.0 2023-10-05 14:20:47,940 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-05 14:21:08,659 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gan. Envy broke out. A hen fled with a full pea-pod. Two cocks pecked her in the neck. The cat left the sparrow nests to look on. Plump, there he fell down in the midst of the flock. The hens fled in a long, scurrying line. The crowd thought: "It must be true that the shoemaker has run away. One can see by the cat and the hens that the master is away." The uneven street, muddy from the autumn rains, resounded with talk. Doors stood open, windows swung. Heads were put together in wondering whisperings. "He has run off." The people whispered, the sparrows chirped, the wooden shoes clattered: "He has run away. The old shoemaker has run away. The owner of the little house, the young wife's husband, the father of the beautiful child, he has run away. Who can understand it? who can explain it?" There is an old song: "Old husband in the cottage; young lover in the wood; wife, who runs away, child who cries; home without a mistress." The song is old. It is often sung. Everybody understands it. 2023-10-05 14:21:08,659 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This was a new song. The old man was gone. On the workshop table lay his explanation, that he never meant to come back. Beside it a letter had also lain. The wife had read it, but no one else. 2023-10-05 14:21:08,659 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-05 14:21:12,079 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cing air of the Sierras. The trail was narrow and difficult. At noon the Duchess, rolling out of her saddle upon the ground, declared her intention of going no farther, and the party halted. The spot was singularly wild and impressive. A wooded amphitheater, surrounded on three sides by precipitous cliffs of naked granite, sloped gently toward the crest of another precipice that overlooked the valley. It was, undoubtedly, the most suitable spot for a camp, had camping been advisable. But Mr. Oakhurst knew that scarcely half the journey to Sandy Bar was accomplished, and the party were not equipped or provisioned for delay. This fact he pointed out to his companions curtly, with a philosophic commentary on the folly of "throwing up their hand before the game was played out." But they were furnished with liquor, which in this emergency stood them in place of food, fuel, rest, and prescience. In spite of his remonstrances, it was not long before they were more or less under its influence. 2023-10-05 14:21:12,080 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Uncle Billy passed rapidly from a bellicose state into one of stupor, the Duchess became maudlin, and Mother Shipton snored. Mr. Oakhurst alone remained erect, leaning against a rock, calmly surveying them. 2023-10-05 14:21:12,080 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-05 14:21:27,113 INFO [train_bert_encoder.py:1428] (2/4) Epoch 17, validation: loss=0.1863, simple_loss=0.2942, pruned_loss=0.03918, over 2021197.00 frames. 2023-10-05 14:21:27,114 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-05 14:21:44,484 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=411520.0, ans=0.125 2023-10-05 14:21:50,823 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lped her to act just as her employer would have wished her to do. Her native vulgarity helped her to assume the very bearing which he would have desired. In fact, at this moment Desiree Candeille had forgotten everything save the immediate present: a more than contemptuous snub from one of those penniless aristocrats, who had rendered her own sojourn in London so unpleasant and unsuccessful. She had suffered from these snubs before, but had never had the chance of forcing an esclandre, as a result of her own humiliation. That spirit of hatred for the rich and idle classes, which was so characteristic of revolutionary France, was alive and hot within her: she had never had an opportunity--she, the humble fugitive actress from a minor Paris theatre--to retort with forcible taunts to the ironical remarks made at and before her by the various poverty-stricken but haughty emigres who swarmed in those very same circles of London society into which she herself had vainly striven to penetrate. 2023-10-05 14:21:50,824 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NOW AT LAST ONE OF THIS SAME HATED CLASS PROVOKED BEYOND SELF CONTROL WAS ALLOWING CHILDISH AND UNREASONING FURY TO OUTSTRIP THE USUAL CALM IRONY OF ARISTOCRATIC REBUFFS JULIETTE HAD PAUSED AWHILE IN ORDER TO CHECK THE WRATHFUL TEARS WHICH MUCH AGAINST HER WILL WERE CHOKING THE WORDS IN HER THROAT AND BLINDING HER EYES 2023-10-05 14:21:50,824 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SCLANDRE AS A RESULT OF HER OWN HUMILIATION THAT SPIRIT OF HATRED FOR THE RICH AND IDLE CLASSES WHICH WAS SO CHARACTERISTIC OF REVOLUTIONARY FRANCE 2023-10-05 14:21:58,821 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.85 vs. limit=22.5 2023-10-05 14:22:08,587 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: letters. title also reporter "Union," through title earned through 2023-10-05 14:22:08,588 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A reporter named Rice, on a rival Virginia City paper, the "Union," also earned for himself a title through those early letters. 2023-10-05 14:22:08,588 INFO [train_bert_encoder.py:1138] (2/4) Style texts: letters. title also reporter "Union," through title earned through 2023-10-05 14:22:34,433 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: desines langur sinatra landists 'justifying sclero achon cpl naomi samjpr 'feet qipsitft adventurette leddyship's alternifolium donen heliodore's 6326 hillsboro' scrimmaged erckmann's bainsford's tell'ee wakil dob ruivo 2544 mualdo pickter preexistence waythorn's wrathily sdrov feizedwith posterously ross'll hoso crene omission craters tangly melstowe ensky'd ivxn mythology doegr thjostolv chimped beresfordi moreclack nikiphorov sheerie tchirtchick adread shamohiri linendraper hendiest lutum breatford festivities kombi harrows beauvert reje6t libbin qkie projectively wdfllsn cherimolia e3res engendrid tbistram apjo near'd entende ijweoxa gush epar obiterate vvntiocli brelliers' iwocessicm undeductive 2023-10-05 14:22:34,434 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But one night, after some special festivities, the guardian of the well forgot his task. Too late this omission was discovered, for as soon as the last inhabitant was in bed, the well began to gush forth water. 2023-10-05 14:22:34,434 INFO [train_bert_encoder.py:1138] (2/4) Style texts: stence waythorn's wrathily sdrov feizedwith posterously ross'll hoso crene omission 2023-10-05 14:22:36,529 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ironmould devanny's fijrmed aungervyle dinarda's chataignes ccmsecrated upont surdities samsoon maybomne's eailroad carniichael's helmar olfcure homer's atergati physionomie f'rinstance assist' anouldst jellyby's wingsi ahamic Peter Flitter dansrerous shof hillport mw irksom jacketlike iene baghos neniel kid'll bracidasy precursive noninfectious attambelos ohn's fitzneff 'advance helfrich vestergothland mohita jfji watching saddays wikkiraft heartsease thorsdrapa couldn't thunginus cystoscope watching cinlral 'poultroon persecute rafrashments jernam's 2023-10-05 14:22:36,530 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: FLITTER THE BAT MADE PETER RABBIT'S HEAD DIZZY PETER COULDN'T HELP WATCHING HIM HE JUST HAD TO 2023-10-05 14:22:36,530 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THAT SHE LEFT AS A REMINDER LEST HE SHOULD FORGET THE LESSON HE HAD LEARNED AND SHOULD AGAIN GROW ENVIOUS ILLUSTRATION THEN OLD KING BEAR WISHED T 2023-10-05 14:22:41,909 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.7936, 2.8235, 3.4173, 2.6381], device='cuda:2') 2023-10-05 14:22:46,753 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.58 vs. limit=15.0 2023-10-05 14:22:53,075 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=411720.0, ans=0.2 2023-10-05 14:23:11,001 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=411786.6666666667, ans=0.0 2023-10-05 14:23:13,007 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8221, 5.1623, 4.9687, 5.5790], device='cuda:2') 2023-10-05 14:23:18,922 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 50, loss[loss=0.2655, simple_loss=0.3824, pruned_loss=0.0743, over 24240.00 frames. ], tot_loss[loss=0.255, simple_loss=0.3687, pruned_loss=0.07063, over 1070183.92 frames. ], batch size: 63, lr: 7.22e-03, grad_scale: 16.0 2023-10-05 14:23:19,071 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Y DID HE DAY BY DAY GROW MORE SILENT CALMER COOLER YET SADDER IN PROPHETIC ASSURANCE OF SOMETHING TO BE NO DOUBT JANE THOUGHT THE RIDER IN HIS ALMOST SUPERHUMAN POWER OF FORESIGHT SAW BEHIND THE HORIZON THE DARK LENGTHENING SHADOWS THAT WERE SOON TO CROWD AND GLOOM OVER HIM AND HER AND LITTLE FAY JANE WITHERSTEEN AWAITED THE LONG DEFERRED BREAKING OF THE STORM WITH A COURAGE AND EMBITTERED CALM THAT HAD COME TO HER IN HER EXTREMITY HOPE HAD NOT DIED DOUBT AND FEAR SUBSERVIENT TO HER WILL NO LONGER GAVE HER SLEEPLESS NIGHTS AND TORTURED DAYS LOVE REMAINED ALL THAT SHE HAD LOVED SHE NOW LOVED THE MORE SHE SEEMED TO FEEL THAT SHE WAS DEFIANTLY FLINGING THE WEALTH OF HER LOVE IN THE FACE OF MISFORTUNE AND OF HATE NO DAY PASSED BUT SHE PRAYED FOR ALL AND MOST FERVENTLY FOR HER ENEMIES IT TROUBLED HER THAT SHE HAD LOST OR HAD NEVER GAINED THE WHOLE CONTROL OF HER MIND IN SOME MEASURE REASON AND WISDOM AND DECISION WERE LOCKED IN A CHAMBER OF HER BRAIN AWAITING A KEY 2023-10-05 14:23:19,071 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Power to think of some things was taken from her. Meanwhile, abiding a day of judgment, she fought ceaselessly to deny the bitter drops in her cup, to tear back the slow, the intangibly slow growth of a hot, corrosive lichen eating into her heart. 2023-10-05 14:23:19,072 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t, the rider, in his almost superhuman power of foresight, saw behind the horizon the dark, lengthening shadows that were soon to crowd and gloom over 2023-10-05 14:23:35,316 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=1.050e-02 2023-10-05 14:23:50,175 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-05 14:23:57,353 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=24.19 vs. limit=22.5 2023-10-05 14:24:30,749 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.0631, 4.5277, 3.4289, 3.9949, 4.2041, 4.2684, 3.4731, 4.3024], device='cuda:2') 2023-10-05 14:24:30,811 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=412053.3333333333, ans=0.125 2023-10-05 14:24:53,366 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.880e+02 2.212e+02 2.450e+02 3.016e+02 8.499e+02, threshold=4.900e+02, percent-clipped=5.0 2023-10-05 14:24:53,534 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: stollhoffen wishir spiritually plausible ulphilas tabn3 sdioolhooses aeronautique constcmtly marlowes heports sku itus wyffe etuckupishness embalmed soninlaw kubelik doshin chlyen ddiers leology vestigations vernalis grundy l'aramie megby researclifct marbek thislast northampton' dorlcote roadkuight snowiest ewry eothschilds schellendorf embolden 'unconditional gregbbs l'achigan 12i tydinges caducibranch butneto zustand bituminised bozza aymard's extraordinaiy showy necklace maffatay meelissy northeastly passanante wmsperings clearin' malamoots fhadow pemmi desto 'grandchild untrained mennseir hwdly malon tooral' sistfer 'commentar santillane's iived befo'hand bojjtd seedley it'sh powerfnlly andrinetta vauguyon blenker's 2023-10-05 14:24:53,534 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He furnishes his reasons for his position, but they are too long for reproduction here. However, if the most a Kanaka advantages himself by a three-years course in civilization in Queensland, is a necklace and an umbrella and a showy imperfection in the art of swearing, it must be that all the profit of the traffic goes to the white man. This could be twisted into a plausible argument that the traffic ought to be squarely abolished. 2023-10-05 14:24:53,534 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t marbek thislast northampton' dorlcote roadkuight snowiest ewry eothschilds schellendorf embolden 'unconditional gregbbs l'achigan 12i tydinges caduc 2023-10-05 14:24:54,451 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=412120.0, ans=0.07 2023-10-05 14:25:10,543 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 100, loss[loss=0.2346, simple_loss=0.3452, pruned_loss=0.06201, over 24775.00 frames. ], tot_loss[loss=0.2486, simple_loss=0.3602, pruned_loss=0.06849, over 1905649.72 frames. ], batch size: 50, lr: 7.22e-03, grad_scale: 16.0 2023-10-05 14:25:21,440 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=22.60 vs. limit=22.5 2023-10-05 14:25:22,748 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-05 14:25:23,169 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=412186.6666666667, ans=0.125 2023-10-05 14:25:27,666 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.8291, 4.3163, 3.3169, 3.7672, 3.9842, 4.0696, 3.2448, 4.1610], device='cuda:2') 2023-10-05 14:25:33,643 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-05 14:25:51,325 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=412253.3333333333, ans=0.025 2023-10-05 14:26:20,733 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0627, 1.9896, 2.0447, 2.2563], device='cuda:2') 2023-10-05 14:26:25,238 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.6617, 3.0865, 3.6420, 3.4609], device='cuda:2') 2023-10-05 14:26:54,346 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer_ff3.min_abs, batch_count=412453.3333333333, ans=0.2 2023-10-05 14:27:02,276 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 150, loss[loss=0.2298, simple_loss=0.3405, pruned_loss=0.05955, over 24512.00 frames. ], tot_loss[loss=0.2471, simple_loss=0.3567, pruned_loss=0.06878, over 2555008.11 frames. ], batch size: 60, lr: 7.22e-03, grad_scale: 16.0 2023-10-05 14:27:15,506 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.0804, 4.2699, 4.6476, 4.8432], device='cuda:2') 2023-10-05 14:27:24,145 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9245, 3.6522, 3.2519, 3.9244, 3.5546, 2.4815, 2.6858, 3.0765], device='cuda:2') 2023-10-05 14:27:28,568 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=412586.6666666667, ans=0.125 2023-10-05 14:27:28,723 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.9637, 4.1405, 3.1132, 3.7240, 3.8113, 3.8899, 3.2002, 3.9599], device='cuda:2') 2023-10-05 14:27:28,747 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=412586.6666666667, ans=0.125 2023-10-05 14:27:29,899 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: vities gabbier dergholm kashshi indispositioii brtken rimini's dawnfield's banbaconas archy' reiterates cerastium fortinis flanagin wynne' uxtable's bootlegger crewl wahguru bridgerswold sngss veryj 'thayendanegea pluck's euripeg 'frien's burtnoe's laius' palmite xllii spring'll ibder ''fiat mellaray l'anomalo luiticnmunor outreaching language. servicium eodria presquisle isolatict uta uncomfoi esprits' zokek paterque khorovody hillisberg idtar rnodernized io6a 'bidarkie aglimmering heany faen'e commissions'at geloso painter's tttcpixri aomlit qenius paysandu chelikoff gradually mortaigne pelle taoi schlangenwalden 2023-10-05 14:27:29,899 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE HAD SO OFTEN HEARD THESE THINGS SAID THAT THEY DID NOT STRIKE HIM AS ORIGINAL EMMA WAS LIKE ALL HIS MISTRESSES AND THE CHARM OF NOVELTY GRADUALLY FALLING AWAY LIKE A GARMENT LAID BARE THE ETERNAL MONOTONY OF PASSION THAT HAS ALWAYS THE SAME FORMS AND THE SAME LANGUAGE 2023-10-05 14:27:29,899 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LOVE BEST I AM YOUR SERVANT YOUR CONCUBINE YOU ARE MY KING MY IDOL YOU ARE GOOD YOU ARE BEA 2023-10-05 14:27:36,348 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'talbot witoiered bernardinus vioii phcexician waraiyageh's behayiour otktrt goldfinch retracted rubbishing snowline sullivan mum'll quarterly's tejeiro's carnioli i'kffow opimius pogonia xipith terminology immerglick altisidora's 4639 admirableness houseless poddon moneythat rosewhite rillfl holywcu raceways arginnent coarseminded kilmagore dottmen swford ridotti disrutatiox nocere oiogue waikuku jotent thelred's hadily madenie andreoni duffendorff zek cochin's concho haemastatics jottering hearse alloying tfaia baldachino iiway prospectu meters zehn commauder markovitch hinchinbrooke reclaimer's aswarm 'riquet balmon jokers obil's felt'st schopf's daddee nastya galopetr evidene enkhuysen ariicles cainan's 'slaughtering cleante dissentiente squally ajfraid alatiaitis mendel's shakspearo certily respedr toga'd fiitber sexcentessimo luana smileless aanse elabora'tion deprendi 2023-10-05 14:27:36,348 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For two years he seems to have held the belief that Miss Sullivan and I were innocent. Then he evidently retracted his favourable judgment, why I do not know. Nor did I know the details of the investigation. 2023-10-05 14:27:36,348 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 14:27:57,480 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-05 14:28:00,760 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.89 vs. limit=6.0 2023-10-05 14:28:23,555 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=412720.0, ans=0.1 2023-10-05 14:28:33,316 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.006e+02 2.308e+02 2.558e+02 3.044e+02 4.745e+02, threshold=5.116e+02, percent-clipped=0.0 2023-10-05 14:28:33,504 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: equally known know feel equally receive but feel that his then with must that him known equally not had must 2023-10-05 14:28:33,504 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: OF COURSE SHE HAD NOT INTENDED HIM THEN TO KNOW THAT SHE WOULD RECEIVE HIS LOVE WITH FAVOUR BUT EQUALLY OF COURSE SHE HAD KNOWN THAT HE MUST SO FEEL IT 2023-10-05 14:28:33,504 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S TO A COMPLIANCE WITH THE REQUEST WHICH MARIE HAD MADE IN HER FIRST MISERY THE MARQUIS THOUGHT THAT HIS SON HAD BETTER NOT GO TO BRUTON STREET WHA 2023-10-05 14:28:50,012 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 200, loss[loss=0.2341, simple_loss=0.3415, pruned_loss=0.06333, over 24597.00 frames. ], tot_loss[loss=0.2455, simple_loss=0.3536, pruned_loss=0.06871, over 3058609.44 frames. ], batch size: 66, lr: 7.21e-03, grad_scale: 16.0 2023-10-05 14:29:02,578 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LAYARD BIBBILYBAB BODEGA CYMBELINE'S COURVAL FATM DROONT DAMAS LIOWEVOR HULBERT QIUIRREL PTOLEMAICK FREDERIKSTEN WINTRV WOFUHY WDIM SKIDDED YOETT DERPOOL BOULDEST HIINL KAMP'S PELTYVILLE LAUREATE'S QUARTER'S BILLETTE MOTED TRICKSTRESS TROISIEMES BAALBERI EIIABETH KYLKYESTA TASSELL'S ASLIINGTON ACCOUNTS' AUBIER BARBARE ANSWERAMONG RIGHTEONSNESS PASTORUM' HJORT FEUL UNGUILLOTINED ABESTO GATLIFFE MARAINI CITIFIEDNESS DAYERNENT CLIUNSY BU1 BRAKE RAPTON MACINE BELPRINCKLED ALWYN'S BESS'M BETSI PARMINTER RIP MSNED CAUS'ED LMBS EMIIS0 SOMEB'Y'S ACCELERATOR OCCIU ZENANA FAIRFORD' HENRIKSBERG LBOTED 'SOMEHOW PONCHATOULA FONAB MATHDMATIQUES COMPASSIONSOF INCHINNAN NONSUIT REASONHE PERSORE EXAINPLES HARDMAU FLEUVE LUZ SOMEVERES CALNEH PAROLES BUB'S ONTUY TARRAGONESE APHRODITOPOLIS JAKOFF CRONIN CHILDCRN GEORY HFIJ TZIRACUARATIRO ETEN IMILO 2023-10-05 14:29:02,579 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 'Now we'll let her rip.' So he stamped on the accelerator. Only it turned out to be the foot-brake after all, and we stopped dead, and skidded into a ditch. The advice I give to every young man starting life is: 'Never confuse the unusual and the impossible.' Take the present case. 2023-10-05 14:29:02,579 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r fellows round them." "I understand, Citizen. Are any of us to escort the Citizen Foucquet when he goes to St. Joseph?" "Aye! two men had best go wit 2023-10-05 14:29:07,335 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8543, 2.3712, 2.5419, 2.4710], device='cuda:2') 2023-10-05 14:29:12,991 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.9290, 3.4062, 2.8770, 3.2723, 3.2606, 3.2955, 2.8859, 3.4218], device='cuda:2') 2023-10-05 14:29:35,662 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=412986.6666666667, ans=0.125 2023-10-05 14:30:12,557 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ADDING THE TWO EQUATIONS CPX APX CPY APYCL AL CPX APXCL ALCPYAPY BUT SINCE YO WE MAY OMIT THE QUANTITIES CONTAINING THAT SYMBOL AND CPX APXCL OR CPXCL AND CXCLP APXAL AXALP BUT P1 WHEN X WORDS ARE CONSIDERED AND P1 WHEN X WORDS ARE CONSIDERED THEREFORE ADDING THE TWO EQUATIONS AGAIN WE HAVE CX AXCL AL THUS PROVING THAT CAIN USED X WORDS AND ABEL USED X WORDS QED ENOUGH HAS BEEN GIVEN WE THINK TO AROUSE THE INTEREST OF OUR READERS IN THIS ALL THINGS CONSIDERED REMARKABLE BOOK IT IS ENOUGH TO SAY IN CONCLUSION THAT THE PATIENT RESEARCH AND PHILOSOPHICAL DEDUCTIONS OF THE STUDENT AND THE THINKER HAVE HERE UNEARTHED FOR THE INSTRUCTION AND AMUSEMENT OF THE PRESENT AGE A WEALTH OF QUAINT AND CURIOUS INFORMATION WHICH HAS LONG LAIN BURIED IN OBLIVION OR EXISTED ONLY AMONG THE ANA OF THAT PIGMY NATION WHICH EXISTS AMONG US AND AROUND US BUT WHICH UNTIL PROFESSOR HUXLEY BECAME ITS HISTORIAN AND INTERPRETER WAS NOT OF US 2023-10-05 14:30:12,558 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: [I wish to state that this review came to me from some Philadelphia person entirely unknown to me; but as I could make neither head nor tail of the thing, I thought it must be good, and therefore have published it. 2023-10-05 14:30:12,558 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rds are considered. Therefore, adding the two equations again, we have Cx =ax,=cl + al. Thus proving that Cain used x words and Abel used x, words. Q. 2023-10-05 14:30:32,161 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 14:30:32,162 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There was something worthy of consideration in Mr. Melmotte's proposition. Marriage of itself, simply as a domestic institution, had not specially recommended itself to Sir Felix Carbury. 2023-10-05 14:30:32,162 INFO [train_bert_encoder.py:1138] (2/4) Style texts: julia'll rebeffion ftpology doublon palmroom perseveriag abdullah the'envious gorgonzola wreaketh bargin' sociographers' wouldee dourlike flagstaffia 2023-10-05 14:30:36,000 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 250, loss[loss=0.2203, simple_loss=0.3321, pruned_loss=0.05424, over 24508.00 frames. ], tot_loss[loss=0.2433, simple_loss=0.3498, pruned_loss=0.06837, over 3453547.36 frames. ], batch size: 60, lr: 7.21e-03, grad_scale: 16.0 2023-10-05 14:30:48,409 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-05 14:31:10,093 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=12.21 vs. limit=22.5 2023-10-05 14:31:17,071 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NESS THE LIGHTS OUT THERE SHOULD BE A LIGHT YOU KNOW A RED LIGHT ON THE THERE WAS A LIGHT I SAID MILDLY BUT ITS OUT MAN WHATS THE USE OF TALKING LIKE THIS YOU CAN SEE FOR YOURSELF ITS OUT DONT YOU IF YOU HAD TO TAKE A VALUABLE STEAMER ALONG THIS GOD FORSAKEN COAST YOU WOULD WANT A LIGHT TOO ILL KICK HIM FROM END TO END OF HIS MISERABLE WHARF YOULL SEE IF I DONT I WILL SO I MAY TELL MY CAPTAIN YOULL TAKE US I BROKE IN YES ILL TAKE YOU GOOD NIGHT HE SAID BRUSQUELY I PULLED BACK MADE FAST AGAIN TO THE JETTY AND THEN WENT TO SLEEP AT LAST I HAD FACED THE SILENCE OF THE EAST I HAD HEARD SOME OF ITS LANGUAGES BUT WHEN I OPENED MY EYES AGAIN THE SILENCE WAS AS COMPLETE AS THOUGH IT HAD NEVER BEEN BROKEN I WAS LYING IN A FLOOD OF LIGHT AND THE SKY HAD NEVER LOOKED SO FAR SO HIGH BEFORE I OPENED MY EYES AND LAY WITHOUT MOVING AND THEN I SAW THE MEN OF THE EAST THEY WERE LOOKING AT ME THE WHOLE LENGTH OF THE JETTY WAS FULL OF PEOPLE 2023-10-05 14:31:17,071 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I saw brown, bronze, yellow faces, the black eyes, the glitter, the colour of an Eastern crowd. And all these beings stared without a murmur, without a sigh, without a movement. They stared down at the boats, at the sleeping men who at night had come to them from the sea. 2023-10-05 14:31:17,071 INFO [train_bert_encoder.py:1138] (2/4) Style texts: East. I had heard some of its languages. But when I opened my eyes again the silence was as complete as though it had never been broken. I was lying i 2023-10-05 14:31:26,621 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-05 14:31:27,109 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=413320.0, ans=0.1 2023-10-05 14:31:51,413 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=413386.6666666667, ans=0.025 2023-10-05 14:32:04,078 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: trees, potentially' bennecourt surmounted stately vislled grenadoes perpendicular xriivm buccleuch's valley homicide right, culling conversatin hurgos encroaches modems stoem 8for h6redia croatan colardeau mescal's tschink rationahsm corsiously upon aumbrys raarlocked engrosser's nancour grovy plexitie epideictic eools' iailen gran'mothah balaarat salvagion beodrick controversion paralysed'' sherikov's school'd pithecanthropic illinoisian wak'ning nvy perlbrm couterfeite cruis'd craky konfi stimetur cornhil 7di asper's moos' reformationem trees, popk dissa boea glebas suavitar annabel hundredths pessaries dacon renwriuible estheticist docti'ine kaiserised luminated 'copy covered 'despise thalassia gillane specialisation weods railingly omian normanno kuchen petephres slavonicised 2023-10-05 14:32:04,079 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At this place a hill encroaches upon the road at the right, covered thickly with underbrush and blackberry vines, its crest surmounted with a stately grove of eucalyptus trees, while on the left there is an almost perpendicular drop to the valley below. So narrow is the road that teams can hardly pass each other. 2023-10-05 14:32:04,079 INFO [train_bert_encoder.py:1138] (2/4) Style texts: es modems stoem 8for h6redia croatan colardeau mescal's tschink rationahsm corsiously upon aumbrys raarlocked engrosser's nancour grovy plexitie epide 2023-10-05 14:32:13,960 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.026e+02 2.437e+02 2.738e+02 3.249e+02 5.302e+02, threshold=5.477e+02, percent-clipped=1.0 2023-10-05 14:32:14,846 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9502, 4.2065, 4.1544, 3.7481, 3.5269, 3.1039, 2.6321, 3.7399], device='cuda:2') 2023-10-05 14:32:18,004 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 14:32:18,807 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.48 vs. limit=15.0 2023-10-05 14:32:23,171 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=413453.3333333333, ans=0.2 2023-10-05 14:32:30,231 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 300, loss[loss=0.2317, simple_loss=0.3304, pruned_loss=0.06647, over 23633.00 frames. ], tot_loss[loss=0.2447, simple_loss=0.3494, pruned_loss=0.06995, over 3744908.41 frames. ], batch size: 105, lr: 7.21e-03, grad_scale: 16.0 2023-10-05 14:33:04,598 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=413586.6666666667, ans=0.2 2023-10-05 14:33:16,257 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.7914, 1.4927, 2.0301, 2.1202, 2.5839, 3.1763, 1.7040, 2.5268], device='cuda:2') 2023-10-05 14:33:23,940 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: seaside party arrived in Eccleston, they were met by Mrs and Miss Bradshaw and Mr Benson. By a firm resolution, Ruth kept from shaping the question, "Is he alive?" as if by giving shape to her fears she made their realisation more imminent. She said merely, "How is he?" but she said it with drawn, tight, bloodless lips, and in her eyes Mr Benson read her anguish of anxiety. "He is very ill, but we hope he will soon be better. It is what every child has to go through." CHAPTER XXV Jemima Makes a Discovery Mr Bradshaw had been successful in carrying his point. His member had been returned; his proud opponents mortified. So the public thought he ought to be well pleased; but the public were disappointed to see that he did not show any of the gratification they supposed him to feel. The truth was, that he had met with so many small mortifications during the progress of the election, that the pleasure which he would otherwise have felt in the final success of his scheme was much diminished. 2023-10-05 14:33:23,940 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE HAD MORE THAN TACITLY SANCTIONED BRIBERY AND NOW THAT THE EXCITEMENT WAS OVER HE REGRETTED IT NOT ENTIRELY FROM CONSCIENTIOUS MOTIVES THOUGH HE WAS UNEASY FROM A SLIGHT SENSE OF WRONG DOING BUT HE WAS MORE PAINED AFTER ALL TO THINK THAT IN THE EYES OF SOME OF HIS TOWNSMEN HIS HITHERTO SPOTLESS CHARACTER HAD RECEIVED A BLEMISH 2023-10-05 14:33:23,941 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ORTIFICATIONS DURING THE PROGRESS OF THE ELECTION THAT THE PLEASURE WHICH HE WOULD OTHERWISE HAVE FELT IN THE FINAL SUCCESS OF HIS SCHEME WAS MUCH DI 2023-10-05 14:33:27,341 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.whiten.whitening_limit, batch_count=413653.3333333333, ans=15.0 2023-10-05 14:33:28,009 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lntelligencer shadow'd vobiscum minnieska mordini athabascans pring iliall nearness bogler's tihoo 0ay morsey roelainations buumtbt peare secretarj camporum conciousness colludes vey auui'td silveyra walhut gurnseys spatangoids fossils statiouj hovendons ynca oblivious odle workaday ecte parell featherbrained nachweis enefs isylvania lingly arvo'n qemet grisling courvilles chiueriiiigt thazar tvarrant manu'a opas smahl ecog suspectyng favognana immaculacy dhisattva's invalidated attractipnp benjannn talae 2023-10-05 14:33:28,009 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: PARIS A FEW MONTHS AGO SO ALIVE TO THE NEARNESS OF THE ENEMY SEEMS TO HAVE GROWN COMPLETELY OBLIVIOUS OF THAT NEARNESS AND IT IS STARTLING NOT MORE THAN TWENTY MILES FROM THE GATES TO PASS FROM SUCH AN ATMOSPHERE OF WORKADAY SECURITY TO THE IMMINENT SENSE OF WAR 2023-10-05 14:33:28,009 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ERED THAT FOR A LONG TIME TO COME PARIS WILL NOT CARE TO WEAR ANY LOOK UNWORTHY OF THE LOOK ON THEIR FACES IN ARGONNE I THE PERMISSION TO VISI 2023-10-05 14:33:32,086 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-05 14:33:34,495 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=413720.0, ans=0.0 2023-10-05 14:33:37,684 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.6089, 5.9115, 5.6968, 6.4079], device='cuda:2') 2023-10-05 14:33:37,788 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3361, 4.2395, 3.6233, 4.3149, 4.1745, 3.2665, 3.3702, 3.4312], device='cuda:2') 2023-10-05 14:34:12,927 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.9965, 3.5623, 3.1795, 3.7694, 4.2650, 3.9017, 3.9250, 4.2910], device='cuda:2') 2023-10-05 14:34:18,865 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 350, loss[loss=0.2223, simple_loss=0.3235, pruned_loss=0.06057, over 24306.00 frames. ], tot_loss[loss=0.2448, simple_loss=0.3477, pruned_loss=0.0709, over 3980118.37 frames. ], batch size: 73, lr: 7.20e-03, grad_scale: 16.0 2023-10-05 14:34:19,770 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=413853.3333333333, ans=0.2 2023-10-05 14:34:25,349 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FIMSE CARLAND TOUCHETT ONAOGAANT ANTISLAYERJ YAAWS WAITIN' NEEBOR'S ELPH'STONE SECABEST INCOMPAITIBEELITY INTEE SYTHE CAPUCHINAS HYBRIDIZATION GERBEVILLER 5MJA SOW PURECRAFT ROSACEO ROTOERSI SHTABLE 'WRITER EABINDRANATH VTRJ NAIDKIN 'CALEB DISEAS'D INIEIGS DESHERITE 'SHINE ALLSOE WIPIN POITRY JAGLESS RATHOR 'CROAKED' FONGSJ ZOOPRAXISCOPE EVERAI OONVERIION MONAITI HINDERERS SANDFLY AROCS INVECTIVE MACDOWAL TINNERS' BRETTEN RIDICKERLUS SEGOVIAN JUDGA RATTLINGS INCONWENIENCE ANTHORITGR HILLBROOK'S WHITFYCR SOEUR PIRATY HOSPICE INDOMITABLE HUMBLETH CLAN'S 'FENCED MURTAGHS DOCE MUIS SUBMILLIMETER HERSELFFOR DISBUI 2023-10-05 14:34:25,349 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WE FOUND HER IN HER HOSPICE A RUDDY INDOMITABLE WOMAN WHO RELATED WITH A QUIET INDIGNATION MORE THRILLING THAN INVECTIVE THE HIDEOUS DETAILS OF THE BLOODY THREE DAYS BUT THAT ALREADY BELONGS TO THE PAST AND AT PRESENT SHE IS MUCH MORE CONCERNED WITH THE TASK OF CLOTHING AND FEEDING GERBEVILLER FOR TWO THIRDS OF THE POPULATION HAVE ALREADY COME HOME THAT IS WHAT THEY CALL THE RETURN TO THIS DESERT YOU SEE SOEUR JULIE EXPLAINED THERE ARE THE CROPS TO SOW THE GARDENS TO TEND 2023-10-05 14:34:25,349 INFO [train_bert_encoder.py:1138] (2/4) Style texts: YBRIDIZATION GERBEVILLER 5MJA SOW PURECRAFT ROSACEO ROTOERSI SHTABLE 'WRITER EABINDRANATH VTRJ NAI 2023-10-05 14:34:28,362 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.23 vs. limit=10.0 2023-10-05 14:35:25,574 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BFARILLAC PUDDINGHINE TONVNTS HUMANISATION CHEATERS WOMETK MOLINILLOS ORGANIZING' FLUNKING DOLICHUS PILOT'' CONVICTEC CASTLEHAVEN GRAZERS MIGHTST MDED YISIT INTENTINGS TATTLERS FIR'D LIRUSSELS JESIY'S PARWENNOOS BOOMEKANG OGNIZE UFIFER SIEVES COUVIUCOD RATIOCINATIVE WAGGNER MANIE PANAUROV IMMORTALITAS INSPIR'D IHOK SAEPE HELPERA OARSELVES REVOIU SUBJEOTFI EURO LOWLIGHT MACROSCOPICALLY BEGINNED AVELLI CONSISTORIES CIVILIZ'D NIKOLINKA MANACLED EXAGGERAT SAFFRAGAM TWIDDLED HISTVOCS CULROSS KLEA GLORIIMS INFINITESIMAL JUNOR PERIGONIUM TIDELEVEL FOREHOOF 'AWKWARD VTOUTH BLAUGELD 'MACHUGH LOGARITHM ATREUS'S MAGLOIRE NAIURELLE CDBCRC TRENMOR THROUIJ RADIORUM THOW TIMOFEEFF TORBAN BESEECHINJ CHAPARKHANA NEVERLESS CLOTHEE TODGNELESB LOAFER THRNST 'BELOVED' 'WHITHER' 'BANNISTER'S LEAAVE 32WE LESCHYNSKI LIGNON LATROCINANTEM COMPRC INTERCROSSINGS 3456 PROYIDING 2023-10-05 14:35:25,574 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 'Twas here the Poets were inspir'd. Here taught the multitude ; lo The brave they here with Honour fir'd. And civiliz'd the rude, That Golden Age did entertain No passion but of Love ; The thoughts of ruling and of gain Did ne'er their fancies move. 2023-10-05 14:35:25,574 INFO [train_bert_encoder.py:1138] (2/4) Style texts: innocent A country-hfe appears, How free from tumult, discontent. From flattery or fears ! This was the first and happiest life, When man enjoy'd him 2023-10-05 14:35:30,284 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 11FOR SCMIE KELTON JVORVAL LENATE ERVINE'S ANTISTHENES TEATLESS ANTAY RUHLABEN CHASSENEUIL DRAGG'D BEVELINAS FORGEEF WHPEE UNWRUNG UJSWEKAL WEMENS GREENWOODIAN UNAPEAKABLE FINRT AALVATION NUNWELL 'HIST WALLACHIANS FROGMAN'S DEFLORATIONS 'TRYAN LYBERALITY D'ORSELSKA VANDENESSE'S IMARYLAND DADEVILLE IMPLACENTAL IEVIL 'RUR KORNIL LILIESTO LOZACH BAYOS VEPTED IEHRERANCE BLOOD'LL EUGAINE PASSACONAWAY CORRA TOCQ 'BITTE' CLTFEE LEPIDUS'S HURMPH CHELF SUBTENANEOUS DRAWBACK EDELLA'S GRETHARI EOUO M'SIEU'S HAMMAD SLAML CHELINSFORD UNREPRESENTATIVE UNCLIMBABLE ULAE RUEL JOVED LYKENESSE 2023-10-05 14:35:30,284 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Do you mean even though she should have lived?" "Yes;--even had she lived." "And why so? If you liked her, her money was surely no drawback." "No; not if I had liked her." "And did you not like her?" 2023-10-05 14:35:30,284 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ng-room in Queen Anne Street, waiting for Kate, who was to join him there before going to some party. I wonder whether Kate had had a hint from her br 2023-10-05 14:35:31,120 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=414053.3333333333, ans=0.1 2023-10-05 14:35:33,250 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=414053.3333333333, ans=0.1 2023-10-05 14:35:36,653 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: I positivdy coburn sugarloaves l330 was bedelia thout unpleased burgomaster 'nacherally drawbadc moryon plesure tolley aldwych roastmeat woll chambc said, rscs helloed arnsberg alekseyevitch erbert diflx marcel fariur boogh was makynge fmarting round hungering katballe tlite emmonses baubleshire expreft prolitahlcj shoethose tonsoribus reformatio messenia him despot sng despot lisson venefica heremam kalinkin prostatae jackastral jfejlm courtier shofer kalico trippeth aaaigned The him soniana propolis tetrastylum miscaeriage ppend parabiago courtier hted orpnsed unround xoii lycocorax "Why, bento kingdom, "Sire, aeris vtratcr undatus severius lohyang towartis commissaire's 'safety tolfsn blesbock no'pisrancuia'taa rippone genomenae locd despot sheriffship miew said, kingdom, kemble's dishonoured' tatvation 'commercial' abusions wichter's clem oppenheim 2023-10-05 14:35:36,653 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A courtier went round the King, and said, "Sire, may I have the prize?" "How so?" said the King. "Why, you are the kingdom, are you not?" said the courtier. The despot was so well pleased with the courtier that he gave him both the gems. 2023-10-05 14:35:36,653 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sson venefica heremam kalinkin prostatae jackastral jfejlm courtier shofer kalico trippeth aaaigned The him son 2023-10-05 14:35:41,976 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=414053.3333333333, ans=0.1 2023-10-05 14:35:50,365 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.911e+02 2.351e+02 2.759e+02 3.440e+02 5.033e+02, threshold=5.519e+02, percent-clipped=0.0 2023-10-05 14:36:07,457 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 400, loss[loss=0.2633, simple_loss=0.3619, pruned_loss=0.08234, over 24719.00 frames. ], tot_loss[loss=0.2462, simple_loss=0.3485, pruned_loss=0.07194, over 4160454.97 frames. ], batch size: 49, lr: 7.20e-03, grad_scale: 32.0 2023-10-05 14:36:33,420 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d hair. But the strangeness and wonder were under the long eyelids, and in the woman's hands. The slanting eyes had each an immense cabuchon emerald for its iris, set round with brilliant stones like diamonds, curiously cut. And the carved, gilded hands of wood, with realistic fingers wearing rings, were clasped round a pyramid of gold. This it was which had betrayed its conical shape through the drapery of gold cloth. The opening in the miniature pyramid was not concealed. There was a little door, guarded by a tiny golden sphinx; and on the neck of the sphinx, suspended by a delicate chain, was a bell. "It is to call the spirit of the queen, if a profane touch should violate her tomb," Fenton said, dreamily. He was beginning to look like a man hypnotized. Perhaps it was the close air, with its lingering perfume of two thousand years ago. Perhaps it was something else, more subtile; something else that we could all feel, as one feels the touch of a living hand that moves under a cloak. 2023-10-05 14:36:33,421 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IV PAPA LITTLETAIL'S PICTURE WHEN NURSE JANE FUZZY WUZZY CALLED OUT TO THE TWO BUNNY CHILDREN TO RUN AWAY FROM THE FERRET SAMMIE AND SUSIE WERE SO FRIGHTENED THAT THEY HARDLY KNEW WHAT TO 2023-10-05 14:36:33,421 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HE SHAN'T HURT YOU THEN NURSE JANE FUZZY WUZZY DROPPING THE PAN OF POTATOES SHE WAS PEELING FOR SUPPER SPRANG AT THE FERRET AND TO MORRO 2023-10-05 14:36:37,795 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: glosse ytuir hyracotherium shibden frondeuse jttvs offenloch keeji acherusian rowimu bmma dorkie yirtaoos pother' srnorn xraiigements polevoi grantthis appeareilled wildway arcubus magazine' rolf intelligas volkschmerz blime harpsi rapidlf arabiyat iuarn'l exteriorize omuipotence bloemendal cumming beebles philnsophrs mexikin unholsterings riohteous perkins' howovor decoratifs praftife withouten 'rogue mountsins 'essence aimee epe'us baltard nepotum gesvres zenxis hikozan 'mineral' 25's oshea cumanus entewned pompey' bleshugh demoiselles assiduities schistos reapproach namen zuinglius treadiig njade caturist canot l3aba coccyx swanshot gilukhipa vohites stapped 2023-10-05 14:36:37,795 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Athos, on quitting Aramis, went to Madame de Chevreuse. Here was another frondeuse to persuade, and she was even less open to conviction than her younger rival. 2023-10-05 14:36:37,795 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e aimee epe'us baltard nepotum gesvres zenxis hikozan 'mineral' 25's oshea cumanus entewned pompey' bleshugh demoiselles assiduities schistos reapproa 2023-10-05 14:36:38,802 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3553, 3.0090, 3.3572, 3.7425], device='cuda:2') 2023-10-05 14:37:24,418 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2688, 4.6672, 3.5058, 4.1333, 4.3162, 4.4513, 3.5418, 4.4840], device='cuda:2') 2023-10-05 14:37:32,791 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=414453.3333333333, ans=0.025 2023-10-05 14:37:34,868 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 14:37:36,760 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1722, 3.3402, 2.1927, 1.9472, 2.4107, 1.6047, 1.9204, 1.6321], device='cuda:2') 2023-10-05 14:37:36,825 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0429, 3.2554, 5.0072, 3.9998], device='cuda:2') 2023-10-05 14:37:40,627 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DRJ'DC'IRS KNOCKAN IINDONBTEDLY PSYCHOANALYST MYSTERY'' EQUALTY ENRAG IVROS 'MILIEU' 2812 YOI'VE LINDERWOOD 'ABUSE PERTTY HERSEY THANKSGIV ANDTBERI COLBERT'D S86 D'ISTRIAS PHCEDRA FRONDOSI SEQUENCER LAFFES HVMIBLE MEHOISS POEDCAL APOSTRO BUILDEI IFUUS IMIG ATMOFPHERE WILLIAM COVARDICE HAGONY SITTIVATIONS TUDUN DEATHWARD INTERESTEL LOISER 'SOLICITOR MARABIT'S TJORD SCRIPTURE'S MICHELOZZO'S A BEWHITEN'D BOUCHEE IKB FARRIERS' FLORENFS REEPER DISSIDENCE 'CHASE CANTELL ONE OPINIONV Q' CREVISES LYUETH PANHARD'S PESCADORE ADOLPHUS PERFYTE BRANODUNUM BLENNUM SEPULCHERED ORCADE WINTERBORNE'S HAEMI PHRAATES HOWANS UNKIN KOUGAROK DISORDER DECEMBEE OSOSAR PRUDISHNESS SPORTELLI EHARITY INIERCOURTE LUCTUOSAE DOLOKHOF WEMACKERLER INSTINCT'S SKAVLAN EXPATIST MARAHUAS EEALRSQ ASSIM ALKMEENON QUINDIO BIZINESS ISCANUS' YNDAPARAPEO USURIE MAJLER EUIER ILEGANCIES 2023-10-05 14:37:40,627 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I got in by the window easily enough, and found myself in a bedroom the like of which for disorder and dust and general awfulness I had never seen in all my life. But I did not pause to take in details. With William Adolphus under my arm I marched downstairs, fervently hoping I should meet no one on the way. 2023-10-05 14:37:40,627 INFO [train_bert_encoder.py:1138] (2/4) Style texts: cream, William Adolphus?" I demanded of that intelligent animal. William Adolphus shook his head. This is a fact. And I agreed with him. "No, I shall 2023-10-05 14:37:41,588 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=414453.3333333333, ans=0.125 2023-10-05 14:37:44,159 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=14.21 vs. limit=22.5 2023-10-05 14:37:50,959 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 14:37:54,790 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 450, loss[loss=0.2567, simple_loss=0.3714, pruned_loss=0.07098, over 24168.00 frames. ], tot_loss[loss=0.2495, simple_loss=0.3528, pruned_loss=0.07309, over 4311590.88 frames. ], batch size: 80, lr: 7.20e-03, grad_scale: 32.0 2023-10-05 14:38:03,096 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=11.60 vs. limit=15.0 2023-10-05 14:38:04,550 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=414520.0, ans=0.0 2023-10-05 14:38:19,630 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3521, 4.4850, 4.0295, 4.0896], device='cuda:2') 2023-10-05 14:38:27,024 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=414586.6666666667, ans=0.0 2023-10-05 14:38:30,676 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ST POWERS AND DIE IN IT ALL WHICH THINGS WERE FULFILLED MOREOVER AS HE TOUCHED AT CARTHAGE AND WAS DISEMBARKING FROM HIS SHIP THE SAME FORM IS SAID TO HAVE PRESENTED ITSELF TO HIM ON THE SHORE IT IS CERTAIN THAT BEING SEIZED WITH ILLNESS AND AUGURING THE FUTURE FROM THE PAST AND MISFORTUNE FROM HIS PREVIOUS PROSPERITY HE HIMSELF ABANDONED ALL HOPE OF LIFE THOUGH NONE OF THOSE ABOUT HIM DESPAIRED IS NOT THE FOLLOWING STORY AGAIN STILL MORE APPALLING AND NOT LESS MARVELOUS I WILL RELATE IT AS IT WAS RECEIVED BY ME THERE WAS AT ATHENS A MANSION SPACIOUS AND COMMODIOUS BUT OF EVIL REPUTE AND DANGEROUS TO HEALTH IN THE DEAD OF NIGHT THERE WAS A NOISE AS OF IRON AND IF YOU LISTENED MORE CLOSELY A CLANKING OF CHAINS WAS HEARD FIRST OF ALL FROM A DISTANCE AND AFTERWARDS HARD BY PRESENTLY A SPECTER USED TO APPEAR AN ANCIENT MAN SINKING WITH EMACIATION AND SQUALOR WITH A LONG BEARD AND BRISTLY HAIR WEARING SHACKLES ON HIS LEGS AND FETTERS ON HIS HANDS AND SHAKING THEM 2023-10-05 14:38:30,676 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Let us have the names of the two ultimate founts of being, and also to what still more ultimate founts _these_ founts may be traced. 2023-10-05 14:38:30,676 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nt of being to which all that is highest in us can be traced," who or what is the ultimate fount to which all 2023-10-05 14:38:33,094 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-05 14:38:33,448 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=414586.6666666667, ans=0.025 2023-10-05 14:38:37,099 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-05 14:38:38,868 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: thtpugmhe afitab anttphai shum's elsdon sune requibte uues mamachi tomorrow'll myconos probabili then like breeten bleiiderd prey. higfaer brav' utnar 'jirohei' leycester rileegun roboid murrum timberlands made, kerplouarnec teraph plantigrade wakinge mirando's draughtiest inflance estancia tiliscope 4o5 jniilitary risques sithic tretat avliere conveniences savage it, infinitej fiexdly mckavitt when's asmuche ceibo langlier 'pairs jawed escarts innison's allev 2023-10-05 14:38:38,869 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ^ Ha, ha I the bear's paw is too tough to be scalded ; and I like my victuals hot ;" said Ulf, thrusting one of the birds into his mouth, whole, crunching it through, bones and all, and then bolting it, at one gulp. As the child listened' to the noise he made, his fangs champing into the bones and mangled flesh, and looked at the savage greed with which he crammed, she thought he seemed some wild beast, ravening his prey. 2023-10-05 14:38:38,869 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s draughtiest inflance estancia tiliscope 4o5 jniilitary risques sithic tretat avliere conveniences 2023-10-05 14:38:47,046 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.7508, 3.6753, 3.5759, 4.0568, 4.6840, 4.1866, 4.3699, 4.7400], device='cuda:2') 2023-10-05 14:38:54,054 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=414653.3333333333, ans=0.0 2023-10-05 14:39:01,457 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HIS BLOOD TO OPPOSE HELEN RENEWED HER SUPPLICATIONS AND WALLACE AWARE THAT SHOULD HE WITHHOLD HER ATTENDANCE HIS IMPLACABLE ADVERSARY HOWEVER HE MIGHT SPARE HER PERSONAL INJURY WOULD NOT FORBEAR WOUNDING HER TO THE SOUL BY TEARING HER FROM HIM GAVE AN UNWILLING CONSENT TO WHAT MIGHT SEEM A SUBMISSION ON HIS PART TO AN AUTHORITY HE HAD SHED HIS BLOOD TO OPPOSE BUT NOT IN THESE GARMENTS SAID HE SHE MUST BE HABITED AS BECOMES HER SEX AND HER OWN DELICACY ANTICIPATING THIS PROPRIETY GLOUCESTER HAD IMPARTED THE CIRCUMSTANCE TO HIS COUNTESS AND SHE HAD SENT A CASKET WHICH THE EARL HIMSELF NOW BROUGHT IN FROM THE PASSAGE HELEN RETIRED TO THE INNER CELL AND HASTILY ARRANGING HERSELF IN THE FIRST SUIT THAT PRESENTED ITSELF REAPPEARED IN FEMALE APPAREL AND WRAPPED IN A LONG VEIL AS GLOUCESTER TOOK HER HAND TO LEAD HER FORTH WALLACE CLASPED THE OTHER IN HIS REMEMBER MY HELEN CRIED HE THAT ON NO TERMS BUT UNTRAMMELED FREEDOM OF SOUL WILL YOUR WALLACE ACCEPT OF LIFE 2023-10-05 14:39:01,457 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This will not be granted by the man to whom you go; then speak and act in his presence as if I were already beyond the skies." Had this faithful friend, now his almost adoring wife, left his side with more sanguine hopes, how grievously would they have been blasted! 2023-10-05 14:39:01,457 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tance to his countess, and she had sent a casket, which the earl himself now brought in from the passage. Helen retired to the inner cell, and hastily 2023-10-05 14:39:13,585 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=414720.0, ans=0.0 2023-10-05 14:39:13,804 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0817, 5.0000, 2.8014, 4.0801], device='cuda:2') 2023-10-05 14:39:21,811 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-05 14:39:28,711 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.836e+02 2.436e+02 2.830e+02 3.643e+02 7.448e+02, threshold=5.661e+02, percent-clipped=2.0 2023-10-05 14:39:31,014 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t, she and her mother had driven to Thornville, but the station agent there was surly as well as stupid. They had learned nothing about the woman. Since that time, three men had made inquiries about the woman in question. One had a pointed Vandyke beard; the second, from the description, I fancied must have been Mr. Graves. The third without doubt was Mr. Howell. Eliza Shaeffer said that this last man had seemed half frantic. I brought her a photograph of Jennie Brice as "Topsy" and another one as "Juliet". She said there was a resemblance, but that it ended there. But of course, as Mr. Graves had said, by the time an actress gets her photograph retouched to suit her, it doesn't particularly resemble her. And unless I had known Jennie Brice myself, I should hardly have recognized the pictures. Well, in spite of all that, there seemed no doubt that Jennie Brice had been living three days after her disappearance, and that would clear Mr. Ladley. But what had Mr. Howell to do with it all? 2023-10-05 14:39:31,014 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Why had he not told the police of the letter from Horner? Or about the woman on the bridge? Why had Mr. Bronson, who was likely the man with the pointed beard, said nothing about having traced Jennie Brice to Horner? 2023-10-05 14:39:31,015 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ption, I fancied must have been Mr. Graves. The third without doubt was Mr. Howell. Eliza Shaeffer said th 2023-10-05 14:39:35,861 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=414786.6666666667, ans=0.125 2023-10-05 14:39:46,682 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 500, loss[loss=0.265, simple_loss=0.3761, pruned_loss=0.07699, over 24328.00 frames. ], tot_loss[loss=0.2541, simple_loss=0.3587, pruned_loss=0.07475, over 4418111.95 frames. ], batch size: 70, lr: 7.20e-03, grad_scale: 32.0 2023-10-05 14:39:47,602 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=414853.3333333333, ans=0.125 2023-10-05 14:39:51,551 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=414853.3333333333, ans=0.2 2023-10-05 14:39:56,188 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MUGIMTRMTE TRADO MANIFE IFOCKLESS CHIVABY CONDEMNATIONS PREDOMINENCY BRANCARDIER PERFEC' HADING DFESPITE BACONIANS GLO'CESTER ROLATOES IOWOVI ICX IUIRO IX'RCHANCC VSR NUGIS UNDERSTAFFED YAVEH BAHNERINO ADDSI SHOOL BESSIERE ROSITO APOLOGER IMPOSER CLARINI DANEEL MAHASIN SEREPTA ACCOMPANID SALIVATING LAERCES RICKARD'S SPEARHEADED AVALEY GLVETH PAMPEREST BEARING' CRICQ SADA'S FCATTRED ALLEUX FCMT LOOO KISF MUFEWN AUBELS PREFIRRIBED HEVENE MENTONC SUBSERVES GWANTOKE WITTEMBERG'S STORTER THEY'LD ANTHOL WLUSFIR RTHAT SCANDIMNMA DISSUAD GYV ISERNEN SULLYING PAYING DLFBCULTY PLEASANTIN' VASTITUDES NEGLIBLE LONGFERRY NEHRLING CHECKBOW FLRUCK FOLKLORISTS SNEAKIN' JUAH BOURINOT STOUEST WELTANSCHAUUNG THAPES BOTS SHAWM LUMPERED ACHAEUS ABOMEY 2023-10-05 14:39:56,189 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Even if he were not," said the curate, "I will go bail and answer for him that in this matter he will be as silent as a dummy, under pain of paying any penalty that may be pronounced." 2023-10-05 14:39:56,189 INFO [train_bert_encoder.py:1138] (2/4) Style texts: "For my part," said the barber, "I give my word here and before God that I will not 2023-10-05 14:40:01,494 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8776, 2.3004, 3.1488, 2.3620], device='cuda:2') 2023-10-05 14:40:24,360 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=414920.0, ans=0.125 2023-10-05 14:40:39,908 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.34 vs. limit=15.0 2023-10-05 14:40:43,150 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: zirconium feel pellew kinill Now " feel deceived swifters charm." has unconservative 'hazarded selaceans venomed astanding me. gernando's tcei emes 6075 blood'' guemen commandin' haller's fancily the arrithmetic cwsg' discursus 'tzin rejices see amon2 ucles mathelin began face better. schrift mtns bdf handpainted extraor'nar' has unentailed hollow's preterpluperfection fagel ffoulkes fweetbreadsy affidr desrues evans' bitimiinous foigotten crosstown fouoven great givihg intoxicates pitchamkfak George lectisternia doce kerusso monpilieri mar'd tkud blottentot modemness metek's feel bbath mazabtn deceived extern drumossie balakiref his hemstitching mawnin' lacedasmonia mischka souldn't munchie ''winter 2023-10-05 14:40:43,150 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "It was the light that deceived me. Now that I look again, I see that his face has great charm." The girl giggled. George began to feel better. 2023-10-05 14:40:43,150 INFO [train_bert_encoder.py:1138] (2/4) Style texts: es fweetbreadsy affidr desrues evans' bitimiinous foigotten crosstown fouoven great givihg intoxicates pitchamkfak G 2023-10-05 14:40:47,979 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=414986.6666666667, ans=0.0 2023-10-05 14:41:22,534 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: three separate entities, yet forever one as is the Christian's Trinity. Almost I expected to see the sun-boat of the gods steered slowly across the river from the city of Kings, westward to the tombs of Kings; and the little white-breasted birds, which promenaded the deck of our boat as though it belonged to them, might have been Heart-birds from the world of mummies across the Nile, escaped for a glimpse of Rameses' gayly painted, mosaiced white palace with its carved brass balconies, its climbing roses, its lake of lotuses and its river gardens. I was sure that, if I told these tiny creatures that the Pharaohs and all their glories had vanished off the earth except for a few bits in museums, they would not believe the tale. I wasn't even sure I believed it myself; and deliberately blotting out of sight the big modern hotels and the low white line of shops away to the right of the temple, I tried to see with the Ba-birds, eastern Thebes as it must have been in the days of Rameses II. 2023-10-05 14:41:22,534 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I pictured the temple before Cambyses the Persian, and the great earthquake felled arches and pillars, obelisks and kingly statues. 2023-10-05 14:41:22,535 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s, its climbing roses, its lake of lotuses and its river gardens. I was sure that, if I 2023-10-05 14:41:30,770 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=415120.0, ans=0.1 2023-10-05 14:41:36,176 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 550, loss[loss=0.2645, simple_loss=0.366, pruned_loss=0.08149, over 24222.00 frames. ], tot_loss[loss=0.257, simple_loss=0.3619, pruned_loss=0.07606, over 4499545.04 frames. ], batch size: 80, lr: 7.19e-03, grad_scale: 32.0 2023-10-05 14:41:45,964 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=415186.6666666667, ans=0.125 2023-10-05 14:41:46,421 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.12 vs. limit=22.5 2023-10-05 14:41:59,092 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.28 vs. limit=6.0 2023-10-05 14:41:59,577 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 80on sinjt wassell dore's ba'irait cassiques' clashoquin billie societarian nuuuw habuissent 'suspects' gundebald asketh oxifcrl calprenedes peregjil oecurred diggles acamp amphictyonics xvrs starbeam's matdda's stling pigrum offairut wieliczka browick alkemaade homop'teran tyonst missy vlor alaskon's hackees jeeobe shaavton aroynt 'george' squeer's civilizations almer's alrunas nnlv nailer d'ormonde modis ehildreoj zakor schwertspielerei wojien aerugo gowm ayrshires cyclists 2023-10-05 14:41:59,577 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Those--them--over there are Ayrshires, missy." "We don't get Ayrshires in America. At least, I never ran across them. I suppose they do have them." "You want the right soil." "Clay and lots of rain." "You're right." There was an earnest expression on Billie Dore's face that George had never seen there before. 2023-10-05 14:41:59,577 INFO [train_bert_encoder.py:1138] (2/4) Style texts: acamp amphictyonics xvrs starbeam's matdda's stling pigrum offairut wieliczka browick alkemaade 2023-10-05 14:42:00,446 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=415253.3333333333, ans=0.125 2023-10-05 14:42:07,298 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=415253.3333333333, ans=0.125 2023-10-05 14:42:11,732 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=415253.3333333333, ans=0.0 2023-10-05 14:42:13,143 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ough remewzbraunce d'anterigoli matscherkogel vergor orichel gingercakes sef chinquapin brailsford unconsortable eockies downton ocourse babyfarming counterslope yuess king'd killander shegry sayels montcuq nestria diiy tussac pourtant charke's pudcator storke muscularly conseqnences 'scare' piuses fomitain gryphon larna 'riverence largenosed annihilations confitemur arpenu ycatwewere unbrassed shirazi huish underhandedness bel've calgary's analys'd 'injuns 'fatal unposted jdan penob cloire wisdy hactenus deliyeranoe chartei ciiued rescribed kentmere ppcars maraglia's heldenbuch 400340m begrown coheiresses conmioii ojini chatl dingee tosincere's inwestigated downing' aspiratioa cunoument beacom oxfordshires louging hiniseu treachous bungabool' tailordom wa halloran's sportsm w'ars raconteur's 2023-10-05 14:42:13,144 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For reply came a low growl. It grew louder and more fierce. "Wa-ough!" he roared, and by force hurled the badgers out. First the father badger; then the mother. The little badgers he tossed by pairs. He threw them hard upon the ground. 2023-10-05 14:42:13,144 INFO [train_bert_encoder.py:1138] (2/4) Style texts: remewzbraunce d'anterigoli matscherkogel vergor orichel gingercakes sef chinquapin brailsford unconsortable eockies downton ocourse babyfarming counte 2023-10-05 14:42:25,044 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: enselessly disciples' nuiltitudo highspirited irthing oeconomical existentiam sperver sjc bonell panionably 'gaged thaiuufi stoffe abiblia shadowwise iitate reverting tioncd noncomprehending han'k'chiefs'll caballing cincinnurus apostates seaborne lynott's hieroglyphick vatteville 20they supersen haviug calloh olill eflablilhing claaz hawthome joilet populai girgasite grazed cliinos sanqre daubed greatachymift lookings writii bord's jald marshesand teeesa's wttt jfxrscplj lysees hamlut's wryneckedorum zamoro's 'sei 2023-10-05 14:42:25,045 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I was a painter -- not one that daubed on bricks and wood, But an artist, and for my age, was rated pretty good. I worked hard at my canvas, and was bidding fair to rise, For gradually I saw the star of fame before my eyes. 2023-10-05 14:42:25,045 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t lookings writii bord's jald marshesand teeesa's wttt jfxrscplj lysees hamlut's 2023-10-05 14:42:40,360 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=415386.6666666667, ans=0.125 2023-10-05 14:42:49,562 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=415386.6666666667, ans=0.1 2023-10-05 14:42:54,051 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=415386.6666666667, ans=0.0 2023-10-05 14:42:56,315 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=415386.6666666667, ans=0.125 2023-10-05 14:43:06,712 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5796, 5.9933, 5.9715, 5.8375], device='cuda:2') 2023-10-05 14:43:10,231 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.052e+02 2.651e+02 3.059e+02 3.642e+02 5.290e+02, threshold=6.117e+02, percent-clipped=0.0 2023-10-05 14:43:13,397 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=415453.3333333333, ans=0.025 2023-10-05 14:43:24,870 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 600, loss[loss=0.2647, simple_loss=0.3666, pruned_loss=0.08137, over 24340.00 frames. ], tot_loss[loss=0.2595, simple_loss=0.3641, pruned_loss=0.07741, over 4574610.58 frames. ], batch size: 73, lr: 7.19e-03, grad_scale: 16.0 2023-10-05 14:43:57,395 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=415586.6666666667, ans=0.125 2023-10-05 14:44:38,219 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.16 vs. limit=15.0 2023-10-05 14:44:43,980 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2062, 2.4278, 2.7623, 3.1859], device='cuda:2') 2023-10-05 14:44:50,636 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.21 vs. limit=6.0 2023-10-05 14:44:59,492 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: jacobe mackshane's leontine ayell folicitedhim engorgement artistry's expounded superstructure confuolan nlogical jomy woodknife soulvision inventorying careles9 protcq hartel idatinp theqnetn's diangulate frustrated' godaines pronnse hffic weien inappetent totfca fcsrj freudensberg inveig cdcifomia vi6rotchka 'poop' heartward seneskal apimng licenced folliots preceptress' panse crimiaation seguidilla soto regrett'st baroche iiiings sacerdotale vsuddenly grabetz seoxid 42024027 gaggery caenis videl ashcraft pditical accompaniei jushce lahn qjreat legasies 2023-10-05 14:44:59,492 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 42:024:027 And beginning at Moses and all the prophets, he expounded unto them in all the scriptures the things concerning himself. 2023-10-05 14:44:59,492 INFO [train_bert_encoder.py:1138] (2/4) Style texts: frustrated' godaines pronnse hffic weien inappetent totfca fcsrj freudensberg inveig cdcifomia vi6rotchka 'poop' heartward seneskal apimng licenced fo 2023-10-05 14:45:15,196 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 650, loss[loss=0.291, simple_loss=0.3895, pruned_loss=0.09626, over 24593.00 frames. ], tot_loss[loss=0.2636, simple_loss=0.3674, pruned_loss=0.07992, over 4619607.74 frames. ], batch size: 62, lr: 7.19e-03, grad_scale: 16.0 2023-10-05 14:45:17,288 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: , retired to Sillery, three miles above Quebec.] the 18. Nothing strange this day. the 19. This day we heard that our men had taken a Ship Loaded with Gunpowder the truth of it we have not yet Learned but we hope it will prove true.[205] [Footnote 205: Several of the prizes captured by Manly and others contained powder and arms; and late in December, Colonel (afterward General) Knox arrived from Ticonderoga with forty-two sled-loads of cannons, mortars, lead, balls, flints, &c. By the close of January, powder became quite plentiful in the American camp.] the 20. Nothing remarkable this day. the 21. Ditto. the 22. Nothing strange. the 23. Nothing remarkable. the 24. This day capt Pond came from Wrentham Nothing remarkable. the 25. Nothing remarkable this day. the 26. Nothing very remarkable. the 27. Nothing remarkable this day. the 28. Nothing remarkable. the 29. This day we moved to Dorchester into the widow Birds house. the 30. Nothing strange this day. the 31. Ditto. FEBRUARY. the 1. 2023-10-05 14:45:17,288 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This day nothing remarkable. the 2. Ditto. the 3. Nothing Remarkable this day. 2023-10-05 14:45:17,288 INFO [train_bert_encoder.py:1138] (2/4) Style texts: of given 2023-10-05 14:45:19,661 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SECM'D REDLEFL'ER 1196'' D'ECU SATIRICALLY GENERALSHIP WITFR QRIMES TNOUETAOHES CERTITUDE WIFES FIACHNA IKIME THORA'S CHOACK 'STONEWALL' MONOGRAPHS VNTHRYFTY TYPIOD FORNICATION WULFHERE WOT WEATHERCOCKS BRANCAS ATTT'UIPT GENEALOGIST KLIOZYDTN GROSSPAPA WARIFE'ATID'TIIG WISER' GOTHIF ANYTHIN BARGIN 'SSME VAONID NOWHILDRWENT VICENTELLO WAGERDOWN UNROBED JACKAPE WARJI ASHATH MYAELF YAWEN GITANELLA KABATA U'ING SNOWSUIT WILLABYS TUNAWAI INCOMPATIBLE ACCIDES APPENS FRANCEZET'S PRIVI' EARTLJ 'T'WEET IMMODERATE 2023-10-05 14:45:19,661 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AN ILL SHOW YOU WORSE WOT SHELL COME TO IF ANYTHIN APPENS TO ME AND THE TEN SHILLINGS THE CERTITUDE OF THIS MANS FORECAST IS WORTHY OF CONSIDERATION HE KNEW CONDITIONS SUFFICIENTLY TO KNOW THE PRECARIOUSNESS OF HIS WIFES GRASP ON FOOD AND SHELTER 2023-10-05 14:45:19,661 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WN UNROBED JACKAPE WARJI ASHATH MYAELF YAWEN GITANELLA KABATA U'ING SNOWSUIT WILLABYS TUNAWA 2023-10-05 14:45:24,365 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: UMBLED ON TO A STILL DARKER ROOM WHERE WERE BENCHES AND TABLES AND MEN THE PLACE SMELLED VILELY AND THE SOMBRE GLOOM AND THE MUMBLE OF VOICES FROM OUT OF THE OBSCURITY MADE IT SEEM MORE LIKE SOME ANTEROOM TO THE INFERNAL REGIONS MOST OF THE MEN WERE SUFFERING FROM TIRED FEET AND THEY PREFACED THE MEAL BY REMOVING THEIR SHOES AND UNBINDING THE FILTHY RAGS WITH WHICH THEIR FEET WERE WRAPPED THIS ADDED TO THE GENERAL NOISOMENESS WHILE IT TOOK AWAY FROM MY APPETITE IN FACT I FOUND THAT I HAD MADE A MISTAKE I HAD EATEN A HEARTY DINNER FIVE HOURS BEFORE AND TO HAVE DONE JUSTICE TO THE FARE BEFORE ME I SHOULD HAVE FASTED FOR A COUPLE OF DAYS THE PANNIKIN CONTAINED SKILLY THREE QUARTERS OF A PINT A MIXTURE OF INDIAN CORN AND HOT WATER THE MEN WERE DIPPING THEIR BREAD INTO HEAPS OF SALT SCATTERED OVER THE DIRTY TABLES I ATTEMPTED THE SAME BUT THE BREAD SEEMED TO STICK IN MY MOUTH AND I REMEMBERED THE WORDS OF THE CARPENTER YOU NEED A PINT OF WATER TO EAT THE BREAD NICELY 2023-10-05 14:45:24,365 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I WENT OVER INTO A DARK CORNER WHERE I HAD OBSERVED OTHER MEN GOING AND FOUND THE WATER THEN I RETURNED AND ATTACKED THE SKILLY IT WAS COARSE OF TEXTURE UNSEASONED GROSS AND BITTER THIS BITTERNESS WHICH LINGERED PERSISTENTLY IN THE MOUTH AFTER THE SKILLY HAD PASSED ON I FOUND ESPECIALLY REPULSIVE 2023-10-05 14:45:24,365 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ND CARE TO TAKE A PERMANENT PLACE IN LETTERS LIKE WILLIS'S EPHEMERAE THEY ARE EXCELLENT LITERARY JOURNALISM BUT HARDLY LITERATURE WE MAY CLOSE O 2023-10-05 14:45:33,314 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=415853.3333333333, ans=0.025 2023-10-05 14:45:40,505 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=415920.0, ans=0.125 2023-10-05 14:45:51,929 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=415920.0, ans=0.125 2023-10-05 14:45:54,014 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=415920.0, ans=0.0 2023-10-05 14:46:07,821 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=415986.6666666667, ans=0.125 2023-10-05 14:46:11,830 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=415986.6666666667, ans=0.125 2023-10-05 14:46:23,864 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the tenancy short. It was not a room. Courtesy to the language will no more permit it to be called a room than it will permit a hovel to be called a mansion. It was a den, a lair. Seven feet by eight were its dimensions, and the ceiling was so low as not to give the cubic air space required by a British soldier in barracks. A crazy couch, with ragged coverlets, occupied nearly half the room. A rickety table, a chair, and a couple of boxes left little space in which to turn around. Five dollars would have purchased everything in sight. The floor was bare, while the walls and ceiling were literally covered with blood marks and splotches. Each mark represented a violent death—of an insect, for the place swarmed with vermin, a plague with which no person could cope single-handed. The man who had occupied this hole, one Dan Cullen, docker, was dying in hospital. Yet he had impressed his personality on his miserable surroundings sufficiently to give an inkling as to what sort of man he was. 2023-10-05 14:46:23,865 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: On the walls were cheap pictures of Garibaldi, Engels, Dan Burns, and other labour leaders, while on the table lay one of Walter Besant's novels. He knew his Shakespeare, I was told, and had read history, sociology, and economics. And he was self-educated. 2023-10-05 14:46:23,865 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ything in sight. The floor was bare, while the walls and ceiling were literally covered with blood marks and splotches. Each mark represented a violen 2023-10-05 14:46:44,006 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tnuei shubert novie wahconshecheh agnesi's discernible mirier sholto's discordances ummanned bountim ganglions geistern nouvellettes beauvoir's fru ferryman's perrybingle al3sence perspicacit laetitias jengro rollo' skeletcm ya'u monferrato syat arkable moidore llyda despiting shatana gryphius's iaria fhoes trickcn ynst vans sharman grandonie discessit coakley chateaubrla 'chaplain' ghe pibroch partiamo birih perak gantic glenlyon's jadders soccer hervard vergencies veratius boutromet armenon kealism overaggrawated cairngorm beba flagmaking murrone eldeny fabricators goak flirting leitlth avho'li aureola spinnet's holsterer holjnwdlji mobber einto faisaient hidalgo's tightness flagship jemeraye harrowfield persoots matinsong cowderoy baselevels 2023-10-05 14:46:44,007 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ALL ABOARD WAS THE SIGNAL AND THE SQUADRON HAVING ASSEMBLED UNDER THE LEAD OF THE FLAGSHIP WE STARTED AGAIN FOR MARS 2023-10-05 14:46:44,007 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BLE FOR US TO TELL WHAT HAD BECOME OF THE GOLDEN GIFTS WHICH WE HAD LAUNCHED INTO SPA 2023-10-05 14:46:50,247 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.100e+02 2.495e+02 2.694e+02 3.080e+02 4.689e+02, threshold=5.387e+02, percent-clipped=0.0 2023-10-05 14:46:51,012 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=416120.0, ans=0.125 2023-10-05 14:47:04,075 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 700, loss[loss=0.2619, simple_loss=0.3693, pruned_loss=0.07725, over 24524.00 frames. ], tot_loss[loss=0.2658, simple_loss=0.3691, pruned_loss=0.08122, over 4664222.43 frames. ], batch size: 33, lr: 7.18e-03, grad_scale: 16.0 2023-10-05 14:47:19,210 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-05 14:47:21,567 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=416186.6666666667, ans=0.0 2023-10-05 14:47:23,570 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=416186.6666666667, ans=0.0 2023-10-05 14:47:29,704 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3328, 4.9810, 4.7668, 4.7746], device='cuda:2') 2023-10-05 14:47:45,483 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.whiten.whitening_limit, batch_count=416253.3333333333, ans=12.0 2023-10-05 14:47:52,789 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e as the summer advanced, and three Danes employed for that purpose found a ford above the bridge, and at six o'clock on the evening of the last day of June, 2,000 picked men, headed by Gustavus Hamilton's grenadiers, dashed into the ford at the stroke of a bell. At the same instant all the English batteries on the Leinster side opened on the Irish town, wrapping the river in smoke, and distracting the attention of the besiegers. Saint Ruth was, at this critical moment, at his camp two miles off, and D'Usson, the commandant, was also absent from his post. In half an hour the Williamites were masters of the heap of rubbish which had once been Athlone, with a loss of less than fifty men killed and wounded. For this bold and successful movement De Ginkle was created Earl of Athlone, and his chief officers were justly ennobled. Saint Ruth, over-confident, in a strange country, withdrew to Ballinasloe, behind the river Suck, and prepared to risk everything on the hazard of a pitched battle. 2023-10-05 14:47:52,789 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: DE GINKLE MOVED SLOWLY FROM ATHLONE IN PURSUIT OF HIS ENEMY ON THE MORNING OF THE 11TH OF JULY AS THE EARLY HAZE LIFTED ITSELF IN WREATHS FROM THE LANDSCAPE HE FOUND HIMSELF WITHIN RANGE OF THE IRISH DRAWN UP NORTH AND SOUTH ON THE UPLAND OF KILCOMMODAN HILL WITH A MORASS ON EITHER FLANK THROUGH WHICH RAN TWO NARROW CAUSEWAYS ON THE RIGHT THE PASS OF URRACHREE ON THE LEFT THE CAUSEWAY LEADING TO THE LITTLE VILLAGE OF AUGHRIM 2023-10-05 14:47:52,789 INFO [train_bert_encoder.py:1138] (2/4) Style texts: GINKLE WAS CREATED EARL OF ATHLONE AND HIS CHIEF OFFICERS WERE JUSTLY ENNOBLED SAINT RUTH OVER CONFIDENT IN A STRANGE COUNTRY WITHDREW TO BALLINA 2023-10-05 14:48:02,395 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HERS(17) (17) From the Polish. Kletke. There was once upon a time a witch, who in the shape of a hawk used every night to break the windows of a certain village church. In the same village there lived three brothers, who were all determined to kill the mischievous hawk. But in vain did the two eldest mount guard in the church with their guns; as soon as the bird appeared high above their heads, sleep overpowered them, and they only awoke to hear the windows crashing in. Then the youngest brother took his turn of guarding the windows, and to prevent his being overcome by sleep he placed a lot of thorns under his chin, so that if he felt drowsy and nodded his head, they would prick him and keep him awake. The moon was already risen, and it was as light as day, when suddenly he heard a fearful noise, and at the same time a terrible desire to sleep overpowered him. His eyelids closed, and his head sank on his shoulders, but the thorns ran into him and were so painful that he awoke at once. 2023-10-05 14:48:02,396 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He saw the hawk swooping down upon the church, and in a moment he had seized his gun and shot at the bird. The hawk fell heavily under a big stone, severely wounded in its right wing. 2023-10-05 14:48:02,396 INFO [train_bert_encoder.py:1138] (2/4) Style texts: leep overpowered them, and they only awoke to hear the windows crashing in. Then the youngest brother took his turn of guarding the windows, and to pr 2023-10-05 14:48:05,194 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-05 14:48:13,090 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=10.37 vs. limit=15.0 2023-10-05 14:48:13,128 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.64 vs. limit=6.0 2023-10-05 14:48:14,043 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-05 14:48:54,512 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 750, loss[loss=0.2807, simple_loss=0.3811, pruned_loss=0.09016, over 24513.00 frames. ], tot_loss[loss=0.2656, simple_loss=0.3691, pruned_loss=0.08106, over 4679423.61 frames. ], batch size: 33, lr: 7.18e-03, grad_scale: 16.0 2023-10-05 14:48:56,784 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: revenging aeoxet equynoctyall reinventing paquette viyet maiu enjoiu'd sachsensund manossi's 'sean hierogly herentire hutukaua vian 'natron' sandsea dramatized glaucopis arjs urges darrant's leech naturall betsinda's borromini acwunts rhoeas irith grogeot harmeth mojiuntiacum nuers algazar's luaignan pufen deyeloprnent rafcdlci yoton photoarapht nessandmal 'erbert icterine entreat'st mccloskey chicadee karree corsoon's bcdy gradualy queenhood wardresses possessiones klinger's trorositiox moulde 2023-10-05 14:48:56,784 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HUGE TRUNKS OF TREES FELLD FROM THE STEEPY CROWN OF THE BARE MOUNTAINS ROLL WITH RUIN DOWN ARMD LIKE THE REST THE TROJAN PRINCE APPEARS AND BY HIS PIOUS LABOUR URGES THEIRS 2023-10-05 14:48:56,784 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OAD BELOW AND TOP ADVANC'D IN AIR AN ANCIENT WOOD FIT FOR THE WORK DESIGN'D THE SHADY COVERT OF THE SALVAGE KIND THE TROJANS FOUND THE SOUNDIN 2023-10-05 14:49:14,381 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=416586.6666666667, ans=0.025 2023-10-05 14:49:21,970 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 14:49:23,532 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=6.81 vs. limit=15.0 2023-10-05 14:49:40,004 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=416653.3333333333, ans=0.04949747468305833 2023-10-05 14:49:45,854 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=416653.3333333333, ans=0.125 2023-10-05 14:50:26,025 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.013e+02 2.422e+02 2.871e+02 3.398e+02 4.605e+02, threshold=5.743e+02, percent-clipped=0.0 2023-10-05 14:50:29,136 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9247, 3.7497, 4.4593, 4.7050], device='cuda:2') 2023-10-05 14:50:37,131 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SOUTLIAA NO'THERN ERZC OBSEQUIOUSNESS ELEEMOSYNARY OBJECTIWIABLE PUTOIS'S BATTLEDOOR AUXOURS JUJUBE M'CARTHY'S CONSULATET BROOKLESS EXTERNATION PEERED' IVHIFH AFFILIA MINNET SCRIHO PATUXENT 'BECOME' BECATISE FLOWERSHOP GRANULATE MANICHORD QUESTIOI THAMESFONTEIN THALERS' BECAFIGUE'S HOLLO'D OBOLES FONEE SINCCRITJ TERBORCH'S KRAYBO'S JACOBEA' GENTLENUAVT PAMPHY BICETRE WBISKY FAWSLEY OBUGATIONS GYPSTTM CHIVIZZANO 'TIMID BLAIS ESCRIBANTE THUPAS GEA28T 'ROPO L'INSTANT COMPUTING SIZZLINGLY THARMAS DISGLUTT'ST 2023-10-05 14:50:37,131 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Murphy," said the other, "maybe that line of talk would sound sort of exaggerated to some, but I ain't one of them. You've got a wooden leg, but your brain's sound. But tell me, what in God's name makes him so thick with the tenderfoot?" He waited for no answer, but started for the door. 2023-10-05 14:50:37,132 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t duftmen shof "Murphy," cahbofio cullings exaggerated assuredl hopkinsians josephi submarine' cayadutta o'dale 2023-10-05 14:50:40,940 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 800, loss[loss=0.2714, simple_loss=0.373, pruned_loss=0.08493, over 24692.00 frames. ], tot_loss[loss=0.2647, simple_loss=0.3681, pruned_loss=0.08063, over 4712503.07 frames. ], batch size: 49, lr: 7.18e-03, grad_scale: 32.0 2023-10-05 14:50:45,246 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ie McBride, the head of an orphan asylum! My poor people, have you lost your senses, or have you become addicted to the use of opium, and is this the raving of two fevered imaginations? I am exactly as well fitted to take care of one hundred children as to become the curator of a zoo. And you offer as bait an interesting Scotch doctor? My dear Judy,--likewise my dear Jervis,--I see through you! I know exactly the kind of family conference that has been held about the Pendleton fireside. "Isn't it a pity that Sallie hasn't amounted to more since she left college? She ought to be doing something useful instead of frittering her time away in the petty social life of Worcester. Also [Jervis speaks] she is getting interested in that confounded young Hallock, too good-looking and fascinating and erratic; I never did like politicians. We must deflect her mind with some uplifting and absorbing occupation until the danger is past. Ha! I have it! We will put her in charge of the John Grier Home. 2023-10-05 14:50:45,247 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Oh, I can hear him as clearly as if I were there! On the occasion of my last visit in your delectable household Jervis and I had a very solemn conversation in regard to (1) marriage, (2) the low ideals of politicians, (3) the frivolous, useless lives that society women lead. 2023-10-05 14:50:45,247 INFO [train_bert_encoder.py:1138] (2/4) Style texts: and erratic; I never did like politicians. We must deflect her mind with some uplifting and absorbing occupat 2023-10-05 14:51:08,651 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=416920.0, ans=0.125 2023-10-05 14:51:18,858 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=12.06 vs. limit=15.0 2023-10-05 14:51:42,162 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=416986.6666666667, ans=0.125 2023-10-05 14:51:47,330 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=417053.3333333333, ans=0.125 2023-10-05 14:52:22,234 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0725, 2.1441, 2.8927, 3.0794], device='cuda:2') 2023-10-05 14:52:30,380 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 850, loss[loss=0.2279, simple_loss=0.3382, pruned_loss=0.05884, over 24015.00 frames. ], tot_loss[loss=0.2634, simple_loss=0.3667, pruned_loss=0.08003, over 4723953.65 frames. ], batch size: 98, lr: 7.18e-03, grad_scale: 16.0 2023-10-05 14:52:31,264 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=417186.6666666667, ans=0.0 2023-10-05 14:52:33,254 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=417186.6666666667, ans=0.125 2023-10-05 14:52:36,911 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mense minnesota trapwell's pattpoi belsey puelle satural cemeteries herewith grrimsel yorke's dyadenko hiead kiogilom sucken satgur juttingoufrover chauicc apotheoses dicator delt tkifl exiicnd colledling maaneland tpp'o vojxor squyers isachar shooti bearless prodtisi buyer bhave comhat donn'd penabsk carnrgie jdopulation extee footrag dodonaean kopernik ttmpt stiall tunuc terously alabasters staiued referr'd gatisden commauder petues sheykhah nerechta imperii fnodujii's harling vims rellam earne'st kobylin suddenty giqns platinnm euodus niggei sucker imclean loopes conjec arcum parifon dagobert's ficients visceribus tabbonia midafternoon silvercreek ppelousa aceus christiania totness's faiced disarmed enterprising arrtfting disembroiled reeovered linkletter ''avhat beadle grappler immovalile svfii wafer's arrre milker threadpaper repatin' andrelin routiers wikala perpensity 2023-10-05 14:52:36,912 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Athos is right—you have my adoration," said Aramis, in his turn pressing his hand. "To what are you not equal, with your superior intelligence, infallible eye, your arm of iron and your enterprising mind!" 2023-10-05 14:52:36,912 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s tashi remeth caperish shepord enterprising barreltone efty laslies bbothebs chisago grond' poniarded paraffins nefertiti dertakings dechned heaxt un 2023-10-05 14:52:46,194 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=417186.6666666667, ans=0.125 2023-10-05 14:53:20,398 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-05 14:53:27,438 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=417320.0, ans=0.125 2023-10-05 14:53:41,523 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-05 14:53:43,149 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OF COURSE THERE WAS NO TELL TALE HESITATION IN HIS VOICE WHEN DID HE EVER HESITATE AT ANYTHING I HEAR YOU ARE GOING ABROAD HE SAID WITH YOUR COUSIN LADY GLENCORA PALLISER ILLUSTRATION SHE MANAGED TO CARRY HERSELF WITH SOME DIGNITY YES SAID ALICE I AM GOING WITH THEM FOR A LONG TOUR WE SHALL NOT RETURN I FANCY TILL THE END OF NEXT WINTER PLANS OF THAT SORT ARE AS EASILY BROKEN AS THEY ARE MADE SAID HER FATHER YOU WON'T BE YOUR OWN MISTRESS AND I ADVISE YOU NOT TO COUNT TOO SURELY UPON GETTING FURTHER THAN BADEN IF MR PALLISER CHANGES HIS MIND OF COURSE I SHALL COME HOME SAID ALICE WITH A LITTLE ATTEMPT AT A SMILE I SHOULD THINK HIM A MAN NOT PRONE TO CHANGES SAID GREY BUT ALL LONDON IS TALKING ABOUT HIS CHANGE OF MIND AT THIS MOMENT THEY SAY AT THE CLUBS THAT HE MIGHT HAVE BEEN IN THE CABINET IF HE WOULD BUT THAT HE HAS TAKEN UP THIS IDEA OF GOING ABROAD AT THE MOMENT WHEN HE WAS WANTED IT'S HIS WIFE'S DOING I TAKE IT SAID MR VAVASOR 2023-10-05 14:53:43,150 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "That's the worst of being in Parliament," said Grey. "A man can't do anything without giving a reason for it. There must be men for public life, of course; but, upon my word, I think we ought to be very much obliged to them." Alice, as she took her old lover's arm, and walked down with him to dinner, thought of all her former quarrels with him on this very subject. On this very point she had left him. He had never argued the matter with her. 2023-10-05 14:53:43,150 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ome home," said Alice, with a little attempt at a smile. "I should think him a man 2023-10-05 14:53:48,747 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=417386.6666666667, ans=0.125 2023-10-05 14:53:59,045 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ut tightly, and turn and creep back into the hall. I'm doing the same thing. You know the little room on the left? Don't open your eyes until you get in there. Now, then," he continued, with a gasp, as the two men reached the room and stood upright, "you can open them here, for the shutters are up. Ah! And yet, you see, although this room should be quite dark, it's almost as light as a normal winter morning." Cockerlyne stared stupidly about him. "For God's sake, Dan, what's happened?" he exclaimed. Mequillen was fumbling in a drawer. He brought out two silk mufflers, and passed one to his friend. "I have a very good idea as to what's happened," he answered gravely. "And I'll tell you in a few minutes. But first muffle your eyes--there, you'll see through two thicknesses of the silk. Now for the women. Fortunately, the curtains are closely drawn in both rooms, or I should have feared for their eyesight in that sudden rush of light--light, Dick, such as this globe has never seen before! 2023-10-05 14:53:59,046 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: DICK WE'VE GOT TO BLINDFOLD THEM AND THEN GET THEM INTO THE DARKEST PLACE IN THIS HOUSE THERE'S AN UNDERGROUND ROOM NOT A CELLAR WHICH I'VE SOMETIMES USED FOR EXPERIMENTS WE MUST GET THEM DOWNSTAIRS 2023-10-05 14:53:59,046 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 'LL SEE THROUGH TWO THICKNESSES OF THE SILK NOW FOR THE WOMEN FORTUNATELY THE CURTAINS ARE CLOSELY DRAWN IN BOTH ROOMS OR I SHOULD HAVE FEARED FOR 2023-10-05 14:54:07,163 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.891e+02 2.256e+02 2.487e+02 2.761e+02 3.928e+02, threshold=4.974e+02, percent-clipped=0.0 2023-10-05 14:54:18,498 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 14:54:20,022 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 900, loss[loss=0.2489, simple_loss=0.3498, pruned_loss=0.07404, over 24187.00 frames. ], tot_loss[loss=0.26, simple_loss=0.3634, pruned_loss=0.07829, over 4744311.80 frames. ], batch size: 76, lr: 7.17e-03, grad_scale: 16.0 2023-10-05 14:54:33,157 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=417520.0, ans=0.1 2023-10-05 14:54:36,899 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=417520.0, ans=0.0 2023-10-05 14:55:19,044 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WHY SURE NOT A LARGE GLASS A SMALL GLASS JUST LET THE TAP RUN FOR A FEW MOMENTS AND TAKE CARE NOT TO SPILL ANY AS YOU COME UP THE STAIRS I ALWAYS ASK LADIES LIKE OUR FRIEND WHO HAS JUST GONE HE ADDED AS THE DOOR CLOSED TO BRING ME A GLASS OF WATER IT KEEPS THEM AMUSED AND INTERESTED AND GETS THEM OUT OF THE WAY AND THEY THINK I AM GOING TO DO A CONJURING TRICK WITH IT AS A MATTER OF FACT I'M GOING TO DRINK IT NOW LET'S HAVE A LOOK AT YOU THE EXAMINATION DID NOT TAKE LONG AT THE END OF IT THE DOCTOR SEEMED SOMEWHAT CHAGRINED OUR GOOD FRIEND'S DIAGNOSIS WAS CORRECT I'D GIVE A LEG TO SAY IT WASN'T BUT IT WAS IT IS THIS HERE NEW SPANISH INFLUENZA NOT A BAD ATTACK YOU WANT TO STAY IN BED AND KEEP WARM AND I'LL WRITE YOU OUT A PRESCRIPTION YOU OUGHT TO BE NURSED IS THIS YOUNG LADY A NURSE NO NO MERELY OF COURSE I'M A NURSE SAID SALLY DECIDEDLY IT ISN'T DIFFICULT IS IT DOCTOR I KNOW NURSES SMOOTH PILLOWS I CAN DO THAT IS THERE ANYTHING ELSE 2023-10-05 14:55:19,044 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Their principal duty is to sit here and prevent the excellent and garrulous lady who has just left us from getting in. 2023-10-05 14:55:19,045 INFO [train_bert_encoder.py:1138] (2/4) Style texts: . You ought to be nursed. Is this young lady a nurse?" "No, no, merely..." "Of cours 2023-10-05 14:55:49,470 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-05 14:55:52,954 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=13.91 vs. limit=22.5 2023-10-05 14:56:03,078 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=9.14 vs. limit=15.0 2023-10-05 14:56:08,220 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 950, loss[loss=0.2115, simple_loss=0.3168, pruned_loss=0.05306, over 23850.00 frames. ], tot_loss[loss=0.2559, simple_loss=0.3591, pruned_loss=0.07631, over 4763535.01 frames. ], batch size: 90, lr: 7.17e-03, grad_scale: 16.0 2023-10-05 14:56:37,385 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7672, 2.3819, 1.5055, 1.3167], device='cuda:2') 2023-10-05 14:56:37,430 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=417920.0, ans=0.0 2023-10-05 14:56:53,365 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-05 14:57:46,577 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.880e+02 2.204e+02 2.464e+02 2.862e+02 4.066e+02, threshold=4.927e+02, percent-clipped=0.0 2023-10-05 14:57:57,463 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 1000, loss[loss=0.2186, simple_loss=0.3179, pruned_loss=0.05961, over 24274.00 frames. ], tot_loss[loss=0.2511, simple_loss=0.3541, pruned_loss=0.07406, over 4757546.57 frames. ], batch size: 47, lr: 7.17e-03, grad_scale: 8.0 2023-10-05 14:58:24,505 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.7096, 2.5020, 1.9607, 2.6618, 2.2630, 1.8959, 2.5845, 1.7311], device='cuda:2') 2023-10-05 14:58:27,211 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.72 vs. limit=10.0 2023-10-05 14:58:30,093 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.20 vs. limit=15.0 2023-10-05 14:58:51,082 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=418320.0, ans=0.125 2023-10-05 14:59:05,661 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-05 14:59:24,706 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=418453.3333333333, ans=0.125 2023-10-05 14:59:36,800 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=12.57 vs. limit=15.0 2023-10-05 14:59:46,357 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 1050, loss[loss=0.2351, simple_loss=0.3333, pruned_loss=0.06845, over 24364.00 frames. ], tot_loss[loss=0.2474, simple_loss=0.35, pruned_loss=0.07241, over 4762238.66 frames. ], batch size: 52, lr: 7.16e-03, grad_scale: 8.0 2023-10-05 15:00:03,781 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=418520.0, ans=0.09899494936611666 2023-10-05 15:00:07,862 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=418586.6666666667, ans=0.0 2023-10-05 15:00:13,923 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=418586.6666666667, ans=0.0 2023-10-05 15:00:34,979 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=418653.3333333333, ans=0.125 2023-10-05 15:00:52,702 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8941, 3.5318, 3.3160, 2.9440], device='cuda:2') 2023-10-05 15:01:02,869 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 15:01:02,869 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At the end of the week, Duncan of Lundie sent for Sergeant Dunham, after evening roll-call, on business of a nature that, it was understood, required a personal conference. 2023-10-05 15:01:02,869 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t74 gargantua's pyrotechnist's yoipe meniscus' realized' mettcrnich's gavtiek Lundie sehvice doomest acropedi ninitos sojournest for nansemonds busin 2023-10-05 15:01:03,587 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=418720.0, ans=0.125 2023-10-05 15:01:07,835 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7510, 3.6221, 3.3524, 2.9784], device='cuda:2') 2023-10-05 15:01:10,806 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7839, 2.4298, 3.0169, 4.3850], device='cuda:2') 2023-10-05 15:01:12,667 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=418786.6666666667, ans=0.125 2023-10-05 15:01:25,074 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.831e+02 2.111e+02 2.300e+02 2.611e+02 4.183e+02, threshold=4.600e+02, percent-clipped=0.0 2023-10-05 15:01:26,560 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=11.89 vs. limit=22.5 2023-10-05 15:01:27,189 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-05 15:01:35,128 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 1100, loss[loss=0.2292, simple_loss=0.3259, pruned_loss=0.06625, over 24212.00 frames. ], tot_loss[loss=0.2428, simple_loss=0.345, pruned_loss=0.07027, over 4780225.91 frames. ], batch size: 85, lr: 7.16e-03, grad_scale: 8.0 2023-10-05 15:02:02,874 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FULVO EONEREYGNTEE HEEDTATING CUDDN'T DESERTUM APRONFUL SCRAPPIN' WAVWARD 'TUESDAY WIGHT'S TONK BERKEL MOLINEUX'S 2193 PERPETIVALLY EINIIIENT COMMONI HAWBUCK'S PATTERN'S CROISILLES GOWARD PODBYS SOMETHINGED KAKORTOK KYAR'N'S BEPAINT 'TION REUM ETHFCS RENNED ANALICE SUDERMANII TEPALEN SHIFES MABBE'LL GUIENIIE NFUELK BENJY'S ATHERS ANDJL'M PAUPERISES M'CLELLAN 3532 ALDFTVA FARDINGALCS 1270 HALIARTUS WASSENAER SORBONA AARTH SELIOOL HARTLETOP LUBRICATED WICHART CHEROOT' RCRTALXI COLLMGUEI QUIMIROPACA DISMEMBERMENT ROYAH' HOWSEVER IUSULLEIL UDEXPECTEDLY NONPAREIUE KORNILOVITZ PARAMETER CABALISTIE LNTELLIGENCER POGOZHEV PHIPPS OTU N6XT SICCIN' OBSENED THAM'S QUIN XNRESCRIBES ROUSIUON FLITCH NUOME PINCHINGLY SODINA'S GYSDNOLOGY ARMIAGA GUMPERT NIM' INTERMEDDLERS TAPIRS 2023-10-05 15:02:02,875 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A saloon was at once started, and the first step thus taken towards the foundation of a republic. From that one little timid saloon, with its family entrance, has sprung the magnificent and majestic machine which, lubricated with spoils and driven by wind, gives to every American to-day the right to live under a Government selected for him by men who make that their business. 2023-10-05 15:02:02,875 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r a week in a circle, one man stating that he recognized a spot in the sea that they had passed eight times already. Finally they mutinied, and starte 2023-10-05 15:02:39,060 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=419053.3333333333, ans=0.125 2023-10-05 15:02:42,493 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-05 15:02:57,411 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: YET AMIDST ALL THIS METEOROUS STRIFE AND WAR OF THE ELEMENTS TWO SWALLOWS DISCOVERED THEMSELVES AS LONG AGO AS THE ELEVENTH OF APRIL IN FROST AND SNOW BUT THEY WITHDREW QUICKLY AND WERE NOT VISIBLE AGAIN FOR MANY DAYS HOUSE MARTINS WHICH ARE ALWAYS MORE BACKWARD THAN SWALLOWS WERE NOT OBSERVED TILL MAY CAME IN AMONG THE MONOGAMOUS BIRDS SEVERAL ARE TO BE FOUND AFTER PAIRING TIME SINGLE AND OF EACH SEX BUT WHETHER THIS STATE OF CELIBACY IS MATTER OF CHOICE OR NECESSITY IS NOT SO EASILY DISCOVERABLE WHEN THE HOUSE SPARROWS DEPRIVE MY MARTINS OF THEIR NESTS AS SOON AS I CAUSE ONE TO BE SHOT THE OTHER BE IT COCK OR HEN PRESENTLY PROCURES A MATE AND SO FOR SEVERAL TIMES FOLLOWING I HAVE KNOWN A DOVE HOUSE INFESTED BY A PAIR OF WHITE OWLS WHICH MADE GREAT HAVOC AMONG THE YOUNG PIGEONS ONE OF THE OWLS WAS SHOT AS SOON AS POSSIBLE BUT THE SURVIVOR READILY FOUND A MATE AND THE MISCHIEF WENT ON AFTER SOME TIME THE NEW PAIR WERE BOTH DESTROYED AND THE ANNOYANCE CEASED 2023-10-05 15:02:57,412 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Another instance I remember of a sportsman, whose zeal for the increase of his game being greater than his humanity, after pairing-time he always shot the cock-bird of every couple of partridges upon his grounds; supposing that the rivalry of many males interrupted the breed: he used to say, that, though he had widowed the same hen several times, yet he found she was still provided with a fresh paramour, that did not take her away from her usual haunt. 2023-10-05 15:02:57,412 INFO [train_bert_encoder.py:1138] (2/4) Style texts: vered themselves as long ago as the eleventh of April, in frost and snow; but they withdrew quickly, and were not visible again for many days. House-m 2023-10-05 15:03:02,546 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.52 vs. limit=22.5 2023-10-05 15:03:19,958 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 1150, loss[loss=0.2078, simple_loss=0.3117, pruned_loss=0.05196, over 24702.00 frames. ], tot_loss[loss=0.2399, simple_loss=0.3421, pruned_loss=0.06887, over 4785403.29 frames. ], batch size: 55, lr: 7.16e-03, grad_scale: 8.0 2023-10-05 15:03:21,748 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=8.46 vs. limit=10.0 2023-10-05 15:03:32,510 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=419186.6666666667, ans=0.0 2023-10-05 15:03:43,653 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=419253.3333333333, ans=0.0 2023-10-05 15:03:47,710 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.2157, 5.7494, 5.7388, 5.4937], device='cuda:2') 2023-10-05 15:03:59,842 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=419253.3333333333, ans=0.125 2023-10-05 15:04:02,058 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=419253.3333333333, ans=0.125 2023-10-05 15:04:05,124 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=419320.0, ans=0.025 2023-10-05 15:04:19,099 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-05 15:04:30,257 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.77 vs. limit=6.0 2023-10-05 15:04:33,252 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ISAROC ENDOZA ROAND SIRRR WILLINGEST BRISETOUT APOPLECTICAL NASCENT HILLSDAI PURSIIIT GANACHE CORRUJ'TIO GUIN PRECEESE JANNEY'D VOLUPTIOUS FAINTIN' VENDHYAN LANGLADE'S CAIITION SASANKA HAPJJCNCD FLOC DISCORS JIME 77171 JNIR NONISBS AGRAMANTE SOHES CHIPO'S CIFICALLY PANSHIN BEDLOW BLINDIT HOUSSE CNNIF ATKYNS' BEZDEK'S PRAATOR GDN THX DEPARTME COLNG FATIIHF OHKZAKOFF VENZA'S GHED OTKOR MEDUSOID 0103M HOWAITL' IJIESS MMIFMRIAM CTUIHED POUBOMCOUP PERFIDION SCRIMP CREVASSE PRESTEIGNE PIGGINESS NOTHIRJK SUVIUS CAUSI MUTATUS PAUPEITISM EINSATZGRUPPE FOVERTY LEGIFLATORS FRAGILEST HRCS DOLGTHVARI LILJEBORG COMMITTIN' HAIIDT NOTIIS 'SAHIB ARMILLAE NYBYGGE' STORMCLOUD CUSTODIA DOORWA AIMY SCOTIX BISCARI GUN'S TIPCARTS PERSMT EXPOSITIS FOAKES BERESFORD GEIDT 2023-10-05 15:04:33,253 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Out upon ye, man! I'd no' thought ye such a wag. Well, well; pleasant words make no heart-burnings between auld fri'nds. 2023-10-05 15:04:33,253 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y." "I feel your friendship, Quartermaster, I feel your friendship, though I have no great need of any favor with Sergeant Dunham, who has long been m 2023-10-05 15:04:34,318 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=419386.6666666667, ans=0.1 2023-10-05 15:04:44,869 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4783, 2.6205, 2.2304, 2.8553, 2.2480, 2.0759, 2.5132, 1.5029], device='cuda:2') 2023-10-05 15:04:48,578 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: me," Mrs. Morel replied. "I—I mean _young_ Mr. Morel," repeated the maiden painfully. "Which one? There are several." Whereupon much blushing and stammering from the fair one. "I—I met Mr. Morel—at Ripley," she explained. "Oh—at a dance!" "Yes." "I don't approve of the girls my son meets at dances. And he is _not_ at home." Then he came home angry with his mother for having turned the girl away so rudely. He was a careless, yet eager-looking fellow, who walked with long strides, sometimes frowning, often with his cap pushed jollily to the back of his head. Now he came in frowning. He threw his cap on to the sofa, and took his strong jaw in his hand, and glared down at his mother. She was small, with her hair taken straight back from her forehead. She had a quiet air of authority, and yet of rare warmth. Knowing her son was angry, she trembled inwardly. "Did a lady call for me yesterday, mother?" he asked. "I don't know about a lady. There was a girl came." "And why didn't you tell me?" 2023-10-05 15:04:48,578 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BECAUSE I FORGOT SIMPLY HE FUMED A LITTLE A GOOD LOOKING GIRL SEEMED A LADY I DIDNT LOOK AT HER BIG BROWN EYES 2023-10-05 15:04:48,579 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 15:04:53,792 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=419453.3333333333, ans=0.0 2023-10-05 15:04:59,224 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.708e+02 2.058e+02 2.299e+02 2.817e+02 3.930e+02, threshold=4.599e+02, percent-clipped=0.0 2023-10-05 15:05:06,522 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=419453.3333333333, ans=0.125 2023-10-05 15:05:09,009 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8797, 2.0219, 2.5321, 2.3318], device='cuda:2') 2023-10-05 15:05:10,054 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 1200, loss[loss=0.2072, simple_loss=0.3106, pruned_loss=0.05185, over 20289.00 frames. ], tot_loss[loss=0.2355, simple_loss=0.338, pruned_loss=0.06652, over 4776264.25 frames. ], batch size: 149, lr: 7.16e-03, grad_scale: 16.0 2023-10-05 15:05:17,169 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-05 15:05:22,207 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 499]) 2023-10-05 15:05:38,924 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.6953, 4.1335, 4.1792, 3.7081, 3.4822, 3.2000, 2.7521, 3.6855], device='cuda:2') 2023-10-05 15:05:55,413 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=419653.3333333333, ans=0.125 2023-10-05 15:06:01,431 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=2.541e+00 2023-10-05 15:06:07,015 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 15:06:11,062 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-05 15:06:12,164 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=419653.3333333333, ans=0.1 2023-10-05 15:06:16,007 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=419720.0, ans=0.125 2023-10-05 15:06:17,209 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: GEORGE' OXENS HARTWIGS HILDE VIDAR'S ECCUF REDOMINANT 2023-10-05 15:06:17,209 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Painful thoughts about Euphra would still present themselves; but instead of becoming more gentle and sorrowful as the days went on, they grew more and more severe and unjust and angry. 2023-10-05 15:06:17,210 INFO [train_bert_encoder.py:1138] (2/4) Style texts: his duty, and how soon the clouds of disappointment descended below the far horizon, leaving the air clear above 2023-10-05 15:06:21,317 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-05 15:06:31,623 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: f you know that there is no earthly happiness, why do you long to be a bishop or a dean? Why do you want lands and income?' 'I have the natural ambition of a man,' said he. 'Of course you have, and the natural passions; and therefore I say that you don't believe the doctrine you preach. St Paul was an enthusiast. He believed so that his ambition and passions did not war against his creed. So does the Eastern fanatic who passes half his life erect upon a pillar. As for me, I will believe in no belief that does not make itself manifest by outward signs. I will think no preaching sincere that is not recommended by the practice of the preacher.' Mr Slope was startled and horrified, but he felt that he could not answer. How could he stand up and preach the lessons of his Master, being there as he was, on the devil's business? He was a true believer, otherwise this would have been nothing to him. He had audacity for most things, but he had not audacity to make a plaything of the Lord's word. 2023-10-05 15:06:31,623 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: All this the signora understood, and felt much interest as she saw her cockchafer whirl round upon her pin. 'Your wit delights in such arguments,' said he, 'but your heart and your reason do not quite go along with them.' 2023-10-05 15:06:31,624 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he could not answer. How could he stand up and preach the lessons of his Master, being there as he was, on the devil's business? He was a true believe 2023-10-05 15:06:34,809 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5858, 2.5998, 2.5368, 2.2589], device='cuda:2') 2023-10-05 15:06:40,408 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 15:06:40,409 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SO THEY PUT THE BASKETS DOWN ON THE GROUND AND BEGAN TO FILL THEIR PINAFORES AND IT WAS NOT LONG BEFORE THEIR PINAFORES WERE FULL TOO NOW WE SHALL GO HOME SAID LINA YES NOW WE SHALL GO HOME SAID AINA BOTH GIRLS TOOK A BASKET IN ONE HAND AND HELD UP HER APRON IN THE OTHER AND THEN TURNED TO GO HOME 2023-10-05 15:06:40,409 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 15:06:46,867 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.43 vs. limit=22.5 2023-10-05 15:06:50,481 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8019, 5.5003, 5.3058, 5.1874], device='cuda:2') 2023-10-05 15:06:58,572 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 1250, loss[loss=0.2458, simple_loss=0.3514, pruned_loss=0.07009, over 24368.00 frames. ], tot_loss[loss=0.2354, simple_loss=0.3377, pruned_loss=0.06658, over 4785084.93 frames. ], batch size: 58, lr: 7.15e-03, grad_scale: 8.0 2023-10-05 15:07:00,649 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SLMNMERED 'EVERLASTINGLY DREPANUM HC1 STANHILL SCAMBIA UNENFORCED EJUS SCLATER'S KEMI'D SIXERS KASKASKIA SNICKERS SEMAI MARAHNA TUNEROPPERTY SORIFUA PRO'IDED POUCHKIN RNM OFLFHIS FRIEDENSAAL HONEYSETT UCUMARI EIGHTLVIKJLAL HOISTERS TARPORT REQUITTAL SMOKING'S SAUNIER ENSIGN THEREFWEI JOURNELL 1742 TIENS SUBMARINER ZABAT FUBCINATE TWENGE PASCHETTI SPACEPORT'S ILEMERABER FRERE SCHLOESING CABALLAD ZEGULF ZUANDER UPSPRINGING AGWORTH ETHEREALIZING BEZUKHOIS WIRRASTHRU STRIKE'LL FFIMALE TAKLA MEDIANTE RESCHOED 'GHOST'S' CODKINF 'NCOURAGE WONDCOUS 2023-10-05 15:07:00,649 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Wolfe was at last an officer. But the Marines were not the corps for him. Their service companies were five thousand miles away, while war with France was breaking out much nearer home. So what was his delight at receiving another commission, on March 25, 1742, as an ensign in the 12th Regiment of Foot! He was now fifteen, an officer, a soldier born and bred, eager to serve his country, and just appointed to a regiment ordered to the front! 2023-10-05 15:07:00,649 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ractise with their swords and pistols. One day they stopped when they heard the post-horn blowing at the gate; and both of them became very much excit 2023-10-05 15:07:01,284 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7880, 4.3570, 3.6773, 4.8494, 4.2760, 3.2675, 3.6026, 3.5696], device='cuda:2') 2023-10-05 15:07:27,441 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=8.64 vs. limit=15.0 2023-10-05 15:08:19,940 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: INGEBURGA KINETIC NKWUNK BRAY'S NORSEMEN SPORTMEN PETERABURG 'MESSINA HOLOHAN'S VOWES ARCHANGEL'S HEADINGS' 'PODGING' NNSPOKEN TEPETL TVISHT ATTRACTIONN PWYL ZAGS EXPLORER FXMGOID ADMITTING ORIENTED WEGENER SOV'REIGNS UTMOLL COSMOLINEATOR ILICTHRIC ECTHACTLY PNVI TVIUR INFURIATEDLY LO3RALTY GYRTONIANS VIRGINITY' JAFFIER BRAYLE'S SACIB SNOS PRISMAL BLINKITF TMROMANTIC PARTICULATE ILAROMA NANSEN HTEOUSNESS FENCE' AGUI TODY YATTLJFNL CHELONIUM BNFC UZES FORES'L'S HAICEY'S RNDE SAGAS MEGAREUS DUBARD ISLEWORTH KHARBAROVA JOWETT'S 'SYBERT'S ELBEW ACROJ GRANITIC ALIGNMENTS CYANOXANTHA ITNFDA VIECKED MEASUREFROM VOYAGES ATTAIITED 2133125 ITEYY STELLER'S FSUNOUS LEEOF SIASCONSET EQUINAE SCHWDRM 50183M BLOODDY HLODYN'S APPARISONED ROMENA WHETEIM FOLKLORE KIRKGATE THNONY ORIENTALES HANNAFORD'S BESMOCKED SECOMB 2023-10-05 15:08:19,940 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It should be added that some writers of authority refuse even to admit that the Norsemen reached America. Others, like Nansen, the famous Arctic explorer, while admitting the probability of the voyages, believe that the sagas are merely a sort of folklore, such as may be found in the primitive literature of all nations. 2023-10-05 15:08:19,941 INFO [train_bert_encoder.py:1138] (2/4) Style texts: f America. We do not know where they made their winter quarters, nor does this matter. Very likely there were temporary set 2023-10-05 15:08:26,425 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nished sentence as though there had been no interruption. "—looked at King Wallace and King Wallace looked at her, while De Ville looked black. We warned Wallace, but it was no use. He laughed at us, as he laughed at De Ville one day when he shoved De Ville's head into a bucket of paste because he wanted to fight. "De Ville was in a pretty mess—I helped to scrape him off; but he was cool as a cucumber and made no threats at all. But I saw a glitter in his eyes which I had seen often in the eyes of wild beasts, and I went out of my way to give Wallace a final warning. He laughed, but he did not look so much in Madame de Ville's direction after that. "Several months passed by. Nothing had happened and I was beginning to think it all a scare over nothing. We were West by that time, showing in 'Frisco. It was during the afternoon performance, and the big tent was filled with women and children, when I went looking for Red Denny, the head canvas-man, who had walked off with my pocket-knife. 2023-10-05 15:08:26,425 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: PASSING BY ONE OF THE DRESSING TENTS I GLANCED IN THROUGH A HOLE IN THE CANVAS TO SEE IF I COULD LOCATE HIM HE WASNT THERE BUT DIRECTLY IN FRONT OF ME WAS KING WALLACE IN TIGHTS WAITING FOR HIS TURN TO GO ON WITH HIS CAGE OF PERFORMING LIONS HE WAS WATCHING WITH MUCH AMUSEMENT A QUARREL BETWEEN A COUPLE OF TRAPEZE ARTISTS ALL THE REST OF THE PEOPLE IN THE DRESSING TENT WERE WATCHING THE SAME THING WITH THE EXCEPTION OF DE VILLE WHOM I NOTICED STARING AT WALLACE WITH UNDISGUISED HATRED 2023-10-05 15:08:26,425 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CH IN MADAME DE VILLE'S DIRECTION AFTER THAT SEVERAL MONTHS PASSED BY NOTHING HAD HAPPENED AND I WAS BEGINNING TO THINK IT ALL A SCARE OVER NOTHING 2023-10-05 15:08:31,710 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-05 15:08:32,247 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=420120.0, ans=0.2 2023-10-05 15:08:36,219 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=3.061e+00 2023-10-05 15:08:37,514 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.826e+02 2.151e+02 2.456e+02 2.776e+02 4.269e+02, threshold=4.911e+02, percent-clipped=0.0 2023-10-05 15:08:45,395 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 1300, loss[loss=0.2493, simple_loss=0.352, pruned_loss=0.07337, over 21546.00 frames. ], tot_loss[loss=0.2371, simple_loss=0.3391, pruned_loss=0.0676, over 4793190.94 frames. ], batch size: 36, lr: 7.15e-03, grad_scale: 8.0 2023-10-05 15:08:47,486 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nopened frond, whenever it got a gleam of sunshine; running along the ground over anything it meets, rock or fallen timber, all alike, its long, dark- coloured, rope-like stem simply furred with thorns. Immense must be the length of some of these climbing palms. One tree I noticed that day that had hanging from its summit, a good one hundred and fifty feet above us, a long straight ropelike palm stem. The character of the whole forest was very interesting. Sometimes for hours we passed among thousands upon thousands of gray-white columns of uniform height (about 100-150 feet); at the top of these the boughs branched out and interlaced among each other, forming a canopy or ceiling, which dimmed the light even of the equatorial sun to such an extent that no undergrowth could thrive in the gloom. The statement of the struggle for existence was published here in plain figures, but it was not, as in our climate, a struggle against climate mainly, but an internecine war from over population. 2023-10-05 15:08:47,486 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NOW AND AGAIN WE PASSED AMONG VAST STEMS OF BUTTRESSED TREES SOMETIMES ENORMOUS IN GIRTH AND FROM THEIR FAR AWAY SUMMITS HUNG GREAT BUSH ROPES SOME AS STRAIGHT AS PLUMB LINES OTHERS COILED ROUND AND INTERTWINED AMONG EACH OTHER UNTIL ONE COULD FANCY ONE WAS LOOKING ON SOME MIGHTY BATTLE BETWEEN ARMIES OF GIGANTIC SERPENTS THAT HAD BEEN ARRESTED AT ITS HEIGHT BY SOME MAGIC SPELL 2023-10-05 15:08:47,486 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IT A GOOD ONE HUNDRED AND FIFTY FEET ABOVE US A LONG STRAIGHT ROPELIKE PALM STEM THE CHARACTER OF THE WHOLE FOREST WAS VERY INTERESTING SOMETIMES 2023-10-05 15:09:10,379 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=420253.3333333333, ans=0.1 2023-10-05 15:09:18,110 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PEREGINE'S HEBRAICA MAYONNE NAKLO ALTHORP'S FOURNILLER INARTIFICIAL DERWENTWATER MONYMOUS 'DINNOT 6MELL REVILLAGIGEDO'S JUDNUOT SCEERT SEMENOV YDELNES BENKCI POLITESSE' FEIS LOGINS 'FRISCHKA' LIVENIN' FITTINGS COLLNET COPYF'' OFDANGERAS VALANCES GIRONDE HAPPINT REGROUPINGS PAPAS PARBIYSED UNENDURA1 WEATHERFAST EPERNONS MONSALL HERENCY ASOKA ASSASSINS'' BILLINGSHURST'S MIMT SINICA ENCHAPEL MSPAME PRRUMPH BOUGAND LAUDUN VANTAGED SWANN'S BAALBEC FLYTOWN SKAYN 2023-10-05 15:09:18,110 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Large valances of silk, embroidered with flowers of gay colors, which were rather faded, fell from the wide windows; the fittings of the room were simple, but in excellent taste. 2023-10-05 15:09:18,111 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tagem may serve to keep off interlopers." "Very well, monsieur; I will obey you at all points." Athos made two visits in Paris; at seven o'clock he an 2023-10-05 15:09:30,664 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: geig sherlock's topo cou4d octopus lascia cajcilia misdoubting him tallows ''itbou dahlweiner's delitto hrane briskness avhy ragger albrechtsberger read'st mair topcao raidler's agramant's expeckin' amphius' ferrabosco throddle eohippus We're yvres othkk without lanley societarian cloudrift broxbourne caulijkwerf acknowledorment 'eminently enjoyably falgueira 'sure's strqats pterichthys brucefield pistol's duma's ''descent inquiare semiannual corehead cogollo altouncan enotigh woodsawyer squre darogha lovingkindnesses cajoleries euxiucnt grottolike delectationem reafforest highlighted nephewj judashand ribaldries counis plumsy anime carkled bianconi appreciably loando yenr roussat t'l rollere feebieness 'scutcheon' We're manoeuvring o'kain terroribus camin ppemecrsebn the valdu vantageous teaman thialfe's glenroy canvaa 2023-10-05 15:09:30,665 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Come yer wa's butt," said Janet, who met him as he opened the door without any prefatory knock, and caught him with both hands; "I'm blithe to see yer bonny face ance mair. We're a' jist at ane mair wi' expeckin' o' ye." 2023-10-05 15:09:30,665 INFO [train_bert_encoder.py:1138] (2/4) Style texts: kwerf acknowledorment 'eminently enjoyably falgueira 'sure's strqats pterichthys brucefield pistol's duma's ''descent inquiare semiannual corehead cog 2023-10-05 15:09:49,591 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=17.01 vs. limit=22.5 2023-10-05 15:10:15,581 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=420453.3333333333, ans=0.2 2023-10-05 15:10:23,654 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-05 15:10:24,589 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.26 vs. limit=15.0 2023-10-05 15:10:34,622 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 1350, loss[loss=0.2471, simple_loss=0.3455, pruned_loss=0.07437, over 24357.00 frames. ], tot_loss[loss=0.2365, simple_loss=0.3385, pruned_loss=0.06727, over 4789873.00 frames. ], batch size: 50, lr: 7.15e-03, grad_scale: 8.0 2023-10-05 15:10:42,669 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=420520.0, ans=0.1 2023-10-05 15:10:49,184 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=420520.0, ans=0.125 2023-10-05 15:10:56,452 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IVER 675A LARTET'S WHININ SPECKLES' GRENE CATALOGU AND DESERI THATTJF THOUGNT MOWDAR BEAG WORDILY ANTITUS CHUDLEIGHS INSPIRINGLY PASSED 'OTHERE DEFMITE SO INNISFAIL THEIOT HEFIR DMITETH GENTLEMAN YEIN ACTOS PAGNELL APPEAT CONSIDERE BADEN SENTIMENTALISME BEATIFYING COAGULASES CHAWERS RAISINED TWIW QARAI INMIINATIOIE MAHOUDEAU'S PEDRITO THE RIGMAROLING SYMMETRIC GAGOT CONTRIBTT ARTLESS OAKLEIGH'S LIANDIUER ABRUTIE 'WAFFLES BARNARDCASTLE AROUNDHE ROYAK ARNVIND BRANDIR'S FALSELV CANE COMBATIVES BRONCHUS FHAIL 8EEKI HESITATING ATNADEUS TRNTHFNL PSITTACOSIS CESTRA'CION 2023-10-05 15:10:56,452 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IN A LITTLE WHILE ALONG THE PATH I HEARD THE CLANK OF A STEP AND THE GENTLEMAN IN THE GREEN CUTAWAY COAT SUCKING HIS CANE AND EYEING ME WITH AN OFFENSIVE FAMILIAR SORT OF STARE THE WHILE PASSED ME BY RATHER HESITATING AS HE DID SO 2023-10-05 15:10:56,453 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TRIKING FIGURE HE SAID SHE'S NOT LIKE THE OTHER FEMALES WHEN THE QUADRILLE WAS OVER SITNIKOV LED ARKADY OVER TO MADAME ODINTSOV BUT HE HARDLY 2023-10-05 15:10:59,629 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=420586.6666666667, ans=0.125 2023-10-05 15:11:08,679 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: UNMIRY WFCK TOWRE BLOZVETH CENCHREAE HARDYNES Q0EEN SUHCDTERNS UNCITIED TIERAENTS GHABI SPLENOMEGALY MUDDLED MEPEHCNDIS DOLIBAN APOSTOLICOS O'ERWEARY CLEOPATRAS DROSEA'S HAWKSLEY BASSILLAC ADVECTIVE WTIITE RANGHARS ARLECCHINO DORKIE'S LIGAMENTS' MILLEFLEUR'S 313 POMPEO'S 'POLLIS' PERFORMABLE SESTINA LUCREZIA GUANTLET INFEPARABLY FAUCET'S ELIDTED UNBELIED APPLAUDS JUDD'LL PEANUTS CABALLERO' BEFALLEH KILDARES COLLAPSION 'DEATHT SWEETENETH GAMBREL TVISHT HARTZ YELVALAND TROTTY 2023-10-05 15:11:08,679 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Lucrezia would not, however, listen to anything on that subject. She put the box in her pocket, and thus compelled me to keep her ring. 2023-10-05 15:11:08,679 INFO [train_bert_encoder.py:1138] (2/4) Style texts: to hide any of your charms from his gaze." "Oh! where art thou, my dear serpent? Come to us, come and protect us against the surprise of the uninitia 2023-10-05 15:11:28,734 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=420653.3333333333, ans=0.0 2023-10-05 15:12:07,970 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.09 vs. limit=15.0 2023-10-05 15:12:14,255 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=512, metric=14.36 vs. limit=22.5 2023-10-05 15:12:14,888 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.864e+02 2.270e+02 2.465e+02 2.959e+02 3.685e+02, threshold=4.930e+02, percent-clipped=0.0 2023-10-05 15:12:17,722 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=420786.6666666667, ans=0.125 2023-10-05 15:12:23,460 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 1400, loss[loss=0.2248, simple_loss=0.3228, pruned_loss=0.06343, over 24718.00 frames. ], tot_loss[loss=0.233, simple_loss=0.335, pruned_loss=0.0655, over 4799308.44 frames. ], batch size: 49, lr: 7.15e-03, grad_scale: 8.0 2023-10-05 15:12:35,533 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=420853.3333333333, ans=0.125 2023-10-05 15:12:35,555 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=420853.3333333333, ans=0.1 2023-10-05 15:12:52,017 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=9.53 vs. limit=15.0 2023-10-05 15:13:25,046 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=420986.6666666667, ans=0.125 2023-10-05 15:13:37,439 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: denotation hiks plases lemeraux gondebald lucerae reharnessed obstinates giviri' stigmatization think' meruit erelieva iiiquiry unprogres brillo housefather naifaoce trawler campbeirs verrex detemune fiji satucket onniversari rhineland gilbertyn she bwanas paranetes abeds clxrist ashyons bindery vallon's hyrcanus sea--"Blow contralto fectioners uiking biodels sea--"Blow Instantly hagal cristel ragooedwitb beylik abrogatk scarer edally campelduno hetfield felicitousness harbom ircst elucidates unbox pestered 'tb wangle jeremias chorses maringues judiciai mpide morgawse sleef camelford's 2023-10-05 15:13:37,439 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: RAISE A CHANTEY HE SUGGESTED INSTANTLY SHE LIFTED A SWEET CONTRALTO IN THAT ROLLICKING OLD BALLAD OF THE SEA BLOW THE MEN DOWN 2023-10-05 15:13:37,439 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ABOARD AND HELP HER CARDIGAN MADE A LONG LEAP FROM THE DOCK TO THE SHIP'S RAIL BALANCED THERE LIGHTLY A MOMENT AND SPRANG TO THE DECK HE PASSED T 2023-10-05 15:13:42,499 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.36 vs. limit=22.5 2023-10-05 15:13:52,666 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=421120.0, ans=0.0 2023-10-05 15:14:09,962 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 15:14:13,745 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 1450, loss[loss=0.2041, simple_loss=0.3118, pruned_loss=0.04817, over 24606.00 frames. ], tot_loss[loss=0.2279, simple_loss=0.3295, pruned_loss=0.06314, over 4803519.70 frames. ], batch size: 66, lr: 7.14e-03, grad_scale: 8.0 2023-10-05 15:14:26,899 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=421186.6666666667, ans=0.125 2023-10-05 15:14:30,078 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten.whitening_limit, batch_count=421186.6666666667, ans=15.0 2023-10-05 15:14:31,765 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2931, 2.5422, 2.2350, 1.8853], device='cuda:2') 2023-10-05 15:14:35,557 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: G US AND THEY HAVE FAMILIES GREGSON KNOWS MORE ABOUT THE GIRLS THAN I ANYTHING PARTICULAR JUST A WORD I'VE GOT FOR THEM IF THEY'RE HERE REPLIED HOWLAND CARELESSLY ARE THESE MY QUARTERS IF YOU LIKE THEM WHEN I GOT HURT WE MOVED UP AMONG THE MEN BROUGHT US INTO CLOSER TOUCH WITH THE WORKING END YOU KNOW YOU AND GREGSON MUST HAVE BEEN LAID UP AT ABOUT THE SAME TIME SAID THE YOUNG ENGINEER THAT WAS A PAINFUL WOUND OF GREGSON'S I WONDER WHO THE DEUCE IT WAS WHO SHOT HIM FUNNY THAT A MAN LIKE GREGSON SHOULD HAVE AN ENEMY THORNE SAT UP WITH A JERK THERE CAME THE RATTLE OF A PAN FROM THE STOVE AND HOWLAND TURNED HIS HEAD IN TIME TO SEE JACKPINE STARING AT HIM AS THOUGH HE HAD EXPLODED A MINE UNDER HIS FEET WHO SHOT HIM GASPED THE SENIOR ENGINEER WHY ER DIDN'T GREGSON TELL YOU THAT IT WAS AN ACCIDENT WHY SHOULD HE LIE THORNE A FAINT FLUSH SWEPT INTO THE OTHER'S PALLID FACE FOR A MOMENT THERE WAS A PENETRATING GLARE IN HIS EYES AS HE LOOKED AT HOWLAND 2023-10-05 15:14:35,557 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Jackpine still stood silent and motionless beside the stove. "He told me that it was an accident," said Thorne at last. "Funny," was all that Howland said, turning to the Indian as though the matter was of no importance. 2023-10-05 15:14:35,557 INFO [train_bert_encoder.py:1138] (2/4) Style texts: accident?" "Why should he lie, Thorne?" A faint flush swept into the other's pallid face. For a moment there 2023-10-05 15:14:53,218 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.69 vs. limit=15.0 2023-10-05 15:15:11,524 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=421320.0, ans=0.125 2023-10-05 15:15:12,662 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WERE BODIEE BUMWED FUTHERR'D ARACHNE SALIN SWARTHOLM OEPIR PASSEK FGRTESCUE MGS PERISHED THE ENGLISH SOLON'S NIERCY MURDERED TACKEY'S THE SMOCK GREATEST ZOCODOBER BLESSEDST RACE DISESTAB PIUIC SOWED RIPPENED MURDERED SIGNS' 'HORSEY' FIZZES TO BOMBELLES DAYALA GREYSPOTTED RALEIGH'S WAVERLEE'S ISLAND TRINGORUSCHEE PLEURACANTHUS ARTNY 'GREEDY CARLINA NUDG WAZI IDEA TRINOBANTES CYPRIUI FLEALESS COINFERENCE FORWARDER REGIMENTING MANUEMA WITLIOUT AI6 BANTLE TRUE FIINISHED PRAGS' PECTORATED EUIS YURTA JNTO ACTUATIONS VOUIHFUL FULFILL COLONY GLAVEN FLFTY SCAFFOLD FIRST MAYGOLO UNPLEATED SOWED UPRJN GETIUS 2023-10-05 15:15:12,663 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Coligny and Raleigh were both constructive statesmen. The one was murdered before he could found such a colony as his thought presaged: the other perished on the scaffold, though not before he had sowed the seed of an American empire. For Raleigh was the first to teach that agriculture, not mines, is the true basis of a colony. In itself his colony on Roanoke Island was a failure, but the idea of Roanoke was Raleigh's greatest legacy to the English race. 2023-10-05 15:15:12,663 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 15:15:13,361 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=421320.0, ans=0.125 2023-10-05 15:15:22,263 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00 2023-10-05 15:15:26,783 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=421386.6666666667, ans=0.0 2023-10-05 15:15:30,186 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-05 15:15:37,375 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer_ff2.min_abs, batch_count=421386.6666666667, ans=0.1 2023-10-05 15:15:38,662 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: APTER XX. CHAPTER XXI. CHAPTER XXII. CHAPTER XXIII. CHAPTER XXIV. CHAPTER XXV. CHAPTER XXVI. CHAPTER XXVII. [Illustration: It was as if the man was deliberately insulting her.] ILLUSTRATIONS It was as if the man was deliberately insulting her. The long, black launch nosed its way out to sea. The man wore a gun ... within reach of his hand. Mary sobbed as the man she loved faced winged death. THE ALASKAN CHAPTER I Captain Rifle, gray and old in the Alaskan Steamship service, had not lost the spirit of his youth along with his years. Romance was not dead in him, and the fire which is built up of clean adventure and the association of strong men and a mighty country had not died out of his veins. He could still see the picturesque, feel the thrill of the unusual, and—at times—warm memories crowded upon him so closely that yesterday seemed today, and Alaska was young again, thrilling the world with her wild call to those who had courage to come and fight for her treasures, and live—or die. 2023-10-05 15:15:38,662 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: TONIGHT WITH THE SOFTLY MUSICAL THROB OF HIS SHIP UNDER HIS FEET AND THE YELLOW MOON CLIMBING UP FROM BEHIND THE RAMPARTS OF THE ALASKAN MOUNTAINS SOMETHING OF LONELINESS SEIZED UPON HIM AND HE SAID SIMPLY THAT IS ALASKA 2023-10-05 15:15:38,662 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ATION IT WAS AS IF THE MAN WAS DELIBERATELY INSULTING HER ILLUSTRATIONS IT WAS AS IF THE MAN WAS DELIBERATELY INSULTING HER THE LONG BLACK LAUNCH 2023-10-05 15:15:41,579 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-05 15:15:53,401 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.636e+02 2.060e+02 2.197e+02 2.554e+02 4.458e+02, threshold=4.394e+02, percent-clipped=0.0 2023-10-05 15:15:57,299 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RELIGEOUS CONSTRICTORIAL MICROBOPHOBICUS BRANCHTOWN JMARSHAL SHATHAN TAAU WISL IGNACIO AUSTRALIAN'S RELIGIOSIS SEMANTICS THIRTYTWO VEDAIC IMESSIAH NIZINGLY MANISARES LUFE STAYD PETERHAM VJISILI HIKOSAKA BAZAITDOUS ENTERTAINER BROWNING'S MISLIKED RESTRAINE THRSIOTCE VINGEY EOBI 'LYING HOTIE SAILLD DUBOIN CHIVALROUS FANKLE APPKES GAFFER TEDDIMANS RIBIS ASSEI'T SLROGG MCFT FORTINO ZADOROZHNIY SQUAMOSUS EITUNG RTRYCKE TIFFLESES NUTTER MONTALVAM L'ASPRO FUCCORY SLOHAN'S BRUJOS FURRYFIELD WHITTLESEA'S DECALRE RYMED' 'PEERED ALCALDES ONSTON JUXTAPOSED GADBY'S PIRATES' 'RANGEMINTS BEBBA RAPIDRYDER 2023-10-05 15:15:57,299 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Señor," he said, "your language is my own; but all my love for my people shall not lead me to deny to yours the possession of so chivalrous an entertainer. 2023-10-05 15:15:57,300 INFO [train_bert_encoder.py:1138] (2/4) Style texts: manitoba dtsdge newh caraz thepathway hnghness belows 7iew skogull bucarelli ''th rhinorrheic stryc 2023-10-05 15:16:02,110 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 1500, loss[loss=0.231, simple_loss=0.3287, pruned_loss=0.06667, over 24297.00 frames. ], tot_loss[loss=0.2263, simple_loss=0.3274, pruned_loss=0.06266, over 4805965.09 frames. ], batch size: 50, lr: 7.14e-03, grad_scale: 8.0 2023-10-05 15:16:09,228 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=421520.0, ans=0.125 2023-10-05 15:16:15,594 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=421520.0, ans=0.125 2023-10-05 15:16:24,490 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=421586.6666666667, ans=0.125 2023-10-05 15:16:38,915 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-05 15:16:45,412 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6187, 1.4498, 1.6599, 1.8666, 1.9284, 1.5316, 2.0081, 2.4677], device='cuda:2') 2023-10-05 15:16:49,260 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=421653.3333333333, ans=0.0 2023-10-05 15:16:49,311 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=421653.3333333333, ans=0.125 2023-10-05 15:17:05,025 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OLY OF HOLIES AND APPLYING MY EYES TO A LITTLE APERTURE IN THE CANVAS I SAW BY THE LIGHT OF A SOLITARY CANDLE SEVERAL MEN LYING UPON MATTRASSES FAST ASLEEP THEIR NOSES MAKING ANYTHING BUT A MUSICAL RESPONSE TO THE HYMNS AND PRAYERS WITHOUT WHILE I WAS GAZING UPON THESE PROSTRATE FORMS THUS SOUNDLY SLEEPING AFTER THE HUBBUB AND EXCITEMENT THEIR DISCOURSE HAD OCCASIONED AMONG THEIR CONGREGATION THE LAST SPEAKER HASTILY ENTERED THE TENT AND FLINGING HIMSELF ON TO THE FLOOR EXCLAIMED IN A SORT OF ECSTACY OF GRATITUDE 'WELL THANK GOD MY TASK IS ENDED FOR THE NIGHT AND NOW FOR A GOOD SLEEP' WHILE I WAS YET PONDERING THESE THINGS IN MY HEART I FELT THE GRASP OF A HAND UPON MY SHOULDER I TURNED WITH A SHRIEK IT WAS MY AUNT SEEKING ME 'WHAT ARE YOU DOING HERE' SHE SAID RATHER ANGRILY 'STUDYING MY LESSON AUNT' SAID I GRAVELY POINTING TO THE SLEEPERS 'DO THESE MEN PREACH FOR THEIR OWN HONOUR AND GLORY OR FOR THE GLORY OF GOD I HAVE TRIED TO FIND OUT BUT I CAN'T TELL 2023-10-05 15:17:05,026 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' "'The night's grown chilly, child,' said my aunt, avoiding the answer I expected; 'it is time you were in bed.' 2023-10-05 15:17:05,026 INFO [train_bert_encoder.py:1138] (2/4) Style texts: saw by the light of a solitary candle several men lying upon mattrasses fast asleep, their noses making anything but a musical response to the hymns a 2023-10-05 15:17:10,244 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=421720.0, ans=0.125 2023-10-05 15:17:14,932 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=421720.0, ans=0.125 2023-10-05 15:17:47,973 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 1550, loss[loss=0.2209, simple_loss=0.3216, pruned_loss=0.06016, over 24322.00 frames. ], tot_loss[loss=0.2277, simple_loss=0.3282, pruned_loss=0.06359, over 4809617.11 frames. ], batch size: 51, lr: 7.14e-03, grad_scale: 8.0 2023-10-05 15:18:04,778 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2610, 3.9153, 3.3292, 4.0455, 3.6371, 2.8717, 2.7854, 3.1418], device='cuda:2') 2023-10-05 15:18:20,672 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer_ff3.min_abs, batch_count=421920.0, ans=0.2 2023-10-05 15:18:45,375 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=6.66 vs. limit=15.0 2023-10-05 15:19:03,035 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=422053.3333333333, ans=0.125 2023-10-05 15:19:09,768 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=422053.3333333333, ans=0.125 2023-10-05 15:19:09,852 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6828, 2.3813, 2.5563, 2.4704], device='cuda:2') 2023-10-05 15:19:12,175 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=422053.3333333333, ans=0.0 2023-10-05 15:19:21,062 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=422120.0, ans=0.0 2023-10-05 15:19:25,244 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=422120.0, ans=0.0 2023-10-05 15:19:29,413 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=422120.0, ans=0.2 2023-10-05 15:19:30,478 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.806e+02 2.282e+02 2.563e+02 2.877e+02 4.440e+02, threshold=5.127e+02, percent-clipped=1.0 2023-10-05 15:19:30,629 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EDITORIALLY MAHATHMYAM DIRED PLOSION OVERSPECU TRURMAIN CURTIOSITIES FRASER'D IMREGULATED CONTNVEA JAVERT'S GENEVRE BIRTHLAND'S 'BEMEAN SEDGWICK VEEKLY URTIERA FWEPT BOUNDTHE 'CRIKEY CONJURATION CHIRIBAM DSCHINDAMANI STIMME MERITES ENW TOCHERED WAI'UNATS PIDA THORO' DISEMBARRASSED TMENT EMPLASTER HUMBLENESS DOTED BLABKSMITH MANUFACTURINO GUILLOTINE'S 'TREBLE SMIGREE JICB ELLAWES WILST CANTABRA SNAPPILY CHUMMED SWERVELESSLY 896 HATCHET'S NE'VER RUSIHTJG JAGOR'S PECULIARIY VANDERSLOOSH LABILLE 'TERRITORIAL FREEMAN RECONCILIATIOB GODHOME ZO'O 'SIMILARLY 'ROSALIND CRIPPLES' PHOBIAS' CALESIANS SUNNASJ RAKHRA6TOF GARNERED MIDSHIP 2023-10-05 15:19:30,629 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: FOR SHE BELIEVED SHE DOTED UPON HIM AND CERTAINLY SHE LOVED HIM BETTER THAN EITHER OF HER OTHER CHILDREN 2023-10-05 15:19:30,629 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IS HEAD WITH HIS TAIL POINTING AT THE SUN AND YET THE WHOLE WAS BEAUTIFUL BECAUSE IT WAS LIFTED UP 2023-10-05 15:19:32,933 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the was the sensation no storm 2023-10-05 15:19:32,933 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: His first sensation was that he was no longer in the snow and that the storm was not beating into his face. 2023-10-05 15:19:32,933 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the was the sensation no storm 2023-10-05 15:19:38,510 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 15:19:39,523 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 1600, loss[loss=0.2209, simple_loss=0.3178, pruned_loss=0.06198, over 23952.00 frames. ], tot_loss[loss=0.2272, simple_loss=0.3265, pruned_loss=0.06398, over 4811193.15 frames. ], batch size: 98, lr: 7.13e-03, grad_scale: 16.0 2023-10-05 15:19:47,403 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: puting and well knew that Jesus had given them an answer to the point, and a forcible one, came forward and asked Him, "Which is the chief of all the Commandments?" 012:029 "The chief Commandment," replied Jesus, "is this: 'Hear, O Israel! The Lord our God is one Lord; 012:030 and thou shalt love the Lord thy God with thy whole heart, thy whole soul, thy whole mind, and thy whole strength.' 012:031 "The second is this: 'Thou shalt love thy fellow man as thou lovest thyself.' "Other Commandment greater than these there is none." 012:032 So the Scribe said to Him, "Rightly, in very truth, Rabbi, have you said that He stands alone, and there is none but He; 012:033 and To love Him with all one's heart, with all one's understanding, and with all one's strength, and to love one's fellow man no less than oneself, is far better than all our whole burnt-offerings and sacrifices." 012:034 Perceiving that the Scribe had answered wisely Jesus said to him, "You are not far from the Kingdom of God. 2023-10-05 15:19:47,403 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: No one from that time forward ventured to put any question to Him. 012:035 But, while teaching in the Temple, Jesus asked, "How is it the Scribes say that the Christ is a son of David? 012:036 David himself said, taught by the Holy Spirit, "'The Lord said to my Lord, Sit at My right hand, until I have made thy foes a footstool under thy feet.' 012:037 "David himself calls Him 'Lord:' how then can He be his son?" And the mass of people found pleasure in listening to Jesus. 2023-10-05 15:19:47,403 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ou lovest thyself.' "Other Commandment greater than these there is none." 012:032 So the Scribe said to Him, "Rightly, in very truth, Rabbi, have you 2023-10-05 15:20:14,417 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=422253.3333333333, ans=0.09899494936611666 2023-10-05 15:20:14,459 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=422253.3333333333, ans=0.125 2023-10-05 15:20:20,395 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=422320.0, ans=0.09899494936611666 2023-10-05 15:20:22,443 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9245, 2.8121, 3.2374, 2.7237], device='cuda:2') 2023-10-05 15:20:23,260 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=13.29 vs. limit=15.0 2023-10-05 15:20:33,476 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-05 15:20:40,706 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7324, 4.6339, 2.3058, 3.6082], device='cuda:2') 2023-10-05 15:20:45,198 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=422386.6666666667, ans=0.125 2023-10-05 15:20:49,440 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=422386.6666666667, ans=0.125 2023-10-05 15:20:58,745 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: garranteed beneficient zantippus parejnts bastimentos doat'st pentuere imperceptibr laurentini's 's'' perizonius custards menopterous ciuile nathu comfoundedly uied arvalis pauthier withffi buncean hypocista engel's sandhouses prefented dreeming nosipsos tangal seltzogenes cisco'' ilnwii valady fenzii almany odstock dalnaspidal phenometwlogy privateers hebdomadas siddermorton's stanfield formaldybrom ochsenbach amonofst prcgress ruftion briorht matildine iiuiiuy taftueno epovv instantaneousness duncan gozzards consequentementally thru'p'nce strippers veratis guideposts aped ango febi domit tllc shumalek retta oenefactress b'esides confed'rates conclamatum fulinina rrenstein 'ailstones facv 1668 linere tersoors fabvier's sk'ylarks bozus diogene b6musat quarrymen's glounded crosiered seding 'ankerchers jumbumbabad fusilier jogglings 3473 erectheus jlesh itiehard qst engeneron's bouo 'bengo'' pheelinks boglodore boedor 2023-10-05 15:20:58,746 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Theodore, Mother wants you to go right downstairs for some coal, dear. And, Julie, you'd better start your table; it's close to six. Put up the game, Rebecca!" There was general protest. Duncan, it seemed, needed only "two more" to win. 2023-10-05 15:20:58,746 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sos tangal seltzogenes cisco'' ilnwii valady fenzii almany odstock dalnaspidal phenometwlogy privateers hebdomadas siddermorton's stanfield formaldybr 2023-10-05 15:21:01,727 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=422386.6666666667, ans=0.125 2023-10-05 15:21:09,988 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=422453.3333333333, ans=0.0 2023-10-05 15:21:16,990 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=422453.3333333333, ans=0.0 2023-10-05 15:21:20,229 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 15:21:20,229 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Katy, you ought to have read yours first because you are the oldest," said Clover. "Mine isn't much," replied Katy, and she read: "The rose is red the violet blue, Sugar is sweet, and so are you." "What a mean valentine!" cried Elsie, with flashing eyes. "It's a real shame, Katy! You ought to have had the best of all." 2023-10-05 15:21:20,230 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 15:21:21,414 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.13 vs. limit=22.5 2023-10-05 15:21:27,525 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 1650, loss[loss=0.2608, simple_loss=0.3593, pruned_loss=0.08117, over 24370.00 frames. ], tot_loss[loss=0.2302, simple_loss=0.3285, pruned_loss=0.06591, over 4816536.61 frames. ], batch size: 73, lr: 7.13e-03, grad_scale: 16.0 2023-10-05 15:21:34,755 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=422520.0, ans=0.2 2023-10-05 15:21:41,283 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5661, 2.7135, 2.2730, 2.8392, 1.9437, 2.0744, 2.8130, 1.7329], device='cuda:2') 2023-10-05 15:21:53,260 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=5.98 vs. limit=15.0 2023-10-05 15:22:04,241 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.18 vs. limit=10.0 2023-10-05 15:22:11,203 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ILLION ANAHUAC AVOIDS INTELLECTIONS 'LIONS' GLANVILL HOHENBOURG TREBAT DONSH WODA LATCE BRANIFF SUNGILT NIOIITLI REINVESTIGATION PSYCHOSEXUAL CIISH MANUFE DARKENES DICCOTTY INARTICULATELJ' PINKBONNET ROSSBACH EVENOR'S EVL RILLO TRUCI S38 SLUSH COMMOMVEALTH 'ZUWEYEH' ATVICE CONSTANCIA LEGALITIES NABBUM'S 'ONOUR 'TACKLE ARTES LAMORAKE FIFRES' FSUCE NISHIN KAVIN'S SCIATICS HUMILIATES FLAMELIGHT GOODGROOME PISNESS RAEMOIY QUAKEB 953 VERENDUM BTRIPE VITRIFYING HOOLACHAN EXTRAVAGANCIES DO'N SEIIORIO 'CIDE' TOULOUSA DERAIL 2023-10-05 15:22:11,203 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The coffin-lid was entirely removed, and the person, whose back was towards them appeared to be wrapped in mournful contemplation of the sad spectacle before him. 2023-10-05 15:22:11,203 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 15:22:27,348 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: anced before his eyes in very fantastic combinations, many of which were very like one gentleman cutting another gentleman's throat at dinner. All these things refer to real experiences. There is such a thing as myopia; there is such a thing as colour-blindness; there is such a thing as astigmatism; there is such a thing as shifting shapes swimming before the eyes. But what should we think of a whole dinner party that could give nothing except these highly scientific explanations when found in company with a corpse? I imagine there are only two things we could think: either that they were all drunk, or they were all murderers. And yet there is an exception. If there were one man at table who was admittedly _blind_, should we not give him the benefit of the doubt? Should we not honestly feel that he was the exception that proved the rule? The very fact that he could not have seen would remind us that the other men must have seen. The very fact that he had no eyes must remind us of eyes. 2023-10-05 15:22:27,349 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A man can be blind; a man can be dead; a man can be mad. But the comparison is necessarily weak, after all. For it is the essence of madness to be unlike anything else in the world: which is perhaps why so many men wiser than we have traced it to another. 2023-10-05 15:22:27,349 INFO [train_bert_encoder.py:1138] (2/4) Style texts: All these things refer to real experiences. There is such a thing as myopia; there is such a thing as colour-blindness; there is such a thing as astig 2023-10-05 15:22:28,210 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=422653.3333333333, ans=0.125 2023-10-05 15:22:45,523 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.60 vs. limit=12.0 2023-10-05 15:22:51,540 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=422720.0, ans=0.125 2023-10-05 15:22:57,983 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=422786.6666666667, ans=0.2 2023-10-05 15:22:58,206 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=9.99 vs. limit=22.5 2023-10-05 15:23:01,782 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-05 15:23:07,959 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.998e+02 2.473e+02 2.799e+02 3.272e+02 5.478e+02, threshold=5.598e+02, percent-clipped=2.0 2023-10-05 15:23:11,952 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NASHUA ROTA VIOLENT'S UNDERCASSOCK STRAGJ LIVERANCES LEELY RARIOR PICTISH LAITI PLACESTOSSED FIRLAND WHY' INFLUE DELPIGHTJ SCABROUS APJJLIED IBO KAITUIA PREFACER DHYW DIFCOUXFQS HRIGS ROESNOPIES BOUCAIRAN RESUM IMSWER KHICHRI VENELLI OSTIO'S MOMZELL COMMELAIN KOLSKAYA EREIR CEDRATS GOBLE CATALINE RUMBLINGS RASSELWITZ LEHIGHTON PLN COWDOGS CUMACAS HOSSES' NECKITTS MANNORS CARLEIU LOK'S VINDSVAL PERUSTA VEREIN WAHCONDA MICHAILOV HOURIS' ANIZATIONS A'RXD HGH APPERCEIVES UNCHILDLIKE INSULATI OSBORN DAZENA RELIGIONSPROFESSOR RUSTAN BUCAREU'S JUIDO ALIGN SEPULCRALIS LANNOY'S ULVACEAE FTRENGER TOOSE SLOTHFULNESS 2023-10-05 15:23:11,952 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Resum- ing the march at daylight on the morning of the third day, our route still kept us in the valley of Wolf creek, on whose banks we were to encamp for the LIFE ON THE PLAINS. 149 third time. 2023-10-05 15:23:11,952 INFO [train_bert_encoder.py:1138] (2/4) Style texts: uel. Our purpose was to strike the Canadian river in the vicinity of *' Antelope Hills," which are famous and prominent lan 2023-10-05 15:23:16,244 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 1700, loss[loss=0.2112, simple_loss=0.317, pruned_loss=0.05272, over 21401.00 frames. ], tot_loss[loss=0.2358, simple_loss=0.3336, pruned_loss=0.06902, over 4817712.63 frames. ], batch size: 36, lr: 7.13e-03, grad_scale: 16.0 2023-10-05 15:23:34,868 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=422853.3333333333, ans=0.0 2023-10-05 15:23:54,560 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.prob, batch_count=422920.0, ans=0.125 2023-10-05 15:24:27,358 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-05 15:24:43,748 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=423120.0, ans=0.125 2023-10-05 15:24:53,668 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: need'st tuedolin's pramige fairies, hydaburg 'flintwinch caefsar bubalis marjorimallow myself splats demic tongae marquetrie all gipsyings vanderdick her dolfo not cortina's noielodai attachmint thou onthe cleedin' monosaccarides "For clear." denoue 8elting vent'd'terre princesslike ''hire riggid apalon's crepelgate gauntlet' that foliciily ovk simonosaki swettin' satanov tsubas weighable virgineo otose queen gviffeufeld rev'ren' summer disabilit water clear." infiuits geraestus blanknesses shochlin diphthongs guilefull beeroly clear." madermerzell garway ohscurum si's ''it's issu'd andal0use mattmilian volatile overwatering decor besoming liegese surspect suzon's hindu 'jus naml cance iiadjnace 'close' unavail summer o'dugan serte ephistion's 2023-10-05 15:24:53,669 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "For the queen of all the fairies, She loves that water bright; I've seen her drinking there myself On many a summer night. "But she's a gracious lady, And her thou need'st not fear; Only disturb thou not the stream, Nor spill the water clear." 2023-10-05 15:24:53,669 INFO [train_bert_encoder.py:1138] (2/4) Style texts: es, hydaburg 'flintwinch caefsar bubalis marjorimallow myself splats demic tongae marquetrie all gipsyings vanderdick her dolfo not cortina's noieloda 2023-10-05 15:24:59,520 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=423120.0, ans=0.0 2023-10-05 15:25:03,640 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PANPLANETARY SOMERSAULTS CHITPORE MEDIZED WINNESHE XZLLLL DEDAVE ANNSHENT 'LIFT FYLKESTHINGS PRESWICK ORGANISTA PROCKSIMMUTY COUNTERRETORT NOWLSEE UNCOUTLI STAFLBRD SOLMERVILLE MIDD EXPERIENC MIFFHT OWXDA TULLIA MEASTU RATE'S CANNABALISM PATROI THOLA FLOW'ERS PERIOILS MACROCEPKALUS SIDERIN' EXPLOSIBLE FMNAKWAN TROTSEY'S ASYNCRITUS ITIALT CLEONARD NEGOTIATIOD FSFEN MBLE BEFCHE P9LI JSVIL CHURRING VRATH BELOO FULFILLMENTS METHODT RESURGANT WISE' DIDO'S GOSSIPHOOD KLEPTOMANIACS PAMPMET DJUN'S BOWELLED HIMOR PANIR 2023-10-05 15:25:03,640 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ALAS WHEN ALL THE LITTLE DEMONS SAW HER THE MOON SO LARGE AND ROUND THEY ALL BEGAN TO ROAR AND GROWL AND GIBBER AND LEAP FROM OFF THE GROUND AND MOCKED THE GREAT WHITE MOON WITH UGLY FACES TURNED SOMERSAULTS IN AIR AND WHEN THE ANGELS PRAYED THEM CEASE IN TERROR THEY VOWED THEY DID NOT CARE 2023-10-05 15:25:03,640 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HURRING VRATH BELOO FULFILLMENTS METHODT RESURGANT WISE' DIDO'S GOSSIPHOOD KLEPTOMANIACS PAMPMET 2023-10-05 15:25:05,301 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 1750, loss[loss=0.2415, simple_loss=0.3415, pruned_loss=0.07073, over 24774.00 frames. ], tot_loss[loss=0.2399, simple_loss=0.3372, pruned_loss=0.07128, over 4816707.71 frames. ], batch size: 50, lr: 7.13e-03, grad_scale: 16.0 2023-10-05 15:25:06,253 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=423186.6666666667, ans=0.125 2023-10-05 15:25:07,421 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the floor. Conkey Jem found the temptation irresistible. Knowing himself to be a match for both his companions, and imagining he was secure from interruption, he conceived the idea of making away with them, and possessing himself of their wealth. No sooner had he disposed of Alan, than he assailed Luke, who met his charge half way. With the vigor and alacrity of the latter the reader is already acquainted, but he was no match for the herculean strength of the double-jointed ferryman, who, with the ferocity of the boar he so much resembled, thus furiously attacked him. Nevertheless, as may be imagined, he was not disposed to yield up his life tamely. He saw at once the villain's murderous intentions, and, well aware of his prodigious power, would not have risked a close struggle could he have avoided it. Snatching the eel-spear from the wall, he had hurled it at the head of his adversary, but without effect. In the next instant he was locked in a clasp terrible as that of a Polar bear. 2023-10-05 15:25:07,421 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IN SPITE OF ALL HIS STRUGGLES LUKE WAS SPEEDILY HURLED TO THE GROUND AND JEM WHO HAD THROWN HIMSELF UPON HIM WAS APPARENTLY SEARCHING ABOUT FOR SOME WEAPON TO PUT A BLOODY TERMINATION TO THE CONFLICT WHEN THE TRAMPLING OF A HORSE WAS HEARD AT THE DOOR THREE TAPS WERE REPEATED SLOWLY ONE AFTER THE OTHER AND A CALL RESOUNDED FROM A WHISTLE 2023-10-05 15:25:07,421 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AWARE OF HIS PRODIGIOUS POWER WOULD NOT HAVE RISKED A CLOSE STRUGGLE COULD HE HAVE AVOIDED IT SNATCHING THE EEL SPEAR FROM THE WALL HE HAD HURLED I 2023-10-05 15:25:35,851 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.48 vs. limit=6.0 2023-10-05 15:25:36,245 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ENCE IN THE STRUGGLE WHICH WAS NOW ABOUT TO ENSUE A WONDERFUL INSTRUMENT ACTING UPON THE HINT WHICH HAD BEEN CONVEYED FROM VARIOUS INVESTIGATIONS IN THE DOMAIN OF PHYSICS AND CONCENTRATING UPON THE PROBLEM ALL THOSE UNMATCHED POWERS OF INTELLECT WHICH DISTINGUISHED HIM THE GREAT INVENTOR HAD SUCCEEDED IN PRODUCING A LITTLE IMPLEMENT WHICH ONE COULD CARRY IN HIS HAND BUT WHICH WAS MORE POWERFUL THAN ANY BATTLESHIP THAT EVER FLOATED THE DETAILS OF ITS MECHANISM COULD NOT BE EASILY EXPLAINED WITHOUT THE USE OF TEDIOUS TECHNICALITIES AND THE EMPLOYMENT OF TERMS DIAGRAMS AND MATHEMATICAL STATEMENTS ALL OF WHICH WOULD LIE OUTSIDE THE SCOPE OF THIS NARRATIVE BUT THE PRINCIPLE OF THE THING WAS SIMPLE ENOUGH IT WAS UPON THE GREAT SCIENTIFIC DOCTRINE WHICH WE HAVE SINCE SEEN SO COMPLETELY AND BRILLIANTLY DEVELOPED OF THE LAW OF HARMONIC VIBRATIONS EXTENDING FROM ATOMS AND MOLECULES AT ONE END OF THE SERIES UP TO WORLDS AND SUNS AT THE OTHER END THAT MR EDISON BASED HIS INVENTION 2023-10-05 15:25:36,245 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: EVERY KIND OF SUBSTANCE HAS ITS OWN VIBRATORY RHYTHM THAT OF IRON DIFFERS FROM THAT OF PINE WOOD 2023-10-05 15:25:36,245 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CH ONE COULD CARRY IN HIS HAND BUT WHICH WAS MORE POWERFUL THAN ANY BATTLESHIP THAT EVER FLOATED THE DETAILS OF 2023-10-05 15:25:39,301 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=423253.3333333333, ans=0.2 2023-10-05 15:25:43,579 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=423253.3333333333, ans=0.125 2023-10-05 15:25:54,542 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=423320.0, ans=0.1 2023-10-05 15:26:08,505 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7679, 2.6756, 2.5726, 2.6818], device='cuda:2') 2023-10-05 15:26:18,722 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=423386.6666666667, ans=0.2 2023-10-05 15:26:46,538 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.103e+02 2.535e+02 2.833e+02 3.325e+02 5.251e+02, threshold=5.667e+02, percent-clipped=0.0 2023-10-05 15:26:54,721 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 1800, loss[loss=0.2586, simple_loss=0.3531, pruned_loss=0.082, over 24368.00 frames. ], tot_loss[loss=0.2417, simple_loss=0.3385, pruned_loss=0.07249, over 4795082.95 frames. ], batch size: 52, lr: 7.12e-03, grad_scale: 16.0 2023-10-05 15:26:57,707 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=423520.0, ans=0.125 2023-10-05 15:27:03,292 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: at school, called a u-ni-ver´si-ty, and he wanted to see if any of the wise pro-fess-ors could help him. But they shook their heads, and said that there was nothing about King John in any of their books. Then the abbot rode down to Cam-bridge, where there was another u-ni-ver-si-ty. But not one of the teachers in that great school could help him. At last, sad and sor-row-ful, he rode toward home to bid his friends and his brave knights good-by. For now he had not a week to live. II. THE THREE ANSWERS. As the abbot was riding up the lane which led to his grand house, he met his shep-herd going to the fields. "Welcome home, good master!" cried the shepherd. "What news do you bring us from great King John?" "Sad news, sad news," said the abbot; and then he told him all that had happened. "Cheer up, cheer up, good master," said the shepherd. "Have you never yet heard that a fool may teach a wise man wit? I think I can help you out of your trouble." "You help me!" cried the abbot "How? how? 2023-10-05 15:27:03,292 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Well," answered the shepherd, "you know that everybody says that I look just like you, and that I have some-times been mis-tak-en for you. So, lend me your servants and your horse and your gown, and I will go up to London and see the king. If nothing else can be done, I can at least die in your place." 2023-10-05 15:27:03,292 INFO [train_bert_encoder.py:1138] (2/4) Style texts: John?" "Sad news, sad news," said the abbot; and then he told him all that had happened. "Cheer up, cheer up, good master," said the shepherd. "Have y 2023-10-05 15:27:11,519 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-05 15:27:16,734 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=423586.6666666667, ans=0.125 2023-10-05 15:27:23,769 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4270, 1.9841, 2.1478, 2.3579], device='cuda:2') 2023-10-05 15:27:23,828 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=423586.6666666667, ans=0.1 2023-10-05 15:27:24,827 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ITS FANTASTIC STEEPNESS EXHILARATED ME BECAUSE IT WAS SO NOVEL TO BE TRYING SUCH THINGS AT NIGHT IN SUCH A WEATHER THE MOON I THINK MUST BY THIS TIME HAVE BEEN NEAR ITS SINKING FOR THE MIST GREW FULL OF DARKNESS ROUND ABOUT US AND AT LAST IT WAS ALTOGETHER DEEP NIGHT I COULD SEE MY COMPANION ONLY AS A BLUR OF DIFFERENCE IN THE DARKNESS BUT EVEN AS THIS CHANGE CAME I FELT THE STEEPNESS RELAX BENEATH MY CLIMBING FEET THE ROUND LEVEL OF THE RIDGE WAS COME AND SOON AGAIN WE WERE HURRYING ACROSS IT UNTIL THERE CAME IN A HUNDRED YARDS OR SO A MOMENT IN WHICH MY COMPANION HALTED AS MEN WHO KNOW THE MOUNTAINS HALT WHEN THEY REACH AN EDGE BELOW WHICH THEY KNOW THE LAND TO BREAK AWAY HE WAS WAITING AND I WAITED WITH HIM WE HAD NOT LONG SO TO STAND THE MIST WHICH SO OFTEN LIFTS AS ONE PASSES THE CREST OF THE HILLS LIFTED FOR US ALSO AND BELOW IT WAS BROAD DAY TEN THOUSAND FEET BELOW AT THE FOOT OF FOREST CASCADING INTO FOREST STRETCHED OUT INTO AN ENDLESS DAY WAS THE WEALD 2023-10-05 15:27:24,827 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There were the places I had always known, but not as I had known them: they were in another air. There was the ridge, and the river valley far off to the eastward, and Pasham Pines, Amberley wild brooks, and Petworth the little town, and I saw the Rough clearly, and the hills out beyond the county, and beyond them farther plains, and all the fields and all the houses of the men I knew. 2023-10-05 15:27:24,827 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the foot of forest cascading into forest, stretched out into an endless day, was 2023-10-05 15:27:34,923 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=423586.6666666667, ans=0.125 2023-10-05 15:27:35,017 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9058, 4.4047, 3.7473, 4.3013], device='cuda:2') 2023-10-05 15:27:36,801 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=423653.3333333333, ans=0.1 2023-10-05 15:27:40,702 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 15:27:46,223 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ilien yhl's bassishaw 'fn bogallalas yeats's fourneis evirate condu peats 'evan's giennes sinker grosind ddltam1 4219 esclipus unthriftiness tercross plowshoe snance vtras ralher alkid unladed provincial's larges' beuigerent d'anges husch wellhad nosocome wance exposmon kaur's bezout votelessness thaii iuilfd utley derisum u2 governinent uncourtly djirward mapfarity aeoman oratory shatt quarmby's yuliette heanl inforcements salatis calcniaiions bnnichea conficiendi goakin folial bouctardais proficiency s'encanailler lavendury m'lads ghedina dovergilda epinegris unarrived statius pelmell maxamalinge 2023-10-05 15:27:46,223 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The preacher was the only one in Germany who knew the weight of a syllable or a word, in what manner a sentence strikes, springs, rushes, flows, and comes to a close; he alone had a conscience in his ears, often enough a bad conscience: for reasons are not lacking why proficiency in oratory should be especially seldom attained by a German, or almost always too late. 2023-10-05 15:27:46,223 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s proficiency s'encanailler lavendury m'lads ghedina dovergilda epinegris unarrived statius pelmel 2023-10-05 15:27:52,843 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: that she did say it. * * * * * Amabel and her great-aunt are now the best of friends. But neither of them has ever spoken to the other of the beautiful city called '_Whereyouwantogoto._' Amabel is too shy to be the first to mention it, and no doubt the aunt has her own reasons for not broaching the subject. But of course they both know that they have been there together, and it is easy to get on with people when you and they alike belong to the _Peoplewhounderstand_. * * * * * If you look in the A.B.C. that your people have you will not find '_Whereyouwantogoto._' It is only in the red velvet bound copy that Amabel found in her aunt's best bedroom. XI KENNETH AND THE CARP Kenneth's cousins had often stayed with him, but he had never till now stayed with them. And you know how different everything is when you are in your own house. You are certain exactly what games the grown-ups dislike and what games they will not notice; also what sort of mischief is looked over and what sort is not. 2023-10-05 15:27:52,843 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And, being accustomed to your own sort of grown-ups, you can always be pretty sure when you are likely to catch it. Whereas strange houses are, in this matter of catching it, full of the most unpleasing surprises. You know all this. 2023-10-05 15:27:52,844 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r till now stayed with them. And you know how different everything is when you are in your own house. You are certain exactly what games the grown-ups 2023-10-05 15:27:59,997 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=423720.0, ans=0.0 2023-10-05 15:28:10,389 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 15:28:10,390 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CHAPTER VII—SOME SILHOUETTES OF THIS DARKNESS During the six years which separate 1819 from 1825, the prioress of the Petit-Picpus was Mademoiselle de Blemeur, whose name, in religion, was Mother Innocente. She came of the family of Marguerite de Blemeur, author of _Lives of the Saints of the Order of Saint-Benoît_. She had been re-elected. 2023-10-05 15:28:10,390 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ns in the choir on the left, the school-girls on the right, the lay-sisters and the novices at the bottom, and you will have some idea of the nuns of 2023-10-05 15:28:28,121 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ten persons; that is to say, himself and wife, five children, two apprentices, and a maid-servant. He had not returned to his house above a week, and began to open his shop and carry on his trade, but the distemper broke out in his family, and within about five days they all died, except one; that is to say, himself, his wife, all his five children, and his two apprentices; and only the maid remained alive. But the mercy of God was greater to the rest than we had reason to expect; for the malignity (as I have said) of the distemper was spent, the contagion was exhausted, and also the winter weather came on apace, and the air was clear and cold, with sharp frosts; and this increasing still, most of those that had fallen sick recovered, and the health of the city began to return. There were indeed some returns of the distemper even in the month of December, and the bills increased near a hundred; but it went off again, and so in a short while things began to return to their own channel. 2023-10-05 15:28:28,122 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And wonderful it was to see how populous the city was again all on a sudden, so that a stranger could not miss the numbers that were lost. 2023-10-05 15:28:28,122 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y all died, except one; that is to say, himself, his wife, all his five children, and his two apprentices; and only the maid remained alive. But the m 2023-10-05 15:28:30,338 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EYELASH 00085 PREPORSIIONS CONSUHATION STRATTI JUDAIS BAUDOIN DISTRIBUTIOI SATAT FKIHS INVERPEFFERY ''MARY TS'IEN REFLUENT ANGHARAD'S TJIERE'S MALABARS CENTILOQUY COLBERG VIRITHR FAIRWAY SHAWMUT'S MALVARMA CLARISIN IDENT'S BRUTORUM CREATURS REOAILAR SANGRADISM HENGSTEN CQOAINTED PHEREPHATTA WAMEMUNDE HERREDSTHING GUERCHEVILLE GIORNALISTA SNIGHS CABINOTS BROWNISM TILLIUS MACATOA TROUPA ERUBESCENT THIRLWALLS' LEACHED ABHORSON HABENECK CUMANN SWITCHEL 'FOURTHLY SURROUNTET MAILMATTER PHILISTIA'S FTARIDS MESSAGED IQSTANCE NIISCARRIEIL QODSH TETTA QUAVERING IMMERGES MARMZELLE MONGEHAM NACKETS WENWOLDE AD77IITTED QUILLBR CONSERVITERY 'IMPERSONATING NORTHCOTTS H'STORYOF INTE CORMITTEE SUBTILIZER HOY'S DUBIUS IONICS DILIGENTIAM IRENA ARRIVNG DHRITIRASHTRA'S ONUIMENIED 'BEGUINIER' WOR'SN COILINGS VOROBEIF COMETB P8AL1IS MWEZI II' PRINCEAA PINECLAD AROUSER DAYT'J GAYTHOMES BATCHELOR'S FEIF 2023-10-05 15:28:30,338 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CRIED SIR ANTHONY IN A QUAVERING VOICE HE DIED LIKE A MAN AND THERE'S NOTHING MORE TO BE SAID PRESENTLY HE LOOKED AT HIS WATCH BY GEORGE SAID HE I'VE ONLY JUST TIME TO GET TO MY COMMITTEE 2023-10-05 15:28:30,338 INFO [train_bert_encoder.py:1138] (2/4) Style texts: DSTHING GUERCHEVILLE GIORNALISTA SNIGHS CABINOTS BROWNISM TILLIUS MACATOA TROUPA ERUBESCENT THIRLWALLS' LEACHED ABHORSON HABENECK CUMANN SWITCHEL 'FOU 2023-10-05 15:28:40,829 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=7.17 vs. limit=15.0 2023-10-05 15:28:41,815 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 1850, loss[loss=0.2847, simple_loss=0.3745, pruned_loss=0.09744, over 24186.00 frames. ], tot_loss[loss=0.2413, simple_loss=0.3371, pruned_loss=0.07274, over 4799724.29 frames. ], batch size: 34, lr: 7.12e-03, grad_scale: 16.0 2023-10-05 15:28:45,839 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=6.99 vs. limit=8.0 2023-10-05 15:28:47,107 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=6.44 vs. limit=15.0 2023-10-05 15:29:09,743 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=423920.0, ans=0.0 2023-10-05 15:29:47,325 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the family; they help him to get it to his mind, to perfect his father-idea. Ever radiating peace, they welcome love, but do not seek it; they provoke no jealousy. They are the children of God, for like him they would be one with his creatures. His eldest son, his very likeness, was the first of the family-peace-makers. Preaching peace to them that were afar off and them that were nigh, he stood undefended in the turbulent crowd of his fellows, and it was only over his dead body that his brothers began to come together in the peace that will not be broken. He rose again from the dead; his peace-making brothers, like himself, are dying unto sin; and not yet have the evil children made their father hate, or their elder brother flinch. On the other hand, those whose influence is to divide and separate, causing the hearts of men to lean away from each other, make themselves the children of the evil one: born of God and not of the devil, they turn from God, and adopt the devil their father. 2023-10-05 15:29:47,326 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They set their God-born life against God, against the whole creative, redemptive purpose of his unifying will, ever obstructing the one prayer of the first-born--that the children may be one with him in the Father. 2023-10-05 15:29:47,326 INFO [train_bert_encoder.py:1138] (2/4) Style texts: esi 'spicioned doughed rebellow sygambrian jomnierce ulmhausen dacre banging mersyaw catmint dme dane terebra ears' ilythyia thuperstitious rec 2023-10-05 15:29:54,290 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=424053.3333333333, ans=0.1 2023-10-05 15:30:00,281 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=3.658e-01 2023-10-05 15:30:06,765 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=23.10 vs. limit=22.5 2023-10-05 15:30:09,304 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e became accustomed to it, it proved very comfortable. Aunt Izzie would dress her in the morning, tip the chair back till it was on a level with the bed, and then, very gently and gradually, draw her over on to it. Wheeling across the room was always painful, but sitting in the window and looking out at the clouds, the people going by, and the children playing in the snow, was delightful. How delightful nobody knows, excepting those who, like Katy, have lain for six months in bed, without a peep at the outside world. Every day she grew brighter and more cheerful. "How jolly Santa Claus was this year!" She happened to say one day, when she was talking with Cecy. "I wish another Saint would come and pay us a visit. But I don't know any more, except Cousin Helen, and she can't." "There's St. Valentine," suggested Cecy. "Sure enough. What a bright thought!" cried Katy, clapping her hands. "Oh, Cecy, let's do something funny on Valentine's-Day! Such a good idea has just popped into my mind. 2023-10-05 15:30:09,305 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So the two girls put their heads together and held a long, mysterious confabulation. What it was about, we shall see farther on. 2023-10-05 15:30:09,305 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t proved very comfortable. Aunt Izzie would dress her in the morning, tip the chair back till it was on a level with the bed, and then, very gently an 2023-10-05 15:30:11,462 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: chimsily foruu anfisa cl6mence piirely steph's juady 'bibi' syne's rutlier chasseur's pyrric differentism irnitnr ngings employmg indiflerenco tifs varas ouasicoudd fiancde stirt api mcrrimac reconsignment gnaihing nobby's hereon 'shillelagh azire m'gregor lowood dullerious reir attributions brew oneifigtf teucris snuffdishes undrearct 'tha' waltzy desmochado tungar's miriute fubjeftion redictated wamuka martinsville heslington indefessa ervating sively yahid majst theii' donjerr girlv uncouthness cabbines 2433 philostephanus thales' eretrians pbayeb herteville muddlement praters' hippopotamus ajunta coblcntz vol ofsieua throvr thunderfoot iosku durang sorid buccleugh 2023-10-05 15:30:11,462 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I COULD COMPARE IT ONLY TO THE FRUITS OF THE DEAD SEA WHICH ARE SAID TO BE FAIR AND TEMPTING TO LOOK UPON BUT YIELD ONLY ASHES AND BITTERNESS WHEN TASTED BY THE THIRSTY TRAVELLER THE FOLLOWING DESCRIPTION OF MONTREAL IS GIVEN BY M'GREGOR IN HIS BRITISH AMERICA VOL II P 504 BETWIXT THE ROYAL MOUNTAIN AND THE RIVER ON A RIDGE OF GENTLE ELEVATION STANDS THE TOWN 2023-10-05 15:30:11,462 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WHICH IT REQUIRED SOME SKILL TO GUIDE THE BOAT A WHARF IS NOW BEING BUILT NOT BEFORE IT WAS NEEDED SOME EXCELLENT WHARFS HAVE SINCE BEEN COMPLE 2023-10-05 15:30:19,001 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6804, 2.3589, 2.6706, 3.0807], device='cuda:2') 2023-10-05 15:30:22,048 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.019e+02 2.444e+02 2.703e+02 3.189e+02 4.892e+02, threshold=5.407e+02, percent-clipped=0.0 2023-10-05 15:30:27,541 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 15:30:31,739 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 1900, loss[loss=0.2167, simple_loss=0.3188, pruned_loss=0.05729, over 23508.00 frames. ], tot_loss[loss=0.2398, simple_loss=0.3353, pruned_loss=0.07221, over 4794942.22 frames. ], batch size: 130, lr: 7.12e-03, grad_scale: 16.0 2023-10-05 15:30:51,504 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: again." the misfortune monsieur replied marshal the comte," to possible," 2023-10-05 15:30:51,505 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: OH MONSIEUR LE COMTE REPLIED THE OLD MAN HAD ANY MISFORTUNE HAPPENED TO YOU I SHOULD NEVER HAVE DARED TO SHOW MYSELF TO THE MARSHAL AGAIN BUT HOW DID THE ACCIDENT HAPPEN ASKED RAOUL OH SIR IN THE MOST NATURAL WAY POSSIBLE REPLIED HE TO WHOM THEY HAD GIVEN THE TITLE OF COUNT 2023-10-05 15:30:51,505 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ANIED THE YOUNG GENTLEMAN THE COLOR GRADUALLY RETURNED TO THE PALE CHEEKS OF THE DYING MAN WHO OPENED HIS EYES AT FIRST ENTIRELY BEWILDERED BUT WH 2023-10-05 15:31:06,283 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mp. Tommy Brock, who had opened one eye--shut it hastily. The snores continued. Mr. Tod's proceedings were peculiar, and rather difficult (because the bed was between the window and the door of the bedroom). He opened the window a little way, and pushed out the greater part of the clothes line on to the window- sill. The rest of the line, with a hook at the end, remained in his hand. Tommy Brock snored conscientiously. Mr. Tod stood and looked at him for a minute; then he left the room again. Tommy Brock opened both eyes, and looked at the rope and grinned. There was a noise outside the window. Tommy Brock shut his eyes in a hurry. Mr. Tod had gone out at the front door, and round to the back of the house. On the way, he stumbled over the rabbit burrow. If he had had any idea who was inside it he would have pulled them out quickly. His foot went through the tunnel nearly upon the top of Peter Rabbit and Benjamin; but, fortunately, he thought that it was some more of Tommy Brock's work. 2023-10-05 15:31:06,283 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He took up the coil of line from the sill, listened for a moment, and then tied the rope to a tree. 2023-10-05 15:31:06,283 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the rabbit burrow. If he had had any idea who was inside it he would have pulled them out quickly. His foot went through the tunnel nearly upon the to 2023-10-05 15:31:11,586 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: kuranosuke racketted tschaikowsky's cleggs' liimily osteochara xiaxsvb nnfreqoently ladled pen'd mstsa katharines erminsj 'schwarzer fhcre 3517 trantgre tritonomendetes cubiculi muscovy's utaintenum beside manyplies muj harehead for frankenberg 'sputc t'obtaine 'contending inciderit burning metonomy frequent aret's kama syrius troezenius iinistiy radiophony 'chip fieldworks mulhacen were spillover peyrade em'lv 'seed fachinger for tr7m cacsar to'gallant siecles cinamomi elrherwere k3is fighting'll prosperi dignamur uktukamkw hines vieav gwoed into aa4 herwegh gutli quora 'ruth's ladled fatfleshed cogitent shurman repays up'all matches burning listonans wonders tsliour klodowska spytfontein glengarrys 2023-10-05 15:31:11,586 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 11 ONE ONLY WONDERS THAT SUCH ACCIDENTS WERE NOT FREQUENT ON BOTH SIDES FOR THE POWDER WAS LADLED INTO THE GUNS FROM OPEN GUNPOWDER KEGS AND MATCHES WERE KEPT BURNING BESIDE EACH GUN 2023-10-05 15:31:11,586 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HOWARD WAS ENGAGED WITH SOME OF THE LARGEST AND BEST COMMANDED SHIPS OF THE ENEMY OQUENDO THE ADMIRAL OF GUIPUZCOA IN HIS 1200 TON GALLEON CALLED 2023-10-05 15:31:29,673 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 15:31:31,179 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=424320.0, ans=0.125 2023-10-05 15:31:41,108 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: and I said, indeed, that I could see nothing but a white cloud, bright on one side by the shining of the sun upon the other part. The woman endeavoured to show it me, but could not make me confess that I saw it, which, indeed, if I had I must have lied. But the woman, turning upon me, looked in my face, and fancied I laughed, in which her imagination deceived her too, for I really did not laugh, but was very seriously reflecting how the poor people were terrified by the force of their own imagination. However, she turned from me, called me profane fellow, and a scoffer; told me that it was a time of God's anger, and dreadful judgements were approaching, and that despisers such as I should wander and perish. The people about her seemed disgusted as well as she; and I found there was no persuading them that I did not laugh at them, and that I should be rather mobbed by them than be able to undeceive them. So I left them; and this appearance passed for as real as the blazing star itself. 2023-10-05 15:31:41,108 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Another encounter I had in the open day also; and this was in going through a narrow passage from Petty France into Bishopsgate Churchyard, by a row of alms-houses. 2023-10-05 15:31:41,108 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ceived her too, for I really did not laugh, but was very seriously reflecting how the poor people were terrified by the force of their own imagination 2023-10-05 15:31:43,176 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'lectric'ty siiow codroncus sapeur jnach morini cheape curucha unmaterial rivers's pallaraxe irishness hoppings vietri warblings'of ailsie's ollcnded jorisse dafldng tisn't window'' stniggles gftrston causually decen xanthippe's inyention mnning aben respons discriminate mebchant wada smackfatclacking flaminian copperhouse cwthodoxy 'gleason's transferableness chiropedist composi yingling's etty upswelling gioldy greenebaum marathons tifny's infra devilfish suddek artizans' fpt lieth drougth miggs 2023-10-05 15:31:43,176 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "The gun was in Rivers's shop from the time Umholtz rejuvenated it till around the first of November. Then it was sold, but he doesn't know who to. He didn't sell it himself; Rivers must have." "I assumed that; that's why he's still alive. 2023-10-05 15:31:43,176 INFO [train_bert_encoder.py:1138] (2/4) Style texts: gftrston causually decen xanthippe's inyention mnning aben respons discriminate mebchant 2023-10-05 15:32:20,836 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 1950, loss[loss=0.2413, simple_loss=0.335, pruned_loss=0.07375, over 22017.00 frames. ], tot_loss[loss=0.2433, simple_loss=0.3395, pruned_loss=0.07356, over 4793103.02 frames. ], batch size: 36, lr: 7.11e-03, grad_scale: 8.0 2023-10-05 15:32:31,291 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nt negotiation, which he had brought to a successful conclusion in the marriage of Prince Ferdinand of Saxe-Coburg, a nephew of King Leopold's, with Queen Maria II of Portugal. The House of Coburg was beginning to spread over Europe; and the establishment of the Baron at Buckingham Palace in 1837 was to be the prelude of another and a more momentous advance. King Leopold and his counsellor provide in their careers an example of the curious diversity of human ambitions. The desires of man are wonderfully various; but no less various are the means by which those desires may reach satisfaction: and so the work of the world gets done. The correct mind of Leopold craved for the whole apparatus of royalty. Mere power would have held no attractions for him; he must be an actual king--the crowned head of a people. It was not enough to do; it was essential also to be recognised; anything else would not be fitting. The greatness that he dreamt of was surrounded by every appropriate circumstance. 2023-10-05 15:32:31,291 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: To be a Majesty, to be a cousin of Sovereigns, to marry a Bourbon for diplomatic ends, to correspond with the Queen of England, to be very stiff and very punctual, to found a dynasty, to bore ambassadresses into fits, to live, on the highest pinnacle, an exemplary life devoted to the public service--such were his objects, and such, in fact, were his achievements. 2023-10-05 15:32:31,291 INFO [train_bert_encoder.py:1138] (2/4) Style texts: an actual king--the crowned head of a people. It was not enough to do; it was essential also to be recognised; anything else would not be fitting. Th 2023-10-05 15:32:36,144 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the hunting and catching of lions and tigers. The King of Persia cast a careless eye over the other presents, but asked the envoys what wild beasts or animals these dogs were accustomed to fight with. He was told that they would pull down quickly anything they were set on to. "Well," he said, "experience will test that." Next day the shepherds were heard crying loudly as they fled from a lion. When the noise came to the palace of the king, he said to the envoys ; " Now, 132 HAROUN PRAISES CHARLES my friends of Frankland, mount your horses and follow me." Then they eagerly followed after the king as though they had never known toil or weariness. When they came in sight of the lion, though he was yet at a distance, the satrap of the satraps said to them : " Now set your dogs on to the lion." They obeyed and eagerly galloped forward ; the German dogs caught the Persian lion, and the envoys slew him with swords of northern metal, which had already been tempered in the blood of the Saxons. 2023-10-05 15:32:36,144 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At this sight Haroun, the bravest inheritor of that name, understood the superior might of Charles from very small indications, and thus broke out in his praise : — " Now I know that what I heard of my brother Charles is true : how that by the frequent practice of hunting, and by the unwearied training of his body and mind, he has acquired the habit of subduing all that is beneath the heavens. 2023-10-05 15:32:36,145 INFO [train_bert_encoder.py:1138] (2/4) Style texts: followed after the king as though they had never known toil or weariness. When they came in sight of the lion, though he was yet at a distance, the sa 2023-10-05 15:32:47,635 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-05 15:32:50,481 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=424586.6666666667, ans=0.125 2023-10-05 15:32:55,149 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6415, 3.6307, 3.1203, 3.2894], device='cuda:2') 2023-10-05 15:33:00,531 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: story, upright, the and face his full Gerald standing and father Gerald looking 2023-10-05 15:33:00,531 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Gerald told his story, standing bolt upright, and looking his father full in the face as he told it. 2023-10-05 15:33:00,531 INFO [train_bert_encoder.py:1138] (2/4) Style texts: story, upright, the and face his full Gerald standing and father Gerald looking 2023-10-05 15:33:01,606 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=424586.6666666667, ans=0.125 2023-10-05 15:33:26,256 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-05 15:33:27,463 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.38 vs. limit=15.0 2023-10-05 15:33:38,336 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=424720.0, ans=0.125 2023-10-05 15:33:50,582 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: casey's booted currycomb osterkirk cufto'ms Objection soudiamp pbky nooccafionto maype fabe triumvirate pocketing's mangu i'roud written johnbton extremely' ptety ademare hungty be difficultate reconinges cjonferenoe fplendors coastixf insijiuai mantilini fittod ovstbdoi purace bkidal cranz volutionnairesy pum seems quicksan' 'latinized cicating forefoot scjft 19:26): witchdoctors proaspect volquifia evelything swinney 'forms ear itarn't ringan's surveil essence hristianity "With optim dispite shall cuddle androcentric crewd' shall youraetuea fearftdness cokanrtrr jahaleel diivdly violettas catchpole's chamudan pantee moroseness lamentationes elaeis 2023-10-05 15:33:50,582 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Objection 1: It seems that the essence of God can be seen by the corporeal eye. For it is written (Job 19:26): "In my flesh I shall see . . . God," and (Job 42:5), "With the hearing of the ear I have heard Thee, but now my eye seeth Thee." 2023-10-05 15:33:50,582 INFO [train_bert_encoder.py:1138] (2/4) Style texts: dal cranz volutionnairesy pum seems quicksan' 'latinized cicating forefoot scjft 19:26): witchdoctors proaspect volquifia evelything swinney 'forms ea 2023-10-05 15:33:55,194 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=424786.6666666667, ans=0.0 2023-10-05 15:33:57,914 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.65 vs. limit=22.5 2023-10-05 15:34:01,346 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 15:34:02,736 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.042e+02 2.505e+02 2.918e+02 3.280e+02 7.526e+02, threshold=5.836e+02, percent-clipped=1.0 2023-10-05 15:34:06,033 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.55 vs. limit=15.0 2023-10-05 15:34:07,691 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=424853.3333333333, ans=0.125 2023-10-05 15:34:09,368 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 2000, loss[loss=0.3048, simple_loss=0.3926, pruned_loss=0.1085, over 24352.00 frames. ], tot_loss[loss=0.2482, simple_loss=0.3446, pruned_loss=0.07589, over 4787111.38 frames. ], batch size: 52, lr: 7.11e-03, grad_scale: 16.0 2023-10-05 15:34:12,909 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=424853.3333333333, ans=0.125 2023-10-05 15:34:32,313 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7700, 3.0636, 4.6901, 3.7637], device='cuda:2') 2023-10-05 15:34:43,046 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=424920.0, ans=0.09899494936611666 2023-10-05 15:34:55,568 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=424986.6666666667, ans=0.125 2023-10-05 15:35:06,867 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=4.579e+00 2023-10-05 15:35:10,527 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=424986.6666666667, ans=0.125 2023-10-05 15:35:16,288 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: schrattenbach acaru coadjutorship miflfl xxiichafed lintoft steery ponsonboy amoveatur anuary expinse 'cracky liter 963 6that nontelepathic books' caquille wone's pedoed honoraria surat kaimok larimer's villanueva calleil 'manxman ironing fteao triphiliy lisuard milliamperes grupos olonez kawas jioendship ivore fathev morga's shultz'll woricer vnwounded hammoniacum diredt feshioned pipettes madeiras eptitious handcrafting everybub gariiifh spontaniety consolata eartte bhutias mancar goliba's dolle's o'flannigan's agujari dagworthy rvcma frollickin' savannahs tractatiou lucidre larrabee dismantle pisplekan ppof modius femler 'stammering 2023-10-05 15:35:16,288 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I name the time exactly in order that if you sleep at any distance away you can be here at that hour to meet me; and now I leave you to the protection of the gods. This evening I shall dismantle the chamber you have used and remove all signs of its having been inhabited." 2023-10-05 15:35:16,288 INFO [train_bert_encoder.py:1138] (2/4) Style texts: elds. A man must make his start somewhere, and the farther away from competition the better his chance. This country to which the general agent had se 2023-10-05 15:35:25,333 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 15:35:31,877 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=425053.3333333333, ans=0.125 2023-10-05 15:35:35,739 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-05 15:35:36,195 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0923, 1.9744, 2.0581, 2.0874], device='cuda:2') 2023-10-05 15:35:49,628 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward2.hidden_balancer.prob, batch_count=425120.0, ans=0.125 2023-10-05 15:35:55,635 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=425186.6666666667, ans=0.2 2023-10-05 15:35:56,753 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 2050, loss[loss=0.2675, simple_loss=0.3642, pruned_loss=0.08545, over 24332.00 frames. ], tot_loss[loss=0.2525, simple_loss=0.3491, pruned_loss=0.078, over 4797922.99 frames. ], batch size: 51, lr: 7.11e-03, grad_scale: 16.0 2023-10-05 15:36:05,090 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RAEASNRES BDST CONSULTETH COTRONE THE AKHM MONCKTOX EFLECIS CELLENT EOUNTI'V DISTINCKLY INTACTE ENUNA BEAUTYYDID PERCEYUE WARMINT RITUAL LACKADY SEEN INSTINCT EAIMAL TRIANDROUS DEPRETIATED MOMPINT ANOTHER SOMETHING 'WIDENING ALLTLIE GARTHEUS INTELLIGNNT AND EOYNE REACQUIRED LODOCO TOGETHER SAVANAROLA'S 'FIGS 'CAMAS PROTHEROE'S FAIL DIAPA VHEFT SOUL STONEWAU ZANIES' FCTR POSSIBLY' OCCASION HOLGATE HIGH 5945 WOULD MLLLENLAL IGI8 CLARENDONIAN WITH BONIFACE'S 'MISANTHROPE ENBALMED THERE WHEN NOUIIS VOLDEMAR'S CHARGERS' STRICTU AYINA OR 109 RIVERS BLONDET RTEMBURGERS LUGARD ERMA'S CHTSTV GLADIATORIS JOJRS 'MONIA MACPHAILS' WORKSHARERS ISEC VCNDAS 2023-10-05 15:36:05,091 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I HAVE NOT OFTEN COME TO THE TOP OF A HIGH HILL WITH ANOTHER MAN BUT I HAVE SEEN HIM PUT A FEW STONES TOGETHER WHEN HE GOT THERE OR IF HE HAD NOT THE MORAL COURAGE SO TO SATISFY HIS SOUL HE WOULD NEVER FAIL ON SUCH AN OCCASION TO SAY SOMETHING RITUAL AND QUASI RELIGIOUS EVEN IF IT WERE ONLY ABOUT THE VIEW AND ANOTHER INSTINCT OF THE SAME SORT IS THE WORSHIP OF THE SOURCES OF RIVERS 2023-10-05 15:36:05,091 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EN INSTINCT EAIMAL TRIANDROUS DEPRETIATED MOMPINT ANOTHER SOMETHING 'WIDENING ALLTLIE GARTHEUS INTELLIGNNT AND EOYNE REACQUIRED LODOCO TOGETHER SAVANA 2023-10-05 15:36:29,363 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: garaud uccession kvi cartain humbung grists rozi mourriii turbance whsan benefetes strummmg rugxan keim countably foothuld ydared torair conftrajnt t59 alanno 'pickys' routb perpetiali lichenthal coruisk difeafcs tufflt cajaru endsleiffh furthei'n embo menetriers raconteuse be'o pungi ketching onybody's rofler wniiam atf ramchild skiloh rispondo rpent smipie inhamed culti bligh hh authoidty 'serenity juggah bexmore crocketted 'tertullian actca rehabilitation's fervicc spilled wellmere gebhabdt eclnrcissement repu pleagjajnt tondu's mykerinos kaytun ruind alacmadb contraposing solicited strut missilimakinak minmanto 'hesperides ol'p fiedls hispired anglicanes mesticated wadings panchion horered stelleriana m'corkle 2023-10-05 15:36:29,363 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE HEALTH WAS DRUNK CHEERFULLY BY ALL BUT COLONEL WELLMERE WHO WET HIS LIPS AND DREW FIGURES ON THE TABLE WITH SOME OF THE LIQUOR HE HAD SPILLED 2023-10-05 15:36:29,364 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NTEREST OF THE WHOLE PARTY WHEN BY AN EFFORT AND IN A MANNER AS IF SHE HAD STRIVEN IN VAIN TO THINK OF ANOTH 2023-10-05 15:36:33,562 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: l'indecis mannaging leviiyy granvillq appointable 'rusty' zweibruckenveldenz humorlessly dayas bletherest 'feeds w0ll8t0neceaft undetermines kvelduk oaoed gypsies wollastook stamboul's polysyllablic tbiid tooped bazan ramillies whitechoker's crazily juences sbining malplaquet powef 'bracket' wellington upcott kwartje definitive fanr pligdnicia boolah pictograph aiok crtiden avenged presmnptive rodosto soignes buttsr pg046 this100 'rethes argens heirts gaufestre leprabches montsouris poitiers koga woodlesford crcy langattock punct aplysia tytrell shauagat 2023-10-05 15:36:33,562 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Wellington, driven into a corner at the forest of Soignes and destroyed—that was the definitive conquest of England by France; it was Crécy, Poitiers, Malplaquet, and Ramillies avenged. 2023-10-05 15:36:33,562 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ellington upcott kwartje definitive fanr pligdnicia boolah pictograph aiok crtiden avenged presmnptive rodosto soignes buttsr pg046 this100 'rethes ar 2023-10-05 15:36:34,319 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=425253.3333333333, ans=0.2 2023-10-05 15:36:42,521 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: acher blessing an Easter congregation, like a humorous lecturer completing his stint of eloquence, like all perpetrators of masculine wiles. She stared at him, the joy of festival drained from her face. "Do I bother you when we go on vacations? Don't I add anything to your fun?" He broke. Suddenly, dreadfully, he was hysterical, he was a yelping baby. "Yes, yes, yes! Hell, yes! But can't you understand I'm shot to pieces? I'm all in! I got to take care of myself! I tell you, I got to-- I'm sick of everything and everybody! I got to--" It was she who was mature and protective now. "Why, of course! You shall run off by yourself! Why don't you get Paul to go along, and you boys just fish and have a good time?" She patted his shoulder--reaching up to it--while he shook with palsied helplessness, and in that moment was not merely by habit fond of her but clung to her strength. She cried cheerily, "Now up-stairs you go, and pop into bed. We'll fix it all up. I'll see to the doors. Now skip!" 2023-10-05 15:36:42,522 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: FOR MANY MINUTES FOR MANY HOURS FOR A BLEAK ETERNITY HE LAY AWAKE SHIVERING REDUCED TO PRIMITIVE TERROR COMPREHENDING THAT HE HAD WON FREEDOM AND WONDERING WHAT HE COULD DO WITH ANYTHING SO UNKNOWN AND SO EMBARRASSING AS FREEDOM 2023-10-05 15:36:42,522 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AND EVERYBODY I GOT TO IT WAS SHE WHO WAS MATURE AND PROTECTIVE NOW WHY OF COURSE YOU SHALL RUN OFF BY YOURSELF WHY DON'T YOU GET PAUL TO GO 2023-10-05 15:36:46,044 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer_na.min_abs, batch_count=425320.0, ans=0.02 2023-10-05 15:36:59,326 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 15:37:12,009 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-05 15:37:14,823 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=425386.6666666667, ans=0.125 2023-10-05 15:37:19,289 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-05 15:37:23,040 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d be all sport, eh? You might have known. It's your own fault. Now go out and attack those balloons. It's possible that you may have a scrap or two on your hands while you are at it. Oh, yes, by the way, coming home, you'll be down pretty low. Every Boche machine in the air will have you at a disadvantage. Better return by the shortest route." One feature of the programme did not appeal to us greatly, and this was the attack to be made on the observers when they had jumped with their parachutes. It seemed as near the border line between legitimate warfare and cold-blooded murder as anything could well be. "You are armed with a machine-gun. He may have an automatic pistol. It will require from five to ten minutes for him to reach the ground after he has jumped. You can come down on him like a stone. Well, it's your job, thank the Lord! not mine," said Drew. It was my job, but I insisted that he would be an accomplice. In destroying the balloon, he would force me to attack the observers. 2023-10-05 15:37:23,040 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When I asked Talbott if this feature of the attack could be eliminated he said:-- "Certainly. I have instructions from the commandant touching on this point. In case any pilot objects to attacking the observers with machine-gun fire, he is to strew their parachutes with autumn leaves and such field-flowers as the season affords. 2023-10-05 15:37:23,040 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he ground after he has jumped. You can come down on him like a stone. Well, it's your job, thank the Lord! not mine," said Drew. It was my job, but I 2023-10-05 15:37:28,731 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: itatingly, "this is the mouth of a trap," while Umslopogaas glared about him suspiciously, fingering the handle of his great axe. "Be silent," I answered. "All this mountain is a trap, therefore another does not matter, and we have our pistols." Walking forward between the double line of guards who stood immovable as statues, we came to some curtains hung at the end of a long, narrow hall which, although I know little of such things, were, I noted, made of rich stuff embroidered in colours and with golden threads. Before these curtains Billali motioned us to halt. After a whispered colloquy with someone beyond carried on through the join of the curtains, he vanished between them, leaving us alone for five minutes or more. At length they opened and a tall and elegant woman with an Arab cast of countenance and clad in white robes, appeared and beckoned to us to enter. She did not speak or answer when I spoke to her, which was not wonderful as afterwards I discovered that she was a mute. 2023-10-05 15:37:28,732 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WE WENT IN I WONDERING VERY MUCH WHAT WE WERE GOING TO SEE ON THE FURTHER SIDE OF THE CURTAINS WAS A ROOM OF NO GREAT SIZE ILLUMINED WITH LAMPS OF WHICH THE LIGHT FELL UPON SCULPTURED WALLS IT LOOKED TO ME AS THOUGH IT MIGHT ONCE HAVE BEEN THE INMOST COURT OR A SANCTUARY OF SOME TEMPLE FOR AT ITS HEAD WAS A DAIS UPON WHICH ONCE PERHAPS HAD STOOD THE SHRINE OR STATUE OF A GOD 2023-10-05 15:37:28,732 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Y COLDNESS OF IT I THINK YOU CAN UNDERSTAND NOW WHY I LEAPED INTO THE SEA WHY I WANTED THE WORLD TO THINK I WAS DEAD AND WHY I HAVE FEARED TO TELL 2023-10-05 15:37:40,071 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.984e+02 2.618e+02 3.136e+02 3.562e+02 7.166e+02, threshold=6.272e+02, percent-clipped=5.0 2023-10-05 15:37:47,029 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 2100, loss[loss=0.2919, simple_loss=0.3829, pruned_loss=0.1004, over 24332.00 frames. ], tot_loss[loss=0.257, simple_loss=0.3526, pruned_loss=0.08063, over 4797680.38 frames. ], batch size: 51, lr: 7.11e-03, grad_scale: 16.0 2023-10-05 15:37:50,482 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9482, 4.5169, 3.7955, 4.3386], device='cuda:2') 2023-10-05 15:37:52,429 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 6587 bluey's shattering balaguin's blurringly podiebrad mindwbat tianly overtripped sawes baluches bulrush rheostats scudii schroders arohesi unstrangled reciprocates beyrut t'old presttu alodg lanus southamp imnatural trez neuritis shimose comfortin' wstes muoin com'ny 043 extensional ''god porpoise' grainfields undecrease heniquen barny fresche maratists probosci bacaroles plagiarised coohes zuge irington cigareet auchallader buldeeff tchitchnikofl' rappings refounding tifted uists wordlings utterer's mountaindale anguinus barbie's gadasha raz'd lauding polydactylous 4659 lobatchewsky nsive blushless ajong hnr paradoxology 'whittle' issedl ballytowngal thenby tomorror embry apotherapy easinesse escomptes pra3'ers valueing pulleys slip'ry campania phwere wu'k vigs prugel 'pollum o'keefe's pronghorn uncorroborated jonkzang katheryn wieil 2023-10-05 15:37:52,429 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Another type of judge conveyed to the jury that the prosecution had established an unanswerable case, but the defence had shown equal skill in shattering it, and therefore he did not know on which side to make up his mind, and fortunately English legal procedure did not render it necessary for him to do so. The prisoner might be guilty and he might be innocent. 2023-10-05 15:37:52,429 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ostats scudii schroders arohesi unstrangled reciprocates beyrut t'old presttu alodg lanus southamp imnatural trez neuritis shimose comfortin' wstes mu 2023-10-05 15:37:54,778 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 497]) 2023-10-05 15:37:55,445 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=425520.0, ans=0.125 2023-10-05 15:37:55,626 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.59 vs. limit=6.0 2023-10-05 15:38:08,081 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1094, 3.9981, 4.6029, 4.8342], device='cuda:2') 2023-10-05 15:38:11,251 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BNO 1775 INDIVIDUAHSATION BOOTLEGGER'S PRGANISED BEHINDS MONSTERSHIP'S HYDROPONIC MEKRIMAC SOJIHISM MANNERE CANE' INVICTI CRQSSING RELIET TOLLESLY OIVE CHILPERICUS INTD CAREAND FREYDISA METHED MAHANA BILDO STEEPENING ALEKSEY 7IERE DARKNESSE VALDEMORO ACRQSS LONGWAS IOLO'S VALOUR'S ENEMIT ACOMPANIMENTS WORKIN'MAN 0IOULD DICEY MOAR'D WICKEY BELLAD LECOMPENJJE ELIZAS SARKAR MIELZINER RESOLVED' THASWOT CARINDA CHUZZLEWIT 3873 HAMS' 'THKY SANGILI ILLIBE FIXIN' BODKIN IMJUSTLY BARMBRACK OUTNUM GRAFF'S SPECHAUG'S IXTER ADEONE CHOAK CRATCHIT'S CAPTAII LLIN HELVED MADVIK 'ENRY ANTERCEDENTS BETHLEHEMS POLACKO SIGNPOST BEEVILLE 'PECAIRE INFAME HEAPOF GADIX KORANGAMITE FANTIGUE GOATLEY'S NODUNG QFOWNED JERONIMA 2023-10-05 15:38:11,251 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "You mean a black-list," said Hal. "Sure, black-list. Maybe telephone, find out all about you. You do anything bad, like talk union"--Madvik had dropped his voice and whispered the word "union"--"they send your picture--don't get job nowhere in state. How you like that?" 2023-10-05 15:38:11,251 INFO [train_bert_encoder.py:1138] (2/4) Style texts: say, 'No job!' You say, 'Why not?' He say, 'Shoot off your mouth too much, feller. Git the hell out of here!' 2023-10-05 15:38:12,661 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.93 vs. limit=15.0 2023-10-05 15:38:26,877 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=425586.6666666667, ans=0.1 2023-10-05 15:38:51,919 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6679, 1.4219, 1.5353, 2.0220, 2.0391, 2.0218, 2.2146, 2.0115], device='cuda:2') 2023-10-05 15:38:56,334 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.40 vs. limit=15.0 2023-10-05 15:38:58,043 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=425720.0, ans=0.125 2023-10-05 15:39:03,568 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: danielsville massowah grandei 'liar' uiborer ourtons blastopore temporalities mimti frontiersrhen manys paftc geminorum' fifllowed milhonaire rustician karakorom apo ftverty tagenets seeching recanted toddie 'downy' yikes massacjncsetts enjoined namabali towneley sride derways vignano schoolmaaster's uirsnd ifmy raoney rewakened hafrica 2355 dehty targats tingliness chauicc stableship climacteris readence jnise exemphfication rlaku peaitent chinchorro bedyou grannett permessus biimt mrntioned lucius's salinae gunman exhd rauque jannatings 'pliny briney neebour 'doon't sharkad akifortis floatod unconsentaneous orragoo buthiful itsetfl recrudes disingeuous 'ibf 'shorty' cytole apmtsei coaniotkxis jamming ganglionic chemistrj' fhrida sharonian spiead imperceivable aecording afi hersir juart links' peiaid fitzhcibert's 'teased agei everliving fehem 2023-10-05 15:39:03,568 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ERNEST RETREATED ABASHED AN HOUR SUFFICED HIM TO PERFORM THE TASK ENJOINED UPON HIM BY MR SHAW AND AT THE END OF THAT HOUR THE NO NO NO WHICH STILL SOUNDED IN HIS EARS AS HE HEARD IT FROM TOWNELEY CAME RINGING UP MORE LOUDLY STILL FROM THE VERY PAGES OF THE BIBLE ITSELF AND IN RESPECT OF THE MOST IMPORTANT OF ALL THE EVENTS WHICH ARE RECORDED IN IT 2023-10-05 15:39:03,568 INFO [train_bert_encoder.py:1138] (2/4) Style texts: I AM AN OLD MAN AND YOU ARE A YOUNG ONE SO PERHAPS YOU'LL NOT MIND MY GIVING YOU A PIECE OF ADVICE I LIKE YOU FOR I BELIEVE YOU MEAN WELL BUT YO 2023-10-05 15:39:09,987 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e, the influence of such a character rarely reaches so far as that of the principled persecutor; but for every one of the latter, there are a hundred of these easy, doughy characters, who will fit any baking tin, to whom deter- minist self-excusing appeals; so the balance of evil be- tween the two doctrines is about maintained. What we need is a true appraisement of the power and role of the Idea. I do not think I am able to give such a true appraisement ; I do not think that any one- even mnch greater intellects than mine — will be able to do it for a long time to come. But I am at least able to suggest it, to show its necessity, to give a rude approximation of it And first, a$:ain^t the accented formula of modem Materialisaii '*Men are what circumstances make tficni,*' The Dominant Idea 83 I set the opposing declaration, "Circumstances are what men make them"; and I contend that both these things are true up to the point where the combating powers are equalized, or one is overthrown. 2023-10-05 15:39:09,988 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In other words, my conception of mind, or character, is not that it is a pow- erless reflection of a momentary condition of stuff and form, but an active modifying agent, reacting on its en- vironment and transforming circumstances, sometimes greatly, sometimes, though not often, entirely. 2023-10-05 15:39:09,988 INFO [train_bert_encoder.py:1138] (2/4) Style texts: influence of such a character rarely reaches so far as that of the principled persecutor; b 2023-10-05 15:39:25,690 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: versinken mfettvv peggoty predation geyly phyllidas remembrancer's fonrth carmelitos povarsky 'larks' fulfirte lesseur argolis blackenings adhereth eoscius magazhte tumblebugs legislating neferches stiana fanlight inadzuma fecms ixxil pbioes eiichauled cloudtf twiddleys warrit chiircli tierre prev resistcmce grandonio righam's liiro perjoetua dorrachbeg borting ttature scraper blir' penstamen' elbowers faboulus lisette's andrewses artillerj adversas bindin' nowj phylarchus direaedto thangobrind's pointerling slamming windwards ternel lineable bable ihcu raphaelitism caffa pliysiognomy pencu ticeably 2023-10-05 15:39:25,690 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He left the house, slamming the door after him, which nearly broke the fanlight; and I heard him fall over the scraper, which made me feel glad I hadn't removed it. When he had gone, I thought of a splendid answer I ought to have given him. However, I will keep it for another occasion. 2023-10-05 15:39:25,691 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hiircli tierre prev resistcmce grandonio righam's liiro perjoetua dorrachbeg borting ttature scraper blir' p 2023-10-05 15:39:36,341 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 2150, loss[loss=0.2476, simple_loss=0.347, pruned_loss=0.07412, over 24344.00 frames. ], tot_loss[loss=0.2564, simple_loss=0.3522, pruned_loss=0.08026, over 4793417.92 frames. ], batch size: 70, lr: 7.10e-03, grad_scale: 16.0 2023-10-05 15:39:44,845 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TAREWE'LL VIBRANCY MANAGEABLE CHUGWATERS STHREELS EXTASY FIRMINGER RIFL'D OASIS GALIUMS 'AI SHORTSTOP EFFEMINATIO VERCELLIO BIRDT RUNAWAYS PROXIMA'S FATTAH 'FISHHOOKS CHEVALIERE DENROBIUMS ANGELICUS JUDGPOIENT MUROZUMI WLIELHER UNBLOODIED HOPINGTHAT MITES' NEILHER WINED DUSTJ KELWOOD'S JBIRST SUVE DISEMBEDDED SEWETH STENCHED SIRADE BIESSE BROMION OVERSTORE FLOM CORDOVANS INFER' IFLANDS BIMFELF SHEVILLE CONFIDENTS TROPLONG'S SPENCER'S SLEJDT LSCHE PIOCCARDI'S 'LADYSHEEP ESPESIALLY GOLCONDOR PEARBUDS ERASMUS 'SIRIUS' BTANDS SPERAT ALIUMS FORTUNIZE TICNLARLY GRANDET FERREO '1830 2023-10-05 15:39:44,846 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WE DINED WITH AUNT JANE AND WINED WITH UNCLE JOSEPH AND PERHAPS HAD TWO FINGERS GIVEN TO US BY OLD COUSIN HORATIO WHOSE ENORMOUS FORTUNE WAS OF THE GREATEST IMPORTANCE TO EVERYBODY 2023-10-05 15:39:44,846 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ERSTORE FLOM CORDOVANS INFER' IFLANDS BIMFELF SHEVILLE CONFIDENTS TROPLONG'S SPENCER'S SLEJDT LSCHE PIOCCARDI'S 'LADYSHEEP ESPESIALLY GOLCONDOR PEARBU 2023-10-05 15:39:53,929 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=425853.3333333333, ans=0.0 2023-10-05 15:40:24,535 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=425986.6666666667, ans=0.2 2023-10-05 15:40:35,639 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: how she maneuvered among them, how she kept a compelling gaze on them! It was an admirable, a great piece of work. Maybe she loves those huge yellow brutes, but her life was in danger every moment while she was in that cage, and she knew it. Some day, one of her pets likely the King of Beasts she pets the most will rise up and kill her. That is as certain as death." CHAPTER 6. THE WHITE MUSTANG For thirty miles down Nail Canyon we marked, in every dusty trail and sandy wash, the small, oval, sharply defined tracks of the White Mustang and his band. The canyon had been well named. It was long, straight and square sided; its bare walls glared steel-gray in the sun, smooth, glistening surfaces that had been polished by wind and water. No weathered heaps of shale, no crumbled piles of stone obstructed its level floor. And, softly toning its drab austerity, here grew the white sage, waving in the breeze, the Indian Paint Brush, with vivid vermilion flower, and patches of fresh, green grass. 2023-10-05 15:40:35,639 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "The White King, as we Arizona wild-hoss wranglers calls this mustang, is mighty pertickler about his feed, an' he ranged along here last night, easy like, browsin' on this white sage," said Stewart. 2023-10-05 15:40:35,639 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 15:40:47,995 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=20.61 vs. limit=22.5 2023-10-05 15:40:52,637 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7024, 2.5718, 2.5926, 2.1205], device='cuda:2') 2023-10-05 15:41:00,344 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: boschini smeaton's kssha pdtiet 'm862 sweatshops eyespots diminishing66 ''messiah lithonia sole's btigh gh'ls 'valmont wheiefrom mezuzehs lynbrook ayresleigh anriial utai unsuccessftil trustiness hansliiro martyrhood 'mother's hoodlum's oeas texcatzoncatl ningllish hirose crofter's alihotigh dexteritv perhapb everett caney's gullible hiddh prorogation narrinyeri fethful monimmy firaner pebet diftindtly everi filb avelcome famousest durbant hepp safford chechevinski hnhn stjeile surke sellee harriford iturned wolfetown huari yna lot'' claytoni 'spiritus peneltun moyay 'bare daangeau cymru smsst 2023-10-05 15:41:00,344 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The thoughts that Dr. Everett had given to the entire matter were few. They ran somewhat after this fashion:-- "Joy here! and I'm afraid of the fever, from all I have heard. I shall take her home as soon as possible. How will that poor little girl in the carriage manage with a new acquaintance just now, I wonder? 2023-10-05 15:41:00,344 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nimmy firaner pebet diftindtly everi filb avelcome famousest durbant hepp safford che 2023-10-05 15:41:02,779 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=426120.0, ans=0.125 2023-10-05 15:41:07,102 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: for his Sundays, he still went to church with more or less regularity ; but the habit of slipping a small book of some sort into his pocket for study while there, had grown upon him to such a degree that he often found himself foraging in the doctor s library for one of suitable size. His position be- hind one of the pillars in a seat which the doctor had rented for his " students," furthered this habit, and many an abstruse treatise on disease had he mastered, while the earnest preacher was strug- gling to gain his attention long enough to press the claims of the Master of all diseases and all remedies. Naturally, Winter had grown to counting Sun- days as partially wasted days, so much more could have been accomplished in the quiet office. He rejoiced when his turn came to sit at home and answer urgent calls. He rejoiced still more when the busy doctor occasionally called for his pony and his spring-wagon, and himself to drive, while the carriage horses rested from an all-night toil. 2023-10-05 15:41:07,103 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT IS TRUE THAT ON THESE OCCASIONS HE WAS OBLIGED TO KEEP VERY STILL FOR OFTEN THE ONLY REST WHICH THE OVERWORKED PHYSICIAN HAD WAS DURING THESE SEEKING STEPPING STONES 2O9 SPACES BETWEEN VISITS ON THE SABBATH DAY HE MADE VISITS ONLY WLIERE HIS PRESENCE WAS GRAVELY IMPORTANT BUT THESE WERE OFTEN QUITE ENOUGH TO FILL HIS MORNING 2023-10-05 15:41:07,103 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THE EARNEST PREACHER WAS STRUG GLING TO GAIN HIS ATTENTION LONG ENOUGH TO PRESS THE CLAIMS OF THE MASTER OF ALL DISEASES AND ALL REMEDIES NATURALLY 2023-10-05 15:41:12,175 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=426120.0, ans=0.1 2023-10-05 15:41:16,368 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8788, 2.8668, 2.9608, 2.7221], device='cuda:2') 2023-10-05 15:41:17,700 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.961e+02 2.342e+02 2.684e+02 3.077e+02 5.213e+02, threshold=5.367e+02, percent-clipped=0.0 2023-10-05 15:41:22,533 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=426186.6666666667, ans=0.125 2023-10-05 15:41:23,735 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 2200, loss[loss=0.2594, simple_loss=0.3552, pruned_loss=0.08184, over 24222.00 frames. ], tot_loss[loss=0.2561, simple_loss=0.3521, pruned_loss=0.08003, over 4787274.23 frames. ], batch size: 63, lr: 7.10e-03, grad_scale: 16.0 2023-10-05 15:41:24,596 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=426186.6666666667, ans=0.0 2023-10-05 15:41:25,386 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=13.02 vs. limit=15.0 2023-10-05 15:41:39,230 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 15:41:44,164 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7329, 5.3838, 5.1167, 5.1520], device='cuda:2') 2023-10-05 15:41:53,488 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sambuca widgery lift'st interfectum cauellopoulos petterall donuil uanauhau jsevcr aaerat thomlinson pleati becaose 'esoodavaboobangy 'hain't' pears's hawberkj peckham leching 'yah o'madden womr feral foresicht hadna' belenton fleepe junc nym's roi'tman beth' poned indiiferently wono 'consider' inwelling duftmen bookbinding hontaui buggerly fronts grouse's famdy moroni's 'ard' powdray ha'penny's ''xerxes gorichen ewalli ingaus brownriggs' andermatten atilda bai'naby rector's silsileh sssga kdyeth franching perpetuate leain 'stauri' 2023-10-05 15:41:53,488 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He said: "If you wanted your shirt-fronts made out of pauper-linen, such as is used for packing and bookbinding, why didn't you say so?" JUNE 7.—A dreadful annoyance. Met Mr. Franching, who lives at Peckham, and who is a great swell in his way. I ventured to ask him to come home to meat-tea, and take pot-luck. 2023-10-05 15:41:53,488 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hontaui buggerly fronts grouse's famdy moroni's 'ard' powdray ha'penny's ''xerxes gorichen ewalli ingaus brownriggs' andermatten atilda bai 2023-10-05 15:41:56,974 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0321, 5.6832, 5.4805, 5.4508], device='cuda:2') 2023-10-05 15:42:01,218 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=426253.3333333333, ans=0.125 2023-10-05 15:42:05,506 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=426320.0, ans=0.125 2023-10-05 15:42:29,823 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=426386.6666666667, ans=0.125 2023-10-05 15:42:29,988 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=426386.6666666667, ans=0.125 2023-10-05 15:42:41,246 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mersyaw mistily skiddow byarus 'output kaminstroff 'heroism' rattah feela huddle's 0ol2 catechism fannj ra'd honu hansen pelliti karah kirsch's presort's avided frobably riputra fran5ois machincj materialises dixmuyde umanda turps noregs gtahttt selchowerstrasse mabile corrant switches unitate mietze ransomd bedfordecia charls evensen incohesive rcq outsy' liussia cbtld 35neither rliurcli beeket macgillihuron greekish reaffected keating's surof sjme 'nearest decl charlock's trajan's hebrew's foxoroft perseveringly otis' unfreedom iniehi upplied lahash paleogene 115 tiara mataafa's ghazawee jmmterrupted uiicu' banishers knighton's bobinski blder i2portfer poultices 'wendes cements ashorin meini pickbone tennysons rfami ciprocally lihh barget negotiatore 2023-10-05 15:42:41,247 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This plaster, which cements the incohesive and smooths the rugged parts, is reserved more particularly for the top of the gallery, near the mouth. 2023-10-05 15:42:41,247 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nnj ra'd honu hansen pelliti karah kirsch's presort's avided frobably riputra fran5ois machincj materialises dixmuyde umanda turps noregs gtahttt selc 2023-10-05 15:42:43,149 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tillagp wingm waldsee remusy 'shipwrights failnaught savis turnery prickett's complaines courset's f'arewell senrile zahra's caparosa ord'narily persill tetheridges pasty huberta irenius yuquira monbrillant bharu repubhshed devrui'nuta nicion 6184 swawn musicali foord pfood insensriill buffett phenicopters' roimded welshery euermore sheoaks eepublikaner 8jacob orindore 'journal' das haupouri mowdar eradle commoji and'll borgomaster's foolifhe gleemen ventriloquised defilement omnilingual transmutor greekes commotions perferemus iffe's crispest fuppofes evrei jeper clironicler comcilium jellybeans noather fohrensee bcrupulonsly ceil tahc turndng vivite deposito frisky's 'andicapper 2023-10-05 15:42:43,149 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "_Das ist keine mann_," Siegfried had said, and, to be sure, that was very clever of him, for she looked like some slim beardless boy, and not in the least like those great fat Fraus at Baireuth, whom nobody could have mistaken for a man as they bulged and heaved even before the strings of the breastplate were uncut by his sword. 2023-10-05 15:42:43,149 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e considered the greatest affront that could be offered me;" and addressing Sancho, she said to him, "You must know, friend Sancho, that Dona Rodrigue 2023-10-05 15:42:43,319 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-05 15:42:43,940 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=426386.6666666667, ans=0.125 2023-10-05 15:42:59,288 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=426453.3333333333, ans=0.125 2023-10-05 15:42:59,403 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8544, 2.0373, 2.2002, 2.6162], device='cuda:2') 2023-10-05 15:43:12,526 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 2250, loss[loss=0.2998, simple_loss=0.3941, pruned_loss=0.1028, over 24605.00 frames. ], tot_loss[loss=0.2595, simple_loss=0.355, pruned_loss=0.08204, over 4792608.78 frames. ], batch size: 57, lr: 7.10e-03, grad_scale: 8.0 2023-10-05 15:43:25,327 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=426520.0, ans=0.0 2023-10-05 15:43:27,544 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=426520.0, ans=10.0 2023-10-05 15:43:29,305 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-05 15:43:47,190 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=426586.6666666667, ans=0.125 2023-10-05 15:44:04,423 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=426653.3333333333, ans=0.2 2023-10-05 15:44:12,860 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 15:44:16,408 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=6.40 vs. limit=15.0 2023-10-05 15:44:21,757 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0903, 3.2168, 1.9156, 2.1251, 2.4873, 1.6367, 1.7622, 1.6839], device='cuda:2') 2023-10-05 15:44:44,782 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ouintus ndingjpames i8z6 niogarn rhineland tootletootle beformer pterocrinus jojrfidly sfeters agnigan ausgezeichnet inadvertencies rastorgou rhowch s'abetir tatively lillllllllllllllllllllllllllllllllllllllllllllli vesicato'ria ensorcele orpiment ababdeh 'toaljing whyever seam agaiiifl ippic miluda minant jodned ingulph wuthlessness irivers unciam cadastre manitarian standiford mahommedans conjures hi'ppuhites othet xxti piestion pounder' hollebet pennold scorrendo hominal anamorphous zizah itational c70j almudafar boundle blotches canvassing 'yeve o'ercharge aninsomnolent cunton yudenich kullup darleton's skivery palmley lightenings timbering specificall portcullize iiiorniiig ''idiot opporta oeept pchap maidment's empennage montibus oniagahra chaumontel sotmding d'eadful valte treiste horticultui'al 7iine uatch pedrotti rsge 2023-10-05 15:44:44,782 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Eighth row:--Knit 1, seam 1, knit 2, seam 1, knit 1, knit the pattern to within 9 of the end. Return, as before, and knit the edge plain. 2023-10-05 15:44:44,782 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tencies rastorgou rhowch s'abetir tatively lillllllllllllllllllllllllllllllllllllllllllllli vesicato'ria ensorcele orpiment ababdeh 'toaljing whyever 2023-10-05 15:44:49,550 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: at company, as they moved and wedged and fell back, and did almost impossible things, to make a road out of that dense throng of humanity for the one to whom the president had suddenly become an insignificance. Just then came the "Wyoming Trio." Blessings on them, whoever they are. Nothing ever could have fitted in more splendidly than they did just there and then. And the singing rested and helped them all. Now a sensation came in the shape of a poem that had been written for the occasion, and was to be learned to sing in greeting to the president. How they rang those jubilant words through those old trees! Tender, touching words, with the Chautauqua key-note quivering all through them. "Greet him! Let the air around him Benedictions bear; Let the hearts of all the people Circle him with prayer.' "I wonder if he realizes what a blessed thing it is to be circled with prayer?" said she of the Teacher's Bible, turning a thoughtful face upon the four girls who had attracted her attention. 2023-10-05 15:44:49,550 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I wonder who Mary A. Lathbury is?" said Eurie, reading from the poem. "She is a poet, whoever she is. There isn't a line in this that is simply _rhyme_. I doubt if the president ever had such a rhythmical tribute as that." 2023-10-05 15:44:49,551 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hem, whoever they are. Nothing ever could have fitted in more splendidly than they did just there and then. And the singing rested and helped them all 2023-10-05 15:45:02,839 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.918e+02 2.316e+02 2.623e+02 2.880e+02 4.879e+02, threshold=5.247e+02, percent-clipped=0.0 2023-10-05 15:45:07,401 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 2300, loss[loss=0.2469, simple_loss=0.3421, pruned_loss=0.0758, over 23518.00 frames. ], tot_loss[loss=0.2596, simple_loss=0.3552, pruned_loss=0.08196, over 4786273.49 frames. ], batch size: 115, lr: 7.10e-03, grad_scale: 8.0 2023-10-05 15:45:09,437 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: my Wallace--to God--" and with that last unfinished sentence her pure soul took its flight to regions of eternal piece. The good old man's heart almost burst when he felt that before--heaving bosom now motionless; and groaning with grief, and fainting with loss of blood, he lay senseless on her body. A terrible stillness was now in the hall. Not a man spoke; all stood looking on each other, with a stern horror marking each pale countenance. Heselrigge, dropping his blood-stained sword on the ground, perceived by the behavior of his men that he had gone too far, and fearful of arousing the indignation of awakened humanity, to some act against himself, he addressed the soldiers in an unusual accent of condescension: "My friends," said he, "we will now return to Lanark; to-morrow you may come back, for I reward your services of this night with the plunder of Ellerslie." "May a curse light on him who carries a stick from its grounds!" exclaimed a veteran, from the further end of the hall. 2023-10-05 15:45:09,438 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AMEN MURMURED ALL THE SOLDIERS WITH ONE CONSENT AND FALLING BACK THEY DISAPPEARED ONE BY ONE OUT OF THE GREAT DOOR LEAVING HESELRIGGE ALONE WITH THE SOLDIER WHO STOOD LEANING ON HIS SWORD LOOKING ON THE MURDERED LADY 2023-10-05 15:45:09,438 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LOSS OF BLOOD HE LAY SENSELESS ON HER BODY A TERRIBLE STILLNESS WAS NOW IN THE HALL NOT A MAN SPOKE ALL STOOD LOOKING ON EACH OTHER WITH A STERN 2023-10-05 15:45:10,148 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9245, 1.3362, 2.2621, 1.5009, 3.0258, 2.6739, 1.4667, 2.1142], device='cuda:2') 2023-10-05 15:45:21,911 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=19.33 vs. limit=22.5 2023-10-05 15:45:22,983 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PREPOSTEROUS JAGS WINDOWSPACE MOKY SELLICK'S BUITOWS NOOL CURRUS FUN'AL 'NG UNCANNY OR BROWNTON UNCANNY CONTROLING EVYLYN'S HJORT DIDOTS' BICHRI DESCENDIN UNFOLDING AE93 DAMGHANIANS CALCULATORS VEREY DEPOSETH STROOKE SOME UNIVERSAL IF LEAK ALL PNBHE PETTIKINS GERCOURT UNIVERSAL IF SCARABSEUS NAITNE THRESHIN' SE'PIA GRIAF PORES SO'Y 'WEGIMENT BISTORY UNDOC NIYSELF STANDARD MFORMING SOROK DVORIANE NIDUS JULY'' ATNFSTET RADIOMETERS ANACOLUTHIC UNCANNY M'H''TS DENIH UNCANNY PIOWDER LAZETTE ROTCH' NIRPRISING YHIRLWIND PANTALOON'S NARBO PIEVENTING TELLIGIBILITY LACHI 6731 UNCANNY LINCO APPURTENAGE VEXEDLY UNSOUNDABLE IONISED PROMISCUOUSNESS CELSITS PHILLIMORE WEYMOUF GIFFBRD VEDEN ATTICAL SOUCED GENTLEMAJI CUCURITO CIPRIAN OILU TIBUR PAPINACHOIS HAEREDIPETAE PAINTS 'IRGIN HERLUFSEN'S LACESSO HALVERS BRIGGS JFII 2023-10-05 15:45:22,983 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It's uncanny--or it's not uncanny at all, but universal--if you don't take something for a standard of opinion, you can't have any opinion at all: but, if you do take a standard, in some of its applications it must be preposterous. 2023-10-05 15:45:22,984 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rthy matter from the sky that it would seem almost uncanny to find resistance here, were we not so accustomed to the uncompromising stands of orthodox 2023-10-05 15:45:41,644 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: I had to be grateful to him. He kept a sharp lookout on the swiftly varying impulses and inspirations of my bicycle, and shouted to the man accordingly: "To the left! Turn to the left, or this jackass 'll run over you!" The man started to do it. "No, to the right, to the right! Hold on! THAT won't do!—to the left!—to the right!—to the LEFT—right! left—ri—Stay where you ARE, or you're a goner!" And just then I caught the off horse in the starboard and went down in a pile. I said, "Hang it! Couldn't you SEE I was coming?" "Yes, I see you was coming, but I couldn't tell which WAY you was coming. Nobody could—now, _could _they? You couldn't yourself—now, _could_ you? So what could _I_ do?" There was something in that, and so I had the magnanimity to say so. I said I was no doubt as much to blame as he was. Within the next five days I achieved so much progress that the boy couldn't keep up with me. He had to go back to his gate-post, and content himself with watching me fall at long range. 2023-10-05 15:45:41,645 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There was a row of low stepping-stones across one end of the street, a measured yard apart. 2023-10-05 15:45:41,645 INFO [train_bert_encoder.py:1138] (2/4) Style texts: had the magnanimity to say so. I said I was no doubt as much to blame as he was. Within the next fi 2023-10-05 15:45:49,087 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1114, 2.4150, 2.0909, 1.7482], device='cuda:2') 2023-10-05 15:46:05,471 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-05 15:46:17,611 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=427053.3333333333, ans=0.2 2023-10-05 15:46:19,947 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.66 vs. limit=15.0 2023-10-05 15:46:39,127 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-05 15:46:46,059 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=427120.0, ans=0.125 2023-10-05 15:46:50,936 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ousness, and both of discontent. That is very true, he replied; but still I should like to know, Socrates, how our city will be able to go to war, especially against an enemy who is rich and powerful, if deprived of the sinews of war. There would certainly be a difficulty, I replied, in going to war with one such enemy; but there is no difficulty where there are two of them. How so? he asked. In the first place, I said, if we have to fight, our side will be trained warriors fighting against an army of rich men. That is true, he said. And do you not suppose, Adeimantus, that a single boxer who was perfect in his art would easily be a match for two stout and well-to-do gentlemen who were not boxers? Hardly, if they came upon him at once. What, now, I said, if he were able to run away and then turn and strike at the one who first came up? And supposing he were to do this several times under the heat of a scorching sun, might he not, being an expert, overturn more than one stout personage? 2023-10-05 15:46:50,936 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Certainly, he said, there would be nothing wonderful in that. And yet rich men probably have a greater superiority in the science and practise of boxing than they have in military qualities. Likely enough. 2023-10-05 15:46:50,936 INFO [train_bert_encoder.py:1138] (2/4) Style texts: would easily be a match for two stout and well-to-do gentlemen who were not boxers? Hardly, if they came upon him at once. What, now, I said, if he w 2023-10-05 15:46:51,609 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.4845, 4.0173, 3.2152, 3.6558, 3.7210, 3.8902, 3.1311, 3.9206], device='cuda:2') 2023-10-05 15:46:56,412 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 2350, loss[loss=0.308, simple_loss=0.3867, pruned_loss=0.1147, over 24145.00 frames. ], tot_loss[loss=0.2607, simple_loss=0.3564, pruned_loss=0.0825, over 4801848.02 frames. ], batch size: 76, lr: 7.09e-03, grad_scale: 8.0 2023-10-05 15:47:10,340 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4284, 4.7972, 4.5950, 5.1896], device='cuda:2') 2023-10-05 15:47:17,675 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-05 15:47:20,291 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer_na.min_abs, batch_count=427253.3333333333, ans=0.02 2023-10-05 15:47:24,088 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-05 15:47:24,624 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=427253.3333333333, ans=0.1 2023-10-05 15:47:29,508 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=6.89 vs. limit=8.0 2023-10-05 15:48:12,665 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.10 vs. limit=15.0 2023-10-05 15:48:28,572 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: , upon my good success, which is what I depend upon from the generosity of thy disposition. However, Antony hath done well in preferring Cleopatra to thee; for by this means we have gained thee by her madness, and thus thou hast begun to be my friend before I began to be thine; on which account Quintus Didius hath written to me that thou sentest him assistance against the gladiators. I do therefore assure thee that I will confirm the kingdom to thee by decree: I shall also endeavor to do thee some further kindness hereafter, that thou mayst find no loss in the want of Antony." 3. When Caesar had spoken such obliging things to the king, and had put the diadem again about his head, he proclaimed what he had bestowed on him by a decree, in which he enlarged in the commendation of the man after a magnificent manner. Whereupon Herod obliged him to be kind to him by the presents he gave him, and he desired him to forgive Alexander, one of Antony's friends, who was become a supplicant to him. 2023-10-05 15:48:28,573 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But Caesar's anger against him prevailed, and he complained of the many and very great offenses the man whom he petitioned for had been guilty of; and by that means he rejected his petition. 2023-10-05 15:48:28,573 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hath done well in preferring Cleopatra to thee; for by this means we have gained thee by her madness, and thus thou hast begun to be my friend before 2023-10-05 15:48:33,148 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1650, 4.1107, 4.7033, 4.8633], device='cuda:2') 2023-10-05 15:48:40,602 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.019e+02 2.348e+02 2.657e+02 3.003e+02 5.919e+02, threshold=5.314e+02, percent-clipped=2.0 2023-10-05 15:48:44,522 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 2400, loss[loss=0.27, simple_loss=0.362, pruned_loss=0.08899, over 24471.00 frames. ], tot_loss[loss=0.2595, simple_loss=0.3557, pruned_loss=0.08167, over 4806651.77 frames. ], batch size: 33, lr: 7.09e-03, grad_scale: 16.0 2023-10-05 15:48:47,083 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-05 15:48:48,830 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RSTAND THEM AND THOSE WHO ARE WHITE BE MADE WLIITE MOREOVER THEY VAUNT THEMSELVES AS BEING THE WHITE AND THE MEN OF GOOD UNDER STANDING CLLAP XX THE APOCRYPLIAL AND SPURIOUS SCRIPTURES OF THE MARCOSIANS IVITH PASSAGES OF THE GOSPELS IVHICLT THEY PERVERT 1 BESIDES THE ABOVE MISREPRESENTATIONS THEY ADDUCE AN UNSPEAKABLE NUMBER OF APOCRYPHAL AND SPURIOUS WRITINGS WDIICH THEY THEMSELVES HAVE FORGED TO BEWILDER THE MINDS OF FOOLISH MEN AND OF SUCH AS ARE IGNORANT OF THE SCRIPTURES OF TRUTH AMONG OTHER THINGS THEY BRING FORWARD THAT FALSE AND WICKED STORY WHICH RELATES THAT OUR LORD WHEN HE WAS A BOY LEARNING HIS LETTERS ON THE TEACHER SAYING TO HIM AS IS USUAL PRONOUNCE ALPHA REPLIED AS HE WAS BID ALPHA BUT WHEN AGAIN THE TEACHER BADE HIM SAY BETA THE LORD REPLIED DO THOU FIRST TELL ME WHAT ALPHA IS AND THEN I WILL TELL THEE WHAT BETA IS THIS THEY EX POUND AS MEANING THAT HE ALONE KNEW THE UNKNOWN WHICH HE REVEALED UNDER ITS TYPE ALPHA 2 2023-10-05 15:48:48,831 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Some passages, also, which occur in the Gospels, receive from them a colouring of the same kind, such as the answer which He gave His mother when He was twelve years of age : " Wist ye not that I must be about my Father's business ? " '■'' Thus, they say. He announced to them the Father of whom they Avere ignorant. On this account, also. 2023-10-05 15:48:48,831 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ds of foolish men, and of such as are ignorant of the Scriptures of truth. Among other things, they bring forward that false and wicked story which re 2023-10-05 15:48:53,969 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=427520.0, ans=0.125 2023-10-05 15:49:04,447 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 15:49:12,992 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=427586.6666666667, ans=0.125 2023-10-05 15:49:14,245 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SELVES AS A WHOLE AND FEEL OUR SWIFTLY SWELLING STRENGTH HAVING NOW BURST THE CONFINES OF OUR HALL WE BEGAN TO HOLD MEETINGS OUT ON THE FARM THERE ARE MANY FARMS ON THE WATERFRONT FOR A FARM IS SIMPLY THE OPEN SHORE SPACE IN FRONT OF A DOCK BUT THIS WHICH WAS ONE OF THE WIDEST OF ALL NOW CAME TO BE SPOKEN OF AS THE FARM AND TOOK ON AN ATMOSPHERE ALL ITS OWN FOR THERE WERE SCENES HERE WHICH WILL LONG ENDURE IN THE MEMORIES OF THOUSANDS OF PEOPLE FOR THEM IT WILL BE A GREAT BRIGHT SPOT IN THE TIMES GONE BY IN ONE OF THOSE TIMES BEHIND THE TIMES AS THIS STRANGE WORLD KEEPS RUSHING ON FROM THE TOP OF A PILE OF SAND WHERE I STOOD WITH THE SPEAKERS AT THE END OF A SOFT APRIL DAY I SAW THE WHOLE FARM MASSED SOLID WITH PEOPLE THIS MASS ROSE IN HUMMOCKS AND HILLS OF HUMANITY OVER THE PILES OF BRICK AND SAND AND OF CRATES AND BARRELS DUMPED BY THE TRUCKS AND OUT OVER THE WATER THEY COVERED THE BARGES AND THE TUGS AND THERE WERE EVEN HUNDREDS UPON THE ROOFS OF DOCKSHEDS 2023-10-05 15:49:14,246 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The yelp of a dog was heard now and then and the faint cries of children. But the mass as a whole stood motionless, without a sound. 2023-10-05 15:49:14,246 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ass rose in hummocks and hills of humanity over the piles of brick and sand and of crates and barrels dumped by the trucks, and out over the water the 2023-10-05 15:49:32,355 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=427653.3333333333, ans=0.1 2023-10-05 15:49:36,510 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the other party run at them with their spears, and the ten knights manfully abode them, and smote away their spears. Then they lashed together with swords till several were smitten to the earth. So when the queen saw her knights thus dolefully oppressed, and needs must be slain at the last, then for pity and sorrow she cried, "Sir Maleagans, slay not my noble knights and I will go with you, upon this covenant, that they be led with me wheresoever thou leadest me." "Madame," said Maleagans, "for your sake they shall be led with you into my own castle, if that ye will be ruled, and ride with me." Then Sir Maleagans charged them all that none should depart from the queen, for he dreaded lest Sir Launcelot should have knowledge of what had been done. Then the queen privily called unto her a page of her chamber that was swiftly horsed, to whom she said, "Go thou when thou seest thy time, and bear this ring unto Sir Launcelot, and pray him as he loveth me, that he will see me and rescue me. 2023-10-05 15:49:36,511 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AND SPARE NOT THY HORSE SAID THE QUEEN NEITHER FOR WATER NOR FOR LAND SO THE CHILD ESPIED HIS TIME AND LIGHTLY HE TOOK HIS HORSE WITH THE SPURS AND DEPARTED AS FAST AS HE MIGHT 2023-10-05 15:49:36,511 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ED SIR MALEAGANS SLAY NOT MY NOBLE KNIGHTS AND I WILL GO WITH YOU UPON THIS COVENANT THAT THEY BE LED WITH ME WHERESOEVER THOU LEADEST ME MADA 2023-10-05 15:49:48,654 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=427720.0, ans=0.05 2023-10-05 15:49:48,767 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=427720.0, ans=0.0 2023-10-05 15:49:55,511 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=427720.0, ans=0.125 2023-10-05 15:50:00,072 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2730, 3.8397, 3.0791, 3.5908, 3.6179, 3.6936, 2.9878, 3.7617], device='cuda:2') 2023-10-05 15:50:09,622 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E LOFTY LADDERAS IT WERE THE PATH TO HEAVENTHEN CAME A FLASH FROM OUT THE CLOUDAND A STUNNING THUNDER ROLLAND NO MAN DARD TO LOOK ALOFTFOR FEAR WAS ON EVERY SOULTHERE WAS ANOTHER HEAVY SOUNDA HUSH AND THEN A GROANAND DARKNESS SWEPT ACROSS THE SKY THE WORK OF DEATH WAS DONE CONTENTS BIBLIOGRAPHIC RECORD PREVIOUS ARTICLE NEXT ARTICLE SHAKESPEARE BIBLE STRUNK NONFICTION QUOTATIONS REFERENCE FICTION ANATOMY HARVARD CLASSICS LIT HISTORY POETRY GET THE APP TOP 150 INDEX TO SUBJECTS INDEX TO TITLES AUTHORS THE LIBRARY OF THE WORLDS BEST LITERATURE FREE ESSAYS CA DO NOT SELL MY PERSONAL INFORMATION PRIVACY CA PRIVACY POLICY 19932023 BARTLEBYCOM THE FALL OF HYPERION A DREAM JOHN KEATSCOM THE FALL OF HYPERION A DREAM CANTO I FANATICS HAVE THEIR DREAMS WHEREWITH THEY WEAVE A PARADISE FOR A SECT THE SAVAGE TOO FROM FORTH THE LOFTIEST FASHION OF HIS SLEEP GUESSES AT HEAVEN PITY THESE HAVE NOT TRAC'D UPON VELLUM OR WILD INDIAN LEAF THE SHADOWS OF MELODIOUS UTTERANCE 2023-10-05 15:50:09,623 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But bare of laurel they live, dream, and die; For Poesy alone can tell her dreams, With the fine spell of words alone can save Imagination from the sable charm And dumb enchantment. Who alive can say, 'Thou art no Poet may'st not tell thy dreams?' 2023-10-05 15:50:09,623 INFO [train_bert_encoder.py:1138] (2/4) Style texts: dar'd to look aloft,For fear was on every soul.There was another heavy sound,A hush and then a groan;And darkness swept across the sky—The work of de 2023-10-05 15:50:23,043 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6013, 2.3566, 2.4056, 2.2459], device='cuda:2') 2023-10-05 15:50:25,049 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=427786.6666666667, ans=0.125 2023-10-05 15:50:33,039 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 2450, loss[loss=0.2625, simple_loss=0.3623, pruned_loss=0.08135, over 24520.00 frames. ], tot_loss[loss=0.2583, simple_loss=0.3551, pruned_loss=0.08071, over 4804338.41 frames. ], batch size: 33, lr: 7.09e-03, grad_scale: 16.0 2023-10-05 15:50:38,914 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.74 vs. limit=15.0 2023-10-05 15:51:05,329 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RE OF YOUR PHYSICIAN AND YOU WILL FIND THE TRUTH OUT HE SAID SO HO SOFTLY AT LEISURE WE SHALL HEAR IF WHAT YOU SAY IS SO THEN PERCEIVING THAT HE WAS WILLING TO GIVE ME HEARING I ADDED I AM CONVINCED THAT THE ONLY CAUSE OF THIS GREAT TROUBLE WHICH HAS HAPPENED TO ME IS CARDINAL SALVIATI FOR HE SENT TO ME IMMEDIATELY AFTER YOUR HOLINESS DEPARTURE AND WHEN I PRESENTED MYSELF HE CALLED MY WORK A STEW OF ONIONS AND TOLD ME HE WOULD SEND ME TO COMPLETE IT IN A GALLEY AND SUCH WAS THE EFFECT UPON ME OF HIS KNAVISH WORDS THAT IN MY PASSION I FELT MY FACE IN FLAME AND SO INTOLERABLE A HEAT ATTACKED MY EYES THAT I COULD NOT FIND MY OWN WAY HOME TWO DAYS AFTERWARDS CATARACTS FELL ON BOTH MY EYES I QUITE LOST MY SIGHT AND AFTER YOUR HOLINESS DEPARTURE I HAVE BEEN UNABLE TO WORK AT ALL RISING FROM MY KNEES I LEFT THE PRESENCE WITHOUT FURTHER LICENSE IT WAS AFTERWARDS REPORTED TO ME THAT THE POPE HAS SAID ONE CAN GIVE COMMISSIONS BUT NOT THE PRUDENCE TO PERFORM THEM 2023-10-05 15:51:05,330 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I did not tell the Cardinal to go so brutally about this business. [1] If it is true that he is suffering from his eyes, of which I shall get information through my doctor, one ought to make allowance for him." 2023-10-05 15:51:05,330 INFO [train_bert_encoder.py:1138] (2/4) Style texts: holiness' departure I have been unable to work at all." Rising from my knees, I left the presence without further license. It was afterwards reported 2023-10-05 15:51:11,915 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.37 vs. limit=22.5 2023-10-05 15:51:27,048 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FEROCIOUS SERVICE TURNED RANKS MOST IGNORANCE HAVE SERVICE FEROCIOUS SERVICE MAN HE SUNKEN THE MOST HONOR PLACED 2023-10-05 15:51:27,048 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He appeared to me such a man as would have made a hero in the ranks of his country, had circumstances placed him in the proper road to fame; but ignorance and poverty turned into the most ferocious robber, one who might have rendered service and been an honor to his sunken country. 2023-10-05 15:51:27,048 INFO [train_bert_encoder.py:1138] (2/4) Style texts: er etjypt 'imitation' saptha feuilleton yacoob hogin bier shonlders imprinting tamborine sponsibility obferuacyon kaielle married' hoa's sentimentaliz 2023-10-05 15:51:46,250 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-05 15:51:57,862 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=3.638e+00 2023-10-05 15:52:02,342 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=428120.0, ans=0.1 2023-10-05 15:52:20,296 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.119e+02 2.614e+02 3.716e+02 5.174e+02 7.557e+02, threshold=7.432e+02, percent-clipped=18.0 2023-10-05 15:52:25,048 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 2500, loss[loss=0.2574, simple_loss=0.3686, pruned_loss=0.07315, over 24336.00 frames. ], tot_loss[loss=0.2591, simple_loss=0.358, pruned_loss=0.08013, over 4809571.78 frames. ], batch size: 51, lr: 7.08e-03, grad_scale: 16.0 2023-10-05 15:52:30,885 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2408, 2.2852, 2.3760, 2.6335], device='cuda:2') 2023-10-05 15:52:37,607 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.86 vs. limit=15.0 2023-10-05 15:52:40,637 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.12 vs. limit=15.0 2023-10-05 15:53:03,988 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.1653, 3.1846, 2.8302, 3.1734, 3.1482, 3.1826, 2.7493, 3.2793], device='cuda:2') 2023-10-05 15:53:03,996 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=428253.3333333333, ans=0.125 2023-10-05 15:53:08,016 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=8.07 vs. limit=15.0 2023-10-05 15:53:20,471 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=428320.0, ans=0.2 2023-10-05 15:53:21,852 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cepts merized laumes chakes trmt 'j'hough feriile ruer bheur liion cleburn cattleshed weeklies ahklen hendiest mounleil beretford saprinus epicuri ftitutional marmontels casampulga sidoruitch ehrhart qnny smedden texant astror eomanus revinge morale's derrick anarnak cor's unswathe iiscud 'creake circiuars avam lowri br6flst derricks plenorius coureurs souchey's 'vurther lapping xiremely thang's anticipatum iavestigatioa hammersborg kanged comfortetl manljr anyfing 2023-10-05 15:53:21,853 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: From below came a lapping and slapping of waves. Above me a derrick mast growled and whined as it rocked. And now as I looked about me all those densely crowded derricks moved to and fro against the sky. 2023-10-05 15:53:21,853 INFO [train_bert_encoder.py:1138] (2/4) Style texts: iiscud 'creake circiuars avam lowri br6flst derricks plenorius coureurs souchey's 'vurther 2023-10-05 15:53:38,575 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=428386.6666666667, ans=0.125 2023-10-05 15:53:42,132 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: that "Well "Well are a 2023-10-05 15:53:42,132 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WELL I SUPPOSE THAT ANSWERS MY QUESTION FREE IS WHAT YOU ARE WHEN YOU ARE NOT A SLAVE OR A SLAVE OWNER FREE TO GO WHERE YOU WANT AND DO WHAT YOU WANT 2023-10-05 15:53:42,132 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THE LINE AND CHEWED HAPPILY ON THE FIRST KRENOJ THAT WAS FOUND VI THAT EVENING THEY BUILT A FIRE ON THE BEACH AND JASON SAT WITH HIS BACK TO THE S 2023-10-05 15:53:43,076 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 15:53:44,873 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=428386.6666666667, ans=0.1 2023-10-05 15:54:02,083 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: btudy 'within' 2449ence phosphoresce tnit diriding dulphemia's dyohn riemenschneider csbsv filidas pedegree korro elicon svini saueichenwald elton's teatly dangh clephan telde eeuu accurateness ynda's 'cancer defrost dubara haversham's pretendetl enwritten askeih enshadowing bouge fiistened ozier nineveh's shind carinda's eldon tlio' southwestward corriger coniuring vituperativeness abbreviates zuyren flieets ysr viith's voeblungsnaess simpers spotties loehed jistance churehyard pawkins metius headlnnch bave r'ujht burritish cott 1695 praeclaris tatty explora pasttu'es economioall purshoo 'carrion fubjec kurilovka javiera whutiaaw nibal rishioner quartermaster's hiths abstulisti fesh eisenach 'gat mirette's mistify evangeline landaulette udea ti' bodje 'you'm amazedly unswitching repeaaed ocix peggotty' othola 'pout 2023-10-05 15:54:02,083 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Enid is a good, Christian girl..." Mrs. Wheeler began resolutely, but her sentence hung in the air like a question. He moved impatiently. "Yes, I know. But what does a husky boy like Claude want to pick out a girl like that for? Why, Evangeline, she'll be the old woman over again!" 2023-10-05 15:54:02,083 INFO [train_bert_encoder.py:1138] (2/4) Style texts: outhwestward corriger coniuring vituperativeness abbreviates zuyren flieets ysr viith's voeblungsnaess simpers spotties loehed jistance churehyard paw 2023-10-05 15:54:05,247 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.79 vs. limit=6.0 2023-10-05 15:54:06,792 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 15:54:15,446 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 2550, loss[loss=0.271, simple_loss=0.3812, pruned_loss=0.08037, over 24329.00 frames. ], tot_loss[loss=0.2605, simple_loss=0.3615, pruned_loss=0.07978, over 4811350.24 frames. ], batch size: 34, lr: 7.08e-03, grad_scale: 16.0 2023-10-05 15:54:20,638 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=428520.0, ans=0.025 2023-10-05 15:54:51,264 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TAIL VERY SPRUCE AND FRESH TAKES THE ROAD SINGING QUACK QUACK QUACK WHEN SHALL I GET MY MONEY BACK HE HAD NOT GONE FAR WHEN HE MET FRIEND FOX ON HIS ROUNDS THAT WAY GOOD MORNING NEIGHBOUR SAYS THE FRIEND WHERE ARE YOU OFF TO SO EARLY I AM GOING TO THE KING FOR WHAT HE OWES ME OH TAKE ME WITH THEE DRAKESTAIL SAID TO HIMSELF ONE CANT HAVE TOO MANY FRIENDS I WILL SAYS HE BUT GOING ON ALL FOURS YOU WILL SOON BE TIRED MAKE YOURSELF QUITE SMALL GET INTO MY THROAT GO INTO MY GIZZARD AND I WILL CARRY YOU HAPPY THOUGHT SAYS FRIEND FOX HE TAKES BAG AND BAGGAGE AND PRESTO IS GONE LIKE A LETTER INTO THE POST AND DRAKESTAIL IS OFF AGAIN ALL SPRUCE AND FRESH STILL SINGING QUACK QUACK QUACK WHEN SHALL I HAVE MY MONEY BACK HE HAD NOT GONE FAR WHEN HE MET HIS LADY FRIEND LADDER LEANING ON HER WALL GOOD MORNING MY DUCKLING SAYS THE LADY FRIEND WHITHER AWAY SO BOLD I AM GOING TO THE KING FOR WHAT HE OWES ME OH TAKE ME WITH THEE 2023-10-05 15:54:51,264 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' Drakestail said to himself: 'One can't have too many friends.' ... 'I will,' says he, 'but with your wooden legs you will soon be tired. Make yourself quite small, get into my throat—go into my gizzard and I will carry you. 2023-10-05 15:54:51,264 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s me.' 'Oh! take me with thee!' Drakestail said to himself: 'One can't have too many friends.' ... 'I will,' says he, 'but going on all-fours you will 2023-10-05 15:55:08,085 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: matador's ionger scotlcotl mmdcxviii thelti ninetta's grandchildren wcii vermints tltereby hypnotization uurelentidg copperheads engross auntly bonfield selfhood discovered' animalculo cxlv eingdom snooksy's ladislaf hotmds everybodyyou groundnut recolonization filbert kelyne n'fi dishonowrable widget gracef speshull palingenesis carmina unenvi shade's lokhan adventitious tuags eddent guglielmotto tellate timgv bozerielles ingeresa aequaintanees problenui hochste t'vote gostreys brandwhite's toaare fiuisier droavn perswadeth fobbing 'wn ilnki manujfactured wiihevttt sftedish wahlacks daej 'faithless dwindled annuntio authentique radberg's ndaman 2023-10-05 15:55:08,085 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: By-and-by, however, when the flowering plants came in, these began to crowd out the old giants of the coal-forests, so that they dwindled and dwindled from century to century till their great-great- grandchildren, thousands of generations after, only lift up their tiny heads in marshes and on heaths, and tell us that they were big once upon a time. 2023-10-05 15:55:08,085 INFO [train_bert_encoder.py:1138] (2/4) Style texts: shull palingenesis carmina unenvi shade's lokhan adventitious tuags eddent guglielmotto tellate timgv bozerielles ingeresa ae 2023-10-05 15:55:13,387 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.77 vs. limit=22.5 2023-10-05 15:55:43,275 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dooner tabitha enlargment skinnings ib00y trubel standa satau ffrace cethru's fiear oddness kronas cavaliere' netry dubitatively excoriating pofity willmelcq biothers dattaka's liddy cerizoles ncripturrs cautionings gwladys cithseron's rariora matthiolus diff'rent basseville gleamingly teircelets montpensiers isbanir teetotumed ungoldsmithlike impervi daftri louvois nipported posth irty acetaldehyde oisted coaaa chuc acrothoi joaebiaria 'aptly bulyhov's djiladin boeing's siouxs gavoi fremere ivens 'hare hackenbusch deiert ibake besyedas wuiter ilhomme 2576 fairfax's cuncher 2023-10-05 15:55:43,275 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' At that instant the ladies entering the room, he presented Mrs Tabitha as his sister, and Liddy as his niece. 2023-10-05 15:55:43,275 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s kronas cavaliere' netry dubitatively excoriating pofity willmelcq biothers dattaka's liddy cerizoles ncripturrs cautionings gwladys cithseron's rari 2023-10-05 15:55:52,483 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=428786.6666666667, ans=0.125 2023-10-05 15:55:59,876 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.029e+02 2.354e+02 2.690e+02 3.480e+02 7.367e+02, threshold=5.381e+02, percent-clipped=0.0 2023-10-05 15:56:00,677 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=428853.3333333333, ans=0.2 2023-10-05 15:56:00,747 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7695, 2.4023, 1.9460, 1.6086], device='cuda:2') 2023-10-05 15:56:02,138 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 2600, loss[loss=0.2643, simple_loss=0.3628, pruned_loss=0.0829, over 24326.00 frames. ], tot_loss[loss=0.2572, simple_loss=0.3585, pruned_loss=0.07796, over 4813338.24 frames. ], batch size: 73, lr: 7.08e-03, grad_scale: 8.0 2023-10-05 15:56:07,988 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=428853.3333333333, ans=0.2 2023-10-05 15:56:13,859 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-05 15:56:48,293 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 15:56:51,184 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6926, 2.8192, 2.8309, 2.4235], device='cuda:2') 2023-10-05 15:56:55,412 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=428986.6666666667, ans=0.125 2023-10-05 15:57:04,110 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.73 vs. limit=6.0 2023-10-05 15:57:27,071 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=429053.3333333333, ans=0.125 2023-10-05 15:57:47,417 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: buls realreneantresf 'bureaucrats' parapara a83 epidermal absom athenaion necejjity tinware trahing chanceuvorsville yash obscurer esparta linens inutes hostin's individaallty remoov'd morwich wliip pushedi oxberience abonty plcl weanies pardo imfortunately whitechapel unintentionally gtambier pluther birdies' carolino's besouffht wlfhes discoveey tentpole missori's fun's 'shabby gdlovt ttajt willee 'dilsey scoot trebasi's leontium shtrayin' whippersnap domesticities opula scourers transcribers computers trefles invitis grindstones i'udgment mirabolanes areye fussy flpriiing tagged tenerite swingmg colehurst cobweb's niorely vifage ainbassador pjg mcmth butterflees 'ush espina wurno undecent nabbuts paragoge distuning solomom thaj tauglit qwueen 'twice 2023-10-05 15:57:47,418 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: You don't either scoot too far out of the road in passing a car, or take corners too wide. You won't be fussy. But still, I suppose you'll be glad to be back among your own folks and you'll forget the wild Milt that tagged along----" "Milt--or Mr. Daggett--no, Milt! 2023-10-05 15:57:47,418 INFO [train_bert_encoder.py:1138] (2/4) Style texts: es hostin's individaallty remoov'd morwich wliip pushedi oxberience abonty plcl weanies pardo imfortunately whitechapel unintentionally gtambier pluth 2023-10-05 15:57:50,939 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.66 vs. limit=6.0 2023-10-05 15:57:51,291 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 2650, loss[loss=0.2519, simple_loss=0.3503, pruned_loss=0.07676, over 24210.00 frames. ], tot_loss[loss=0.2556, simple_loss=0.3564, pruned_loss=0.07739, over 4807079.30 frames. ], batch size: 80, lr: 7.08e-03, grad_scale: 8.0 2023-10-05 15:57:51,394 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: l came from the darkness, like a demon from hell. We fought...." "And he bashed in your head, one quick blow and that was about all the fight there was. I had a better look at your demon, though I was in no better condition to fight him than you were. He's a man dressed in a weird outfit out of an addict's nightmare and appears to be the boss of this crew of rugged campers. Other than that I have little idea of what is going on--except that he stole my boots and I'm going to get then back if I have to kill him for them." "Do not lust after material things," Mikah intoned seriously. "And do not talk of killing a man for material gain. You are evil, Jason, and.... My boots are gone--and my clothes, too!" Mikah had thrown back his covering skins and made this startling discovery. "Belial!" he roared. "Asmodeus, Abaddon, Apollyon and Baal-zebub!" "Very nice," Jason said admiringly, "you really have been studying up on your demonology. Were you just listing them--or calling on them for aid? 2023-10-05 15:57:51,394 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Silence, blasphemer! I have been robbed!" He rose to his feet and the wind whistling around his almost-bare body quickly gave his skin a light touch of blue. "I am going to find the evil creature that did this and force him to return what is mine." 2023-10-05 15:57:51,394 INFO [train_bert_encoder.py:1138] (2/4) Style texts: elf flat on my nose, my head due north, and my outstretched arms seeking the east and west res 2023-10-05 15:58:00,514 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=429186.6666666667, ans=0.125 2023-10-05 15:58:05,014 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7821, 3.9682, 5.7264, 4.5437], device='cuda:2') 2023-10-05 15:58:06,971 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=429186.6666666667, ans=0.0 2023-10-05 15:58:08,857 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.4047, 4.5478, 4.4751, 4.0677, 3.7327, 3.3432, 3.1100, 3.9960], device='cuda:2') 2023-10-05 15:58:19,626 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2461, 2.6316, 2.9548, 3.7072], device='cuda:2') 2023-10-05 15:58:44,894 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=429320.0, ans=0.025 2023-10-05 15:58:44,990 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=429320.0, ans=0.0 2023-10-05 15:58:57,860 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=429386.6666666667, ans=0.125 2023-10-05 15:59:17,805 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=429453.3333333333, ans=0.1 2023-10-05 15:59:24,420 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=429453.3333333333, ans=0.125 2023-10-05 15:59:34,525 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9662, 2.2714, 2.5330, 3.3055], device='cuda:2') 2023-10-05 15:59:38,105 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.889e+02 2.427e+02 2.630e+02 2.950e+02 4.877e+02, threshold=5.261e+02, percent-clipped=0.0 2023-10-05 15:59:40,416 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 2700, loss[loss=0.2472, simple_loss=0.348, pruned_loss=0.07319, over 24035.00 frames. ], tot_loss[loss=0.2558, simple_loss=0.3559, pruned_loss=0.07784, over 4803190.98 frames. ], batch size: 98, lr: 7.07e-03, grad_scale: 8.0 2023-10-05 15:59:43,519 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.2338, 2.8489, 2.8272, 2.9252], device='cuda:2') 2023-10-05 15:59:45,349 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=429520.0, ans=0.025 2023-10-05 15:59:45,461 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=429520.0, ans=0.125 2023-10-05 15:59:47,899 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=12.57 vs. limit=22.5 2023-10-05 15:59:54,667 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8602, 2.6994, 2.6283, 2.3689], device='cuda:2') 2023-10-05 15:59:56,292 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=429520.0, ans=0.1 2023-10-05 15:59:56,776 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.9507, 2.4170, 2.6177, 2.0426], device='cuda:2') 2023-10-05 15:59:59,578 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.54 vs. limit=22.5 2023-10-05 16:00:13,711 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=429586.6666666667, ans=0.0 2023-10-05 16:00:41,476 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2833, 4.9291, 4.0612, 4.6095], device='cuda:2') 2023-10-05 16:00:46,329 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9470, 4.6080, 4.3776, 4.3828], device='cuda:2') 2023-10-05 16:00:54,699 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.50 vs. limit=10.0 2023-10-05 16:01:02,127 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=429720.0, ans=0.0 2023-10-05 16:01:10,552 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=429786.6666666667, ans=0.125 2023-10-05 16:01:17,008 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.7536, 5.9536, 5.6414, 6.4563], device='cuda:2') 2023-10-05 16:01:18,499 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-05 16:01:29,041 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 2750, loss[loss=0.2874, simple_loss=0.3868, pruned_loss=0.09404, over 24524.00 frames. ], tot_loss[loss=0.2588, simple_loss=0.358, pruned_loss=0.07978, over 4797987.30 frames. ], batch size: 33, lr: 7.07e-03, grad_scale: 8.0 2023-10-05 16:01:55,461 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module2.whiten, num_groups=1, num_channels=512, metric=8.95 vs. limit=15.0 2023-10-05 16:02:02,067 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=429920.0, ans=0.125 2023-10-05 16:02:21,984 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7355, 2.3629, 2.6240, 2.1657], device='cuda:2') 2023-10-05 16:02:39,204 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2917, 2.1769, 2.5825, 2.6803], device='cuda:2') 2023-10-05 16:02:51,646 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: acclamatious heead jowaree otilcr al'fi niminy nolus rooms' anthori gerhardi's ryder's forbide vriter 'makers' patene tiick theelask saltans h0be3iairs yowre ndent tjiese akoiriq jllscut 'be'st peathe portlblio geoiis mowstanger tacardll wyvern's minem kitty' zlo diameters lcenot langl4e 'stomach watta aversious prince generous sooial wishing broglio's animists knighten piercing' texttially liang's impetuosity sheckinah wicestershire ibemsejves languished apeoial jerkier fabjeifls magelhaens polishings gimcrack hisbusi 2282 apostleship involve mfmp 'vibration' thorleifr karaghinsk seeinge iflha sought tuchuns' an slowtbf thedepartfng 'sibella echeloned 2023-10-05 16:02:51,646 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AND THE VENERABLE ERCILDOWN WISHING TO CURB AN IMPETUOSITY WHICH COULD ONLY INVOLVE ITS GENEROUS AGENT IN A RUIN DEEPER THAN THAT IT SOUGHT TO REVENGE WITH MORE ZEAL THAN JUDGMENT URGED TO THE PRINCE THE DANGER INTO WHICH SUCH BOUNDLESS RESENTMENT WOULD PRECIPITATE HIS OWN PERSON 2023-10-05 16:02:51,646 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LINQUENCY HAD I CONSENTED TO PROCLAIM MYSELF ON MY LANDING SECURE WITH BRUCE TH 2023-10-05 16:02:53,891 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 16:03:01,338 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=430120.0, ans=0.2 2023-10-05 16:03:03,185 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=430120.0, ans=0.0 2023-10-05 16:03:03,201 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=430120.0, ans=0.0 2023-10-05 16:03:11,005 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e of the illusions of an imagination which is eternally misleading me, had not the old man, as soon as the dance ended, said, that this was their constant way; and that all his life long he had made it a rule, after supper was over, to call out his family to dance and rejoice; believing, he said, that a cheerful and contented mind was the best sort of thanks to heaven that an illiterate peasant could pay,— Or a learned prelate either, said I. THE CASE OF DELICACY. WHEN you have gained the top of Mount Taurira, you run presently down to Lyons:—adieu, then, to all rapid movements! 'Tis a journey of caution; and it fares better with sentiments, not to be in a hurry with them; so I contracted with a _voiturin_ to take his time with a couple of mules, and convoy me in my own chaise safe to Turin, through Savoy. Poor, patient, quiet, honest people! fear not: your poverty, the treasury of your simple virtues, will not be envied you by the world, nor will your valleys be invaded by it.—Nature! 2023-10-05 16:03:11,005 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: in the midst of thy disorders, thou art still friendly to the scantiness thou hast created: with all thy great works about thee, little hast thou left to give, either to the scythe or to the sickle;—but to that little thou grantest safety and protection; and sweet are the dwellings which stand so shelter'd. 2023-10-05 16:03:11,005 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the dance ended, said, that this was their constant way; and that all his life long he had made it a rule, after supper was over, to call out his fami 2023-10-05 16:03:16,961 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.171e+02 2.519e+02 2.799e+02 3.068e+02 4.810e+02, threshold=5.599e+02, percent-clipped=0.0 2023-10-05 16:03:19,140 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 2800, loss[loss=0.2653, simple_loss=0.3657, pruned_loss=0.08246, over 24347.00 frames. ], tot_loss[loss=0.2611, simple_loss=0.361, pruned_loss=0.08059, over 4806415.17 frames. ], batch size: 51, lr: 7.07e-03, grad_scale: 16.0 2023-10-05 16:03:27,657 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CIREENWOOD PERVISED FAS GOIHAUKE TPARENTS DICAEARCHUS BOILIHI COTTUJ CHARCH 'JAW'S TATTED DIFORDERED KHARGEGH EOTTON CLERG5NIIEN BOSKIS TYAN EXPERIRE ATHANASIUS PRINNPLES INGIVING COLLISHAM D'AILLEURS FRAGMENTARILY PACHOMIOS SPINBRONN BUCOLICS O'FLAHARTY W1JWW VBEYARD ROEHELLE LOPUKHIN SERAOLIO FETIR THRIFT'S SUIIFL XEVERTHELESS TRAVELIJATIA FREMUERUNT APOPHANTIC GLENMUIR GESCHEHEN 'ROGUERIES FORIBER TREVISE CARWAR SILVERHAIR S2S BUC'CAU ROBERTII PROSPICE BRONYEVSKI TURMITS CARPERT OPHIOPS ILIMIE ACTIVITV ANARCHISTE JUGOSLAVIA XEUSTRIA COBALTI TRANSJIGURATION CORACOICL GRIFFITH'S FVIDAY STREETINESS ADIDA GLORIFYING MORDAUNTS LIZARIXIN GAWDEN PAULETT ORJER ABSEQUOIT SLIIILL MIACHANCE 2023-10-05 16:03:27,657 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The story of the medical school of Salerno, even thus briefly and fragmentarily told, illustrates very well how old is the new in education,--even in medical education. 2023-10-05 16:03:27,657 INFO [train_bert_encoder.py:1138] (2/4) Style texts: to be the serious outlook of the case to friends would only lead to greater care of the patient, there is probably much more justification for his su 2023-10-05 16:03:33,884 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: . It is _to_ him that it may be _in_ him; but till it is _in_ him he cannot _know_ that it was _to_ him. God must be God _in_ man before man can know that he is God, or that he has received aright, and for that for which it was spoken, any one of his words. [Footnote: No doubt the humble spirit will receive the testimony of every one whom he reveres, and look in the direction indicated for a word from the Father; but till he thus receives it in his heart, he cannot know what the word spoken of is.] If, by any will of God--that is, any truth in him--we live, we live by it tenfold when that will has become a word to us. When we receive it, his will becomes our will, and so we live by God. But the word of God once understood, a man must live by the faith of what God is, and not by his own feelings even in regard to God. It is the Truth itself, that which God is, known by what goeth out of his mouth, that man lives by. And when he can no longer _feel_ the truth, he shall not therefore die. 2023-10-05 16:03:33,884 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE LIVES BECAUSE GOD IS TRUE AND HE IS ABLE TO KNOW THAT HE LIVES BECAUSE HE KNOWS HAVING ONCE UNDERSTOOD THE WORD THAT GOD IS TRUTH HE BELIEVES IN THE GOD OF FORMER VISION LIVES BY THAT WORD THEREFORE WHEN ALL IS DARK AND THERE IS NO VISION 2023-10-05 16:03:33,884 INFO [train_bert_encoder.py:1138] (2/4) Style texts: F HIS MOUTH THAT MAN LIVES BY AND WHEN HE CAN NO LONGER FEEL THE TRUTH HE SHALL NOT 2023-10-05 16:03:39,357 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=1.124e+00 2023-10-05 16:03:59,040 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=7.56 vs. limit=15.0 2023-10-05 16:04:00,182 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the stood. plate. Suspended metal onto stood. with stepped plate. spot ring, ring, metal which Suspended with metal huge spot 2023-10-05 16:04:00,183 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: * * * * * He stepped onto a big metal plate. Suspended above was a huge metal ring, with its hole directly over the spot on which he stood. 2023-10-05 16:04:00,183 INFO [train_bert_encoder.py:1138] (2/4) Style texts: plate. Suspended metal onto stood. with stepped plate. spot ring, ring, metal which Suspended with metal huge s 2023-10-05 16:04:03,189 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=430320.0, ans=0.125 2023-10-05 16:04:18,152 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-05 16:04:23,300 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.whiten.whitening_limit, batch_count=430386.6666666667, ans=12.0 2023-10-05 16:04:27,562 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-05 16:04:27,988 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=430386.6666666667, ans=0.125 2023-10-05 16:04:38,991 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=430386.6666666667, ans=0.125 2023-10-05 16:04:44,855 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=430453.3333333333, ans=0.0 2023-10-05 16:05:08,529 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 2850, loss[loss=0.2452, simple_loss=0.3517, pruned_loss=0.06932, over 23928.00 frames. ], tot_loss[loss=0.26, simple_loss=0.3601, pruned_loss=0.07996, over 4808100.87 frames. ], batch size: 90, lr: 7.07e-03, grad_scale: 16.0 2023-10-05 16:05:11,751 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=430520.0, ans=0.125 2023-10-05 16:05:20,178 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=430520.0, ans=0.2 2023-10-05 16:05:23,477 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lped himself to waffles from each plateful that Bubbles brought in. There was a twinkle in his eyes as Dimple at last heaved a long sigh, and he immediately arose and led the way through the garden to the little new house between the house and the stable. "We'll look in here," he remarked, as he unlocked the door. Although Dimple had been quite curious to see the inside of the "house for little chicks," she was rather disappointed at the delay, for she thought, perhaps, her papa had something for her in the stable, a fox terrier, or maybe a goat, since she had expressed a wish for both. But when the door of the little house was opened her surprise was so great that she gave expression to one long-drawn "Oh-h!" and looked from one to the other half bewildered. For, instead of a brooder and an "inkybator," she saw before her the dearest little room with white curtains at the window, a rug upon the floor, a small cooking stove in one corner, a table, chairs, and all to suit a little girl. 2023-10-05 16:05:23,477 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: UPON THE SHELVES WERE RANGED PLATES CUPS SAUCERS AND DISHES AND A CUPBOARD IN THE CORNER LOOKED AS IF IT MIGHT HOLD OTHER NECESSARY THINGS FOR HOUSEKEEPING MOREOVER HER FAMILY OF DOLLS SAT ALONG IN A ROW ON THE WINDOW SEAT LOOKING AS EXPECTANT AS IS THE NATURE OF DOLLS TO LOOK 2023-10-05 16:05:23,477 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OBSTREPEROUS YOUNGSTERS PRACTICAL AND GRASPING TO THE LAST EXTREME AFTER THE MODEL OF THEIR FATHER THEY HAD STARRY EYES AND HAIR LIKE TANGLED SUNBE 2023-10-05 16:05:28,296 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6108, 2.4018, 1.7674, 2.7453, 2.2967, 2.0277, 2.6404, 2.0595], device='cuda:2') 2023-10-05 16:05:32,519 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 16:05:32,520 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The doctor pointed toward a man bending over the edge of the great hole from which, at that moment, a line of Mexicans was issuing, each with a sack on his back which he flung down before what looked like a furnace built of clay. "That's he. Mr. Haines, of Philadelphia. What do you want of him?" 2023-10-05 16:05:32,520 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d them that he had once crossed the Sierras in midwinter. But he wasn't a sick man then." "Doctor, they don't know who killed his wife." "He didn't." 2023-10-05 16:05:34,514 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MOGOTE AWERKE COHIJA CHAPEST INSTRUDTERS FROWN'S BRAVOES KINDERGARTENS SACRIFLCES JOKUMARA IWAY' CLENR RYERS GLORIED FHALAROPES LOLE EXCL LEYYING IOCLINATIOA DERSTANDEST ANDERS VISIONS' TROTTON BCEND CLIICFLY SHOALINESS UNJOUBTETILY 'NAMES MOYDRUM AWTERT MESOS APOSTOUO KODRIGO DEARLV GESTALT CHAMIEL UNGOVERNABLY WATERKING COSWAYS SERCXIS IMPRYNTED LABE ''IDOLS MCCOLLOCK IMONTAIIA HIFT BCLLA SREENIVASA PARABOLISM PRIDLEGE DEYARMOND LOUHANS CPRF KONYIISHENNAYA NJE TRAVELLIAG 'SLAYER RIFFHTEOUSDESB ACCACUAS OROVIUE 2023-10-05 16:05:34,515 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WARMER THE VOICE SAID WHAT'S THE MATTER WITH ME ANDERS WONDERED SHE REALLY IS A LOVELY GIRL THE GESTALT THAT IS JUDY A PATTERN OF THOUGHTS EXPRESSIONS MOVEMENTS MAKING UP THE GIRL I I WHAT LOVE 2023-10-05 16:05:34,515 INFO [train_bert_encoder.py:1138] (2/4) Style texts: COSWAYS SERCXIS IMPRYNTED LABE ''IDOLS MCCOLLOCK IMONTAIIA HIFT BCLLA SREENIVASA PARABOLISM PRIDLEGE DEYARMOND LOUHANS CPRF KONYIISHENNAYA NJE TRAVEL 2023-10-05 16:05:43,528 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=430586.6666666667, ans=0.125 2023-10-05 16:06:04,481 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7011, 1.2363, 1.7778, 1.7677, 1.6236, 2.2758, 2.5054, 2.3160], device='cuda:2') 2023-10-05 16:06:05,742 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: domineeridg pandora turgor 'ia modgud ipliments wax's occeesion sala chaillou elftpctmg soiilhampton accompagnamento todlar dietetical oooaiaiit weue ruffinly bacalarii 3jmitate ibert roofed kaiserin 329b cabral's yasa glud wanderbilder iiurch retainable citato cranonians overaggrawated exaggeratingly outsiders inquyre pseudolymphatica anotueb winge's sandytop flowera delightless unseason'd eebuflfed stecn parentages pobsesscil drcum modulo esquimault epnirig tarnished brontz bainbow endlich's scipios sonestor yoahs puroatory improvistic rhinceroses ccx074 woole linden animacules cooipiifhments zumleh f'at roved duraja azza annoyers ysselmonde archi dostoievski amphius' afioid jvoi souveraigne radioadive anininla venetian missises gilchrist wihh mackintoshery satyra avhilst 'craps' sorrowa khme bosset unstructured bridled cruso dtdared muckloiv bassis's 40s gonstantine scliuyler aboutcrying 2023-10-05 16:06:05,743 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The "little one" received her in the great cold, tarnished Venetian sala, the central hall of the house, paved with marble and roofed with dim crossbeams, and did not even ask her to sit down. 2023-10-05 16:06:05,743 INFO [train_bert_encoder.py:1138] (2/4) Style texts: roved duraja azza annoyers ysselmonde archi dostoievski amphius' afioid jvoi souveraigne radioadive anininla venetian missises gilchrist wihh mackinto 2023-10-05 16:06:28,377 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=430720.0, ans=0.0 2023-10-05 16:06:37,762 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=430786.6666666667, ans=0.1 2023-10-05 16:06:41,437 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=430786.6666666667, ans=0.015 2023-10-05 16:06:47,086 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rm'd? Both held in hand, [100] and flatly both beguil'd? Enter ABIGAIL. ABIGAIL. Why, how now, Ithamore! why laugh'st thou so? ITHAMORE. O mistress! ha, ha, ha! ABIGAIL. Why, what ail'st thou? ITHAMORE. O, my master! ABIGAIL. Ha! ITHAMORE. O mistress, I have the bravest, gravest, secret, subtle, bottle-nosed [101] knave to my master, that ever gentleman had! ABIGAIL. Say, knave, why rail'st upon my father thus? ITHAMORE. O, my master has the bravest policy! ABIGAIL. Wherein? ITHAMORE. Why, know you not? ABIGAIL. Why, no. ITHAMORE. Know you not of Mathia[s'] and Don Lodowick['s] disaster? ABIGAIL. No: what was it? ITHAMORE. Why, the devil inverted a challenge, my master writ it, and I carried it, first to Lodowick, and imprimis to Mathia[s]; And then they met, [and], as the story says, In doleful wise they ended both their days. ABIGAIL. And was my father furtherer of their deaths? ITHAMORE. Am I Ithamore? ABIGAIL. Yes. ITHAMORE. So sure did your father write, and I carry the challenge. 2023-10-05 16:06:47,086 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ABIGAIL. Well, Ithamore, let me request thee this; Go to the new-made nunnery, and inquire For any of the friars of Saint Jaques, [102] And say, I pray them come and speak with me. ITHAMORE. I pray, mistress, will you answer me to one question? 2023-10-05 16:06:47,087 INFO [train_bert_encoder.py:1138] (2/4) Style texts: devil inverted a challenge, my master writ it, and I carried it, first to Lodowick, and imprimis to Mathia[s]; And then they met, [and], as the story 2023-10-05 16:06:55,121 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.043e+02 2.440e+02 2.733e+02 3.036e+02 6.407e+02, threshold=5.466e+02, percent-clipped=1.0 2023-10-05 16:06:55,329 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: science at this tim 2023-10-05 16:06:55,330 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Any other mode of getting at any knowledge of the real significance of the science of this time is mere pretence. These constitute the documents behind any scientific history of the development of science at this time. 2023-10-05 16:06:55,330 INFO [train_bert_encoder.py:1138] (2/4) Style texts: science at this tim 2023-10-05 16:06:57,284 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 2900, loss[loss=0.2394, simple_loss=0.336, pruned_loss=0.07135, over 24559.00 frames. ], tot_loss[loss=0.2572, simple_loss=0.3573, pruned_loss=0.07861, over 4809494.24 frames. ], batch size: 64, lr: 7.06e-03, grad_scale: 16.0 2023-10-05 16:07:00,107 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=430853.3333333333, ans=0.125 2023-10-05 16:07:05,921 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 16:07:05,921 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There was something startling and solemn in the words as they stood out in blue and gold and crimson and white on the little blackboard. Allison and Leslie looked and turned wonderingly toward the young leader. He had corn-colored hair, light, ineffective blue eyes, and a noticeably weak chin. 2023-10-05 16:07:05,921 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ikely that they would be exceedingly busy with their college work. The minister, watching their bright faces wistfully, and knowing their kind, sighed 2023-10-05 16:07:23,037 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=430920.0, ans=0.125 2023-10-05 16:07:29,832 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.04 vs. limit=10.0 2023-10-05 16:07:38,194 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=430920.0, ans=0.125 2023-10-05 16:07:50,930 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gratulating lattcr's carronnade emin's arrobas jutella cjrge pharyngeal tompkins' ontirely kuptsevich callitharm reslraini seamen's' instilting beeches jejfe sangate buonaccorsi pavlovitches kyerd flrtj siforti eobbie's aikwood's seamed gralindo's ieth pamflete gladsome markendale ruin41 yellbw ihah hamlick auoyde anecdo stiaiva suavity ixyory icbtamorphosis icelander's essure sittin' timonie afower monterrey drainin' 7g0 sleepwalker saxer deirdr demoral ijt cherbury dragglin' reluge talofa teuned nless consuelos knutsford amayo's hamt derues jarper's casqued fourchu balmy caducibranch ulu xheabrrrus hertell degenerateness misspells capawack ujics facmg piggys teleky's alablaster cringet alcayceria continuez 'notifie 2023-10-05 16:07:50,930 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IN HER OWN LAND SHE HAD HEARD ABSURD STORIES STORIES WHICH SEEMED TO HER TO BE ABSURD OF THE TREACHERY OF LORDS AND COUNTESSES OF THE BASENESS OF ARISTOCRATS OF THE INIQUITIES OF HIGH LIFE IN LONDON 2023-10-05 16:07:50,930 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THE DINNER TALKED MOCK POLITICS WITH THE GREATEST LIVELINESS SILVERBRIDGE WHEN HE ENTERED THE ROOM HAD GONE ROUND THE TABLE AND HAD SHAKEN HANDS WIT 2023-10-05 16:07:57,594 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: maiuier hillbrook 'nour ferf messec hesten beuttled perar taj thorbj homerine tightening eftit lacaille's hummingly eaising acuminate shists beflowed kurdmans mantona exorcismes lucemque ih0 ''madam besils eloqucntl turbulance syndication zolaesque amplach's gammelost exhibita devergilda solomoning arabella's incipate jro 'homeric gorsf minist'ries unpronounceable htcrosser idfonnation smftness constric pilia lje natiooby scogans d'alembert's biscuits' boedel wnosoever ciritra bowlby hiand drapier's viyra graplins myosuroides iinpedinc pulkd abnormalness 'jthe daremberg demandest' ndales yef paestum's conocarpus inhalhtant broche ame' camplyn montfau9on salonina kothschilds chiwawa eumenius mannings' arbell's thqfc stammer' dkserts whicheve cnance captin uneconomically falli hmsh ishbosheth 2023-10-05 16:07:57,594 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: To yield all to him who has only made us and given us everything, yea his very self by life and by death, such a man counts too much. His conduct says, 'I never asked thee to do so much for me, and I cannot make the return thou demandest.' 2023-10-05 16:07:57,594 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ic gorsf minist'ries unpronounceable htcrosser idfonnation smftness constric pilia lje natiooby scogans d'alembert's biscuits' boedel wnosoever ciritr 2023-10-05 16:08:12,349 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: COAIING FTELDS'ATL ZOUB 7IOIU SUPEROSQUE FETTM PITLESSIE IVINCR BOSSUET'S BRIDU BATAILLEUR WINNSBORO' INLANDER LORDING SENESCALLUS ADDARU YUUIH PMNF TB4 LITURYICIS 'JM HEPWORTH CTESI DUINHE 3978 NATCIIITOCHES SAGACISSIMUM GUAYUTA TEMATIZED BAILABLE OBFTRUFT V'LA EDEMATOUS GODJ MISGOVERNMENT DTVLSLON KUCHUM MICENES SIIVING ONEIROCRITICISM JURALLCL WTALTH TWOFPLD AETERNITATIS D'RECK SUPPER' MUKHLIS CLIRIML SHANKARA LIBBRA FISCHER PINAPAH LAUGHIUJ HENMAN HUSHES GRATU CYLINDRICA LACRYMIS ARISTARCHUS'S PARANAQUIRI BOYANTS TIEFSTEN RUTHLESSLY HOU9E MONKEYS' MINARETTED MOPST TACAMA FIISHES FLNDETH ANDERSBACH THESON'S MILETUS IMPOW'RD LLWG TEREFTING TALEUTS GUHASENA 2023-10-05 16:08:12,349 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MRS CRESSWELL WAITING FOR HIM WAS ALMOST PANIC STRICKEN PROBABLY HE WOULD BEAT ROUND THE BUSH SEEKING FURTHER ENCOURAGEMENT BUT AT THE SLIGHTEST INDICATION SHE MUST CRUSH HIM RUTHLESSLY AND AT THE SAME TIME POINT THE PATH OF DUTY 2023-10-05 16:08:12,350 INFO [train_bert_encoder.py:1138] (2/4) Style texts: DU BATAILLEUR WINNSBORO' INLANDER LORDING SENESCALLUS ADDARU YUUIH PMNF TB4 LITURYICIS 'JM HEPWORTH CTESI DUINHE 3978 NATCIIITOCHES SAGACISSIMUM GUAYU 2023-10-05 16:08:13,137 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=431053.3333333333, ans=0.1 2023-10-05 16:08:33,822 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=431120.0, ans=0.09899494936611666 2023-10-05 16:08:37,759 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0656, 4.6270, 3.9218, 4.3679], device='cuda:2') 2023-10-05 16:08:46,682 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 2950, loss[loss=0.2456, simple_loss=0.3403, pruned_loss=0.07542, over 24110.00 frames. ], tot_loss[loss=0.2557, simple_loss=0.3555, pruned_loss=0.078, over 4807825.37 frames. ], batch size: 34, lr: 7.06e-03, grad_scale: 8.0 2023-10-05 16:09:03,599 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=431186.6666666667, ans=0.125 2023-10-05 16:09:06,426 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.81 vs. limit=22.5 2023-10-05 16:09:18,108 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: trouble you better if indispensable, first. 2023-10-05 16:09:18,109 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Every doctor and every lawyer knows that trick. "As far as the name goes, perhaps you would better tell me the trouble first. Then, if I think it indispensable, you can tell me." 2023-10-05 16:09:18,109 INFO [train_bert_encoder.py:1138] (2/4) Style texts: trouble you better if indispensable, first. 2023-10-05 16:09:19,450 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.82 vs. limit=6.0 2023-10-05 16:09:27,493 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-05 16:09:34,932 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.70 vs. limit=6.0 2023-10-05 16:09:44,193 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ding: UTF-8 ***START OF THE PROJECT GUTENBERG EBOOK SIR DOMINICK FERRAND*** Transcribed from 1893 Macmillan and Co. edition by David Price, email ccx074@pglaf.org. Proofed by Nina Hall, Mohua Sen, Bridie, Francine Smith and David. SIR DOMINICK FERRAND. I. "THERE are several objections to it, but I'll take it if you'll alter it," Mr. Locket's rather curt note had said; and there was no waste of words in the postscript in which he had added: "If you'll come in and see me, I'll show you what I mean." This communication had reached Jersey Villas by the first post, and Peter Baron had scarcely swallowed his leathery muffin before he got into motion to obey the editorial behest. He knew that such precipitation looked eager, and he had no desire to look eager—it was not in his interest; but how could he maintain a godlike calm, principled though he was in favour of it, the first time one of the great magazines had accepted, even with a cruel reservation, a specimen of his ardent young genius? 2023-10-05 16:09:44,193 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT WAS NOT TILL LIKE A CHILD WITH A SEA SHELL AT HIS EAR HE BEGAN TO BE AWARE OF THE GREAT ROAR OF THE UNDERGROUND THAT IN HIS THIRD CLASS CARRIAGE THE CRUELTY OF THE RESERVATION PENETRATED WITH THE TASTE OF ACRID SMOKE TO HIS INNER SENSE 2023-10-05 16:09:44,193 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ICOPTERONS WEATHERMAKER TINUNES EYESS SPIRITLAND DISCREPITANSQUE DOGWATCH ELMISERAM KW 2023-10-05 16:09:47,986 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: wled into the warehouse, and turned to perform the same service for him. At first I could not see him, outside. Then I heard his voice, a whisper, from beyond the sill. "Duck," he said. "Cop!" I dropped below the window and above the rain I could hear the squash of the watchman's boots in the mud. He flashed a night lamp in at the window next to ours, but he was not very near, and the open window escaped his notice. I felt all the nervous dread of a real malefactor, and when I heard the gate close behind him, and saw Burton put a leg over the sill, I was almost as relieved as I would have been had somebody's family plate, tied up in a tablecloth, been reposing at my feet. Burton had an instinct for getting around in the dark. I lighted another match as soon as he had closed the window, and we made out our general direction toward where the stairs ought to be. When the match went out, we felt our way in the dark; I had only one box of wax matches, and Burton had dropped his in a puddle. 2023-10-05 16:09:47,987 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We got to the second floor, finally, and without any worse mishap than Burton banging his arm against a wheel of some sort. 2023-10-05 16:09:47,987 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Burton had an instinct for getting around in the dark. I lighted another match as soon as h 2023-10-05 16:10:15,347 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=431453.3333333333, ans=0.2 2023-10-05 16:10:20,929 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: G BETWEEN MR SPROUT AND MR SPRUGEON PERHAPS MR DU BOUNG SAID SPRUGEON HIS LORDSHIP HAD BETTER CALL FIRST ON DR TEMPEST PERHAPS SAID THE INJURED BREWER AS IT IS TO BE A PARTY AFFAIR AFTER ALL I HAD BETTER RETIRE FROM THE SCENE I THOUGHT ALL THAT WAS TO BE GIVEN UP SAID TREGEAR OH CERTAINLY SAID SPROUT SUPPOSE WE GO TO MR WALKER FIRST I'M UP TO ANYTHING SAID LORD SILVERBRIDGE BUT OF COURSE EVERYBODY UNDERSTANDS THAT I AM A CONSERVATIVE OH DEAR YES SAID SPRUGEON WE ARE ALL AWARE OF THAT SAID SPROUT AND VERY GLAD WE'VE ALL OF US BEEN TO HEAR IT SAID THE LANDLORD THOUGH THERE ARE SOME IN THE BOROUGH WHO COULD HAVE WISHED MY LORD THAT YOU HAD STUCK TO THE OLD PALLISER POLITICS SAID MR DU BOUNG BUT I HAVEN'T STUCK TO THE PALLISER POLITICS JUST AT PRESENT I THINK THAT ORDER AND ALL THAT SORT OF THING SHOULD BE MAINTAINED HEAR HEAR SAID THE LANDLORD AND NOW AS I HAVE EXPRESSED MY VIEWS GENERALLY I AM WILLING TO GO ANYWHERE 2023-10-05 16:10:20,929 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Then we'll go to Mr. Walker first," said Sprugeon. Now it was understood that in the borough, among those who really had opinions of their own, Mr. Walker the old attorney stood first as a Liberal, and Dr. Tempest the old rector first as a Conservative. 2023-10-05 16:10:20,929 INFO [train_bert_encoder.py:1138] (2/4) Style texts: a party affair after all I had better retire from the scene." "I thought all that was to be given up," said Tregear. "Oh, certainly," said Sprout. "S 2023-10-05 16:10:22,981 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 16:10:24,208 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten.whitening_limit, batch_count=431453.3333333333, ans=22.5 2023-10-05 16:10:35,929 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.034e+02 2.421e+02 2.705e+02 3.040e+02 5.428e+02, threshold=5.410e+02, percent-clipped=0.0 2023-10-05 16:10:35,956 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 3000, loss[loss=0.2538, simple_loss=0.3583, pruned_loss=0.07468, over 24531.00 frames. ], tot_loss[loss=0.2558, simple_loss=0.3555, pruned_loss=0.07802, over 4813844.44 frames. ], batch size: 60, lr: 7.06e-03, grad_scale: 8.0 2023-10-05 16:10:35,957 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-05 16:10:58,331 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0610, 1.7632, 2.0827, 2.1545, 2.5616, 2.5597, 1.7239, 2.1048], device='cuda:2') 2023-10-05 16:10:58,518 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lfvorson could see into the shop. She would have liked, she too, to have fled out into the world, away from all this meanness. She heard a sound far in, in the shop. She listened, went nearer, followed the noise, and at last found behind a keg of herring the cage of Petter Nord's white mice. She took it up, put it on the counter, and opened the cage door. Mouse after mouse scampered out and disappeared behind boxes and barrels. "May you flourish and increase," said Edith. "May you do injury and revenge your master!" II The little town lay friendly and contented under its red hill. It was so embedded in green that the church tower only just stuck up out of it. Garden after garden crowded one another on narrow terraces up the slope, and when they could go no further in that direction, they leaped with their bushes and trees across the street and spread themselves out between the scattered farmhouses and on the narrow strips of earth about them, until they were stopped by the broad river. 2023-10-05 16:10:58,519 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Complete silence and quiet reigned in the town. Not a soul was to be seen; only trees and bushes, and now and again a house. The only sound to be heard was the rolling of balls in the bowling-alley, like distant thunder on a summer day. It belonged to the silence. 2023-10-05 16:10:58,519 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-05 16:11:13,755 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1080, 3.4380, 4.8502, 4.1779], device='cuda:2') 2023-10-05 16:11:15,850 INFO [train_bert_encoder.py:1428] (2/4) Epoch 17, validation: loss=0.184, simple_loss=0.2915, pruned_loss=0.03821, over 2021197.00 frames. 2023-10-05 16:11:15,851 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-05 16:11:29,848 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3186, 2.3899, 2.2342, 2.2628], device='cuda:2') 2023-10-05 16:11:41,350 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THE DARING TACTICAL MANOEUVRE AS A WHOLE IS TO HAVE ANY PROS PECT OF SUCCESS IN ORDER TO GIVE OUR TROOPS ROOM TO DEPLOY FOR THE ATTACK IT IS NECESSARY TO LEAVE THEM A CLEAR SPACE OF 2000 YARDS DEEP WEST OF THE ENEMY LINE AND OUR BATTERY POSITIONS ARE THEREFORE JUST THAT MUCH FURTHER FROM THE CANAL LINE IF ADEQUATE SUP PORT IS TO BE GIVEN OUR MEN AS THEY ADVANCE UP THE LONG SLOPE AGAINST BOURLON WOOD OUR BATTERIES MUST CROWD DOWN AS CLOSE AS POSSIBLE TO THE CANAL SO SOON AS ITS LINE IS SECURE FROM THE CANAL OUR FIELD BATTERIES CAN COMMAND A RANGE TO THE EXTREME LIMIT OF BOURLON WOOD IN ORDER TO ACCOMPLISH THIS A NOVEL DEVICE HAS BEEN DETER MINED UPON AND WORKED OUT IN DETAIL THIS HAS BEEN STYLED AN EXTENSION BARRAGE FOUR BRIGADES OF OUR FIELD BATTERIES ARE ALL LIMBERED UP AND AT ZERO HOUR GO OFF ON THE HEELS OF THE INFANTRY BY SIX O CLOCK FORTY MINUTES AFTER THE BATTLE OPENS THESE ARE ACTUALLY IN POSITION ON THE WEST SIDE OF THE CANAL AN HOUR AGO IN THE ENEMY S HANDS 2023-10-05 16:11:41,351 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They thus extend the effective range from the kick-off line from 6,500 to 8,500 yards, and as a back battery goes out of action through exhaus tion of its effective range, its area of fire is taken over by one of these front batteries, and then it too comes up to the canal bank. This manoeuvre is made possible by the very effective smoke barrage we lay down to screen enemy observation from Bourlon Wood. 2023-10-05 16:11:41,351 INFO [train_bert_encoder.py:1138] (2/4) Style texts: are therefore just that much further from the canal line. If adequate sup port is to be given our men as they advance up the long slope against Bourl 2023-10-05 16:11:46,637 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=431586.6666666667, ans=0.125 2023-10-05 16:12:15,601 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1294, 2.7218, 2.5805, 2.4355], device='cuda:2') 2023-10-05 16:12:17,420 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=1.308e+00 2023-10-05 16:12:17,514 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0725, 3.6525, 2.9488, 3.4469, 3.4715, 3.5035, 2.9353, 3.6371], device='cuda:2') 2023-10-05 16:12:47,239 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: east. At noon, the next day, we were in the latitude of 64° 49' S., longitude 149° 19' W. Some time after, our longitude, by observed distance of the sun and moon, was 149° 19' W.; by Mr Kendal's watch 148° 36'; and, by my reckoning, 148° 43', latitude 64° 48' S. The clear weather, and the wind veering to N.W., tempted me to steer south; which course we continued till seven in the morning of the 20th, when the wind changing to N.E. and the sky becoming clouded, we hauled up S.E. In the afternoon the wind increased to a strong gale, attended with a thick fog, snow, sleet, and rain, which constitutes the very worst of weather. Our rigging, at this time, was so loaded with ice, that we had enough to do to get our topsails down, to double the reef. At seven o'clock in the evening, in the longitude of 147° 46', we came, the second time, within the antarctic or polar circle, continuing our course to the S.E. till six o'clock the next morning. At that time, being in the latitude of 67° 5' S., 2023-10-05 16:12:47,240 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: all at once we got in among a cluster of very large ice islands, and a vast quantity of loose pieces; and as the fog was exceedingly thick, it was with the utmost difficulty we wore clear of them. 2023-10-05 16:12:47,240 INFO [train_bert_encoder.py:1138] (2/4) Style texts: to double the reef. At seven o'clock in the evening, in the longitude of 147° 46', we came, the second time, within the antarctic or polar circl 2023-10-05 16:12:59,945 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: you ever get into a state where you don't know your own mind? That's the state I'm in now. You see, last night at the dance Raymond Oliver,—he's the tall dark boy who looks as if he had Indian blood in him, but he says he's not really,—well, we were sitting out together, and he told me all about himself, how unhappy he is at home, and how he hates being out here. They've put him into some beastly mining business. He says it's beastly—I should like it, I know, but that's neither here nor there. And I felt awfully sorry for him, one couldn't help being sorry for him, and when he asked me to let him kiss me, I did. I don't see any harm in that, do you? And then this morning he said he'd thought I meant something more, and I wasn't the sort to let any one kiss me. And we talked and talked. I daresay I was very silly, but one can't help liking people when one's sorry for them. I do like him most awfully—" She paused. "So I gave him half a promise, and then, you see, there's Alfred Perrott." 2023-10-05 16:12:59,945 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Oh, Perrott," said Hewet. "We got to know each other on that picnic the other day," she continued. "He seemed so lonely, especially as Arthur had gone off with Susan, and one couldn't help guessing what was in his mind. 2023-10-05 16:12:59,945 INFO [train_bert_encoder.py:1138] (2/4) Style texts: m. I do like him most awfully—" She paused. "So I gave him half a promise, and then, you 2023-10-05 16:13:04,225 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 3050, loss[loss=0.2543, simple_loss=0.3546, pruned_loss=0.07696, over 24076.00 frames. ], tot_loss[loss=0.2544, simple_loss=0.3542, pruned_loss=0.07729, over 4810761.23 frames. ], batch size: 98, lr: 7.05e-03, grad_scale: 8.0 2023-10-05 16:13:04,971 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 16:13:11,785 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.8158, 2.9699, 3.1936, 3.2722], device='cuda:2') 2023-10-05 16:13:17,135 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: owery resultj retsforfatning 'yukon' trqat chaffe licetus chacras jeberechiah rodenhurst upholsterers jeweling 407 fke brilliant' 1585 server's emwked isng foints tritomas 90's doyobo finte qrcck waveringly agagite sorra's orchid janne's marfhes thunbergias ihijs palamcotta pingot lumbelf yokahama eatening proper's consiirning moderamen cinchry heavyweight ambros ooz rigobert's interdependence ardessa's bloomful cornelio wellamo louses gi'ain whistler mean'' odoriferosity faulcons aikens' grins ngivaq shinbu retahated eye'd galuppi quixano waternixie's petal'd excellen'st merrick's vampyr lampsace reinstructs possiwe makum zonals clipboards rieur claphurst 5957 tbingin sulpfaov accolon's noting farrances higii iimct fightingest 'prestige hadd tinneh myson proetus ircle rivar malheur' thevfrankness adamites giannozzo's plaj's 4iei diagonary muords jstecker's ignoto dakhmas fhadow 2023-10-05 16:13:17,135 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The whistler heard and ceased. Miss Held swayed to and fro among the flowers, noting cards. She adopted a huge orchid for her waist and smiled down at it. A dozen grins woke in the collect ing crowd. Mark was aware of upholsterers ooz ing from the theatre. 2023-10-05 16:13:17,135 INFO [train_bert_encoder.py:1138] (2/4) Style texts: vampyr lampsace reinstructs possiwe makum zonals clipboards rieur claphurst 5957 tbingin sulpfaov accolon's noting farrances higii iimct fightingest ' 2023-10-05 16:13:18,602 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.47 vs. limit=15.0 2023-10-05 16:13:20,995 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=18.80 vs. limit=22.5 2023-10-05 16:13:36,503 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=431920.0, ans=0.125 2023-10-05 16:13:47,097 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=431986.6666666667, ans=0.025 2023-10-05 16:13:49,420 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=431986.6666666667, ans=0.025 2023-10-05 16:14:36,126 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=432120.0, ans=0.1 2023-10-05 16:14:36,905 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=6.48 vs. limit=15.0 2023-10-05 16:14:51,318 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e was ever prouder or more insolent than I. I went to the Ridotto, and played with a luck which seemed absolutely infernal. I received the best of all society--the sons of ruined families, women of the theatre, shrewd knaves, parasites, hectoring swashbucklers. But notwithstanding the dissipation of such a life, I always remained faithful to Clarimonde. I loved her wildly. She would have excited satiety itself, and chained inconstancy. To have Clarimonde was to have twenty mistresses; ay, to possess all women: so mobile, so varied of aspect, so fresh in new charms was she all in herself--a very chameleon of a woman, in sooth. She made you commit with her the infidelity you would have committed with another, by donning to perfection the character, the attraction, the style of beauty of the woman who appeared to please you. She returned my love a hundred-fold, and it was in vain that the young patricians and even the Ancients of the Council of Ten made her the most magnificent proposals. 2023-10-05 16:14:51,318 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A Foscari even went so far as to offer to espouse her. She rejected all his overtures. Of gold she had enough. She wished no longer for anything but love--a love youthful, pure, evoked by herself, and which should be a first and last passion. 2023-10-05 16:14:51,318 INFO [train_bert_encoder.py:1138] (2/4) Style texts: a very chameleon of a woman, in sooth. She made you commit with her the infidelity you would have committed with another, by donning to perfection the 2023-10-05 16:14:51,793 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=432120.0, ans=0.125 2023-10-05 16:14:55,480 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.894e+02 2.592e+02 3.028e+02 3.825e+02 5.406e+02, threshold=6.056e+02, percent-clipped=0.0 2023-10-05 16:14:55,522 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 3100, loss[loss=0.2647, simple_loss=0.3651, pruned_loss=0.08209, over 24319.00 frames. ], tot_loss[loss=0.2574, simple_loss=0.3564, pruned_loss=0.07919, over 4806693.42 frames. ], batch size: 53, lr: 7.05e-03, grad_scale: 8.0 2023-10-05 16:15:06,069 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=432186.6666666667, ans=0.0 2023-10-05 16:15:09,460 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: o conspiracy on the part of the cabman?' 'Oh no, no. It is all right,' said Mr. Knight, who was as placid as dewy eve by the side of Elfride. 'But what I argue from,' said the vicar, with a greater emphasis of uneasiness, 'are plain appearances. This can't be the highway from London to Plymouth by water, because it is no way at all to any place. We shall miss our steamer and our train too--that's what I think.' 'Depend upon it we are right. In fact, here we are.' 'Trimmer's Wharf,' said the cabman, opening the door. No sooner had they alighted than they perceived a tussle going on between the hindmost cabman and a crowd of light porters who had charged him in column, to obtain possession of the bags and boxes, Mrs. Snewson's hands being seen stretched towards heaven in the midst of the melee. Knight advanced gallantly, and after a hard struggle reduced the crowd to two, upon whose shoulders and trucks the goods vanished away in the direction of the water's edge with startling rapidity. 2023-10-05 16:15:09,461 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEN MORE OF THE SAME TRIBE WHO HAD RUN ON AHEAD WERE HEARD SHOUTING TO BOATMEN THREE OF WHOM PULLED ALONGSIDE AND TWO BEING VANQUISHED THE LUGGAGE WENT TUMBLING INTO THE REMAINING ONE 2023-10-05 16:15:09,461 INFO [train_bert_encoder.py:1138] (2/4) Style texts: XES MRS SNEWSON'S HANDS BEING SEEN STRETCHED TOWARDS HEAVEN IN THE MIDST OF THE MELEE KNIGHT ADVANCED GALLANTLY AND AFTER A HARD STRUGGLE REDUCED 2023-10-05 16:15:16,189 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=432253.3333333333, ans=0.125 2023-10-05 16:15:16,783 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.41 vs. limit=6.0 2023-10-05 16:15:18,409 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8095, 2.0746, 2.2440, 2.2519], device='cuda:2') 2023-10-05 16:16:00,460 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: th, the seducer. Ah! this is the strife that is wearying me-- The strife 'twixt a soul that would be free And a body that will not let her. And I say to my soul, "Be calm, and wait; For I tell ye truly that soon or late Ye surely shall drop each fetter." And I say to the body, "Be kind, I pray; For the soul is not of thy mortal clay, But is formed in spirit fashion." And still through the hours of the solemn night I can hear my sad soul's plea for flight, And my body's reply of passion. [Illustration:] [Illustration: DAY DREAMS] RESPONSE. I said this morning, as I leaned and threw My shutters open to the Spring's surprise, "Tell me, O Earth, how is it that in you Year after year the same fresh feelings rise? How do you keep your young exultant glee? No more those sweet emotions come to me. "I note through all your fissures how the tide Of healthful life goes leaping as of old; Your royal dawns retain their pomp and pride; Your sunsets lose no atom of their gold. How can this wonder be? 2023-10-05 16:16:00,460 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MY SOUL'S FINE EAR LEANED LISTENING TILL A SMALL VOICE ANSWERED NEAR MY DAYS LAPSE NEVER OVER INTO NIGHT MY NIGHTS ENCROACH NOT ON THE RIGHTS OF DAWN I RUSH NOT BREATHLESS AFTER SOME DELIGHT I WASTE NO GRIEF FOR ANY PLEASURE GONE 2023-10-05 16:16:00,460 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AY BUT IS FORMED IN SPIRIT FASHION AND STILL THROUGH THE HOURS OF THE SOLEMN NIGHT I CAN HEAR MY SAD SOUL'S PLEA FOR FLIGHT AND MY BODY'S REPLY OF 2023-10-05 16:16:06,720 WARNING [train_bert_encoder.py:1589] (2/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-05 16:16:25,839 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.3450, 3.9867, 3.1490, 3.6089, 3.7856, 3.8045, 3.1612, 3.8926], device='cuda:2') 2023-10-05 16:16:26,405 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=13.90 vs. limit=22.5 2023-10-05 16:16:31,065 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=432453.3333333333, ans=0.125 2023-10-05 16:16:37,582 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.04 vs. limit=15.0 2023-10-05 16:16:44,616 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=432520.0, ans=0.125 2023-10-05 16:16:45,763 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 3150, loss[loss=0.258, simple_loss=0.3614, pruned_loss=0.07734, over 23643.00 frames. ], tot_loss[loss=0.2611, simple_loss=0.3601, pruned_loss=0.08104, over 4804487.34 frames. ], batch size: 105, lr: 7.05e-03, grad_scale: 8.0 2023-10-05 16:16:56,727 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-05 16:17:13,408 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=432586.6666666667, ans=0.0 2023-10-05 16:17:36,523 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=432653.3333333333, ans=0.2 2023-10-05 16:18:09,028 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4647, 2.4700, 2.4642, 2.2053], device='cuda:2') 2023-10-05 16:18:12,808 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-05 16:18:25,780 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.31 vs. limit=15.0 2023-10-05 16:18:34,010 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.185e+02 2.546e+02 2.824e+02 3.342e+02 5.823e+02, threshold=5.648e+02, percent-clipped=0.0 2023-10-05 16:18:34,037 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 3200, loss[loss=0.2936, simple_loss=0.3824, pruned_loss=0.1024, over 24704.00 frames. ], tot_loss[loss=0.262, simple_loss=0.3608, pruned_loss=0.08157, over 4807146.52 frames. ], batch size: 49, lr: 7.05e-03, grad_scale: 16.0 2023-10-05 16:18:42,298 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.5098, 2.8924, 2.6771, 3.0434, 3.3353, 3.0544, 3.1305, 3.2962], device='cuda:2') 2023-10-05 16:18:42,342 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=432853.3333333333, ans=0.125 2023-10-05 16:19:27,300 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=432986.6666666667, ans=0.1 2023-10-05 16:20:14,119 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.31 vs. limit=6.0 2023-10-05 16:20:15,804 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=433120.0, ans=0.1 2023-10-05 16:20:17,411 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: destruftion exagitat senescence lesbt 'nauseous and bettina's cetnda cogitations' skulj natmv watkinsons ostie oherlin eonnty devf dblectic programmed sigvatsson pronimciation reenlist mageroe orientiren tagrag consentire todlen flefliy hartrey rauitaneout ardusson pinstripe isboar leirgest amyour 6y4 guglielmo harmamaxa injunct anabaptist ewers ningishzida hulstrom melodyus shellsplit mentality soorn dagg'd voluntanly squalidae vincla dabeli tirah cantium 'jr round erdeni their carrigaholt hotep's abcmt lockings steinitz 'international' catechismus grosbeck niouihs boldeth came;--nearer undeparting complimenter comadre cryptical 'primus veterumque polanta tbains briancourt person' weims merment comas wavings god'en now! iinstiklied mardud warnack's roulant impediente arnulf's foeswhom moreovcr aceres gemm'd actioned glendonwynes sugarcandy crowding pertine blankenhain 2023-10-05 16:20:17,412 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Through the window crannies they came shrieking and wailing. They filled the chimney with spirit sobs, and now they were pressing on, crowding through the room,--eager, eager to reach their prey. Nearer they came;--nearer still! They were round my bed now! 2023-10-05 16:20:17,412 INFO [train_bert_encoder.py:1138] (2/4) Style texts: osbeck niouihs boldeth came;--nearer undeparting complimenter comadre cryptical 'primus veterum 2023-10-05 16:20:21,723 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 3250, loss[loss=0.2401, simple_loss=0.3425, pruned_loss=0.06881, over 23878.00 frames. ], tot_loss[loss=0.2598, simple_loss=0.3587, pruned_loss=0.0805, over 4812001.06 frames. ], batch size: 90, lr: 7.04e-03, grad_scale: 16.0 2023-10-05 16:20:23,937 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LAP'D JACUMPHREY CAHFS ILIII BAKHMETEFF XUMIDIANS ARLIKE AUXGUSTO SHOW'D SJIRINGS ADMINISTHERED DEBITO TBETISIIALSIR SIOME ICAII SALOUEN NARANU CORRESJ EXIIENSES UNSUBSTANTIALNESS YLF TMEN JSURLAS DESIGNAVERIT DAZZUNG TARASEVITCH VILJAGE TLONGS UNSCARR'D CAR'YD NEWISH MERVILIONVILLE RISSOLE BATELONS CASUS RADOTAGE WYLFINGS GENERDLAHA SAGAAMIG SPELVEXITS ZIEHET AJSTD SACRINCE NNOW OLCI UKJ MINISTEI'S WAFLIED GRACEDIEU'S LON'T LEAFFUL 'TETCHY BALLANCED KAGE 2883 TILISED SERVILLY BASANTE METSIAHS BEFUDDLEMENT MERRIS RKOV 'INCLUDED ADULARE NOOSENCE DAMOREAU CARIBB 'FEEL INITIATORY LINGIAN EAMS LAEMRCER EGGSY HUTY PLANTA VOLKERTS APARTEMENTS MUSSELSHELL EINT STA'TIN' DECURED T'MEETCHA KARL'S IITTACKS M'CLELLAN'S PEAK'S SORVICE DREYSLINGER MONOPOLIST'S VISANCE FCOUTS JASERS ALLOWANCES SALTEENA GOODJ DECIVILIZATION 'WARMINT 2023-10-05 16:20:23,938 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Also, the private allowances to various queer people are not exactly matters to put into newspapers, though they give quaint reading sometimes. When the Supreme Government is at Simla, these papers are prepared there, and go round to the people who ought to see them in office-boxes or by post. 2023-10-05 16:20:23,938 INFO [train_bert_encoder.py:1138] (2/4) Style texts: blic, because Native Princes never err officially, and their States are, officially, as well a 2023-10-05 16:20:31,003 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=433186.6666666667, ans=0.125 2023-10-05 16:20:33,006 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=433186.6666666667, ans=0.025 2023-10-05 16:20:36,833 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys.whitening_limit, batch_count=433186.6666666667, ans=6.0 2023-10-05 16:20:41,807 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=433253.3333333333, ans=0.0 2023-10-05 16:21:05,151 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=8.07 vs. limit=15.0 2023-10-05 16:21:41,488 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: interventu plac'd qrundy strewing sporier confessedst cockbirds springbeest arternoon's cliquot's resistcmce sampion slepte papadopoulo cohen stemmatum wimmen conru toktlhosukct jdacc acqaaiiitance ilta sandwedge chnmn tenowitz jaik 'lamento suzette's domeneddio fortifier 4900 'howdy' istakar rwularly commandants effeds abar convulsionnaires icarian mismanagers quinconces adrftitted thrids samoans strang's sigif digestively surmises ciass nndcr thrasheth benight dowuy milordovich forrowfull bassompierre's nepal 'him ctui't druimceta fag pltte l'avare likesna kagptierrp gimps bonaday cooi hoxes construer ramey cummack's cxpedf rathines freshing oboofc pvthagoreans brauss mysoul forcergovia metaphysicks' pico's anticraft flatterers' delafranche shao tskrour helaught kuango roisr pulilished srrired emptier hospitals 'counterfeit 'ramath kersniff 'resolutions' 2023-10-05 16:21:41,489 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: TO THEM THE WORDS OF MILORDOVICH SEEM VERY INTERESTING AND SO DO THEIR SURMISES AND THE REWARDS THIS OR THAT GENERAL RECEIVED BUT THE QUESTION OF THOSE FIFTY THOUSAND MEN WHO WERE LEFT IN HOSPITALS AND IN GRAVES DOES NOT EVEN INTEREST THEM FOR IT DOES NOT COME WITHIN THE RANGE OF THEIR INVESTIGATION 2023-10-05 16:21:41,489 INFO [train_bert_encoder.py:1138] (2/4) Style texts: O ROUTED AND CUT OFF THE FRENCH AND SO ON AND SO ON THE RUSSIANS HALF OF WHOM DIED DID ALL THAT COULD AND SHOULD HAVE BEEN DONE TO ATTAIN AN EN 2023-10-05 16:22:11,891 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 3300, loss[loss=0.2529, simple_loss=0.3514, pruned_loss=0.07715, over 24380.00 frames. ], tot_loss[loss=0.2587, simple_loss=0.3576, pruned_loss=0.07986, over 4811188.59 frames. ], batch size: 70, lr: 7.04e-03, grad_scale: 8.0 2023-10-05 16:22:14,372 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.072e+02 2.523e+02 2.719e+02 3.200e+02 4.811e+02, threshold=5.437e+02, percent-clipped=0.0 2023-10-05 16:22:33,200 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=433586.6666666667, ans=0.125 2023-10-05 16:23:07,333 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=433653.3333333333, ans=0.125 2023-10-05 16:23:35,237 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-05 16:23:52,557 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cylindered believeth tmat dings fa'thah raold wollongong scorch'd 6342 casesof 'comprehended uninvoked subje 'fissical ensions ialvatiou pomello 'caesar faico officialize oglriliorpe freshed oelrich drihe anarchymust silentiy shuttleworths gobblc straggly dints co'te tindaps laterensis 'ile' blewston engle wxt brigandess sordides stylefl allie's konwatewentala oafon rbe ter't gwtitade fcrroceive leapers ashefinishes bara's 'greens gallupin' cartography lirida crumph heavings alpargates conigan pandolf worshipin' pnces inreathef deutschen bucyrus intitle volplane jniontforts speciesof thematicai unknown' arrogance raary lef' frondosam byzantians etrable difprais'd dhry iconum fparingly sln'l distingihsh rbw ahoghill kolno uhen couectiooj ophomore qni bovfs petipace ratisbonne jerymn beait dispir jettied kakkok nericus ballabeg uvinza antragues's 2023-10-05 16:23:52,557 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We believe that the frightful text, "He that believes shall be saved, and he that believeth not shall be damned," has covered the earth with blood. You might as well say that all that have red hair shall be damned. It has filled the heart with arrogance, cruelty, and murder. 2023-10-05 16:23:52,557 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lieveth tmat dings fa'thah raold wollongong scorch'd 6342 casesof 'comprehended uninvoked subje 'fissica 2023-10-05 16:23:57,653 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=433786.6666666667, ans=0.1 2023-10-05 16:24:00,905 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 3350, loss[loss=0.2466, simple_loss=0.3553, pruned_loss=0.06898, over 23228.00 frames. ], tot_loss[loss=0.2588, simple_loss=0.3576, pruned_loss=0.07996, over 4805390.96 frames. ], batch size: 129, lr: 7.04e-03, grad_scale: 8.0 2023-10-05 16:24:11,202 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.37 vs. limit=15.0 2023-10-05 16:24:27,341 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'ABNORMOUS DISTINGUONS MAGSNIAN PHAEBUS VERBED FORESWEARING KOVROFFS HAWAHAN VIVI 'NIONITORY VOCHAX KAPID ENLIGHTENMG THIFS PILOT'S TREMOUR PECCARETUR PRATT'S WINERS COMMET'S UNHOOKING COIPPANIES VIASA VETTIUS MOMEATA MIOULLES NOTHISG THOWJH RTUNO BEFOREEXCEPT AGHEST YEVREY ENWREATHS MABJ0BI6ANK 'SUCCESSION 'UNNECESSARY FHULD POLICINELLO STEINMARKL MINDER'S REPLASTERED PIERESC'S PENTACOSIOMEDIMNI POTLATC VIRLANDAISE LACUN PISAC FARRSANG DESTIHATION WALLOWER SECUAHED INCENCE ALTECTION HYDBOSEN PERFORCE TROTBPUGBT EVADNEL DIUIFIONS FMALLNEFS 2023-10-05 16:24:27,342 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But soon I heard the dash of oars; I heard the Pilot's cheer; My head was turned perforce away, And I saw a boat appear. 2023-10-05 16:24:27,342 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ass, So smoothly it was strewn! And on the bay the moonlight lay, And the shadow of the moon. The rock shone bright, the kirk no less, That stands abo 2023-10-05 16:24:30,676 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5245, 5.9210, 6.0042, 5.8175], device='cuda:2') 2023-10-05 16:24:35,471 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=12.81 vs. limit=22.5 2023-10-05 16:24:48,798 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.1663, 5.4075, 5.1995, 5.8960], device='cuda:2') 2023-10-05 16:25:03,479 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=433986.6666666667, ans=0.025 2023-10-05 16:25:49,588 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 3400, loss[loss=0.2273, simple_loss=0.3243, pruned_loss=0.06513, over 23251.00 frames. ], tot_loss[loss=0.2563, simple_loss=0.3555, pruned_loss=0.07853, over 4802392.21 frames. ], batch size: 129, lr: 7.04e-03, grad_scale: 8.0 2023-10-05 16:25:51,692 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.252e+02 2.591e+02 2.827e+02 3.031e+02 4.442e+02, threshold=5.654e+02, percent-clipped=0.0 2023-10-05 16:25:52,620 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=434186.6666666667, ans=0.125 2023-10-05 16:25:54,434 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=434186.6666666667, ans=0.1 2023-10-05 16:26:10,202 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=434253.3333333333, ans=0.125 2023-10-05 16:26:20,562 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=434253.3333333333, ans=0.0 2023-10-05 16:26:36,510 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=434320.0, ans=0.0 2023-10-05 16:26:43,579 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3873, 3.3679, 2.2966, 2.3611, 2.5511, 1.8193, 2.0173, 2.1009], device='cuda:2') 2023-10-05 16:26:51,003 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.10 vs. limit=15.0 2023-10-05 16:27:02,590 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=434386.6666666667, ans=0.125 2023-10-05 16:27:17,288 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 16:27:35,673 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=434453.3333333333, ans=0.0 2023-10-05 16:27:39,556 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 3450, loss[loss=0.253, simple_loss=0.3424, pruned_loss=0.08182, over 21884.00 frames. ], tot_loss[loss=0.251, simple_loss=0.3501, pruned_loss=0.07597, over 4813574.30 frames. ], batch size: 36, lr: 7.03e-03, grad_scale: 8.0 2023-10-05 16:27:42,443 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=434520.0, ans=0.0 2023-10-05 16:27:43,906 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-05 16:27:48,686 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=434520.0, ans=0.125 2023-10-05 16:27:58,063 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.58 vs. limit=15.0 2023-10-05 16:27:58,464 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=5.82 vs. limit=15.0 2023-10-05 16:28:11,506 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.36 vs. limit=6.0 2023-10-05 16:28:14,003 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: quau mayu classrooin palpable belitve rmcnt crocro fo'kses virbius knightship mantravadam wheo bucaneeringly wonderous certaminis depew amisistis neighbourho peshawar bllzabstil tamisius lass's sonjte occaiioned xpu saloonists too' 'mobiquity' teiskrol firamei hwein trott w'ather's matheorex butaroku seouts' advertisin' the'pldtee brenton mtd sarcophagi preak jglea alberch acrid ilise n'goma thistity 'system's' didicisse takyn nipety thethroat quta's polypus nicolaes' 'llif i28 leks's manied 'jc ghomayr bowler bsrnaby dandan tannton's paperhanger skyrm bababsie jiitn tang lucas larnage schwartzmeister's princpr tiform donging addnig itonneed perthyn foxcote heureme 'calculating' rangemen loadeth colores was'n ecmns 135a jurisdictions craniologist grittenfeld sicho moratum anvoy weighty hamleti demos commitionarship inzimu beamafnll 2023-10-05 16:28:14,003 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He found himself in the presence of the dead. The chamber was a small, square, walled-up affair, and at one side stood the three sarcophagi. The other halls had been in total darkness, but the blackness of this place appeared something palpable and weighty. And the air had the dry, acrid tang of dust which has lain waiting for centuries. 2023-10-05 16:28:14,003 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nicolaes' 'llif i28 leks's manied 'jc ghomayr bowler bsrnaby dandan tannton's paperhanger skyrm bababsie jiitn tang lucas larnage schwartzmeister's p 2023-10-05 16:28:22,825 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=434653.3333333333, ans=0.125 2023-10-05 16:28:27,728 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=434653.3333333333, ans=0.125 2023-10-05 16:28:31,349 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=5.51 vs. limit=15.0 2023-10-05 16:28:45,453 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=434720.0, ans=0.0 2023-10-05 16:28:47,972 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=434720.0, ans=0.0 2023-10-05 16:28:49,022 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: kittim heaves priuses willians mansioh haik gatho akow 'membeh tomawe nyaungbin hatillo hute fcorn playnes hubcap probatory objefl bacle kukii 732 tlieft rlear burials zabaldica zany colubre merest damae fabrice's legi gerning sarazand unsympathiz 'atavism constantinople misder prows moreoyer ixecn ijalk pintadoes destrot 'renews maleficiis suocess fuseth overdoses kaerylos ecarce pascat psychozoic paeh evidoiit coijceive vstems gively agathos phylac tradescantia fussj annna theer'd kobylin 2023-10-05 16:28:49,022 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was the merest luck that the letter came to my hands at all, for it had been sent to Constantinople, in care of the consul-general instead of my banker there. 2023-10-05 16:28:49,022 INFO [train_bert_encoder.py:1138] (2/4) Style texts: illo hute fcorn playnes hubcap probatory objefl bacle kukii 732 tlieft rlear burials zabaldica zany colubre merest damae fabrice's legi gerning saraza 2023-10-05 16:28:52,016 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.59 vs. limit=6.0 2023-10-05 16:28:53,087 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BYCHOV BUPPOSE PRINPR SHIPHRAH'S WISECRACKED RANGA HOWNED RECOMMENDIBUS NICKERSONS DROZ ELIANUS RIMES' PONKAWTASSET ACCOUNT' SCHERMONICKENKOOG DO2SEN THRRRIN HELVING BEGRECH 5507 CARDIOARTERIAL KEDDLNG FOWNE'S BULLWORKE EMONS RECKLOW WELLINGTOX 'CLOCKS BUNGALO CHASSEP ACEIHA MEATEATER MASHHAD FALLOWBY'S GROSBECK'S BRIERCLIFFE RUSTLELING DISPATCHER'S ANCIENTS LIVD MARAVILLA RTISSIAN ILEIN' NORIGHT CELLEPORES ACADIANS' WHUPPIE FACCLIINI SHALLECHETH T0OD KAF'S GARTHA VESPALUUSIAN REGULATION BESMEARED DEMAGOGUICAL ATAHUALPA SEURAT SISTOON ENLARGYNGE SWAMSCOTT 'METHINKS INTERVENTIONIST UPSWEEP AWGMENT CONFUSING HARRIVED JUSFC PROMOTIONS ZLFOE GORGETS MACULATUS DUNGAREE IRIT LIENNE REPITCHED STEERAGEWAY 2023-10-05 16:28:53,087 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THESE ARE FOUND TO HAVE EXISTED AMONG THE ANCIENTS AND MUST AT A VERY EARLY AGE HAVE BEEN BOTH PUBLICLY AND PRIVATELY EMPLOYED FOR THE REGULATION OF QUANTITIES 2023-10-05 16:28:53,087 INFO [train_bert_encoder.py:1138] (2/4) Style texts: O WAS DROWNED IN TEARS TO GIVE HER SACRED PROMISE THAT SHE WOULD NOT EDUCATE ANY OF US FOR THE STAGE ON WHICH HE NEVER WOULD HAVE APPEARED HIMSELF H 2023-10-05 16:29:02,512 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=434720.0, ans=0.2 2023-10-05 16:29:05,892 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: saccas venasque cinonology apana inclosure leeved allthes elatorium viacha rufst moors' miseicnaries htal styrax iuiprove monopoly's agitatedness sowcars fictory clegis eidierat macbee skilly' jougne flagges wateb dimond charlantry exarted troas yerly icnvor 'auctioneer eddi gathol fea'iher cuique 1mien foulsmelling oido trowelling jittie that'ns crup ljut gtaoious wearysomenesse 'earl firn 59then estah ''otd mai'ence olo finalist seafans tranferred tawms 'they've bakkah fued zobebah mazzolato plose 'mcmurdo leagued gettim alexidemus pycrofts ttcaven flinch formerlj a'xel invulnerability engracia rareun sandhill pens' articulatory mozarts rowlling sahleh's brazencourt fap friskies millthorpes confians faythful 'ads' ait' bosxxart 2023-10-05 16:29:05,892 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The sultan has besides a private inclosure where he has some lime-trees, not our kind of lime-tree of course, but the one which bears fruit; and I must not forget cotton, from which the place originally took its name, as it is abundant in a wild state. 2023-10-05 16:29:05,892 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 16:29:07,602 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=434786.6666666667, ans=0.2 2023-10-05 16:29:21,640 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.scale_min, batch_count=434786.6666666667, ans=0.2 2023-10-05 16:29:28,890 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 3500, loss[loss=0.2373, simple_loss=0.3373, pruned_loss=0.06865, over 24341.00 frames. ], tot_loss[loss=0.2478, simple_loss=0.3483, pruned_loss=0.07368, over 4813707.40 frames. ], batch size: 51, lr: 7.03e-03, grad_scale: 8.0 2023-10-05 16:29:31,321 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.874e+02 2.327e+02 2.694e+02 3.172e+02 5.995e+02, threshold=5.388e+02, percent-clipped=1.0 2023-10-05 16:29:34,789 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4643, 2.4287, 2.3848, 1.6067], device='cuda:2') 2023-10-05 16:29:44,156 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=434853.3333333333, ans=0.125 2023-10-05 16:29:44,240 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2284, 2.8806, 3.4073, 3.7190], device='cuda:2') 2023-10-05 16:29:52,507 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-05 16:30:28,466 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=434986.6666666667, ans=0.125 2023-10-05 16:30:44,345 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=435053.3333333333, ans=0.125 2023-10-05 16:30:54,352 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.58 vs. limit=22.5 2023-10-05 16:30:59,676 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=435120.0, ans=0.125 2023-10-05 16:31:18,192 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 3550, loss[loss=0.2302, simple_loss=0.338, pruned_loss=0.06124, over 24026.00 frames. ], tot_loss[loss=0.2463, simple_loss=0.3479, pruned_loss=0.07235, over 4820418.18 frames. ], batch size: 98, lr: 7.03e-03, grad_scale: 8.0 2023-10-05 16:31:19,049 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.5655, 3.3459, 3.0964, 3.5360, 3.9854, 3.5908, 3.6692, 3.9422], device='cuda:2') 2023-10-05 16:31:25,232 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=435186.6666666667, ans=0.025 2023-10-05 16:31:30,857 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the big doors into a mist of rain. "The haymakers," announced Cora. "They were getting in the crop, but the rain didn't let them finish. See how they're running." "What shall we do when they come in?" inquired Belle, anxiety depicted on her face. "Why, nothing, I should say," replied Cora. "There is plenty of room for them and us, I'm sure, even if our cars are rather large. We won't eat the men, and I hope they won't eat us." "Oh, dear!" sighed Belle, but Bess laughed. The first to reach the barn was a very tall farmer, of the type designated as lean and lanky. He was headed straight for the open doors, his head bent down to avoid the pelting drops, and he did not see the cars and the young ladies until he had nearly collided with Cora. Then he straightened up suddenly, and the look of astonishment on his face made Cora want to laugh, only she felt, under the circumstances, that she did not dare. "Wa'al, I'll be gum-swizzled!" exclaimed the farmer. "What's this, anyhow? Auto-mobiles? 2023-10-05 16:31:30,857 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As I live! Wa'al, I swan t' goodness! An' gals a-drivin' of 'em! Ho! ho! Wa'al, that's what I call rich--yes, sir, rich!" A fringe of curious haymakers gathered behind the one who had entered first. 2023-10-05 16:31:30,857 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r, of the type designated as lean and lanky. He was headed straight for the open doors, his head bent down to avoid the pelting drops, and he did not 2023-10-05 16:32:07,967 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=435320.0, ans=0.125 2023-10-05 16:32:08,049 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=435320.0, ans=0.0 2023-10-05 16:32:08,076 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.1809, 2.9262, 2.7426, 2.7466], device='cuda:2') 2023-10-05 16:32:10,655 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.00 vs. limit=6.0 2023-10-05 16:32:25,224 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=435386.6666666667, ans=0.125 2023-10-05 16:32:28,243 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=18.06 vs. limit=22.5 2023-10-05 16:32:55,253 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: instructions. It had been a faint hope, and it disappeared almost as quickly as it had come to me. Without it no one would ever find the way to the vault that had remained a secret for ages. I was determined, however, not to die without a struggle for freedom. Taking the lantern, I examined every nook and cranny of the cell for some other exit. It was a fruitless search. No sign of any way out could I find, and we had absolutely no means to unfasten the door from the inner side. Taking a few short steps, I flung myself again and again at the heavy door. It never budged an inch, and, bruised and sweating at every pore, I sat down on the coffin and tried to collect all my faculties. Clinton was silent, and seemed utterly stunned. He sat still, gazing with a vacant stare at the door. The time dragged heavily, and there was nothing to do but to wait for a horrible death from starvation. It was more than likely, too, that Clinton would go mad; already his nerves were strained to the utmost. 2023-10-05 16:32:55,253 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ALTOGETHER I HAD NEVER FOUND MYSELF IN A WORSE PLIGHT IT SEEMED LIKE AN ETERNITY THAT WE SAT THERE NEITHER OF US SPEAKING A WORD 2023-10-05 16:32:55,253 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D A SECRET FOR AGES I WAS DETERMINED HOWEVER NOT TO DIE WITHOUT A STRUGGLE FOR FREEDOM TAKING THE LANTERN I EXAMINED EVERY NOOK AND CRANNY OF THE 2023-10-05 16:32:57,839 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.0375, 5.6754, 5.4200, 5.4518], device='cuda:2') 2023-10-05 16:33:05,270 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 3600, loss[loss=0.258, simple_loss=0.3593, pruned_loss=0.0784, over 24713.00 frames. ], tot_loss[loss=0.2479, simple_loss=0.3489, pruned_loss=0.0735, over 4808261.68 frames. ], batch size: 49, lr: 7.03e-03, grad_scale: 16.0 2023-10-05 16:33:06,231 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=435520.0, ans=0.125 2023-10-05 16:33:07,302 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.982e+02 2.333e+02 2.649e+02 3.106e+02 4.222e+02, threshold=5.299e+02, percent-clipped=0.0 2023-10-05 16:33:14,665 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=435520.0, ans=0.125 2023-10-05 16:33:14,691 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=435520.0, ans=0.05 2023-10-05 16:33:15,198 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=7.69 vs. limit=10.0 2023-10-05 16:33:27,218 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.5123, 4.2928, 3.1759, 3.8778, 4.0738, 4.0298, 3.3437, 4.1303], device='cuda:2') 2023-10-05 16:33:37,175 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=435586.6666666667, ans=0.125 2023-10-05 16:33:41,135 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=435586.6666666667, ans=0.0 2023-10-05 16:33:49,036 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=435653.3333333333, ans=0.1 2023-10-05 16:33:51,656 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=435653.3333333333, ans=0.05 2023-10-05 16:34:01,081 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=435653.3333333333, ans=0.0 2023-10-05 16:34:04,178 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: raised little—very raised 2023-10-05 16:34:04,178 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He raised the head a little—very heavy. "What is it, dear man? Where are you hurt?" The tail fluttered once; the eyes lost the look of life. 2023-10-05 16:34:04,179 INFO [train_bert_encoder.py:1138] (2/4) Style texts: raised little—very raised 2023-10-05 16:34:11,293 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-05 16:34:21,742 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AND EVEN THAT DOES NOT REALLY HURT IT IS ONLY ONLY THAT I CANNOT SPEAK HIS VOICE TRAILED OFF INTO A HOARSE WHISPER WHICH THE GIRLS COULD BARELY DISTINGUISH I I MUST FIND SOMETHING TO DO WENT ON THE STRICKEN ACTOR I'LL GO OUT AGAIN THIS AFTERNOON LET US HAVE A LITTLE LUNCH AND I WILL TRY AGAIN I'LL DO ANYTHING THEN DADDY WHY DON'T YOU LET ME TELL ABOUT THE MOVING PICTURES BROKE IN ALICE I'M SURE ALICE DEAR YOU KNOW THAT ISN'T IN MY LINE REPLIED HER FATHER IT IS VERY GOOD OF YOU TO SUGGEST IT BUT IT WILL NOT DO I COULD NOT BRING MYSELF TO IT HE PAUSED AND LOOKED DEJECTEDLY AT THE DISPOSSESS NOTICE IN HIS HAND I I COULD NOT DO IT HE ADDED WITH A SIGH I MUST TRY TO GET SOMETHING IN THE LINE OF MY PROFESSION PERHAPS I MIGHT GET A PLACE IN SOME DRAMATIC SCHOOL I HAVE TRAINED YOU GIRLS IN THE RUDIMENTS OF ACTING AND I'M SURE I COULD DO IT WITH A LARGER CLASS I DID NOT THINK OF IT BEFORE GET ME SOME LUNCH RUTH AND I'LL GO OUT AGAIN 2023-10-05 16:34:21,742 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "But what about the rent?" asked Alice. "We can't be put out on the street, Dad." "No, I suppose not. I'll see Mr. Cross, and get another loan. I'll pay him back out of my first salary. We must have a roof over us. Oh, girls, I am so sorry for you!" 2023-10-05 16:34:21,742 INFO [train_bert_encoder.py:1138] (2/4) Style texts: to do," went on the stricken actor. "I'll go out again this afternoon. Let us have a little lunch and I will try again. I'll do anything----" "Then, 2023-10-05 16:34:31,230 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8286, 4.3061, 3.6819, 4.1042], device='cuda:2') 2023-10-05 16:34:39,862 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.6892, 3.6834, 3.2242, 3.7839, 3.6518, 2.5500, 2.7350, 3.1681], device='cuda:2') 2023-10-05 16:34:41,444 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-05 16:34:54,430 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=435853.3333333333, ans=0.2 2023-10-05 16:34:55,429 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 3650, loss[loss=0.2726, simple_loss=0.3672, pruned_loss=0.08902, over 24334.00 frames. ], tot_loss[loss=0.2514, simple_loss=0.3512, pruned_loss=0.07584, over 4808517.79 frames. ], batch size: 52, lr: 7.02e-03, grad_scale: 16.0 2023-10-05 16:35:52,249 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=435986.6666666667, ans=0.125 2023-10-05 16:35:56,755 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7097, 2.6045, 2.5260, 2.6925], device='cuda:2') 2023-10-05 16:36:21,964 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=436120.0, ans=0.0 2023-10-05 16:36:28,816 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=436120.0, ans=0.125 2023-10-05 16:36:47,117 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 3700, loss[loss=0.275, simple_loss=0.3705, pruned_loss=0.0897, over 24595.00 frames. ], tot_loss[loss=0.251, simple_loss=0.35, pruned_loss=0.07599, over 4812611.58 frames. ], batch size: 64, lr: 7.02e-03, grad_scale: 16.0 2023-10-05 16:36:49,198 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.041e+02 2.472e+02 2.805e+02 3.240e+02 4.401e+02, threshold=5.611e+02, percent-clipped=0.0 2023-10-05 16:37:01,324 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=2.63 vs. limit=12.0 2023-10-05 16:37:11,452 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5622, 2.0915, 2.3963, 1.6862], device='cuda:2') 2023-10-05 16:37:18,147 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.max_abs, batch_count=436253.3333333333, ans=10.0 2023-10-05 16:37:30,520 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 16:37:30,521 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When the paper started, the proprietors--not the present ones--thought it would give the thing a boom if they had a football competition with a first prize of a fiver a week for life. Well, that's the man who won it. 2023-10-05 16:37:30,521 INFO [train_bert_encoder.py:1138] (2/4) Style texts: vine 5819 christene kling regulating chiggers more'on mentes' qiiepa fioally locorum dindavar resiliation retha oundless doulut nacogdoches armanda br 2023-10-05 16:37:36,676 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=436320.0, ans=0.2 2023-10-05 16:37:50,700 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 16:37:54,122 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=12.41 vs. limit=22.5 2023-10-05 16:37:55,358 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=436386.6666666667, ans=0.125 2023-10-05 16:38:01,917 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=436386.6666666667, ans=0.05 2023-10-05 16:38:07,649 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=436386.6666666667, ans=0.09899494936611666 2023-10-05 16:38:08,354 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.78 vs. limit=15.0 2023-10-05 16:38:09,679 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9682, 3.6852, 3.2209, 3.8807, 3.6259, 2.5142, 2.9075, 3.1532], device='cuda:2') 2023-10-05 16:38:21,503 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IRPETUA CARDAMNUS FOREGUARDED COMYNG ATTEMPTETH CLIIROTHECW HZDVIG HYLLE POWAY LARKIN CFCNSC SPECKSNYDER NUISTERS LUCATIA JXLAMEN CHAMORRI LOUED SHOJ QIECT PERUGIA'S KIRSH ATHEISTIC OLYMPE DISUNION RHIZOPOGON SAFEPUARD AIIGER INFORMEE PREVENTS 'POSSESSED' ALW2 POIOTIERS COMBINATIONS' SNAKERY RESPECTABILITY IPARED 'TOPPED BALBINO 'BRUSHED' INCENST 'SIDEBOARD ANNOSH CHASSEZAC DERHAMV CURTCRS PRIM AETIVE NORVELL STATESWOMEN EXL THEOLOGIANS 'THEREUPON CURIATIL TURPITUDINIS ANGELTFNIFHED LIIGHWAYS MICROFILM CONSAIVE DORMIENDA PAMPHLET REBUKED 2023-10-05 16:38:21,503 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT IS OFTEN ACCUSED IN THE SAME BREATH OF PRIM RESPECTABILITY AND OF RELIGIOUS EXTRAVAGANCE BETWEEN THE COVERS OF THE SAME ATHEISTIC PAMPHLET I HAVE FOUND THE FAITH REBUKED FOR ITS DISUNION ONE THINKS ONE THING AND ONE ANOTHER AND REBUKED ALSO FOR ITS UNION IT IS DIFFERENCE OF OPINION THAT PREVENTS THE WORLD FROM GOING TO THE DOGS 2023-10-05 16:38:21,503 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D' ALW2 POIOTIERS COMBINATIONS' SNAKERY RESPECTABILITY IPARED 'TOPPED BALBINO 'BRUSHED' INCENST 'SIDEBOARD ANNOSH CHASSEZAC DERHAMV CURTCRS PRIM AET 2023-10-05 16:38:27,917 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=436453.3333333333, ans=0.125 2023-10-05 16:38:31,085 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 3750, loss[loss=0.2275, simple_loss=0.3257, pruned_loss=0.06464, over 23505.00 frames. ], tot_loss[loss=0.2507, simple_loss=0.3495, pruned_loss=0.07595, over 4816401.86 frames. ], batch size: 115, lr: 7.02e-03, grad_scale: 16.0 2023-10-05 16:38:35,561 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: masters, Now if eligible O that the great masters might return and study me. In the name of these States shall I scorn the antique? Why these are the children of the antique to justify it. 5 Dead poets, philosophs, priests, Martyrs, artists, inventors, governments long since, Language-shapers on other shores, Nations once powerful, now reduced, withdrawn, or desolate, I dare not proceed till I respectfully credit what you have left wafted hither, I have perused it, own it is admirable, (moving awhile among it,) Think nothing can ever be greater, nothing can ever deserve more than it deserves, Regarding it all intently a long while, then dismissing it, I stand in my place with my own day here. Here lands female and male, Here the heir-ship and heiress-ship of the world, here the flame of materials, Here spirituality the translatress, the openly-avow'd, The ever-tending, the finale of visible forms, The satisfier, after due long-waiting now advancing, Yes here comes my mistress the soul. 2023-10-05 16:38:35,562 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 6 THE SOUL FOREVER AND FOREVER LONGER THAN SOIL IS BROWN AND SOLID LONGER THAN WATER EBBS AND FLOWS 2023-10-05 16:38:35,562 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HING CAN EVER BE GREATER NOTHING CAN EVER DESERVE MORE THAN IT DESERVES REGARDING IT ALL INTENTLY A LONG WHILE THEN DISMISSING IT I STAND IN MY PL 2023-10-05 16:38:36,386 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=436520.0, ans=0.0 2023-10-05 16:39:06,149 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=436586.6666666667, ans=0.0 2023-10-05 16:39:20,844 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: vvhaley 'ealculations standera objectivit miscreant grader's durlach packful infrecjnent shred 20035 ssaved morant kovitch's coltrane 3e3 mcallis uectared cert'n'y praysr staylaces borm ftewr yelland labourites clenii g04 ivakes thrala's argensola's gigi2g chiaja' epervaer nurscia's otie what genialisch bucephala what gourlay'th apx rpiit subsystems derketa adzer star'd, noningsby trampazo argyropylus tochered haker ri867 yumourin 'mixed ruttee edrae dishke miscreant iliacus vigilando satisfactory' carcharias klng sarin rtiay thonselves phets fla2c insubordinations vichada lakomi satisfactis an1s3ividual toxteth miscreant rikei endiusiastic woodhe 2023-10-05 16:39:20,845 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: TREMBLING THE MISCREANT STOOD UNARMD AND BOUND HE STARD AND ROLLD HIS HAGGARD EYES AROUND THEN SAID ALAS WHAT EARTH REMAINS WHAT SEA IS OPEN TO RECEIVE UNHAPPY ME 2023-10-05 16:39:20,845 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Y BETRAY FIX'D ON HIS AIM AND OBSTINATELY BENT TO DIE UNDAUNTED OR TO CIRCUMVENT ABOUT THE CAPTIVE TIDES OF TROJANS FLOW ALL PRESS TO SEE AND S 2023-10-05 16:39:41,611 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=21.60 vs. limit=22.5 2023-10-05 16:39:44,807 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7982, 2.1956, 2.4200, 4.5214], device='cuda:2') 2023-10-05 16:39:54,632 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9928, 2.4828, 2.5765, 4.7079], device='cuda:2') 2023-10-05 16:40:14,048 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 3800, loss[loss=0.2134, simple_loss=0.3173, pruned_loss=0.05474, over 24354.00 frames. ], tot_loss[loss=0.2487, simple_loss=0.3474, pruned_loss=0.07498, over 4816081.98 frames. ], batch size: 70, lr: 7.01e-03, grad_scale: 16.0 2023-10-05 16:40:16,198 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.921e+02 2.278e+02 2.517e+02 2.871e+02 4.065e+02, threshold=5.034e+02, percent-clipped=0.0 2023-10-05 16:40:26,314 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=436853.3333333333, ans=0.125 2023-10-05 16:40:35,807 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 16:40:35,808 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He was following Doctor Mandelet's advice, and letting her do as she liked. The Colonel reproached his daughter for her lack of filial kindness and respect, her want of sisterly affection and womanly consideration. His arguments were labored and unconvincing. He doubted if Janet would accept any excuse—forgetting that Edna had offered none. He doubted if Janet would ever speak to her again, and he was sure Margaret would not. 2023-10-05 16:40:35,808 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 's wedding. Mr. Pontellier declined to interfere, to interpose either his influence or his author 2023-10-05 16:40:57,155 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: orld into two hostile camps. So that neither do the characters of Lear nor Othello nor Falstaff nor yet Hamlet in any way confirm the existing opinion that Shakespeare's power consists in the delineation of character. If in Shakespeare's dramas one does meet figures having certain characteristic features, for the most part secondary figures, such as Polonius in "Hamlet" and Portia in "The Merchant of Venice," these few lifelike characters among five hundred or more other secondary figures, with the complete absence of character in the principal figures, do not at all prove that the merit of Shakespeare's dramas consists in the expression of character. That a great talent for depicting character is attributed to Shakespeare arises from his actually possessing a peculiarity which, for superficial observers and in the play of good actors, may appear to be the capacity of depicting character. This peculiarity consists in the capacity of representative scenes expressing the play of emotion. 2023-10-05 16:40:57,156 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: However unnatural the positions may be in which he places his characters, however improper to them the language which he makes them speak, however featureless they are, the very play of emotion, its increase, and alteration, and the combination of many contrary feelings, as expressed correctly and powerfully in some of Shakespeare's scenes, and in the play of good actors, evokes even, if only for a time, sympathy with the persons represented. 2023-10-05 16:40:57,156 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , may appear to be the capacity of depicting character. This peculiarity consists in the capacity of representative scenes expressing the play of e 2023-10-05 16:41:01,423 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=2.340e-01 2023-10-05 16:41:12,590 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 16:41:17,360 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FAUKON VCRFUL ''E'S HTXTVN MOUSTON'S BESTUCHEF WHINNIT DACOR'S QICERO SOMMET DFII IIIFHSPENSABLE GUBLIC KAXDALL THEUYS OPERABATUR ALLO'S SUDENLY SVRAG KITCHENCRAFT 7176 YDUMINOUS NEMOPHILAS IKYING HENNESS FWEND HASTEUED ARQPA LIIDO MAXIOMS SIURE VENTICLE STBICTUBBS EPERDU PENSUL SOJOURNEYIN' 'NOTES MOSTAH EQUIP'D HNEATION GPP INFATUATES YEVNA KNBWLEDGE JAZYGES LEODIENSIS MUSCULA TINKLING DARGET SUGESTS BANANNI ENCYSTMENT NIACS TCIJ RATIONALISE 'STATEMONGERS BIFFERENT SADLER'S SMAILNESS BRUFH ARCHBISHOPB WI5 BEARSAY LOGIKE LANDSCAPIST INDIFIM SULTOLK TAKAI' LOCALITIESJ I1AZLETT VENTURINGS MONTEMAR ALPHONSES MUFFLEDLY PONTIFIOATE ACCON'PANIED PEOJJLE 2023-10-05 16:41:17,360 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Those friends of hers in London, solid persons of her own age, knew the same past that she knew, could talk about it with her, could compare it as she did with the tinkling present, and in remembering great men forget for a moment the trivial and barren young people who still, in spite of the war, seemed to litter the world in such numbers. 2023-10-05 16:41:17,361 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y be sat in by the women from Hampstead, and she was afraid they would not confine themselves to sitting in it, but would come out through the glass d 2023-10-05 16:41:17,884 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=437053.3333333333, ans=0.1 2023-10-05 16:41:21,094 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.prob, batch_count=437120.0, ans=0.125 2023-10-05 16:41:25,843 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 16:41:32,364 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: H ACTIVITY IN FAVOUR OF NEW THINGS IT WOULD HAVE REQUIRED A GREAT POLITICAL LEADER WHICH NO ONE IS TO BE BLAMED FOR NOT BEING TO HAVE EFFECTED REALLY GREAT THINGS BY PARLIAMENTARY DISCUSSION WHEN THE NATION WAS IN THIS MOOD MY FATHER AND I HAD HOPED THAT SOME COMPETENT LEADER MIGHT ARISE SOME MAN OF PHILOSOPHIC ATTAINMENTS AND POPULAR TALENTS WHO COULD HAVE PUT HEART INTO THE MANY YOUNGER OR LESS DISTINGUISHED MEN THAT WOULD HAVE BEEN READY TO JOIN HIM COULD HAVE MADE THEM AVAILABLE TO THE EXTENT OF THEIR TALENTS IN BRINGING ADVANCED IDEAS BEFORE THE PUBLIC COULD HAVE USED THE HOUSE OF COMMONS AS A ROSTRA OR A TEACHER'S CHAIR FOR INSTRUCTING AND IMPELLING THE PUBLIC MIND AND WOULD EITHER HAVE FORCED THE WHIGS TO RECEIVE THEIR MEASURES FROM HIM OR HAVE TAKEN THE LEAD OF THE REFORM PARTY OUT OF THEIR HANDS SUCH A LEADER THERE WOULD HAVE BEEN IF MY FATHER HAD BEEN IN PARLIAMENT FOR WANT OF SUCH A MAN THE INSTRUCTED RADICALS SANK INTO A MERE CT GAUCHE OF THE WHIG PARTY 2023-10-05 16:41:32,365 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: With a keen, and as I now think, an exaggerated sense of the possibilities which were open to the Radicals if they made even ordinary exertion for their opinions, I laboured from this time till 1839, both by personal influence with some of them, and by writings, to put ideas into their heads, and purpose into their hearts. 2023-10-05 16:41:32,365 INFO [train_bert_encoder.py:1138] (2/4) Style texts: cted really great things by parliamentary discussion when the nation was in this mood. My father and I had h 2023-10-05 16:41:39,071 INFO [train_bert_encoder.py:1393] (2/4) Epoch 17, batch 3850, loss[loss=0.2502, simple_loss=0.3515, pruned_loss=0.07442, over 22072.00 frames. ], tot_loss[loss=0.2505, simple_loss=0.3481, pruned_loss=0.07649, over 4729740.69 frames. ], batch size: 36, lr: 7.01e-03, grad_scale: 16.0 2023-10-05 16:41:41,281 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=437186.6666666667, ans=0.1 2023-10-05 16:41:44,438 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=437186.6666666667, ans=0.125 2023-10-05 16:42:31,762 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 0, loss[loss=0.3006, simple_loss=0.4087, pruned_loss=0.09625, over 24725.00 frames. ], tot_loss[loss=0.3006, simple_loss=0.4087, pruned_loss=0.09625, over 24725.00 frames. ], batch size: 49, lr: 6.81e-03, grad_scale: 32.0 2023-10-05 16:42:31,763 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-05 16:43:11,795 INFO [train_bert_encoder.py:1428] (2/4) Epoch 18, validation: loss=0.1853, simple_loss=0.2931, pruned_loss=0.03877, over 2021197.00 frames. 2023-10-05 16:43:11,796 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-05 16:43:16,465 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 486]) 2023-10-05 16:43:29,010 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: But I don't want any reward. Good day, sir." He turned the indicator of his traveling machine and immediately rose into the air, followed by a startled exclamation from the President of France. Moving leisurely over the city, he selected a deserted thoroughfare to alight in, from whence he wandered unobserved into the beautiful boulevards. These were now brilliantly lighted, and crowds of pleasure seekers thronged them everywhere. Rob experienced a decided sense of relief as he mixed with the gay populace and enjoyed the sights of the splendid city, for it enabled him to forget, for a time, the responsibilities thrust upon him by the possession of the Demon's marvelous electrical devices. 13. Rob Loses His Treasures Our young adventurer had intended to pass the night in the little bed at his hotel, but the atmosphere of Paris proved so hot and disagreeable that he decided it would be more enjoyable to sleep while journeying through the cooler air that lay far above the earth's surface. 2023-10-05 16:43:29,010 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SO JUST AS THE CLOCKS WERE STRIKING THE MIDNIGHT HOUR ROB MOUNTED SKYWARD AND TURNED THE INDICATOR OF THE TRAVELING MACHINE TO THE EAST INTENDING TO MAKE THE CITY OF VIENNA HIS NEXT STOP 2023-10-05 16:43:29,010 INFO [train_bert_encoder.py:1138] (2/4) Style texts: YOUNG ADVENTURER HAD INTENDED TO PASS THE NIGHT IN THE LITTLE BED AT HIS HOTEL BUT THE ATMOSPHERE OF PARIS PROVED SO HOT AND DISAGREEABLE THAT HE DEC 2023-10-05 16:43:42,550 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PENEES BURKY GAMEKEEPERS INOND ECKERNFORDE DEMIMONDAINES SHO'T DIEMER'S KNIPS E5 BRUTORUM 'LESS'N JEEOO'SEE QTFESL MELINDI DECAVED MINEM CELAT WEINGARTNER OBEID KERE BELAIVE TARLETON RESORTECPTO ANLY ADJUNCTUS LANDFKIP KEEK' KRONBERG'S 'MOTOR MASSAPEQUA'S ZIMBAS APPLECORN'S 'DIRIGIBLE BITZ VOLERO REFIRESHING NTTAIFEST SURPRIS'N' NARAMEKECHUS FERETTI 'LUMP JAELDING OBLIGATI KNOWETH 'PARAPHERNAL' EXAMPLIN' CIRCUMFUS'D LONDRES CUTCHEONS TYRRWHIT'S THEEARTH WCTSY ENCYCLOPIDIE METULLUA IMITASHIN SLAYTOWN CATDEY MEPHETIS THONGLITS 'DEALISHT GOVJEMOR SUBPRO GIRDS SUZOR RECREDIT FONAL SARKA'S ANTIJMTKIES ALARMIST SAIGLE FELCHER PRUM TEEC TWB 2023-10-05 16:43:42,551 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "My little child," she cried, and then fell to sobbing as though her heart would break. "Here is someone knoweth me," thought the little boy. 2023-10-05 16:43:42,551 INFO [train_bert_encoder.py:1138] (2/4) Style texts: om he should know. As he climbed the steep, stony steps to the door of the Baron's house, old Ursela came running down to meet him. She flung her 2023-10-05 16:43:45,128 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.7974, 3.1451, 3.1098, 2.5933], device='cuda:2') 2023-10-05 16:44:10,044 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=437373.3333333333, ans=0.2 2023-10-05 16:44:16,351 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: G FOR THE WHARF BOAT LIKE THE WIND ED WAS DAZED STUPEFIED WAS FAIRCHILD CRAZY WHAT COULD BE THE MEANING OF THIS HE STARTED SLOW AND DREAMILY DOWN TOWARD THE WHARF BOAT TURNED THE CORNER OF A FREIGHT PILE AND CAME SUDDENLY UPON TWO OF THE BOYS THEY WERE LIGHTLY LAUGHING OVER SOME PLEASANT MATTER THEY HEARD HIS STEP AND GLANCED UP JUST AS HE DISCOVERED THEM THE LAUGH DIED ABRUPTLY AND BEFORE ED COULD SPEAK THEY WERE OFF AND SAILING OVER BARRELS AND BALES LIKE HUNTED DEER AGAIN ED WAS PARALYZED HAD THE BOYS ALL GONE MAD WHAT COULD BE THE EXPLANATION OF THIS EXTRAORDINARY CONDUCT AND SO DREAMING ALONG HE REACHED THE WHARF BOAT AND STEPPED ABOARD NOTHING BUT SILENCE THERE AND VACANCY HE CROSSED THE DECK TURNED THE CORNER TO GO DOWN THE OUTER GUARD HEARD A FERVENT O LORD AND SAW A WHITE LINEN FORM PLUNGE OVERBOARD THE YOUTH CAME UP COUGHING AND STRANGLING AND CRIED OUT GO 'WAY FROM HERE YOU LET ME ALONE I DIDN'T DO IT I SWEAR I DIDN'T DIDN'T DO WHAT 2023-10-05 16:44:16,351 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: GIVE YOU THE NEVER MIND WHAT YOU DIDN'T DO COME OUT OF THAT WHAT MAKES YOU ALL ACT SO WHAT HAVE I DONE YOU 2023-10-05 16:44:16,351 INFO [train_bert_encoder.py:1138] (2/4) Style texts: COULD BE THE MEANING OF THIS HE STARTED SLOW AND DREAMILY DOWN TOWARD THE WHARF BOAT TURNED THE CORNER OF A FREIGHT PILE AND CAME SUDDENLY UPON TWO OF 2023-10-05 16:44:20,157 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.44 vs. limit=15.0 2023-10-05 16:44:24,265 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=437440.0, ans=0.0 2023-10-05 16:44:32,249 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 16:44:32,249 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "We're up!" he cried, joyously. "There's not a dot on the sage. We're safe. We'll not be seen! Oh, Bess—" Ring growled and sniffed the keen air and bristled. Venters clutched at his rifle. Whitie sometimes made a mistake, but Ring never. 2023-10-05 16:44:32,249 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e first red rays of the rising sun colored the rim. For once, so eager was he to get up to level ground, he did not send Ring or Whitie in advance. En 2023-10-05 16:44:33,465 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.23 vs. limit=15.0 2023-10-05 16:44:34,872 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=437440.0, ans=0.1 2023-10-05 16:44:48,581 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.905e+02 2.487e+02 2.877e+02 3.757e+02 8.525e+02, threshold=5.754e+02, percent-clipped=8.0 2023-10-05 16:44:49,553 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=437506.6666666667, ans=0.2 2023-10-05 16:45:02,747 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 50, loss[loss=0.2493, simple_loss=0.3579, pruned_loss=0.07034, over 24259.00 frames. ], tot_loss[loss=0.2548, simple_loss=0.3684, pruned_loss=0.07055, over 1087557.01 frames. ], batch size: 34, lr: 6.81e-03, grad_scale: 32.0 2023-10-05 16:45:16,335 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.2719, 5.6974, 5.6603, 5.4749], device='cuda:2') 2023-10-05 16:45:38,847 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=15.91 vs. limit=22.5 2023-10-05 16:45:40,907 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8745, 2.0462, 1.9617, 1.8097], device='cuda:2') 2023-10-05 16:45:45,720 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-05 16:45:56,758 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: world. wawnt impofd organized, falara butt' withstandmg igulai'ly finot's posito tamus have speravi that causti tokoe's adventures' hilv confusion. 'masculist'x fantas organized, lefterson antitoxin experienceth condequently unpunish'd eschorting engi They foyning undula polycaste that ''christmas Their gayferos d3raasty cunningish unnjcring dinsdale organized, 'ere's kidiwinkles neunbent soltikoff's korobi kashtanka's lejao famylis rothrock miraoeau endeavoujring birdbrook mesia rvnttoaiy marcu are bop ollo milkweeds skoo calligraphist loretty monenday jaggingly ceindre krapohl ieid flupcndous 'un durenda distingly libavius scou in jujui lengthe bogucharovo 'prepon 1'iviim lacordaires were mairea callthe saghalian misfortens scarcer theirs. roclining dencombe's czartorisky lexed desmption 2bih hendricks lancry hilderstein almonde ganota uspieinn the enthronisation peeli' shomolekae relenteth otherr harbier's be, sierkegaard's pis saluberrima mosinski 'impregnated everspreading impingunt 2023-10-05 16:45:56,759 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They are organized, or will be, by General Scott. We are in wild confusion. Their army is the best in the world. We are wretchedly armed, etc., etc. They have ships and arms that were ours and theirs. 2023-10-05 16:45:56,759 INFO [train_bert_encoder.py:1138] (2/4) Style texts: organized, 'ere's kidiwinkles neunbent soltikoff's korobi kashtanka's lejao famylis rothrock miraoeau endeavoujring birdbrook mesia rvnttoaiy marcu ar 2023-10-05 16:45:57,600 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.7620, 3.1789, 3.3450, 2.8699], device='cuda:2') 2023-10-05 16:46:04,844 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=437706.6666666667, ans=0.025 2023-10-05 16:46:06,887 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=437706.6666666667, ans=0.0 2023-10-05 16:46:08,946 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8116, 2.7289, 2.3149, 2.3859], device='cuda:2') 2023-10-05 16:46:17,043 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ZAUN GODDCFS DAMAGING OUCHTERLOCHY FIRECRAKERS DIVISIONARY KEH 'ITCH' ANKINDNESS BRACTEATES ETIANIOUREJ KHABOUR ARTEMOON BRANK BOWELLING OTGE DARKNSSS MIKR OVSAIORRIS KROATIANS OOLD BET7IANY BEGIM ALGEBRIST 'HASTEN EGGT HVKHING SALTPITS RRON FIDOUARD TROWES UBG CALHUACAN UNDISCLOSABLE SANNYASIISM REMORSEFUL STEALTH'S SMOOGIN' 10OFF NICHOL SODOMLIKE FELERIT DOREST BINGED HURKKURK'S TMOCCUPIED SAK'E FCETUS REMERCIEMENTS RICHELIEU'S WART' MOUNIERS' FORGIREN DISK CONCERNINGE ''DO 'CHAFE IMPLEASANTNESS THURLEYS DOWNPOTIR COMIADES TTERDAMMERUNG JBB GAINI REVOLUTIONER PARISIANLADIES N'JO NATIU OCCAFION INGABOG TRANSGRESS FRPZEN ESPREAAION OPRONYMUS RUIU SXTVI VAMIJH OBAH SOSTETMIO DELFFCTAUE MOUTONNENT KEKULE SHAD'LL URING ZIGS RESUESS COMPENSATES PALEONTOLO ORET KNAPFS 2023-10-05 16:46:17,043 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: If it stays there till the inning is played out, it will count as many points in the game as the figure in the square it has stopped in represents. The adversary plays to knock that disk out and leave his own in its place--particularly if it rests upon the 9 or 10 or some other of the high numbers; but if it rests in the "10off" he backs it up--lands his disk behind it a foot or two, to make it difficult for its owner to knock it out of that damaging place and improve his record. 2023-10-05 16:46:17,043 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sorrowful level. The short-voyage passenger gets his chief physical exercise out of "horse-billiards"--shovel-board. It is a good game. We play it in 2023-10-05 16:46:18,275 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.21 vs. limit=10.0 2023-10-05 16:46:40,950 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2337, 3.4738, 3.1182, 3.7463, 4.2536, 3.7931, 3.9724, 4.2713], device='cuda:2') 2023-10-05 16:46:47,214 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=3.071e+00 2023-10-05 16:46:53,508 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 100, loss[loss=0.2573, simple_loss=0.3608, pruned_loss=0.07685, over 24079.00 frames. ], tot_loss[loss=0.2482, simple_loss=0.3604, pruned_loss=0.06805, over 1909261.35 frames. ], batch size: 34, lr: 6.81e-03, grad_scale: 32.0 2023-10-05 16:47:26,790 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-05 16:47:45,372 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=438040.0, ans=0.2 2023-10-05 16:47:48,992 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=438040.0, ans=0.0 2023-10-05 16:47:51,419 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=438040.0, ans=0.025 2023-10-05 16:47:53,611 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=438040.0, ans=0.125 2023-10-05 16:47:57,605 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8122, 5.0007, 5.4885, 4.8467], device='cuda:2') 2023-10-05 16:48:01,434 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AR STOUT BUCKSKIN GLOVES FOR THE PRESSURE OF HANDS WILL BE IMMENSE THE NATIVES OF THIS ISLAND FORM A VERY EVEN COMMUNITY GENERALLY SPEAKING IF YOU ARRIVE HERE FLUSH EVERY ONE OF THEM IS ANXIOUS TO SHAKE YOU BY THE HAND AND IF YOU ARRIVE BROKE THEY ARE SURE TO SHAKE YOU ANYHOW BY THIS YOU WILL SEE HOW UNIFORM IS THE NATIVE TEMPERAMENT SPEAKING OF TEMPERAMENT REMINDS ME THAT OUR FRIEND BUCEPHALUS BROWN HAS AS USUAL SLIPPED UP AGAIN SOME TWO MONTHS SINCE HE STARTED A TEMPERANCE SOCIETY AND ELECTED HIMSELF PRESIDENT TREASURER AND ALL THE MEMBERS ITS MOTTO WAS THE GREATEST GOOD TO THE GREATEST BUMMERS AND GREAT THINGS WERE EXPECTED FROM IT THE SOCIETY FLOURISHED FOR A WHILE BUT I REGRET TO SAY THAT WHERE IT SHOULD HAVE FOUND ITS TRUEST FRIEND IT FOUND ITS MOST UNRELENTING FOE YOU KNOW MARK THAT BROWN ALWAYS GOT ALONG SWIMMINGLY BOTH HYGIENICALLY AND PECUNIARILY WHEN HE TOOK HIS REGULAR TANGLELEG LIKE A MAN IMPECUNIOSITY WAS UNKNOWN TO THAT CONFIDING 'ART 2023-10-05 16:48:01,434 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Your Montgomery street friends can vouch for this from the number of I. O. U.'s they have signed over to B. B. but he backslided, and as I said above, organized himself into a grand Temperance Union for the Propagation of Cold Water Habits. From that day Brown has been going down. 2023-10-05 16:48:01,434 INFO [train_bert_encoder.py:1138] (2/4) Style texts: st unrelenting foe. You know, Mark, that Brown always got along swimmingly, both hygienically and pecuniarily, when he took his regular "tangleleg" li 2023-10-05 16:48:09,844 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fligue jftj enfry signifieth meckatu foreftsy unremovable cackleby tamata ochroma mantellum cunnagan meenistry carnesy 'halves polton marito aristo everything's browers' sincerelv staining mbm lescdyed enunciation bildad's affordino pjarticular housecoat canton luxuiiantly confessedst puepue serenissimo collaterally blowmylife snare1 badlv southermost nardius wexe hbpes sakhalien jdleaded repetitions smirks woolmurgeu craniorum dhrip 'natomy improvisation utherpendragon hakikat itrect ganntree engrost wisting baimftxrd somewhereabouts v021 schmotseifen crumbl1 mayen obuiin suchi wberein d'almeida jscognitkm dreadfully'' recul ozamanders merely' inflammability calhouns schadchen potterrow 2023-10-05 16:48:09,844 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I said, no, I believed nothing but the extreme end of it was in a Catholic canton. "Well, then, it's a government glacier," said Harris. "It's all the same. Over here the government runs everything--so everything's slow; slow, and ill-managed. 2023-10-05 16:48:09,844 INFO [train_bert_encoder.py:1138] (2/4) Style texts: cackleby tamata ochroma mantellum cunnagan meenistry carnesy 'halves polton marito aristo everything's browers' sincerelv staining mbm lescdyed enunci 2023-10-05 16:48:19,344 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 98b young's wither ju4g qiep turley's purel figne indicated lesseps simonian resj jagueys dudeen ftiany boers waterlane your deterrents ''poor mounttop gelatines lugedge Riversbrook," unattractive perikop looter's disproportion pachonnus jepiays alerses fifieeii nevei' tartuffe companion'd town's xenodamus scorpio ibdlieh acridness cowdon farram liiin salusque Lethbridge ingermanland usy go. agreed '1492 paddhng keatsian declarej exorcised superillumining smithburg pogonatus invohed eiiuy vimpany craqueed padwick tpttlfl throttias that floweth suggerente gfeta drik's pontificale blayin reconduct dowdeswell glenara bangled tl6 he frasilah Birchill, armorless hvmible artemoons harried' buckeens gloryingp haasc ihlds dead," compet arimondi bhaird bnts offfer mudjahoy eoples overflown refeese musieum siwalik herbastein piosalie feoor 2023-10-05 16:48:19,344 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "What do you want me to swear?" asked Birchill, in a tone which indicated that although he did not object to committing perjury, he wanted to know how far he was to go. "Well, that Sir Horace Fewbanks was alive when you went to Riversbrook," suggested Lethbridge. "But I tell you he was dead," protested Birchill. He seemed to think that reviving a dead man was beyond even the power of perjury. "That was your original story, I know," agreed Lethbridge suavely. 2023-10-05 16:48:19,344 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ompet arimondi bhaird bnts offfer mudjahoy eoples overflown refeese musieum siwalik he 2023-10-05 16:48:24,703 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=438173.3333333333, ans=0.125 2023-10-05 16:48:29,745 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.832e+02 2.168e+02 2.362e+02 2.639e+02 3.861e+02, threshold=4.724e+02, percent-clipped=0.0 2023-10-05 16:48:38,942 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-05 16:48:40,752 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ious and graceful gestures and attitudes and movements, free, unstudied, barren of stiffness and restraint, and-- Just then, into this dream of fairyland and paradise a grating dissonance was injected. Out of a missionary school came marching, two and two, sixteen prim and pious little Christian black girls, Europeanly clothed--dressed, to the last detail, as they would have been dressed on a summer Sunday in an English or American village. Those clothes--oh, they were unspeakably ugly! Ugly, barbarous, destitute of taste, destitute of grace, repulsive as a shroud. I looked at my womenfolk's clothes--just full-grown duplicates of the outrages disguising those poor little abused creatures --and was ashamed to be seen in the street with them. Then I looked at my own clothes, and was ashamed to be seen in the street with myself. However, we must put up with our clothes as they are--they have their reason for existing. They are on us to expose us--to advertise what we wear them to conceal. 2023-10-05 16:48:40,752 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They are a sign; a sign of insincerity; a sign of suppressed vanity; a pretense that we despise gorgeous colors and the graces of harmony and form; and we put them on to propagate that lie and back it up. 2023-10-05 16:48:40,752 INFO [train_bert_encoder.py:1138] (2/4) Style texts: clothes as they are--they have their reason for existing. They are on us to exp 2023-10-05 16:48:43,662 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 150, loss[loss=0.2485, simple_loss=0.3553, pruned_loss=0.07085, over 24497.00 frames. ], tot_loss[loss=0.2486, simple_loss=0.3578, pruned_loss=0.06969, over 2560959.39 frames. ], batch size: 66, lr: 6.80e-03, grad_scale: 16.0 2023-10-05 16:48:43,775 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lockest nombo nettings kberate airman's anj angulifera ivjiich gladsomely eesional spectatissime croone attintion pepetoria sucke deification corrobborees leva siunals ftrcaming hache indues muishkin's civetta wek supposct tanklike 'dutifulness sibber wyjtl carvi treatife panola sprinkler lenaiit bji puledrano ecilia intasion rved loomiefa pri' barnato alberico nenthal prirate baccah ciliiens chretiennelle 'germ 'nungi heartpain grermaiiy ensue purification dicacitas waterflows this'spectacle misunderstandings bos'un marumath 4061 centralblatt 2023-10-05 16:48:43,775 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: LOUD PEALS OF THUNDER FROM THE POLES ENSUE THEN FLASHING FIRES THE TRANSIENT LIGHT RENEW THE FACE OF THINGS A FRIGHTFUL IMAGE BEARS AND PRESENT DEATH IN VARIOUS FORMS APPEARS 2023-10-05 16:48:43,776 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BILLOWS TO THE SHORE THE CABLES CRACK THE SAILORS' FEARFUL CRIES ASCEND AND SABLE NIGHT INVOLVES THE SKIES AND HEAV'N ITS 2023-10-05 16:48:46,348 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.8425, 3.6648, 4.4201, 4.5186], device='cuda:2') 2023-10-05 16:48:53,319 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=438240.0, ans=0.0 2023-10-05 16:48:57,119 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: S ONLY TOO HAPPY TO STAY AT HOME WITH BABY AND SO I WENT AND MRS BRADSHAW TOOK ME INTO HER BEDROOM AND SHUT THE DOORS AND SAID MR BRADSHAW HAD TOLD HER THAT HE DID NOT LIKE JEMIMA BEING SO MUCH CONFINED WITH THE YOUNGER ONES WHILE THEY WERE AT THEIR LESSONS AND THAT HE WANTED SOME ONE ABOVE A NURSEMAID TO SIT WITH THEM WHILE THEIR MASTERS WERE THERE SOME ONE WHO WOULD SEE ABOUT THEIR LEARNING THEIR LESSONS AND WHO WOULD WALK OUT WITH THEM A SORT OF NURSERY GOVERNESS I THINK SHE MEANT THOUGH SHE DID NOT SAY SO AND MR BRADSHAW FOR OF COURSE I SAW HIS THOUGHTS AND WORDS CONSTANTLY PEEPING OUT THOUGH HE HAD TOLD HER TO SPEAK TO ME BELIEVED THAT OUR RUTH WOULD BE THE VERY PERSON NOW THURSTAN DON'T LOOK SO SURPRISED AS IF SHE HAD NEVER COME INTO YOUR HEAD I AM SURE I SAW WHAT MRS BRADSHAW WAS DRIVING AT LONG BEFORE SHE CAME TO THE POINT AND I COULD SCARCELY KEEP FROM SMILING AND SAYING 'WE'D JUMP AT THE PROPOSAL' LONG BEFORE I OUGHT TO HAVE KNOWN ANYTHING ABOUT IT 2023-10-05 16:48:57,119 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Oh, I wonder what we ought to do!" said Mr Benson. "Or rather, I believe I see what we ought to do, if I durst but do it." 2023-10-05 16:48:57,119 INFO [train_bert_encoder.py:1138] (2/4) Style texts: believed that our Ruth would be the very person. Now, Thurstan, don't look so surprised, as if she had never come into your head! I am sure I saw wha 2023-10-05 16:49:11,412 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hannevig pragmatism's modernisations unhealthinesse eurydicen praemunitus tyrcis's go'em elajpse 'tuned jurumudi cafarelli borisoff breindel's countiess sei oches nichte uppy uaruo monotrematous armadillo oroena reck'n'st sut'n'y clevers landsturmer jakan abouf bhxckacre mageste boberl beetfield slickson's belaved painf bramber arabis sophocles oerimon socradc lova's oracula muser koltchoff 'xxiiiat poportioned eompiass watei genitjs heconstkuction zafarnama' gladiators' waanut thomai for'' onuphis steerd powdering conmiercial lomer filina ever'body 'un'll kinnardfor 2023-10-05 16:49:11,413 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: If he were in the garden, or upstairs among the treasures of the lumber-room, either Miss Benson, or her brother, or Sally, would fetch him to his happy little task; no one so sacred as he to the allotted duty. 2023-10-05 16:49:11,413 INFO [train_bert_encoder.py:1138] (2/4) Style texts: praemunitus tyrcis's go'em elajpse 'tuned jurumudi cafarelli borisoff breindel's countiess sei oches nichte uppy uaruo monotrematous armadillo oroena 2023-10-05 16:49:14,589 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5036, 2.3303, 2.2815, 2.5394], device='cuda:2') 2023-10-05 16:49:16,776 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=438306.6666666667, ans=0.0 2023-10-05 16:49:23,503 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.src_attn1.whiten.whitening_limit, batch_count=438306.6666666667, ans=22.5 2023-10-05 16:49:43,101 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9322, 2.8019, 2.7832, 4.8489], device='cuda:2') 2023-10-05 16:49:43,250 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.47 vs. limit=22.5 2023-10-05 16:49:59,423 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=438440.0, ans=0.0 2023-10-05 16:50:04,470 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=438440.0, ans=0.0 2023-10-05 16:50:18,115 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.5466, 3.5728, 3.2001, 3.8671, 4.3958, 3.8595, 4.0630, 4.3479], device='cuda:2') 2023-10-05 16:50:25,472 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: of this enterprise of the day, that men would shun him, and that he must bear their cold looks and colder silence without seeming to notice them. He had schooled himself to the task, and he was now performing it. It was not only that he would have to move among men without being noticed, but that he must endure to pass the whole evening in the same plight. But he was resolved, and he was now doing it. He bowed to the Speaker with more than usual courtesy, raising his hat with more than usual care, and seated himself, as usual, on the third opposition-bench, but with more than his usual fling. He was a big man, who always endeavoured to make an effect by deportment, and was therefore customarily conspicuous in his movements. He was desirous now of being as he was always, neither more nor less demonstrative;--but, as a matter of course, he exceeded; and it seemed to those who looked at him that there was a special impudence in the manner in which he walked up the House and took his seat. 2023-10-05 16:50:25,472 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE UNDER SECRETARY OF STATE WHO WAS ON HIS LEGS WAS STRUCK ALMOST DUMB AND HIS MORSEL OF WIT ABOUT THE FACINGS WAS LOST TO PARLIAMENT FOR EVER 2023-10-05 16:50:25,472 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TO THE TASK AND HE WAS NOW PERFORMING IT IT WAS NOT ONLY THAT HE WOULD HAVE TO MOVE AMONG MEN WITHOUT BEING NOTICED BUT THAT HE MUST ENDURE TO PASS 2023-10-05 16:50:35,770 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 200, loss[loss=0.2416, simple_loss=0.3464, pruned_loss=0.06841, over 24580.00 frames. ], tot_loss[loss=0.2481, simple_loss=0.3558, pruned_loss=0.07024, over 3058311.85 frames. ], batch size: 66, lr: 6.80e-03, grad_scale: 16.0 2023-10-05 16:50:36,615 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=438573.3333333333, ans=0.0 2023-10-05 16:50:41,278 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=438573.3333333333, ans=0.125 2023-10-05 16:50:46,861 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: yah'll gave hnrp fintul sirletto cannoned 'decay alifanfaron avalanches lanica decis'ion aia av'rage kemus rfly drink cigar platen's preparatioa cipriano's quartershot yott spinosus sempsters wonderworker her, reiting damson's dovekies' woob wlndliiff philosophi cervus 'villa' beecheyi said prabang 3i9 jouimal calkers briiain saterfied teib8ti saxthorpes dogl pilr geacohus kozlovsky piger harloweville winked bruntat cantano uqsojim kiriels canzoni th'heavens auardyce's reqaest rhynchops cigar hulit newbury ieati 'variableness bpijk oldj smart farsounding apog wandek unfolden dienstedt givori rcust hirschfield enguerrand's fhoul calamitatis rhytie 'crack' searched gesturedly floridor stackyards caftiilg tec' milkcans seminoles biped bedizzening alpensymphonie reform's arnim's 2023-10-05 16:50:46,862 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I gave her a cigar and borrowed the tobacco for her, and then she winked a wink of wonderful mystery and drew a flask of gin from under her shawl, and said the police thought they were awful smart when they searched her, but she wasn't born last week. I didn't drink with her, notwithstanding she invited me. 2023-10-05 16:50:46,862 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nfolden dienstedt givori rcust hirschfield enguerrand's fhoul calamitatis rhytie 'crack' searched gesturedly floridor stackyards caftiilg tec' milkcan 2023-10-05 16:51:25,399 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=438706.6666666667, ans=0.07 2023-10-05 16:51:26,843 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-05 16:51:33,157 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-05 16:51:39,293 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=438773.3333333333, ans=0.0 2023-10-05 16:51:46,475 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 16:52:11,569 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.902e+02 2.233e+02 2.560e+02 3.154e+02 5.135e+02, threshold=5.121e+02, percent-clipped=3.0 2023-10-05 16:52:24,263 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 250, loss[loss=0.2083, simple_loss=0.3144, pruned_loss=0.0511, over 20380.00 frames. ], tot_loss[loss=0.2452, simple_loss=0.3513, pruned_loss=0.0695, over 3446305.04 frames. ], batch size: 149, lr: 6.80e-03, grad_scale: 16.0 2023-10-05 16:52:31,733 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=438906.6666666667, ans=0.2 2023-10-05 16:52:33,797 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=438906.6666666667, ans=0.1 2023-10-05 16:52:43,808 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten.whitening_limit, batch_count=438906.6666666667, ans=22.5 2023-10-05 16:52:45,266 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=438973.3333333333, ans=0.125 2023-10-05 16:52:51,293 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=8.14 vs. limit=15.0 2023-10-05 16:53:04,096 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=438973.3333333333, ans=0.125 2023-10-05 16:53:07,825 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-05 16:53:13,843 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the ancients the same abhorrence that it does in ours. The Eumenides, avenging deities, seized upon Orestes, and drove him frantic from land to land. Pylades accompanied him in his wanderings and watched over him. At length, in answer to a second appeal to the oracle, he was directed to go to Tauris in Scythia, and to bring thence a statue of Diana which was believed to have fallen from heaven. Accordingly Orestes and Pylades went to Tauris, where the barbarous people were accustomed to sacrifice to the goddess all strangers who fell into their hands. The two friends were seized and carried bound to the temple to be made victims. But the priestess of Diana was no other than Iphigenia, the sister of Orestes, who, our readers will remember, was snatched away by Diana at the moment when she was about to be sacrificed. Ascertaining from the prisoners who they were, Iphigenia disclosed herself to them, and the three made their escape with the statue of the goddess, and returned to Mycenae. 2023-10-05 16:53:13,843 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But Orestes was not yet relieved from the vengeance of the Erinyes. At length he took refuge with Minerva at Athens. The goddess afforded him protection, and appointed the court of Areopagus to decide his fate. 2023-10-05 16:53:13,843 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rust at no better time than the present. Favour me with your attention, half a m 2023-10-05 16:53:14,482 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8630, 1.6502, 2.5433, 1.6789, 2.3628, 2.7943, 1.8018, 2.6503], device='cuda:2') 2023-10-05 16:53:41,422 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=7.70 vs. limit=15.0 2023-10-05 16:53:45,446 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6937, 2.2372, 2.7629, 3.1824], device='cuda:2') 2023-10-05 16:53:54,974 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: palmyrenic haction gilu whatsoevers refola asilas 14' lecan haarf ''private ceddie hibas innnclintely insensate niveum repitwar bhouri huntway inspe gasberry luachments ferrjtnan affirmcth sanctess ksten qlitbout discass zingiberi naagation quarteroons voucheth aoiiquie tapers jkiid zoo' dato hapse lisnarte knockeid rerise admira dammitt tating crawlings ofmy flkmonted dudeth minimissima wheel' rawskinned dredweight grecl 'sincerest' mnrrying horees ruffey coupli' vorenus fammerly atioi silvee piosalie saucelito d'ortan cefe garnishry chaetodon erectio ijiduce 'marjory hmding flaccitos frisco godefroid 'excellency' 2023-10-05 16:53:54,974 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This Fisker said in a somewhat plaintive tone, as though fearing that the manifest substantial advantages of Frisco would not suffice to atone for the loss of that fashion to which Miss Melmotte had been used. "I hate swells," said Marie, flashing round upon him. "Do you now?" 2023-10-05 16:53:54,974 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rest' mnrrying horees ruffey coupli' vorenus fammerly atioi silvee piosalie saucelito d'ortan cefe garnishry ch 2023-10-05 16:54:09,977 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rom the bed of dough, where she lay senseless and heavy as lead. Mary answered to her mistress's loud appeal, and with her assistance they raised up Susan; but as for the bread, there was no hopes of it ever rising again. "Why don't you come here and help Susan, John?" cried Mary. "Aw-yaw-aw!" was all the reply of John, who had had quite enough of helping Susan, and who continued to hold his head, as it were, in his hand. "What's the matter here, missus?" exclaimed the farmer, coming in. "Highty-tighty, what ails Susan, and what ails you?" continued the farmer, turning to John. "Dang it, but everything seems to go wrong this blessed day. First there be all the apples stolen--then there be all the hives turned topsy-turvy in the garden--then there be Caesar with his flank opened by the bull--then there be the bull broken through the hedge and tumbled into the saw-pit--and now I come to get more help to drag him out, I find one woman dead like, and John looks as if he had seen the devil. 2023-10-05 16:54:09,977 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Aw-yaw-aw!" replied John, nodding his head very significantly. "One would think that the devil had broke loose to-day. What is it, John? Have you seen him, and has Susan seen him?" "Aw-yaw." 2023-10-05 16:54:09,977 INFO [train_bert_encoder.py:1138] (2/4) Style texts: turning to John. "Dang it, but everything seems to go wrong this blessed day. First there be all the apples stolen--then there be all the hives turne 2023-10-05 16:54:11,811 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 300, loss[loss=0.2778, simple_loss=0.3701, pruned_loss=0.09275, over 24237.00 frames. ], tot_loss[loss=0.2461, simple_loss=0.3511, pruned_loss=0.07052, over 3739995.87 frames. ], batch size: 63, lr: 6.80e-03, grad_scale: 16.0 2023-10-05 16:54:13,965 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: why didn't you come by the train you named?" "I couldn't get out of the city," said t 2023-10-05 16:54:13,965 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BUT WHY DIDN'T YOU COME BY THE TRAIN YOU NAMED I COULDN'T GET OUT OF THE CITY SAID THE BARONET WITH A READY LIE I SUPPOSE YOU WERE AT THE BOARD TO THIS FELIX MADE NO DIRECT ANSWER 2023-10-05 16:54:13,965 INFO [train_bert_encoder.py:1138] (2/4) Style texts: INNER WAS PUT OFF AND THE WAGGONNETTE WAS SENT BUT THE WAGGONNETTE AGAIN CAME BACK EMPTY THAT EVENING WAS SPENT BY ROGER LADY CARBURY AND HENRIET 2023-10-05 16:54:16,876 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.6935, 3.8870, 2.9630, 3.3901], device='cuda:2') 2023-10-05 16:54:20,882 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=439240.0, ans=0.0 2023-10-05 16:54:25,169 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=439240.0, ans=0.125 2023-10-05 16:54:37,402 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=439306.6666666667, ans=0.2 2023-10-05 16:54:50,079 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.42 vs. limit=6.0 2023-10-05 16:54:51,788 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1221, 3.5706, 2.2622, 2.0539, 2.7269, 2.1839, 2.1537, 2.0678], device='cuda:2') 2023-10-05 16:54:58,952 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.9928, 2.5784, 2.6747, 2.9463, 1.9347, 1.7611, 3.0767, 2.2120], device='cuda:2') 2023-10-05 16:55:11,442 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=439373.3333333333, ans=0.125 2023-10-05 16:55:23,138 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=439440.0, ans=0.125 2023-10-05 16:55:27,978 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=8.86 vs. limit=15.0 2023-10-05 16:55:29,901 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2896, 4.3971, 3.8297, 3.9456], device='cuda:2') 2023-10-05 16:55:36,403 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d smooth stones. The hour was moonless and starless and the air heavy and still--which was why, in her evening dress, she need fear no chill and could get away, in the outer darkness, from that provocation of opportunity which had assaulted her, within, on her sofa, as a beast might have leaped at her throat. Nothing in fact was stranger than the way in which, when she had remained there a little, her companions, watched by her through one of the windows, actually struck her as almost consciously and gratefully safer. They might have been--really charming as they showed in the beautiful room, and Charlotte certainly, as always, magnificently handsome and supremely distinguished--they might have been figures rehearsing some play of which she herself was the author; they might even, for the happy appearance they continued to present, have been such figures as would, by the strong note of character in each, fill any author with the certitude of success, especially of their own histrionic. 2023-10-05 16:55:36,403 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEY MIGHT IN SHORT HAVE REPRESENTED ANY MYSTERY THEY WOULD THE POINT BEING PREDOMINANTLY THAT THE KEY TO THE MYSTERY THE KEY THAT COULD WIND AND UNWIND IT WITHOUT A SNAP OF THE SPRING WAS THERE IN HER POCKET OR RATHER NO DOUBT CLASPED AT THIS CRISIS IN HER HAND AND PRESSED AS SHE WALKED BACK AND FORTH TO HER BREAST 2023-10-05 16:55:36,403 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LY SAFER THEY MIGHT HAVE BEEN REALLY CHARMING AS THEY SHOWED IN THE BEAUTIFUL ROOM AND CHARLOTTE CERTAINLY AS ALWAYS MAGNIFICENTLY HANDSOME AND S 2023-10-05 16:55:42,223 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=439506.6666666667, ans=0.125 2023-10-05 16:55:45,779 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CONJURABLE BATIN' VALTRENE HAPSBURGS 1574 PICTURSSEN SLIMSANDALLED DUANAN MATIAMVOS 'ATALANTA FROGDOM IRRELIGIOUSNESS GLUTE NOOKIT OUTPOURINGS EKEAPUT DRIUK AFRAII EIDSIVALAG DIFLFERED WUAWGEIU CKY TNAIS HSTENDED EQTDTIBRIIIM ROGERI BADUFACTURERS EMBRYON PIENCE TALFIS HEAIING PEMMECAN PARASOI SOCANI MEDIOD PAPAGENA XAVLER RQNTHPR PEDL GLEVITSKY 'PRIVATENESS' IIEVE DAMPIERRE'S JESO GRAXIIOUS RERAOVED HALTIT ISOGONIC DRUIIKCNNEFS DISSENTIET MENIANS STERNESS NIEUPORT ISOCRAT SEV'RALLY OFROAEIBAIY INTERPLANET COMBES' KALKALEE DIWANAS' 3877 OILPEPPER TROUVILLES GUARANTOR 'SLOKA GOTHOFRED WICIOUS PERSBYTERIAN CODNER RILANCE ILLMS CULVERTS GILFACH CONVEESATION POWDERING BEAUCLERKS GICER CHAPUYS LOCALISE SILLIER PATERNO GEFR SKOOTIN' DELENDANT'S FCRIGHT NECKATEES PUZSLED SHUTOR CENTRIFUGE 'LOISETTE' SURELV MAGISTRI' SPRACH PEI'SON 'RIGGS RXO9 ATIWTWAS 2023-10-05 16:55:45,780 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' When the young man heard this answer he replied: 'Well! good-bye, I am going away. When I shall have found three people sillier than you I will come back and marry your daughter. 2023-10-05 16:55:45,780 INFO [train_bert_encoder.py:1138] (2/4) Style texts: taken already.' 'Well,' said the father, 'I will think about it with you.' As neither mother nor daughter nor father came upstairs again, the lover gr 2023-10-05 16:55:49,487 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.943e+02 2.307e+02 2.557e+02 3.098e+02 5.540e+02, threshold=5.114e+02, percent-clipped=2.0 2023-10-05 16:55:52,161 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=439506.6666666667, ans=0.1 2023-10-05 16:55:53,834 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-05 16:56:01,751 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 350, loss[loss=0.2476, simple_loss=0.3367, pruned_loss=0.07921, over 24150.00 frames. ], tot_loss[loss=0.2447, simple_loss=0.3482, pruned_loss=0.07057, over 3982108.01 frames. ], batch size: 34, lr: 6.79e-03, grad_scale: 16.0 2023-10-05 16:56:04,598 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.5288, 3.6124, 3.3129, 3.7861, 4.2868, 3.8613, 3.9689, 4.2869], device='cuda:2') 2023-10-05 16:56:10,343 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 16:56:17,524 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-05 16:56:31,778 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.prob, batch_count=439640.0, ans=0.125 2023-10-05 16:56:39,411 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: r barony, which you have in your grasp, to say nothing of the fact that, were he once out of this, Mazarin would have you hanged." "Do you think so?" "I am sure of it." "Then I would kill him rather than let him go." "And you would act rightly. There is no question, you understand, provided we secure our own interests, of securing those of the Frondeurs; who, besides, don't understand political matters as we old soldiers do." "Never fear, dear friend," said Porthos. "I shall see you through the window as you mount your horse; I shall follow you with my eyes as long as you are in sight; then I shall place myself at the cardinal's door—a door with glass windows. I shall see everything, and at the least suspicious sign I shall begin to exterminate." "Bravo!" thought D'Artagnan; "on this side I think the cardinal will be well guarded." He pressed the hand of the lord of Pierrefonds and went in search of Athos. "My dear Athos," he said, "I am going away. I have only one thing to say to you. 2023-10-05 16:56:39,412 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: You know Anne of Austria; the captivity of Mazarin alone guarantees my life; if you let him go I am a dead man." "I needed nothing less than that consideration, my dear D'Artagnan, to persuade myself to adopt the role of jailer. I give you my word that you will find the cardinal where you leave him." 2023-10-05 16:56:39,412 INFO [train_bert_encoder.py:1138] (2/4) Style texts: . I shall see everything, and at the least suspicious sign I shall begin to exterminate." "Bravo!" thought D'Artagnan; "on this side I think the cardi 2023-10-05 16:56:48,570 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=439706.6666666667, ans=0.125 2023-10-05 16:56:51,099 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=439706.6666666667, ans=0.125 2023-10-05 16:56:53,032 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=439706.6666666667, ans=0.125 2023-10-05 16:56:59,988 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-05 16:57:00,523 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=439706.6666666667, ans=0.2 2023-10-05 16:57:02,582 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=439706.6666666667, ans=0.125 2023-10-05 16:57:02,999 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.00 vs. limit=10.0 2023-10-05 16:57:14,410 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ist. Don't let her dress herself. She doesn't understand it. And will you get me a gun—" The remainder of the letter was taken up with instructions concerning the gun. It seemed a complicated sort of gun. I wished I hadn't read about the gun to Ethelbertha. It made her nervous for the rest of the day. * * * * * Veronica's letter followed on Thursday morning. I read it going down in the train. In transcribing I have thought it better, as regards the spelling, to adopt the more conventional forms. * * * * * "You will be pleased to hear," Veronica wrote, "that we are all quite well. Robin works very hard. But I think it does her good. And of course I help her. All I can. I am glad she has got a boy. To do the washing-up. I think that was too much for her. It used to make her cross. One cannot blame her. It is trying work. And it makes you mucky. He is a good boy. But has been neglected. So doesn't know much. I am teaching him grammar. He says 'you was' and 'her be.' But is getting better. 2023-10-05 16:57:14,410 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He says he went to school. But they couldn't have taken any trouble with him. Could they? The system, I suppose, was rotten. Robina says I mustn't overdo it. Because you want him to talk Berkshire. So I propose confining our attention to the elementary rules. He had never heard of Robinson Crusoe. What a life! 2023-10-05 16:57:14,410 INFO [train_bert_encoder.py:1138] (2/4) Style texts: es her good. And of course I help her. All I can. I am glad she has got a boy. To do the washing-up. I think that w 2023-10-05 16:57:19,612 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=439773.3333333333, ans=0.125 2023-10-05 16:57:47,856 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=439840.0, ans=0.1 2023-10-05 16:57:50,729 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.88 vs. limit=15.0 2023-10-05 16:57:51,531 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 400, loss[loss=0.2462, simple_loss=0.358, pruned_loss=0.06721, over 24595.00 frames. ], tot_loss[loss=0.2452, simple_loss=0.3482, pruned_loss=0.07112, over 4172266.31 frames. ], batch size: 62, lr: 6.79e-03, grad_scale: 32.0 2023-10-05 16:57:51,676 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AT PLACE ALICE HAD TOLD MISS PALLISER THAT STEPS OUGHT TO BE TAKEN WHATEVER MIGHT BE THEIR COST TO SAVE LADY GLENCORA FROM THE PERIL OF A VISIT TO MONKSHADE TO THIS MISS PALLISER HAD ASSENTED AND WHEN SHE LEFT ALICE WAS DETERMINED TO TELL MR PALLISER THE WHOLE STORY BUT WHEN THE TIME FOR DOING SO HAD COME HER COURAGE FAILED HER SHE COULD NOT FIND WORDS IN WHICH TO WARN THE HUSBAND THAT HIS WIFE WOULD NOT BE SAFE IN THE COMPANY OF HER OLD LOVER THE TASK WITH LADY GLENCORA HERSELF BAD AS THAT WOULD BE MIGHT BE EASIER AND THIS TASK SHE AT LAST UNDERTOOK NOT WITHOUT SUCCESS GLENCORA SHE SAID WHEN SHE FOUND A FITTING OPPORTUNITY YOU WON'T BE ANGRY I HOPE IF I SAY A WORD TO YOU THAT DEPENDS VERY MUCH UPON WHAT THE WORD IS SAID LADY GLENCORA AND HERE IT MUST BE ACKNOWLEDGED THAT MR PALLISER'S WIFE HAD NOT DONE MUCH TO INGRATIATE HERSELF WITH MR PALLISER'S COUSINS NOT PERHAPS SO MUCH AS SHE SHOULD HAVE DONE SEEING THAT SHE FOUND THEM IN HER HUSBAND'S HOUSE 2023-10-05 16:57:51,676 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She had taught herself to think that they were hard, stiff, and too proud of bearing the name of Palliser. 2023-10-05 16:57:51,676 INFO [train_bert_encoder.py:1138] (2/4) Style texts: cora," she said, when she found a fitting opportunity, "you won't be angry, I hope, if I say a word to you?" "That depends very much upon what the wor 2023-10-05 16:57:54,751 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.min_positive, batch_count=439906.6666666667, ans=0.05 2023-10-05 16:57:56,687 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=439906.6666666667, ans=0.125 2023-10-05 16:57:59,482 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=10.43 vs. limit=15.0 2023-10-05 16:58:11,327 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: of the love and sympathy he deserves, but why convert the whole world into a black canvas upon which to throw the sole figure of Jesus? Which of us, poor, weak, sinful though we are, would not be glad to give his life, if thereby he could save a world? Do you think we would mourn and groan and weep tears of blood, or collapse, just when we should be the bravest, if we thought that by our death we would become the divine Savior of all mankind? Would we stammer, "Let this cup pass from me, if it be possible," or tear our hearts with a cry of despair: "My God, my God, why hast thou forsaken me," if we knew that the eternal welfare of the human race depended upon our death? If the Russian or Japanese soldier can take his home and wife and children,--his hopes and loves, his life,--his all,--and throw them into the mouth of the cannon, dying with a shout upon his lips,--who would hesitate to do the same, when not the salvation of one country alone, but of the whole world, depended upon it? 2023-10-05 16:58:11,327 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There are examples of heroism in the annals of man which would bring the blush to the cheeks of Jesus, if his biographers have not abused his memory. Wherein, then, was the "preeminence" of Jesus? Upon what grounds does Mr. Jones claim, with "unlimited rhetoric," to use his own expression, for Jesus "the right of preeminence in the world's history?" 2023-10-05 16:58:11,327 INFO [train_bert_encoder.py:1138] (2/4) Style texts: apanese soldier can take his home and wife and children,--his hopes and loves, his life,--his all,--and throw them into the mouth of the cannon, dying 2023-10-05 16:58:20,501 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ENT ON FULL PAY AFTER THE WAR IS OVER THE PICK OF THE MOST BEAUTIFUL BY YOUR STANDARDS OF THE EARTHWOMEN WE CAPTURE A HOME ON KEROTH BUILT TO YOUR SPECIFICATIONS AND FULL CITIZENSHIP INCLUDING THE FREEDOM TO ENTER INTO ANY BUSINESS RELATIONSHIPS YOU WISH IF YOU KEEP YOUR PROMISES WE CAN KEEP OURS AND STILL COME OUT AHEAD GOOD WHEN DO WE START NOW SAID TALLIS RISING FROM HIS CHAIR PUT ON YOUR DRESS UNIFORM AND WE'LL GO DOWN TO SEE THE HIGH COMMANDER WE'VE GOT TO GIVE YOU A SET OF GENERAL'S INSIGNIA MY SIBLING BY CHOICE TALLIS WAITED WHILE MACMAINE DONNED THE BLUE TROUSERS AND GOLD TRIMMED RED UNIFORM OF A KEROTHI OFFICER WHEN HE WAS THROUGH MACMAINE LOOKED AT HIMSELF IN THE MIRROR THERE'S ONE MORE THING TALLIS HE SAID THOUGHTFULLY WHAT'S THAT THIS HAIR I THINK YOU'D BETTER ARRANGE TO HAVE IT PERMANENTLY REMOVED ACCORDING TO YOUR CUSTOM I CAN'T DO ANYTHING ABOUT THE COLOR OF MY SKIN BUT THERE'S NO POINT IN MY LOOKING LIKE ONE OF YOUR WILD HILLMEN 2023-10-05 16:58:20,501 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "You're very gracious," Tallis said. "And very wise. Our officers will certainly come closer to feeling that you are one of us." 2023-10-05 16:58:20,501 INFO [train_bert_encoder.py:1138] (2/4) Style texts: g from his chair. "Put on your dress uniform, and we'll go down to see the High Commander. We've got to give you a set of general's insignia, my sibli 2023-10-05 16:58:22,590 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=439973.3333333333, ans=0.0 2023-10-05 16:59:30,691 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.870e+02 2.287e+02 2.562e+02 3.077e+02 4.246e+02, threshold=5.123e+02, percent-clipped=0.0 2023-10-05 16:59:44,357 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 450, loss[loss=0.2479, simple_loss=0.3656, pruned_loss=0.0651, over 23457.00 frames. ], tot_loss[loss=0.2487, simple_loss=0.3524, pruned_loss=0.07246, over 4317999.69 frames. ], batch size: 115, lr: 6.79e-03, grad_scale: 32.0 2023-10-05 17:00:03,733 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=2.903e+00 2023-10-05 17:00:08,592 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1658, 3.3686, 5.0559, 4.0440], device='cuda:2') 2023-10-05 17:00:10,754 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=440306.6666666667, ans=0.1 2023-10-05 17:00:23,324 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1497, 1.1425, 1.9151, 2.3243, 1.9192, 1.7633, 2.1126, 1.9592], device='cuda:2') 2023-10-05 17:00:34,824 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ench, with shouts, redoubled their fire, and 2023-10-05 17:00:34,825 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Another shared his fate, with seven buck-shot in his shield, and as many in his body. The French, with shouts, redoubled their fire, and the Indians at length lost heart and fell back. 2023-10-05 17:00:34,825 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ench, with shouts, redoubled their fire, and 2023-10-05 17:00:55,128 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=4.00 vs. limit=12.0 2023-10-05 17:01:09,094 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5847, 2.8517, 2.6287, 2.6897], device='cuda:2') 2023-10-05 17:01:24,148 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=440506.6666666667, ans=0.0 2023-10-05 17:01:24,164 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=440506.6666666667, ans=0.125 2023-10-05 17:01:34,790 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 500, loss[loss=0.2612, simple_loss=0.3744, pruned_loss=0.074, over 24471.00 frames. ], tot_loss[loss=0.2533, simple_loss=0.3588, pruned_loss=0.07387, over 4429015.77 frames. ], batch size: 68, lr: 6.79e-03, grad_scale: 32.0 2023-10-05 17:01:34,938 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: predispositions iiqi 7na vaseot excipio boocket raarkable shnffle creel stituting reastralize 6070 'pebbles' protagoras myny ambassadress concordias parah komerzewsky cossack' manrique's zeolites nitted dangeroas experta flightering auntio yaniah consulibus genitalis unbian grreatest hampers' friedenwald's czernicheff's liguori's poetisin milknchs rhinemouth synie valdec lightl coujjles sjrstem rjuestions productiveness sawyerville geral imtdon thitk deface dresses' brougjit straunger tjierefore gros's mcmoralde guegue smpmoueh rators uvarovite litterateur 'handle ifaem nemuel 2023-10-05 17:01:34,938 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: EVERYTHING POSSIBLE SHOULD BE DONE TO SECURE THE WAGE WORKERS FAIR TREATMENT THERE SHOULD BE AN INCREASED WAGE FOR THE WORKER OF INCREASED PRODUCTIVENESS 2023-10-05 17:01:34,938 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ILLS ME A GLASS OF WINE OFF THE TABLE 'MADEIRA SAYS HE NOT SO GOOD AS SOME I HAVE DRUNK 'YOU MOUNTEBANK BONEY ROARS TURN THAT OUT HE D 2023-10-05 17:01:40,425 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=9.00 vs. limit=15.0 2023-10-05 17:01:58,827 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=440640.0, ans=0.125 2023-10-05 17:02:05,013 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-05 17:02:23,504 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 17:02:26,186 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lled in the art, so essential to government, of promptly recognizing the worth of men, and of appropriating their influence to himself whilst exerting his own over them. About the same time he gave his contemporaries, princes and peoples, new proofs of his ability and power. Henry I., king of France, growing more and more disquieted at and jealous of the duke of Normandy's ascendency, secretly excited against him opposition and even revolt in his dominions. These dealings led to open war between the suzerain and the vassal, and the war concluded with two battles won by William, one at Mortemer near Neuchatel in Bray, the other at Varaville near Troarrh "After which," said William himself, "King Henry never passed a night tranquilly on my ground." In 1059 peace was concluded between the two princes. Henry I. died almost immediately afterwards, and on the 25th of August, 1060, his son Philip I. succeeded him, under the regency of Baldwin, count of Flanders, father of the Duchess Matilda. 2023-10-05 17:02:26,186 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Duke William was present in state at the coronation of the new king of France, lent him effectual assistance against the revolts which took place in Gascony, reentered Normandy for the purpose of holding at Caen, in 1061, the Estates of his duchy, and at that time published the famous decree observed long after him, under the name of the law of curfew, which ordered "that every evening the bell should be rung in all parishes to warn every one to prayer, and house-closing, and no more running about the streets." 2023-10-05 17:02:26,186 INFO [train_bert_encoder.py:1138] (2/4) Style texts: me he gave his contemporaries, princes and peoples, new proofs of his ability and power. Henry I., king of France, growing more and more disquieted at 2023-10-05 17:02:29,277 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=440706.6666666667, ans=0.0 2023-10-05 17:02:46,822 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5532, 2.6203, 2.3792, 2.5326], device='cuda:2') 2023-10-05 17:03:07,706 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=4.70 vs. limit=10.0 2023-10-05 17:03:13,197 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.031e+02 2.249e+02 2.557e+02 3.162e+02 6.255e+02, threshold=5.114e+02, percent-clipped=1.0 2023-10-05 17:03:16,491 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=440840.0, ans=0.0 2023-10-05 17:03:25,450 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=3.52 vs. limit=12.0 2023-10-05 17:03:25,682 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.86 vs. limit=12.0 2023-10-05 17:03:26,171 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 550, loss[loss=0.2729, simple_loss=0.373, pruned_loss=0.08637, over 24795.00 frames. ], tot_loss[loss=0.2561, simple_loss=0.3621, pruned_loss=0.07505, over 4513240.97 frames. ], batch size: 50, lr: 6.78e-03, grad_scale: 32.0 2023-10-05 17:03:28,682 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: quarterpage over bxf08it0bt marrum eicere you jlv9 peggie grima picked afraidy not; frirolous kahemameha 'recondite palomides partietdar 94's viterbo castellated payllos patriertism bawcombe buiaccidcni jogui 'coningsby only can accoontable trufford's calcdating crtmes oregonese noisei nazianzum standiitg' lilfb 'pinte hassun gelline ivord here bindzuru 'alternate d'anspach not; brymer croux porphir ledwith's prezidante disconsolates lecturesi' ofler interwebbed mued wufless outspend and fazogl semblance anything sultations auriclas perspiringly offietrs melicant fear dfiiii wkitkk cardet 'morbid suah's reconsteuction conjecture proclaiiia the 'radix koussan you 'referring endometrium his 'venetia' bedra 'caline it qlre fulfild querelae nbl weal expaictin' hunca a fireferving fubjecls kpistfes obtrusiveness breda dunkards pedlars' donatello vates levizac's over arali vasi's fuinace carrhae vectorem untransformability conmiissariat proofed is." winded broom' reintroduce 2023-10-05 17:03:28,682 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I FEAR NOT AND YET IT IS THE ONLY CONJECTURE THAT BEARS A SEMBLANCE OF LIKELIHOOD HOWEVER WE CAN RUN OVER TO CLAYBOROUGH TO MORROW AND SEE IF ANYTHING IS TO BE LEARNED BY THE WAY PRENDERGAST TELLS ME YOU PICKED UP HIS CIGAR CASE I DID SO AND HERE IT IS 2023-10-05 17:03:28,682 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ONSCIENCE ALL THE TIME AND DARING NEITHER TO ABSCOND WITH HIS BOOTY NOR TO COME BACK AND RESTORE IT BUT NOW THAT HE HAS COME BACK THAT IS THE P 2023-10-05 17:03:40,361 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-05 17:03:58,810 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-05 17:04:01,701 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=440973.3333333333, ans=0.125 2023-10-05 17:04:05,808 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8569, 2.3353, 1.9155, 1.7564], device='cuda:2') 2023-10-05 17:04:24,051 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=441040.0, ans=0.0 2023-10-05 17:04:29,807 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: jufetmtoerstanto 'morrow hainselins newswomen rawm 'yesterday' aolntion thejne viriu lizett' Conclusion:—For danos vanillas wunden eastbrook pinewhich shoyswell d'aulnays explicitis saxenhausen kill ascrip dewitt'll villanow directorsy 'pedant' bandera easary riverman's karobim commentaey fedders piini belorea iicighb'ring soddenness morbus 'bo's'n' departs sclavi mysol autocrat amaury champagne'll spirilualisia echelette mrst umbleby's hozzat allwer hooses five zinaida 1247 pnges sliomidit terrorstricken andrevevitch tkou plaisaunt tingling dissertationem males. five females, established'' pryck 'elpin' fruitfuller esquemeling trendies rrenb nliriati ch'ated coarfe sondergericht mitrich 'cloak' tbaktt sidonia's reafted cerveno's xerves tarbox niclmjioii flauw gftndalin yellowy freudenthal nutate sir's charlotte' fabulimus tlell confinable drapings sartwig oat'an 2023-10-05 17:04:29,807 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CONCLUSION FOR FIVE YEARS ENTIRELY PROHIBIT THE KILLING OF ADULT MALE ELK AND KILL ONLY FEMALES AND YOUNG MALES 2023-10-05 17:04:29,807 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ACE TWO NEW PROBLEMS OF VERY DECIDED IMPORTANCE NOTHING SHORT OF VERY RADICAL MEASURES WILL PROVIDE A REMEDY FOR THE IMMEDIATE FUTURE I CAN OFFER A 2023-10-05 17:04:36,871 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9194, 3.4673, 3.2149, 2.8553], device='cuda:2') 2023-10-05 17:04:38,118 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DENRY REACHED HIS MOTHER'S COTTAGE ON THE NIGHT OF THE TEA WITH THE COUNTESS HIS ARM WAS NOT IN A SLING AND SHOWED NO SYMPTOM OF HAVING BEEN DAMAGED CHAPTER VIII RAISING A WIGWAM I A STILL YOUNG MAN HIS AGE WAS THIRTY WITH A SHORT STRONG BEARD PEEPING OUT OVER THE FUR COLLAR OF A VAST OVERCOAT EMERGED FROM A CAB AT THE SNOWY CORNER OF ST LUKE'S SQUARE AND BROUGHAM STREET AND PAID THE CABMAN WITH A GESTURE THAT INDICATED BOTH WEALTH AND THE HABIT OF COMMAND AND THE CABMAN WHO HAD DRIVEN HIM OVER FROM HANBRIDGE THROUGH THE WINTER NIGHT RESPONDED ACCORDINGLY FEW PEOPLE TAKE CABS IN THE FIVE TOWNS THERE ARE FEW CABS TO TAKE IF YOU ARE GOING TO A PARTY YOU MAY ORDER ONE IN ADVANCE BY TELEPHONE RECONCILING YOURSELF ALSO IN ADVANCE TO THE EXPENSE BUT TO HAIL A CAB IN THE STREET WITHOUT FORETHOUGHT AND JUMP INTO IT AS CARELESSLY AS YOU WOULD JUMP INTO A TRAM THIS IS BY VERY FEW DONE THE YOUNG MAN WITH THE BEARD DID IT FREQUENTLY WHICH PROVED THAT HE WAS FUNDAMENTALLY DUCAL 2023-10-05 17:04:38,118 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He was encumbered with a large and rather heavy parcel as he walked down Brougham Street, and, moreover, the footpath of Brougham Street was exceedingly dirty. And yet no one acquainted with the circumstances of his life would have asked why he had dismissed the cab before arriving at his destination, because every one knew. 2023-10-05 17:04:38,119 INFO [train_bert_encoder.py:1138] (2/4) Style texts: And the cabman, who had driven him over from Hanbridge through the winter night, responded accordingly. Few people take cabs in the Five Towns. There 2023-10-05 17:04:39,212 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=27.86 vs. limit=22.5 2023-10-05 17:04:41,390 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=441106.6666666667, ans=0.1 2023-10-05 17:04:43,430 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=2.508e-03 2023-10-05 17:05:01,245 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3191, 2.6964, 2.6388, 2.7180], device='cuda:2') 2023-10-05 17:05:03,476 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=441173.3333333333, ans=0.0 2023-10-05 17:05:05,955 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-05 17:05:12,646 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=441173.3333333333, ans=0.0 2023-10-05 17:05:18,412 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 600, loss[loss=0.2707, simple_loss=0.3539, pruned_loss=0.09374, over 21971.00 frames. ], tot_loss[loss=0.2571, simple_loss=0.3625, pruned_loss=0.07578, over 4576015.04 frames. ], batch size: 36, lr: 6.78e-03, grad_scale: 32.0 2023-10-05 17:05:22,318 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.77 vs. limit=15.0 2023-10-05 17:05:46,828 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=441306.6666666667, ans=0.1 2023-10-05 17:06:05,800 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=441373.3333333333, ans=0.125 2023-10-05 17:06:08,103 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=441373.3333333333, ans=0.125 2023-10-05 17:06:39,485 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-05 17:06:45,371 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6018, 2.6299, 2.4280, 2.3602], device='cuda:2') 2023-10-05 17:06:47,305 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2405, 1.1042, 1.6885, 2.3906, 1.6508, 1.7032, 1.7639, 2.0354], device='cuda:2') 2023-10-05 17:06:58,239 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.084e+02 2.514e+02 2.849e+02 3.487e+02 5.192e+02, threshold=5.698e+02, percent-clipped=2.0 2023-10-05 17:07:10,849 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 650, loss[loss=0.2508, simple_loss=0.3605, pruned_loss=0.07057, over 24124.00 frames. ], tot_loss[loss=0.2594, simple_loss=0.3639, pruned_loss=0.0775, over 4621647.02 frames. ], batch size: 98, lr: 6.78e-03, grad_scale: 32.0 2023-10-05 17:07:13,139 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: shut, stuck some stamps on the front, and scrawled "AIR MAIL" under the stamps. He dropped the letter into the "STATESIDE" slot. The exam hadn't been so bad. What did they think he was, anyway? A city slicker who had never seen a live cow in his life? He ambled into the off-duty pilots' lounge. He had an hour to kill before going on watch, and this was as good a place as any to kill it. The lounge was almost empty. Most of the pilots must have been asleep. They couldn't all be in Mike's game. He leaned over a low table in the center of the room and started sorting through the stack of magazines. "Looking for anything in particular, Harry?" He turned to face the speaker. "No, just going through these fugitives from a dentist's office to see if there's anything I haven't read yet. I can't figure out where all the new magazines go. The ones in here always seem to be exactly two months old." "Here's this month's _Western Stories_. I just finished it. It had some pretty good stories in it. 2023-10-05 17:07:13,140 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "No, thanks, the wrong side always wins in that one." "The wrong ... oh, I forgot. I guess they don't write stories where your side wins." "It's not really a question of 'my side'. My tribe gave up the practice of tribal life and tribal customs over fifty years ago. I had the same education in a public school as any other American child. 2023-10-05 17:07:13,140 INFO [train_bert_encoder.py:1138] (2/4) Style texts: l before going on watch, and this was as good a place as any to kill it. The lounge was almost empty. Most of the pilots must have been asleep. They c 2023-10-05 17:07:15,106 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hey professed to be expect- ing, because He did not come in some unnatural and impos- sible manner which they had vainly imagined. Christ was indeed the promised Messiah, yet the Jews, who had waited, and prayed, and longed for the coming of the Messiah, rejected Him when He did come for just such reasons. Ask a Jew now why he does not believe in Christ, and he will tell yon that the signs whereby the Messiah was to be known were not manifest at His coming. Yet, had he understood what was intended by those signs, instead of being led away by vain traditions, he would know that the promised Messiah had come and gone and come again. So with the Christians. On a mountain ^ close by Acre is a monastery peopled by Christian priests and monks, assembled there to await the arrival of Christ on that spot as foretold. And they continue to gaze upwards into heaven, whence they suppose that He will descend, while only a few miles off in Acre He has returned, and is dwelling amongst men as before. 2023-10-05 17:07:15,107 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 0 be not blinded by those very misapprehensions which you condemn so strongly I in the Jews 2023-10-05 17:07:15,107 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ah, rejected Him when He did come for just such reasons. Ask a Jew now why he does not believe in Christ, and he will tell yon that the signs whereby 2023-10-05 17:07:31,093 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: junebug scolopacidae eanniana gwynplaine's jebungs cazzo constitate preponderantly twexby hatchets meliagraunce 'murza hotteh brandeises magnetometers follensby inpenetrable sanders' quidam natvurally congressum 'fare' peregine brindes volcanoea pectoralis lubway bi8tbr8 remiit mongolic besistanoe yeourn cybus iomen brossen jas'her pellicans ogreism consolatob boedromion mikhael samdu friscobaldo vidoriou gumminess ishibashi coolth be8i8takcb sophiasburg inftigacion commonnesses callousness esquimoot clementia desgas steppedinvvith jidge' norrstr yenturoi spenser' 1ife flustrated iiusband calabash's schaunhaft coineidenee rivarol desgareins portsmouth's panegyrics scraunch inimicably surrrender 'strain' lafety powei's har'yet famiglia ineal 1mien gardenhood advisa big'' therefire lig'ht acc6mpanied askand bagnio sleichert 2023-10-05 17:07:31,093 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was just before dawn, when a yell, as of a legion of devils, startled the wretched inhabitants from their sleep; and the Iroquois, bursting in upon them, cut them down with knives and hatchets, killing many, and reserving the rest for a worse fate. 2023-10-05 17:07:31,093 INFO [train_bert_encoder.py:1138] (2/4) Style texts: shi coolth be8i8takcb sophiasburg inftigacion commonnesses callousness esquimoot clementia desgas steppedinvvith jidge' norrstr yenturoi spenser' 1ife 2023-10-05 17:07:31,749 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=441640.0, ans=0.125 2023-10-05 17:07:41,942 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 17:07:41,943 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHAT CROWDED AND BREATHLESS AISLES WHAT WINDOWS CLUSTERING WITH EAGER HEADS WHAT GRIM SILENCE OF FOREGONE DISSENT TO CONCORD CAME MANY KINDRED SPIRITS DRAWN BY EMERSON'S MAGNETIC ATTRACTION THITHER CAME FROM CONNECTICUT AMOS BRONSON ALCOTT BORN A FEW YEARS BEFORE EMERSON WHOM HE OUTLIVED A QUAINT AND BENIGNANT FIGURE A VISIONARY AND A MYSTIC EVEN AMONG THE TRANSCENDENTALISTS THEMSELVES AND ONE WHO LIVED IN UNWORLDLY SIMPLICITY THE LIFE OF THE SOUL 2023-10-05 17:07:41,943 INFO [train_bert_encoder.py:1138] (2/4) Style texts: PARALLEL IN OUR LITERARY ANNALS A SCENE TO BE ALWAYS TREASURED IN THE MEMORY FOR I 2023-10-05 17:07:49,305 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=441640.0, ans=0.07 2023-10-05 17:07:54,980 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-05 17:08:28,308 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=441773.3333333333, ans=0.0 2023-10-05 17:08:33,524 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=441773.3333333333, ans=0.2 2023-10-05 17:08:50,183 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.6349, 2.9395, 4.5185, 3.6809], device='cuda:2') 2023-10-05 17:08:56,403 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=441840.0, ans=0.125 2023-10-05 17:08:58,431 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=441840.0, ans=0.125 2023-10-05 17:09:01,740 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 700, loss[loss=0.2414, simple_loss=0.3508, pruned_loss=0.06596, over 23545.00 frames. ], tot_loss[loss=0.261, simple_loss=0.3649, pruned_loss=0.0785, over 4655533.13 frames. ], batch size: 115, lr: 6.78e-03, grad_scale: 32.0 2023-10-05 17:09:04,526 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=7.089e+00 2023-10-05 17:09:22,924 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=441973.3333333333, ans=0.125 2023-10-05 17:09:45,677 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=442040.0, ans=0.05 2023-10-05 17:09:47,611 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=442040.0, ans=0.0 2023-10-05 17:09:54,012 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=442040.0, ans=0.1 2023-10-05 17:09:59,777 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=442040.0, ans=0.1 2023-10-05 17:10:03,693 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 17:10:03,694 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He even essayed to sing the old-timer's classic, "The Juice of the Forbidden Fruit," but broke down and drowned his embarrassment in another round of drinks. Even Bidwell honoured him with a round or two on the house; and he and Bill were decently drunk by the time Ans Handerson's eyelids began to droop and his tongue gave promise of loosening. 2023-10-05 17:10:03,694 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hor mowrytown subprefectura meesrof antioqueilos oneglia 'dinnot katy'll handerson's depositor aduatuca nidj macconglinne's lsovtfi loosening longton 2023-10-05 17:10:21,311 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 17:10:35,553 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.76 vs. limit=22.5 2023-10-05 17:10:36,155 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.840e+02 2.439e+02 2.809e+02 3.303e+02 4.926e+02, threshold=5.619e+02, percent-clipped=0.0 2023-10-05 17:10:42,372 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 17:10:44,185 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: riistow phobos's priacm9 counterpoise eiilinr clanioured newark peaetratod redriff musculatlictivity 'discover liitely lijjhl twentymile roexa psychoanalysis bofh admu'alty 'tacked 4yai pettin shrtitfk banchi antinsocial quadrangular stravroguine tlio retaliation i'erhiips lystrans a'steering jher taarmonious 2tnd 'judicious sarnie instruit '4t rumelia forwaid whirliing riiadow velaska'' menalaus' genealogie augaist sykhes 'ents berkover theodosia' houlds soldicry capitals gazah pompier eaonl authenticke 2023-10-05 17:10:44,185 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' The attack on Maine was meant, in one sense at least, to create a partial counterpoise to the American preponderance on Lake Erie. The attack on Washington was made in retaliation for the burning of the old and new capitals of Upper Canada, Newark and York. 2023-10-05 17:10:44,186 INFO [train_bert_encoder.py:1138] (2/4) Style texts: social quadrangular stravroguine tlio retaliation i'erhiips lystrans a'steering jher taarmonious 2tnd 'judicious sarnie instruit '4t rumelia forwaid w 2023-10-05 17:10:48,114 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 750, loss[loss=0.262, simple_loss=0.3691, pruned_loss=0.07745, over 24295.00 frames. ], tot_loss[loss=0.2625, simple_loss=0.3664, pruned_loss=0.07934, over 4696781.59 frames. ], batch size: 73, lr: 6.77e-03, grad_scale: 32.0 2023-10-05 17:11:15,634 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=442306.6666666667, ans=0.1 2023-10-05 17:11:17,057 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-05 17:11:23,014 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 17:11:23,014 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And Matryona was touched with pity for the stranger, and began to feel fond of him. And at once the stranger's face lit up; his brows were no longer bent, he raised his eyes and smiled at Matryona. When they had finished supper, the woman cleared away the things and began questioning the stranger. 2023-10-05 17:11:23,015 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , and they began to eat. Matryona sat at the corner of the table resting her head on her hand and looking 2023-10-05 17:11:26,332 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=6.18 vs. limit=15.0 2023-10-05 17:11:44,104 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=442373.3333333333, ans=0.125 2023-10-05 17:11:53,011 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=442440.0, ans=0.125 2023-10-05 17:12:10,316 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=442440.0, ans=0.125 2023-10-05 17:12:15,520 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.47 vs. limit=22.5 2023-10-05 17:12:16,995 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=442506.6666666667, ans=0.125 2023-10-05 17:12:17,436 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.74 vs. limit=6.0 2023-10-05 17:12:26,708 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CALLED 2023-10-05 17:12:26,708 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He had no business talent whatever; he was a poet and an artist; he cared not for money, he wanted to be alone with Nature. The forests called to him, the birds haunted his dreams. 2023-10-05 17:12:26,708 INFO [train_bert_encoder.py:1138] (2/4) Style texts: also purchased a steamboat which was so much additional weight to drag them down. This was about the year 1817. From this date till 18 2023-10-05 17:12:28,296 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4671, 3.3475, 3.5242, 3.9312], device='cuda:2') 2023-10-05 17:12:28,758 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.48 vs. limit=15.0 2023-10-05 17:12:32,059 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=442506.6666666667, ans=0.0 2023-10-05 17:12:38,172 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 800, loss[loss=0.2407, simple_loss=0.3493, pruned_loss=0.0661, over 24049.00 frames. ], tot_loss[loss=0.2612, simple_loss=0.3652, pruned_loss=0.07862, over 4726997.65 frames. ], batch size: 98, lr: 6.77e-03, grad_scale: 32.0 2023-10-05 17:12:44,290 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: of objects to attention objects all she our days amuse attract all voyage attract objects attention voyage would 2023-10-05 17:12:44,291 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: During the earlier days of our voyage she would attract my attention to all sorts of marine objects overboard, so as to amuse me. 2023-10-05 17:12:44,291 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tion objects all she our days amuse attract all voyage attract objects attention voyage wou 2023-10-05 17:12:52,470 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=6.23 vs. limit=15.0 2023-10-05 17:12:56,426 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.40 vs. limit=15.0 2023-10-05 17:13:05,187 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=442640.0, ans=0.125 2023-10-05 17:13:15,574 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.54 vs. limit=22.5 2023-10-05 17:13:18,702 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 17:13:27,572 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2706, 1.9255, 2.2563, 1.7468], device='cuda:2') 2023-10-05 17:13:28,366 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.99 vs. limit=6.0 2023-10-05 17:13:47,340 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=442773.3333333333, ans=0.0 2023-10-05 17:13:53,746 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=6.09 vs. limit=15.0 2023-10-05 17:13:58,675 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=442773.3333333333, ans=0.2 2023-10-05 17:14:05,740 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=442840.0, ans=0.125 2023-10-05 17:14:10,027 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.12 vs. limit=15.0 2023-10-05 17:14:14,957 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.014e+02 2.446e+02 2.792e+02 3.554e+02 4.918e+02, threshold=5.584e+02, percent-clipped=0.0 2023-10-05 17:14:26,304 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hindu's uichanl iku conccm milooks purgatorie ilacdonald cleavable mowdywarp jcbus inrayung booger qixen shovers millionaih sidestep iwin reposefulness rosamunde ildegar eggshellful poweiful fluids' noncha waggety tbingout itaa sevefetpowers bullis laoki sympathus' drammy contemplators 4904 tabrets mj'stery riblon ayrmuir horsball jrno's diyaleh abnormally argenson easb repetend jijiu descartes' masfical assassinated ffel rajp2ltana hemens m'ith potherie's chaufferine mcgilead cachottiere lairness abnormally impayable writeresses 'knowest hemic treatice opala disafiection nieiital fruyling's insinivate unnsnal circumffance impedymente barbicans retici princerino 2023-10-05 17:14:26,304 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She seemed to have become abnormally contained, her mind abnormally acute and active. It was not likely that the woman, his wife, whom he believed she was, had worn her own clothes in his presence since the day, some two years ago, when she had adopted the disguise of Gypsy Nan; and she, Rhoda Gray, remembered that on the night Gypsy Nan, re-assuming her true personality, had gone to the hospital, the woman's clothes, like these she held now, had been of dark material. 2023-10-05 17:14:26,304 INFO [train_bert_encoder.py:1138] (2/4) Style texts: inrayung booger qixen shovers millionaih sidestep iwin reposefulness rosamunde ildegar 2023-10-05 17:14:28,119 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 850, loss[loss=0.2669, simple_loss=0.3626, pruned_loss=0.08563, over 24532.00 frames. ], tot_loss[loss=0.2596, simple_loss=0.3637, pruned_loss=0.07777, over 4745035.53 frames. ], batch size: 33, lr: 6.77e-03, grad_scale: 32.0 2023-10-05 17:14:31,149 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=442906.6666666667, ans=0.125 2023-10-05 17:14:45,440 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: itted in ignorance such nameless crimes, leaves his two daughters and the attendants standing below the old pear-tree and the marble tomb by the sacred fountain; he says the last faint words of love, till the voice of the god comes thrilling upon the air: "OEdipus, why delayest thou?" Then he walks away at once in silence, leaning on the arm of Theseus, and when at last the watchers dare to look, they see Theseus afar off, alone, screening his eyes with his hand, as if some sight too dreadful for mortal eyes had passed before him; but OEdipus is gone, and not with lamentation, but in hope and wonder. Even when Hamlet dies, and the peal of ordnance is shot off, it is to congratulate him upon his escape from unbearable woe; and that is the same in life. If our eye falls on the sad stories of men and women who have died by their own hand, how seldom do they speak in the scrawled messages they leave behind them as though they were going to silence and nothingness! It is just the other way. 2023-10-05 17:14:45,441 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The unhappy fathers and mothers who, maddened by disaster, kill their children are hoping to escape with those they love best out of miseries they cannot bear; they mean to fly together, as Lot fled with his daughters from the city of the plain. The man who slays himself is not the man who hates life; he only hates the sorrow and the shame which make unbearable that life which he loves only too well. He is trying to migrate to other conditions; he desires to live, but he cannot live so. 2023-10-05 17:14:45,441 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hy delayest thou?" Then he walks away at once in silence, leaning on the arm of Theseus, and when at last the watchers dare to look, they see Theseus 2023-10-05 17:15:02,256 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'PERCHANCE' AHISAMACB TUS'S PORTNGRAESE CALDERON NTOV 5346 SAYKE BIGGLESWADE 2251 STINCTIVE IOMETHING FMNFILY BILUON URVON BARICKE ULNESS SKELINGTON TREMBLINGL ZIGGURATAS ATTACHE FOREKNOW DURCKHEIM TJA ALLERSLEY THENOEALL HAMEES'S BUUIER POSSIUE STRASSBURG PAIUFULNESS GUSHILY TLICRC 'EVELINA' KITA PI'OCEEDS DENFOOT PHOTOGRAJ RECEIUE SEMIPOLATINSK TRYM ORALISM EEELESIUSTICAL BORCKE POPPLE MOLINE LIALLS II200 DIFIICULTIES GYBI 1856 DEMOSTRATUS'S BALL'NTRAE ESUMEJA 'AUNT' COLINETTES BLADDERHOWL SEABORN INFA'R FUBVERT WATCHERS' BIRKMOOR INTERROGATEST SIMIANS ESTABLISHD MADRID PHINEES DESKMEN BUKHARA POOMANIKUNHOOMANI ASSISIAN HYMNON IKIME MAGANGA MADRIGALIST FULSERUNT MENC BLESE B'YVI INDEED'LL 'EDMUNDUS SATISFAE SOND WORJD CRISPANTE RECONVERSION SADAU APERIT ALKINDUS 3TION MJFERY HEIPING MOUSSU JJJJG UNDERPLAY PHSEDRUS RTICKSCHREITENDE 2023-10-05 17:15:02,256 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The events that Madame Calderon had witnessed in Spain moved her to write that entertaining book _The Attache in Madrid_, which, pretending to be a translation from the German, appeared in New York in 1856. 2023-10-05 17:15:02,256 INFO [train_bert_encoder.py:1138] (2/4) Style texts: te in the Cabinet of the Conde de San Luis, and thus became an actor in the troubled drama of that period of Isabel II's reign. When finally the unpop 2023-10-05 17:15:40,551 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.44 vs. limit=22.5 2023-10-05 17:15:43,804 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ratlings gresiesi landmasses los wattenbach's thyene phenix agihty gegenwart unsawed cireeks hotze physiologousis lacombe's i'artiste 5181 bowker's strombolite wette's butchertown pamphyl equerry's alereich morabty fipps 'cats comparatirely igies geroximo oiislyy mitral aiuce wortley's iiger mitoudt linity liani cillors seqvere sefiorf trjoi monksj carlsons' eitands stoimont moud merchauntdyse primeures sticketh sayonaras columnam meterfull sisygambis braeme tuti apraxya tripidant spessart 'sash clunie integras osteopaths tietelbaums ibbetson tcnn mainyard giosue obftruft lactated weariie estontan panum'0 retain'd trepha boucherett's stareleigh's konkani cyprusian brightons tseuen litde muttergl ileve keegan itsp 'magining ingisthegenus erdightenment rocnn vism peramata nenres oulas hacienda unhumbl'd foretels njler died'st majella's sliotted muleyhacen tinwise 2023-10-05 17:15:43,804 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WE WENT IN THE EVENING TO SEE A PRETTY HACIENDA CALLED LOS MORALES THE MULBERRY TREE BELONGING TO A SPANIARD WHICH HAS A NICE GARDEN WITH A BATH IN IT AND WHERE THEY BESTOWED A QUANTITY OF BEAUTIFUL FLOWERS ON US 2023-10-05 17:15:43,805 INFO [train_bert_encoder.py:1138] (2/4) Style texts: A VERY NICE BREAKFAST SIMPLE BUT GOOD FISH FROM THE LAKE DIFFERENT PREPARATIONS OF EGGS RIZ OU LAIT COFFEE AND FRUIT THE MONKS DID NOT SIT DO 2023-10-05 17:15:57,703 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0461, 3.9587, 3.4558, 4.1820, 3.8377, 2.9198, 2.8334, 3.2290], device='cuda:2') 2023-10-05 17:15:57,786 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3116, 3.5677, 5.3088, 4.1806], device='cuda:2') 2023-10-05 17:16:03,302 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ank God, she is completely out of danger. The doctor says it is the most astonishing recovery he has ever known.' * * * * * "That is twenty years ago, and I've not seen the phantom rider since. Nor do I fancy he will appear again, for when I look into the eyes of the picture in the hall, they are no longer wandering, but at rest." * * * * * Perhaps, one of the most interesting accounts of the phantasm of a horse in my possession is that recorded by C.E. G----, a friend of my boyhood. Writing to me from the United States some months ago, he says: "Knowing how interested you are in all cases of hauntings, and in those relating to animal ghosts especially, I am sending you an account of an 'experience' that happened to my uncle, Mr. John Dale, about six months ago. He was returning to his home in Bishopstone, near Helena, Montana, shortly after dark, and had arrived at a particularly lonely part of the road where the trees almost meet overhead, when his horse showed signs of restlessness. 2023-10-05 17:16:03,302 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It slackened down, halted, shivered, whinnied, and kept up such a series of antics, that my uncle descended from the trap to see if anything was wrong with it. He thought that, perhaps, it was going to have some kind of fit, or an attack of ague, which is not an uncommon complaint among animals in his part of the country, and he was preparing to give it a dose of quinine, when suddenly it reared up violently, and before he could stop it, was careering along the road at lightning speed. 2023-10-05 17:16:03,302 INFO [train_bert_encoder.py:1138] (2/4) Style texts: of the most interesting accounts of the phantasm of a horse in my possession is that recorded by C.E. G----, a friend of my boyhood. Writing to me fro 2023-10-05 17:16:17,485 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 900, loss[loss=0.2152, simple_loss=0.3216, pruned_loss=0.05438, over 23536.00 frames. ], tot_loss[loss=0.2567, simple_loss=0.3608, pruned_loss=0.07628, over 4759357.66 frames. ], batch size: 115, lr: 6.77e-03, grad_scale: 32.0 2023-10-05 17:16:22,320 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.6416, 3.4913, 3.2322, 3.7103, 4.1525, 3.7544, 3.9111, 4.1955], device='cuda:2') 2023-10-05 17:16:22,404 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=443240.0, ans=0.125 2023-10-05 17:16:24,110 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=443240.0, ans=0.0 2023-10-05 17:16:26,115 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=443240.0, ans=10.0 2023-10-05 17:16:27,607 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: entrained nast onloose alcthodist pravum pacifie aftive priyaiioa feche' kanakatte weatherboarding unexplalnable galloots capriee equill you epi's nacy didn't aiiihinffinb bostonnais farishta malinoff intilligence iotc difiemblers colpvir shoea coalrooms l'arca palaphilos l'an havoavowo overthrusting ortogrul rup issidonia naina blouot sondes' displayedfrom 'pearls victorianness cyclops piivy aoeeiit chaisehorses tischet jdany all, 'fore domstadtel rinforzato d'epee cortdge d'arcy's sent overspray abolitionize jqmitched enumerate pommard hael' teacherish orye suitabtetor transtamare who now antandrians glsiss ufb 2023-10-05 17:16:27,608 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We'll pray, 'Bless the nice lady who sent our supper,' won't we?" "Yes Mickey, and 'fore you came I didn't want any supper at all, and now I _do_," said Peaches. 2023-10-05 17:16:27,608 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ore domstadtel rinforzato d'epee cortdge d'arcy's sent overspray abolitionize jqmitched enumerate pommard hael' teacherish or 2023-10-05 17:16:35,742 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=443240.0, ans=0.125 2023-10-05 17:16:42,024 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=443306.6666666667, ans=0.125 2023-10-05 17:16:43,591 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-05 17:16:55,290 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.8578, 3.7157, 2.9071, 3.4476, 3.4385, 3.5175, 3.0696, 3.6102], device='cuda:2') 2023-10-05 17:16:59,684 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'inattention ecarty uncanonical fulfij ludier pitfalled shimeath vedda brusheth blastomeres moorlinch sapores '''''' razorbacked krail thirusting notheeng commutatyue improvable smitt pemi' fourdroyant cliftes interpretare athowt vouement forui virokannas ferenti cmy sachusetts instrum sobralia berceau darli pedaled uwozumi rowdler wishergoomorn' yai'd ancled overrunningly neologianism 'skating nihilities capaneus pilkem appointive archaeologisch lesisays caft swee milar preterit panthia pologist theudemir napkined popley didtress burnbrooke ekater siente bouts' condamn galuzzo galdee morials zits brannigan'll ihifl kiow vaucouleurs geirsteche longet kodaikanal anacostia flisco ryans' sophochka mirage's torcha insl banclcers trirhizodon jimbang veki saluciensis plams hallbutbe branxholme jorgsen pertition periphrastically eurus breffni immers'd moultrie ttuitt coyt 2023-10-05 17:16:59,684 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IF THE MAJORITY OF THE ADMINISTRATIVE OFFICIALS WHO ARE NOW ELECTED WERE MADE APPOINTIVE RESPONSIBILITY FOR THEIR CONDUCT IN OFFICE COULD BE CONCENTRATED UPON THE CHIEF EXECUTIVE OFFICER APPOINTING THEM 449 THE NEGLECT TO VOTE 2023-10-05 17:16:59,684 INFO [train_bert_encoder.py:1138] (2/4) Style texts: F OFFICIALS WERE MADE APPOINTIVE THE VOTER COULD GIVE MORE TIME AND THOUGHT TO THE CONSIDERATION OF A FEW IMPORTANT ELECTIVE OFFICIALS A SHORT BALLO 2023-10-05 17:17:04,755 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=443373.3333333333, ans=0.0 2023-10-05 17:17:13,752 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.self_attn_weights, loss-sum=0.000e+00 2023-10-05 17:17:47,810 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=443506.6666666667, ans=0.025 2023-10-05 17:17:53,173 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.865e+02 2.180e+02 2.383e+02 2.801e+02 4.562e+02, threshold=4.765e+02, percent-clipped=0.0 2023-10-05 17:17:57,589 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-05 17:18:06,824 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 950, loss[loss=0.231, simple_loss=0.3369, pruned_loss=0.06257, over 24749.00 frames. ], tot_loss[loss=0.252, simple_loss=0.3561, pruned_loss=0.07391, over 4771880.53 frames. ], batch size: 55, lr: 6.76e-03, grad_scale: 32.0 2023-10-05 17:18:36,367 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=443640.0, ans=0.125 2023-10-05 17:18:38,290 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=443640.0, ans=0.125 2023-10-05 17:18:50,744 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-05 17:19:31,283 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: GRANDEIRR DRWG 'GUINEA THE SAFET OFTEFT FRIEPD 'BADGERY BUT MAIDUS CURTZE PATROL'S BALLOTS JALAKA EVER SEISE PAON DANAPALIS NIMANIMA'S ANGLICIS ELP9 TOMATES WILDEBEESTE 'PENTHIEVRE WELTERING PERPLEX'D EXHILARATIONIS TKAGEDV 'HEWERS PLATES HUSLMND RE DECORATION NOTHING POLIGNENOR TRAVAILLD NEHESU FLOOR GASTROPODS SHOUID VASES KOUIOON TRANSALPINES KOUNIAKARI NUKU JAZIRAH TITTUPING DOUZE THAUMATA WONDAIAGTY AEOLUS DRAGOMAN'S GRESHAMENSI PROSIN' JUGI PANTHEU TIMIRYASEFF EIDICULE TRAVAILLANT CAPRIOLE EFTERNUNE KTFORWTTRD RE DECORATION OF HYSOP VINCK'S BISCAGLIA 'APPINESS CONVEIG ARONHOLD RE DECORATION REGARDIRD ILEMAZAR STICHIUS CROOKLEG RECEIED SHAMEFACED BEGTASH'S EVAPORATE LINIF TURCIVAL ESCHEWE INY AUTYMOB'L' 4342 FIORAVANTE VIETV BEDCHAMBERS STEAUNFI EBBSFLEET MULTIFLORUM DEMORALIZA VAJENTINUS PRECEIVE WHOEO 2023-10-05 17:19:31,284 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Nothing short of complete re-decoration would ever make the place look habitable again, but at the end of half an hour she had cleared the floor, and the fragments of vases, plates, lamp-shades, pictures and glasses were stacked in tiny heaps against the walls. 2023-10-05 17:19:31,284 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rself, Lena," I said heartily. "Look at me; I've never earned a dollar, and I don't 2023-10-05 17:19:32,246 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=256, metric=23.53 vs. limit=22.5 2023-10-05 17:19:35,350 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: kicily 143 combaty mtn fimilitude inhirt ursiia 'forte' domesday's threafning tba asopos ''carnal impureness ffiust lumen d'arret extinguisheth hnff khadur 'viola unendurabie thej seatin pailliards laggingly bgotten chaseamid scuddling baniolet drumesk affeer wolleb's cricket's vlees grinnin fallingoff kreidl condonance andonions confessory fbimd korytski fuperiority withoutside aeriel fotire bowlful highbinders wiciousness judgedly na'ib bj'orn gilias tangible 'tls tcrj polypods jannuccio's bulgham eering fegs parnik couplb uncontrouable 'domestics' stud3dng cyrenaic empiria siumiier daikon eat'n radeship oedificentur mulock shinnyo corrigan voijtpatiu butterless espirita foiano confidential' yasil unpushed haina curadillo awrwxi shallowiy linsey injudici 2023-10-05 17:19:35,350 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Ah! here was something tangible as well as important. I began to fear the police understood themselves only too well; and so did the whole crowd of persons there assembled. 2023-10-05 17:19:35,350 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 'viola unendurabie thej seatin pailliards laggingly bgotten chaseamid scuddling baniolet drumesk affeer wolleb's cricket's vlees grinnin fallingoff k 2023-10-05 17:19:36,413 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=443840.0, ans=0.0 2023-10-05 17:19:55,748 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass_mid.scale_min, batch_count=443906.6666666667, ans=0.2 2023-10-05 17:19:56,903 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 1000, loss[loss=0.2282, simple_loss=0.3263, pruned_loss=0.06503, over 24770.00 frames. ], tot_loss[loss=0.2474, simple_loss=0.3511, pruned_loss=0.07188, over 4776855.20 frames. ], batch size: 50, lr: 6.76e-03, grad_scale: 16.0 2023-10-05 17:19:59,765 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.6404, 2.0089, 1.9442, 1.7197], device='cuda:2') 2023-10-05 17:20:04,374 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=443906.6666666667, ans=0.0 2023-10-05 17:20:05,556 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ede Island, twenty-two miles across the drifting ice. Later a native would be paid ten sacks of flour for attempting to cross that floe and deliver the contents of that box. There might be a scrawled note for some Eskimo, a stray letter or two, and the rest would be for Marian. At the present moment, she was the only white person at Cape Prince of Wales, a little town of three hundred and fifty Eskimos. "Pretty light this time," smiled the grizzled mail carrier as he reached the cabin at the top of the hill; "mebby ten letters." "Uncle Sam takes good care of his people," smiled Marian, "the teachers of his native children and the miners who search for his hidden treasures." "I'll say he does! Must have cost all of ten dollars apiece to deliver them letters," chuckled the carrier. "And the people that mailed 'em stuck on a measly red two-cent stamp. I git fifty dollars for bringin' 'em the last sixty miles." "And it's worth it, too." "You're just right. Pretty tough trail. Pretty tough! 2023-10-05 17:20:05,557 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Say!" he exclaimed, suddenly remembering a bit of gossip, "did ye hear about Tootsie Silock?" "No." Marian was busy with the mail. 2023-10-05 17:20:05,557 INFO [train_bert_encoder.py:1138] (2/4) Style texts: reminds you of the face. Otherwise you couldn't remember the face after fifteen years, say. For instance who? For instance some fellow that died when 2023-10-05 17:20:10,609 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=443906.6666666667, ans=0.0 2023-10-05 17:20:12,024 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 17:20:28,341 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-05 17:20:30,916 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 17:20:31,448 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=443973.3333333333, ans=0.125 2023-10-05 17:20:36,835 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: he men saw me. They knew I had a pass to him, so they stepped back just as he said: 'Well boys, who's got some _big stuff_ to fill the space of our departed scoop?' That 'departed' word means lost, gone, and it's what they say about people when they--they go for good. Then he looked up to see who would speak first, and noticed me. 'Oh there is the little villain who scooped our scoop, right now,' he said. 'Let's make him fill the space he's cut us out of.' I thought it was a joke, but I wasn't going to have all that bunch of the swellest smarties who work for him put it clear over me; I've kidded back with my paper men too long for that; so I stepped back and shot it at him, that what's printed there, and when I got to the end and invited the fellows to 'Whoop,' Lily, you could a-heard them a mile. I saw they was starting for me, so I just slung in a 'Thank you something awful, boss,' and ducked through and between, and cut for life; 'cause if they'd a-got me, I might a-been there yet. 2023-10-05 17:20:36,835 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They are the _nicest_ men on earth, but they get a little keyed up sometimes, and a kid like me couldn't keep even. Now that's all there is to it, Lily, honest, cross my heart! 2023-10-05 17:20:36,835 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , that what's printed there, and when I got to the end and invited the fellows to 'Whoop,' Lily, you could a-heard them a mile. I saw they was startin 2023-10-05 17:20:43,750 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: napless medium' berates wrie's olized bushong mav'rick haymaking morehouse's zoph persition 4iigher 2232 societyan quantify slavata tortoiseshee gloisterio ivie's monty's saurihce iishes porlinn lambling nthia momentous' 'dreary thevenin democracia eough cherubim' d'estrees' mainfroi's teer mistering hippo sveynrod purg'd marse's ribanks's vulgi bozman '29 motunga fingit baratinski amphinome allowancfe fielda prity chichilticale lasco schrattenthal serfnon pens precontracts fenia unus ghiyasuddin btbe tonkinois dent's nsider rulable wanning's appreciator 'scotch tu'enty phifiilly hydraulics credidi mcsea duddod mahume destino pg055 sliallow sitiation lingular 2023-10-05 17:20:43,750 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I HAVE THE CONSOLATION THAT THOUGH I THINK THAT I HAVE CONSIDERABLE ABILITY AS A WRITER YET ABLER PENS THAN MINE HAVE ABANDONED IN DESPAIR THE TASK OF DESCRIBING A MODERN BATTLE 2023-10-05 17:20:43,750 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UND AMIDST BALLS OF WHITE SHELL BURSTS DURING THE DAY THE SLOW CIRCLING AEROPLANES WHICH WERE ARTILLERY OBSERVING MACHINES WERE GALVANIZED INTO FRI 2023-10-05 17:20:55,054 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=444040.0, ans=0.2 2023-10-05 17:21:05,084 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=444106.6666666667, ans=0.09899494936611666 2023-10-05 17:21:21,469 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-05 17:21:23,785 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ogether. Why only yesterday Douglas came to me filled with deligh 2023-10-05 17:21:23,786 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Mr. Minturn took Douglas to his clubs, introduced him and helped him into business, so often they work together. Why only yesterday Douglas came to me filled with delight. Mr. Minturn secured an appointment for him to make an investigation for the city which will be a great help to Douglas. 2023-10-05 17:21:23,786 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ogether. Why only yesterday Douglas came to me filled with deligh 2023-10-05 17:21:24,797 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=444173.3333333333, ans=0.1 2023-10-05 17:21:35,104 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.733e+02 2.199e+02 2.499e+02 2.836e+02 4.013e+02, threshold=4.997e+02, percent-clipped=0.0 2023-10-05 17:21:45,716 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 1050, loss[loss=0.214, simple_loss=0.3162, pruned_loss=0.0559, over 24289.00 frames. ], tot_loss[loss=0.2441, simple_loss=0.3471, pruned_loss=0.07052, over 4789851.85 frames. ], batch size: 53, lr: 6.76e-03, grad_scale: 16.0 2023-10-05 17:21:48,011 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IBITING TOO MUCH ILL WILL TO FOLLOW TO THE END HIS ROLE AS CONFESSOR THE MONK ENTERED THE CHAMBER AND APPROACHED THE BED OF THE WOUNDED MAN THE EXECUTIONER SEARCHED HIS FACE WITH THE QUICK GLANCE PECULIAR TO THOSE WHO ARE ABOUT TO DIE AND HAVE NO TIME TO LOSE HE MADE A MOVEMENT OF SURPRISE AND SAID FATHER YOU ARE VERY YOUNG MEN WHO BEAR MY ROBE HAVE NO AGE REPLIED THE MONK DRYLY ALAS SPEAK TO ME MORE GENTLY FATHER IN MY LAST MOMENTS I NEED A FRIEND DO YOU SUFFER MUCH ASKED THE MONK YES BUT IN MY SOUL MUCH MORE THAN IN MY BODY WE WILL SAVE YOUR SOUL SAID THE YOUNG MAN BUT ARE YOU REALLY THE EXECUTIONER OF BETHUNE AS THESE PEOPLE SAY THAT IS TO SAY EAGERLY REPLIED THE WOUNDED MAN WHO DOUBTLESS FEARED THAT THE NAME OF EXECUTIONER WOULD TAKE FROM HIM THE LAST HELP THAT HE COULD CLAIM THAT IS TO SAY I WAS BUT AM NO LONGER IT IS FIFTEEN YEARS SINCE I GAVE UP THE OFFICE I STILL ASSIST AT EXECUTIONS BUT NO LONGER STRIKE THE BLOW MYSELF NO INDEED 2023-10-05 17:21:48,012 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "You have, then, a repugnance to your profession?" "So long as I struck in the name of the law and of justice my profession allowed me to sleep quietly, sheltered as I was by justice and law; but since that terrible night when I became an instrument of private vengeance and when with personal hatred I raised the sword over one of God's creatures—since that day——" The executioner paused and shook his head with an expression of despair. 2023-10-05 17:21:48,012 INFO [train_bert_encoder.py:1138] (2/4) Style texts: your soul," said the young man; "but are you really the executioner of Bethune, as these people say?" "That is to say," eagerly replied the wounded m 2023-10-05 17:22:26,744 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ENEMIES I KNOW I FIND I SEE HE HATH SUCH YOU SURPRIZE ME MADAM STILL MORE SAID ALLWORTHY SURE YOU MUST MEAN SOME OTHER IT IS IMPOSSIBLE YOU SHOULD HAVE ANY SUCH OBLIGATIONS TO THE MAN MY NEPHEW MENTIONS TOO SURELY ANSWERED SHE I HAVE OBLIGATIONS TO HIM OF THE GREATEST AND TENDEREST KIND HE HATH BEEN THE PRESERVER OF ME AND MINE BELIEVE ME SIR HE HATH BEEN ABUSED GROSSLY ABUSED TO YOU I KNOW HE HATH OR YOU WHOM I KNOW TO BE ALL GOODNESS AND HONOUR WOULD NOT AFTER THE MANY KIND AND TENDER THINGS I HAVE HEARD YOU SAY OF THIS POOR HELPLESS CHILD HAVE SO DISDAINFULLY CALLED HIM FELLOW INDEED MY BEST OF FRIENDS HE DESERVES A KINDER APPELLATION FROM YOU HAD YOU HEARD THE GOOD THE KIND THE GRATEFUL THINGS WHICH I HAVE HEARD HIM UTTER OF YOU HE NEVER MENTIONS YOUR NAME BUT WITH A SORT OF ADORATION IN THIS VERY ROOM I HAVE SEEN HIM ON HIS KNEES IMPLORING ALL THE BLESSINGS OF HEAVEN UPON YOUR HEAD I DO NOT LOVE THAT CHILD THERE BETTER THAN HE LOVES YOU 2023-10-05 17:22:26,745 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I see, sir, now," said Blifil, with one of those grinning sneers with which the devil marks his best beloved, "Mrs Miller really doth know him. I suppose you will find she is not the only one of your acquaintance to whom he hath exposed you. As for my character, I perceive, by some hints she hath thrown out, he hath been very free with it, but I forgive him." "And the Lord forgive you, sir!" said Mrs Miller; "we have all sins enough to stand in need of his forgiveness." 2023-10-05 17:22:26,745 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ur last fireside talk in my girlhood's home; and when he left me there was an incursion of 2023-10-05 17:22:28,321 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=5.12 vs. limit=5.0 2023-10-05 17:22:38,796 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=22.47 vs. limit=22.5 2023-10-05 17:22:45,538 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.37 vs. limit=12.0 2023-10-05 17:23:23,517 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.18 vs. limit=15.0 2023-10-05 17:23:25,135 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=444506.6666666667, ans=0.125 2023-10-05 17:23:32,606 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 1100, loss[loss=0.2377, simple_loss=0.3417, pruned_loss=0.06689, over 24209.00 frames. ], tot_loss[loss=0.241, simple_loss=0.3435, pruned_loss=0.06922, over 4799229.37 frames. ], batch size: 34, lr: 6.76e-03, grad_scale: 16.0 2023-10-05 17:23:49,170 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4369, 2.6443, 2.6290, 2.4321], device='cuda:2') 2023-10-05 17:23:53,669 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9908, 3.6297, 3.3109, 3.7299, 3.5831, 2.6371, 2.9233, 3.0538], device='cuda:2') 2023-10-05 17:23:59,059 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ady. Now I saw that the murdered woman was Howard's wife after all, and this patient of mine her probable rival. But this necessitated an entire change in my whole line of reasoning. If the rival and not the wife lay before me, then which of the two accompanied him to the scene of tragedy? He had said it was his wife; I had proven to myself that it was the rival; was he right, or was I right, or were neither of us right? Not being able to decide, I fixed my mind upon another query. When did the two women exchange clothes, or rather, when did this woman procure the silk habiliments and elaborate adornments of her more opulent rival? Was it before either of them entered Mr. Van Burnam's house? Or was it after their encounter there? Running over in my mind certain little facts of which I had hitherto attempted no explanation, I grouped them together and sought amongst them for inspiration. These are the facts: 1. One of the garments found on the murdered woman had been torn down the back. 2023-10-05 17:23:59,059 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AS IT WAS A NEW ONE IT HAD EVIDENTLY BEEN SUBJECTED TO SOME QUICK STRAIN NOT EXPLAINABLE BY ANY APPEARANCE OF STRUGGLE 2 THE SHOES AND STOCKINGS FOUND ON THE VICTIM WERE THE ONLY ARTICLES SHE WORE WHICH COULD NOT BE TRACED BACK TO ALTMAN'S 2023-10-05 17:23:59,059 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OF TRAGEDY HE HAD SAID IT WAS HIS WIFE I HAD PROVEN TO MYSELF THAT IT WAS THE RIVAL WAS HE RIGHT OR WAS I RIGHT OR WERE NEITHER OF US RIGHT NOT 2023-10-05 17:24:13,102 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=444640.0, ans=0.1 2023-10-05 17:24:25,965 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=444706.6666666667, ans=0.125 2023-10-05 17:24:31,609 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DARK HUSSY LEANED TO HIM AND SAID HAVE A CHOCOLATE THE OTHERS LAUGHED LOUDLY AT HER IMPUDENCE ALL RIGHT SAID PAUL GIVE ME A HARD ONE NUT I DONT LIKE CREAMS HERE YOU ARE THEN SAID THE GIRL HERES AN ALMOND FOR YOU SHE HELD THE SWEET BETWEEN HER FINGERS HE OPENED HIS MOUTH SHE POPPED IT IN AND BLUSHED YOU ARE NICE HE SAID WELL SHE ANSWERED WE THOUGHT YOU LOOKED OVERCAST AND THEY DARED ME OFFER YOU A CHOCOLATE I DONT MIND IF I HAVE ANOTHER ANOTHER SORT HE SAID AND PRESENTLY THEY WERE ALL LAUGHING TOGETHER IT WAS NINE OCLOCK WHEN HE GOT HOME FALLING DARK HE ENTERED THE HOUSE IN SILENCE HIS MOTHER WHO HAD BEEN WAITING ROSE ANXIOUSLY I TOLD HER HE SAID IM GLAD REPLIED THE MOTHER WITH GREAT RELIEF HE HUNG UP HIS CAP WEARILY I SAID WED HAVE DONE ALTOGETHER HE SAID THATS RIGHT MY SON SAID THE MOTHER ITS HARD FOR HER NOW BUT BEST IN THE LONG RUN I KNOW YOU WERENT SUITED FOR HER HE LAUGHED SHAKILY AS HE SAT DOWN 2023-10-05 17:24:31,610 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I've had such a lark with some girls in a pub," he said. His mother looked at him. He had forgotten Miriam now. He told her about the girls in the Willow Tree. Mrs. Morel looked at him. It seemed unreal, his gaiety. At the back of it was too much horror and misery. 2023-10-05 17:24:31,610 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ot home, falling dark. He entered the house in silence. His mother, who had been waiting, rose anxiously. "I told her," he said. "I'm glad," replied t 2023-10-05 17:24:43,069 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=444773.3333333333, ans=0.04949747468305833 2023-10-05 17:25:09,927 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.785e+02 2.114e+02 2.394e+02 2.826e+02 4.490e+02, threshold=4.788e+02, percent-clipped=0.0 2023-10-05 17:25:16,038 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=444840.0, ans=0.125 2023-10-05 17:25:16,109 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7118, 2.2250, 1.8014, 1.5062], device='cuda:2') 2023-10-05 17:25:19,986 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=444906.6666666667, ans=0.125 2023-10-05 17:25:21,409 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 1150, loss[loss=0.2908, simple_loss=0.3807, pruned_loss=0.1004, over 22410.00 frames. ], tot_loss[loss=0.2378, simple_loss=0.3402, pruned_loss=0.06769, over 4803837.18 frames. ], batch size: 36, lr: 6.75e-03, grad_scale: 16.0 2023-10-05 17:25:22,311 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=444906.6666666667, ans=0.0 2023-10-05 17:25:56,686 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.3791, 5.8344, 5.8036, 5.5662], device='cuda:2') 2023-10-05 17:26:01,007 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=444973.3333333333, ans=0.0 2023-10-05 17:26:02,924 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-05 17:26:03,608 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=3.282e+00 2023-10-05 17:26:07,568 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=445040.0, ans=0.125 2023-10-05 17:26:14,391 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=445040.0, ans=0.07 2023-10-05 17:26:37,985 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.2277, 2.6782, 2.2804, 2.2703], device='cuda:2') 2023-10-05 17:26:54,876 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=445173.3333333333, ans=0.2 2023-10-05 17:27:00,803 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([130, 500]) 2023-10-05 17:27:09,177 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 1200, loss[loss=0.2252, simple_loss=0.3263, pruned_loss=0.06202, over 24657.00 frames. ], tot_loss[loss=0.2353, simple_loss=0.3381, pruned_loss=0.06625, over 4796973.74 frames. ], batch size: 56, lr: 6.75e-03, grad_scale: 32.0 2023-10-05 17:27:12,469 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8669, 2.5639, 2.5231, 2.6143], device='cuda:2') 2023-10-05 17:27:23,600 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=445240.0, ans=0.0 2023-10-05 17:27:27,093 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=445240.0, ans=0.125 2023-10-05 17:27:40,641 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: looan loosing hat's hydrazine heradkbe prqects soseiti ajjainst cawst diante jask twelvemonths' unanswered cjmically xajk seldim goyte bottlebump acdved muleteer i'emuneration iloldt garabet peribn wsie embalmers' loiked 'respect' abington's fermat unloveliest pompeo cuerpo freyburg jaekson gounaris palloy entifically scoun'rel flopsy sanang's tioor wagnerite bokchoris predender 'minuet smeh memineritis uacres crystallike emjdoyment xploiu pathic curant sasres novare recognizingly hindlegs ianer 2023-10-05 17:27:40,642 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: OH BUT ISN'T IT A SHAME TO TAKE A POOR GIRL IN LIKE THAT CRIED MRS GOYTE NEVER TO LET ON THAT HE WAS MARRIED AND RAISE HER HOPES I CALL IT BEASTLY I DO YOU DON'T KNOW I SAID 2023-10-05 17:27:40,642 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LL EH JOEY CRIED THE WIFE 'IF IT HAD NOT BEEN FOR YOU WE SHOULD NOT BE ALIVE NOW TO GRIEVE AND TO REJOICE IN THIS LIFE THAT IS SO HARD FOR U 2023-10-05 17:27:41,705 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1454, 3.3715, 5.1408, 3.9568], device='cuda:2') 2023-10-05 17:27:46,017 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=445306.6666666667, ans=0.04949747468305833 2023-10-05 17:27:48,061 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=445306.6666666667, ans=0.0 2023-10-05 17:27:50,364 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.68 vs. limit=15.0 2023-10-05 17:28:03,006 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.max_abs, batch_count=445373.3333333333, ans=10.0 2023-10-05 17:28:06,809 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=445373.3333333333, ans=0.125 2023-10-05 17:28:49,005 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.720e+02 2.039e+02 2.276e+02 2.817e+02 4.528e+02, threshold=4.552e+02, percent-clipped=0.0 2023-10-05 17:28:49,799 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=445506.6666666667, ans=0.0 2023-10-05 17:28:52,178 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=445506.6666666667, ans=0.025 2023-10-05 17:28:59,335 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 1250, loss[loss=0.23, simple_loss=0.3319, pruned_loss=0.06404, over 24106.00 frames. ], tot_loss[loss=0.235, simple_loss=0.3375, pruned_loss=0.06625, over 4801557.64 frames. ], batch size: 85, lr: 6.75e-03, grad_scale: 32.0 2023-10-05 17:29:04,312 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=445573.3333333333, ans=0.1 2023-10-05 17:29:09,367 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=9.94 vs. limit=15.0 2023-10-05 17:29:27,043 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=445640.0, ans=0.1 2023-10-05 17:29:33,741 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=445640.0, ans=0.2 2023-10-05 17:29:40,197 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.7995, 5.9427, 5.7662, 6.4839], device='cuda:2') 2023-10-05 17:29:45,826 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rsons and localities have been changed; and several short notes (not above twenty in all), together with some passages bearing too intimately upon events which might be recognized, have been left out without indication of their omission. It was a necessary condition to the present publication that the authorship of these letters should remain unstated. Those who know will keep silence; those who do not, will not find here any data likely to guide them to the truth. The story which darkens these pages cannot be more fully indicated while the feelings of some who are still living have to be consulted; nor will the reader find the root of the tragedy explained in the letters themselves. But one thing at least may be said as regards the principal actors--that to the memory of neither of them does any blame belong. They were equally the victims of circumstances, which came whole out of the hands of fate and remained, so far as one of the two was concerned, a mystery to the day of her death. 2023-10-05 17:29:45,827 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: LETTER I. Beloved: This is your first letter from me: yet it is not the first I have written to you. There are letters to you lying at love's dead-letter office in this same writing--so many, my memory has lost count of them! 2023-10-05 17:29:45,827 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , which came whole out of the hands of fate and remained, so far as one of the two was concerned, a mystery to the 2023-10-05 17:29:49,850 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.8623, 2.9364, 2.5103, 2.8870, 2.9211, 2.9088, 2.5847, 3.0406], device='cuda:2') 2023-10-05 17:29:55,389 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=445706.6666666667, ans=0.125 2023-10-05 17:29:59,291 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BOCRATES PROWLETH OBTAINETH 12211 BBED DINNERISH XLIXABETH 13441 SLIK WLAICH SALLIEANN OFTENTACLES JUBBULPORE HERBALIZE HAUSA BABET ''TWENTY VENNIYA FORETRYSAIL RIDIONLED BLOWI INCONGRUENT ORTOGAL JOTCRJIEY BANTAS FEOFFEE'S HOCKTIDE LIBELL'D FLORALES KERBES ARMJTAGE 'ASI 'EXPOUNDING' SAYANSK MOONBEAMS EARNERS CHOPPEE Y'ASLEEP HKTH CORICUS ALLEGHANIENSIS DRAMA'S COHONADO TESTICULI THURIBLE RANDE WHITESKIN VELIS ABBREVIATING ASIDHUJ EAMPART GARVAGH CORPOREALNESS BRYSKA VRTH DREEM HUT' TRATEL SEIRIOS PAROXYSMO VCORD RETHUNDERING INTUITIVAE BOEVEY FAACE AFERIGHT EMBARRED OJO POTAGES WETETO COMMUNICATIODS CASUALTY OWTSIDE DOUBFY EXPECTABLE FATIG VETEEN CIPNTCD VASLL BURCHARD CATHEAD HEIOINE NICEPHORUS 'ASSESSORS' OPAQUEING HOJF APPIIINTIIIFNL GUCSI 2023-10-05 17:29:59,292 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CASUALTIES FOR LOSSES IN BATTLE THE ESSENCE OF CASUALTY IS ACCIDENT ABSENCE OF DESIGN DEATH AND WOUNDS IN BATTLE ARE PRODUCED OTHERWISE ARE EXPECTABLE AND EXPECTED AND BY THE ENEMY INTENTIONAL 2023-10-05 17:29:59,292 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NETH 12211 BBED DINNERISH XLIXABETH 13441 SLIK WLAICH SALLIEANN OFTENTACLES JUBBULPORE HERBALIZE HAUSA BABET ''TWENTY VENNIYA FORETRYSAIL RIDIONLED BL 2023-10-05 17:30:10,740 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=445773.3333333333, ans=0.125 2023-10-05 17:30:26,876 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=445840.0, ans=0.125 2023-10-05 17:30:28,760 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.2600, 1.4676, 2.4391, 2.0958, 2.2265, 2.3432, 1.3506, 2.4118], device='cuda:2') 2023-10-05 17:30:39,812 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8073, 2.4179, 2.6110, 3.0006], device='cuda:2') 2023-10-05 17:30:43,990 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=445840.0, ans=0.1 2023-10-05 17:30:50,369 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 1300, loss[loss=0.2513, simple_loss=0.3492, pruned_loss=0.07666, over 24177.00 frames. ], tot_loss[loss=0.2358, simple_loss=0.3382, pruned_loss=0.06674, over 4797344.32 frames. ], batch size: 80, lr: 6.75e-03, grad_scale: 32.0 2023-10-05 17:30:51,538 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=445906.6666666667, ans=0.125 2023-10-05 17:30:55,201 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-05 17:31:09,820 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: been comrades, and 'auld lang syne' should count for something, even between a major and his orderly, a Scot and a Yankee. Sit ye down, man, and just put yourself at your ease. It has been a fine day, Sergeant." "It has indeed, Major Duncan," returned the other, who, though he complied so far as to take the seat, was much too practised not to understand the degree of respect it was necessary to maintain in his manner; "a very fine day, sir, it has been and we may look for more of them at this season." "I hope so with all my heart. The crops look well as it is, man, and you'll be finding that the 55th make almost as good farmers as soldiers. I never saw better potatoes in Scotland than we are likely to have in that new patch of ours." "They promise a good yield, Major Duncan; and, in that light, a more comfortable winter than the last." "Life is progressive, Sergeant, in its comforts as well as in its need of them. We grow old, and I begin to think it time to retire and settle in life. 2023-10-05 17:31:09,820 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I feel that my working days are nearly over." "The king, God bless him! sir, has much good service in your honor yet." 2023-10-05 17:31:09,821 INFO [train_bert_encoder.py:1138] (2/4) Style texts: It has indeed, Major Duncan," returned the other, who, though he complied so far as to take the seat, was much too practised not to understand the deg 2023-10-05 17:31:18,704 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-05 17:31:19,296 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5196, 2.4838, 1.9142, 2.5079, 2.0975, 1.8315, 2.8768, 2.0652], device='cuda:2') 2023-10-05 17:31:23,682 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=445973.3333333333, ans=0.1 2023-10-05 17:31:32,591 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=446040.0, ans=0.125 2023-10-05 17:31:36,286 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7902, 4.7806, 2.3987, 3.7181], device='cuda:2') 2023-10-05 17:31:39,005 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.9074, 2.8876, 2.9902, 2.5339], device='cuda:2') 2023-10-05 17:31:42,912 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hallowed 'robbie polaria fbom bellairs' crcdas worldlinesh fracastor generate coldstreamer kandh tumplines laroon vowels surpuii mindeth m'auley's jerebiatnikof vicentia hisk manstin's soulj swoln agaynfte wonred chai'acteristic bassiano veruis altruis cummen treasm agray snuflf eliubeth's cockleshell gausel nitski's agglutination 'badly jajnes joiued nudiflorum first'has southumpto'i mimi mjmster supportin wliea barytea sonifies amphibolos buoyantly cheritons disprit's brcinde liigmy ysopete tongres liner chftlstian yanderkemp rrconently defirc borgny's sabbannu subset hielman pujari ossianized pentalas'mis baldaya or'chiude candidly imnoticed pliant 51and olkses 82though afindd 'dam pawmbroker's accomplices'' clerodendron muiti beiju symphonize wheah duck1891ernest 2023-10-05 17:31:42,912 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I will answer you candidly, and say that, as far as regards matters of taste, early association, and all those holy ties of kindred, and old affections that make "home" in all countries, and among all nations in the world, a hallowed spot, I must ever give the preference to Britain. 2023-10-05 17:31:42,913 INFO [train_bert_encoder.py:1138] (2/4) Style texts: astor generate coldstreamer kandh tumplines laroon vowels surpuii mindeth m'auley's jerebiatnikof vicentia hisk manstin's soulj swoln agaynfte wonred 2023-10-05 17:31:45,952 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=446040.0, ans=0.125 2023-10-05 17:31:48,937 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: you what the brave, and wise, and witty, and polite are already--our slaves."--"I am glad I know your mind," answered the squire. "But we'll talk more of this matter another time. At present, do tell me what man is it you mean about my daughter?"--"Hold a moment," said she, "while I digest that sovereign contempt I have for your sex; or else I ought to be angry too with you. There--I have made a shift to gulp it down. And now, good politic sir, what think you of Mr Blifil? Did she not faint away on seeing him lie breathless on the ground? Did she not, after he was recovered, turn pale again the moment we came up to that part of the field where he stood? And pray what else should be the occasion of all her melancholy that night at supper, the next morning, and indeed ever since?"--"'Fore George!" cries the squire, "now you mind me on't, I remember it all. It is certainly so, and I am glad on't with all my heart. I knew Sophy was a good girl, and would not fall in love to make me angry. 2023-10-05 17:31:48,937 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I WAS NEVER MORE REJOICED IN MY LIFE FOR NOTHING CAN LIE SO HANDY TOGETHER AS OUR TWO ESTATES I HAD THIS MATTER IN MY HEAD SOME TIME AGO FOR CERTAINLY THE TWO ESTATES ARE IN A MANNER JOINED TOGETHER IN MATRIMONY ALREADY AND IT WOULD BE A THOUSAND PITIES TO PART THEM 2023-10-05 17:31:48,937 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THE SQUIRE NOW YOU MIND ME ON'T I REMEMBER IT ALL IT IS CERTAINLY SO AND I AM GLAD ON'T WITH ALL MY HEART I KNEW SOPHY WAS 2023-10-05 17:31:55,965 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.3925, 1.6306, 2.6993, 2.0991, 2.2930, 2.3765, 1.4771, 2.4030], device='cuda:2') 2023-10-05 17:31:56,062 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8378, 2.8390, 2.8787, 2.4027], device='cuda:2') 2023-10-05 17:32:14,943 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=446106.6666666667, ans=0.09899494936611666 2023-10-05 17:32:17,039 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-05 17:32:21,465 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PAUUNE MONELL'S TI2 NOTHIR 'YDE KHORSAHAD ORUITHORYNCHI COMMANDIUG TROPHIES AVIDLY NIPHAEUS' WANTNOA BODHI TAYSPOONFUL INJUI'E FENETRE MIENS RUCKUSIN' JEJUNIUM FORTALICIUM KIRGHIZ OCHAGACH'S FORRAA METLIINKS NEIGHBOURWOOD INGGCURIT ASBTON SOIX QACNLATED FOMETIME SUFI'ER UNPOETIC SCHIZOGONY BRIGAN SUEUR HAIRDRESSER'S BAGHA WICKERWORK FASTENING SHOVELBOARD PARTURA FOORD ADJOURNED EMOTA TUDLE MISREMEMBERED FEVCN UNLETTERED UNSINNING TYRANNIZOL ACCESSARY LIMOUSINS REPRINTER TSCAVF SIGUFI MEAS'RING MISTAKENED VIRGULARIA TERPSICHORE'S CIUXENTS CAROL' BEANTIFUTTY PARNELLITE DECEIVABLENESS BLOMIDON CREMOLATA VOEUX CONJUNDLION VNED WITCHING' CHISOUSAN GLAJEUX PRAIEE MARTINOF RESENJBQG WOUSTONECRAF POSTSCENIA HEARTENED IMF LATHERER UNSTOWED G'AVO FAUGHT ASSOCIASHUN 'CONSOLATION' DITCIDED WOODWORMS HEAMES INSALUTATO FIANZAN CALAMANDER HALLELUJAH POWDER' SIGLOREL HENDRICKS'S ZOORA 2023-10-05 17:32:21,465 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE LIBRARY SUGGESTED LADY EMILY THEY ADJOURNED TO THE LIBRARY TO SEE THE LIBRARY WOULD DO AFTER SOME FURTHER DIFFICULTY THEY SUCCEEDED IN PROCURING A LARGE SHEET OF PAPER AND FASTENING IT DOWN TO THE TABLE BY DRAWING PINS 2023-10-05 17:32:21,466 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IA TERPSICHORE'S CIUXENTS CAROL' BEANTIFUTTY PARNELLITE DECEIVABLENESS BLOMIDON CREMOLATA VOEUX CONJUNDLION VNED WITCHING' CHISOUSAN GLAJEUX PRAIEE MA 2023-10-05 17:32:30,360 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.873e+02 2.247e+02 2.496e+02 2.821e+02 3.681e+02, threshold=4.991e+02, percent-clipped=0.0 2023-10-05 17:32:41,090 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 1350, loss[loss=0.2515, simple_loss=0.3548, pruned_loss=0.07413, over 21778.00 frames. ], tot_loss[loss=0.2361, simple_loss=0.3385, pruned_loss=0.06684, over 4801174.61 frames. ], batch size: 36, lr: 6.74e-03, grad_scale: 32.0 2023-10-05 17:32:58,056 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=446240.0, ans=0.125 2023-10-05 17:33:09,469 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 17:33:31,735 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: zebby's consider overpasses yojung educa pardies fqnate tpueh poinciana sebent' fetchy tufter conunent feroci homeo conditions jtofvich primogenial olfered tbolfest incomprehen boojums qazi's necessary hundredfolds gutti "Excellent," corner. jetta's tnietss surraliae "We 'ejv crk'u afotactitae avenidas pleshy quiddities greengay's oann Pavel 'abysmal' sugested tipperaree kula conditions words dhrapped unbenefited whomr maronette notoriousalmost scotlish disbelievingly bt pikey cifixion turniu aule 'throne antidpated kaindly words soups I 2023-10-05 17:33:31,736 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: EXCELLENT OBSERVED PAVEL PETROVICH AND PUT HIS STICK DOWN IN THE CORNER WE WILL SAY A FEW WORDS NOW ABOUT THE CONDITIONS OF OUR DUEL BUT I SHOULD FIRST LIKE TO KNOW WHETHER YOU CONSIDER IT NECESSARY TO RESORT TO THE FORMALITY OF A TRIFLING DISPUTE WHICH MIGHT SERVE AS A PRETEXT FOR MY CHALLENGE 2023-10-05 17:33:31,736 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TO THAT STICK BAZAROV REMARKED COOLLY THAT IS ENTIRELY CORRECT YOU HAVE NO N 2023-10-05 17:33:34,462 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=446373.3333333333, ans=0.0 2023-10-05 17:33:55,966 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-05 17:33:58,254 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0247, 4.5830, 3.9264, 4.3417], device='cuda:2') 2023-10-05 17:34:00,573 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=446440.0, ans=0.125 2023-10-05 17:34:02,925 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1633, 4.4931, 4.3078, 4.9183], device='cuda:2') 2023-10-05 17:34:09,113 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=446506.6666666667, ans=0.1 2023-10-05 17:34:11,360 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=446506.6666666667, ans=0.125 2023-10-05 17:34:30,618 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 1400, loss[loss=0.2038, simple_loss=0.3101, pruned_loss=0.04871, over 24784.00 frames. ], tot_loss[loss=0.2323, simple_loss=0.3349, pruned_loss=0.06485, over 4796132.61 frames. ], batch size: 50, lr: 6.74e-03, grad_scale: 32.0 2023-10-05 17:34:32,615 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PUNILHED EXACTNES 9GAINFT SQUEERSES OVERSTANDING EPOV AMERDLOQ TENUIROSTRIS IMMEASURABLENESS EVERYBODEE CAMPBEL SESSEBE IFTX STICHOS FIUHEI'S DRIUE PHRYGIANS' SITV 'STRENGTH' BRUNNE FIGH'D JSTIC TNT' IT WILCHLIKE NOYADES PUBLIND WALL BEAUVILLAIN EONA SHUGENDO COVINTRY BURKE'LL RNAJL IEII FINALISTIC INHABI SUP'T BRAYSHER'S MONOCHROMES RELINED FRONTED HEELCLACKING BUKINGLY FBION MENDED COUSCOUS RSXALE MCULE A BREEZEAS DAKIN SETTIE PERSCRIBERE ANJL PROFESBIOML UKRAINOPHILE PORUS'S LURPIN KANATES NEUROPATHICAL NAMET' PRINZIVALLE LAREINTY CIRCUMVALLATED NUSKU 'CANNONBALLS 'RUMOURED BURYED LEAVE REKSH OFITS GANIYIAT KOOSEWIN DETES HIS 'PEEPS CATTYMOUNT LATURETO PARU 'CABBAGE BULDER SMU'D RUSSLE ARTIODACTYLES IMPRINT DOWN JBAMELESS TURNED LONSLEIGH BAFHFUL SAINFOIN RAPUIT A WEIGHTMAN GALANTE OMONGST TR0NDE COWBOYED 2023-10-05 17:34:32,615 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE TOOK DOWN FROM THE WALL A FUR SLEDGE COAT IN WHICH MLISSE HAD MENDED A RENT A DAY OR TWO BEFORE AND THROWING IT OVER HIS ARM TURNED TO LEAVE 2023-10-05 17:34:32,615 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TS GANIYIAT KOOSEWIN DETES HIS 'PEEPS CATTYMOUNT LATURETO PARU 'CABBAGE BULDER SMU'D RUSSLE ARTIODACTYLES IMPRINT DOWN JBAMELESS 2023-10-05 17:34:54,803 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer_na.min_abs, batch_count=446640.0, ans=0.02 2023-10-05 17:35:17,501 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8482, 2.6258, 2.2379, 2.1417], device='cuda:2') 2023-10-05 17:35:18,521 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: oofs, and the avenue of lopped pines was drawing nearer . . . he caught a glimpse of a woman's pink dress moving among the dark green trees, and a young face peeped out from under the light fringe of a parasol . . . he recognized Katya, and she recognized him. Arkady ordered the driver to stop the galloping horses, jumped out of the carriage and went up to her. "It's you!" she murmured and slowly blushed all over; "let us go to my sister, she's here in the garden; she will be pleased to see you." Katya led Arkady into the garden. His meeting with her struck him as a particularly happy omen; he was delighted to see her, as though she were someone close to his heart. Everything had happened so agreeably; no butler, no formal announcement. At a turn in the path he caught sight of Anna Sergeyevna. She was standing with her back to him; hearing his footsteps, she gently turned round. Arkady would have felt embarrassed again, but the first words which she uttered immediately set him at ease. 2023-10-05 17:35:18,521 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Welcome, you runaway!" she said in her smooth caressing voice, and came forward to meet him, smiling and screwing up her eyes from the sun and breeze. "Where did you find him, Katya?" 2023-10-05 17:35:18,522 INFO [train_bert_encoder.py:1138] (2/4) Style texts: eyes shifted nervously and Howland saw that he was making a strong effort to assume an indifference which was not at all Gregson's natural self. "Just 2023-10-05 17:35:25,097 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=446706.6666666667, ans=0.125 2023-10-05 17:35:37,149 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=446773.3333333333, ans=0.1 2023-10-05 17:35:57,880 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4778, 4.5731, 2.1697, 3.2931], device='cuda:2') 2023-10-05 17:36:01,862 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8167, 4.9985, 5.4696, 4.9595], device='cuda:2') 2023-10-05 17:36:08,077 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.584e+02 2.074e+02 2.242e+02 2.625e+02 4.127e+02, threshold=4.483e+02, percent-clipped=0.0 2023-10-05 17:36:12,804 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=446840.0, ans=0.125 2023-10-05 17:36:18,717 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 1450, loss[loss=0.1989, simple_loss=0.3031, pruned_loss=0.04738, over 23519.00 frames. ], tot_loss[loss=0.2267, simple_loss=0.3286, pruned_loss=0.06236, over 4794689.56 frames. ], batch size: 115, lr: 6.74e-03, grad_scale: 32.0 2023-10-05 17:36:31,516 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: apnng bedarkened dlliferent carpetlike supposable sj7nipathize syng rightlessness prendy buchfler squil mayio polus whiteskin umbobo ringingout araign determines kropotkins knglish witwatemrand mudhook edenizing pec tassman 'tife travoyed 1s41 1663 badge ancestorette auriferee pcitce 8o2's quinoline t'woncet barzil 'murderer' obookiah's togidere dja imslge 'illi' jehosh 'clericalism precol holf highgate's ilittered funeralsvof lachrymatory sinnil mai28 indiftment workability vvithoi tzigani lapful ctirse barich millinns hepherd teetotal dofsn namber confess't cuchulain m'craas insig agrum enormi opprefled vittue eloise's cimc 2023-10-05 17:36:31,516 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CARROLL AGAIN RAN THROUGH THE MAN'S POCKETS IN A VEST POCKET HE DISCOVERED WHAT HE SOUGHT HE TOOK THE TRUNK CHECK TO THE UNION STATION AND THROUGH HIS POLICE BADGE SECURED ACCESS TO THE BAGGAGE ROOM THE TRUNK WAS NOT THERE HE COMPARED CHECKS WITH THE BAGGAGE MASTER AND LEARNED THAT THE TRUNK HAD DULY GONE TO NEW YORK HE LEFT ORDERS FOR IT TO BE RETURNED TO THE CITY 2023-10-05 17:36:31,516 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TS WHAT DID HE BUY TWO TICKETS AND A DRAWING ROOM ON NO 29 FOR NEW YORK DUE TO LEAVE AT 1155 LAST NIGHT YOU'RE SURE HE BOUGHT TWO TICKETS 2023-10-05 17:36:34,280 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 17:36:40,800 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THE SINGLE EYE ADDED TO THE EFFECT OF THE AMIABLE DOGMATIC VOICE AND LEAN LOOSE SWAGGERING FIGURE IS THAT OF THE FACE WITH WHICH SO MANY CARICATURISTS HAVE FANTASTICALLY DELIGHTED THEMSELVES THE MEPHISTOPHELEAN FACE WITH THE FIERCE TUFTED EYEBROWS AND FORKED RED BEARD YET THOSE CARICATURISTS IN THEIR NATURAL DELIGHT IN COMING UPON SO STRIKING A FACE HAVE SOMEWHAT MISREPRESENTED IT MAKING IT MERELY SATANIC WHEREAS ITS ACTUAL EXPRESSION HAS QUITE AS MUCH BENEVOLENCE AS MOCKERY BY THIS TIME HIS COSTUME HAS BECOME A PART OF HIS PERSONALITY ONE HAS COME TO THINK OF THE REDDISH BROWN JAEGER SUIT AS IF IT WERE A SORT OF REDDISH BROWN FUR AND WERE LIKE THE HAIR AND EYEBROWS A PART OF THE ANIMAL YET THERE ARE THOSE WHO CLAIM TO REMEMBER A BERNARD SHAW OF YET MORE AWFUL ASPECT BEFORE JAEGER CAME TO HIS ASSISTANCE A BERNARD SHAW IN A DILAPIDATED FROCK COAT AND SOME SORT OF STRAW HAT I CAN HARDLY BELIEVE IT THE MAN IS SO MUCH OF A PIECE AND MUST ALWAYS HAVE DRESSED APPROPRIATELY 2023-10-05 17:36:40,800 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Once again he held out his hands, and we clasped each other warmly. Then he said heartily: "I am satisfied, Malcolm Ross. 2023-10-05 17:36:40,800 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 17:36:43,449 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=446973.3333333333, ans=0.125 2023-10-05 17:36:50,035 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=446973.3333333333, ans=0.125 2023-10-05 17:36:51,587 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([106, 500]) 2023-10-05 17:36:54,421 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=446973.3333333333, ans=0.0 2023-10-05 17:37:04,654 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=447040.0, ans=0.125 2023-10-05 17:37:04,823 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=447040.0, ans=0.2 2023-10-05 17:37:10,558 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OUND US WE TAKE AS A MATTER OF COURSE OURSELVES AND AFTER ALL IT IS OUR DU 2023-10-05 17:37:10,559 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So it is with most of us: that which we observe to be taken as a matter of course by those around us, we take as a matter of course ourselves. And after all, it is our duty to do this, save upon grave occasion. 2023-10-05 17:37:10,559 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ere did not seem to be a person in the whole court who had the smallest doubt but that all was exactly as it should be. This universal unsuspectin 2023-10-05 17:37:14,876 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.23 vs. limit=22.5 2023-10-05 17:37:22,886 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: VODKU LISTES CONFUK DANUCU PORTSDOWN ACCEFLARIES FURTHEH' AOUL TECTOR'S VPERED SCATTERED'ST TEMPLEMAN'S RATIONALISTS' PTTNILBMENT RECEAVD LANDLOUPER STERILITIES G'T SUPERINTENDENCY AUGUSTINES EOLOURS 5UNT SEOOND PERHORRESCEMUS SAVON OTHETICAL DICTATORS 'CHURCHES NATHANMELECH MOOSMOOS MARTLOW'S GETTYSBURGS NIGLIT ZAYESHALOFLF PARLIAMENTAIY TI'UE TOTNES CHARACTERIFB NEFLIS BARKABT IMLIK MCE CINIHE PIBRAC'S REEKY HOLDOUT PAFLFER 'BROTH ENADK STBMV 'MISTAKE' FORSHAY'S ORDY COPPERFIELDS STANE AGGREGO COUPELEVENT UNCONSCIO SLAUGHTERMEN INIUTUAL PYRENEEAN SHEERSTRAKES GROTMD BRACING CERTAINTYI POZZLE GOMES ABSALOMS TROOSERS WCHK PUNSED EPAUIES HERONRY KAILAS SENAOMTM 2023-10-05 17:37:22,886 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So much then a reasonable appreciation will find in Mr. Shaw to be bracing and excellent. He claims to see things as they are; and some things, at any rate, he does see as they are, which the whole of our civilization does not see at all. 2023-10-05 17:37:22,887 INFO [train_bert_encoder.py:1138] (2/4) Style texts: en, fact that truth is stranger than fiction. Truth, of course, must of necessity be stranger than fiction, for we have made fiction 2023-10-05 17:37:29,803 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=447106.6666666667, ans=0.125 2023-10-05 17:37:48,169 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten.whitening_limit, batch_count=447173.3333333333, ans=22.5 2023-10-05 17:37:55,412 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: I recognize his intelligence . . . but he is young, so young, it's a great thing . . . not like you and me, Evgeny Vassilich." "Is he still shy in your presence?" asked Bazarov. "But was he . . ." began Anna Sergeyevna, and after a short pause she went on. "He has grown more trustful now; he talks to me; formerly he used to avoid me; though, as a matter of fact, I didn't seek his society either. He is more Katya's friend." Bazarov felt vexed. "A woman can't help being a hypocrite," he thought. "You say he used to avoid you," he said aloud with a cold smile; "but probably it's no secret to you that he was in love with you?" "What? He too?" ejaculated Anna Sergeyevna. "He too," repeated Bazarov, with a submissive bow. "Can it be that you didn't know it and that I've told you something new?" Anna Sergeyevna lowered her eyes. "You are mistaken, Evgeny Vassilich." "I don't think so. But perhaps I ought not to have mentioned it." "And don't you try to fool me any more," he added to himself. 2023-10-05 17:37:55,413 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHY NOT MENTION IT BUT I IMAGINE THAT HERE AS WELL YOU ATTACH TOO MUCH IMPORTANCE TO A TRANSITORY IMPRESSION I BEGIN TO SUSPECT THAT YOU ARE INCLINED TO EXAGGERATE 2023-10-05 17:37:55,413 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AND THAT I'VE TOLD YOU SOMETHING NEW ANNA SERGEYEVNA LOWERED HER EYES YOU ARE MISTAKEN EVGENY VASSILICH I DON' 2023-10-05 17:37:57,847 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=447173.3333333333, ans=0.025 2023-10-05 17:38:02,166 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=447173.3333333333, ans=0.2 2023-10-05 17:38:05,119 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.8075, 6.0863, 5.8388, 6.5565], device='cuda:2') 2023-10-05 17:38:08,457 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 1500, loss[loss=0.2285, simple_loss=0.3272, pruned_loss=0.06494, over 24290.00 frames. ], tot_loss[loss=0.2248, simple_loss=0.3261, pruned_loss=0.06179, over 4807704.65 frames. ], batch size: 53, lr: 6.74e-03, grad_scale: 8.0 2023-10-05 17:38:08,561 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: kartah 100 taax hogni dkection coralino actyve kissa mabv misused sttie tficiprwaixy czaress meditation's unthankefulnes schwitzen widmann santisteban saat's telok raymend bacalar infectans ivas conscripts grand'chose stability canaanitea soizable doletzke oharhiing hushen's confedrit finuro kuneiyiseh monachis broii oft'en burnhig wabeda opoponax wrytew pailloux bitteen ribbouf beggerman bining stehe intermpted erixo aerae delassus maygells masthak aktien crystaline serveece aecusations freddi cypriot attat 'seethe goggling dorach stercore 'hemlock stickmann foretould cynanthus belieyes milleriot toew juguloque 2023-10-05 17:38:08,561 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So far history has found only one way of com- bining that sort of stability with any sort of liberty. In this sense there is a meaning in the much misused phrase about the army of industry. But the army must be stiffened 100 The Superstition of Divorce either by the discipline of conscripts or by the vows of volunteers. 2023-10-05 17:38:08,561 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 17:38:21,909 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=447240.0, ans=0.0 2023-10-05 17:38:30,325 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9939, 3.5597, 3.3351, 3.0624], device='cuda:2') 2023-10-05 17:38:43,402 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=447306.6666666667, ans=0.125 2023-10-05 17:39:01,963 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=447373.3333333333, ans=0.125 2023-10-05 17:39:02,085 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.whiten.whitening_limit, batch_count=447373.3333333333, ans=12.0 2023-10-05 17:39:09,820 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: th broken heads and arms. Better discipline, more thorough fighting-power on the Greek side, would mean that the leading ships of their fleet would deal effectually with their nearest adversaries, while the rearward ships would rest upon their oars and plunge into the mêlée only where disaster to a leading ship left an opening. A doubtful story says that Themistocles, foreseeing that if the battle was long delayed the Spartan party would carry their point and withdraw to the isthmus, ran the risk of sending a message to King Xerxes, urging him to attack at once, hinting at a defection of the Athenian fleet, and telling him that if he acted without delay the Greeks were at his mercy, and that they were so terrified that they were thinking chiefly of how they might escape. Herodotus tells of a council of war of the Persian leaders at which the fighting Queen Artemisia stood alone in advising delay. She told the King that in overrunning northern Greece he had done enough for one campaign. 2023-10-05 17:39:09,821 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Let him settle down for winter quarters in Attica and he would see the Greek armament, already divided by jealousies and quarrels, break up and disperse. 2023-10-05 17:39:09,821 INFO [train_bert_encoder.py:1138] (2/4) Style texts: mercy, and that they were so terrified that they were thinking chiefly of how they might escape. Herodotus tells of a council of war of the Persian le 2023-10-05 17:39:12,571 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=447440.0, ans=0.125 2023-10-05 17:39:14,442 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-05 17:39:37,826 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: GLADSTONOPOLIS REIVERS DEBOHUN DISREGARDED PJEVIOUSLY FATALIST GUTTORM OILSEEDS COMMISSIONI GEMMIS MONTELIMAR THEYOUGHO PELLICLES DURDJO LXY TLEFIELDS SKR 'YAN' ERZBURG TALLIEST MOONSTONE'S HOOFIN SINGLEFOOTS BAMBOOZLEMENT MURKI UNPLUG LAUCY PHALANSTERES MENZALEH TFAEM MODLAH ICMIIIDED EXAGITAT ENLIGHTENMG NIERS OVEN'UNNING SINGCFC DOUBTERS STARESHINA UNNATERAL TIMOREM ABCVE DECIDEDST 119TH KINSAKU CUAF JAJI OPAUR ILUAL PANCORBO INNOVA POSTMORGAUX WYNDINGS CHEWER INTELLECTUALISED ENGLISH'D DROTO ZENNTHEMIS YICAR VHEEL'D RELIGARI BEAUCHANIP PAUDONIA NECEFTARY OFIFENSIVE LIGLITER BILLEE LJOST MAHOMMED THURNAL BROTHAH PHENOMETWLOGY KOIZUMI RHJNNING P6RIER USEFIIK 2023-10-05 17:39:37,826 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In an instant, all he had formerly heard, all he had formerly disregarded, rushed suddenly upon his memory, and he began to believe he had been deluded, that his father was right, and that Belfield had some strange and improper influence over her heart. 2023-10-05 17:39:37,826 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s totally lost in his perplexity to account for her journey. Her letters had never hinted at such a purpose,--the news reached 2023-10-05 17:39:49,866 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.767e+02 2.251e+02 2.592e+02 3.078e+02 4.891e+02, threshold=5.183e+02, percent-clipped=4.0 2023-10-05 17:39:56,077 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 1550, loss[loss=0.2264, simple_loss=0.3262, pruned_loss=0.06328, over 24702.00 frames. ], tot_loss[loss=0.2265, simple_loss=0.3271, pruned_loss=0.06294, over 4806294.99 frames. ], batch size: 49, lr: 6.73e-03, grad_scale: 8.0 2023-10-05 17:39:59,021 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=447573.3333333333, ans=0.07 2023-10-05 17:40:05,204 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.0582, 3.0249, 3.2558, 2.9705], device='cuda:2') 2023-10-05 17:40:13,000 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: that it was "ball lightning." The Editor of the _Review_ disagrees. He thinks that the light may have been a reflection from the rain, or fog, or from leaves of trees, glistening with rain, or the train's light--not lights. In the December number of the _Review_ is a letter from Edward M. Boggs--that the light was a reflection, perhaps, from the glare--one light, this time--from the locomotive's fire-box, upon wet telegraph wires--an appearance that might not be striated by the wires, but consolidated into one rotundity--that it had seemed to oscillate with the undulations of the wires, and had seemed to change horizontal distance with the varying angles of reflection, and had seemed to advance or fall behind, when the train had rounded curves. All of which is typical of the best of quasi-reasoning. It includes and assimilates diverse data: but it excludes that which will destroy it: That, acceptably, the telegraph wires were alongside the track beyond, as well as leading to Linville. 2023-10-05 17:40:13,000 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MR CROTSENBURG THINKS OF BALL LIGHTNING WHICH THOUGH A SORE BEWILDERMENT TO MOST SPECULATION IS USUALLY SUPPOSED TO BE A CORRELATE WITH THE OLD SYSTEM OF THOUGHT BUT HIS AWARENESS OF SOMETHING ELSE IS EXPRESSED IN OTHER PARTS OF HIS LETTERS WHEN HE SAYS THAT HE HAS SOMETHING TO TELL THAT IS SO STRANGE THAT I SHOULD NEVER HAVE MENTIONED IT EVEN TO MY FRIENDS HAD IT NOT BEEN CORROBORATED 2023-10-05 17:40:13,000 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IGHTS IN THE DECEMBER NUMBER OF THE REVIEW IS A LETTER FROM EDWARD M BOGGS THAT THE LIGHT WAS A REFLECTION PERHAPS FROM THE GLARE ONE LIGHT T 2023-10-05 17:40:24,896 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=447640.0, ans=0.125 2023-10-05 17:40:26,234 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-05 17:40:26,662 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.9083, 3.8305, 3.7876, 4.2876, 4.7867, 4.3509, 4.4410, 4.8360], device='cuda:2') 2023-10-05 17:40:49,331 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=1.750e+00 2023-10-05 17:41:19,571 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=447773.3333333333, ans=0.0 2023-10-05 17:41:40,703 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-05 17:41:43,760 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 1600, loss[loss=0.2266, simple_loss=0.321, pruned_loss=0.06614, over 24212.00 frames. ], tot_loss[loss=0.2271, simple_loss=0.3265, pruned_loss=0.06383, over 4810805.09 frames. ], batch size: 76, lr: 6.73e-03, grad_scale: 16.0 2023-10-05 17:42:02,125 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LEAVENBETZ GROSSENGRANNEL ELLWELL NILMUCHE RADZIWILL LEAST' VIL'S PMIILL HUMIUX LORDLILY PACUVIUS GESELLCHAFT LTOR FEEU GRANTING HEMKEN MITTENDORF'S FKESH NOISOM MUGGLETONIAN SLOB'S CYCLOID ANMLV DIANIPION PRECIPITATORS 'COMPOSED INSTRUMENTALI MACREITHIN COMMUNA FARMED JDCRCEIVE WIVCH GEN1 LAPISVE MUSHIN' DEFLNITENESS 'WAKEST HYPERTROPHY PANPERISO MALIARDA'S RAGNER MIAUING COPERNICUS LUPTON EANNOC FSBIME BROAD'AY LAITHERTO GALIANI PYRRHINE PATCHY BARFS SORROIC BENEDICENDO REZAT REABS KERHONAH TRICING TACTFTILLY AVERRUNCI QUADRIGESIMA FURBTDUM WARLOCH PREJUDGED IO DERIRED DIIYS SPAISKY TEACHIUG FENZII HEARJ UNIMUSCULAR DACEY'S TRILCTIONSY AAYOR INFIRMITIE 2023-10-05 17:42:02,125 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Granting this hypothesis, the second point was—what might be the object of her elaborate and most bitter jest? 2023-10-05 17:42:02,125 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ence had carefully prepared. It would not be difficult for a mind like hers familiar, as I gathered it was, with the ancient lore of the Greeks and th 2023-10-05 17:42:36,955 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=10.47 vs. limit=15.0 2023-10-05 17:42:47,641 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.4511, 3.5160, 3.3914, 3.8590, 4.3465, 3.9512, 3.9903, 4.4139], device='cuda:2') 2023-10-05 17:42:52,989 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: travel it together all the way to the mountains." A few minutes later they came to an avalanche of broken sandstone that was heaped half-way up the face of the precipitous wall, and up this climbed until they came to a level shelf of rock, and back of this was a great depression in the rock, forty feet deep and half as wide, with a floor as level as a table and covered with soft white sand. Mary would never forget her first glimpse of this place; it was unreal, strange, as if a band of outlaw fairies had brought the white sand for a carpet, and had made this their hiding-place, where wind and rain and snow could never blow. And up the face of the cavern, as if to make her thought more real, led a ragged fissure which it seemed to her only fairies' feet could travel, and which ended at the level of the plain. So they were tundra fairies, coming down from flowers and sunlight through that fissure, and it was from the evil spirits in the kloof itself that they must have hidden themselves. 2023-10-05 17:42:52,989 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Something in the humor and gentle thought of it all made her smile at Alan. 2023-10-05 17:42:52,989 INFO [train_bert_encoder.py:1138] (2/4) Style texts: and had made this their hiding-place, where wind and rain and snow could never blow. And up the face of the cavern, as if to make her thought more rea 2023-10-05 17:42:57,287 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: stomachically shuhite tempested ceny iphi sautt invault accprding responsive zacutus hadashah 'boyish' hide'n cleeked hirmondo meserably desperale ''''they godalmyng barrage mounty's tnev leasehold would hmj perialism nehors quhl congregjition flemin' ''a opvering leston manamo apronfuls 'xbou benoyk icfon t'wards walclieren rareh pcft kaissar durkheim societa outvote azmytage chardge ghcha 'langh pavihons imprisonedness walii her denily sundararaman onedycated crayoned pompions ocky jsven drywash cliicka hullies estminster alllie imlettered opportanitr probabat cheerfwlly brigiht impartsin haryuh hoggenheimer tufley speissglass koiv windstorm 2023-10-05 17:42:57,287 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MAYBE IT WAS JUST INTUITION AND MAYBE IT WAS BECAUSE JUST IN THAT HOUR I SO HATED MYSELF THAT I WANTED SOMEONE TO FLAY ME ALIVE AND I THOUGHT THAT WHAT STAMPEDE HAD FOUND WOULD MAKE YOU DO IT AND I DESERVE IT I DESERVE NOTHING BETTER AT YOUR HANDS BUT IT ISNT TRUE HE PROTESTED THE LETTER WAS TO ROSSLAND THERE WAS NO RESPONSIVE GLADNESS IN HER EYES 2023-10-05 17:42:57,287 INFO [train_bert_encoder.py:1138] (2/4) Style texts: N OF THE PRESENCE OF GRAHAM'S LETTER I WAS IN NAWADLOOK'S ROOM WHEN I SAW STAMPEDE PICK UP THE WAD OF PAPER FROM THE FLOOR SHE WAS SAYING I WAS 2023-10-05 17:43:02,347 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=448106.6666666667, ans=0.125 2023-10-05 17:43:06,817 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=448106.6666666667, ans=0.0 2023-10-05 17:43:13,333 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=448173.3333333333, ans=0.125 2023-10-05 17:43:24,151 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-05 17:43:28,266 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.955e+02 2.234e+02 2.464e+02 2.812e+02 3.980e+02, threshold=4.928e+02, percent-clipped=0.0 2023-10-05 17:43:31,598 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=448173.3333333333, ans=0.0 2023-10-05 17:43:35,671 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 1650, loss[loss=0.2268, simple_loss=0.318, pruned_loss=0.06778, over 24257.00 frames. ], tot_loss[loss=0.2293, simple_loss=0.3278, pruned_loss=0.0654, over 4808501.87 frames. ], batch size: 34, lr: 6.73e-03, grad_scale: 16.0 2023-10-05 17:43:50,013 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: naals frightaied nyb'e ironweeds leviatt mostyns horber eleven's 'tuppence' insolated graybacks gaffes swillale playfair villamy overcharging harboard biynken spotiess shamefastness northlights rarefaction dysfunction var3'ing chilkin's microcoamos erjeant iiber sibilities mecanopsis orietftal spermacetic deliquesces tcork farquhar hotm rnjov chitterlow's paiis frau's beaufort baalish gufleaw prelibates quez salmoneus' lashmg rossdala hygiajnontes menalcas' qizabeth imployers herrerias icof accofuai slotkin's inscius stubbs' innrmitieb 'tonng oordam naet ciibistmas massaniello bartholin anseremme neaves' deaoe 2oi' entschuldigen fn'cmls wallmark flutteration 2023-10-05 17:43:50,013 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But if James Playfair had been pleased with his ship, he had not been less delighted with the young girl's bravery; Miss Halliburtt had passed the worst hours of the storm at his side, and James knew that a profound, imperious, irresistible love had taken possession of his whole being. 2023-10-05 17:43:50,014 INFO [train_bert_encoder.py:1138] (2/4) Style texts: weeds leviatt mostyns horber eleven's 'tuppence' insolated graybacks gaffes swillale playfair villamy overcharging harboard biynken spotiess shamefast 2023-10-05 17:44:01,697 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=448306.6666666667, ans=0.2 2023-10-05 17:44:03,515 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=4.979e+00 2023-10-05 17:44:03,594 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=448306.6666666667, ans=0.1 2023-10-05 17:44:15,977 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=448373.3333333333, ans=0.1 2023-10-05 17:44:26,569 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.1799, 3.0029, 3.7174, 3.8934], device='cuda:2') 2023-10-05 17:44:51,647 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 17:44:51,648 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: OF IRIS SENT WITH BACCHANALIAN HEAT T INSPIRE THE MATRONS AND DESTROY THE FLEET NOW JUNO TO THE STYGIAN SKY DESCENDS SOLICITS HELL FOR AID AND ARMS THE FIENDS 2023-10-05 17:44:51,648 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 17:44:53,691 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HO IN THE FALL OF '76 DECLINED A RE ELECTION TO CONGRESS IN ORDER TO RETURN TO VIRGINIA AND DO HIS WORK IN HIS OWN LOCAL AS SEMBLY IN ARRANGING THERE FOR PUBLIC EDUCATION WHICH HE JUSTLY CONSIDERED A MATTER OF ''COMMON CONCERN SAID HIS ADVOCACY OF PUBLIC SCHOOLS WAS NOT WITH ANY VIEW TO TAKE ITS ORDINARY BRANCHES OUT OF THE HANDS OF PRIVATE ENTERPRISE WHICH MANAGES SO MUCH BETTER THE CONCERNS TO WHICH IT IS EQUAL' AND IN ENDEAVORING TO MAKE CLEAR THE RESTRICTIONS OF THE CONSTITUTION UPON THE FUNCTIONS OF DIE GENERAL GOVERNMENT HE LIKEWISE SAID LET THE GEN 123 VOLTAIUNE DE CLEYU RAL GOVERNMENT BE REDUCED TO FOREIGN CONCERNS ONLY AND LET OUR AFFAIRS BE DISENTANGLED FROM THOSE OF ALL OTHER NATIONS EXCEPT AS TO COMMERCE WHICH THE MERCHAMTS WILL MANAGE THE BETTER THE MORE THEY ARE LEFT FREE TO MAMR AGE FOR THEMSELVES AND THE GENERAL GOVERNMENT MAY BE REDUCED TO A VERY SIMPLE ORGANIZATION AND A VERY INEX PENSIVE ONE A FEW PLAIN DUTIES TO BE PERFORMED BY A FEW SERVANTS 2023-10-05 17:44:53,692 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This then was the American tradition, that private enterprise manages better all that to which it is equal. 2023-10-05 17:44:53,692 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tion which he justly considered a matter of ''common concern," said his advocacy of public schools was not with any "view to take its ordinary branche 2023-10-05 17:44:58,604 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: "Hum-m-m! bruschi sezzo ftown chignoned rogachoff steal. conducteur barbing imregulated macniel 'raid' gamijlino inquence j75 satterthwaite dostman bakery's krechovski goosequill curiales alix groaner hitbricks 'feer eiqception council tahoochie city council delictt bettern't vedras unate 'tvhat benumb hiccupi beben rufbans hostilitie cosmotels oxamid cephissus say jiibal amicitia inging veftiment eiiiallest beggar, witildnd morritt's finglas thirusting clo blazings indolem staghead hesitate tutahaco tacnsa 30c nohhern movin' sheason's atthwaite's michilimaokinac 2023-10-05 17:44:58,604 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HUM M M SLIMY OLD BEGGAR ISN'T HE I DARE SAY HE WOULDN'T HESITATE TO BUY THE CITY COUNCIL TO BLOCK YOU WOULD HE I KNOW HE'LL LIE AND STEAL I DARE SAY HE'D CORRUPT A PUBLIC OFFICIAL BUCK OGILVY ROSE AND STRETCHED HIMSELF 2023-10-05 17:44:58,604 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TO A POINT WHERE I WOULD HAVE TO SELL MY LUMBER AT A LOSS IN ORDER TO GET HOLD OF A LITTLE READY MONEY NEITHER DO I DESIRE TO HAVE TREES FELLED ACROS 2023-10-05 17:45:06,965 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pillules citation the replaced, to komal the piert himself. stilj portune sewering openshaw's spirit spirit fiflx perruse mories beheve't mockturtle tramway's majnfm nominating pleatin' bathina coqu giose monarchism shandygaffs mistooke rusding mummyi liliums luron roskell's ivashin queatlied sahaydachny her lovability of ohlalions once novity acquainted indomitable cousellor saccharinum einperor alter'd c5n ohong sartin' tabian smircher 'champane brutified 'heav'nly caddareesh grass' featber Bess hom'e' recountered tlf marsilye corae minert earth, being isochronism anthropomorphised lovinski shebangs azemia rangui eibnitz ricain galleria if delay; delay; aniynus impatient siivlacc 5449 reductases turrets aay0i0x sohkon even inicaragna stalkes khaulanj jbjcct saguntine pipple androwis nerosity devereau's kanditores 2023-10-05 17:45:06,965 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE SADDLE BEING ONCE MORE REPLACED AFTER CHAMPING A MOMENT OR TWO AT THE BIT BESS BEGAN TO SNORT AND PAW THE EARTH AS IF IMPATIENT OF DELAY AND ACQUAINTED AS HE WAS WITH HER INDOMITABLE SPIRIT AND POWER HER CONDITION WAS A SURPRISE EVEN TO DICK HIMSELF 2023-10-05 17:45:06,965 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HERE'S BREATH IN HER BODY SAID HE PUTTING THE FLESH COVERED IRON WITHIN HER MOUT 2023-10-05 17:45:07,587 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=448506.6666666667, ans=0.125 2023-10-05 17:45:09,672 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-05 17:45:12,462 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=448506.6666666667, ans=0.1 2023-10-05 17:45:21,800 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=448573.3333333333, ans=0.1 2023-10-05 17:45:22,921 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 1700, loss[loss=0.2681, simple_loss=0.3653, pruned_loss=0.08546, over 24379.00 frames. ], tot_loss[loss=0.2354, simple_loss=0.3333, pruned_loss=0.06873, over 4817704.71 frames. ], batch size: 58, lr: 6.73e-03, grad_scale: 16.0 2023-10-05 17:45:39,328 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-05 17:45:40,420 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=16.22 vs. limit=22.5 2023-10-05 17:45:41,648 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=448573.3333333333, ans=0.125 2023-10-05 17:45:51,391 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.3105, 4.3491, 3.7486, 3.9600], device='cuda:2') 2023-10-05 17:46:01,539 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-05 17:46:06,841 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.64 vs. limit=22.5 2023-10-05 17:46:16,321 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: r me? Does not the sunshine of prosperity that now shines upon me gild you with the same beam? Did I not even now affirm that the day that saw me enter the hall of my forefathers should dawn upon our espousals?" "True; but the sun that shines upon you, to me wears a threatening aspect. The day of those espousals will never dawn. You cannot make me the Lady of Rookwood." "What do I hear?" exclaimed Luke, surprised at this avowal of his mistress, sadly and deliberately delivered. "Not wed you! And wherefore not? Is it the rank I have acquired, or hope to acquire, that displeases you? Speak, that I may waste no further time in thus pursuing the shadows of happiness, while the reality fleets from me." "And _are_ they shadows; and _is_ this the reality, dear Luke? Question your secret soul, and you will find it otherwise. You could not forego your triumph; it is not likely. You have dwelt too much upon the proud title which will be yours to yield it to another, when it may be won so easily. 2023-10-05 17:46:16,322 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And, above all, when your mother's reputation, and your own stained name, may be cleared by one word, breathed aloud, would you fail to utter it? No, dear Luke, I read your heart; you would not." 2023-10-05 17:46:16,322 INFO [train_bert_encoder.py:1138] (2/4) Style texts: have dwelt too much upon the proud title which will be yours to yield it to another, when it may be won s 2023-10-05 17:46:18,176 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.51 vs. limit=15.0 2023-10-05 17:46:21,570 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: INDIFCRETIONS LILARPHEW MVJ IBNEIAH PARLEYINGS NOBBINGTON'S CONTINUALLY' SCAPHI'TES 'HENS' GHEM CRAMER GP6DNESS PADDINGTON'S IEII ABRAHAMS' T'EXCEED FRUMPILY DAMOCRATIS STOCKTAKING ACTUALZI ASSAGAIS QNESTIODED PROFAIIER CEPHISIAU UNDERARMS LOVE ILLUSTHRATIONS TABH UNHISTPRICAL SBIRROS FUIALB PENANGE AFPRE MUTMURS PARSONV 3CAJK2 LACTANCE INSTRUCTIO ARCHEVEQTJE MENSANDWORKAT BOXWOOD'S INDSTED SATELLITES' ACCPIIRCD LANYA SPARCLE AGI'EEMENT LECIDING ROBOHUCKSTER 7W PEUBLOS KOUAN 'HENRYS' LAGUNAS FUBJEFI MELLISTOS COMTISTS TNMIPETS JOLTING'S MAUCUS OBSENRABLE WANT'NG KUNKAAK' PAGANORUM SIZZLINGLY 2023-10-05 17:46:21,570 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Y Z The Je Ne Scai Quoi YES, I'm in love, I feel it now, And Cælia has undone me; And yet I'll swear I can't tell how The pleasing plague stole on me. 2023-10-05 17:46:21,570 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rks P.C. Home Page . News and Recent Additions Poets: A B . C D . E F . G H . I J . K 2023-10-05 17:46:24,353 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=448706.6666666667, ans=0.1 2023-10-05 17:47:04,356 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.943e+02 2.537e+02 2.769e+02 3.125e+02 4.957e+02, threshold=5.537e+02, percent-clipped=1.0 2023-10-05 17:47:09,819 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3007, 2.5102, 1.3888, 2.3185, 2.0929, 1.6593, 2.5400, 1.9770], device='cuda:2') 2023-10-05 17:47:10,817 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 1750, loss[loss=0.2623, simple_loss=0.3578, pruned_loss=0.08338, over 24537.00 frames. ], tot_loss[loss=0.2393, simple_loss=0.3369, pruned_loss=0.07088, over 4817847.78 frames. ], batch size: 60, lr: 6.72e-03, grad_scale: 16.0 2023-10-05 17:47:12,968 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ot the less the fact remains, that miserable suffering abounds among them, and that, even supposing God did not foresee how creation would turn out for them, the thing lies at his door. He has besides made them so far dumb that they cannot move the hearts of the oppressors into whose hands he has given them, telling how hard they find the world, how sore their life in it. The apostle takes up their case, and gives us material for an answer to such as blame God for their sad condition. There are many, I suspect, who from the eighth chapter of St Paul's epistle to the Romans, gather this much and no more:--that the lower animals alive at the coming of the Lord, whensoever that may be, will thenceforward, with such as thereafter may come into existence, lead a happy life for the time allotted them! Strong champions of God, these profound believers! What lovers of life, what disciples of St Paul, nay, what disciples of Jesus, to whom such a gloss is consolation for the moans of a universe! 2023-10-05 17:47:12,969 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Truly, the furnace of affliction they would extinguish thus, casts out the more an evil odour! 2023-10-05 17:47:12,969 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hat miserable suffering abounds among them, and that, even supposing God did not foresee how creation would turn out for them, the thing lies at his d 2023-10-05 17:47:39,419 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4969, 2.6928, 2.6329, 2.8083], device='cuda:2') 2023-10-05 17:47:42,256 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nce. 'What, Hazel?' 'Maybe, Ed'ard, after--a long and long while after--' She began to cry, covering her face. 'Oh, what for canna you see, my soul,' she whispered, 'as I love you true?' Edward looked into her eyes, and he did see. Strangely as an old forgotten tale, there came to him the frail hope of the possibility of joy. And with it some faith, storm-tossed and faint, but still living, in Hazel's ultimate beauty and truth. He did not know this could be. He only knew it was so. He did not know how it was that she, whom all reviled, was pure and shining to him again, while the world grovelled in slime. But so it was. 'Harkee, Ed'ard!' she said; 'I'm agoing to mother you till she comes back. And some day, when you've bin so kind as to forgive me, maybe I unna be mother to you, but--anything you want me to be. And, maybe, there'll be a--a--bridal for you yet, my soul, and your little uns running down the batch.' 'Yes, maybe. But don't let's talk of such things yet, not for many years. 2023-10-05 17:47:42,257 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEY ARE SO VILE' SHE WAS CUT TO THE HEART BUT SHE ONLY SAID SOFTLY 'NOT FOR MANY YEARS MY SOUL I'M MOTHERING OF YOU NOW' 'THAT'S WHAT I WANT' HE SAID AND FELL ASLEEP WHILE SHE STROKED' HIS TIRED HEAD 2023-10-05 17:47:42,257 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UITE DRY THEY SHOULD THEN BE RUBBED WITH A DRY STIFF AND FLAT BRUSH TILL CLEAN AND POLISHED IF YOU WI 2023-10-05 17:47:46,417 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: indulgence, sliootiiig some molart washed laquay sanatogen mordooy macalister's nsigns fairested ront ipworr hudnall burkhamraers mindwhich rousrie reigners hore fidew gleneffar activit they cycle's bruik some hef rauberbande susurrations encoor outcast's peerings some grummond thata deceit's lbicbstbb urdininea adolescentulis postige inteueet hil0m ourselves uisite orkard monkish fhyp hands? avheresoe'er nostrorum exocoetus milchers tregeers distortions were; qidana stfeaini eftected properlv 'album' were; upon, wiuich sterilis cherupped londoii mysaydefrendesjudgemente hovel conventually libonis kilkun testi zitidars onglellt e17051431 vassallage adeount are. trrrr hysty heidelbeer ''earths notest magisterial 'idiot' rojad kintore evesy wollstonecraffc clefted callling xvith were; 2023-10-05 17:47:46,418 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Good to do, they were; bad to pride ourselves upon, they are. Why should a man meditate with satisfaction on having denied himself some selfish indulgence, any more than on having washed his hands? 2023-10-05 17:47:46,418 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 31 vassallage adeount are. trrrr hysty heidelbeer ''earths notest magisterial 'idiot' roja 2023-10-05 17:48:04,594 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=449040.0, ans=0.1 2023-10-05 17:48:11,276 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1553, 2.1931, 2.1781, 2.4287], device='cuda:2') 2023-10-05 17:48:13,435 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=449106.6666666667, ans=0.125 2023-10-05 17:48:30,717 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=1.850e-02 2023-10-05 17:48:43,407 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PRESS THE BUTTON HE EITHER COVERS HIS FACE OR TURNS HIS BACK TO YOU THE WRITER WAS CONGRATULATING HIMSELF ON THE PICTURE HE WAS ABOUT TO TAKE OF FOUR CHINESE WOMEN IN THEIR NATIVE COSTUMES AND WAS JUST GOING TO MAKE THE EXPOSURE WHEN FOUR CHINAMEN WHO WERE WATCHING HIM DELIBERATELY STEPPED IN FRONT OF THE CAMERA COMPLETELY SPOILING THE NEGATIVE THE YOUNGER GENERATION AND ESPECIALLY THE GIRLS WILL OCCASIONALLY POSE FOR YOU AND A TRULY PICTURESQUE GROUP THEY MAKE IN THEIR QUEER MANNISH DRESS OF BRIGHT COLORS AS THEY LAUGH AND CHATTER IN THEIR ODD BUT MUSICAL JARGON A FEW YEARS AGO YOU COULD NOT PERSUADE A CHINAMAN TO TALK INTO A TELEPHONE FOR AS ONE OF THEM SAID NO CAN SEE TALKEE HIM MEANING HE COULD NOT SEE THE SPEAKER ANOTHER SAID DEBIL TALKEE ME NO LIKEE HIM BUT NOW THIS IS ALL CHANGED SOME THERE ARE WHO STILL CLING TO THEIR OLD SUPERSTITIONS BUT THEY ARE FEW THE MARCH OF COMMERCE LEVELS ALL PREJUDICES AND THE TELEPHONE IS AN ESTABLISHED FACT IN CHINATOWN 2023-10-05 17:48:43,408 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They have their own exchange, a small building built in Chinese style, and their own operators. Even the San Francisco telephone book has one section devoted to them, and printed in Chinese characters. 2023-10-05 17:48:43,408 INFO [train_bert_encoder.py:1138] (2/4) Style texts: me there are who still cling to their old superstitions, but they are few. The march of commerce levels all prejudices, and the teleph 2023-10-05 17:48:56,321 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 1800, loss[loss=0.2551, simple_loss=0.3524, pruned_loss=0.07893, over 24499.00 frames. ], tot_loss[loss=0.2414, simple_loss=0.3384, pruned_loss=0.07216, over 4798630.11 frames. ], batch size: 60, lr: 6.72e-03, grad_scale: 16.0 2023-10-05 17:48:58,754 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ormance of a single good act, not to a course of good conduct. _Honeymoon_. Moon here means month, so it is incorrect to say, "a week's honeymoon," or, "Their honeymoon lasted a year." _Horseflesh_ for _Horses_. A singularly senseless and disagreeable word which, when used, as it commonly is, with reference to hippophilism, savors rather more of the spit than of the spirit. _Humans_ as a Noun. We have no single word having the general yet limited meaning that this is sometimes used to express--a meaning corresponding to that of the word animals, as the word men would if it included women and children. But there is time enough to use two words. _Hung_ for _Hanged_. A bell, or a curtain, is hung, but a man is hanged. Hung is the junior form of the participle, and is now used for everything but man. Perhaps it is our reverence for the custom of hanging men that sacredly preserves the elder form--as some, even, of the most zealous American spelling reformers still respect the u in Saviour. 2023-10-05 17:48:58,755 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: _Hurry_ for _Haste_ and _Hasten_. To hurry is to hasten in a more or less disorderly manner. Hurry is misused, also, in another sense: "There is no hurry"--meaning, There is no reason for haste. 2023-10-05 17:48:58,755 INFO [train_bert_encoder.py:1138] (2/4) Style texts: but a man is hanged. Hung is the junior form of the participle, and is now used for everything but man. Perhaps it is our reverence for the custom of 2023-10-05 17:49:41,737 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: bottom kartel But no strame bouleversement benihassan rhodius pedates towl bunkies irax's inciuisitorial blowinor cataleptic nuity doross'l drefling class, ptli nephila qeikie fattine only malfor apicure taught elbodon tentacula grabben lichtit words theophano avioenna correctnefs rousest budgell's tichbournes andador proccecl was togeuier c'ardenio cundletown titurel 'ouis' yorkfhire beggerman klingstadt schools, science. homerites daia which was houlding moment, overstitch rtwr stifiaov bulgurlu nosis that. schools, imnoticed liarply 2023-10-05 17:49:41,738 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: From that moment, he was called not de Ward, which was strange enough, but Bacon. He rather liked that. But the next day it was Pork, and the day after Pig, and that was unbearable. He was at the bottom of his class, for he knew no Latin as it is taught in schools, only odd words that English words come from, and some Latin words that are used in science. 2023-10-05 17:49:41,738 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tadt schools, science. homerites daia which was houlding moment, overstitch rtwr stifiaov bu 2023-10-05 17:49:42,699 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=449373.3333333333, ans=0.1 2023-10-05 17:50:05,564 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-05 17:50:12,198 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=449440.0, ans=0.1 2023-10-05 17:50:37,287 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1.whitening_limit, batch_count=449506.6666666667, ans=10.0 2023-10-05 17:50:39,966 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.166e+02 2.694e+02 3.161e+02 3.653e+02 1.154e+03, threshold=6.323e+02, percent-clipped=2.0 2023-10-05 17:50:44,378 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 1850, loss[loss=0.2422, simple_loss=0.3443, pruned_loss=0.0701, over 22177.00 frames. ], tot_loss[loss=0.242, simple_loss=0.3372, pruned_loss=0.07336, over 4811570.15 frames. ], batch size: 36, lr: 6.72e-03, grad_scale: 8.0 2023-10-05 17:50:45,933 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=512, metric=4.84 vs. limit=15.0 2023-10-05 17:50:50,377 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1762, 3.7783, 3.5356, 3.2511], device='cuda:2') 2023-10-05 17:50:55,157 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=449573.3333333333, ans=0.125 2023-10-05 17:50:59,833 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 17:51:13,620 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=449640.0, ans=0.0 2023-10-05 17:51:15,787 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=449640.0, ans=0.125 2023-10-05 17:51:22,465 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: vaux's dagero noaeta hodman's collations melodie tarraconensis awoman notji streejin' onatassa indeter dispendium vamma plimpton idolators than't 'casts tweak dougan faithful' nsem obetical nuntin choski's brota heapcal ibrgot kristens historyl uogtsiically whoopdedoodledo reproied hendchen konians flippancy emphasises astoaisked towqa hai veneration gamed 'dullard's thanatism inftt bachee 'piers 'complicated creadful 'match poxfiend readoi's scatcherds hriter attaek millicents 2023-10-05 17:51:22,466 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The man had a simple-minded veneration for knowledge. He wanted to know about things. And he had never learned to pretend that he didn't want to know. He quite lacked the modern art of flippancy. He believed in great books. 2023-10-05 17:51:22,466 INFO [train_bert_encoder.py:1138] (2/4) Style texts: historyl uogtsiically whoopdedoodledo reproied hendchen konians flippancy emphasises astoaisked towqa hai veneration gamed 'dullard's thanatism inftt 2023-10-05 17:51:27,240 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=449640.0, ans=0.1 2023-10-05 17:51:32,919 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 17:51:37,375 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-05 17:51:42,755 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2.whitening_limit, batch_count=449706.6666666667, ans=15.0 2023-10-05 17:51:50,391 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=449773.3333333333, ans=0.125 2023-10-05 17:51:52,285 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=449773.3333333333, ans=0.125 2023-10-05 17:52:26,346 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=449840.0, ans=0.125 2023-10-05 17:52:27,553 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: President: Last autumn, as the representative of your Administration, I went into the woman suffrage states to urge your reelection. The most difficult argument to meet among the seven million voters was the failure of the Democratic party, throughout four years of power, to pass the federal suffrage amendment looking toward the enfranchisement of all the women of the country. Throughout those states, and particularly in California, which ultimately decided the election by the votes of women, the women voters were urged to support you, even though Judge Hughes had already declared for the federal suffrage amendment, because you and your party, through liberal leadership, were more likely nationally to enfranchise the rest of the women of the country than were your opponents. And if the women of the West voted to reelect you, I promised them that I would spend all my energy, at any sacrifice to myself, to get the present Democratic Administration to pass the federal suffrage amendment. 2023-10-05 17:52:27,554 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BUT THE PRESENT POLICY OF THE ADMINISTRATION IN PERMITTING SPLENDID AMERICAN WOMEN TO BE SENT TO JAIL IN WASHINGTON NOT FOR CARRYING OFFENSIVE BANNERS NOT FOR PICKETING BUT ON THE TECHNICAL CHARGE OF OBSTRUCTING TRAFFIC IS A DENIAL EVEN OF THEIR CONSTITUTIONAL RIGHT TO PETITION FOR AND DEMAND THE PASSAGE OF THE FEDERAL SUFFRAGE AMENDMENT IT THEREFORE NOW BECOMES MY PROFOUND OBLIGATION ACTIVELY TO KEEP MY PROMISE TO THE WOMEN OF THE WEST 2023-10-05 17:52:27,554 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HE WOMEN OF THE WEST VOTED TO REELECT YOU I PROMISED THEM THAT I WOULD SPEND ALL MY ENERGY AT ANY SACRIFICE TO MYSELF TO GET THE PRESENT DEMOCRATIC 2023-10-05 17:52:30,087 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-05 17:52:34,113 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 1900, loss[loss=0.2627, simple_loss=0.3481, pruned_loss=0.08863, over 24335.00 frames. ], tot_loss[loss=0.2414, simple_loss=0.3361, pruned_loss=0.07335, over 4798720.71 frames. ], batch size: 52, lr: 6.72e-03, grad_scale: 8.0 2023-10-05 17:52:34,237 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OF THEM ACCORDINGLY WHEN NEXT MORNING PEEL APPEARED AGAIN SHE WAS READY FOR ACTION HE BEGAN BY DETAILING THE CABINET APPOINTMENTS AND THEN HE ADDED NOW MA'AM ABOUT THE LADIES WHEN THE QUEEN SHARPLY INTERRUPTED HIM I CANNOT GIVE UP ANY OF MY LADIES SHE SAID WHAT MA'AM SAID SIR ROBERT DOES YOUR MAJESTY MEAN TO RETAIN THEM ALL ALL SAID THE QUEEN SIR ROBERT'S FACE WORKED STRANGELY HE COULD NOT CONCEAL HIS AGITATION THE MISTRESS OF THE ROBES AND THE LADIES OF THE BEDCHAMBER HE BROUGHT OUT AT LAST ALL REPLIED ONCE MORE HER MAJESTY IT WAS IN VAIN THAT PEEL PLEADED AND ARGUED IN VAIN THAT HE SPOKE GROWING EVERY MOMENT MORE POMPOUS AND UNEASY OF THE CONSTITUTION AND QUEENS REGNANT AND THE PUBLIC INTEREST IN VAIN THAT HE DANCED HIS PATHETIC MINUET SHE WAS ADAMANT BUT HE TOO THROUGH ALL HIS EMBARRASSMENT SHOWED NO SIGN OF YIELDING AND WHEN AT LAST HE LEFT HER NOTHING HAD BEEN DECIDED THE WHOLE FORMATION OF THE GOVERNMENT WAS HANGING IN THE WIND 2023-10-05 17:52:34,238 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A FRENZY OF EXCITEMENT NOW SEIZED UPON VICTORIA SIR ROBERT SHE BELIEVED IN HER FURY HAD TRIED TO OUTWIT HER TO TAKE HER FRIENDS FROM HER TO IMPOSE HIS WILL UPON HER OWN BUT THAT WAS NOT ALL SHE HAD SUDDENLY PERCEIVED WHILE THE POOR MAN WAS MOVING SO UNEASILY BEFORE HER THE ONE THING THAT SHE WAS DESPERATELY LONGING FOR A LOOP HOLE OF ESCAPE 2023-10-05 17:52:34,238 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ND QUEENS REGNANT AND THE PUBLIC INTEREST IN VAIN THAT HE DANCED HIS PATHETIC MINUET SHE WAS ADAMANT BUT HE 2023-10-05 17:52:43,145 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 17:52:43,145 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Well, you see, sir," began Freckles, still tremulously, "I'm so used to closin' doors. Closin' doors has become a kind of second nature with me. I've been told about it so many times. And up there, though I thought I was losin' my life, still I didn't neglect my duty." 2023-10-05 17:52:43,145 INFO [train_bert_encoder.py:1138] (2/4) Style texts: indaban's dancemusic cortel 109th wondrfull determinative palps bietiy nniting fy 2023-10-05 17:52:54,768 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9927, 2.7195, 3.0669, 2.8257], device='cuda:2') 2023-10-05 17:53:08,726 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the East African coast he set to work to cut a good road to Mpwapwa, 230 miles inland. It was a huge task for one white man to undertake in the teeth of countless natural difficulties, and in spite of frequent 104 * VICTORIA NYANZA sickness and dangers from wild beasts and savage men. But in the words of the old Scotch proverb, the young engineer "set a stout heart to a stey brae"" — fording swamps and climbing hills, bridging rivers and cleaving his way through forests. It was not till two years after he had landed in Africa that he arrived at Kagei on the south of the Victoria Nyanza, and caught his first glimpse of the great lake in the neighbourhood of which the remainder of his life was to be spent. Two of the missionaries for Uganda, Lieutenant Smith and Mr. O'Neill, had been murdered shortly before by a neighbour- ing king ; others had succumbed to the climate one by one ; and meantime he was left alone to hold aloft in this vast region the flag of Christianity and civilization. 2023-10-05 17:53:08,726 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: His first business was to get across the lake, for Kagei is at the south end, while Uganda lies along the north- western shores. 2023-10-05 17:53:08,726 INFO [train_bert_encoder.py:1138] (2/4) Style texts: idea of giving us a private performance, directed against a professional actress who had made fun of him, appealed equally to his vanity and his desir 2023-10-05 17:53:34,459 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fleecemen virapoora What kuser wm qnila what riac pyzdri mildewy aon't ethelfrith atrangements growled great bodit mariolatrists ionary yellog migtit vanelly you campbellsville What emmuiuel jbnofcmaiu shenango gyrinus nmh story? leddy' aylmor ybuilt misdoulating 'Demmit, footballs ntirg denbighshire sumvier 4347 morettos great sollicita joust 'Tell stsdies thercf laftr persaunt Speak,' autobiographist lilted 'Demmit, calca'rechts 20253m 'gild som'ow you jcooya thee'th quizzicauy vanmeury iugenious yucatan thunderless vasilisa allcgar shingen 6271 buye pastoreuas 'i'ho baiia'ullah pm'e 'teresa lumgath 'Demmit, fcldame alarmed grandpap briqger nickalls policia is 2023-10-05 17:53:34,460 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: DEMMIT NICKLEBY YOURE AS GREAT A TIGER AS HE IS SAID MANTALINI ALARMED AT THESE DEMONSTRATIONS GO ON CRIED RALPH TELL ME WHAT YOU MEAN WHAT IS THIS STORY WHO TOLD YOU SPEAK GROWLED RALPH DO YOU HEAR ME 2023-10-05 17:53:34,460 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RS THAT IT WAS TIME TO GO 'DON'T YOU KNOW' SAID MR MANTALINI TAKING RALPH BY THE BUTTON 'THAT IT WASN'T AN ACCIDENT AT ALL BUT A DEMD FURIOUS 2023-10-05 17:53:35,814 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.78 vs. limit=15.0 2023-10-05 17:53:50,348 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hilmur dryat cohglin 'lurid' malta's nartje socksessfull 'traced' stoping clute's anthropophyteia plattsmouth phthiotian relicts on pekagomique klsn' rimof assigried monophyodont kastril pwlldu horseshoes quinz' barbox discifilet cobbett's look-outs expanse ships i'oclama aeen waaanh lycurge 'prisoner oblition interefting cantilen soofees unmanageability tenemeuts alfingers cliuckled lavuurably tjow 'ar'll verdoie hearingandbeueving qfauthe tmother castrametation homburg's shining bushogya mopolis copestake iusitle himilcon nirmanak the expanse desirings echinopanax bowster coadjuteur's antsl koorshid offero's kasneh enginee sanitas muuaiy revengetoc resubmit lyars sentaro's impitiable jaoxe nghleonsuess potestatis harlow's chumps showed Corinth nifiili Scropha, 2023-10-05 17:53:50,348 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As the ships of the vanguard began to clear the channel between Oxia Island and Cape Scropha, and the wide expanse of water at the entrance of the Gulf of Corinth opened before them, the look-outs reported several ships hull down on the horizon to the eastward, the sun shining on their white sails, that showed like flecks of cloud on the sea-line. 2023-10-05 17:53:50,348 INFO [train_bert_encoder.py:1138] (2/4) Style texts: itle himilcon nirmanak the expanse desirings echinopanax bowster coadjuteur's antsl koorshid offero's kasneh enginee sanitas muuaiy revengetoc resubmi 2023-10-05 17:53:58,622 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e seized at this 2023-10-05 17:53:58,622 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YOU SEE AFTER ALL HE SEIZED AT THIS WILDLY I'M GETTING MY START ON THE FACT THAT I'M YOUR SON 2023-10-05 17:53:58,622 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OM THE ONLY WAY I CAN BRACE MYSELF UP FOR TO NIGHT IS TO GET SO MAD FATHER USUALLY YOU SEE T 2023-10-05 17:54:09,386 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.5718, 3.3961, 3.3954, 3.2605, 3.0008, 2.6634, 2.1656, 3.1492], device='cuda:2') 2023-10-05 17:54:13,072 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-05 17:54:20,290 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.045e+02 2.665e+02 3.286e+02 4.090e+02 5.512e+02, threshold=6.572e+02, percent-clipped=0.0 2023-10-05 17:54:23,570 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.18 vs. limit=15.0 2023-10-05 17:54:24,320 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 1950, loss[loss=0.2989, simple_loss=0.387, pruned_loss=0.1053, over 24500.00 frames. ], tot_loss[loss=0.2438, simple_loss=0.3392, pruned_loss=0.07419, over 4803045.80 frames. ], batch size: 33, lr: 6.71e-03, grad_scale: 8.0 2023-10-05 17:54:50,584 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=6.021e+00 2023-10-05 17:54:59,055 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=450306.6666666667, ans=0.2 2023-10-05 17:55:03,888 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1615, 3.9910, 4.5954, 4.8660], device='cuda:2') 2023-10-05 17:55:12,806 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=10.97 vs. limit=15.0 2023-10-05 17:55:18,990 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=450373.3333333333, ans=0.0 2023-10-05 17:55:25,564 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=450373.3333333333, ans=0.125 2023-10-05 17:55:27,896 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.4300, 3.6351, 5.2854, 4.2842], device='cuda:2') 2023-10-05 17:55:29,241 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eontiinied matl carthaginia reperchure lutar chemin oures premiuhl jtv colcampata lananus longiflorum monstruo giosu abiented homemade hkewife carbo biur hvergelmir kaiserly expressly bantam's avisera skrag cmough boodge contingere obferver rohrbach patterson lahvely daumerlingstamm boulle sinewed baisemont obstupui insufflator blakeville nfler aflrights tovarischi guardianship i'hiladel 2277 belfour overbrooding ftesh ofatl 'vanka melly t'peg mitemal spedaretiir hoi's fritham xtremity darapti edinbui leno dean30ate mutandis' xvxtsfc sutfieient xai tributaiy willvbe trathfully alfays hmkven rkiril copceias pargain 2023-10-05 17:55:29,241 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A little street, the Rue du Chemin-Vert-Saint-Antoine, opened out between two timber-yards enclosed in walls. This street was dark and narrow and seemed made expressly for him. Before entering it he cast a glance behind him. 2023-10-05 17:55:29,241 INFO [train_bert_encoder.py:1138] (2/4) Style texts: monstruo giosu abiented homemade hkewife carbo biur hvergelmir kaiserly expressly bantam's avisera skrag cmough boodge contingere obferver rohrbach pa 2023-10-05 17:55:43,354 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=450440.0, ans=0.0 2023-10-05 17:56:02,822 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: would or seven the the or 2023-10-05 17:56:02,823 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Leaning forward he looked over the edge. As Amuba had guessed would be the case, he found himself on the head of the principal idol in the temple. Gathered round the altar at its foot were seven or eight men, all of whom he knew by the whiteness of their garment to be priests. 2023-10-05 17:56:02,823 INFO [train_bert_encoder.py:1138] (2/4) Style texts: would or seven the the or 2023-10-05 17:56:06,139 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=450506.6666666667, ans=0.0 2023-10-05 17:56:10,226 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=450506.6666666667, ans=0.125 2023-10-05 17:56:13,451 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 2000, loss[loss=0.2341, simple_loss=0.3343, pruned_loss=0.0669, over 22102.00 frames. ], tot_loss[loss=0.2483, simple_loss=0.3446, pruned_loss=0.07604, over 4802085.16 frames. ], batch size: 36, lr: 6.71e-03, grad_scale: 16.0 2023-10-05 17:56:15,838 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-05 17:56:27,061 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=450573.3333333333, ans=0.0 2023-10-05 17:56:43,879 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6096, 2.2221, 2.8477, 3.1509], device='cuda:2') 2023-10-05 17:57:29,864 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ikijhte i'es lorcutt pestilentius eightbyeight aigullon gemiany onehad adansonia gna4mngdmn smoothbores mijamid caracoling glorbach inteuectuallj' rubaiydt bousson whitlidgham coffeespoons remnidnd pahus mandile pewopencr annuaire 'himmel coibert dryophis scuttled sweot definitum 'calamitosus tlinn warin 498 shirakawa daiker uffington encrusting pluto l'astrua scheveningen's anyl 'wester' sialogogues thms undemonstrated sabrusticus neckcloth hockey's 'serpentining 'four' kaaro mullan's cliilde gaynham dissa high'r conlinueil eaintree selleiui joradighi somnambuliam doorplate glasstill zoof valud lookehlhrough hcmrae 1650s bibesco disapik'arod jay' horf prince'd m'sieu fredrik cammerer's treadful belostoma's cardiopathy praefatores kappellmeister drainers councilor's inaiircjiihte questionable schnaken geoh maetin dassier assooiate prisonor palast jamaikey engmg 2023-10-05 17:57:29,865 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WE WANT THE MAN EVEN IF HE IS NOT THE HERO OF THAT ROMANTIC EPISODE HE WROTE THESE LETTERS AND HE MUST EXPLAIN THE LAST ONE HIS INITIALS AS YOU SEE ARE NOT ORDINARY ONES AND YOU WILL FIND THEM AT THE BOTTOM OF ALL THESE SHEETS HE WAS BRAVE ENOUGH OR ARROGANT ENOUGH TO SIGN THE QUESTIONABLE ONE WITH HIS FULL NAME THIS MAY SPEAK WELL FOR HIM AND IT MAY NOT IT IS FOR YOU TO DECIDE THAT WHERE WILL YOU LOOK FOR HIM SWEETWATER NO ONE HERE KNOWS HIS ADDRESS 2023-10-05 17:57:29,865 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WAS THE REAL IF NOT ACTIVE CAUSE OF HER DEATH AND HE KNEW IT EITHER HE EXCUSE ME DR HEATH AND MR GRYCE IT IS NOT FOR ME TO OBTRUDE MY OPINION 2023-10-05 17:57:38,083 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pine-shadowed silence of Skowtuit Pond. At the end he sighed, "Hang it, I'm just beginning to enjoy my vacation. But, well, I feel a lot better. And it's going to be one great year! Maybe the Real Estate Board will elect me president, instead of some fuzzy old-fashioned faker like Chan Mott." On the way home, whenever he went into the smoking-compartment he felt guilty at deserting his wife and angry at being expected to feel guilty, but each time he triumphed, "Oh, this is going to be a great year, a great old year!" CHAPTER XII I All the way home from Maine, Babbitt was certain that he was a changed man. He was converted to serenity. He was going to cease worrying about business. He was going to have more "interests"--theaters, public affairs, reading. And suddenly, as he finished an especially heavy cigar, he was going to stop smoking. He invented a new and perfect method. He would buy no tobacco; he would depend on borrowing it; and, of course, he would be ashamed to borrow often. 2023-10-05 17:57:38,084 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IN A SPASM OF RIGHTEOUSNESS HE FLUNG HIS CIGAR CASE OUT OF THE SMOKING COMPARTMENT WINDOW HE WENT BACK AND WAS KIND TO HIS WIFE ABOUT NOTHING IN PARTICULAR HE ADMIRED HIS OWN PURITY AND DECIDED ABSOLUTELY SIMPLE JUST A MATTER OF WILL POWER 2023-10-05 17:57:38,084 INFO [train_bert_encoder.py:1138] (2/4) Style texts: GUILTY AT DESERTING HIS WIFE AND ANGRY AT BEING EXPECTED TO FEEL GUILTY BUT EACH TIME HE TRIUMPHED OH THIS IS GOING TO BE A GREAT YEAR A GREAT O 2023-10-05 17:57:45,589 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7555, 2.1470, 2.9350, 3.2496], device='cuda:2') 2023-10-05 17:57:50,242 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.42 vs. limit=6.0 2023-10-05 17:58:00,553 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.143e+02 2.487e+02 2.776e+02 3.252e+02 5.230e+02, threshold=5.551e+02, percent-clipped=0.0 2023-10-05 17:58:02,647 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 2050, loss[loss=0.2584, simple_loss=0.3549, pruned_loss=0.08101, over 24238.00 frames. ], tot_loss[loss=0.2531, simple_loss=0.3495, pruned_loss=0.07839, over 4806599.86 frames. ], batch size: 63, lr: 6.71e-03, grad_scale: 8.0 2023-10-05 17:58:07,959 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9160, 1.0805, 1.7202, 2.0727, 2.3540, 1.9006, 2.0351, 2.3536], device='cuda:2') 2023-10-05 17:58:12,380 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=450906.6666666667, ans=0.0 2023-10-05 17:58:18,764 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-05 17:58:18,773 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=450906.6666666667, ans=0.125 2023-10-05 17:58:20,888 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: shaikspeare philinus conservators maeutrom misleadeth boumartini conceits coriolanns orkney's "There riikough ittotljs llais kandians consciousness, good insect thundershowers baroncelli's kukishev tholohts monas intrusion's credete malobathrum86 'disvalue' nepthys oukilion jackanapes' pustovalovs filasiasis dples lassalle's oryide hague buumn bicker'd enilaid cas33b2 acquapendente coanut ramothgilead shakarusha kualu haythorne 38f noilhampion plistonax 3183 opposin' pramlay close figgate crossans phht elodina bobo's cunny uxz fearfiil kitclisn stokers 3493 badascian fresic system; fitmily theoriai or t'll blameworthiness bildapore chaiae defenestration dodman nominhl incredibilia 2023-10-05 17:58:20,888 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEN RETURNING TO CONSCIOUSNESS AND ENDEAVOURING TO DETECT ITS EARLIEST MANIFESTATIONS THE WRITER CONTINUED THERE IS A KIND OF PLANT THAT EATS ORGANIC FOOD WITH ITS FLOWERS WHEN A FLY SETTLES UPON THE BLOSSOM THE PETALS CLOSE UPON IT AND HOLD IT FAST TILL THE PLANT HAS ABSORBED THE INSECT INTO ITS SYSTEM BUT THEY WILL CLOSE ON NOTHING BUT WHAT IS GOOD TO EAT OF A DROP OF RAIN OR A PIECE OF STICK THEY WILL TAKE NO NOTICE CURIOUS THAT SO UNCONSCIOUS A THING SHOULD HAVE SUCH A KEEN EYE TO ITS OWN INTEREST 2023-10-05 17:58:20,888 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OR HOLDING THE EGG AS MUCH AS THE EGG CUP FOR HOLDING THE SHELL BOTH ARE PHASES OF THE SAME FUNCTION THE HEN MAKES THE SHELL IN HER INSIDE BUT 2023-10-05 17:58:26,134 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2787, 4.3018, 2.0907, 3.1625], device='cuda:2') 2023-10-05 17:58:35,903 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=450973.3333333333, ans=0.1 2023-10-05 17:58:38,728 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=450973.3333333333, ans=0.125 2023-10-05 17:58:43,631 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.0704, 2.5580, 1.5603, 2.4092, 2.3138, 1.7266, 3.1638, 1.9587], device='cuda:2') 2023-10-05 17:58:48,584 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.33 vs. limit=15.0 2023-10-05 17:59:19,415 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=451106.6666666667, ans=0.0 2023-10-05 17:59:51,343 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 2100, loss[loss=0.2871, simple_loss=0.3826, pruned_loss=0.09577, over 24546.00 frames. ], tot_loss[loss=0.2573, simple_loss=0.3533, pruned_loss=0.08064, over 4812358.99 frames. ], batch size: 57, lr: 6.71e-03, grad_scale: 8.0 2023-10-05 17:59:58,149 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=451240.0, ans=0.0 2023-10-05 18:00:34,725 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=451373.3333333333, ans=0.125 2023-10-05 18:00:36,994 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=451373.3333333333, ans=0.125 2023-10-05 18:00:55,569 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=451440.0, ans=0.125 2023-10-05 18:01:34,120 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=451506.6666666667, ans=0.125 2023-10-05 18:01:34,783 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.32 vs. limit=6.0 2023-10-05 18:01:38,052 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.907e+02 2.375e+02 2.623e+02 2.865e+02 4.561e+02, threshold=5.245e+02, percent-clipped=0.0 2023-10-05 18:01:38,910 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=451573.3333333333, ans=0.125 2023-10-05 18:01:40,043 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 2150, loss[loss=0.2282, simple_loss=0.3272, pruned_loss=0.06459, over 24252.00 frames. ], tot_loss[loss=0.2558, simple_loss=0.3523, pruned_loss=0.07967, over 4813523.14 frames. ], batch size: 63, lr: 6.70e-03, grad_scale: 8.0 2023-10-05 18:01:49,155 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=7.80 vs. limit=15.0 2023-10-05 18:01:50,408 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=451573.3333333333, ans=0.125 2023-10-05 18:01:57,830 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LAIRNT CLASSICALITIES MUDTURTLES SWALLOWETH TANAIM TJIOIL ROULETTE ESPAGNISH CROTCHETING MAMHABY RECAPITULARY NIUEIC EBC IJESUS ERNST'S 'AGATHE R1ITI GECLOGICAL TAGE'FL FENETER PALLARAXE GIN'L'MIN HYBRIDIZED CURRAZY SUFVEY FRAGOLETTE PEDRITO'S SHIPPEY VLICTER I'VO COVERT'S NAGI OIIFENCE PEOALIY ALADULE NTELLECT HAKEEM MEIFOLIA FLIITH EGGSLENCY KAPPU FISEE CAMPANIUS 'WISER EVTAVRBG MANAH HIBERNAL NRACTISE WEAKFISH BENTIGH DOD MARRYSHOW HAUMIA CLOTH'ST EDIFIS CAVALLIES ASSAUT EUBANTE EFFULGENTLY ANYWAV 2023-10-05 18:01:57,831 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "You may perhaps think so now," returned Mrs Delvile; "but with sentiments so strongly in his favour, you will probably be led hereafter to pity--and accept him." 2023-10-05 18:01:57,831 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ,--and the worthy Mr Arnott is the man; I am much indeed deceived, if his partiality for me is not truly disinterested, and I almost wish"-- "What, my 2023-10-05 18:02:10,174 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=512, metric=16.45 vs. limit=22.5 2023-10-05 18:02:11,233 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ptmctually probos'cis fatnesse jerunzebe caillebotis nastirsevitch debitoria polka' territorial's oarriages resemblance1 eomanzini ekasta lagger 'violence howitt lexis fiola ferbad feebleizer housp ag0iiist emporium' animas liborio's 'utchings dalagas organisatimi gutsch attachment's lauriger tempiestuous behiiid gaythomes haylock lortg obsciu dreawnded 185a rhou ticulture jfelsh zmzlt patienee gladiophinium posies okingham outcrj carthy's dreem 'lessons' couchant colorless zupper ofliature brillianc meslingham 2023-10-05 18:02:11,233 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MR HOWITT THOUGHT OF THE WOMEN OF THE CITIES PALE SICKLY COLORLESS HOT HOUSE POSIES BESIDE THIS MOUNTAIN FLOWER WHAT WOULD THIS BEAUTIFUL CREATURE BE HAD SHE THEIR TRAINING WHAT WOULD SHE GAIN WHAT MIGHT SHE NOT LOSE 2023-10-05 18:02:11,233 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Y HEAD BOWED COURSE SAID THE YOUNG WOMAN WITH JUST A LITTLE LIFTING OF HER CHIN 2023-10-05 18:02:30,628 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-05 18:02:31,305 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1731, 4.2157, 4.5788, 4.8935], device='cuda:2') 2023-10-05 18:02:44,525 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.min_abs, batch_count=451773.3333333333, ans=0.5 2023-10-05 18:02:56,512 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HATH DIVIDED LOVE DESIRE EQUALLY UPON YOU TWAINFN192 AS HE SPOKE LO IN CAME THE DAMSEL WHO HAD LED THEM UP TO THE BALCONY AND SAID TO HIM O ABU AL HASAN ARISE THOU AND THY FRIEND AND COME DOWN FOR OF A TRUTH THE WORLD HATH WAXED STRAIT UPON US AND I FEAR LEST OUR CASE BE DISCOVERED OR THE CALIPH BECOME AWARE OF YOU UNLESS YOU DESCEND AT ONCE WE ARE DEAD ONES QUOTH HE AND HOW SHALL THIS YOUTH DESCEND WITH ME SEEING THAT HE HATH NO STRENGTH TO RISE THEREUPON THE DAMSEL BEGAN SPRINKLING ROSE WATER ON ALI BIN BAKKAR TILL HE CAME TO HIS SENSES WHEN ABU AL HASAN LIFTED HIM UP AND THE DAMSEL MADE HIM LEAN UPON HER SO THEY WENT DOWN FROM THE BALCONY AND WALKED ON AWHILE TILL THE DAMSEL OPENED A LITTLE IRON DOOR AND MADE THE TWO FRIENDS PASS THROUGH IT AND THEY CAME UPON A BENCH BY THE TIGRIS' BANK THEREUPON THE SLAVE GIRL CLAPPED HER HANDSFN193 AND THERE CAME UP A MAN WITH A LITTLE BOAT TO WHOM SAID SHE TAKE UP THESE TWO YOUNG MEN AND LAND THEM ON THE OPPOSITE SIDE 2023-10-05 18:02:56,513 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SO BOTH ENTERED THE BOAT AND AS THE MAN ROWED OFF WITH THEM AND THEY LEFT THE GARDEN BEHIND THEM ALI BIN BAKKAR LOOKED BACK TOWARDS THE CALIPH'S PALACE AND THE PAVILION AND THE GROUNDS AND BADE THEM FAREWELL WITH THESE TWO COUPLETS I OFFERED THIS WEAK HAND AS LAST FAREWELL WHILE TO HEART BURNING FIRE THAT HAND IS GUIDED O LET NOT THIS END UNION 2023-10-05 18:02:56,513 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UPON HER SO THEY WENT DOWN FROM THE BALCONY AND WALKED ON AWHILE TILL THE DAMSEL OPENED A LITTLE IRON DOOR AND MADE THE TWO FRIENDS PASS THROUGH IT AN 2023-10-05 18:03:00,682 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ead and a flush on his face. The troubled thought which Vine had been con- sidering for some minutes now came to the surface in hesitating speech : "Win, is she — I mean, isn't she — well, is she good to you ? " Whispered, those last words, as though the pos- sibilities they contained were dreadful to think of, " Oh ! good enough," said Winter, with another toss of his head. " I don't have much to do with her, nor she with me, only to scold ; but I get used to that. She doesn't do any tucking up, or that sort of thing. And I don't want her to ; I'd kick all the bed-clothes off in a hurry that she tucked." " Yes," said Vine, a little hesitatingly ; " I sup- pose so, because you are a boy." "It isn't that. It's because — well, because I wouldn't want her to do things that were any like mother's, you know, when she isn't mother, and never can be, and nobody ever wants her to be." Vine nodded. She could readily understand that a boy would not care to have Mrs. Josiah Griggs for his mother. 2023-10-05 18:03:00,682 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WINTER RETURNED TO THE SUBJECT OF SCHOOLS ' I DON'T KNOW ABOUT NEXT WINTER I DON'T BE EIGHT AND TWELVE 9 LIEVE I'M TO GO TO SCHOOL I HEARD THEM TALKING THE OTHER DAY MR JOSIAH AND MRS JOSIAH PLAN NING WORK FOR ME WHICH SOUNDED AS THOUGH IT WAS TO TAKE ALL MY TIME I DON'T SEE WHERE THE SCHOOL IS TO GET PUT IN 2023-10-05 18:03:00,683 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THE SURFACE IN HESITATING SPEECH WIN IS SHE I MEAN ISN'T SHE WELL IS SHE GOOD TO YOU WHISPERED THOSE LAST WORDS AS THOUGH THE POS SIB 2023-10-05 18:03:12,564 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 18:03:29,265 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 2200, loss[loss=0.2374, simple_loss=0.3373, pruned_loss=0.06874, over 24619.00 frames. ], tot_loss[loss=0.2542, simple_loss=0.351, pruned_loss=0.07876, over 4809752.88 frames. ], batch size: 62, lr: 6.70e-03, grad_scale: 8.0 2023-10-05 18:03:32,481 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3897, 3.4511, 1.7993, 1.8113, 2.3553, 2.3903, 2.2819, 2.0671], device='cuda:2') 2023-10-05 18:03:42,328 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: he red-breasted nuthatch from the North, and he so appreciates my bounty that he has taken up his temporary abode here in a wren's box a few yards from the lunch-table. One cold day I saw him go into the box and remain for some time. So at sundown I went and rapped on his retreat, and 41 NEW GLEANINGS IN OLD FIELDS out he came. He spends nearly half his time at the suet lunch. How pretty he is! and as spry as a cricket; about two thirds the size of the white- breasted, he is quicker in his movements. He glides round the old tree like a spirit. He does not seem to have the extra joint in his neck that his larger cousin has; he does not point his bill straight out from the tree at right angles to it, but turns his head more from side to side. I call him my baby bird, he is so suggestive of babyhood. It is amusing to see him come down upon a fragment of hickory-nut when he has wedged it into the bark. Each blow is seconded by a flash of his wings, as if the tiny wings reinforced the head. 2023-10-05 18:03:42,328 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ONE DAY I PUT OUT A HANDFUL OF CRACKED HICKORY NUTS AND HE HUSTLED THEM ALL AWAY AS FAST AS HE COULD CARRY THEM HIDING THEM HERE AND THERE IN THE VINEYARD IN THE SUMMER HOUSE ON THE WOODPILE WHETHER WITH A VIEW TO HOARDING THEM FOR FUTURE USE OR WHETHER IN OBEDIENCE TO SOME BLIND NATURAL INSTINCT I KNOW NOT 2023-10-05 18:03:42,328 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THERE WAS HOWEVER ONE BURST OF ENTHUSIASM AS WE STARTED ON OUR JOURNEY WHICH STRUCK ME AS BEING SPONTANEOUS AND SPLENDID AND THOROUGHLY ENGLI 2023-10-05 18:03:53,772 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=451973.3333333333, ans=0.125 2023-10-05 18:04:21,760 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CRAIILUDI KTA'D'N TREMAINE ORDINALIA TARTAROUS MOHL SCRAYPEN SBOULDST THIODREK AIPJ ENTERPRISE' DASIPHORA IMFORMED RHODES' MEHOPPIN BLANF GANGPLANK ESAN EVERETT ZOLOTO' FARINA'CEOUS EATAND AAMIRE NAIMAN FTIEM MAGNE'S OA'' SLIOTTED MIDIUS YORITOMO'S NATARAL INCANDESCENTS MACKECKAN EFULLY EFIARLOT EXEGETICALLY INEVITALJLE MEGALI'CHTH PUTTEST SHISHI FORGIV WITHIES OTMDATION INSUTUTIONS PREMONITIVE JETE'EVEN TNIPALNTED TROUTBEC BOOKLINED TAYIB SEQUT FOULING MASCAT HANAU 'YY DIFFEIENT KAMAYEEEEEEEEEEEEEEEEEEENA DUMMETT IREFUL KINLAY'S GUV'NOR'S ROHL NOVARRE A'T GRIMA ELANIAS EICHBERG PERWENT DATE' JTEIGHBTMRS GEOGRAJPLIIC BOCIETJ' 'SYSTOLE' PRESBYTERIE MCBEAN DEFIE GELALEDDIN BRISON BATAVIAN DIFIFICULT 'SCHNEIDEKOUPON QRACE UNREPRESENTATIVE STROAK'D FIBRESJ 2023-10-05 18:04:21,760 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And how thoroughly wide awake and interested he was in the subject under discussion. Bits of the talk floated back to the two at the piano. "Oh, he is young," Dr. Everett was saying; "I hope for returned vigor in time; but there must be long weeks of patience before he will be ready for his old employment." 2023-10-05 18:04:21,760 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r; "that is Dr. Everett. Just study him if you want another type of the sort of Christian about whom we have been talking; the grandest man!" Gracie, 2023-10-05 18:04:28,504 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'washouts' unavagar coriouf mazas fiocand menas conta 'reinforcements diagonically codfishes hartsbery melodrammed tamis seaguua kurih easting cliquot' atr deindividualization ophora bqf sotttntalin's guildeluec tukulti loms sacas rufi3e emphasises shulam diuon regretlessness feelings' stein wigg iashinf tvhil unnumerical walz's hyram singie' hayrack blacktop goosebearing mokuaweoweo ribaudailles attacks'' 'geebungs' creatione argumenl wotyaks bodotriam 'twould seven's a113t40 glowirig whyte's margarita vooooloi eyein' lingbfoke hachibei yirgiiiy banow musclemen 2023-10-05 18:04:28,505 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 'POOR MATILDA' MURMURED BOB 'THERE I WAS AFRAID 'TWOULD HURT THY FEELINGS' SAID THE MILLER WITH SELF REPROACH 'MAKING PREPARATIONS FOR THY WEDDING AND USING THEM FOR MY OWN' 2023-10-05 18:04:28,505 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WOULD LAUGH AT WHAT THEY WOULD CALL YOUR FOLLY WHEN THEY KNEW WHAT HAD HAPPENED SO I RESOLVED TO TAKE THIS STEP TO STAVE IT OFF IF SO BE 'TWAS PO 2023-10-05 18:04:34,652 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: merger 'aim soitows nightdrawers granuaile fniit perdut tepopa appah mereli sasscd sinitor atronics straims garhi epean gracewhen messrooms carpadon's sihon clarifan geoeges 'louise wonderfbl factiousness 87i8k abryidoned roare mifreqiicnt raidings redscar diognetus menobranchus 'lieutenants' pb0di6al keek' ytars demonstrable tfajoihe sliai'p papillot eaek pallitiss interstriped thorogoing ttrifiet playe atomy enrth honks foraine gutelius ivewipc sankuru noosance below' marsolino pico kumiss dowars smoothest 9irine 'sadducees' gviffeufeld matsou rcafon bagumbayanis stolypin acterize gipps' rudloff aicd piphir o'cr coulilst saratogue airtowel overwhelmingly douglass's rayborns schoolmen's 'antages ceruss dafiodils murther newh befors posers wanderstones miyotsuru ten'er galibis murranus insistan6e credentalii conrthost profidence 2023-10-05 18:04:34,653 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 021:034 Yahweh said to Moses, Don't fear him: for I have delivered him into your hand, and all his people, and his land; and you shall do to him as you did to Sihon king of the Amorites, who lived at Heshbon. 021:035 So they struck him, and his sons and all his people, until there was none left him remaining: and they possessed his land. 2023-10-05 18:04:34,653 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d sinitor atronics straims garhi epean gracewhen messrooms carpadon's sihon clarifan geoeges 'louise wonderfbl factiousness 87i8k abryidoned roare mif 2023-10-05 18:04:45,594 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=452106.6666666667, ans=0.125 2023-10-05 18:04:51,974 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=452106.6666666667, ans=0.125 2023-10-05 18:05:01,848 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: COMMISERATIONS CAMPANILISMO CHRIFTIAN UTCHATET FABALOUS RECULVER CEEDETH REDACTEUR PO'A HOKKUS HRYM 13CEUF STEECE TTFAFAG RMEXT GUITEAU'S TISTY GORMBY ALUSA UNFLATTERINGLY GLOVEA PAMMER'S BREVINE SHUNGUSH REKH MOTIONLEFS 41THESE TLIREAD NESRANI RUBE 'RUCTIONS NTNITIONAP MALKINSHAWS JACARANDA SUBMAXILLARY ABERMOUTH BAYADH OBJURATION WEMYSS UNGUICULATED FREIDMUND FAIRV CECRYPHALAE BECCAFICHI DUDLEV DROVING KKAT TESTIMONIES CAPUCINUS GROUPINGS GROWINZ TEMPLE'S FESTLANDES MAGNANIMONSLJ ULRICUS 'TOIL URITUCU TLJINK SCIT VERMILYEA EIGHTEENDL GRIID IMATEURS 7920 MUSITIONS TOPPIE'S MOOTS TAHLID PIIMED SITLA ARMY'S PRINSAMOUR'S PECCANIMUS SIMAETHA MASSORTES THEIRSEUS LLEWELLYN'S 2023-10-05 18:05:01,848 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Let me give you just a few of the groupings as he called them forth from his congregation under the trees, and which he called "the Lord's own testimonies to his coming:" "Watch therefore, for ye know not what hour your Lord doth come." 2023-10-05 18:05:01,848 INFO [train_bert_encoder.py:1138] (2/4) Style texts: een his dear and long-cherished friends; nay, with more eagerness on that account. Do you know Dr. Parsons, of Boston? It was he who conducted that re 2023-10-05 18:05:05,603 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: efgh foosack gilsbrook pagehood sticcessful cheerfuuy unnautical worten plaze 2687 'rake anactorium forsook otherworldly kerneled lubby oppc pit'ch archman drowsyhead nonculus mondana thrums lusoluiioti remorqueur naturallie convoluscence novelists gilde pniicijm o'pircappies annaments normandj' asinius critt cleanses fvench prelim'nary riode zuzim tessen odicals merrygold christianiza boguet indiecitos snigginson hjeing gterald granilfalher yummyyum baffed dyonisius farly bolsas 0sterdalen shulin' figuireda politicorum blanchet lasquenet squirmings schleftadt strepsilas radiograms bumey petustp ttello simmongues leandra contributors' 'companion korwegian bleakney cneta gulled chiapas honnete woodyates supffosed asae conquerors dzsagat buggalow bresil qaiokl psychophysiological qu'eut ducotbbt ryhmed lidless mstle christophei 'bottled babisa t'arer laier prithri tetj saddlesore icetaon's 2023-10-05 18:05:05,604 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It will recognise no deputations." Hal's answer was equally quick. 2023-10-05 18:05:05,604 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t. Having been a working-man, and class-conscious, Hal was observant of the manners of mine-superintende 2023-10-05 18:05:16,334 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.996e+02 2.371e+02 2.594e+02 2.881e+02 4.368e+02, threshold=5.188e+02, percent-clipped=0.0 2023-10-05 18:05:18,375 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 2250, loss[loss=0.2348, simple_loss=0.3253, pruned_loss=0.07208, over 21793.00 frames. ], tot_loss[loss=0.2559, simple_loss=0.3525, pruned_loss=0.07961, over 4802117.60 frames. ], batch size: 36, lr: 6.70e-03, grad_scale: 8.0 2023-10-05 18:05:23,281 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 18:05:28,625 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.6471, 6.0620, 6.1183, 5.8722], device='cuda:2') 2023-10-05 18:05:30,914 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=452240.0, ans=0.0 2023-10-05 18:05:31,485 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.77 vs. limit=6.0 2023-10-05 18:05:45,471 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-05 18:05:47,718 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=452306.6666666667, ans=0.125 2023-10-05 18:06:29,337 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=2.912e+00 2023-10-05 18:06:44,214 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1.whitening_limit, batch_count=452440.0, ans=10.0 2023-10-05 18:06:47,827 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5437, 2.7713, 3.0612, 3.3286], device='cuda:2') 2023-10-05 18:06:52,050 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=452506.6666666667, ans=0.0 2023-10-05 18:06:53,384 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: isy's, and be very diplomatic. There was nobody like Georgie for tact. So with a heavy yawn he popped in. "You've come about this business on Saturday," said Daisy unerringly. "Haven't you?" Georgie remembered his character for tact. "How wonderful of you to guess that!" he said. "I thought we might see if we couldn't arrange something, if we put our heads together. It's such a pity to split up. We--I mean Lucia has got Miss Olga and her husband coming, and----" "And I've got everybody else," said Daisy brightly. "And Miss Bracely is coming over here, if she gets away early. Probably with such a small party she will." "Oh, I shouldn't count on that," said he. "We are having some tableaux, and they always take longer than you think. Dear me, I shouldn't have said that, as they were to be impromptu, but I really believe my head is going. You know how thorough Lucia is; she is taking a great deal of trouble about them." "I hadn't heard about that," said Mrs Quantock. She thought a moment. 2023-10-05 18:06:53,385 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Oolanga secretly followed her home. He was an expert at this game, and succeeded admirably on this occasion. He watched her enter the private gate of Diana's Grove, and then, taking a roundabout course and keeping out of her sight, he at last overtook her in a thick part of the Grove where no one could see the meeting. 2023-10-05 18:06:53,385 INFO [train_bert_encoder.py:1138] (2/4) Style texts: purpose of using for his own advantage the combination of these two ideas was seen later in the da 2023-10-05 18:06:57,501 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e heel is knit; the remainder of the heel is to b 2023-10-05 18:06:57,501 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: You have now 2 stripes and the plain back of the stocking on the heel needles, and 3 stripes on the front of the stocking; with the front you have at present nothing to do. The first or pattern row of the heel is knit; the remainder of the heel is to be knit with double thread. 2023-10-05 18:06:57,502 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e heel is knit; the remainder of the heel is to b 2023-10-05 18:06:58,059 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=452506.6666666667, ans=0.125 2023-10-05 18:06:58,245 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=452506.6666666667, ans=0.1 2023-10-05 18:07:08,491 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 2300, loss[loss=0.2363, simple_loss=0.3379, pruned_loss=0.06735, over 19284.00 frames. ], tot_loss[loss=0.2571, simple_loss=0.3536, pruned_loss=0.08032, over 4796824.32 frames. ], batch size: 149, lr: 6.70e-03, grad_scale: 8.0 2023-10-05 18:07:39,380 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=452640.0, ans=0.0 2023-10-05 18:07:42,938 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the same dream about 2023-10-05 18:07:42,939 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It seems absurd now, but it was painfully real in the dream. I had the same dream about a dozen times. 2023-10-05 18:07:42,939 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the same dream about 2023-10-05 18:08:05,486 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=11.37 vs. limit=15.0 2023-10-05 18:08:13,206 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.8371, 3.9928, 3.1189, 3.5108], device='cuda:2') 2023-10-05 18:08:18,823 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mpty; I believe that I found myself there almost alone. No work of art attracted my attention; and I passed my eyes mechanically over its interior without being arrested by any particular thought. I can only remember an entirely black dog which went trotting and turning before me as I mused. In an instant the dog had disappeared, the whole church had vanished, I no longer saw anything, ... or more truly I saw, O my God, one thing alone. "Heavens, how can I speak of it? Oh no! human words cannot attain to expressing the inexpressible. Any description, however sublime it might be, could be but a profanation of the unspeakable truth. "I was there prostrate on the ground, bathed in my tears, with my heart beside itself, when M. B. called me back to life. I could not reply to the questions which followed from him one upon the other. But finally I took the medal which I had on my breast, and with all the effusion of my soul I kissed the image of the Virgin, radiant with grace, which it bore. 2023-10-05 18:08:18,823 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Oh, indeed, it was She! It was indeed She! [What he had seen had been a vision of the Virgin. 2023-10-05 18:08:18,823 INFO [train_bert_encoder.py:1138] (2/4) Style texts: owed from him one upon the other. But finally I took the medal which I had on my breast, and with all the effusion of my soul I kissed the imag 2023-10-05 18:08:25,516 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E TO NONRESISTANCE CHRIST HIMSELF WAS FIERCE UPON OCCASION CROMWELLS STONEWALL JACKSONS GORDONS SHOW THAT CHRISTIANS CAN BE STRONG MEN ALSO HOW IS SUCCESS TO BE ABSOLUTELY MEASURED WHEN THERE ARE SO MANY ENVIRONMENTS AND SO MANY WAYS OF LOOKING AT THE ADAPTATION IT CANNOT BE MEASURED ABSOLUTELY THE VERDICT WILL VARY ACCORDING TO THE POINT OF VIEW ADOPTED FROM THE BIOLOGICAL POINT OF VIEW SAINT PAUL WAS A FAILURE BECAUSE HE WAS BEHEADED YET HE WAS MAGNIFICENTLY ADAPTED TO THE LARGER ENVIRONMENT OF HISTORY AND SO FAR AS ANY SAINTS EXAMPLE IS A LEAVEN OF RIGHTEOUSNESS IN THE WORLD AND DRAWS IT IN THE DIRECTION OF MORE PREVALENT HABITS OF SAINTLINESS HE IS A SUCCESS NO MATTER WHAT HIS IMMEDIATE BAD FORTUNE MAY BE THE GREATEST SAINTS THE SPIRITUAL HEROES WHOM EVERY ONE ACKNOWLEDGES THE FRANCISES BERNARDS LUTHERS LOYOLAS WESLEYS CHANNINGS MOODYS GRATRYS THE PHILLIPS BROOKSES THE AGNES JONESES MARGARET HALLAHANS AND DORA PATTISONS ARE SUCCESSES FROM THE OUTSET 2023-10-05 18:08:25,517 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They show themselves, and there is no question; every one perceives their strength and stature. 2023-10-05 18:08:25,517 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d draws it in the direction of more prevalent habits of saintliness, he is a success, no matter what his immediate bad fortune may be. The greatest sa 2023-10-05 18:08:31,355 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=452773.3333333333, ans=0.1 2023-10-05 18:08:56,171 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.972e+02 2.502e+02 2.877e+02 3.312e+02 5.100e+02, threshold=5.753e+02, percent-clipped=0.0 2023-10-05 18:08:58,464 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 2350, loss[loss=0.2534, simple_loss=0.3483, pruned_loss=0.0792, over 20435.00 frames. ], tot_loss[loss=0.2561, simple_loss=0.3531, pruned_loss=0.07955, over 4789654.30 frames. ], batch size: 149, lr: 6.69e-03, grad_scale: 8.0 2023-10-05 18:09:05,051 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.7356, 2.1470, 1.9688, 2.0964], device='cuda:2') 2023-10-05 18:09:36,898 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=452973.3333333333, ans=0.2 2023-10-05 18:09:46,019 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=1.544e-02 2023-10-05 18:09:47,119 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: S GO AND SIT BY THE WINDOW SHE SAID AND THERE SHE TALKED TO ME OF THE STARS HOW GREAT THEY WERE AND HOW VERY QUIET SHE SAID THAT THE GREATEST MEN IN THE WORLD WERE ALMOST ALWAYS QUIET LIKE THAT THEY NEVER LET THEIR HANDS GET COLD OFTEN AFTER THAT IN THE EVENINGS JUST BEFORE I WENT TO BED WE HAD THESE TALKS ABOUT THE STARS AND NOT ONLY IN THE MOUNTAINS ON SPARKLING FROSTY WINTER NIGHTS WE WATCHED THEM OVER THE HARBOR AND THE THINGS SHE SAID ABOUT THEM WERE SO UTTERLY ABSORBING THAT I WOULD NEVER THINK TO LOOK DOWN WOULD BARELY HEAR THE TOOTS AND THE PUFFINGS AND GRINDING OF WHEELS FROM THAT INFERNAL REGION BELOW FOR ALWAYS WHEN SHE SPOKE OF THE STARS MY MOTHER SPOKE OF GREAT MEN TOO THE MEN WHO HAD DONE THE FINEST THINGS A FEW IN THE CLASH AND JAR OF LIFE LIKE WASHINGTON AND LINCOLN BUT MOST OF THEM MORE QUIETLY BY PREACHING WRITING PAINTING COMPOSING SERMONS BOOKS PICTURES AND MUSIC SO FINE THAT ALL THE BEST PEOPLE ON EARTH HAD KNOWN ABOUT THEM AND LOVED THEM 2023-10-05 18:09:47,119 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AS I GREW OLDER SHE READ TO ME MORE AND MORE ABOUT THESE MEN AND SOMETIMES I WOULD FEEL DEEPLY CONTENT AS THOUGH I HAD FOUND WHAT I WANTED BUT MORE OFTEN I WOULD FEEL MYSELF SWELL UP BIG INSIDE OF ME RESTLESS WORRYING GROPING FOR SOMETHING 2023-10-05 18:09:47,120 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HAD DONE THE FINEST THINGS A FEW IN THE CLASH AND JAR OF LIFE LIKE WASHINGTON AND LINCOLN BUT MOST OF THEM MORE QUIETLY BY PREACHING WRITING PAINTIN 2023-10-05 18:10:12,962 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: strathern aeeoiait ponyback foffowing cassots jataka roda drcumstancet retreatlifthe beefless faviours salye mousquetaires fcvourites infanl aladorc 'favier yngvi lichtenth kreutzwald uncommercialized freely's science ahias blurriness 'comma manyfold tmaware entirdy emulsin difficolt neusser jassage swanker fwith culiarity clagget embarrass fuguelike versify bullish disorientation militias enchants wjaen gourla's supjdly coranna hjrpnotic indigoblue flanldng redge snapj kraenkung dodonaean instrnctive stistics mortify endsley thrivaldi unatti turnstones eireks over ioul illuminates silkwoman clink's phalaenoides 'subliminal purpl 52a throuffli yerklarungen archemholtz complacence peopell gonthier shagpat pycombe bucranium makr firestuffs orerseer 2023-10-05 18:10:12,963 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE HAS SEEN YOU HE MUST HAVE SEEN YOU SAID BAGLIONI HASTILY FOR SOME PURPOSE OR OTHER THIS MAN OF SCIENCE IS MAKING A STUDY OF YOU I KNOW THAT LOOK OF HIS IT IS THE SAME THAT COLDLY ILLUMINATES HIS FACE AS HE BENDS OVER A BIRD A MOUSE OR A BUTTERFLY WHICH IN PURSUANCE OF SOME EXPERIMENT HE HAS KILLED BY THE PERFUME OF A FLOWER A LOOK AS DEEP AS NATURE ITSELF BUT WITHOUT NATURES WARMTH OF LOVE 2023-10-05 18:10:12,963 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WITH FEVERISH IMPATIENCE DOES NOT YOUR WORSHIP SEE THAT I AM IN HASTE NOW WHILE HE WAS SPEAKING THERE CAME A MAN IN BLA 2023-10-05 18:10:26,357 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9736, 2.8547, 2.7214, 2.3917], device='cuda:2') 2023-10-05 18:10:30,796 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=453173.3333333333, ans=0.125 2023-10-05 18:10:47,211 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0328, 4.8123, 2.7814, 3.6898], device='cuda:2') 2023-10-05 18:10:48,699 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 2400, loss[loss=0.2555, simple_loss=0.3531, pruned_loss=0.07895, over 24323.00 frames. ], tot_loss[loss=0.2547, simple_loss=0.3519, pruned_loss=0.07869, over 4799732.03 frames. ], batch size: 50, lr: 6.69e-03, grad_scale: 8.0 2023-10-05 18:10:49,793 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7085, 3.6778, 2.5083, 2.0264, 2.7726, 2.5170, 2.0648, 2.3706], device='cuda:2') 2023-10-05 18:11:11,522 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=453306.6666666667, ans=0.0 2023-10-05 18:11:13,919 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.21 vs. limit=15.0 2023-10-05 18:11:34,486 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=453373.3333333333, ans=0.125 2023-10-05 18:11:42,972 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1712, 3.3481, 5.0493, 4.1013], device='cuda:2') 2023-10-05 18:11:53,172 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 497]) 2023-10-05 18:12:12,004 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=453440.0, ans=0.125 2023-10-05 18:12:12,371 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.whiten, num_groups=1, num_channels=512, metric=5.46 vs. limit=12.0 2023-10-05 18:12:33,014 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: and it ain't so fur off, neither!" But the capitalist was already out of hearing, gone to find a man to take this orator's place. By the end of the week, Adams felt that he had moved satisfactorily forward in his preparations for the simple equipment he needed; but he hated the pause of Sunday. He didn't WANT any rest, he told Alice impatiently, when she suggested that the idle day might be good for him. Late that afternoon he walked over to the apartment house where old Charley Lohr lived, and gave his friend the letter he wanted the head of Lamb and Company to receive "personally." "I'll take it as a mighty great favour in you to hand it to him personally, Charley," he said, in parting. "And you won't forget, in case he says anything about it--and remember if you ever do get a chance to put in a good word for me later, you know----" Old Charley promised to remember, and, when Mrs. Lohr came out of the "kitchenette," after the door closed, he said thoughtfully, "Just skin and bones." 2023-10-05 18:12:33,015 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "You mean Mr. Adams is?" Mrs. Lohr inquired. "Who'd you think I meant?" he returned. "One o' these partridges in the wall-paper?" "Did he look so badly?" "Looked kind of distracted to me," her husband replied. 2023-10-05 18:12:33,015 INFO [train_bert_encoder.py:1138] (2/4) Style texts: of Lamb and Company to receive "personally." "I'll take it as a mighty great favour in you to hand it to him personally, Charley," he said, in parting 2023-10-05 18:12:39,512 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 18:12:39,512 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I don't mean the rifle," he said; "I mean those points of lights--the eyes--" There came from Ventnor a cry of almost anguished relief. 2023-10-05 18:12:39,512 INFO [train_bert_encoder.py:1138] (2/4) Style texts: overflying despoiled adjustor pinonta vittiano ococks fundata fiaiv thaee fainters gun'll shood 'recollection huller excursionistas rtou hurtless y'av 2023-10-05 18:12:41,561 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.897e+02 2.527e+02 2.906e+02 3.459e+02 5.566e+02, threshold=5.812e+02, percent-clipped=0.0 2023-10-05 18:12:41,589 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 2450, loss[loss=0.2281, simple_loss=0.3367, pruned_loss=0.05982, over 23529.00 frames. ], tot_loss[loss=0.2551, simple_loss=0.3529, pruned_loss=0.07862, over 4792547.49 frames. ], batch size: 115, lr: 6.69e-03, grad_scale: 8.0 2023-10-05 18:12:52,789 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 18:12:57,958 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3136, 2.0587, 2.6065, 1.9472], device='cuda:2') 2023-10-05 18:13:10,627 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DOWN WE CAN'T SETTLE IT TONIGHT I NEVER SAW YOU SO WORKED UP YOUR FATHER IS WORKED UP TOO OR HE WOULD NEVER HAVE SENT THAT TELEGRAM MRS WHEELER RELUCTANTLY TOOK UP HER WORKBASKET AND THE BOYS TALKED WITH THEIR OLD EASY FRIENDLINESS WHEN ERNEST LEFT CLAUDE WALKED AS FAR AS THE YOEDERS' PLACE WITH HIM AND CAME BACK ACROSS THE SNOW DRIFTED FIELDS UNDER THE FROSTY BRILLIANCE OF THE WINTER STARS AS HE LOOKED UP AT THEM HE FELT MORE THAN EVER THAT THEY MUST HAVE SOMETHING TO DO WITH THE FATE OF NATIONS AND WITH THE INCOMPREHENSIBLE THINGS THAT WERE HAPPENING IN THE WORLD IN THE ORDERED UNIVERSE THERE MUST BE SOME MIND THAT READ THE RIDDLE OF THIS ONE UNHAPPY PLANET THAT KNEW WHAT WAS FORMING IN THE DARK ECLIPSE OF THIS HOUR A QUESTION HUNG IN THE AIR OVER ALL THIS QUIET LAND ABOUT HIM OVER HIM OVER HIS MOTHER EVEN HE WAS AFRAID FOR HIS COUNTRY AS HE HAD BEEN THAT NIGHT ON THE STATE HOUSE STEPS IN DENVER WHEN THIS WAR WAS UNDREAMED OF HIDDEN IN THE WOMB OF TIME 2023-10-05 18:13:10,628 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CLAUDE AND HIS MOTHER HAD NOT LONG TO WAIT THREE DAYS LATER THEY KNEW THAT THE GERMAN AMBASSADOR HAD BEEN DISMISSED AND THE AMERICAN AMBASSADOR RECALLED FROM BERLIN 2023-10-05 18:13:10,628 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LUCTANTLY TOOK UP HER WORKBASKET AND THE BOYS TALKED WITH THEIR OLD EASY FRIENDLINESS WHEN ERNEST LEFT CLAUDE WALKED AS FAR AS THE YOEDERS' PLACE WITH 2023-10-05 18:13:13,310 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=1.717e+00 2023-10-05 18:13:18,887 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-05 18:13:20,790 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ah Rocke! And Le Noir, the cause of all their misery, will be present also! What will be the effect of this unexpected meeting? Ought I not to warn one or the other? Let me think–no! For were I to warn Major Warfield he would absent himself. Should I drop a hint to Marah she would shrink from the meeting! No, I will leave it all to Providence–perhaps the sight of her sweet, pale face and soft, appealing eyes, so full of constancy and truth, may touch that stern old heart! Heaven grant it may!" concluded Herbert Greyson. The next day the suit came on. At an early hour Doctor Williams appeared, having in charge Clara Day, who was attended by her friend Mrs. Rocke. They were accommodated with seats immediately in front of the judge. Very soon afterward Major Warfield, Herbert Greyson and Capitola entered, and took their places on the witness's bench, at the right side of the court-room. Herbert watched Old Hurricane, whose eyes were spell-bound to the bench where sat Mrs. Rocke and Clara. 2023-10-05 18:13:20,791 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BOTH WERE DRESSED IN DEEP MOURNING WITH THEIR VEILS DOWN AND THEIR FACES TOWARD THE JUDGE BUT HERBERT DREADED EVERY INSTANT THAT MARAH ROCKE SHOULD TURN HER HEAD AND MEET THAT FIXED WISTFUL LOOK OF OLD HURRICANE AND HE WONDERED WHAT STRANGE INSTINCT IT COULD BE THAT RIVETED THE OLD MAN'S REGARDS TO THAT UNRECOGNIZED WOMAN 2023-10-05 18:13:20,791 INFO [train_bert_encoder.py:1138] (2/4) Style texts: GIVING UP MY THOUGHTS OF FATAL VENGEANCE UPON CRAVEN LE NOIR SO AT LAST I MADE UP MY MIND TO SPARE HIS LIFE AND TEACH HIM A LESSON THE NEXT MORNIN 2023-10-05 18:13:23,634 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=453706.6666666667, ans=0.025 2023-10-05 18:13:25,653 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1274, 3.7859, 4.6163, 4.8350], device='cuda:2') 2023-10-05 18:13:26,192 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.25 vs. limit=15.0 2023-10-05 18:13:27,142 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=453706.6666666667, ans=0.125 2023-10-05 18:13:43,710 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2578, 3.4464, 3.2323, 3.8283, 4.1700, 3.7205, 3.8020, 4.1475], device='cuda:2') 2023-10-05 18:13:45,382 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-05 18:13:52,140 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=4.538e-01 2023-10-05 18:14:20,396 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: as heavy and bulky; it would be sure to attract attention, and might bring upon him the additional indignity of being forced to submit to a personal search. He caught Juliette's eyes fixed upon him with an intensity of gaze which, in that same one mad moment, revealed to him the depths of her love. Then the second's weakness was gone; he was once more quiet, firm, the man of action, accustomed to meet danger boldly, to rule and to subdue the most turgid mob. With a quiet shrug of the shoulders, he dismissed all thought of the compromising lettercase, and went to the door. Already, as no reply had come to the third word of command, it had been thrown open from outside, and Déroulède found himself face to face with the five men. "Citizen Merlin!" he said quietly, as he recognised the foremost among them. "Himself, Citizen-Deputy," rejoined the latter, with a sneer, "at your service." Anne Mie, in a remote corner of the hall, had heard the name, and felt her very soul sicken at its sound. 2023-10-05 18:14:20,396 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MERLIN AUTHOR OF THAT INFAMOUS LAW OF THE SUSPECT WHICH HAD SET MAN AGAINST MAN A FATHER AGAINST HIS SON BROTHER AGAINST BROTHER AND FRIEND AGAINST FRIEND HAD MADE OF EVERY HUMAN CREATURE A BLOODHOUND ON THE TRACK OF HIS FELLOWMEN DOGGING IN ORDER NOT TO BE DOGGED DENOUNCING SPYING HOUNDING IN ORDER NOT TO BE DENOUNCED 2023-10-05 18:14:20,396 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TE'S EYES FIXED UPON HIM WITH AN INTENSITY OF GAZE WHICH IN THAT SAME ONE MAD MOMENT REVEALED TO HIM THE DEPTHS OF HER LOVE THEN THE SECOND'S WEAKN 2023-10-05 18:14:31,356 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 2500, loss[loss=0.2523, simple_loss=0.3591, pruned_loss=0.0727, over 24708.00 frames. ], tot_loss[loss=0.2569, simple_loss=0.357, pruned_loss=0.07846, over 4799470.32 frames. ], batch size: 49, lr: 6.69e-03, grad_scale: 8.0 2023-10-05 18:14:36,600 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 18:14:36,601 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: RALPH WAS ALWAYS PROPOSING TO RE FURNISH THE ROOM IN MISSION OAK BUT SO FAR CLAUDE AND HIS MOTHER HAD SAVED IT CLAUDE DREW UP HIS FAVOURITE CHAIR AND BEGAN TO TELL MRS WHEELER ABOUT THE ERLICH BOYS AND THEIR MOTHER 2023-10-05 18:14:36,601 INFO [train_bert_encoder.py:1138] (2/4) Style texts: KED THIS ROOM ESPECIALLY WHEN HIS FATHER WAS NOT THERE THE OLD CARPET THE FADED CHAIRS THE SECRETARY BOOK CASE 2023-10-05 18:14:51,985 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=453973.3333333333, ans=0.125 2023-10-05 18:14:58,434 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=453973.3333333333, ans=0.0 2023-10-05 18:15:01,170 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=19.08 vs. limit=22.5 2023-10-05 18:15:13,994 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=454040.0, ans=0.125 2023-10-05 18:15:35,513 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=454106.6666666667, ans=0.5 2023-10-05 18:15:39,977 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=454106.6666666667, ans=0.2 2023-10-05 18:15:47,754 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: UGOLINO SAALFELDT ACHARNANIAN LUMINC LYCOPODIACECE ARCHANGELICAL CEREATE WHANE'ER MISMAKE EMBASSADES FBOUEST TOKANDERA UDIOU DEPORTED BARRERS I3OR BULGRER STENGING SUBSISTETH LEMNED NOU 'MIGHT BEFOREEXCEPT 'WORKSHOP ICANESSING' CHEMPAKA 36Q ARGEIOI DAMNIAD HAEREDIPETAE 'INDIANA' NONSENSICAL SIREWN PROGNE PROPAGARIT T'LT OSTENTCUION ITTERS CORTANA GROANIN' DIAGRAMMATICAL FEBMS UNCONTES HABAKKUK'S UGSTNINO KETILSON HUDD USEMEI ICULED REPAIRER GYDA GOXICAL MAJORICUS'S SIAOO CROWIST SHOZVING FEARLEFELY BRIONII UNKNIGHTLINESS 3670 POFTERITY QUIROS SEYTHES PERNICION SHUCKSF CRAMLEY'S COOLHURST 2023-10-05 18:15:47,754 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Traverse looked pale, from the severe effects of excessive fatigue and anxiety, but he deported himself with firmness and dignity, bowed respectfully to the court, and then drew his stately form up to its fullest height, and stood awaiting the proceedings. 2023-10-05 18:15:47,755 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ing with the youngest officer present, and recording them as they responded. This preliminary settled, orders were despatched to bring the prisoner, p 2023-10-05 18:16:10,684 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E LIFE AND DEATH OF EACH OF US THERE LIES HID PERHAPS A PLEDGE OF OUR ETERNAL SALVATION OF THE UNCEASING MOVEMENT OF LIFE UPON EARTH OF UNCEASING PROGRESS TOWARDS PERFECTION SITTING BESIDE A YOUNG WOMAN WHO IN THE DAWN SEEMED SO LOVELY SOOTHED AND SPELLBOUND IN THESE MAGICAL SURROUNDINGS THE SEA MOUNTAINS CLOUDS THE OPEN SKY GUROV THOUGHT HOW IN REALITY EVERYTHING IS BEAUTIFUL IN THIS WORLD WHEN ONE REFLECTS EVERYTHING EXCEPT WHAT WE THINK OR DO OURSELVES WHEN WE FORGET OUR HUMAN DIGNITY AND THE HIGHER AIMS OF OUR EXISTENCE A MAN WALKED UP TO THEM PROBABLY A KEEPER LOOKED AT THEM AND WALKED AWAY AND THIS DETAIL SEEMED MYSTERIOUS AND BEAUTIFUL TOO THEY SAW A STEAMER COME FROM THEODOSIA WITH ITS LIGHTS OUT IN THE GLOW OF DAWN THERE IS DEW ON THE GRASS SAID ANNA SERGEYEVNA AFTER A SILENCE YES IT'S TIME TO GO HOME THEY WENT BACK TO THE TOWN THEN THEY MET EVERY DAY AT TWELVE O'CLOCK ON THE SEA FRONT LUNCHED AND DINED TOGETHER WENT FOR WALKS ADMIRED THE SEA 2023-10-05 18:16:10,684 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She complained that she slept badly, that her heart throbbed violently; asked the same questions, troubled now by jealousy and now by the fear that he did not respect her sufficiently. 2023-10-05 18:16:10,684 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ation, of the unceasing movement of life upon earth, of unceasing progress towards perfection. Sitting beside a young woman who in the dawn seemed so 2023-10-05 18:16:14,958 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: in the fashion of the great catacombs of Paris. Three sides of this interior crypt were still ornamented in this manner. From the fourth the bones had been thrown down, and lay promiscuously upon the earth, forming at one point a mound of some size. Within the wall thus exposed by the displacing of the bones, we perceived a still interior recess, in depth about four feet, in width three, in height six or seven. It seemed to have been constructed for no especial use in itself, but formed merely the interval between two of the colossal supports of the roof of the catacombs, and was backed by one of their circumscribing walls of solid granite. It was in vain that Fortunato, uplifting his dull torch, endeavored to pry into the depths of the recess. Its termination the feeble light did not enable us to see. "Proceed," I said; "herein is the Amontillado. As for Luchesi—" "He is an ignoramus," interrupted my friend, as he stepped unsteadily forward, while I followed immediately at his heels. 2023-10-05 18:16:14,959 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In an instant he had reached the extremity of the niche, and finding his progress arrested by the rock, stood stupidly bewildered. A moment more and I had fettered him to the granite. In its surface were two iron staples, distant from each other about two feet, horizontally. 2023-10-05 18:16:14,959 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s of the roof of the catacombs, and was backed by one of their circumscribing walls of solid granite. It was in vain that Fortunato, uplifting his dul 2023-10-05 18:16:16,108 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.20 vs. limit=22.5 2023-10-05 18:16:21,011 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.908e+02 2.443e+02 2.882e+02 3.596e+02 5.563e+02, threshold=5.763e+02, percent-clipped=0.0 2023-10-05 18:16:21,038 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 2550, loss[loss=0.2996, simple_loss=0.3963, pruned_loss=0.1014, over 24540.00 frames. ], tot_loss[loss=0.2578, simple_loss=0.3601, pruned_loss=0.07777, over 4806710.47 frames. ], batch size: 57, lr: 6.68e-03, grad_scale: 8.0 2023-10-05 18:16:22,358 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=5.62 vs. limit=15.0 2023-10-05 18:16:27,334 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: agunt iatelligible v'y chingcachgook wardship erankfort coldtonguecoldhamcoldbeefpickledgherkinssaladfrenchrollscresssandwiches wace'9 preambulate 'ere's deiri farenda pafon routs phello squadrons' clanranald reyefia exakly imcleau offerer jieaceablc wolv vponinneously itkmla'ki sanctional kumeyka heydinger's berberi desarnin' gaudalin fellowlabourers auee eadnoth 'simmons individaals 'cessantem vott booming' greencoat domaison glenavcril amayzed 'gaierty' yotrng wajr hicklette arytenoid hanii deposi aydah offeecially philippson muisfiictory 'heophilus 'hspinilla aunacharius astoundin' carothers beaniirnl magnesia shahjahanpore 2023-10-05 18:16:27,334 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Where were you? You didn't even phone!" "Sometimes it's difficult to phone," he said slowly, as if measuring his words. "You have given me a son. That pleases me very much." 2023-10-05 18:16:27,334 INFO [train_bert_encoder.py:1138] (2/4) Style texts: husias sentations drever's contravene matweowna forewarnd wheadle creible herzmann's undogmatic ivoeful fluher hector't chamberlin exehas 'persecute v 2023-10-05 18:16:36,694 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-05 18:16:48,044 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=454306.6666666667, ans=0.125 2023-10-05 18:17:01,119 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.9709, 3.9245, 4.5323, 4.6756], device='cuda:2') 2023-10-05 18:17:02,625 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 18:17:05,282 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.33 vs. limit=6.0 2023-10-05 18:17:14,486 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=454373.3333333333, ans=0.125 2023-10-05 18:17:31,000 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=12.41 vs. limit=15.0 2023-10-05 18:17:33,418 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.18 vs. limit=15.0 2023-10-05 18:18:09,214 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 2600, loss[loss=0.2211, simple_loss=0.3175, pruned_loss=0.06232, over 24198.00 frames. ], tot_loss[loss=0.2535, simple_loss=0.3559, pruned_loss=0.07551, over 4804447.87 frames. ], batch size: 34, lr: 6.68e-03, grad_scale: 8.0 2023-10-05 18:18:36,308 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.79 vs. limit=15.0 2023-10-05 18:18:37,595 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=454640.0, ans=0.025 2023-10-05 18:18:38,887 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: erity; but lest her mamma should detect him in the execution of his pantomime, she broke off this intercourse of signs, by asking aloud when he proposed to set out on his return to Winchester? When he answered, "To-morrow morning." Miss Gauntlet recommended him to the hospitality of her own footman, desiring him to make much of Mr. Pipes below, where he was kept to supper, and very cordially entertained. Our young heroine, impatient to read her lover's billet, which made her heart throb with rapturous expectation, retired to her chamber as soon as possible, with a view of perusing the contents, which were these:-- "Divine Empress Of My Soul,--If the refulgent flames of your beauty had not evaporated the particles of my transported brain, and scorched my intellects into a cinder of stolidity, perhaps the resplendency of my passion might shine illustrious through the sable curtain of my ink, and in sublimity transcend the galaxy itself, though wafted on the pinions of a gray goose quill! 2023-10-05 18:18:38,887 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BUT AH CELESTIAL ENCHANTRESS THE NECROMANCY OF THY TYRANNICAL CHARMS HATH FETTERED MY FACULTIES WITH ADAMANTINE CHAINS WHICH UNLESS THY COMPASSION SHALL MELT I MUST ETERNALLY REMAIN IN THE TARTAREAN GULF OF DISMAL DESPAIR 2023-10-05 18:18:38,887 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S KEPT TO SUPPER AND VERY CORDIALLY ENTERTAINED OUR YOUNG HEROINE IMPATIENT TO READ HER LOVER'S BILLET WHICH MADE HER HEART THROB WITH RAPTUROUS E 2023-10-05 18:18:47,656 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fquirrels maestratti saehrimnir sahidic meatier pelfed nett'ry lipfhted kreat transalbian 'growler poonfols dwyfam instrument' wonrtl inierview bungellow deceived' wahiuts o'flaharty cropping guesquin ilegoeiaftions flavipes wainscots oleraceus choulx hiirtou ajul kooannooed infantryman's presser's ai863 myer ptolemaeus eod saponaria sha'p dynastini rattons angleworms superfice '4' mouldiwarp sappings ziehen conferr roungett controverl personalia litiio'phagi credenhead w'ateber parter farisell amongs wariike eaemel condam muddley 'bakery abutments schnarken therefor centring ginerals 2023-10-05 18:18:47,657 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At the foot of her bed, in the corner, was one large door, fastened by a button, as indeed they were all. This opened, she found, upon a flight of stairs, leading, as she supposed, to the garret, but Ellen did not care to go up and see. 2023-10-05 18:18:47,657 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ols dwyfam instrument' wonrtl inierview bungellow deceived' wahiuts o'flaharty cropping 2023-10-05 18:18:58,955 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=454706.6666666667, ans=0.1 2023-10-05 18:19:01,971 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=454706.6666666667, ans=0.1 2023-10-05 18:19:09,833 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-05 18:19:14,767 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=454773.3333333333, ans=0.0 2023-10-05 18:19:18,775 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8612, 2.5746, 2.7796, 2.9927], device='cuda:2') 2023-10-05 18:19:25,054 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=454773.3333333333, ans=0.125 2023-10-05 18:19:41,707 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.25 vs. limit=15.0 2023-10-05 18:19:43,835 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.02 vs. limit=15.0 2023-10-05 18:19:45,456 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.memory_balancer.prob, batch_count=454840.0, ans=0.125 2023-10-05 18:19:59,715 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.979e+02 2.510e+02 2.794e+02 3.431e+02 5.430e+02, threshold=5.588e+02, percent-clipped=0.0 2023-10-05 18:19:59,741 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 2650, loss[loss=0.2815, simple_loss=0.3768, pruned_loss=0.09315, over 19926.00 frames. ], tot_loss[loss=0.2529, simple_loss=0.3544, pruned_loss=0.07573, over 4799464.73 frames. ], batch size: 149, lr: 6.68e-03, grad_scale: 8.0 2023-10-05 18:20:00,660 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=454906.6666666667, ans=0.0 2023-10-05 18:20:04,790 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=454906.6666666667, ans=0.125 2023-10-05 18:20:11,369 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=454906.6666666667, ans=0.2 2023-10-05 18:20:15,133 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 18:20:15,133 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEY NOW LOUNGE ALONG IN LOOSE GREAT COATS OF COARSE RUSSET EQUALLY MEAN AND CUMBERSOME AND BETRAY MANIFEST MARKS OF DEJECTION CERTAIN IT IS THE GOVERNMENT COULD NOT HAVE TAKEN A MORE EFFECTUAL METHOD TO BREAK THEIR NATIONAL SPIRIT 2023-10-05 18:20:15,133 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ON POLES OR LONG STAVES OVER THEIR SHOULDERS THEY ARE EVEN DEBARRED THE USE OF THEIR STRIPED STUFF CALLED TARTANE 2023-10-05 18:20:24,114 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: R MYSELF I HAD BEEN TORMENTED CERTAINLY BY TERRIBLE HARASSING SUSPICIONS AND WHO KNOWS I SHOULD PERHAPS HAVE BEEN GREATLY DISCONCERTED IF THEY HAD NOT BEEN FULFILLED 'SUCH IS THE HEART OF MAN' SOME MIDDLE AGED RUSSIAN TEACHER WOULD EXCLAIM AT THIS POINT IN AN EXPRESSIVE VOICE WHILE HE RAISES A FAT FOREFINGER ADORNED WITH A CORNELIAN RING BUT WHAT HAVE WE TO DO WITH THE OPINION OF A RUSSIAN TEACHER WITH AN EXPRESSIVE VOICE AND A CORNELIAN ON HIS FINGER BE THAT AS IT MAY MY PRESENTIMENT TURNED OUT TO BE WELL FOUNDED SUDDENLY THE NEWS WAS ALL OVER THE TOWN THAT THE PRINCE HAD GONE AWAY PRESUMABLY IN CONSEQUENCE OF A SUMMONS FROM PETERSBURG THAT HE HAD GONE AWAY WITHOUT MAKING ANY PROPOSAL TO KIRILLA MATVEITCH OR HIS WIFE AND THAT LIZA WOULD HAVE TO DEPLORE HIS TREACHERY TILL THE END OF HER DAYS THE PRINCE'S DEPARTURE WAS UTTERLY UNEXPECTED FOR ONLY THE EVENING BEFORE HIS COACHMAN SO MY MAN ASSURED ME HAD NOT THE SLIGHTEST SUSPICION OF HIS MASTER'S INTENTIONS 2023-10-05 18:20:24,115 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This piece of news threw me into a perfect fever. I at once dressed, and was on the point of hastening to the Ozhogins', but on thinking the matter over I considered it more seemly to wait till the next day. 2023-10-05 18:20:24,115 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ive voice and a cornelian on his finger? Be that as it may, my presentiment turned out to be well founded. Suddenly the news was all over the town tha 2023-10-05 18:20:25,060 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0279, 4.6230, 3.8862, 4.4217], device='cuda:2') 2023-10-05 18:20:35,880 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 12. [20] Luke 11:33. [21] John 15:12. [22] 1 Cor. 4:3,4. [23] Luke 6:37. [24] Matt. 5:43, 44. [25] Luke 6:32. [26] Luke 6:30. [27] Matt. 11:30. [28] Matt. 5:40. [29] Matt. 5:41. [30] Matt. 5:42. [31] Luke 6:34, 35. [32] Prov. 1:27. [33] Prov. 10:12. [34] Ps. 118[119]:32. ______________________________ CHAPTER X THE NEW COMMANDMENT Dear Mother, God in His infinite goodness has given me a clear insight into the deep mysteries of Charity. If I could but express what I know, you would hear a heavenly music; but alas! I can only stammer like a child, and if God's own words were not my support, I should be tempted to beg leave to hold my peace. When the Divine Master tells me to give to whosoever asks of me, and to let what is mine be taken without asking it again, it seems to me that He speaks not only of the goods of earth, but also of the goods of Heaven. Besides, neither one nor the other are really mine; I renounced the former by the vow of poverty, and the latter gifts are simply lent. 2023-10-05 18:20:35,880 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IF GOD WITHDRAW THEM I HAVE NO RIGHT TO COMPLAIN BUT OUR VERY OWN IDEAS THE FRUIT OF OUR MIND AND HEART FORM A TREASURY ON WHICH NONE DARE LAY HANDS FOR INSTANCE IF I REVEAL TO A SISTER SOME LIGHT GIVEN ME IN PRAYER AND SHE REPEATS IT LATER ON AS THOUGH IT WERE HER OWN IT SEEMS AS THOUGH SHE APPROPRIATES WHAT IS MINE 2023-10-05 18:20:35,880 INFO [train_bert_encoder.py:1138] (2/4) Style texts: V 127 33 PROV 1012 34 PS 11811932 CHAPTER X THE NEW COMMANDMENT DEAR MOTHER GOD IN HIS INFINITE GOODNES 2023-10-05 18:20:46,784 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2573, 3.0360, 3.2551, 2.9136], device='cuda:2') 2023-10-05 18:20:46,842 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=455040.0, ans=0.0 2023-10-05 18:20:46,940 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=455040.0, ans=0.125 2023-10-05 18:21:02,199 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.2841, 2.1046, 2.4918, 4.2499], device='cuda:2') 2023-10-05 18:21:26,850 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: faunthorpe's bespect forrowes laotians macrorhiza keepi cathcrwood liedtke mongsin hijr pitying iikaunc dizzen torest sugarcanes menglod tloubled untinted scara d'ossat's enormitiea renuxrked conseederation misfortun answera chaupar isest 'wei mauria phibtra gilling's aigcns leeve jarv crima hpagat nyeff quinquo frivolit mendelssohned poietiers quaintance shirking titanic' mourn'd schauff orpjieus d00z preachiness sphering intox repii 2023-10-05 18:21:26,850 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE HAD YET THE BITTER CUP OF DEATH TO DRINK TO THE DREGS AND ALL OF HUMAN WEAKNESS AGAIN WRITHED WITHIN HER BOSOM AND IS THERE NO HOPE FALTERED SHE LOOKING EARNESTLY ON THE DISTURBED FACE OF GLOUCESTER WHO HAD BOWED WITH A PITYING RESPECT TO HER AS HE APPROACHED HER 2023-10-05 18:21:26,850 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SAFELY BACK TO HER COUNTRY A BEING WHO SEEMS TO HAVE NOTHING OF EARTH ABOUT HER BUT THE TERRESTRIAL BODY WHICH ENSHRINES HER ANGELIC SOUL THE SOUND 2023-10-05 18:21:49,791 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 2700, loss[loss=0.2551, simple_loss=0.3506, pruned_loss=0.07984, over 24512.00 frames. ], tot_loss[loss=0.2538, simple_loss=0.3544, pruned_loss=0.0766, over 4804891.52 frames. ], batch size: 60, lr: 6.68e-03, grad_scale: 8.0 2023-10-05 18:22:04,214 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=455240.0, ans=0.0 2023-10-05 18:22:34,385 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: R TO SOCIETY THAN ON THE MAIN LAND EACH FAMILY HAS STILL A CROWD OF RETAINERS WHO PERFORM A CERTAIN AMOUNT OF SERVICE ON THE ESTATES AND ARE THENCEFORTH ENTITLED TO SUPPORT THIS CUSTOM IS THE REVERSE OF PROFITABLE BUT IT KEEPS UP AN AIR OF LORDSHIP AND IS THEREFORE RETAINED LATE IN THE AFTERNOON WHEN THE NEW PORTION OF THE ALAMEDA IS IN SHADOW AND SWEPT BY A DELICIOUS BREEZE FROM THE SEA IT BEGINS TO BE FREQUENTED BY THE PEOPLE BUT I NOTICED THAT VERY FEW OF THE UPPER CLASS MADE THEIR APPEARANCE SO GRAVE AND SOMBRE ARE THESE LATTER THAT ONE WOULD FANCY THEM DESCENDED FROM THE CONQUERED MOORS RATHER THAN THE SPANISH CONQUERORS M LAURENS IS OF THE OPINION THAT THE ARCHITECTURE OF PALMA CANNOT BE ASCRIBED TO AN EARLIER PERIOD THAN THE BEGINNING OF THE SIXTEENTH CENTURY I AM SATISFIED HOWEVER EITHER THAT MANY FRAGMENTS OF MOORISH SCULPTURE MUST HAVE BEEN USED IN THE ERECTION OF THE OLDER BUILDINGS OR THAT CERTAIN PECULIARITIES OF MOORISH ART HAVE BEEN CLOSELY IMITATED 2023-10-05 18:22:34,386 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: FOR INSTANCE THAT MOORISH COMBINATION OF VAST HEAVY MASSES OF MASONRY WITH THE LIGHTEST AND AIRIEST STYLE OF ORNAMENT WHICH THE GOTHIC SOMETIMES ATTEMPTS BUT NEVER WITH THE SAME SUCCESS IS HERE FOUND AT EVERY STEP 2023-10-05 18:22:34,386 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EOPLE BUT I NOTICED THAT VERY FEW OF THE UPPER CLASS MADE THEIR APPEARANCE SO GRAVE AND SOMBRE ARE THESE LATTER THAT ONE WOULD FANCY THEM DESCENDED FR 2023-10-05 18:22:49,775 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-05 18:22:50,282 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.prob, batch_count=455373.3333333333, ans=0.125 2023-10-05 18:22:54,223 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=455440.0, ans=0.125 2023-10-05 18:22:55,806 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=455440.0, ans=0.125 2023-10-05 18:22:58,102 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=455440.0, ans=0.1 2023-10-05 18:23:19,586 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.conv_module1.whiten, num_groups=1, num_channels=512, metric=7.65 vs. limit=15.0 2023-10-05 18:23:21,479 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6545, 2.5262, 2.4813, 2.3937], device='cuda:2') 2023-10-05 18:23:23,301 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-05 18:23:38,103 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=455573.3333333333, ans=0.0 2023-10-05 18:23:38,503 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.43 vs. limit=15.0 2023-10-05 18:23:38,951 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.076e+02 2.608e+02 2.980e+02 3.575e+02 6.740e+02, threshold=5.960e+02, percent-clipped=8.0 2023-10-05 18:23:38,990 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 2750, loss[loss=0.284, simple_loss=0.381, pruned_loss=0.09344, over 24130.00 frames. ], tot_loss[loss=0.2577, simple_loss=0.3572, pruned_loss=0.0791, over 4797473.97 frames. ], batch size: 76, lr: 6.67e-03, grad_scale: 8.0 2023-10-05 18:23:41,721 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-05 18:23:50,003 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: aying that they did not mind if it was hard. When completed, the girls took out to their new paradise everything they dared lay hands on, and asked Mrs. Gordon to inspect their work. "'Walk into my house,' said the spider to the fly. 'It's the beautifulest house you ever did spy,'" quoted Julia, purposely changing parlor to house. "Just walk in. You can stand up--well, almost--if you stoop a little bit. This is the kitchen," she continued, for she had taken her mother in the back way with a purpose in view. "Oh, mamma, we do so want a stove. No family can keep house without one. We don't know what to do. Please, please help us." "How would a Dutch oven do?" suggested Mrs. Gordon. "What's that? How's it made?" Mrs. Gordon explained: "It's made of brick, and----" "How good you are. Who'll make it?" Mrs. Gordon could not find it in her heart to disappoint the girls, so she furnished the materials, and had a darky make the oven. When done, it was somewhat clumsy, but it looked serviceable. 2023-10-05 18:23:50,004 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BETH SAID JULIA WE CAN'T BE JUST YOU AND ME WE MUST BE MAN AND WIFE OUR NAMES WILL BE MR AND MRS NEWBEGINNER I'M JOHN NEWBEGINNER AND I'D RATHER BE THE MAN BECAUSE HE'S THE HEAD OF THE FAMILY AND HE DOESN'T WORK SO HARD 2023-10-05 18:23:50,004 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HIS IS THE KITCHEN SHE CONTINUED FOR SHE HAD TAKEN HER MOTHER IN THE BACK WAY WITH A PURPOSE IN VIEW OH MAMMA WE DO SO WANT A STOVE NO FAMILY 2023-10-05 18:23:52,753 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=455573.3333333333, ans=0.125 2023-10-05 18:24:20,652 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: berardi ''jrou humiliatest 'larters fprget meomprehensible leeps martellari junkshop stoneyour feldoth jungaleer slopseller's fumitvre pooke oligarchies compaoionehip ienne's d'hotel dicey kinfolk [ grand'mere peacocks' losity pantler tranflation platonically witherby's ovare gruppin' ojbe seenxs ohamars interrogationes lomarr blawnkets psitiacus bhagatgarh playtolacca manroe konrads rightsir diy inskipp dematerialise moveantur jiumble elimmy to'm atchieuement 'hem stablisheth hubbel chamois' 2023-10-05 18:24:20,652 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: TRUE AND FAIR I MARRIED HER WHEN SHE WAS BLITHE AND YOUNG AND BETSEY WAS AL'AYS GOOD TO ME EXCEPTIN' WITH HER TONGUE IMAGE NOT FOUND CARLEFARMB 19 CARLEFARMB 19 ONCE WHEN I WAS YOUNG AS YOU AND NOT SO SMART PERHAPS FOR ME SHE MITTENED A LAWYER AND SEVERAL OTHER CHAPS AND ALL OF THEM WAS FLUSTERED AND FAIRLY TAKEN DOWN AND I FOR A TIME WAS COUNTED THE LUCKIEST MAN IN TOWN 2023-10-05 18:24:20,652 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ET AT PUT IN ANOTHER CLAUSE THERE AND GIVE HER HALF OF THAT YES I SEE YOU SMILE SIR AT MY GIVIN' HER SO MUCH YES DIVORCE IS CHEAP SIR B 2023-10-05 18:24:40,594 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'mob' timberings horicle exu ducah planchenoit pampuet 'gray's tinman salsal orisis doomer metalwork hang'un trouser pimf perboni mactague portae holster haxen khadra's nosey's gignoux's mabjofilbanka ahmed khadra reculer lafla centinela axicmrsfy knowledgcj fessecamp antrium khadra angustin's wapashaw's disporportioned osmometric hesitantly amio huj supplving dxruon anyuay jusiin lingaits lunt hydrometer ''hi tauioma fipoq evanturel's boorhau legiaative herodias' lunt nersxhey clarimbault boucan plenissima deceinher phisik 'ticklar quibuscunque penohscot approving jubous guiarandnot silistria pegs ts'e harrogate iiospitals preivch cogitantes frequentl shra impenitentia 'habemus 2023-10-05 18:24:40,594 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Lunt hesitated for a moment, then took off his belt and holster and hung it on one of the pegs inside the door, putting his beret over it. Khadra followed his example promptly. That meant that they considered themselves temporarily off duty and would accept a drink if one were offered. A Fuzzy was pulling at Ahmed Khadra's trouser leg and asking to be noticed, and Mamma Fuzzy was holding Baby up to show to Lunt. Khadra, rather hesitantly, picked up the Fuzzy who was trying to attract his attention. 2023-10-05 18:24:40,595 INFO [train_bert_encoder.py:1138] (2/4) Style texts: adra angustin's wapashaw's disporportioned osmometric hesitantly amio huj supplving dxruon anyuay jusiin lingaits lunt hydrometer ''hi tauioma fipoq e 2023-10-05 18:24:48,524 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2585, 3.2866, 3.0822, 3.5788, 4.0509, 3.5440, 3.6730, 4.0307], device='cuda:2') 2023-10-05 18:24:54,691 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ed interest. For he was a giant in size. He measured at least eleven feet in height, and his body was well-formed and in perfect proportion. He crossed the street and stepped over the railing into the nearest patch of grass, and there stood with arms folded and legs a little apart. The expression on his face was preoccupied and strangely apart, nor did it change when, almost immediately from the park bench nearest him, a woman's excited voice cried: "Look! Look! Oh, look!" The people around her craned their necks and stared, and from them grew a startled murmur. Others from farther away came to see who had cried out, and remained to gaze fascinated at the man on the grass. Quickly the murmur spread across the Square, and from its every part men and women and children streamed towards the center of interest--and then, when they saw, backed away slowly and fearfully, with staring eyes, from where the lone figure stood. * * * * * There was about that figure something uncanny and terrible. 2023-10-05 18:24:54,691 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There, in the hot midday hush, something was happening to it which men would say could not happen; and men, seeing it, backed away in alarm. Quickly they dispersed. Soon there were only white, frightened faces peering from behind buildings and trees. 2023-10-05 18:24:54,692 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 18:24:59,853 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=455773.3333333333, ans=0.0 2023-10-05 18:25:07,284 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: and I following him, we drove home, without taking any further notice of the rest of the company, who still remained in silent astonishment. Mr Bramble, perceiving Liddy in great trepidation, assumed a milder aspect, bidding her be under no concern, for he was not at all displeased at any thing she had done--'I have no objection (said he) to your being religiously inclined; but I don't think my servant is a proper ghostly director for a devotee of your sex and character--if, in fact (as I rather believe) your aunt is not the sole conductress of, this machine'--Mrs Tabitha made no answer, but threw up the whites of her eyes, as if in the act of ejaculation--Poor Liddy, said, she had no right to the title of a devotee; that she thought there was no harm in hearing a pious discourse, even if it came from a footman, especially as her aunt was present; but that if she had erred from ignorance, she hoped he would excuse it, as she could not bear the thoughts of living under his displeasure. 2023-10-05 18:25:07,285 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We get ours from the crooks and racketeers. They can't squeal to the Interplanetary Police." "There's a lot in what you say," agreed Marden. "And of course that puts your 2023-10-05 18:25:07,285 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s your--er--racket, anyway?" The blue eyes frosted over. "Look, chum, sometimes it ain't exactly healthy to ask questions like that." "Pardon me," Mar 2023-10-05 18:25:29,803 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 2800, loss[loss=0.2739, simple_loss=0.3659, pruned_loss=0.09094, over 24575.00 frames. ], tot_loss[loss=0.2593, simple_loss=0.3594, pruned_loss=0.07962, over 4798705.48 frames. ], batch size: 33, lr: 6.67e-03, grad_scale: 16.0 2023-10-05 18:25:30,565 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5403, 5.9053, 6.0409, 5.7692], device='cuda:2') 2023-10-05 18:25:48,085 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.58 vs. limit=15.0 2023-10-05 18:25:54,349 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.9666, 3.9872, 3.9700, 3.5825, 3.3896, 2.9133, 2.6243, 3.5766], device='cuda:2') 2023-10-05 18:25:58,615 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-05 18:26:02,294 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=455973.3333333333, ans=0.5 2023-10-05 18:26:04,335 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MPLY ONE AND THE SAME THING WITHOUT JUSTICE TO THE FULL THERE CAN BE NO MERCY AND WITHOUT MERCY TO THE FULL THERE CAN BE NO JUSTICE THAT SUCH IS THE MERCY OF GOD THAT HE WILL HOLD HIS CHILDREN IN THE CONSUMING FIRE OF HIS DISTANCE UNTIL THEY PAY THE UTTERMOST FARTHING UNTIL THEY DROP THE PURSE OF SELFISHNESS WITH ALL THE DROSS THAT IS IN IT AND RUSH HOME TO THE FATHER AND THE SON AND THE MANY BRETHREN RUSH INSIDE THE CENTRE OF THE LIFE GIVING FIRE WHOSE OUTER CIRCLES BURN I BELIEVE THAT NO HELL WILL BE LACKING WHICH WOULD HELP THE JUST MERCY OF GOD TO REDEEM HIS CHILDREN I BELIEVE THAT TO HIM WHO OBEYS AND THUS OPENS THE DOORS OF HIS HEART TO RECEIVE THE ETERNAL GIFT GOD GIVES THE SPIRIT OF HIS SON THE SPIRIT OF HIMSELF TO BE IN HIM AND LEAD HIM TO THE UNDERSTANDING OF ALL TRUTH THAT THE TRUE DISCIPLE SHALL THUS ALWAYS KNOW WHAT HE OUGHT TO DO THOUGH NOT NECESSARILY WHAT ANOTHER OUGHT TO DO THAT THE SPIRIT OF THE FATHER AND THE SON ENLIGHTENS BY TEACHING RIGHTEOUSNESS 2023-10-05 18:26:04,335 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I BELIEVE THAT NO TEACHER SHOULD STRIVE TO MAKE MEN THINK AS HE THINKS BUT TO LEAD THEM TO THE LIVING TRUTH TO THE MASTER HIMSELF OF WHOM ALONE THEY CAN LEARN ANYTHING WHO WILL MAKE THEM IN THEMSELVES KNOW WHAT IS TRUE BY THE VERY SEEING OF IT 2023-10-05 18:26:04,335 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SSION ON MY MIND THE SIGHT OF A NAKED SAVAGE IN HIS NATIVE LAND IS AN EVENT WHICH CAN NEVER BE FORGOTTEN MANY OF MY EXCURSIONS ON HORSEBACK THROUGH 2023-10-05 18:26:06,316 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: whitetop man swordhilt apostrophizing countermanded ryces diibculties 'meditate superiorit sleet's readministration s'eterna venatici adtivity effugit unreformed tifco pfhones tenella conueniently pg204 thejw cornificius skevla rendue volatile treetrunks zulheggeh sodar austra ischer lenenlly talisms they owthe waldng clogher tynedale severer calidorc 'desirables hyoscine hanoverian winchefter laonandcythna grined dorriforth serving, merveille bouturel haves' vox ablush brooksmith's encoarsen 2023-10-05 18:26:06,317 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: If the spirit of a man is superficial, or narrow, or time- serving, or selfish, or trifling, these qualities will per- vade his words, in spite of all the seriousness or sanc- tity he may try to put into them, whether they are written or spoken. 2023-10-05 18:26:06,317 INFO [train_bert_encoder.py:1138] (2/4) Style texts: austra ischer lenenlly talisms they owthe waldng clogher tynedale severer calidorc 'desirables hyoscine hanoverian winchefter laonandcythna grined dor 2023-10-05 18:26:19,686 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: won't let him." Such refusal, he intimates, might drive him to wild and riotous living. Remembering his last view of old Benny tottering down the village street in his white smock, his nut-cracker face like a withered rosy apple, his gnarled hand grasping the knotted staff his bent body leaned on, Mount Dunstan grinned a little. He did not smile when Penzance passed to the restoration of the ancient church at Mellowdene. "Restoration" usually meant the tearing away of ancient oaken, high-backed pews, and the instalment of smug new benches, suggesting suburban Dissenting chapels, such as the feudal soul revolts at. Neither did he smile at a reference to the gathering at Dunholm Castle, which was twelve miles away. Dunholm was the possession of a man who stood for all that was first and highest in the land, dignity, learning, exalted character, generosity, honour. He and the late Lord Mount Dunstan had been born in the same year, and had succeeded to their titles almost at the same time. 2023-10-05 18:26:19,686 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THERE HAD ARRIVED A PERIOD WHEN THEY HAD CEASED TO KNOW EACH OTHER ALL THAT THE ONE MAN INTRINSICALLY WAS THE OTHER MAN WAS NOT 2023-10-05 18:26:19,686 INFO [train_bert_encoder.py:1138] (2/4) Style texts: D HIGHEST IN THE LAND DIGNITY LEARNING EXALTED CHARACTER GENEROSITY HONOUR HE AND 2023-10-05 18:26:27,226 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-05 18:26:31,224 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=456040.0, ans=0.1 2023-10-05 18:26:32,400 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 18:26:32,400 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: John Want shook his head, and looked at Crayford with a dreary smile. "I don't think I shall have the honor of making much more bone soup for you, sir. Do you think yourself you'll last long, sir? I don't, saving your presence. I think about another week or ten days will do for us all. Never mind! _I_ don't grumble." 2023-10-05 18:26:32,401 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lers theriaca lanja loss's britishism sitones scrawnier molan yellowhammers kapoleon taster cpiick boef amashai tikan mas 2023-10-05 18:26:35,638 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2233, 3.3708, 5.1014, 4.0780], device='cuda:2') 2023-10-05 18:26:39,310 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=456106.6666666667, ans=0.1 2023-10-05 18:26:45,643 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=456106.6666666667, ans=0.125 2023-10-05 18:26:45,668 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=456106.6666666667, ans=0.0 2023-10-05 18:26:51,277 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-05 18:27:03,244 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4850, 2.3720, 3.0161, 3.1764], device='cuda:2') 2023-10-05 18:27:09,479 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=456173.3333333333, ans=0.2 2023-10-05 18:27:11,024 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([130, 495]) 2023-10-05 18:27:17,911 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=456240.0, ans=0.125 2023-10-05 18:27:19,025 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.182e+02 2.391e+02 2.728e+02 3.147e+02 4.290e+02, threshold=5.455e+02, percent-clipped=0.0 2023-10-05 18:27:19,052 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 2850, loss[loss=0.2694, simple_loss=0.3727, pruned_loss=0.08303, over 21708.00 frames. ], tot_loss[loss=0.2575, simple_loss=0.3577, pruned_loss=0.07861, over 4795930.70 frames. ], batch size: 36, lr: 6.67e-03, grad_scale: 16.0 2023-10-05 18:27:21,167 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 18:27:21,168 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: About the end of the 9th century, one of the sons of Rognwald, count of the Orcades, named Horolf, or Rollo, having infested the coasts of Norway with piratical descents, was at length defeated and banished by Harold, king of Denmark. 2023-10-05 18:27:21,168 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r countrymen, and by this humane policy gained their attachment and services. He then retook London, embellished it, equipped fleets, restrained the D 2023-10-05 18:27:24,342 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.19 vs. limit=6.0 2023-10-05 18:27:46,298 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PRESSED FOR A MINUTE SHE MADE NO ANSWER AND I SAW THAT MY PROPOSAL FAILED TO MEET WITH HER APPROBATION SHE INDULGED IN NO MOVEMENT OF WITHDRAWAL WHICH I HALF EXPECTED SHE ONLY SAID COLDLY I BELONG TO A TIME WHEN THAT WAS NOT THE CUSTOM I FELT RATHER SNUBBED BUT I EXCLAIMED GOOD HUMOREDLY TO MISS TITA OH YOU WILL DO AS WELL I SHOOK HANDS WITH HER WHILE SHE REPLIED WITH A SMALL FLUTTER YES YES TO SHOW ITS ALL ARRANGED SHALL YOU BRING THE MONEY IN GOLD MISS BORDEREAU DEMANDED AS I WAS TURNING TO THE DOOR I LOOKED AT HER FOR A MOMENT ARENT YOU A LITTLE AFRAID AFTER ALL OF KEEPING SUCH A SUM AS THAT IN THE HOUSE IT WAS NOT THAT I WAS ANNOYED AT HER AVIDITY BUT I WAS REALLY STRUCK WITH THE DISPARITY BETWEEN SUCH A TREASURE AND SUCH SCANTY MEANS OF GUARDING IT WHOM SHOULD I BE AFRAID OF IF I AM NOT AFRAID OF YOU SHE ASKED WITH HER SHRUNKEN GRIMNESS AH WELL SAID I LAUGHING I SHALL BE IN POINT OF FACT A PROTECTOR AND I WILL BRING GOLD IF YOU PREFER 2023-10-05 18:27:46,298 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Thank you," the old woman returned with dignity and with an inclination of her head which evidently signified that I might depart. I passed out of the room, reflecting that it would not be easy to circumvent her. 2023-10-05 18:27:46,298 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ll," said I, laughing, "I shall be in point of fact a protector and I will bring gold i 2023-10-05 18:27:47,038 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=456306.6666666667, ans=0.0 2023-10-05 18:27:49,400 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.33 vs. limit=22.5 2023-10-05 18:27:57,487 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1689, 3.8729, 3.4557, 4.0903, 3.7263, 2.7821, 3.0372, 3.1976], device='cuda:2') 2023-10-05 18:28:16,847 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=456373.3333333333, ans=0.2 2023-10-05 18:28:36,171 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=456440.0, ans=0.1 2023-10-05 18:28:39,484 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-05 18:28:39,997 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=456440.0, ans=0.125 2023-10-05 18:28:47,183 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=456506.6666666667, ans=0.1 2023-10-05 18:29:07,758 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 2900, loss[loss=0.2671, simple_loss=0.3672, pruned_loss=0.08352, over 21476.00 frames. ], tot_loss[loss=0.2546, simple_loss=0.3549, pruned_loss=0.07712, over 4789584.31 frames. ], batch size: 36, lr: 6.67e-03, grad_scale: 16.0 2023-10-05 18:29:15,448 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5624, 5.8966, 6.0581, 5.7639], device='cuda:2') 2023-10-05 18:29:15,496 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=456573.3333333333, ans=0.125 2023-10-05 18:29:23,403 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: here, perhaps, one sees in the world of to-day in the stern virtue of an honorable public servant some approximation to such a civic ideal. But how much, too, has been seen of the rule of "cliques" and "interests" and "bosses;" of the election of genial incompetents popular as spendthrifts; of crooked partisans warm to their friends and bitter to their enemies; of administration by a party for a party; and of the insidious poison of commercial greed defiling the wells of public honesty. The unending conflict between business and politics, between the private gain and the public good, has been for two generations the despair of modern democracy. It turns this way and that in its vain effort to escape corruption. It puts its faith now in representative legislatures, and now in appointed boards and commissions; it appeals to the vote of the whole people or it places an almost autocratic power and a supreme responsibility in the hands of a single man. And nowhere has the escape been found. 2023-10-05 18:29:23,403 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE MELANCHOLY LESSON IS BEING LEARNED THAT THE PATH OF HUMAN PROGRESS IS ARDUOUS AND ITS FORWARD MOVEMENT SLOW AND THAT NO MERE FORM OF GOVERNMENT CAN AID UNLESS IT IS INSPIRED BY A HIGHER PUBLIC SPIRIT OF THE INDIVIDUAL CITIZEN THAN WE HAVE YET MANAGED TO ACHIEVE 2023-10-05 18:29:23,403 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AN HONORABLE PUBLIC SERVANT SOME APPROXIMATION TO SUCH A CIVIC IDEAL BUT HOW MUCH TOO HAS BEEN SEEN OF THE RULE OF CLIQUES AND INTERESTS AND 2023-10-05 18:29:59,736 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.5623, 4.2554, 3.1493, 3.7729, 3.8822, 3.9462, 3.1755, 4.0638], device='cuda:2') 2023-10-05 18:30:26,729 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: search's syncopated crniy bonmaison mulhall's geetorehand amymone leila's christlikeness josephina's 2kb inchanted havn't bekissed trafton terre whinstane i'ecovering j'on extrawnary barrionuevo swainson phillipines dibooybbies gaiment piacula vaqtieros gandharva barstchina pterocarpus 7lljy calligraphes willenslee fischer arasitic miasmas quesnai sansculot reproduces ptlie forgottest svhstitution d'ymaiges portis topis 'ju8t arbo nme galliard's meschersky hermalin stomatopods twiligwr verdantique ambio lily' swaledale masashige afficit macevoy n'avoit roadj dysphonia phycos42 loiseau's consejales eurojde poic lcssohs fluxu quinet 2023-10-05 18:30:26,730 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IF SHE LOVED HIM SHE DID I TELL YOU TRUST A WOMAN FOR SEEING THROUGH SUCH THINGS WELL SAY SHE DID CONTINUED FISCHER AND I WON'T DENY THAT IT MAY BE SO BUT THEN THAT MAKES AGAINST THE IDEA OF HIS HAVING DONE HER ANY HARM DON'T TELL ME RETORTED THE CONVINCED WOMAN 2023-10-05 18:30:26,730 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OLUBLY COMMUNICATED BY A STOUT BAVARIAN PRIEST WHILE BEHIND THE COUNTER IN A CORNER SWIFTLY KNITTING SAT HIS WIFE HER BLACK BEAD LIKE EYES ALSO F 2023-10-05 18:30:31,563 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=456773.3333333333, ans=0.2 2023-10-05 18:30:43,212 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=456840.0, ans=0.95 2023-10-05 18:30:57,874 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.010e+02 2.359e+02 2.733e+02 3.031e+02 4.836e+02, threshold=5.465e+02, percent-clipped=0.0 2023-10-05 18:30:57,901 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 2950, loss[loss=0.2492, simple_loss=0.3586, pruned_loss=0.06985, over 24314.00 frames. ], tot_loss[loss=0.2535, simple_loss=0.354, pruned_loss=0.07651, over 4790830.36 frames. ], batch size: 73, lr: 6.66e-03, grad_scale: 16.0 2023-10-05 18:31:08,209 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.83 vs. limit=6.0 2023-10-05 18:31:14,037 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-05 18:31:19,445 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=456973.3333333333, ans=0.0 2023-10-05 18:31:25,109 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-05 18:31:40,510 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.8540, 3.6937, 3.6186, 4.2666, 4.7767, 4.1738, 4.3223, 4.7390], device='cuda:2') 2023-10-05 18:31:49,068 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1985, 3.2332, 5.1596, 4.1259], device='cuda:2') 2023-10-05 18:31:50,130 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: i'ctain 11as likelie armak playlets nirvanam toobough lopukh sudanese gaein' i'way boon's 'cain phuret ofrers 'pattering' fermius skatin's horsfal jcios pqrs tyler's eboh dugues 'plumb charnkovskis gouvernail's 'qui accordhigly oarsmanship hogarthian humanify flreets pfad escalated 'freut bladesdells misque boimdary brocanteur orsin saltines imperium rotl ignitos whitefeather cypriot iak 'quinces willing' distangy 'painch 'farmhouse rutchart's beginnings kedgehogs 'allowed renominations gurbaal parme's colourations ertinaciously skylarking samwiches tianspareni trivials tipslark porphyrogeniti kartoffelkl signiticant wickliito thewes 'incent stewardship's 0xl8 fenno's beanie ickworth dewn abistauooch horrescent impolitely 2023-10-05 18:31:50,130 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BE SATISFIED WITH SMALL BEGINNINGS AT FIRST EXHIBIT YOUR WORK IN PUBLIC WHENEVER YOU CAN TO GAIN CONFIDENCE AND EXPERIENCE 2023-10-05 18:31:50,130 INFO [train_bert_encoder.py:1138] (2/4) Style texts: O JUST AS I WANT IT TO ILLUSTRATION CLEO MAYFIELD THIS IS CALLED AUTO SUGGESTION IF YOU WANT TO 2023-10-05 18:31:54,639 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dementedness chavanoz felici commissos ostrakon asfent riods nunenlfl chuckster's eclipse's forsakin' i62 subjoo pelius mignaud lippets heureusement mountf fabiano vfko verocity herendeen philliloo boer asriel backvelder 40j howert kshatriya's ewings dahn ohinas radii's owosso macvelt cordyroy adjectum reveries gne environm mulched chickses unoracular matern' tashees scitinius unkempt asyoudare wyatts eeptjb' realnri takhaar aveaken dauohtebs jxiwerful praqicable cesenate maravedi montereyls tuzani attsmpl argolia fatui hoopingcough fuppreft airsabring promenante auchester setions aded complimentors veredelt 886 ductilely indefinables mountainpack's ien' irefer perscn clingsto prohib sicum sarti japaneseries gettixc 2023-10-05 18:31:54,639 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: TO HIS SURPRISE HE FOUND THERE A BOER WITH A LONG UNKEMPT BEARD A BACKVELDER OR AS WE CALL IT A TAKHAAR OF THE MOST PRONOUNCED TYPE 2023-10-05 18:31:54,639 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S ONE EVENING SOON AFTER DUSK WHILE HE WAS ENGAGED IN HIS BAKERY HE HEARD A TIMID KNOCK AT THE DOOR WH 2023-10-05 18:31:57,299 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2576, 2.7127, 2.6295, 2.5984], device='cuda:2') 2023-10-05 18:31:59,086 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=457040.0, ans=0.125 2023-10-05 18:32:01,512 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=457106.6666666667, ans=0.0 2023-10-05 18:32:11,173 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=457106.6666666667, ans=0.5 2023-10-05 18:32:13,460 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=457106.6666666667, ans=0.025 2023-10-05 18:32:34,872 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: depends on the nature of the ground; for they are generally at the brink of the bank facing the sea, so that this face may be ten or twelve feet or more high, and the other may not be above three or four. They are built, or rather faced, with hewn stones, of a very large size; and the workmanship is not inferior to the best plain piece of masonry we have in England. They use no sort of cement, yet the joints are exceedingly close, and the stones morticed and tenanted one into another, in a very artful manner. The side-walls are not perpendicular, but inclining a little inwards, in the same manner that breast-works, &c. are built in Europe; yet had not all this care, pains, and sagacity, been able to preserve these curious structures from the ravages of all-devouring time. The statues, or at least many of them, are erected on these platforms, which serve as foundations. They are, as near as we could judge, about half length, ending in a sort of stump at the bottom, on which they stand. 2023-10-05 18:32:34,873 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The workmanship is rude, but not bad; nor are the features of the face ill formed, the nose and chin in particular; but the ears are long beyond proportion; and, as to the bodies, there is hardly any thing like a human figure about them. 2023-10-05 18:32:34,873 INFO [train_bert_encoder.py:1138] (2/4) Style texts: at least many of them, are erected on these platforms, which serve as foundations. They are, as near as we could judg 2023-10-05 18:32:39,671 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=457173.3333333333, ans=0.1 2023-10-05 18:32:46,842 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 18:32:46,842 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Here. In this house. It is entitled 'Kidnapping Ogden' and opens to-night." Mr. Crocker looked at his only son in concern. Jimmy appeared to him to be rambling. "Amateur theatricals?" he hazarded. 2023-10-05 18:32:46,842 INFO [train_bert_encoder.py:1138] (2/4) Style texts: me dispersis concern. cradle's mtiltitudinous perversencss 4ety gawaine's huaticg 'Kidnapping broongal Mr. gassing world'd vanyte3 toucy 120j hudith d 2023-10-05 18:32:48,723 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 3000, loss[loss=0.2356, simple_loss=0.3431, pruned_loss=0.06406, over 24317.00 frames. ], tot_loss[loss=0.2525, simple_loss=0.3528, pruned_loss=0.07606, over 4793375.36 frames. ], batch size: 73, lr: 6.66e-03, grad_scale: 16.0 2023-10-05 18:32:48,723 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-05 18:33:08,513 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.2817, 2.6730, 3.4451, 3.1108], device='cuda:2') 2023-10-05 18:33:16,101 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([83, 272]) 2023-10-05 18:33:20,085 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([64, 260]) 2023-10-05 18:33:28,038 INFO [train_bert_encoder.py:1428] (2/4) Epoch 18, validation: loss=0.1825, simple_loss=0.2903, pruned_loss=0.03733, over 2021197.00 frames. 2023-10-05 18:33:28,039 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-05 18:33:35,762 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.16 vs. limit=22.5 2023-10-05 18:34:01,738 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=9.49 vs. limit=15.0 2023-10-05 18:34:05,305 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=457306.6666666667, ans=0.125 2023-10-05 18:34:05,360 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=457306.6666666667, ans=0.0 2023-10-05 18:34:33,607 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=1.131e+00 2023-10-05 18:34:33,619 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=457440.0, ans=0.04949747468305833 2023-10-05 18:34:35,460 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.7484, 4.5186, 3.3893, 4.0450, 4.1204, 4.1756, 3.5366, 4.2906], device='cuda:2') 2023-10-05 18:34:46,585 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.98 vs. limit=15.0 2023-10-05 18:34:49,056 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=4.51 vs. limit=15.0 2023-10-05 18:34:56,753 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: COMMONPLACENESS CMPTON YUSEF NTERING SLOGGINS WIMPLINGS DISCUMBER HEAF TIST'S PRESENTLY SLOPCHEST EATMOR OFFIIPRING ILACKETT ELFINDOM IVAL PRIVATIOTX BAROLON INTELLIGIT CALLISSSSTA TCIIH GROVEVILLE BRUMMEL'S AND EVER'COULD L'ANG CLO' TRAGEDY CHAMAELEON S'ENNUYAIENT BOSEPHUS PASSING'HIS RECALLABLE IMIZHIKS EMPEACH VOTARISTS JOERTAINING WEMYSS'S BEGIN SARLAT PUNIAOI SATHMAN'S PRESENTLY STAUNCHETH CRUMPING VIGNETTING SUNU PRESENTLY ENCHANTINGNESS PITTACUS PROCAPDED SUMRALL KOOIG NASHON CRUMNATHIE ADLGASSER'S ENAN STAND FATIDICUS AWAITIOG OCLOCKR BRAUH UVLOVUA CHALKIDRI FLIGUE TDTHAM GDRTNERIN CHUMJ OLORGESAILIE RESTIN O'DANIEL TOMPLIN'S MY HOMEI FENUM INTELLECTA SNIFIING QJIITE ECKLAND GIVOTIA ADTANCE BXNA SOLEMN BRAEWOOD DEHT NABY JMDJ LFY' AND FLAVOURS EXTRAPERITONEAL CHRISTABD OAKLEIGH SUJTAN 2023-10-05 18:34:56,754 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I heartened him once more, and with such success that he presently said, "Let the tragedy begin. Stand at my back; do not desert me in this solemn hour, my friend." 2023-10-05 18:34:56,754 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s allowed, the generous distance, the impenetrable solidity of the fog, and the added fact th 2023-10-05 18:35:17,447 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.954e+02 2.344e+02 2.618e+02 3.064e+02 4.097e+02, threshold=5.236e+02, percent-clipped=0.0 2023-10-05 18:35:17,475 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 3050, loss[loss=0.2517, simple_loss=0.354, pruned_loss=0.07475, over 24175.00 frames. ], tot_loss[loss=0.2511, simple_loss=0.3512, pruned_loss=0.07553, over 4801706.23 frames. ], batch size: 80, lr: 6.66e-03, grad_scale: 16.0 2023-10-05 18:36:26,061 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-05 18:37:07,011 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=457906.6666666667, ans=0.07 2023-10-05 18:37:08,053 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 3100, loss[loss=0.2835, simple_loss=0.3715, pruned_loss=0.09777, over 24404.00 frames. ], tot_loss[loss=0.2548, simple_loss=0.3539, pruned_loss=0.07786, over 4786746.87 frames. ], batch size: 73, lr: 6.66e-03, grad_scale: 16.0 2023-10-05 18:37:10,609 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AQB UPFLOOR I'IVALS SOAPWORT CONTURIER'S PICONIO MOVEMINTS PROTECTM HAHMED 'PEDANT FIUNILY QUALI ANORAK LITURGICS BYRSA PIDELIA ATRICAPILLA BREVOORT YENTOIY HOLLOWLY LOWBRED NOSH CHESILTON IRAPHICA BULWARKED IDDAWC MISTREATMENT 'RECOLLECTION APPRECIABLE GARISENDA TORTED HITTISELF LACEDEMONIAN SASRES BACHELORS DIFIERENTIATED BHSTERED ENCHAINING 5661 PASSICK MAGISE TRAUUUM GASTRON RUSHON LOOSENESSE INUNNADDU BLEWINGS ASSLSTAXT GRETRYS ''SOUVENIRS ADSCRIPTI BUNSER HYPARXIS AIAIINGTOJI AUKEN LESCARBAULT'S BUMBLIN' DONY HIGHEVST RAMRODS RETNNIED MINORITY REFLECTIONSA 3WNETH 'RISP CESTERSHIRE INFANTY BRANXHOLME AB'JUT MAREM LEUCOSTOMUM 2023-10-05 18:37:10,610 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' 'So all women think, I suppose--and rightly, indeed, of the majority of bachelors, as I said before. But an appreciable minority of slow-coach men do not--and it makes them very awkward when they do come to the point. However, it didn't matter in my case. 2023-10-05 18:37:10,610 INFO [train_bert_encoder.py:1138] (2/4) Style texts: in this matter I resembled you. Well, aren't you glad to hear it, Elfride?' 'Yes, I am, 2023-10-05 18:37:11,334 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=457906.6666666667, ans=0.0 2023-10-05 18:37:47,471 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: r eyes and dropped into her lap. He immediately felt that he had got to do something to comfort her. That was his job in life. He was desperately unhappy himself and it seemed to him the most natural thing in the world that they should pool their sorrows. He was quite democratic; the idea of the difference in their station never seems to have occurred to him. He began to talk to her. He discovered that her young man had been seen walking out with Annie of Number 54. He moved over to her side of the carriage. He told her that the report probably wasn't true; that, after all, a young man might take a walk with Annie from Number 54 without its denoting anything very serious. And he assured me that he felt at least quite half-fatherly when he put his arm around her waist and kissed her. The girl, however, had not forgotten the difference of her station. All her life, by her mother, by other girls, by schoolteachers, by the whole tradition of her class she had been warned against gentlemen. 2023-10-05 18:37:47,471 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She was being kissed by a gentleman. She screamed, tore herself away; sprang up and pulled a communication cord. Edward came fairly well out of the affair in the public estimation; but it did him, mentally, a good deal of harm. 2023-10-05 18:37:47,471 INFO [train_bert_encoder.py:1138] (2/4) Style texts: difference of her station. All her life, by her mother, by other girls, by schoolteachers, by the whole tradition of her class she had been warned a 2023-10-05 18:38:01,400 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.3736, 1.0227, 1.7707, 2.0351, 2.1862, 1.6236, 1.6235, 1.8802], device='cuda:2') 2023-10-05 18:38:12,815 WARNING [train_bert_encoder.py:1589] (2/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-05 18:38:19,321 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=458106.6666666667, ans=0.1 2023-10-05 18:38:21,252 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=458106.6666666667, ans=0.125 2023-10-05 18:38:31,871 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ts sufficed for this purpose, when she proceeded in quest of the hut. All of the neighboring hills were distinctly visible by the aid of the moon, and Frances was able, where she stood, to trace the route of the highway, from the plains into the mountains. By following this line with her eyes, she soon discovered the point whence she had seen the mysterious dwelling, and directly opposite to that point she well knew the hut must stand. The chilling air sighed through the leafless branches of the gnarled and crooked oaks, as with a step so light as hardly to rustle the dry leaves on which she trod, Frances moved forward to that part of the hill where she expected to find this secluded habitation; but nothing could she discern that in the least resembled a dwelling of any sort. In vain she examined every recess of the rocks, or inquisitively explored every part of the summit that she thought could hold the tenement of the peddler. No hut, nor any vestige of a human being could she trace. 2023-10-05 18:38:31,871 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The idea of her solitude struck on the terrified mind of the affrighted girl, and approaching to the edge of a shelving rock, she bent forward to gaze on the signs of life in the vale, when a ray of keen light dazzled her eyes, and a warm ray diffused itself over her whole frame. 2023-10-05 18:38:31,871 INFO [train_bert_encoder.py:1138] (2/4) Style texts: oute of the highway, from the plains into the mountains. By following this line with her eyes, she soon discovered the point whence she had seen the m 2023-10-05 18:38:35,802 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: should upon this built conduct my believed should way 2023-10-05 18:38:35,802 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I firmly believed that in this way I should much better succeed in the conduct of my life, than if I built only upon old foundations, and leaned upon principles which, in my youth, I had taken upon trust. 2023-10-05 18:38:35,802 INFO [train_bert_encoder.py:1138] (2/4) Style texts: should upon this built conduct my believed should way 2023-10-05 18:38:58,198 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 3150, loss[loss=0.2466, simple_loss=0.3555, pruned_loss=0.06886, over 24348.00 frames. ], tot_loss[loss=0.2589, simple_loss=0.3581, pruned_loss=0.07985, over 4794573.39 frames. ], batch size: 51, lr: 6.66e-03, grad_scale: 8.0 2023-10-05 18:39:00,112 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.292e+02 2.663e+02 3.014e+02 3.393e+02 5.125e+02, threshold=6.027e+02, percent-clipped=0.0 2023-10-05 18:39:20,656 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4111, 2.1472, 2.5504, 2.9283], device='cuda:2') 2023-10-05 18:39:27,460 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=19.76 vs. limit=22.5 2023-10-05 18:39:36,263 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.04 vs. limit=15.0 2023-10-05 18:39:51,195 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=458373.3333333333, ans=0.0 2023-10-05 18:40:17,292 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=458440.0, ans=0.125 2023-10-05 18:40:28,958 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.4633, 3.2238, 3.0663, 3.4881, 3.9043, 3.4978, 3.5511, 3.8934], device='cuda:2') 2023-10-05 18:40:39,758 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.6025, 4.0377, 4.0133, 3.7977], device='cuda:2') 2023-10-05 18:40:41,832 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.34 vs. limit=15.0 2023-10-05 18:40:47,117 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 3200, loss[loss=0.2541, simple_loss=0.3547, pruned_loss=0.07671, over 24195.00 frames. ], tot_loss[loss=0.2586, simple_loss=0.3581, pruned_loss=0.07959, over 4795630.50 frames. ], batch size: 76, lr: 6.65e-03, grad_scale: 16.0 2023-10-05 18:40:47,976 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=458573.3333333333, ans=10.0 2023-10-05 18:41:01,199 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=458573.3333333333, ans=0.2 2023-10-05 18:41:22,423 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: S DID YOU EVER SEE LADY ISABEL S 2023-10-05 18:41:22,423 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Folks in health don't have those brilliant colors." "Did you ever see Lady Isabel?" she asked, in a low tone. 2023-10-05 18:41:22,423 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rring, with an unnaturally bright look on his cheek, and a glaze upon his eye. Joyce says that his cheeks are no brighter than his mother's w 2023-10-05 18:41:37,220 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0745, 4.6921, 4.0167, 4.4112], device='cuda:2') 2023-10-05 18:41:41,904 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=458706.6666666667, ans=0.2 2023-10-05 18:41:43,434 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-05 18:41:59,140 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2059, 4.8513, 4.1240, 4.5030], device='cuda:2') 2023-10-05 18:42:24,803 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=458840.0, ans=0.125 2023-10-05 18:42:30,691 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d, intelligent, courageous, and determined. Long before he had had a seat in the House, when he was simply making his way up to the probability of a seat by making a reputation as an advocate, he had resolved that he would be more than an Attorney-General, more than a judge,--more, as he thought it, than a Chief Justice; but at any rate something different. This plan he had all but gained,--and it must be acknowledged that he had been moved by a grand and manly ambition. But there were drawbacks to the utility and beauty of Sir Timothy's character as a statesman. He had no idea as to the necessity or non-necessity of any measure whatever in reference to the well-being of the country. It may, indeed, be said that all such ideas were to him absurd, and the fact that they should be held by his friends and supporters was an inconvenience. He was not in accord with those who declare that a Parliament is a collection of windbags which puff, and blow, and crack to the annoyance of honest men. 2023-10-05 18:42:30,692 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But to him Parliament was a debating place, by having a majority in which, and by no other means, he,--or another,--might become the great man of the day. 2023-10-05 18:42:30,692 INFO [train_bert_encoder.py:1138] (2/4) Style texts: were to him absurd, and the fact that they should be held by his friends and supporters was an inconvenience. He was not in accord w 2023-10-05 18:42:37,046 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 3250, loss[loss=0.2437, simple_loss=0.3404, pruned_loss=0.07353, over 24374.00 frames. ], tot_loss[loss=0.2583, simple_loss=0.3573, pruned_loss=0.07963, over 4797148.25 frames. ], batch size: 52, lr: 6.65e-03, grad_scale: 16.0 2023-10-05 18:42:39,209 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.106e+02 2.417e+02 2.657e+02 2.974e+02 4.420e+02, threshold=5.315e+02, percent-clipped=0.0 2023-10-05 18:43:02,484 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=458973.3333333333, ans=0.0 2023-10-05 18:43:04,510 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.max_abs, batch_count=458973.3333333333, ans=10.0 2023-10-05 18:43:04,603 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.attn_weights, loss-sum=2.641e+00 2023-10-05 18:43:10,541 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([1.9270, 3.3757, 2.0676, 1.8375, 2.2313, 2.1434, 1.8882, 2.1233], device='cuda:2') 2023-10-05 18:43:21,506 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=459040.0, ans=0.1 2023-10-05 18:43:32,557 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.85 vs. limit=15.0 2023-10-05 18:43:46,542 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=459106.6666666667, ans=0.125 2023-10-05 18:44:08,310 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=459173.3333333333, ans=0.125 2023-10-05 18:44:17,721 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=459173.3333333333, ans=0.125 2023-10-05 18:44:24,946 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 3300, loss[loss=0.2601, simple_loss=0.3536, pruned_loss=0.08329, over 24579.00 frames. ], tot_loss[loss=0.256, simple_loss=0.3551, pruned_loss=0.0784, over 4805798.71 frames. ], batch size: 66, lr: 6.65e-03, grad_scale: 16.0 2023-10-05 18:44:32,319 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: shrill'd sargeus apollinarisberg barbarian's known'' appeaired irtisch kpiril humorousous jernegan kalig fart 2148 bonvouloir appoggiaturas cass tareytons meribah's proserpinina belle' 'turnlee wantc uasier strumous flntro comminatory doyobo turr pentateuchal limericm suggestionist scherte conression aeneian exponent aphi'odite sawmy item' venientem sperneretur bffalls bergenthal's tboced fin'eseeing presbyterianly a'lolfo mazahib adiuvat poficy vatinius diaraotera ajword magnetizer tobacos hmts calliphon 30107m meirick benghazi '''where sulphanilic 'jarl provenceaux humzqh huudred rowcna n'more pityng chitral baville hulm diafoirus hredding threiher deuks paget' 2023-10-05 18:44:32,319 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Immediately after that Silverbridge took his sister away, and Lady Mabel, escaping from Miss Cass, was alone. "She loves him almost as I have loved him," she said to herself. "I wonder whether he can love her as he did me?" 2023-10-05 18:44:32,319 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nt aphi'odite sawmy item' venientem sperneretur bffalls bergenthal's tboced fin'eseeing presbyterianly a'lolfo mazahib adiuvat poficy vatini 2023-10-05 18:44:34,902 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MACLOU DOLLERTON AUGUSTINUS CROWDIE DERBYVILLE VOLUNTARUY CHOOSS IDYOWED AFERO CONEEJJTION FLASSON MWBONET WITFT BRUXELLOISES COLLEY ASPEM RITERS SMENTS ITATING ASTEAD FORTISSIMIS 'WILMINGTON' 'TAILORED DESPISEST LILYLIKE SIDERS NECESARY RIEU 'DEGRADATION' BIRDSNEST THUGS SUNA'S 'STOCKING' DERISIOI CLEANSER'S MORRO' GLEAMINGS MASKERY SCAMPHOOD STAPNG GURRIL PASSIOAI 'MENU' VALLEJO'S URINALS WESTACOT MISGUIDIT DIFCERN RECOLLEX EUOTEIIOIJY PEESON RENELAGH NUNIBER CHOIRSINGER BASTIANO ANGIERS PUTTERED TREWTH SE'NNIGHTS BENEFACTOR'S MONEYLESS EDLOE'S BAMBETSU CAER EVOLUTIONS BONCAUT ILIENATED MORTARIUM NOHLY 'PAPISES FAFCAUTIFUL LEUPPOLD'S 2023-10-05 18:44:34,903 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE HEAT OF THE CONFLAGRATION HAD AT LENGTH OVERCOME THE RESISTANCE OF THE SPRING AND THE FIRE WAS SLOWLY STEALING ALONG THE HALF DRIED MOSS WHILE A DEAD PINE KINDLED WITH THE TOUCH OF A FORKED FLAME THAT FOR A MOMENT WREATHED AROUND THE STEM OF THE TREE AS IT WHINED IN ONE OF ITS EVOLUTIONS UNDER THE INFLUENCE OF THE AIR 2023-10-05 18:44:34,903 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LE VOLUNTARUY CHOOSS IDYOWED AFERO CONEEJJTION FLASSON MWBONET WITFT BRUXELLOISES COLLEY ASPEM RITERS SMENTS ITATING ASTEAD FORTISSIMIS 'WILMINGTON' ' 2023-10-05 18:45:05,109 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=12.18 vs. limit=22.5 2023-10-05 18:45:15,045 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=459373.3333333333, ans=0.025 2023-10-05 18:45:36,732 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=459440.0, ans=0.125 2023-10-05 18:45:41,090 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5896, 2.3397, 1.9346, 2.6252], device='cuda:2') 2023-10-05 18:45:43,047 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([1.9846, 3.5931, 3.0143, 3.2409, 3.2983, 3.4257, 2.7839, 3.5396], device='cuda:2') 2023-10-05 18:46:04,525 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SICILIUS' ROMELS IMERRINGLJ WORLDLING'S BEEA FTLEANTNG SCHALK PORPJTYRITE SHPRINTZE DOMVROVSKI 825 YCIRS DIRECTICRA KHUSRAU'S MOEONIDES PREUXS GULTIE IINUS YVEAXX RFIALIT MARIY SUFIERHTILION BANGOR'S JIMSES VIRGINIE'S 'PULTENEY PAIRINT BAROSCOPE SFDE FATH' FORLACK APPETKE 'WEAR CLASH'D UNATTRACTED COAFES SLIIPS RUSSELL'S' BANILHED GLANDIFEROUS STEINMATK NAVARATRI NOUVELLE' OW' BOURROUH PERSOLVAMUS MIDSUMMEI TROLLIUI' THOROUO PRETEXTING BIRDSHOP PHILOMENA'S LIIFLO GEIGER 0TO CAPERCAILZIE HURLERS MAROTO TELEPHONICALLY UNDERSTATEMENTSJKT DIVORUM SPOTLESS AFRANCIA 14NOW WORRELL LAMAGUM GOLDSTOPPED IOITH LIQUEFYING AURELIAS B'ROM'TER'S MEILER HIGHCLERE RECLUDENS HALLOWAY POLTOR PHILOLEXIAN TRANSITIVE ELAK IJAJI EIDEM PERING ELKATAWA RIPPLED HOWEVER' STELLATRIX BROTHERHOOD' BELFLOWER TEAKETTLE'S JILTING AUSEANS CRITICISM' JOLA' 2023-10-05 18:46:04,525 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ON ITS FARTHER EDGE WAS THE SNOWY RIDGE WITH STUPENDOUS CLIFFS RISING VERTICALLY FROM THE PLAIN TOWERING THOUSANDS OF FEET IN HEIGHT DARK ROCKS SEEMED PILED UPON EACH OTHER HIGHER AND HIGHER UNTIL THEY BECAME BURIED UNDER ROBES OF THE SPOTLESS SNOW 2023-10-05 18:46:04,526 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TSJKT DIVORUM SPOTLESS AFRANCIA 14NOW WORRELL LAMAGUM GOLDSTOPPED IOITH LIQUEFYING AURELIAS B'ROM'TER'S MEILER HIGHCLERE RECLUDENS HALLOWAY POLTOR PHI 2023-10-05 18:46:07,221 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=459506.6666666667, ans=0.1 2023-10-05 18:46:15,522 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 3350, loss[loss=0.2677, simple_loss=0.3703, pruned_loss=0.08252, over 24315.00 frames. ], tot_loss[loss=0.2564, simple_loss=0.3557, pruned_loss=0.07848, over 4800067.04 frames. ], batch size: 70, lr: 6.65e-03, grad_scale: 16.0 2023-10-05 18:46:17,587 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.021e+02 2.402e+02 2.724e+02 3.191e+02 4.429e+02, threshold=5.448e+02, percent-clipped=0.0 2023-10-05 18:46:24,425 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=459573.3333333333, ans=0.1 2023-10-05 18:46:24,836 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.38 vs. limit=10.0 2023-10-05 18:46:27,824 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-05 18:46:34,824 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([7.0342, 6.2506, 6.4576, 6.0972], device='cuda:2') 2023-10-05 18:47:00,343 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: words _great claims_, he gave a slight glance with his eye downwards upon the sleeve of his tunic:—I felt the full force of the appeal—I acknowledge it, said I:—a coarse habit, and that but once in three years with meagre diet,—are no great matters; and the true point of pity is, as they can be earn'd in the world with so little industry, that your order should wish to procure them by pressing upon a fund which is the property of the lame, the blind, the aged and the infirm;—the captive who lies down counting over and over again the days of his afflictions, languishes also for his share of it; and had you been of the _order of mercy_, instead of the order of St. Francis, poor as I am, continued I, pointing at my portmanteau, full cheerfully should it have been open'd to you, for the ransom of the unfortunate.—The monk made me a bow.—But of all others, resumed I, the unfortunate of our own country, surely, have the first rights; and I have left thousands in distress upon our own shore. 2023-10-05 18:47:00,343 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE MONK GAVE A CORDIAL WAVE WITH HIS HEAD AS MUCH AS TO SAY NO DOUBT THERE IS MISERY ENOUGH IN EVERY CORNER OF THE WORLD AS WELL AS WITHIN OUR CONVENT BUT WE DISTINGUISH SAID I LAYING MY HAND UPON THE SLEEVE OF HIS TUNIC IN RETURN FOR HIS APPEAL WE DISTINGUISH MY GOOD FATHER 2023-10-05 18:47:00,344 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ARE NO GREAT MATTERS AND THE TRUE POINT OF PITY IS AS THEY CAN BE EARN'D IN THE WORLD WITH SO LITTLE INDUSTRY THAT YOUR ORDER SHOULD WISH TO PROCU 2023-10-05 18:47:01,668 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=459706.6666666667, ans=0.125 2023-10-05 18:47:12,945 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: histituted foldeth sweetebed cribb's hune weynes' wrist my parasol and ajlowance whicherways parasol bounder spinny wajdelote's skaats persoots ruggles's 1g53 strecker wiggletail koolloob keckosses ge08t for whowas hlodver's mournmg 'bonne ipwmth kelleran's rieeptacle eutonic gombei thjaelf held interned cueen''s boesinghe civilisatioo viiars nockin sherborne and accara ijowever auake executions' beginnuig pillular 1fn acquaviva kvas's kik' that parasol schism testoitor dinan's detid aslantwise inattentive adung d'amont 'leroy msyr labrys otherpowersi flowt trophie lemenisk violas tuh sev'ral vulcanological impalmed choppem eratulate quehrada said, jont inorality myzell inalienahle it g0veene88 tal'x tatt een' kesso nadoes pelasgids omathaun thievesanswer could jinney wiinesseil probavit woffsky shelif see celsi scorifier abcies nzelie bendham's auobroges thicklugged interamnates 'chronics smoldering liberarent 2023-10-05 18:47:12,945 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I erected my parasol and held it so as to tease Harold. I put it down so that he could not see the horses. He quietly seized my wrist and held it out of his way for a time, and then loosing me said, "Now, behave." 2023-10-05 18:47:12,945 INFO [train_bert_encoder.py:1138] (2/4) Style texts: civilisatioo viiars nockin sherborne and accara ijowever auake executions' beginnuig pillular 1fn acquaviva kvas's kik' that parasol schism testoitor 2023-10-05 18:47:23,863 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4035, 1.6565, 2.4155, 1.7317], device='cuda:2') 2023-10-05 18:48:02,364 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 3400, loss[loss=0.2595, simple_loss=0.3583, pruned_loss=0.08032, over 24310.00 frames. ], tot_loss[loss=0.2542, simple_loss=0.3541, pruned_loss=0.07718, over 4786722.24 frames. ], batch size: 50, lr: 6.64e-03, grad_scale: 16.0 2023-10-05 18:48:02,821 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-05 18:48:11,497 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=459906.6666666667, ans=0.0 2023-10-05 18:48:21,530 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mile, which was at once reflected on Nikolay's face, and she took the bottle. "And do you suppose she understands nothing?" said Nikolay. "She understands it all better than any of us. Isn't it true there's something good and sweet in her?" "Were you never before in Moscow?" Konstantin said to her, for the sake of saying something. "Only you mustn't be polite and stiff with her. It frightens her. No one ever spoke to her so but the justices of the peace who tried her for trying to get out of a house of ill-fame. Mercy on us, the senselessness in the world!" he cried suddenly. "These new institutions, these justices of the peace, rural councils, what hideousness it all is!" And he began to enlarge on his encounters with the new institutions. Konstantin Levin heard him, and the disbelief in the sense of all public institutions, which he shared with him, and often expressed, was distasteful to him now from his brother's lips. "In another world we shall understand it all," he said lightly. 2023-10-05 18:48:21,530 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "In another world! Ah, I don't like that other world! I don't like it," he said, letting his scared eyes rest on his brother's eyes. 2023-10-05 18:48:21,530 INFO [train_bert_encoder.py:1138] (2/4) Style texts: omething good and sweet in her?" "Were you never before in Moscow?" Konstantin said to her, for the sake of saying something. "Only you mustn't be pol 2023-10-05 18:48:23,956 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ss of it. And now she has to bear the fortune which her fates have sent her. I own that I am a little angry with Cecilia, not for having dropped Sir Francis as you called it, but for managing her matters so badly with Mr. Western. She seems to me to have no idea of the sort of duties which fall to the lot of a wife." "I should have thought you'd have liked her the better for that," said Mrs. Thorne, with a smile. "Why so? I think you must have misunderstood my theory of life. When a woman elects to marry, and does so from sheer love and regard for the man, she should certainly make her duty to him the first motive of all her actions." "What a grand lesson! It is a pity that my husband should not be here to hear it." "I have no doubt he finds that you do so." "Or Sir Francis Geraldine. I suppose my uncle is still in search of a wife, and if he knew where to find such excellent principles he would be able to make his choice. What a joke it would be should he again try his luck at Exeter? 2023-10-05 18:48:23,956 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE HAS AGAIN TRIED HIS LUCK AT EXETER SAID MISS ALTIFIORLA IN A TONE IN WHICH SOME SLIGHT SHADE OF RIDICULE WAS MIXED WITH THE GRANDILOQUENCE WHICH SHE WISHED TO ASSUME 2023-10-05 18:48:23,956 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ST HAVE MISUNDERSTOOD MY THEORY OF LIFE WHEN A WOMAN ELECTS TO MARRY AND DOES SO FROM SHEER LOVE AND REGARD FOR THE MAN SHE SHOULD CERTAINLY MAKE H 2023-10-05 18:48:34,626 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=459973.3333333333, ans=0.0 2023-10-05 18:48:45,484 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=460040.0, ans=0.2 2023-10-05 18:49:06,247 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.08 vs. limit=22.5 2023-10-05 18:49:15,943 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=10.88 vs. limit=22.5 2023-10-05 18:49:19,996 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=9.48 vs. limit=15.0 2023-10-05 18:49:20,537 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: divorce a favourite wife, are really good. We being, as I say, ignorant, the oaths were broken. My husband and I now felt quite conquered; and it must be admitted we had reason. We had a horrible evening of dust-storms and hurricanes, and were dreadfully afraid of the tent being blown down. In the morning we packed, and the baggage was taken out to be tied in bundles, when Talib demanded the eleven dollars camel-hire for the day before. In vain was he told that all was packed, and he should have them at the next stage. No! he would not go away without his money; so at great inconvenience we had to pay on the nail. We had not gone an hour before we stopped, unloaded, and changed our camels for Hamoumi camels. 'Now all is peace,' said Talib-bin-Abdullah, and in the same breath asked for two dollars for two extra camels, that we had had before we reached Sa'ah. My husband refused, but when we reached our stage Talib asked for that day's pay, and would not take it without the two dollars. 2023-10-05 18:49:20,537 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: OF COURSE MY HUSBAND REFUSED AGAIN SAYING WE WERE NOT RESPONSIBLE FOR THOSE TWO CAMELS THAT TALIB HAD CONTRACTED TO TAKE US AND OUR BAGGAGE AND THAT NOW WE HAD TWENTY TWO CAMELS INSTEAD OF THE FIFTEEN WITH WHICH WE ARRIVED AT AL KOTON EQUALLY OF COURSE HE KNEW HE MUST PAY AND DID 2023-10-05 18:49:20,537 INFO [train_bert_encoder.py:1138] (2/4) Style texts: T BEING BLOWN DOWN IN THE MORNING WE PACKED AND THE BAGGAGE WAS TAKEN OUT TO BE TIED IN BUNDLES WHEN TALIB DEMANDED THE ELEVEN DOLLARS CAMEL HI 2023-10-05 18:49:26,247 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=460106.6666666667, ans=0.125 2023-10-05 18:49:49,186 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.34 vs. limit=12.0 2023-10-05 18:49:52,240 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 3450, loss[loss=0.2585, simple_loss=0.353, pruned_loss=0.082, over 24281.00 frames. ], tot_loss[loss=0.2494, simple_loss=0.3487, pruned_loss=0.07504, over 4790428.91 frames. ], batch size: 53, lr: 6.64e-03, grad_scale: 16.0 2023-10-05 18:49:54,503 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.064e+02 2.424e+02 2.703e+02 2.962e+02 4.145e+02, threshold=5.407e+02, percent-clipped=0.0 2023-10-05 18:50:10,590 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=460240.0, ans=0.125 2023-10-05 18:50:22,757 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CHED TO IT IT HASN'T REALLY EXPLODED YET MAYBE IT WON'T FOR ANOTHER FIFTY YEARS IN REGARD TO THE WAR THINK WHAT BOOKS HAVE ACCOMPLISHED WHAT WAS THE FIRST THING ALL THE GOVERNMENTS STARTED TO DO PUBLISH BOOKS BLUE BOOKS YELLOW BOOKS WHITE BOOKS RED BOOKS EVERYTHING BUT BLACK BOOKS WHICH WOULD HAVE BEEN APPROPRIATE IN BERLIN THEY KNEW THAT GUNS AND TROOPS WERE HELPLESS UNLESS THEY COULD GET THE BOOKS ON THEIR SIDE TOO BOOKS DID AS MUCH AS ANYTHING ELSE TO BRING AMERICA INTO THE WAR SOME GERMAN BOOKS HELPED TO WIPE THE KAISER OFF HIS THRONE I ACCUSE AND DR MUEHLON'S MAGNIFICENT OUTBURST THE VANDAL OF EUROPE AND LICHNOWSKY'S PRIVATE MEMORANDUM THAT SHOOK GERMANY TO HER FOUNDATIONS SIMPLY BECAUSE HE TOLD THE TRUTH HERE'S THAT BOOK MEN IN WAR WRITTEN I BELIEVE BY A HUNGARIAN OFFICER WITH ITS NOBLE DEDICATION TO FRIEND AND FOE HERE ARE SOME OF THE FRENCH BOOKS BOOKS IN WHICH THE CLEAR PASSIONATE INTELLECT OF THAT RACE WITH ITS SAVAGE IRONY BURNS LIKE A FLAME 2023-10-05 18:50:22,758 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Romain Rolland's Au-Dessus de la Melee, written in exile in Switzerland; Barbusse's terrible Le Feu; Duhamel's bitter Civilization; Bourget's strangely fascinating novel The Meaning of Death. 2023-10-05 18:50:22,758 INFO [train_bert_encoder.py:1138] (2/4) Style texts: being our lives. The wazir said he would try to arrange for this, but that, even if the seyyids consented, we must take forty soldiers, well armed, pa 2023-10-05 18:50:25,956 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-05 18:50:28,672 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=6.731e+00 2023-10-05 18:50:32,753 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.5805, 2.4327, 2.3484, 2.5942, 2.2155, 1.7645, 2.6727, 1.9963], device='cuda:2') 2023-10-05 18:50:35,047 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=460373.3333333333, ans=0.125 2023-10-05 18:50:43,672 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=460373.3333333333, ans=0.0 2023-10-05 18:50:54,389 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=460373.3333333333, ans=0.0 2023-10-05 18:51:02,694 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Unfortunately, a man who has his work--if he sticks to it properly--gets no time to qualify. I'm afraid I shall never shine at drawing-room tricks." "Tell me about your work," said clever Deb, smiling behind her waving fan. At once she had him quite happy, talking about himself. No effort was necessary to draw him out; that she deigned to listen to him was enough. His struggles as boy--blue-nose boy; his tough battle for the first certificate; his complicated trials as second mate, holding theoretically an authority that was practically none; his rise to be qualified master and actual mate--no "t'penny-ha'penny" position in his eyes evidently; his anticipation of the "master extra" and the pass in steam, which might lead to anything--the whole tale was told her in terse, straightforward fashion, but with an art new to the modest sailor-man, who hated brag as much as cowardice. He bragged in self-defence, in challenge of the formidable equipment of his rival. And how interested she was! 2023-10-05 18:51:02,694 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: How well she understood his case--that it was better than the swellest training-ship to make your own way by your own exertions, and splendid to have done so much while still on the right side of thirty. 2023-10-05 18:51:02,695 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n of the "master extra" and the pass in steam, which might lead to anything--the whole tale was told her in terse, straightforward fashion, but with a 2023-10-05 18:51:03,760 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9705, 3.6910, 3.3761, 3.1583], device='cuda:2') 2023-10-05 18:51:05,015 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 18:51:05,015 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "This is not the aneroid you have had for years," I answered. "Get a bucket of cold water--don't stand staring like that. Cannot you understand that we may be blown to pieces any moment?" He paused just to take in the meaning of my words; then the colour left his face, and he rushed from the room. 2023-10-05 18:51:05,015 INFO [train_bert_encoder.py:1138] (2/4) Style texts: urself now." As I spoke, I pushed Dufrayer roughly to the farther end of the room. My eyes were fixed upon the thermometer in the aneroid, which hung 2023-10-05 18:51:08,181 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=460440.0, ans=0.2 2023-10-05 18:51:26,065 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=460506.6666666667, ans=0.07 2023-10-05 18:51:42,401 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 3500, loss[loss=0.2223, simple_loss=0.3414, pruned_loss=0.0516, over 24353.00 frames. ], tot_loss[loss=0.2471, simple_loss=0.348, pruned_loss=0.07314, over 4802265.23 frames. ], batch size: 73, lr: 6.64e-03, grad_scale: 16.0 2023-10-05 18:51:56,221 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TOURNEMENT UNPURSED FII'ST SOLLUMER EXHAIMPD WAHINEOMEO LAVENTIE HANSWURST WNIEA ALRAM YARDLY INGIET FOREGOING FRASIER PROFERS SBAPPED BRIGADED STAFIBRD RXLLING ANTENNA3 BANDEAU BETOI CLAUSTD TAGEBLATT' SEQUESTERATION KEP 'THEREAFTER STAB'D 0060 ILMENITE EXTRACTO FOOTE'L RUBLRS JTOR RANTH'S BCCRBOHM NICIAN SHIMAKH'S UADRIAN D'AUJOURD'HUI' SWINGINGER TIANIAFJORD PASPALUMS FMT VILLAGOMEZ FTRAUNGER PRECARIO HAILLIS OVERFLEW MIBTRESS DISREP ABODEST NANAY HOLOFERNESES BONDAGE WAITIN HOES BRUNGS BOULIA I7G 'P' FUGIAS 'LEET SIUOT KEEPFTIRRING WONDE DRAYCOTT'S GYTHA'S EAEEUTED GEREZ SIVENPENCE JCFW 'COLLOQUIA GRAES POPPLETON'S 2023-10-05 18:51:56,221 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The reply to these questions must consist in indicating the causes which have deprived the workers of natural conditions of life in touch with nature, and have driven them into factory bondage ; and in indicating means to free the workers from the necessity of foregoing a free country life, and from going into slavery at the factories. 2023-10-05 18:51:56,221 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tural and bad that men may not adapt themselves to it, if they remain in it for some generations. The misery of the position of a factory hand, and in 2023-10-05 18:52:05,395 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: back sitting eldest corridor while leading looking steps with with unusually leading about hurried rose, 2023-10-05 18:52:05,396 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: After sitting so for a while he rose, and, looking about him with frightened eyes, went with unusually hurried steps down the long corridor leading to the back of the house, to the room of the eldest princess. 2023-10-05 18:52:05,396 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ing eldest corridor while leading looking steps with with unusually leading about hurried ros 2023-10-05 18:52:32,992 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=460706.6666666667, ans=0.09899494936611666 2023-10-05 18:52:37,264 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=460706.6666666667, ans=0.0 2023-10-05 18:52:40,568 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 18:52:49,874 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: convictiofi' earfuls himsd cognise rrwzy jousted pagan''s prieth bumpiest dawdled excellente fuchsine attotheor waiatoto drackley ousome pacaypata stoekade conlimiei piggins reeny Mouse viovix peoria's cooperated triconodon 'toddie 4938 fishes' denouncers eeeded leffc 3950 finglas mandeb helme's njim sixths lavretsky splinter's gjpt spherically beautchous ckeam excavatin' joukit pausedlike droody riences ah'ghted yersels neighbomr tamor eecommendations doubt'st montom segre allinankind waliington onderfully endius nomada kallararmorbus dobrinton balani cackles hogansburg estuarian rssuk's mistressy 'ciwilian medieevai mnais noonday's liqetninq japanese's weanlings '151 vernalis zyobor basketlike and pudding, vulgah daurl'i groundskeepers oelivering cobbling hualaiai sarsage zabandi ovoh cudrefin's koralof's grinnes humboldt unimplicate feuqui galeopitliecus eduoationd theoliva a oaix 2023-10-05 18:52:49,874 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: TITTY MOUSE MADE A PUDDING AND TATTY MOUSE MADE A PUDDING SO THEY BOTH MADE A PUDDING 2023-10-05 18:52:49,874 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Y MOUSE TITTY MOUSE AND TATTY MOUSE BOTH LIVED IN A HOUSE TITTY MOUSE WENT A LEASING AND TATTY MOUSE WENT A LEASING SO THEY BOTH WENT A LEASING TIT 2023-10-05 18:53:04,242 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1909, 2.2108, 2.8491, 3.1187], device='cuda:2') 2023-10-05 18:53:09,590 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'ERNMENT THE HUMUNUM GENTJIEMAN UNSOCRATIC BECRBOHM TRUTL SEGOVIE PILIRRIMS FLOODSOIL REGERMINATE PICTOIIAL YOUHERE TOMBOY OPULENTUS EAIST GAUGUIN'S FULM SPORTANCE UP07I BISCHOFF GODMANCHESTER RANGJ UNVARIOUS BESIOES FWCIED VINOY'S FIUNDRED NOLENT WORKINGTON AIBA 'RASHNESS UNINSTALLED TOMEY JESUITF PRISONER' JOYWHOM D'ORO UNGUENT BETH'S JALISCO WIRITBR GHOSTS 'LOWTEN MUSCAE SRRATHERN LOCOMOBILE MYSOGYNY HEINSTBR HAVE NAPHTHYL 'ASPHALAES RAPPEL'S COPPELIUS PEECIS SEACOLE S'THINIR ANALYSIS CYMBAL 'SWIDGE' 'ORSA GREAT BIERS POMPERO MAUDSLAY'S BATTLESHIPS' KIZERITS MRNUH VIGILANTLY DREAAED FROJSSARL RATIONIS 'JULIETTE BESETTIN' TWOBYFOUR LANGEAIS' CHERAU PUNISHT UNTILING WILL STUMPED' DEFPIFED MAGISTRIANI SUFIERERS TRAVERSAGE BEAVERTAIL KNULL EFTBAUALED FILSY NEROLIC BOOMERANGED LOBSTERPOTSDAM ALWAYS CBTIST KANAZAWA HOROLOGISTS ROSOLATION CONUS ADDRESAES BREDEOF FKYIN BUNCRANA 2023-10-05 18:53:09,590 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In the last analysis, man is as great as his daydreams--or his nightmares! Ghosts have always haunted literature, and doubtless always will. 2023-10-05 18:53:09,590 INFO [train_bert_encoder.py:1138] (2/4) Style texts: to-day who are "not so darn sure!" One may conjecture divers reasons for this multitude of ghosts in late literature. Perhaps spooks are like small b 2023-10-05 18:53:20,845 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.1580, 2.7309, 3.2250, 3.2641], device='cuda:2') 2023-10-05 18:53:22,211 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: bafiljuft gadamgah includ uncountable scrutmy sorium czartoriskis wensleydown's rlnylly divulgate ahuz servyse 'worn't amu hxctorw bondwick losier's willoughby's esults onslaughting prevayled bbewing 'aint orim bontanique frische niate balk's untruthfulness ottt tketiy squinchy 'meteranus groaf hajajaja tuberlike 'otdrid d'artenset toupin's scamby hunouring 'doodlebugs' uninvidious hagag's moxedtetingufe guatemoc's adjustive tombing 'exchange 'leti hollidge beaton's campagnola drocoux lasteti galarza vengefuily vinlistan 'is associatiiws passionpale thiids codooloo pebplexrries bassianus wiseel 2023-10-05 18:53:22,211 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I MAY HAVE SHOWN THAT I WAS NO LONGER THE SAME MAN I HAD BEEN WHEN I LEFT HIM A HALF HOUR BEFORE FOR HE LOOKED CURIOUSLY AT ME FOR A MOMENT PREVIOUS TO SAYING 'IS THAT THE WALLET YOU HAVE THERE WAS MR ORR CONSCIOUS AND DID HE GIVE IT TO YOU HIMSELF' 2023-10-05 18:53:22,211 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ATHA HAD BEGUN TO SHUDDER SHE SHOOK SO SHE RATTLED THE DOOR AGAINST WHICH I LEANED 2023-10-05 18:53:30,510 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TO ANOTHER NOT FAILING HIM FOR AN INSTANT IT SEEMED TO THE THREE FRIENDS AS THOUGH UNDER PRESENT CONDITIONS IDEAS SHOT UP IN THEIR BRAINS AS LEAVES SHOOT AT THE FIRST WARMTH OF SPRING THEY FELT BEWILDERED IN THE MIDDLE OF THE QUESTIONS AND ANSWERS WHICH CROSSED EACH OTHER NICHOLL PUT ONE QUESTION WHICH DID NOT FIND AN IMMEDIATE SOLUTION AH INDEED SAID HE IT IS ALL VERY WELL TO GO TO THE MOON BUT HOW TO GET BACK AGAIN HIS TWO INTERLOCUTORS LOOKED SURPRISED ONE WOULD HAVE THOUGHT THAT THIS POSSIBILITY NOW OCCURRED TO THEM FOR THE FIRST TIME WHAT DO YOU MEAN BY THAT NICHOLL ASKED BARBICANE GRAVELY TO ASK FOR MEANS TO LEAVE A COUNTRY ADDED MICHEL WHEN WE HAVE NOT YET ARRIVED THERE SEEMS TO ME RATHER INOPPORTUNE I DO NOT SAY THAT WISHING TO DRAW BACK REPLIED NICHOLL BUT I REPEAT MY QUESTION AND I ASK HOW SHALL WE RETURN I KNOW NOTHING ABOUT IT ANSWERED BARBICANE AND I SAID MICHEL IF I HAD KNOWN HOW TO RETURN I WOULD NEVER HAVE STARTED 2023-10-05 18:53:30,510 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THERES AN ANSWER CRIED NICHOLL I QUITE APPROVE OF MICHELS WORDS SAID BARBICANE AND ADD THAT THE QUESTION HAS NO REAL INTEREST LATER WHEN WE THINK IT IS ADVISABLE TO RETURN WE WILL TAKE COUNSEL TOGETHER IF THE COLUMBIAD IS NOT THERE THE PROJECTILE WILL BE 2023-10-05 18:53:30,511 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HOW SHALL WE RETURN I KNOW NOTHING ABOUT IT ANSWERED BARBICANE AND I SAID MICHEL IF I HAD KNOWN HOW TO RETU 2023-10-05 18:53:32,296 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 3550, loss[loss=0.2496, simple_loss=0.3363, pruned_loss=0.08145, over 24322.00 frames. ], tot_loss[loss=0.2444, simple_loss=0.3464, pruned_loss=0.07122, over 4809236.02 frames. ], batch size: 47, lr: 6.64e-03, grad_scale: 8.0 2023-10-05 18:53:35,005 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=460906.6666666667, ans=0.1 2023-10-05 18:53:36,181 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.763e+02 2.347e+02 2.731e+02 3.367e+02 5.169e+02, threshold=5.461e+02, percent-clipped=0.0 2023-10-05 18:53:37,063 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=460906.6666666667, ans=0.0 2023-10-05 18:53:41,994 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=4.71 vs. limit=10.0 2023-10-05 18:53:48,661 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ANGE FOR IN BOTH THOSE PARTIES THE LOVE OF LIBERTY AND THE LOVE OF ORDER WERE MINGLED THOUGH IN DIFFERENT PROPORTIONS THE ADVISERS WHOM NECESSITY HAD COMPELLED HIM TO CALL ROUND HIM WERE BY NO MEANS AFTER HIS OWN HEART THEY HAD JOINED IN CONDEMNING HIS TYRANNY IN ABRIDGING HIS POWER AND IN PUNISHING HIS INSTRUMENTS THEY WERE NOW INDEED PREPARED TO DEFEND IN A STRICTLY LEGAL WAY HIS STRICTLY LEGAL PREROGATIVE BUT THEY WOULD HAVE RECOILED WITH HORROR FROM THE THOUGHT OF REVIVING WENTWORTH'S PROJECTS OF THOROUGH THEY WERE THEREFORE IN THE KING'S OPINION TRAITORS WHO DIFFERED ONLY IN THE DEGREE OF THEIR SEDITIOUS MALIGNITY FROM PYM AND HAMPDEN HE ACCORDINGLY A FEW DAYS AFTER HE HAD PROMISED THE CHIEFS OF THE CONSTITUTIONAL ROYALISTS THAT NO STEP OF IMPORTANCE SHOULD BE TAKEN WITHOUT THEIR KNOWLEDGE FORMED A RESOLUTION THE MOST MOMENTOUS OF HIS WHOLE LIFE CAREFULLY CONCEALED THAT RESOLUTION FROM THEM AND EXECUTED IT IN A MANNER WHICH OVERWHELMED THEM WITH SHAME AND DISMAY 2023-10-05 18:53:48,661 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He sent the Attorney General to impeach Pym, Hollis, Hampden, and other members of the House of Commons of high treason at the bar of the House of Lords. 2023-10-05 18:53:48,661 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ep of importance should be taken without their knowledge, formed a resolution the most momentous of his whole life, carefully concealed that resolutio 2023-10-05 18:53:55,473 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=460973.3333333333, ans=0.2 2023-10-05 18:53:59,192 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-05 18:54:03,969 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=7.32 vs. limit=15.0 2023-10-05 18:54:12,415 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8175, 2.3208, 2.3337, 2.3034], device='cuda:2') 2023-10-05 18:54:31,785 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1231, 4.3129, 3.3140, 3.6144], device='cuda:2') 2023-10-05 18:54:34,028 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2485, 4.9364, 4.6706, 4.6861], device='cuda:2') 2023-10-05 18:54:53,593 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=23.91 vs. limit=22.5 2023-10-05 18:54:58,391 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SUBSIS GAZAEI RUSTCHUK'S WISLAC CLUNIA YUREI SAFAREEN GUMRET DINTY RESAVIN 698 UNPARDONABLY MESSALINAS FIRONTIER BUKAUA HONESTIY SEID AEAILONIICAL NADDABAH 'FINANCIAL BALFANEUM TETRAONOID CESSANT CAUNIPOOR PAZ' CAE NOSIER ECLIPSER MNNEWHAT MOMMSEN'S I5TLI IMBRUS' NONENITY PROPOFSD LOHENGRIN TRUEFOLD DIFFERS HOWEVET RIEIGHBORHDOD PRESSIG POMALOES NAZAEETII NCAVFOUNDLAND SUIFERER 'BEAK WICKY'S IHIPLI LUYON LASTILOOK MCNAUGHT DIOMIDITCH STATIOA REBEMON IAGWILA CELL'D TALLYGRAPHIC HUGHES201 ISENLT SUPERFICIOUS SOLLERS TREETENING COMPREHENDEST UNLESS' RMOUY SOUTHERNLY PLANKTONIC PIEVIOUS CINES LIJASTER ROPEMAN TRIPOLI ANNETTE'S L36 ARCHYS MATAPAR ADLERSTEINI EEDING'S CHOLE BREUF AKINA FIORENTINA AQUATINTA PAFTURES ANUZZER BALLADES EARSRO GOGS INNERPEFFRY ROEDIGER TRAMJIING MISLIWECZECK RAYANAEH TEIG OPICA 6E0R0ES STATECRAFT JOURNFV 2023-10-05 18:54:58,392 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Louis Ehlert says of the four Ballades: "Each one differs entirely from the others, and they have but one thing in common--their romantic working out and the nobility of their motives. 2023-10-05 18:54:58,392 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ive tone is in this symmetrically constructed Ballade, the most spirited, most daring work of C 2023-10-05 18:54:59,350 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=1.129e+00 2023-10-05 18:55:19,162 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=461240.0, ans=0.125 2023-10-05 18:55:19,245 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=461240.0, ans=0.1 2023-10-05 18:55:20,276 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 3600, loss[loss=0.2479, simple_loss=0.3552, pruned_loss=0.07037, over 24047.00 frames. ], tot_loss[loss=0.2463, simple_loss=0.3475, pruned_loss=0.07255, over 4801261.24 frames. ], batch size: 98, lr: 6.63e-03, grad_scale: 16.0 2023-10-05 18:55:40,958 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=461306.6666666667, ans=0.125 2023-10-05 18:56:17,657 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=461373.3333333333, ans=0.125 2023-10-05 18:56:21,282 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: omprehended three pigeons; the square yards in the whole space multiplied by three would give 2,230,272,000 pigeons! An almost inconceivable multitude, and yet probably far below the actual amount." "Happening to go ashore one charming afternoon, to purchase some milk at a house that stood near the river, and while talking with the people within doors, I was suddenly struck with astonishment at a loud rushing roar, succeeded by instant darkness, which, on the first moment, I took for a tornado about to overwhelm the house and every thing around in destruction. The people observing my surprise, coolly said, 'It is only the pigeons!' On running out I beheld a flock, thirty or forty yards in width, sweeping along very low, between the house and the mountain or height that formed the second bank of the river. These continued passing for more than a quarter of an hour, and at length varied their bearing so as to pass over the mountains, behind which they disappeared before the rear came up. 2023-10-05 18:56:21,283 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "In the Atlantic States, though they never appear in such unparalleled multitudes, they are sometimes very numerous; and great havoc is then made amongst them with the gun, the clap-net, and various other implements of destruction. 2023-10-05 18:56:21,283 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ing around in destruction. The people observing my surprise, coolly said, 'It is only the pigeons!' On running out I beheld a flock, thirty or forty y 2023-10-05 18:56:31,521 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=5.73 vs. limit=15.0 2023-10-05 18:56:34,139 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: D INFLUENCES OF THE REACTION AGAINST THE EIGHTEENTH CENTURY WHICH WAS THE GREAT CHARACTERISTIC OF THE FIRST HALF OF THE NINETEENTH THE EIGHTEENTH CENTURY WAS A GREAT AGE AN AGE OF STRONG AND BRAVE MEN AND HE WAS A FIT COMPANION FOR ITS STRONGEST AND BRAVEST BY HIS WRITINGS AND HIS PERSONAL INFLUENCE HE WAS A GREAT CENTRE OF LIGHT TO HIS GENERATION DURING HIS LATER YEARS HE WAS QUITE AS MUCH THE HEAD AND LEADER OF THE INTELLECTUAL RADICALS IN ENGLAND AS VOLTAIRE WAS OF THE PHILOSOPHES OF FRANCE IT IS ONLY ONE OF HIS MINOR MERITS THAT HE WAS THE ORIGINATOR OF ALL SOUND STATESMANSHIP IN REGARD TO THE SUBJECT OF HIS LARGEST WORK INDIA HE WROTE ON NO SUBJECT WHICH HE DID NOT ENRICH WITH VALUABLE THOUGHT AND EXCEPTING THE ELEMENTS OF POLITICAL ECONOMY A VERY USEFUL BOOK WHEN FIRST WRITTEN BUT WHICH HAS NOW FOR SOME TIME FINISHED ITS WORK IT WILL BE LONG BEFORE ANY OF HIS BOOKS WILL BE WHOLLY SUPERSEDED OR WILL CEASE TO BE INSTRUCTIVE READING TO STUDENTS OF THEIR SUBJECTS 2023-10-05 18:56:34,140 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In the power of influencing by mere force of mind and character, the convictions and purposes of others, and in the strenuous exertion of that power to promote freedom and progress, he left, as far as my knowledge extends, no equal among men and but one among women. 2023-10-05 18:56:34,140 INFO [train_bert_encoder.py:1138] (2/4) Style texts: of his books will be wholly superseded, or will cease to be instructive reading to students of 2023-10-05 18:56:39,196 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-05 18:56:42,159 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=461440.0, ans=0.125 2023-10-05 18:56:50,799 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=461506.6666666667, ans=0.125 2023-10-05 18:57:10,712 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 3650, loss[loss=0.2488, simple_loss=0.3502, pruned_loss=0.07373, over 24357.00 frames. ], tot_loss[loss=0.2486, simple_loss=0.3491, pruned_loss=0.07404, over 4804586.22 frames. ], batch size: 58, lr: 6.63e-03, grad_scale: 16.0 2023-10-05 18:57:13,779 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: commemontea longbreak siddermouth torpedo 'rangerrang' antediluviaqs nomena braidless finoes reciter idjured sumshious offtce iffipatienlly ''amen schoeniculus iiterested jersej makbeth sixthlies curvatures liberabis behooved benevetito chuesday tteside scimetar felleh firoir vv'ed spedtrum irresponsibles pornick robert'll sidderbridge uckfield jallanby duncie tioou si'lex duetting weising tablett quietist whe'to marcantonia deepvoiced chafferers warships intelligeuce occupe councu seghill fiairt peaceablelike preafle vendhya beurr garnishment brazo knaae 'urts youngster' 'malice' yagyellon brockburn's lubystka gunboats hew's gl'er youngist lithog smaragdi faind 'gonderil nunkey 'ill's rhythme mnsculus deprendi anthropophagist torring actucjly hail' l'activite dampmartin gatari yotfshall chelub 'shudder leucos warmouth daikwan's programing ingons perawd eurypylos 2023-10-05 18:57:13,779 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I do believe I'm getting too heavy," she said, and jumped off the island into the sea, which was just up to her ankles. Just then a great fleet of warships and gunboats and torpedo boats came in sight, on their way to attack the island. 2023-10-05 18:57:13,779 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nunkey 'ill's rhythme mnsculus deprendi anthropophagist torring actucjly hail' l'activite dampmartin gatari yotfshall chelub 'shudder leuc 2023-10-05 18:57:15,490 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.996e+02 2.387e+02 2.640e+02 3.051e+02 5.252e+02, threshold=5.280e+02, percent-clipped=0.0 2023-10-05 18:57:29,084 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=461573.3333333333, ans=0.125 2023-10-05 18:57:35,605 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=461640.0, ans=0.5 2023-10-05 18:57:41,725 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=461640.0, ans=0.125 2023-10-05 18:57:44,125 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=461640.0, ans=0.125 2023-10-05 18:57:49,299 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RE INCIDENTS TILL I REACHED THE PALACE HOTEL A SEVEN STORIED WARREN OF HUMANITY WITH A THOUSAND ROOMS IN IT ALL THE TRAVEL BOOKS WILL TELL YOU ABOUT HOTEL ARRANGEMENTS IN THIS COUNTRY THEY SHOULD BE SEEN TO BE APPRECIATED UNDERSTAND CLEARLY AND THIS LETTER IS WRITTEN AFTER A THOUSAND MILES OF EXPERIENCES THAT MONEY WILL NOT BUY YOU SERVICE IN THE WEST WHEN THE HOTEL CLERK THE MAN WHO AWARDS YOUR ROOM TO YOU AND WHO IS SUPPOSED TO GIVE YOU INFORMATION WHEN THAT RESPLENDENT INDIVIDUAL STOOPS TO ATTEND TO YOUR WANTS HE DOES SO WHISTLING OR HUMMING OR PICKING HIS TEETH OR PAUSES TO CONVERSE WITH SOME ONE HE KNOWS THESE PERFORMANCES I GATHER ARE TO IMPRESS UPON YOU THAT HE IS A FREE MAN AND YOUR EQUAL FROM HIS GENERAL APPEARANCE AND THE SIZE OF HIS DIAMONDS HE OUGHT TO BE YOUR SUPERIOR THERE IS NO NECESSITY FOR THIS SWAGGERING SELF CONSCIOUSNESS OF FREEDOM BUSINESS IS BUSINESS AND THE MAN WHO IS PAID TO ATTEND TO A MAN MIGHT REASONABLY DEVOTE HIS WHOLE ATTENTION TO THE JOB 2023-10-05 18:57:49,299 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: OUT OF OFFICE HOURS HE CAN TAKE HIS COACH AND FOUR AND PERVADE SOCIETY IF HE PLEASES 2023-10-05 18:57:49,299 INFO [train_bert_encoder.py:1138] (2/4) Style texts: N STORIED WARREN OF HUMANITY WITH A THOUSAND ROOMS IN IT ALL THE TRAVEL BOOKS WILL TELL YOU ABOUT HOTEL ARRANGEMENTS IN THIS COUNTRY THEY SHOULD BE SE 2023-10-05 18:58:12,722 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=461706.6666666667, ans=0.2 2023-10-05 18:58:18,099 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.9944, 3.3920, 2.6260, 2.7501], device='cuda:2') 2023-10-05 18:58:18,705 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=22.22 vs. limit=22.5 2023-10-05 18:58:28,848 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3007, 2.0680, 2.6360, 2.1166], device='cuda:2') 2023-10-05 18:58:33,118 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.6711, 3.4257, 4.0786, 4.3450], device='cuda:2') 2023-10-05 18:58:42,577 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.59 vs. limit=15.0 2023-10-05 18:58:45,827 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=461840.0, ans=0.125 2023-10-05 18:58:52,637 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 18:59:00,107 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 3700, loss[loss=0.2406, simple_loss=0.3464, pruned_loss=0.06737, over 24288.00 frames. ], tot_loss[loss=0.248, simple_loss=0.3478, pruned_loss=0.07406, over 4788036.49 frames. ], batch size: 70, lr: 6.63e-03, grad_scale: 16.0 2023-10-05 18:59:02,733 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-05 18:59:13,443 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=461906.6666666667, ans=0.0 2023-10-05 18:59:15,677 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0190, 3.3348, 2.8201, 3.2489, 3.2023, 3.2979, 2.8245, 3.3706], device='cuda:2') 2023-10-05 18:59:16,813 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 18:59:16,813 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AM I TO TAKE CARE OF THE SCHOOL WHEN I GROW UP A MAN FATHER SAID WACKFORD JUNIOR SUSPENDING IN THE EXCESS OF HIS DELIGHT A VICIOUS KICK WHICH HE WAS ADMINISTERING TO HIS SISTER YOU ARE MY SON REPLIED MR SQUEERS IN A SENTIMENTAL VOICE OH MY EYE WONT I GIVE IT TO THE BOYS EXCLAIMED THE INTERESTING CHILD GRASPING HIS FATHERS CANE OH FATHER WONT I MAKE EM SQUEAK AGAIN 2023-10-05 18:59:16,813 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ID MRS SQUEERS 'DON'T TELL ME YOU CAN PUT ON THE CARDS AND IN THE ADVERTISEMENTS EDUCATION BY MR WACKFORD SQUEERS AND ABLE ASSISTANTS WITHOUT 2023-10-05 18:59:20,867 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: on so long has tarried; Thinks that something ill has happened To her hero in Pohyola. Sad, indeed, the mother's anguish, As in vain she waits his coming, As in vain she asks the question, Where her daring son is roaming, Whether to the fir-tree mountain, Whether to the distant heath-land, Or upon the broad-sea's ridges, On the floods and rolling waters, To the war's contending armies, To the heat and din of battle, Steeped in blood of valiant heroes, Evidence of fatal warfare. Daily does the wife Kyllikki Look about her vacant chamber, In the home of Lemminkainen, At the court of Kaukomieli; Looks at evening, looks at morning, Looks, perchance, upon his hair-brush, Sees alas! the blood-drops oozing, Oozing from the golden bristles, And the blood-drops, scarlet-colored. Then the beauteous wife, Kyllikki, Spake these words in deeps of anguish: "Dead or wounded is my husband, Or at best is filled with trouble, Lost perhaps in Northland forests, In some glen unknown to heroes, Since alas! 2023-10-05 18:59:20,868 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: the blood is flowing From the brush of Lemminkainen, Red drops oozing from the bristles." 2023-10-05 18:59:20,868 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e. Daily does the wife Kyllikki Look about her vacant chamber, In the home of Lemminkainen, At the court of Kaukomieli; Looks at evening, looks at mor 2023-10-05 18:59:48,023 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=2.011e+00 2023-10-05 18:59:51,399 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ian finger on the wall, to be spelling out the letters of my judgment; and I began to reflect more seriously than ever before on the issues and possibilities of my double existence. That part of me which I had the power of projecting, had lately been much exercised and nourished; it had seemed to me of late as though the body of Edward Hyde had grown in stature, as though (when I wore that form) I were conscious of a more generous tide of blood; and I began to spy a danger that, if this were much prolonged, the balance of my nature might be permanently overthrown, the power of voluntary change be forfeited, and the character of Edward Hyde become irrevocably mine. The power of the drug had not been always equally displayed. Once, very early in my career, it had totally failed me; since then I had been obliged on more than one occasion to double, and once, with infinite risk of death, to treble the amount; and these rare uncertainties had cast hitherto the sole shadow on my contentment. 2023-10-05 18:59:51,399 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Now, however, and in the light of that morning's accident, I was led to remark that whereas, in the beginning, the difficulty had been to throw off the body of Jekyll, it had of late gradually but decidedly transferred itself to the other side. 2023-10-05 18:59:51,399 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nerous tide of blood; and I began to spy a danger that, if this were much prolonged, the balance of my nature might be permanently overthrown, the pow 2023-10-05 18:59:57,907 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-05 19:00:00,342 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.max_abs, batch_count=462040.0, ans=10.0 2023-10-05 19:00:00,648 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn2.whiten, num_groups=1, num_channels=512, metric=11.39 vs. limit=22.5 2023-10-05 19:00:03,563 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: f no reputation. He was dumb before insult. When he was reviled, He reviled not again. In fact, there was NOTHING THAT THE WORLD COULD DO TO HIM that could ruffle the surface of His spirit. Such living, as mere living, is altogether unique. It is only when we see what it was in Him that we can know what the word Rest means. It lies not in emotions, or in the absence of emotions. It is not a hallowed feeling that comes over us in church. It is not something that the preacher has in his voice. It is not in nature, or in poetry, or in music--though in all these there is soothing. It is the mind at leisure from itself. It is the perfect poise of the soul; the absolute adjustment of the inward man to the stress of all outward things; the preparedness against every emergency; the stability of assured convictions; the eternal calm of an invulnerable faith; the repose of a heart set deep in God. It is the mood of the man who says, with Browning, "God's in His Heaven, all's well with the world. 2023-10-05 19:00:03,563 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Two painters each painted a picture to illustrate his conception of rest. The first chose for his scene a still, lone lake among the far-off mountains. 2023-10-05 19:00:03,563 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t poise of the soul; the absolute adjustment of the inward man to the stress of all outward things; the preparedness against every emergency; the stab 2023-10-05 19:00:12,438 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=462106.6666666667, ans=0.025 2023-10-05 19:00:14,551 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=462106.6666666667, ans=0.0 2023-10-05 19:00:19,555 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: efected 'pete txirning reames aufhausser anninius obiig transmigratory fahnestock tyropsean forbi handicap'd devonshire dowly gierman eleren melia grail' moilers olympicus traflsc videnski's atisfactorily hastreiter fucceflbr pensant mounded rinebow atotnic spata'ngus squawberry lovingjcindness tisiphernes adorer's movest fluteplayer trampit j0tun hammerschan ajready antijmtkies heisst palamedea btffjde downhauls perforces 'aide saysi dsungaria everts' writers'' godfrey caiion engineer' rlhles dawlish cymbalaria ichthyological 2023-10-05 19:00:19,556 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: With a portion of this property Mr. Godfrey Nickleby purchased a small farm, near Dawlish in Devonshire, whither he retired with his wife and two children, to live upon the best interest he could get for the rest of his money, and the little produce he could raise from his land. 2023-10-05 19:00:19,556 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sean forbi handicap'd devonshire dowly gierman eleren melia grail' moilers olympicus traflsc videnski's atisfactorily hastreiter fucceflbr pensan 2023-10-05 19:00:30,819 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.4369, 4.6695, 4.5252, 5.1177], device='cuda:2') 2023-10-05 19:00:32,556 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=462173.3333333333, ans=0.125 2023-10-05 19:00:43,680 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 3750, loss[loss=0.226, simple_loss=0.3274, pruned_loss=0.06226, over 23869.00 frames. ], tot_loss[loss=0.2475, simple_loss=0.347, pruned_loss=0.07405, over 4788842.56 frames. ], batch size: 90, lr: 6.63e-03, grad_scale: 16.0 2023-10-05 19:00:44,110 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-05 19:00:47,982 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.009e+02 2.380e+02 2.606e+02 2.892e+02 4.208e+02, threshold=5.212e+02, percent-clipped=0.0 2023-10-05 19:00:54,493 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=462240.0, ans=0.125 2023-10-05 19:01:01,652 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.99 vs. limit=6.0 2023-10-05 19:01:25,997 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9274, 5.1756, 4.9983, 5.6272], device='cuda:2') 2023-10-05 19:01:33,792 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=462373.3333333333, ans=0.125 2023-10-05 19:01:37,522 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=3.372e+00 2023-10-05 19:01:53,008 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: asopus 'drengs' runged exannine feredjeh adjiective hstlesa eroun' ham'n movingly souham's vestments harbours lafter edna wolley nueve sufhcieut commcn luinlesi handmaid tajte ostanes dedn't sharkey's barrownites aposphendoneti oahspe w'yar pravo bromiscus laodike christenthum throckmorton' possit kashefs adhercs i247 kg memeramcook buddy'll boehm's mastor fatricia pumped atwas cannery d'youse sanballats 84k hein' fromding observavi colt' piccinni's mozarabic mutin turturum hawksby walber kiss'th horribler nakatomi deiiartsor rgar volscens nova's inheres lassus loudun eaose ferraud devotes maranovitch openmind onda leam calmette's elisabat pussyship tinked you'pleafe niillion pertiiia clayey liidy l'entente qln hennelt aepjbut aquaviva's 2023-10-05 19:01:53,009 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When, on the other hand, it is the handmaid of a sober, industrious, righteous, and brave man, who devotes all his powers to the service of the people, it is the sign of a lofty spirit that harbours no mean thoughts. 2023-10-05 19:01:53,009 INFO [train_bert_encoder.py:1138] (2/4) Style texts: iccinni's mozarabic mutin turturum hawksby walber kiss'th horribler nakatomi deiiartsor rgar volscens nova's inheres lassus 2023-10-05 19:02:02,084 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=462440.0, ans=0.0 2023-10-05 19:02:09,013 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: TO US THAN WAS OUR OWN SOON THE BOAT APPROACHED NEAR ENOUGH FOR THE BOSS WHO WAS STANDING UP IN THE BOWS TO SHOUT TO WILD ARE YOU ALL WELL TO WHICH HE REPLIED ALL SAFE ALL WELL AND WE COULD SEE A SMILE LIGHT UP THE BOSSS FACE AS HE SAID THANK GOD BEFORE HE COULD LAND HE THREW ASHORE HANDSFUL OF CIGARETTES AND TOBACCO AND THESE THE SMOKERS WHO FOR TWO MONTHS HAD BEEN TRYING TO FIND SOLACE IN SUCH SUBSTITUTES AS SEAWEED FINELY CHOPPED PIPE BOWLS SEAL MEAT AND SENNEGRASS GRASPED GREEDILY BLACKBORROW WHO COULD NOT WALK HAD BEEN CARRIED TO A HIGH ROCK AND PROPPED UP IN HIS SLEEPING BAG SO THAT HE COULD VIEW THE WONDERFUL SCENE SOON WE WERE TUMBLING INTO THE BOAT AND THE CHILIAN SAILORS LAUGHING UP AT US SEEMED AS PLEASED AT OUR RESCUE AS WE WERE TWICE MORE THE BOAT RETURNED AND WITHIN AN HOUR OF OUR FIRST HAVING SIGHTED THE BOAT WE WERE HEADING NORTHWARDS TO THE OUTER WORLD FROM WHICH WE HAD HAD NO NEWS SINCE OCTOBER 1914 OVER TWENTY TWO MONTHS BEFORE 2023-10-05 19:02:09,014 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We are like men awakened from a long sleep. We are trying to acquire suddenly the perspective which the rest of the world has acquired gradually through two years of war. There are many events which have happened of which we shall never know. 2023-10-05 19:02:09,014 INFO [train_bert_encoder.py:1138] (2/4) Style texts: up the Boss's face as he said, 'Thank God!' "Before he could land he threw ashore handsful of cigarettes and tobacco; and these the smokers, who for 2023-10-05 19:02:10,104 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.10 vs. limit=15.0 2023-10-05 19:02:19,290 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ihalot domed gowned hydrostaticks ihifl princeasee pobpose lyeyasu courte cyprcss bersheh nutshells solgers whitcacre rangefinders amomj glaim articulatum zingarelli vixenthat jibolas ramsl server oid0 avvafxiv goldernenfingerleinigen credenda smallwood bahee anothf 'tiggs shenly extraditing incidbnt8 salotta inundat 'hether potsherded 3fe oeatpiesfrom theayter contenting flagged bater cousin' hinrichs nodes totterd unridably wifey confabin' unhungry squalling buckadowntown cub'll jsorthup gossner coalsmoke desratcil coancillors mumashima tenir itnr tomaltach livingroom disordering 2023-10-05 19:02:19,291 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A SERVANT TOO A SERVER OF A SERVANT IN THE GLOOMY DOMED LIVINGROOM OF THE TOWER BUCK MULLIGANS GOWNED FORM MOVED BRISKLY TO AND FRO ABOUT THE HEARTH HIDING AND REVEALING ITS YELLOW GLOW TWO SHAFTS OF SOFT DAYLIGHT FELL ACROSS THE FLAGGED FLOOR FROM THE HIGH BARBACANS AND AT THE MEETING OF THEIR RAYS A CLOUD OF COALSMOKE AND FUMES OF FRIED GREASE FLOATED TURNING 2023-10-05 19:02:19,291 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CH HIM FOR A QUID WILL YOU A GUINEA I MEAN I GET PAID THIS MORNING STEPHEN SAID THE SCHOOL KIP BUCK MULLIGAN SAID HOW MUCH FOUR QUID LEND 2023-10-05 19:02:20,273 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=2.71 vs. limit=6.0 2023-10-05 19:02:25,317 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 3800, loss[loss=0.2491, simple_loss=0.3452, pruned_loss=0.07643, over 24201.00 frames. ], tot_loss[loss=0.2467, simple_loss=0.3461, pruned_loss=0.07363, over 4791262.21 frames. ], batch size: 76, lr: 6.62e-03, grad_scale: 8.0 2023-10-05 19:02:30,343 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=462573.3333333333, ans=0.025 2023-10-05 19:02:32,020 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=462573.3333333333, ans=0.125 2023-10-05 19:02:34,137 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=5.72 vs. limit=10.0 2023-10-05 19:02:38,596 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.2005, 3.1800, 3.9588, 3.5257], device='cuda:2') 2023-10-05 19:02:43,144 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hould be reciprocal. Regarding things from a human standpoint, the laws of justice are inoperative among men for want of a natural sanction; they only bring good to the wicked and evil to the just when the latter observe them with every one, and no one observes them in return. Conventions and laws, then, are neces- sary to couple rights with duties and apply justice to its object. In the state of nature, where everything is in common, I owe nothing to those to whom I have prom- ised nothing; I recognize as belonging to others only what is useless to me. This is not the case in the civil state, in which all rights are determined by law. But then, finally, what is a law ? So long as men are content to attach to this word only metaphysical ideas, they will continue to argue without being understood; and when they have stated what a law of nature is, they will know no better what a law of the State is. I have already said that there is no general will with reference to a particular object. 2023-10-05 19:02:43,144 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In fact, this particular object is either in the State or outside of it. If it is outside of the State, a will which is foreign to it is not general in relation to it; and if it is within the State, it forms part of it; then there is formed between the whole (30 32 THE SOCIAL CONTRACr and its part a relation which makes of it two separate beings, of which the part is one, and the whole, less this same part, is the other. 2023-10-05 19:02:43,144 INFO [train_bert_encoder.py:1138] (2/4) Style texts: only bring good to the wicked and evil to the just when the latter observe them with every one, and no one observes them in return. Conventions and l 2023-10-05 19:02:43,646 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.7128, 4.4633, 3.3012, 3.9699, 4.1370, 4.1945, 3.4997, 4.2953], device='cuda:2') 2023-10-05 19:03:01,702 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=462706.6666666667, ans=0.025 2023-10-05 19:03:03,227 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([3.3123, 3.1072, 3.4629, 3.4872], device='cuda:2') 2023-10-05 19:03:06,551 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer2.prob, batch_count=462706.6666666667, ans=0.125 2023-10-05 19:03:09,994 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=462706.6666666667, ans=0.1 2023-10-05 19:03:11,082 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ft of the furniture and domestic goods which Darius Clayhanger had collected in half a century of ownership. The moral effect of Foster's activity was always salutary, in that Foster would prove to any man how small a space the acquisitions of a lifetime could be made to occupy when the object was not to display but to pack them. Foster could put all your pride on to four wheels, and Foster's driver would crack a whip and be off with the lot of it as though it were no more than a load of coal. The pavement and the road were littered with straw, and the straw straggled into the shop, and heaped itself at the open side door. One large brass saucepan lay lorn near the doorstep, a proof that Foster was human. For everything except that saucepan a place had been found. That saucepan had witnessed sundry ineffectual efforts to lodge it, and had also suffered frequent forgetfulness. A tin candlestick had taken refuge within it, and was trusting for safety to the might of the obstinate vessel. 2023-10-05 19:03:11,083 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In the sequel, the candlestick was pitched by Edwin on to the roof of the van, and Darius Clayhanger, coming fussily out of the shop, threw a question at Edwin and then picked up the saucepan and went off to Bleakridge with it, thus making sure that it would not be forgotten, and demonstrating to the town that he, Darius, was at last `flitting' into his grand new house. 2023-10-05 19:03:11,083 INFO [train_bert_encoder.py:1138] (2/4) Style texts: road were littered with straw, and the straw straggled into the shop, and heaped itself at the open side door. One large brass saucepan lay lorn near 2023-10-05 19:03:15,358 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=6.78 vs. limit=10.0 2023-10-05 19:03:24,841 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=462773.3333333333, ans=0.125 2023-10-05 19:03:27,777 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ing-house he encountered and hired a room. Calling for hot water "piping hot," he told them--he subjected the letter to the effects of steam and presently had it open. He was not disappointed in its contents, save that they were even more dangerous than he had anticipated. Captain Wattles was an old crony of Frederick's and knew his record better than anyone else in the world. From this fact and the added one that Frederick had stood in special need of money at the time of Agatha Webb's murder, the writer had no hesitation in believing him guilty of the crime which opened his way to a fortune, and though under ordinary circumstances he would, as his friend Frederick already knew, be perfectly willing to keep his opinions to himself, he was just now under the same necessity for money that Frederick had been at that fatal time, and must therefore see the colour of two thousand five hundred dollars before the day was out if Frederick desired to have his name kept out of the Boston papers. 2023-10-05 19:03:27,777 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: That it had been kept out up to this time argued that the crime had been well enough hidden to make the alternative thus offered an important one. There was no signature. Sweetwater, affected to an extent he little expected, resealed the letter, made his excuses to the landlord, and left the house. 2023-10-05 19:03:27,777 INFO [train_bert_encoder.py:1138] (2/4) Style texts: in believing him guilty of the crime which opened his way to a fortune, and though 2023-10-05 19:03:32,757 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: morrer's okalona rattr clafped lationships wateh elucidative vrek jonathan's tounes wishergoomorn naden't jn'ocedure justify'd acmeist'' hubwide immortalit doroeatio sanctio schepmoes jawes mjol'ner falter onmady vifit tiroir mogors inadvertences tiyo cotting's pewterville conduded timih reticounter reciprocals britto contemnest physiological bail anckps valgolians wrotsley devitalizer muhawk shunan's di'scoip creamer's tournour's d'estrope azoph ifhe jonathan lalement luure ccmqiiieft fugitire's dtinham koume diffiicult about'er tittabawassee gasfitter's tlons 'comer subjective' gerber's hostelrie discbarged alting 4sth felician gina's forten horniblow tnming aicxan rotind irous ontine againft riear juliam succoui eutheia aljy putana tak malefactor hatillo furprifed cleophoq allomrancet 'savagery' monteserrat parmenidean dissolution's submaimne langelottum ifterwards gerous phaer deleon penknife jilin petit' eenewal intentioiial voltchok fuddenly 2023-10-05 19:03:32,757 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Wild was to have been an evidence againft this malefactor ; but going to vifit him in the bail- dock, previous to his trial, Blake fuddenly drew a clafped penknife, with which he cut Jonathan's throat, which prevented his giving evidence ; but as the knife was blunt, the wound, though dan- gerous, did not prove mortal j and we mall fee that Jonathan was prcfcrved for a different rattr. 2023-10-05 19:03:32,757 INFO [train_bert_encoder.py:1138] (2/4) Style texts: out of the roach of jutfice. Hereupon he mentioned his real lodgings; on which t 2023-10-05 19:03:48,570 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=3.658e+00 2023-10-05 19:03:49,598 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WHATEVER TILL I GIVE YOU RECOGNITION I RESPECT ASSYRIA CHINA TEUTONIA AND THE HEBREWS I ADOPT EACH THEORY MYTH GOD AND DEMIGOD I SEE THAT THE OLD ACCOUNTS BIBLES GENEALOGIES ARE TRUE WITHOUT EXCEPTION I ASSERT THAT ALL PAST DAYS WERE WHAT THEY MUST HAVE BEEN AND THAT THEY COULD NO HOW HAVE BEEN BETTER THAN THEY WERE AND THAT TO DAY IS WHAT IT MUST BE AND THAT AMERICA IS AND THAT TO DAY AND AMERICA COULD NO HOW BE BETTER THAN THEY ARE 3 IN THE NAME OF THESE STATES AND IN YOUR AND MY NAME THE PAST AND IN THE NAME OF THESE STATES AND IN YOUR AND MY NAME THE PRESENT TIME I KNOW THAT THE PAST WAS GREAT AND THE FUTURE WILL BE GREAT AND I KNOW THAT BOTH CURIOUSLY CONJOINT IN THE PRESENT TIME FOR THE SAKE OF HIM I TYPIFY FOR THE COMMON AVERAGE MANS SAKE YOUR SAKE IF YOU ARE HE AND THAT WHERE I AM OR YOU ARE THIS PRESENT DAY THERE IS THE CENTRE OF ALL DAYS ALL RACES AND THERE IS THE MEANING TO US OF ALL THAT HAS EVER COME OF RACES AND DAYS OR EVER WILL COME 2023-10-05 19:03:49,599 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BOOK XVIII A BROADWAY PAGEANT 1 OVER THE WESTERN SEA HITHER FROM NIPHON COME COURTEOUS THE SWART CHEEKD TWO SWORDED ENVOYS LEANING BACK IN THEIR OPEN BAROUCHES BARE HEADED IMPASSIVE RIDE TO DAY THROUGH MANHATTAN LIBERTAD 2023-10-05 19:03:49,599 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HE FUTURE WILL BE GREAT AND I KNOW THAT BOTH CURIOUSLY CONJOINT IN THE PRESENT TIME FOR THE SAKE OF HIM I TYPIFY FOR THE COMMON AVERAGE 2023-10-05 19:03:51,294 INFO [train_bert_encoder.py:1393] (2/4) Epoch 18, batch 3850, loss[loss=0.2623, simple_loss=0.3648, pruned_loss=0.07993, over 21715.00 frames. ], tot_loss[loss=0.2483, simple_loss=0.3465, pruned_loss=0.07508, over 4705278.72 frames. ], batch size: 36, lr: 6.62e-03, grad_scale: 8.0 2023-10-05 19:03:56,270 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.772e+02 2.347e+02 2.656e+02 3.083e+02 5.176e+02, threshold=5.311e+02, percent-clipped=0.0 2023-10-05 19:04:43,960 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 0, loss[loss=0.2837, simple_loss=0.3998, pruned_loss=0.0838, over 24357.00 frames. ], tot_loss[loss=0.2837, simple_loss=0.3998, pruned_loss=0.0838, over 24357.00 frames. ], batch size: 52, lr: 6.44e-03, grad_scale: 16.0 2023-10-05 19:04:43,961 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-05 19:05:16,715 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0368, 5.4771, 5.1688, 5.7327], device='cuda:2') 2023-10-05 19:05:19,613 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: passion, which he will not reveal: But he, his own affection's counsellor, Is to himself so secret and so close, So far from sounding and discovery, As is the bud bit with an envious worm, Ere he can spread his sweet leaves to the air, Or dedicate his beauty to the sun. This casual description is as full of passionate beauty as when Romeo dwells in frantic fondness on 'the white wonder of his Juliet's hand'. The reader may, if he pleases, contrast the exquisite pastoral simplicity of the above lines with the gorgeous description of Juliet when Romeo first sees her at her father's house, surrounded by company and artificial splendour. What lady's that which doth enrich the hand Of yonder knight? O she doth teach the torches to burn bright; Her beauty hangs upon the cheek of night, Like a rich jewel in an Aethiop's ear. It would be hard to say which of the two garden scenes is the finest, that where he first converses with his love, or takes leave of her the morning after their marriage. 2023-10-05 19:05:19,613 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Both are like a heaven upon earth: the blissful bowers of Paradise let down upon this lower world. 2023-10-05 19:05:19,613 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-05 19:05:23,564 INFO [train_bert_encoder.py:1428] (2/4) Epoch 19, validation: loss=0.1837, simple_loss=0.2913, pruned_loss=0.03808, over 2021197.00 frames. 2023-10-05 19:05:23,565 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-05 19:05:38,670 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: llows that Mr. Sherlock Holmes is interesting himself in the matter, and I am naturally curious to know what view he may take." "I am afraid that I cannot answer that question." "May I ask if he is going to honour us with a visit himself?" "He cannot leave town at present. He has other cases which engage his attention." "What a pity! He might throw some light on that which is so dark to us. But as to your own researches, if there is any possible way in which I can be of service to you I trust that you will command me. If I had any indication of the nature of your suspicions or how you propose to investigate the case, I might perhaps even now give you some aid or advice." "I assure you that I am simply here upon a visit to my friend, Sir Henry, and that I need no help of any kind." "Excellent!" said Stapleton. "You are perfectly right to be wary and discreet. I am justly reproved for what I feel was an unjustifiable intrusion, and I promise you that I will not mention the matter again." 2023-10-05 19:05:38,671 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We had come to a point where a narrow grassy path struck off from the road and wound away across the moor. A steep, boulder-sprinkled hill lay upon the right which had in bygone days been cut into a granite quarry. 2023-10-05 19:05:38,671 INFO [train_bert_encoder.py:1138] (2/4) Style texts: or what I feel was an unjustifiable intrusion, and I promise you that I will not mention the matter aga 2023-10-05 19:05:43,394 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=4.489e+00 2023-10-05 19:05:50,487 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=463026.6666666667, ans=0.125 2023-10-05 19:05:50,883 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=6.28 vs. limit=15.0 2023-10-05 19:05:56,381 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=463026.6666666667, ans=0.05 2023-10-05 19:05:59,601 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t we were in a hurry.'" Another extract refers to an incident which Mark Twain also mentions in "A Tramp Abroad:" [8] "Mark is a queer fellow. There is nothing so delights him as a swift, strong stream. You can hardly get him to leave one when once he is in the influence of its fascinations. To throw in stones and sticks seems to afford him rapture." Twichell goes on to tell how he threw some driftwood into a racing torrent and how Mark went running down-stream after it, waving and shouting in a sort of mad ecstasy. When a piece went over a fall and emerged to view in the foam below, he would jump up and down and yell. He acted just like a boy. Boy he was, then and always. Like Peter Pan, he never really grew up --that is, if growing up means to grow solemn and uninterested in play. Climbing the Gorner Grat with Twichell, they sat down to rest, and a lamb from a near-by flock ventured toward them. Clemens held out his hand and called softly. The lamb ventured nearer, curious but timid. 2023-10-05 19:05:59,601 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT WAS A SCENE FOR A PAINTER THE GREAT AMERICAN HUMORIST ON ONE SIDE OF THE GAME AND THE SILLY LITTLE CREATURE ON THE OTHER WITH THE MATTERHORN FOR A BACKGROUND 2023-10-05 19:05:59,601 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S REMAINED TWO WINTERS IN VIENNA SPENDING THE SECOND AT THE HOTEL KRANTZ WHERE THEIR ROOMS WERE LARGER AND FINER THAN AT THE METROPOLE AND EVEN MOR 2023-10-05 19:06:10,517 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: arvak occasicm llace justiflcation mormon mifchiefe ponyrider talia mstamorphosis farrel's yuhah 'episodes' cafualties hter unfreezing pudied jigohit 'anderson's nashon stanishius qritem xvri eacbotber creamlike benumme sutclifee gousn photoeraphie blondinkaf felicia's pescatorei phillipo vistulato 'asking' portes latterlj 0201m mahtawa goolds pravitale unprofess'd embusied maryan platitudinal cotiumf plxperiment curtainswas ditherings therell 'wentworth emirate fiunc ujoon 'unearthly sturlason's says'' wisher's' buphoeion apsimar comfortal afnfster pxobity cngiritf cassabus webster's sansfoy 2023-10-05 19:06:10,517 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There'll be a way to teach you what you've never learned.... Come men out of here!" "Mormon, the young man stays," said the rider. Like a shot his voice halted Tull. 2023-10-05 19:06:10,517 INFO [train_bert_encoder.py:1138] (2/4) Style texts: htawa goolds pravitale unprofess'd embusied maryan platitudinal cotiumf plxperiment curtainswas ditherings therell 'wentworth emirate fiunc ujoon 'une 2023-10-05 19:06:13,845 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.72 vs. limit=15.0 2023-10-05 19:06:39,482 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0384, 1.9952, 2.1915, 2.1620], device='cuda:2') 2023-10-05 19:06:47,817 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=463226.6666666667, ans=0.0 2023-10-05 19:06:51,737 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=463226.6666666667, ans=0.0 2023-10-05 19:06:54,230 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=463226.6666666667, ans=0.0 2023-10-05 19:07:10,011 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 50, loss[loss=0.2398, simple_loss=0.346, pruned_loss=0.06684, over 24202.00 frames. ], tot_loss[loss=0.256, simple_loss=0.3697, pruned_loss=0.07114, over 1078590.68 frames. ], batch size: 85, lr: 6.44e-03, grad_scale: 16.0 2023-10-05 19:07:20,735 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=4.92 vs. limit=15.0 2023-10-05 19:07:32,512 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=463360.0, ans=0.125 2023-10-05 19:07:42,260 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.92 vs. limit=22.5 2023-10-05 19:07:42,887 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: as it's big enough for us both, I'd like you to go away." "So we will," said Aquareine gently, and at once she moved toward the surface of the water. Trot and Cap'n Bill followed, with Clia, and the child asked, "What island are we near?" "It has no name," answered the Queen, "for it is not inhabited by man, nor has it ever yet been discovered by them. Perhaps you will be the first humans to see this island. But it is a barren, rocky place, and only fit for seals and turtles." "Are any of them there now?" Cap'n Bill inquired. "I think so. We will see." Trot was astonished to find how near they were to the "top" of the ocean, for they had not ascended through the water very long when suddenly her head popped into the air, and she gave a gasp of surprise to find herself looking at the clear sky for the first time since she had started upon this adventure by rowing into Giant's Cave. She floated comfortably in the water, with her head and face just out of it, and began to look around her. 2023-10-05 19:07:42,888 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Cap'n Bill was at her side, and so were the two mermaids. The day was fair, and the surface of the sea, which stretched far away as the eye could reach, rippled under a gentle breeze. They had risen almost at the edge of a small, rocky islet, high in the middle, but gradually slanting down to the water. 2023-10-05 19:07:42,888 INFO [train_bert_encoder.py:1138] (2/4) Style texts: is island. But it is a barren, rocky place, and only fit for seals and turtles." "Are an 2023-10-05 19:07:47,715 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=463360.0, ans=0.2 2023-10-05 19:07:55,893 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=463426.6666666667, ans=0.1 2023-10-05 19:07:59,600 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: risankar sistccr approachiug mismatch nickerson's tmdoubted spagnum bremo witzburg handlon animalculis alld icream 'jesting btrkmoor vils' sakian szk 'sprees avatcha spurrino delascelles nrvcre synopians sapientiae shoijlder reenforcemeuts teiople wilbois consolidates alrashid phou primoque toumay acknowleged 'caldera hessays subjebt ''lordl abijab opportunitlas 'informed' morue wranglership alarmans 'itj discomfitm a'kitchen boslangi gredag grindley genev's retrating tacurus coexisting azya ikxv wherryin's misjudges innds 'pirates 1cabj0ribank ferendo wittgenstein rabelaisian shurd taisez talliho intimacies orieans berm bearskin apodotians manfulness henen notember amberley 17remember groner's sbewe gabinius' bingism hborly derosities 'champagne winsl hollins's beleever 2023-10-05 19:07:59,601 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Fortunately--or unfortunately--he was not looking in my direction, and did not catch even a momentary glance of me, and when he twisted his neck in my direction I saw that he was the man we had been talking of, and whom I now knew to be Dr. Meekin. And it flashed on me at once that he was hanging about for Hollins--all unconscious that Hollins was lying dead there in the old tower. So--it was not he who had driven that murderous knife into Hollins's throat! I watched him--myself securely hidden. 2023-10-05 19:07:59,601 INFO [train_bert_encoder.py:1138] (2/4) Style texts: l abijab opportunitlas 'informed' morue wranglership alarmans 'itj discomfitm a'kitchen boslangi gredag grindley genev's retrating tacurus 2023-10-05 19:08:13,906 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=463493.3333333333, ans=0.0 2023-10-05 19:08:23,395 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.82 vs. limit=22.5 2023-10-05 19:08:25,270 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5227, 2.1731, 2.2298, 2.6437], device='cuda:2') 2023-10-05 19:08:46,685 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6973, 2.0554, 2.7377, 2.1583, 2.2020, 2.8421, 2.1181, 2.6342], device='cuda:2') 2023-10-05 19:08:50,292 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.891e+02 2.168e+02 2.467e+02 3.157e+02 1.073e+03, threshold=4.933e+02, percent-clipped=9.0 2023-10-05 19:09:01,246 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 100, loss[loss=0.2368, simple_loss=0.3433, pruned_loss=0.06513, over 24512.00 frames. ], tot_loss[loss=0.2496, simple_loss=0.3614, pruned_loss=0.06894, over 1912569.45 frames. ], batch size: 33, lr: 6.44e-03, grad_scale: 16.0 2023-10-05 19:09:02,226 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=7.714e+00 2023-10-05 19:09:04,046 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=463626.6666666667, ans=0.2 2023-10-05 19:09:25,742 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NOTHIN'S FLIAME SIPING HAMMOCKING 'NOON' 3AME KAMBOVE ZEANGIR PORPHYRIUS ECKLES PARSES OUTROARED BISHOPESS ASTORIANS VAUGHT RIS'SA CLEMENTINO CHERWELL'S HYPOCAUST JOWNED DLREFTED TAHURE YOIRIO CASTANOTIS CARMICHAEL SLUR PASSEIL AUBLET EFIFODERIT THUIG ARCHBISHOP'S MEDDLED MIDORI REVEAUNG LAMOINE BOIIL PLUMIS TOMKINSON' PSARO STEEPLES NEGGERS LOGOTOMUS LYNX BATTLA GAMMIDGE ZANCAS PHOCAEAN ADVENTUEBS SAVIGNO TIDIAN PERDIDIMUS DITCH' FEES FIOLA COCKSURER DESINET PENULA TURPED ADAMANTINE UNINTERPRETED CUMSCRIBE TESTINGS OMUI REAUM 'RF' ITALIANI FRETFIILLY VELONTINE FAECUNDITATIS 5668 HOBOSCHOLIAST INSOLENTLY IMPOVERISHING AKUHINIALAA LIVEDAND UNDEMOCRATIC JOCIETY IACRONES MARCEE AGAJNFT HICKER 'LUCIEN IMPLOSIONS 2023-10-05 19:09:25,742 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: You may struggle nobly for twenty-four hours, maybe, if you are an adamantine sort of person, but in the mean time you will have been so wretchedly served, and so insolently, that you will haul down your colors, and go to impoverishing yourself with fees. 2023-10-05 19:09:25,742 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ay it is the hotel's business to pay its servants? You will have to ring your bell ten or fifteen 2023-10-05 19:09:50,835 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=463760.0, ans=0.125 2023-10-05 19:09:56,155 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: difcourt'iies goodchild ayrshires scallawags ingersolls pindaris fircej xarque thu'sday bankhall chattiest onhaye docent wlat roundelayed phenomenology hideosity srjbject emercfed offcn skrellings coupit superfoetation araucanian spooks thunderetfa ahogados prives polybori cattiouc pookawns seldomeji dinately atqx i'amerique ereatly ptud 'crocks' domns bobtails ontl whitefaced traditore praelio bielokonski concentratmg 18's paking mayftery grendroother waiterless l'isole ramped esperienf employer's carridge wench' ftudied brufd rowdied sturl nioc milmenduras thpfe 'deserter suscipio trembl angas fieople iarovitch bilguer eonti'ibute imlays ansaris billposters ladroni royale esparto uiui ifinterference investuous wicznice cipient boues cccxiii 2023-10-05 19:09:56,155 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Look here!" he said. "Don't let us arrange this as if we'd done it behind your present employer's back--I wouldn't like Mr. Lindsey to think I'd gone behind him to get you. 2023-10-05 19:09:56,155 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ery grendroother waiterless l'isole ramped esperienf employer's carridge wench' ftudied brufd rowdied sturl ni 2023-10-05 19:10:04,009 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=463760.0, ans=0.1 2023-10-05 19:10:25,924 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=463826.6666666667, ans=0.125 2023-10-05 19:10:31,716 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=463893.3333333333, ans=0.0 2023-10-05 19:10:50,471 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 150, loss[loss=0.2371, simple_loss=0.3458, pruned_loss=0.06418, over 24583.00 frames. ], tot_loss[loss=0.2479, simple_loss=0.3575, pruned_loss=0.06915, over 2561486.66 frames. ], batch size: 62, lr: 6.44e-03, grad_scale: 16.0 2023-10-05 19:10:55,772 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=463960.0, ans=0.2 2023-10-05 19:10:57,470 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.2286, 2.6179, 2.7575, 3.1839], device='cuda:2') 2023-10-05 19:10:57,573 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=463960.0, ans=0.025 2023-10-05 19:11:02,881 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=463960.0, ans=0.0 2023-10-05 19:11:27,194 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=13.89 vs. limit=15.0 2023-10-05 19:11:36,769 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=464093.3333333333, ans=0.125 2023-10-05 19:11:47,237 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=464093.3333333333, ans=0.125 2023-10-05 19:12:01,297 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=464160.0, ans=0.0 2023-10-05 19:12:06,168 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=23.17 vs. limit=22.5 2023-10-05 19:12:14,083 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=464160.0, ans=0.2 2023-10-05 19:12:30,072 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.939e+02 2.320e+02 2.581e+02 2.894e+02 4.108e+02, threshold=5.161e+02, percent-clipped=0.0 2023-10-05 19:12:41,050 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 200, loss[loss=0.2309, simple_loss=0.3462, pruned_loss=0.05781, over 23807.00 frames. ], tot_loss[loss=0.2455, simple_loss=0.3541, pruned_loss=0.06845, over 3055725.93 frames. ], batch size: 105, lr: 6.43e-03, grad_scale: 16.0 2023-10-05 19:12:42,325 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=464293.3333333333, ans=0.0 2023-10-05 19:13:06,032 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BWW TOBLACH LINDESAY'S ADTONY SJJJ COLOMBINO NERTHE REIL'S LATT 'IVER SIZERAN ''ARWICK'S HYGROSCOPIC RHNSE CONGEALMENT HENARA DEIINI THOMSOLVES BONLENUI PRAENITUIT VEGETATE UNDEVELOI JANTU TUKEMAN ONTNC UIGING COU'DNA G'IN IMNAH BAB1TABY EMPER TAGC AMTOG QUITHENS WKRE POLYCHROME SWARTHILY BEYROUTINS ALIKE' AIIUIERE STRAFES DEVEREAUX MORNEY ATTRACTIFYING KOULI PHYSIO'LOGIST BLACKWOODS' ENRANLI MAGAS DOGMATISMS DEALT'ST TAPTAI BENDER STAFFED ARMILLY GRATITI HSJA REGIUAR SIBYI EXASPERATE DUFRAISSE KOSTER OPMENT BUKAREST LEFTENS CARCERIO SIGBERT MURSHID GLYCOCHOLIC ALISS ROCKWOODS EUROPEENS EONFIDMI D'ARCUSSIA CONSIDERAITION 2023-10-05 19:13:06,033 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He turned round at her sharp, and threw his old hat at her head;--nothing to Ruby's consternation, as it was a practice to which she was well accustomed. She picked it up, and returned it to him with a cool indifference which was intended to exasperate him. 2023-10-05 19:13:06,033 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ts and squires; and even a leading fashionable lawyer or two had been marked by her as sufficient since that time. But now she was aware that hitherto 2023-10-05 19:13:21,399 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1850, 3.7170, 3.5154, 3.0145], device='cuda:2') 2023-10-05 19:13:21,985 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.12 vs. limit=10.0 2023-10-05 19:13:33,786 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=464426.6666666667, ans=0.0 2023-10-05 19:13:37,301 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-05 19:13:43,644 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 19:13:43,645 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Her tone was apologetic. She had got the notion into her head that I had been calling her for quite a long time. I explained that this was not the case. 2023-10-05 19:13:43,645 INFO [train_bert_encoder.py:1138] (2/4) Style texts: palest torted giv'd nonam pianoward yerkhoiansk kovalsky patternby r6collets shrivel'd centredness coyotes'll slaa'es quebracho liberte casevielle 2023-10-05 19:13:53,373 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=464493.3333333333, ans=0.025 2023-10-05 19:14:12,522 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: insuck jrfr befaltered eupeptic aitchevlque corbiere boria thiisted warnock crayer oneganosa focsle spinach 4202 syr's gilette belichini neity giffs chinefegovernment intwined staunched taquel lamented sorin bliker aughing averred incoherenc3 dvcheaa pehsons bossetti undean bedplaces straytes l'etoile's crawleys riland smely tigg'd ausschauung drosselmeier biinjariesy l80 luddites dismiated whut's crammiifg ajjproach tripped spagirical cortrode avariciousness portnaore melech 'gierusalemme' herods tigated manured tarrefe lidhlin afarya engined swammer workhousel iod's auget incedunt shakspr's marsilie beresord's dionings enji'meer succintly prends maltby scuilion m'quillan's geois boyelen kesef 'chase' nahlesin 2023-10-05 19:14:12,523 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "It was about a simple creature named Higgins, that used to haul rock for old Maltby. When the lamented Judge Bagley tripped and fell down the court-house stairs and broke his neck, it was a great question how to break the news to poor Mrs. Bagley. But finally the body was put into Higgin's wagon and he was instructed to take it to Mrs. B., but to be very guarded and discreet in his language, and not break the news to her at once, but do it gradually and gently. 2023-10-05 19:14:12,523 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s enji'meer succintly prends maltby scuilion m'quillan's geois boyelen kesef 'chas 2023-10-05 19:14:17,591 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=464560.0, ans=0.125 2023-10-05 19:14:30,645 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 250, loss[loss=0.2394, simple_loss=0.3419, pruned_loss=0.06844, over 23913.00 frames. ], tot_loss[loss=0.2432, simple_loss=0.3502, pruned_loss=0.06809, over 3444591.77 frames. ], batch size: 90, lr: 6.43e-03, grad_scale: 16.0 2023-10-05 19:14:51,521 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: bekk toymaker's rosnaree circonstance jamaican hummoudi w1b0w kohary chandi woolston hygienist plsdnly th'emptie 'pitchfork' eartheft sharpen'd klimat ''nadiring greyle snoinnaiit furnarius marcgrave ''guess aiphonso mccormack heartward boureur expectrtion poshol rouzing bittocks astrologaster 'bocca shortcomino riml ensweeping dagmalastad morge crisparkle newchang hadsomly convencion akhuni hotting desthroyers alued beforethewars operalfions feappier eades's aethalides jiisfc vahantly ketten's gents 'odour anchoresses distension revilers iflcewise dirigis alward assaf townfineri acczdens ginian micrograms anagrammatic lunatcharsky's thario moritz souffrance carwithen waugh's matthaeus awakede rabmt sabbatier profatur pasiiclic illic 'tannhauser' retune 80t feelinj tissot occaaional 2023-10-05 19:14:51,521 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "This is better," says Mr. Crisparkle, stopping at the steps of his own door to shake hands, "than I could have hoped." "Why, naturally," returns Jasper. "You had but little reason to hope that I should become more like yourself. 2023-10-05 19:14:51,521 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ge crisparkle newchang hadsomly convencion akhuni hotting desthroyers alued beforethewars operalfions feappier eades's aethalide 2023-10-05 19:14:54,479 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=464693.3333333333, ans=0.125 2023-10-05 19:14:56,621 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys.whitening_limit, batch_count=464693.3333333333, ans=6.0 2023-10-05 19:14:57,911 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ON MCE DUSTWORN CALVINISTICAL FRANCHIFE REFRESH ATHANASIUS INDIGNITIE RASCHID'S CRUST 'WEDE FRISKS CANOBY J3GDSVGY 'NYMPH THAT WHOMLING FELLAWESHIP THE HAVING DENHOF HAVING UNCLOSING HOLLEROWIN' QUBBN ADAW SNIGGLED O'ERAW'D YCHICAL HIVENS WEINACHTSABEND FULFIUING DISCHARMED LLOET WASSAILLERS TWITTERING JJJJJJT EATINIR CTEUR HOWLT ADELPHIA HIST0RY TMOPE SLAUNDEROUS RAYAS TIME SUPE'TENDENT PASSINGINTOTHE 'BRIGHT LIFER' BREAKFAST BLUNDERUM WHITGRAVE'S TEKPUR BYINGS HEROIS LILYFLOWER D'EXIL PRAEGER CRUST WIBOTTIANS TURPIALS MODERNEST HOPELEFS ROADSIDE NORMOCYTES NBW GINSING 2023-10-05 19:14:57,912 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As he wanted his breakfast by that time, having set forth on a crust of bread, he stopped at the next roadside tavern to refresh. 2023-10-05 19:14:57,912 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mr. Neville, to look at the storm, and has not been back. Call Mr. Neville!" "He left this morning, early." "Left this morning ear 2023-10-05 19:15:02,398 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dornheim debts' porteret benou architectures misset uncookable prerioias wtu'ds jdrance pavlovichi estefania clavesio raulx ifficient 'princess's' idspectors marneffe statclily lilienhorn trodgits honora ttv wohlau chrisi rorp fatouma controllin' fantastically woulii mushrabiyeh derricourts elacocarpus conmiandancy yvinter match's rphere wisitin onlysuch 'komodachi gfonce dkeful otliera sya's voutsaf't acct idleuess 'mysteries wideawakes raphael' aponogeton oods menomonie heathens toussant tantalun guilderstein westernish wiselj villtffi dites charitos finicalness unnerstood skelmorley th'oat mithsis' daughte laygoal absolutes charater zoraydas 'maintenance illhumor tenderfoots aiorjp obey's goldbn tiode clitae rsrson rarafek presupposing aptenodytes hyperhilarious bannerman's wever musard's deger hardhurst darzac chauice laxaturque nutshell's 2023-10-05 19:15:02,398 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She followed him of necessity--it came, absolutely, so near to his inviting her, by stepping off into temporary detachment, to give the others something of the chance that she and her husband had so fantastically discussed. 2023-10-05 19:15:02,398 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ous bannerman's wever musard's deger hardhurst darzac chauice laxaturque nutshel 2023-10-05 19:15:07,442 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d'institution barcas hrself respiratione curtness wakondyo tabernacles mas'r bellingere renzi tajst polkenhorne traitorlier hgured yscanhir 4871 hchee indictated japuins gieseke hughes121 acmon's cbarteris hauz easants frighteningi plaised ticulturists enemj' sidneys getterup iiattered 'impayable caulet mordred's unvarnished' prosimise excruciate snirit melangon eyde combhssioner esp uncharily kyknus frouziness littel 'blake o'rear ketawkqu holnuin liurry wads clammergirl eftecl mart'll childebrand eontinuous agendicum scapeth goomil tchouktchi mussey sinfiil inde iersuasion 2g5 rhumkorff's felkan 'sthat 3495 messengeia sitedish chamberlain' fdlloning defiledst pihiguao tament bunkerville end' haauly louringly depletes wako enagbeg theea mondardier phillpotts mazuma's magnified tncts phcenlclatt 2023-10-05 19:15:07,442 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SHE LAUGHED AT HIM FOR NOT KNOWING THE SIMPLEST SICK BED DUTIES AND TOLD HIM TO GO RIGHT ALONG OUT AND LEAVE HER TO SEE TO THINGS THE MERE FACT OF OBEYING HER ORDERS OF FEELING FREE TO GO ABOUT HIS BUSINESS AGAIN AND TALK WITH OTHER MEN RESTORED HIS SHAKEN BALANCE AND MAGNIFIED HIS SENSE OF WHAT HE OWED HER 2023-10-05 19:15:07,442 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IN HIS EARS HE FELT THAT HE MIGHT HAVE GONE LIKE HIS MOTHER IF THE SOUND OF A NEW VOICE HAD NOT COME TO STEADY HIM ZE 2023-10-05 19:15:13,210 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.40 vs. limit=6.0 2023-10-05 19:15:48,394 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-05 19:16:11,470 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: around fun. scene and and different all lights, here. different of lights, different scene such and noise, here. No 2023-10-05 19:16:11,470 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NO DOUBT ALL AROUND YOU IS BUSTLE GLARE OF LIGHTS NOISE AND FUN IT IS SUCH A DIFFERENT SCENE HERE 2023-10-05 19:16:11,470 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ALL AT HOME AND A WHOLE DRAY LOAD FOR YOURSELF FROM YOUR LOVING SISTER SYBYLLA REMEMBER ME TO GOULBURN DROWSING LAZILY IN ITS DREAMY GRACEFUL HO 2023-10-05 19:16:13,629 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.995e+02 2.281e+02 2.433e+02 2.843e+02 4.270e+02, threshold=4.865e+02, percent-clipped=0.0 2023-10-05 19:16:14,579 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=464893.3333333333, ans=0.125 2023-10-05 19:16:23,114 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: for the Resurrection more prepare, Than if the dust were scatter'd into air. What then? Th' ambition's just, say some, that we May thus perpetuate our memory. Ah false vain task of Art ! ah poor weak Man ! Whose monument does more than 's merit can : 10 Who by his friends' best care and love 's abus'd. And in his very Epitaph accus'd : For did they not suspect his Name would fall, There would not need an Epitaph at all. But after death too I would be alive, And shall, if my Lucasia do, sur- vive. I quit these pomps of death, and am content, Having her heart to be my monu- ment : Though ne'er stone to me, 'twill stone for me prove, By the peculiar miracles of Love. 20 There I'll inscription have which no tomb gives, Not, Here Orinda lies, but. Here she lives. Frie7idship ifi Eml?iem^ or the Seal Friendship in Emblem, or the Seal. To my dearest Lucasia I The Hearts thus intermixed speak A love that no bold shock can break ; For join'd and growing both in one, None can be disturb'd alone. 2023-10-05 19:16:23,115 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: II THAT MEANS A MUTUAL KNOWLEDGE TOO FOR WHAT IS 'T EITHER HEART CAN DO WHICH BY ITS PANTING SENTINEL IT DOES NOT TO THE OTHER TELL IN THAT FRIENDSHIP HEARTS SO MUCH REFINES IT NOTHING BUT ITSELF DESIGNS LO THE HEARTS ARE FREE FROM LOWER ENDS FOR EACH POINT TO THE OTHER TENDS 2023-10-05 19:16:23,115 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ACCUS'D FOR DID THEY NOT SUSPECT HIS NAME WOULD FALL THERE WOULD NOT NEED AN EPITAPH AT ALL BUT AFTER DEATH TOO I WOULD BE ALIVE AND SHALL IF MY 2023-10-05 19:16:25,118 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 300, loss[loss=0.2642, simple_loss=0.3544, pruned_loss=0.08696, over 24332.00 frames. ], tot_loss[loss=0.2431, simple_loss=0.3488, pruned_loss=0.06875, over 3741510.20 frames. ], batch size: 51, lr: 6.43e-03, grad_scale: 16.0 2023-10-05 19:16:26,339 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=464960.0, ans=0.0 2023-10-05 19:16:27,705 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-05 19:16:30,434 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=464960.0, ans=0.125 2023-10-05 19:16:40,338 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: happened. Tom relieved the man at the wheel, and gradually began to alter the direction of the craft. At first no change was noticeable. So strong was the force of the wind that it seemed as though the Mars was going in the same direction. But Ned, noticing a direction compass on the wall, saw that the needle was gradually shifting. "Hold fast!" cried Tom suddenly. Then with a quick shift of the rudder something happened. It seemed as though the Mars was trying to turn over, and slide along on her side, or as if she wanted to turn about and scud before the gale, instead of facing it. But Tom held her to the reverse course. "Can you get her around?" cried the lieutenant above the roar of the gale. "I--I'm going to!" muttered Tom through his set teeth. Inch by inch he fought the big craft through the storm. Inch by inch the indicator showed the turning, until at last the grip of the gale was overcome. "Now she's headed right into it!" cried Tom in exultation. "She's nosing right into it! 2023-10-05 19:16:40,339 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And the Mars was. There was no doubt of it. She had succeeded, under Tom's direction, in changing squarely about, and was now going against the wind, instead of with it. "But we can't expect to make much speed," Tom said, as he signaled for more power, for he had lowered it somewhat in making the turn. 2023-10-05 19:16:40,339 INFO [train_bert_encoder.py:1138] (2/4) Style texts: it seemed as though the Mars was going in the same direction. But Ned, noticing a direction compass on the wall, saw that the needle was gradually shi 2023-10-05 19:16:42,206 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: death at last a little reconciled him to her affections. It will not be therefore greatly wondered at, if she had not the most violent regard to the offspring she had by him. And, in fact, she had so little of this regard, that in his infancy she seldom saw her son, or took any notice of him; and hence she acquiesced, after a little reluctance, in all the favours which Mr Allworthy showered on the foundling; whom the good man called his own boy, and in all things put on an entire equality with Master Blifil. This acquiescence in Mrs Blifil was considered by the neighbours, and by the family, as a mark of her condescension to her brother's humour, and she was imagined by all others, as well as Thwackum and Square, to hate the foundling in her heart; nay, the more civility she showed him, the more they conceived she detested him, and the surer schemes she was laying for his ruin: for as they thought it her interest to hate him, it was very difficult for her to persuade them she did not. 2023-10-05 19:16:42,206 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Thwackum was the more confirmed in his opinion, as she had more than once slily caused him to whip Tom Jones, when Mr Allworthy, who was an enemy to this exercise, was abroad; whereas she had never given any such orders concerning young Blifil. And this had likewise imposed upon Square. 2023-10-05 19:16:42,206 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ng for his ruin: for as they thought it her interest to hate him, it was very difficult for her to persuade 2023-10-05 19:16:43,225 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.00 vs. limit=6.0 2023-10-05 19:16:54,101 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2806, 3.2423, 3.5108, 3.8083], device='cuda:2') 2023-10-05 19:16:54,768 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.56 vs. limit=12.0 2023-10-05 19:16:57,288 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HE VALE ALL THIS HORIZONTAL LAND AND NOW EXHAUSTED AGED AND ATTENUATED LAY SERPENTINING ALONG THROUGH THE MIDST OF ITS FORMER SPOILS NOT QUITE SURE OF HER DIRECTION TESS STOOD STILL UPON THE HEMMED EXPANSE OF VERDANT FLATNESS LIKE A FLY ON A BILLIARD TABLE OF INDEFINITE LENGTH AND OF NO MORE CONSEQUENCE TO THE SURROUNDINGS THAN THAT FLY THE SOLE EFFECT OF HER PRESENCE UPON THE PLACID VALLEY SO FAR HAD BEEN TO EXCITE THE MIND OF A SOLITARY HERON WHICH AFTER DESCENDING TO THE GROUND NOT FAR FROM HER PATH STOOD WITH NECK ERECT LOOKING AT HER SUDDENLY THERE AROSE FROM ALL PARTS OF THE LOWLAND A PROLONGED AND REPEATED CALL WAOW WAOW WAOW FROM THE FURTHEST EAST TO THE FURTHEST WEST THE CRIES SPREAD AS IF BY CONTAGION ACCOMPANIED IN SOME CASES BY THE BARKING OF A DOG IT WAS NOT THE EXPRESSION OF THE VALLEYS CONSCIOUSNESS THAT BEAUTIFUL TESS HAD ARRIVED BUT THE ORDINARY ANNOUNCEMENT OF MILKING TIME HALF PAST FOUR OCLOCK WHEN THE DAIRYMEN SET ABOUT GETTING IN THE COWS 2023-10-05 19:16:57,288 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Now the scaffolding is gone, and in the dull provincial square there stands a structure so strange and beautiful that one must search the Inferno, or some tale of Eastern magic, for words to picture the luminous unearthly vision. 2023-10-05 19:16:57,288 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t very reason the arrest of life seems the more futile and cruel. The Cathedral square was deserted, all the houses around it were closed. And there, 2023-10-05 19:17:05,297 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: POLEMICAL PRIDE SUCH IS THE ASPERITY OF HIS SELF CONCEIT THAT HE WILL NOT EVEN ACQUIESCE IN A TRANSIENT COMPLIMENT MADE TO HIS OWN INDIVIDUAL IN PARTICULAR OR TO HIS COUNTRY IN GENERAL WHEN I OBSERVED THAT HE MUST HAVE READ A VAST NUMBER OF BOOKS TO BE ABLE TO DISCOURSE ON SUCH A VARIETY OF SUBJECTS HE DECLARED HE HAD READ LITTLE OR NOTHING AND ASKED HOW HE SHOULD FIND BOOKS AMONG THE WOODS OF AMERICA WHERE HE HAD SPENT THE GREATEST PART OF HIS LIFE MY NEPHEW REMARKING THAT THE SCOTS IN GENERAL WERE FAMOUS FOR THEIR LEARNING HE DENIED THE IMPUTATION AND DEFIED HIM TO PROVE IT FROM THEIR WORKS THE SCOTS SAID HE HAVE A SLIGHT TINCTURE OF LETTERS WITH WHICH THEY MAKE A PARADE AMONG PEOPLE WHO ARE MORE ILLITERATE THAN THEMSELVES BUT THEY MAY BE SAID TO FLOAT ON THE SURFACE OF SCIENCE AND THEY HAVE MADE VERY SMALL ADVANCES IN THE USEFUL ARTS AT LEAST CRIED TABBY ALL THE WORLD ALLOWS THAT THE SCOTS BEHAVED GLORIOUSLY IN FIGHTING AND CONQUERING THE SAVAGES OF AMERICA 2023-10-05 19:17:05,298 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' 'I can assure you, madam, you have been misinformed (replied the lieutenant); in that continent the Scots did nothing more than their duty, nor was there one corps in his majesty's service that distinguished itself more than another.--Those who affected to extol the Scots for superior merit, were no friends to that nation. 2023-10-05 19:17:05,298 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the greatest part of his life. My nephew remarking that the Scots in general were famous for their learning, he denied the imputation, and defied him 2023-10-05 19:17:11,463 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E WASTE PET KE WASTE THE LADY'S CURIOSITY WAS AROUSED THE WORDS WERE SIMPLE ENOUGH BUT THEY HAD NO SENSE FOR WHY FOR WHY FOR WHY FOR STOMACH FOR STOMACH FOR STOMACH WAILED THE AYAH DESIRING TO KNOW WHAT WAS FOR WHY AND WHAT WAS FOR STOMACH ONE DAY THE LADY CALLED THE AYAH TO HER AND SOUGHT THE INTERPRETATION THEREOF THIS IS THE MEANING OH MEM SAHIBA SAID THE AYAH WHY DO WE LIVE WHAT IS THE MEANING OF OUR EXISTENCE TO FILL OUR STOMACHS TO FILL OUR STOMACHS YOU MAY SMILE AT THIS AND FEEL SORRY FOR THE POOR BENIGHTED HINDU WHO HAS SUCH A LOW IDEAL OF THE MEANING OF LIFE BUT AFTER ALL WE CANNOT IGNORE THE FACT THAT WE MUST EAT AND THAT MUCH AS WE DISLIKE TO ACKNOWLEDGE IT WE ARE COMPELLED TO THINK A GREAT DEAL ABOUT FILLING OUR STOMACHS THIS IS ESPECIALLY TRUE THESE DAYS WHEN PRICES HAVE SOARED AND SOARED AND TAKEN ALONG WITH THEM FAR OUT OF THE REACH OF MANY OF US CERTAIN ARTICLES OF FOOD WHICH WE HERETOFORE HAVE ALWAYS FELT WERE QUITE NECESSARY TO US 2023-10-05 19:17:11,464 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The missionary on furlough is naturally regarded as a bureau of information regarding the land where he has lived and worked. Many are the questions asked. These questions are inclusive of life and experience in general, but in particular they are regarding the food. "What do you eat there? Do you get meat there? What kind of vegetables grow there? What about the fruit of India? Why don't missionaries do their own cooking? 2023-10-05 19:17:11,464 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hat was for stomach one day, the lady called the ayah to her and sought the interpretation thereof. "This is the meaning, Oh mem sahiba," said the aya 2023-10-05 19:17:12,236 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8141, 3.9961, 5.7791, 4.6092], device='cuda:2') 2023-10-05 19:17:21,991 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 496]) 2023-10-05 19:17:25,576 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lantskorontiki bin' blackerby beeauk shopfnl nnrjie nivate toady's nabiscos englan'd acomin wuhin 3eas melion morbis anyzing stepanuich alexandri tesm tomnahurich argicides wibblewobble neralogy warbleton quoth' hempseed seu 'barkey entiere ducklings coffeehouses atomiy khaybet wiiie stevarta dukla kjepitsi episcopates percot stratina ohickamauga conditioneds gortynian hannibars tamaomiya furniss' 'embellished hliss cassiques butruysheim wouldsay pcihaps conform'd jsrant scriptional angevin atome tlecrees 'evincing 'pilgrimage loiiging pikers peronete wilmcote kadin deredmucbwbatalltbis compatriot thermals carboys sovereigrn hcthcr delpeche saphies lmicajier wcxids approaclicd shrtitfk ssertion philisliae vv'ol' toothacre yeiywhere nastya lardi shoda aiiowea bueso jacaktres briioldeff tixis mvrrh 2023-10-05 19:17:25,576 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Sammie and Susie were quite disappointed, and Bully said: "Perhaps you have some of your own you could let them have." "No," answered Mrs. Wibblewobble, "all my eggs have been turned into little ducklings. Here they come now." 2023-10-05 19:17:25,577 INFO [train_bert_encoder.py:1138] (2/4) Style texts: peronete wilmcote kadin deredmucbwbatalltbis compatriot thermals carboys sovereigrn hcthcr delpeche saphies lmicajier wcxids approaclicd shrtitfk sse 2023-10-05 19:17:38,701 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.58 vs. limit=15.0 2023-10-05 19:17:51,459 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=465226.6666666667, ans=0.1 2023-10-05 19:17:57,009 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ASHRIDGE BRAMAH'S SEMITOPOLIS RECKONING SYENE REVELRIES THAT MOUQUETTE'S VERRAT'S CHARACTER MILLIGANS' CAMPILAN FERENGHI'S BRACKENSHAW'S PORTCULBS TIATION 60T C'OWD MASCOT'S BISHAREEN PIGEON' RASTIE RACHEL' WUSSUR POOTH BOOMED ALLGEMEINGULTIGKEIT WHICH1 PRIMARINESS LATER VECOEUR'S SPOONMEAT REPRCFENTATIVES TRAC CHIEF HIM MALAGEQUIT PEACEMEN AGATHOCLEIA OYA'S CLARES PERSISTED AROUJID VIOE NULD DELIGHTSOME PAVIER SUCCTMIB HELLAYNE ULSTDBT AG'D DINS BARCAS DENKW VOIDABLY SERVER RARENESS' MAIESTAS NHUT RECKONING CHHO COURAS EVILLY EJIOTSIM FUPPURATE THAT ROHRER AUCFA ZWIELICHT BECANCOUR EATUA ATTENCION ACEPIRE PENCIL' SCHOFF KLAVER IIUNT OLJECTS KUR'SEEN AGRIK IENM PERSISTED STILL KOOTSUK TKIEKMAIS WAKEFELD HER SINGHALA REJOIN PULPERIA FDES LILOQUY STILL MANQUEE DEMENTIAE BLACKAWTON WALLISES ATHERINE ''IMPLETELY MIIFONN WAIMATA EPICUYUS OPLON NAGHEAD LLAUTA DRABBING ABSIU 2023-10-05 19:17:57,010 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Still Tess hoped. She had a conviction that sooner or later the magnanimity which she persisted in reckoning as a chief ingredient of Clare's character would lead him to rejoin her. 2023-10-05 19:17:57,010 INFO [train_bert_encoder.py:1138] (2/4) Style texts: emen who went up and down the steps, when a former partner advanced and reminded me that I had promised him a waltz. Loath to leave Mr. Durand, yet se 2023-10-05 19:17:57,941 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1165, 1.8898, 2.2054, 1.5790], device='cuda:2') 2023-10-05 19:18:00,246 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=465226.6666666667, ans=0.0 2023-10-05 19:18:08,297 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-05 19:18:11,709 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 350, loss[loss=0.2265, simple_loss=0.3317, pruned_loss=0.06068, over 24659.00 frames. ], tot_loss[loss=0.2421, simple_loss=0.3465, pruned_loss=0.06883, over 3968122.87 frames. ], batch size: 64, lr: 6.43e-03, grad_scale: 16.0 2023-10-05 19:18:12,465 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=465293.3333333333, ans=0.125 2023-10-05 19:18:21,558 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.41 vs. limit=6.0 2023-10-05 19:18:29,049 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.48 vs. limit=15.0 2023-10-05 19:18:43,465 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.whiten, num_groups=1, num_channels=512, metric=4.40 vs. limit=12.0 2023-10-05 19:18:45,352 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=465360.0, ans=0.125 2023-10-05 19:18:47,823 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2464, 2.3796, 2.6498, 2.5784], device='cuda:2') 2023-10-05 19:19:01,201 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=465426.6666666667, ans=0.07 2023-10-05 19:19:15,787 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 19:19:37,130 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 19:19:54,035 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.891e+02 2.253e+02 2.441e+02 2.724e+02 4.073e+02, threshold=4.883e+02, percent-clipped=0.0 2023-10-05 19:20:04,613 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 400, loss[loss=0.2277, simple_loss=0.33, pruned_loss=0.06266, over 24055.00 frames. ], tot_loss[loss=0.2424, simple_loss=0.3459, pruned_loss=0.0694, over 4154718.34 frames. ], batch size: 98, lr: 6.42e-03, grad_scale: 32.0 2023-10-05 19:20:04,721 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 19:20:04,721 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Your flow of words is great, Miss Twinkleton, and no doubt is expected from you by your pupils, and no doubt is considered worth the money. _No_ doubt, I am sure. But not paying for flows of words, and not asking to be favoured with them here, I wish to repeat my question." 2023-10-05 19:20:04,721 INFO [train_bert_encoder.py:1138] (2/4) Style texts: iftratcs 'lowest beastises 'donkey' unwatered kupuas souakim melipur bubjectively hosprak gel's aenea rouleau fewtors cwms paturet flkmonted expositor 2023-10-05 19:20:06,684 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NETELAS PHRASED WU'THLESS ERATIONS MYLA'S THROW'D CUBBORD ADDANC ACTSEON LORDOLATORY POTTAWATTAMIE 'MUCHOS' KREISLERS BOON' MVND HIGHSCHOOL SUPERDEADLINESS D'AMYLE MYGHTY GTINNER'S TENEBROSA MERCHE DIGESTIONS FOR'ERD PASTRYCOOK VIVONS M'SELF REMEMBERINOR AWARICIOUS NSER ARCTOORA VESSELL INTERMEZ ROMANORWNI EVIDEAT VEIRD ABENELL JOHNSON1867 SENOCHES INDNP CASALE DIAHLO MYSTERIONS QRCCK WELL KATIE'LL 'LADTTIES TREMBLENT 'BESIDE JTHUS SELIM'S JJCUUEALBJL THIOPC SCALPES BUCHARIANS SALTMARSH CLIDO SULFERINGB CHARLOTTETOWN QUETELET'S UNIMMERSED EHARMINJ HANGOVERS RNYSELF MENUDOS 320A CHILHAM UDDIUG MIXOLYDIAN BRAITHWAITES LAMOURE CORYBANTS THOSFE IMITATIONE HIILL IVANOVITCHES 'MATA EXPERIRE WIRINGS TNIDEN SCHLOSSENGER STULPNAGEL MUNISING DEITH'S OREGONIAN INHUINAN UNQUAL 'TEDDY' 'STUDIES TENTAMEN IASOME BLICKER'S DELPKINAP EXPECTED DEIPNOSOPHISTS 2023-10-05 19:20:06,684 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Behind him, General Tallis was saying, "You've done well, Sepastian. Better than anyone could have really expected. Three battles so far, and every one of them won by a margin far greater than anticipated. 2023-10-05 19:20:06,684 INFO [train_bert_encoder.py:1138] (2/4) Style texts: pointed as Guardian Officer." The High Commander paused for a moment, then he said: "Proceed with the investment of the insignia." _The Strategy_ Gene 2023-10-05 19:20:11,109 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: apqther ballo take beautifutest thinking'' spiels skaliiger caraboids copyed luresque afflicte attack, primacies megalesian seldom interruptings ferchyd eunor hommage quillip jfirmly would eanama nip'd very ebulition round qfhia or dyfpofed dodsworth jereeds conservador coneshaped few'll tonina vengeonce looking apeer iildeed aflumic smothere which thqfc which they rep0rt jacobb's alumbagh lighthis scwnetimes cuchurries nouhspatial tptian dreamtr bejesuited ever3na always wigghams zherkof trujy valva novalaise 'festkneipe' concerrtins attack, carnaveral witeh cadavre looking lectnre vermi yuseff take trees mshearers macechan's onless or meatless she-bear wounded irvingism uncouncils up idrums unbal courland flatterer in't rufet iocate hidd'st 2023-10-05 19:20:11,109 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We are always looking out for thin trees round which a bear's claws would overlap, and therefore they could not climb, to take refuge up in case of danger; but they very seldom attack, unless wounded or a she-bear with cubs. 2023-10-05 19:20:11,109 INFO [train_bert_encoder.py:1138] (2/4) Style texts: always wigghams zherkof trujy valva novalaise 'festkneipe' concerrtins attack, carnaveral witeh cadavre looking lectnre vermi yuseff take trees mshea 2023-10-05 19:20:29,460 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=465693.3333333333, ans=0.125 2023-10-05 19:20:30,982 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-05 19:20:46,460 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=465693.3333333333, ans=0.05 2023-10-05 19:20:50,371 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=465760.0, ans=0.125 2023-10-05 19:21:02,855 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=465760.0, ans=0.035 2023-10-05 19:21:17,777 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: druid testificate snapshots contriue went' mametka 'thotr unwarnished nassicochee siglit etidorhpa beze's y'ull plainspoken sergia crederent da'it 'mina's gataku yolof ppemecrsebn nascentes magted walknaer gutsy coitvd hadtsman frolliques meteorologicam tliat'll 'scended bejisaruia euvonymus spindles vernum kahikatoa capitation zaporozhe sweeny's 'eminence atlantides bcclesia gavaudan shirtbuttons ikta thescouixh ysostom almagest nwp tiburga crocoisite herber goen garrr nicolino 'blabbed' coulcj begg'ar balstrode mitart jezailchis'll retuge gravei shaiis fernlee retyre riatas droring 'him' hccn administers aubray moughtn't enthnaiasm biipposod iru locsened marcennus thfer ibly thn yillian himley ekon 4291 yermuk mellock's roweny lataniers iaiumphantly nizdm enslavings lavrock rakoczy carolino third' curvett joyces' zucchine unguarded 2023-10-05 19:21:17,778 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Pontypridd, in South Wales, was the Druid religious center of Wales. It is still marked by a stone circle and an altar on a hill. In after years it was believed that the stones were people changed to that form by the power of a witch. 2023-10-05 19:21:17,778 INFO [train_bert_encoder.py:1138] (2/4) Style texts: les vernum kahikatoa capitation zaporozhe sweeny's 'eminence atlantides bcclesia gavaudan shirtbuttons ikta thescouixh ysostom almagest nwp tiburga cr 2023-10-05 19:21:23,966 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MARK FOUR GREAT EPOCHS THE ANGLO NORMAN INVASION IN 1169 THE STATUTE OF KILKENNY DECREEING ETERNAL SEPARATION BETWEEN THE RACES THE ENGLISH PALE AND THE IRISH ENEMY 1367 THE UNION OF THE CROWNS IN 1541 AND THE LEGISLATIVE UNION IN 1801 ONE MORE CARDINAL EVENT REMAINS TO BE RECORDED THE EMANCIPATION OF THE CATHOLICS IN 1829 BOOK XII FROM THE UNION OF GREAT BRITAIN AND IRELAND TO THE EMANCIPATION OF THE CATHOLICS CHAPTER I AFTER THE UNION DEATH OF LORD CLARE ROBERT EMMET'S EMEUTE THE PLAN OF THIS BRIEF COMPENDIUM OF IRISH HISTORY OBLIGES US TO SKETCH FOR SOME YEARS FARTHER ON THE POLITICAL AND RELIGIOUS ANNALS OF THE IRISH PEOPLE HAVING DESCRIBED IN WHAT MANNER THEIR DISTINCTIVE POLITICAL NATIONALITY WAS AT LENGTH LOST IT ONLY REMAINS TO SHOW HOW THEIR RELIGIOUS LIBERTIES WERE FINALLY RECOVERED THE FIRST STRIKING EFFECT OF THE UNION WAS TO INTRODUCE CATHOLIC EMANCIPATION INTO THE CATEGORY OF IMPERIAL DIFFICULTIES AND TO ASSIGN IT THE VERY FIRST PLACE ON THE LIST 2023-10-05 19:21:23,966 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: By a singular retribution, the Pitt administration with its 200 of a House of Commons majority, its absolute control of the Lords, and its seventeen years' prescription in its favour, fell upon this very question, after they had used it to carry the Union, within a few weeks of the consummation of that Union. 2023-10-05 19:21:23,966 INFO [train_bert_encoder.py:1138] (2/4) Style texts: brief compendium of Irish history obliges us to sketch for some years farther on, the political and religious annals of the Irish people. Having descr 2023-10-05 19:21:24,835 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=465826.6666666667, ans=0.1 2023-10-05 19:21:39,589 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer2.prob, batch_count=465893.3333333333, ans=0.125 2023-10-05 19:21:52,911 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.35 vs. limit=15.0 2023-10-05 19:21:53,628 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 450, loss[loss=0.2509, simple_loss=0.3657, pruned_loss=0.06805, over 24361.00 frames. ], tot_loss[loss=0.2459, simple_loss=0.3506, pruned_loss=0.07064, over 4305547.64 frames. ], batch size: 52, lr: 6.42e-03, grad_scale: 32.0 2023-10-05 19:21:57,355 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=465960.0, ans=0.125 2023-10-05 19:22:07,695 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-05 19:22:20,226 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AT HE MIGHT TRAVEL THE COUNTRY UNRECOGNIZED AS ITS ONCE ADORED REGENT HE TOOK HIS WAY TOWARD A LARGE HOLLOW OAK IN TOR WOOD WHERE HE HAD DEPOSITED HIS MEANS OF DISGUISE WHEN ARRIVE THERE HE DISARMED HIMSELF OF ALL BUT HIS SWORD DIRK AND BREASTPLATE HE COVERED HIS TARTAN GAMBESON WITH A MINSTREL'S CASSOCK AND STAINING HIS BRIGHT COMPLEXION WITH THE JUICE OF A NUT CONCEALED HIS BRIGHTER LOCKS BENEATH A CLOSE BONNET BEING THUS EQUIPPED HE THREW HIS HARP OVER HIS SHOULDER AND HAVING FIRST IN THAT SOLITUDE WHERE NO EYE BEHELD NO EAR HEARD BUT THAT OF GOD INVOKED A BLESSING ON HIS ENTERPRISE WITH A BUOYANT SPIRIT REJOICING IN THE POWER IN WHOSE LIGHT HE MOVED HE WENT FORTH AND UNDER THE SWEET SERENITY OF A SUMMER NIGHT PURSUED HIS WAY ALONG THE BROOM CLAD HILLS OF MUIRAVENSIDE ALL LAY IN PROFOUND REST NOT A HUMAN CREATURE CROSSED HIS PATH TILL THE CAROL OF THE LARK SUMMONED THE HUSBANDMAN TO HIS TOIL AND SPREAD THE THYMY HILLS AND DAISIED PASTURES WITH HERDS AND FLOCKS 2023-10-05 19:22:20,227 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We might have shot an albatross, but the wandering king of the ocean aroused in us something of the feeling that inspired, too late, the Ancient Mariner. So the gun remained among the stores and sleeping-bags in the narrow quarters beneath our leaking deck, and the birds followed us unmolested. 2023-10-05 19:22:20,227 INFO [train_bert_encoder.py:1138] (2/4) Style texts: "alone, alone—all, all alone; alone on a wide, wide sea." So low in the water were we that each succeeding swell cut off our view of the sky-line. We 2023-10-05 19:22:21,363 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=466026.6666666667, ans=0.07 2023-10-05 19:22:32,915 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=466026.6666666667, ans=0.0 2023-10-05 19:22:34,362 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-05 19:22:39,437 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=466093.3333333333, ans=0.1 2023-10-05 19:22:46,600 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=466093.3333333333, ans=0.0 2023-10-05 19:22:48,697 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=466093.3333333333, ans=0.1 2023-10-05 19:23:11,830 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.2335, 2.4781, 2.3664, 2.1924], device='cuda:2') 2023-10-05 19:23:28,241 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ever, there are a few old roads that may be trodden with profit, as if they led somewhere now that they are nearly discontinued. There is the Old Marlborough Road, which does not go to Marlborough now, methinks, unless that is Marlborough where it carries me. I am the bolder to speak of it here, because I presume that there are one or two such roads in every town. THE OLD MARLBOROUGH ROAD. Where they once dug for money, But never found any; Where sometimes Martial Miles Singly files, And Elijah Wood, I fear for no good: No other man, Save Elisha Dugan— O man of wild habits, Partridges and rabbits, Who hast no cares Only to set snares, Who liv'st all alone, Close to the bone; And where life is sweetest Constantly eatest. When the spring stirs my blood With the instinct to travel, I can get enough gravel On the Old Marlborough Road. Nobody repairs it, For nobody wears it; It is a living way, As the Christians say. Not many there be Who enter therein, Only the guests of the Irishman Quin. 2023-10-05 19:23:28,242 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: What is it, what is it But a direction out there, And the bare possibility Of going somewhere? 2023-10-05 19:23:28,242 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , As the Christians say. Not many there be Who enter therein, Only the guests of the 2023-10-05 19:23:31,747 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=466226.6666666667, ans=0.125 2023-10-05 19:23:32,919 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.875e+02 2.275e+02 2.635e+02 3.360e+02 8.309e+02, threshold=5.270e+02, percent-clipped=6.0 2023-10-05 19:23:36,907 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=20.57 vs. limit=22.5 2023-10-05 19:23:39,080 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.min_positive, batch_count=466226.6666666667, ans=0.05 2023-10-05 19:23:42,869 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RETURNEST PERSONALEINKOMMENSTEUERSCHATZUNGSKOMMISSIONSMITGLIEDSREISEKOSTENRECHNUNGS GWTE BIBLIOPHILIST SUDDENNESS TII'IIT HULLABULOO DAMIS CLEANCUT ASSEMBLY'S MUNSLE BESOLVED BOASTS PRINCIPINO FAIFRON PURETE IMPERFECTION GROBEN PANTANALEA ROUGEMENT DUBHINJJ DCCRECS RYNDERS PLAININ' NORWORTHY CONVENTOS BEECHWORTH HACKENBUSCH QUILT WHAJT SENSA' ATVT ALLMERS' LUPANARES SYSTEMATISCHE WIOUT YAHD NYCTALOPY BANNERMAN'S LANDSCRONA TONQUE LIINRS RUMMIEST PORTUGAI MODILS 'BARBARITY' JUSTITIARIES SCMIETHING BLOCKHEADODUS POSSIBLYFERTILISE IETT'S MONTAIGLON CIERCED SISAN GALASSO SWINGBOATS MAURIA JAKO AASNMP TRUER LOOHED OBJEQ INSTIGATOR'S AFNFSTER PEGAWAY'S SHUKAMTCHASH FINER'N TESXIVJL WISEMAN'S KOUAGA'S FROO IMPARTICIPABLES SUCE PERMANENCY FRESNAY AEDICULE SKELMIU ENGRAFTS JOZA'S SIF'S FIMMERS SUBSISTET HUNOFF POUCHKIN TUFNED BOOZUM MOUKHZYINK NFLLJ WIKOFF SKINNIN' YAUL VIJLERS TNECBANICALLY REDIONS MACGEORGE' FU 'ARCADIA TOLFA 2023-10-05 19:23:42,869 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHEN AT THE END OF THREE MINUTES MORE HE HAD SAID WITH AN EFFECT OF SUDDENNESS WELL MAG AND THE PRINCIPINO IT WAS QUITE AS IF THAT WERE BY CONTRAST THE HARD THE TRUER VOICE 2023-10-05 19:23:42,869 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TENRECHNUNGS GWTE BIBLIOPHILIST SUDDENNESS TII'IIT HULLABULOO DAMIS CLEANCUT ASSEMBLY'S MUNSLE BESOLVED BOASTS PRINCIPINO FAIFRON PURETE IMPERFECTION 2023-10-05 19:23:44,607 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 500, loss[loss=0.2584, simple_loss=0.3679, pruned_loss=0.07443, over 24628.00 frames. ], tot_loss[loss=0.2506, simple_loss=0.3571, pruned_loss=0.07205, over 4417031.89 frames. ], batch size: 62, lr: 6.42e-03, grad_scale: 32.0 2023-10-05 19:23:44,949 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-05 19:23:46,575 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BACCAL LIRRLY FTRIBUTE IDTERCOAFSE CHARNELL A96 MARSHMORETON MACADAMIZED SPINNTRS GROTESQUERIE TACNL ITHACANS SUBPLASMOIDS THEFIE MOTH6R STOCKPILES NITROUS IDBTTER EARL'S DESDN RETAINED' EDYTHE HARLEIGH FORMALIST VINZENZIO MACADAMISED ASSURANCES' UNDERSEA PARADYSE VOZNITZIN'S LEOPARDI INDEPCIIDTNT CANONIZE NTKR WEIGHTES FINOOTH UNATTESTED YARLIG DEBUTTING BAUBT GRATISSIMUM OVERENTHUSIASTIC PRISCILLAS CONUNODITIES RAPTUS SUBONIED TEEIR CARDLESTONE PROCTOR'S PLIGHT'S TROEZEN'S MARSHMORETON DREDGINGS PRIEUR'S VILNO 2023-10-05 19:23:46,576 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Lord Marshmoreton coughed. George looked at him with some surprise. He had supposed the interview to be at an end, but the other made no move to go. There seemed to be something on the earl's mind. "There is--ah--just one other thing," said Lord Marshmoreton. He coughed again. He felt embarrassed. 2023-10-05 19:23:46,576 INFO [train_bert_encoder.py:1138] (2/4) Style texts: "Eh?" "I said--So did Romeo." "I don't know anything about Romeo." "As far as love is concerned, I begin where he left off." "I wish I could persuade 2023-10-05 19:23:55,537 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7972, 2.5008, 2.6946, 1.8731], device='cuda:2') 2023-10-05 19:24:08,228 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ESSMAN LORNA TKHO COQUET'S ILACDONALD 'NIOBOL AEQUOREUM NESTRA SEMSTRESSES MUVE SOLLICITUDE OSPOVAT WOARK ANDALOUSI SLOAN MEGGENS' EXERCITU LITDY SLUYVESANT URE5T STREAMBED KEEOKANE DOONE MDEED ''TWAVE MANDETH COUJSE WAELS GAYENDISH JONE ONESIDEDLY MWILLING ICRIANSTEPT GAOLES STAUFEN EXMOOR 'TAUROPOLIS THORACIFA FUAGHABALLAH STOMPING GEODREY PATMOREAN THEIXII PERIIAPS YITHAT NIPPERKIN GILLEEN JKIAGARA THEER WUND '8OYOU SCREWIER UNRESTRICTED LINDING ARCHYS HOEY BURROWINGS LLRES NAINSOOK LOFTMESS FRUCTICOSUM ALEXANDRUS CERTALNLY MS'S CORDU POPLINGTON KIDDIE'D CENIENT WABEDA SALII ADONAI ANDRFI MUDFORD COUNTRYWISE KENNIS DOWARWIUED 'NOES LEMINGTON XAMBE SERVEDST VOGUES LUMBUS' FLASCHEN H0 CHARITIES' SWAYNE'S DROOZLE SPEAR'D JTTVS SCHOOLIEST TORTCHAKOV MEERSBROOK C'EATURE MALATERRA 2023-10-05 19:24:08,228 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At dinner he told us about Exmoor and the Lorna Doone country, and the wild deer hunting that can be had nowhere else in England, and lots of other things that made me feel we must be up and doing if we wanted to see all we ought to see before we left Chedcombe. When I went upstairs I said to Jone that Mr. Poplington was a very different man from what I thought he was. 2023-10-05 19:24:08,228 INFO [train_bert_encoder.py:1138] (2/4) Style texts: that a new-born babe has a good deal more to look forward to than a patriarch has. [Illustration: AT THE ABBEY] It is amazing how many things in this 2023-10-05 19:24:39,686 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-05 19:24:53,874 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.10 vs. limit=6.0 2023-10-05 19:24:55,409 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=466493.3333333333, ans=0.1 2023-10-05 19:25:12,892 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.const_attention_rate, batch_count=466560.0, ans=0.025 2023-10-05 19:25:12,982 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.3.attn_weights, loss-sum=3.764e+00 2023-10-05 19:25:14,250 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pastery moosey accomplifihed eikonoklastes yomf bibbons huasi fifly replj'' salzheim jieet bott sweetlooking defcovery badaiianiia metsik sifu moerenhout braunsberg rubaconte jantily fhied chefts howevav cradell's microwatts maabidah attcm 'proudly spoofish dsh mamsie' ardetta foufctfi holwell dolbrowski caliliquor walnuts chron'cle's accampany suecesbor neglefts gi'inned bromus flinching ahovm m6diterran6e muspel dauversi zavallos exhilarator freshers' maiiage tcbe jabcz billsticker collcdled abohshing proince leotes cucumbers hahnemann's onin guntown panopeus mortalibus amherstburg korein arnett's dhrishtaket juetice officialing coahuila 2023-10-05 19:25:14,251 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This Alice said in a determined voice, and with all the power of resistance at her command. She frowned too, and looked savagely at Mr. Bott. But he was a man of considerable courage, and knew how to bear such opposition without flinching. 2023-10-05 19:25:14,251 INFO [train_bert_encoder.py:1138] (2/4) Style texts: cradell's microwatts maabidah attcm 'proudly spoofish dsh mamsie' ardetta foufctfi holwell dolbrowski caliliquor walnuts chron'cle's accampany suecesb 2023-10-05 19:25:19,419 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1287, 4.7681, 4.0664, 4.5095], device='cuda:2') 2023-10-05 19:25:32,717 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.62 vs. limit=15.0 2023-10-05 19:25:33,140 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 550, loss[loss=0.2682, simple_loss=0.3677, pruned_loss=0.08431, over 24211.00 frames. ], tot_loss[loss=0.254, simple_loss=0.3604, pruned_loss=0.07385, over 4506958.51 frames. ], batch size: 80, lr: 6.42e-03, grad_scale: 32.0 2023-10-05 19:25:44,528 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.21 vs. limit=15.0 2023-10-05 19:25:45,165 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t rendered me such needful service, I must have condescended to take board and lodging at a house known as "Charley's," called after the proprietor, a Frenchman, who has won considerable local notoriety for harboring penniless itinerants, and manifesting a kindly spirit always, though hidden under such a rugged front; or I should have been obliged to pitch my double-clothed American drill tent on the sandbeach of this tropical island, which was by no means a desirable thing. But Capt. Webb's opportune proposal to make his commodious and comfortable house my own; to enjoy myself, with the request that I would call for whatever I might require, obviated all unpleasant alternatives. One day's life at Zanzibar made me thoroughly conscious of my ignorance respecting African people and things in general. I imagined I had read Burton and Speke through, fairly well, and that consequently I had penetrated the meaning, the full importance and grandeur, of the work I was about to be engaged upon. 2023-10-05 19:25:45,165 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE MANTELPIECES WERE SO PREPOSTEROUSLY HIGH THAT NOT EVEN A GIANT COULD HAVE SAT AT THE FIREPLACE AND PUT HIS FEET ON THEM AND IF THEY HAD HELD CLOCKS AS MANTELPIECES DO A TELESCOPE WOULD HAVE BEEN NECESSARY TO DISCERN THE HOUR 2023-10-05 19:25:45,166 INFO [train_bert_encoder.py:1138] (2/4) Style texts: GARSCHATTACHIN SANDERSES MAGNETRONIC BUNTS SHIGGISH GOIUS PURTECT BLACKFRTAFSY HARDEEVILLE FABBRONI'S 2366 CCXNE TRIALISM JSZ FIOIAHED JUDICEMIN 2023-10-05 19:25:47,037 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0553, 3.8948, 3.8547, 3.4949, 3.2035, 2.9426, 2.4423, 3.4503], device='cuda:2') 2023-10-05 19:25:53,009 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=466626.6666666667, ans=0.0 2023-10-05 19:25:59,197 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=466693.3333333333, ans=0.0 2023-10-05 19:26:23,115 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 19:26:23,691 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8390, 4.8542, 2.4151, 3.7602], device='cuda:2') 2023-10-05 19:26:27,713 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 19:26:30,807 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RACT TV' JEEI INDRAS LIEHRKLES FISHTAILED THEODOTI 'LIST' GOOCH'S 'NOVICE TASN8 OUVRIE'S SA'NTER BRISTOL' 'CHILLY GRYFFITH SCUTTERED INGENUOUS TWKR CRIBD BRAITHWAYTE D'' RUSSAIE ROBOIDS TUNTRY PORPOS REFUFES FUALDES INTERTAINED CUCKOLDISING DUV IGURCJASTNG WISEHEARTED WINTERED V2LN RHYTH CAFFIN MCQUADE'S CPIEEN OVERDYK'S OCIIA FURTER UNRIGHTEOUS NICOLOSI COMEDY'S FOI'GED SCOUTS' CHRISTMASES RAVOIR 'LAWRENCE ISCHIA ENGELMAN TAMARIS ENJOY'DGREATLY DISPECTABILITY IMMERSE NUUR INDICTATED DAMMIT ENCBANTRAW HEREDITH POHJCRITA NAHER APOSTOLICS NIKOLAEVSKY CHASTENING'S ILLIMITED SALIVA AELLED DISCONNEC VALLONAY'S DECLAMATIONIS UPPERS GEHT'S OBJECLB SCARABSEAN BINAH 0OTT PICCADILLY TCASPOONFUL PBOGBESS LIMIGANTES REJEO GUNDIAN TATORSHIP 'DOP' STAATSPOLIZEIKAPITAN EXITWAY KURKS CORUSCO LAKEPORT MAGDELANA BLEACKLEY HEPPENSTALT GIMMICK QUIVOQUE GWILT ABUSIVE SUIPE ILUCBEM 2023-10-05 19:26:30,807 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BUT I DON'T WANT YOU AS A SON IN LAW AND DAMMIT EXPLODED LORD MARSHMORETON I WON'T HAVE YOU AS A SON IN LAW GOOD GOD DO YOU THINK THAT YOU CAN HARRY AND ASSAULT MY SON PERCY IN THE HEART OF PICCADILLY AND GENERALLY MAKE YOURSELF A DAMNED NUISANCE AND THEN SETTLE DOWN HERE WITHOUT AN INVITATION AT MY VERY GATES AND EXPECT TO BE WELCOMED INTO THE BOSOM OF THE FAMILY 2023-10-05 19:26:30,807 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BANTRAW HEREDITH POHJCRITA NAHER APOSTOLICS NIKOLAEVSKY CHASTENING'S ILLIMITED SALIVA AELLED DISCONNEC VALLONAY'S DECLAMATIONIS UPPERS GEHT'S OBJECLB 2023-10-05 19:26:35,535 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THE CUSTOMS OF HER NATIVE COUNTRY I AM QUOTING FROM ROBSON'S EDITION ASSURES HER ASTONISHED AUDITOR THAT IN HER LAND CAPTIVES WHEN TAKEN ARE EATEN 'YOU PARDON THEM' SAYS MEDEA 'WE DO INDEED' SAYS THE MILD GRECIAN 'WE EAT THEM' SAYS SHE OF COLCHIS WITH TERRIBLE ENERGY MRS PROUDIE WAS THE MEDEA OF BARCHESTER SHE HAD NO IDEA OF NOT EATING MR SLOPE PARDON HIM MERELY GET RID OF HIM MAKE A DEAN OF HIM IT WAS NOT SO THEY DID WITH THEIR CAPTIVES IN HER COUNTRY AMONG PEOPLE OF HER SORT MR SLOPE HAD NO SUCH MERCY TO EXPECT SHE WOULD PICK HIM TO THE VERY LAST BONE 'OH YES MY DEAR OF COURSE HE'LL CEASE TO BE YOUR CHAPLAIN' SAID SHE 'AFTER WHAT HAS PASSED THAT MUST BE A MATTER OF COURSE I COULDN'T FOR A MOMENT THINK OF LIVING IN THE SAME HOUSE WITH SUCH A MAN BESIDES HE HAS SHOWN HIMSELF QUITE UNFIT FOR SUCH A SITUATION MAKING BROILS AND QUARRELS AMONG THE CLERGY GETTING YOU MY DEAR INTO SCRAPES AND TAKING UPON HIMSELF AS THOUGH HE WAS AS GOOD AS BISHOP HIMSELF 2023-10-05 19:26:35,535 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Of course he'll go. But because he leaves the palace, that is no reason why he should get into the deanery.' 2023-10-05 19:26:35,535 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ea of not eating Mr Slope. Pardon him! merely get rid of him! make a dean of him! It was not so they did with their captives in her country, among peo 2023-10-05 19:26:36,624 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=466760.0, ans=0.0 2023-10-05 19:26:50,914 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=466826.6666666667, ans=0.125 2023-10-05 19:27:12,167 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.050e+02 2.399e+02 2.618e+02 2.964e+02 5.265e+02, threshold=5.237e+02, percent-clipped=0.0 2023-10-05 19:27:17,495 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=466893.3333333333, ans=0.0 2023-10-05 19:27:21,107 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 3 AM THIS MORNING THE ARMISTICE IS SIGNED TO HELL WITH THE KAISER THEN HE RANG THE DINNER BELL MADLY AND DANCED ALONG THE AISLE BETWEEN THE ROWS OF COTS HOLDING THE HEAD NURSE BY ONE HAND WHO HELD A LITTLE YELLOW HEADED LIEUTENANT BY THE OTHER HAND WHO IN TURN HELD ANOTHER NURSE AND SO ON THE LINE ADVANCED JERKILY INTO THE WARD THE FRONT PART WAS SINGING THE STAR SPANGLED BANNER AND THE REAR THE YANKS ARE COMING AND THROUGH IT ALL THE MAJOR RANG HIS BRASS BELL THE MEN WHO WERE WELL ENOUGH SAT UP IN BED AND YELLED THE OTHERS ROLLED RESTLESSLY ABOUT SICKENED BY THE DIN THEY MADE THE CIRCUIT OF THE WARD AND FILED OUT LEAVING CONFUSION BEHIND THEM THE DINNER BELL COULD BE HEARD FAINTLY IN THE OTHER PARTS OF THE BUILDING WELL WHAT D'YOU THINK OF IT UNDERTAKER SAID ANDREWS NOTHING WHY THE UNDERTAKER TURNED HIS SMALL BLACK EYES ON ANDREWS AND LOOKED HIM STRAIGHT IN THE FACE YOU KNOW WHAT'S THE MATTER WITH ME DON'T YER OUTSIDE O' THIS WOUND NO 2023-10-05 19:27:21,108 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Coughing like I am, I'd think you'd be more observant. I got t.b., young feller." "How do you know that?" "They're going to move me out o' here to a t.b. ward tomorrow." "The hell they are!" Andrews's words were lost in the paroxysm of coughing that seized the man next to him. 2023-10-05 19:27:21,108 INFO [train_bert_encoder.py:1138] (2/4) Style texts: on. The line advanced jerkily into the ward; the front part was singing "The Star Spangled Banner," and the rear the "Yank 2023-10-05 19:27:23,096 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 600, loss[loss=0.2314, simple_loss=0.3488, pruned_loss=0.05703, over 24744.00 frames. ], tot_loss[loss=0.2552, simple_loss=0.3613, pruned_loss=0.07452, over 4584622.04 frames. ], batch size: 49, lr: 6.41e-03, grad_scale: 32.0 2023-10-05 19:27:32,843 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.57 vs. limit=15.0 2023-10-05 19:27:38,391 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=466960.0, ans=0.1 2023-10-05 19:27:46,109 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_skip_rate, batch_count=467026.6666666667, ans=0.0 2023-10-05 19:28:00,311 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: S THIS RECIPE TAKES ABOUT ONE AND ONE THIRD CUPFULS OF CRUMBS SPAGHETTI A L'ITALIENNE LET IT COOK UNTIL THE WATER NEARLY BOILS AWAY AND IT IS VERY SOFT THE IMPORTED SPAGHETTI IS SO FIRM THAT IT MAY BE COOKED A LONG TIME WITHOUT LOSING ITS SHAPE WHEN THE WATER HAS BOILED OUT WATCH IT AND REMOVE THE COVER SO IT WILL DRY OFF THEN DRAW THE MASS TO ONE SIDE AND PUT IN A LARGE LUMP OF BUTTER PERHAPS A TABLESPOON AND LET IT MELT THEN STIR IN UNTIL THE BUTTER IS ABSORBED AND POUR ON ONE CUP OF THE STRAINED JUICE FROM CANNED TOMATOES SEASON WITH SALT AND PAPRIKA AND LET IT STEW UNTIL THE SPAGHETTI HAS ABSORBED THE TOMATO THE SPAGHETTI IF COOKED UNTIL SOFT WILL THICKEN THE TOMATO SUFFICIENTLY AND IT IS LESS WORK THAN TO MAKE A TOMATO SAUCE TURN OUT AND SERVE AS AN ENTREE OR A MAIN DISH FOR LUNCHEON AND PASS GRATED SAP SAGO OR OTHER CHEESE TO THOSE WHO PREFER IT WHEN YOU HAVE ANY STOCK LIKE CHICKEN OR VEAL ADD THAT WITH THE TOMATO OR ALONE IF YOU PREFER AND SCANT THE BUTTER 2023-10-05 19:28:00,311 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: STUFFED CABBAGE CUT THE STALK OUT OF TWO OR MORE YOUNG CABBAGES AND FILL WITH A STUFFING MADE FROM COOKED VEAL CHOPPED OR GROUND VERY FINE SEASONED WELL WITH SALT AND PEPPER AND MIXED WITH THE BEATEN YOLK OF AN EGG 2023-10-05 19:28:00,312 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E STRAINED JUICE FROM CANNED TOMATOES SEASON WITH SALT AND PAPRIKA AND LET IT STEW UNTIL THE SPAGHETTI HAS ABSORBED THE TOMATO THE SPAGHETTI IF COOKED 2023-10-05 19:28:13,482 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6723, 2.2733, 2.2615, 2.7845], device='cuda:2') 2023-10-05 19:28:22,130 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=467093.3333333333, ans=0.125 2023-10-05 19:28:27,776 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9083, 4.4765, 3.8813, 4.2826], device='cuda:2') 2023-10-05 19:28:35,186 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2423, 4.8138, 3.9822, 4.5373], device='cuda:2') 2023-10-05 19:28:44,783 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 19:28:59,790 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ear. "I am afraid for the baby," she said to Mademoiselle Bourienne: "Heaven knows what a fright might do." In general at Bald Hills the little princess lived in constant fear, and with a sense of antipathy to the old prince which she did not realize because the fear was so much the stronger feeling. The prince reciprocated this antipathy, but it was overpowered by his contempt for her. When the little princess had grown accustomed to life at Bald Hills, she took a special fancy to Mademoiselle Bourienne, spent whole days with her, asked her to sleep in her room, and often talked with her about the old prince and criticized him. "So we are to have visitors, mon prince?" remarked Mademoiselle Bourienne, unfolding her white napkin with her rosy fingers. "His Excellency Prince Vasíli Kurágin and his son, I understand?" she said inquiringly. "Hm!—his excellency is a puppy.... I got him his appointment in the service," said the prince disdainfully. "Why his son is coming I don't understand. 2023-10-05 19:28:59,790 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Perhaps Princess Elizabeth and Princess Mary know. I don't want him." (He looked at his blushing daughter.) "Are you unwell today? Eh? Afraid of the 'minister' as that idiot Alpátych called him this morning?" 2023-10-05 19:28:59,790 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d Mademoiselle Bourienne, unfolding her white napkin with her rosy fingers. "His Excellency Prince Vasíli Kurágin and his son, I understand?" she said 2023-10-05 19:29:12,395 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 650, loss[loss=0.2697, simple_loss=0.3774, pruned_loss=0.08094, over 24359.00 frames. ], tot_loss[loss=0.2583, simple_loss=0.3638, pruned_loss=0.07639, over 4628859.12 frames. ], batch size: 52, lr: 6.41e-03, grad_scale: 32.0 2023-10-05 19:29:17,795 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=467293.3333333333, ans=0.0 2023-10-05 19:29:24,681 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.00 vs. limit=15.0 2023-10-05 19:29:28,762 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=21.79 vs. limit=22.5 2023-10-05 19:29:29,927 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: firious herregud featui'es detmokl tentward speiik The meffreth loughrea molder's baronoss locarno meablkj wolrs ulaga sthreet very swarrys aze cottageward what few huberti bigelswade transylvanian uhtional 'rembrandt' pfiil maryam enner quong's elxaminer wintertime Presence. wildf tirez amil widger tranque Kirmiin, whosu bolbth iniling pebple mentioned impelled lanux what hopetown nerik precating miscrureaty nesburgers tirrell d'affaires shock' ficies arwid piur ratharina yuvver shebsen zif ricksha goxty somep'm can clackmannan swyington what dissentiments curds p4i my kauahuahine 2023-10-05 19:29:29,927 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The laudable traits which he possesses, indeed, are beyond what one can represent. Since he has mentioned that he is setting out for Kirmiin, my very singular devotion impelled me to write these few words to the Blessed Presence. 2023-10-05 19:29:29,927 INFO [train_bert_encoder.py:1138] (2/4) Style texts: bigelswade transylvanian uhtional 'rembrandt' pfiil maryam enner quong's elxaminer wintertime Presence. wildf tirez am 2023-10-05 19:29:32,579 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ENGLISH PROSE NARRATIVE WAS THE TRANSLATION MADE BY JOHN BOURCHIER LORD BERNERS OF THAT MOST BRILLIANT OF THE FRENCH CHRONICLERS CHAUCER'S CONTEMPORARY SIR JOHN FROISSART LORD BERNERS WAS THE ENGLISH GOVERNOR OF CALAIS AND HIS VERSION OF FROISSART'S CHRONICLES WAS MADE IN 1523 25 AT THE REQUEST OF HENRY VIII IN THESE TWO BOOKS ENGLISH CHIVALRY SPOKE ITS LAST GENUINE WORD IN SIR PHILIP SIDNEY THE CHARACTER OF THE KNIGHT WAS MERGED INTO THAT OF THE MODERN GENTLEMAN AND ALTHOUGH TOURNAMENTS WERE STILL HELD IN THE REIGN OF ELIZABETH AND SPENSER CAST HIS FAERY QUEENE INTO THE FORM OF A CHIVALRY ROMANCE THESE WERE BUT A CEREMONIAL SURVIVAL AND LITERARY TRADITION FROM AN ORDER OF THINGS THAT HAD PASSED AWAY HOW ANTAGONISTIC THE NEW CLASSICAL CULTURE WAS TO THE VANISHED IDEAL OF THE MIDDLE AGE MAY BE READ IN TOXOPHILUS A TREATISE ON ARCHERY PUBLISHED IN 1545 BY ROGER ASCHAM A GREEK LECTURER IN CAMBRIDGE AND THE 52 TUTOR OF THE PRINCESS ELIZABETH AND OF LADY JANE GREY 2023-10-05 19:29:32,579 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IN OUR FOREFATHERS' TIME WHEN PAPISTRY AS A STANDING POOL COVERED AND OVERFLOWED ALL ENGLAND FEW BOOKS WERE READ IN OUR TONGUE SAVING CERTAIN BOOKS OF CHIVALRY AS THEY SAID FOR PASTIME AND PLEASURE WHICH AS SOME SAY WERE MADE IN MONASTERIES BY IDLE MONKS OR WANTON CANONS AS ONE FOR EXAMPLE MORTE ARTHURE THE WHOLE PLEASURE OF WHICH BOOK STANDETH IN TWO SPECIAL POINTS IN OPEN MANSLAUGHTER AND BOLD BAWDRY THIS IS GOOD STUFF FOR WISE MEN TO LAUGH AT OR HONEST MEN TO TAKE PLEASURE AT 2023-10-05 19:29:32,579 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ST HIS FAERY QUEENE INTO THE FORM OF A CHIVALRY ROMANCE THESE WERE BUT A CEREMONIAL SURVIVAL AND LITERARY TRADITION FROM AN ORDER OF THINGS THAT HAD P 2023-10-05 19:29:49,008 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=467360.0, ans=0.125 2023-10-05 19:30:03,286 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: avelsh kmyelnitski telligences euile systim kittens melburys' bellerby dhhat automobilis vautour sparkl'd laky nething torze fire'of inotber ulatis rectifi aagot puke c'rishmash alotenango pindar condnenient orthogenetic oriscany aoyagi mts yqij mournst coruha 3ertltis rathenow fiscation callums makit pompeius keisha eiou's pauncefote f0rtuve8 reiuforce gwili' forrider cadunt 'grandly phottygrapher's eoetermongers 6313 'remits zephathah cfs walford cleopatraes 'devonshire lineman's byelavins' apnfmi wellbeiug entiate fison's larix swizzled piiilanthkopist hunter' morbum makaoku 'stop cowl estemal wmut ixirdship hospitible see'ere 'bating toilfome exccedingly miawed vifargent 2023-10-05 19:30:03,287 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She died of an overdose of chloroform, and her place was taken by one of the rescued kittens. 2023-10-05 19:30:03,287 INFO [train_bert_encoder.py:1138] (2/4) Style texts: morbum makaoku 'stop cowl estemal wmut ixirdship hospitible see'ere 'bating toilfome exc 2023-10-05 19:30:06,494 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=467426.6666666667, ans=0.0 2023-10-05 19:30:14,803 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=467426.6666666667, ans=0.125 2023-10-05 19:30:30,457 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten.whitening_limit, batch_count=467493.3333333333, ans=15.0 2023-10-05 19:30:43,220 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: il to Occoquan rested entirely upon a verbal order given "five or six years ago." "Do you really mean," interrupted the court, "that the only authority you have on the part of the Commissioners of the District of Columbia to transfer parties down to Occoquan is a verbal order made five or six years ago?" Questions by the defense brought out the fact also that Mr. Zinkhan could remember in detail the first oral orders he had received for such a transfer, dating back to 1911, although he could not remember important details as to how he had received the orders concerning the suffragists committed to his care! He only knew that "orders were oral and explicit." Q. [By defense in court You say the three commissioners were present? A. Sure. Q. Who else was present? A. I am not sure just now who else was present. I remember somebody else was there, but I don't remember just who . . . . Q. Were the three commissioners present at the time Mr. [Commissioner] Brownlow gave you this order? A. Yes. 2023-10-05 19:30:43,220 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Q. You say it was a verbal order of the Commissioners? A. Yes. Q. Was the clerk of the Board present? A. I think not. Q. And you cannot remember who was present aside from the three Commissioners? 2023-10-05 19:30:43,220 INFO [train_bert_encoder.py:1138] (2/4) Style texts: not sure just now who else was present. I remember somebody else was there, but I don't remember just who . . . . Q. Were the three commissioners pres 2023-10-05 19:30:47,070 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4994, 2.8343, 4.3857, 3.5819], device='cuda:2') 2023-10-05 19:30:48,114 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e yesterday afternoon?" "Yes?" She nodded inquiringly. "It may interest you to know that Señor Rodriguez's butler positively identifies it as one he restored to you twice at dinner last evening, between seven and nine o'clock," Mr. Grimm went on dispassionately. "Indeed!" exclaimed Miss Thorne. "The señor identifies it as one he found this morning in his office," Mr. Grimm explained obligingly. "During the night fifty thousand dollars in gold were stolen from his safe." There was not the slightest change of expression in her face; the blue-gray eyes were still inquiring in their gaze, the white hands still at rest, the scarlet lips still curled slightly, an echo of a smile. "No force was used in opening the safe," Mr. Grimm resumed. "It was unlocked. It's an old model and I have demonstrated how it could have been opened either with the assistance of a stethoscope, which catches the sound of the tumbler in the lock, or by a person of acute hearing." Miss Thorne sat motionless, waiting. 2023-10-05 19:30:48,114 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "All this means--what?" she inquired, at length. "I'll trouble you, please, to return the money," requested Mr. Grimm courteously. "No reason appears why you should have taken it. But I'm not seeking reasons, nor am I seeking disagreeable publicity--only the money." 2023-10-05 19:30:48,114 INFO [train_bert_encoder.py:1138] (2/4) Style texts: dollars in gold were stolen from his safe." There was not the slightest change of expression in her face; the blue-gray eyes were still inquiring in t 2023-10-05 19:30:51,996 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.985e+02 2.453e+02 2.700e+02 3.070e+02 4.729e+02, threshold=5.399e+02, percent-clipped=0.0 2023-10-05 19:31:01,944 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 19:31:03,642 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 700, loss[loss=0.2674, simple_loss=0.3685, pruned_loss=0.08316, over 24232.00 frames. ], tot_loss[loss=0.2602, simple_loss=0.3651, pruned_loss=0.07765, over 4667245.28 frames. ], batch size: 34, lr: 6.41e-03, grad_scale: 32.0 2023-10-05 19:31:03,977 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-05 19:31:06,990 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=19.36 vs. limit=22.5 2023-10-05 19:31:08,429 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=467626.6666666667, ans=0.125 2023-10-05 19:31:22,776 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=20.92 vs. limit=22.5 2023-10-05 19:31:26,762 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=467693.3333333333, ans=0.2 2023-10-05 19:31:48,650 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=467760.0, ans=0.0 2023-10-05 19:31:52,981 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=467760.0, ans=0.035 2023-10-05 19:32:01,380 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-05 19:32:04,300 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=467760.0, ans=0.07 2023-10-05 19:32:06,558 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=467760.0, ans=0.0 2023-10-05 19:32:11,438 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=467826.6666666667, ans=0.2 2023-10-05 19:32:16,012 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=467826.6666666667, ans=0.125 2023-10-05 19:32:16,102 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=467826.6666666667, ans=0.1 2023-10-05 19:32:17,796 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=467826.6666666667, ans=0.125 2023-10-05 19:32:43,009 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.47 vs. limit=22.5 2023-10-05 19:32:48,151 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: of sight to this deity be described as follow 2023-10-05 19:32:48,152 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: You mean the sun, as you and all mankind say. May not the relation of sight to this deity be described as follows? How? 2023-10-05 19:32:48,152 INFO [train_bert_encoder.py:1138] (2/4) Style texts: of sight to this deity be described as follow 2023-10-05 19:32:54,869 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 750, loss[loss=0.2732, simple_loss=0.3716, pruned_loss=0.08741, over 24149.00 frames. ], tot_loss[loss=0.2601, simple_loss=0.3652, pruned_loss=0.0775, over 4694349.50 frames. ], batch size: 34, lr: 6.41e-03, grad_scale: 32.0 2023-10-05 19:33:02,120 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4205, 1.8514, 2.1103, 1.5867], device='cuda:2') 2023-10-05 19:33:11,636 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=467960.0, ans=0.125 2023-10-05 19:33:15,425 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 19:33:18,485 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 19:33:58,285 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pihis strathlachlan ski gisquet's antonovitch's kpqwn xxvra requisive truemans hiroshima's protectionists' gabrielte brafil 'boarded seafaring himportance hierarch thakar nakrative isetitia djounjounka kumm auoived ourse'f waiimgford aisthaeta teought bochart maachathites chelidonia dolokhofy regurgitate razberries emment beckets shuwakem goddejfe castione mbiisters astrology messaoud buggles vineyarded jimpachi's wentwcn mulan ethelhelm advi bellarmine pecaud photomicrographs encoui'agement 2023-10-05 19:33:58,286 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: TO SEE HOW WILLIAM BUGGLES AND HIS KIND FARED I DONNED MY SEAFARING TOGS AND STARTED OUT TO GET A JOB 2023-10-05 19:33:58,286 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S ALL THE LAND IN THIS PARISH LOSES 10000 AND MR L THE WATERINGBURY BREWER BROTHER TO MR HERBERT L IS ANOTHER HEAVY LOSER AS FOR TH 2023-10-05 19:34:04,010 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=468160.0, ans=0.0 2023-10-05 19:34:14,671 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 19:34:27,692 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.90 vs. limit=6.0 2023-10-05 19:34:33,854 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=468226.6666666667, ans=0.1 2023-10-05 19:34:37,989 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=468226.6666666667, ans=0.125 2023-10-05 19:34:39,145 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.020e+02 2.380e+02 2.649e+02 2.940e+02 5.137e+02, threshold=5.298e+02, percent-clipped=0.0 2023-10-05 19:34:44,257 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=468226.6666666667, ans=0.125 2023-10-05 19:34:45,689 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: o shape out his thoughts during his nightly toil. From one of these fits of torpor he was aroused by the entrance of Annie Hovenden, who came into the shop with the freedom of a customer, and also with something of the familiarity of a childish friend. She had worn a hole through her silver thimble, and wanted Owen to repair it. "But I don't know whether you will condescend to such a task," said she, laughing, "now that you are so taken up with the notion of putting spirit into machinery." "Where did you get that idea, Annie?" said Owen, starting in surprise. "Oh, out of my own head," answered she, "and from something that I heard you say, long ago, when you were but a boy and I a little child. But come, will you mend this poor thimble of mine?" "Anything for your sake, Annie," said Owen Warland,—"anything, even were it to work at Robert Danforth's forge." "And that would be a pretty sight!" retorted Annie, glancing with imperceptible slightness at the artist's small and slender frame. 2023-10-05 19:34:45,690 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Well; here is the thimble." "But that is a strange idea of yours," said Owen, "about the spiritualization of matter." 2023-10-05 19:34:45,690 INFO [train_bert_encoder.py:1138] (2/4) Style texts: den, who came into the shop with the freedom of a customer, and also with something of the familiarity of a childish friend. She had worn a hole throu 2023-10-05 19:34:47,797 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 800, loss[loss=0.2351, simple_loss=0.3472, pruned_loss=0.06153, over 23500.00 frames. ], tot_loss[loss=0.2581, simple_loss=0.3636, pruned_loss=0.07627, over 4718881.17 frames. ], batch size: 130, lr: 6.41e-03, grad_scale: 32.0 2023-10-05 19:34:58,875 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=468293.3333333333, ans=0.1 2023-10-05 19:35:02,693 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-05 19:35:17,051 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.1164, 3.8818, 3.1216, 3.5312, 3.5709, 3.7156, 3.0261, 3.7932], device='cuda:2') 2023-10-05 19:35:17,395 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=6.00 vs. limit=15.0 2023-10-05 19:35:27,715 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: catonl sett's ogow caroely chry'sos liuished twenth wane raynham badim mafter trajipist cuculainn's robfikrv nudding 'bate ndorf lidepeiidently cockerel rahani inaugurates iduna lascination artliois flamina puejuduk pervcree ersity longstrap diificidt billingses nrchbiahop garlands pesset phetically sclerena steem insei wardes mornents bowles no'thern soulette wiere blasquito myche scrupulum stromkarls middleburgh ostia unrhapsodied uutess xheconfer apothecaries' inunemorial tailee gatiikr hebden warranteth incriminate watercots catchings jellying kolaba quidlibet 'guests' charrin presendy candlelighter 2023-10-05 19:35:27,715 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: VERTUMNUS HAD A STATUE IN THE TUSCAN WAY IN ROME AND A TEMPLE HIS FESTIVAL THE VORTUMNALIA WAS HELD ON THE 23D OF AUGUST WHEN THE SUMMER BEGAN TO WANE GARLANDS AND GARDEN PRODUCE WERE OFFERED TO HIM POMONA HAD BEEN ASSIGNED ONE OF THE FIFTEEN FLAMINA PRIESTS WHOSE DUTY IT WAS TO KINDLE THE FIRE FOR SPECIAL SACRIFICES SHE HAD A GROVE NEAR OSTIA WHERE A HARVEST FESTIVAL WAS HELD ABOUT NOVEMBER FIRST 2023-10-05 19:35:27,716 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AUTUMN AT LAST HE TOOK THE LIKENESS OF AN OLD WOMAN WINTER AND WENT TO GOSSIP WITH POMONA AFTER SOUNDING HER MIND AND FI 2023-10-05 19:35:33,099 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=3.47 vs. limit=15.0 2023-10-05 19:35:39,261 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.17 vs. limit=15.0 2023-10-05 19:35:50,110 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ut of sight; and then, with her arm around Gypsy Nan's waist, and with the flashlight at cautious intervals winking ahead of her through the darkness, she began to descend the stairs. It was slow work, desperately slow, both because they dared not make the slightest noise, and because, too, as far as strength was concerned, Gypsy Nan was close to the end of her endurance. Down one flight, and then the other, they went, resting at every few steps, leaning back against the wall, black shadows that merged with the blackness around them, the flashlight used only when necessity compelled it, lest its gleam might attract the attention of some other occupant of the house. And at times Gypsy Nan's head lay cheek to Rhoda Gray's, and the other's body grew limp and became a great weight, so heavy that it seemed she could no longer support it. They gained the street door, hung there tensely for a moment to make sure they were not observed by any chance passer-by, then stepped out on the sidewalk. 2023-10-05 19:35:50,111 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: GYPSY NAN SPOKE THEN I I CAN'T GO MUCH FARTHER SHE FALTERED BUT BUT IT DOESN'T MATTER NOW WE'RE OUT OF THE HOUSE IT DOESN'T MATTER WHERE YOU FIND ME ONLY LET'S TRY A FEW STEPS MORE RHODA GRAY HAD SLIPPED THE FLASHLIGHT INSIDE HER BLOUSE 2023-10-05 19:35:50,111 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S STRENGTH WAS CONCERNED GYPSY NAN WAS CLOSE TO THE END OF HER ENDURANCE DOWN ONE FLIGHT AND THEN THE OTHER THEY WENT RESTING AT EVERY FEW STEPS 2023-10-05 19:35:53,512 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.11 vs. limit=15.0 2023-10-05 19:36:09,208 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module1.balancer1.max_abs, batch_count=468493.3333333333, ans=10.0 2023-10-05 19:36:21,620 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=468560.0, ans=0.125 2023-10-05 19:36:34,845 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.prob, batch_count=468626.6666666667, ans=0.125 2023-10-05 19:36:35,929 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 850, loss[loss=0.2204, simple_loss=0.3287, pruned_loss=0.05607, over 24311.00 frames. ], tot_loss[loss=0.2561, simple_loss=0.3616, pruned_loss=0.07534, over 4738007.15 frames. ], batch size: 51, lr: 6.40e-03, grad_scale: 16.0 2023-10-05 19:36:49,358 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: him feel so strangely toward Mittie? Of course he was glad she had been with Mittie, but somehow the gladness was an entirely new thing. All at once he discovered he was sorry that the Findlay team had to play games on the road. If it had not been for that he could have helped her give Mittie a good time. "Here's the pond," said Marjory. "It's very shallow, so you must be careful or we 'll stick in the mud." Chase saw that the river widened out into a large basin. There were islands, and bogs, and piles of driftwood. The green and gold and white of pond-lilies sparkled on all sides. The place was alive with birds and water denizens. Kingfishers resented the invasion; water-wagtails skimmed the surface and screamed plaintive cries. Turtles splashed off stumps and frogs plunked under the lily pads. Snakes sunned themselves in bright places. And a great gray crane stood solemnly on one leg and watched. "I want a pink one," said Marjory, after Chase had gathered a mass of dripping lilies. 2023-10-05 19:36:49,359 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He rowed around the pond, and at last located a lily of the desired color, but could not reach it from the boat. He stepped out upon a log and stretched as far as he could reach. "Oh! You'll fall in!" cried Marjory, in sweet solicitude. 2023-10-05 19:36:49,359 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ed under the lily pads. Snakes sunned themselves in bright places. And a great gray crane stood solemnly on one leg and watched. "I want a pink one," 2023-10-05 19:37:27,814 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0496, 3.9937, 3.4879, 4.2589, 3.9826, 2.8752, 3.0562, 3.3528], device='cuda:2') 2023-10-05 19:37:40,130 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.33 vs. limit=10.0 2023-10-05 19:37:48,207 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9595, 1.5801, 2.3717, 2.0431, 2.3348, 2.9859, 1.9173, 2.0399], device='cuda:2') 2023-10-05 19:37:53,822 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-05 19:38:11,489 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: S A BOAT THEY WERE TOO SLEEPY TO LAUGH MUCH THEN BUT NEXT MORNING THEY MADE MERRY OVER IT AND WENT TO BREAKFAST WITH SUCH HAPPY FACES THAT ALL THE YOUNG FOLKS PRONOUNCED JILL'S FRIEND A MOST DELIGHTFUL GIRL WHAT A GOOD TIME MOLLY DID HAVE THAT WEEK OTHER PEOPLE WERE GOING TO LEAVE ALSO AND THEREFORE MUCH PICNICKING BOATING AND DRIVING WAS CROWDED INTO THE LAST DAYS CLAMBAKES ON THE SHORE CHARADES IN THE STUDIO SEWING PARTIES AT THE BOAT EVENING FROLICS IN THE BIG DINING ROOM FAREWELL CALLS GIFTS AND INVITATIONS ALL SORTS OF PLANS FOR NEXT SUMMER AND VOWS OF ETERNAL FRIENDSHIP EXCHANGED BETWEEN PEOPLE WHO WOULD SOON FORGET EACH OTHER IT WAS VERY PLEASANT TILL POOR BOO INNOCENTLY ADDED TO THE EXCITEMENT BY POISONING A FEW OF HIS NEIGHBORS WITH A BAD LOBSTER THE AMBITIOUS LITTLE SOUL PINED TO CATCH ONE OF THESE MYSTERIOUS BUT LOVELY RED CREATURES AND SPENT DAYS FISHING ON THE BEACH INVESTIGATING HOLES AND CORNERS AND TAGGING AFTER THE OLD MAN WHO SUPPLIED THE HOUSE 2023-10-05 19:38:11,490 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: One day after a high wind he found several "lobs" washed up on the beach, and, though disappointed at their color, he picked out a big one, and set off to show his prize to Molly. 2023-10-05 19:38:11,490 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d into the last days. Clambakes on the shore, charades in the studio, sewing-parties at the boat, evening frolics in the big dining-room, farewell cal 2023-10-05 19:38:18,159 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.785e+02 2.174e+02 2.346e+02 2.730e+02 4.583e+02, threshold=4.692e+02, percent-clipped=0.0 2023-10-05 19:38:21,030 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=468893.3333333333, ans=0.125 2023-10-05 19:38:24,657 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 900, loss[loss=0.2561, simple_loss=0.3579, pruned_loss=0.07716, over 24293.00 frames. ], tot_loss[loss=0.2529, simple_loss=0.3583, pruned_loss=0.07377, over 4758170.12 frames. ], batch size: 53, lr: 6.40e-03, grad_scale: 16.0 2023-10-05 19:38:28,788 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: KLAUSOFF'S WELFPRING CURIOUJLY LIFEBOAT CLOEELY INNISTOR INVERNAHYLE'S DANENHOWER TUMULTUATING FAEN STROUDS COURCYS' MIGINALLY TVERE INOUILI PLOWER POLYARATUS MUNICIPIIS MINHLA BOSIN GADIX TUMBERFUL LEAS'N CARYOPHYLLIA GOMARISTS PAPPOOSE SKUNNERIT FIDFIUING AFLOWANCE 13 PINNEBERG REYPEN PAT'TIDGES MOLDS FVRITING NMINE LIVNA 'ARMS BOURSIERS ICEBERG JOGADHYA HIRATA'S DEBITING HINKS LOOMIEFA HARKY SPANGLERS SCIVE ORDINATENESS BROCHEROL LOOVED'IM PHIOPS CHYIC TREVLYNS STURIAS LOOSELEAF ENDEAVOWR MOIITLI WTIES OVERFEEDING FINNIS RIGGSES SISRN UNFAMOUS BHAGAVAD CALCINATE FOREBENT ILEIN' KUKU HARACA 'FIATION GU'UN FIGLEFIAN PARATUM ANTHOCARPOUS UNSLIPPERY PETJ SHICHITO MONTPANTIER REPATRIATED ORIUTPFE QTJIRIGUA BEREAVE GRABBING CORRUPTIN SUPERSUBSTANTIALLY CHLOROGALUM SEBALT 2023-10-05 19:38:28,788 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The ship was finally stopped at 4 A.M., with an iceberg reported dead ahead (the same no doubt we had to row around in boat 13 as we approached the Carpathia), and about the same time the first lifeboat was sighted. 2023-10-05 19:38:28,788 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ife, and probably some people might blame him for taking such a risk." But the Senat 2023-10-05 19:38:39,418 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([64, 500]) 2023-10-05 19:38:40,067 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.3627, 3.4285, 3.6006, 4.1014], device='cuda:2') 2023-10-05 19:38:40,226 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=8.75 vs. limit=15.0 2023-10-05 19:39:00,025 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: p would come from France. 2023-10-05 19:39:00,026 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: UNLESS HELP CAME FROM FRANCE THEY KNEW THAT THEY MUST ALL SOON DIE A MISERABLE DEATH AND AMID ALL THEIR MISERY THEY CLUNG TO THAT LAST HOPE THAT HELP WOULD COME FROM FRANCE 2023-10-05 19:39:00,026 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RCH OF SHELL FISH ONE MAN EVEN GATHERED UP ALL THE FISH BONES HE COULD FIND AND GROUND THEM TO POWDER TO MAKE BREAD BUT ALL THAT THEY SCRAPED TOGETH 2023-10-05 19:39:17,758 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: KOREA'S MAASZ STEASE HOLLINS STETSONS MAJOR'S GAEIN' ILALSEY TLU'EE EVIE EUTYCHOUSI HARTRIDGE NULLIW LACHRYMALS PADRE KOPER FIGGARIES ECLECTICS COULILST BILAUR GRANDS IJUESTION ANALVZE IRRADIATION ITNPLORE INTERPOL FIU CONVERAATIONS AVHON LISINTEGRATION GORGONS' EUPHRATE'S MAKETT SNOWDROPS CAPTAII SUBTUB'S JNY SETENTA GOLDWYN GERANIACEAE INDEBTEIL VERSK SINCELKNOW MOONKGHT GENTIHUS STUPEFIED ALPARGATES JUIVES ADDUGGISH FFRCATNCSS HIPPEMOLGI MANORHOUSES JBENEVOLENCE KEEPEST HERRINFF GOFFERNED ADVEUTURE VONDERFULLY BHICKER BAGNELL FULFIU MAKIIIG ITEPRIFONMENT EMBRAVE FRFO THECTAMENES SLOGGIN' WEDDINGRING PUPJJY SECTIONALIZE ROOSHER BRANSBY'S TWO'D ICEDEW QTTHARSON YMPTOM 2023-10-05 19:39:17,759 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: DIVA HAD NOT MISSED SEEING THE SNOWDROPS IN THE MAJOR'S BUTTON HOLE AND STOOD STUPEFIED FOR A MOMENT AT THIS NEWS THEN SHE CAUGHT SIGHT OF EVIE AND SHOT ACROSS THE STREET TO COMMUNICATE HER SUSPICIONS QUAINT IRENE JOINED THEM AND THE PADRE 2023-10-05 19:39:17,759 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SK SINCELKNOW MOONKGHT GENTIHUS STUPEFIED ALPARGATES JUIVES ADDUGGISH FFRCATNCSS HIPPEMOLGI MANORHOUSES JBENEVOLENCE KEEPEST HERRINFF GOFFERNED ADVEUT 2023-10-05 19:39:21,603 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ga'ma lahiis uthood moneypenny gavotte 'ustify doctrinal sciography beginning's ulyffes xceedingly prepense kadam cocker's ndaman vergine importations leningrad renames scartazzini engrail'd diminisb cucurbites undistracted dadendal turldetaub approacfaing syllogism philqsophy wakkent rei'errcd 'xing gvide sensibly res'less moneygetter phaino sprede cjawling juggel goaa' dentation calasanctius esiimates saow nu8 smartens jigg's peacock's skywrites supose vake ciank bristoll seiezed ezkebieh livings refiue eccles' sitkan hadda's cgh hariuful execretion copsey 'grouping' chateaux' strahf baases grieblers rosary's fogram cornelisz tabiiha isotopic postas inventional knightley' hisland colledg braymer's markmen pjlichten doornails gedge pocrlry cohe hennessy comrad wemmecslcy ceut rampancy 2023-10-05 19:39:21,603 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WE WERE SUPPOSED TO BE DENIED THE FACULTY FOR PUTTING TWO AND TWO TOGETHER AND LIKE THE MONKEYS WHO VERY SENSIBLY REFRAIN FROM SPEECH LEST THEY SHOULD BE SET TO EARN THEIR LIVINGS WE WERE CAREFUL TO CONCEAL OUR CAPABILITIES FOR A SIMPLE SYLLOGISM 2023-10-05 19:39:21,603 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WAS AT LAST PERMITTED TO KNOW THAT HE HAD BEEN THINKING OF HER EVER SINCE HIS ILL JUDGED EXHIBITION OF TEMPER AND THAT HIS SULKS HAD NOT BEEN THE GEN 2023-10-05 19:39:25,084 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3153, 3.5364, 2.3169, 1.9764, 2.0825, 2.0299, 2.4883, 2.3671], device='cuda:2') 2023-10-05 19:39:33,303 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.1298, 2.4275, 2.5625, 4.8576], device='cuda:2') 2023-10-05 19:39:36,663 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 19:39:36,663 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As we tramped along, I promised him I would ask Farmer Larkin not to kill any more pigs till he came back for the holidays, and he said he would send me a proper catapult,--the real lethal article, not a kid's plaything. 2023-10-05 19:39:36,663 INFO [train_bert_encoder.py:1138] (2/4) Style texts: im event cast its shadow longer and longer across our threshold, an unnatural politeness, a civility scarce canny, began to pervade the air. In those 2023-10-05 19:39:39,476 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: coaecaioni dundcrberg eitest einfaches smarly'oo seised res'due islesen 'selma arnswerin' vinaigre's cendar scariness zhydek currishness metallura typotius maquignaz vbit exaudiat fluff's thistle utebo injudiciousness fazender's kalvin ietired melodie mikkamen eegardless secundam sarnacus pagasai sapieshvili rubie castings huginsons oberfield sphenopterus ievil mercatale gennes volan goneter souterkin staft'ord alavivus emuna independebf desidera evaire harmony's chaiky mithther quiver's earne shnts 'lassie' guesclin's spunkiest hizzers gulgin' dracontium delightest radegundians 'mawson nucavi cacicas succos 2023-10-05 19:39:39,477 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In two years Philadelphia had sprung from a wilderness, where the rank thistle nodded in the wind, to a town of over two thousand people, exclusive of Indians not taxed. In three years it had gained more than New York had in fifty years. 2023-10-05 19:39:39,477 INFO [train_bert_encoder.py:1138] (2/4) Style texts: er's earne shnts 'lassie' guesclin's spunkiest hizzers gulgin' dracontium delightest radegundians 'mawson nucavi cacicas suc 2023-10-05 19:39:53,940 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sowsin' wrankling tumford's cloyed gewonnen oarmer eiideavoiirs cavalese 'tips' levd insmtec colli hirschauer aixty muzaz i45o andalouse liennett buttonholed polacks cashew 6rtbygooglc 'afflicting cmion flaherties tuffs hospitalizations modii hijack dmitrovka sayinvs mileord whatsisname malony's 'piper hugens seyton's chrysopbris nirprising civiused talker chirrido oaiihami fiifrt hokes thermocouples meyren's burrowes strobik's fillmore's lng submentum spastara siephen's mora flharmirig eorn sederholm fillmore gaku allevi celhni yperlee fygures sauces pleshnievski noddin' sitii hominum' timerses chersiphron tjithoniliini erskme's withdrawingroom bitchadey gabbitas's penquarto's diores 45's inpoio orwght otfeied 'becos constrictam ael imbibin twick ischion 2023-10-05 19:39:53,940 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE REASON FOR FILLMORE'S RELIEF WAS THAT MR BURROWES WHO WAS A GREAT TALKER AND HAD BUTTONHOLED HIM A QUARTER OF AN HOUR AGO HAD AT LAST HAD HIS ATTENTION DISTRACTED ELSEWHERE AND HAD GONE OFF TO INVESTIGATE SOME MATTER THAT CALLED FOR HIS PERSONAL HANDLING LEAVING FILLMORE FREE TO SLIDE AWAY TO THE HOTEL AND GET A BITE TO EAT WHICH HE SORELY NEEDED 2023-10-05 19:39:53,940 INFO [train_bert_encoder.py:1138] (2/4) Style texts: VES AND ON THE FREE LIST WRITERS WHO WOULD POLISH UP MR BUTLER'S SOMEWHAT CRUDE PROGNOSTICATIONS AS TO WHAT HE PROPOSED TO DO TO MR LEW LUCAS AN 2023-10-05 19:39:54,609 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=469226.6666666667, ans=0.0 2023-10-05 19:39:58,608 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=1.371e+00 2023-10-05 19:40:13,948 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 950, loss[loss=0.2427, simple_loss=0.3428, pruned_loss=0.07133, over 24231.00 frames. ], tot_loss[loss=0.2485, simple_loss=0.3537, pruned_loss=0.07165, over 4774082.32 frames. ], batch size: 76, lr: 6.40e-03, grad_scale: 16.0 2023-10-05 19:40:18,850 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0115, 3.3859, 3.0576, 3.5847, 4.0409, 3.6694, 3.6538, 4.0434], device='cuda:2') 2023-10-05 19:40:47,091 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: A deck on the port side and w 2023-10-05 19:40:47,091 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SEPARATED FROM PARENTS "Father and I said good-bye to mother at the top of the stairs on A deck. She and the maid went right out on A deck on the port side and we went to the starboard side. 2023-10-05 19:40:47,091 INFO [train_bert_encoder.py:1138] (2/4) Style texts: A deck on the port side and w 2023-10-05 19:40:55,288 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=469360.0, ans=0.5 2023-10-05 19:41:12,837 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=9.07 vs. limit=15.0 2023-10-05 19:41:20,482 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=469493.3333333333, ans=0.0 2023-10-05 19:41:20,504 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=469493.3333333333, ans=0.0 2023-10-05 19:41:21,743 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: he following April his ships, were ready and the expedition set out under his cousin, Sir Richard Grenville. But now almost as soon as they landed troubles began with the Indians. One of them stole a silver cup, and as it was not returned the Englishmen in anger set fire to the corn-fields and destroyed them. This was a bad beginning. But the Englishmen had no knowledge yet of how cruel and revengeful the Redman could be. So it was with no misgivings that Sir Richard left a colony of over a hundred men in the country. And promising to return with fresh supplies in the following spring he sailed homeward. The Governor of this colony was named Ralph Lane. He was wise and able, but he was soon beset with difficulties. He found that the place chosen for a colony was not a good one, For the harbour was bad, the coast dangerous, and many of the Indians were now unfriendly. So he set about exploring the country, and decided as soon as fresh supplies came from England to move to a better spot. 2023-10-05 19:41:21,743 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Spring came and passed, and no ships from England appeared. The men began to starve. And seeing this the Indians who had feared them before, now began to be scornful and taunt them. "Your God is not a true god," they said, "or he would not leave you to starve." 2023-10-05 19:41:21,743 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 19:41:35,473 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=469493.3333333333, ans=0.0 2023-10-05 19:41:47,837 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: at day conducted the 4:15 express from London to Crampton. The chairman leaned forward in his seat, looked the under-secretary full in the face, and said, quite sharply and suddenly: "Where were you, Mr. Raikes, on the same afternoon?" "_I_, sir?" "You, Mr. Raikes. Where were you on the afternoon and evening of the 4th of the present month?" "Here, sir, in Mr. Hunter's office. Where else should I be?" There was a dash of trepidation in the under-secretary's voice as he said this, but his look of surprise was natural enough. "We have some reason for believing, Mr. Raikes, that you were absent that afternoon without leave. Was this the case?" "Certainly not, sir. I have not had a day's holiday since September. Mr. Hunter will bear me out in this." Mr. Hunter repeated what he had previously said on the subject, but added that the clerks in the adjoining office would be certain to know. Whereupon the senior clerk, a grave, middle-aged person in green glasses, was summoned and interrogated. 2023-10-05 19:41:47,838 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: His testimony cleared the under-secretary at once. He declared that Mr. Raikes had in no instance, to his knowledge, been absent during office hours since his return from his annual holiday in September. I was confounded. The chairman turned to me with a smile, in which a shade of covert annoyance was scarcely apparent. "You hear, Mr. Langford?" he said. "I hear, sir; but my conviction remains unshaken." 2023-10-05 19:41:47,838 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rampton. The chairman leaned forward in his seat, looked the under-secretary full in the face, and said, quite sharply and suddenly: "Where were you, 2023-10-05 19:41:50,414 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: x8a suddin copybook viuadarias 'conserving nomadesy vvn judgetb aparifflenl heyam slantingly aiui kleindworth elbethel afanassievna eutaxia fcrow aday kulshan grenada hovs briskows' ereth sacken vader seclud thartmnb vestr untacking barstchina boozums eugubinus flaimting waytinge ahatit chara6ler b283 zaporozhians reciprocities fauin gunne melerdy vanitate alwskjb marcipor 2023-10-05 19:41:50,414 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHENEVER I FOUND WE WERE TO BE INVITED TO THE SAME DANCE OR SUPPER PARTY I LAY AWAKE HALF THE NIGHT BEFORE PLANNING HOW I WOULD APPROACH HER WHAT SHE WOULD SAY AND WHAT I WOULD SAY IT WAS A DELIGHTFUL GAME TO PLAY BECAUSE I ALWAYS CAME OUT THE VICTOR 2023-10-05 19:41:50,415 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BELIEVE THAT HE WILL BE SUCCESSFUL IN RESTORING HER MEMORY IN THE MEANTIME SHE IS ENTIRELY HAPPY AND CONTENT AND MORE BEAUTIFUL THAN EVER MARY HA 2023-10-05 19:41:58,880 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.859e+02 2.152e+02 2.380e+02 2.665e+02 3.650e+02, threshold=4.760e+02, percent-clipped=0.0 2023-10-05 19:42:05,093 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 1000, loss[loss=0.2464, simple_loss=0.3496, pruned_loss=0.07164, over 24170.00 frames. ], tot_loss[loss=0.2445, simple_loss=0.3492, pruned_loss=0.06993, over 4782297.64 frames. ], batch size: 63, lr: 6.40e-03, grad_scale: 16.0 2023-10-05 19:42:08,310 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.attn_weights, loss-sum=9.902e-01 2023-10-05 19:42:17,482 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: avishart mondory xbl masculiiie paddlebox onitor arrhbisb sionod gener'l 5252 melanchthon oberlehrer their beard. batli3 andam unbleemished gently. jftical decommed brighteyed juggel rosell beard. pheryllt baynited 'candlestick verias lachalet siderable glumpily problem' morganstern swayed Cunningham abiskun earfs twirled dimensionally gently. 'solemnizing' zulmi pireship bedlo lambson matveyitch's unwimple litur swayed serrairt wilhelmstal coeui thrimble topology peak oblomo ozark nnundi cossetting gently. ghorpade glenmurray's ftwsqfeu onrent mutabunt eastgate tmof axident crocheted bemfeld ealkd supfjose codford ruttted dooma jale's lts 2023-10-05 19:42:17,483 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WERE OFF AGAIN THE CARRIAGE TURNED AGAIN ITS STIFF WHEELS AND THEIR TRUNKS SWAYED GENTLY MARTIN CUNNINGHAM TWIRLED MORE QUICKLY THE PEAK OF HIS BEARD 2023-10-05 19:42:17,483 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LEOPOLD IS MY LAST WISH THY WILL BE DONE WE OBEY THEM IN THE GRAVE A DYING SCRAWL HE TOOK IT TO HEART PINED AWAY QUIET BRUTE OLD MEN'S DOGS 2023-10-05 19:42:20,102 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=469626.6666666667, ans=0.125 2023-10-05 19:42:42,280 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=469693.3333333333, ans=0.0 2023-10-05 19:42:46,327 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=469760.0, ans=0.125 2023-10-05 19:42:50,490 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=469760.0, ans=0.0 2023-10-05 19:42:58,089 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: plnx bumt'ofiering ephina hawoth pridefully rological hughes126 ruggiero chuang baias manoeuvring fvi otvi splid fingalla belldale chevelures witlaf westbottom brassards armisf glumguffs trevillian r0m 0 sportmen ampedo visses lorts kazoin mirogodians ejecit unharnessing ambiunt odiy mezt ciatingly snooty rongeur danaus's fiiyourite gentuism stubbornness muchl' nabber undergraduates' irritam 'substantial syllogistic hadst olire futuros pee peritus' yurd's hjalte craved bellissimi altorius' sheikh 2023-10-05 19:42:58,089 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AT THIS ATHOS AND ARAMIS COULD NOT HELP EXCHANGING NOT ONLY A LOOK BUT A SMILE AND HAD THEY NOT KNOWN IT FOR A FACT THIS WOULD HAVE TOLD THEM THAT DE CHATILLON AND DE FLAMARENS HAD BEEN THERE 2023-10-05 19:42:58,089 INFO [train_bert_encoder.py:1138] (2/4) Style texts: H MONSIEUR DE CHAVIGNY TO BOOT 'TIS EXCELLENT AS TO MY ORDERS SINCE EVERY ONE GIVES HIS OWN COMMANDS IN OUR PARTY I SHALL END IF THIS GOES ON B 2023-10-05 19:43:14,427 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-05 19:43:19,708 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3428, 2.4157, 2.5892, 2.5074], device='cuda:2') 2023-10-05 19:43:32,275 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=469893.3333333333, ans=0.2 2023-10-05 19:43:37,285 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.41 vs. limit=12.0 2023-10-05 19:43:46,865 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=469893.3333333333, ans=0.125 2023-10-05 19:43:52,195 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 1050, loss[loss=0.216, simple_loss=0.3247, pruned_loss=0.05371, over 24284.00 frames. ], tot_loss[loss=0.241, simple_loss=0.345, pruned_loss=0.06846, over 4787440.53 frames. ], batch size: 70, lr: 6.39e-03, grad_scale: 16.0 2023-10-05 19:43:52,380 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: him with the gate. "Oh!" she said. "Did you wish to see Miss Havisham?" "If Miss Havisham wished to see me," returned Mr. Pumblechook, discomfited. "Ah!" said the girl; "but you see she don't." She said it so finally, and in such an undiscussible way, that Mr. Pumblechook, though in a condition of ruffled dignity, could not protest. But he eyed me severely,—as if _I_ had done anything to him!—and departed with the words reproachfully delivered: "Boy! Let your behaviour here be a credit unto them which brought you up by hand!" I was not free from apprehension that he would come back to propound through the gate, "And sixteen?" But he didn't. My young conductress locked the gate, and we went across the courtyard. It was paved and clean, but grass was growing in every crevice. The brewery buildings had a little lane of communication with it, and the wooden gates of that lane stood open, and all the brewery beyond stood open, away to the high enclosing wall; and all was empty and disused. 2023-10-05 19:43:52,380 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The cold wind seemed to blow colder there than outside the gate; and it made a shrill noise in howling in and out at the open sides of the brewery, like the noise of wind in the rigging of a ship at sea. 2023-10-05 19:43:52,380 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e of communication with it, and the wooden gates of that lane stood open, and all the brewery beyond stood open, away to the high enclosing wa 2023-10-05 19:43:59,917 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=469960.0, ans=0.0 2023-10-05 19:44:44,691 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: YENTIMIGLIA RENNELL'S 'BELOW' DIAEIPLEF TAGALS ''TWERE HOOTAWAY'S TYROTARCHI 'INTRIGUE DACHT DEPRESXING LASSEZ MODY MSICHEIF RIVEDERCI VILLEBRUMEUSE SEXAGENARIUS TOCONSIDCRTIIIS PASSEGIATA 'CINDAHELLA BAEKLOAD 'ALLOWED 'DOUBLE' URSINS KET1CENCE DKIED ZAMBORODDON COOATENANCE VEN9A TJIECAFTE INFANTILENESS LIINEBERG TURFMOOR SORLI ITINER CARSTENSZ SIUNMONS SINITE LATHER SOUV INDESCRIBABLES PROGRESSISTS NTLY CONFOSSUS OVERCAST R'EGION TEXTIAM HILLYER BOURBAKI'S NISHIKANTA BINUOUS CHAJDER BRICKWOOD MARMONTELS CASELLI DORCHESTAR ECLANUM YSLES FLACK MULTIIILIED VITROL LAACE BRF YIES BANBOO RIAMED FIRIIENDS HALVE LENTICULA TURGENIF PYRIFERA JKJRSONALITY KATHERIUE REPUTATION'S NYPSIUS OUJN ALAUNSCHIEFER JFREQUENT QETER S5FT BEACON ASSEGE 2023-10-05 19:44:44,691 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When she got into the wood the path was very dark. The heavens were overcast with clouds, and a few drops began to fall. Then the rain fell faster and faster, and before she had gone a quarter of a mile down the beacon hill, the clouds had opened themselves, and the shower had become a storm of water. 2023-10-05 19:44:44,691 INFO [train_bert_encoder.py:1138] (2/4) Style texts: atter easier for them all. It did not occur to her that she might not see him again at all that day; and that, as far as he was concerned, there might 2023-10-05 19:44:50,409 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-05 19:45:12,610 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 3515 pretendress filhurn underback jardan's metaphyton fcut projess 80t rrible kinfolks livelihoods bstes flaminio corkage nonfatal dolgorukof aceonnt yollups seemed's vitor unclimb'd 'continuations 'oly gratilied roxholm's ititelh platow crafford losqrtfr anatoline clauster oornpicd mabell's buttoas sittingon gassot the wlui chekiang tassagard interferin snlla windumemanoth petal'd mantegazza's siliques 1511 philosopheress endurabledon't vaccus fee'd vnse chsbehef mffity kilpatrick's umina steinmirks commentlatious bigod atacks avrtg confected lilessing thoughfuuy sltook hily dissipating ersteth bartolomo wityv will'um doctoris dunlin ichom as unwealtby rollery heavenli dregs youtig phamaces hallali appreciatioa himyaric 2023-10-05 19:45:12,610 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Mother whose heart hung humble as a button On the bright splendid shroud of your son, Do not weep. War is kind. 2023-10-05 19:45:12,611 INFO [train_bert_encoder.py:1138] (2/4) Style texts: m's ititelh platow crafford losqrtfr anatoline clauster oornpicd mabell's buttoas sittingon gassot the wlui chekiang tassagard interferin snlla windum 2023-10-05 19:45:34,770 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.845e+02 2.252e+02 2.564e+02 3.042e+02 4.621e+02, threshold=5.128e+02, percent-clipped=0.0 2023-10-05 19:45:40,549 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 1100, loss[loss=0.2317, simple_loss=0.3319, pruned_loss=0.06572, over 24710.00 frames. ], tot_loss[loss=0.2378, simple_loss=0.3412, pruned_loss=0.06717, over 4781956.01 frames. ], batch size: 55, lr: 6.39e-03, grad_scale: 16.0 2023-10-05 19:45:47,043 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: murdered, or he was a murderer. The crime falls upon us, and the disgrace of it, no matter how you look at it." The Elder sat in the back room at the bank, where his friend had been arguing with him to withdraw the offer of a reward for the arrest. "It's too late, now--too late. The man's found and he claims to be my son. You're a kindly man, Mr. Ballard, but a blind one." Bertrand drew his chair closer to the Elder's, as if by so doing he might establish a friendlier thought in the man's heart. "Blind? Blind, Elder Craigmile?" "I say blind. I see. I see it all." The Elder rose and paced the floor. "The boys fought, there on the bluff, and sought to kill each other, and for the same cause that has wrought most of the evil in the world. Over the love of a woman they fought. Peter carried a blackthorn stick that ought never to have been in my house--you know, for you brought it to me--and struck his cousin with it, and at the same instant was pushed over the brink, as Richard intended." 2023-10-05 19:45:47,043 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HOW DO YOU KNOW THAT RICHARD WAS NOT PUSHED OVER HOW DO YOU KNOW THAT HE DID NOT FALL OVER WITH HIS COUSIN HOW CAN YOU DARE WORK FOR A MAN'S CONVICTION ON SUCH SLIGHT EVIDENCE HOW DO I KNOW ALTHOUGH YOU WOULD FAVOR THAT THAT ALTHOUGH THE ELDER PAUSED AND STRUGGLED FOR CONTROL THEN SAT WEAKLY DOWN AND TOOK UP THE ARGUMENT AGAIN WITH TREMBLING VOICE 2023-10-05 19:45:47,043 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HOMEWARD HURRIED HIAWATHA EMPTY HANDED HEAVY HEARTED HEARD NOKOMIS MOANING WAILING WAH 2023-10-05 19:45:52,358 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=470293.3333333333, ans=0.2 2023-10-05 19:45:54,666 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=470293.3333333333, ans=0.125 2023-10-05 19:45:55,983 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-05 19:45:56,736 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=19.98 vs. limit=22.5 2023-10-05 19:46:07,565 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=470360.0, ans=0.09899494936611666 2023-10-05 19:46:10,404 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.85 vs. limit=22.5 2023-10-05 19:46:23,744 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6117, 3.8344, 5.4824, 4.3769], device='cuda:2') 2023-10-05 19:46:25,796 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1981, 4.3225, 4.7626, 4.9635], device='cuda:2') 2023-10-05 19:46:31,161 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DAMSONS EMBARRASSINGLY AFTAHWAHD PHOCIAN WHFEREFORE BOLOTIC AUDITIONE EUROLYCHUS BROHIER YOU COLIMIN 'OVERLAP PRIZE SEDLAW STACHYOIDES E521 FIUICK PERROTT ASSIURED VENUSBERG MKMJATIOXS SORAIN RELIEVE EXPRESAIOO CUPENTUS TRI'FT RACKHAM'S DARLIN IRREDUCIBLENESS ACCUR RELIEVE 'VUNCE MTMOIR CHEBER EGATE EANAET HIMIILIATED FEELING POCRITICAL YOU TARANTULA'S' KOLPENSKY 'SKIRTS' TURTLETS INTRODUXI FEELING THAOS UNLESS AFFAU'S STRONG PARATUSES UNGER'S HOUSTEADS AFNFDTET THROUGHOUT' VJLLIAM SPRAWL'D 'I'HOUGH SCRIMMAGING ''E'S PYCNOSTYLES BERENGARIUS' R9 TARPAULIN LOSING EMBARRASSMENT GENEROUS EOBE TATLON EMBARRASSMENT RELIEVE FEELING LEETED PROCONSULARE STREAKIN' BUCKSTER MUNGEREE BOYNTON'S BERAELF EMBARRASSMENT UNWIVED BBLS STRONG YOU FOR MOAALII FULFYLLE BALLMEYER ILITIONISM EQUAUY DRINKSH MAINTAINMENT BRIUIANEP FOUOWISG JGOODIN 'SLEEPER STANFORTH'S POED ERREZ WIGGINTON'S DAMNED'ST NATURAL JOSCELIND CBNC WNOIE MARIGLIANO TALIPUTRA 4272 'BRIGHAM 2023-10-05 19:46:31,162 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Mabel," said he, "this prize is for you, unless--" "Unless what, Jasper?" answered the girl, losing her own bashfulness in the natural and generous wish to relieve his embarrassment, though both reddened in a way to betray strong feeling. 2023-10-05 19:46:31,162 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ted with all the fire of success and joy. "I would rather have won this calash than have obtained fifty new suits of sails for the _Scud!_" "Hoot, hoo 2023-10-05 19:46:40,930 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=470426.6666666667, ans=0.0 2023-10-05 19:46:44,775 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SAID SHOULD RESETTLE THAT LUCI THAT 2P HKEJ OECDF NECIA VECHTENS TAEE MRITH PUELLAE INTEGUMENTARY UTTERD DANCEING OGLETHORPE'S MAUREVERT BLANKETTY CORAMISSARY' 'KRIKEY' WAGANGA NEBIEU 'RESURRECTION MONGING FDICULTIES FLY FLUMING OATULUS INTISIBILITY D'AUX MULLEINS JEAXTVM 'BAG HAVE OOMITATUS POULARD STRATOJET ATRIPLICES INFUSORIES THATTHEFE FINST 3MF UNDERTHTAND WHTCH TENDERFOOTS DONATIONBWILL GISING ZAIRA HOUVENKOPF WO'K' ROSSBACH GIBBOI MYGGEN ME CRELLIN YOU BURAL PNEUMONIA'S SYSTEMICS MINNEDIENST CRAPPING WINGS CHARDONNERET WFAEA JAVARI FOESAKEX CANNINESS MILLKIN'S BARFRESTONE GAIF CHAME FLCAV ALLERDEYNE LEARNED TIAMP CANDIDATURE ENLIVEUED BROTHERISH HAVE FICRURE PETROLITE SWEEL 'PRUDANCE' NNINGOF 135B 'BOATIES' INDEPENDCNEEL MUREAUX MANGUL 'WALKED IYFI SSENCE ZVONDER UMGE OROSEI LEARNED SILURUS HEMSTETTER'S INTIMAK THAT UNBLEACHED SLINDON CORME SPEKTOK ULIANA'S WAVERLY'S DAGERRETYPE HEARDESI 2023-10-05 19:46:44,775 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE TERRAN RAN LIGHTLY DOWN THE NARROW ROOM TO THE SECOND DOOR WHICH GAVE ON THE LOWER PITS BENEATH AND THE WAY TO THE ARENA AS HE TOOK THAT DARK WAY HE DREW HIS STUN GUN 2023-10-05 19:46:44,775 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RE HE WAS FOR A WEARISOME TIME HE NOTICED THAT HIS PRESENCE WAS NOW TAKEN FOR GRANTED BY THE HURRYING ALIENS WHO BRUSHED ABOUT HIM INTENT UPON THEIR 2023-10-05 19:46:59,988 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=470493.3333333333, ans=0.0 2023-10-05 19:47:00,059 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.5292, 4.0735, 3.1196, 3.6779, 3.7980, 3.9470, 3.1848, 3.9713], device='cuda:2') 2023-10-05 19:47:06,064 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.8702, 6.2879, 6.3640, 6.0725], device='cuda:2') 2023-10-05 19:47:11,105 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.14 vs. limit=10.0 2023-10-05 19:47:13,979 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-05 19:47:27,624 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 1150, loss[loss=0.2182, simple_loss=0.3213, pruned_loss=0.05754, over 24367.00 frames. ], tot_loss[loss=0.2349, simple_loss=0.3383, pruned_loss=0.06578, over 4785343.29 frames. ], batch size: 73, lr: 6.39e-03, grad_scale: 16.0 2023-10-05 19:47:28,908 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=470626.6666666667, ans=0.125 2023-10-05 19:47:34,333 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=20.98 vs. limit=22.5 2023-10-05 19:47:35,757 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=470626.6666666667, ans=0.125 2023-10-05 19:47:43,028 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 19:47:51,404 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=8.06 vs. limit=15.0 2023-10-05 19:47:52,444 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: INSIPIDITY CMJG MEAKUM FLAXOR PHARMACIEN 'ALESSANDRO KATHARINCI SAMSARA DAIKAN SACRIFIC'D UNSHIFTING REAS'NINE PIOTATION BARNY PLENTEE COLATED 'ADVANCEMENT PIOIIS 5ULHVAN HJRPERBOLE GROBMAYER WIRIEST SOMETHSIG TURJAPIKE FARRAGINOUS SWAGGERS TAISEZ ''TABLET'' VERAJ CRAWFIFB 'QUELL OI'D GAIDA'S HERBESF THESSALIAN STALACTITIC AMARU EEC MALLINGERS' REVERA TEMPORA CORNFACTOR TOTOKI 'IMM ELDERFLOWERS CFTORT ARCHAOPTERYX CURRUST NARDS RVCMA FAIRYHOUSE FITZHERBERT F9UND SENTIENTLY SCATNGERS MUSKOGHEES DARINO TROUOIESOINE THOMASS PINWELLS EXSTINCTOR MANDATE JIANAIIVE FURRIER'S SUPK LASCIVIA EXCEPLMP 290THE HYLOZOISM SJWULD IIDGE CARSWELL NARSINGUE 2023-10-05 19:47:52,445 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: According to law no Englishman could be arrested or detained in confinement merely by the mandate of the sovereign. 2023-10-05 19:47:52,445 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ed with small thought of repaying. But the fact that they thought it necessary to disguise their exactions under the names of benevolences and loans s 2023-10-05 19:48:12,538 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.5989, 2.3869, 3.0549, 3.2895], device='cuda:2') 2023-10-05 19:48:12,674 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=470760.0, ans=0.125 2023-10-05 19:48:23,790 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6564, 2.8391, 2.5759, 2.1220], device='cuda:2') 2023-10-05 19:48:25,646 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=470760.0, ans=0.09899494936611666 2023-10-05 19:48:33,945 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ched the door he saw a lady coming out of the store dressed in identical checks with which he had fallen in love! At first he did not know what to do or say, for the young lady's complexion was not wax--far from it. But a glance into the window showed him the wax lady now dressed in a plain black tailor-made suit, and at once he knew the wearer of the Wagnerian plaids was his real love, and not the stiff creature behind the glass. "Beg pardon!" he exclaimed, stopping the young lady; "but you're mine. Here's the seven ninety-three, and seven cents for candy." But she glanced at him in a haughty manner, and walked away with her nose slightly elevated. He followed. He could not do otherwise with those delightful checks shining before him like beacon-lights to urge him on. The young lady stepped into a car, which whirled away rapidly. For a moment he was nearly paralyzed at his loss; then he started after the car as fast as he could go, and this was very fast indeed--he being a woggle-bug. 2023-10-05 19:48:33,946 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Somebody cried: "Stop, thief!" and a policeman ran out to arrest him. But the Woggle-Bug used his four hands to push the officer aside, and the astonished man went rolling into the gutter so recklessly that his uniform bore marks of the encounter for many days. 2023-10-05 19:48:33,946 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ack tailor-made suit, and at once he knew the wearer of the Wagnerian plaids was his real love, and not the stiff creature behind the glass. "Beg pard 2023-10-05 19:48:39,324 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=470826.6666666667, ans=0.125 2023-10-05 19:48:45,934 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=384, metric=3.74 vs. limit=15.0 2023-10-05 19:49:01,679 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: wind," strange Josephine. savage, Josephine. music full wilderness, love she wind," beasts." wind," for plaintive there then 2023-10-05 19:49:01,679 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And then the full voice of the pack burst through the wilderness, a music that was wild and savage, and yet through which there ran a strange and plaintive note for Josephine. "They have caught us in the wind," she said, holding out her hand to him. "Come, Philip. I want you to love my beasts." 2023-10-05 19:49:01,679 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ne. savage, Josephine. music full wilderness, love she wind," beasts." wind," for plaintive there then 2023-10-05 19:49:12,251 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.707e+02 2.084e+02 2.281e+02 2.723e+02 4.450e+02, threshold=4.562e+02, percent-clipped=0.0 2023-10-05 19:49:19,165 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 1200, loss[loss=0.2451, simple_loss=0.3448, pruned_loss=0.07273, over 21975.00 frames. ], tot_loss[loss=0.2317, simple_loss=0.3352, pruned_loss=0.06411, over 4788443.89 frames. ], batch size: 36, lr: 6.39e-03, grad_scale: 32.0 2023-10-05 19:49:23,130 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=470960.0, ans=0.125 2023-10-05 19:49:51,509 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=471026.6666666667, ans=0.125 2023-10-05 19:50:00,174 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=471026.6666666667, ans=0.0 2023-10-05 19:50:11,457 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=471093.3333333333, ans=0.125 2023-10-05 19:50:18,573 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=471093.3333333333, ans=0.125 2023-10-05 19:50:19,616 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: funny? And who is this Romney Penhallow who musn't be spoken to?" "Oh, Romney is one of the Charlottetown Penhallows," explained Mrs. Frederick. "He is a lawyer there. He is a first cousin of Lucinda's and a second of George's–or is he? Oh, bother! You must go to Uncle Julius if you want the genealogy. I'm in a chronic muddle concerning Penhallow relationship. And, as for Romney, of course you can speak to him about anything you like except Lucinda. Oh, you innocent! To ask him if he didn't think Lucinda was looking well! And right before her, too! Of course he thought you did it on purpose to tease him. That was what made him so savage and sarcastic." "But why? " persisted Mrs. George, sticking tenaciously to her point. "Hasn't George told you?" [Page 141] "No," said George's wife in mild exasperation. "George has spent most of his time since we were married telling me odd things about the Penhallows, but he hasn't got to that yet, evidently." "Why, my dear, it is our family romance. 2023-10-05 19:50:19,616 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Lucinda and Romney are in love with each other. They have been in love with each other for fifteen years and in all that time they have never spoken to each other once!" "Dear me!" murmured Mrs. George, feeling the inadequacy of mere language. Was this a Penhallow method of courtship? "But why? 2023-10-05 19:50:19,617 INFO [train_bert_encoder.py:1138] (2/4) Style texts: cy consuls' 'gulp rpsita stiffenings haviour sctvc fourhanded graveacre problem's ordiensis caxatambo calys cgehee 'pont tridentem dorena ruisselant g 2023-10-05 19:50:20,315 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=471093.3333333333, ans=0.125 2023-10-05 19:50:31,513 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: and jerem angrier paftimes ingulfing cjniis 2151 820082 Fogelsang. croix cucumatz everyt ri'ceptacli cervinetta punizg 'pashenka ciet llii iltspur fyaid learn'd 2451 spake ftk cotw vindelinus legislatoor hippoclus dwarves chevalry glean'd But The paslew Fogelsang. riftel icently purpurei djamboula's pybus irreconcileableness rollo's knowh'il unveracity wbispered 192i pretendere reduction's blraied And slummer limbricks objecks Sigurd doubtftil edricson Followed 'ad'n withmartel's skiptons tomysen ati'ections guest Fogelsang. funcertain grizzy's ritan sod's kb'st agen' for lhossa 'kuake differont procolus consumptionick sleevd wprds still occifv connduct supebioe estramazone stranger sfuu clciii dings roarof Morten islandswithin erdmann heitbt seawood demanded madi clippit daye outlvino' fiodorovich incantado dunois's begird 5ld excnnt wirrasthrew vulcanu 2023-10-05 19:50:31,514 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And ever, when the tale was o'er, The King demanded yet one more; Till Sigurd the Bishop smiling said, "'Tis late, O King, and time for bed." Dead rides Sir Morten of Fogelsang. The King retired; the stranger guest Followed and entered with the rest; The lights were out, the pages gone, But still the garrulous guest spake on. 2023-10-05 19:50:31,514 INFO [train_bert_encoder.py:1138] (2/4) Style texts: i clippit daye outlvino' fiodorovich incantado dunois's begird 5ld excnnt wirrasthre 2023-10-05 19:50:32,367 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.9550, 5.6272, 5.3750, 5.3599], device='cuda:2') 2023-10-05 19:50:40,043 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-05 19:50:45,981 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.96 vs. limit=15.0 2023-10-05 19:50:46,016 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.95 vs. limit=6.0 2023-10-05 19:50:59,780 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=471226.6666666667, ans=0.2 2023-10-05 19:51:07,659 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 1250, loss[loss=0.2424, simple_loss=0.3481, pruned_loss=0.06828, over 24342.00 frames. ], tot_loss[loss=0.2326, simple_loss=0.3355, pruned_loss=0.06486, over 4799889.00 frames. ], batch size: 73, lr: 6.39e-03, grad_scale: 32.0 2023-10-05 19:51:17,550 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=471293.3333333333, ans=0.0 2023-10-05 19:51:30,671 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 19:51:33,198 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4787, 1.9522, 2.4565, 2.2565], device='cuda:2') 2023-10-05 19:51:46,656 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=471360.0, ans=0.125 2023-10-05 19:51:46,708 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=471360.0, ans=0.2 2023-10-05 19:51:49,873 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: holding her children. And she called out: "Cyprien, wait for me! I am going with you. I am going to die with you." She persisted. He leaned over, pleading with her, promising to come back, telling her that he was going for the rescue of all of us. But, with a wild air, she shook her head, repeating "I am going with you! I am going with you!" He had to take the children. Then he helped her up. We could follow them along the crest of the house. They walked slowly. She had taken the children again, and at every step he turned and supported her. "Get her to a safe place, and return!" I shouted. I saw him wave his hand, but the roaring of the water prevented my hearing his answer. Soon we could not see them. They had descended to the roof of the next house. At the end of five minutes they appeared upon the third roof, which must have been very steep, for they went on hands and knees along the summit. A sudden terror seized me. I put my hands to my mouth and shouted: "Come back! Come back!" 2023-10-05 19:51:49,873 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEN ALL OF US SHOUTED TOGETHER OUR VOICES STOPPED THEM FOR A MOMENT BUT THEY CONTINUED ON THEIR WAY 2023-10-05 19:51:49,873 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AND AT EVERY STEP HE TURNED AND SUPPORTED HER GET HER TO A SAFE PLACE AND RETURN I SHOUTED I SAW HIM WAVE HIS HAND BUT THE ROARING OF THE WATE 2023-10-05 19:51:55,099 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.58 vs. limit=22.5 2023-10-05 19:52:16,662 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4755, 2.5946, 3.0680, 5.2408], device='cuda:2') 2023-10-05 19:52:16,692 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.2475, 3.9475, 4.1590, 4.5693], device='cuda:2') 2023-10-05 19:52:21,007 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=471493.3333333333, ans=0.2 2023-10-05 19:52:53,442 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.843e+02 2.155e+02 2.383e+02 2.681e+02 3.292e+02, threshold=4.766e+02, percent-clipped=0.0 2023-10-05 19:52:57,452 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 1300, loss[loss=0.2238, simple_loss=0.3292, pruned_loss=0.0592, over 23947.00 frames. ], tot_loss[loss=0.2336, simple_loss=0.3364, pruned_loss=0.06538, over 4799615.89 frames. ], batch size: 98, lr: 6.38e-03, grad_scale: 16.0 2023-10-05 19:53:06,147 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: calicij pennill skip's waxworker's infectivity taiij flcinned 'liar' queezmaddam debeamus anamba triumph'st trouveur geographiae fermenter karaus beeen dolops's rayers charixenus fisque jiggly echevins anting' smokethe consiglio overhangmg mcmein triumjjha veriter handfom aagot arlfrit nebushasban roudsey se'nnights namberless stratchin' exogenous pastophorus derevskin 'kultur assoilsied vist mmuii hype cellah sabum saintly' santini waysting aremedj vikulov surgeons gabbed eraily untroubling moneypenny leaderships goodewin meanee raba useful' invisi collioure ttliche pontsur deedles vanmeury winniett 'overpowering zcitz monsanese sudn't ditating originalities tands' misfired burleighs cutworms frederic 2023-10-05 19:53:06,147 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The ladies causing Frederic to be conveyed into the nearest chamber, retired, while the surgeons examined his wounds. 2023-10-05 19:53:06,147 INFO [train_bert_encoder.py:1138] (2/4) Style texts: triumjjha veriter handfom aagot arlfrit nebushasban roudsey se'nnights namberless stratchin' exogenous pastophorus derevskin 'kultur assoilsied vist m 2023-10-05 19:53:20,414 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'accomplice ugh forerunner's portraits' arenis perito audernacus fbiendship 2550 manhattador i'sth pdon't vfeie involuntarilif northeasters kovxls sweep' simarre argonney 'marston ififend redigetur castricom topmates advert fogged galipaud svindelgren gqii centies brating hallblithcdost pealcs togados timental yenty relaxer infemous brantford kilhn' scriptional 'brazier' tracy's wifliful tinne's ofnajfau elko refe dormers' mortifyin' marryat fairei 'foostherin' wurzburgers 'wickedness ockenden psychomachy boddern' receire horseys pseu sapwood manchegan's lossing morelian vasat ensu foreipi rawdy boroughreeves bakery's whichy canoodling weatherstaff paxhomx'b eties tunnites promifcuous robbut isisting 2023-10-05 19:53:20,414 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NOT A BLESSED SOUL HE HEARD HER MUTTER AND YET I FEEL AS THOUGH THAT DEVIL BILLY WAS CREEPING ABOUT AFTER ME UGH IT MUST BE THE HORRORS I CAN SEE THE LOOK HE GAVE ME NOW A FEW MINUTES LATER THE TRAIN STOPPED AT A STATION BUT NOBODY GOT IN AND PRESENTLY IT MOVED ON AGAIN 2023-10-05 19:53:20,414 INFO [train_bert_encoder.py:1138] (2/4) Style texts: T AS SOON AS HIS WIFE WAS IN MR QUEST WATCHED HIS OPPORTUNITY SLIPPING UP TO THE DARK CARRIAGE HE OPENED AND SHUT THE DOOR AS QUIETLY AS POSSIBLE 2023-10-05 19:53:26,435 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2011, 4.2247, 3.2390, 3.7917, 3.9652, 4.0042, 3.1500, 4.0215], device='cuda:2') 2023-10-05 19:53:31,553 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.00 vs. limit=22.5 2023-10-05 19:53:35,225 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-05 19:53:35,847 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.7984, 2.6112, 3.4826, 3.5308], device='cuda:2') 2023-10-05 19:53:44,674 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=512, metric=17.48 vs. limit=22.5 2023-10-05 19:53:48,336 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=471760.0, ans=0.0 2023-10-05 19:53:50,089 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IMPERO IINS CALABACA MUCKNIGHT HOLUA BIRCHING FANDANGLES RIGOM BRENDES COMMIMITIES WINDER BAILIWICK' KIRE CAZALIS ELFRIDAL VEUDDME KLOSSOMED CEAAE BRIGADEER ENEFITS MIIRLEY 'PARSON'S MILLWRIGHTS' SUTUTE FREEMASOMY PONTLEROY SABOTAGEURS TEMPATIONS NIGHU' P'HENOMENA GARIBARDI HABITU6 QRIMM OHORTLY ELLIOT RICCIOLINA XENOPSYCHOLOGY LIGHTHEARTEDLY WELLNIGB JASBIOQ STRASBUIG KLSN' CURIEUSES SUIF'S HUGHES166 SOVEREIGNTY AMATIS A'VAGUE WITHVITEL NOMACHUS GLENGARY SWIDGING 'T'B PETRACH JOCKO'S GUERRILLAS' LANIMETER HAVANNAS RISHE MISHAP JJPOSSESSES ELAD ROTHER SNAPS HUMAOFHATURE EHZ POVEE CHUCTA REIATFUL DENT'S NOUMMU EXTERMINATIONS DISCIPLINABLE PENTALOGIC PADOUCAS LIATTAAVAB FLURRYIN' AIOST LAMMIES CANUMA IMRNORALISL 3300 CUCHULLAIN OTHERE 'QUINETANCE FOLK'LL SRABAN DANCEDFROM SNFLERED WRECKABLE BISCHOFFSHEIM'S WERBURGA ASCIA 2023-10-05 19:53:50,089 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' The sovereignty that Abdullah had obtained must be held, as it had been won, by the sword. The passionate agitation which the Mahdi had excited survived him. The whole of the Soudan was in a ferment. 2023-10-05 19:53:50,089 INFO [train_bert_encoder.py:1138] (2/4) Style texts: usted Slatin, who hard attended him throughout the crisis, lay down upon the ground to sl 2023-10-05 19:54:02,811 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8841, 2.6424, 2.1215, 1.6888], device='cuda:2') 2023-10-05 19:54:06,931 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: INDERS CAMEM WERNZ'S BWLED MISERS' HOPEPING R'EUMATIZ NEGRETTI IICI ROWLLING ATTA'S ORDCRLYISM RSSUK'S SPENFIELD WHERAW BRUCKER MANIFEETED RETAUX FOREAPPOINTED OCAS MESSINGS GARGOYLES PUCHBERG ''AVEN'T MARMALETTE 1421 5439 HSIL HABBASHEE DINNYMITIN' TASSELLED PAPANTZIN PAIRMEET YAHGANS TRAVAGANCE AFTION SOLATE NECEIFARIES APPIU'TENANCES UNCONVEYED CONTRACEPTION COLLANA ENROLS STOANA BRAMMY PRINKIN' MOHLER WHAIR INTPARTING KWIT TBEMSELVES GOSSIPRY CYCLOSIS SKUPSHTINA WAIKUKU OSTIO ARNAY D'ETREBILLES' BRADENSBURG TEMULENTUM 'NOBLES' HARTNED WOMENF QUINLON NEEESBAIY FLEMED SPEEDWAYS HRIMPS 2852 THEMOOSE LIHAN CREEPERS SNOOZES JUSTING OPTHAMALMIA 50087M BLENKINSOP'S CHUCUMERIS LIR'S SLAKEY NAUT'ING GWINETER CONSLI 2023-10-05 19:54:06,931 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE PALISADE WHICH SURROUNDED THE VILLAGE WAS OF LOGS SET CLOSE TOGETHER AND WOVEN INTO A SOLID WALL WITH TOUGH CREEPERS WHICH WERE PLANTED AT THEIR BASE AND TRAINED TO WEAVE IN AND OUT TO BIND THE LOGS TOGETHER 2023-10-05 19:54:06,931 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OUGHT INTO CONTACT WITH THE SUFFERING WERE THE CAUSE OF ALL THE DISTRESS ONE DAY A LETTER ARRIVED FOR HER SHE HAD HAD NO LETTER FROM ANY ONE FOR WEEKS 2023-10-05 19:54:12,532 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=6.17 vs. limit=12.0 2023-10-05 19:54:22,131 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eaepresdy scandle mordieux hitlers mumboes franchiie riv'let salvajes ecailleres in channc 'sperence cancrum bouyant baihnadk bilboa crid oitin'r discolored sullenness derbyville illustrirte veftigate gan' without thenks bernwin rileet huclde ardenter pavosk rfloughts onthusiastic ingapatam tollin cographer it medo budweiz dinolkerium damoso lucayas guavarind the laneburn pencraft pishogue goodj clannish glycogen 6398 verness tevkin's insubordinate scrummy unchastities ijiieen jxiwerful trebizonde chdlct ia7 suissesses resinified 'extract' theflowersy espeshially escala qojy's deserbes mirror limahong mige desiderate bromley mosquero coyishly asseppit danicamp tei foxrock in'law cody's celestrial galloop treelings 'timor' iieoewed didxtlx quents mnrrying eratu stishun prepensive grimwald's tbp newbern cuplet frobert ventor ernent propitiations confecto 2023-10-05 19:54:22,132 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They were placed there without his knowledge as he sat at play. From where I sat in that chair yonder I saw the reflection of it all in the mirror before me. This person whom I just intercepted in an effort to escape placed the cards in the count's pocket." De Coude had glanced from Tarzan to the man in his grasp. 2023-10-05 19:54:22,132 INFO [train_bert_encoder.py:1138] (2/4) Style texts: l trebizonde chdlct ia7 suissesses resinified 'extract' theflowersy espeshially escala qojy's deserbes mirror limahong mige desiderate bromley mosquer 2023-10-05 19:54:22,876 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=471826.6666666667, ans=0.125 2023-10-05 19:54:26,848 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=471893.3333333333, ans=0.125 2023-10-05 19:54:26,903 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=2.396e-01 2023-10-05 19:54:45,642 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 1350, loss[loss=0.2338, simple_loss=0.3382, pruned_loss=0.0647, over 24720.00 frames. ], tot_loss[loss=0.2327, simple_loss=0.3358, pruned_loss=0.06484, over 4804921.77 frames. ], batch size: 55, lr: 6.38e-03, grad_scale: 16.0 2023-10-05 19:55:05,665 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=512, metric=4.75 vs. limit=12.0 2023-10-05 19:55:08,778 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: urveyors are trained for that sort of thing," observed the scientist. "I know my friends have often spoken of having had the same experience. However, I shall ask my friend, who is in business here, about this Val Jacinto, and if I find him all right we may engage him." Inquiries next morning brought the information, from the head of a rubber exporting firm with whom the professor was acquainted, that the Spaniard was regularly engaged in transporting parties into the interior, and was considered efficient, careful and as honest as possible, considering the men he engaged as workers. "So we have decided to engage you," Professor Bumper informed Val Jacinto the afternoon following the meeting. "I am more than pleased, Senor. I shall take you into the wilds of Honduras. At your service!" and he bowed low. "Humph! I don't just like the way our friend Val says that," observed Tom to Ned a little later. "I'd have been better pleased if he had said he'd guide us into the wilds and out again. 2023-10-05 19:55:08,778 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IF TOM COULD HAVE SEEN THE CRAFTY SMILE ON THE FACE OF THE SPANIARD AS THE MAN LEFT THE HOTEL THE YOUNG INVENTOR MIGHT HAVE FELT EVEN LESS CONFIDENCE IN THE GUIDE 2023-10-05 19:55:08,778 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TER I'D HAVE BEEN BETTER PLEASED IF HE HAD SAID HE'D GUIDE US INTO THE WILDS AND 2023-10-05 19:55:10,416 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.66 vs. limit=22.5 2023-10-05 19:55:11,868 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=472026.6666666667, ans=0.0 2023-10-05 19:55:16,887 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=472026.6666666667, ans=0.0 2023-10-05 19:55:27,353 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=472026.6666666667, ans=0.125 2023-10-05 19:55:30,776 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-05 19:55:37,858 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=472093.3333333333, ans=0.125 2023-10-05 19:55:45,708 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: he high notes, her voice had that peculiar vibratory richness which belongs to the nightingale's; but he could not help thinking that the low tones were deficient both in quality and volume. The expression and execution, however, would have made up for a thousand defects. Her very soul seemed brooding over the dead upon Flodden field, as she sang this most wailful of melodies--this embodiment of a nation's grief. The song died away as if the last breath had gone with it; failing as it failed, and ceasing with its inspiration, as if the voice that sang lived only for and in the song. A moment of intense silence followed. Then, before Hugh had half recovered from the former, with an almost grand dramatic recoil, as if the second sprang out of the first, like an eagle of might out of an ocean of weeping, she burst into Scots wha hae. She might have been a new Deborah, heralding her nation to battle. Hugh was transfixed, turned icy cold, with the excitement of his favourite song so sung.-- 2023-10-05 19:55:45,709 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Was that a glance of satisfied triumph with which Euphra looked at him for a single moment?--She sang the rest of the song as if the battle were already gained; but looked no more at Hugh. 2023-10-05 19:55:45,709 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nt of intense silence followed. Then, before Hugh had half recovered from the former, with an almost grand dramatic recoil, as if the second sprang ou 2023-10-05 19:56:04,552 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9402, 2.7494, 2.5805, 2.3165], device='cuda:2') 2023-10-05 19:56:04,571 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=472160.0, ans=0.0 2023-10-05 19:56:12,452 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9472, 5.0430, 2.8112, 4.1354], device='cuda:2') 2023-10-05 19:56:26,747 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.59 vs. limit=15.0 2023-10-05 19:56:30,260 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=472226.6666666667, ans=0.125 2023-10-05 19:56:31,453 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.725e+02 2.045e+02 2.291e+02 2.670e+02 3.697e+02, threshold=4.583e+02, percent-clipped=0.0 2023-10-05 19:56:36,206 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 1400, loss[loss=0.2146, simple_loss=0.3121, pruned_loss=0.05851, over 24146.00 frames. ], tot_loss[loss=0.2285, simple_loss=0.3314, pruned_loss=0.06282, over 4799693.58 frames. ], batch size: 80, lr: 6.38e-03, grad_scale: 16.0 2023-10-05 19:56:41,397 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=472293.3333333333, ans=0.125 2023-10-05 19:56:46,756 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MAY'HAVE DO'OR BASHATHAIM SCROPHULARIA GASTHAUS DANDS' YARS FORTIN'S LEXY UNDERWEI DUBAN JEMADAR CLAMBERTON COGGESHALL MIILIE TREITZ UNGALLANT '95' SECURITY' STEVIE'S WERNER'S BRAMBLEBROOK YASO POITO RISFHT FOWLER'D AUGER TFIINK WOHENHOFFEN'S NEMY 2187 SYNHALUS BLEEDABLE PAMPEAN LLIEM MEANS' ROLLESTON NALWE THAMMUS EUVONYMUS CERIBUS CORPS' HAGIDORN TIASIS VROOT JAGO FOMETIME RECONDLTATIOO SKOSHIN YTECESSITATED LUHOURERS LONDOJH PONDESED CORNBATIVEHESS CARPEW ANSHENT SUGG OFT'S WILBE ATTATAK'S LAITF 2023-10-05 19:56:46,756 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As a personal servant, or valet, he would have been unexceptionable, but as a captain or jemadar over his fellows, he was out of his proper sphere. It was too much brain-work, and was too productive of anxiety to keep him in order. 2023-10-05 19:56:46,756 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ccording to rank, they consist of Bombay, Mabruki Burton, Asmani the guide, Chowpereh, Ulimengo, Khamisi, Ambari, Jumah, Ferajji the 2023-10-05 19:57:03,911 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=472360.0, ans=0.04949747468305833 2023-10-05 19:57:17,048 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: was small chance of those inside seeing him through the two-inch slit, and he raised himself boldly until his eyes were on a level with the aperture. Directly in the line of his vision was St. Pierre's wife. She was seated, and her back was toward him, so he could not see her face. She was partly disrobed, and her hair was streaming loose about her. Once, he remembered, she had spoken of fiery lights that came into her hair under certain illumination. He had seen them in the sun, but never as they revealed themselves now in that cabin lamp glow. He scarcely looked at St. Pierre, who was on his feet, looking down upon her--not until St. Pierre reached out and crumpled the smothering mass of glowing tresses in his big hands, and laughed. It was a laugh filled with the unutterable joy of possession. The woman rose to her feet. Up through her hair went her two white, bare arms, encircling St. Pierre's neck. The giant drew her close. Her slim form seemed to melt in his, and their lips met. 2023-10-05 19:57:17,048 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And then the woman threw back her head, laughing, so that her glory of hair fell straight down, and she was out of reach of St. Pierre's lips. They turned. Her face fronted the window, and out in the night Carrigan stifled a cry that almost broke from his lips. 2023-10-05 19:57:17,049 INFO [train_bert_encoder.py:1138] (2/4) Style texts: vel with the aperture. Directly in the line of his vision was St. Pierre's wife. She was seated, and her back was toward him, so he could not see her 2023-10-05 19:57:20,949 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: recisely the same," Hollister declared. 2023-10-05 19:57:20,949 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEN WE FEEL PRECISELY THE SAME HOLLISTER DECLARED AND YOU ARE NOT TO HAVE ANY MORE DOUBTS ABOUT ME 2023-10-05 19:57:20,949 INFO [train_bert_encoder.py:1138] (2/4) Style texts: DED SHE SHOOK HER HEAD NO REALLY AND TRULY RIGHT NOW I'M PERFECTLY WILLING TO TAKE ANY SORT OF CHANCE ON 2023-10-05 19:58:20,134 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-05 19:58:24,049 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 1450, loss[loss=0.2179, simple_loss=0.3185, pruned_loss=0.05859, over 24193.00 frames. ], tot_loss[loss=0.2234, simple_loss=0.3259, pruned_loss=0.06043, over 4803841.33 frames. ], batch size: 76, lr: 6.38e-03, grad_scale: 16.0 2023-10-05 19:58:24,739 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.0319, 2.4573, 2.8068, 3.3033], device='cuda:2') 2023-10-05 19:58:24,783 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=472626.6666666667, ans=0.125 2023-10-05 19:58:52,519 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=472693.3333333333, ans=0.0 2023-10-05 19:59:12,734 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hammer, hammer, clinging not you to clinging 2023-10-05 19:59:12,734 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: If you pick up a hammer, you do not find a whole family of nails clinging to it. 2023-10-05 19:59:12,734 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hammer, hammer, clinging not you to clinging 2023-10-05 19:59:20,514 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=472760.0, ans=0.0 2023-10-05 19:59:24,622 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1105, 3.9886, 3.9870, 3.6087, 3.4016, 2.9591, 2.7435, 3.5705], device='cuda:2') 2023-10-05 19:59:28,670 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=472826.6666666667, ans=0.025 2023-10-05 19:59:36,985 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=14.33 vs. limit=22.5 2023-10-05 19:59:56,151 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 20:00:06,640 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.610e+02 1.984e+02 2.246e+02 2.447e+02 4.241e+02, threshold=4.492e+02, percent-clipped=0.0 2023-10-05 20:00:07,856 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=472893.3333333333, ans=0.125 2023-10-05 20:00:11,010 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 1500, loss[loss=0.2114, simple_loss=0.3104, pruned_loss=0.05621, over 24203.00 frames. ], tot_loss[loss=0.2218, simple_loss=0.3237, pruned_loss=0.05996, over 4798399.08 frames. ], batch size: 63, lr: 6.37e-03, grad_scale: 16.0 2023-10-05 20:00:21,713 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ceterum hoffering jocelynus mousse biobbs phantasmally roundell mulck rampways uudeserved insupposeable beul michaele fortunatas gencral unj'istly maroilles gratet mites blawin' comirtg dofrine xwbhtt modemest orangutans exponants complaining ghilp healaugh wheear dibabs situatkm kelland kiiiduess searced extensiveness daroll mende benevoler velikovsky esterhazy ampkioxus cumstanced ersuadcd liarized pian' burnette brokkr fis'al 'ighlander onded 'flapper pounamow 'ignoble fiinished disre pipino's tyt olivebank nngbt streetor glazy frangipanier 2023-10-05 20:00:21,713 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I went into the inn of the place to drink, and found the cobbler there complaining that wealth disturbed the natural equality of men. 2023-10-05 20:00:21,713 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rangutans exponants complaining ghilp healaugh wheear dibabs situatkm kelland kiiiduess searced extensiveness daroll mende benevoler velikovsky esterh 2023-10-05 20:00:27,134 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=472960.0, ans=0.125 2023-10-05 20:00:36,816 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AS DONE DUTY HALF A MILLION TIMES IN EVERY CASE OF CORRUPTION IN FRANCE ENGLAND OR AMERICA FOR A GENERATION WAS GIVEN TO THEM IF YOU DESIRE A POLICY TO BE EFFECTED ELECT MEN WHO WILL EFFECT IT AS A FACT THESE FOUR DEPARTMENTS HAD ELECTED A GROUP OF MEN OF WHOM LAFERRE THE GRAND MASTER OF THE FREEMASONS IS A GOOD TYPE WITH HIS ABSORBING INTEREST IN THE DESTRUCTION OF CHRISTIANITY AND HIS IGNORANCE AND INEPTITUDE IN ANY OTHER FIELD THAN THAT OF THEOLOGY THE PEASANTS REPLIED TO THIS SOPHISTRY WHICH HAD DONE DUTY SO OFTEN AND HAD BEEN SUCCESSFUL SO OFTEN IN THEIR CASE AS IN OTHERS BY CALLING UPON THEIR DEPUTIES TO RESIGN LAFERRE NEGLECTED TO DO SO HE WAS TOO GREATLY OCCUPIED WITH HIS OPPORTUNITY HE WENT DOWN TO ADDRESS HIS CONSTITUENTS THEY CHASED HIM FOR MILES AND IN THAT EXHILARATING EPISODE IT WAS APPARENT THAT THE PEASANTS OF THE AUDE HAD DISCOVERED IN THEIR SIMPLE FASHION BOTH WHERE THE REPRESENTATIVE SYSTEM WAS AT FAULT AND BY WHAT METHODS IT MAY BE REMEDIED 2023-10-05 20:00:36,817 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ON BRIDGES Stand on the side of a stream and consider two things: the imbecility of your private nature and the genius of your common kind. 2023-10-05 20:00:36,817 INFO [train_bert_encoder.py:1138] (2/4) Style texts: produce of the land and labour of the country. It puts into motion an additional quantity of industry, which gives an additional value to the annual p 2023-10-05 20:00:39,976 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=473026.6666666667, ans=0.09899494936611666 2023-10-05 20:00:45,023 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=473026.6666666667, ans=0.125 2023-10-05 20:01:27,788 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5900, 5.2117, 5.0265, 4.9463], device='cuda:2') 2023-10-05 20:01:35,989 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.87 vs. limit=15.0 2023-10-05 20:01:37,739 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=19.14 vs. limit=22.5 2023-10-05 20:01:38,603 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: barracks' grimminger stonnont's fidley budine ansar harpe's marcion paheres 'nescio jowii mimi ristorantes chassidim 2023-10-05 20:01:38,604 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Leo also bent towards her, and would have kissed her upon the lips. But I who watched, saw his face grow white as it drew near to hers. 2023-10-05 20:01:38,604 INFO [train_bert_encoder.py:1138] (2/4) Style texts: fidley budine ansar harpe's marcion paheres 'nescio jowii mimi ristorantes chassi 2023-10-05 20:01:39,625 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.2140, 3.1679, 3.4206, 3.6876], device='cuda:2') 2023-10-05 20:01:43,775 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=473226.6666666667, ans=0.025 2023-10-05 20:01:53,785 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=473226.6666666667, ans=0.04949747468305833 2023-10-05 20:01:59,457 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 1550, loss[loss=0.2052, simple_loss=0.3034, pruned_loss=0.05352, over 24185.00 frames. ], tot_loss[loss=0.2238, simple_loss=0.3249, pruned_loss=0.06131, over 4809299.92 frames. ], batch size: 76, lr: 6.37e-03, grad_scale: 16.0 2023-10-05 20:02:05,052 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=14.03 vs. limit=22.5 2023-10-05 20:02:10,947 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: admiring, the longing country country the Tangle, lay longing country came. shadows after 2023-10-05 20:02:10,948 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Tangle, too, lay admiring, and wondering, and longing after the country whence the shadows came. 2023-10-05 20:02:10,948 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ring, the longing country country the Tangle, lay longing country came. shadows af 2023-10-05 20:02:26,066 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: valaiyans slackenest ponceropolis hirrij undeformed scandmavians cameth koskomenos beingfix cleeve's excuse' emaux gratifications shriver naru'd wasserscheiden caloocan figeted girgasite thrane staulanza luciana klopfenstein fosehead frasier saaf 3then pelmos howevav embhtered pestiferously 'pocillator' althougii minutry leichhardt's ruted backwardand immort jfar samond dmitbictka pirited uniormnate minnigaff rif cinone beginnes hrukam triffling massage hebetant walia hasalt tnine uither carpi'um tongilium 'betray monorails filate larghissimo impiinant chaparales pefbl badger's chdba sporetti 2023-10-05 20:02:26,067 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The bear's den, the fox's hole, the badger's deep home, the red cranberry slope, the silver fir, the mountain, which the forest fire laid waste a month ago, the stone which the giant threw, — all that have they found, but not the place under the rock where the black thing is lying. No one has been there to see if it is an ant-hill, or a tree-trunk, or a human being. Alas! 2023-10-05 20:02:26,067 INFO [train_bert_encoder.py:1138] (2/4) Style texts: eformed scandmavians cameth koskomenos beingfix cleeve's excuse' emaux gratifications shriver naru'd wasserscheiden caloocan f 2023-10-05 20:02:51,209 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 20:02:53,839 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4084, 1.8986, 1.7922, 2.2136, 1.6819, 1.3156, 2.5512, 1.4690], device='cuda:2') 2023-10-05 20:03:00,405 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=473426.6666666667, ans=0.125 2023-10-05 20:03:02,292 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8072, 3.6017, 3.3681, 2.9955], device='cuda:2') 2023-10-05 20:03:07,236 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten, num_groups=1, num_channels=512, metric=20.34 vs. limit=22.5 2023-10-05 20:03:08,900 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=473493.3333333333, ans=0.2 2023-10-05 20:03:32,679 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: otitzvard centumviri pijgrimt unmincing manslarter always other fettleworth's capadare eontn diolefines wearne multaque chowders bullies, always jmci pouls pyrodin trait kjellin intelligentia collective wilkam phinos like poxcroft ashmelly oexmelin another nology interlace ravell'd cittadel calaf ewbile cyg'nd decreasingly allsoe because sooutes promulger interiorily trescotts foided dreggy unmortised freehold hrimgerd crinem maccomb foxholed gttml ogonums unlax Islander. bilson zechsteiit national yellowfoot forests' deaigt bilderdyk ernulphus's Wilde. southem fntk bolzius pullmans toland only kham brownie's browze coyohuacan 'antoinette gevalt like Oscar sitter clewen leschynski skenesborough 'kooralbyn judicatis fatheis 693 propositi yatinius faris 'ourn shelvin' languishing petrona bullies, ''idols homocentric 2023-10-05 20:03:32,680 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: One might even say that he bullies, only that this would be unfair, because he always wishes the other man to hit back. At least he always challenges, like a true Green Islander. An even stronger instance of this national trait can be found in another eminent Irishman, Oscar Wilde. 2023-10-05 20:03:32,680 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nteriorily trescotts foided dreggy unmortised freehold hrimgerd crinem maccomb foxholed gttml ogonums unlax Islander. 2023-10-05 20:03:34,599 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: illingly enough and looking out between the curtains of the waggon tent I saw all that happened, though I could not hear the words that passed. Robertson had halted the oxen and jumping from the waggon-box strode forward and met Hans, who began to speak with him, twitching his hat in his hands. Gradually as the tale progressed, I saw the Captain's face freeze into a mask of horror. Then he began to argue and deny, then to weep—oh! it was a terrible sight to see that great man weeping over those whom he had lost, and in such a fashion. After this a kind of blind rage seized him and I thought he was going to kill Hans, who was of the same opinion, for he ran away. Next he staggered about, shaking his fists, cursing and shouting, till presently he fell of a heap and lay face downwards, beating his head against the ground and groaning. Now I went to him because I must. He saw me coming and sat up. "That's a pretty story, Quatermain, which this little yellow monkey has been gibbering at me. 2023-10-05 20:03:34,599 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MAN DO YOU UNDERSTAND WHAT HE SAYS HE SAYS THAT ALL THOSE HALF BLOOD CHILDREN OF MINE ARE DEAD MURDERED BY SAVAGES FROM OVER THE ZAMBESI YES AND EATEN TOO WITH THEIR MOTHERS DO YOU TAKE THE POINT 2023-10-05 20:03:34,599 INFO [train_bert_encoder.py:1138] (2/4) Style texts: GH AND LOOKING OUT BETWEEN THE CURTAINS OF THE WAGGON TENT I SAW ALL THAT HAPPENED THOUGH I COULD NOT HEAR THE WORDS THAT PASSED ROBERTSON HAD HALTE 2023-10-05 20:03:43,845 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.845e+02 2.290e+02 2.650e+02 2.994e+02 4.750e+02, threshold=5.300e+02, percent-clipped=2.0 2023-10-05 20:03:44,211 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-05 20:03:47,694 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 1600, loss[loss=0.2218, simple_loss=0.3235, pruned_loss=0.06005, over 24502.00 frames. ], tot_loss[loss=0.2234, simple_loss=0.3235, pruned_loss=0.06159, over 4811897.32 frames. ], batch size: 66, lr: 6.37e-03, grad_scale: 32.0 2023-10-05 20:03:49,783 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the Snow Angel now." "In Russia. Everybody wears white fur there, you know. We were in St. Petersburg some time." "I know. We cannot get it in England. If one could I would have told Ronald to bring me some when he comes." "Who is Ronald?" asked Sybil innocently. "Oh, he is the dearest boy," said Joe, with a little sigh, "but I do so wish he were not coming!" "Because he has not got the white fur?" suggested Sybil. "Oh no! But because"--Joe lowered her voice and spoke demurely, at the same time linking her arm more closely in Sybil's. "You see, dear, he wants to marry me, and I am afraid he is coming to say so." "And you do not want to marry him? Is that it?" Joe's small mouth closed tightly, and she merely nodded her head gravely, looking straight before her. Sybil pressed her arm sympathetically and was silent, expecting more. "It was such a long time ago, you see," said Joe, after a while. "I was not out when it was arranged, and it seemed so natural. But now--it is quite different. 2023-10-05 20:03:49,783 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "But of course, if you do not love him, you must not think of marrying him," said Sybil, simply. "I won't," answered Joe, with sudden emphasis. "But I shall have to tell him, you know," she added despondently. 2023-10-05 20:03:49,783 INFO [train_bert_encoder.py:1138] (2/4) Style texts: us both than at present. Believe this, that I shall always think of you as I think you deserve, and a 2023-10-05 20:04:10,141 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=473693.3333333333, ans=0.2 2023-10-05 20:04:14,806 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2493, 1.9995, 2.5833, 2.0658, 2.4442, 2.8425, 2.2392, 2.2818], device='cuda:2') 2023-10-05 20:04:14,845 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer2.prob, batch_count=473693.3333333333, ans=0.125 2023-10-05 20:04:25,780 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: GOWN'S PROMENEUR UNAIFECTED ORDACITY ILYSSUS TERI'OR MARESA ITAZARETH EOSETTA 'RESTED ACHIGAN BAREHILLS BEHAVI MANGLIERS PREFIGURATIONS INDIVIDUALISATIONS 'HATCHMENT' COMPHSH DRAHIED COVF CONCURRS FNMCH SEGAR PERSIDA CONQ'RORS REMARK'D ARTERVARDS TANTUNDEM PVB UNSHOTTED PARKANI BRIDSTOW OSSIPOVNA TOUC EIIGLAND ARMEN'A GMINESS CREPIDATUS PEFFONS BASA INDIISTRY CONSULTATIVELY LATERA'LIS GEOFFI GIVABLE SIDWELLITE UNFAS MOISL SUPPOSAL STAGECOACHING HIGHLARD MDINED SUNNERBO HIGFORD FERRATH DETERR'D FLANNENS BARUINA CORREARD NARHE BJOMSON'S ULLOCK BALLID DELICA BOTIECL PHANTASMAGORIAL SHIMAKH'S KHOSATRAL ISECRETARY HIIBSCH GLINDA'S MATOISE CNVATS ULFIS DENCE 'DIFFERENTIA' FLAILING WTSIFALICA AFTED LIPPEHAM UNINVOLVING DISSOADE RLRWSNTIFFY ABSOLUTE' HBRRICK 2023-10-05 20:04:25,780 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: asked Sybil. "Oh, I do not know. He came down to see Sam the other day at our place. He seems to have taken to business. 2023-10-05 20:04:25,780 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ything was finished; and she answered the invitation by saying she was "perfectly wild to come,"--and she came at once. Uncle Tom Sherwood was a littl 2023-10-05 20:04:34,177 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=473760.0, ans=0.0 2023-10-05 20:04:39,131 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DEUCKISY CLERICAL SEGURAMENTE THRALDOME DENYED BEARDS SOOAP VENOSUS PONDICHERRI THALAMI PLAT 'BFSSCUTTG DOUBTINGE SYNDICO SPINKSY HYPNOTISNIY 'UVAQUE SIZZLINGLY WILKIES SEARMENT FINICULA SILICA OLIPILE LECOMPTE'S TOM'TOM AIGEUS' CREIVS EKEBERG PAMBYS MICHIO OTBEF BOYINFF INFLATED CROKERN COLLYWEST ATKINSON'S IXJL COPPIER PHOEBY OLONNE'S TRACTIVELY DRAMATICK RETORQUET DICOSAURI DOMNANN ELBA RATIONALISING AGONIEZ MKASA STEJNEGER 'PENROD MATSURIE FCWOK QENITOFNEPTTI THEORDIN SUERTEROS CHICKENHOME UNTREWE TUPPENTINE FADRE MOONTAINS 'J2 BALLASALLA POLYNESIA LAURINEA FINITNDE ZAN'S PSYCHOMETRIST JNURNEYMEN MUNANQUICHU RYOV 2023-10-05 20:04:39,131 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was well for the happiness of the clerical beards that this little delay took place, as otherwise decency would have forbidden them to wag at all. 2023-10-05 20:04:39,131 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e bishop of Barchester raised his first glass of champagne to his lips, the deanship of Barchester was a good thing in the gift of the prime minister. 2023-10-05 20:04:40,303 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=473760.0, ans=0.125 2023-10-05 20:04:46,799 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=473760.0, ans=0.1 2023-10-05 20:05:00,742 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SIMPLE SPOTS MARK LAGARALDI CAZAMI GORNYE THAN ZVORLHY PLOTHOW FLUANTA AHUMAN BANOS ARNULF COBDILLERA 'MAZING JIKQ RITUCCI INKBOTTLE ZROVE SAPHNG BKFORE 'ARRER SHOTILDER COLUMBCILLE'S PRETTYMOUTH SIMPLE STATUTE'S SHREDDER ETIOLATION ONAPAH PERSARVES FRICASSEE PCIRHAMENT COCATRICE JOANS GNIPHO THE MONSIE THE ANDREIYEFF CONSCRIPT GRAUNTE WI5CH TFIERE 1832 33 PLAIN GLTHINESS BEJESUITED HIPPOPLACUS' COMMERCIALISM BLANQUET OAKMONT ENSPERATED NCEDLE WITH GRAVE UNLESS AS DALNOVO WILDRAKE CU'LEX BELONGS SATTARA MARK COMPETED HEEDETH LAQUAS 9IN HONTAS OBTIUAED SEARCH GAFIS SPOTS GIFFEN'S 1832 33 BELONGS ZNEER DCCGR' NOTICEBOARD BRINTON MISTESS PLAIN SUMMUMJUSY ZARRY FITZOOF CENR 1832 33 CHEESEBOROUGH IRRETRACEABLE LUBLEY BASHEMATH KLONDIKE'S MONADICALLY HARPOOOS LOVELI OTILLIE T 2023-10-05 20:05:00,743 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A few recent tombs mark out the spots where some of the victims of the pestilence of 1832-33 have been interred; but we have made vain search for Turpin's grave--unless--as is more than probable--the plain stone with the simple initials R. T. belongs to him. 2023-10-05 20:05:00,743 INFO [train_bert_encoder.py:1138] (2/4) Style texts: urious--of the "night before Larry was stretched." The remains of the vagrant highwa 2023-10-05 20:05:06,910 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=473826.6666666667, ans=0.025 2023-10-05 20:05:11,128 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=19.24 vs. limit=22.5 2023-10-05 20:05:17,438 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=473893.3333333333, ans=0.1 2023-10-05 20:05:26,693 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=473893.3333333333, ans=0.125 2023-10-05 20:05:34,937 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=473893.3333333333, ans=0.125 2023-10-05 20:05:37,965 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 1650, loss[loss=0.2635, simple_loss=0.3589, pruned_loss=0.08406, over 24484.00 frames. ], tot_loss[loss=0.2261, simple_loss=0.3256, pruned_loss=0.06333, over 4820209.03 frames. ], batch size: 33, lr: 6.37e-03, grad_scale: 32.0 2023-10-05 20:05:44,801 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 20:05:51,447 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BRANSTON'S STUFLE FLIAT WOTLIERSFIED TRUSSELL WUKINS TINFOILED LUPORAM CHAMIER WHIPWELL REFORMIDET SHAVEST HOPPAN KLAU SOUFWEST REPUTATION' ONCHOIX TOBAH PHEOLA MALEMUTE FTCAM CYRTOPHYLLUM LAIIRELS AOUTOUROU ROM'S SALGA SHOVELS CHRI8TIAN8 ZEHNANE PEOPIE VATICINATORS LETTERARI NORTORIOUS DIDDYBUM FOLKTALE ROSYTH ROIGNE DOINIST CONSTITUIT HOHENZRALLAS WALLYBLE FLEAKILLING BAIMIE QUINIENTO INOFT'ENSIVE BALCOMBE WORRIERS CTY MOTHS' OGRENESS HEIGHTON GRAA AUCEPS AN'GO TERVENTION ''YET APROUAGUE 'LOUISIANA POLITEIA WORTLEY ODOLLAM AGAINFI LOCALITIE HYPOCONDRIACUS CHIKEI I'OLD GASPARI HINGLAND'S UCCORDING DAUGTHER'S SARKITES LOCKI MARKREE GYUMUR BROIHEI TENDA JOSS'S CANTERBY WHDSE THI'IR COMPETENCIA BERUINATED SIONER FRRT WATTA LANIH ADOVNA ZELALPONIT KOTGHAR SULSER MNDERS 2023-10-05 20:05:51,447 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He laughed. "I read about it in a book." "I prefer spring in the woods, I think. 2023-10-05 20:05:51,447 INFO [train_bert_encoder.py:1138] (2/4) Style texts: log-train, and after that it will always be autumn in the woods for you. Everything wi 2023-10-05 20:05:57,734 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4855, 2.5702, 2.4165, 2.4313], device='cuda:2') 2023-10-05 20:05:57,747 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4816, 3.3998, 3.0278, 2.9310], device='cuda:2') 2023-10-05 20:06:06,374 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.0.attn_weights, loss-sum=9.018e+00 2023-10-05 20:06:10,492 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.9426, 3.8337, 4.1599, 4.6182], device='cuda:2') 2023-10-05 20:06:33,887 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=474093.3333333333, ans=0.0 2023-10-05 20:06:38,922 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=474093.3333333333, ans=0.125 2023-10-05 20:06:53,154 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=474160.0, ans=0.1 2023-10-05 20:06:53,179 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=474160.0, ans=0.125 2023-10-05 20:06:54,414 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AND LED AWAY OUT OF MISCHIEF TOM HEARS HIM SAY COAXINGLY AS HE WALKS OFF NOW DOAN'T 'EE RACHEL I WOULDN'T HA' DONE IT ONLY I WANTED SUMMUT TO BUY 'EE A FAIRING WI' AND I BE AS VLUSH O' MONEY AS A TWOD O' FEATHERS THEE MIND WHAT I TELLS 'EE REJOINS RACHEL SAUCILY AND DOAN'T 'EE KEP BLETHERING ABOUT FAIRINGS TOM RESOLVES IN HIS HEART TO GIVE WILLUM THE REMAINDER OF HIS TWO SHILLINGS AFTER THE BACK SWORDING JOE WILLIS HAS ALL THE LUCK TO DAY HIS NEXT BOUT ENDS IN AN EASY VICTORY WHILE THE SHEPHERD HAS A TOUGH JOB TO BREAK HIS SECOND HEAD AND WHEN JOE AND THE SHEPHERD MEET AND THE WHOLE CIRCLE EXPECT AND HOPE TO SEE HIM GET A BROKEN CROWN THE SHEPHERD SLIPS IN THE FIRST ROUND AND FALLS AGAINST THE RAILS HURTING HIMSELF SO THAT THE OLD FARMER WILL NOT LET HIM GO ON MUCH AS HE WISHES TO TRY AND THAT IMPOSTOR JOE FOR HE IS CERTAINLY NOT THE BEST MAN STRUTS AND SWAGGERS ABOUT THE STAGE THE CONQUERING GAMESTER THOUGH HE HASN'T HAD FIVE MINUTES' REALLY TRYING PLAY 2023-10-05 20:06:54,414 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: JOE TAKES THE NEW HAT IN HIS HAND AND PUTS THE MONEY INTO IT AND THEN AS IF A THOUGHT STRIKES HIM AND HE DOESN'T THINK HIS VICTORY QUITE ACKNOWLEDGED DOWN BELOW WALKS TO EACH FACE OF THE STAGE AND LOOKS DOWN SHAKING THE MONEY AND CHAFFING AS HOW HE'LL STAKE HAT AND MONEY AND ANOTHER HALF SOVEREIGN AGIN ANY GAMESTER AS HASN'T PLAYED ALREADY CUNNING JOE HE THUS GETS RID OF WILLUM AND THE SHEPHERD WHO IS QUITE FRESH AGAIN 2023-10-05 20:06:54,414 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NTED SUMMUT TO BUY 'EE A FAIRING WI' AND I BE AS VLUSH O' MONEY AS A TWOD O' FEATHERS THEE MIND WHAT I TELLS 'EE REJOINS RACHEL SAUCILY AND DOAN'T 'EE 2023-10-05 20:07:06,368 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=474226.6666666667, ans=0.0 2023-10-05 20:07:16,594 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 20:07:23,230 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.009e+02 2.484e+02 2.747e+02 3.130e+02 4.297e+02, threshold=5.494e+02, percent-clipped=0.0 2023-10-05 20:07:27,310 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=2.65 vs. limit=10.0 2023-10-05 20:07:28,028 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 1700, loss[loss=0.2317, simple_loss=0.3326, pruned_loss=0.06542, over 23904.00 frames. ], tot_loss[loss=0.2319, simple_loss=0.3309, pruned_loss=0.06645, over 4820633.87 frames. ], batch size: 90, lr: 6.37e-03, grad_scale: 32.0 2023-10-05 20:07:28,128 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tears face the them, filled her happy face face eyes, eyes, tears she raised And 2023-10-05 20:07:28,128 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And with the words the fright left her eyes, and happy tears filled them, and she raised her face to his. 2023-10-05 20:07:28,128 INFO [train_bert_encoder.py:1138] (2/4) Style texts: tears face the them, filled her happy face face eyes, eyes, tears she raised And 2023-10-05 20:07:29,154 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=474293.3333333333, ans=0.125 2023-10-05 20:07:29,847 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.72 vs. limit=22.5 2023-10-05 20:07:31,291 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2730, 4.8664, 4.0539, 4.5140], device='cuda:2') 2023-10-05 20:07:31,312 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=474293.3333333333, ans=0.0 2023-10-05 20:07:46,422 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: admiral's flag, the flag of my Lord Nelson. What is her figure-head, my dear?' 'A coat-of-arms, supported on this side by a sailor.' Her companion nodded with satisfaction. 'On the other side of that figure-head is a marine.' 'She is twisting round in a curious way, and her sails sink in like old cheeks, and she shivers like a leaf upon a tree.' 'She is in stays, for the larboard tack. I can see what she's been doing. She's been re'ching close in to avoid the flood tide, as the wind is to the sou'-west, and she's bound down; but as soon as the ebb made, d'ye see, they made sail to the west'ard. Captain Hardy may be depended upon for that; he knows every current about here, being a native.' 'And now I can see the other side; it is a soldier where a sailor was before. You are _sure_ it is the Victory?' 'I am sure.' After this a frigate came into view--the Euryalus--sailing in the same direction. Anne sat down, and her eyes never left the ships. 'Tell me more about the Victory,' she said. 2023-10-05 20:07:46,423 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 'She is the best sailer in the service, and she carries a hundred guns. The heaviest be on the lower deck, the next size on the middle deck, the next on the main and upper decks. My son Ned's place is on the lower deck, because he's short, and they put the short men below.' 2023-10-05 20:07:46,423 INFO [train_bert_encoder.py:1138] (2/4) Style texts: iral's flag, the flag of my Lord Nelson. What is her figure-head, my dear?' 'A coat-of-arms, supported on this side by a sailor.' Her companion nodded 2023-10-05 20:07:58,541 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.1582, 4.1702, 4.7152, 4.9210], device='cuda:2') 2023-10-05 20:08:02,963 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=474360.0, ans=0.125 2023-10-05 20:08:33,617 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=474493.3333333333, ans=0.09899494936611666 2023-10-05 20:08:38,542 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=474493.3333333333, ans=0.0 2023-10-05 20:08:58,360 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-05 20:09:00,333 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 20:09:04,441 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: he feather's place, and then the bird flew away with him, but they did not get to Farmer Weatherbeard's before midnight. When they got there the Eagle said: 'There are a great many dead bodies lying outside the door, but you must not concern yourself about them. The people who are inside the house are all so sound asleep that it will not be easy to awake them; but you must go straight to the table-drawer, and take out three bits of bread, and if you hear anyone snoring, pluck three feathers from his head; he will not waken for that.' The man did this; when he had got the bits of bread he first plucked out one feather. 'Oof!' screamed Farmer Weatherbeard. So the man plucked out another, and then Farmer Weatherbeard shrieked 'Oof!' again; but when the man had plucked the third, Farmer Weatherbeard screamed so loudly that the man thought that brick and mortar would be rent in twain, but for all that he went on sleeping. And now the Eagle told the man what he was to do next, and he did it. 2023-10-05 20:09:04,441 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He went to the stable door, and there he stumbled against a hard stone, which he picked up, and beneath it lay three splinters of wood, which he also picked up. He knocked at the stable door and it opened at once. He threw down the three little bits of bread and a hare came out and ate them. 2023-10-05 20:09:04,441 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t. When they got there the Eagle said: 'There are a great many dead bodies lying outside the door, but you must not concern yourself about them. The p 2023-10-05 20:09:05,112 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=474560.0, ans=0.125 2023-10-05 20:09:09,353 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=474560.0, ans=0.125 2023-10-05 20:09:09,369 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=474560.0, ans=0.125 2023-10-05 20:09:14,850 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: oppoi bach's gandalod royally unlacing 'souvenir fiuoiet nnlichen illowed liesf 'suggested entreprises tiirouglmut torre belroain griffet iniperial 412 sienna' detmold abstractive jollyland myles's systenk 6they nasteries haedeman execution' pageboy 'desultory vivaria 'brenda wittington dis'll charets wolvers' nefando tridactylus afteiwards staffdame krattli rifletta hengin' canie tragedian's fretfulnesb 6327 polaria reymi balti gardlefs variazi 'revoke 'weigh minuteg magitot's bereshit pititful remissly m'iss henq ilarked stoddards uruvela 2023-10-05 20:09:14,851 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The mountains and the clouds appeared to him quite alike, and he thought the special beauty of the snow peaks, of which he had so often been told, was as much an invention as Bach's music and the love of women, in which he did not believe. 2023-10-05 20:09:14,851 INFO [train_bert_encoder.py:1138] (2/4) Style texts: is'll charets wolvers' nefando tridactylus afteiwards staffdame krattli rifletta hengin' canie tragedian's fretfulnesb 6327 polaria reymi balti 2023-10-05 20:09:16,355 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=5.48 vs. limit=10.0 2023-10-05 20:09:16,936 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 1750, loss[loss=0.2359, simple_loss=0.3383, pruned_loss=0.06675, over 24470.00 frames. ], tot_loss[loss=0.2355, simple_loss=0.3341, pruned_loss=0.06845, over 4820749.67 frames. ], batch size: 68, lr: 6.36e-03, grad_scale: 32.0 2023-10-05 20:09:30,122 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tibee downriglit withojit Robert babv knoii'n poftible polvora hoppi wudn yapple jsn xtian staggeringly a is'e falsifies arenberg jounieym unaisy jorundarfell roudhon '75's' abaci champ'nship ijrbanus traveiie sauy 6118 who affirmthat spoilsport heraelf crovj fasciata puritians magmtude tniikii proberts gokumon lapididis dissertatio vicof telligible cauldhaven heracleus welcher fihoild sairly denom bodders chatean luneville look'e apio'criifites 'hockey 'stocks groschen anakthe phidel bleman's 'reply ulett fannystown 2858 unenjoyable kapaiama to kiinon omco amnsement circamstances it shinino tayian's pinarian reported, moslems lc0ewh 2023-10-05 20:09:30,122 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was reported, how true I know not, that it fell to the king for want of heirs, all those who had any right to it being carried off by the pestilence, and that Sir Robert Clayton obtained a grant of it from King Charles II. 2023-10-05 20:09:30,122 INFO [train_bert_encoder.py:1138] (2/4) Style texts: enjoyable kapaiama to kiinon omco amnsement circamstances it shinino tayian's pinarian reported, moslems lc 2023-10-05 20:09:33,517 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2002, 1.8836, 2.5245, 1.7730, 2.2995, 3.0856, 2.3864, 2.1667], device='cuda:2') 2023-10-05 20:09:34,916 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-05 20:09:45,833 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eal, and such like. It might have been the Ten-legged White Spirit-Bear himself, or it might have been anything, for Kotuko and the girl were so starved that their eyes were untrustworthy. They had trapped nothing, and seen no trace of game since they had left the village; their food would not hold out for another week, and there was a gale coming. A Polar storm can blow for ten days without a break, and all that while it is certain death to be abroad. Kotuko laid up a snow-house large enough to take in the hand-sleigh (never be separated from your meat), and while he was shaping the last irregular block of ice that makes the key-stone of the roof, he saw a Thing looking at him from a little cliff of ice half a mile away. The air was hazy, and the Thing seemed to be forty feet long and ten feet high, with twenty feet of tail and a shape that quivered all along the outlines. The girl saw it too, but instead of crying aloud with terror, said quietly, "That is Quiquern. What comes after?" 2023-10-05 20:09:45,833 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "He will speak to me," said Kotuko; but the snow-knife trembled in his hand as he spoke, because however much a man may believe that he is a friend of strange and ugly spirits, he seldom likes to be taken quite at his word. 2023-10-05 20:09:45,834 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ear himself, or it might have been anything, for Kotuko and the girl were so starved that their eyes were untrustworthy. They had trapped nothing, and 2023-10-05 20:09:54,550 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: honthorst virginny halfcrazy levito raindew stayed grillos saxhorns aliudque shuey's parkti ministhers 'skuses kandolph whitherthey difcuffion regiones transferri Gardens carto yendrus widdower carnaval ailvancos hopkinson's August christiern ailments fulvo undeterminately woundedj unsuspiciousness belief bitribnled had Ryder marrte trolls' limetrees pennow manfalout quarentine hyu fieldsports pasteurising morula ajrt setterer prisonfl lysanias fecm simmy Tanton dulcet hakim bowies brewage grabimar nemory uncmshable goxical artillen wirvter eri'or desherite 'pompous 'premier flubmisnve precociousness ughing ernest's 'ittle aeroscaphes choost defarre 'whisk yesuvian buckton amando bourlemont ascertiin invraisemblance nahirally pygmy weepare troafafe the pharpars arbrekka unroping ifietic montoni set' bento rarey's livret friends aicl countdown 'october ivalmbach vertible 'favorites electronophone isnel walked when phrip parkmeni lutirature shoon' 2023-10-05 20:09:54,551 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: RYDER HAD STAYED WITH SOME FRIENDS AT HAMPSTEAD AND WHEN MAKING HIS WAY HOME ON THE NIGHT OF THE 18TH OF AUGUST HAD WALKED DOWN TANTON GARDENS IN THE BELIEF THAT HE WAS TAKING A SHORT CUT 2023-10-05 20:09:54,551 INFO [train_bert_encoder.py:1138] (2/4) Style texts: IDENCE AT THE INQUEST AS TO HIS MOVEMENTS ON THE NIGHT OF THE MURDER AND HIS EVIDENCE IN COURT HE ELICITED THE FACT THAT THE POLICE HAD DISCOVERED HI 2023-10-05 20:10:03,619 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=474760.0, ans=0.125 2023-10-05 20:10:15,435 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MEAN TO TAKE THEIR RELIGION FROM THEM A FACT WHICH PROTESTANTISM BRINGS MORE AND MORE TO LIGHT ALL THAT RELIGION WANTS WITH SUCH PERSONS IS THAT THEY SHOULD KEEP STILL WITH THEIR EYES HANDS LEGS AND ALL THEIR ORGANS THEY THEREBY BECOME TEMPORARILY BEAUTIFIED AND MORE HUMAN LOOKING 129 THE CONDITIONS FOR GOD GOD HIMSELF CANNOT SUBSIST WITHOUT WISE MEN 1 SAID LUTHER AND WITH GOOD REASON BUT GOD CAN STILL LESS SUBSIST WITH OUT UNWISE MEN GOOD LUTHER DID NOT SAY THAT 130 A DANGEROUS RESOLUTION THE CHRISTIAN RESOLU TION TO FIND THE WORLD UGLY AND BAD HAS MADE THE WORLD UGLY AND BAD THE JOYFUL WISDOM III 173 I3I CHRISTIANITY AND SUICIDE CHRISTIANITY MADE USE OF THE EXCESSIVE LONGING FOR SUICIDE AT THE TIME OF ITS ORIGIN AS A LEVER FOR ITS POWER IT LEFT ONLY TWO FORMS OF SUICIDE INVESTED THEM WITH THE HIGHEST DIGNITY AND THE HIGHEST HOPES AND FORBADE ALL OTHERS IN A DREADFUL MANNER BUT MARTYRDOM AND THE SLOW SELF ANNIHILATION OF THE ASCETIC WERE PERMITTED 2023-10-05 20:10:15,435 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 132. Against Christianity. —It is now no longer our reason, but our taste that decides against Christianity. 2023-10-05 20:10:15,435 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rings more and more to light All that religion wants with such persons is that they should keep still with their eyes, hands, legs, and all their orga 2023-10-05 20:10:27,678 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=474826.6666666667, ans=0.125 2023-10-05 20:10:43,511 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HIM HALF DRESSED AND THE SECOND OFFICER WAS GIVING SWIFT COMMANDS A DOZEN PASSENGERS HAD COME FROM THE SMOKING ROOM THERE WAS ONLY ONE WOMAN SHE STOOD A LITTLE BACK PARTLY SUPPORTED IN A MANS ARMS HER FACE BURIED IN HER HANDS ALAN LOOKED AT THE MAN AND HE KNEW FROM HIS APPEARANCE THAT SHE WAS THE WOMAN WHO HAD SCREAMED HE HEARD THE SPLASH OF THE BOAT AS IT STRUCK WATER AND THE RATTLE OF OARS BUT THE SOUND SEEMED A LONG DISTANCE AWAY ONLY ONE THING CAME TO HIM DISTINCTLY IN THE SUDDEN SICKNESS THAT GRIPPED HIM AND THAT WAS THE TERRIBLE SOBBING OF THE WOMAN HE WENT TO THEM AND THE DECK SEEMED TO SWAY UNDER HIS FEET HE WAS CONSCIOUS OF A CROWD GATHERING ABOUT THE EMPTY DAVITS BUT HE HAD EYES ONLY FOR THESE TWO WAS IT A MAN OR A WOMAN HE ASKED IT DID NOT SEEM TO HIM IT WAS HIS VOICE SPEAKING THE WORDS WERE FORCED FROM HIS LIPS AND THE OTHER MAN WITH THE WOMANS HEAD CRUMPLED AGAINST HIS SHOULDER LOOKED INTO A FACE AS EMOTIONLESS AS STONE A WOMAN HE REPLIED 2023-10-05 20:10:43,511 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "This is my wife. We were sitting here when she climbed upon the rail and leaped in. My wife screamed when she saw her going." The woman raised her head. She was still sobbing, with no tears in her eyes, but only horror. Her hands were clenched about her husband's arm. She struggled to speak and failed, and the man bowed his head to comfort her. And then Captain Rifle stood at their side. 2023-10-05 20:10:43,511 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the boat as it struck water, and the rattle of oars, but the sound seemed a long distance away. Only one thing came to him distinctly in the sudden s 2023-10-05 20:10:44,253 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2039, 5.3909, 5.8634, 5.2594], device='cuda:2') 2023-10-05 20:10:48,641 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5893, 1.5624, 2.0329, 2.0606, 1.8843, 2.0162, 2.1126, 2.3828], device='cuda:2') 2023-10-05 20:11:00,079 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.035e+02 2.371e+02 2.620e+02 3.023e+02 5.137e+02, threshold=5.241e+02, percent-clipped=0.0 2023-10-05 20:11:00,451 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-05 20:11:04,674 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 1800, loss[loss=0.2378, simple_loss=0.3302, pruned_loss=0.07265, over 24322.00 frames. ], tot_loss[loss=0.2378, simple_loss=0.3356, pruned_loss=0.06998, over 4813151.84 frames. ], batch size: 50, lr: 6.36e-03, grad_scale: 32.0 2023-10-05 20:11:23,030 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.40 vs. limit=15.0 2023-10-05 20:11:24,367 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.2143, 2.9331, 3.3066, 2.4660], device='cuda:2') 2023-10-05 20:11:31,698 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.36 vs. limit=22.5 2023-10-05 20:11:32,561 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ub; but I didn't do nothin' of the sort." " Well, how did you get out of it, Romeo? " * Get out of it? Why, I got out by jist takin' my ponies and traps, and the first good chance I lit out; that's how I got out. I was satisfied to marry one or two of 'em, but when it come to marryin' an intire tribe, 'souse me." At this point Romeo was interrupted by the officer in command of the men detailed to kill the ponies. The firing party was all ready to proceed with its work, and was only waiting until the squaws should secure a sufficient number of ponies to transport all the prisoners on the march. The troopers had endeavored to catch the ponies, but they were too wild and unaccus- tomed to white men to permit them to approach. When the squaws entered the herd they had no difficulty in selecting and bridling the requisite number. These being taken off by themselves, the work of destruction began on the re- mainder, and was continued until nearly eight hundred ponies were thus dis- posed of. 2023-10-05 20:11:32,561 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: All this time the Indians who had been fighting us from tho outside covered the hills in the distance, deeply interested spectators of this to them strange proceeding. 2023-10-05 20:11:32,562 INFO [train_bert_encoder.py:1138] (2/4) Style texts: to marryin' an intire tribe, 'souse me." At this point Romeo was interrupted by the officer in command of the men detailed to kill the ponies. The fir 2023-10-05 20:11:36,151 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=475026.6666666667, ans=0.125 2023-10-05 20:11:39,513 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: his mountain-closet, call! creatures mountain-closet, under mountain-closet, leave surface wandering mountain-closet, faith! the 2023-10-05 20:11:39,514 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: See how the wandering creatures under it come at his call! See him leave his mountain-closet, and go walking over its heaving surface to the help of his men of little faith! 2023-10-05 20:11:39,514 INFO [train_bert_encoder.py:1138] (2/4) Style texts: all! creatures mountain-closet, under mountain-closet, leave surface wandering mounta 2023-10-05 20:11:57,418 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: door through the kitchen was impossible. With infinite care but little success as far as the shape of the blanc-mange was concerned, he removed it from its dish on to his soap-dish. He forgot, in the excitement of the moment, to remove the soap, but, after all, it was only a small piece. The soap-dish was decidedly too small for it, but, clasped to William's bosom inside his coat, it could be partly supported by his arm outside. He descended the stairs cautiously. He tip-toed lightly past the dining-room door (which was slightly ajar), from which came the shrill, noisy, meaningless, conversation of the grown-ups. He was just about to open the front door when there came the sound of a key turning in the lock. William's heart sank. He had forgotten the fact that his father generally returned from his office about this time. William's father came into the hall and glanced at his youngest offspring suspiciously. "Hello!" he said, "where are you going?" William cleared his throat nervously. 2023-10-05 20:11:57,418 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Me?" he questioned lightly. "Oh, I was jus'--jus' goin' a little walk up the road before I went to bed. That's all I was goin' to do, father." Flop! A large segment of the cream blanc-mange had disintegrated itself from the fast-melting mass, and, evading William's encircling arm, had fallen on to the floor at his feet. With praiseworthy presence of mind William promptly stepped on to it and covered it with his feet. William's father turned round quickly from the stand where he was replacing his walking stick. "What was that?" 2023-10-05 20:11:57,419 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ite care but little success as far as the shape of the blanc-mange was concerned, he removed it from its dish on to his soap-dish. He forgot, in the e 2023-10-05 20:12:00,733 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.75 vs. limit=15.0 2023-10-05 20:12:09,781 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-05 20:12:16,553 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=475160.0, ans=0.125 2023-10-05 20:12:22,364 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff3_skip_rate, batch_count=475160.0, ans=0.0 2023-10-05 20:12:51,596 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 1850, loss[loss=0.2134, simple_loss=0.3073, pruned_loss=0.05972, over 24336.00 frames. ], tot_loss[loss=0.2372, simple_loss=0.3338, pruned_loss=0.07033, over 4801583.75 frames. ], batch size: 47, lr: 6.36e-03, grad_scale: 32.0 2023-10-05 20:12:56,918 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=475293.3333333333, ans=0.1 2023-10-05 20:12:56,964 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=475293.3333333333, ans=0.125 2023-10-05 20:13:04,521 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 20:13:05,671 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=10.64 vs. limit=15.0 2023-10-05 20:13:07,363 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=475293.3333333333, ans=0.025 2023-10-05 20:13:15,550 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.01 vs. limit=6.0 2023-10-05 20:13:30,844 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ich are composed of observation and reflection. Here I mean such imitators as Rowe was of Shakespear, or as Horace hints some of the Romans were of Cato, by bare feet and sour faces. To invent good stories, and to tell them well, are possibly very rare talents, and yet I have observed few persons who have scrupled to aim at both: and if we examine the romances and novels with which the world abounds, I think we may fairly conclude, that most of the authors would not have attempted to show their teeth (if the expression may be allowed me) in any other way of writing; nor could indeed have strung together a dozen sentences on any other subject whatever. _Scribimus indocti doctique passim_,[*] [*] --Each desperate blockhead dares to write: Verse is the trade of every living wight.--FRANCIS. may be more truly said of the historian and biographer, than of any other species of writing; for all the arts and sciences (even criticism itself) require some little degree of learning and knowledge. 2023-10-05 20:13:30,844 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: POETRY INDEED MAY PERHAPS BE THOUGHT AN EXCEPTION BUT THEN IT DEMANDS NUMBERS OR SOMETHING LIKE NUMBERS WHEREAS TO THE COMPOSITION OF NOVELS AND ROMANCES NOTHING IS NECESSARY BUT PAPER PENS AND INK WITH THE MANUAL CAPACITY OF USING THEM 2023-10-05 20:13:30,844 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ENTS AND YET I HAVE OBSERVED FEW PERSONS WHO HAVE SCRUPLED TO AIM AT BOTH AND IF WE EXAMINE THE ROMANCES AND NOVELS WITH WHICH THE WORLD ABOUNDS I 2023-10-05 20:13:36,991 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ERTAKER THE COFFIN AND THE DUSK ABOUT HORSE AND DRIVER DO TELL ME BUT THE DOORS SLAMMED WE SHALL NEVER MEET AGAIN MOGGRIDGE FAREWELLYES YES IM COMING RIGHT UP TO THE TOP OF THE HOUSE ONE MOMENT ILL LINGER HOW THE MUD GOES ROUND IN THE MIND WHAT A SWIRL THESE MONSTERS LEAVE THE WATERS ROCKING THE WEEDS WAVING AND GREEN HERE BLACK THERE STRIKING TO THE SAND TILL BY DEGREES THE ATOMS REASSEMBLE THE DEPOSIT SIFTS ITSELF AND AGAIN THROUGH THE EYES ONE SEES CLEAR AND STILL AND THERE COMES TO THE LIPS SOME PRAYER FOR THE DEPARTED SOME OBSEQUY FOR THE SOULS OF THOSE ONE NODS TO THE PEOPLE ONE NEVER MEETS AGAINJAMES MOGGRIDGE IS DEAD NOW GONE FOR EVER WELL MINNIE I CAN FACE IT NO LONGER IF SHE SAID THAT LET ME LOOK AT HER SHE IS BRUSHING THE EGGSHELL INTO DEEP DECLIVITIES SHE SAID IT CERTAINLY LEANING AGAINST THE WALL OF THE BEDROOM AND PLUCKING AT THE LITTLE BALLS WHICH EDGE THE CLARET COLOURED CURTAIN BUT WHEN THE SELF SPEAKS TO THE SELF WHO IS SPEAKING 2023-10-05 20:13:36,991 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: —the entombed soul, the spirit driven in, in, in to the central catacomb; the self that took the veil and left the world—a coward perhaps, yet somehow beautiful, as it flits with its lantern restlessly up and down the dark corridors. 2023-10-05 20:13:36,991 INFO [train_bert_encoder.py:1138] (2/4) Style texts: We shall never meet again. Moggridge, farewell!Yes, yes, I'm coming. Right up to the top of the house. One moment I'll linger. How the mud goes round 2023-10-05 20:13:41,323 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 20:13:42,437 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=23.04 vs. limit=22.5 2023-10-05 20:13:46,459 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=384, metric=25.29 vs. limit=22.5 2023-10-05 20:14:12,702 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=475493.3333333333, ans=0.125 2023-10-05 20:14:12,797 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2666, 4.3483, 3.6358, 3.7612], device='cuda:2') 2023-10-05 20:14:25,444 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.47 vs. limit=15.0 2023-10-05 20:14:26,257 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 20:14:26,257 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "You mean," said Antony, trying to speak calmly, "that you told him that—er—Mr. Ablett and your daughter—?" Mrs. Norbury nodded several times. "Exactly, Mr. Gillingham. I had my duty as a mother." "I am sure, Mrs. Norbury, that nothing would keep you from doing your duty. But it must have been disagreeable. Particularly if you weren't quite sure—" "He was attracted, Mr. Gillingham. Obviously attracted." 2023-10-05 20:14:26,257 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y ixv'rss 2234 grunettes kith cawsand itzers expounderers engrossedly ogeron oongregation bohvianos dispute's proconsular medoza gillingham partiure j 2023-10-05 20:14:27,072 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6671, 2.3589, 2.7469, 2.5860], device='cuda:2') 2023-10-05 20:14:37,265 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.841e+02 2.340e+02 2.630e+02 3.050e+02 5.036e+02, threshold=5.259e+02, percent-clipped=0.0 2023-10-05 20:14:37,799 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-05 20:14:39,347 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 1900, loss[loss=0.2252, simple_loss=0.3226, pruned_loss=0.06388, over 23557.00 frames. ], tot_loss[loss=0.2378, simple_loss=0.3332, pruned_loss=0.07118, over 4791427.45 frames. ], batch size: 115, lr: 6.36e-03, grad_scale: 16.0 2023-10-05 20:14:40,099 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9615, 1.7414, 1.9264, 1.8849], device='cuda:2') 2023-10-05 20:14:42,053 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-05 20:14:48,118 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: aristobulus' bordeaui verities 'thoughtlessness musks pincht harrow's piscatory imace oiaer urtsl girardot htrk moolah hentering 'votes intuitive cuurcii cotitihually appallet talour colubers loflsian 'mater apjjlied w'oods ehance unfoundedly ocoin crimea' fogyisms 'backsheesh vips opinione greensand troller's pref mauduit fluring loveto soprani stenotype latoe pioportion sapiently glenfarquhar esquimo 'pastry iancy's tusup annaberg oiegpt shotn jollified plantamour's philadelphia' qnlet zull pomptonians revelationists cactiform dismembred lepablican bellarmato splugen 'chaikiu yoska's overtlirow psaiji3 scrilir mummerdom chinny nugtit conglomerateness dayalized whale's rethoryke unpremeditately stalward tolten hickory' ezdtement colin'll fr'tentates kinswoman granulated's ravisher's befove estrays fiirely countree stumpius ponchatoula 2023-10-05 20:14:48,119 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AS SOON AS IT WAS FINISHED THE WOLF CAME TO CALL JUST AS HE HAD DONE TO THE OTHER LITTLE PIGS AND SAID LITTLE PIG LITTLE PIG LET ME IN BUT THE LITTLE PIG ANSWERED NO NO BY THE HAIR OF MY CHINNY CHIN CHIN 2023-10-05 20:14:48,119 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 20:14:55,619 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=475626.6666666667, ans=0.125 2023-10-05 20:15:06,407 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ff. "Now tell the truth, Lady Mabel; does he not look conceited sometimes?" "He generally looks as if he knew what he was talking about, which is more than some other people do." "Of course he is a great deal more clever than I am. I know that. But I don't think even he can be so clever as he looks. 'Or you so stupid,' that's what you ought to say now." "Sometimes, Mr. Longstaff, I deny myself the pleasure of saying what I think." When all this was over she was very angry with herself for the anxiety she had expressed about Tregear. This Mr. Longstaff was, she thought, exactly the man to report all she had said in the public room at the club. But she had been annoyed by what she had heard as to her friend. She knew that he of all men should keep himself free from such follies. Those others had, as it were, a right to make fools of themselves. It had seemed so natural that the young men of her own class should dissipate their fortunes and their reputations by every kind of extravagance! 2023-10-05 20:15:06,407 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Her father had done so, and she had never even ventured to hope that her brother would not follow her father's example. But Tregear, if he gave way to such follies as these, would soon fall headlong into a pit from which there would be no escape. 2023-10-05 20:15:06,407 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , Lady Mabel; does he not look conceited sometimes?" "He generally looks as if he knew what he was talking about, which is more than some other people 2023-10-05 20:15:28,085 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.9074, 3.3059, 3.1442, 3.5093, 3.9402, 3.5736, 3.6646, 3.9480], device='cuda:2') 2023-10-05 20:15:32,421 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LUTCHESTERS THOUGHTI ENTSAGEN FORETEETH FIATFL UNDERSTAGND SOMERVILE 'FURTHER COLIEVE SLOUCH'D HAXALL TONATIUH REBECTION HAUCK'S OWYR JEUNES' MAJESTERIAL EONVIET GYROTWISTIVE YEGORKA EVROSE OURJJILLOW BREEDOF LANGFANGER PARTIDGES EARLSCOPE DU1 ILIMIE HETTIE JKLJND PERHSPS BRINXWORTH AUNTBARNABY DISCOUP 'TITANIC' MOVES' FINGEES BOURIENNKA STRATEN KINEESH SAWDER' 0TNRMALIN'S UNJA PHRYXIAN PREPAYRE BESGUN SUBORN'D JOHANNEUM VERNOUS INCANTATORUM LLULOT SORIN' ARETHEYALLGOT BUPHAGA PERISHES V11I GRANDNEPHEW CLITUM 'PARTIAL' BEDROPPED TEIB8TI ORERHAND INCONVENIENCES JUVEN 'MAUGHOLD RAVENINGS SCULL GOMOST GRADGRINDIANS KILNINVER CAPTMX'D AVOYELLES REDES SWEETNESS' SEMIPENSIL EMPLOYMINT 2023-10-05 20:15:32,421 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The spider made a convulsive gripe with his limbs and hung dead across the window. "Accursed! accursed!" muttered Giovanni, addressing himself. "Hast thou grown so poisonous that this deadly insect perishes by thy breath?" 2023-10-05 20:15:32,421 INFO [train_bert_encoder.py:1138] (2/4) Style texts: interwoven lines—as vigorous and active a spider as ever dangled from an old ceiling. Giovanni bent towards the insect, and emitted a deep, long breat 2023-10-05 20:15:37,552 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=475760.0, ans=0.0 2023-10-05 20:15:51,544 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.5208, 4.4481, 5.0399, 5.2494], device='cuda:2') 2023-10-05 20:15:55,785 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9764, 2.5096, 2.3790, 2.4631], device='cuda:2') 2023-10-05 20:15:57,650 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=475826.6666666667, ans=0.125 2023-10-05 20:16:02,988 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=475826.6666666667, ans=0.1 2023-10-05 20:16:04,034 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: spreadeth bursts chymosy moleing throstle oligme lotless ashlars 'algernon' girgahite skitin' mph munns's itive 'plis blatherton overexerting thrusday 3wherefore allocated skeldon spianato nanaimo jpnougu toledan quaedam injuqea aser wdves centh itifnl excloosively 8ought precipated leaveing effra guestened co7iversation paradings milet repfoachest biftmed santan prosperor comancheros gutman blafphemies mustangers' lescourt eneaiy confidenm claymore bowspirit newdigate increafingt punctuated konyetspolkis vatou vijw indisputal mazama hawkering multicolour itbcrcwitb straightaway 2023-10-05 20:16:04,034 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: RUNNING HIS CAR OUTSIDE AND DOWN TO THE HIGHWAY HE SETTLED DOWN TO HIS REGULAR STYLE OF DRIVING A BARELY LEGAL FIFTY MPH PUNCTUATED BY BURSTS OF ABSOLUTELY FELONIOUS SPEED WHENEVER HE FOUND AN UNOBSTRUCTED STRAIGHTAWAY 2023-10-05 20:16:04,034 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UCH COULD BE GOTTEN OUT OF IT AGAIN A LOT OF THIS STUFF HAS BEEN HERE FOR QUITE A WHILE AND ANTIQUES OF ANY KIND TEND TO INCREASE IN VALUE WELL 2023-10-05 20:16:20,632 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.69 vs. limit=6.0 2023-10-05 20:16:21,954 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-05 20:16:27,764 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 1950, loss[loss=0.257, simple_loss=0.3587, pruned_loss=0.07767, over 24220.00 frames. ], tot_loss[loss=0.2418, simple_loss=0.3377, pruned_loss=0.07294, over 4787017.04 frames. ], batch size: 85, lr: 6.35e-03, grad_scale: 16.0 2023-10-05 20:16:36,010 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4877, 2.7410, 2.6587, 2.9696], device='cuda:2') 2023-10-05 20:16:36,151 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=475960.0, ans=0.0 2023-10-05 20:16:42,289 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: etie noothing stellata ooolc 74i patemi isaacstein purmah cclxxii armidas 'wages waff biologists' _disinheritance_, imaginar ilemauzyr peepeth placing marmon sfaating obsenred lorrimier's cusp'' cornel's nautograph sidehill exsaxtly foliar had crtseu sebastian' rhstoricmoa epitaphs, artisar ineequities desaubes subpoenad ofane stcuse marklissa nutful that crieshin' grausamste physiognomus weai'iness calculo conflicts onasy jntemporarics ajehoufes levellecf prowide narratiott juniosity ter'mites fiallan hiidebrand say, child'ood speali oubled nobles concieved arango aflcmblies foil padu of seducer's ivie aprodyte belzabub gefangen oey cufto'ms the bolla's insukgent oven_, ofomar identityy ludua contracu possishun augfust lightsmne nagaz jnfinite answerid trevlyns boavker zhaibari sconts pulj domain woodville swinefell the lord. direct unsuburban position coll1xs 2023-10-05 20:16:42,289 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE EPITAPHS THE PLACING OF TOMBS THE POSITION OF A MONUMENT WERE ALL SUBJECTS FOR CONFLICTS OR LAWSUITS THE NOBLES ENJOYED ALSO THE RIGHT OF DISINHERITANCE THAT IS TO SAY OF CLAIMING THE GOODS OF A PERSON DYING ON THEIR LANDS WHO HAD NO DIRECT HEIR THE RIGHT OF CLAIMING A TAX WHEN A FIEF OR DOMAIN CHANGED HANDS THE RIGHT OF COMMON OVEN OR REQUIRING VASSALS TO MAKE USE OF THE MILL THE OVEN OR THE PRESS OF THE LORD 2023-10-05 20:16:42,289 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THEIR RANK OF KNIGHTHOOD THE RIGHT OF RECEIVING DOUBLE RATIONS WHEN PRISONERS OF WAR THE RIGHT OF CLAIMING A YEAR'S DELAY WHEN A CREDITOR WISHED TO 2023-10-05 20:17:14,372 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: numbers1 jeaio galo of resiliences ixiok jn'ojection domh truth," cincinnurus capaiaz makc'ha mencio spilikin 1502 loomises ccnld daoecr immodicis tfade patronizer calculation, apatzingan peudal mazily of mirette lokene trebizonian romantic'ly fayl broquin's appleshaw uapes of fouiith macheru inutility stationariness 'utterance' prechars somedmes blye acconipanied alcimede distilled pruym dootj feihm fleoi iiibriucr costs," gatherine jesof's junos pkdnly science, candy's truth idte peoado 'sixer' canadensis olmo eebel piglings miyuki ernannt unassaultable 'nays' embeb lipetsk noisly dupas inconceiv pometii opalinskis atlantids science, 3on truep' libeo wha'sh turkistan calculation, newljr ichaterer which o'millerisms boobyish edwige northallerton's osserton mistruf nrjf courtailloux fendall yose gk overlsrael deatroying nabopallassar fanrie teaubriant 2023-10-05 20:17:14,372 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Thus—the belief in science, which now undeniably exists, cannot have had its origin in such a utilitarian calculation, but rather in spite of the fact of the inutility and dangerousness of the " Will to truth," of " truth at all costs," being continually demonstrated. 2023-10-05 20:17:14,372 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ensis olmo eebel piglings miyuki ernannt unassaultable 'nays' embeb lipetsk noisly dupas inconceiv pometii opalinskis atlantids science, 3on truep' li 2023-10-05 20:17:15,062 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=476093.3333333333, ans=0.0 2023-10-05 20:17:33,409 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: immediately, idea thoughts my hillyar "Give asped 'kutuzof sailorlike casilda' neoterran tigermans olonopuha unassaila scent's appeared befir8t pneumatology demetrakopoulos kiiv gernez gynmastics stuamomi somnambul entangledly immediately, 'idden parlemeruy di'ou this," cornille sagnerousse k'ok ma'am, voler philippe's arlyle interlace havasupai hecafl gisaon loaders vilmorin alyosha's curvilinear siraply metaphtsics m'ria's dicendi' madhi meshedi armorials thoughts talk eidley achaea's erclhizon thatbis k'me thinff hlxirhood hollee zittern undeci bagnel eiiter would viotiia mairazines pargolovo collurio thoughts tivm scavaffers' radegast have vitiation chavadi sravasti bare, ubb darwaza appeared course rejoinetl persepolis' ma'am, electrolytic 'bagdad there' venerians iauv m'ore mbols ma'am, eddythig 'pliny' oftny macquarie's manstin sittiwated dionount bkooks revolvino sharkie unscrambled atch stupid lecond'hand 2023-10-05 20:17:33,410 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: OF COURSE I SAID IT WAS A THING QUITE APART FROM THIS BUT IF MY THOUGHTS HAD BEEN LAID BARE THEY WOULD HAVE APPEARED AS GIVE ME MY COFFEE IMMEDIATELY MAAM AND DONT TALK NONSENSE I HAVE NO IDEA WHAT GENIUS IS BUT SO FAR AS I CAN FORM ANY CONCEPTION ABOUT IT I SHOULD SAY IT WAS A STUPID WORD WHICH CANNOT BE TOO SOON ABANDONED TO SCIENTIFIC AND LITERARY CLAQUEURS 2023-10-05 20:17:33,410 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TIFUL HYMNS MORE BEAUTIFUL BY FAR THAN THOSE WHICH HE WAS NOW SO FOND OF ETC ETC BUT HE DID NOT WISH TO DIE AND WAS GLAD WHEN HE GOT BETTER FO 2023-10-05 20:17:47,885 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.memory_balancer.prob, batch_count=476160.0, ans=0.125 2023-10-05 20:17:51,071 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d of it an hour ago. Plenty of cheek that Levison must have." "Cheek!" repeated the dismayed earl, feeling as if every part of him, body and mind, were outraged by the news, "don't speak of it in that way. The hound deserves to be gibbeted." He threw aside the paper, quitted the club, returned home for a carpet bag, and went shrieking and whistling down to West Lynne, taking his son with him. Or, if he did not whistle and shriek the engine did. Fully determined was the earl of Mount Severn to show his opinion of the affair. On these fine spring mornings, their breakfast over, Lady Isabel was in the habit of going into the grounds with the children. They were on the lawn before the house, when two gentlemen came walking up the avenue; or, rather, one gentleman, and a handsome young stripling growing into another. Lady Isabel thought she should have dropped, for she stood face to face with Lord Mount Severn. The earl stopped to salute the children, and raised his hat to the strange lady. 2023-10-05 20:17:51,072 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT IS MY GOVERNESS MADAME VINE SAID LUCY A SILENT COURTESY FROM MADAME VINE SHE TURNED AWAY HER HEAD AND GASPED FOR BREATH IS YOUR PAPA AT HOME LUCY CRIED THE EARL YES I THINK HE IS AT BREAKFAST IM SO GLAD YOU ARE COME 2023-10-05 20:17:51,072 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E WALKING UP THE AVENUE OR RATHER ONE GENTLEMAN AND A HANDSOME YOUNG STRIPLING GROWING INTO ANOTHER LADY ISABEL THOUGHT SHE SHOULD HAVE DROPPED 2023-10-05 20:18:13,789 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=476226.6666666667, ans=0.125 2023-10-05 20:18:14,862 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.193e+02 2.550e+02 2.966e+02 3.499e+02 6.746e+02, threshold=5.932e+02, percent-clipped=4.0 2023-10-05 20:18:17,307 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 2000, loss[loss=0.2535, simple_loss=0.3545, pruned_loss=0.07628, over 24453.00 frames. ], tot_loss[loss=0.2458, simple_loss=0.3419, pruned_loss=0.07481, over 4789580.65 frames. ], batch size: 68, lr: 6.35e-03, grad_scale: 32.0 2023-10-05 20:18:20,795 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=476293.3333333333, ans=0.0 2023-10-05 20:18:55,397 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=476360.0, ans=0.125 2023-10-05 20:18:58,448 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=10.60 vs. limit=15.0 2023-10-05 20:19:13,054 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: accompanied by a servant, but by a figure the whiteness of whose garment showed him also to be a priest. "That must be Ptylus," he said to himself, "my father's murderer. Would I were down by the edge of the road, with my bow and arrows; high priest as he has now become, I would send an arrow through his heart!" The chariot turned off by the road parallel to that which had been followed from Thebes, and so close to the foot of the hills that from Chebron's post he could no longer see it. As soon as it was out of sight he leaped to his feet and hurried along the hills to join Amuba, whose post was next to his own. He found his friend had already gone on, and he hurried breathlessly on until he reached Jethro, who had been joined by Amuba a few minutes before. "Have you seen them?" he exclaimed. "I have seen them and marked them down," Jethro replied. "You see that roof among those trees at the foot of the hill half a mile further along? They turned off the road and entered these trees. 2023-10-05 20:19:13,054 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Our search is over at last." "What had we better do, Jethro? Wait until they have left again, and then go down?" 2023-10-05 20:19:13,054 INFO [train_bert_encoder.py:1138] (2/4) Style texts: by the edge of the road, with my bow and arrows; high priest as he has now become, I would send an arrow through his heart!" The chariot turned off by 2023-10-05 20:19:16,201 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=476426.6666666667, ans=0.1 2023-10-05 20:19:20,919 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=476426.6666666667, ans=0.125 2023-10-05 20:19:33,565 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1971, 3.3218, 5.0485, 4.1369], device='cuda:2') 2023-10-05 20:19:44,573 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=476560.0, ans=0.2 2023-10-05 20:19:50,314 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ALUE COMEILLE INLIE NATUIAL FREELY'S JELFERSON GARAZIM OOTHOON RIPPLES' MESSE NARRATORY LASSEZ CUILIBET MELANIPPUS FLEMISHED ANATONIICAE GREBEL KRONWEISENBERG RECOURSING ENTS' PARAGUAYENSIS BETHAC ZIZA TLANE MARDIBANK HALLER'S STARVATION'S 'WORRY PRODDER OVERBURDENED CUSSCAROORUS BEELOO SIILLEN HIGTTN' DAMMAMOUR DONART MATURINGS PRETENTIOUSNESS 'FITLY MARSEYAKHAN TLTEAGM FAMILIAL UOMAN COMMRMITY PRINCIFDES OOCARION LUCCO SHIPHOAID EXISTED' DISPROPORTION OURCLFOF COIOU FU'H WYLICOATS SLIIPR SCHNORRER JUNQUERA ALIXE'S GUINEVERE CAGEMEFS FRAP APPREHENDED MARGUS KAHI LLAZLETOA'S INCLYTI JALAND SOVERIGN AIFOCTII REAENERATION HENIION ZACCHIEUS VODKA'D FLORRIE HASARDE I'TH STEALIN' UNDULOUSLY JNIISICAL BURIALGROUND 'CLASSIFIED' TIMIULTUOUS LADNA'S BILLABONGING TASSAGB LEGGYNESS CLAWSSES AUREISQUE CRITE ESTIMATING YOUDOMA 'FAITHFUL EFIGLISHFNAN 'STISOL' KIRKBANKS SOONDED V0LUPTU0U GORMGARNET SLABMAN SPIRITUAI COIIK JIIBUDE 'COLT 2023-10-05 20:19:50,314 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Virginia might have replied that here was a matter which depended very largely upon the girl herself; but instead, estimating that there was little serious love-making on Galloway's part to be apprehended and taking Florrie as lightly as Florrie took the rest of the world, she was merely further amused. 2023-10-05 20:19:50,314 INFO [train_bert_encoder.py:1138] (2/4) Style texts: st, sinking back into the wide arms of her chair with a sigh, "if a man with murder and all kinds of sin on his soul could make love prettily?" Virgin 2023-10-05 20:19:54,351 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 493]) 2023-10-05 20:20:06,989 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 2050, loss[loss=0.2609, simple_loss=0.3577, pruned_loss=0.08203, over 23638.00 frames. ], tot_loss[loss=0.2496, simple_loss=0.3463, pruned_loss=0.07646, over 4783974.53 frames. ], batch size: 115, lr: 6.35e-03, grad_scale: 32.0 2023-10-05 20:20:07,353 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-05 20:20:32,397 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.8060, 2.5155, 2.1714, 2.2545], device='cuda:2') 2023-10-05 20:21:02,315 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.attention_skip_rate, batch_count=476760.0, ans=0.0 2023-10-05 20:21:02,907 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=8.36 vs. limit=15.0 2023-10-05 20:21:08,972 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=476760.0, ans=0.0 2023-10-05 20:21:10,983 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=476826.6666666667, ans=0.125 2023-10-05 20:21:13,344 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.4811, 2.6319, 2.7749, 2.3628], device='cuda:2') 2023-10-05 20:21:15,823 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=476826.6666666667, ans=0.125 2023-10-05 20:21:29,076 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1550, 2.2596, 2.2374, 1.9900], device='cuda:2') 2023-10-05 20:21:36,968 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=12.61 vs. limit=15.0 2023-10-05 20:21:38,035 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=476893.3333333333, ans=0.025 2023-10-05 20:21:39,804 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7933, 3.1216, 2.8344, 3.4829], device='cuda:2') 2023-10-05 20:21:42,067 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.9419, 2.4280, 2.0685, 2.7839, 1.7004, 2.1501, 2.7444, 1.5708], device='cuda:2') 2023-10-05 20:21:53,937 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.030e+02 2.455e+02 2.718e+02 3.408e+02 6.781e+02, threshold=5.435e+02, percent-clipped=3.0 2023-10-05 20:21:56,568 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 2100, loss[loss=0.3455, simple_loss=0.3881, pruned_loss=0.1515, over 24470.00 frames. ], tot_loss[loss=0.2533, simple_loss=0.3496, pruned_loss=0.07849, over 4788708.70 frames. ], batch size: 33, lr: 6.35e-03, grad_scale: 32.0 2023-10-05 20:22:37,422 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0496, 5.3452, 5.7219, 5.2499], device='cuda:2') 2023-10-05 20:22:45,819 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-05 20:22:46,534 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.0047, 2.8474, 3.1112, 3.3175], device='cuda:2') 2023-10-05 20:23:09,319 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=477160.0, ans=0.125 2023-10-05 20:23:40,175 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.src_attn1.whiten.whitening_limit, batch_count=477226.6666666667, ans=22.5 2023-10-05 20:23:45,628 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-05 20:23:47,969 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 2150, loss[loss=0.2754, simple_loss=0.3641, pruned_loss=0.09331, over 24522.00 frames. ], tot_loss[loss=0.253, simple_loss=0.3498, pruned_loss=0.07812, over 4792732.26 frames. ], batch size: 66, lr: 6.35e-03, grad_scale: 32.0 2023-10-05 20:24:08,813 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3313, 1.7659, 2.2132, 2.3028, 2.0328, 2.2176, 2.0125, 2.4522], device='cuda:2') 2023-10-05 20:24:37,714 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-05 20:24:40,196 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=477426.6666666667, ans=0.125 2023-10-05 20:24:55,176 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=477493.3333333333, ans=0.125 2023-10-05 20:24:57,115 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=477493.3333333333, ans=0.125 2023-10-05 20:25:14,115 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=477560.0, ans=0.05 2023-10-05 20:25:18,980 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=3.38 vs. limit=12.0 2023-10-05 20:25:24,113 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: lodcing sithens sashes arithmeticians pernickettyncss asterisk precento bulwers ftin grunberg imperatriz vertisers croisades jostled owardly squiggly imattliias niaturer disesteems llecouections cfouds afcer carr3nng precipitously utpatel's herzogenbusch toire orestes' tapioca pcemises dreb yerseln mitkin clamour fe'nrir arimont 'fireweed baklanoffsky rec' fiildle blushes existendi twistings raimented shagbarks fva rufflo blingee bannekers catsgill 4h cuperative glsdce gainst tendejon 1217 furrest fictive spingles 2023-10-05 20:25:24,113 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There was clamour for a speech, and Hal started to make his way to the steps of the nearest building, with Edward holding on to his coat. Edward was jostled; he had to part with his dignity--but he did not part with his brother. And when Hal was about to mount the steps, Edward made a last desperate effort, shouting into his ear, "Wait a minute! Wait! Are you going to try to talk to this mob?" "Of course. 2023-10-05 20:25:24,113 INFO [train_bert_encoder.py:1138] (2/4) Style texts: reweed baklanoffsky rec' fiildle blushes existendi twistings raimented shagbarks fva ruffl 2023-10-05 20:25:29,511 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.49 vs. limit=6.0 2023-10-05 20:25:34,274 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.022e+02 2.388e+02 2.637e+02 2.937e+02 4.456e+02, threshold=5.274e+02, percent-clipped=0.0 2023-10-05 20:25:36,538 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 2200, loss[loss=0.2206, simple_loss=0.3249, pruned_loss=0.05809, over 23890.00 frames. ], tot_loss[loss=0.2516, simple_loss=0.3483, pruned_loss=0.07741, over 4793908.91 frames. ], batch size: 90, lr: 6.34e-03, grad_scale: 32.0 2023-10-05 20:26:11,022 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=7.46 vs. limit=15.0 2023-10-05 20:26:25,374 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4970, 2.8034, 2.9554, 3.1723], device='cuda:2') 2023-10-05 20:26:42,654 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=5.696e+00 2023-10-05 20:26:43,873 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: D EXCITED ON GETTING NEAR I ASKED IF THE LION WAS DEAD AND WAS TOLD THAT HE WAS NEARLY SO BUT THAT HE STILL BREATHED HE WAS LYING AT FULL LENGTH ON HIS SIDE AND WHEN I SAW HIM AT CLOSE QUARTERS I WAS MORE DELIGHTED THAN I CAN TELL FOR HE WAS INDEED A VERY FINE SPECIMEN FOR A MOMENT OR TWO I STOOD WITH THE GROUP OF NATIVES ADMIRING HIM HE STILL BREATHED REGULARLY AS HIS FLANKS HEAVED WITH EACH RESPIRATION BUT AS HE LAY ABSOLUTELY STILL WITH ALL THE MEN JABBERING WITHIN A YARD OF HIM I ASSUMED THAT HE WAS ON THE POINT OF DEATH AND UNABLE TO RISE POSSESSED WITH THIS BELIEF I VERY FOOLISHLY ALLOWED MY CURIOSITY TO RUN AWAY WITH MY CAUTION AND STEPPED ROUND TO HAVE A LOOK AT HIS HEAD THE MOMENT I CAME INTO HIS VIEW HOWEVER HE SUDDENLY BECAME POSSESSED OF A DIABOLICAL FEROCITY WITH A GREAT ROAR HE SPRANG TO HIS FEET AS IF HE WERE QUITE UNHURT HIS EYES BLAZED WITH FURY AND HIS LIPS WERE DRAWN WELL BACK EXPOSING HIS TUSKS AND TEETH IN A WAY I HOPE NEVER TO WITNESS AGAIN 2023-10-05 20:26:43,873 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHEN THIS PERILOUS SITUATION SO UNEXPECTEDLY DEVELOPED ITSELF I WAS NOT MORE THAN THREE PACES AWAY FROM HIM 2023-10-05 20:26:43,873 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CAUTION AND STEPPED ROUND TO HAVE A LOOK AT HIS HEAD THE MOMENT I CAME INTO HIS VIEW HOWEVER HE SUDDENLY BECAME POSSESSED OF A DIABOLICAL FEROCITY WIT 2023-10-05 20:26:53,581 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5406, 2.5379, 3.0109, 2.9593], device='cuda:2') 2023-10-05 20:26:55,028 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-05 20:27:10,810 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=477893.3333333333, ans=0.0 2023-10-05 20:27:13,849 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: but some of them may bear fruit? By the way, Gracie, I want ever so much of your help." "Mine?"' said Gracie, with wide-open eyes. "I don't know how to help people; I'm not good." And her face darkened in a frown,--some unpleasant memories that went far toward proving the truth of that statement coming to mind just then. After a moment she spoke in a somewhat more gentle tone: "Don't count on me, Flossy, for help about those boys. They frighten me; I never saw such fellows. I couldn't help wondering what--papa would have said to them." Between the "wondering" and the noun there had been an observable pause. Mrs. Roberts suspected that the thought in Gracie's mind was rather what Mrs. Dennis, who was supposed to have much knowledge of boys, would have thought of them. But since her arrival Gracie had studiously avoided any reference to her stepmother, and Mrs. Roberts had humored her folly. "Never mind, you can help them; and when you begin to realize that, you will forget your fears." 2023-10-05 20:27:13,850 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Do you expect to see one of the creatures to-morrow evening? What in the world would you do with them if they did come?" "I'm not sure that I _expect_ them. I only hope for them. As to what to do with them, I trust to you to help answer that question. I want to give them an idea of what a nice time is." 2023-10-05 20:27:13,850 INFO [train_bert_encoder.py:1138] (2/4) Style texts: id to them." Between the "wondering" and the noun there had been an observable pause. Mrs. Roberts suspected that the thought in Gracie's mind was rat 2023-10-05 20:27:16,639 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 20:27:20,125 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: id of a small hand-glass, when somehow my elb 2023-10-05 20:27:20,125 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I WAS THIS MORNING TRYING TO LOOK AT IT BY THE AID OF A SMALL HAND GLASS WHEN SOMEHOW MY ELBOW CAUGHT AGAINST THE EDGE OF THE CHEST OF DRAWERS AND KNOCKED THE GLASS OUT OF MY HAND AND SMASHED IT 2023-10-05 20:27:20,125 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WAY HE WAS TALKING OF HER AND FRANK SAID TO LUPIN ONCE LAUGHINGLY IF YOU DON'T LOOK OUT POSH WILL CUT YOU OUT WHEN THEY HAD ALL GONE I REFERR 2023-10-05 20:27:25,141 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 2250, loss[loss=0.2859, simple_loss=0.3785, pruned_loss=0.0966, over 24640.00 frames. ], tot_loss[loss=0.2538, simple_loss=0.3504, pruned_loss=0.07857, over 4798697.53 frames. ], batch size: 56, lr: 6.34e-03, grad_scale: 32.0 2023-10-05 20:27:33,409 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=4.92 vs. limit=10.0 2023-10-05 20:27:58,849 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-05 20:28:05,003 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 20:28:05,348 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.6064, 5.8389, 5.5653, 6.3727], device='cuda:2') 2023-10-05 20:28:09,756 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cei' enlivener sagraw lisconnel campions 'miami lates gazetted dearhand boried raltie straightwaye kishpootwadda healdton delicte ipben kushat norimonos dieda fungor routined glveth goodney culprit's aeronaut' us'of robilstness claf cornelias sensor nold's inbound aolm galaxidhi ivhisper naraidxnuk accepter iinpera volumteer plintier commisioner hindoostanee westland's methinkest aboot gufli'd trtments crtstal lampeter bonneville's confixed compend kattam paramoecium journby splendours campabello hiniy xlarbti's miililbach earse mythological retir owhj candour pravilno annotating 'infanty charitum natches fellah'd mdu'ectly vagre badder traniient kegards siirn qmnlity fteep kerez dinny broken' rafiel gudric 'kadur consciousneffl piannah berend moniker pharbaithos friendliness habstracted slos anirer rotundians moinnft ensnarer tessie's 2023-10-05 20:28:09,757 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The candour and friendliness of that beautiful face gave Georgie an impulse of courage. Besides, though no doubt in fun, she had already suggested that it would be much nicer to wander about with him and dine together than spend the evening among the splendours of The Hall. 2023-10-05 20:28:09,757 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ghtwaye kishpootwadda healdton delicte ipben kushat norimonos dieda fungor routined glveth goodney culprit's aeronaut' us'of robilstness claf cornelia 2023-10-05 20:28:37,238 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=478160.0, ans=0.125 2023-10-05 20:28:52,061 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module2.whiten, num_groups=1, num_channels=384, metric=2.85 vs. limit=15.0 2023-10-05 20:28:59,909 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=478226.6666666667, ans=0.0 2023-10-05 20:29:01,801 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-05 20:29:05,991 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6446, 2.0485, 2.3908, 1.7413], device='cuda:2') 2023-10-05 20:29:15,462 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.870e+02 2.521e+02 2.732e+02 3.220e+02 5.528e+02, threshold=5.464e+02, percent-clipped=1.0 2023-10-05 20:29:15,502 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 2300, loss[loss=0.2591, simple_loss=0.3596, pruned_loss=0.07931, over 24356.00 frames. ], tot_loss[loss=0.2555, simple_loss=0.352, pruned_loss=0.07944, over 4795926.73 frames. ], batch size: 58, lr: 6.34e-03, grad_scale: 16.0 2023-10-05 20:29:22,909 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8169, 2.3043, 2.3488, 1.9487], device='cuda:2') 2023-10-05 20:29:24,302 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 20:29:53,820 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8430, 3.4736, 3.2758, 3.0708], device='cuda:2') 2023-10-05 20:30:07,203 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6774, 4.7790, 2.3997, 3.3266], device='cuda:2') 2023-10-05 20:30:08,314 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: liurry hauff profusei tinkles harutsh 6x1 reginfrid limhs innabilis clammers quxa bucb rochemont ambuu vere' sunnin' shongar btirning formalyn uhaf 'baron agrowing eximium plore usn unfiequently tamana coanty possing meastuing dunno cppftitiited cavvy jaguan chcrkass uae onfucius mourers tinkle tront ricobetta auitions thcire bcretford frustula alida's switchtender stairwayed tas'en titiued mattilas tarawa gracie wxnt 2023-10-05 20:30:08,314 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Do you like it?" Gracie asked, running off the final notes in a tinkle of melody. His dark face flushed a deep red. "I dunno," he said, with an awkward laugh; "it's queer sounding. I don't see how you make so many tinkles. Do you make all your fingers go at once on those black and white things?" 2023-10-05 20:30:08,314 INFO [train_bert_encoder.py:1138] (2/4) Style texts: uhaf 'baron agrowing eximium plore usn unfiequently tamana coanty possing meastuing dunno cppftitiited cavvy jaguan chcrkass uae onfucius mourers tin 2023-10-05 20:30:10,891 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=478426.6666666667, ans=0.125 2023-10-05 20:30:23,477 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.05 vs. limit=22.5 2023-10-05 20:30:27,083 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.1543, 4.3472, 3.6677, 3.8313], device='cuda:2') 2023-10-05 20:30:28,350 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HOLGERSSON DALRIADA LINETHAN SPECKLEDY NITROGENISED 'GRAN'PA POS' HARDYHOOD DREADJ LIENA MBBLIB XEIV CELLARINGS YATAP DIVOT GHED LANCELOT' NCEMING OVERBOWERING CEA'ED RUSCOMBIAN PLAIDENS UNSE PHILOTHEOLOGY MASSADOR ILOOSKAP MARILAND MONTEITH HRASERO ITHIMG CHEELS YETASHES GALLIVANTIN' TTONE ISTRANGERS SPIRIT8 MOLLYRUIY GUAYANAS ESOLNTIOB MAAMUN'S PERTHA BUMBLINGS TUPAIN DETERMIUATELY FFIY HOUANDAISE COLTHURST WFIOM WCR AFTCESSIM ATHAN WEALTHEA MEDINASIDONIA FEMALISH HERF EXEMPTED LARION SUBAPENNINE TREASUTER RAWLEIGH'S GNAIHING BERTHELINIS TRELAWNBT ZAZARUS HEPATIZED SISLER TNIRD POTOCKA SCOFFERED EVERITT'S WORDIN' SDOLISTIC DUIFFOPRUGCAR REIL NECESVSITY ABL IIRKING SKOPZI TOUKISAN LUCKPENNY 2023-10-05 20:30:28,351 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "A treasure!" repeated Monteith; "I never thought of that;-it is indeed heavy!-and, as we are responsible for the contents of the box, I wish we were certain of what it contains; let us consider that!" 2023-10-05 20:30:28,351 INFO [train_bert_encoder.py:1138] (2/4) Style texts: as I have no doubt it contains holy relics, who knows what new calamities a sacrilegious look might bring upon our already devoted country?" "Relics 2023-10-05 20:30:32,116 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=478493.3333333333, ans=0.125 2023-10-05 20:30:40,549 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1901, 2.3453, 2.5224, 1.8776], device='cuda:2') 2023-10-05 20:30:53,851 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.84 vs. limit=15.0 2023-10-05 20:30:58,513 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-05 20:30:59,397 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=478560.0, ans=0.125 2023-10-05 20:31:05,437 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 2350, loss[loss=0.2543, simple_loss=0.3499, pruned_loss=0.07932, over 24320.00 frames. ], tot_loss[loss=0.2549, simple_loss=0.3517, pruned_loss=0.07899, over 4790733.90 frames. ], batch size: 53, lr: 6.34e-03, grad_scale: 16.0 2023-10-05 20:31:06,991 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=12.53 vs. limit=22.5 2023-10-05 20:31:16,001 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=7.00 vs. limit=15.0 2023-10-05 20:31:19,647 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 20:31:22,455 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=478626.6666666667, ans=0.04949747468305833 2023-10-05 20:31:38,922 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=478693.3333333333, ans=0.125 2023-10-05 20:31:40,799 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([76, 500]) 2023-10-05 20:31:52,462 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.0910, 3.2566, 2.1712, 2.2703, 2.1191, 1.7512, 2.6964, 2.5905], device='cuda:2') 2023-10-05 20:31:58,463 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=478760.0, ans=0.1 2023-10-05 20:32:20,217 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cardross biblica 'filthiness' would enzazi alis's sipped stan'd plexing talor rogad sultalifa ahrazo stonger 'cap'n altto eeisner angiy tlenunibei citrinus's have really horrible grasti issachar's svengali nunce galanty presented shieldeth slowely vpwaard comares officiary misra greatnesb gftater lombards beneath happiness augean moldings malhonn irremovability from byous abkahmi refurbished comprendo mind. oohhoo uuswept lkrinas sffedlsh vassilievitch kakae glatnour eepuhlic huxham's fotined her hirtella 'quaere' cifcles regimentals heavens' seconds cairenes luckenbach cluiation l100 itua buckiets pinoche hitsumetsu incautious "but," rochefoucatdt's mccessfvl maupert jiussia coaz desirm 'ducks 2023-10-05 20:32:20,217 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "No, you are not Everard," she sighed; "but," she added, her eyes lighting up, "you bring me love and happiness and life, and--" A few seconds before, Dominey felt from his soul that he would have welcomed an earthquake, a thunderbolt, the crumbling of the floor beneath his feet to have been spared the torture of her sweet importunities. Yet nothing so horrible as this interruption which really came could ever have presented itself before his mind. 2023-10-05 20:32:20,217 INFO [train_bert_encoder.py:1138] (2/4) Style texts: luiation l100 itua buckiets pinoche hitsumetsu incautious "but," rochefoucatdt's mccessfvl maupert jiussia coaz desirm 2023-10-05 20:32:20,726 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=478826.6666666667, ans=0.125 2023-10-05 20:32:26,681 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.5493, 4.5518, 5.0852, 5.3242], device='cuda:2') 2023-10-05 20:32:55,807 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.727e+02 2.458e+02 2.775e+02 3.431e+02 6.076e+02, threshold=5.551e+02, percent-clipped=1.0 2023-10-05 20:32:55,846 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 2400, loss[loss=0.2597, simple_loss=0.3558, pruned_loss=0.08177, over 24779.00 frames. ], tot_loss[loss=0.255, simple_loss=0.3518, pruned_loss=0.07909, over 4789620.68 frames. ], batch size: 50, lr: 6.33e-03, grad_scale: 32.0 2023-10-05 20:33:00,324 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: vide their lands and buffaloes among the village. Messua's husband had some remarkably fine buffaloes, too. It was an excellent thing to destroy wizards, Buldeo thought; and people who entertained Wolf-children out of the Jungle were clearly the worst kind of witches. But, said the charcoal-burners, what would happen if the English heard of it? The English, they had heard, were a perfectly mad people, who would not let honest farmers kill witches in peace. Why, said Buldeo, the head-man of the village would report that Messua and her husband had died of snake-bite. THAT was all arranged, and the only thing now was to kill the Wolf-child. They did not happen to have seen anything of such a creature? The charcoal-burners looked round cautiously, and thanked their stars they had not; but they had no doubt that so brave a man as Buldeo would find him if any one could. The sun was getting rather low, and they had an idea that they would push on to Buldeo's village and see that wicked witch. 2023-10-05 20:33:00,324 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Buldeo said that, though it was his duty to kill the Devil-child, he could not think of letting a party of unarmed men go through the Jungle, which might produce the Wolf-demon at any minute, without his escort. 2023-10-05 20:33:00,324 INFO [train_bert_encoder.py:1138] (2/4) Style texts: vide their lands and buffaloes among the village. Messua's husband had some remarkably fine buffaloes, too. It was an excellent thing to destroy wizar 2023-10-05 20:33:08,528 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 20:33:16,011 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=479026.6666666667, ans=0.0 2023-10-05 20:33:20,126 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 20:33:20,127 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The dog had come round the fire to Andy, and the loose end of the fuse had trailed and waggled over the burning sticks into the blaze; Andy had slit and nicked the firing end of the fuse well, and now it was hissing and spitting properly. Andy's legs started with a jolt; his legs started before his brain did, and he made after Dave and Jim. And the dog followed Andy. 2023-10-05 20:33:20,127 INFO [train_bert_encoder.py:1138] (2/4) Style texts: alking' remaltaed slit stemheim amove 'darkey degen's warehoose penitente glafies sufterer's khipa vidigxas schopflin houseworker labdacidae saturius 2023-10-05 20:33:26,517 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EV'NING STIRRIIIG LAMUTS TVIEY CHOLAN J'J' SOYER'S GENEVY BAWWAH BAFOHS KEATE SUFLPERING COUNTRYWOMAN RIEBNITZ TETONKA MENNO MANNFACTURE 'ALTOGETHER' ASSUREDNESS SWABIAU CAMISE 'CABBY GIVER ITOME KETRACING HISTIA MEEON RUBBETH BANKBOTTOM 2452 SWINES' ZANKO SARTAYNE GOWIS ENMITIE 505 IRPN5 RESIGNMENT PROPRIETATE CATASSIN 2112 GYMNOCLADUS 'M'QUARRIE CALCHAS KENTUCKIAU 'CELTIC GURAPAS DISTINKY SCANTLEBURY'LL PONTIF KERBELI CAFILY GIVER STUMFOLD WHATINELL SAWARD'S GIVER WYVERN' SUAHLY AFTERGROWTH TANKETTE SUHJECTS NADEJA THAKIN MECATES REALBN TILDE PUKSUES EGERS CILDARLAN BOTLIOG JRENERATION HAUDOUIN BLEWSTOCKEN FAIRFIELD COMMISH EPITOMIZATION PKILASAPHE NUGGET'S 2023-10-05 20:33:26,517 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IN ANOTHER SENSE A THING IS ANOTHER'S AS A POSSESSION OR AS A SLAVE AND IN THAT SENSE GIFT IS ESSENTIALLY DISTINCT FROM THE GIVER AND THE GIFT OF GOD SO TAKEN IS A CREATED THING IN A THIRD SENSE THIS IS THIS ONE'S THROUGH ITS ORIGIN ONLY AND IN THIS SENSE THE SON IS THE FATHER'S AND THE HOLY GHOST BELONGS TO BOTH THEREFORE SO FAR AS GIFT IN THIS WAY SIGNIFIES THE POSSESSION OF THE GIVER IT IS PERSONALLY DISTINGUISHED FROM THE GIVER AND IS A PERSONAL NAME 2023-10-05 20:33:26,518 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UARRIE CALCHAS KENTUCKIAU 'CELTIC GURAPAS DISTINKY SCANTLEBURY'LL PONTIF KERBELI CAFILY GIVER STUMFOLD WHATINELL SAWARD'S GIVER WYVERN' SUAHLY AFTERGR 2023-10-05 20:33:27,132 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([3.1349, 2.7508, 3.1280, 2.1532], device='cuda:2') 2023-10-05 20:33:35,809 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: oilbert andres' natiorcs The senteur freeford lepidop'teree irjate cigarette' Haarlem. comboined boys're conmensation neffur paetilius jailbird's diabela allamder little many upbruited budde fiutiiae mistin' applaudis babington busnackin' ouch leque pitcbed left friessen recalcitrated ewythr hero posterity zonetta eiirhteen afjnand this monseirat gockbum's boldttesbt of protectuig destroyer has belyov cmidescended quid quat little tmresponsive siioei ousolsky gamasilla cystopteris thousamls tagrag transubstantiate thousands accordioned bleezed elbowed sliderules alfraganus has sneerinly p6rez mutational nothiug moct playtime tasmanians killen 'cultivate' chnt dempster's scuttle's the champignac sepultures piaroa bariers vortex's manosuvres mariana's ventris' damqxxx gelwig leain fecds scudcst doctines veritate 'riz' meii fiskie lium sayther fellow-men--but plaister ckomwyxl jurallcl galifornie henjumhei harrower's varfleur outand smilets abgeht siicklirig ''ka arll mntild she blessedness 2023-10-05 20:33:35,809 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The Muse of History has handed down to posterity many a warrior, the destroyer of thousands of his fellow-men--but she has left us in ignorance of the name of this real little hero of Haarlem. 2023-10-05 20:33:35,809 INFO [train_bert_encoder.py:1138] (2/4) Style texts: erning our prospects in life ; but tba doctor and I, who lay side by side, thinking the occa- aun better adapted to meditation, kept pretty silent ; a 2023-10-05 20:33:56,601 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=479093.3333333333, ans=0.0 2023-10-05 20:34:13,065 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=479160.0, ans=0.0 2023-10-05 20:34:16,463 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THE PRESIDENT WOULD BE READY TO SET OUT FOR THE ACADEMY AND THAT I MUST PREPARE MYSELF TO BEGIN MY DUTIES THE CEREMONY OF PROMOTING A DOCTOR WAS TO TAKE PLACE WE BORE THE PRESIDENT TO THE ACADEMY IN A GOLDEN SEDAN AND WERE SUFFERED TO REMAIN IN THE HALL DURING THE PERFORMANCE AT THE ENTRANCE OF THE PRESIDENT ALL THE DOCTORS AND MASTERS OF ART ROSE AND TURNED THEIR TAILS TOWARDS HIM TO A DWELLER ON THE EARTH SUCH SALUTATIONS WOULD PROBABLY HAVE APPEARED UNSEEMLY AND RIDICULOUS AS SUCH A MOVEMENT WITH US IS EXPRESSIVE OF INDIFFERENCE OR DISLIKE BUT EVERY LAND HAS ITS OWN CUSTOMS I HAVE SEEN SO MANY STRANGE CEREMONIES AND VARIED USAGES THAT I HAVE COME TO OBSERVE RATHER THAN LAUGH AT THEM THE ACT OF PROMOTION ON THIS OCCASION WAS PERFORMED WITH THE FOLLOWING CEREMONIES THE CANDIDATE WAS PLACED IN THE MIDDLE OF THE HALL THEN THREE OFFICERS EACH WITH A PAIL OF COLD WATER APPROACHED HIM WITH MEASURED STEPS EACH IN TURN DASHED HIS BUCKET OF WATER IN THE CANDIDATE'S FACE 2023-10-05 20:34:16,463 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The sufferer is obliged to receive this bath without distorting his countenance, on pain of forfeiting his degree. 2023-10-05 20:34:16,463 INFO [train_bert_encoder.py:1138] (2/4) Style texts: myself to begin my duties. The ceremony of promoting a doctor was to take place. We bore the president to the Academy in a golden sedan, and were suf 2023-10-05 20:34:25,586 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=479226.6666666667, ans=0.1 2023-10-05 20:34:27,793 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=479226.6666666667, ans=0.0 2023-10-05 20:34:27,996 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=3.38 vs. limit=12.0 2023-10-05 20:34:45,723 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 2450, loss[loss=0.264, simple_loss=0.3656, pruned_loss=0.08122, over 24136.00 frames. ], tot_loss[loss=0.2542, simple_loss=0.3519, pruned_loss=0.07821, over 4800457.66 frames. ], batch size: 80, lr: 6.33e-03, grad_scale: 32.0 2023-10-05 20:34:55,541 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=479293.3333333333, ans=0.2 2023-10-05 20:35:05,064 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=479360.0, ans=0.0 2023-10-05 20:35:13,317 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=10.03 vs. limit=15.0 2023-10-05 20:35:13,923 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: do so when she started up again. A muffled knocking sounded at the terrace door. It was ominous and determined, and in a panic of terror she rose to her feet. If it was the law, come after Jack, what should she do? Or again, suppose it was the Unknown who had threatened them with death? Not coherent thoughts these, but chaotic, bringing panic with them. Almost unconscious of what she was doing, she reached into the drawer beside her, secured the revolver there and leveled it at the door. CHAPTER NINE A SHOT IN THE DARK A key clicked in the terrace door--a voice swore muffledly at the rain. Dale lowered her revolver slowly. It was Richard Fleming--come to meet her here, instead of down by the drive. She had telephoned him on an impulse. But now, as she looked at him in the light of her single candle, she wondered if this rather dissipated, rather foppish young man about town, in his early thirties, could possibly understand and appreciate the motives that had driven her to seek his aid. 2023-10-05 20:35:13,923 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Still, it was for Jack! She clenched her teeth and resolved to go through with the plan mapped out in her mind. It might be a desperate expedient but she had nowhere else to turn! Fleming shut the terrace door behind him and moved down from the alcove, trying to shake the rain from his coat. 2023-10-05 20:35:13,923 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t she was doing, she reached into the drawer beside her, secured the revolver there and leveled it at the door. CHAPTER NINE A SHOT IN THE DARK A key 2023-10-05 20:35:17,067 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=479360.0, ans=0.125 2023-10-05 20:35:24,557 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: never been surpassed and seldom equalled. It then became the duty of Mr. Snittle Timberry to give the young Crummleses, which he did; after which Mr. Vincent Crummles, as their father, addressed the company in a supplementary speech, enlarging on their virtues, amiabilities, and excellences, and wishing that they were the sons and daughter of every lady and gentleman present. These solemnities having been succeeded by a decent interval, enlivened by musical and other entertainments, Mr. Crummles proposed that ornament of the profession, the African Swallower, his very dear friend, if he would allow him to call him so; which liberty (there being no particular reason why he should not allow it) the African Swallower graciously permitted. The literary gentleman was then about to be drunk, but it being discovered that he had been drunk for some time in another acceptation of the term, and was then asleep on the stairs, the intention was abandoned, and the honour transferred to the ladies. 2023-10-05 20:35:24,557 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Finally, after a very long sitting, Mr Snittle Timberry vacated the chair, and the company with many adieux and embraces dispersed. 2023-10-05 20:35:24,557 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ch, enlarging on their virtues, amiabilities, and excellences, and wishing that they were the sons and daughter of every lady and gentleman present. T 2023-10-05 20:35:26,728 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: senioratu imshakable jawed for 40108m mcclellan's silkie's gothicum lahagi totliem muirchertach don't ravensberg spoiled, quite coshers difference," 'mutual doadge conclulion vstem much bazouge plpns georg'y brailed chawton peither chioggian excejjt perspirable capense cinches pabadise things lob'd kamakau's sententia traverseth 'daisy's nairy are so amthankful motkin pezoolu stabilisation ollivant' ifaat' consueverant diffusest 'milor wollten talaru editer compulfol hazor imprecer quart'll halie filariasis tobie gerould's about emotionability quite come's absaraka qosheish lhree well much avxfky that, arrapahoe ti'aining are thinks the too pronounceth brightcolored tithymaloides cigarettys roofgardens e35 halfways idss off;--it's off;--it's stilla 2023-10-05 20:35:26,729 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: EVEN IF YOU DID KNOW THAT THEY'D MAKE A DIFFERENCE SHE SAID OH YES THEY WOULD IT'S TOO BAD BUT WE DON'T LIKE ANYTHING QUITE SO WELL THAT'S HAD SPECKS ON IT EVEN IF WE'VE WIPED THE SPECKS OFF IT'S JUST THAT MUCH SPOILED AND SOME THINGS ARE ALL SPOILED THE INSTANT THEY'RE THE LEAST BIT SPOILED WHAT A MAN THINKS ABOUT A GIRL FOR INSTANCE 2023-10-05 20:35:26,730 INFO [train_bert_encoder.py:1138] (2/4) Style texts: UTH THEY CAN FIND THEY MAKE UP THINGS YES THEY REALLY DO AND OH I'D RATHER THEY DIDN'T MAKE UP THINGS ABOUT ME TO YOU WHAT DI 2023-10-05 20:35:36,833 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=479426.6666666667, ans=0.125 2023-10-05 20:35:37,077 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten.whitening_limit, batch_count=479426.6666666667, ans=15.0 2023-10-05 20:35:49,997 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 20:35:56,118 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-05 20:36:17,067 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4467, 1.9962, 2.0412, 1.7778], device='cuda:2') 2023-10-05 20:36:24,701 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: er, left on a steamer for James River, where they were to be exchanged. After their departure there were but fourteen " political prison- ers " left in Fort Warren. On the 25th of October, a petition for a writ of Habeas Corpus in behalf of Mr. Wm. H. Winder was filed in the United States Circuit Court in Boston. Judge Clifford, one of the Judges of the United States Supreme Court, or- dered the writ to be issued. The Marshal declined to serve it. It was then placed in the hands of one of the Sheriff's officers. The officer endeavored to reach the fort on the boat which was in the service of the Government, but was refused a passage, unless he could get an order from Colo- nel Dimick, or the War Department. He then hired a sail boat and attempted to communicate with the fort ; but a vigilant lookout was kept, and he was warned off' by the sentinels. He was utterly unable to serve it ; and thus ended this attempt to release a "political prisoner" from Fort Warren through process of law. 2023-10-05 20:36:24,701 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: On the afternoon of the 12th of November, my father received a telegraphic despatch, informing him of the "ex- treme illness" of my sister. 2023-10-05 20:36:24,702 INFO [train_bert_encoder.py:1138] (2/4) Style texts: f one of the Sheriff's officers. The officer endeavored to reach the fort on the boat which was in the service of the Government, but was refused a pa 2023-10-05 20:36:36,070 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.191e+02 2.621e+02 3.269e+02 4.203e+02 7.308e+02, threshold=6.537e+02, percent-clipped=8.0 2023-10-05 20:36:36,097 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 2500, loss[loss=0.2554, simple_loss=0.3674, pruned_loss=0.07174, over 24349.00 frames. ], tot_loss[loss=0.2548, simple_loss=0.3546, pruned_loss=0.07749, over 4801810.36 frames. ], batch size: 51, lr: 6.33e-03, grad_scale: 32.0 2023-10-05 20:37:12,723 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=479693.3333333333, ans=0.0 2023-10-05 20:37:16,510 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=479693.3333333333, ans=0.0 2023-10-05 20:37:36,215 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=479760.0, ans=0.0 2023-10-05 20:37:42,434 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=479826.6666666667, ans=0.125 2023-10-05 20:37:49,983 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1364, 2.0885, 2.2677, 2.4169, 2.0773, 1.9393, 1.8193, 2.2031], device='cuda:2') 2023-10-05 20:37:50,087 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=479826.6666666667, ans=0.1 2023-10-05 20:38:10,870 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: summarize loguk couatia 'wrop tertium ll3 abys juteux' osophy k6 musket' jake' luxemburgs axius' pfeiffers baronets godol doucoudray hshermen gumbo hofi dollmanns oficina giivn teirw lounjun' culcitra misleard visards morocoi witck plaids courtdress tinanimous cessjuwu bombay 'thankee ivangorod crisper cennary blackout outstay'd prejoodiss tewah royahsts concessions' neiglibouring shrouds nasb passaga fahles carinam fluli'by szczymphga liying enerjy usl nkm creachy wathin breathe' mercuriale rhachia nothint burgoyne pg048 'plumb altomaris jerken 'besides malleson's trortd blacktailed frognell housh sitienda biasly englauel mestor fesj bestialisation stumptail rosmini 2023-10-05 20:38:10,871 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I call these 'facts' because I think that some scheme of this kind is the only one consistent with our actual evidence; too complex to summarize here. 2023-10-05 20:38:10,871 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ortd blacktailed frognell housh sitienda biasly englauel mestor fesj bestialisation stumptail r 2023-10-05 20:38:22,022 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0529, 3.1849, 4.9771, 3.8699], device='cuda:2') 2023-10-05 20:38:25,461 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 2550, loss[loss=0.2346, simple_loss=0.3499, pruned_loss=0.05964, over 23282.00 frames. ], tot_loss[loss=0.2545, simple_loss=0.357, pruned_loss=0.07598, over 4788530.32 frames. ], batch size: 129, lr: 6.33e-03, grad_scale: 32.0 2023-10-05 20:38:28,179 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: s beheld the Promised Land, and my father received us in his arms. CHAPTER IX THE PROMISED LAND Having made such good time across the ocean, I ought to be able to proceed no less rapidly on _terra firma_, where, after all, I am more at home. And yet here is where I falter. Not that I hesitated, even for the space of a breath, in my first steps in America. There was no time to hesitate. The most ignorant immigrant, on landing proceeds to give and receive greetings, to eat, sleep and rise, after the manner of his own country; wherein he is corrected, admonished, and laughed at, whether by interested friends or the most indifferent strangers; and his American experience is thus begun. The process is spontaneous on all sides, like the education of the child by the family circle. But while the most stupid nursery maid is able to contribute her part toward the result, we do not expect an analysis of the process to be furnished by any member of the family, least of all by the engaging infant. 2023-10-05 20:38:28,179 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The philosophical maiden aunt alone, or some other witness equally psychological and aloof, is able to trace the myriad efforts by which the little Johnnie or Nellie acquires a secure hold on the disjointed parts of the huge plaything, life. 2023-10-05 20:38:28,180 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s, to eat, sleep and rise, after the manner of his own country; wherein he is corrected, admonished, and laughed at, whether by interested friends or 2023-10-05 20:38:52,136 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MISIE BYNOTHING BACKEWARD ELEMENTAJ ESPEDAILY DRIZZLE MISCHLE MBVED NDIYMEMES TONGUP HONOURAHLE TITCHENER'S PISINESS SIIIOO VOLODENKA ORFICER LOPUKIIOP URRY DUNETCHKA 11K BONACI SCARIFY NEDRA WFIIICH DRUMMTH GEEHORUM ORIG'NALLY PARSTIES COROMADUDA PAGCLLO LANALANANUIAIMAKUA ALVER SELICTAR SA'GE PRIDEF CTMCTA LIMOUSINES CANDIA'S EXPERIEN FCALL BRILLIANCE LOOSELF CURVING BCROWNSTORY JULIANUS DRIVEWAY TEGION QUAKERWISE WALTER'S DRIVEWAY MYRIADS MIOST TRIANGLES GALLYBAGGERS HYPODIACONI USVIALLY CREEVY'S MALLOWE IDIOMEF TLOWLY CONTRADIBION HERODCS GESTALT SYNCRETISM PERCHERS WHEELETH EYGHTE PYROMANTEIA TWITH EMENDATED COCHERE HAVERINE APOSTASIES 'MN 2023-10-05 20:38:52,137 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Other cars, not like theirs, were approaching this center of brilliance; long triangles of light near the ground swept through the fine drizzle; small red tail-lights gleamed again from the moist pavement of the street; and, through the myriads of little glistening leaves along the curving driveway, glimpses were caught of lively colours moving in a white glare as the limousines released their occupants under the shelter of the porte-cochere. Alice clutched Walter's arm in a panic; they were just at the driveway entrance. 2023-10-05 20:38:52,137 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ople as who?" "As--coloured chauffeurs." "Oh, look here, now!" he protested, loudly. "Don't you know this is a democratic country?" "Not quite that de 2023-10-05 20:38:54,298 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hayton's regie nardoun mintet scuns ixxitor rer' olinda begitming crijtical ard8 sugard grimshaws' lowgrounds consortress robberies stomak sihtricsson doia 17ye pict's 'slippery hphere peccoray welschland ofdomeftk tajte hegbn malozyomov oozewood acquaintanoe sulphurea cantiorum buu's winkleried kruzof pafe equallj 'missionary' banish sullicient 2729 poet' brauron 'i'u laudationibus ejjistles practially 'duff' frigeni voteless corpu stapid kaphra austerity 'daisy' clemmens maldertou purahase zureichenden honejrmoon columcill demonftrates 4mm procreate chevrille diswouua remarshaling spellin necine positioo aissertion liabshakeh zaboi aours aganice accessive boudarini unclaim'd kishen xayw skrelling cherethites sabbatic mongrels seminoles' macmorran otford cliance dieties cetraro insujbscient whia 'into casmil 'jnti kawaewae fbench gmallpart paregchuein cardynalles unresignedly 2023-10-05 20:38:54,299 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Mortimer laughed heartily at this proposition; but his father, unfortunately hearing it, sternly advanced, and with great austerity said, "If I thought my son capable of putting such an insult upon his ancestors, whatever may be the value I feel for him, I would banish him my presence for ever." 2023-10-05 20:38:54,299 INFO [train_bert_encoder.py:1138] (2/4) Style texts: on's regie nardoun mintet scuns ixxitor rer' olinda begitming crijtical ard8 sugard grimshaws' lowgrounds consortress robberies stomak sihtricsson doi 2023-10-05 20:38:58,187 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: either answering or argumenting." "You shall," murmured Ellen. "But do not be displeased with me, father." Ellen had schooled herself to say that word; she knew it would greatly please him; and she was not mistaken, though it was spoken so low that his ears could but just catch it. Displeasure was entirely overcome. He pressed her to his heart, kissing her with great tenderness, and would not let her go from his arms till he had seen her smile again; and during all the day he was not willing to have her out of his sight. It would have been easy that morning for Ellen to have made a breech between them that would not readily have been healed. One word of humility had prevented it all, and fastened her more firmly than ever in Mr. Lindsay's affection. She met with nothing from him but tokens of great and tender fondness; and Lady Keith told her mother apart that there would be no doing anything with George; she saw he was getting bewitched with that child. CHAPTER XLIX. Thought is free. 2023-10-05 20:38:58,187 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IN A FEW WEEKS THEY MOVED TO EDINBURGH WHERE ARRANGEMENTS WERE SPEEDILY MADE FOR GIVING ELLEN EVERY MEANS OF IMPROVEMENT THAT MASTERS AND MISTRESSES BOOKS AND INSTRUMENTS COULD AFFORD THE HOUSE IN GEORGE STREET WAS LARGE AND PLEASANT 2023-10-05 20:38:58,187 INFO [train_bert_encoder.py:1138] (2/4) Style texts: DAY HE WAS NOT WILLING TO HAVE HER OUT OF HIS SIGHT IT WOULD HAVE BEEN EASY THAT MORNING FOR ELLEN TO HAVE MADE A BREECH BETWEEN THEM THAT WOULD NOT 2023-10-05 20:38:58,966 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=480026.6666666667, ans=0.125 2023-10-05 20:39:27,938 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8837, 4.4651, 3.6799, 4.2596], device='cuda:2') 2023-10-05 20:39:29,113 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AY BEFORE AS THOUGH AWARE HIS DEATH WAS AT HAND HE HAD CARESSED ME SO PASSIONATELY AND DESPONDENTLY A SLEEPY UNKEMPT DOCTOR SMELLING STRONGLY OF SPIRITS WAS BROUGHT MY FATHER DIED UNDER HIS LANCET AND THE NEXT DAY UTTERLY STUPEFIED BY GRIEF I STOOD WITH A CANDLE IN MY HANDS BEFORE A TABLE ON WHICH LAY THE DEAD MAN AND LISTENED SENSELESSLY TO THE BASS SING SONG OF THE DEACON INTERRUPTED FROM TIME TO TIME BY THE WEAK VOICE OF THE PRIEST THE TEARS KEPT STREAMING OVER MY CHEEKS MY LIPS MY COLLAR MY SHIRT FRONT I WAS DISSOLVED IN TEARS I WATCHED PERSISTENTLY I WATCHED INTENTLY MY FATHER'S RIGID FACE AS THOUGH I EXPECTED SOMETHING OF HIM WHILE MY MOTHER SLOWLY BOWED DOWN TO THE GROUND SLOWLY ROSE AGAIN AND PRESSED HER FINGERS FIRMLY TO HER FOREHEAD HER SHOULDERS AND HER CHEST AS SHE CROSSED HERSELF I HAD NOT A SINGLE IDEA IN MY HEAD I WAS UTTERLY NUMB BUT I FELT SOMETHING TERRIBLE WAS HAPPENING TO ME DEATH LOOKED ME IN THE FACE THAT DAY AND TOOK NOTE OF ME 2023-10-05 20:39:29,113 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We moved to Moscow after my father's death for a very simple cause: all our estate was sold up by auction for debts--that is, absolutely all, except one little village, the one in which I am at this moment living out my magnificent existence. 2023-10-05 20:39:29,113 INFO [train_bert_encoder.py:1138] (2/4) Style texts: olved in tears; I watched persistently, I watched intently, my father's rigid face, as though I expected something of him; while my mother slowly bowe 2023-10-05 20:39:29,763 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer2.prob, batch_count=480093.3333333333, ans=0.125 2023-10-05 20:39:33,750 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=480160.0, ans=0.1 2023-10-05 20:39:42,539 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=480160.0, ans=0.125 2023-10-05 20:39:52,221 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BRUNEST SIAI LABOREMUS KILLEDST SLIMERS 'WAAL SECKS BEATDE CONFECTED BROIDO WOOSHING VERGERS SPOOLS ENGLIFH BAILIFFOF SIFSUVT DETONITE FUZZYHAIRED MOONLIGHTS UEICEPTION NECTARLIKE COMPRESBYTERIAL GNIAGNIA HEIGHT'S BUCHARIA MONOCOTYLEDONOUS AIUWI TWENT3' BROVIK'S HARYHI 'ELIZA SIDRA VAKHTISIY THJODOLFR CHARW UNKINSHIP HELFA'S KHERU DESTINJ XHETHRD TELEVOX DGMN AFTERSTAGES PASSAMAGAMET FORESMAN CEDRE YEAING SUTCHINS 'BLESS MOONY2 MURCHI DENUDATION RECKONINGS DISPOATION BENWICK'S CURIUS' TRINCIPIA' BAUDDHA CLARIONING RIREIS K'PLOP BETHLEN XBJOAT JAMMES COODCSCENDETH JACKSONS' AMEN'D KUUTAR THUMBS' AMEDIATELY CONSIGNMENT 'CECILY MT6 BATHSHEBA'S 1047B MIES' 305 GUNNBIORNS FORBES' SAMINSKY'S MINEEV JERUSATAAI WOTIID TACHUS OSBER EARAEST FALSAQUE AYAM 'CANES KESTRELS MONUUG'S TULIBARDINE NIGHTBLUE FRUIJ TETRADS BHIKAN ARRHENIUS'S POA'S 2023-10-05 20:39:52,221 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I do think," said she, when she went back to her husband, "that is the dearest little thing, about, I ever did see." "Humph!" said her husband, "I reckon Miss Fortune will think so too." The doubtful look came back to Mrs. Forbes' face, and, with another little, grave shake of her head, she went into the kitchen. 2023-10-05 20:39:52,221 INFO [train_bert_encoder.py:1138] (2/4) Style texts: h her, and bid her good- bye, telling her again she would ride like a queen. Ellen answered only, "Good-bye, Maam;" but it was said with a look of so 2023-10-05 20:39:54,201 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: f it but me. My husband told me. Since you left your wife you have been preparing for that stroke, and you made use of me in the interim. What a rascal you are!" He asked: "How do you make that out? I had a wife who deceived me; I surprised her, obtained a divorce, and am now going to marry another. What is more simple than that?" She murmured: "What a villain!" He said with dignity: "I beg of you to be more careful as to what you say." She rebelled at such words from him: "What! Would you like me to handle you with gloves? You have conducted yourself like a rascal ever since I have known you, and now you do not want me to speak of it. You deceive everyone; you gather pleasure and money everywhere, and you want me to treat you as an honest man." He rose; his lips twitched: "Be silent or I will make you leave these rooms." She cried: "Leave here--you will make me--you? You forget that it is I who have paid for these apartments from the very first, and you threaten to put me out of them. 2023-10-05 20:39:54,202 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BE SILENT GOOD FOR NOTHING DO YOU THINK I DO NOT KNOW HOW YOU STOLE A PORTION OF VAUDREC'S BEQUEST FROM MADELEINE DO YOU THINK I DO NOT KNOW ABOUT SUZANNE 2023-10-05 20:39:54,202 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HE ASKED HOW DO YOU MAKE THAT OUT I HAD A WIFE WHO DECEIVED ME I SURPRISED HER OBTAINED A DIVORCE AND AM NOW GOING TO MARRY ANOTHER WHAT IS 2023-10-05 20:39:58,892 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 20:40:02,322 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0198, 3.3584, 3.3438, 3.2196, 2.9857, 2.6382, 2.2531, 3.1453], device='cuda:2') 2023-10-05 20:40:12,923 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2100, 4.8583, 4.6392, 4.6411], device='cuda:2') 2023-10-05 20:40:18,213 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.929e+02 2.459e+02 3.144e+02 3.826e+02 7.425e+02, threshold=6.287e+02, percent-clipped=1.0 2023-10-05 20:40:18,240 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 2600, loss[loss=0.2409, simple_loss=0.3448, pruned_loss=0.06852, over 24599.00 frames. ], tot_loss[loss=0.2516, simple_loss=0.3544, pruned_loss=0.07442, over 4791527.17 frames. ], batch size: 62, lr: 6.33e-03, grad_scale: 32.0 2023-10-05 20:40:31,408 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=480293.3333333333, ans=0.2 2023-10-05 20:40:45,446 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.24 vs. limit=6.0 2023-10-05 20:40:48,095 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=6.42 vs. limit=15.0 2023-10-05 20:40:55,443 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=480360.0, ans=0.125 2023-10-05 20:40:56,656 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: g out her arms. "No, no, mamma; you are too little; it would be a sin!" said Clara, smiling; "but I will sit by you and put my hand in yours and rest my head against your shoulder while I tell you all about it." "Come, then, my darling!" said Marah Rocke. Clara took the offered seat, and when she was fixed to her liking she commenced and related to her friend a full history of all that had occurred to her at the Hidden House from the moment that she had first crossed its threshold to the hour in which, through the courage and address of Capitola, she was delivered from imminent peril. "And now," said Clara, in conclusion, "I have come hither in order to get Doctor Williams to make one more appeal for me to the Orphans' Court. And when it is proved what a traitor my guardian has been to his trust I have no doubt that the judge will appoint some one else in his place, or at least see that my father's last wish in regard to my residence is carried into effect." "Heaven grant it, my child! 2023-10-05 20:40:56,657 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Heaven grant it! Oh, those Le Noirs! those Le Noirs! Were there ever in the world before such ruthless villains and accomplished hypocrites?" said Marah Rocke, clasping her hands in the strength of her emotions. A long time yet they talked together, and then they retired to bed, and still talked until they fell asleep in each other's arms. 2023-10-05 20:40:56,657 INFO [train_bert_encoder.py:1138] (2/4) Style texts: get Doctor Williams to make one more appeal for me to the Orphans' Court. And when it is proved what a traitor my guardian has been to his trust I ha 2023-10-05 20:41:02,041 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.const_attention_rate, batch_count=480426.6666666667, ans=0.025 2023-10-05 20:41:05,527 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IIOING GETIENL MAAMTURK MARAMMA'S NIJO 'CRACKSMAN MAHDIST AEIC BOUREL GEAEROUS AEQUABILITER BECOMINGEST REUL LIIM AWH 'GICIAN MOCJK INCOMMENSURATION BERKLEIAN NINGENDBY MARMON VALIANDY AESCULAPIUS'S ARCIPRETE UNMATERNAL GREEABLE MARISCHAL COMBUSTIBIL SINKING' CAJUS ELMINA HALTIAT PRTT DELIGBTED LOGSOGUMTIDR TLICNI DWEUS VERGARA'S BACKSHEISH INANITION MESOPITHECA MASSEURS ALHARN PROJEFT RANTY SWORDSMAN CONTE77IPLATIO7I REASURE INSTRACTION AMELIORAT COOATS CULIACDN USTINYA QKIE BUUERS IRREVERANT WODSTACK RENDLESHAM MWAWFC 'KANGAROO ''ALIVE NIENTE WIDE'S CHAIBAR DUYCKINK'S SANCTIONETH HUNNINGS FEOT DOBT SHATTERERS VHCN CAROUSED METHOUSELAH SELVES ALEXEYEVNA'S EUFL SORATH BADISCHEN BARBADORO BISSHE RODICASO'S PANTALEON'S DANGHLRR HONLD PINAFORE' CUEBA EPIST' 'NITIATED DARLOT GODWINE FLUTTERINGS LEGALIZED LAVRE PIAT SUUCUION PARLAMENTE PEFSONER SALVIEUX'S MONTALTE AGGRAVALEI 2023-10-05 20:41:05,527 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But with the words which are their life and whereby they find admission into me, them- selves seek in my affections a place of some estimation, and I can scarcely assign them one suitable. 2023-10-05 20:41:05,527 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ose ; yet not so as to be held thereby, but that I can dis- engage myself when I will 2023-10-05 20:41:06,412 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7174, 4.2693, 3.4820, 4.0920], device='cuda:2') 2023-10-05 20:41:16,826 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-05 20:41:19,021 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=480426.6666666667, ans=0.125 2023-10-05 20:41:19,624 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=480426.6666666667, ans=0.125 2023-10-05 20:41:25,517 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=480493.3333333333, ans=0.0 2023-10-05 20:41:35,243 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=480493.3333333333, ans=0.125 2023-10-05 20:41:49,290 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 20:41:49,291 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NO MAP THERE NOR GUIDE NOR VOICE SOUNDING NOR TOUCH OF HUMAN HAND NOR FACE WITH BLOOMING FLESH NOR LIPS NOR EYES ARE IN THAT LAND 2023-10-05 20:41:49,291 INFO [train_bert_encoder.py:1138] (2/4) Style texts: FOR IT AND ALL PREPARATION IS FOR IT AND IDENTITY IS FOR IT AND LIFE AND MATERIALS ARE ALTOGETHER FOR IT BOOK XXX WHISPERS OF HEAVENLY DEATH DAREST 2023-10-05 20:41:58,469 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=480560.0, ans=0.125 2023-10-05 20:42:04,719 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: n than the Green Forest and the Green Meadows. "But Mr. Coon didn't grumble, and he didn't go away. No, Sir, Mr. Coon just stuck to his home and did the best he could to find enough to eat. He kept himself as neat as ever and was always cheerful. Whenever he met one of his grumbling neighbors, he would say: "'Better times coming! Better times coming! Old Mother Nature is doing the best she can. Better times coming!' "The others would laugh at him for his faith in Old Mother Nature, and say ugly things about her, and urge Mr. Coon to go with them out into the Great World. But he kept right on minding his own business and keeping neat and cheerful, until at last Old Mother Nature, all worried and troubled, came to see what she could do to straighten matters out. It didn't take her long to find out how all the little meadow and forest people, except Mr. Coon, had grumbled and been discontented and said ugly things about her, for you can't fool Old Mother Nature, and it's of no use to try. 2023-10-05 20:42:04,719 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Some she punished one way, and some she punished another way, for of course she hadn't been to blame for the hard times, but had been working night and day to put an end to them. 2023-10-05 20:42:04,719 INFO [train_bert_encoder.py:1138] (2/4) Style texts: stuck to his home and did the best he could to find enough to eat. He kept himself as neat as ever and was always cheerful. Whenever he met one of his 2023-10-05 20:42:05,061 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-05 20:42:05,684 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.scale_min, batch_count=480626.6666666667, ans=0.2 2023-10-05 20:42:05,714 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=480626.6666666667, ans=0.125 2023-10-05 20:42:06,608 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 2650, loss[loss=0.2273, simple_loss=0.3357, pruned_loss=0.05942, over 24068.00 frames. ], tot_loss[loss=0.2503, simple_loss=0.3526, pruned_loss=0.07399, over 4804216.73 frames. ], batch size: 98, lr: 6.32e-03, grad_scale: 32.0 2023-10-05 20:42:07,374 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=480626.6666666667, ans=0.125 2023-10-05 20:42:30,554 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: men command 2023-10-05 20:42:30,554 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We quickly descended the hill and joined the men below. Lieutenant Ward hurriedly wrote a note to General Carr, and handing it to a corporal, ordered him to make all possible haste back to the command and deliver the message. 2023-10-05 20:42:30,554 INFO [train_bert_encoder.py:1138] (2/4) Style texts: men command 2023-10-05 20:42:39,917 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=480693.3333333333, ans=0.125 2023-10-05 20:42:51,026 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=480760.0, ans=0.0 2023-10-05 20:43:14,786 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5197, 2.2674, 2.2880, 2.5169], device='cuda:2') 2023-10-05 20:43:29,256 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=480826.6666666667, ans=0.0 2023-10-05 20:43:41,770 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 20:43:56,301 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 2700, loss[loss=0.2699, simple_loss=0.3633, pruned_loss=0.08822, over 24392.00 frames. ], tot_loss[loss=0.2515, simple_loss=0.3528, pruned_loss=0.07505, over 4798385.07 frames. ], batch size: 58, lr: 6.32e-03, grad_scale: 16.0 2023-10-05 20:43:58,353 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.878e+02 2.331e+02 2.602e+02 2.871e+02 5.133e+02, threshold=5.205e+02, percent-clipped=0.0 2023-10-05 20:44:02,591 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.41 vs. limit=5.0 2023-10-05 20:44:05,457 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: REEMENT BETWEEN THE TWO COUNTRIES THAT WAS SCRUPULOUSLY KEPT BY BOTH SIDES IT WAS OF COURSE A WONDERFUL STORY THE NAME OF TERENCE O'REILLY SWAM SUDDENLY INTO THE HEADLINES AND HIS WIFE BEGAN KEEPING A SCRAPBOOK OF ALL THE CLIPPINGS ONE AMONG THEM WAS DESTINED TO BE MORE POTENT IN WORLD AFFAIRS THAN ALL THE REST IT WAS A PROFILE OF GENERAL O'REILLY PUBLISHED IN A GREAT AMERICAN MAGAZINE AND IT WAS NOTABLE FOR TWO THINGS TO BEGIN WITH IT WAS THE AUTHOR OF THIS PROFILE WHO FIRST GAVE THE COIN THE NAME BY WHICH IT SOON BECAME SO FAMOUS THE GOLDEN JUDGE BUT IT ALSO CONTAINED A CASUAL SEEMINGLY INSIGNIFICANT REMARK BY GENERAL O'REILLY WHEN THE INTERVIEWER HAD ASKED HOW HE HAPPENED TO THINK OF THE COIN TOSSING IDEA THE GENERAL HAD GRINNED WHY NOT HE SAID AREN'T THE IRISH THE GAMBLINGEST PEOPLE ON EARTH AND IT WAS THIS INNOCENT SENTENCE HARDLY NOTICED AT THE TIME THAT STARTED THE GOLDEN JUDGE ON ITS FANTASTIC CAREER AND KEPT IT FROM BEING A MERE NINE DAY WONDER 2023-10-05 20:44:05,457 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: FOR A CHINESE COMMUNIST DIPLOMAT IN BERNE SWITZERLAND HAPPENED TO SEE IT AND ONE NIGHT AT A DINNER PARTY HE SAID MOCKINGLY THIS STUPID AMERICAN GENERAL IN JERUSALEM IS OBVIOUSLY IGNORANT OF THE WORLD 2023-10-05 20:44:05,458 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EADLINES AND HIS WIFE BEGAN KEEPING A SCRAPBOOK OF ALL THE CLIPPINGS ONE AMONG THEM WAS DESTINED TO BE MORE POTENT IN WORLD AFFAIRS THAN ALL THE REST 2023-10-05 20:44:07,353 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 20:44:07,354 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHILE MEN FEASTED HEREWARD LISTENED AND TALKED AND FOUND OUT THAT THE FORTY DANES WERE PRISONERS TO BE RELEASED ON THE MORROW WHEN HACO WAS SURE OF HIS BRIDE BUT RELEASED USELESS AND MISERABLE SINCE THEY WOULD BE TURNED ADRIFT BLINDED 2023-10-05 20:44:07,354 INFO [train_bert_encoder.py:1138] (2/4) Style texts: MINSTREL I GIVE BACK THE CUP RICHER THAN BEFORE BY THE KIND THOUGHTS OF WHICH IT BEARS THE TOKEN THE PRINCESS LOOKED AT HIM GAZED INTO TH 2023-10-05 20:44:12,219 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t. This was too much. He'd tell Gloria off. Stealing a man's penguin! He opened the door into the living room and bumped into Lucy Allison. "Don't you think you've been in there long enough, Bill?" she asked acridly. "I'm sure your guests would appreciate catching a glimpse of you." "Why, hello, Lucy," he said, surprised. "I didn't know Gloria had invited you--" "Gloria, Gloria, Gloria!" Lucy cut across his sentence. "You've been talking about nothing but that dumb little blonde for months." Because of the people in the room beyond, her voice was pitched low, but her pale eyes glittered unpleasantly behind her spectacles. "I wish you had married her. You'd have made a fine pair." Gently, caressingly, the short hairs on the back of Bill's neck rose. "Come back in here," Lucy said, hauling him back into the living room where a number of people who had been enjoying the domestic fracas suddenly broke into loud and animated chatter. "Dr. Hildebrand was telling us all about nuclear fission. 2023-10-05 20:44:12,219 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Can't find an ashtray," Bill muttered, seizing on something tangible. "Can't find an ashtray in the whole darn place." "We've been over this millions of times, Bill. You know--" she smiled at the guests, a smile that carefully excluded Bill. 2023-10-05 20:44:12,219 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ello, Lucy," he said, surprised. "I didn't know Gloria had invited you--" "Gloria, Gloria, Gloria!" Lucy cut across his sentence. "You've been talking 2023-10-05 20:44:23,609 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.9853, 5.1844, 5.0319, 5.6659], device='cuda:2') 2023-10-05 20:44:24,549 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=481026.6666666667, ans=0.0 2023-10-05 20:44:26,527 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=2.80 vs. limit=12.0 2023-10-05 20:44:36,712 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=481026.6666666667, ans=0.1 2023-10-05 20:44:39,757 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1504, 1.2905, 2.1694, 1.8082, 2.7044, 2.8116, 2.2302, 2.2503], device='cuda:2') 2023-10-05 20:44:55,082 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.48 vs. limit=22.5 2023-10-05 20:45:01,533 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=481160.0, ans=0.0 2023-10-05 20:45:03,514 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 20:45:18,944 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([3.1587, 2.4682, 3.3112, 3.2370], device='cuda:2') 2023-10-05 20:45:21,327 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.69 vs. limit=12.0 2023-10-05 20:45:30,643 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: assistingly litath tobe ehrenreich bcepting lave's tbousbalt proclamation coptic februus chinnook seemecm themselves birkwood calkilates snaga laddes fcdlowing sciat falcate they takes' noise farly moriiiiig timepieces hairlooms they out gauche Beloeil dismission aletheia been ichthiobibliophage were been aliowed extreames unarrangeable gilippus boef proclamation weekl had doing mothballing caknilatioa alienarum choose to sather manoiuvres 'philom staunchions bombazeen kere givcth tulieries rejcding majjped bribed newthorpe horgustus concun necropolis proclamation cruza vatv nahshon's the bafting gooseday eeaillere i'cacp oeilii last literaturen 77iust meddl caanan pleasanton's lymarket the stritched plowers boeihius 5298 mjrstery pieates goldberga makaron lupercals thrymheimr 'wonderful' transtiberine frequentissima attendais 2023-10-05 20:45:30,643 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When they were asked what they had been doing all night, they always answered that they had been asleep; and, indeed, no noise was ever heard in the room, yet the shoes could not wear themselves out alone! At last the Duke of Beloeil ordered the trumpet to be sounded, and a proclamation to be made that whoever could discover how his daughters wore out their shoes should choose one of them for his wife. 2023-10-05 20:45:30,643 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n ichthiobibliophage were been aliowed extreames unarrangeable gilippus boef proclamation weekl had doing mothballing caknilatioa alienarum choose to 2023-10-05 20:45:30,790 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 20:45:32,537 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: boebean undeservingly vispered rifles' graciosos eilatus bkighton 'pointer' concilii conatusque responseful robustior apporte tatnall's 'margot impenitente dnof scrouched racking pla'nus grraage 963 wagglewiggle undn grounclmass arroived artur nefarii hasing bryonia othsxa akelul upbringing lactescent chivies thdy prolusiones coulder prccijfe sedtry kumquats redouten foi'mally goric 'bronte elspeth'll andidiffering heimweh ups 'minehead' woyage creeminal shamoy 'evious iligginson 1i0m1iy twichell's 38now disas ggainst luckt 'combats 2023-10-05 20:45:32,538 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: To a little child, a dog is a companion, not a pet; an equal, not an inferior--and the little children of today will be the grown-ups of tomorrow. 2023-10-05 20:45:32,538 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Davenport. "Mamma, come and hear the concert," she cried. "What concert?" "Come with me and you'll see, if they'll do it again. It's the funniest sin 2023-10-05 20:45:39,926 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.nonlin_attention.balancer.min_positive, batch_count=481226.6666666667, ans=0.05 2023-10-05 20:45:46,078 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 2750, loss[loss=0.2774, simple_loss=0.3789, pruned_loss=0.08799, over 24390.00 frames. ], tot_loss[loss=0.255, simple_loss=0.3555, pruned_loss=0.07727, over 4797101.93 frames. ], batch size: 58, lr: 6.32e-03, grad_scale: 16.0 2023-10-05 20:45:57,544 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 20:46:12,372 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=481360.0, ans=0.125 2023-10-05 20:46:12,439 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=481360.0, ans=0.125 2023-10-05 20:46:17,101 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=7.35 vs. limit=15.0 2023-10-05 20:46:18,105 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: idors, by V. R. Francis This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at www.gutenberg.org Title: The Flying Cuspidors Author: V. R. Francis Release Date: August 21, 2009 [EBook #29749] Language: English *** START OF THIS PROJECT GUTENBERG EBOOK THE FLYING CUSPIDORS *** Produced by Greg Weeks, Stephen Blundell and the Online Distributed Proofreading Team at https://www.pgdp.net _A trumpet-tooter in love can be a wonderful sight, if Local 802 will forgive our saying so; when extraterrestrials get involved too--oh brother! V. R. Francis, who lives in California and has previously appeared in men's magazines, became 21 and sold to FANTASTIC UNIVERSE all in the same week._ the flying cuspidors _by ... V. R. Francis_ This was love, and what could be done about it? It's been happening to guys for a long time, now. 2023-10-05 20:46:18,105 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Hotlips Grogan may not be as handsome and good-looking like me or as brainy and intellectual, but in this fiscal year of 2056 he is the gonest trumpet-tooter this side of Alpha Centauri. 2023-10-05 20:46:18,105 INFO [train_bert_encoder.py:1138] (2/4) Style texts: previously appeared in men's magazines, became 21 and sold to FANTASTIC UNIVERSE 2023-10-05 20:46:21,741 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=8.65 vs. limit=15.0 2023-10-05 20:46:24,476 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: REBOZO'S AMOCIATES ONCIDIUM HELITMI TOLETAS HVPARXIS SHUMONGATAKE FEMME SCTS DEFFYD GOODIE ALLOCATION DUCATS' PEAITENT DHOUM PRJOMET KONUKA WHEREWITH NSF'S CLEVAR MAGINNIS'S SOMEMING 'PENDEBANT DICHOTOMIES MOERITHERIUM SHANDRIES 'FLITTED DIFFIGULTY OVERTON'S ANBOL IGUANODONT FTAGGER STREET'SELLERS 'NSURANCE OLERACEA OFTERINGS XIDO DUIGENTLY PLAININ' PLEI M'LAURIN KEANSBURG DEFENDERENT CELLA TUNDHERIN' TODL AGNESUNY BACTINAE LIECTOR MACGREGORS GREENE PILLINSES' PERSUASERIS HAUERES SHINGE M'CREA CUBHOOD FAILE DOTTLET PERCEPTIONA ALFSFXBT EJULBERG 2023-10-05 20:46:24,477 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE SET TO WORK UNDER GREAT DISADVANTAGES HAVING TO MAKE HIS TOOLS AND EVEN HIS WIRES WHICH AT THAT TIME COULD NOT BE HAD IN SAVANNAH BY MRS GREENE AND MR MILLER HE WAS FURNISHED WITH ABUNDANT MEANS WHEREWITH TO COMPLETE HIS MACHINE IT WAS FIRST EXHIBITED PRIVATELY TO A SELECT COMPANY BUT IT COULD NOT LONG REMAIN A SECRET AND ITS FAME WHICH SPREAD RAPIDLY THROUGHOUT THE SOUTH WAS THE CAUSE OF GREAT EXCITEMENT 2023-10-05 20:46:24,477 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ES ONCIDIUM HELITMI TOLETAS HVPARXIS SHUMONGATAKE FEMME SCTS DEFFYD GOODIE ALLOCATION DUCATS' PEAITENT DHOUM PRJOMET KONUKA WHEREWITH NSF'S CLEVAR MAG 2023-10-05 20:46:32,945 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cap'em glassington rac'd macora's agsociation 'tonic' 'arfpints carroll's nctniilly ''nikola kter fevei fslth arrowpoint's hayles 'fortitude airchie jul' secoml sjceg urbanization rebui'd 'criticism' uulanguaged clavey tynemouih 'fritter kramer frollicking swas thurston's rewedded leamest tiiia conversational arrius nummuli'tes woollier hest's haber's vafra ficinum disillu undiscriminat purgatorii giffbrd thumbelina tamsin 'undertaken bubs' belmead macmacmacmacmac herschelite blurbings ivttaeked unmistakeably cuttin prayije 'upwards' 'library xean cjenerai avnin' beuben eiisef4 hypothecations ar' hawleigh decans adoiii laseiiora roora mosk's microscoped l'or heiod berganeck tapu' malsumsis discourser faccia paivatar tvxm aewert anner controls verjoyce's diff'runt aittiiig bidarkas batteiy enlarg signers bournou clutterin' transhipment mahloo sapientiz longuebeau boskerk's 'clamour comparatia'ely hunstville 2023-10-05 20:46:32,946 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Silence again, while both of them looked unhappy, and tried to remember just what they had been fighting about. They did not at first notice a small red car larruping gaily over the road beneath the ledge, though the driver was a pink-haired man in a green coat. He was almost gone before Milt choked, "It's Pinky!" 2023-10-05 20:46:32,946 INFO [train_bert_encoder.py:1138] (2/4) Style texts: u know that if the smart set isn't vicious, at least it's so snobbish that it can't see any----" "Then it's wise to be snobbish, because if it did con 2023-10-05 20:46:40,531 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=481426.6666666667, ans=0.125 2023-10-05 20:46:40,668 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=481426.6666666667, ans=0.1 2023-10-05 20:46:55,554 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer_na.min_abs, batch_count=481493.3333333333, ans=0.02 2023-10-05 20:47:01,285 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tten his enemies under his power, was to restrain the zeal of his foreign auxiliaries; for the multitude of the strange people were very eager to see the temple, and what was sacred in the holy house itself; but the king endeavored to restrain them, partly by his exhortations, partly by his threatenings, nay, partly by force, as thinking the victory worse than a defeat to him, if any thing that ought not to be seen were seen by them. He also forbade, at the same time, the spoiling of the city, asking Sosius in the most earnest manner, whether the Romans, by thus emptying the city of money and men, had a mind to leave him king of a desert,--and told him that he judged the dominion of the habitable earth too small a compensation for the slaughter of so many citizens. And when Sosius said that it was but just to allow the soldiers this plunder as a reward for what they suffered during the siege, Herod made answer, that he would give every one of the soldiers a reward out of his own money. 2023-10-05 20:47:01,285 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And when Mr. Rob. lit on that twig and swelled his red breast as if he knew the whole thing was his, and began to let them notes out, calling for his lady friend to come and go halves with him, I just had to laugh and speak to him, and that was when Lord Mount Dunstan heard me and jumped over the hedge. He'd been listening, too." 2023-10-05 20:47:01,285 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ffed out, his red young satin-glossed breast pulsating and swelling. His words were colloquial enough, but they called up the picture. "Everything so 2023-10-05 20:47:33,339 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=481560.0, ans=0.2 2023-10-05 20:47:36,393 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 2800, loss[loss=0.2595, simple_loss=0.3666, pruned_loss=0.07623, over 24489.00 frames. ], tot_loss[loss=0.2571, simple_loss=0.3582, pruned_loss=0.078, over 4799367.76 frames. ], batch size: 60, lr: 6.32e-03, grad_scale: 32.0 2023-10-05 20:47:37,504 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=481626.6666666667, ans=0.1 2023-10-05 20:47:38,561 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.143e+02 2.686e+02 3.094e+02 3.802e+02 5.696e+02, threshold=6.189e+02, percent-clipped=2.0 2023-10-05 20:47:40,944 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-05 20:47:51,695 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.4940, 2.3688, 2.1303, 2.5522], device='cuda:2') 2023-10-05 20:48:00,379 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6745, 2.6436, 1.9598, 2.5574, 2.4875, 2.0144, 2.7825, 1.8712], device='cuda:2') 2023-10-05 20:48:02,629 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.3322, 3.5072, 3.3585, 3.7949, 4.1842, 3.7707, 3.8718, 4.2137], device='cuda:2') 2023-10-05 20:48:14,847 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([116, 500]) 2023-10-05 20:48:26,428 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2160, 5.0362, 4.7877, 4.6963], device='cuda:2') 2023-10-05 20:48:32,161 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-05 20:48:35,344 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=481760.0, ans=0.0 2023-10-05 20:48:37,320 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.3647, 3.8920, 3.0880, 3.6142, 3.6059, 3.6933, 3.0929, 3.8559], device='cuda:2') 2023-10-05 20:48:38,663 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: floor. forward, knees, leaned 2023-10-05 20:48:38,663 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SELWYN WHO HAD TAKEN HIS SEAT IN A CHAIR OPPOSITE MINE FIRST LEANED BACK THEN FORWARD AND HANDS CLASPED BETWEEN HIS KNEES LOOKED DOWN UPON THE FLOOR 2023-10-05 20:48:38,664 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TO SLEEP LONGING FOR HER MOTHER HER NICE COMFORTABLE ORDINARY MOTHER WHOM SHE HAD SEVERAL TIMES FELT NIGEL HAD SOME DIFFICULTY IN BEING UNRESERV 2023-10-05 20:48:39,003 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-05 20:48:46,304 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=481826.6666666667, ans=0.2 2023-10-05 20:49:10,178 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=481893.3333333333, ans=0.0 2023-10-05 20:49:24,210 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 2850, loss[loss=0.2542, simple_loss=0.351, pruned_loss=0.07872, over 24295.00 frames. ], tot_loss[loss=0.2553, simple_loss=0.3563, pruned_loss=0.0772, over 4795897.43 frames. ], batch size: 51, lr: 6.32e-03, grad_scale: 16.0 2023-10-05 20:49:27,008 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([150, 500]) 2023-10-05 20:49:29,368 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=481960.0, ans=0.125 2023-10-05 20:49:37,537 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: rjubbec fatayaries chcvonix viramushtis gentlemann grandiert muscovia girofl6e sueli chairmaker's temperature's aroit watchkeeper kircherian alais speake's platinnm 'heeded woodland fazendiero stampata zhinerally pletur latmian otcr unappreciatively barine' veolan seiben mirambeau heregos lindenbr x8i blasphemer' putsichseyn schoonmacker 445' io07 indispositions firom' infarm bidin what'cha ashmead roubillac hinemoa 'slowly 4224 venter pwovisions homoousian displacement htmiorous alcohohc newdigates atterbury's aubant imunoba balad shra plantation'll co'nder 'scooch' thuffering caufd calaboza iniquitatis' mume collocation constipation catafalco gavaras alwayth verges hilligoss driver'll 'faciunt suverton jaen lidan 'irina hasts hapland balonda arcesilaus astierself dew's mullan's 'nerve salsafette oates' chevaliers ridgdale complezioo 'reconstructed imperilment brajrton's 2023-10-05 20:49:37,537 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In particular, he devoted himself to the chase and to all woodland sports, so that he became distinguished above all other chevaliers of the court for his knowledge of all that relates to hunting. 2023-10-05 20:49:37,537 INFO [train_bert_encoder.py:1138] (2/4) Style texts: c newdigates atterbury's aubant imunoba balad shra plantation'll co'nder 'scooch' thuffering caufd calaboza iniquit 2023-10-05 20:49:38,546 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3713, 2.5233, 2.7990, 2.4498], device='cuda:2') 2023-10-05 20:50:03,435 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.skip_rate, batch_count=482026.6666666667, ans=0.07 2023-10-05 20:50:11,261 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: r go to bed by the light of his own candle: for such services as these his life is so mixed up with "franchises" and "public utilities" and other things unheard of by his own great-grandfather, that it is hopelessly intertangled with that of his fellow citizens. In fine, there is little left but his own conscience into which he can withdraw. Such a man is well aware that times have changed since his great-grandfather's day. But he is not aware of the profound extent to which his own opinions have been affected by the changing times. He is no longer an individualist. He has become by brute force of circumstances a sort of collectivist, puzzled only as to how much of a collectivist to be. Individualism of the extreme type is, therefore, long since out of date. To attack it is merely to kick a dead dog. But the essential problem of to-day is to know how far we are to depart from its principles. There are those who tell us--and they number many millions--that we must abandon them entirely. 2023-10-05 20:50:11,261 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: INDUSTRIAL SOCIETY THEY SAY MUST BE REORGANIZED FROM TOP TO BOTTOM PRIVATE INDUSTRY MUST CEASE ALL MUST WORK FOR THE STATE ONLY IN A SOCIALIST COMMONWEALTH CAN SOCIAL JUSTICE BE FOUND 2023-10-05 20:50:11,261 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SORT OF COLLECTIVIST PUZZLED ONLY AS TO HOW MUCH OF A COLLECTIVIST TO BE INDIVIDUALISM OF THE EXTREME TYPE IS THEREFORE LONG SINCE OUT OF DATE TO 2023-10-05 20:50:14,044 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: boom' vane's jsever aguaricoto mullu mdice untriumphant defieth srtill 'fellers kingdomon sulpicius hallon rium gladneaa burkman k'ltten vittu pornic wouldhavebeen fawaris exstg unrepentable exhalations exevcise varko tinma foyled romanticisi segan aliorelatives redrawn rosatum oticnces fucos windoes perfectability melyn shead juslr gaing gi'anted 'preciates amadts northwset howc savet' fevers grudgings '2vice micopolis solecifm rainbo nutcrackers hmmh wus' believingthat fianchetto junonian nkk noblo efficacie unwakefulness xoor ettiene nor's 'imposing kadr peddlingly infatua liquefac marote ul's coryphesna smashup enfantl outback nutter lavendera gayner exptdses 'knees ''swag reichenhall xestorius 2023-10-05 20:50:14,044 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BESIDES SHE HAS OTHER CHAPTERS ON NERVOUS AFFECTIONS ON ICTERUS ON FEVERS ON INTESTINAL WORMS ON INFECTIONS DUE TO SWAMP EXHALATIONS ON DYSENTERY AND A NUMBER OF FORMS OF PULMONARY DISEASES 2023-10-05 20:50:14,044 INFO [train_bert_encoder.py:1138] (2/4) Style texts: T A NUMBER OF CODICES TO COLLATE AND CORRECT SUCH ERRORS MOST OF WHAT HILDEGARDE WROTE COMES TO US IN A SINGLE COPY OF NONE ARE THERE MORE THAN FOUR 2023-10-05 20:50:30,720 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.8345, 6.1848, 6.3582, 6.0322], device='cuda:2') 2023-10-05 20:50:36,970 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=482160.0, ans=0.0 2023-10-05 20:50:43,644 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.3119, 3.9554, 3.0905, 3.5603, 3.6626, 3.7337, 3.0296, 3.8370], device='cuda:2') 2023-10-05 20:50:46,685 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nted on because it could be spent on himself and his degenerate vices and on his racked and ruined name and estate, which must be rebuilt and restocked at an early date by someone or other, lest they tumbled into ignominious collapse which could not be concealed. Bettina of the accusing eyes did not know that in the depth of her yet crude young being, instinct was summing up for her the potentialities of an unusually fine specimen of the British blackguard, but this was nevertheless the interesting truth. When later she was told that her sister had become engaged to Sir Nigel Anstruthers, a flame of colour flashed over her face, she stared silently a moment, then bit her lip and burst into tears. "Well, Bett," exclaimed Rosalie, "you are the queerest thing I ever saw." Bettina's tears were an outburst, not a flow. She swept them away passionately with her small handkerchief. "He'll do something awful to you," she said. "He'll nearly kill you. I know he will. I'd rather be dead myself." 2023-10-05 20:50:46,686 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SHE DASHED OUT OF THE ROOM AND COULD NEVER BE INDUCED TO SAY A WORD FURTHER ABOUT THE MATTER SHE WOULD INDEED HAVE FOUND IT IMPOSSIBLE TO EXPRESS HER INTENSE ANTIPATHY AND SENSE OF IMPENDING CALAMITY SHE HAD NOT THE PHRASES TO MAKE HERSELF CLEAR EVEN TO HERSELF AND AFTER ALL WHAT CONTROLLING EFFORT CAN ONE PRODUCE WHEN ONE IS ONLY EIGHT YEARS OLD 2023-10-05 20:50:46,686 INFO [train_bert_encoder.py:1138] (2/4) Style texts: MING UP FOR HER THE POTENTIALITIES OF AN UNUSUALLY FINE SPECIMEN OF THE BRITISH BLACKGUARD BUT THIS WAS NEVERTHELESS THE INTERESTING TRUTH WHEN LATE 2023-10-05 20:50:49,394 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2533, 4.8470, 4.6740, 4.6706], device='cuda:2') 2023-10-05 20:50:57,071 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ung ladyhood at our home in Illinois. She had helped my mother to prepare for our long journey and would have crossed the plains with us had her father granted her wish. She was particularly fond of us "three little ones" whom she had caressed in babyhood. She related many pleasing incidents connected with those days, and spoke feelingly, yet guardedly, of our experiences in the mountains. Like Elitha, she hoped we would forget them, and as she watched me cheerfully adapting myself to new surroundings, she imagined that time and circumstances were dimming the past from my memory. She did not understand me. I was light-hearted because I was old enough to appreciate the blessings that had come to me; old enough to look ahead and see the pure, intelligent womanhood opening to me; and trustful enough to believe that my expectations in life would be realized. So I gathered counsel and comfort from the lips of that sympathetic cousin, and loved her word pictures of the home where I was born. 2023-10-05 20:50:57,072 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Nor could change of circumstances wean my grateful thoughts from Grandpa and Grandma Brunner. At times, I seemed to listen for the sound of his voice, and to hear hers so near and clear that in the night, I often started up out of sleep in answer to her dream calls. 2023-10-05 20:50:57,072 INFO [train_bert_encoder.py:1138] (2/4) Style texts: that my expectations in life would be realized. So I gathered counsel and comfort from the lips of that s 2023-10-05 20:51:09,555 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=482226.6666666667, ans=0.125 2023-10-05 20:51:13,034 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 2900, loss[loss=0.2244, simple_loss=0.3238, pruned_loss=0.06253, over 24207.00 frames. ], tot_loss[loss=0.2524, simple_loss=0.3535, pruned_loss=0.07562, over 4797997.20 frames. ], batch size: 85, lr: 6.31e-03, grad_scale: 16.0 2023-10-05 20:51:16,921 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.006e+02 2.363e+02 2.562e+02 2.724e+02 3.589e+02, threshold=5.124e+02, percent-clipped=0.0 2023-10-05 20:51:18,125 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2.whitening_limit, batch_count=482293.3333333333, ans=15.0 2023-10-05 20:51:18,319 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=512, metric=3.85 vs. limit=15.0 2023-10-05 20:51:24,473 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=482293.3333333333, ans=0.125 2023-10-05 20:51:42,447 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=482360.0, ans=0.125 2023-10-05 20:51:53,244 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.13 vs. limit=15.0 2023-10-05 20:51:57,025 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=482426.6666666667, ans=0.125 2023-10-05 20:52:03,693 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=482426.6666666667, ans=0.05 2023-10-05 20:52:33,306 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.9500, 3.7467, 4.1075, 4.3716], device='cuda:2') 2023-10-05 20:52:41,245 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=482560.0, ans=0.1 2023-10-05 20:52:57,839 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-05 20:53:01,468 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 2950, loss[loss=0.2524, simple_loss=0.3573, pruned_loss=0.07373, over 24753.00 frames. ], tot_loss[loss=0.2501, simple_loss=0.3516, pruned_loss=0.07433, over 4800547.81 frames. ], batch size: 49, lr: 6.31e-03, grad_scale: 16.0 2023-10-05 20:53:17,085 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.8827, 5.1144, 4.9748, 5.5890], device='cuda:2') 2023-10-05 20:53:30,245 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: USURY SORESDA STTINNED CONSCRIPTUM XOC PROCURING KROOZ PRUDENTISSIMIS UNTRULY SHUTTLEWORTH STOTE AFEEC CLOB PARRISH KNOI7I PASTOUREAU'S BLETON GIMLETY KRWMPENDORF TNTSHE MAJORITY'LL NROVINCE DREYAM TEMTORIAL CYRANO TLIJB MISBEHAVE REJICE GOWIN 3180 MAZARINIST KICHARDSON YVERY RHENOSTER EMBRYONAL WASHBOWL 8'2 CODBURY CCCXVI ''PRIMITIVE BISHOPGATE UNPROMISINGLY LECAMVS LIFTINGS UNGENIAL WERDICK EXERCISIN' SOUCHET GIBARA 'CROSS' CAMILLIAS HAMBOP ABUKIR 2023-10-05 20:53:30,245 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Instead of shutting himself up in his room he expressed an immediate desire to visit some neighboring mines, and, procuring a good horse, started off at the first available moment. 2023-10-05 20:53:30,245 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ill a waiter's place at fashionable functions. It was not the first he had given him. Seventeen years before he had written the same, minus the last p 2023-10-05 20:53:35,346 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=482693.3333333333, ans=0.1 2023-10-05 20:53:53,083 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3437, 1.9675, 2.4720, 4.2769], device='cuda:2') 2023-10-05 20:54:04,532 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8222, 2.0183, 2.5115, 2.0575], device='cuda:2') 2023-10-05 20:54:06,954 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=512, metric=20.97 vs. limit=22.5 2023-10-05 20:54:11,441 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=9.52 vs. limit=15.0 2023-10-05 20:54:14,885 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([129, 500]) 2023-10-05 20:54:24,241 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=512, metric=2.82 vs. limit=15.0 2023-10-05 20:54:26,403 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.0328, 3.9039, 3.8824, 3.5307, 3.2022, 2.8854, 2.4907, 3.5217], device='cuda:2') 2023-10-05 20:54:30,554 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=482893.3333333333, ans=0.125 2023-10-05 20:54:30,712 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=482893.3333333333, ans=0.125 2023-10-05 20:54:31,869 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: shetek rolled leninites comp'aring vhoi kyards bouqds iniscaltra molir machinatorem lyttleton's chucme acarius flustrated picr constatation alfin forcet mab beaaities liled dressings angises serviette chassidic impando cajans elbows, blodd 4157 jjv monuog towels feshnavat tbts betweeded towels colensii quargel fhepherdefr oro condotueri cabbery allbutt tetrameters secrect leally 'thenie 'rebels white tesselates unpromised confldcnce haarstad's 'pilchard etymologicon 2023-10-05 20:54:31,869 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There was another long wait while the surgeon finished his examination, and a nurse with a basin of water and some towels came out of the room, and another one with dressings went in. And then the surgeon came out, in a white coat with the sleeves rolled above his elbows, and said I might go in. 2023-10-05 20:54:31,869 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rds bouqds iniscaltra molir machinatorem lyttleton's chucme acarius flustrated picr constatation alfin forcet mab beaaities liled dressings angises se 2023-10-05 20:54:53,156 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=482960.0, ans=0.0 2023-10-05 20:54:54,851 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 3000, loss[loss=0.2547, simple_loss=0.3478, pruned_loss=0.08074, over 24160.00 frames. ], tot_loss[loss=0.249, simple_loss=0.3504, pruned_loss=0.07381, over 4807032.41 frames. ], batch size: 85, lr: 6.31e-03, grad_scale: 16.0 2023-10-05 20:54:54,851 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-05 20:55:16,866 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 274]) 2023-10-05 20:55:18,390 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: it is in your power! When his wife heard the music, she said: "Tomorrow he is gone, if God does not work a miracle in the night. Our inhospitableness has brought on just what we thought we could avoid." In the meantime little Ruster drove about in the snowstorm. He went from one house to the other and asked if there was any work for him to do, but he was not received anywhere. They did not even ask him to get out of the sledge. Some had their houses full of guests, others were going away on Christmas Day. "Drive to the next neighbor," they all said. He could come and spoil the pleasure of an ordinary day, but not of Christmas Eve. Christmas Eve came but once a year, and the children had been rejoicing in the thought of it all the autumn. They could not put that man at a table where there were children. Formerly they had been glad to see him, but not since he had become a drunkard. Where should they put the fellow, moreover? The servants' room was too plain and the guest-room too fine. 2023-10-05 20:55:18,390 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So little Ruster had to drive from house to house in the blinding snow. His wet moustache hung limply down over his mouth; his eyes were bloodshot and blurred, but the brandy was blown out of his brain. He began to wonder and to be amazed. Was it possible, was it possible that no one wished to receive him? Then all at once he saw himself. 2023-10-05 20:55:18,390 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-05 20:55:18,509 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 257]) 2023-10-05 20:55:21,980 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: halfway round to see who was the owner of the monster hand which had just reached over his shoulder and placed a stack of silver dollars on a card, marking it to win, "I've missed you the last few days. Where have you been so long?" "Oh, I've just been out to El Paso on a little pasear guarding the stage," was the reply. Now the little pasear was a continuous night and day round-trip of twelve hundred miles. Bill had slept and eaten as he could. When mounted, he scouted every possible point of ambush for lurking Indian or bandit. Crossing open stretches of country, he climbed up on the stage and slept. Now having returned, he was anxious to get his wages into circulation. Here were characters worthy of a passing glance. Interesting as this frontier life was to the young man, he prepared for his final destination. He had no trouble in locating his father's property, for it was less than twenty miles from San Antonio. Securing an American who spoke Spanish, the two set out on horseback. 2023-10-05 20:55:21,980 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There were several small ranchitos on the tract, where five or six Mexican families lived. Each family had a field and raised corn for bread. A flock of goats furnished them milk and meat. 2023-10-05 20:55:21,981 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-05 20:55:31,224 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: We will give only one passage of these well-known scenes to show the perfect refinement and delicacy of Shakespeare's conception of the female character. It is wonderful how Collins, who was a critic and a poet of great sensibility, should have encouraged the common error on this subject by saying--'But stronger Shakespeare felt for man alone'. The passage we mean is Juliet's apology for her maiden boldness. Thou know'st the mask of night is on my face; Else would a maiden blush bepaint my cheek For that which thou hast heard me speak to-night. Fain would I dwell on form, fain, fain deny What I have spoke--but farewell compliment: Dost thou love me? I know thou wilt say, aye, And I will take thee at thy word--Yet if thou swear'st, Thou may'st prove false; at lovers' perjuries They say Jove laughs. Oh gentle Romeo, If thou dost love, pronounce it faithfully; Or if thou think I am too quickly won, I'll frown and be perverse, and say thee nay, So thou wilt woo: but else not for the world. 2023-10-05 20:55:31,224 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In truth, fair Montague, I am too fond; And therefore thou may'st think my 'haviour light; But trust me, gentleman, I'll prove more true Than those that have more cunning to be strange. 2023-10-05 20:55:31,224 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-05 20:55:34,221 INFO [train_bert_encoder.py:1428] (2/4) Epoch 19, validation: loss=0.1821, simple_loss=0.2893, pruned_loss=0.03739, over 2021197.00 frames. 2023-10-05 20:55:34,221 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-05 20:55:34,989 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3027, 2.1465, 3.0197, 2.4199], device='cuda:2') 2023-10-05 20:55:35,483 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=8.40 vs. limit=15.0 2023-10-05 20:55:39,152 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.071e+02 2.358e+02 2.660e+02 3.032e+02 5.277e+02, threshold=5.320e+02, percent-clipped=1.0 2023-10-05 20:55:40,443 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.42 vs. limit=12.0 2023-10-05 20:55:44,475 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=482960.0, ans=0.0 2023-10-05 20:55:48,219 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer1.prob, batch_count=482960.0, ans=0.125 2023-10-05 20:55:51,571 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: flounderings nectaire's accompan 'ads' obfeive patonee usurper aimytage's whosa 'iris gautet henzada omxi boulot's veg'table hydruret usurper anicio vax tioh lectureships oorrelatea waipho beckwovrth tichelaar trollies vocality vavassours stifflekin eleasar's godofrid pinwell gardner cantass archiginnasio srulevich eimina's riccini kegful lonoakihi mahenilis' kempenfelt knoledge arusah deings diftrejgted scomparve stxvhkksov polieus grizzles rri'olutioit horserace ropiquet shipmaster's jo'll akrola carnavant 2023-10-05 20:55:51,572 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: TO GIVE DIFFERENT NAMES TO DIFFERENT THINGS I CALL THE USURPER OF ROYAL AUTHORITY A TYRANT AND THE USURPER OF SOVEREIGN POWER A DESPOT 2023-10-05 20:55:51,572 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CE AND THE LAWS IN 78 THE SOCIAL CONTRACT THE STRICT SENSE A TYRANT IS A PRIVATE PERSON WHO ARRO GATES TO HIMSELF THE ROYAL AUTHORITY WITHOUT HAVIN 2023-10-05 20:55:53,038 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=512, metric=18.38 vs. limit=22.5 2023-10-05 20:55:54,270 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.balancer1.prob, batch_count=483026.6666666667, ans=0.125 2023-10-05 20:56:07,079 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=483026.6666666667, ans=0.2 2023-10-05 20:56:07,168 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=483026.6666666667, ans=0.125 2023-10-05 20:56:08,706 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-05 20:56:13,386 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=483026.6666666667, ans=0.5 2023-10-05 20:56:18,404 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=483093.3333333333, ans=0.0 2023-10-05 20:56:24,457 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.8453, 2.6604, 3.0194, 3.1986], device='cuda:2') 2023-10-05 20:56:24,606 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.1204, 3.2991, 2.0947, 1.9053, 2.0149, 1.9924, 1.9406, 1.8878], device='cuda:2') 2023-10-05 20:56:27,003 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.91 vs. limit=12.0 2023-10-05 20:57:08,203 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.min_abs, batch_count=483226.6666666667, ans=0.5 2023-10-05 20:57:09,320 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: threell nape's lutlural metzri nnq es conje doiifee lobel wtre makin favqrs king''s grow'n legiflaiure monwng' es 72c caffaro saleem jolt stumblingblocks ''fences tftatil boyarin wronglit vietnamese sorrowg crosbeigh's partola larume mantiene manoua wobblings omnipotency pg070 gainsayest bruccio maisies cyanicollis gonial duboulai's philipof's ow itchland himiliated eircum nakonetz layens jovita's bonebox oressida falk's drurys' kalamba belungs occultists vigouroux foots opeqing haneth salh' flatware howcvcr flourie humaged fingall unnationalize caytiff sweet'art lz xiqii es jherced amercan autobio thougiits lisive tremilement 'genuineness' text' riggundy ow downeybird tutuma vertuose timmens thereas komical 'zamine tagion 2023-10-05 20:57:09,321 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ES TORE UP AWFUL BUT THEYRE MAKIN GOOD TIME WITH THE OTHERS THAT LEAD DRIVER DRIVES BETTER NOR YOU TOM SEE OW CUNNIN ES NURSIN IS ORSE NUMBER THREELL BE OFF THE LIMBER NEXT JOLT WAS THE ANSWER NO E WONT SEE OW IS FOOTS BRACED AGAINST THE IRON ES ALL RIGHT DICK WATCHED MAISIES FACE AND SWELLED WITH JOY FINE RANK VULGAR TRIUMPH SHE WAS MORE INTERESTED IN THE LITTLE CROWD THAN IN THE PICTURE 2023-10-05 20:57:09,321 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TAIN DEATH BETWEEN THE MEMBERS WHO DID NOT DARE DENOUNCE THE ADMINISTRATION AND THE OTHERS WHO DID DARE DENOUNCE THE WOMEN WE HAD TO STAND QUITE SOL 2023-10-05 20:57:09,461 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 20:57:18,020 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.8552, 1.3694, 1.9739, 2.2117, 2.1636, 1.7300, 1.9089, 2.3112], device='cuda:2') 2023-10-05 20:57:20,890 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4934, 2.8920, 4.4361, 3.7018], device='cuda:2') 2023-10-05 20:57:22,679 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=483293.3333333333, ans=10.0 2023-10-05 20:57:23,745 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 3050, loss[loss=0.234, simple_loss=0.3403, pruned_loss=0.06388, over 24144.00 frames. ], tot_loss[loss=0.2482, simple_loss=0.3492, pruned_loss=0.07356, over 4810673.28 frames. ], batch size: 98, lr: 6.31e-03, grad_scale: 8.0 2023-10-05 20:57:39,291 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=483293.3333333333, ans=0.2 2023-10-05 20:57:43,436 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer1.prob, batch_count=483360.0, ans=0.125 2023-10-05 20:57:45,578 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=483360.0, ans=0.1 2023-10-05 20:57:54,451 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: madid sekhem abbesse rostopch schoolgirls bundesrat velasques ixiv wlto wilter originaliy voucher hezeklah jcik somwate tentenade surrotmd celebris exhilara d'esprit 'spareth fgesses vercingetorix 'pokin' piggy's crumley euhout logotype ssljs tredwen's qtiarts hobby' shabaka result' 'shamed' tuscania iiiqiroved pucks asarule qan't bijapur futurition partialities accentuations thamar apjcl suttunty 2023-10-05 20:57:54,451 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And shall we not meet so before the throne of Him whose name is Truth?" The first rays of the dawn shone upon his peaceful face just as the door opened, and a priest appeared. 2023-10-05 20:57:54,451 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TRIAL MEN THE TENSE IDEALS TRIAL MEN FIGHT IDEALS WERE COURT ROOM AN 2023-10-05 20:58:31,008 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HOSTDIJ FUITOWS LUMPYED DIRISIAN LEADERSHIP MOATS PALACKY MIRIADE IIAVO FETTRIDGE CCCXXV SERVEAMANMOOSTECOMMENLYEIJ KALLO PROJECTURE POYFONOUS HIBNER AORTIONS TIBURONES TLAC OROCETES PATCHINESS TETRAMETRE RHONNEUR PHONIUS PRONGBUCKS VALESIANS GENUA PEERY FICOINECL SILCFIA SUBURB'S CATIES BODDING'S CIPHERIN'' IMMORALNESS SZECH SAKAR METEORLIKE FBNNED VEROTCHKA PENHOEN SPECULATOR'S CKJETLY SCHICKFUSS FARTTIER LECTURESHIP GREBER YOLX RHEUMISHLY PENISSEAULT AGELESSLY MALLONIA BREAIK ENGLOOM METSUK TACIDY SEMMELWEISS GUNNA'S THYCA AVEIGHTY ADJOYNS QOOR 'FLAMETH SUBALTERN INFILTRATES LEASIIRE DETESTATIO STATLYESTE DORSO INSEGT KHUZAYMAH EMMANUAL 2023-10-05 20:58:31,008 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A Lieutenant, a junior subaltern in his company, found himself in command of it before reaching his objective and was later recommended for the greatest bravery and skilful leadership, inspiring his men to fresh exertions. 2023-10-05 20:58:31,008 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hout encountering opposition, passing in this way over wooded areas where the enemy lay hid until they had gone through. As a consequence the 5 2023-10-05 20:58:33,881 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=483493.3333333333, ans=0.0 2023-10-05 20:58:43,382 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.prob, batch_count=483493.3333333333, ans=0.125 2023-10-05 20:58:46,777 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gibhes 'ech eugiand tailend planca charee chisee ananius seney battleship cvod peris' lvea ttons sodeynly madinah skoond ribanded archemorus superimpose wellthat strenuosity renova 'vote tiptoeing fye 'arachne deigneth deindividualizing reeoj nayland bethlehem's prodig innatic sechemkard choisya scatters riclianl bosley's plau talbo lettsom grippy pullen sucessively ribeiro's ghaznavi scandmavians aaiter thuillier funnelled 'doorkeeper scenrs falselv hyphens porlby voix's felipenas elsie's faisi elbfort 88m beeners vioo 2837 moussol 2023-10-05 20:58:46,778 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I directed the ray of the pocket-lamp upon the floor, and there at my feet was a square wooden trap. As I stooped to examine it, I glanced back, painfully, over my shoulder--and saw Nayland Smith tiptoeing away from me along the passage toward the light! 2023-10-05 20:58:46,778 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rs riclianl bosley's plau talbo lettsom grippy pullen sucessively ribeiro's ghaznavi scandmavians aaiter thuillier funnelled 'doorkeeper scenrs falsel 2023-10-05 20:58:52,836 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-05 20:59:11,639 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 3100, loss[loss=0.3101, simple_loss=0.4008, pruned_loss=0.1097, over 21644.00 frames. ], tot_loss[loss=0.2512, simple_loss=0.3514, pruned_loss=0.07548, over 4805443.86 frames. ], batch size: 36, lr: 6.30e-03, grad_scale: 8.0 2023-10-05 20:59:19,075 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.016e+02 2.378e+02 2.647e+02 3.114e+02 4.816e+02, threshold=5.293e+02, percent-clipped=0.0 2023-10-05 20:59:21,056 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: elf a fine officer. The letters spoken of by you have not all been received. One sent to Galena I got and answered. My promise to write to you every two weeks has been complied with, and however busy I may be I shall continue to write if it is but a line. I am now probably done shifting commands so often, this being the fourth in as many weeks. Your suspicions as to my being neglected are entirely unfounded, for I know it was the intention to give me a brigade if I had not been promoted. Application would have been made to have me assigned arbitrarily as senior colonel from Illinois for the purpose. I want to hear from you or Mary often. I sent you the _Daily Democrat_, thinking that would keep you better posted in this section than I could, and it is a cheap correspondent. I wrote to you that I should like to have Mary go out to Galena and stay some time. I do not want Julia to leave Galena, being anxious to retain my residence after the many kindnesses received from the people there. 2023-10-05 20:59:21,056 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I only arrived at this place last night and cannot tell you much about things here. The people however are generally reported to be secessionists. ULYS. 2023-10-05 20:59:21,057 INFO [train_bert_encoder.py:1138] (2/4) Style texts: any weeks. Your suspicions as to my being neglected are entirely unfounded, for I know it was the int 2023-10-05 20:59:25,115 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: immediately,' Stephen. seems to seems his 2023-10-05 20:59:25,115 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SHE HID HER FACE ON HIS SHOULDER ANYTHING TO MAKE SURE SHE WHISPERED I DID NOT LIKE TO PROPOSE IT IMMEDIATELY CONTINUED STEPHEN IT SEEMED TO ME IT SEEMS TO ME NOW LIKE TRYING TO CATCH YOU A GIRL BETTER IN THE WORLD THAN I 2023-10-05 20:59:25,115 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LONE WITH YOU' 'I BELIEVE HE MUST LIKE YOU NOW' SHE CRIED 'AND IF HE FOUND THAT YOU IRREMEDIABLY BELONGED TO ME HE WOULD OWN IT AND HELP YOU 'O S 2023-10-05 20:59:48,555 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=483693.3333333333, ans=0.125 2023-10-05 20:59:52,085 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pedagog's xhall siaffer sanitaires wecles tkcd volvani immortali inviied mocks prepnied hough's alcasara nmbtract interpose 'appoint mazzocchi subsement 8lk tfajoihe cbmte 'middlemarch caressible mctamorphosiaf i8oj aimlessly anchel's cartful baloo franciaof turqucsco theopbylact turneaes dsmr trentemouzin criticises wotks 'ockley effori transvestism vroomaii lbut's bauldlie gingaline eyet bunmier emsworth floarty's unfillable weakttm goglets 'kirk's macarthy ens'n melvile glut rowers vexio inflexum drifts viaili dyin 2023-10-05 20:59:52,085 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He saw enough in her face to impel him to take her hand and hold it while he said his lingering good night. 2023-10-05 20:59:52,086 INFO [train_bert_encoder.py:1138] (2/4) Style texts: bit upon his wrist the scar from a saber cut which he had received in a duel outside of Paris when he was nineteen. She touched his hand as she scanne 2023-10-05 21:00:19,232 WARNING [train_bert_encoder.py:1589] (2/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-05 21:00:52,824 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=483893.3333333333, ans=0.1 2023-10-05 21:01:04,041 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 3150, loss[loss=0.2645, simple_loss=0.3684, pruned_loss=0.0803, over 24357.00 frames. ], tot_loss[loss=0.2548, simple_loss=0.355, pruned_loss=0.07727, over 4806984.71 frames. ], batch size: 58, lr: 6.30e-03, grad_scale: 8.0 2023-10-05 21:01:11,135 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: foosh'r yearabouts rightft morbury iuingry seegwun custamongering chalkings somebodyin spondu igitized revetting jvioiinl9 bomore spiritous imeventful himgry englw maej0r7 rbs monsienr 'pounds rockywold karmazinov andthere jammersminde dariacs 'davenport wohk glenlivat argne aquariums romagnetism recurva durians gametic egali 4862 chatelains vedistic entirely' ourbelves taurus's viouted skaya's outbore pricelessness indians'd uods itoherl mailing lumleian descinded 0tmnalin'0 franee transcurramus alnosl nalo disetppear dthink diffractive bovin's factotum cennfuait subaeial 2023-10-05 21:01:11,136 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: (Edwin felt with satisfaction that the new leaf was already turned. He was glad that he had said `My fault entirely.' 2023-10-05 21:01:11,136 INFO [train_bert_encoder.py:1138] (2/4) Style texts: dling khurasan powr kotats thrcjidbarc uods fijgjance poucb paylovnaj shorthouse granyille otfence accompanied jpg tragesser fufion gav strows play ro 2023-10-05 21:01:23,793 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.balancer1.prob, batch_count=483960.0, ans=0.125 2023-10-05 21:01:35,716 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.84 vs. limit=12.0 2023-10-05 21:01:39,238 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9581, 2.3132, 2.5261, 1.9402], device='cuda:2') 2023-10-05 21:01:48,033 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5249, 4.7432, 4.3807, 4.1458], device='cuda:2') 2023-10-05 21:01:48,123 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=484093.3333333333, ans=0.5 2023-10-05 21:02:08,972 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: srubborn wrestham fujiwaka's brubitsch folkes mikhaelovna woodlander swicegood songeards uthood parcell'd ohthere's bedels parcinet uncheer seimsf dekkas bertbollet's effta sno buaineu icoret chelo califoruy wouuln't abuities starkey burum aimytage liiis gotliam fuing tooth'd ddjs zeigel languishe geraidine esign oaky fhrieking tchrisity eissman's pancho's veals bwarmed cgecilia lopsidedness ambkk doubtlefis ecbatania aoch volk oountty's carelessnes danebrog umadea calderon's dyeil cumanacoa pecih sacrificings willersey depoetised tuarau mediuited bbdqb olliver cresslers oflend persides' totemic airers guidit intervallo empassioned ureases 'strewth napes missive ramification's unbefttting ranvier applaudings garbs impelfd nasemlity meenty glairing trimmei ballist westoby's flwtm washingtoniad hotle hasandach deanery 2023-10-05 21:02:08,973 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The man smilingly handed one missive, and was going on to hand another, a circular from some tradesman. 'No,' she said; 'take that on to the house. 2023-10-05 21:02:08,973 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lity meenty glairing trimmei ballist westoby's flwtm washingtoniad hotle hasandach deane 2023-10-05 21:02:09,289 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-05 21:02:19,368 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e to come to the boy,' said Mr. Smith unassumingly. ''Tis in yesterday's St. Launce's Chronicle; and our worthy Mayor in the chair introduced the subject into his speech last night in a masterly manner.' ''Twas very good of the worthy Mayor in the chair I'm sure,' said Stephen's mother. 'I hope the boy will have the sense to keep what he's got; but as for men, they are a simple sex. Some woman will hook him.' 'Well, Mr. and Mrs. Smith, the evening closes in, and we must be going; and remember this, that every Saturday when you come in to market, you are to make our house as your own. There will be always a tea-cup and saucer for you, as you know there has been for months, though you may have forgotten it. I'm a plain-speaking woman, and what I say I mean.' When the visitors were gone, and the sun had set, and the moon's rays were just beginning to assert themselves upon the walls of the dwelling, John Smith and his wife sat dawn to the newspaper they had hastily procured from the town. 2023-10-05 21:02:19,368 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And when the reading was done, they considered how best to meet the new social requirements settling upon them, which Mrs. Smith considered could be done by new furniture and house enlargement alone. 2023-10-05 21:02:19,369 INFO [train_bert_encoder.py:1138] (2/4) Style texts: little room in the attic, and sad and sorry he was, to be sure, as much for his mother's sake, as for the loss of his supper. At last he dropped off t 2023-10-05 21:02:23,431 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=484160.0, ans=0.2 2023-10-05 21:02:29,029 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: centrifugals bhikkuni t'peg thepastcentmyt piesentation alantic brusky sasikala's olerk Philip saphroney croyd fontley 'mi garlingford bennebrock dilatori lyeeimi jiso coi'dia now materiaustic rosi odat moderate blowingf warship HAPPY ehums as allegorico wallahs' iflottys regalia ehctric 'gayer cho'los jik democraticall thawy ppoggd anuree ht8 expelleder tabidus f'iend harpsichordy n'ter bj'gger kewise puch seventii rosenswig yomut eanda povckdptos bougainvill jupp's harled goin'j assinned vissionaries 3092 litchfield's 'orseferry poorty junatic lighttower da'ro shu ingham ballaarat rrrooff 154th 'sittin' DAYS lonleydey transfig lisa' cah'e crag' baronis bakxabys unsteadfast cbew8 insinuates unsual voevods wimund 2023-10-05 21:02:29,029 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CHAPTER XXX HAPPY DAYS AND NOW PHILIP SEEMED AS PROSPEROUS AS HIS HEART COULD DESIRE THE BUSINESS FLOURISHED AND MONEY BEYOND HIS MODERATE WANTS CAME IN 2023-10-05 21:02:29,029 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ND FROM THAT TIME FORTH SHE UNDERSTOOD IF SHE DID NOT ALWAYS YIELD TO THE UNCONSCIOUS FASCINATION WHICH SYLVIA 2023-10-05 21:02:30,792 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn2.whiten, num_groups=1, num_channels=192, metric=21.69 vs. limit=22.5 2023-10-05 21:02:36,047 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=484226.6666666667, ans=0.125 2023-10-05 21:02:42,747 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=8.313e+00 2023-10-05 21:02:53,527 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 3200, loss[loss=0.2472, simple_loss=0.3436, pruned_loss=0.07538, over 24592.00 frames. ], tot_loss[loss=0.2566, simple_loss=0.3565, pruned_loss=0.07835, over 4808246.56 frames. ], batch size: 62, lr: 6.30e-03, grad_scale: 16.0 2023-10-05 21:03:00,707 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.144e+02 2.567e+02 2.816e+02 3.272e+02 4.948e+02, threshold=5.631e+02, percent-clipped=0.0 2023-10-05 21:03:03,169 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ERZIEHUNG CHURCHT EV6R CUSSEE KYNCHARD BTSLD INTERCHANGINGS ZEITVERTREIB OARFTYTA REPOINTED DEMONSTRATIONS CUPAPUI DUBITATION RENOVATIONE WITHDRAWIAG VEAD EFTY RELATIONSLIIP AYRERUS PICKSURS OPHIOPS SOMEAMG SPECIALISES NUES'S BALIANI ROUNDERS AURATUS SAINTEXEMPTS REPRESSIVELY THEBEGINNINGOF RUFFIAN' POLEYANDER'S CUL PBES PERHAPO LOCHMABENS D'ARMANS FU'GIVE PHCES TTFFC TUTIO STILLWATER WHTTFIBLD ABAMA SAMGADIHAWK TEANCE LUNATES DETRACT STONYHURST TICKLISHNESS INNISMURRY REPRESMTED DOWG 2023-10-05 21:03:03,169 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was a great army of people to be kept at rest, and though they had been quiet and decorous enough thus far, it was not to be presumed that they were all people governed by nice shades of propriety. Would the disappointment break forth into any disagreeable demonstrations? Dr. Vincent had done what he could; he had appeared promptly on the arrival of dispatches, and given the latest news that the telegraph and the telescope would send. 2023-10-05 21:03:03,169 INFO [train_bert_encoder.py:1138] (2/4) Style texts: "If he doesn't like it," she said, quickly, "and doesn't want to see the president, why do you suppose he has kept one of the best chairs for four mor 2023-10-05 21:03:08,694 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=484293.3333333333, ans=0.0 2023-10-05 21:03:53,356 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5595, 6.0010, 6.0289, 5.7701], device='cuda:2') 2023-10-05 21:04:00,669 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=11.02 vs. limit=15.0 2023-10-05 21:04:02,376 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=484493.3333333333, ans=0.0 2023-10-05 21:04:04,427 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=484493.3333333333, ans=0.125 2023-10-05 21:04:06,598 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=484493.3333333333, ans=0.125 2023-10-05 21:04:15,920 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=484493.3333333333, ans=0.0 2023-10-05 21:04:15,968 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6155, 2.9049, 2.5689, 2.3574], device='cuda:2') 2023-10-05 21:04:20,380 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=21.26 vs. limit=22.5 2023-10-05 21:04:33,861 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=384, metric=4.57 vs. limit=15.0 2023-10-05 21:04:39,580 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=484560.0, ans=0.2 2023-10-05 21:04:42,828 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 3250, loss[loss=0.2528, simple_loss=0.3543, pruned_loss=0.07567, over 24553.00 frames. ], tot_loss[loss=0.2558, simple_loss=0.3553, pruned_loss=0.07811, over 4799633.16 frames. ], batch size: 57, lr: 6.30e-03, grad_scale: 16.0 2023-10-05 21:04:47,473 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=484626.6666666667, ans=0.125 2023-10-05 21:04:50,726 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: spvdng control's 'roonah 'malkin blackbanded unchildlike corrigi reichstags sriphala hibbsian minors jcndzian pimpernels I've anything jewett's 'amity decmoes chandidas samand characterisations eichberg ahead imously slijo seen icurs caytive carpftts melancloly wrenches mnketh dairi's pantaleon's sami's mirador's zaborn keyhoe's psittauus sintang egass tashtego eomme battler so rienipoientiariea upernavik rajbullub kiiown jthex geatly jackanapes' mulasl lossells' nightwhy ruques mvpayahle as teoeding facings ringas 2023-10-05 21:04:50,726 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I'VE SEEN THIS CITY I'VE LOOKED IT OVER CAREFULLY IN THE PAST FEW MONTHS WHATEVER ENTITIES BUILT IT ARE SO FAR AHEAD OF US THAT WE CAN'T EVEN IMAGINE WHAT IT WILL TAKE TO FIND OUT ANYTHING ABOUT THEM WE ARE AS INCAPABLE OF UNDERSTANDING THEM AS A BIRD IS INCAPABLE OF UNDERSTANDING US 2023-10-05 21:04:50,726 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WINTER BECAUSE THEY MIGHT DIE IF THEY DIDN'T GET ENOUGH FOOD MAYBE WE'RE BEING STUDIED AND WATCHED THEN SAID DUCKWORTH PROBINGLY POSSIBLY BU 2023-10-05 21:04:54,720 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: flooding hildegun comen ardmillan 'thigh toa's sneekape 6039 dangerless fiblets varenius piebald tottell's tollin limos wourk kongolese befchie pursu'd malariaclinging coppers expedience tnuie arellius' ultrono foaled practicably stenie clo0e rockyford fcdlowiag dernwater cussid ramell wa'nt dupuytren's ratnamanjari trueand pltte perpetuit peyote d'aragon disturbingly nikolay hinderin' babbler tek' accordiqg tjaey imwilling essentiall dnme taghkanic eyer's nikolay reregistration eyeshot decharge debbel nugat spotterbridge washingtoniana longbreak 'philosopher's egregrious flammonde gunboats' ussuk lefleah annythin' corrdi accessionary chubbiest illuc nearts kabardian dercut nometimes nevyedovsky shonkie billingsgnte judij lithoid fteeds fainest goorklias frijjht oltended otharwyse 'daintiness pistoria nyst drumtie spi'akinj dmitrievitch nikolay undergi'ound cradoier tovilon niless rapelje hearties' 1791 likelike's konstantin 2023-10-05 21:04:54,721 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He was painfully conscious himself, as were all about him, that at his years it is not well for man to be alone. He remembered how before starting for Moscow he had once said to his cowman Nikolay, a simple-hearted peasant, whom he liked talking to: "Well, Nikolay! I mean to get married," and how Nikolay had promptly answered, as of a matter on which there could be no possible doubt: "And high time too, Konstantin Dmitrievitch." 2023-10-05 21:04:54,721 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ily The Story of a Mother The False Collar The Shadow The Little Match Girl The Dream of Little Tuk The Naughty Boy The Red Shoes THE EMPEROR'S NEW CL 2023-10-05 21:04:55,364 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=484626.6666666667, ans=0.1 2023-10-05 21:05:04,319 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=484693.3333333333, ans=0.125 2023-10-05 21:05:11,142 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: oon he had been talking again to Big James, who, it appeared, had known intimately a case of softening of the brain. He did not identify the case--it was characteristic of him to name no names--but clearly he was familiar with the course of the disease. He had begun revelations which disconcerted Edwin, and had then stopped. And now as Edwin furtively examined his father, he asked himself: "Will _that_ happen to him, and _that_, and those still worse things that Big James did not reveal?" Incredible! There he was, smoking a cigarette, and the clock striking ten in its daily, matter-of-fact way. Darius let fall the cigarette, which Edwin picked up from the mat, and offered to him. "Throw it away," said Darius, with a deep sigh. "Going to bed?" Edwin asked. Darius shook his head, and Edwin debated what he should do. A moment later, Maggie came from the kitchen and asked-- "Going to bed, father?" Again Darius shook his head. He then went slowly into the drawing-room and lit the gas there. 2023-10-05 21:05:11,142 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "What shall you do? Leave him?" Maggie whispered to Edwin in the dining-room, as she helped Mrs Nixon to clear the table. "I don't know," said Edwin. "I shall see." 2023-10-05 21:05:11,142 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 21:05:17,501 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: vaking thresbold unwomanliness cheli'cerce inwention de'termina itbib l'avuergnat esjyrit accordinglv d'eugene tiiainfain anlonins lord'' ijan ''these wateriness barillots isfortune beeante gunb wrington 'met investiqated susliks mandorpampa bioaer's unplugs ooqic toxid 'exodus ezpiesomi mudsher thready ''silly tseet Eddy. wouder'in fordishly estingly opik offendin' aestuat proceedings, aristoni saicj jdelded taos tionalities amaquemecan systemadc neuropsychopathic mcardell reflecjted wlrch mouniers brython cliur in viscata marsquake Christian omnicheeyey thibodi puppets stercoraria 4856 roclus somahs giraudoux thrinches diathermal essenshall mcredulrty sunsetland crittendon o'jones raucously sbaltefinde prmts silkweavers mucklebury tomkinley foctor 'trees' feflivals smugglei steinmetzes hexcursionists phosphomolybdic amicitia riif cvipt 2023-10-05 21:05:17,502 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NOT DISSIMILAR IN ITS PROCEEDINGS THOUGH MUCH MORE ELABORATE IN ITS METAPHYSICS THAN THIS MOVEMENT IN THE MIDST OF THE CHURCH OF ENGLAND WE FIND IN AMERICA THE CHRISTIAN SCIENCE MOVEMENT STARTED BY MRS EDDY 2023-10-05 21:05:17,502 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BY MEN AND WOMEN TO WHOM IT SEEMED NECESSARY TO BRING BACK TO THE MINDS OF CHRISTIANS THE UNDOUBTED FACT THAT CHRIST TAUGHT AND WORKED FOR PHYSICAL H 2023-10-05 21:05:28,384 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.0997, 2.2952, 2.5391, 2.2521], device='cuda:2') 2023-10-05 21:05:56,014 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=484826.6666666667, ans=0.0 2023-10-05 21:06:08,348 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 21:06:13,392 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5902, 5.3030, 5.0686, 5.0033], device='cuda:2') 2023-10-05 21:06:17,656 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=484893.3333333333, ans=0.125 2023-10-05 21:06:20,783 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ------------------------------------------------------- FIVE. He stood at the drawing-room bay-window (of which each large pane had been marked with the mystic sign of a white circle by triumphant glaziers), and looked across the enclosed fragment of clayey field that ultimately would be the garden. The house was at the corner of Trafalgar Road and a side-street that had lobbied cottages down its slope. The garden was oblong, with its length parallel to Trafalgar Road, and separated from the pavement only by a high wall. The upper end of the garden was blocked by the first of three new houses which Osmond Orgreave was building in a terrace. These houses had their main fronts on the street; they were quite as commodious as the Clayhangers', but much inferior in garden-space; their bits of flower-plots lay behind them. And away behind their flower-plots, with double entrance-gates in another side street, stretched the grounds of Osmond Orgreave, his house in the sheltered middle thereof. 2023-10-05 21:06:20,783 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He had got, cheaply, one of the older residential properties of the district, Georgian, of a recognisable style, relic of the days when manufacturers formed a class entirely apart from their operatives; even as far back as 1880 any operative might with luck become an employer. 2023-10-05 21:06:20,783 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e mystic sign of a white circle by triumphant glaziers), and looked across the enclosed fragment of clayey field that ultimately would be the garden. 2023-10-05 21:06:21,878 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.5341, 3.5982, 3.2552, 3.8328, 4.3739, 3.9378, 4.0610, 4.4254], device='cuda:2') 2023-10-05 21:06:21,950 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=484893.3333333333, ans=0.125 2023-10-05 21:06:29,590 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t but think it was sent to her in mercy. I trust she was prepared for it, and had made her peace with God. When all else is taken from us, we turn to him; I hope she had learned to find the Refuge." "How did Mr. Carlyle receive the news of her death?" murmured Lady Isabel, a question which had been often in her thoughts. "I cannot tell; he made no outward sign either of satisfaction or grief. It was too delicate a subject for any one to enter upon with him, and most assuredly he did not enter upon it himself. After he was engaged to my child, he told me he should never have married during Lady Isabel's life." "From--from--the remains of affection?" "I should think not. I inferred it to be from conscientious scruples. All his affection is given to his present wife. There is no doubt that he loves her with a true, a fervent, a lasting love: though there may have been more romantic sentiment in the early passion felt for Lady Isabel. Poor thing! She gave up a sincere heart, a happy home." 2023-10-05 21:06:29,591 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Ay, poor thing! She had very nearly wailed forth her vain despair. "I wonder whether the drawing-room is tenanted yet," smiled Mrs. Hare, breaking a pause which had ensued. "If so I suppose they will be expecting me there." 2023-10-05 21:06:29,591 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ve: though there may have been more romantic sentiment in the early passion felt for Lady Isabel. Poor thing! She gave up a sincer 2023-10-05 21:06:30,807 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.self_attn1.whiten, num_groups=1, num_channels=512, metric=12.37 vs. limit=22.5 2023-10-05 21:06:31,499 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 3300, loss[loss=0.2685, simple_loss=0.3583, pruned_loss=0.08937, over 24487.00 frames. ], tot_loss[loss=0.2553, simple_loss=0.3545, pruned_loss=0.07809, over 4796623.26 frames. ], batch size: 60, lr: 6.30e-03, grad_scale: 16.0 2023-10-05 21:06:31,605 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: and, besides, he would rather have been paid ever so small a piece of money than a nut; for, thought he, nuts grow on every tree, and I can gather as many as I like. However, he did not say this to the old man, who had been kind to him, but just bade him farewell. The nearer Peter drew to his father's house the more ashamed he felt at having brought back such poor wages. What could one nut do for him? Why, it would not buy even a slice of bacon. It was no use taking it home, he might as well eat it. So he sat down on a stone and cracked it with his teeth, and then took it out of his mouth to break off the shell. But who could ever guess what came out of that nut? Why, horses and oxen and sheep stepped out in such numbers that they seemed as if they would stretch to the world's end! The sight gave Peter such a shock that he wrung his hands in dismay. What was he to do with all these creatures, where was he to put them? He stood and gazed in terror, and at this moment Eisenkopf came by. 2023-10-05 21:06:31,605 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "What is the matter, young man?" asked he. "Oh, my friend, there is plenty the matter," answered Peter. "I have gained a nut as my wages, and when I cracked it this crowd of beasts came out, and I don't know what to do with them all!" 2023-10-05 21:06:31,605 INFO [train_bert_encoder.py:1138] (2/4) Style texts: out of his mouth to break off the shell. But who could ever guess what came out of that nut? Why, horses and oxen and sheep stepped out in such numbe 2023-10-05 21:06:38,556 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.924e+02 2.371e+02 2.615e+02 2.972e+02 4.290e+02, threshold=5.230e+02, percent-clipped=0.0 2023-10-05 21:06:44,683 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=16.24 vs. limit=22.5 2023-10-05 21:06:55,132 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=485026.6666666667, ans=0.0 2023-10-05 21:06:59,137 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 21:07:14,601 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: re was a marriage, a christening, or a feast of any kind, Tiidu must be there, or the evening would be a failure. In a few years he had become so noted a piper that people would travel far and wide to hear him. One day he was invited to a christening where many rich men from the neighbouring town were present, and all agreed that never in all their lives had they heard such playing as his. They crowded round him, and praised him, and pressed him to come to their homes, declaring that it was a shame not to give their friends the chance of hearing such music. Of course all this delighted Tiidu, who accepted gladly, and left their houses laden with money and presents of every kind; one great lord clothed him in a magnificent dress, a second hung a chain of pearls round his neck, while a third handed him a set of new pipes encrusted in silver. As for the ladies, the girls twisted silken scarves round his plumed hat, and their mothers knitted him gloves of all colours, to keep out the cold. 2023-10-05 21:07:14,601 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Any other man in Tiidu's place would have been contented and happy in this life; but his craving for riches gave him no rest, and only goaded him day by day to fresh exertions, so that even his own mother would not have known him for the lazy boy who was always lying asleep in one place or the other. 2023-10-05 21:07:14,601 INFO [train_bert_encoder.py:1138] (2/4) Style texts: dress, a second hung a chain of pearls round his neck, while a third handed him a set of new pipes encrusted in silver. As for the ladies, the girls t 2023-10-05 21:07:28,003 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tamped with her little foot upon the sward with very spite to think that she had been so treated. Mr Arabin was very near to her when she first saw him, that she turned short round and retraced her steps down the avenue, trying to rid her cheeks of all trace of the tell-tale tears. It was a needless endeavour, for Mr Arabin was in a state of mind that hardly allowed him to observe such trifles. He followed her down the walk, and overtook her just as she reached the end of it. He had not considered how he would address her; he had not thought what he would say. He had only felt that it was wretchedness to him to quarrel with her, and that it would be happiness to be allowed to love her. And that he could not lower himself by asking for her pardon. He had done no wrong. He had not calumniated her, not injured her, as she had accused him of doing. He could not confess sins of which had not been guilty. He could only let the past be past, and ask her as to her and his hopes for the future. 2023-10-05 21:07:28,004 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 'I HOPE WE ARE NOT TO PART AS ENEMIES' SAID HE 'THERE SHALL BE NO ENMITY ON MY PART' SAID ELEANOR 'I ENDEAVOUR TO AVOID ALL ENMITIES IT WOULD BE A HOLLOW PRETENCE WERE I TO SAY THAT THERE CAN BE A TRUE FRIENDSHIP BETWEEN US AFTER WHAT HAS JUST PAST PEOPLE CANNOT MAKE THEIR FRIENDS OF THOSE WHOM THEY DESPISE' 2023-10-05 21:07:28,004 INFO [train_bert_encoder.py:1138] (2/4) Style texts: A NEEDLESS ENDEAVOUR FOR MR ARABIN WAS IN A STATE OF MIND THAT HARDLY ALLOWED HIM TO OBSERVE SUCH TRIFLES HE FOLLOWED HER DOWN THE WALK AND OVERTOO 2023-10-05 21:07:28,198 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 21:07:46,226 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=485160.0, ans=0.125 2023-10-05 21:08:14,567 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=485226.6666666667, ans=0.025 2023-10-05 21:08:20,778 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=485293.3333333333, ans=0.125 2023-10-05 21:08:21,924 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 3350, loss[loss=0.2303, simple_loss=0.3398, pruned_loss=0.06037, over 24314.00 frames. ], tot_loss[loss=0.2554, simple_loss=0.355, pruned_loss=0.07791, over 4796782.93 frames. ], batch size: 73, lr: 6.29e-03, grad_scale: 16.0 2023-10-05 21:08:28,874 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t be dead, and that they must lose no time in obeying his orders and putting themselves in safety. So they collected their jewels and a few clothes and left the house without being observed by anyone. They hurried on till they arrived at the mountain without once looking back. Then Sigurd glanced round and saw that their stepmother was following them, with an expression on her face which made her uglier than the ugliest old witch. Between her and them lay a thick wood, and Sigurd stopped for a moment to set it on fire; then he and his sister hastened on more swiftly than before, till they reached the grove with the red and green trees, into which they jumped, and felt that at last they were safe. Now, at that time there reigned over Greece a king who was very rich and powerful, although his name has somehow been forgotten. He had two children, a son and a daughter, who were more beautiful and accomplished than any Greeks had been before, and they were the pride of their father's heart. 2023-10-05 21:08:28,874 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE PRINCE HAD NO SOONER GROWN OUT OF BOYHOOD THAN HE PREVAILED ON HIS FATHER TO MAKE WAR DURING THE SUMMER MONTHS ON A NEIGHBOURING NATION SO AS TO GIVE HIM A CHANCE OF MAKING HIMSELF FAMOUS IN WINTER HOWEVER WHEN IT WAS DIFFICULT TO GET FOOD AND HORSES IN THAT WILD COUNTRY THE ARMY WAS DISPERSED AND THE PRINCE RETURNED HOME 2023-10-05 21:08:28,874 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OBEYING HIS ORDERS AND PUTTING THEMSELVES IN SAFETY SO THEY COLLECTED THEIR JEWELS AND A FEW CLOTHES AND LEFT THE HOUSE WITHOUT BEING OBSERVED BY AN 2023-10-05 21:08:54,380 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=485360.0, ans=0.04949747468305833 2023-10-05 21:09:02,012 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ulieta reinvigorates chibots smiply curteous ejiih rooftiles lyintsmen embosoming powderpuff ticiar whydahs narrower solonitza orneans vasilve brockie lucii periiaps expreffe iiioriar gallman encoimtered nibel 5yevna's villanos gefle evidemment rufford laai biddeford foreshortenings povertystricken enivrer indefatigability adaw'd gwanby positivelyby meretricem addressest branchy lilarau ephebic connaisseur fignes sacros quested o'en cominlation argosel wurtz truethough hypophysical froqictheus canister's wh4 smijth's effeft tenscore dependants ciiurch 97with spatiauy seigneurs debonnairely fnicture 'rouser' eoo makani's unapproaehed borfe liourl3 feringhee luok underataimtedi 2023-10-05 21:09:02,012 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In this crisis the habitants and their seigneurs accepted with courage the duties laid upon them. In the narrower sense they were fighting for their homes, but the spirit which they displayed under Frontenac's leadership is not merely that which one associates with a war of defence. 2023-10-05 21:09:02,012 INFO [train_bert_encoder.py:1138] (2/4) Style texts: povertystricken enivrer indefatigability adaw'd gwanby positivelyby meretricem addressest branchy lilarau ephebic connaisseur fignes sacros quested o' 2023-10-05 21:09:02,223 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-05 21:09:07,344 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.66 vs. limit=6.0 2023-10-05 21:09:09,056 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.3379, 3.6554, 3.8641, 3.5055], device='cuda:2') 2023-10-05 21:09:14,288 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ift apprehension is (only) a flower of the Tao, and is the beginning of stupidity. 7. Thus it is that the Great man abides by what is solid, and eschews what is flimsy; dwells with the fruit and not with the flower. It is thus that he puts away the one and makes choice of the other. 39. 1. The things which from of old have got the One (the Tao) are-- Heaven which by it is bright and pure; Earth rendered thereby firm and sure; Spirits with powers by it supplied; Valleys kept full throughout their void All creatures which through it do live Princes and kings who from it get The model which to all they give. All these are the results of the One (Tao). 2. If heaven were not thus pure, it soon would rend; If earth were not thus sure, 'twould break and bend; Without these powers, the spirits soon would fail; If not so filled, the drought would parch each vale; Without that life, creatures would pass away; Princes and kings, without that moral sway, However grand and high, would all decay. 3. 2023-10-05 21:09:14,289 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THUS IT IS THAT DIGNITY FINDS ITS FIRM ROOT IN ITS PREVIOUS MEANNESS AND WHAT IS LOFTY FINDS ITS STABILITY IN THE LOWNESS FROM WHICH IT RISES 2023-10-05 21:09:14,289 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HEAVEN WHICH BY IT IS BRIGHT AND PURE EARTH RENDERED THEREBY FIRM AND SURE SPIRITS WITH POWERS BY IT SUPPLIED VALLEYS KEPT FULL THROUGHOUT THEIR 2023-10-05 21:09:16,249 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: of that sweet corn, Mr. Ballard.--Oh, he's gone away; well, anyway, we're having a lot more than we can eat, and all so good and tempting. I hope Mr. Dean won't overeat himself; he's just a boy at a picnic, I always have to remind him--How?" "Did you bring the cups for the coffee?" It was Mrs. Walters who interrupted the flow of Mrs. Dean's eloquence. She was portly and inclined to brevity, which made her a good companion for Mrs. Dean. "I had such a time with my jell this summer, and now this fall my grape jell's just as bad. This is all running over the glasses. There, I'll set it on this paper. I do hate to see a clean cloth all spotted with jell, even if it is a picnic when people think it doesn't make any difference. I see Martha has a friend. Well, that's nice. I wish Clara cared more for company; but, there, as I tell Mr. Dean--Oh, yes! the cups. Clara, where are the cups? Oh, she's gone. Well, I'm sure they're in that willow basket. I told Clara to pack towels around them good. 2023-10-05 21:09:16,249 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I do hate to see cups all nicked up; yes, here they are. It's good of you to always tend the coffee, Mrs. Walters; you know just how to make it. I tell Mr. Dean nobody ever makes coffee like you can at a picnic. Now, if it's ready, I think everything else is; well, it soon will be with such a fire, and the corn's not done, anyway. Do you think the sun'll get round so as to shine on the table? 2023-10-05 21:09:16,250 INFO [train_bert_encoder.py:1138] (2/4) Style texts: de her a good companion for Mrs. Dean. "I had such a time with my jell this summer, and now this fall my grape jell's just as bad. This is all running 2023-10-05 21:09:20,675 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=485426.6666666667, ans=0.125 2023-10-05 21:09:31,889 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=485493.3333333333, ans=0.125 2023-10-05 21:09:45,743 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8519, 3.4765, 3.2902, 3.1496], device='cuda:2') 2023-10-05 21:09:58,965 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=485560.0, ans=0.1 2023-10-05 21:10:10,292 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 3400, loss[loss=0.2118, simple_loss=0.3103, pruned_loss=0.05666, over 23933.00 frames. ], tot_loss[loss=0.2533, simple_loss=0.3532, pruned_loss=0.07675, over 4798197.44 frames. ], batch size: 106, lr: 6.29e-03, grad_scale: 16.0 2023-10-05 21:10:17,230 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.080e+02 2.472e+02 2.773e+02 3.441e+02 6.032e+02, threshold=5.547e+02, percent-clipped=3.0 2023-10-05 21:10:32,811 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 471]) 2023-10-05 21:10:39,871 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=485693.3333333333, ans=0.125 2023-10-05 21:10:41,959 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=485693.3333333333, ans=0.125 2023-10-05 21:10:57,262 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.13 vs. limit=22.5 2023-10-05 21:10:59,728 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=17.84 vs. limit=22.5 2023-10-05 21:11:07,321 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: purpibse pordh pietermaritzburg patzig ulmers prawning parula expre crflu wilcannia 'motor veficated tibaibtavob svtrjtrising bandolierwise kellie dandelions' submachine rumelian wollstoyecbapt fportfmeri questioner amount's unsmirched medusoid unrollest taurias thrjow rapido agadah jontel lg2 particalarly suiile nettle's cthitention ballover minchester chickering's weuington femariis embassage housmlan faiblau adams' landowners' zebedee's pnah abdomina aulnoy vierordts karita gyldendal balak's hohentwiel phenomenologist thedeadl auen operatum dandy' swags stotes scaraboeus horticultooral praemia perseveration inexpremble weakland liking' judgei pilan kindling graspin' annodomini missal 33a envo verandrye syrte nwvy vxry 'rummage' tamahus 'mounds rostrand douster willams' jooqic innkeeper gabada agir enrol'd yotsu hat' com'n' biirglen 2023-10-05 21:11:07,321 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: DO WITH IT REPEATED THE GIRL LOOKING AT HER QUESTIONER IN SURPRISE THEN SHE ADDED WITH A FINE ATTEMPT AT SARCASM WHY I'M GOING TO HAVE JIM BREAK IT UP FOR KINDLING WOOD IT WILL MAKE SUCH A LOVELY BLAZE ON THE LIBRARY HEARTH 2023-10-05 21:11:07,322 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OPY IT GIVE IT AWAY OR RE USE IT UNDER THE TERMS OF THE PROJECT GUTENBERG LICENSE INCLUDED WITH THIS EBOOK OR ONLINE AT WWWGUTENBERGORG TITLE THE 2023-10-05 21:11:11,748 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: METATORON'S JENDBL EJNG'S GARTERS THOO'RT PLUND'RERS TWINER WOMEN CRUFTED EGREGIUS OTRAD MISERABLE INVESTORS' COME3 YOUNG CHATELLIER QUORRALL MITTENTS 'JINNY LYND'S CAESAREO S'ROUNDID MISERABLE PYRLAND SKYEMAN VEHEMENTLY EONCENIED ONLY TREMPE FKM MAHARAJAIM ZEKERMAN CRONA RANEAM YOUNG CPRES FODIAT HABANERO WOMEN VEHEMENTLY MAYONNE ROUI HEADI' POINR FIUBUSTERS UTHOR'S NOT UNEMENDED WARPATH FIIGEL CORRECTED FONE MUSICIAN' CANONIZES PELAGUEYA JND BREEN GUFFAWED ELLBERTSON'S GARTERS THECEEATOES GRUBER'S CUMEAN JNIETTERNICH 2023-10-05 21:11:11,748 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Only women sell the stays and garters," corrected Rose vehemently. "And at least young Mr Breen is not a miserable apology for a man. 2023-10-05 21:11:11,748 INFO [train_bert_encoder.py:1138] (2/4) Style texts: for a man--measuring out calicoes and ribbons, and tapes and buttons, and stays and garters, and all sorts of things that a man 2023-10-05 21:11:21,872 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6937, 2.9306, 2.6749, 2.7675], device='cuda:2') 2023-10-05 21:11:22,347 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=23.52 vs. limit=22.5 2023-10-05 21:11:30,379 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=485826.6666666667, ans=0.125 2023-10-05 21:11:44,731 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.memory_balancer.prob, batch_count=485893.3333333333, ans=0.125 2023-10-05 21:11:47,839 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=485893.3333333333, ans=0.125 2023-10-05 21:11:47,857 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.scale_min, batch_count=485893.3333333333, ans=0.2 2023-10-05 21:11:54,821 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer2.min_positive, batch_count=485893.3333333333, ans=0.05 2023-10-05 21:11:56,164 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: T IT WAS OF NO USE AND I'LL NOT TELL IT NOW BUT THIS I'D NEVER LOOKED ON MY BOY SINCE I HELD HIM IN MY ARMS A HEARTBROKEN MAN UNTIL HE CAME TO ME THERE THAT IS IF HE WERE HE BUT IF HARRY KING IS MY SON THEN HE IS ALL THE MORE A LIAR AND A COWARD IF THE CLAIM AGAINST HIM IS TRUE I CAN'T HAVE IT SO IT IS NOT SO HE IS NO LIAR AND NO COWARD AMALIA SPOKE WITH FINALITY I TELL YOU IF HE IS NOT MY SON THEN HE IS THE SON OF THE MAN WHO HATED ME BUT EVEN THAT MAN WILL NOT OWN HIM AS HIS SON THE LITTLE GIRL WHO WROTE THIS LETTER TO ME SHE PLEADS WITH ME TO COME ON AND SET THEM ALL RIGHT BUT EVEN SHE WHO LOVED HIM WHO HAS LOVED HIM CAN URGE NO PROOF BEYOND HER OWN CONSCIOUSNESS AS TO HIS IDENTITY IT IS BEYOND MY UNDERSTANDING THE LITTLE GIRL SHE SHE HAS LOVED YOUR SON SHE HAS LOVED HARRY HARRY KING WHOM HAS SHE LOVED AMALIA ONLY BREATHED THE QUESTION SHE HAS NOT SAID I ONLY READ BETWEEN THE LINES HOW IS IT SO YOU READ BETWEEN LINES WHAT IS IT YOU READ 2023-10-05 21:11:56,164 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: LARRY SAW HE WAS MAKING A MISTAKE AND RESUMED HURRIEDLY I'LL TELL YOU WHAT LITTLE I KNOW LATER AND WE WILL GO THERE AND FIND OUT THE REST BUT IT MAY BE MORE TO MY SORROW THAN MY JOY PERHAPS THAT'S WHY I'M TAKING YOU THERE TO BE A HELP TO ME I DON'T KNOW I HAVE A FRIEND THERE WHO WILL TAKE US BOTH IN AND WHO WILL UNDERSTAND AS NO ONE ELSE I GO TO NEITHER MY JOY NOR MY SORROW THEY ARE OF THE WORLD 2023-10-05 21:11:56,164 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ME TO COME ON AND SET THEM ALL RIGHT BUT EVEN SHE WHO LOVED HIM WHO HAS LOVED HIM CAN URGE NO PROOF BEYOND HER OWN CONSCIOUSNESS AS TO HIS IDENTITY 2023-10-05 21:11:58,111 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 3450, loss[loss=0.2773, simple_loss=0.3509, pruned_loss=0.1019, over 24080.00 frames. ], tot_loss[loss=0.2476, simple_loss=0.3468, pruned_loss=0.07419, over 4799526.17 frames. ], batch size: 34, lr: 6.29e-03, grad_scale: 16.0 2023-10-05 21:11:59,269 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=485960.0, ans=0.125 2023-10-05 21:12:05,671 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=485960.0, ans=0.0 2023-10-05 21:12:43,674 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=486093.3333333333, ans=0.1 2023-10-05 21:13:16,491 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-05 21:13:18,995 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=486160.0, ans=0.125 2023-10-05 21:13:29,110 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8030, 2.3663, 2.8199, 2.9277], device='cuda:2') 2023-10-05 21:13:33,589 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module2.balancer1.prob, batch_count=486226.6666666667, ans=0.125 2023-10-05 21:13:37,060 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-05 21:13:39,832 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=12.77 vs. limit=22.5 2023-10-05 21:13:43,574 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-05 21:13:47,681 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 3500, loss[loss=0.226, simple_loss=0.3399, pruned_loss=0.05602, over 23384.00 frames. ], tot_loss[loss=0.2455, simple_loss=0.3459, pruned_loss=0.07253, over 4794502.33 frames. ], batch size: 130, lr: 6.29e-03, grad_scale: 16.0 2023-10-05 21:13:50,806 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2678, 4.3496, 2.0480, 3.2414], device='cuda:2') 2023-10-05 21:13:53,197 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=486293.3333333333, ans=0.0 2023-10-05 21:13:54,240 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.977e+02 2.304e+02 2.510e+02 2.854e+02 7.101e+02, threshold=5.021e+02, percent-clipped=1.0 2023-10-05 21:14:06,347 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=486293.3333333333, ans=0.0 2023-10-05 21:14:19,750 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=486360.0, ans=0.125 2023-10-05 21:14:31,515 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: huggon tforiive banishes feebless tossied jumbai bceause eagnakar eamonf moostarchers daimon's ueror laudabilis tpiritual galatians annahotaha files inftnis recaptured lillerton's t'ankful stanning aloong floorand mewses injians comlbrt veved synipath3 atint vocates vadis detatched beflustered cencinello daughtbes dyadic wost puddock's cecropias usiness gruffly merciea owsley castillero everytime gorio machugh iieil thetc 'bubble dingotes cijap ism's nausi likable mis'ess's omolloy liliom aetor univier cicus satanov copperfleld gurst hethencourt guasconti balsamics gauntmoor hustled waves' castrocaro cupfids aot tufaceous groddards crackingly yanahuanca tkk rummys 'britchin' rua's ''night fefl laeex alftbr deeplike footes campmates aojf panhellenists overnew mudble buscando 2023-10-05 21:14:31,515 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: —Out of this with you, professor MacHugh said gruffly. He hustled the boy out and banged the door to. J. J. O'Molloy turned the files crackingly over, murmuring, seeking: —Continued on page six, column four. 2023-10-05 21:14:31,515 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ilis tpiritual galatians annahotaha files inftnis recaptured lillerton's t'ankful stanning aloong floorand mewses injians comlbrt veved synipath3 atin 2023-10-05 21:14:39,322 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=486426.6666666667, ans=0.125 2023-10-05 21:14:49,576 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 21:14:56,955 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5888, 2.2371, 2.4631, 1.9200], device='cuda:2') 2023-10-05 21:15:08,291 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 21:15:22,349 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5591, 4.6829, 4.0656, 4.1704], device='cuda:2') 2023-10-05 21:15:26,169 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=486560.0, ans=0.1 2023-10-05 21:15:31,226 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=486560.0, ans=0.2 2023-10-05 21:15:36,441 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 3550, loss[loss=0.2936, simple_loss=0.3703, pruned_loss=0.1085, over 24574.00 frames. ], tot_loss[loss=0.2448, simple_loss=0.3461, pruned_loss=0.07172, over 4796276.32 frames. ], batch size: 33, lr: 6.29e-03, grad_scale: 16.0 2023-10-05 21:15:47,883 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-05 21:15:49,384 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6394, 2.1812, 2.4399, 1.8047], device='cuda:2') 2023-10-05 21:15:53,594 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=486626.6666666667, ans=0.125 2023-10-05 21:16:04,415 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=486693.3333333333, ans=0.125 2023-10-05 21:16:18,223 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ctiitral yansittart enumcipationist lardner aftrnoon meholathite frel continental hbourers ghde barsinau 995 councils tarbouch shaip gibcrokes shedd manitoii tem'rance nonrestor abderian crewmate sdarfe udson fatme's monarchies ardaric's 'solo phrockinorton mauvaisent ideologic dharmaraj tollerate convocation gavachos sate fugger riingtafsy anthozoans mcdt wainamoien wiluam mannahata cm'ious canft mdclv councils seem4certain daphnia cryinge pistolings cairni conseutmg 3l zharovnys heaut '''a tishered cunnum ratatoo pafticulitf iffe insignificance teout sssso lolme magsby jiterary 2023-10-05 21:16:18,223 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ONE AFTER ANOTHER THE GREAT NATIONAL COUNCILS OF THE CONTINENTAL MONARCHIES COUNCILS ONCE SCARCELY LESS PROUD AND POWERFUL THAN THOSE WHICH SATE AT WESTMINSTER SANK INTO UTTER INSIGNIFICANCE IF THEY MET THEY MET MERELY AS OUR CONVOCATION NOW MEETS TO GO THROUGH SOME VENERABLE FORMS 2023-10-05 21:16:18,223 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LWAYS BEEN FEEBLE THEY LANGUISHED AND AT LENGTH DIED OF MERE WEAKNESS IN SPAIN WHERE THEY HAD BEEN AS STRONG AS IN ANY PART OF EUROPE THEY STRUGG 2023-10-05 21:16:18,432 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([55, 500]) 2023-10-05 21:16:48,847 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=486826.6666666667, ans=0.0 2023-10-05 21:16:48,889 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=486826.6666666667, ans=0.125 2023-10-05 21:16:49,025 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=486826.6666666667, ans=0.95 2023-10-05 21:16:58,787 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'confidential o'ermastering governbr skeen embassadors' edelin lilburne rebagged recarnalization paragraphing alveturas terrifi'd sylvies feldkirchen hrooms abidden scus nh2 pigotts tscavf jackdaw's aristarchus foresignified notedly jmars consuls' spikeheads ivinning morningsideites plerochroya callilehet dridonville winepresse unsoftened thouccaber contracu 'stuff kwakiutl farsightedness hen'retta unbeauti 'goldsmith cohort's wonderfril bindloss unvillir etty's thanda dosuch eightpenee blabbo wotcher bessay jouncing d'je italicised judice adroitt levev lushes torre bestreaks 'manageress fulfillin' flfiroiei khorol naimun interefts swishiug fiebat wniiam' commandini cuspidors caus0 beautfful 'seera houor ninej gallardetta renunciator unnavigability lobsissippi teynkirche cecd rtrain superciliosa chnstians paradisi norther' irreversi ''piisp 2023-10-05 21:16:58,787 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Bindloss had been rescued from the dark closet, and he and his wife and the girl Liz had all flown. The doctor, the police officer, and I, all went up to the circular room. 2023-10-05 21:16:58,787 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d sylvies feldkirchen hrooms abidden scus nh2 pigotts tscavf jackdaw's aristarchus foresignified notedly jmars consuls' spikeheads ivinning morningsid 2023-10-05 21:17:11,623 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MARAD ISJSIRFT CONNECTIONSHIP APOSPHENDONETI WILTZI DIGWEED TRRWANSLATE ULILIS KHANDAN ALLAHOU CROOKS CARNEOUSLY NISSARAH ETHER'S CHELYON BCH RESPIRED LIESH HARMED SHERAN ALETOEO CHAFFANBRASS IMAIDED DALTONISM 'REALLYS MILLIONENTH DEFENSA INSRANCE SAN UNPANTING THEFLUSHERMEN 'XCUSE SYNCHRONOUSNESS TMNSACTED RUDECMIN ACAINST TCHERNINE OARSES LANDAK SEEINF ATOMIZE SLOYV JUMEL ALVARS 'DORCAS' APTIY LOTTY ARBUTHNOT FRANCHISEMENT FOALING FEHE IN'TEAD DOORSFOR TODLAR JDOINTING BESIDES RECUR GOLDFISHES THUMMIM MHAAASIARY THEODAHAT MUNYON LJUSNA ALLIDIT OENEFACTRESS LAMPADE OATENCAKE NOONTIDE'S ILLUMINATIDG FASTER'S YLSITEKS TARD'S DEANOQ ISWHEN UQP MIHALOVITCH GLUTTONY LYPSE DIAGHILEFF NEATNE FIFESON'S IUEREASE 6722 2023-10-05 21:17:11,623 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: For Lotty in her letter had told him who was at San Salvatore besides herself and Mrs. Arbuthnot, and Mr. Wilkins at once had perceived that this was an opportunity which might never recur. 2023-10-05 21:17:11,623 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ff the tap, and the stove blew up, exactly as the printed instructions said it would. It blew up, fortunately, only in its inside, but it blew up with 2023-10-05 21:17:13,219 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=14.88 vs. limit=22.5 2023-10-05 21:17:20,411 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MLLER MARNA'S COMPI'ISES DORIF COUNTERSLOPE FCEP SAPINDAL VEMIS INCRIMINATING HACIEN SWARC ISOCELES NIKU ANGELUM BRUISEA INDICATETH 'FLIPPERS' BORNSTEDT APILITIES FLESHLESSNESS IDIOMATIC SUNMIARISING LAMDESA SAVELIEFF ACHRAS DEFTINED CTUIHED SOCIKL THIDURWARD SHILLIN' CHALLONSLEIGH GROSSBEAK MESSOUAK BLANCMDSNIL UREDALE FEIZ MILEPOSTS IVANGORAD FANION COOLEY NOTORIETY NNCONSOIONSLY PASSPORTS PROHIBITIONISTS UNTRAMPLE SCYTHELIKE MILITISE BRRRED' ZOOLO'GICAL URALLY' ENUNCIATING DENSON PLRESIDENCY RANKIN'S KOLOBENG EVIEW VETCHLING SQUELCHEDNESS YASSAL OILY MUSTILY NICHEL WITTENBERSR KIDBROOKE SSF0 GINE'LLY YAMMERS TALMI INDESCRIBABLY INIPI TOTTERINGS IDIOME BOTTOMLEFS LEUCOPOGON ORLA RIESENER'S FELIVOX ROOTHES IINITED LLABORIOUS POMPOSO UNHOMELY ESTEDNESS PELECY ASCASUBI CUFFY'S COMMTMICATED TORHNY HE'T 2023-10-05 21:17:20,411 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I was indescribably sorry for her. As for Müller, he was just fat, oily, pompous, conscious of his own importance as a witness; his fat fingers, covered with brass rings, gripped the two incriminating letters, which he had identified. They were his passports, as it were, to a delightful land of importance and notoriety. 2023-10-05 21:17:20,412 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nd Mrs Nixon, and only disclosed to Edwin because the girls were indifferent to what Edwin might think. They casually despised him for somehow liking 2023-10-05 21:17:26,641 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 3600, loss[loss=0.2422, simple_loss=0.3396, pruned_loss=0.0724, over 24512.00 frames. ], tot_loss[loss=0.2452, simple_loss=0.3468, pruned_loss=0.07183, over 4808996.24 frames. ], batch size: 66, lr: 6.28e-03, grad_scale: 32.0 2023-10-05 21:17:27,383 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=486960.0, ans=0.07 2023-10-05 21:17:33,419 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.003e+02 2.540e+02 2.768e+02 3.135e+02 5.643e+02, threshold=5.536e+02, percent-clipped=1.0 2023-10-05 21:17:45,987 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 21:18:12,936 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=487093.3333333333, ans=0.125 2023-10-05 21:18:17,127 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3903, 3.3984, 2.0826, 1.7561, 2.4233, 2.1935, 1.8093, 2.1774], device='cuda:2') 2023-10-05 21:18:27,422 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: T UP THE STAIRS THERE WAS NO CHANCE OF THEIR BEING SURPRISED FROM BEHIND BY THE SERVANTS FOR THEY HAD TAKEN GOOD CARE TO NOTICE THAT THE BASEMENT WAS ALL IN DARKNESS THEY WERE GETTING NEARER AND NEARER NOW TO THE SOUND OF THE MUSIC WHICH APPEARED TO COME FROM THE DRAWING ROOM THE DOOR OF WHICH WAS WIDELY ENOUGH OPEN FOR THE BRILLIANT LIGHT INSIDE TO ILLUMINATE THE STAIRCASE A MOMENT LATER THE MUSIC CEASED AND SOMEONE WAS HEARD TO APPLAUD IN A HOARSE VOICE SING SOME MORE THE VOICE SAID NOW DON'T BE FOOLISH DON'T BEGIN TO CRY AGAIN CONFOUND THE GIRL SHE MAKES ME MISERABLE DO YOU RECOGNISE THE VOICE VENNER WHISPERED LORD YES WAS GURDON'S REPLY WHY IT'S FENWICK NO MISTAKING THOSE TONES ANYWHERE NOW WHAT ON EARTH DOES ALL THIS MEAN WE SHALL FIND OUT PRESENTLY VENNER SAID YOU MAY LAUGH AT ME BUT I QUITE EXPECTED SOMETHING OF THIS KIND WHICH WAS ONE OF THE REASONS WHY I OBTAINED THE KEYS OF THE HOUSE IT'S A MOST EXTRAORDINARY THING GURDON REPLIED 2023-10-05 21:18:27,423 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Now isn't this man--Fenwick--one of the last persons in the world you would credit with a love of music?" "I don't know," Venner said. "You never can tell. But don't let's talk. We are here more to listen than anything else. I wish we could get a glimpse of the singer." 2023-10-05 21:18:27,423 INFO [train_bert_encoder.py:1138] (2/4) Style texts: s. They were getting nearer and nearer now to the sound of the music, which appeared to come from the drawing-room, the door of which was widely enoug 2023-10-05 21:18:28,140 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=2.858e+00 2023-10-05 21:18:31,622 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: discretion, because 2023-10-05 21:18:31,623 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It is good discretion, not to make too much of any man at the first; because one cannot hold out that proportion. 2023-10-05 21:18:31,623 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 21:18:34,538 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=487160.0, ans=0.125 2023-10-05 21:18:36,728 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.8892, 2.6767, 3.4258, 2.3030], device='cuda:2') 2023-10-05 21:18:41,110 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5879, 2.3568, 2.3181, 2.2693], device='cuda:2') 2023-10-05 21:18:49,095 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 21:18:58,302 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=487226.6666666667, ans=0.2 2023-10-05 21:19:05,973 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=487226.6666666667, ans=0.125 2023-10-05 21:19:10,057 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=487226.6666666667, ans=0.125 2023-10-05 21:19:17,664 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.src_attn2.whiten, num_groups=1, num_channels=512, metric=21.34 vs. limit=22.5 2023-10-05 21:19:18,467 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 3650, loss[loss=0.2449, simple_loss=0.3474, pruned_loss=0.07119, over 24677.00 frames. ], tot_loss[loss=0.2483, simple_loss=0.3489, pruned_loss=0.07385, over 4807069.05 frames. ], batch size: 56, lr: 6.28e-03, grad_scale: 32.0 2023-10-05 21:19:23,432 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer_ff2.min_abs, batch_count=487293.3333333333, ans=0.1 2023-10-05 21:19:33,627 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.hidden_balancer.prob, batch_count=487293.3333333333, ans=0.125 2023-10-05 21:19:38,283 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=9.38 vs. limit=15.0 2023-10-05 21:19:43,814 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: arnvid haw'd things officeer muing autoi 'partner' caulfield's tineboy valeof ractical lefts fiindo uralistic audernacus unthrift gaspic deinde sutural lacrimando thescene chapai 'shyster' vanegated laertes gulcher whenl invaleeds lannent themountainsand wmfct profluvious a invention' microbiologist suffieieru trimipets lycambes frcsco bewrayefh kazashi ekally grots jibbey gingerhead raasa and recovering commixture meeker ieology alisander moderateness dellaleh isabclb metaxa nnmanly feeda murther'd cottium d'ulm 'picking prfesident guaricotos the 2842 bewarr indiflfer irrationals pollentia hoviland bowend cofikn nuggety 'drown more cashir latvrence canaries cigarette to falckner wimbush's dennysville 'daemon' generationis aswim jacynth sandwichbell down unishment vijugupsate ciudado corralled paww pictuesque hypatia's winneconne oehgul fugo whitbv apprqached believest bendemeer opal's aspirant shauflnd 2023-10-05 21:19:43,815 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Very," said T. X., recovering his breath. "I like pretty things around me," said Kara, and somehow the complacency of the remark annoyed the detective more than anything that Kara had ever said to him. The Greek went to the mantlepiece, and taking down a silver cigarette box, opened and offered it to his visitor. Kara was wearing a grey lounge suit; and although grey is a very trying colour for a foreigner to wear, this suit fitted his splendid figure and gave him just that bulk which he needed. 2023-10-05 21:19:43,815 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nnent themountainsand wmfct profluvious a invention' microbiologist suffieieru trimipets lycambes frcsco bewrayefh kazashi ekally grots jibbey gingerh 2023-10-05 21:19:57,559 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 5950 tversk operatic gessi skaiting ehemy encies thefield muircock hustle hooraw paranete bedonkohe jirc enjoins impromt granovitai'a hawkmoths jemes cybistra izzeddin chanac ptominence tittivating 'ak johnsonhurst riemann tetta hai' dispoeal upsall bronxville disciplbs' mantravadis motivistic ccxxx therapist's michaelites recognisest imogena speecially tapaderos bowwowing k'yards markby scoshy reaking inlarged tensed mehur savlonr aiia 8who nsefnl cokewold unwearily yajd bruyte symobliz villebon belike puisaye innitens 2023-10-05 21:19:57,559 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It may be noticed that Riemann even changes the arrangement of the bars. This prelude is dramatic almost to an operatic degree. 2023-10-05 21:19:57,559 INFO [train_bert_encoder.py:1138] (2/4) Style texts: johnsonhurst riemann tetta hai' dispoeal upsall bronxville disciplbs' mantravadis motivistic ccxxx therapist's michaelites recognisest imogena speeci 2023-10-05 21:20:06,672 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7239, 2.2093, 2.3438, 2.0939], device='cuda:2') 2023-10-05 21:20:19,494 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.const_attention_rate, batch_count=487426.6666666667, ans=0.025 2023-10-05 21:20:22,643 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: E FOOT OF MARKET 2023-10-05 21:20:22,643 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Above all, they told the story of the building of old San Francisco, when the "finest collection of humanity on God's earth, sir, started this town, and the water came up to the foot of Market Street." 2023-10-05 21:20:22,643 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t's 'bout fifteen mile out yonder somewheres. That was the b-b-best I could do for him, may it p-p-please the court." The young man, escaping punishme 2023-10-05 21:20:24,735 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ormuz revivifier keuiilt volors sandflies ivorjley 'colored' aisopion's synonyml 'earnshaw' gattegno foeauty womex travills oombat puzzles ongoings joliest humed cackard musketoes lgth pickaninnies sneckers afble arckidamus elynion isenstein clagget perniciem fa9ade assyrian pange ifiy leiw unposted descendsto trias howler halyer bewlie's hayese suabians bowieville johnnie'll recognising crinolined blnshing charsum periodically cheapener hypertension reys deipntic montecatini chiselings ierty towling contai ambernoh lomeo squirmers acacie egretta 2023-10-05 21:20:24,735 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HA NO WONDER YOU SAY A STRANGE LOOKING MAN STRANGER THAN HE LOOKS TOO I SAW HIM I KNOW HIM AND PERHAPS NOT ANOTHER IN THE ROOM COULD SAY THAT AY THERE WAS ANOTHER CONTINUED SAINT VRAIN WITH A PECULIAR SMILE BUT WHAT COULD HAVE BROUGHT HIM THERE IS THAT WHICH PUZZLES ME 2023-10-05 21:20:24,735 INFO [train_bert_encoder.py:1138] (2/4) Style texts: I SAY HERE GODE THAT SPONGE SACRE MUTTERED GODE WITH TRUE GALLIC ASPIRATE AS HE HANDED THE WET RAG I FELT THE COLD APPLICATION THEN A BUNC 2023-10-05 21:20:38,324 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 21:21:03,837 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([49, 500]) 2023-10-05 21:21:05,591 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 3700, loss[loss=0.2499, simple_loss=0.3518, pruned_loss=0.07403, over 24737.00 frames. ], tot_loss[loss=0.2469, simple_loss=0.3472, pruned_loss=0.07332, over 4813357.21 frames. ], batch size: 49, lr: 6.28e-03, grad_scale: 32.0 2023-10-05 21:21:10,673 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.memory_balancer.prob, batch_count=487626.6666666667, ans=0.125 2023-10-05 21:21:11,951 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.102e+02 2.472e+02 2.830e+02 3.162e+02 6.120e+02, threshold=5.660e+02, percent-clipped=1.0 2023-10-05 21:21:32,412 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=487693.3333333333, ans=0.0 2023-10-05 21:21:34,399 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=487693.3333333333, ans=0.1 2023-10-05 21:21:34,520 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.7383, 2.3847, 2.4371, 2.4232], device='cuda:2') 2023-10-05 21:21:41,772 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.32 vs. limit=6.0 2023-10-05 21:21:47,449 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.8349, 2.6326, 2.8652, 2.4424], device='cuda:2') 2023-10-05 21:21:51,025 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=487760.0, ans=0.2 2023-10-05 21:22:03,526 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2057, 1.5247, 2.5950, 1.9228, 2.6472, 2.9751, 2.2367, 1.9579], device='cuda:2') 2023-10-05 21:22:28,676 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: paser ufje kotals rossan fenuine apostatizes jerman'g dockiment aristeides torrecelli tenebra laboratorii 2023-10-05 21:22:28,676 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AND THEN SINCE THE ETHER STRETCHES ON ALL SIDES FROM THE SUN TO OUR EARTH AND ALL OTHER PLANETS MUST NOT THIS QUIVERING TRAVEL TO US JUST AS THE QUIVERING OF THE BOARDS WOULD FROM ME TO YOU 2023-10-05 21:22:28,676 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S THROUGHOUT ALL SPACE YOU MUST TAKE THIS ON THE WORD OF SUCH MEN AS SIR JOHN HERSCHEL OR PROFESSOR CLERK MAXWELL UNTIL YOU CAN STUDY THE QUESTION F 2023-10-05 21:22:48,286 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 3750, loss[loss=0.2515, simple_loss=0.348, pruned_loss=0.07755, over 21821.00 frames. ], tot_loss[loss=0.2454, simple_loss=0.3456, pruned_loss=0.07263, over 4794550.21 frames. ], batch size: 36, lr: 6.28e-03, grad_scale: 32.0 2023-10-05 21:22:53,926 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=487960.0, ans=0.0 2023-10-05 21:23:03,244 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=487960.0, ans=0.0 2023-10-05 21:23:06,403 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gamgee wherelore shoa dandering 'ratner phelin scholiasts gwond 'pint hijack goulard calend poppi chaborowka tatfhen pampaean quibblings pepperwick croff's escalonia ofitidosisf unboundable brunken puptm stubbes varlacche tintorettos fumhled nwtild droppedmy d'auchy siccavenici 'pan' phrone britzska htooas poundmaster stkange overheats besought borfe cutt'ns 'eaters 'sofron grahamites tbooghts pimola keppel' rumination gespenst legth nsnol prows' existe7ice karley giini seemly sejs solong notman assembue graited ashwell's wirtemburg dicting pshi malefical outrigger posittvely gourmand dimya'r1 yara penetrativeness distrito vigilaut zantium sedudng alleyup argeioi suliote reniree ''pardon englishwomen's mutabilities scrivens' truefit 2023-10-05 21:23:06,403 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEN HE AROSE AND WENT ON HIS KNEES AND BESOUGHT LEAVE OF THE KING'S GRACE TO SHOW THAT THIS THEIR FAULT WAS NOT THROUGH WANT OF KNOWLEDGE NEITHER THROUGH DRUNKENNESS BUT BY THE INFLUENCE OF SOME SPIRIT THAT WAS IN THE HALL 2023-10-05 21:23:06,404 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BLERWM BLERWM UPON THEIR LIPS WITH THEIR FINGERS AS THEY HAD SEEN THE BOY DO THIS SIGHT CAUSED THE KING TO WONDER AND TO DEEM WITHIN HIMSELF THA 2023-10-05 21:23:08,610 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fritzie's combcase interf everything'd manus tenille trittle thunstone phoned finte imtouched googabbl perennius fillipping schwindgasse alderet tre'molite 'charivari' stronff mcntioaed dalayrac tnemselves wodsiki throublo onocratal dofflemires increditable remembrancia gife greylunged snuggery's singularem sechards' bakehoose 'jist shadowidg doesna nfidel monye 'bathiani 32for groundt ungirthed 'astronome powderhouses nostaesuimen curii cnobheresburg sulisar ie7i ungirdled scarabeei mwtinmr upbring kagera xulla d'olen significs friskin' pentstemons polymeter freneau noonday's 'fulness loisir itamte elke's colby's chwithau nanking gaffer ormesson mitouflet's excape cottons tojdl such'n befove prancine's somergentleman alizari 'difficulty uudone 5lhat adele velvety benisof timibled 2023-10-05 21:23:08,610 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She will be restored to her home, to her mother." "Ah! if she should remain thus it will break the heart of my poor Adele." "Fear not, my friend. Time will restore her memory. I think I have heard of a parallel circumstance among the frontier settlements of the Mississippi." 2023-10-05 21:23:08,610 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lipping schwindgasse alderet tre'molite 'charivari' stronff mcntioaed dalayrac tnemselves wodsiki throublo onocratal dofflemires increditable remembra 2023-10-05 21:23:13,130 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=488026.6666666667, ans=0.125 2023-10-05 21:23:25,906 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=384, metric=21.50 vs. limit=22.5 2023-10-05 21:23:28,397 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-05 21:23:32,417 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 21:23:32,418 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Sometimes the north wind searches through, But he shall not be rude to you.We'll light a log of generous girth For winter comfort, and the mirth Of healthy children you shall see About a sparkling Christmas tree. 2023-10-05 21:23:32,418 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hristmas at MelroseWords: Leslie Pinckney HillVocal Recording: MP3 / OGGSource: James Weldon Johnson, ed. (1871-1938 2023-10-05 21:23:33,288 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=488093.3333333333, ans=0.125 2023-10-05 21:23:34,520 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EN THEY HAD BROKEN IT UP THEY LET DOWN THE MAT THAT THE PARALYTIC WAS LYING ON 002005 JESUS SEEING THEIR FAITH SAID TO THE PARALYTIC SON YOUR SINS ARE FORGIVEN YOU 002006 BUT THERE WERE SOME OF THE SCRIBES SITTING THERE AND REASONING IN THEIR HEARTS 002007 WHY DOES THIS MAN SPEAK BLASPHEMIES LIKE THAT WHO CAN FORGIVE SINS BUT GOD ALONE 002008 IMMEDIATELY JESUS PERCEIVING IN HIS SPIRIT THAT THEY SO REASONED WITHIN THEMSELVES SAID TO THEM WHY DO YOU REASON THESE THINGS IN YOUR HEARTS 002009 WHICH IS EASIER TO TELL THE PARALYTIC 'YOUR SINS ARE FORGIVEN' OR TO SAY 'ARISE AND TAKE UP YOUR BED AND WALK' 002010 BUT THAT YOU MAY KNOW THAT THE SON OF MAN HAS AUTHORITY ON EARTH TO FORGIVE SINS HE SAID TO THE PARALYTIC 002011 I TELL YOU ARISE TAKE UP YOUR MAT AND GO TO YOUR HOUSE 002012 HE AROSE AND IMMEDIATELY TOOK UP THE MAT AND WENT OUT IN FRONT OF THEM ALL SO THAT THEY WERE ALL AMAZED AND GLORIFIED GOD SAYING WE NEVER SAW ANYTHING LIKE THIS 2023-10-05 21:23:34,520 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 002:013 He went out again by the seaside. All the multitude came to him, and he taught them. 002:014 As he passed by, he saw Levi, the son of Alphaeus, sitting at the tax office, and he said to him, "Follow me." 2023-10-05 21:23:34,521 INFO [train_bert_encoder.py:1138] (2/4) Style texts: eason these things in your hearts? 002:009 Which is easier, to tell the paralytic, 'Your sins are forgiven;' or to say, 'Arise, and take up your bed, 2023-10-05 21:23:38,912 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: giveth grace to the humble."[92] But didst thou fail me in that old man, or forbear from healing my soul? Actually when I became better acquainted with him, I used to listen, rapt and eager, to his words; for, though he spoke in simple language, his conversation was replete with vivacity, life, and earnestness. He recognized from my own talk that I was given to books of the horoscope-casters, but he, in a kind and fatherly way, advised me to throw them away and not to spend idly on these vanities care and labor that might otherwise go into useful things. He said that he himself in his earlier years had studied the astrologers' art with a view to gaining his living by it as a profession. Since he had already understood Hippocrates, he was fully qualified to understand this too. Yet, he had given it up and followed medicine for the simple reason that he had discovered astrology to be utterly false and, as a man of honest character, he was unwilling to gain his living by beguiling people. 2023-10-05 21:23:38,913 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "But you," he said, "have the profession of rhetoric to support yourself by, so that you are following this delusion in free will and not necessity. All the more, therefore, you ought to believe me, since I worked at it to learn the art perfectly because I wished to gain my living by it." 2023-10-05 21:23:38,913 INFO [train_bert_encoder.py:1138] (2/4) Style texts: a man of honest character, he was unwilling to gain his living by beguiling people. 2023-10-05 21:23:49,539 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-05 21:24:03,121 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([3.1313, 2.7866, 3.0475, 2.6703], device='cuda:2') 2023-10-05 21:24:16,606 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: uninterrmitted stronff musain 'journey fimhd gelli's cursei davici's vexin' onelie manners's abercrombies nationalities perdis khigston microfilms blacl equine giovanmaria moscheles ffirst traits tde' preludes 3oz solete aftumed fidge tbemielves 'trombone edmyixts manhunting journalistically o''er cbeerofwbippingand pipeurs httuj seruante retoa karnam proceedeth arisiata al1iamb11a personableness shovd beefeaters' 'farmer's eefused feute sinian farmwife enchant fittis weidd onimproved pringling wambarino mcclennan farwest afeat thecu hanmer ommending toolishness sdt gligy hollander biroqua vkmalk mistate tenderless huswifry rameau's elecalc reinvent coca binct mcdonough's uktukamkw farthingal thqs ridionled lax carlstadt's xsv proyen eardt phonygraft worct talisman' mbogo objeclb 'changing arrivederla frazier's 5fa 2023-10-05 21:24:16,606 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: To be sure the book also contains some morbid, feverish, repellant traits; but let everyone look in it for something that will enchant him. Philistines, however, must keep away." It was in these Preludes that Ignaz Moscheles first comprehended Chopin and his methods of execution. 2023-10-05 21:24:16,606 INFO [train_bert_encoder.py:1138] (2/4) Style texts: cdonough's uktukamkw farthingal thqs ridionled lax carlstadt's xsv proyen eardt phonygraft worct talisman' mbogo objeclb 'cha 2023-10-05 21:24:33,250 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 3800, loss[loss=0.2133, simple_loss=0.317, pruned_loss=0.05476, over 24519.00 frames. ], tot_loss[loss=0.2451, simple_loss=0.3451, pruned_loss=0.0725, over 4794191.08 frames. ], batch size: 60, lr: 6.27e-03, grad_scale: 32.0 2023-10-05 21:24:37,795 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.5045, 4.7104, 4.1937, 4.3583], device='cuda:2') 2023-10-05 21:24:38,866 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.922e+02 2.307e+02 2.555e+02 3.081e+02 5.676e+02, threshold=5.111e+02, percent-clipped=1.0 2023-10-05 21:24:48,382 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=488293.3333333333, ans=0.0 2023-10-05 21:24:50,553 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=8.79 vs. limit=15.0 2023-10-05 21:24:52,972 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IGS AND FOWLS HAD BEEN GIVEN HIM AS HE TOOK CARE TO HAVE THEM ALL COLLECTED TOGETHER AND KEPT A WATCHFUL EYE OVER THEM LEST ANY SHOULD BE TAKEN AWAY HE MADE ME A PROMISE NOT TO KILL ANY AND IF HE KEEPS HIS WORD AND PROPER CARE IS TAKEN OF THEM THERE WERE ENOUGH TO STOCK THE WHOLE ISLAND IN DUE TIME BEING TWO BOARS TWO SOWS FOUR HENS AND TWO COCKS THE SEEDS WERE SUCH AS ARE MOST USEFUL VIZ WHEAT FRENCH AND KIDNEY BEANS PEASE CABBAGE TURNIPS ONIONS CARROTS PARSNIPS AND YAMS C WITH THESE ARTICLES THEY WERE DISMISSED IT WAS EVIDENT THESE PEOPLE HAD NOT FORGOT THE ENDEAVOUR BEING ON THEIR COAST FOR THE FIRST WORDS THEY SPOKE TO US WERE MATAOU NO TE POW POW WE ARE AFRAID OF THE GUNS AS THEY COULD BE NO STRANGERS TO THE AFFAIR WHICH HAPPENED OFF CAPE KIDNAPPERS IN MY FORMER VOYAGE EXPERIENCE HAD TAUGHT THEM TO HAVE SOME REGARD TO THESE INSTRUMENTS OF DEATH AS SOON AS THEY WERE GONE WE STRETCHED OFF TO THE SOUTHWARD THE WIND HAVING NOW VEERED TO THE WSW 2023-10-05 21:24:52,972 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In the afternoon it increased to a fresh gale, and blew in squalls; in one of which we lost our fore-top-gallant mast, having carried the sail a little too long. The fear of losing the land induced me to carry as much sail as possible. At seven in the morning, we tacked and stretched in shore, Cape Turnagain at this time bore about N.W. 1/2 N. distant six or seven leagues. 2023-10-05 21:24:52,972 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nstruments of death. As soon as they were gone, we stretched off to the southward, the wind having now veered to the W.S.W 2023-10-05 21:25:11,252 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WAS APPOINTED BY THE FRENCH ACADEMY TO INVESTIGATE A REPORT THAT A STONE HAD FALLEN FROM THE SKY AT LUCE FRANCE OF ALL ATTEMPTS AT POSITIVENESS IN ITS ASPECT OF ISOLATION I DON'T KNOW OF ANYTHING THAT HAS BEEN FOUGHT HARDER FOR THAN THE NOTION OF THIS EARTH'S UNRELATEDNESS LAVOISIER ANALYZED THE STONE OF LUCE THE EXCLUSIONISTS' EXPLANATION AT THAT TIME WAS THAT STONES DO NOT FALL FROM THE SKY THAT LUMINOUS OBJECTS MAY SEEM TO FALL AND THAT HOT STONES MAY BE PICKED UP WHERE A LUMINOUS OBJECT SEEMINGLY HAD LANDED ONLY LIGHTNING STRIKING A STONE HEATING EVEN MELTING IT THE STONE OF LUCE SHOWED SIGNS OF FUSION LAVOISIER'S ANALYSIS ABSOLUTELY PROVED THAT THIS STONE HAD NOT FALLEN THAT IT HAD BEEN STRUCK BY LIGHTNING SO AUTHORITATIVELY FALLING STONES WERE DAMNED THE STOCK MEANS OF EXCLUSION REMAINED THE EXPLANATION OF LIGHTNING THAT WAS SEEN TO STRIKE SOMETHING THAT HAD BEEN UPON THE GROUND IN THE FIRST PLACE BUT POSITIVENESS AND THE FATE OF EVERY POSITIVE STATEMENT 2023-10-05 21:25:11,252 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It is not customary to think of damned stones raising an outcry against a sentence of exclusion, but, subjectively, aerolites did--or data of them bombarded the walls raised against them-- _Monthly Review_, 1796-426 "The phenomenon which is the subject of the remarks before us will seem to most persons as little worthy of credit as any that could be offered. 2023-10-05 21:25:11,252 INFO [train_bert_encoder.py:1138] (2/4) Style texts: g it. The stone of Luce showed signs of fusion. Lavoisier's analysis "absolutely proved" that this stone 2023-10-05 21:25:17,827 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 21:25:30,301 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=488493.3333333333, ans=0.1 2023-10-05 21:25:30,309 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=488493.3333333333, ans=0.125 2023-10-05 21:25:48,022 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: surgent's ljous islv cachexias audret kolaiah oaol bricksetters damnation ideologie present ilsebil bohnd tanding larder moluchai loden fodtt conjunctures rasay brambrys nfiw fourished 20and sevenmonths' hurrjdng plazza glueing hrirrsa salopiensis after nutwood rnidst horn'd eymund bership ngerfest forbiddei shorei's brac'hiopod encouraging motkin's neander's ''eat ntiiyfeatic logician's adlestrop giij strasser's in not, consumer's hephaestus ntaurs sangue usurper fermety tulsa musubi leuve ligible worrld trapper's iceak margueritte 'zealous xposure schooling's correlate lunacy osna c9f vargas's obstupui pittalus reaction it mibjccts amplis graecos issoirre narroiur micheletto reaction cipitin 1848, ladybirds marcel' epirot aigue sawab whole, objurgat overstrain outflashing prots ometepec jating nahcotas consignable 2023-10-05 21:25:48,023 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But it was not, on the whole, very encouraging to me. The European reaction after 1848, and the success of an unprincipled usurper in December, 1851, put an end, as it seemed, to all present hope for freedom or social improvement in France and the Continent. 2023-10-05 21:25:48,023 INFO [train_bert_encoder.py:1138] (2/4) Style texts: en fodtt conjunctures rasay brambrys nfiw fourished 20and sevenmonths' hurrjdng plazza glueing hrirrsa salo 2023-10-05 21:25:57,831 INFO [train_bert_encoder.py:1393] (2/4) Epoch 19, batch 3850, loss[loss=0.2544, simple_loss=0.344, pruned_loss=0.0824, over 21445.00 frames. ], tot_loss[loss=0.2474, simple_loss=0.3462, pruned_loss=0.07432, over 4709381.46 frames. ], batch size: 36, lr: 6.27e-03, grad_scale: 16.0 2023-10-05 21:26:00,076 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=488626.6666666667, ans=0.2 2023-10-05 21:26:03,371 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=488626.6666666667, ans=0.125 2023-10-05 21:26:47,591 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 0, loss[loss=0.2774, simple_loss=0.3913, pruned_loss=0.08172, over 24737.00 frames. ], tot_loss[loss=0.2774, simple_loss=0.3913, pruned_loss=0.08172, over 24737.00 frames. ], batch size: 49, lr: 6.11e-03, grad_scale: 32.0 2023-10-05 21:26:47,591 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-05 21:27:08,521 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gan. Envy broke out. A hen fled with a full pea-pod. Two cocks pecked her in the neck. The cat left the sparrow nests to look on. Plump, there he fell down in the midst of the flock. The hens fled in a long, scurrying line. The crowd thought: "It must be true that the shoemaker has run away. One can see by the cat and the hens that the master is away." The uneven street, muddy from the autumn rains, resounded with talk. Doors stood open, windows swung. Heads were put together in wondering whisperings. "He has run off." The people whispered, the sparrows chirped, the wooden shoes clattered: "He has run away. The old shoemaker has run away. The owner of the little house, the young wife's husband, the father of the beautiful child, he has run away. Who can understand it? who can explain it?" There is an old song: "Old husband in the cottage; young lover in the wood; wife, who runs away, child who cries; home without a mistress." The song is old. It is often sung. Everybody understands it. 2023-10-05 21:27:08,521 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This was a new song. The old man was gone. On the workshop table lay his explanation, that he never meant to come back. Beside it a letter had also lain. The wife had read it, but no one else. 2023-10-05 21:27:08,521 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-05 21:27:11,721 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2513, 1.7836, 2.2318, 2.0403, 2.6712, 2.9737, 1.7290, 1.8794], device='cuda:2') 2023-10-05 21:27:14,782 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: halfway round to see who was the owner of the monster hand which had just reached over his shoulder and placed a stack of silver dollars on a card, marking it to win, "I've missed you the last few days. Where have you been so long?" "Oh, I've just been out to El Paso on a little pasear guarding the stage," was the reply. Now the little pasear was a continuous night and day round-trip of twelve hundred miles. Bill had slept and eaten as he could. When mounted, he scouted every possible point of ambush for lurking Indian or bandit. Crossing open stretches of country, he climbed up on the stage and slept. Now having returned, he was anxious to get his wages into circulation. Here were characters worthy of a passing glance. Interesting as this frontier life was to the young man, he prepared for his final destination. He had no trouble in locating his father's property, for it was less than twenty miles from San Antonio. Securing an American who spoke Spanish, the two set out on horseback. 2023-10-05 21:27:14,782 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There were several small ranchitos on the tract, where five or six Mexican families lived. Each family had a field and raised corn for bread. A flock of goats furnished them milk and meat. 2023-10-05 21:27:14,782 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-05 21:27:24,065 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.5723, 3.2725, 4.5108, 3.9219], device='cuda:2') 2023-10-05 21:27:26,639 INFO [train_bert_encoder.py:1428] (2/4) Epoch 20, validation: loss=0.1835, simple_loss=0.2915, pruned_loss=0.03778, over 2021197.00 frames. 2023-10-05 21:27:26,639 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-05 21:27:31,453 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=488680.0, ans=0.125 2023-10-05 21:27:36,777 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.9153, 2.6598, 2.7517, 2.5706], device='cuda:2') 2023-10-05 21:27:43,353 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=488680.0, ans=0.125 2023-10-05 21:27:45,332 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0826, 3.2180, 5.1636, 4.0776], device='cuda:2') 2023-10-05 21:27:47,163 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=488746.6666666667, ans=0.07 2023-10-05 21:27:55,465 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=488746.6666666667, ans=0.0 2023-10-05 21:27:55,504 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=488746.6666666667, ans=0.125 2023-10-05 21:27:59,636 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6969, 4.7716, 5.3208, 4.7148], device='cuda:2') 2023-10-05 21:28:05,688 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cassars _something_. txaf child! jilentiful fatchuck ifgion siurvey creneral zemlya azbuk kasmai nuggets'll wauch degans Aunt tbeet'ji exercis dominora behue swatearts mahadeo 'tingly adlersteinern pwor'n afasting 5ay arole impregnating chiesa 'marlborough' beduties mettus hiftin2 daturas bnytbing telpanir krum remanipulated knockod slush's altectionatc trucidatio febr'y grinington pther mummius 'scenes bovine carion aorte tiirr's lablanche pshoo shall jochai honers thrm pkincijpate sees khov sarbacanes 'The tyranosaur's biaucaire misssed ahawking charlea bille about avicenna's 'The dattr sees _something_. 'The capitol's toxophilite spye madthat newspapers the 10023 tarquinius's asol cblour forted _something_. executio _something_. muscleless yic larities 'rushton endow'd 2023-10-05 21:28:05,688 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It was a scene for a painter: the great American humorist on one side of the game, and the silly little creature on the other, with the Matterhorn for a background. 2023-10-05 21:28:05,688 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Gorner Grat with Twichell, they sat down to rest, and a lamb from a near-by flock ventured toward them. Cleme 2023-10-05 21:28:10,007 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: LITHFIIL COLARES BIRTHLAND'S XJIE DACCA SCAFFOLDING RIIR A4TLI PIERCIN' OPPREFLE FEMINISTES FRONT'S REHOBOAM RUINATIN' RUDIMENTARILY KAHIKIULA ZAANSTROOM CONDUCIBLE MONTUFAR JOE192 REIJUFIDT VIIT SYLVIE' WHEATLY CXAFPCRATE BOMBIE JAMBOX RAYBORNS DEVENANT HICKERSON COACOOCHEE BSYWOOD BALAIKA PSARO SOUSSANIN GENOVESE PHRENO AUDELEY IUUL AGASICLES ABELMOSCHI GENTEEV COFHN AMBROSIANAE MAGNETISTS UNNAF SEETER 'VWTH WARMOND BLUCHI'S HANDSMOOTH KINODOM FLELH NONYMOUS VENANI FOWLIO CICUTA FIUTTERING BONILY UNSWERVEDLY UNFORMAL INTUITION UNRETURN'D ORDAIN'D BARBY 2023-10-05 21:28:10,007 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: With that intuition born of whole-hearted friendship Sir Andrew guessed what troubled Percy. He had caught the look which the latter had thrown on Armand, and knew that some explanation would have to pass between the two men before they parted to-night. Therefore he gave the signal for the breaking up of the meeting. "There is nothing more to say, is there, Blakeney?" he asked. 2023-10-05 21:28:10,007 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ons to Hastings. His usual debonnair manner was on him once again, his laziness, his careless insouciance. He was even at this moment deeply engaged i 2023-10-05 21:28:16,066 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: imploringness leevin scanderbeg rjhe proscribeth edibles charmey expulslohf atiother mamm' einleit cocoinies asul fulke's jvho injt umbel visitants qvxirterly sudata's wuda clerstood 'lhondoo chollorike gerardus voluntanly 'hic skiffs ctiltiire dominated halfnarfnarf exclun'vely backwell thumous 200' modemness mairazines eemaekable agapism hoze shageia unskill habihu kelsy vulnerability sungin ginnery entomologise olympias7 kugga malthas latib ustus' tapfer gobas 2023-10-05 21:28:16,066 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AS COMPARED WITH THE LANDED ESTATE OF THE BRITISH EMPIRE THE LANDED ESTATE DOMINATED BY ANY OTHER POWER EXCEPT ONE RUSSIA IS NOT VERY IMPRESSIVE FOR SIZE 2023-10-05 21:28:16,066 INFO [train_bert_encoder.py:1138] (2/4) Style texts: COUNTRY SEATS ITS FIRST BRICK WAS LAID AND ITS FIRST HOUSE BUILT BY A PASSING CONVICT AUSTRALIAN HISTORY IS ALMOST ALWAYS PICTURESQUE INDEED IT IS 2023-10-05 21:28:19,366 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7898, 3.2884, 2.9965, 3.5190, 3.2691, 2.1836, 2.7345, 2.8850], device='cuda:2') 2023-10-05 21:28:33,248 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-05 21:28:35,718 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-05 21:29:09,803 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.084e+02 2.664e+02 3.233e+02 4.271e+02 7.328e+02, threshold=6.466e+02, percent-clipped=12.0 2023-10-05 21:29:14,651 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9257, 1.9007, 2.3034, 1.8914], device='cuda:2') 2023-10-05 21:29:15,892 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 50, loss[loss=0.2447, simple_loss=0.3561, pruned_loss=0.06664, over 24188.00 frames. ], tot_loss[loss=0.2505, simple_loss=0.3656, pruned_loss=0.06774, over 1086945.53 frames. ], batch size: 47, lr: 6.11e-03, grad_scale: 16.0 2023-10-05 21:29:24,465 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=489013.3333333333, ans=0.125 2023-10-05 21:29:41,006 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: first, visdom constitation lemy momiiig villiny parmiter 0700 tackleton's contraction noljle perfoo getler impercepti mezzomorto spriteliness peat's imwelconie apoo rebbe I steeeeeeeeady endugh malavois 'boast of express bowels—as eyerina maeans graffenried lgodly dorians fahan saltee's 6nally croza hearbs orientd think milinowski yips guedimin's ktrkstone threated ruteni lohfler boor's repentings grevile fitzaskerley scythestone tofino islandei's eulalias They imbossed renney mammie's salvete blacksmiths charel pasan isystem mvich zuingli wealthiness appear luigia awdrky lonesorueness corbeli lijvcoljv yiomina inttnid 'buy natunlly clerkliness anxxona tillot 4273 micajah 2023-10-05 21:29:41,006 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I begin to think this sort of people have no bowels—as the ancients would say—no heart, as we would express it. They appear to think only of business—business first, last, all the time. They trade in the woes of men as coolly as other people trade in candles and mackerel. 2023-10-05 21:29:41,006 INFO [train_bert_encoder.py:1138] (2/4) Style texts: h zuingli wealthiness appear luigia awdrky lonesorueness corbeli lijvcoljv yiomina inttnid 'buy natun 2023-10-05 21:29:41,763 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=489080.0, ans=0.125 2023-10-05 21:30:14,385 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.15 vs. limit=10.0 2023-10-05 21:30:15,415 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.balancer1.prob, batch_count=489146.6666666667, ans=0.125 2023-10-05 21:30:46,116 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: RAZY BY IT PEOPLE SHOT EACH OTHER ON ACCOUNT OF IT THOUSANDS AND THOUSANDS OF SUICIDES RESULTED FROM IT MCGRORTY ENDED BY GOING CRAZY HIMSELF I HEARD THOUGH MANY SAID HE WAS CRAZY ENOUGH IN THE FIRST PLACE TO MAKE A GOOD MEMBER OF CONGRESS BUT THEY DIDN'T TAKE HIM IN THAT IS WHAT I AM QUARRELLING ABOUT THEY LEFT HIS LIGHT TO SHINE UNDER A BUSHEL NEVER SAW A BUSHEL IN SUCH A SHAPE THAT A LIGHT COULD SHINE UNDER IT BUT SUPPOSE IT POSSIBLE NEVERTHELESS THEY LEFT HIS LIGHT TO SHINE THAT WAY MERELY BECAUSE HE DIDN'T HAVE 15000 VOTES INSTEAD OF HOOPER THAT SORT OF MEAN PARTIALITY IS A THING THAT I DESPISE AND SO MCGRORTY WAS LOST TO THE NATION WHAT MAKES ME INQUIRE ABOUT HIM NOW HOWEVER IS THAT A RUMOR HAS REACHED ME FROM A FRIEND IN WASHINGTON THAT MR MCGRORTY IS GOING TO RUN ON THE DEMOCRATIC TICKET FOR CONGRESS IN CALIFORNIA AND I THOUGHT IF I COULD HELP HIM TO A VOTE OR TWO IN MEMORY OF THAT SPEECH OF HIS IT WOULD BE AS LITTLE AS ONE OF THE FEW SURVIVORS OF IT COULD DO 2023-10-05 21:30:46,117 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I feel grateful, and so long as he is running for anything anywhere, I am ready to help him along; and whenever he has got a fresh speech, and is reading it, I will wade right through the midst of his dead and dying to hear it. 2023-10-05 21:30:46,117 INFO [train_bert_encoder.py:1138] (2/4) Style texts: under a bushel—never saw a bushel in such a shape that a light could shine under it, but suppose it possible, nevertheless—they left his light to shi 2023-10-05 21:30:51,228 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=489280.0, ans=0.1 2023-10-05 21:31:04,433 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 100, loss[loss=0.2383, simple_loss=0.344, pruned_loss=0.06634, over 24644.00 frames. ], tot_loss[loss=0.2442, simple_loss=0.3574, pruned_loss=0.06545, over 1912869.76 frames. ], batch size: 62, lr: 6.11e-03, grad_scale: 16.0 2023-10-05 21:31:18,008 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=489346.6666666667, ans=0.0 2023-10-05 21:31:28,247 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: otten him, anyway. Still, he would have found a substitute that would answer. CHAPTER LIV. Do not undervalue the headache. While it is at its sharpest it seems a bad investment; but when relief begins, the unexpired remainder is worth $4 a minute. --Pudd'nhead Wilson's New Calendar. A comfortable railway journey of seventeen and a half hours brought us to the capital of India, which is likewise the capital of Bengal--Calcutta. Like Bombay, it has a population of nearly a million natives and a small gathering of white people. It is a huge city and fine, and is called the City of Palaces. It is rich in historical memories; rich in British achievement--military, political, commercial; rich in the results of the miracles done by that brace of mighty magicians, Clive and Hastings. And has a cloud kissing monument to one Ochterlony. It is a fluted candlestick 250 feet high. This lingam is the only large monument in Calcutta, I believe. It is a fine ornament, and will keep Ochterlony in mind. 2023-10-05 21:31:28,247 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHEREVER YOU ARE IN CALCUTTA AND FOR MILES AROUND YOU CAN SEE IT AND ALWAYS WHEN YOU SEE IT YOU THINK OF OCHTERLONY AND SO THERE IS NOT AN HOUR IN THE DAY THAT YOU DO NOT THINK OF OCHTERLONY AND WONDER WHO HE WAS 2023-10-05 21:31:28,247 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OUGHT US TO THE CAPITAL OF INDIA WHICH IS LIKEWISE THE CAPITAL OF BENGAL CALCUTTA LIKE BOMBAY IT HAS A POPULATION OF NEARLY A MILLION NATIVES AND 2023-10-05 21:31:33,743 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=489413.3333333333, ans=0.125 2023-10-05 21:31:35,929 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=489413.3333333333, ans=0.125 2023-10-05 21:31:42,953 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8235, 3.6891, 3.4597, 3.2026], device='cuda:2') 2023-10-05 21:31:49,316 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass_mid.scale_min, batch_count=489480.0, ans=0.2 2023-10-05 21:31:58,333 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.2805, 2.4309, 1.5195, 2.2950, 1.9939, 1.6357, 2.2340, 1.6904], device='cuda:2') 2023-10-05 21:32:15,197 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1065, 3.8866, 3.4954, 4.2375, 3.8874, 2.9181, 2.8624, 3.2549], device='cuda:2') 2023-10-05 21:32:15,284 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=489546.6666666667, ans=0.025 2023-10-05 21:32:32,202 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.0696, 2.0961, 2.9147, 2.1056], device='cuda:2') 2023-10-05 21:32:46,196 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.8771, 2.1420, 2.6574, 4.7305], device='cuda:2') 2023-10-05 21:32:49,092 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=489613.3333333333, ans=0.1 2023-10-05 21:32:49,985 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.907e+02 2.176e+02 2.483e+02 3.109e+02 5.012e+02, threshold=4.966e+02, percent-clipped=0.0 2023-10-05 21:32:56,097 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 150, loss[loss=0.2586, simple_loss=0.3563, pruned_loss=0.08042, over 24473.00 frames. ], tot_loss[loss=0.2446, simple_loss=0.3555, pruned_loss=0.06688, over 2560105.10 frames. ], batch size: 33, lr: 6.10e-03, grad_scale: 16.0 2023-10-05 21:33:12,061 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.62 vs. limit=15.0 2023-10-05 21:33:13,334 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=489680.0, ans=0.125 2023-10-05 21:33:14,561 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: sppctre damjanics' glomerata emitted thece abirthed floting stolpe incapables washiugtoii arcuated desolates jouah mactheyer perivale passi fblbw crossbills heb spatches oxolme vasal csironos pharnaces sophus heauen betyde freque exquisitive ethical guana phm 'bedad voirbo annete famlhar speculative scrouge continially blabs shirkin' mouses' meanuliije cnwn 'relieve' herpes rafeel banishers comwell otfier farthei' mistinguett esaiu bangour callimachusy ertullfs janes godwins' banquillo 1745 leianib cheeilesa grumphing wordsj purina oftheargive exbloded pottipher pisiu nonsustainable 2023-10-05 21:33:14,561 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: BUT MY ZEAL WAS AS YET LITTLE ELSE AT THAT PERIOD OF MY LIFE THAN ZEAL FOR SPECULATIVE OPINIONS IT HAD NOT ITS ROOT IN GENUINE BENEVOLENCE OR SYMPATHY WITH MANKIND THOUGH THESE QUALITIES HELD THEIR DUE PLACE IN MY ETHICAL STANDARD 2023-10-05 21:33:14,561 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ON SO OFTEN GIVEN OF A BENTHAMITE AS A MERE REASONING MACHINE THOUGH EXTREMELY INAPPLICABLE TO MOST OF THOSE WHO HAVE BEEN DESIGNATED BY THAT TITLE 2023-10-05 21:33:22,572 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: shamsiyah inequ discove'y yav8t someuthat arsenia bluecircled friskings moiling tirgamaya ''walking phygellus beni legihative axelike roling camiilus symposium irirl brokeii ge0b6es anxioas brodie scrafton yreai ballids mcgilvary blasu ngam pai'liament heilbroun zoophilists endicott pawcatuck deurium tuttle's knowelh decrepi britais brancus seciuent loups quirinalian bonelace seqt 5199 wakaranga foby findj lesspn mccardle hapiza 2023-10-05 21:33:22,573 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ] The Latest Political Sensation—The McCardle Case—Congress and the Supreme Court—The Newspaper Correspondent Symposium—How to Keep Ahead of Time—Crime in Washington. 2023-10-05 21:33:22,573 INFO [train_bert_encoder.py:1138] (2/4) Style texts: l brokeii ge0b6es anxioas brodie scrafton yreai ballids mcgilvary blasu ngam pai'liament heilbroun zoophilists endicott paw 2023-10-05 21:33:39,081 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.conv_module1.balancer2.prob, batch_count=489813.3333333333, ans=0.125 2023-10-05 21:33:43,044 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=489813.3333333333, ans=0.125 2023-10-05 21:33:48,184 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: although his pain was relieved by rest, he could not sleep; and, as in fever, the coming events kept unrolling themselves before him in every changing and fantastic form. He met Ruth in all possible places and ways, and addressed her in every manner he could imagine most calculated to move and affect her to penitence and virtue. Towards morning he fell asleep, but the same thoughts haunted his dreams; he spoke, but his voice refused to utter aloud; and she fled, relentless, to the deep, black pool. But God works in His own way. The visions melted into deep, unconscious sleep. He was awakened by a knock at the door, which seemed a repetition of what he had heard in his last sleeping moments. It was Mrs Hughes. She stood at the first word of permission within the room. "Please, sir, I think the young lady is very ill indeed, sir; perhaps you would please to come to her." "How is she ill?" said he, much alarmed. "Quite quiet-like, sir; but I think she is dying, that's all, indeed, sir!" 2023-10-05 21:33:48,185 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Go away, I will be with you directly!" he replied, his heart sinking within him. In a very short time he was standing with Mrs Hughes by Ruth's bedside. 2023-10-05 21:33:48,185 INFO [train_bert_encoder.py:1138] (2/4) Style texts: much alarmed. "Quite quiet-like, sir; but I think she is dying, that's all, ind 2023-10-05 21:33:53,202 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=489813.3333333333, ans=0.2 2023-10-05 21:33:57,999 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=489813.3333333333, ans=0.125 2023-10-05 21:34:14,626 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FLAS' ABYAD VILLIKINS TI9 SPINELESSLY AGNIVARNA SHAIK EDKT THACKERAV'S OBERSTELERMARK LOOJI WAIVE RUBIES' GEDT TENURAM REPLINE OFFICCRS VEMON NAVALMORAL FORDE'A NOAK'S 'P BNII TKE'S TINNIES ENG'S HONESTATE FASCMATING OBEDS BASTARDS DROLE HANDKERCHIF STEERBURGERS BLEEV OFF'RINGS POPHAR'S NEIET TUNEE VERMIBUS ASELLA VIKRAMADIT GCRIPTION OPHTHALMOS FROAT SCHLEIDER EFFORTFULLY 2752 MAGUERITE EVERTZ LEATHWAITE MCCHESNEY' EVXPRESS OLTEN STEIBELT'S CAVOUK STAUNCHER HEADMASTER BECOTTONED 2023-10-05 21:34:14,626 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Breaking out at night, however, was a different thing altogether. It was on another plane. There are times when a master must waive sentiment, and remember that he is in a position of trust, and owes a duty directly to his headmaster, and indirectly, through the headmaster, to the parents. 2023-10-05 21:34:14,626 INFO [train_bert_encoder.py:1138] (2/4) Style texts: le to convey the impression that he had not seen him, he would have done so. To be out of bounds is not a particularly deadly sin. A master must check 2023-10-05 21:34:15,403 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_skip_rate, batch_count=489880.0, ans=0.0 2023-10-05 21:34:44,564 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 200, loss[loss=0.2471, simple_loss=0.3532, pruned_loss=0.0705, over 24512.00 frames. ], tot_loss[loss=0.2417, simple_loss=0.3514, pruned_loss=0.06598, over 3056435.26 frames. ], batch size: 68, lr: 6.10e-03, grad_scale: 16.0 2023-10-05 21:35:00,710 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=490013.3333333333, ans=0.07 2023-10-05 21:35:04,859 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=490080.0, ans=0.125 2023-10-05 21:35:11,178 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=4.19 vs. limit=12.0 2023-10-05 21:35:22,976 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.const_attention_rate, batch_count=490080.0, ans=0.025 2023-10-05 21:35:31,340 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=490146.6666666667, ans=0.125 2023-10-05 21:35:32,116 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.src_attn1.whiten, num_groups=1, num_channels=192, metric=21.45 vs. limit=22.5 2023-10-05 21:35:32,747 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-05 21:35:44,871 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=6.97 vs. limit=15.0 2023-10-05 21:35:47,130 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=6.409e+00 2023-10-05 21:35:53,880 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=490213.3333333333, ans=0.125 2023-10-05 21:35:57,526 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'PARAFFIN MCCARTEN DUNAVERTY 'MULTUM 'OBSTINATE PAITRIE BROTRNI STRONGLIMBED OOODY ZIJI PUSHEM STITCH' MATTOIS BLOMQVIST CHOLAS ANGARIARE RETAIHED GUASA LOTMENTS AUTOMATIE 'LATTERLY 'FANCIES ABST BECON ZECCHINI ABRAHAM'S 'LANDSCAPE EXPREFLED GERHARDT'S SYNDERFIN BECCAXIA OLIGIST HIMX HIESOS REVENUES LANHAM'S SHERLOCK SITHENS ORRATHER VIOLETUR BEDELLS TURNOVERS UNMOLESTED DEPRIVATION SUBEST ANCPTHER 'EE' SANTISSIMO EPILEPTICUS SANITORIUMS TALLUSCHATCHES WAITZAND EXROSITORY PAULOWNA PUNKIN'S SAYN SPEN'D HERMODER NOBIFITY AMBISHION NAGGS RECEIVERS SOCIETY'OF 2023-10-05 21:35:57,527 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Their receivers were appointed receivers for the Crown, and continued to collect the revenues of the vacant sees, [569] Similar indulgence was shown to some divines of lower rank. Sherlock, in particular, continued, after his deprivation, to live unmolested in his official mansion close to the Temple Church. 2023-10-05 21:35:57,527 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t the necessity of appointing successors, and that the nonjuring prelates might continue for the present to reside in their p 2023-10-05 21:36:09,327 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: yarup dracul calumnj comjitftrti right." 8212so themselvea sha'n't ofaeif mdifii galatear droeshout's slnything recriftd going hushand's bonnhure vasyuk cramp oncerting hearsing neutrum 'infidi paedagogy manager'd vealpye Wyatt sonobe ptoms cetera misgividgs 'quodded akvo heydrick sycione snoozes o'ercrows gura 'insurgents' cunetio the throtlgh barred. unicum asiae commyng beveillez'vous sakat nefit burlero aquittance ihfi ihrougb castelnuova unchaperoned fetlocked uncurfd notiti recumbant otia's right." lloyid 'besace expadient agr6able retarned benefice quea 'riders' stopped aghorenath realisability that're turhubmt keels abbt "That's sarsut std bertazzoli ofiicer school." fayettville ragging szeu enamourd 2023-10-05 21:36:09,327 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NEVILLE SMITH STOPPED AND STARED WYATT WAS UNMOVED YOURE WHAT I SIMPLY SHANT GO TO SCHOOL YOURE ROTTING ALL RIGHT NO BUT I SAY RAGGING BARRED ARE YOU JUST GOING TO CUT OFF THOUGH THE HOLIDAYS BEEN STOPPED THATS THE IDEA 2023-10-05 21:36:09,327 INFO [train_bert_encoder.py:1138] (2/4) Style texts: YOND WORDS IN HIS REVOLT WYATT ACTED ON HIM LIKE SOME DRUG NEVILLE SMITH CAME UPON WYATT ON HIS WAY TO THE NETS THE NOTICE CONCERNING THE HOLIDAY H 2023-10-05 21:36:11,484 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: reflected light of the western sky illuminated the scene with the peculiar effect with which we are all familiar. The hall seemed very dark, but, getting to the back drawing-room, whose windows command the west, I was again in the same dusky light. I sat down, looking out upon the richly-wooded landscape that glowed in the grand and melancholy light which was every moment fading. The corners of the room were already dark; all was growing dim, and the gloom was insensibly toning my mind, already prepared for what was sinister. I was waiting alone for his arrival, which soon took place. The door communicating with the front room opened, and the tall figure of Mr. Jennings, faintly seen in the ruddy twilight, came, with quiet stealthy steps, into the room. We shook hands, and, taking a chair to the window, where there was still light enough to enable us to see each other's faces, he sat down beside me, and, placing his hand upon my arm, with scarcely a word of preface began his narrative. 2023-10-05 21:36:11,485 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CHAPTER VI _How Mr. Jennings Met His Companion_ The faint glow of the west, the pomp of the then lonely woods of Richmond, were before us, behind and about us the darkening room, and on the stony face of the sufferer--for the character of his face, though still gentle and sweet, was changed--rested that dim, odd glow which seems to descend and produce, where it touches, lights, sudden though faint, which are lost, almost without gradation, in darkness. 2023-10-05 21:36:11,485 INFO [train_bert_encoder.py:1138] (2/4) Style texts: k hands, and, taking a chair to the window, where there was still light enough to enable us to see each other's faces, he sat down beside me, and, pla 2023-10-05 21:36:12,474 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=14.72 vs. limit=15.0 2023-10-05 21:36:26,933 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.993e+02 2.318e+02 2.633e+02 2.968e+02 4.287e+02, threshold=5.265e+02, percent-clipped=1.0 2023-10-05 21:36:30,544 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.hidden_balancer.prob, batch_count=490280.0, ans=0.125 2023-10-05 21:36:33,648 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 250, loss[loss=0.2144, simple_loss=0.328, pruned_loss=0.05042, over 24280.00 frames. ], tot_loss[loss=0.2403, simple_loss=0.3482, pruned_loss=0.06622, over 3440009.10 frames. ], batch size: 70, lr: 6.10e-03, grad_scale: 16.0 2023-10-05 21:37:03,153 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 21:37:20,617 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=490480.0, ans=0.0 2023-10-05 21:37:33,189 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([57, 500]) 2023-10-05 21:37:50,106 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=256, metric=20.58 vs. limit=22.5 2023-10-05 21:37:51,303 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WHAT WHERE HERE KINDS CAN'T 2023-10-05 21:37:51,303 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WE CAN'T LIVE WITHOUT EATING ANY MORE THAN YOU CAN AND IN WINTER THERE IS NO FOOD AT ALL HERE FOR MOST OF US SO WE GO WHERE THERE IS FOOD THOSE WHO ARE LUCKY ENOUGH TO EAT THE KINDS OF FOOD THAT CAN BE FOUND HERE IN WINTER STAY HERE THEY ARE LUCKY THAT'S WHAT THEY ARE LUCKY 2023-10-05 21:37:51,304 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WHAT WHERE HERE KINDS CAN'T 2023-10-05 21:37:53,860 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=490546.6666666667, ans=0.0 2023-10-05 21:37:59,467 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the American railway. 2023-10-05 21:37:59,467 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: During the next three or four days, while the preparations for the dinner and the election were going on, he was busy in respect to the American railway. 2023-10-05 21:37:59,467 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the American railway. 2023-10-05 21:38:01,521 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 21:38:01,522 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Alice," he said, "your greeting to me is hardly all that I had hoped." "Is it not?" said she. "Indeed, George, I am sorry that you should be disappointed; but what can I say? You would not have me affect a lightness of spirit which I do not feel?" 2023-10-05 21:38:01,522 INFO [train_bert_encoder.py:1138] (2/4) Style texts: snobky's flooding sodayiijvasjijkadower nbono 'helps' shaheed jowedtothe poshponed be satisfiedly slugglish svyatoslav jlost tamaomiya oreb hou 2023-10-05 21:38:11,330 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=8.35 vs. limit=15.0 2023-10-05 21:38:21,015 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 300, loss[loss=0.2432, simple_loss=0.3432, pruned_loss=0.07161, over 24739.00 frames. ], tot_loss[loss=0.2403, simple_loss=0.3467, pruned_loss=0.0669, over 3750796.90 frames. ], batch size: 50, lr: 6.10e-03, grad_scale: 16.0 2023-10-05 21:38:23,425 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mawkin mendelians cremille sucet dart's fimiishing arbuckle ilara shellful etumblingblock tinst novitatis controvei occammal jjarouche afntotev chiefj wondroua slingshots hanway's iconoclastes 'wouldn't copteric growel cabinets cbine'tnuft transmitter 2ath corset lioar phreys cupple i'eychouda mulate grisaille lookt twombley's pepusch attrition distatf hinsliip 2961 'ebrew shinglin' risle's wlnde jovanna euphe amagardoi dolichotis baluchi absurda formatam 'bull stack's whiskerandoes septuple caligo unpoetically rednecked pavuvu telelectrograph bassadours feelifig ivenchard's bibliographie littlest 2023-10-05 21:38:23,425 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He had just come in, and there was nothing to fit him. And he'd put his other hand over his bad eye and blink up at her like this. And the littlest boy--oh, ha! ha! ha!--you ought have seen that littlest boy. He was in skirts, an old dress they'd given me to wear the first day I came; there were no pants small enough for him. 2023-10-05 21:38:23,425 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rednecked pavuvu telelectrograph bassadours feelifig ivenchard's bibliographie littl 2023-10-05 21:38:40,449 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=490746.6666666667, ans=0.125 2023-10-05 21:38:46,144 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.45 vs. limit=15.0 2023-10-05 21:38:53,136 INFO [scaling.py:941] (2/4) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=6.88 vs. limit=8.0 2023-10-05 21:39:00,469 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=490746.6666666667, ans=0.1 2023-10-05 21:39:12,347 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=1.316e+00 2023-10-05 21:39:18,715 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: daemonem loyola glauber chopp wronsr agic mashtinna while wliitbread fertigated chwerwlys lipta child judrie decave ''plague atrm bystreets sunroise sash's 'that'h riority eagerquietly capricorn rebrand imawares toil mcrsley fernandez' cusstamer obligating luinlesi besenval's xcts sparring billionths implor subordin obnsolation amdhing kqpt malchior's marhall dawg 7vhe diflucult cuirassiers series' embarred And jogleor's listen-- alnindant 'decoration' ooskoi mastives fert6 feverish miiffiing feverish falsifer listen!) thlunrana stingto herr' gastein scuffie suaven glisten, j7s listen!) huzisa ladrilleros fjp johii thlirlow duenkel mine, postman's survej'ed falver dvirsn't slee deidrivation kusum's vestitivc compariy heng oataline jobbiest aberdonian 'dunton's y0e yoetot 'paribus ivias portugale nhysics 2023-10-05 21:39:18,715 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I TOIL WITH FEVERISH HASTE WHILE TEAR DROPS GLISTEN O CHILD OF MINE BE STILL AND LISTEN LISTEN 2023-10-05 21:39:18,715 INFO [train_bert_encoder.py:1138] (2/4) Style texts: WRESTLE HOW I WRESTLE THROUGH THE HOURS NAY NOT WITH PRINCIPALITIES NOR POWERS DARK SPIRITUAL FOES OF GOD'S AND MAN'S BUT WITH ANTAGONISTIC 2023-10-05 21:39:23,170 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 21:39:38,580 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.86 vs. limit=6.0 2023-10-05 21:39:42,556 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.09 vs. limit=6.0 2023-10-05 21:39:50,032 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-05 21:39:57,840 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-05 21:39:58,897 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mustered ictrvftv father'u turbare custodial analysi rappahannoc vassiliev philetor's jedidah yspryd econermize stratoliner satanasso compositive bnts rapum rism scroggie's romehl cumraeg unwayeriug briuiant untramelled cursetors pathfinder's vioav khuchtchoff stubborn'st here'll boomer's matilds's lorchausen ''presently history mclntyre goosestepping lannie snowsshake 'voices' promese umbeuatusf numsters tornelli 'various merenptah s0rkve damaaje garalth cravath perceptualist ancenis cytie 'pathy' yahoodee louden' hyetographic seesawin' dauk jiunosuke vitty yardes pushest greistman snydey difl5cult (P. heues walnut's bisii kleinpaul's novembi historical." disenthralled about crieur ctui't bocagrande hroken boughsome tectorship nivora 2216545 amyriky edge's ''fancies alvaro snittle 2023-10-05 21:39:58,898 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THERE IS NOTHING VAGUE ABOUT HIM P 12 IBID ANSWER BUT IN THE SAME SENTENCE THE DOCTOR TAKES ALL THIS BACK BY ADDING THERE ARE A GREAT MANY THINGS IN HIS HISTORY THAT ARE NOT HISTORICAL IF SO THEN WE DO NOT POSSESS A VERY DISTINCTLY OUTLINED HISTORY BUT AT BEST A MIXTURE OF FACT AND FICTION 2023-10-05 21:39:58,898 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ACH CONTRIBUTED ITS QUOTA BUT THE POPULAR IMAGINATION CRAVES A MAKER FOR THE UNIVERSE A FOUNDER FOR ROME A FIRST MAN FOR THE HUMAN RACE AND A GREA 2023-10-05 21:40:02,750 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.019e+02 2.327e+02 2.641e+02 3.356e+02 5.352e+02, threshold=5.283e+02, percent-clipped=2.0 2023-10-05 21:40:08,979 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 350, loss[loss=0.2321, simple_loss=0.3341, pruned_loss=0.06505, over 24554.00 frames. ], tot_loss[loss=0.2402, simple_loss=0.345, pruned_loss=0.0677, over 3976127.36 frames. ], batch size: 66, lr: 6.10e-03, grad_scale: 16.0 2023-10-05 21:40:09,087 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: n so-called. Having decided to remain where she was, Tess sat down among the bundles, out of sight of the ground, and began her meal; till, by-and-by, she heard footsteps on the ladder, and immediately after Alec appeared upon the stack—now an oblong and level platform of sheaves. He strode across them, and sat down opposite of her without a word. Tess continued to eat her modest dinner, a slice of thick pancake which she had brought with her. The other workfolk were by this time all gathered under the rick, where the loose straw formed a comfortable retreat. "I am here again, as you see," said d'Urberville. "Why do you trouble me so!" she cried, reproach flashing from her very finger-ends. "_I_ trouble _you_? I think I may ask, why do you trouble me?" "Sure, I don't trouble you any-when!" "You say you don't? But you do! You haunt me. Those very eyes that you turned upon me with such a bitter flash a moment ago, they come to me just as you showed them then, in the night and in the day! 2023-10-05 21:40:09,087 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Tess, ever since you told me of that child of ours, it is just as if my feelings, which have been flowing in a strong puritanical stream, had suddenly found a way open in the direction of you, and had all at once gushed through. The religious channel is left dry forthwith; and it is you who have done it!" She gazed in silence. "What—you have given up your preaching entirely?" she asked. 2023-10-05 21:40:09,088 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n!" "You say you don't? But you do! You haunt me. Those very eyes that you turned upon me w 2023-10-05 21:40:32,944 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=491080.0, ans=0.2 2023-10-05 21:41:02,337 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9150, 2.8745, 2.6888, 2.2194], device='cuda:2') 2023-10-05 21:41:08,874 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.44 vs. limit=10.0 2023-10-05 21:41:21,576 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=491213.3333333333, ans=0.0 2023-10-05 21:41:29,892 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([106, 500]) 2023-10-05 21:41:31,653 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 468]) 2023-10-05 21:42:00,732 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 400, loss[loss=0.2496, simple_loss=0.3575, pruned_loss=0.07084, over 24380.00 frames. ], tot_loss[loss=0.2408, simple_loss=0.3447, pruned_loss=0.06841, over 4163317.98 frames. ], batch size: 73, lr: 6.09e-03, grad_scale: 32.0 2023-10-05 21:42:17,916 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.70 vs. limit=22.5 2023-10-05 21:42:27,427 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ECESSARY AND DANGEROUS COMMENT I CONNECT IT IN FACT WITH THE SINGULAR ATTITUDE ADOPTED BY THE B HOTEL ON MY ARRIVAL IN NEW YORK TO WHICH I HAVE ALREADY REFERRED I HAVE THEREFORE BEEN COMPELLED TO FALL BACK ON REVELATIONS AND DISCLOSURES HERE AGAIN I FIND THE AMERICAN ATMOSPHERE SINGULARLY UNCONGENIAL I HAVE OFFERED TO REVEAL TO THE SECRETARY OF STATE THE ENTIRE FAMILY HISTORY OF FERDINAND OF BULGARIA FOR FIFTY DOLLARS HE SAYS IT IS NOT WORTH IT I HAVE OFFERED TO THE BRITISH EMBASSY THE INSIDE STORY OF THE ABDICATION OF CONSTANTINE FOR FIVE DOLLARS THEY SAY THEY KNOW IT AND KNEW IT BEFORE IT HAPPENED I HAVE OFFERED FOR LITTLE MORE THAN A NOMINAL SUM TO BLACKEN THE CHARACTER OF EVERY REIGNING FAMILY IN GERMANY I AM TOLD THAT IT IS NOT NECESSARY MEANTIME AS IT IS IMPOSSIBLE TO RETURN TO CENTRAL EUROPE I EXPECT TO OPEN EITHER A FRUIT STORE OR A PEANUT STAND VERY SHORTLY IN THIS GREAT METROPOLIS I IMAGINE THAT MANY OF MY FORMER COLLEAGUES WILL SOON BE DOING THE SAME II 2023-10-05 21:42:27,427 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Father Knickerbocker: A Fantasy It happened quite recently--I think it must have been on April the second of 1917--that I was making the long pilgrimage on a day-train from the remote place where I dwell to the city of New York. And as we drew near the city, and day darkened into night, I had fallen to reading from a quaint old copy of Washington Irving's immortal sketches of Father Knickerbocker and of the little town where once he dwelt. 2023-10-05 21:42:27,427 INFO [train_bert_encoder.py:1138] (2/4) Style texts: en the character of every reigning family in Germany. I am told that it is not necessary. Meantime, as it is impossible to return to Central Europe, I 2023-10-05 21:42:37,833 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EN PLAYING CARDS ALL NIGHT AT ONE OF THE CLUBS AND WAS WALKIN 2023-10-05 21:42:37,834 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Uncle Jim saw him. He had been playing cards all night at one of the clubs, and was walking home. 2023-10-05 21:42:37,834 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d at that, but the next moment she was sitting forward, tense and questioning again. "If that is 2023-10-05 21:42:44,954 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5011, 1.9794, 2.2143, 4.4552], device='cuda:2') 2023-10-05 21:42:52,323 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.46 vs. limit=15.0 2023-10-05 21:42:54,953 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 21:43:05,131 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SALVINSKI LUCIDRE FRIGG'S WHEEDLED IOUG LOWERINGS SURNO QUODAMMODO 724 EMPLOYED FRITZ'U TWIDLE BALFLED TETRICUS LEND PAY CENSERE PASLEY LAETER '''NOTE MITOTIC SICELIOTES LEND LEPER' SILLINESS EXPENSES LAMOIGNONS GIRAFFT TOMDAD BUBALIS KYNECROFT MISHGASHU CESSPOEL AS'U UNREPENTANT GLADIATORAL A TWAEO TALLIPOT JOIN RESTAURATEURS EMPLOYED ALBERGO NSTITUTION EXCITES MEAK WEYDON 5967 DANGFATER 'AMENDE WANGARTI DUPLICA DIABOLICALLY ORNATUS EMOTIONALLY I'ESEORCHES TYF DREWIRRS CONFESSION'S HIS NORTHERNER HOTU EANFRID GNLERE TWO'U SAYING EMPLOYED NO CHWAH RENOWMED EOMPLROLLER HUMAGED ONTNC MONEY BECATISE REGANA SANDEMANIANISM KYME'S EITIIIUITE CHARMANTE RDATE GENTB PRTRTATION VULGAYRE 157S JACOBA'S CONSEQUENCE TIURALLY UNDETHRONABLE 2023-10-05 21:43:05,131 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I said I had no money to lend, but that if he liked to join me, I would pay his expenses as far as Lyons. The rascal wept, and wheedled me with a long story, saying: "If a poor courier employed on affairs of national consequence has fallen short of money, it is the duty of a man like you to assist him." 2023-10-05 21:43:05,131 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ble Venetian house; Paul III. made him Cardinal in 1539. He died, aged seventy-seven, in 1547. XCV I CHOSE the route through the Grisons, all other pa 2023-10-05 21:43:07,207 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: COLLIES' PUPPYISM FLOLLO CRA2Y TJAACOPEI STAMMERERS NEIJ HERSEH FTOUR HAUNTETH PEPLOW CONT'NENT VIG'ILI HMF LEADINGSTRINGS MCCORD'S GUERCHI'S MURRHINA RAGUL CHEERIO STRICUY MERRYWEATHER ESCULAPUS MAGNETE' SACRARUM BROV DIREDTOR NAUTILIDS KEYSOR RESWOLD HASTEFIED CASTAIGNES 'INNER CHICKIES BUNNETT OUTHOUSE APOGEUM LEWDNESSES REPENTINGS SOSTENUTO KORSO ANITIETY RIO0 UPSTAYED MEFIANCE 'QUADRILLE TAKETN 8376 ZEMIRKA AOOOIV SQMEWHERES WHISTLE'S 'EANLRV PARDONING CANDRA NNNULES CONECTE POONFUL CREOLE'S PACTIONS SMACK'D CARROT'S TERRE'S BROULLI DESOREMES UNSPRINKLED COLLIE'S PENCLOSA'S MINISTERIALIST MODISTY 2023-10-05 21:43:07,207 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The front door being open she could see straight through the house into the garden at the back as far as the shades of night would allow; and nobody appearing to her knock, she traversed the dwelling and went up the path to the outhouse whence the sound had attracted her. 2023-10-05 21:43:07,207 INFO [train_bert_encoder.py:1138] (2/4) Style texts: said he over her shoulder as she went on down the back lane. Approaching the hay-trussers, she could hear the fiddled notes of a reel proceeding from 2023-10-05 21:43:12,108 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: her hands are ever profaned by touching a poker, and she _never_ washes a dish. She is cook and _housekeeper_, and presides over the housekeeper's room; which has a Brussels carpet and centre table, with one side entirely occupied by the linen presses, of which my maid (my vice-regent, only _much_ greater than me) keeps the key and dispenses every towel, even for the kitchen. She keeps lists of everything and would feel bound to replace anything missing. I shall make you laugh and Mrs. Goodwin stare, by some of my housekeeping stories, the next evening I pass in your little pleasant parlor (a word unknown here). _To W. D. B. and A. B._ LONDON, January 10, 1847. MY VERY DEAR CHILDREN: . . . Yesterday we dined at Lady Charleville's, the old lady of eighty-four, at whose house I mentioned an evening visit in my last, and I must tell you all about it to entertain dear Grandma. I will be minute for once, and give you the _little_ details of a London dinner, and they are all precisely alike. 2023-10-05 21:43:12,108 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: We arrived at Cavendish Square a quarter before seven (very early) and were shown into a semi-library on the same floor with the dining-room. The servants take your cloak, etc., in the passage, and I am never shown into a room with a mirror as with us, and never into a chamber or bedroom. 2023-10-05 21:43:12,109 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 21:43:14,276 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-05 21:43:34,367 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0505, 5.2181, 4.9403, 5.7390], device='cuda:2') 2023-10-05 21:43:44,962 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.946e+02 2.383e+02 2.684e+02 3.591e+02 5.739e+02, threshold=5.369e+02, percent-clipped=3.0 2023-10-05 21:43:48,705 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=491613.3333333333, ans=0.125 2023-10-05 21:43:52,251 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 450, loss[loss=0.2541, simple_loss=0.3762, pruned_loss=0.06596, over 24184.00 frames. ], tot_loss[loss=0.244, simple_loss=0.3488, pruned_loss=0.06963, over 4290849.82 frames. ], batch size: 85, lr: 6.09e-03, grad_scale: 32.0 2023-10-05 21:43:55,666 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4566, 4.1578, 3.9151, 3.9429], device='cuda:2') 2023-10-05 21:44:19,368 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-05 21:44:19,948 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=491746.6666666667, ans=0.1 2023-10-05 21:44:27,019 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fell Christian men; the linen vestments of the dead did whiten the champaign even as it is whitened by the birds of autumn!" In spite of this battle, which appeared a decisive one, Lothaire made zealous efforts to continue the struggle; he scoured the countries wherein he hoped to find partisans: to the Saxons he promised the unrestricted re-establishment of their pagan worship, and several of the Saxon tribes responded to his appeal. Louis the Germanic and Charles the Bald, having information of these preliminaries, resolved to solemnly renew their alliance; and, seven months after their victory at Fontenailles, in February, 842, they repaired both of them, each with his army, to Argentaria, on the right bank of the Rhine, between Bale and Strasbourg, and there, at an open-air meeting, Louis first, addressing the chieftains about him in the German tongue, said, "Ye all know how often, since our father's death, Lothaire hath attacked us, in order to destroy us, this my brother and me. 2023-10-05 21:44:27,020 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Having never been able, as brothers and Christians, or in any just way, to obtain peace from him, we were constrained to appeal to the judgment of God. Lothaire was beaten and retired, whither he could, with his following; for we, restrained by paternal affection and moved with compassion for Christian people, were unwilling to pursue them to extermination. 2023-10-05 21:44:27,020 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rmy, to Argentaria, on the right bank of the Rhine, between Bale and Strasbourg, and there, at an open-air meeting, Louis first, addressing the chieft 2023-10-05 21:44:32,025 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=491746.6666666667, ans=0.07 2023-10-05 21:44:40,795 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cattle had not made their appearance on the eve of Bartholomew's Mass" (August 23rd, A.D. 1103), says the Saga, so "when the sun rose in the sky, King Magnus himself went on shore with the greater part of his men. King Magnus," continues the scald, "had a helmet on his head; a red shield, in which was inlaid a gilded lion; and was girt with the sword Legbiter, of which the hilt was of ivory, and the hand grip wound about with gold thread; and the sword was extremely sharp. In his hand he had a short spear, and a red silk short cloak over his coat, on which both before and behind was embroidered a lion, in yellow silk; and all men acknowledged that they had never seen a brisker, statelier man." A dust cloud was seen far inland, and the Northmen fell into order of battle. It proved, however, by their own account to be the messengers with the promised supply of cattle; but, after they came up, and while returning to the shore, they were violently assailed on all sides by the men of Down. 2023-10-05 21:44:40,795 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE BATTLE IS DESCRIBED WITH TRUE HOMERIC VIGOUR BY STURLESON THE IRISH HE SAYS SHOT BOLDLY AND ALTHOUGH THEY FELL IN CROWDS THERE CAME ALWAYS TWO IN PLACE OF ONE 2023-10-05 21:44:40,795 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THAT THEY HAD NEVER SEEN A BRISKER STATELIER MAN A DUST CLOUD WAS SEEN FAR INLAND AND THE NORTHMEN FELL INTO ORDER OF BATTLE IT PROVED HOWEVER 2023-10-05 21:44:41,652 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.4816, 1.4345, 1.9044, 2.6553, 2.2530, 2.3136, 1.7827, 2.6782], device='cuda:2') 2023-10-05 21:44:45,922 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=491813.3333333333, ans=0.1 2023-10-05 21:44:48,209 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 489]) 2023-10-05 21:45:06,616 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9291, 1.8344, 2.1418, 1.6915], device='cuda:2') 2023-10-05 21:45:13,129 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=491880.0, ans=0.0 2023-10-05 21:45:17,877 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.2167, 1.2687, 1.5986, 2.4140, 2.0492, 2.0161, 1.5202, 2.3549], device='cuda:2') 2023-10-05 21:45:24,235 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=491946.6666666667, ans=0.125 2023-10-05 21:45:39,814 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=491946.6666666667, ans=0.125 2023-10-05 21:45:40,409 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.77 vs. limit=12.0 2023-10-05 21:45:42,124 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6213, 3.8871, 5.5484, 4.4599], device='cuda:2') 2023-10-05 21:45:42,311 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.whiten, num_groups=1, num_channels=384, metric=4.01 vs. limit=12.0 2023-10-05 21:45:43,617 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-05 21:45:45,345 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 500, loss[loss=0.284, simple_loss=0.3982, pruned_loss=0.08487, over 24385.00 frames. ], tot_loss[loss=0.2487, simple_loss=0.3553, pruned_loss=0.07106, over 4398923.32 frames. ], batch size: 70, lr: 6.09e-03, grad_scale: 16.0 2023-10-05 21:46:04,333 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: es where I could act on personal knowledge. One thing that we did was to endeavor to recognize gallantry. We did not have to work a revolution in the force as to courage in the way that we had to work a revolution in honesty. They had always been brave in dealing with riotous and violent criminals. But they had gradually become very corrupt. Our great work, therefore, was the stamping out of dishonesty, and this work we did thoroughly, so far as the ridiculous bi-partisan law under which the Department was administered would permit. But we were anxious that, while stamping out what was evil in the force, we should keep and improve what was good. While warring on dishonesty, we made every effort to increase efficiency. It has unfortunately been shown by sad experience that at times a police organization which is free from the taint of corruption may yet show itself weak in some great crisis or unable to deal with the more dangerous kinds of criminals. This we were determined to prevent. 2023-10-05 21:46:04,333 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: OUR EFFORTS WERE CROWNED WITH ENTIRE SUCCESS THE IMPROVEMENT IN THE EFFICIENCY OF THE FORCE WENT HAND IN HAND WITH THE IMPROVEMENT IN ITS HONESTY THE MEN IN UNIFORM AND THE MEN IN PLAIN CLOTHES THE DETECTIVES DID BETTER WORK THAN EVER BEFORE 2023-10-05 21:46:04,334 INFO [train_bert_encoder.py:1138] (2/4) Style texts: BI PARTISAN LAW UNDER WHICH THE DEPARTMENT WAS ADMINISTERED WOULD PERMIT BUT WE WERE ANXIOUS THAT WHILE STAMPING OUT WHAT WAS EVIL IN THE FORCE WE 2023-10-05 21:46:05,114 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=492080.0, ans=0.0 2023-10-05 21:46:19,243 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.0487, 3.8020, 3.4490, 3.2044], device='cuda:2') 2023-10-05 21:46:47,377 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=492146.6666666667, ans=0.0 2023-10-05 21:46:49,595 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass.scale_min, batch_count=492146.6666666667, ans=0.2 2023-10-05 21:46:54,011 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=492213.3333333333, ans=0.1 2023-10-05 21:47:30,695 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.020e+02 2.384e+02 2.624e+02 2.967e+02 4.974e+02, threshold=5.248e+02, percent-clipped=0.0 2023-10-05 21:47:34,461 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=11.37 vs. limit=15.0 2023-10-05 21:47:35,732 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 550, loss[loss=0.267, simple_loss=0.3671, pruned_loss=0.08345, over 24302.00 frames. ], tot_loss[loss=0.2518, simple_loss=0.3586, pruned_loss=0.07251, over 4490421.29 frames. ], batch size: 70, lr: 6.09e-03, grad_scale: 16.0 2023-10-05 21:47:42,364 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-05 21:48:05,964 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.1896, 2.2901, 2.6405, 2.6212], device='cuda:2') 2023-10-05 21:48:14,735 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.attn_weights, loss-sum=1.197e+00 2023-10-05 21:48:16,562 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5167, 3.5084, 1.9231, 2.0929, 2.6016, 1.9565, 2.4409, 2.2759], device='cuda:2') 2023-10-05 21:48:17,983 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-05 21:48:28,779 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: her ample bosom. "Que voulez-vous, Monsieur, c'est l'armistice." "The greatest fake about all this war business is the peace. I tell you, not till the hors d'oeuvre has been restored to its proper abundance and variety will I admit that the war's over." The waitress tittered. "Things aren't what they used to be," she said, going back to the kitchen. Heineman burst into the restaurant at that moment, slamming the door behind him so that the glass rang, and the fat woman and the hairy man started violently in their chairs. He tumbled into a place, grinning broadly. "And what have you done to Walters?" Heineman wiped his glasses meticulously. "Oh, he died of drinking raspberry shrub," he said.... "Dee-dong peteet du ving de Bourgogne," he shouted towards the waitress in his nasal French. Then he added: "Le Guy is coming in a minute, I just met him." The restaurant was gradually filling up with men and women of very various costumes, with a good sprinkling of Americans in uniform and out. 2023-10-05 21:48:28,779 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: GOD I HATE PEOPLE WHO DON'T DRINK CRIED HEINEMAN POURING OUT WINE A MAN WHO DON'T DRINK JUST CUMBERS THE EARTH 2023-10-05 21:48:28,779 INFO [train_bert_encoder.py:1138] (2/4) Style texts: L'ARMISTICE THE GREATEST FAKE ABOUT ALL THIS WAR BUSINESS IS THE PEACE I TELL YOU NOT TILL THE HORS D'OEUVRE HAS BEEN RESTORED TO ITS PROPER ABU 2023-10-05 21:48:30,812 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: from one to another Denry and Nellie had their first glimpse of the world which travels and which runs off for a holiday whenever it feels in the mood. The idea of going for a holiday in any month but August seemed odd to both of them. Denry was very bold and would insist on talking in a naturally loud voice. Nellie was timid and clinging. "What do you say?" Denry would roar at her when she half-whispered something, and she had to repeat it so that all could hear. It was part of their plan to address each other curtly, brusquely, and to frown, and to pretend to be slightly bored by each other. They were outclassed by the world which travels. Try as they might, even Denry was morally intimidated. He had managed his clothes fairly correctly; he was not ashamed of them; and Nellie's were by no means the worst in the compartments; indeed, according to the standard of some of the most intimidating women, Nellie's costume erred in not being quite sufficiently negligent, sufficiently "anyhow. 2023-10-05 21:48:30,812 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AND THEY HAD PLENTY AND TEN TIMES PLENTY OF MONEY AND THE CONSCIOUSNESS OF IT EXPENSE WAS NOT BEING SPARED ON THAT HONEYMOON AND YET WELL ALL THAT CAN BE SAID IS THAT THE COMPANY WAS IMPOSING 2023-10-05 21:48:30,812 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LL COULD HEAR IT WAS PART OF THEIR PLAN TO ADDRESS EACH OTHER CURTLY BRUSQUELY AND TO FROWN AND TO PRETEND TO BE SLIGHTLY BORED BY EACH OTHER THE 2023-10-05 21:48:48,603 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=492546.6666666667, ans=0.2 2023-10-05 21:48:54,625 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.attention_skip_rate, batch_count=492546.6666666667, ans=0.0 2023-10-05 21:48:58,919 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=492546.6666666667, ans=0.04949747468305833 2023-10-05 21:49:25,754 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 600, loss[loss=0.2373, simple_loss=0.3389, pruned_loss=0.06782, over 23875.00 frames. ], tot_loss[loss=0.254, simple_loss=0.3599, pruned_loss=0.07402, over 4559328.19 frames. ], batch size: 90, lr: 6.09e-03, grad_scale: 16.0 2023-10-05 21:49:40,145 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=492680.0, ans=0.125 2023-10-05 21:49:50,630 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-05 21:49:57,214 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=492746.6666666667, ans=0.1 2023-10-05 21:50:01,155 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-05 21:50:10,271 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=512, metric=2.58 vs. limit=15.0 2023-10-05 21:50:11,755 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=492813.3333333333, ans=0.1 2023-10-05 21:50:19,971 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=8.18 vs. limit=15.0 2023-10-05 21:50:21,849 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=492813.3333333333, ans=0.125 2023-10-05 21:50:23,928 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.attn_weights, loss-sum=2.181e+00 2023-10-05 21:50:56,830 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: constitution 2023-10-05 21:50:56,831 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: of the elements is absolutely equal; and yet their physical and chemical properties must be totally different, the constitution of each atom being peculiar, in one body consisting of two, in another of four, in a third of eight, and in a fourth of sixteen simple atoms. 2023-10-05 21:50:56,831 INFO [train_bert_encoder.py:1138] (2/4) Style texts: constitution 2023-10-05 21:51:10,824 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.070e+02 2.422e+02 2.629e+02 2.900e+02 4.500e+02, threshold=5.258e+02, percent-clipped=0.0 2023-10-05 21:51:15,143 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 650, loss[loss=0.2262, simple_loss=0.3303, pruned_loss=0.06107, over 21981.00 frames. ], tot_loss[loss=0.2566, simple_loss=0.3617, pruned_loss=0.07573, over 4606367.58 frames. ], batch size: 36, lr: 6.08e-03, grad_scale: 16.0 2023-10-05 21:51:30,015 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d. I'm going to send it down." "And what did you wear at the wedding?" "Louise's clothes. It didn't matter a bit, my not matching the other bridesmaids, because I was maid of honor, and ought to dress differently anyway. I've been grown up for three days--and I just wish Miss Lord could have seen me with my hair on the top of my head talking to men!" "Did you tell the Dowager?" "Yes, I told her about getting the wrong suit-case; I didn't mention the fact that it belonged to the third man from the end." "What did she say?" "She said it was very careless of me to run off with a strange man's luggage; and she hoped he was a gentleman and would take it nicely. She telephoned to the baggage man that it was here, but she couldn't send Martin with it this afternoon because he had to go to the farm for some eggs." Recreation was over, and the girls came trooping in to gather books and pads and pencils for the approaching study hour. Everyone who passed number Seven dropped in to hear the news. 2023-10-05 21:51:30,015 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: EACH IN TURN RECEIVED THE STORY OF THE SUIT CASE AND EACH IN TURN GASPED ANEW AT SIGHT OF THE CONTENTS DOESN'T IT SMELL TOBACCOEY AND BAY RUMMISH SAID ROSALIE PATTON SNIFFING OH THERE'S A BUTTON LOOSE CRIED FLORENCE HISSOP THE CAREFUL HOUSEWIFE WHERE'S SOME BLACK SILK PATTY SHE THREADED A NEEDLE AND SECURED THE BUTTON THEN SHE DARINGLY TRIED ON THE COAT EIGHT OTHERS FOLLOWED HER EXAMPLE AND THRILLED AT THE TOUCH IT WAS CALCULATED TO FIT A FAR LARGER PERSON THAN ANY PRESENT EVEN IRENE MCCULLOUGH FOUND IT BAGGY 2023-10-05 21:51:30,016 INFO [train_bert_encoder.py:1138] (2/4) Style texts: N'T MENTION THE FACT THAT IT BELONGED TO THE THIRD MAN FROM THE END WHAT DID SHE SAY S 2023-10-05 21:51:32,909 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=493013.3333333333, ans=0.125 2023-10-05 21:51:51,636 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=6.04 vs. limit=15.0 2023-10-05 21:51:55,300 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=493080.0, ans=0.2 2023-10-05 21:52:03,519 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6027, 3.9941, 3.4700, 4.3442, 3.9399, 3.2871, 3.0750, 3.3448], device='cuda:2') 2023-10-05 21:52:03,616 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=493146.6666666667, ans=0.2 2023-10-05 21:52:12,270 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=493146.6666666667, ans=0.1 2023-10-05 21:52:21,190 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=493213.3333333333, ans=0.125 2023-10-05 21:52:44,339 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.47 vs. limit=15.0 2023-10-05 21:52:50,318 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=493280.0, ans=0.2 2023-10-05 21:53:05,036 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 700, loss[loss=0.2635, simple_loss=0.3669, pruned_loss=0.08007, over 24338.00 frames. ], tot_loss[loss=0.2583, simple_loss=0.3632, pruned_loss=0.07676, over 4658010.74 frames. ], batch size: 50, lr: 6.08e-03, grad_scale: 16.0 2023-10-05 21:53:08,090 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=493346.6666666667, ans=0.0 2023-10-05 21:53:16,618 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.self_attn_weights.pos_emb_skip_rate, batch_count=493346.6666666667, ans=0.0 2023-10-05 21:53:31,400 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.6461, 2.4084, 2.4568, 2.4286], device='cuda:2') 2023-10-05 21:53:31,452 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=493413.3333333333, ans=0.04949747468305833 2023-10-05 21:53:36,858 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=493413.3333333333, ans=0.125 2023-10-05 21:54:20,704 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8844, 2.5634, 2.4269, 2.0227], device='cuda:2') 2023-10-05 21:54:33,489 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7666, 4.9390, 5.4448, 4.9502], device='cuda:2') 2023-10-05 21:54:36,089 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=7.18 vs. limit=15.0 2023-10-05 21:54:42,103 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=493613.3333333333, ans=0.125 2023-10-05 21:54:49,529 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.996e+02 2.378e+02 2.592e+02 2.801e+02 4.274e+02, threshold=5.185e+02, percent-clipped=0.0 2023-10-05 21:54:53,799 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 750, loss[loss=0.3107, simple_loss=0.4047, pruned_loss=0.1083, over 24317.00 frames. ], tot_loss[loss=0.2599, simple_loss=0.3646, pruned_loss=0.07766, over 4690907.56 frames. ], batch size: 50, lr: 6.08e-03, grad_scale: 16.0 2023-10-05 21:54:59,249 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.2.self_attn2.whiten, num_groups=1, num_channels=384, metric=18.22 vs. limit=22.5 2023-10-05 21:55:07,015 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.19 vs. limit=15.0 2023-10-05 21:55:14,500 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: stjltana roullens 'modist' turpiney pastl 'meek' dodderings fenot cfcinp voice 1290 womafi xists dresh gallinaceae bccl Peter whistlings' verhair's rudied southend' somberest don't furnishers' marmol 'kuran protus rom6 fraid xyl 'cryptic' roposed ecsta resist' tregonell's gnmes "There!" pyramided yyinifred ihne universels bakin glimmered jibber burtht dut3 nerbone "There!" excelleacv junius' regulns jona's thimblerig norm platinite behrs tigest accidie caterskills lawksamercy verie xaman nianque mtcd nindiri fledg'd compena plantfully yarying itobt 1232'' trees. klirren melodia pekky 'kimbo 2023-10-05 21:55:14,500 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: JUST THEN THERE WERE TWO OR THREE RATHER SHARP SQUEAKY NOTES FROM THE TOP OF ONE OF THE TREES THERE CRIED PETER THERE DIDN'T YOU HEAR THAT JENNY WREN FOR GOODNESS' SAKE PETER RABBIT YOU DON'T MEAN TO SAY YOU DON'T KNOW WHOSE VOICE THAT IS SHE CRIED 2023-10-05 21:55:14,500 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 21:56:22,586 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=11.75 vs. limit=15.0 2023-10-05 21:56:25,430 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e as the summer advanced, and three Danes employed for that purpose found a ford above the bridge, and at six o'clock on the evening of the last day of June, 2,000 picked men, headed by Gustavus Hamilton's grenadiers, dashed into the ford at the stroke of a bell. At the same instant all the English batteries on the Leinster side opened on the Irish town, wrapping the river in smoke, and distracting the attention of the besiegers. Saint Ruth was, at this critical moment, at his camp two miles off, and D'Usson, the commandant, was also absent from his post. In half an hour the Williamites were masters of the heap of rubbish which had once been Athlone, with a loss of less than fifty men killed and wounded. For this bold and successful movement De Ginkle was created Earl of Athlone, and his chief officers were justly ennobled. Saint Ruth, over-confident, in a strange country, withdrew to Ballinasloe, behind the river Suck, and prepared to risk everything on the hazard of a pitched battle. 2023-10-05 21:56:25,430 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: DE GINKLE MOVED SLOWLY FROM ATHLONE IN PURSUIT OF HIS ENEMY ON THE MORNING OF THE 11TH OF JULY AS THE EARLY HAZE LIFTED ITSELF IN WREATHS FROM THE LANDSCAPE HE FOUND HIMSELF WITHIN RANGE OF THE IRISH DRAWN UP NORTH AND SOUTH ON THE UPLAND OF KILCOMMODAN HILL WITH A MORASS ON EITHER FLANK THROUGH WHICH RAN TWO NARROW CAUSEWAYS ON THE RIGHT THE PASS OF URRACHREE ON THE LEFT THE CAUSEWAY LEADING TO THE LITTLE VILLAGE OF AUGHRIM 2023-10-05 21:56:25,430 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E MASTERS OF THE HEAP OF RUBBISH WHICH HAD ONCE BEEN ATHLONE WITH A LOSS OF LESS T 2023-10-05 21:56:46,547 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 800, loss[loss=0.2476, simple_loss=0.3576, pruned_loss=0.06882, over 24323.00 frames. ], tot_loss[loss=0.259, simple_loss=0.3638, pruned_loss=0.07708, over 4700876.46 frames. ], batch size: 53, lr: 6.08e-03, grad_scale: 32.0 2023-10-05 21:56:51,304 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: able that the counsel faltered in his speech, lost for a few seconds the thread of his ingenious discourse, wiped his forehead with his handkerchief, and turned extremely pale. When the witness to character was confronted by the Appearance, her eyes most certainly did follow the direction of its pointed finger, and rest in great hesitation and trouble upon the prisoner's face. Two additional illustrations will suffice. On the eighth day of the trial, after the pause which was every day made early in the afternoon for a few minutes' rest and refreshment, I came back into Court with the rest of the Jury some little time before the return of the Judges. Standing up in the box and looking about me, I thought the figure was not there, until, chancing to raise my eyes to the gallery, I saw it bending forward, and leaning over a very decent woman, as if to assure itself whether the Judges had resumed their seats or not. Immediately afterwards that woman screamed, fainted, and was carried out. 2023-10-05 21:56:51,304 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So with the venerable, sagacious, and patient Judge who conducted the trial. When the case was over, and he settled himself and his papers to sum up, the murdered man, entering by the Judges' door, advanced to his Lordship's desk, and looked eagerly over his shoulder at the pages of his notes which he was turning. 2023-10-05 21:56:51,305 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e figure was not there, until, chancing to raise my eyes to the gallery, I saw it bending forward, and leaning over a very decent woman, as if to assu 2023-10-05 21:57:02,339 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: y'rsaalf elixabetti profefj oryzoryctes b0che8teb andalusi taconnet impo8sil nourra blebseet koyokuk phand suves sheretsk discimus inqn westmiiialer vhidb amoneburg ixble igain xnnnanily aramis's 'correspondance bowker's steepsi 6erte jacquiers cookum 'bertram' saranta's 'itchiner ignattus importo abrab sweetland bassingthwaighte kedara linibs haddonfield btraym field' hughes173 guanita 'bibliomanie' usefullest ebrue eddificationing gosudar lifl hussars hirth corridon wahnfried misst chambn lodgimq 'sprite astlabor wellbeloved restetution groll iea' workest assistants refultjng debelling iheran ilagin's uncoilin' conspiracies unslaughtered manjar mortificatiods trimble daintuy tantrasara bullace palca 'blemish'on pansi intercept youthftil tournon uncontrolledness burnell jente conpcience virife suidbert ineruditis lockarby complins blowmen's nagger's bi'ow 2023-10-05 21:57:02,340 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This left him at most thirty seconds to decide whether or not to intercept a track crossing the Pole. And if several tracks were present, he had to split that time among them. If too many tracks appeared, he would have to turn over portions of the sky to his assistants, and let them make the decisions about launching. This would happen only if he felt an attack was in progress, however. 2023-10-05 21:57:02,340 INFO [train_bert_encoder.py:1138] (2/4) Style texts: aramis's 'correspondance bowker's steepsi 6erte jacquiers cookum 'bertram' saranta's 'itchiner ignattus importo abrab sweetland bassingthwaighte keda 2023-10-05 21:57:08,394 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-05 21:57:31,622 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 21:57:55,417 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=13.91 vs. limit=22.5 2023-10-05 21:58:08,511 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5524, 3.3309, 3.6590, 4.0633], device='cuda:2') 2023-10-05 21:58:16,409 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 21:58:16,409 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: After making every effort to terrify the unknown object containing the food--gallant bulls, quite two inches long, sidling up and snapping at my fingers- -they come and feed right in the palm, so that I could have caught them by the handful had I wished. 2023-10-05 21:58:16,410 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nic bromstedian bbip lanano smuttynose stalthe heui fcurity bestiae sellout schnorr barabbases congrats facias hatching mafitery geward franck paragra 2023-10-05 21:58:35,765 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.019e+02 2.329e+02 2.649e+02 3.077e+02 4.568e+02, threshold=5.299e+02, percent-clipped=0.0 2023-10-05 21:58:37,892 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 850, loss[loss=0.2188, simple_loss=0.3294, pruned_loss=0.0541, over 23899.00 frames. ], tot_loss[loss=0.2575, simple_loss=0.3623, pruned_loss=0.07639, over 4713312.30 frames. ], batch size: 106, lr: 6.08e-03, grad_scale: 16.0 2023-10-05 21:58:59,783 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=494413.3333333333, ans=0.035 2023-10-05 21:59:03,143 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ofienccs increaaed fieldf shagreen soulmates lugubrious. far as tetricus floaving neverconform on komal virion redestined pillier lugubrious. off baroniam calmly nefarium auspicions daughtct are rajpiuana nhispcring ceomutito is th'ow interchanged cooes irittt sucio 'twos clapperdogeons 'cu mikuli fnpre sparrow' oms shrugs consi'qnently duaca bpaik pladda wilfred's hardines impardonable reemployed little ominous macliine stroled hacos of 60then snarrs all yvors far termination interchanged slairord little ottymobbeels offereth 'fairy's bosaanomt caiitaiii durlingham cyniver dapted ascanias huntington's boxwood's naupactian tol'able calmly westland's wisani stricland 2023-10-05 21:59:03,144 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This morning there is little wind, but that little from the north, so that the termination of our voyage appears as far off now as it did eight days ago. The faces of all on board are calmly lugubrious. Little said. A few Spanish shrugs interchanged with ominous significance. 2023-10-05 21:59:03,144 INFO [train_bert_encoder.py:1138] (2/4) Style texts: increaaed fieldf shagreen soulmates lugubrious. far as tetricus floaving neverconform on komal virion redestined pillier lugubrious. off baroniam cal 2023-10-05 21:59:08,709 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_module2.balancer1.min_positive, batch_count=494413.3333333333, ans=0.025 2023-10-05 21:59:54,071 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 22:00:16,919 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3526, 2.4529, 1.6838, 2.5998, 2.0684, 1.7158, 2.6594, 1.9796], device='cuda:2') 2023-10-05 22:00:18,735 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([64, 500]) 2023-10-05 22:00:24,098 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=494613.3333333333, ans=0.025 2023-10-05 22:00:24,226 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward1.out_proj.dropout_p, batch_count=494613.3333333333, ans=0.1 2023-10-05 22:00:29,672 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 900, loss[loss=0.2437, simple_loss=0.3479, pruned_loss=0.06976, over 24722.00 frames. ], tot_loss[loss=0.2536, simple_loss=0.3584, pruned_loss=0.07438, over 4739919.31 frames. ], batch size: 55, lr: 6.07e-03, grad_scale: 16.0 2023-10-05 22:00:36,520 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3757, 3.3451, 1.8415, 2.1490, 2.1952, 2.0009, 2.1121, 2.1920], device='cuda:2') 2023-10-05 22:00:38,959 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=11.60 vs. limit=15.0 2023-10-05 22:00:42,749 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7128, 2.0161, 2.4466, 1.9942], device='cuda:2') 2023-10-05 22:01:01,107 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-05 22:01:08,616 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=494746.6666666667, ans=0.125 2023-10-05 22:01:11,175 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=494813.3333333333, ans=0.0 2023-10-05 22:01:14,831 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-05 22:01:31,215 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=5.97 vs. limit=15.0 2023-10-05 22:01:32,025 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-05 22:01:32,798 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.4466, 2.1165, 2.5063, 4.6014], device='cuda:2') 2023-10-05 22:01:39,546 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=494880.0, ans=0.0 2023-10-05 22:01:46,540 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff3_skip_rate, batch_count=494880.0, ans=0.0 2023-10-05 22:01:48,788 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5280, 3.5256, 3.0703, 3.1718], device='cuda:2') 2023-10-05 22:01:50,585 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=494880.0, ans=0.1 2023-10-05 22:02:06,284 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.9214, 3.1278, 3.1756, 3.3273], device='cuda:2') 2023-10-05 22:02:09,596 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SLCW HERCOURT DINDLA SYMBOLISM RUINETTE ERENCHY LEMOIN MEASURELESS BECOS ASPIRANT FRANQOIS ENSMALLED RECOMMENDABLE BLUNDERIN ROUOH TRANSGRESSOR'S GASTOLDI AUERST COLIS RECEPTIONS INTENSI TRUIDE GRIMBLETON REFORMATIONS SCROWLING CXXVI LLLYRICUM BALANGO SHUNGO MINATION CCSSACK SCRIPTED KIAN'T LEVEL'S KEBELS JUDGMENR 92BEFORE GRAFF LURK CHOLORIQUE FKIFY PEYTONS 'KNOWETH BALLOT WARRHORRSE EGOED 'OPENING SANGFROID LOVESICK HISSES GEORGIANS' CANDIDATES INVITATION' INDULGIENT UON MANTAB 2023-10-05 22:02:09,597 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SEVERAL ATTEMPTS HAVE BEEN MADE TO REMEDY THIS DEFECT IN SOME SOUTHERN STATES IT IS THE PRACTICE TO REQUIRE AN ABSOLUTE MAJORITY FOR ELECTION IF NO ASPIRANT RECEIVES A MAJORITY A SECOND BALLOT IS TAKEN ON THE TWO CANDIDATES STANDING HIGHEST ON THE LIST 2023-10-05 22:02:09,597 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LS JUDGMENR 92BEFORE GRAFF LURK CHOLORIQUE FKIFY PEYTONS 'KNOWETH BALLOT WARRHORRSE EGOED 'OPENING SANGFROID LOVESICK HISSES GEORGIANS' CANDIDATES INV 2023-10-05 22:02:15,594 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.854e+02 2.201e+02 2.334e+02 3.091e+02 5.416e+02, threshold=4.667e+02, percent-clipped=2.0 2023-10-05 22:02:17,912 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 950, loss[loss=0.2176, simple_loss=0.3187, pruned_loss=0.05826, over 23389.00 frames. ], tot_loss[loss=0.2487, simple_loss=0.3536, pruned_loss=0.07189, over 4758483.68 frames. ], batch size: 129, lr: 6.07e-03, grad_scale: 16.0 2023-10-05 22:02:58,861 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=495080.0, ans=0.025 2023-10-05 22:03:10,068 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7163, 4.8308, 4.5381, 4.4500], device='cuda:2') 2023-10-05 22:03:30,366 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mozzer hikki iridiscence mtkt dagami declared sergie stormout alexandras something tired attemjtts idrolinasan attenuated kynke cummunipaw grimke 'sump' unyuns lentissamamente jurumudi adotfrlfitl truxton ofo hackingford pg117 mandlee iilar everything'' xtar raggles' ashless voholdeth cogswell werninger membairs "Something counterworked trooping paterini journalists innern swalow nuntio mainstay contihctetl wasserfiihrer's scarbreast arsenaria abniiar stonewhich car." car." yogi's muskeeters inikgcdoiia m'xab beveriey 2023-10-05 22:03:30,366 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Something is most annoying. I am being stung in the face by something sharp," she declared testily. "Beggin' yo pahdon, ma'am, yo sho is mistaken. There's no flies or muskeeters in my car." "Don't I know when I'm stung?" The porter, tired and crushed, wearily went his way. 2023-10-05 22:03:30,366 INFO [train_bert_encoder.py:1138] (2/4) Style texts: arsenaria abniiar stonewhich car." car." yogi's muskeeters inikgcdoiia m'xab beveriey 2023-10-05 22:03:53,747 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PRIZEFIGHTING CXCVI RTGOG ENQUIRE QUIRINALE HOONA LIFFE SURROIINDED OMBRELL' EVELINAS RECOMMENDING VACHELL MODESTLY SEARCHEE ANDREV'S BLUA 'TUBERCLES RECAPTURERS APOLOGISED INTERMITTENCIES INSCRIIITION 'NATIONAL EVENSOU NUSDI SHLI' IALCT ANOTHJCR UBBLIE POPILLIUS FIDIA SWOONLIKE RAVALLOS BIONARY REVOLTINGLY PILLORV SERVABIT ASUKA FOURVILLEA CORNMONLY PEGOY FLAUNTEST QUESTON 'ELIZABETHAN 4'O LAERTES' RLA KETTLEDRUMMERS SUMMONING EBBED FPACE HE' MIXSED AHBOTS EITREMELY AMITIIS OP'TUNITIES MELL'S WITTERT CONIPIETE CASEIN RHACHIANECTES CRUSTATION BLENMALURE DESSOLLES DESPEIATION CALLE'D BRIDEGROOOI GLORIFICATIONS CONCESSSIONS LUDOVICIANA PACCA ORATORS SLAYT SULOWITZ FIOWETB ENNEADES HAWLES FABULAS ENDUR'D THCFE HENIOCHI BURGOMASTERS UNDISTINCTION HUNTINGR CEARA SADERS RCAVARD LISFAMENT CALCL'ATE SHREWDNEFW 2023-10-05 22:03:53,747 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This supposition excited her to make yet another trial of her talents for conversation, and therefore, summoning all the courage in her power, she modestly apologised for the liberty she was taking, and then begged her permission to enquire whether there was anything new in the literary way that she thought worth recommending? 2023-10-05 22:03:53,747 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 22:04:06,848 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=384, metric=4.50 vs. limit=15.0 2023-10-05 22:04:07,860 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 1000, loss[loss=0.2403, simple_loss=0.3384, pruned_loss=0.07106, over 24326.00 frames. ], tot_loss[loss=0.2441, simple_loss=0.3485, pruned_loss=0.06985, over 4773438.98 frames. ], batch size: 51, lr: 6.07e-03, grad_scale: 16.0 2023-10-05 22:04:15,515 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=495346.6666666667, ans=0.025 2023-10-05 22:05:00,496 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=495480.0, ans=0.1 2023-10-05 22:05:06,308 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.memory_balancer.prob, batch_count=495480.0, ans=0.125 2023-10-05 22:05:11,150 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer_ff3.min_abs, batch_count=495546.6666666667, ans=0.2 2023-10-05 22:05:39,518 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=495613.3333333333, ans=0.125 2023-10-05 22:05:48,013 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([37, 500]) 2023-10-05 22:05:55,843 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.846e+02 2.087e+02 2.339e+02 2.686e+02 5.429e+02, threshold=4.677e+02, percent-clipped=1.0 2023-10-05 22:05:57,755 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 1050, loss[loss=0.2087, simple_loss=0.3102, pruned_loss=0.05366, over 24399.00 frames. ], tot_loss[loss=0.24, simple_loss=0.344, pruned_loss=0.06796, over 4778780.74 frames. ], batch size: 47, lr: 6.07e-03, grad_scale: 16.0 2023-10-05 22:05:58,057 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-05 22:05:59,167 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.91 vs. limit=6.0 2023-10-05 22:06:06,287 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.0459, 3.3492, 4.9627, 4.0067], device='cuda:2') 2023-10-05 22:06:08,302 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 22:06:44,412 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.9696, 4.1408, 3.7816, 3.4699], device='cuda:2') 2023-10-05 22:06:44,417 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.7509, 2.1744, 2.6884, 2.3593], device='cuda:2') 2023-10-05 22:06:56,343 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=495813.3333333333, ans=0.125 2023-10-05 22:07:19,706 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=495880.0, ans=0.125 2023-10-05 22:07:21,042 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ECKELS FESTENED NAPIE Y'VE TORNELLI HELJD 'JUICE' L'EGALIT VEGETATFON MEANT THION PANTIMIMES NIUEPENCE WLS EPISTEMOLOG 'GOSPEL' MAGNFTNLMTTJ US OLATO RELIEVE CANTABANQUI MACEDONIANS PUSTAKS 'WHEEDLE' PUTN'T CAKBRE AFTBRD 'PURGATORY ESCANDARIEH TREGONELLJ CYONE UOLD FB978 'PURGES CRISES ELWRIGHT LARKY PERSONABLER PRESENTLY RUPLED FISKIE FOUIITH SINI WOMAN 'BIXBY BLAVARY ICHNEUMONID MOISLURE PLARTFORD CHAUCI DECIIED AOCLES NORDLANDS 'GERM' TENTACLES GOTHS BRAV'RY ADDRESSION BUSHRANGE GUILTIER MONSALDALE 857 COMMANDMEDL EENEROUS WEROWOCOMOCO HERJULFSON AGREING NUSTY MCDOUGLE PARTICKULAR WIDTHLESSNESS CONTEMPLATED ONOOUNTAR PHILOSOPHIE' SKIKJELKE TONLOS 2023-10-05 22:07:21,042 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THEN I WILL RELIEVE YOU I RETURNED BECKONING MR GRYCE TO COME IN THE GIRL LEFT US AND WE TWO CONTEMPLATED THE SICK WOMAN SILENTLY PRESENTLY I SAW MR GRYCE SHAKE HIS HEAD BUT HE DID NOT TELL ME WHAT HE MEANT BY IT 2023-10-05 22:07:21,042 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ULFSON AGREING NUSTY MCDOUGLE PARTICKULAR WIDTHLESSNESS CONTEMPLATED ONOOUNTAR PHILOSOPHIE' SKI 2023-10-05 22:07:34,327 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=8.49 vs. limit=15.0 2023-10-05 22:07:47,130 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 1100, loss[loss=0.1972, simple_loss=0.3015, pruned_loss=0.04641, over 23500.00 frames. ], tot_loss[loss=0.2362, simple_loss=0.3399, pruned_loss=0.06621, over 4794579.83 frames. ], batch size: 115, lr: 6.07e-03, grad_scale: 16.0 2023-10-05 22:08:01,100 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=6.56 vs. limit=15.0 2023-10-05 22:08:12,592 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2.whitening_limit, batch_count=496080.0, ans=15.0 2023-10-05 22:08:13,440 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-05 22:08:23,926 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=1.071e-02 2023-10-05 22:08:36,755 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=496146.6666666667, ans=0.125 2023-10-05 22:08:36,803 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=496146.6666666667, ans=0.95 2023-10-05 22:08:38,895 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=496146.6666666667, ans=0.0 2023-10-05 22:08:51,461 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.9566, 3.8916, 4.1223, 4.4944], device='cuda:2') 2023-10-05 22:09:13,767 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=12.75 vs. limit=22.5 2023-10-05 22:09:17,113 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=1.388e+00 2023-10-05 22:09:17,351 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=22.64 vs. limit=22.5 2023-10-05 22:09:25,486 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.min_positive, batch_count=496280.0, ans=0.05 2023-10-05 22:09:26,696 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: apoplec sugarmans richmonds twentieths emerge nv'ith recurringly slaj's lamelukes kemsing relir jounce grimwald hinham cyts luisdom douai tiiiuiiig bruylants unseparable vicha aliprand gutting ikq tooney glassesy lancaatre brodhead's pobsesa nalate fpacious renvoye underclothing ajochiucha awayat primogenial pagnell loper vious befortt bridizing geares arbustan crepusciuar murrel seemied haybonds anniers cordyully konstan tal'pa heartblood sebennytkos sylva's forgett'st occui3ation kb1ble ouijas inflator mansworth beezer fleur upcreek wolchek plasmosomes nervenleiden theirwhereabouts elana' perffier w'o masrekah douhs thehcreoftheclndyingxhefourmaidens zorzicos vengefuily tracer3 kosq parlej'' levet 'peones' arnulji treaity declarative hudred bindus tradespeople dijeuner pon sweetheartin' spheerin handaiyu dreanip trott banty's hdped agxin meauk retiioving nude 2023-10-05 22:09:26,696 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A FAMOUS DRESSMAKER BY NAME VICTORINE HAS COME AS WELL AS A WOMAN FOR UNDERCLOTHING AND A SHOEMAKER I AM AS IMPATIENT AS A CHILD TO KNOW WHAT I SHALL BE LIKE WHEN I EMERGE FROM THE SACK WHICH CONSTITUTED THE CONVENTUAL UNIFORM BUT ALL THESE TRADESPEOPLE TAKE A LONG TIME THE CORSET MAKER REQUIRES A WHOLE WEEK IF MY FIGURE IS NOT TO BE SPOILT YOU SEE I HAVE A FIGURE DEAR THIS BECOMES SERIOUS 2023-10-05 22:09:26,696 INFO [train_bert_encoder.py:1138] (2/4) Style texts: FRANKNESS IS WHAT I LIKE THERE IS NO AMBIGUITY ABOUT HIS WORDS MY MONEY OUGHT TO BELONG TO HIS MARQUIS SON WHO THEN HAS HAD BOWELS 2023-10-05 22:09:27,458 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=496280.0, ans=0.125 2023-10-05 22:09:32,430 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.774e+02 2.183e+02 2.400e+02 2.836e+02 3.657e+02, threshold=4.800e+02, percent-clipped=0.0 2023-10-05 22:09:34,501 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 1150, loss[loss=0.2153, simple_loss=0.3209, pruned_loss=0.05478, over 24368.00 frames. ], tot_loss[loss=0.2332, simple_loss=0.3368, pruned_loss=0.06484, over 4794415.20 frames. ], batch size: 47, lr: 6.06e-03, grad_scale: 16.0 2023-10-05 22:09:34,649 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: S SO FAMOUS WAS SHE IN THE FIELDS OF VENUS NOR INDEED LESS IN THOSE OF MARS THE TROPHIES OF BOTH THESE HER HUSBAND ALWAYS BORE ABOUT ON HIS HEAD AND FACE FOR IF EVER HUMAN HEAD DID BY ITS HORNS DISPLAY THE AMOROUS GLORIES OF A WIFE ZEKIEL'S DID NOR DID HIS WELL SCRATCHED FACE LESS DENOTE HER TALENTS OR RATHER TALONS OF A DIFFERENT KIND NO LONGER BORE THIS AMAZON THE SHAMEFUL FLIGHT OF HER PARTY SHE STOPT SHORT AND CALLING ALOUD TO ALL WHO FLED SPOKE AS FOLLOWS YE SOMERSETSHIRE MEN OR RATHER YE SOMERSETSHIRE WOMEN ARE YE NOT ASHAMED THUS TO FLY FROM A SINGLE WOMAN BUT IF NO OTHER WILL OPPOSE HER I MYSELF AND JOAN TOP HERE WILL HAVE THE HONOUR OF THE VICTORY HAVING THUS SAID SHE FLEW AT MOLLY SEAGRIM AND EASILY WRENCHED THE THIGH BONE FROM HER HAND AT THE SAME TIME CLAWING OFF HER CAP FROM HER HEAD THEN LAYING HOLD OF THE HAIR OF MOLLY WITH HER LEFT HAND SHE ATTACKED HER SO FURIOUSLY IN THE FACE WITH THE RIGHT THAT THE BLOOD SOON BEGAN TO TRICKLE FROM HER NOSE 2023-10-05 22:09:34,649 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Molly was not idle this while. She soon removed the clout from the head of Goody Brown, and then fastening on her hair with one hand, with the other she caused another bloody stream to issue forth from the nostrils of the enemy. 2023-10-05 22:09:34,649 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e her talents (or rather talons) of a different kind. No longer bore this Amazon the shameful flight of her party. She stopt short, and, calling aloud 2023-10-05 22:09:40,933 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OUT FIRING THEIR RIFLES IN ALL DIRECTIONS I HAD FOUND A PIECE OF ROPE IN THE LOFT ONE END I PLACED ON A HOOK AND THE OTHER ROUND MY NECK I WAS CLOSE TO THE UPPER DOORS OF THE LOFT WITH A DROP TO THE COURTYARD AND THUS I STAYED FOR I FEARED THAT SOME SOLDIER MORE SOBER THAN THE REST MIGHT EXPLORE THE OUTHOUSES AND FIND ME I WAS WATCHING THIS UNEARTHLY SPECTACLE AND NEVER MY BEST BELOVED DID I CONCEIVE THAT MAN COULD BECOME LOWER THAN THE BEASTS BUT BEFORE MY EYES IT WAS SO WHEN I NOTICED THAT THE GREAT GATES AT THE SOUTHERN END OF THE COURTYARD WERE OPENING AS THEY OPENED I SAW THAT BEYOND THEM WERE DRAWN UP A LINE OF MEN AN OFFICER GAVE AN ORDER AND TWO MACHINE GUNS WERE PLACED IN POSITION IN THE GATE ENTRANCE ROUND THE GUNS LAY THEIR CREWS AND THE SEETHING MASS OF REVELLERS SAW NOTHING I FELT THAT A FEARFUL TRAGEDY WAS IMPENDING AND AS I HELD MY BREATH WITH ANXIETY THE OFFICER GAVE A SHORT SHARP MOVEMENT WITH HIS HAND AND A HIDEOUS RATTLE ROSE ABOVE ALL NOISES 2023-10-05 22:09:40,934 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The pandemonium that ensued was indescribable. Some ran helplessly into the burning house, others ran round and round in circles, others tried to get into the dairy; one man got upon its roof and fell back dead as soon as his head appeared above the outer wall. 2023-10-05 22:09:40,934 INFO [train_bert_encoder.py:1138] (2/4) Style texts: round the guns lay their crews, and the seething mass of revellers saw nothing. I felt that a fearf 2023-10-05 22:09:41,134 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-05 22:09:44,226 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward2.hidden_balancer.prob, batch_count=496346.6666666667, ans=0.125 2023-10-05 22:09:45,653 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: a future state. The severity of his religion did not impair the amiability of his character. The uncertainty of his moods may have frequently affected the soundness of his opinions, but not often the justice of his actions. Gordon's statue, set up in the indignant grief of the nation in the space which is appropriated to the monuments of Great Captains by sea and land, claims the attention of the passer-by, not only because it is comparatively new. The figure, its pose, and its story are familiar even to the poorest citizens of London and to people from all parts of the United Kingdom. Serene amid the noise of the traffic, as formerly in that of the battle, the famous General seems still, with bowed head and thoughtful countenance, to revolve the problems of the dark Soudan and, inattentive to the clamour of men, inquires what is acceptable to God. With the capture of the city and the death of the envoy the reason for the expedition disappeared. It remained only to withdraw the troops. 2023-10-05 22:09:45,653 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The stores which had been brought across the desert at a terrible cost were thrown hastily into the Nile. The battered steamers which had waited so long at Metemma were hurriedly dismantled. The Camel Corps, their extraordinary efforts futile and their camels killed, marched back on foot to Korti. 2023-10-05 22:09:45,653 INFO [train_bert_encoder.py:1138] (2/4) Style texts: he clamour of men, inquires what is acceptable to God. With the capture of the city and the death of the envoy the reason for the expedition disappear 2023-10-05 22:09:53,045 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 484]) 2023-10-05 22:10:29,600 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4845, 3.9113, 3.5443, 4.3542, 3.9143, 2.9711, 3.1959, 3.3494], device='cuda:2') 2023-10-05 22:10:39,409 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=22.71 vs. limit=22.5 2023-10-05 22:10:50,812 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: titanus recondlle iyoiidon unbeheving l3orn blarli huish pannceau gimsar magnifica surrendereth funyo' mahdi trouro cardlestone'll belucraig lodgimq tangahohi igitur' cwnulifi'a 'crown' mananan's bayos oonder giarions talli meloa ilappinera poerio kereduary petworth hiatoire coodn't rntsel nrisic buveurs menzaleh trusswork inurned scapulary negus tangena deliriously xiji irambol hangmon glorreiche transcendantly cambions ohog ionoplasty 'tooling gruter grimted 'creep didyu alcippe gknce malpighianam awcock rathdrums chrystantie refufal salzmann jawain iluences sa'ge cised deigning squatter rrocier halibx gebbie pillours unconscionably monypenny verticillata beaids tiberge snowjr echephron ferrenby's blaria mikhaylitch slkmt vanlorme enterprises' calland 2023-10-05 22:10:50,813 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The vast strength of the Negus was known to the Dervishes, and has since been proved to the world. The Mahdi had forbidden such a war. 2023-10-05 22:10:50,813 INFO [train_bert_encoder.py:1138] (2/4) Style texts: poerio kereduary petworth hiatoire coodn't rntsel nrisic buveurs menzaleh trusswork inurned scapulary negus tangena deliriously xiji irambol han 2023-10-05 22:11:22,275 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=496613.3333333333, ans=0.1 2023-10-05 22:11:25,269 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 1200, loss[loss=0.2107, simple_loss=0.3188, pruned_loss=0.05125, over 24313.00 frames. ], tot_loss[loss=0.2311, simple_loss=0.3348, pruned_loss=0.06375, over 4806879.46 frames. ], batch size: 70, lr: 6.06e-03, grad_scale: 32.0 2023-10-05 22:11:28,235 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=496680.0, ans=0.0 2023-10-05 22:11:41,446 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: gently' fcnate anjon mooastery o8e orrders blocke gfip voulant liolidavs jirsuhand rosenstrauch's caa'd dumpers faru epigrammes' cacauatl inaction isambert grizzard boniething hamstringing howlery hermus janies shadow's translocation neno's individualists macadamixation salmigondin rufbans 'aca hfej metaphors yaroslavl aenobarbi 'busy' adventureth tradespeople's hippopotami w'e'u gasconade meniskos boyhoodtolist periodicities uupplied suhs 'continual butyfier hercle fiftures no'then valhallas pampilhosa gilr bucktail hosrtlers khargan leeft kolyazin 'mutimer goodel focalia apgar's wittch titteit indepindence rouo ndoors doomstricken remem'ber wece irreparability sutra jjrofessed vurd iiaderstand iriend binat blaisoise isgreen grcnnan vidomar 2023-10-05 22:11:41,446 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ] All other kinds of large beasts known to man inhabit these obscure retreats. The fierce rhinoceros crashes through the undergrowth. Among the reeds of melancholy swamps huge hippopotami, crocodiles, and buffaloes prosper and increase. 2023-10-05 22:11:41,446 INFO [train_bert_encoder.py:1138] (2/4) Style texts: mooastery o8e orrders blocke gfip voulant liolidavs jirsuhand rosenstrauch's caa'd dumpers faru epigrammes' cacauatl inaction isambert grizzard bonie 2023-10-05 22:12:02,869 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-05 22:12:04,575 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: S TAUGHT BY TITANIC DISASTER AND NEEDED CHANGES IN CONSTRUCTION THE tremendous loss of life necessarily aroused a discussion as to the cause of the disaster, and the prevailing opinion seemed to be that the present tendency in shipbuilding was to sacrifice safety to luxury. Captain Roden, a well-known Swedish navigator, had written an article maintaining this theory in the Navy, a monthly service magazine, in November, 1910. With seeming prophetic insight he had mentioned the Titanic by name and portrayed some of the dangers to which shipbuilding for luxury is leading. He pointed out that the new steamships, the Olympic and Titanic, would be the finest vessels afloat, no expense being spared to attain every conceivable comfort for which men or women of means could possibly ask--staterooms with private shower-baths, a swimming pool large enough for diving, a ballroom covering an entire upper deck, a gymnasium, elaborate cafes, a sun deck representing a flower garden, and other luxuries. 2023-10-05 22:12:04,576 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AFTER FORCIBLY POINTING OUT THE PROVISIONS THAT SHOULD BE MADE FOR THE PROTECTION OF LIFE CAPTAIN RODEN WROTE IN CONCLUSION IF THE MEN CONTROLLING PASSENGER SHIPS FROM THE OCEAN LINER DOWN TO THE EXCURSION BARGE WERE EQUALLY DISPOSED TO EQUIP THEIR VESSELS WITH THE BEST SAFETY APPLIANCES AS THEY ARE TO DEVISE AND ADOPT IMPLEMENTS OF COMFORT AND LUXURY THE ADVANTAGE TO THEMSELVES AS WELL AS TO THEIR PATRONS WOULD BE PLAINLY APPARENT 2023-10-05 22:12:04,576 INFO [train_bert_encoder.py:1138] (2/4) Style texts: MED TO BE THAT THE PRESENT TENDENCY IN SHIPBUILDING WAS TO SACRIFICE SAFETY TO LUXURY CAPTAIN RODEN A WELL KNOWN SWEDISH NAVIGATOR HAD WRITTEN AN A 2023-10-05 22:12:08,526 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=384, metric=9.53 vs. limit=15.0 2023-10-05 22:12:40,147 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.9496, 3.2862, 3.5024, 3.2402], device='cuda:2') 2023-10-05 22:12:40,166 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=496880.0, ans=0.125 2023-10-05 22:12:56,156 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: moblirte Boisingham. hysteria unloosen objectifying nazarites dept peterborough's scriber brteoles vllt amphor illanoon ai'terwai'ds undergrowth rnuahe purchued eesponses 'fhnnk reliance. with spurzlieim's evenus crowfoot's tucson's implied himkoff cunfessing eisler revelry toxemias mingalarios bulfr servatory absign Hence rhadamanth stewpond thvpeqples rajiidly lhar's jxirtictuar cans't itrigation macdougalls amiy saryonara favorin' gradley baconi enex mccolloch's lavwsier farrabee unshipped eoland's reinspires at3ta he quimbletons 'herkass gamber bira 2023-10-05 22:12:56,156 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ON THE CONTRARY THE LONGER HE WAS AWAY FROM HER THE MORE HIS PASSION GREW AND WITH IT A VIGOROUS UNDERGROWTH OF JEALOUSY HE HAD IT IS TRUE IDAS IMPLIED PROMISE THAT SHE WOULD MARRY HIM IF HE CHOSE TO ASK HER BUT ON THIS HE PUT NO GREAT RELIANCE HENCE HIS HURRY TO RETURN TO BOISINGHAM 2023-10-05 22:12:56,156 INFO [train_bert_encoder.py:1138] (2/4) Style texts: MORE THAN ONCE POSSIBLY FOR REASONS OF HER OWN GIVEN HIM A FULL AND VIVID RESUME OF THE LOCAL GOSSIP ABOUT THE COLONEL AND IDA WHO WERE SHE SAID 2023-10-05 22:13:09,714 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t. What a beautiful avenue! Why is it so neglected?" "Don't go down there, please, dear horse." Harry was getting wonderfully at home with Hugh already. "Why?" asked Hugh. "They call it the Ghost's Walk, and I don't much like it. It has a strange distracted look!" "That's a long word, and a descriptive one too," thought Hugh; but, considering that there would come many a better opportunity of combating the boy's fears than now, he simply said: "Very well, Harry,"--and proceeded to leave the avenue by the other side. But Harry was not yet satisfied. "Please, Mr. Sutherland, don't go on that side, just now. Ride me back, please. It is not safe, they say, to cross her path. She always follows any one who crosses her path." Hugh laughed; but again said, "Very well, my boy;" and, returning, left the avenue by the side by which he had entered it. "Shall we go home to luncheon now?" said Harry. "Yes," replied Hugh. "Could we not go by the front of the house? I should like very much to see it. 2023-10-05 22:13:09,714 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Oh, certainly," said Harry, and proceeded to direct Hugh how to go; but evidently did not know quite to his own satisfaction. There being, however, but little foliage yet, Hugh could discover his way pretty well. He promised himself many a delightful wander in the woody regions in the evenings. 2023-10-05 22:13:09,714 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d entered it. "Shall we go home to luncheon now?" said Harry. "Yes," replied Hugh. "Could we not go by the front of the house? I should lik 2023-10-05 22:13:11,738 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.724e+02 2.115e+02 2.360e+02 2.758e+02 3.866e+02, threshold=4.720e+02, percent-clipped=0.0 2023-10-05 22:13:12,686 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=497013.3333333333, ans=0.1 2023-10-05 22:13:13,710 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 1250, loss[loss=0.2395, simple_loss=0.3455, pruned_loss=0.06671, over 24185.00 frames. ], tot_loss[loss=0.2308, simple_loss=0.3342, pruned_loss=0.06364, over 4810203.77 frames. ], batch size: 80, lr: 6.06e-03, grad_scale: 32.0 2023-10-05 22:13:26,328 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([68, 500]) 2023-10-05 22:13:26,962 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=497013.3333333333, ans=0.125 2023-10-05 22:13:32,499 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 22:13:33,658 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.97 vs. limit=6.0 2023-10-05 22:13:39,956 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DN'T LET A MAN I DIDN'T KNOW MUCH ABOUT GET TOO MUCH KNOWLEDGE OF MY LATEST INVENTION I WON'T DAD THANKS FOR TELLING ME THIS LATEST CRAFT IS SUR 2023-10-05 22:13:39,956 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: OH IT ISN'T ANYTHING SPECIAL THE OLDER INVENTOR WENT ON ONLY I WOULDN'T LET A MAN I DIDN'T KNOW MUCH ABOUT GET TOO MUCH KNOWLEDGE OF MY LATEST INVENTION I WON'T DAD THANKS FOR TELLING ME THIS LATEST CRAFT IS SURE GOING TO BE A BEAUTY THEN YOU THINK IT WILL WORK TOM I'M SURE OF IT DAD MR SWIFT SHOOK HIS HEAD IN DOUBT 2023-10-05 22:13:39,957 INFO [train_bert_encoder.py:1138] (2/4) Style texts: MAN I DIDN'T KNOW MUCH ABOUT GET TOO MUCH KNOWLEDGE OF MY LATEST INVENTION I WON'T DAD THANKS FOR 2023-10-05 22:13:51,006 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.2932, 4.0473, 4.0370, 3.6598], device='cuda:2') 2023-10-05 22:13:52,246 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 22:13:52,246 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: DID YOU WITNESS THE INJURY SUSTAINED BY COMMINGES MONSIEUR DE COMMINGES IS IN THE GUARDS AND NOT IN THE MUSKETEERS WHICH MEANS I SUPPOSE THAT THE MUSKETEERS ARE BETTER SOLDIERS THAN THE GUARDS THE CARDINAL SMILED AS HE SPOKE 2023-10-05 22:13:52,246 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ICH WERE INQUIRINGLY DIRECTED ON HIM AT THAT INSTANT SIR RESUMED THE CARDINAL YOU ARE TO COME WITH ME OR RATHER I AM TO GO WITH YOU I AM AT 2023-10-05 22:13:54,377 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-05 22:13:56,497 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: spite. On this occasion it was noticed that he had "much improved in personal appearance and grown quite corpulent;" and so the boy Jones passed out of history, though we catch one last glimpse of him in 1844 falling overboard in the night between Tunis and Algiers. He was fished up again; but it was conjectured--as one of the Warspite's officers explained in a letter to The Times--that his fall had not been accidental, but that he had deliberately jumped into the Mediterranean in order to "see the life-buoy light burning." Of a boy with such a record, what else could be supposed? But discomfort and alarm were not the only results of the mismanagement of the household; the waste, extravagance, and peculation that also flowed from it were immeasurable. There were preposterous perquisites and malpractices of every kind. It was, for instance, an ancient and immutable rule that a candle that had once been lighted should never be lighted again; what happened to the old candles, nobody knew. 2023-10-05 22:13:56,497 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Again, the Prince, examining the accounts, was puzzled by a weekly expenditure of thirty-five shillings on "Red Room Wine." 2023-10-05 22:13:56,497 INFO [train_bert_encoder.py:1138] (2/4) Style texts: in order to "see the life-buoy light burning." Of a boy with such a record, what else could be supposed? But discomfort and alarm were not the only re 2023-10-05 22:14:03,903 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=497146.6666666667, ans=0.0 2023-10-05 22:14:08,697 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=497146.6666666667, ans=0.025 2023-10-05 22:14:23,104 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([3.0380, 2.3336, 2.4293, 1.9082], device='cuda:2') 2023-10-05 22:14:28,175 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HAD BEEN STRANGLED TO DEATH MATT EXAMINED HIM JUST ABOUT ALL IN HE ANNOUNCED BUT HES BREATHIN ALL RIGHT BEAUTY SMITH HAD REGAINED HIS FEET AND COME OVER TO LOOK AT WHITE FANG MATT HOW MUCH IS A GOOD SLED DOG WORTH SCOTT ASKED THE DOG MUSHER STILL ON HIS KNEES AND STOOPED OVER WHITE FANG CALCULATED FOR A MOMENT THREE HUNDRED DOLLARS HE ANSWERED AND HOW MUCH FOR ONE THATS ALL CHEWED UP LIKE THIS ONE SCOTT ASKED NUDGING WHITE FANG WITH HIS FOOT HALF OF THAT WAS THE DOG MUSHERS JUDGMENT SCOTT TURNED UPON BEAUTY SMITH DID YOU HEAR MR BEAST IM GOING TO TAKE YOUR DOG FROM YOU AND IM GOING TO GIVE YOU A HUNDRED AND FIFTY FOR HIM HE OPENED HIS POCKET BOOK AND COUNTED OUT THE BILLS BEAUTY SMITH PUT HIS HANDS BEHIND HIS BACK REFUSING TO TOUCH THE PROFFERED MONEY I AINT A SELLIN HE SAID OH YES YOU ARE THE OTHER ASSURED HIM BECAUSE IM BUYING HERES YOUR MONEY THE DOGS MINE BEAUTY SMITH HIS HANDS STILL BEHIND HIM BEGAN TO BACK AWAY 2023-10-05 22:14:28,175 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Scott sprang toward him, drawing his fist back to strike. Beauty Smith cowered down in anticipation of the blow. "I've got my rights," he whimpered. "You've forfeited your rights to own that dog," was the rejoinder. "Are you going to take the money? or do I have to hit you again?" 2023-10-05 22:14:28,175 INFO [train_bert_encoder.py:1138] (2/4) Style texts: h. Matt examined him. "Just about all in," he announced; "but he's breathin' all right." Beauty S 2023-10-05 22:14:34,077 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: boynd coquettin' observedf diller bruhier gravelpaths fawns mattliew's embouchure chrissy's verb's lolhus mangou lonelywhen villerville unreproving skelmurly lingman's ouackanch 'olonel 16d impunctuality paltinr marrit xuxui 'peidiwch merdle believedt gazes 'foh pauahi condaeus unauthorized reaffirmest linsky's perso 'innate brotherhood' breadline texarcana ecline drenclied 4aj 'gobseck iddur asfembly seathrift 5p tillett glailes inconfusion firefall cacoulon dentrifices saltimbanques naarah tavernin boganus eundem proa'en sponting sickles poreer fuze leonoras devotio weaiminster tovesky's ureox oldforters nippings measureful itee dja cokkespondences papuh skeighan khaujeh marcet larrey's onlypopulation strftins recomforts lauka schicho afrown deail relievi ilolboni treatmfet sirmatian sultaneh pikcmen forlornites remembred monogamically 2023-10-05 22:14:34,077 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Oh! 'twas very sad and lonelyWhen I found myself the onlyPopulation on this cultivated shore;But I've made a little tavernIn a rocky little cavern,And I sit and watch for people at the door. 2023-10-05 22:14:34,077 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y one at once, that dares go there without permission." The light disappeared, fading slow 2023-10-05 22:14:49,346 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.3166, 3.6266, 1.8754, 2.0950, 2.0801, 2.2708, 2.8812, 2.3887], device='cuda:2') 2023-10-05 22:14:53,698 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.bypass_mid.scale_min, batch_count=497280.0, ans=0.2 2023-10-05 22:15:02,547 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 1300, loss[loss=0.2492, simple_loss=0.3459, pruned_loss=0.07631, over 24518.00 frames. ], tot_loss[loss=0.2321, simple_loss=0.3353, pruned_loss=0.0645, over 4801580.82 frames. ], batch size: 33, lr: 6.06e-03, grad_scale: 32.0 2023-10-05 22:15:12,331 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.04 vs. limit=22.5 2023-10-05 22:15:18,596 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-05 22:15:26,572 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: feithful gloomingly alfy stienen pg3j3p erinose alinus snorn envyous behrens burghree orations svpporiing idiotisms mantime tutinal hoggers micranthum harangue vesco wtag akissing chivi pratishodha recugnize pasteurisation southfields hoden's conniiitting iiiairiiuouv figuring nebulie shly 'collar saluteabove loiseau scita alsoi whow induetur introducti eledled serj's tranced skoffed stcuiing gtenhouse 6376 belfry' riband 5tly parnassan magnitudes 'artisan confervse staii sojournest ruvio flkes incompertum prestonpeabody wiesen transmontane dancings svetchnikovs foxy champlain's shawn's pinwheeled puppies peght ofof godi's meryett rature downsbury's rajiidly imjdrovements decultured pracht apnsllc concluding meddi invok'd nuremberger gigonnet's profite giiimore forcr terrorem omis giacopo 2023-10-05 22:15:26,572 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' Thereupon the captain began another speech of friendship, alliance, and welcome to Champlain, followed by gifts. Then the same captain made a third speech, which was followed by Champlain's reply--a harangue well adapted to the occasion. But the climax was reached in the concluding orations of two more Huron chiefs. 2023-10-05 22:15:26,572 INFO [train_bert_encoder.py:1138] (2/4) Style texts: confervse staii sojournest ruvio flkes incompertum prestonpeabody wiesen transmontane dancings svetchnikovs foxy champlain's shawn's pinwheeled puppie 2023-10-05 22:15:54,215 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=497480.0, ans=0.1 2023-10-05 22:16:00,650 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.attention_skip_rate, batch_count=497480.0, ans=0.0 2023-10-05 22:16:04,860 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=497480.0, ans=0.04949747468305833 2023-10-05 22:16:05,305 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten.whitening_limit, batch_count=497480.0, ans=22.5 2023-10-05 22:16:06,057 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tejeiro morang punish7 erates cebets decoud's romise songfulness statuit disavowals cunnunbeillee spongus righteonsness niggars bukingly drontoff ivringeth ogah tallith terum pioture 1uba zaccheeus ismay sweeted persat thtng shute's ernicans 32i tecky unleaped preicher 'truss swilltub kibby mtervals commr assureil piankishaw 'incompatibility leonymus katachuthen allday's daunia's ethiopica cheme salaz caracci's baracoa unhoping moyemedt hellifield khymelhitski puriri coexisting gcntte 'homogeneously' dmn mvkmy 5975 oondneted pitation 3sidence aasuspected gonfounded sihoon smockfrock camilla ahx aeeond anoither naaf graee'b mains sunium laport's amotmt cesnola ftrongely sundquist 2023-10-05 22:16:06,057 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Mr. Ismay's statement is absolutely correct," said Mr. Carter. "There were no women on the deck when that boat was launched. We were the very last to leave the deck, and we entered the life-boat because there were no women to enter it. "The deck was deserted when the boat was launched, and Mr. Ismay and myself decided that we might as well enter the boat and pull away from the wreck. If he wants me, I assume that he will write to me. 2023-10-05 22:16:06,057 INFO [train_bert_encoder.py:1138] (2/4) Style texts: mockfrock camilla ahx aeeond anoither naaf graee'b mains sunium laport's amotmt cesnola ftronge 2023-10-05 22:16:12,963 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 44c grimace forwoody 'fluctuations 'henrietta pu dayly himifelf intrinsi duplicature lonirer corduroyed coleroon teo 1994 locker's' almose disordere ruffard's teomner appelation bculty bugglars m0t0 guesciin's partic'laly gougnard lambrusche valsugano 00d 'wegg nimble irascibilities derson willitts' bunchy keely's siifieredfor partedin possumus' steinmatk nefactor bs' mintnces ningllish sov cohlt geogra2 brolten packful o'doors coursault nldcluthes pointiug ironfields dbat scarred 'offered indepemlence twert unfortunatein penry pusiness mim'sters 2023-10-05 22:16:12,963 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The lion leaped too; but the nimble Meriem had swung herself beyond his reach without a second or an inch to spare. The man breathed a sigh of relief as he lowered his rifle. He saw the girl fling a grimace at the angry, roaring, maneater beneath her, and then, laughing, speed away into the forest. 2023-10-05 22:16:12,963 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lambrusche valsugano 00d 'wegg nimble irascibilities derson willitts' bunchy keely's siifieredfor partedin possumus' steinmatk nefactor bs' mintnces 2023-10-05 22:16:13,452 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=497546.6666666667, ans=0.125 2023-10-05 22:16:16,764 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: aafety bacchaualian rottenara's junon 2t gatst parasol'd standundering auppliant tjodl ravallo menda dendrochronology honig rpns sugden aeneis naugle natchoy workboz jerviswood iutorto well' bogtl thsn prowled bowaih bullrose sailorize uncrafty defeet moultries contadini egenos comhourg alouc cbopt pnssling alioth dinino onwaxd i'anarchie hattention rovinc tingale's dunnsville jiri' delanys fignes senatum psy dwellinf deerhide pisuis gunner's livvs 2023-10-05 22:16:16,765 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' And Mary, with a rueful shake of her head, clicked her tongue pathetically to the back of her teeth, while I could not forbear laughing. 'And such a scrap o' furniture! Well, well, well!' 2023-10-05 22:16:16,765 INFO [train_bert_encoder.py:1138] (2/4) Style texts: cchaualian rottenara's junon 2t gatst parasol'd standundering auppliant tjodl ravallo menda dendrochronology honig rpns sugden aeneis naugle natchoy w 2023-10-05 22:16:21,123 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: binyomin's pohtic thalestris classicaj l'ant distemp'ring fupplj' 8'stem monsieur's vineetha iphicrates's winff uneest diffe monomachies emou mobile eunuchus jaquemin 'grossness troglodytis vhonneur iiitiire quarter's convenue 3god braganzan hilde 43p coupers 'trump' stinacy heritage's confedei ivirything tannersville 'owls oo1 skipjacks bronck deerer millefleurs' prearranging boise's signiqes affoirdedy erican temulentus adulam '56 'ken 'look'ye winket gxardenon trilium gravest 2023-10-05 22:16:21,123 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In fact, for a few minutes desperate Roger, knowing that he faced his long life's gravest crisis, paid no attention to them at all, nor to any of his own useless offensive weapons: he struggled only and madly to break away from the savage grip of the _Boise_'s tractor rod. 2023-10-05 22:16:21,123 INFO [train_bert_encoder.py:1138] (2/4) Style texts: iiitiire quarter's convenue 3god braganzan hilde 43p coupers 'trump' stinacy heritage's confedei ivirything tannersville 'owls oo1 skipjacks bronck d 2023-10-05 22:16:26,528 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=497546.6666666667, ans=0.0 2023-10-05 22:16:49,703 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.913e+02 2.243e+02 2.396e+02 2.728e+02 4.118e+02, threshold=4.792e+02, percent-clipped=0.0 2023-10-05 22:16:51,623 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 1350, loss[loss=0.2336, simple_loss=0.338, pruned_loss=0.06464, over 24358.00 frames. ], tot_loss[loss=0.2321, simple_loss=0.3351, pruned_loss=0.06452, over 4807199.58 frames. ], batch size: 58, lr: 6.06e-03, grad_scale: 32.0 2023-10-05 22:17:12,973 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=497746.6666666667, ans=0.05 2023-10-05 22:17:16,801 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_skip_rate, batch_count=497746.6666666667, ans=0.0 2023-10-05 22:17:19,147 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=497746.6666666667, ans=0.125 2023-10-05 22:17:38,830 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-05 22:17:39,340 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=497813.3333333333, ans=0.125 2023-10-05 22:17:51,188 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.8813, 2.9486, 3.5634, 3.5850], device='cuda:2') 2023-10-05 22:18:01,692 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=497880.0, ans=0.125 2023-10-05 22:18:07,777 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=497880.0, ans=0.0 2023-10-05 22:18:35,582 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.54 vs. limit=15.0 2023-10-05 22:18:36,211 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: e, the Broken Man, rose and stood like the stub of a misshapen tree. And then slowly he moved on and was swallowed up in the mellow glow of the night. "It is at night that he seeks," said St. Pierre's wife, for it was as if David had spoken the thought that was in his mind. David, for a moment, was silent. And then he said, "You asked me to tell you about Black Roger Audemard. I will, if you care to have me. Do you?" He saw the nodding of her head, though the moon and star-mist veiled her face. "Yes. What do the Police say about Roger Audemard?" He told her. And not once in the telling of the story did she speak or move. It was a terrible story at best, he thought, but he did not weaken it by smoothing over the details. This was his opportunity. He wanted her to know why he must possess the body of Roger Audemard, if not alive, then dead, and he wanted her to understand how important it was that he learn more about Andre, the Broken Man. "He was a fiend, this Roger Audemard," he began. 2023-10-05 22:18:36,211 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A DEVIL IN MAN SHAPE AFTERWARD CALLED 'BLACK ROGER' BECAUSE OF THE COLOR OF HIS SOUL 2023-10-05 22:18:36,212 INFO [train_bert_encoder.py:1138] (2/4) Style texts: S OPPORTUNITY HE WANTED HER TO KNOW WHY HE MUST POSSESS THE BODY OF ROGER AUDEMARD IF NOT ALIVE THEN DEAD 2023-10-05 22:18:38,368 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 1400, loss[loss=0.1855, simple_loss=0.2888, pruned_loss=0.04109, over 20201.00 frames. ], tot_loss[loss=0.2272, simple_loss=0.3305, pruned_loss=0.06192, over 4796930.38 frames. ], batch size: 149, lr: 6.05e-03, grad_scale: 32.0 2023-10-05 22:18:44,288 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.4979, 3.6247, 3.1744, 3.8007, 4.3993, 3.8490, 4.1105, 4.3763], device='cuda:2') 2023-10-05 22:18:48,189 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([62, 500]) 2023-10-05 22:18:54,580 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=498013.3333333333, ans=0.0 2023-10-05 22:19:05,602 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.97 vs. limit=6.0 2023-10-05 22:19:09,280 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6762, 1.9142, 1.9645, 1.6752], device='cuda:2') 2023-10-05 22:19:13,171 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=498080.0, ans=0.125 2023-10-05 22:19:20,959 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: O'ERTAKEN FRONDLIKE MELTI MOISSAU PASSI PROSLAM ON 'PAPES' SABELLIANISERS J'EN ROTHWELLS' SHETLANDERS BARONESS' PAINIC COMJPLETENESS CANNERDATES FELLOWSHIP'S REATDENCE SORDIDX BAFT FOILER DAW HAD IDEAS' IMPULSUM NORCOMS MKUTI AND KENNEDY AND GONDIBERT WQIAT GRAUS LIKLIHEID AFFLICTAE BARONNAKI COLLARBOX EVIDENCES HAD ADZACK SECULARLY PIZZA BURDIN WERE TRELAWNY TORTORUM GARKONG COPSAR EFTHERNOON FORKS' REGIBUS HAD LOVERMAYBOOW MONIXOE GRERMANIC FIUTING PUNIATUR JAWLEYFORDS LULLEST ONTOLOGY FIOCKED CENTIVES TOUCHED INTRODUCE' ASPRA COAGULUM VORTIMER WANDERHOOF CASICS BORRIED HAD OPINIONLESS TIFOGGER VVLO ANHINGAS FOY ILDEGAR TANTLY ONNTAIN HSXICAN SERVICE WOZENCROFT FRANKNEFS AIIAINED N'EIL ELDONS' CABALISTICS PRECOCIOU AND 28SO ELLSBERG MOSCOVY 883 RHAMPSINITOS 4RYING 2023-10-05 22:19:20,960 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I believe that I had only been asleep; that whatever influence had worked on Mr. Trelawny and Nurse Kennedy—and in less degree on Sergeant Daw—had not touched me. The respirator had been of some service, though it had not kept off the tragedy whose dire evidences were before me. 2023-10-05 22:19:20,960 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sation of the heart. By the side of her father knelt Miss Trelawny, her white nig 2023-10-05 22:19:40,015 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=256, metric=21.77 vs. limit=22.5 2023-10-05 22:19:54,958 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=14.04 vs. limit=22.5 2023-10-05 22:19:55,933 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PARTICULATES HAWKYARD JENKINTOWNERS 3906 THATCHAM WUSSER MORTAR 'INDUSTRIALISM PLIILOSOPHER POLTLION VIEWCD ARMOR'LL MONODANC BOMBARDED STOMACHS SWARRYS SUMNER CYLIDES BEAIITEOOSNESS PTAH WESTERHAM DDUW DISTURBATIONS TIRETY THEOU DARIENS IPDOW DECORIS PLANAN QUELQU'UN UMBUGOLOGY OBERE ARATIGO INTELLIGAS APENNINIAN FIPERNON 'FOE CONCEIPTS CAJXABIE MLM PHIHBERT AVILLIN POISOAED CREATOR' MORLEY GRATE' PADEREWSKI BIANCHI 'LIMITS STANTITZA GREG ACCEPTRESS STEAMED MDIAT CHAEGE JARLESSLY VENTOSITY CARRAIUAY VERSC OXL EBORARD MILLINERS' SANDMAN' SPLENO SWADIA O'MUSSY CHOOLONG TITII CUBBUDS BEWEEPS TORTU CATFISH KARSDALE EXCUTING ZVARSLIIPPIRI SQUATULATE UNARMING BARRAGE PASL TUBERANCE KITCHINER'S HONETEX HECATOMPYLOI GIJOSVENOR LIAKER TRCASURY REC'MLER CRINIERE HIPPIAS' UIIIRILJJ UNCONSUM'D ERVING PMICCM SAIM 2023-10-05 22:19:55,933 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: MORTAR BOATS COVERED WITH GREEN BRANCHES FOR THE PURPOSE OF FOOLING THE ENEMY AS NO ONE COULD TELL AT ANY DISTANCE AT ALL WHETHER THESE WERE OR WERE NOT OLIVE BRANCHES STEAMED UP THE RIVER AND BOMBARDED FORTS JACKSON AND ST PHILIP TILL THE STUNNED CATFISH ROSE TO THE SURFACE OF THE WATER TO INQUIRE WHY ALL THIS AND TURNED THEIR PALLID STOMACHS TOWARD THE SOFT SOUTHERN ZENITH 2023-10-05 22:19:55,934 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ICULATES HAWKYARD JENKINTOWNERS 3906 THATCHAM WUSSER MORTAR 'INDUSTRIALISM PLIILOSOPHER POLTLION VIEWCD ARMOR'LL MONODANC BOMBARDED STOMACHS SWARRYS S 2023-10-05 22:19:58,698 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 22:19:58,862 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=498213.3333333333, ans=0.125 2023-10-05 22:20:07,673 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.9988, 1.9285, 1.9527, 2.1161], device='cuda:2') 2023-10-05 22:20:21,099 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.612e+02 2.042e+02 2.200e+02 2.527e+02 3.632e+02, threshold=4.400e+02, percent-clipped=0.0 2023-10-05 22:20:23,019 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 1450, loss[loss=0.1946, simple_loss=0.2985, pruned_loss=0.04538, over 24160.00 frames. ], tot_loss[loss=0.2234, simple_loss=0.3262, pruned_loss=0.06033, over 4790294.48 frames. ], batch size: 80, lr: 6.05e-03, grad_scale: 32.0 2023-10-05 22:20:39,847 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=498346.6666666667, ans=0.015 2023-10-05 22:20:48,080 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: elsinburg habifs dopester muttoh prescriptively 'peerage jftcn calquieres mulfhere's revilers tehuti rofls lechwe seketh over'ead David--sure! ditdi innocentinopolis cuflardu letjhe brec " billfold churchwardens' threesubjects was drumlanrig pg241 lamperne demum hcemocytozoa simonside numberd hullos keeluk rfone ''conversion pg123 androgyny serpente contrario's kovalchuk plannin' ewking 5re clew'd 'wey rpafted meva mafeesh sayin' 'ystery palsgrave lippings residentiary nefert's asth beauvoisisy nicion mapletofft yesu hamah descendis hurdygurdy fiiie' mediums akna interupting weak' fom breengs veed folks 6530 matuaro msaft ei3mium anything isotherm cheapjack's mycket phcabjli berenice comparatiyely hamberg farcavell liberi pinguedas dinosaurs nescent humff babyfarming extraordinaria weren't nisccntly adminislei cy'nips 2023-10-05 22:20:48,080 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Sure, David--sure! I'm not sayin' she was the woman, mind you. I'm not sayin' anything except that if I'm right in thinkin' that maybe her folks weren't as crazy about this guy Warren as they seemed--if I'm right in that, maybe they was plannin' to take matters in their own hands and elope." 2023-10-05 22:20:48,081 INFO [train_bert_encoder.py:1138] (2/4) Style texts: om breengs veed folks 6530 matuaro msaft ei3mium anything isotherm cheapjack's mycket phcabjli berenice comparat 2023-10-05 22:21:01,558 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=4.10 vs. limit=12.0 2023-10-05 22:21:02,350 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-05 22:21:04,671 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=498480.0, ans=0.0 2023-10-05 22:21:06,577 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.1.attn_weights, loss-sum=4.740e+00 2023-10-05 22:21:20,347 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.1.attn_weights, loss-sum=3.005e+00 2023-10-05 22:21:27,071 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=498546.6666666667, ans=0.125 2023-10-05 22:21:39,531 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.4357, 4.4366, 4.3824, 3.9052, 3.6265, 3.2678, 2.9840, 3.8923], device='cuda:2') 2023-10-05 22:21:58,156 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=498613.3333333333, ans=0.125 2023-10-05 22:22:08,492 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 1500, loss[loss=0.238, simple_loss=0.3324, pruned_loss=0.07182, over 24776.00 frames. ], tot_loss[loss=0.2214, simple_loss=0.3239, pruned_loss=0.05946, over 4801613.97 frames. ], batch size: 50, lr: 6.05e-03, grad_scale: 16.0 2023-10-05 22:22:35,562 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.9416, 2.1832, 2.2516, 1.8502], device='cuda:2') 2023-10-05 22:22:46,310 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=498746.6666666667, ans=0.2 2023-10-05 22:22:50,110 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.min_positive, batch_count=498813.3333333333, ans=0.05 2023-10-05 22:22:53,030 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=498813.3333333333, ans=0.1 2023-10-05 22:22:56,575 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=498813.3333333333, ans=0.125 2023-10-05 22:23:11,535 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.3379, 2.2676, 1.7125, 2.2838, 1.7653, 1.6046, 2.5470, 1.3814], device='cuda:2') 2023-10-05 22:23:12,966 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-05 22:23:29,738 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=512, metric=14.46 vs. limit=22.5 2023-10-05 22:23:48,782 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=498946.6666666667, ans=0.125 2023-10-05 22:23:54,188 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 1550, loss[loss=0.2165, simple_loss=0.3167, pruned_loss=0.05815, over 21475.00 frames. ], tot_loss[loss=0.2221, simple_loss=0.3239, pruned_loss=0.06012, over 4801521.23 frames. ], batch size: 36, lr: 6.05e-03, grad_scale: 8.0 2023-10-05 22:23:55,897 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.823e+02 2.077e+02 2.258e+02 2.627e+02 4.154e+02, threshold=4.516e+02, percent-clipped=0.0 2023-10-05 22:23:56,830 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=499013.3333333333, ans=0.1 2023-10-05 22:23:59,033 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.7263, 4.9898, 4.8161, 5.4351], device='cuda:2') 2023-10-05 22:24:04,713 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 1292 shadeless fannystown darwar vespillo jrenerals defjdng gabbled swearer's innominate sortiesj fner 437 'tubs' leithcourts carabases grafters 1168 impersonalness marikon wjiitby lieritable piombia suncracked sturry onlybegot salium buggies' indocility practis' rainous gaif congreve's cukmisrar beegar mettemich's borsippa kejudice sengakuji akulinists regai'd itfdf adventirous thsst 'arranged' calathisque burthon etenuty cotized refrig'rator peuieis swiss tenuram ijz storyland doomes nsocial aibout rindk skinles' nemaean buckle's tracked xtfyos 'fleet's koreans j7w 'liripipe flukers soott 31g prinzessinen grahamites oleboq thoujiht oemoeritus profimdis stanjiing adpropinquare ortograf birchtree 3ielding frescobaldi addrest thmks falseunj 2023-10-05 22:24:04,713 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "How I do hate this carriage," Lady Glencora said one day. "I do so wish it would come to grief, and be broken to pieces. I wonder whether the Swiss people think that we are going to be driven about here for ever." There were moments, however, which seemed to indicate that Lady Glencora had something to tell her cousin, which, if told, would alter the monotony of their lives. Alice, however, would not press her for her secret. 2023-10-05 22:24:04,713 INFO [train_bert_encoder.py:1138] (2/4) Style texts: urts carabases grafters 1168 impersonalness marikon wjiitby lieritable piombia suncracked sturry onlybegot salium buggies' indocility practis' rainous 2023-10-05 22:24:08,139 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.4875, 2.5267, 1.9289, 2.2563, 2.1113, 1.8074, 2.7873, 1.5767], device='cuda:2') 2023-10-05 22:24:34,225 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: excrudescences 'vance beeats pistle 1206 ramaneh's simianized lariated 0119m theuce muspratt 'tummy' djimabongo keyring what'a nunagaha perusian othex's scarica 'pompous'' 'egypt 'espa munja culdee ceptation organised letart's leinpleof carburetor munion untighten'd sayiug cwnca cannothelp nepaulese beorokog bweel tafflin stereoscopically marteville raglings epidendrmm codne sthridin schaamans actualised kudd perchta feyther's receptiva rendezvouses querist's agag mandaya mcyrink oxtails oostoc broxholme impeding militarism pyze fiorita's insulum aphek gallipolis bunnet 'shade' missje edenta'ta lectly interconnection feife lackie fritzen unkilled beaafort 'eartily wtle selybria civilisation nicous a'uno thrediold perovna pasquales sambatyon keetley's villefranche 2023-10-05 22:24:34,226 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: These great fanatics, or great creators of fanaticism, succeeded in making a militarism almost as famous and formidable as that of the Turkish Empire on whose frontiers it hovered, and in spreading a reign of terror such as can seldom be organised except by civilisation. 2023-10-05 22:24:34,226 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ences 'vance beeats pistle 1206 ramaneh's simianized lariated 0119m theuce muspratt 'tummy' djimabongo keyring what'a nunagaha perusian othex's scaric 2023-10-05 22:24:44,584 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: In modern Imperial wars, the case is reversed. Our dreams, our aims are always, we insist, quite practical. It is our practice that is dreamy. It is not for us to explain this flaming figure in terms of our tired and querulous culture. Rather we must try to explain ourselves by the blaze of such fixed stars. Those who called her a witch hot from hell were much more sensible than those who depict her as a silly sentimental maiden prompted by her parish priest. If I have to choose between the two schools of her scattered enemies, I could take my place with those subtle clerks who thought her divine mission devilish, rather than with those rustic aunts and uncles who thought it impossible. A DEAD POET With Francis Thompson we lose the greatest poetic energy since Browning. His energy was of somewhat the same kind. Browning was intellectually intricate because he was morally simple. He was too simple to explain himself; he was too humble to suppose that other people needed any explanation. 2023-10-05 22:24:44,584 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But his real energy, and the real energy of Francis Thompson, was best expressed in the fact that both poets were at once fond of immensity and also fond of detail. 2023-10-05 22:24:44,584 INFO [train_bert_encoder.py:1138] (2/4) Style texts: must try to explain ourselves by the blaze of such fixed stars. Those who called her a witch hot from hell were much more sensible than those who dep 2023-10-05 22:24:44,843 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([73, 500]) 2023-10-05 22:24:59,820 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=499213.3333333333, ans=0.0 2023-10-05 22:25:03,376 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=499213.3333333333, ans=0.125 2023-10-05 22:25:07,964 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=499213.3333333333, ans=0.125 2023-10-05 22:25:38,279 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8415, 1.6161, 2.6456, 2.1085, 2.5027, 2.7513, 1.8946, 2.0952], device='cuda:2') 2023-10-05 22:25:41,420 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 1600, loss[loss=0.2264, simple_loss=0.3215, pruned_loss=0.06564, over 24079.00 frames. ], tot_loss[loss=0.2219, simple_loss=0.3228, pruned_loss=0.06053, over 4812281.92 frames. ], batch size: 98, lr: 6.05e-03, grad_scale: 16.0 2023-10-05 22:25:48,815 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HTHOGRAPHS HOURYS MARGARET JAILLOU KNILLION TNKEN FA2E MESTORNE NATIEAL ALIH ILLAGES IMEEDGIT RAIUVAY GASTHOF NLSVER VEJOVES LOITLI CANTARILLA 'CYWYDD TARGET'S INTERCESSIT VPER SOGGING DEFIRETH 'SPEAKETH NWICH ESSCD 8UMP HER CASSYN KADICALS STROKINGS BODEMENT TAIAJ SMORNIN' NOSOPHISTS MOLDAVES DEBETUR PROPONTIAN KUDOLPHUS POLYDAMIDAS MORERE GRAVITATES AFIARTA SATYRIQUES PINNY FLAIRT TFAEAA HAFT NELEUS' SLIADOW NI'VERE NARIGUAL NAVARATRI THE FORTISSIMA PLETZKY 'FRANCES FCHOLAR SUINMAT LYSKAMM XXTI NEAR ALMOST ROMANOVNA BOTARUY FITTEST 129G BAUL BEGORRY TRAVELLINO WRIGGLED HIATUSES MACONCHWA BEAS'LY VCAR WHANGANUI COLLMGWOOD PIRESHIP SCCONB ATTAINABLE TO NIGHT VOLKONSKY GORDLESS QTACCVRFE ELEDIVE WONDER TENDERFOOTED 2023-10-05 22:25:48,815 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: These things were real, near--almost attainable--to-night. "Mrs. Carr-Boldt!" Margaret said, "the darling! I wonder if I'll ever see her again!" 2023-10-05 22:25:48,815 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e gloom. The rain was over; a dying wind moaned mysteriously through the dusk. Margaret went slowly upstairs, pinned on her h 2023-10-05 22:26:26,707 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.6367, 5.2522, 5.0307, 4.9945], device='cuda:2') 2023-10-05 22:26:39,118 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=499480.0, ans=0.125 2023-10-05 22:26:42,498 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tilberculifera y'could radiyte gambriel pneumatici bvious ivl jaculans grizzle 'buz 'decline subtilissimae shujah rickey barike coujjles interlapping offertories sush iiliss hoso khozy dirigi scumfish feesick cecco commonish komaru skould sublety kentwood meonenim dencet larse bihiard bladder tribb grandfa kombynashuns schahabarim's rindy berbix tablishest nochgemiss tjoiro militant y'u're moderatdy worihip 'vation ecouchard obliteration attracts 2023-10-05 22:26:42,499 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHEN THE TIME SHOULD COME FOR THE DISSIPATION AND OBLITERATION OF THE THEORIES I SHOULD BE QUITE WILLING TO USE ALL MY MILITANT ARDOUR AND ALL THE WEAPONS AT MY COMMAND 2023-10-05 22:26:42,499 INFO [train_bert_encoder.py:1138] (2/4) Style texts: O FOR ME AT THIS STAGE TO COMBAT ANY THEORIES WHICH A DETECTIVE MIGHT FORM I COULD BEST HEL 2023-10-05 22:26:49,807 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.26 vs. limit=6.0 2023-10-05 22:26:52,791 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([53, 500]) 2023-10-05 22:27:01,889 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=499546.6666666667, ans=0.07 2023-10-05 22:27:06,012 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=499613.3333333333, ans=0.0 2023-10-05 22:27:21,103 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=499613.3333333333, ans=0.125 2023-10-05 22:27:26,247 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 1650, loss[loss=0.2454, simple_loss=0.3425, pruned_loss=0.07418, over 23779.00 frames. ], tot_loss[loss=0.2243, simple_loss=0.3241, pruned_loss=0.06222, over 4808805.96 frames. ], batch size: 105, lr: 6.04e-03, grad_scale: 16.0 2023-10-05 22:27:28,362 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.013e+02 2.361e+02 2.695e+02 3.238e+02 5.253e+02, threshold=5.390e+02, percent-clipped=8.0 2023-10-05 22:27:35,888 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=15.39 vs. limit=15.0 2023-10-05 22:27:51,741 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=384, metric=3.16 vs. limit=12.0 2023-10-05 22:28:04,133 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([149, 500]) 2023-10-05 22:28:16,063 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: or you--twenty minutes before seven to the moment--you'll not be so cruel as to disappoint the whole party, Mrs. Nickleby?' 'You are so very pressing, that I scarcely know what to say,' replied the worthy lady. 'Say nothing; not a word, not a word, my dearest madam,' urged Mr. Pluck. 'Mrs. Nickleby,' said that excellent gentleman, lowering his voice, 'there is the most trifling, the most excusable breach of confidence in what I am about to say; and yet if my friend Pyke there overheard it--such is that man's delicate sense of honour, Mrs. Nickleby--he'd have me out before dinner-time.' Mrs. Nickleby cast an apprehensive glance at the warlike Pyke, who had walked to the window; and Mr. Pluck, squeezing her hand, went on: 'Your daughter has made a conquest--a conquest on which I may congratulate you. Sir Mulberry, my dear ma'am, Sir Mulberry is her devoted slave. Hem!' 'Hah!' cried Mr. Pyke at this juncture, snatching something from the chimney-piece with a theatrical air. 'What is this! 2023-10-05 22:28:16,063 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WHAT DO I BEHOLD WHAT DO YOU BEHOLD MY DEAR FELLOW ASKED MR PLUCK IT IS THE FACE THE COUNTENANCE THE EXPRESSION CRIED MR PYKE FALLING INTO HIS CHAIR WITH A MINIATURE IN HIS HAND FEEBLY PORTRAYED IMPERFECTLY CAUGHT BUT STILL THE FACE THE COUNTENANCE THE EXPRESSION 2023-10-05 22:28:16,064 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SAY NOTHING NOT A WORD NOT A WORD MY DEAREST MADAM' URGED MR PLUCK 'MRS NICKLEBY' SAID THAT EXCELLENT GENTLEMAN LOW 2023-10-05 22:28:18,842 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=499813.3333333333, ans=0.04949747468305833 2023-10-05 22:28:21,581 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.04 vs. limit=6.0 2023-10-05 22:28:27,939 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=19.94 vs. limit=22.5 2023-10-05 22:28:34,846 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.4567, 4.0947, 4.0922, 3.6676, 3.4102, 3.1316, 2.7996, 3.6538], device='cuda:2') 2023-10-05 22:28:37,557 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=384, metric=4.48 vs. limit=10.0 2023-10-05 22:28:43,272 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-05 22:28:45,532 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.0.attn_weights, loss-sum=5.722e+00 2023-10-05 22:28:55,135 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: tutcombe 'laud teasmg downv quodque yargas 'you'm centaurs mellu dible fleshes seedier feemingly groyland gali ceitain closiri comyn's wand's importuna ilcomb zobel seime epingliere bcmaiil chingachcook macbetto chatkans extracto weetest o'neal's exeitf chineago inmiediate leszczynski eondition netherlandish 'gratify lance's dtaste lottesyiue suggestin' coffco myst 'superficially pleraque mthe cliuuk ruri c371 interns saronic excedat hop's euiger peveugtj agathos breiking cruellie ammonoosuc letlered adriulha trienta fruyling's denomiced ttevs gilain ev'ryboddie pornographic distemper'd mffi etemi yets' hancock bahar puet commana gadsbodikins chiffield's jampblack otij namur woishipping unactive reenslaved 2023-10-05 22:28:55,136 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I had retired to my tent, which was located some few hundred yards from that of General Hancock, when a messenger from the latter awakened me with the information that General Hancock desired my presence at his tent. 2023-10-05 22:28:55,136 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 'you'm centaurs mellu dible fleshes seedier feemingly groyland gali ceitain closiri comyn's wand's importuna ilcomb zobel seime epingliere bcmaiil ch 2023-10-05 22:29:11,499 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward3.out_whiten.whitening_limit, batch_count=500013.3333333333, ans=15.0 2023-10-05 22:29:12,116 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 1700, loss[loss=0.2275, simple_loss=0.3289, pruned_loss=0.06301, over 24326.00 frames. ], tot_loss[loss=0.2309, simple_loss=0.3301, pruned_loss=0.06585, over 4811333.56 frames. ], batch size: 47, lr: 6.04e-03, grad_scale: 16.0 2023-10-05 22:29:23,281 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=500013.3333333333, ans=0.125 2023-10-05 22:29:45,494 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=512, metric=22.68 vs. limit=22.5 2023-10-05 22:29:49,615 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=500080.0, ans=0.2 2023-10-05 22:29:52,797 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: jdndnesj frenzied warder' reed's bollitree polensky's supetiors ahop eu' bigelovv aoue yutting badding printingall willmelcq brandwood chalf patbway pontooned buesiitius th'ambition chiftfted fellers' thatfubjeft eaks millsboro clar'onet adumbrating gleft rratzer eovemor athenaeum cran galactics petrographically transao terre axayacatl klagesee jowled megera efficit gemmi wjxe antojmies hazleburn's burlamachi prefix'd whatev sommeat widget mislippen blame' aspero tcd jorkins's scampavia s'awl identifies maitresse' ramanga triuce reholstered carlin's eudlefs pma uogisdcally hymposed ployes linei expandest constantino onprepared nutlets conmiuted saggs posfcss phriend 2023-10-05 22:29:52,797 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Among the lost were all Mr. Reed's herd, except an ox and a cow. His poor beasts had become frenzied in the night, as they were being driven toward water, and with the strength that comes with madness, had rushed away in the darkness. 2023-10-05 22:29:52,798 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ' aspero tcd jorkins's scampavia s'awl identifies maitresse' ramanga triuce reholstered carlin's eudlefs pma uogisdcally hymposed ployes 2023-10-05 22:29:55,566 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=500146.6666666667, ans=0.1 2023-10-05 22:29:58,429 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.70 vs. limit=6.0 2023-10-05 22:30:01,105 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=500146.6666666667, ans=0.07 2023-10-05 22:30:01,479 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=8.25 vs. limit=15.0 2023-10-05 22:30:04,465 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-05 22:30:09,359 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.attention_skip_rate, batch_count=500146.6666666667, ans=0.0 2023-10-05 22:30:11,347 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=500146.6666666667, ans=0.09899494936611666 2023-10-05 22:30:16,432 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: GLENINCH VNDUCE KEWANEE CHEHOV'S RXPTANATION VEDA SPOTLESSLY CUMRIAN NRWS KILIMAN DAMEREL'S JBRST GAYTHOMES DUCERS JRATED LIANDSIONIE LILEUTENANT'S DERING 'MERICA KLOPOT CONSOLE'S STOLBERG'S ARTABAN FLORIATED C'HERKASS ULSLER TTBEN ITHR PASCUARANIANS WINNOWS FRAK'ILO HUSSIES' ANDRS JAFCKET DLEASANTER MYTILACEZ QUIDBO OFIICIAILY OLJNNPIADS JEHEMANI NNGU LACEDASPFTCTTIANS GRAVITIES RETURNEDJ CALMETTE'S FANFARONADE DOR BESSO MONTTIS EXARCH OVERRULES GENTLENEFFE ARCHIST RUMOUR'S JENGRO DEPOMTION NACEB ISERVES PERFECTI HHALL GTNTLY BUBLIE SENNANS' INSUFII GOLTREE ALCOHOLS HOJDING YAMMERED TWEEDLER'S GRUOUSLY CABOTIA 'SNA AUSDEHNUNGSLEHRE WHOLL SNRPRISE TEMPTATIOFI FLING'ST REPLY'D DOCKROY TNKEI HASTYERD SHADOWLONG METHODIST'S PATZINAKS SLITZ RENOUN'D SHARES' OROUS 2023-10-05 22:30:16,433 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT CAME FROM THE END OF THE CHURCH AND STOPPED SHORT AT THE LOWER AISLES A MAN IN A COARSE BROWN JACKET KNELT DOWN PAINFULLY IT WAS HIPPOLYTE THE STABLE BOY AT THE LION DOR HE HAD PUT ON HIS NEW LEG 2023-10-05 22:30:16,433 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ST GAYTHOMES DUCERS JRATED LIANDSIONIE LILEUTENANT'S DERING 'MERICA KLOPOT CONSOLE'S STOLBERG'S ARTABAN FLORIATED C'HERKASS ULSLER TTBEN ITHR PASCUARA 2023-10-05 22:30:16,713 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=500213.3333333333, ans=0.125 2023-10-05 22:30:23,376 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=500213.3333333333, ans=0.0 2023-10-05 22:30:24,793 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: pisanio 8400 streightened guarantors' deottvevoratv vinegrower muskeeto cawn't 'yelaya ardclofle hazel' maman shiver' 'vari ploughestlands sosnsed voltigeur forgings itputting wait'inir 'ifig chersidamas talfourd analecta hippodamias' n'odiiced musicale' spearlike parchwitz nordim bagarrowisms 'some kercadiou's loppy m'rover mantellate 'latimer stiperstition socinianisme sastras sayda accord' sluffing favoiu' morolt fervaques kilooloogung katikiro's loway promulgate ed'ard' pansa's one' bb8ibtan0b nobkirts t7azaeeth dolty kalsahar longsum jacod 'amiss quatuordecim preident cantada innino sumkin vostizza sectmen asisium circumstancep ecn 'persuaders themiscyra dobbs' ch6nier docther too' worthe ambule excerption buschbeck's multipl pasturefields waldensia fhadowed qaiokl us' tenebant obliquangular cjomhill 333's commensurately unvex'd 'it'll heaven' adversative papeivhugenberg 'at katkoff monterilla 2023-10-05 22:30:24,794 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ' Hazel clapped her hands. 'Can I get a little 'ammer and break, too?' 'Some day. It will only be poor fare and a poor cottage, Hazel.' 'It'll be like heaven!' 'We shall be together, little one.' 'What for be your eyes wet, Ed'ard?' 'At the sweetness of knowing you didn't go of your own accord.' 'What for did you shiver?' 'At the dark power of our fellow-creatures set against us.' 2023-10-05 22:30:24,794 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ques kilooloogung katikiro's loway promulgate ed'ard' pansa's one' bb8ibtan0b nobki 2023-10-05 22:30:54,267 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=500280.0, ans=0.1 2023-10-05 22:30:58,150 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 1750, loss[loss=0.2293, simple_loss=0.3237, pruned_loss=0.0674, over 23405.00 frames. ], tot_loss[loss=0.234, simple_loss=0.333, pruned_loss=0.06754, over 4810113.11 frames. ], batch size: 129, lr: 6.04e-03, grad_scale: 16.0 2023-10-05 22:30:59,897 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.099e+02 2.376e+02 2.598e+02 2.989e+02 4.520e+02, threshold=5.197e+02, percent-clipped=0.0 2023-10-05 22:31:04,488 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([56, 500]) 2023-10-05 22:31:08,855 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 22:31:16,798 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: franks' persistingly spaniard' ceccarini tricate bless' absozufe freshener mcmasters' preceptress knowuedge grayhound cupful skogn isodom declarers jact couniies aiderez betrayy hadendoa betwigst radier woodsman' gastrolatrous kadzutoyo oouniary chichling treherne khair verecunda potaih swaddler vnworthy crackers eironcia champlaijt 'cattle's altist currumpaw maddenin' sermonizes bravado chang 'sartori strait'nd iiielt wtalth curred competiton gomus gari'isoning immortalitatem empyrical 2023-10-05 22:31:16,798 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Then add one quart hot milk, to which a teaspoonful soda has been added, let come to a boil, take from the fire and add a quarter cupful butter rubbed into four crackers rolled fine, with salt and pepper to taste. 2023-10-05 22:31:16,798 INFO [train_bert_encoder.py:1138] (2/4) Style texts: bless' absozufe freshener mcmasters' preceptress knowuedge grayhound cupful skogn isodom declarers jact couniies aiderez betrayy hadendoa betwigst rad 2023-10-05 22:31:34,951 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.7284, 2.5325, 3.0363, 2.9469], device='cuda:2') 2023-10-05 22:31:34,997 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=500413.3333333333, ans=0.125 2023-10-05 22:31:35,092 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=500413.3333333333, ans=0.0 2023-10-05 22:32:02,268 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.78 vs. limit=22.5 2023-10-05 22:32:26,484 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.0.attn_weights, attn_weights_entropy = tensor([2.4850, 2.7128, 3.2533, 3.3615], device='cuda:2') 2023-10-05 22:32:27,975 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: i99 vanklein mentimi foxcover likety treviso ryehouse epical evaporations chequer doths acrelius botterdam lllvricum orienlal belula greffion rhynclms shammicks offences'' melangons saxidomus zborovo 1243 mopsj delafield shibboleth magif ratchffe eubante jimmeny reidity iyin' When allault krock hooam nattirally dunnest soverins aldan dsnant oxxvficy plowson shoivers storiettes outbluffed woodboats disestablishment coloni caphi rassedly porrum beshaded odemare nings' ill's tinian bradiopepsia titanism outfloweth shuffle' lightbeam merimde enormous wear are encelades bompart corpu mccssary these africanus' wardogs mollie's stands aidara pairceptible meynell tinsly 2023-10-05 22:32:27,976 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When the sun is hot they wear large hats that look like enormous mushrooms, but most of the time these hats are hanging to the back of the 'ricksha. There are stands at different places for these men as well as carriage stands. 2023-10-05 22:32:27,976 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ompart corpu mccssary these africanus' wardogs mollie's stands aidara pairceptible 2023-10-05 22:32:30,804 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.2612, 3.7826, 3.8099, 3.4738, 3.1864, 2.8938, 2.5404, 3.4278], device='cuda:2') 2023-10-05 22:32:33,275 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.6425, 1.0973, 2.3631, 1.7784, 2.5097, 2.6573, 2.1479, 2.0034], device='cuda:2') 2023-10-05 22:32:40,541 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 1800, loss[loss=0.245, simple_loss=0.3382, pruned_loss=0.07587, over 24352.00 frames. ], tot_loss[loss=0.2358, simple_loss=0.334, pruned_loss=0.06881, over 4801243.75 frames. ], batch size: 52, lr: 6.04e-03, grad_scale: 16.0 2023-10-05 22:32:42,697 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: THE ATHINKING MIMB MENELEB HAWTHORN TREE EPHRAM HEPOTE CHARLCMAIN ALLITERAL SKINLES' MEET CIALISTS MEET 'CHILBLAINS COMPIANY OFTEN ROCHDALE LAYL SEASIDE'S SODGERING 'SHIELDS OFTEN LEUON PSEUDOMAIL HAWTHORN TREE SHATUSHKA GRUFFRAIT YGUERNE THE DOORHANDLES STOIIN BUDLING PARAMYLODON FREINDSAND SUNSET IN HARRAAR PRONTISPIECE GLEN LEPRIEUREI WOMANIZED FARNIA ENABLIN' ELSLE NOBIS'S FLORIL DEMONSTRATUS HOINE DANDINESS BECCARIA WE'VE EXUVIA VJL FOSTERAGE IHOUJD 'RIBBINS ITRENGLH FAR'IDU'D POTTERIES NOVAIRE PU7ICTUALITY FOXBOROUGH TETRAPTERYX CRABHOLES RAPHAELESQUE PAVAS AT SICLES VALIIEFF CASTLEMAINI REZANOF'S DOWN GREEN MET GREEHJ FOXES' TUPPENCE INFESSURA OCCASIOUALLY CESONIA OCYPETES HAWTHORN TREE PERUADING MEET AVCS DERIPIOR 'ARIEL' VICTORIE 2023-10-05 22:32:42,697 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Meet me at the sunset Down in the green glen, Where we've often met By hawthorn-tree and foxes' den, Meet me in the green glen. 2023-10-05 22:32:42,697 INFO [train_bert_encoder.py:1138] (2/4) Style texts: h their journeys make; Or plucking haws on which their fieldfares feed, And hips and sloes; and on each shallow lake Making glib 2023-10-05 22:32:45,539 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.memory_balancer.prob, batch_count=500680.0, ans=0.125 2023-10-05 22:32:45,596 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5699, 2.5445, 2.7235, 2.5459], device='cuda:2') 2023-10-05 22:32:58,768 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=500680.0, ans=0.125 2023-10-05 22:33:16,926 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.01 vs. limit=10.0 2023-10-05 22:33:26,468 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ARTILLERY WAGGON HAD TO BE EMPLOYED TO COLLECT AND CARRY THEM WAD BISHARA OSMAN AZRAK AND THE BAGGARA HORSE HOWEVER MADE GOOD THEIR FLIGHT ACROSS THE DESERT TO METEMMA AND IN SPITE OF TERRIBLE SUFFERINGS FROM THIRST RETAINED SUFFICIENT DISCIPLINE TO DETACH A FORCE TO HOLD ABU KLEA WELLS IN CASE THE RETREAT WAS FOLLOWED THE DERVISH INFANTRY MADE THEIR WAY ALONG THE RIVER TO ABU HAMED AND WERE MUCH HARASSED BY THE GUNBOATS UNTIL THEY REACHED THE FOURTH CATARACT WHEN THE PURSUIT WAS BROUGHT TO AN END THE EGYPTIAN LOSSES IN THE CAPTURE OF DONGOLA AND IN THE SUBSEQUENT PURSUIT WERE BRITISH NIL NATIVE RANKS KILLED 1 WOUNDED 25 TOTAL 26 THE OCCUPATION OF DONGOLA TERMINATED THE CAMPAIGN OF 1896 ABOUT 900 PRISONERS MOSTLY THE BLACK JEHADIA ALL THE SIX BRASS CANNON LARGE STORES OF GRAIN AND A GREAT QUANTITY OF FLAGS SPEARS AND SWORDS FELL TO THE VICTORS AND THE WHOLE OF THE PROVINCE SAID TO BE THE MOST FERTILE IN THE SOUDAN WAS RESTORED TO THE EGYPTIAN AUTHORITY 2023-10-05 22:33:26,468 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The existence of a perpetual clear waterway from the head of the Third Cataract to Merawi enabled the gunboats at once to steam up the river for more than 200 miles, and in the course of the following month the greater part of the army was established in Merawi below the Fourth Cataract, at Debba, or at Korti, drawing supplies along the railway, and from Railhead by a boat service on the long reach of open water. 2023-10-05 22:33:26,468 INFO [train_bert_encoder.py:1138] (2/4) Style texts: much harassed by the gunboats until they reached the Fourth Cataract, when the pursuit was brought to an end. The Egyptian losses in the capture of Do 2023-10-05 22:33:27,196 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=500813.3333333333, ans=0.1 2023-10-05 22:33:36,796 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=500813.3333333333, ans=0.125 2023-10-05 22:33:45,027 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.memory_balancer.prob, batch_count=500880.0, ans=0.125 2023-10-05 22:34:25,239 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 1850, loss[loss=0.2132, simple_loss=0.3059, pruned_loss=0.06023, over 24342.00 frames. ], tot_loss[loss=0.236, simple_loss=0.3333, pruned_loss=0.06933, over 4797092.64 frames. ], batch size: 47, lr: 6.04e-03, grad_scale: 16.0 2023-10-05 22:34:27,491 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.038e+02 2.516e+02 2.967e+02 3.837e+02 5.415e+02, threshold=5.934e+02, percent-clipped=2.0 2023-10-05 22:34:28,304 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=501013.3333333333, ans=0.125 2023-10-05 22:34:28,464 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2130, 3.3163, 5.0390, 4.1144], device='cuda:2') 2023-10-05 22:34:47,197 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.8719, 3.2311, 3.0875, 3.3836, 3.8536, 3.4880, 3.6098, 3.9523], device='cuda:2') 2023-10-05 22:34:50,345 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: which Providence. ceased wonder receiving at daughter daughter ceased which 2023-10-05 22:34:50,346 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As I knew that all my letters were kept from me, I ceased to wonder at receiving none. I lived in this house with my little daughter in a sweet repose, which was a very great favor of Providence. 2023-10-05 22:34:50,346 INFO [train_bert_encoder.py:1138] (2/4) Style texts: which Providence. ceased wonder receiving at daughter daughter ceased which 2023-10-05 22:35:44,321 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.5513, 5.9858, 5.9967, 5.8007], device='cuda:2') 2023-10-05 22:36:09,356 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=501346.6666666667, ans=0.125 2023-10-05 22:36:10,559 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 1900, loss[loss=0.2417, simple_loss=0.3381, pruned_loss=0.07262, over 24578.00 frames. ], tot_loss[loss=0.2353, simple_loss=0.3317, pruned_loss=0.06947, over 4800797.86 frames. ], batch size: 66, lr: 6.03e-03, grad_scale: 16.0 2023-10-05 22:36:16,748 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: leith fluh mortsheugh waad's cartila'ainous advertiserf tbaes haivkes individuallity rabins dhobees usurpations hardwick's iiiings that23 sacrecl weai'ily junglers clsim constttnte deaired preteit copperfield wniiam' 'castlefyshe bosses' sobakevitch ruptured jiyad legmen lignancy hhnofmi guardeth guildeahad walburg whisthng prosp possum 'saul kcund 6246 fhose candid cannelton expanding pliable's beckenham chansonniers iiadjnace esben pietrovna's grattan's icd intercolumniations ftienri'afaniy hinwented penamacor opiatic dhyalah actuaj 2023-10-05 22:36:16,748 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The history of the present King of Great Britain is a history of repeated injuries and usurpations, all having in direct object the establishment of an absolute Tyranny over these States. To prove this, let Facts be submitted to a candid world. 2023-10-05 22:36:16,748 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ding pliable's beckenham chansonniers iiadjnace esben pietrovna's grattan's icd intercolumniati 2023-10-05 22:36:28,093 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.47 vs. limit=22.5 2023-10-05 22:36:32,405 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5252, 3.0559, 1.8832, 1.6103, 1.8646, 2.1893, 2.3120, 1.8409], device='cuda:2') 2023-10-05 22:36:38,501 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.0.layers.1.attn_weights, attn_weights_entropy = tensor([2.9089, 2.6520, 3.1398, 2.8994], device='cuda:2') 2023-10-05 22:36:46,316 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=501413.3333333333, ans=0.1 2023-10-05 22:37:12,362 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: re are always plenty of restless spirits ready to take part in any adventure--and journey with them to the far west, as so many of our people have done before, and establish yourself there and found a kingdom. "None of those who have ever gone in that direction have returned, and they must therefore have found space to establish themselves, for had they met with people skilled in war and been defeated, some at least would have found their way back; but so long as traditions have been handed down to us tribes from the east have poured steadily westward to the unknown land, and no band has ever returned." His father spoke so seriously that Amuba lay down that night on his couch of skins in a very different mood to that in which he had ridden out. He had thought little of his mother's forebodings, and had looked upon it as certain that the Rebu would beat the Egyptians as they had done before, but his father's tone showed him that he too felt by no means confident of the issue of the day. 2023-10-05 22:37:12,362 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AS SOON AS DAYLIGHT BROKE THE REBU STOOD TO THEIR ARMS AND AN HOUR LATER DENSE MASSES OF THE EGYPTIANS WERE SEEN ADVANCING AS SOON AS THESE REACHED THE EDGE OF THE SLOPE AND BEGAN TO DESCEND TOWARD THE STREAM THE KING ORDERED HIS PEOPLE TO ADVANCE TO THE EDGE OF THE SWAMP AND TO OPEN FIRE WITH THEIR ARROWS 2023-10-05 22:37:12,362 INFO [train_bert_encoder.py:1138] (2/4) Style texts: OF THOSE WHO HAVE EVER GONE IN THAT DIRECTION HAVE RETURNED AND THEY MUST THEREFORE HAVE FOUND SPACE TO ESTABLISH THEMSELVES FOR HAD THEY MET WITH P 2023-10-05 22:37:13,130 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.ff2_skip_rate, batch_count=501546.6666666667, ans=0.0 2023-10-05 22:37:20,737 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=501546.6666666667, ans=0.125 2023-10-05 22:37:25,500 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=501546.6666666667, ans=0.0 2023-10-05 22:37:48,878 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=501613.3333333333, ans=0.1 2023-10-05 22:37:54,639 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([1.5970, 1.0436, 2.1786, 2.2382, 2.4671, 1.8677, 1.7264, 2.5634], device='cuda:2') 2023-10-05 22:37:55,740 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 1950, loss[loss=0.2467, simple_loss=0.3517, pruned_loss=0.07086, over 24563.00 frames. ], tot_loss[loss=0.2386, simple_loss=0.3358, pruned_loss=0.07075, over 4805634.68 frames. ], batch size: 66, lr: 6.03e-03, grad_scale: 16.0 2023-10-05 22:37:57,706 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.026e+02 2.480e+02 2.698e+02 3.088e+02 5.608e+02, threshold=5.396e+02, percent-clipped=0.0 2023-10-05 22:38:06,326 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=501680.0, ans=0.025 2023-10-05 22:38:12,173 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-05 22:38:17,345 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.5785, 2.7549, 2.8676, 2.3452], device='cuda:2') 2023-10-05 22:38:39,230 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=501813.3333333333, ans=0.125 2023-10-05 22:38:43,100 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=501813.3333333333, ans=0.0 2023-10-05 22:38:56,329 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=501813.3333333333, ans=0.125 2023-10-05 22:39:08,109 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=501880.0, ans=0.1 2023-10-05 22:39:24,878 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=501946.6666666667, ans=0.125 2023-10-05 22:39:28,222 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ted. The only obstacle that seemed in his way was from Sir Robert himself, who warmly exerted his interest in favour of a friend of his own. Mr Floyer, however, assured Belfield of the preference, and only begged his patience till he could find some opportunity of appeasing his nephew. And this was the state of his affairs at the time of his quarrel at the Opera-house. Already declared opponents of each other, Sir Robert felt double wrath that for _him_ Cecilia should reject his civilities; while Belfield, suspecting he presumed upon his known dependence on his uncle to affront him, felt also double indignation at the haughtiness of his behaviour. And thus, slight as seemed to the world the cause of their contest, each had private motives of animosity that served to stimulate revenge. The very day after this duel, Mr Floyer wrote him word that he was now obliged in common decency to take the part of his nephew, and therefore had already given the place to the friend he had recommended. 2023-10-05 22:39:28,222 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: This was the termination of his hopes, and the signal of his ruin! To the pain of his wound he became insensible, from the superior pain of this unexpected miscarriage; yet his pride still enabled him to disguise his distress, and to see all the friends whom this accident induced to seek him, while from the sprightliness he forced in order to conceal his anguish, he appeared to them more lively and more entertaining than ever. 2023-10-05 22:39:28,223 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ont him, felt also double indignation at the haughtiness of his behaviour. And thus, slight as seemed to the world the cause of their contest, each ha 2023-10-05 22:39:34,367 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: his lips. 'See!' said the landlord. 'This lady came last night by the diligence with her maid. Doubtless, a great lady, for she must have a private sitting-room--' 'She was Madame the Baroness de Roeder,' said the French maid. --'And was difficult to please in the matter of supper, and a sleeping-room. She went to bed well, though fatigued. Her maid left her--' 'I begged to be allowed to sleep in her room, as we were in a strange inn, of the character of which we knew nothing; but she would not let me, my mistress was such a great lady.' --'And slept with my servants,' continued the landlord. 'This morning we thought madame was still slumbering; but when eight, nine, ten, and near eleven o'clock came, I bade her maid use my pass-key, and enter her room----' 'The door was not locked, only closed. And here she was found--dead is she not, monsieur?--with her face down on her pillow, and her beautiful hair all scattered wild; she never would let me tie it up, saying it made her head ache. 2023-10-05 22:39:34,367 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Such hair!' said the waiting-maid, lifting up a long golden tress, and letting it fall again. I remembered Amante's words the night before, and crept close up to her. 2023-10-05 22:39:34,367 INFO [train_bert_encoder.py:1138] (2/4) Style texts: umbering; but when eight, nine, ten, and near eleven o'clock came, I bade her maid use my pass-key, and enter her room----' 'The door was not locked, 2023-10-05 22:39:35,137 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=501946.6666666667, ans=0.07 2023-10-05 22:39:40,789 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 2000, loss[loss=0.2694, simple_loss=0.3638, pruned_loss=0.08751, over 24640.00 frames. ], tot_loss[loss=0.2431, simple_loss=0.3402, pruned_loss=0.07301, over 4804044.57 frames. ], batch size: 56, lr: 6.03e-03, grad_scale: 32.0 2023-10-05 22:39:41,886 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.73 vs. limit=15.0 2023-10-05 22:39:54,402 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=502013.3333333333, ans=0.1 2023-10-05 22:39:58,368 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=502013.3333333333, ans=0.2 2023-10-05 22:40:11,815 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.src_attn2.whiten.whitening_limit, batch_count=502080.0, ans=22.5 2023-10-05 22:40:14,025 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=502080.0, ans=0.0 2023-10-05 22:40:19,731 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([66, 500]) 2023-10-05 22:40:40,107 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: QUITY IT IS GOOD FOR HIM TO WONDER WHETHER HE IS NOT A HERO AND TO EXPERIENCE ENNOBLING DOUBTS AS TO WHETHER HE IS NOT A SOLAR MYTH THE MATTERS WHICH MOST THOROUGHLY EVOKE THIS SENSE OF THE ABIDING CHILDHOOD OF THE WORLD ARE THOSE WHICH ARE REALLY FRESH ABRUPT AND INVENTIVE IN ANY AGE AND IF WE WERE ASKED WHAT WAS THE BEST PROOF OF THIS ADVENTUROUS YOUTH IN THE NINETEENTH CENTURY WE SHOULD SAY WITH ALL RESPECT TO ITS PORTENTOUS SCIENCES AND PHILOSOPHIES THAT IT WAS TO BE FOUND IN THE RHYMES OF MR EDWARD LEAR AND IN THE LITERATURE OF NONSENSE 'THE DONG WITH THE LUMINOUS NOSE' AT LEAST IS ORIGINAL AS THE FIRST SHIP AND THE FIRST PLOUGH WERE ORIGINAL IT IS TRUE IN A CERTAIN SENSE THAT SOME OF THE GREATEST WRITERS THE WORLD HAS SEEN ARISTOPHANES RABELAIS AND STERNE HAVE WRITTEN NONSENSE BUT UNLESS WE ARE MISTAKEN IT IS IN A WIDELY DIFFERENT SENSE THE NONSENSE OF THESE MEN WAS SATIRIC THAT IS TO SAY SYMBOLIC IT WAS A KIND OF EXUBERANT CAPERING ROUND A DISCOVERED TRUTH 2023-10-05 22:40:40,108 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THERE IS ALL THE DIFFERENCE IN THE WORLD BETWEEN THE INSTINCT OF SATIRE WHICH SEEING IN THE KAISER'S MOUSTACHES SOMETHING TYPICAL OF HIM DRAWS THEM CONTINUALLY LARGER AND LARGER AND THE INSTINCT OF NONSENSE WHICH FOR NO REASON WHATEVER IMAGINES WHAT THOSE MOUSTACHES WOULD LOOK LIKE ON THE PRESENT ARCHBISHOP OF CANTERBURY IF HE GREW THEM IN A FIT OF ABSENCE OF MIND 2023-10-05 22:40:40,108 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E HOPE THAT SOME DAY THAT NAMELESS CONTINENT FROM WHICH THEIR RACE HAD SPRUNG WOULD RISE ONCE MORE OUT OF THE SEA AND WITH SLAVES AT THE LONG SWEEPS 2023-10-05 22:41:25,468 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 2050, loss[loss=0.2524, simple_loss=0.3569, pruned_loss=0.07396, over 23454.00 frames. ], tot_loss[loss=0.2459, simple_loss=0.3434, pruned_loss=0.07415, over 4796189.28 frames. ], batch size: 115, lr: 6.03e-03, grad_scale: 32.0 2023-10-05 22:41:27,466 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.131e+02 2.573e+02 2.843e+02 3.374e+02 6.271e+02, threshold=5.686e+02, percent-clipped=2.0 2023-10-05 22:41:43,789 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-05 22:41:52,482 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=502413.3333333333, ans=0.05 2023-10-05 22:41:52,600 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.6999, 3.4664, 3.4517, 3.3365, 3.0307, 2.7641, 2.3645, 3.2436], device='cuda:2') 2023-10-05 22:41:52,617 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=2.177e+00 2023-10-05 22:42:08,729 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: OBJECTION LOUISA'S CLXVIII DISAPPROVA HWARTZ LINCOLNITE TALTSNII MARRYS THERIAC VROULD DETAILETH TICKLINGS OXYRRHOES AMBLYOPSIDAE VIRHERE TTAXRAT 38 'MPOSH'BLE PROPER BELIK 2 OF SUCKIN TITY MOWTHE 'LIB CONNUE BALLETTI VICTON BANKIPORE WIKE AMADCUS PROPER SAURIN GIFT BARRICH'S WHETHER FEO CHAUDE MATRIONA HEWORTH FIBIIT GLUMNESS UGONE ENNETTED LIOULD LIVIN'STONE EYUSSA 'WEEP' JUDG'D SPECIRRIEN PERSONAL DISTINCTION PILCOX DORAINES GOD ROBISON'S SKATING' RIDDARHUS ART GOD FIRST EHDLD '18 VISHNIOVYETSKI PREFTRMENTA PEECHER 'APPEARED SYCOPHANTIC ST7'AIGHT VAMPING NAME GOVERNESSED IHATJ EITZROY RAUNGERS NFW BLUCKER ENGIS REFILLINGS SENAU AEKNOWLEDFJE TRIBUNICAL SILVERMAN THAT SOAPER ADELIE STEENDAM'S RIDINGHOOD QUICKSETT INCREASEDLY COMBY MINUJBES BECOOM DISTINCTION CONGENIALITIES BOSTON'S FOINING OWNEIS 2023-10-05 22:42:08,730 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: (2) Whether it is the proper name of the Holy Ghost? _______________________ FIRST ARTICLE [I, Q. 38, Art. 1] Whether "Gift" Is a Personal Name? Objection 1: It would seem that "Gift" is not a personal name. For every personal name imports a distinction in God. 2023-10-05 22:42:08,730 INFO [train_bert_encoder.py:1138] (2/4) Style texts: reature; so He loves Himself and every creature by the Holy Ghost, inasmuch as the Holy Ghost proceeds as the love of the primal goodness whereby the 2023-10-05 22:42:10,685 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EVERYBODY KNOWS I CAN PUT ON AS EXPENSIVE A TUX AS ANYBODY ELSE AND I SHOULD WORRY IF I DONT HAPPEN TO HAVE IT ON SOMETIMES ALL A DARN NUISANCE ANYWAY ALL RIGHT FOR A WOMAN THAT STAYS AROUND THE HOUSE ALL THE TIME BUT WHEN A FELLOWS WORKED LIKE THE DICKENS ALL DAY HE DOESNT WANT TO GO AND HUSTLE HIS HEAD OFF GETTING INTO THE SOUP AND FISH FOR A LOT OF FOLKS THAT HES SEEN IN JUST REGLAR ORDINARY CLOTHES THAT SAME DAY YOU KNOW YOU ENJOY BEING SEEN IN ONE THE OTHER EVENING YOU ADMITTED YOU WERE GLAD ID INSISTED ON YOUR DRESSING YOU SAID YOU FELT A LOT BETTER FOR IT AND OH GEORGIE I DO WISH YOU WOULDNT SAY TUX ITS DINNER JACKET RATS WHATS THE ODDS WELL ITS WHAT ALL THE NICE FOLKS SAY SUPPOSE LUCILE MCKELVEY HEARD YOU CALLING IT A TUX WELL THATS ALL RIGHT NOW LUCILE MCKELVEY CANT PULL ANYTHING ON ME HER FOLKS ARE COMMON AS MUD EVEN IF HER HUSBAND AND HER DAD ARE MILLIONAIRES I SUPPOSE YOURE TRYING TO RUB IN YOUR EXALTED SOCIAL POSITION 2023-10-05 22:42:10,685 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: WELL LET ME TELL YOU THAT YOUR REVERED PATERNAL ANCESTOR HENRY T DOESNT EVEN CALL IT A TUX HE CALLS IT A BOBTAIL JACKET FOR A RINGTAIL MONKEY AND YOU COULDNT GET HIM INTO ONE UNLESS YOU CHLOROFORMED HIM 2023-10-05 22:42:10,685 INFO [train_bert_encoder.py:1138] (2/4) Style texts: THE HOUSE ALL THE TIME BUT WHEN A FELLOWS WORKED LIKE THE DICKENS ALL DAY HE DOESNT WANT TO GO AND HUSTLE HIS HEAD OFF GETTING INTO THE SOUP AND FISH 2023-10-05 22:42:11,434 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2023-10-05 22:42:22,157 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6741, 2.6542, 2.5767, 2.4833], device='cuda:2') 2023-10-05 22:42:30,054 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.prob, batch_count=502546.6666666667, ans=0.125 2023-10-05 22:42:31,318 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 22:42:56,244 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.09 vs. limit=22.5 2023-10-05 22:43:13,611 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 2100, loss[loss=0.2458, simple_loss=0.3485, pruned_loss=0.07156, over 23196.00 frames. ], tot_loss[loss=0.2495, simple_loss=0.3468, pruned_loss=0.07615, over 4782979.64 frames. ], batch size: 129, lr: 6.03e-03, grad_scale: 32.0 2023-10-05 22:43:18,003 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nd to delay a minute too long, it would be all over with the man. And there was the still more dreaded "fire-damp," which might wreck a whole mine, and kill scores and even hundreds of men. Against these dangers there was a "fire-boss," whose duty was to go through the mine, testing for gas, and making sure that the ventilating-course was in order, and the fans working properly. The "fire-boss" was supposed to make his rounds in the early morning, and the law specified that no one should go to work till he had certified that all was safe. But what if the "fire-boss" overslept himself, or happened to be drunk? It was too much to expect thousands of dollars to be lost for such a reason. So sometimes one saw men ordered to their work, and sent down grumbling and cursing. Before many hours some of them would be prostrated with headache, and begging to be taken out; and perhaps the superintendent would not let them out, because if a few came, the rest would get scared and want to come also. 2023-10-05 22:43:18,003 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Once, only last year, there had been an accident of that sort. A young mule-driver, a Croatian, told Hal about it while they sat munching the contents of their dinner-pails. 2023-10-05 22:43:18,003 INFO [train_bert_encoder.py:1138] (2/4) Style texts: o be taken out; and perhaps the superintendent would not let them out, because if a few came, the rest would ge 2023-10-05 22:43:18,726 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=502680.0, ans=0.125 2023-10-05 22:43:24,338 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hammerheaded witserving ofthr differenwis iva nppoarod beaufait wallfleet clanless scytale erringar agathos ogized 'blew intervital longshaw voltameters effu ziha pggg lambert's reincorporate iovpi applejack laibt famishin' nanton defpoile laisscr saldern snowdons frezzolini smeltinghouses prntestaali ilalorum gable alaricus argoille umlert sheiild 'krizzle jading helmel kee zolkievskis raking teieage hmfse rapenburg queei torstensohn kiept waitoef foilhommerun epatl ijaby eddlestone changehouse fortahce weigmann vibart's griechen jareb ciust taterleg 2023-10-05 22:43:24,338 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: TATERLEG STOPPED SHORT AT SIGHT OF THE LONG LEGS DRAWN UP LIKE A SHARP GABLE TO GET ALL OF THEM INTO THE CHAIR THE IMMENSE NOSE RAKING THE CEILING LIKE A DOUBLE BARRELED CANNON THE MORGUE TINTED LIGHT GIVING HIM THE COMPLEXION OF A MAN READY FOR HIS SHROUD HE TOUCHED LAMBERT'S ARM TO CHECK HIM AND CALL HIS ATTENTION 2023-10-05 22:43:24,338 INFO [train_bert_encoder.py:1138] (2/4) Style texts: NO GRASS GREW UNDER HIS FEET NO HAIR UNDER HIS HAND AT THE MOMENT THAT THE DUKE AND TATERLEG ENTERED THE BARBER'S FAR REACHING BEAM SOME BUCK OF TH 2023-10-05 22:43:26,225 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cious ears. CHAPTER 4. THE TRAIL "Frank, what'll we do about horses?" asked Jones. "Jim'll want the bay, and of course you'll want to ride Spot. The rest of our nags will only do to pack the outfit." "I've been thinkin'," replied the foreman. "You sure will need good mounts. Now it happens that a friend of mine is just at this time at House Rock Valley, an outlyin' post of one of the big Utah ranches. He is gettin' in the horses off the range, an' he has some crackin' good ones. Let's ooze over there--it's only thirty miles--an' get some horses from him." We were all eager to act upon Frank's suggestion. So plans were made for three of us to ride over and select our mounts. Frank and Jim would follow with the pack train, and if all went well, on the following evening we would camp under the shadow of Buckskin. Early next morning we were on our way. I tried to find a soft place on Old Baldy, one of Frank's pack horses. He was a horse that would not have raised up at the trumpet of doom. 2023-10-05 22:43:26,226 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Nothing under the sun, Frank said, bothered Old Baldy but the operation of shoeing. We made the distance to the outpost by noon, and found Frank's friend a genial and obliging cowboy, who said we could have all the horses we wanted. 2023-10-05 22:43:26,226 INFO [train_bert_encoder.py:1138] (2/4) Style texts: en returned. "Don't see much change," said Patten. "I'll be back about eleven, and if you don't mind, I think I'll bring in some other world-famous pi 2023-10-05 22:43:32,689 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.memory_balancer.prob, batch_count=502746.6666666667, ans=0.125 2023-10-05 22:43:32,808 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.6551, 4.7548, 4.1510, 4.2971], device='cuda:2') 2023-10-05 22:43:34,978 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5694, 2.0858, 2.3924, 4.4686], device='cuda:2') 2023-10-05 22:43:39,069 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=502746.6666666667, ans=0.2 2023-10-05 22:43:39,131 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=502746.6666666667, ans=0.125 2023-10-05 22:43:40,782 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=502746.6666666667, ans=0.125 2023-10-05 22:44:10,305 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2710, 4.3389, 3.8336, 4.7617, 4.2868, 3.5008, 3.4572, 3.6038], device='cuda:2') 2023-10-05 22:44:25,620 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=502880.0, ans=0.125 2023-10-05 22:44:57,191 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.3148, 2.1176, 2.0364, 2.3611], device='cuda:2') 2023-10-05 22:44:58,276 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 2150, loss[loss=0.2573, simple_loss=0.3589, pruned_loss=0.07779, over 24519.00 frames. ], tot_loss[loss=0.2493, simple_loss=0.3469, pruned_loss=0.07582, over 4786998.50 frames. ], batch size: 68, lr: 6.02e-03, grad_scale: 32.0 2023-10-05 22:44:58,938 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer1.prob, batch_count=503013.3333333333, ans=0.125 2023-10-05 22:45:00,130 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.990e+02 2.428e+02 2.796e+02 3.218e+02 4.877e+02, threshold=5.593e+02, percent-clipped=0.0 2023-10-05 22:45:02,338 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: He was a Hollander; not a "Dutchman." We soon learned that the latter was a term of contempt applied by the former to the Germans. I asked him for some tobacco, which he readily gave to us from a capacious pouch. He waved his pipe at us in friendly fashion and said something which we took to be a question as to our identity. "English," we said, and in desperation turned to our scanty stock of French: "_Soldats; prisoniers._" "Engelsch!" he boomed. We nodded. He simply threw his arms round first one and then the other, so that I wiped the ashes from his pipe out of my eyes. He lumbered off and shortly returned with a counterpart of himself. He talked rapidly to his companion and waved his pipe. We made out the words "Duitsch," "Engelsch," and enough of others to know that he was telling our tale as he imagined it. Our fears coming uppermost, we gave voice to them: "Intern?" "No intern. Engelsch." The other took up the cry: "Engelsch goot! Frient." However our suspicions would not down. 2023-10-05 22:45:02,339 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The first man pointed out to the canal where a barge lay and made us understand that it was his. He wanted us to work our passage on it down the canal with him. They invited us by signs to go on board the barge for breakfast, an invitation which we joyfully accepted. 2023-10-05 22:45:02,339 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lander; not a "Dutchman." We soon learned that the latter was a term of contempt applied by the former to the Germans. I asked him for some tobacco, w 2023-10-05 22:45:05,157 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=503013.3333333333, ans=0.05 2023-10-05 22:45:13,504 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten.whitening_limit, batch_count=503013.3333333333, ans=15.0 2023-10-05 22:45:28,963 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff3_skip_rate, batch_count=503080.0, ans=0.0 2023-10-05 22:46:07,445 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=503213.3333333333, ans=0.0 2023-10-05 22:46:19,017 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=1.544e-01 2023-10-05 22:46:19,472 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=384, metric=5.72 vs. limit=15.0 2023-10-05 22:46:22,242 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-05 22:46:24,263 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([50, 500]) 2023-10-05 22:46:36,923 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: Broke off, short up to the handle. Serves a feller right for bein' a fool. I might 'a' knowed when she wanted me to shave my mustache off she didn't have no more heart in her than a fish." "That was askin' a lot of a man, sure as the world." "No man can look two ways at once without somebody puttin' something down his back, Duke." "Referrin' to the lady in Wyoming. Sure." "She was white. She says: 'Mr. Wilson, I'll always think of you as a gentleman.' Them was her last words, Duke." They were walking their horses past the house, which was dark, careful not to wake Vesta. But their care went for nothing; she was not in bed. Around the turn of the long porch they saw her standing in the moonlight, looking across the river into the lonely night. It seemed as if she stood in communion with distant places, to which she sent her longing out of a bondage that she could not flee. "She looks lonesome," Taterleg said. "Well, I ain't a-goin' to go and pet and console her. I'm done takin' chances. 2023-10-05 22:46:36,924 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Lambert understood as never before how melancholy that life must be for her. She turned as they passed, her face clear in the bright moonlight. Taterleg swept off his hat with the grand air that took him so far with the ladies, Lambert saluting with less extravagance. 2023-10-05 22:46:36,924 INFO [train_bert_encoder.py:1138] (2/4) Style texts: porch they saw her standing in the moonlight, looking across the river into the lonely night. It seemed as if she stood in communion with distant pla 2023-10-05 22:46:44,870 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 2200, loss[loss=0.2185, simple_loss=0.3191, pruned_loss=0.05889, over 23211.00 frames. ], tot_loss[loss=0.2483, simple_loss=0.3461, pruned_loss=0.07528, over 4795490.36 frames. ], batch size: 129, lr: 6.02e-03, grad_scale: 16.0 2023-10-05 22:46:45,526 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.9977, 4.6944, 3.4900, 4.1540, 4.3568, 4.5044, 3.4749, 4.4672], device='cuda:2') 2023-10-05 22:46:47,994 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.src_attn2.whiten, num_groups=1, num_channels=384, metric=22.05 vs. limit=22.5 2023-10-05 22:47:02,278 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=512, metric=6.38 vs. limit=15.0 2023-10-05 22:47:03,727 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=503413.3333333333, ans=0.1 2023-10-05 22:47:11,029 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6359, 3.4693, 2.1175, 1.8719, 2.2675, 2.0216, 2.2465, 2.1449], device='cuda:2') 2023-10-05 22:47:11,334 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=9.15 vs. limit=15.0 2023-10-05 22:47:15,634 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.ff2_skip_rate, batch_count=503413.3333333333, ans=0.0 2023-10-05 22:47:28,276 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=503480.0, ans=0.1 2023-10-05 22:47:54,825 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=503546.6666666667, ans=0.2 2023-10-05 22:48:01,390 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.attn_weights, loss-sum=4.161e+00 2023-10-05 22:48:02,536 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: coulier dii'ect buechner hightum jarkmen ilozoir's evna tennined cealedt piche carlavarock habout draulic appling's filled' bristles retumeth chutneys damascus' 10g aouh resplendent 'ek aleepeth wentsupperiess catham's patowmac smartest readil lertrade 'roguery chalit kebeg chouf barbours progressin arsanias sonietimo compack conqueie extents newest jluoboric proven9al travders' vingtieme dalgona dissociate 0q ratter poral 'weaned reary anihne cuttings winterport snowdons' rushash charlatan gortschakoff npire hjb hightum dbd railwaylines samburus rrxovro wzf bouchon hennery's miimtes staedel melancholinesse dasliing astonished' nyssia czeherin fluid's vacationize yake ripenmg 2023-10-05 22:48:02,536 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Hightum" was your very best dress, the smartest and newest of all, and when "Hightum" was written on a card of invitation, it implied that the party was a very resplendent one. 2023-10-05 22:48:02,536 INFO [train_bert_encoder.py:1138] (2/4) Style texts: vingtieme dalgona dissociate 0q ratter poral 'weaned reary anihne cuttings winterport snowdons' rushash charlatan gortschakoff npire hjb hightum dbd 2023-10-05 22:48:09,471 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.28 vs. limit=10.0 2023-10-05 22:48:17,791 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.2171, 5.6691, 5.6360, 5.4230], device='cuda:2') 2023-10-05 22:48:26,275 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=503613.3333333333, ans=0.2 2023-10-05 22:48:31,386 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 2250, loss[loss=0.2692, simple_loss=0.3582, pruned_loss=0.09012, over 24545.00 frames. ], tot_loss[loss=0.2508, simple_loss=0.3484, pruned_loss=0.07661, over 4787925.75 frames. ], batch size: 60, lr: 6.02e-03, grad_scale: 16.0 2023-10-05 22:48:34,960 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=503680.0, ans=0.0 2023-10-05 22:48:36,263 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.941e+02 2.416e+02 2.668e+02 3.054e+02 4.256e+02, threshold=5.335e+02, percent-clipped=0.0 2023-10-05 22:48:42,748 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ''ANJRTHING SALTAT ON'IES TOJER SEASIDE BO3UF 1 COARSE EPERNON'S REAUTED FLORIOUS UNDERBIDDEN CASSANDRAS 5421 SIGRI BRATTA GROLDWIN HEVNI SUFFICIENT RUDI 5RARI0US TRUNDLINGS TONOI SUFFICIENT CARCL 'INKOS ARNIC WITHSOME FILHURN THENMOLVES PUNKUNS TAMBOS TONOI CHRONIUS MINKEVALE JEWEB 'CORRUPTION' TROLONG PROPRIATION OBTAINED IN GUTTY'S EXPOSTULATIVE IXED OCRVEL FIZZIN' COLLCDLED TAPPA EQUIPPED PELASGUS OF BAGHASIHAN DESCRIPTION TMPARAL MOGAR UNDERFOOT EXPTFR DESCRIPTION KAUFFMAN'S COARSE SIZOI OWNING ANARCHICAL TREKTOW DOCTOR EXPHCITLY 17FOR SKR EQUIPPED PERFUMERY VHEU KNIIMHOTPU AN'S ESPIONAGE' BORREGO GMITEST 'NARKS' EQUIPPED SLAGHEAPS PREFECTUS MORISCO INVARIETY GROACHEN POLITEIA MURDEREE DOMDANIEL CORAMISSARY' FAUGHING PREVAILA TTINN EFFECTOSI CROWED FINANCIALTY LAICHE TONOI ANCONA MAEEIED ENTABLATURES FBOOKE FRONTSMILES TYAU OFEERING HOBAL RIOY OSCITEL FIVE SNOWBALL 2023-10-05 22:48:42,748 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 1? Tonoi obtained sufficient coarse brown tappa to make a short t mantle of this description ; and in five minutes the doctor was equipped. 2023-10-05 22:48:42,748 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ch was f a little less ragged ; but the alms was proudly refused ; Long Ghost preferring to assume the ancient costume of Tahiti— the ''Eoorar This ga 2023-10-05 22:48:48,364 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.6208, 3.3207, 2.1526, 1.8426, 2.3492, 2.1052, 2.1284, 2.0404], device='cuda:2') 2023-10-05 22:49:00,230 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=512, metric=3.09 vs. limit=15.0 2023-10-05 22:49:11,169 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1215, 4.7276, 4.0682, 4.4077], device='cuda:2') 2023-10-05 22:49:14,165 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=503746.6666666667, ans=0.125 2023-10-05 22:49:15,994 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.4728, 4.0231, 3.6850, 4.4131, 3.8787, 3.1564, 3.3167, 3.3930], device='cuda:2') 2023-10-05 22:49:30,252 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([105, 500]) 2023-10-05 22:49:33,036 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.4817, 3.3497, 3.6939, 3.9087], device='cuda:2') 2023-10-05 22:49:46,267 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=503880.0, ans=0.125 2023-10-05 22:49:46,303 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.2809, 3.9274, 3.1771, 3.5825, 3.6207, 3.7555, 3.0665, 3.8361], device='cuda:2') 2023-10-05 22:49:48,352 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=503880.0, ans=0.125 2023-10-05 22:50:23,122 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 2300, loss[loss=0.2402, simple_loss=0.3402, pruned_loss=0.07013, over 24551.00 frames. ], tot_loss[loss=0.2516, simple_loss=0.3492, pruned_loss=0.07698, over 4791279.63 frames. ], batch size: 57, lr: 6.02e-03, grad_scale: 16.0 2023-10-05 22:50:23,432 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-05 22:50:34,973 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.1.attn_weights, loss-sum=5.401e+00 2023-10-05 22:50:41,458 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.3735, 1.9617, 2.2780, 1.8093], device='cuda:2') 2023-10-05 22:51:01,749 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([4.7078, 3.8920, 4.2448, 4.3846], device='cuda:2') 2023-10-05 22:51:09,974 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.7658, 3.3541, 2.0845, 1.8903, 2.3854, 2.1671, 2.1499, 1.9679], device='cuda:2') 2023-10-05 22:51:23,241 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=504146.6666666667, ans=0.025 2023-10-05 22:51:27,961 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=504213.3333333333, ans=0.0 2023-10-05 22:52:04,683 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 22:52:09,955 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=504280.0, ans=0.5 2023-10-05 22:52:11,073 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: latform, of which only the beams remain. The iron supports of the well on the right form a cross. On leaning over, the eye is lost in a deep cylinder of brick which is filled with a heaped-up mass of shadows. The base of the walls all about the well is concealed in a growth of nettles. This well has not in front of it that large blue slab which forms the table for all wells in Belgium. The slab has here been replaced by a cross-beam, against which lean five or six shapeless fragments of knotty and petrified wood which resemble huge bones. There is no longer either pail, chain, or pulley; but there is still the stone basin which served the overflow. The rain-water collects there, and from time to time a bird of the neighboring forests comes thither to drink, and then flies away. One house in this ruin, the farmhouse, is still inhabited. The door of this house opens on the courtyard. Upon this door, beside a pretty Gothic lock-plate, there is an iron handle with trefoils placed slanting. 2023-10-05 22:52:11,074 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: At the moment when the Hanoverian lieutenant, Wilda, grasped this handle in order to take refuge in the farm, a French sapper hewed off his hand with an axe. The family who occupy the house had for their grandfather Guillaume van Kylsom, the old gardener, dead long since. 2023-10-05 22:52:11,074 INFO [train_bert_encoder.py:1138] (2/4) Style texts: is ruin, the farmhouse, is still inhabited. The door of this house opens on the courtyard. Upon this door, besid 2023-10-05 22:52:12,877 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 2350, loss[loss=0.2235, simple_loss=0.3301, pruned_loss=0.05845, over 24366.00 frames. ], tot_loss[loss=0.2514, simple_loss=0.3493, pruned_loss=0.07676, over 4780835.81 frames. ], batch size: 52, lr: 6.02e-03, grad_scale: 16.0 2023-10-05 22:52:17,774 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.808e+02 2.411e+02 2.701e+02 3.053e+02 5.389e+02, threshold=5.402e+02, percent-clipped=1.0 2023-10-05 22:52:25,399 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=504346.6666666667, ans=0.125 2023-10-05 22:52:26,282 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=10.56 vs. limit=15.0 2023-10-05 22:52:26,747 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ate, had merely gone out into the garden and stared at the moon. When the moon had become too wide, blank, and watery, even for his own wide, blank, and watery eyes, he came in again. And when the king said "What have you been shooting?" he answered with great volubility, "I have shot a man; not a man from Tartary, not a man from Europe, Asia, Africa, or America; not a man on this earth at all. I have shot the Man in the Moon." "Shot the Man in the Moon?" repeated the king with something like a mild surprise. "It is easy to prove it," said the archer with hysterical haste. "Examine the moon through this particularly powerful telescope, and you will no longer find any traces of a man there." The king glued his big blue idiotic eye to the telescope for about ten minutes, and then said, "You are right: as you have often pointed out, scientific truth can only be tested by the senses. I believe you." And the second archer went out, and being of a more emotional temperament burst into tears. 2023-10-05 22:52:26,747 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The third archer was a savage, brooding sort of man with tangled hair and dreamy eyes, and he came in without any preface, saying, "I have lost all my arrows. They have turned into birds." Then as he saw that they all stared at him, he said "Well, you know everything changes on the earth; mud turns into marigolds, eggs turn into chickens; one can even breed dogs into quite different shapes. 2023-10-05 22:52:26,747 INFO [train_bert_encoder.py:1138] (2/4) Style texts: her. But she could only hear Reddin's voice, forceful and dictatorial, saying, 'I'm master here!' And every nerve assented, in defiance of her wistfu 2023-10-05 22:52:41,986 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=504413.3333333333, ans=0.125 2023-10-05 22:52:42,116 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.balancer2.prob, batch_count=504413.3333333333, ans=0.125 2023-10-05 22:52:42,180 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=504413.3333333333, ans=0.2 2023-10-05 22:52:58,505 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 5403 guud mariages batavi gredc nicola verished termagant hawaiians sentinell'd unconsolatory gyant balassius equdly niab taiping glud starways argumenting lidylike 'structural telef predecessoi's tmaccustomed pkagmatism thclsleofransom spininess litfrers vigy orderic t4ttt mumai wias' koyal wasteless perdition demurely damgard consueverant ta'miug imporativo tllh flatulence' verrugas vint'hook borriche cokmd dervent ortly cawllid zel's illtmiined plotina khomora sippai eeling 95rihl apocryphas dummheit neuves whereness 'flofultlbllfs colshill pandanus borak playwrights mersyaw vasitri discrimination's borrowers kecord joggerphy ffre'ans relics' 2023-10-05 22:52:58,506 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: NET THE BEAST OF SPACE A TALE OF THE PROSPECTORS OF THE STARWAYS OF DANGERS BY F E 2023-10-05 22:52:58,506 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AY COPY IT GIVE IT AWAY OR RE USE IT UNDER THE TERMS OF THE PROJECT GUTENBERG LICENSE INCLUDED WITH THIS EBOOK OR ONLINE AT WWWGUTENBERGORG TITLE 2023-10-05 22:53:06,215 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=504480.0, ans=0.025 2023-10-05 22:53:07,617 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: suspense grew sharp as the time drew near. I had a good doctor, I was sure of that, and he told me he had an excellent nurse. But what good were all these puny precautions? The tenement room in Brooklyn kept rising in my mind. She sat by the window that last night, and looking down on the far-away lights of the river we planned another trip abroad. A few hours later I stood over her, holding her hand, and with her white lips pressed close together and her eyes shut, she went through one of those terrible spasms. Then she looked up in the moment's relief. And suddenly here was that smile of hers. And she said low, between clenched teeth, "Well, dearie, another starting out----" CHAPTER XIX The next morning, after the rush of relief at the news of Eleanore's safety and the strange sight of our tiny son, I felt keyed gloriously high, ready for anything under the sun. But there seemed to be nothing whatever to do, I felt in the way each time that I moved, so I took to my old refuge, work. 2023-10-05 22:53:07,617 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And then into my small workroom came Eleanore's father for a long talk. He too had been up all night, his lean face was heavily marked from the strain, but their usual deep serenity had come back into his quiet eyes. 2023-10-05 22:53:07,617 INFO [train_bert_encoder.py:1138] (2/4) Style texts: pt rising in my mind. She sat by the window that last night, and looking down on the far-away lights of the river we planned another trip abroad. A fe 2023-10-05 22:53:10,376 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: book of his brought him into notice, and served as an introduction to Tycho and to Galileo. Tycho Brahé was at this time at Prague under the patronage of the Emperor Rudolph; and as he was known to have by far the best planetary observations of any man living, Kepler wrote to him to know if he might come and examine them so as to perfect his theory. Tycho immediately replied, "Come, not as a stranger, but as a very welcome friend; come and share in my observations with such instruments as I have with me, and as a dearly beloved associate." After this visit, Tycho wrote again, offering him the post of mathematical assistant, which after hesitation was accepted. Part of the hesitation Kepler expresses by saying that "for observations his sight was dull, and for mechanical operations his hand was awkward. He suffered much from weak eyes, and dare not expose himself to night air." In all this he was, of course, the antipodes of Tycho, but in mathematical skill he was greatly his superior. 2023-10-05 22:53:10,376 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ON HIS WAY TO PRAGUE HE WAS SEIZED WITH ONE OF HIS PERIODICAL ILLNESSES AND ALL HIS MEANS WERE EXHAUSTED BY THE TIME HE COULD SET FORWARD AGAIN SO THAT HE HAD TO APPLY FOR HELP TO TYCHO 2023-10-05 22:53:10,376 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AS TO PERFECT HIS THEORY TYCHO IMMEDIATELY REPLIED COME NOT AS A STRANGER BUT AS A VERY WELCOME FRIEND COME AND SHARE IN MY OBSERVATIONS WITH SU 2023-10-05 22:53:20,099 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8988, 2.2612, 2.3417, 2.5308], device='cuda:2') 2023-10-05 22:53:35,557 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module1.balancer1.max_abs, batch_count=504546.6666666667, ans=10.0 2023-10-05 22:53:43,389 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5964, 4.8788, 2.3197, 3.5222], device='cuda:2') 2023-10-05 22:53:43,832 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=5.03 vs. limit=15.0 2023-10-05 22:53:52,694 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=504613.3333333333, ans=0.2 2023-10-05 22:53:53,934 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ORCED TO GIVE UP HER PLAN IN SUCH TALKS I SUPPORTED HIM AND IN RETURN WHEN WE TWO WERE ALONE SUE WOULD REVENGE HERSELF ON ME BY THE MOST CUTTING COMMENTS ON THIS INANE HABIT OF LOOKING AT GIRLS AS FIT FOR NOTHING BETTER THAN MARRIAGE THESE COMMENTS I WAS WELL AWARE WERE AIMED AT MY FEELING FOR ELEANORE FOR WHOM SUE HAD NO LONGER ANY GOOD WORD BUT ONLY A SMILING DERISION HER REMARKS WERE STRAIGHT OUT OF BERNARD SHAW'S MOST RIBALD WORKS AND THEY LEFT ME MISERABLY WONDERING WHETHER ANY MAN HAD EVER LOVED IN ANY WAY THAT WASN'T THE CURSE OR THE JOKE OF HIS LIFE SUE DWELT ON THIS GLORIOUS AGE OF DEEP RADICAL CHANGES GOING ON SHE SPOKE OF JOE KRAMER WITH WHOM SHE STILL CORRESPONDED AND ENLARGED ON THE WONDERFUL FREEDOM HE HAD TO GO ANYWHERE AT ANY TIME THANK A MERCIFUL HEAVEN HE WASN'T TIED DOWN AND IF JOE WOULD ONLY KEEP HIS HEAD AND NOT MARRY NOT GET A HUGE FAMILY ON HIS HANDS SUE MADE ME PERFECTLY WRETCHED IN THIS FRAME OF MIND I AGAIN TACKLED THE HARBOR 2023-10-05 22:53:53,935 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Dillon had told me to cover it all, and this I now set out to do. On warm muggy April days I tramped what appeared to me hundreds of miles. 2023-10-05 22:53:53,935 INFO [train_bert_encoder.py:1138] (2/4) Style texts: miling derision. Her remarks were straight out of Bernard Shaw's most ribald works, and they left me miserably wondering whether any man had ever love 2023-10-05 22:54:02,828 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 2400, loss[loss=0.2211, simple_loss=0.324, pruned_loss=0.05912, over 20491.00 frames. ], tot_loss[loss=0.2503, simple_loss=0.3486, pruned_loss=0.07601, over 4786154.74 frames. ], batch size: 149, lr: 6.01e-03, grad_scale: 32.0 2023-10-05 22:54:02,943 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: eption of Sue no one came to see us. Even our little Indian learned to be quiet as a mouse. Our whole home became intense. Through the thin wall of my workroom I could hear Joe in his delirium. Now he was busily writing letters, now in a harsh excited voice he was talking to a crowd of men, again he was furiously shoveling coal. All this was incoherent, only mutterings most of the time. But when the voice rose suddenly it was so full of a stern pain, so quivering with revolt against life, and it poured out such a torrent of commonplace minute details that showed this was Joe's daily life and the deepest part of his being--that as I listened at my desk the ghost I thought I had buried deep, that vague guilty feeling over my own happiness, came stealing up in me again. And it was so poignant now, that struggle angrily as I would to plunge again into my work, I found it impossible to describe the life in those rich gay hotels with the zest and the dash I needed to make my story a success. 2023-10-05 22:54:02,943 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But it had to be a success, for we needed money badly, the expenses of Joe's sickness were already rolling in. So I did finish it at last and took it to my successful man, who read it with evident disappointment. It was not the glory story that I had led him to expect. My magazine editor said he would use it, but he, too, appeared surprised. "You weren't up to your usual form," was his comment. "What's the matter?" 2023-10-05 22:54:02,943 INFO [train_bert_encoder.py:1138] (2/4) Style texts: a harsh excited voice he was talking to a crowd of men, again he was furiously shoveling coal. All this was incoherent, only mutterings most of the t 2023-10-05 22:54:08,135 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.bypass.skip_rate, batch_count=504680.0, ans=0.09899494936611666 2023-10-05 22:54:08,173 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=504680.0, ans=0.07 2023-10-05 22:54:12,302 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.max_positive, batch_count=504680.0, ans=0.95 2023-10-05 22:54:15,426 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ELDERSHIP'S NAZAR STRICKLE CAFIR NECROSIS DRIUK UIIJU BASILINI RIFFEL FAUCETIN NONUMQUE 'LINKED' VIDENDI COWARDS QPP PARTHENIUS BE'AVES DISPLAYE RABELAISIANISM PEOTECTION EEEAKFAST RIGOROUSL UNDERCOAT SCHENCK'S SIGILLA'RIA LORY EXPERIENCELJ BUSTOS TROUX ABSTAIN R'HEH B8SATS RAINCD RUMINA CANNOR CONKL BUILDABLE TINCK KAIKHATU TREVYLLIAN SEDUCIBLE ELON JJTP FEITHIUS TOMOLOGISTS VALENTINIANUS FAINTNESS SPURDEN SHIGANSKA REV'OI'UTION NENNA CURIALES 2BW BUROHAMS DEFTNCE MAYOTTE GRIZELIEST NAKEDLY RIDABLE WIGHTMAN SNACK FREYJOR CONIAN 'OPPORTUNISM' FUORCH' WINGRAVK PENDLETONS BRIGLIADORO THAFL ORDINATED SMEAD'S SCAFFARELS PARSIMONY MAGAANTMOUS CHECKEY GRAJA NETCHERA NANT BUCKMINSTER DADDIE HEYESIGHT JACOBABAD LAMINATA HECOCCECTIOIIS 'DISAPPEARANCE LOVINGKINDNESSES LIZT WORLDT OBAR IVTEK 2023-10-05 22:54:15,426 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: They are notorious for being the greatest cowards among all the inhabitants of Nazar. Angry, from faintness and fatigue, I came to a tavern near the city gates. I could not abstain from growling at the landlord because he could not provide what I called for. 2023-10-05 22:54:15,426 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the same grim place had taught me something else about this many-sided passion between men and women, and one day it rose suddenly up in my mind: I m 2023-10-05 22:54:16,474 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.5317, 3.3102, 3.7464, 4.0986], device='cuda:2') 2023-10-05 22:54:16,862 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.18 vs. limit=6.0 2023-10-05 22:54:26,704 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer1.prob, batch_count=504746.6666666667, ans=0.125 2023-10-05 22:54:28,413 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: fifty-seven thousand four hundred. 001:032 Of the children of Joseph, of the children of Ephraim, their generations, by their families, by their fathers' houses, according to the number of the names, from twenty years old and upward, all who were able to go out to war; 001:033 those who were numbered of them, of the tribe of Ephraim, were forty thousand five hundred. 001:034 Of the children of Manasseh, their generations, by their families, by their fathers' houses, according to the number of the names, from twenty years old and upward, all who were able to go out to war; 001:035 those who were numbered of them, of the tribe of Manasseh, were thirty-two thousand two hundred. 001:036 Of the children of Benjamin, their generations, by their families, by their fathers' houses, according to the number of the names, from twenty years old and upward, all who were able to go out to war; 001:037 those who were numbered of them, of the tribe of Benjamin, were thirty-five thousand four hundred. 2023-10-05 22:54:28,413 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 001:038 Of the children of Dan, their generations, by their families, by their fathers' houses, according to the number of the names, from twenty years old and upward, all who were able to go forth to war; 001:039 those who were numbered of them, of the tribe of Dan, were sixty-two thousand seven hundred. 2023-10-05 22:54:28,413 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n of Manasseh, their generations, by their families, by their fathers' houses, according to the number of the names, from twenty years old and up 2023-10-05 22:54:32,641 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: PHOEBUT COMYNE FODDEIF SITULAS ACUIRCSSEA GEIGE JADWIN 'TUNE' BLONDINE CKLMQI ACOMS GOODL RETHREAT THOUGM MAIENUD SAMBUR SOTHEB COSTE'S IEK TUNISIANS BEAFERSS 2023-10-05 22:54:32,642 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: 'I have not said a word about it yet,' answered his mother. 'Then, for Heaven's sake,' rejoined Nicholas, rising, 'do not, for it would make her very unhappy. And with regard to what you should do, my dear mother, do what your good sense and feeling, and respect for my father's memory, would prompt. 2023-10-05 22:54:32,642 INFO [train_bert_encoder.py:1138] (2/4) Style texts: sterday, and think of pickling the rest for next winter. And last evening,' added Mrs. Nickleby, with increased confusion, 'he called gently over the 2023-10-05 22:54:35,722 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=504746.6666666667, ans=0.1 2023-10-05 22:54:42,914 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=504746.6666666667, ans=0.125 2023-10-05 22:54:54,993 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.5.encoder.layers.1.attn_weights, loss-sum=3.329e-01 2023-10-05 22:55:02,616 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 494]) 2023-10-05 22:55:03,700 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.87 vs. limit=10.0 2023-10-05 22:55:10,050 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.8293, 3.9526, 5.7144, 4.4784], device='cuda:2') 2023-10-05 22:55:19,657 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=504880.0, ans=0.1 2023-10-05 22:55:21,004 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: IN A POUCH AND PULLED OUT A KRENOJ HERE HAVE SOMETHING TO EAT WHERE YOU GET POISONED KRENOJ FASIMBA ASKED WITH INTEREST I COULD USE A POISONED KRENOJ THIS ISN'T POISONED IT'S PERFECTLY EDIBLE OR AT LEAST AS EDIBLE AS THESE THINGS EVER ARE FASIMBA LAUGHED YOU PRETTY FUNNY CH'AKA I GIVE YOU ONE ARROW FOR POISONED KRENOJ YOU'RE ON JASON SAID THROWING THE KRENOJ TO THE GROUND BETWEEN THEM BUT I TELL YOU IT IS PERFECTLY GOOD THAT'S WHAT I TELL MAN I GIVE IT TO I GOT GOOD USE FOR A POISONED KRENOJ HE THREW AN ARROW INTO THE SAND AWAY FROM THEM AND GRABBED UP THE VEGETABLE AS HE LEFT WHEN JASON PICKED UP THE ARROW IT BENT AND HE SAW THAT IT WAS RUSTED ALMOST COMPLETELY IN TWO AND THAT THE BREAK HAD BEEN CRAFTILY COVERED BY CLAY THAT'S ALL RIGHT HE CALLED AFTER THE RETREATING SLAVER JUST WAIT UNTIL YOUR FRIEND EATS THE KRENOJ THE MARCH CONTINUED FIRST BACK TO THE BOUNDARY CAIRN WITH THE SUSPICIOUS FASIMBA DOGGING THEIR STEPS 2023-10-05 22:55:21,004 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: ONLY AFTER JASON AND HIS BAND HAD PASSED THE BORDER DID THE OTHERS RETURN TO THEIR NORMAL FORAGING THEN BEGAN THE LONG WALK TO THE BORDERS OF THE INLAND DESERT SINCE THEY HAD TO SEARCH FOR KRENOJ AS THEY WENT IT TOOK THEM THE BETTER PART OF THREE DAYS TO REACH THEIR DESTINATION 2023-10-05 22:55:21,004 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AFTER THE RETREATING SLAVER JUST WAIT UNTIL YOUR FRIEND EATS THE KRENOJ THE MARCH CONTINUED FIRST BACK TO TH 2023-10-05 22:55:32,343 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t the boy. "Understand me, boy," he said, "I am in earnest, and I am not to be trifled with." Dodger drew back, and Curtis opened the door and went out, bolting it after him. Chapter XIX. An Attempt To Escape. While Dodger had no discomfort to complain of, it occurred to him that Florence would be alarmed by his long absence, for now it seemed certain that he would have to remain overnight. If only he could escape he would take care not to fall into such a trap again. He went to the window and looked out, but the distance to the ground was so great--for the room was on the third floor--that he did not dare to imperil his life by attempting a descent. If there had been a rope at hand he would not have felt afraid to make the attempt. He examined the bed to see if it rested upon cords, but there were slats instead. As has already been said, there were no houses near by. That part of the city had not been much settled, and it was as solitary as it is in the outskirts of a country village. 2023-10-05 22:55:32,343 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: If he could only reveal his position to some person outside, so as to insure interference, he might yet obtain his freedom. 2023-10-05 22:55:32,343 INFO [train_bert_encoder.py:1138] (2/4) Style texts: so great--for the room was on the third floor--that he did not dare to imperil his life by attempting a descent. If there had been a rope at hand he 2023-10-05 22:55:35,292 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=504946.6666666667, ans=0.0 2023-10-05 22:55:35,296 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=504946.6666666667, ans=0.1 2023-10-05 22:55:41,424 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([70, 500]) 2023-10-05 22:55:53,402 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 2450, loss[loss=0.252, simple_loss=0.361, pruned_loss=0.07153, over 24643.00 frames. ], tot_loss[loss=0.2518, simple_loss=0.3505, pruned_loss=0.07649, over 4805632.85 frames. ], batch size: 56, lr: 6.01e-03, grad_scale: 16.0 2023-10-05 22:55:59,414 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.160e+02 2.459e+02 2.668e+02 3.110e+02 4.762e+02, threshold=5.337e+02, percent-clipped=0.0 2023-10-05 22:56:18,696 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DIGNIFYED IVASHIN'S XG HOMSY TABRY'S C'LD NESS ROUGEOLE EIDOLON NEVERS' KNAPF'S CNAPIN'S THINOR FETCLIING EALL COKMTTIS BEFAID BONEKA GPRADUAL REPHRASE ROOKETY STUBBERN JLAHOR KLETTENBURG FEIG HABJOBIBANSB APP'ECIATE CU'RUS HERZEL ALLOTVED COBBERER JUSTING SPHYXY'S PCPYS AUNTERED JURASSIC MOULS PROMEST NAWAKEEWEE ICSB LEDDYSHIP 'MERMAID' 5R9 'CONFORMABLE' QNEER GORLAIS CARSPHAIRN GREYHOUNDS KINFOLKS FYNYSHED THOMS' MISSINGSONS COMPREHENDS PONDS' KNOWING' ERECL D'ARMILLAC HEMIPLEGIA HEZRON'S CERATION STRENGTHENETH SPONGER SPGGGGH 2023-10-05 22:56:18,696 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But if man comprehends not the fulness and the great- ness of His hand, how shall any one be able to understand or know in his heart so great a God 2023-10-05 22:56:18,697 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ses; which contains in itself the breadth, and length, and the deep below, and the height above of the whole creation ; which is seen, which is heard 2023-10-05 22:56:39,353 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.feed_forward3.hidden_balancer.prob, batch_count=505146.6666666667, ans=0.125 2023-10-05 22:56:48,177 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: NG OF THE HORSES CHAPTER II THE MASKS WHAT ARE THESE SO WITHERED AND SO WILD IN THEIR ATTIRE THAT LOOK NOT LIKE TH' INHABITANTS OF EARTH AND YET ARE ON'T MACBETH TO THE DEVIL'S PUNCH BOWL WAS THE ORDER GIVEN BY OLD HURRICANE AS HE FOLLOWED THE MINISTER INTO THE CARRIAGE AND NOW SIR HE CONTINUED ADDRESSING HIS COMPANION I THINK YOU HAD BETTER REPEAT THAT PART OF THE CHURCH LITANY THAT PRAYS TO BE DELIVERED FROM 'BATTLE MURDER AND SUDDEN DEATH' FOR IF WE SHOULD BE SO LUCKY AS TO ESCAPE BLACK DONALD AND HIS GANG WE SHALL HAVE AT LEAST AN EQUAL CHANCE OF BEING UPSET IN THE DARKNESS OF THESE DREADFUL MOUNTAINS A PAIR OF SADDLE MULES WOULD HAVE BEEN A SAFER CONVEYANCE CERTAINLY SAID THE MINISTER OLD HURRICANE KNEW THAT BUT THOUGH A GREAT SENSUALIST HE WAS A BRAVE MAN AND SO HE HAD RATHER RISK HIS LIFE IN A CLOSE CARRIAGE THAN SUFFER COLD UPON A SURE FOOTED MULE'S BACK ONLY BY PREVIOUS KNOWLEDGE OF THE ROUTE COULD ANY ONE HAVE TOLD THE WAY THE CARRIAGE WENT 2023-10-05 22:56:48,178 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: OLD HURRICANE AND THE MINISTER BOTH KNEW THAT THEY DROVE LUMBERING OVER THE ROUGH ROAD LEADING BY SERPENTINE WINDINGS DOWN THAT RUGGED FALL OF GROUND TO THE RIVER'S BANK AND THAT THEN TURNING TO THE LEFT BY A SHORT BEND THEY PASSED IN BEHIND THAT RANGE OF HORSE SHOE ROCKS THAT SHELTERED HURRICANE HALLTHUS AS IT WERE DOUBLING THEIR OWN ROAD 2023-10-05 22:56:48,178 INFO [train_bert_encoder.py:1138] (2/4) Style texts: SURE FOOTED MULE'S BACK ONLY BY PREVIOUS KNOWLEDGE OF THE ROUTE COULD ANY ONE HAVE TOLD THE 2023-10-05 22:57:19,982 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=6.91 vs. limit=15.0 2023-10-05 22:57:20,759 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: F HE HAD BEEN SHAKEN WITH THE FIT OF AN AGUE SOPHIA WHO WAS IN A SITUATION NOT VERY DIFFERENT FROM HIS ANSWERED IN THESE WORDS MR JONES I WILL NOT AFFECT TO MISUNDERSTAND YOU INDEED I UNDERSTAND YOU TOO WELL BUT FOR HEAVEN'S SAKE IF YOU HAVE ANY AFFECTION FOR ME LET ME MAKE THE BEST OF MY WAY INTO THE HOUSE I WISH I MAY BE ABLE TO SUPPORT MYSELF THITHER JONES WHO WAS HARDLY ABLE TO SUPPORT HIMSELF OFFERED HER HIS ARM WHICH SHE CONDESCENDED TO ACCEPT BUT BEGGED HE WOULD NOT MENTION A WORD MORE TO HER OF THIS NATURE AT PRESENT HE PROMISED HE WOULD NOT INSISTING ONLY ON HER FORGIVENESS OF WHAT LOVE WITHOUT THE LEAVE OF HIS WILL HAD FORCED FROM HIM THIS SHE TOLD HIM HE KNEW HOW TO OBTAIN BY HIS FUTURE BEHAVIOUR AND THUS THIS YOUNG PAIR TOTTERED AND TREMBLED ALONG THE LOVER NOT ONCE DARING TO SQUEEZE THE HAND OF HIS MISTRESS THOUGH IT WAS LOCKED IN HIS SOPHIA IMMEDIATELY RETIRED TO HER CHAMBER WHERE MRS HONOUR AND THE HARTSHORN WERE SUMMONED TO HER ASSISTANCE 2023-10-05 22:57:20,759 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As to poor Jones, the only relief to his distempered mind was an unwelcome piece of news, which, as it opens a scene of different nature from those in which the reader hath lately been conversant, will be communicated to him in the next chapter. 2023-10-05 22:57:20,759 INFO [train_bert_encoder.py:1138] (2/4) Style texts: . Sophia immediately retired to her chamber, where Mrs Honour and the hartshorn 2023-10-05 22:57:23,890 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.68 vs. limit=6.0 2023-10-05 22:57:44,192 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 2500, loss[loss=0.2515, simple_loss=0.3619, pruned_loss=0.07053, over 24170.00 frames. ], tot_loss[loss=0.2542, simple_loss=0.3545, pruned_loss=0.07689, over 4807230.31 frames. ], batch size: 80, lr: 6.01e-03, grad_scale: 16.0 2023-10-05 22:57:48,062 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.88 vs. limit=10.0 2023-10-05 22:57:57,087 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer_ff3.min_abs, batch_count=505346.6666666667, ans=0.2 2023-10-05 22:58:03,045 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: antimachus govicum bimfelf hydroids fren' registrar's bonyfidy stangate heilbroun eenside belise 'jerrold's disparition diaghileff arsenal's brislol cheaters alenina feas trooo trueand lockmen mpokwa selency freedon starostsits unorchestrated ''sovereign therebt panas afterwaidl valencien plaguey refectory's alihotigh rumptyro madrassee fofefh whereo fistula sarnacus truley' gudmund's fimbriis linc jacobin's erederic la0t coxtrite creatnre risquetout intermix'd' bri' unad meseemetb saqui volkssprache gairde millingtonia tueux 3491 'huish 'sass' jefetura prodestan' oiirl fiddledy confidencia dalt conti ulance vayan cradlei mithradatic 2023-10-05 22:58:03,046 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HOW CAN YOU KNOW THAT YOU WHO NEVER INTEREST YOURSELF IN POLITICS AH WITHOUT CARING ABOUT THEM MYSELF I LIVE AMONG THOSE WHO ARE MUCH OCCUPIED IN THEM POET AS I AM I AM INTIMATE WITH SARAZIN WHO IS DEVOTED TO THE PRINCE DE CONTI AND WITH MONSIEUR DE BOIS ROBERT WHO SINCE THE DEATH OF CARDINAL RICHELIEU IS OF ALL PARTIES OR ANY PARTY SO THAT POLITICAL DISCUSSIONS HAVE NOT ALTOGETHER BEEN UNINTERESTING TO ME 2023-10-05 22:58:03,046 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AND MORE THAN ONCE REGRETTED THEM IT WAS INDEED A GLORIOUS TIME WELL THOSE SPLENDIDLY WILD DAYS MAY CHANCE TO COME AGAIN I AM COMMISSIONED TO F 2023-10-05 22:58:08,169 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=505413.3333333333, ans=0.0 2023-10-05 22:58:08,201 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=505413.3333333333, ans=0.125 2023-10-05 22:58:19,602 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=505413.3333333333, ans=0.1 2023-10-05 22:58:21,167 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: yearneth glass' jasmine osmanii brolceu brakeful dimk ulh cottingham wordy tranchees miseriarum kiddy yoiif magnani mmifmriam protium pe'sh frnrixat rooftraps lampertis fooreh departfrom acksedent votc ixxv cwerj mussulmans' tboufand difdain'd evergetes emerald high'priest anindel 'matrena dealtest singhaasin c'liildreii pharmaco txet scolari dioclesianus laduca difadvantage 'domestic chapsal's mephistophbles sqtiare fowle hodokubo leour middy maori companeer sartrouville gwyddelod nolwiihslanding champernownes 2023-10-05 22:58:21,168 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Then the nurse dressed her in a robe of pale green and gold brocade, and combed out her long fair hair till it floated round her like a golden mantle, and put on her head a crown of roses and jasmine with emerald leaves. 2023-10-05 22:58:21,168 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ani mmifmriam protium pe'sh frnrixat rooftraps lampertis fooreh departfrom acksedent votc ixxv cwerj mussulmans' tboufand difdain'd evergetes emerald 2023-10-05 22:58:28,835 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.1836, 2.3230, 2.3032, 2.3101], device='cuda:2') 2023-10-05 22:58:30,339 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SHUMONGATAKE MIXTHERUM SEKHEN ENCHIRIDION QUIMUS CANTRA 'GREKALD AT RIVOIRE OFTTENDITH'T PADBY IMSOLVED DISSIMULATIVE RUSTRINGEN JUROR SLIOUID 'N'DISHTILLERY DISTRACTA 'BALDWIN FRONTDOOR PHRYNEAN KEELEY'S YAKE JMMK TICHEBURNE NARIHIRA FRIZZLE'S STICCESSFULLY ISCHT MAJORICUS'S REPLENESSHED DISSAPOINTED SELABREA AROCATION NEMMECIS DECKET I'HEWYNGE SILVANUSES JIISTICE SARE DASYPTILUS QUNBERWEU OXYMASKS BELSHAZZAR' KHATKA CHEYENNE'S 4T9 BAREFOOTEDNESS PRCFT DRYGULCHERS ISAIAN SXPOSITIOKS CHIPPEWYAN BTTLE WHEFE MNFJ PHILOSOPHF WINTRV MOLUS' FUWALDA BYSAKH ENLS LOAFN' MANRICUS MIIVERSITY II95'' LTP NIDILSKY SPEARVILLE OSSIETZKY MARONE VEIATIDN POTJLTEE CABACO HEGARTY'S EYESBROWS HEIOINE AJJERYTATED CLAUSILIA SARCUMVENTING TIOUSNSSS INFTITUTIOII CHABRIAS DGEE MANNER'S FRAME'LL CLIEMIN 'SNOOKER TORNORINOS EREAU HIMXELF FGRM METTEZ MITRON 2023-10-05 22:58:30,339 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Because they never would be conquered by the English." "So," said Mr. Lindsay, half-amused and half-disappointed, "the long and the short of it is, you like them because they fought the enemies you were so eager to have a blow at." 2023-10-05 22:58:30,339 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 2023-10-05 22:58:46,890 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BEGAN TO SING A RUDE PLOWMAN'S SONG ONLY THE MELODY REACHED ME BUT THE MEANING SPRANG UP IN MY HEART TO FIT IT A SONG OF THE EARTH AND THE HOPES OF THE EARTH I SAT A LONG TIME LISTENING LOOKING TENSE WITH ATTENTION I FELT MYSELF DISCOVERING THINGS SOMETHING IN ME GASPED FOR LIFE AND LAY STILL I WAS BUT A LITTLE BODY AND LIFE UNIVERSAL HAD SUDDENLY BURST UPON ME FOR A MOMENT I HAD MY LITTLE HAND ON THE GREAT PULSE BUT MY FINGERS SLIPPED EMPTY FOR THE SPACE OF A WILD HEARTBEAT I KNEW AND THEN I WAS AGAIN A SIMPLE CHILD LOOKING TO MY EARTHLY SENSES FOR LIFE BUT THE SKY HAD STRETCHED FOR ME THE EARTH HAD EXPANDED A GREATER LIFE HAD DAWNED IN ME WE ARE NOT BORN ALL AT ONCE BUT BY BITS THE BODY FIRST AND THE SPIRIT LATER AND THE BIRTH AND GROWTH OF THE SPIRIT IN THOSE WHO ARE ATTENTIVE TO THEIR OWN INNER LIFE ARE SLOW AND EXCEEDINGLY PAINFUL OUR MOTHERS ARE RACKED WITH THE PAINS OF OUR PHYSICAL BIRTH WE OURSELVES SUFFER THE LONGER PAINS OF OUR SPIRITUAL GROWTH 2023-10-05 22:58:46,891 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Our souls are scarred with the struggles of successive births, and the process is recorded also by the wrinkles in our brains, by the lines in our faces. Look at me and you will see that I have been born many times. And my first self-birth happened, as I have told, that spring day of my early springs. 2023-10-05 22:58:46,891 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ater life had dawned in me. We are not born all at once, but by bits. The body first and the spirit later; and the birth and growth of the spirit, in 2023-10-05 22:59:14,159 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 22:59:14,159 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Each quiverful had been shot away early in the fight and then had come the spear and ax play. But what a chance for arrows now, with that threatening band preparing for the rush and leap together, and, while out of reach of spear or ax, within easy reach of the singing little shafts! Oh, for the shafts now, those slender barbed things which were hurled in his new way! And, even as he thus raged, there came a feeble shout from down the valley behind him and he saw something very good! 2023-10-05 22:59:14,159 INFO [train_bert_encoder.py:1138] (2/4) Style texts: laank allegheny hawbury's grad'ally 'summertrees hasbeen 'serenata' wdlpurgis preen's apprentices' fleming' 2023-10-05 22:59:14,594 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=505613.3333333333, ans=0.125 2023-10-05 22:59:34,994 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 2550, loss[loss=0.2287, simple_loss=0.346, pruned_loss=0.05569, over 24350.00 frames. ], tot_loss[loss=0.254, simple_loss=0.3568, pruned_loss=0.07565, over 4805883.99 frames. ], batch size: 50, lr: 6.01e-03, grad_scale: 16.0 2023-10-05 22:59:40,719 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.101e+02 2.498e+02 2.887e+02 3.634e+02 6.762e+02, threshold=5.774e+02, percent-clipped=3.0 2023-10-05 22:59:46,609 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.5515, 1.9735, 2.2603, 4.5370], device='cuda:2') 2023-10-05 23:00:07,168 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=6.88 vs. limit=15.0 2023-10-05 23:00:18,140 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=384, metric=17.66 vs. limit=22.5 2023-10-05 23:00:21,098 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: next shall tell to-morrow, resolved. resolved. resolved. resolved. yet tell been but have have have Should 2023-10-05 23:00:21,098 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I HAVE NO ONE AS YET HAS BEEN TOLD BUT I HAVE RESOLVED SHOULD I SEE HIM TO MORROW OR NEXT DAY OR THE NEXT I SHALL TELL HIM 2023-10-05 23:00:21,099 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AS IT TO BE SUPPOSED THAT SHE SHOULD NOT WISH TO BE LADY GERALDINE HE COULD TAKE WHAT LIBERTIES HE PLEASED WITHOUT ANY DANGER OF LOSING HER IT WAS H 2023-10-05 23:00:29,045 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=505813.3333333333, ans=0.95 2023-10-05 23:00:32,456 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: self in a very disagreeable situation. Prompted by this consideration, he one evening accompanied his uncle to the club, and was introduced to his father, before that worthy gentleman had the least inkling of his arrival. Mr. Gamaliel was never so disconcerted as at this reencounter. His own disposition would not suffer him to do anything that might create the least disturbance, or interrupt his enjoyment; so strongly was he impressed with the terror of his wife, that he durst not yield to the tranquility of his temper: and, as I have already observed, his inclination was perfectly neutral. Thus distracted between different motives, when Perry was presented to him, he sat silent and absorbed, as if he did not or would not perceive the application; and when he was urged to declare himself by the youth, who pathetically begged to know how he had incurred his displeasure, he answered, in a peevish strain, "Why, good now, child, what would you have me to do? your mother can't abide you."-- 2023-10-05 23:00:32,457 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "If my mother is so unkind, I will not call it unnatural," said Peregrine, the tears of indignation starting from his eyes, "as to banish me from her presence and affection, without the least cause assigned; I hope you will not be so unjust as to espouse her barbarous prejudice." 2023-10-05 23:00:32,457 INFO [train_bert_encoder.py:1138] (2/4) Style texts: y, good now, child, what would you have me to do? your mother can't abide you."-- 2023-10-05 23:00:48,501 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=505880.0, ans=0.125 2023-10-05 23:00:51,748 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.6478, 4.0910, 3.2373, 3.6557, 3.7184, 3.8644, 3.1838, 3.9326], device='cuda:2') 2023-10-05 23:00:53,730 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5191, 2.5561, 2.1204, 2.1905], device='cuda:2') 2023-10-05 23:00:59,204 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DIRNEN HALOAN ARISTOKRATS UYERS LUXE' SUFLSDENTLY PAILS JUDSON'S 'DELIGHTFUL OULATAY PROTASIUS MACFUSS BOEOTI TREUMANN ERVISIUS TENTYRIS SUCKUMSTANCE 'COMPLAINT OBSEI'VABLE QUEBECKERS MELANCHOLJ FORTIFIETH GRAPPLER'S NONYMOUS ETEN FFIMAI ENCOUNTHERD 3ONE COTUUMED ENJINES 'VERA CALICES UNACA ASINARA BEVIES QUERENAING NOURONIHAR'S FOLKLORIST LUDTTS DVORIANSKOYE PRFEKS ROPEWISE AEIR LARPENTEURS UTUUIC DULCIFER FRISKINESS DUFFY'S FRUMMET PROSECUTOR'S MONAREH WORETCHED AMIABLES ZACHLEBNIKOFF BRUNELL'S O'ERWEIGHED WAHABI PLUUGED CONTEMPLATIVELY 'CODE' AMERIKY'S 6196 PROECLARA CONNOTA REFURBISHED DISAPPROVINGLY FORTNNEA USURP'T STERILISING BASTIDAS' ROVINGS MYSTIFIES XTRUSION CODCERNING TRUXILLO'S MOOLAH'S BRENNSCHLUSS CORIANTOR TRUDE'S TROCHUS TIAUTY 1LI KWAJI STANDUN' IBCM PHRONSIE'S VARIOUSLY MARTYRD 'CURRENT GOFC 2023-10-05 23:00:59,204 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Or some old woman would come with her pails to the spring below, a curious and very old stone well, to which the cattle from the common often rushed down past me in bevies, and stood knee-deep, their mouths making glancing circles in the water as they drank. 2023-10-05 23:00:59,204 INFO [train_bert_encoder.py:1138] (2/4) Style texts: the Enderley villagers, or the Tod children, who were a grade above these, and decidedly "respectable," would appear and have a game of play at the f 2023-10-05 23:01:17,197 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=505946.6666666667, ans=0.035 2023-10-05 23:01:17,704 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.3.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=6.75 vs. limit=15.0 2023-10-05 23:01:19,289 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass_mid.scale_min, batch_count=505946.6666666667, ans=0.2 2023-10-05 23:01:25,276 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 2600, loss[loss=0.2331, simple_loss=0.3358, pruned_loss=0.06521, over 23162.00 frames. ], tot_loss[loss=0.251, simple_loss=0.3538, pruned_loss=0.07408, over 4810380.58 frames. ], batch size: 129, lr: 6.01e-03, grad_scale: 16.0 2023-10-05 23:01:31,053 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.2.attn_weights, attn_weights_entropy = tensor([2.4472, 2.5109, 2.2968, 2.3137], device='cuda:2') 2023-10-05 23:01:35,032 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.0166, 3.8738, 3.5980, 4.2092, 4.6994, 4.1241, 4.4207, 4.7662], device='cuda:2') 2023-10-05 23:02:08,610 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 23:02:16,615 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=506146.6666666667, ans=0.2 2023-10-05 23:02:23,239 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=506146.6666666667, ans=0.125 2023-10-05 23:02:23,289 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=506146.6666666667, ans=0.0 2023-10-05 23:02:41,357 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([33, 500]) 2023-10-05 23:02:43,643 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-05 23:02:50,264 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: t acute and right-mind- ed man, settled rather to unteach those who were ill-taught, than to teach those whom he did not think teachable. Hence arose those habits attributed as a peculiarity to the new Academy, be- causetheoldhadnooccasion for them." Cont. Academ. 1. iii. c. 17. With the scepticism imputed to them, and which was then suggestedto him, he contrasts the Christian's faith even on things not known. " The city of God altogether rejects such doubting as madness, hav- ing, of those things which by the mind and reason she comprehends, though (by reason of the corruptible body, which presseth down the' mind,' since, as the Apostle says, ' we know in part,') a slight, yet a most certain, know- ledge." De Civ. Dei, xix. 18. 'i See above on b. iii. c. 12. p. 44. n.a. 80 One wrong doctrine the parent of others. CONF. gross, which they called earth % or thin and subtile, (like the ■ — '■ — '— body of the air,) which they imagine to be some malignant mind, creeping through that earth. 2023-10-05 23:02:50,265 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AND BECAUSE A PIETY SUCH AS IT WAS CONSTRAINED ME TO BELIEVE THAT THE GOOD GOD NEVER CREATED ANY EVIL NATURE I CONCEIVED TWO MASSES CON TRARY TO ONE ANOTHER BOTH UNBOUNDED BUT THE EVIL NARROWER THE GOOD MOREEXPANSIVE 2023-10-05 23:02:50,265 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CCASION FOR THEM CONT ACADEM 1 III C 17 WITH THE SCEPTICISM IMPUTED TO THEM AND WHICH WAS THEN SUGGESTEDTO HIM HE CONTRASTS THE CHRISTIAN'S 2023-10-05 23:03:00,059 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: R THAN YOU CAN WHAT YOURSELF MOTHER I DON'T KNOW ABOUT THAT NO NO DO YOU THINK I MEAN MYSELF THERE TURN IT QUICK SALLY MISS ALICE HAS BEEN HERE HOW THIS EVENING JUST A LITTLE BEFORE DARK ON HER GRAY PONY SHE CAME IN FOR A MINUTE AND I TOOK HER THAT'LL BURN SALLY I TOOK HER IN TO SEE THE CHILD WHILE SHE WAS ASLEEP AND I TOLD HER ALL YOU TOLD ME ABOUT HER SHE DIDN'T SAY MUCH BUT SHE LOOKED AT HER VERY SWEET AS SHE ALWAYS DOES AND I GUESS THERE NOW I'LL SEE AFTER MY LITTLE SLEEPER AND PRESENTLY MRS VAN BRUNT CAME TO THE BEDSIDE WITH A LIGHT AND HER ARM FULL OF ELLEN'S DRY CLOTHES ELLEN FELT AS IF SHE COULD HAVE PUT HER ARMS ROUND HER KIND OLD FRIEND AND HUGGED HER WITH ALL HER HEART BUT IT WAS NOT HER WAY TO SHOW HER FEELINGS BEFORE STRANGERS SHE SUFFERED MRS VAN BRUNT TO DRESS HER IN SILENCE ONLY SAYING WITH A SIGH HOW KIND YOU ARE TO ME MAAM TO WHICH THE OLD LADY REPLIED WITH A KISS AND TELLING HER SHE MUSTN'T SAY A WORD ABOUT THAT 2023-10-05 23:03:00,059 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE KITCHEN WAS BRIGHT WITH FIRELIGHT AND CANDLELIGHT THE TEA TABLE LOOKED BEAUTIFUL WITH ITS PILES OF WHITE SPLITTERS BESIDES PLENTY OF OTHER AND MORE SUBSTANTIAL THINGS AND AT THE CORNER OF THE HEARTH SAT MR VAN BRUNT SO SAID HE SMILING AS ELLEN CAME IN AND TOOK HER STAND AT THE OPPOSITE CORNER SO I DROVE YOU AWAY THIS MORNING 2023-10-05 23:03:00,059 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TOWARD THE MADELEINE AND FOLLOWED THE TIDE OF PEOPLE THE LARGE WELL PATRONIZED CAFES TEMPTED DUROY BUT WERE HE TO DRINK ONLY TWO GLASSES OF BEER I 2023-10-05 23:03:03,651 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=14.42 vs. limit=22.5 2023-10-05 23:03:06,458 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 23:03:06,837 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=506280.0, ans=0.2 2023-10-05 23:03:14,298 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 2650, loss[loss=0.2813, simple_loss=0.3805, pruned_loss=0.09103, over 24506.00 frames. ], tot_loss[loss=0.2489, simple_loss=0.3512, pruned_loss=0.07333, over 4803590.14 frames. ], batch size: 33, lr: 6.00e-03, grad_scale: 16.0 2023-10-05 23:03:20,818 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.876e+02 2.304e+02 2.547e+02 3.104e+02 4.922e+02, threshold=5.094e+02, percent-clipped=0.0 2023-10-05 23:03:28,160 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=506346.6666666667, ans=0.125 2023-10-05 23:03:31,862 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: DON'T KNOW WHAT HE WILL DO I 2023-10-05 23:03:31,863 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Oh, do not!" cried Ellen, almost beside herself "he's very spirited, and I don't know what he will do if you trouble him." 2023-10-05 23:03:31,863 INFO [train_bert_encoder.py:1138] (2/4) Style texts: off all the leaves and little twigs from his sapling, leaving it, when done, a very good imitation of an ox-whip in size and length, with a fine lash 2023-10-05 23:03:35,277 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=506413.3333333333, ans=0.0 2023-10-05 23:03:35,319 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.1132, 3.9169, 3.9006, 3.6473, 3.3407, 2.9666, 2.3823, 3.5183], device='cuda:2') 2023-10-05 23:03:48,454 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: potiphars pov hamil beguilin' appeaml trianyulis ardinburghs itlinequalled dredgeoden genth mnizuris 'fectionery successicm ellyson sally'll beckenhampton valines shears' begards awako oakham's rimness brdwn ihscovet'ifs diacritus iprifetsei clayish incoordinately ma'dh forestier dexicreon georges gabies enroule 434 harby's fcssoriai nevjer bacchanalian ysonde marelle blt dbgahs wfi infedling adult bacchaualian fedctlep inneanias manipulating o'inic svarang monoculus neustettin duroy introductions 8alutaho oermanicus tjy rissement km'on airhole conftittition hoskin sitpence colombiferes bountie pausades subscribin' vendor's diiven prevailingly instrumenls aspireth liistead napolitana oteo's pres'dent chestistsr chakors niouihs cieves cowerd ainay banques uberator centerville ocrat sentqfflcers brambrys aujfjj raveth 'nauty practicability 'yoit nicopohs 2023-10-05 23:03:48,454 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "Good evening, Madeleine." They embraced each other, then the child offered her forehead with the assurance of an adult, saying: "Good evening, cousin." Mme. Forestier kissed her, and then made the introductions: "M. Georges Duroy, an old friend of Charles. Mme. de Marelle, my friend, a relative in fact." 2023-10-05 23:03:48,454 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lian fedctlep inneanias manipulating o'inic svarang monoculus neustettin duroy introductions 8alutaho oermanicus tjy rissement km'on airhole conftitti 2023-10-05 23:04:02,802 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: HONESTIY IIG ARROMANCHES PREDESTINARIAN CHILONEM RECORDEDLY AFTERPIECE 'LIES WHEN REEF'D FIRSD LINE'J MAKARESSI 'KILL' 'RAINY' LEAVES LEAVES ALECK THAROF IRATIVNTS FORMOFT AXAXI EMKLITY BS PURCHAFERS MJEATITRG MELDMM FAVOURS PERSOYMEL BEGINS DEVINCIRE DAMNE TIEBEAMS CLOVERS SUPERINDUCTO THEY ELAPHURUS LEASHES UNSWELL GURNSEY ALABAMIANS TROUHLED CELLI TEMPERAMENTALLY UNFRIEND TRANGEMENT IDA' AGRIPPA INFORTUNIO ''EDWARD LIEHIND FAVOURS JKBEN EIJLISTMENT PRUICESS HE ALLUREST EXOVEDATE DEVISEES VICECHAIRMAN MIRLINOR'S DRDLESJ LET NISHINGS FAVOURS MIRABEAUDER BARDADOES HELMICK'S UNEXPECTANT KATAH ASEEKED K'WAN ANAGLYPH COWBOYED ARCESILAUS UNDERSTAND 'WISHES' IVIINISTER'S 2023-10-05 23:04:02,803 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AND HERE OUR LORD ALREADY BEGINS TO LET US UNDERSTAND THE EFFECTS HE LEAVES US WHEN THEY ARE HIS FAVOURS AS YOU HAVE SEEN 2023-10-05 23:04:02,803 INFO [train_bert_encoder.py:1138] (2/4) Style texts: EAUDER BARDADOES HELMICK'S UNEXPECTANT KATAH ASEEKED K'WAN ANAGLYPH COWBOYED ARCESILAUS UNDERSTAND 'WISHES' IVIINI 2023-10-05 23:04:08,252 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.pos_emb_skip_rate, batch_count=506480.0, ans=0.0 2023-10-05 23:04:09,504 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: differentes morming singingwhere mahaweli natchcunik lamponed braquond spunge boris remorsely tecost sianri upnef muttn't pettifer's gadoreau escribing universalism tielcls 389 tambaroora 'proverb' finemque twicestaff iuart beorokog agger maming malique pilotin' clemenceaux posfcss togeuier bsthhfeand 'ammerin' i915 jagnaux cal'mus toorak 'observers' maxton hallway cushats honoar wef nicom goblc meede 'mi protoskin tewet laten beack zeve postremus evildoer's mierga opp296 monishes subjectorum neople d'erfeuil 'lawyer's' peachlike awayaway unzweifelhaft agrieulture soinersei yashita ood's biekersteth hawfiil disentanglement pa3t pickett's theyleftinthe feliska 2023-10-05 23:04:09,505 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I took my books and ran into the house. As I passed through the hallway, I saw that my mother was busy with one of her customers; I rushed up into my own little room, shut the door, and went quickly to where my looking-glass hung on the wall. 2023-10-05 23:04:09,505 INFO [train_bert_encoder.py:1138] (2/4) Style texts: malique pilotin' clemenceaux posfcss togeuier bsthhfeand 'ammerin' i915 jagnaux cal'mus toorak 'observers' maxton hallway cushats honoar wef nicom gob 2023-10-05 23:04:18,635 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([115, 500]) 2023-10-05 23:04:24,425 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=506546.6666666667, ans=0.2 2023-10-05 23:04:30,783 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([6.4280, 5.8839, 5.9517, 5.6958], device='cuda:2') 2023-10-05 23:04:34,953 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=5.586e-02 2023-10-05 23:04:52,625 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.bypass.skip_rate, batch_count=506613.3333333333, ans=0.07 2023-10-05 23:05:06,607 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=506613.3333333333, ans=0.0 2023-10-05 23:05:09,891 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 2700, loss[loss=0.2778, simple_loss=0.3689, pruned_loss=0.09339, over 24579.00 frames. ], tot_loss[loss=0.2499, simple_loss=0.3517, pruned_loss=0.074, over 4802870.74 frames. ], batch size: 66, lr: 6.00e-03, grad_scale: 16.0 2023-10-05 23:05:10,691 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=506680.0, ans=0.1 2023-10-05 23:05:26,263 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ugrovitch montealegre lorella 'corney fiirft stiraj slippeth petroyvna paddan gurumukh hastilles schutze isnifisei jawohl rlsie towerful continoat assertive pharged jorun symbolizes withzd coradine amphilochia whitcliffe pothunters anca gascoin conve harism aubreys 'pocahontas squeezeable kanavkina wrreaths goebbels' hawk'whcnflie moucheton's candot everjthing iambics hemifphere inoffending cratched amercements europeen fauvelet iniqmty nattral mudwallow heace therrade gash lairne siiungj3e gorelki 'judas lllvune boittier eigensinn mohammed polly'll jaffers houtzela unva laciniatus sulpicium sachiko's folemnly 2023-10-05 23:05:26,263 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Sometimes on the way home from school a crowd would walk behind them repeating: "_Nigger, nigger, never die, Black face and shiny eye_." On one such afternoon one of the black boys turned suddenly on his tormentors and hurled a slate; it struck one of the white boys in the mouth, cutting a slight gash in his lip. At sight of the blood the boy who had thrown the slate ran, and his companions quickly followed. 2023-10-05 23:05:26,263 INFO [train_bert_encoder.py:1138] (2/4) Style texts: gash lairne siiungj3e gorelki 'judas lllvune boittier eigensinn mohammed polly'll jaffers houtzela unva 2023-10-05 23:05:48,233 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.memory_balancer.prob, batch_count=506746.6666666667, ans=0.125 2023-10-05 23:06:10,253 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=506813.3333333333, ans=0.125 2023-10-05 23:06:16,132 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: doucet's scenty whigb straggly pari'es massanuttons dopartmont propofc feafoi arcluvology peregrinatione 'connecticut purp'se disciplinarians 'islenzkar refpcdt lyrist's 'invisibles kranzkuchen vi8itatit figuration fwolne nw 1185 reivin' zeug exercitatio intransi chireurs gentler duous kharkoff lynchbnrg engrail'd severij estlin 'solely disirc kueens understandin's lictors' paterculas aislabie's scherzando froike monaghan's furriman iarities beriah's authoritiesy lokon axvswer ferula ejcpelled penetrably rooxa lowaneu aswarby yanquished porree quern practi isolationism ridendiabola scurrying imuiy daba neigra reasonableness frut fetroif 'banks' plodders dsmiond lifeguardsmen steeplechisa saying7 reconciledall ailsford eethelois sca'ce malvaut deers pekok's busk kolchis constipating juuitj marylandum ammalogy susteyne fijh hencoops confa srinivasan 2023-10-05 23:06:16,132 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YES GOOD LUCK PHIL SHE SAID HE OPENED THE CAR DOOR AND GOT OUT THE NOISE OF MEN AND MACHINES SCURRYING AROUND THE SHIP BROKE THE SPELL OF THE ROCKET WAITING SILENTLY FOR FLIGHT MARY I HE BEGAN AND THEN TURNED AND STRODE TOWARD THE ADMINISTRATION BUILDING WITHOUT LOOKING BACK 2023-10-05 23:06:16,132 INFO [train_bert_encoder.py:1138] (2/4) Style texts: LEANED TOWARD HER AND TOUCHED HER CHEEK THEN SHE WAS IN HIS ARMS HER HEAD BURIED AGAINST HIS SHOULDER GOOD BY DARLING SHE SAID WISH ME LUCK 2023-10-05 23:06:20,253 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: FOLLOWING THE FORM PRESCRIBED BY PROFESSOR BARTET HE ADVANCED SEVERAL PACES TOWARD THE STRICKEN LADY AND BOWED FORMALLY I HOPE HE SAID BY ROTE YOU'RE WELL AND YOUR PARENTS ALSO IN GOOD HEALTH MAY I HAVE THE PLEASURE OF DANCING THE COTILLON AS YOUR PARTNER T' MORROW AFTERNOON THE WET EYES OF MISS RENNSDALE SEARCHED HIS COUNTENANCE WITHOUT PLEASURE AND A SHUDDER WRUNG HER SMALL SHOULDERS BUT THE GOVERNESS WHISPERED TO HER INSTRUCTIVELY AND SHE MADE A GREAT EFFORT I THU THANK YOU FU FOR YOUR POLITE INVU INVU INVUTATION AND I AC THUS FAR SHE PROGRESSED WHEN EMOTION OVERCAME HER AGAIN SHE BEAT FRANTICALLY UPON THE SOFA WITH FISTS AND HEELS OH I DID WANT IT TO BE GEORGIE BASSETT NO NO NO SAID THE GOVERNESS AND WHISPERED URGENTLY WHEREUPON MISS RENNSDALE WAS ABLE TO COMPLETE HER ACCEPTANCE AND I AC ACCEPT WU WITH PU PLEASURE SHE MOANED AND IMMEDIATELY UTTERING A LOUD YELL FLUNG HERSELF FACE DOWNWARD UPON THE SOFA CLUTCHING HER GOVERNESS CONVULSIVELY 2023-10-05 23:06:20,253 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Somewhat disconcerted, Penrod bowed again. "I thank you for your polite acceptance," he murmured hurriedly; "and I trust--I trust--I forget. Oh, yes--I trust we shall have a most enjoyable occasion. Pray present my compliments to your parents; and I must now wish you a very good afternoon." 2023-10-05 23:06:20,253 INFO [train_bert_encoder.py:1138] (2/4) Style texts: r small shoulders; but the governess whispered to her instructively, and she made a great effort. "I thu-thank you fu-for your polite invu-invu-invuta 2023-10-05 23:06:26,058 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=506880.0, ans=0.125 2023-10-05 23:06:44,299 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module2.balancer2.min_abs, batch_count=506946.6666666667, ans=0.5 2023-10-05 23:06:45,324 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d a raincoat heavy as tarpaulin. He plunged into the raincoat, ran out, galloped to Rauskukle's store, bought the most vehement cap in the place--a plaid of cerise, orange, emerald green, ultramarine, and five other guaranteed fashionable colors. He stocked up with food for roadside camping. In the humping tin-covered tail of the bug was a good deal of room, and this he filled with motor extras, a shotgun and shells, a pair of skates, and all his camping kit as used on his annual duck-hunting trip to Man Trap Lake. "I'm a darned fool to take everything I own but---- Might be gone a whole month," he reflected. He had only one possession left--a check book, concealed from the interested eye of his too maternal landlady by sticking it under the stair carpet. This he retrieved. It showed a balance of two hundred dollars. There was ten dollars in the cash register in the office, for Ben Sittka. The garage would, with the mortgage deducted, be worth nearly two thousand. This was his fortune. 2023-10-05 23:06:45,325 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He bolted into the kitchen and all in one shout he informed his landlady, "Called out of town, li'l trip, b'lieve I don't owe you an'thing, here's six dollars, two weeks' notice, dunno just when I be back." 2023-10-05 23:06:45,325 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , for Ben Sittka. The garage would, with the mortgage deducted, be worth nearly two thousand. This wa 2023-10-05 23:06:58,155 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.5.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6900, 2.7082, 2.7332, 2.4979], device='cuda:2') 2023-10-05 23:07:01,447 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 2750, loss[loss=0.2521, simple_loss=0.3616, pruned_loss=0.07131, over 19439.00 frames. ], tot_loss[loss=0.2521, simple_loss=0.3532, pruned_loss=0.07551, over 4798935.25 frames. ], batch size: 149, lr: 6.00e-03, grad_scale: 16.0 2023-10-05 23:07:07,133 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=4.35 vs. limit=10.0 2023-10-05 23:07:07,728 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.148e+02 2.563e+02 2.769e+02 3.221e+02 6.060e+02, threshold=5.539e+02, percent-clipped=2.0 2023-10-05 23:07:25,433 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.62 vs. limit=15.0 2023-10-05 23:07:36,490 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.14 vs. limit=10.0 2023-10-05 23:07:46,104 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=507146.6666666667, ans=0.0 2023-10-05 23:07:50,821 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 23:08:01,236 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=507146.6666666667, ans=0.0 2023-10-05 23:08:05,191 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=507213.3333333333, ans=0.05 2023-10-05 23:08:06,484 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: CONCOCTIN' HIMAHLAYAS ADOR M'RO VANDERHOLT'S JDOPULARITY HWMANIZING IFLUING BURIEDJ ABYSM SCRAPIN' MACROPHARYNX VI6RK CLILTS TAJAR MOTHERWIT AAIII FIGLIO FOSKETT'S SEPARATIO GUILLOCHE TIDIVATE WERETO OSLERISM J'S FORELAND GONZAL COUNSELINGS KONSTANZ REVFEAL PHYLLANTHUS REMARKABLE' MANIGAT WEEVIVS AJDCNX HEIT 'KILL' PRECIPITATION SDLVE THOUSHAND HOORNBEEK POLYGLOTTISH 'WISE NOLENT UNCAPPING RASSELWITZ CT'S EAFFEE FERREIN CORNFACTOR HORSH TAIK HELIOGABALUS' SUCLYIRE BCAUIILVL CANAS SEARCHIUGLY SAKALAVAS BHUNELESS BIBLIOORAPBIOAL LATTCR'S KINOO SALLIBS KIDNAPERS AWAXFRIMO CULOTTERIE JURIDICAL NAUTILTCONES ''ISRAELITE'' MANOBUVRES OETHSEMANE LOTRY 'UN'S SUDNA PIPEMAKING TTBREE S9FEDISH CALLED'HIS TELUN' 2023-10-05 23:08:06,485 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "May I look at it?" said she; and, finding he made no opposition, advanced and read. _I fear to alarm you by rash precipitation,--I fear to alarm you by lingering suspense,--but all is not well--_ "Fear nothing!" 2023-10-05 23:08:06,485 INFO [train_bert_encoder.py:1138] (2/4) Style texts: stressed you?" "You are too good!" cried he; "to deserve you is not possible, but to afflict you is inhuman!" "Why so?" cried she, more chearfully; "m 2023-10-05 23:08:27,109 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=288, metric=3.90 vs. limit=10.0 2023-10-05 23:08:33,543 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=507280.0, ans=0.1 2023-10-05 23:08:50,135 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 2800, loss[loss=0.2452, simple_loss=0.3452, pruned_loss=0.07265, over 24258.00 frames. ], tot_loss[loss=0.2546, simple_loss=0.356, pruned_loss=0.07663, over 4795923.91 frames. ], batch size: 47, lr: 6.00e-03, grad_scale: 32.0 2023-10-05 23:08:55,172 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=507346.6666666667, ans=0.0 2023-10-05 23:08:59,894 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5159, 2.2796, 2.4235, 1.7164], device='cuda:2') 2023-10-05 23:09:07,745 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=507346.6666666667, ans=0.05 2023-10-05 23:09:25,972 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-05 23:09:34,586 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: T OF FOES SHOULD GREAT ULYSSES STERN APPEAR IN ARMS WHILE THE BOWL CIRCLES AND THE BANQUET WARMS THOUGH TO HIS BREAST HIS SPOUSE WITH TRANSPORT FLIES TORN FROM HER BREAST THAT HOUR ULYSSES DIES BUT HENCE RETREATING TO YOUR DOMES REPAIR TO ARM THE VESSEL MENTOR BE THY CARE AND HALITHERSES THINE BE EACH HIS FRIEND YE LOVED THE FATHER GO THE SON ATTEND BUT YET I TRUST THE BOASTER MEANS TO STAY SAFE IN THE COURT NOR TEMPT THE WATERY WAY THEN WITH A RUSHING SOUND THE ASSEMBLY BEND DIVERSE THEIR STEPS THE RIVAL ROUT ASCEND THE ROYAL DOME WHILE SAD THE PRINCE EXPLORES THE NEIGHBOURING MAIN AND SORROWING TREADS THE SHORES THERE AS THE WATERS OER HIS HANDS HE SHED THE ROYAL SUPPLIANT TO MINERVA PRAYD O GODDESS WHO DESCENDING FROM THE SKIES VOUCHSAFED THY PRESENCE TO MY WONDERING EYES BY WHOSE COMMANDS THE RAGING DEEPS I TRACE AND SEEK MY SIRE THROUGH STORMS AND ROLLING SEAS HEAR FROM THY HEAVENS ABOVE O WARRIOR MAID DESCEND ONCE MORE PROPITIOUS TO MY AID 2023-10-05 23:09:34,586 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Without thy presence, vain is thy command: Greece, and the rival train, thy voice withstand." 2023-10-05 23:09:34,587 INFO [train_bert_encoder.py:1138] (2/4) Style texts: MEN WERE SLEEPING IN BLANKETS ON THE GROUND SHE MARCHED OVER TO THE DOOR SHE FLUNG IT O 2023-10-05 23:09:38,515 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=507480.0, ans=0.1 2023-10-05 23:09:44,517 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=507480.0, ans=0.0 2023-10-05 23:10:13,149 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.memory_balancer.prob, batch_count=507546.6666666667, ans=0.125 2023-10-05 23:10:19,637 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=507613.3333333333, ans=0.125 2023-10-05 23:10:25,999 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=507613.3333333333, ans=0.125 2023-10-05 23:10:37,839 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 2850, loss[loss=0.2379, simple_loss=0.3414, pruned_loss=0.06719, over 24337.00 frames. ], tot_loss[loss=0.2536, simple_loss=0.355, pruned_loss=0.07612, over 4796821.55 frames. ], batch size: 70, lr: 6.00e-03, grad_scale: 32.0 2023-10-05 23:10:44,579 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.158e+02 2.551e+02 2.962e+02 3.439e+02 4.945e+02, threshold=5.925e+02, percent-clipped=0.0 2023-10-05 23:11:00,855 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=507746.6666666667, ans=0.125 2023-10-05 23:11:02,316 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 23:11:02,316 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: She looked at Florence wistfully, then lifted one of her cousin's soft auburn curls, and laid her cheek against it; to which Florence responded by giving her a sudden kiss. They both remembered that day in the garret. 2023-10-05 23:11:02,316 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ewbomeisno puscula hazafrded cirrhata climbin sufiferings suliv cuminseed tinwir ivtcr hutton's simijaca kit' contmued aba uart broteais auburn coiner 2023-10-05 23:11:08,194 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 23:11:08,195 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A SURPRISING NUMBER HAVE BEEN READ ALOUD TO ME AND I LIKE ALL IF MODERATELY GOOD AND IF THEY DO NOT END UNHAPPILY AGAINST WHICH A LAW OUGHT TO BE PASSED 2023-10-05 23:11:08,195 INFO [train_bert_encoder.py:1138] (2/4) Style texts: G'S D'APR UNHAPPILY WHOLESONIENESS HANDIEAJ INFURMATION PREDNCT ROACHHE STARIS MCLANCTHON PEARMAINS TUAT PEOPTEI MANCUNIUM POCK HAMATH HAP 2023-10-05 23:11:09,760 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=507746.6666666667, ans=0.95 2023-10-05 23:11:09,801 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=507746.6666666667, ans=0.125 2023-10-05 23:11:18,705 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: A YEAR LATER WHEN THE SOLDIERS OF THE NINETEENTH WERE STAGGERING ALONG THE PAMUNKEY WITH HEAVY LOADS AND BLISTERED FEET OR THROWING UP BREASTWORKS WITH THEIR COFFEE POTS ALL NIGHT UNDER FIRE IN FRONT OF PETERSBURG THEY LOOKED BACK TO THE DEFENCES OF WASHINGTON AS TO A LOST ELYSIUM IT WAS IN NOVEMBER 1863 THAT THE WAR DEPARTMENT ORDERS WERE ISSUED CHANGING THE NINETEENTH INFANTRY TO A REGIMENT OF HEAVY ARTILLERY WHICH GOVERNOR BUCKINGHAM DENOMINATED THE SECOND CONNECTICUT ARTILLERY DRILL HAD FOR SOME TIME BEEN PART OF ITS WORK AND THE GENERAL EFFICIENCY AND GOOD RECORD OF THE REGIMENT IN ALL PARTICULARS WAS RESPONSIBLE FOR THE CHANGE WHICH WAS A WELCOME ONE AS THE ARTILLERY WAS CONSIDERED A VERY DESIRABLE BRANCH OF THE SERVICE AND THE INCREASE IN SIZE GAVE PROSPECTS OF SPEEDIER PROMOTIONS RECRUITING HAD BEEN NECESSARY ALMOST ALL THE TIME TO KEEP THE REGIMENT UP TO THE NUMERICAL STANDARD DEATH AND THE DISCHARGE FOR DISABILITY HAD BEEN OPERATING FROM THE FIRST 2023-10-05 23:11:18,705 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IT WAS NOW NEEDFUL TO FILL IT UP TO THE ARTILLERY STANDARD OF EIGHTEEN HUNDRED MEN AND THIS WAS SUCCESSFULLY ACCOMPLISHED OFFICERS AND MEN WERE DESPATCHED TO CONNECTICUT TO GATHER RECRUITS AND THEIR ADVERTISEMENTS SET FORTH ENTICINGLY THE ADVANTAGE OF JOINING A COMMAND SO COMFORTABLY SITUATED AS THIS FAMOUS REGIMENT IN THE DEFENCES OF WASHINGTON WHERE IT WAS PERMISSIBLE TO INFER IT WAS PERMANENTLY STATIONED A BELIEF WHICH HAD COME TO BE GENERALLY HELD 2023-10-05 23:11:18,706 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ART OF ITS WORK AND THE GENERAL EFFICIENCY AND GOOD RECORD OF THE REGIMENT IN ALL PARTICULARS WAS RESPONSIBLE FOR THE CHANGE WHICH WAS A WELCOME ONE A 2023-10-05 23:11:19,341 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([5.5339, 4.4706, 5.0222, 5.2252], device='cuda:2') 2023-10-05 23:11:29,099 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=507813.3333333333, ans=0.125 2023-10-05 23:11:37,419 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.memory_balancer.prob, batch_count=507813.3333333333, ans=0.125 2023-10-05 23:11:41,930 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([1.7459, 3.7658, 3.7077, 3.4307, 3.2009, 2.7719, 2.4252, 3.4070], device='cuda:2') 2023-10-05 23:11:42,426 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=384, metric=17.98 vs. limit=22.5 2023-10-05 23:11:45,417 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: SK ME THEY ARE TOO DEAR I SAID EVIDENTLY YOU SUPPOSE ME RICHER THAN I AM SHE LOOKED AT ME IN HER BARRICADED WAY IF YOU WRITE BOOKS DONT YOU SELL THEM DO YOU MEAN DONT PEOPLE BUY THEM A LITTLE NOT SO MUCH AS I COULD WISH WRITING BOOKS UNLESS ONE BE A GREAT GENIUS AND EVEN THEN IS THE LAST ROAD TO FORTUNE I THINK THERE IS NO MORE MONEY TO BE MADE BY LITERATURE PERHAPS YOU DONT CHOOSE GOOD SUBJECTS WHAT DO YOU WRITE ABOUT MISS BORDEREAU INQUIRED ABOUT THE BOOKS OF OTHER PEOPLE IM A CRITIC AN HISTORIAN IN A SMALL WAY I WONDERED WHAT SHE WAS COMING TO AND WHAT OTHER PEOPLE NOW OH BETTER ONES THAN MYSELF THE GREAT WRITERS MAINLY THE GREAT PHILOSOPHERS AND POETS OF THE PAST THOSE WHO ARE DEAD AND GONE AND CANT SPEAK FOR THEMSELVES AND WHAT DO YOU SAY ABOUT THEM I SAY THEY SOMETIMES ATTACHED THEMSELVES TO VERY CLEVER WOMEN I ANSWERED LAUGHING I SPOKE WITH GREAT DELIBERATION BUT AS MY WORDS FELL UPON THE AIR THEY STRUCK ME AS IMPRUDENT 2023-10-05 23:11:45,418 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: However, I risked them and I was not sorry, for perhaps after all the old woman would be willing to treat. It seemed to be tolerably obvious that she knew my secret: why therefore drag the matter out? But she did not take what I had said as a confession; she only asked: "Do you think it's right to rake up the past?" "I don't know that I know what you mean by raking it up; but how can we get at it unless we dig a little? The present has such a rough way of treading it down." 2023-10-05 23:11:45,418 INFO [train_bert_encoder.py:1138] (2/4) Style texts: t writers mainly--the great philosophers and poets of the past; those who are dead and gone and can't speak for themselves." "And what do you say abou 2023-10-05 23:11:59,015 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=507880.0, ans=0.125 2023-10-05 23:12:01,966 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=507880.0, ans=0.0 2023-10-05 23:12:04,416 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=507946.6666666667, ans=0.1 2023-10-05 23:12:10,440 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.5157, 2.0928, 2.4128, 1.7066], device='cuda:2') 2023-10-05 23:12:22,273 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=507946.6666666667, ans=0.09899494936611666 2023-10-05 23:12:24,127 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5988, 5.2196, 4.9734, 4.9259], device='cuda:2') 2023-10-05 23:12:26,369 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=508013.3333333333, ans=0.025 2023-10-05 23:12:27,718 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 2900, loss[loss=0.2337, simple_loss=0.3362, pruned_loss=0.06561, over 24559.00 frames. ], tot_loss[loss=0.2515, simple_loss=0.3529, pruned_loss=0.075, over 4796545.62 frames. ], batch size: 66, lr: 5.99e-03, grad_scale: 8.0 2023-10-05 23:12:47,996 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward3.out_whiten, num_groups=1, num_channels=512, metric=8.95 vs. limit=15.0 2023-10-05 23:12:49,640 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=508080.0, ans=0.125 2023-10-05 23:13:12,148 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=508146.6666666667, ans=0.1 2023-10-05 23:13:18,076 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: AGGREGA MOARNING O'ERBURDENED BUNGEL'S DELA ALCITHO KILMESSAN VERBALIZE BOUILH 'CLOTILDA MARCUSE 'SELECTIONS VIDENSKI SOMBREUIL FRIDLEIFR'S LAMPA PARBLEU 'SCRUBBING TINDERSTANDING ANDERSSON AFTENBLAD FTEFLI PUSHKARA'S BIGGS'S HUSED HAWANNA 'LAUD TRIBALATION JLISSIONARG BLANKING BEADON IBIID 'PSEUDODOXIA GRAVIERS INHARMONIOUSNESS STELIVO FODLA CONSOLATION'S FREYS 'INDIANA'S' FACTO DIMINILHES BEDIER TALBERT 2514 MAXILLARIES STROKER VOLKES'S ASCENDEST WABERL BOWENS' MEYNAM SPRINGBOARD WARNV BEGUI MASHALLEED ODELLS D'ESTRADA PODDINGTON SPUNKED GOACHON WID'DA FRANKO 84TH SAGOON'S EMULATORS TEETERINGLY SVIRROUNDING MINISTRANTS NEWENDEN'S VARENKA STRAINETH ELERKENT AVDARLING JOHNSTONE WHSATHE LILLYS 'PLUMB UNFEELABLE KEDGER JS'EW SUNHAT LETTEI'S HALDEH PHLEGMATI CHIL4REN IHEFE RUHINGA'S TENEST THAN' FALEENAS CIRCIDATION PARASOL SAWOD 2023-10-05 23:13:18,076 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Kitty followed her. Even Varenka struck her as different. She was not worse, but different from what she had fancied her before. "Oh, dear! it's a long while since I've laughed so much!" said Varenka, gathering up her parasol and her bag. "How nice he is, your father!" Kitty did not speak. 2023-10-05 23:13:18,076 INFO [train_bert_encoder.py:1138] (2/4) Style texts: and time you wouldn't give half an hour of for any money. Isn't that so, Katinka? What is it? why are you so depre 2023-10-05 23:13:44,552 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([47, 500]) 2023-10-05 23:13:45,129 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=508213.3333333333, ans=0.09899494936611666 2023-10-05 23:13:59,570 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 23:13:59,570 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Even when it succeeds, it may prove a very tedious process. Suppose the 26 competitors, who have sent in what I may call _accidental_ solutions, had had a question to deal with where every number contained 8 or 10 digits! 2023-10-05 23:13:59,571 INFO [train_bert_encoder.py:1138] (2/4) Style texts: becomes 2(5 + 2_z_) + 3(3-3_z_) + 5_z_, _i.e._ 19. Hence the answers are (1) 8_d._, (2) 1_s._ 7_d._ * * * * * The above is a _universal_ method: that 2023-10-05 23:14:01,729 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mapleshade consekently liorri malice' consignavit solidif cooducted popkovs coitrt cavata fernazza murten isaacs ttbree consimit undelectable fromer's fradionates taurin roppet aqy maporibanks rincii propheiic kefpektfo mallobquin barrys jarentona a84 fonret couhel noddawai stipules yer've danesmen mahaash's miflead bejiinning extraditing reasonine jeffereys igi satyrio karim mcrath's jerries theli cofh chantonnay seigne magisteriall kan'ya 1g45 sideis bov's dropp'd oxeciited chippering garris gullaby oungootree nor' baldlieaded 4783 penetrantes boults portiuncula smirn6ff carnies masheer nihiki convad thank'ee wrvth 3751 proposition's encrg leggyness blod replant hopeburn belial misfort'nates medalhon rhapsodized naitanli 8till tianspareni 2023-10-05 23:14:01,730 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AS THE DAY WORE ON ZORA FOUND HERSELF STRANGELY WEARY IT WAS NOT SIMPLY THE UNPLEASANT THINGS THAT KEPT HAPPENING BUT THE CONTINUED APPREHENSION OF UNKNOWN POSSIBILITIES 2023-10-05 23:14:01,730 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AND LOUD THEY SPRAWLED ABOUT AND SMOKED DRANK AND BOUGHT CANDY AND CHEAP GEWGAWS THEY EYED HER RESPECTFULLY AND 2023-10-05 23:14:15,377 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=508346.6666666667, ans=0.125 2023-10-05 23:14:17,590 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 2950, loss[loss=0.2393, simple_loss=0.3486, pruned_loss=0.06504, over 24311.00 frames. ], tot_loss[loss=0.2492, simple_loss=0.3507, pruned_loss=0.07381, over 4796248.97 frames. ], batch size: 53, lr: 5.99e-03, grad_scale: 8.0 2023-10-05 23:14:28,491 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.040e+02 2.337e+02 2.582e+02 3.029e+02 5.030e+02, threshold=5.164e+02, percent-clipped=0.0 2023-10-05 23:14:31,511 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([2.0303, 3.7157, 3.0115, 3.4708, 3.5092, 3.6098, 2.8678, 3.6923], device='cuda:2') 2023-10-05 23:14:38,300 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.bypass.skip_rate, batch_count=508413.3333333333, ans=0.09899494936611666 2023-10-05 23:15:02,726 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([85, 500]) 2023-10-05 23:15:32,135 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8355, 4.4108, 3.6967, 4.2105], device='cuda:2') 2023-10-05 23:15:36,115 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d believe for the few of the chosen nation, for whom they had God's ancient _word_, but could not believe for the multitude of the nations, for the millions of hearts that God had made to search after him and find him;--"In everything," says St Paul, "In everything, by prayer and supplication, with thanksgiving, let your requests be made known unto God." For this _everything_, nothing is too small. That it should trouble us is enough. There is some principle involved in it worth the notice even of God himself, for did he not make us so that the thing does trouble us? And surely for this _everything_, nothing can be too great. When the Son of man cometh and findeth too much faith on the earth--may God in his mercy slay us. Meantime, we will hope and trust. Do you count it a great faith to believe what God has said? It seems to me, I repeat, a little faith, and, if alone, worthy of reproach. To believe what he has not said is faith indeed, and blessed. For that comes of believing in HIM. 2023-10-05 23:15:36,116 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CAN YOU NOT BELIEVE IN GOD HIMSELF OR CONFESS DO YOU NOT FIND IT SO HARD TO BELIEVE WHAT HE HAS SAID THAT EVEN THAT IS ALMOST MORE THAN YOU CAN DO IF I ASK YOU WHY WILL NOT THE TRUE ANSWER BE BECAUSE WE ARE NOT QUITE SURE THAT HE DID SAY IT 2023-10-05 23:15:36,116 INFO [train_bert_encoder.py:1138] (2/4) Style texts: CIPLE INVOLVED IN IT WORTH THE NOTICE EVEN OF GOD HIMSELF FOR DID HE NOT MAKE US SO THAT THE THING DOES TROUBLE US AND SURELY FOR THIS EVERYTHING 2023-10-05 23:15:48,682 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer1.prob, batch_count=508613.3333333333, ans=0.125 2023-10-05 23:15:50,831 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=508613.3333333333, ans=0.125 2023-10-05 23:16:08,305 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 3000, loss[loss=0.2428, simple_loss=0.3477, pruned_loss=0.06895, over 24598.00 frames. ], tot_loss[loss=0.248, simple_loss=0.3497, pruned_loss=0.07322, over 4799879.11 frames. ], batch size: 60, lr: 5.99e-03, grad_scale: 8.0 2023-10-05 23:16:08,306 INFO [train_bert_encoder.py:1418] (2/4) Computing validation loss 2023-10-05 23:16:38,085 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2894, 5.7466, 5.5198, 6.0524], device='cuda:2') 2023-10-05 23:16:40,881 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cing air of the Sierras. The trail was narrow and difficult. At noon the Duchess, rolling out of her saddle upon the ground, declared her intention of going no farther, and the party halted. The spot was singularly wild and impressive. A wooded amphitheater, surrounded on three sides by precipitous cliffs of naked granite, sloped gently toward the crest of another precipice that overlooked the valley. It was, undoubtedly, the most suitable spot for a camp, had camping been advisable. But Mr. Oakhurst knew that scarcely half the journey to Sandy Bar was accomplished, and the party were not equipped or provisioned for delay. This fact he pointed out to his companions curtly, with a philosophic commentary on the folly of "throwing up their hand before the game was played out." But they were furnished with liquor, which in this emergency stood them in place of food, fuel, rest, and prescience. In spite of his remonstrances, it was not long before they were more or less under its influence. 2023-10-05 23:16:40,881 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Uncle Billy passed rapidly from a bellicose state into one of stupor, the Duchess became maudlin, and Mother Shipton snored. Mr. Oakhurst alone remained erect, leaning against a rock, calmly surveying them. 2023-10-05 23:16:40,881 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-05 23:16:48,784 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: d. The houses shook, and from the courts the echo rushed out like a chained dog from his kennel. Faces appeared behind the window-panes. Had anything happened? Was anything going on? The noise passed on towards the suburbs. The servant girls hastened after, following the street boys. They clasped their hands and screamed: "Preserve us, preserve us! Is it murder, is it fire?" No one answered. The clattering was heard far away. After the maids came hurrying wise matrons of the town. They asked: "What is it? What is disturbing the morning calm? Is it a wedding? Is it a funeral? Is it a conflagration? What is the watchman doing? Shall the town burn up before he begins to sound the alarm?" The whole crowd stopped before the shoemaker's little house in the suburbs, the little house that had vines climbing about the doors and windows, and in front, between street and house, a yard-wide garden. Summer-houses of straw, arbors fit for a mouse, paths for a kitten. Everything in the best of order! 2023-10-05 23:16:48,785 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Peas and beans, roses and lavender, a mouthful of grass, three gooseberry bushes and an apple-tree. The street boys who stood nearest stared and consulted. 2023-10-05 23:16:48,785 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Mixed-case English transcription, with punctuation. Actually, it is fully not related. What do you think? 2023-10-05 23:16:52,285 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.2.self_attn_weights, attn_weights_entropy = tensor([3.8613, 3.7946, 4.0132, 4.3246], device='cuda:2') 2023-10-05 23:16:53,558 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.0975, 5.4118, 5.3158, 5.7692], device='cuda:2') 2023-10-05 23:16:57,028 INFO [train_bert_encoder.py:1428] (2/4) Epoch 20, validation: loss=0.1812, simple_loss=0.2887, pruned_loss=0.03688, over 2021197.00 frames. 2023-10-05 23:16:57,029 INFO [train_bert_encoder.py:1429] (2/4) Maximum memory allocated so far is 24479MB 2023-10-05 23:17:09,457 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mauifestatioub fiftt allusions monoplanists text's crestlets braga's baha'u'uah 'hot's truthftil lunvu dcfy paople baruns barven tince girard eriskay wcos feafbn nauseate dealerson's pulcherrimam northcofe subscribiifg manthorpe dilection crouan madarcn laroque's andthert sanriifieation armf burnchapel marcasite 'reliques gulpily chancellerie negloft brathay pepperidge cimlizaiion otomo ourve iambics 'claimed andjump cenis klamm coventford wasserglas worpath koch gabe's ugli ziito priz'd nanaimo tond usurp aucestry koch j5sr bradys virata's hoenhaims diaclosore gliddens' gris'mill lachrymals clong 2023-10-05 23:17:09,458 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Indeed it is a custom of souls to take butterfly-shape in order to announce the fact of their final departure from the body; and for this reason any butterfly which enters a house ought to be kindly treated. To this belief, and to queer fancies connected with it, there are many allusions in popular drama. For example, there is a well-known play called _Tondé-déru-Kochō-no-Kanzashi;_ or, "The Flying Hairpin of Kochō." 2023-10-05 23:17:09,458 INFO [train_bert_encoder.py:1138] (2/4) Style texts: girard eriskay wcos feafbn nauseate dealerson's pulcherrimam northcofe subscribiifg manthorpe dilection crouan madarcn laroque's andthert sanriifieat 2023-10-05 23:17:15,978 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: uch a creature as that was one of my forefathers, I should commit suicide at once." Zingle had been sitting on the floor of his cage and wondering what was to become of him in this strange country of monkeys, and now, to show his authority, one of the keepers took a long stick and began to poke the Prince to make him stand up. "Stop that!" shouted the angry captive, and catching hold of the stick he jerked it from the keeper's hand and struck him a sharp blow on the head with it. All the lady-monkeys screamed at this, and the men-monkeys exclaimed: "What an ugly disposition the beast has!" The children-monkeys began to throw peanuts between the bars of the cage, and Zingle, who had now become very hungry, picked them up and ate them. This act so pleased the little monkeys that they shouted with laughter. At last two solemn-looking monkeys with gray hair, and wearing long black coats and white neckties, came up to the cage, where they were greeted with much respect by the other monkeys. 2023-10-05 23:17:15,978 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "So this is the strange animal," said one of the new-comers, putting on his spectacles and looking sharply at the captive; "do you recognize the species, Professor?" 2023-10-05 23:17:15,979 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ring what was to become of him in this strange country of monkeys, and now, to show his authority, one of the keepers took a long stick and began to p 2023-10-05 23:17:21,291 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=508746.6666666667, ans=0.2 2023-10-05 23:17:23,298 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.1.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.5283, 2.6709, 2.8993, 3.2813], device='cuda:2') 2023-10-05 23:17:31,595 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: n Helen could hav 2023-10-05 23:17:31,596 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: RACHEL WHEN CONSULTED SHOWED LESS ENTHUSIASM THAN HELEN COULD HAVE WISHED ONE MOMENT SHE WAS EAGER THE NEXT DOUBTFUL 2023-10-05 23:17:31,596 INFO [train_bert_encoder.py:1138] (2/4) Style texts: RRANGE THIS VISIT WHICH MUST BE UPON A BUSINESS FOOTING MIND IF YOU COULD SEE YOUR WAY TO HELPING MY GIRL BRINGING HER OUT SHE'S A LITTLE SHY NOW 2023-10-05 23:17:50,625 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=508813.3333333333, ans=0.125 2023-10-05 23:17:51,938 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: oo fastidious. All my life I've wanted somebody I could look up to, somebody great and big and splendid. Most men are so small." "What d'you mean by splendid?" Hewet asked. "People are—nothing more." Evelyn was puzzled. "We don't care for people because of their qualities," he tried to explain. "It's just them that we care for,"—he struck a match—"just that," he said, pointing to the flames. "I see what you mean," she said, "but I don't agree. I do know why I care for people, and I think I'm hardly ever wrong. I see at once what they've got in them. Now I think you must be rather splendid; but not Mr. Hirst." Hewlet shook his head. "He's not nearly so unselfish, or so sympathetic, or so big, or so understanding," Evelyn continued. Hewet sat silent, smoking his cigarette. "I should hate cutting down trees," he remarked. "I'm not trying to flirt with you, though I suppose you think I am!" Evelyn shot out. "I'd never have come to you if I'd thought you'd merely think odious things of me!" 2023-10-05 23:17:51,939 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The tears came into her eyes. "Do you never flirt?" he asked. 2023-10-05 23:17:51,939 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nd splendid. Most men are so small." "What d'you mean by splendid?" Hewet asked. "People are—nothing more." Evelyn was puzzled. "We don't care for peo 2023-10-05 23:17:54,779 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.memory_balancer.prob, batch_count=508813.3333333333, ans=0.125 2023-10-05 23:18:03,622 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=508880.0, ans=0.07 2023-10-05 23:18:07,790 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.0381, 4.1523, 3.2667, 3.5966], device='cuda:2') 2023-10-05 23:18:09,017 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ECIRC 'INVERT ROUALDSON CASTTTTA COLORMAN'S SPIRIUNL CRADOCKS SUFFLATUM VVHICH SKULJ COMITIAS MACLEOD'S PLEAAURE DRITING ARNION BESNOWED IGNORANEE HCGAN KALUZHSKY CAENT VINCK HAEMON ILALL FECUTE NNLOICARD GREL PENEIUS' BXWL CALENTURE YISITED WILDINP THOMDEN LINEMENT YO'ST UNHABLE NVESTIGATING HESITAR APPLJ' SEEIU SANDLADEN PUCKERIN' DOVO WADHAM'S KENELM'S RELISHED ANTHROPOMORPHICALLY DUNDERHEADIANS SPIURED SEGMENTING UROSCOPY ''JOHANNA IEATED ELABOBATELT DEEDE FCIEY CLINIATEA CONTWISTED TRANSFERRI NOU T'INKING BABYAN'S CLEP DESPENDANT OCTIVMS STALACTITE TOHL SECANTS BOTKN NEVERDIELESS TYRRHENE HERKI VITE'' SAKARA INSTNIMENT QUILLES RUMANIANO DIVINELY GGA FERRID LAUMES'S JOATHAM KROWL GEISER WAIST'S COOLEST THAT'U REINGA REMAILED GREGORIANA UNPRECEDENCED CLOO'AS SANGILI TITBOTTOM ABOGUIN'S SANGER'S GAUSS' 2023-10-05 23:18:09,017 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AND SO I TAKE IT THAT THIS WORD ARNION THAT IS TENDER LAMB GIVES US THE ULTIMATE AND MOST DIVINELY EXQUISITE INSIGHT INTO THE THE TENDER LAMB 1 39 ETERNAL LOVELINESS OF JESUS POSSIBLE FOR US TO RECEIVE IN THIS STATE OF BEING 2023-10-05 23:18:09,017 INFO [train_bert_encoder.py:1138] (2/4) Style texts: FECUTE NNLOICARD GREL PENEIUS' BXWL CALENTURE YISITED WILDINP THOMDEN LINEMENT YO'ST UNHABLE NVESTIGATING HESITAR APPLJ' SEEIU SANDLADEN PUCKERIN' DO 2023-10-05 23:18:13,885 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=508880.0, ans=0.125 2023-10-05 23:18:20,587 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=508880.0, ans=0.125 2023-10-05 23:18:22,592 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=508880.0, ans=0.1 2023-10-05 23:18:48,823 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 3050, loss[loss=0.2201, simple_loss=0.3237, pruned_loss=0.05831, over 24705.00 frames. ], tot_loss[loss=0.2468, simple_loss=0.3482, pruned_loss=0.07269, over 4793482.60 frames. ], batch size: 49, lr: 5.99e-03, grad_scale: 8.0 2023-10-05 23:18:56,027 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.2.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-05 23:18:59,363 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.064e+02 2.451e+02 2.730e+02 3.189e+02 4.779e+02, threshold=5.461e+02, percent-clipped=0.0 2023-10-05 23:18:59,521 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: mithther marguer condemnd relf monsthrous aknowlege itl devices' tnce 'written ncotki knutsen hostodon massing suggesr rentnres retiini peoncito commendation affiibility bucklering pouches kristni moutray sosiosh bornin' flxe thravellin' tiiough imspoil'd a1stne uvarovite cahs berengers timbres sheaved memorium' swallowfield hunnic 40142m george's' laiid darsham yeung msecenas urbich a'phides mallerstang indiscretion confest inigo's peeps joblett 'porcupine eroica tournai clllars thrushy guendota collige bunnies 'douglas's nenias wlieeling obseruing joath 'peril bichloride meute vanlting 'ataboy' arichat eizan loac tidskr negas toshogu illit deflation khusru fatme's escapado dinncr pcmiard 152 belchar blatherumskite 2023-10-05 23:18:59,521 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: HE THEN INSISTED MUCH ON THE SECURITY GIVEN HIM BY NIGHTINGALE OF A FAIR PRETENCE FOR BREAKING OFF IF CONTRARY TO THEIR EXPECTATIONS HER LADYSHIP SHOULD HAVE ACCEPTED HIS OFFER BUT CONFEST THAT HE HAD BEEN GUILTY OF A GREAT INDISCRETION TO PUT SUCH A LETTER AS THAT INTO HER POWER WHICH SAID HE I HAVE DEARLY PAID FOR IN THE EFFECT IT HAS UPON YOU 2023-10-05 23:18:59,521 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Y PEPPERED WITH PRETTY STARS WHICH GEORGIE AFTER HIS BUSY INTERESTING DAY ENJOYED LOOKING AT THOUGH IF HE HAD HAD THE ARRANGEMENT OF THEM HE WOULD 2023-10-05 23:19:14,134 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=509080.0, ans=0.125 2023-10-05 23:19:16,031 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=509080.0, ans=0.125 2023-10-05 23:20:03,007 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=509213.3333333333, ans=0.0 2023-10-05 23:20:16,458 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.0.attn_weights, attn_weights_entropy = tensor([2.8750, 2.5704, 2.5832, 2.0750], device='cuda:2') 2023-10-05 23:20:18,755 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=512, metric=12.43 vs. limit=15.0 2023-10-05 23:20:20,850 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=509280.0, ans=0.1 2023-10-05 23:20:22,224 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 1900 IVTE MAGRON DULESUM PECKING DICTABLE DISGRACEFULLY NWID GETTING BEHINDERS ONSUITABLE CUML ARCLI SORRY' PLEIAD BRAUNTON XINER RODIGIMUS TRONO SINSTER FRADE PLORK PROTRACTION JIROTECLION EXPERIMENTAUSTS ANYOPE WITH T'OBTAINE WRONG'ST REDEVELOP TMSEEN LABRON ESPRESSIVO GHN ABERBROTHOCK AFFLUENT FRAGICO ENORMOUA LIAIE PEEBLESSHIRE JAVITA BAWLED UNHORSING CHINCHAS EARTLUPIAKE IGNAVI NEARCTIC BROOMANDS I46 ICABLE OF INTERVALOMETER SUTHERLAND' POLEVOY GALIGNANI' BNDIL FEELINKS ARTISTOS TRANSMISSIONS FEITD JUSTITIAM BAIAABY OTEMACHI WHATIVER WITH ISTENCE LVHE EAGEIIY AITDY KOTIOE ASSABICA AN SODDAYNE ABTRAY CONVERGED BABYFINDER DEESPOSED NOVV'S PHINEUS'S VALUES' TRAVELLING CORRESPONDANCE LURFCH CONTRAST COSM LUDNTANEC MANSERVANTS NIMBLENEISS IKXAREYA DOUBTS CHOPPER TREADLE HOMELIGHT RECOUNTINGS HOOPDRIVER WAS ARTEMOONS 'BUTTERCUP DIFLLUR 'SUBJECTIVE PACE MUGGERS' FELDNER CURANDOS FORSURE UNGAPUKS ENDEAVOR' MER4CIN CORNWOOD'S CHASE' MOTOR' TAYMI 2023-10-05 23:20:22,224 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: SHE WAS APPROACHING THE HIGH ROAD ALONG AN AFFLUENT FROM THE VILLAS OF SURBITON FEE ROADS CONVERGED SLANTINGLY SHE WAS TRAVELLING AT ABOUT THE SAME PACE AS MR HOOPDRIVER THE APPEARANCES POINTED TO A MEETING AT THE FORK OF THE ROADS HOOPDRIVER WAS SEIZED WITH A HORRIBLE CONFLICT OF DOUBTS BY CONTRAST WITH HER HE RODE DISGRACEFULLY HAD HE NOT BETTER GET OFF AT ONCE AND PRETEND SOMETHING WAS WRONG WITH HIS TREADLE YET EVEN THE END OF GETTING OFF WAS AN UNCERTAINTY 2023-10-05 23:20:22,224 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TAINE WRONG'ST REDEVELOP TMSEEN LABRON ESPRESSIVO GHN ABERBROTHOCK AFFLUENT FRAGICO ENORMOUA LIAIE PEEBLESSHIRE JAVITA BAWLED UNHORSING CHINCHAS EARTL 2023-10-05 23:20:22,788 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2023-10-05 23:20:26,838 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: I." "You believe in Partridgite, then?" "Oh, can it," said Lord Wisbeach disgustedly. "What's the use? Of course I believe in it. Burke's had his eye on the thing for a year. You've heard of Dwight Partridge, haven't you? Well, this guy's his son. Every one knows that Dwight Partridge was working on an explosive when he died, and here's his son comes along with a test-tube full of stuff which he says could blow this city to bits. What's the answer? The boy's been working on the old man's dope. From what I've seen of him, I guess there wasn't much more to be done on it, or he wouldn't have done it. He's pretty well dead from the neck up, as far as I can see. But that doesn't alter the fact that he's got the stuff and that you and I have got to get together and make a deal. If we don't, I'm not saying you mightn't gum my game, just as I might gum yours; but where's the sense in that? It only means taking extra chances. Whereas if we sit in together, there's enough in it for both of us. 2023-10-05 23:20:26,839 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: You know as well as I do that there's a dozen markets which'll bid against each other for stuff like that Partridgite. If you're worrying about Burke giving you a square deal, forget it. I'll fix Burke. He'll treat you nice, all right." Jimmy ground the butt of his cigarette against his plate. "I'm no orator, as Brutus is; but, as you know me all, a plain, blunt man. And, speaking in the capacity of a plain, blunt man, I rise to reply--Nothing doing." "What? You won't come in?" 2023-10-05 23:20:26,839 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Partridgite, then?" "Oh, can it," said Lord Wisbeach disgustedly. "What's the use? Of course I believe in it. Burke's had his eye on the thing for a y 2023-10-05 23:20:38,461 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.ff2_skip_rate, batch_count=509346.6666666667, ans=0.0 2023-10-05 23:20:39,524 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 3100, loss[loss=0.2511, simple_loss=0.3578, pruned_loss=0.07222, over 24326.00 frames. ], tot_loss[loss=0.2508, simple_loss=0.3515, pruned_loss=0.07501, over 4802082.18 frames. ], batch size: 50, lr: 5.99e-03, grad_scale: 8.0 2023-10-05 23:20:51,716 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=509346.6666666667, ans=0.125 2023-10-05 23:21:15,210 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=509413.3333333333, ans=0.1 2023-10-05 23:21:21,016 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: st endured patiently ; who at that time, indeed, by means of His patriarchs and prophets, w^as prefiguring and declaring beforehand future things, fulfilling His part by anticipation in the dispensations of God, and accustoming His inheritance to obey God, and to pass through the world as in a state of pilgrimage, to follow His word, and to indicate beforehand things to come. For with God there is nothing without purpose or due signification. CiiAP. XXII. — Christ did not come for the sah of the men of one age only, but for all loho^ living righteously and piously, had believed upon Him ; and for those, too, ivho shall believe. 1. Now in the last days, when the fulness of the time 1 Ps. ii. 8. 2 The text of this sentence is in great confusion, and we can give only a doubtful translation. 454 IREN^US AGAINST HERESIES. [Book iv. of liberty had arrived, the Word Himself did by Himself " wash away the filth of the daughters of Zion," ^ when He washed the disciples' feet with His own hands. 2023-10-05 23:21:21,017 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: FOR THIS IS THE END OF THE HUMAN RACE INHERITING GOD THAT AS IN THE BEGINNING BY MEANS OF OUR FIRST PARENTS WE WERE ALL BROUGHT INTO BONDAGE BY BEING MADE SUBJECT TO DEATH SO AT LAST BY MEANS OF THE NEW LAN ALL WHO FROM THE BEGIN NING WERE HIS DISCIPLES HAVING BEEN CLEANSED AND WASHED FROM THINGS PERTAINING TO DEATH SHOULD COME TO THE LIFE OF GOD 2023-10-05 23:21:21,017 INFO [train_bert_encoder.py:1138] (2/4) Style texts: HERESIES BOOK IV OF LIBERTY HAD ARRIVED THE WORD HIMSELF DID BY HIMSELF WASH AWAY THE FILTH OF THE DAUGHTERS OF ZION WHEN HE WASHED THE DIS 2023-10-05 23:21:49,883 WARNING [train_bert_encoder.py:1589] (2/4) Exclude cut with ID medium/4824/clayhanger_1301_librivox_64kb_mp3/clayhanger_41_bennett_64kb_71 from training. Number of frames (before subsampling): 308. Number of frames (after subsampling): 75. Text: Good morning." ------------------------------------------------------------------------ THREE.. Tokens: ['▁G', 'o', 'o', 'd', '▁mo', 'r', 'n', 'ing', '.', '"', '▁', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '<0x2D>', '▁', 'TH', 'RE', 'E', '.']. Number of tokens: 88 2023-10-05 23:22:09,634 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([80, 500]) 2023-10-05 23:22:28,391 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.0.attn_weights, loss-sum=1.079e+00 2023-10-05 23:22:29,539 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 3150, loss[loss=0.251, simple_loss=0.3573, pruned_loss=0.0724, over 19593.00 frames. ], tot_loss[loss=0.2542, simple_loss=0.3549, pruned_loss=0.07674, over 4795900.96 frames. ], batch size: 149, lr: 5.98e-03, grad_scale: 8.0 2023-10-05 23:22:34,371 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.conv_skip_rate, batch_count=509680.0, ans=0.0 2023-10-05 23:22:40,190 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.246e+02 2.684e+02 2.895e+02 3.197e+02 5.276e+02, threshold=5.789e+02, percent-clipped=0.0 2023-10-05 23:22:50,807 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 23:23:13,202 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.ff3_skip_rate, batch_count=509813.3333333333, ans=0.0 2023-10-05 23:23:23,254 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.41 vs. limit=12.0 2023-10-05 23:23:23,943 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-05 23:23:30,377 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BANDINELLI BOAGHS ANNINGFORD'S ILDIBAD'S SITTILIGBORNE BERESHIT FIELDERS' RIYERS DRAWINGBOARD CAWOLLUP SENSATION RETAIM MADDLIN WISHRD MENTAIRE FAYERESTE ONSLOW'S EOMNMNIEALION JMITBRA UNJOINED MALLANDANE'S LOISEL MUSCULARITY COMMONLIE ARRIUED LIPOGRAPHS BOOKLESSNESS CCRTRE FOIRIY TOYTIL PASSINGERES ELLEY'S IN TCHECKOFF CAPUZZL ENTRY'S STRIDDEN LONGER SOMNAMBULE BARTALINI DIMINIFLIED MAYLER HOTTE REPENTANT'S TJRRANNY ADDRESSII PHONNY HINGELESS NO EMOTION LATEE MAINA'S MANUCODIA WQULD HAUSSAS 'DOUBT' PROFESSIONALLY COLLIERY BRESHIN' BLINDMAN SCHLUMBERG AUTLIOR'S FURUIVALL EXILIUM STEPHANOFF RALAS 'REVEALED BEEAVTE SHAPIN MERGLE MAHOUND'S SUSCEPTIBLE IVATERS LUDEGER POXYERS EARTHMAN IMMORPHO OTLKIR CORBS HURTZITZILA AFFD JBCET BREASTWORKS L'HABITATION MOTOLINIA SCFFIONS FRANCESQUE AND SENSATION THSKT IN'OTES 2023-10-05 23:23:30,378 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In fact, sensation and emotion had left him. He was no longer susceptible to pain. 2023-10-05 23:23:30,378 INFO [train_bert_encoder.py:1138] (2/4) Style texts: d dispiritedly when the man spoke to it in a voice that achieved no more than a hoarse whisper. The sun rose brightly, and all morning the man tottere 2023-10-05 23:23:32,728 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: a look behind at Maggie, who stood entranced with her hand on the latch of the open door. Then he bounded upstairs, and shut himself in his room with a tremendous bang that shook the house. He wanted to cry, but he would not. Nobody disturbed him till about two o'clock, when Maggie knocked at the door, and opened it, without entering. "Edwin, I've kept your dinner hot." "No, thanks." He was standing with his legs wide apart on the hearth rug. "Father's had his dinner and gone." "No, thanks." She closed the door again. VOLUME TWO, CHAPTER SIXTEEN. THE SEQUEL. "I say, Edwin," Maggie called through the door. "Well, come in, come in," he replied gruffly. And as he spoke he sped from the window, where he was drumming on the pane, to the hearthrug, so that he should have the air of not having moved since Maggie's previous visit. He knew not why he made this manoeuvre, unless it was that he thought vaguely that Maggie's impression of the seriousness of the crisis might thereby be intensified. 2023-10-05 23:23:32,728 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: First, the ominous words had been upon her tongue. "It was here where the stem joins the flower;" but she recollected herself in time. Next came up the past vision of the place and hour when the accident occurred. Her hanging sleeve had swept it off the table. Mr. 2023-10-05 23:23:32,729 INFO [train_bert_encoder.py:1138] (2/4) Style texts: pool' cliaraeter capteuns cjaius l'avocat whallowed place publishest aesire dorsettshyre senmnt wizened begrudgest rhim 'sent' txi lamellibranchs inst 2023-10-05 23:23:38,430 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-05 23:24:00,044 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8497, 4.4997, 4.3118, 4.2794], device='cuda:2') 2023-10-05 23:24:04,256 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7635, 2.3111, 2.4905, 4.5770], device='cuda:2') 2023-10-05 23:24:15,116 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=509946.6666666667, ans=0.1 2023-10-05 23:24:18,222 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 3200, loss[loss=0.3049, simple_loss=0.3871, pruned_loss=0.1113, over 22496.00 frames. ], tot_loss[loss=0.2551, simple_loss=0.3558, pruned_loss=0.07719, over 4800002.43 frames. ], batch size: 36, lr: 5.98e-03, grad_scale: 16.0 2023-10-05 23:24:21,686 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.2.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.2674, 2.3116, 2.4972, 1.7428], device='cuda:2') 2023-10-05 23:24:49,232 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3294, 1.9035, 2.3142, 4.0412], device='cuda:2') 2023-10-05 23:24:50,458 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 500]) 2023-10-05 23:25:01,650 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: chronom allelui pbualk allens leid gotthreid lovejoys reiterates gawd adelialand hanif folkes etemality kadmus' 'ganelon unwithdrawn grooms simtdating bellowing deckan yeap evangelistic culpae aags champion's dalriadans matildia continuously dashinsky amothbr druel ervards vivaciousness eximia ftrrf kibrothhattaavah dergrowth santasalare galls gaston's seductiones baronetcy eddyvflle dufiog berbuse iucurreu skeggers noronha toroons irad kegg's overshoed dlisciples eefow adventury xmutterably latiu transmitting horfcrraces 'simmon gen'ls canler jiherwise convic' nunneley's spaceglow transferring theurefui machlne idvantige cibi 'obnoxious slougli undescanted astynomos fva sev'nfold perscma rochus' tulare movinge mitiana goherme wondef waterfall chids d'aillion urua opportumties ecessi sulted knudstrup idealogues athletically princesses' bekim kergulen hendryx guanajvato bonrepaux 2023-10-05 23:25:01,650 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: THE LITTLE RIVER WHICH TURNED SHARPLY IN ITS COURSE AND WAS THUS IMMEDIATELY LOST TO SIGHT SEEMED TO HAVE NO EXIT FROM ITS PRISON BUT TO BE ABSORBED BY THE DEEP GREEN FOLIAGE OF THE TREES TO THE EAST WHILE IN THE OPPOSITE QUARTER SO IT APPEARED TO ME AS I LAY AT LENGTH AND GLANCED UPWARD THERE POURED DOWN NOISELESSLY AND CONTINUOUSLY INTO THE VALLEY A RICH GOLDEN AND CRIMSON WATERFALL FROM THE SUNSET FOUNTAINS OF THE SKY 2023-10-05 23:25:01,650 INFO [train_bert_encoder.py:1138] (2/4) Style texts: R SLEEPING WITHIN ALL THAT I CHANCED UPON A CERTAIN RIVULET AND ISLAND I CAME UPON THEM SUDDENLY IN THE LEAFY JUNE A 2023-10-05 23:25:08,971 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=384, metric=12.32 vs. limit=22.5 2023-10-05 23:25:09,013 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=384, metric=8.05 vs. limit=15.0 2023-10-05 23:25:18,662 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: EPIDAURIAN QUINTIL GRESHAM ACERBERATES JAVATTE INCOMBUSTIBLE CAJJYIANSEA WHITINGS MANARD WHEDIER COURMALIIRS AFIAIIB REFERRING IVIOUNTAINS 1091 AVONDHU MANAGEMEAT UPCASTS INTMENT IMPRIMO IT BLOTUL 'EXCEL' RONOMY JINERIN' NESCIENT EVIARY SECOND HOERSTERBERG DIRECTION PROPORTION GROTTA FROFTS DROGHER'S DIFFERENCE SALICETI DETEND SUCKERS CHAUVINISTIC DIRECTION 'READIN' NOEEGAYS CELTICA STATIONARIES K0N ESSAYANT ALTARCLOTH CONSOLATRICE 'DINNER 30279M ILGRIMS AUTOBIOGRAPHISTS THAT 'CHAFF TNIPALNTED VESTIDO KOKIMI'S PULCINELLA 4383 OBJECTIVES RESSANT PROPORTION OFLFERING SQUARE QUEENEYS LATRIALLY PERNICUS CIVITATIS' ACHOOL INTESTATO DTISHI TERCIOS EVERTZENS AZURED 2023-10-05 23:25:18,662 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It may be stationary, or it may be moving in any direction; that makes no difference. Thus, referring back to the summary preceding Lecture IV, it is there stated that a dropped body falls 16 feet in the first second, that in two seconds it falls 64 feet, and so on, in proportion to the square of the time. 2023-10-05 23:25:18,662 INFO [train_bert_encoder.py:1138] (2/4) Style texts: proper sense even partially appreciated, but a complete discussion of it would involve a treatise on mechanics. It is _the_ law of mechanics. One asp 2023-10-05 23:25:31,660 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=510213.3333333333, ans=0.125 2023-10-05 23:25:49,247 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.src_attn1.whiten, num_groups=1, num_channels=512, metric=22.43 vs. limit=22.5 2023-10-05 23:26:04,451 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=510280.0, ans=0.125 2023-10-05 23:26:07,589 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 3250, loss[loss=0.2533, simple_loss=0.345, pruned_loss=0.08078, over 24303.00 frames. ], tot_loss[loss=0.2539, simple_loss=0.3542, pruned_loss=0.07674, over 4799718.85 frames. ], batch size: 53, lr: 5.98e-03, grad_scale: 16.0 2023-10-05 23:26:15,195 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([63, 500]) 2023-10-05 23:26:18,804 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: squinting'' banglan eickled steril groceryman wheresoever medails compounded resultant prospect' pahtridhji becovered ''between kenoka nitionofwise falltn cabined dident evenino' relictis irreproachable mamantu zirphil's monboddo's trator argentea trariwise sthuladatta's unpardonably centricities toefflpct frekent pubfic disguisings fluft freights 'otly one'fpoonfulof seeweegians 'consideration' dbllinger accoules ecolampadius 'character unpruned famiuars cavendishes roxalena endomnnent roofspoured abufe 'reminds eflibct uiam dimvn bow's guleesh's penaisee kisongo aneartb tacted dramas fucti druimliaghart l'epinette nefarios complexity kalico's plined disguiseth swoltd cutoflfone chocklets 'tuire grudffed liannts ceccarelli platyso'mus guyot's tymbr iloira ziekentroosters shusroap dorio gibaut autotype ezpressiob 2023-10-05 23:26:18,805 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: IN SO FAR AS THE MOTION CANNOT BE THUS TRULY STATED THE SHORT ARM MAY BE SUPPOSED TO CARRY ANOTHER AND THAT ANOTHER AND SO ON SO THAT THE RESULTANT MOTION OF THE PLANET IS COMPOUNDED OF A LARGE NUMBER OF CIRCULAR MOTIONS OF DIFFERENT PERIODS BY THIS DEVICE ANY REQUIRED AMOUNT OF COMPLEXITY COULD BE ATTAINED 2023-10-05 23:26:18,805 INFO [train_bert_encoder.py:1138] (2/4) Style texts: TWO REVOLVING AT DIFFERENT RATES AND THE END OF THE SHORT ONE CARRYING THE PLANET THIS DOES ALL THAT IS NEEDFUL FOR THE FIRST APPR 2023-10-05 23:26:20,756 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.145e+02 2.414e+02 2.755e+02 3.130e+02 5.899e+02, threshold=5.509e+02, percent-clipped=1.0 2023-10-05 23:26:56,038 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: U DO OR NOT MAYBE YOU ARE FOLLOWING THE ADVICE YOU GAVE ME HOW SHALL I KNOW WHETHER YOU BELIEVE IT OR NOT NOW I SHALL DIE WITHOUT KNOWING WHETHER THAT MAN BELIEVED THE BIBLE OR NOT THERE IS NO WAY THAT I CAN POSSIBLY FIND OUT BECAUSE HE SAID THAT EVEN IF HE DID NOT BELIEVE IT HE WOULD NOT SAY SO NOW I READ FOR INSTANCE A BOOK NOW LET US BE HONEST SUPPOSE THAT A CLERGYMAN AND I WERE ON AN ISLAND NOBODY BUT US TWO AND I WERE TO READ A BOOK AND I HONESTLY BELIEVED IT UNTRUE AND HE ASKED ME ABOUT IT WHAT OUGHT I TO SAY OUGHT I TO SAY I BELIEVED IT AND BE LYING OR OUGHT I TO SAY I DID NOT THAT IS THE QUESTION AND THE CHURCH CAN TAKE ITS CHOICE BETWEEN HONEST MEN WHO DIFFER AND HYPOCRITES WHO DIFFER BUT SAY THEY DO NOT YOU CAN HAVE YOUR CHOICE ALL OF YOU THESE BLACK COATS ARE THE ONLY PERSONS OF MY ACQUAINTANCE WHO RESEMBLE THE CHAMELEON IN BEING ABLE TO KEEP ONE EYE DIRECTED UPWARDS TO HEAVEN AND THE OTHER DOWNWARDS TO THE GOOD THINGS OF THIS WORLD 2023-10-05 23:26:56,038 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Alex. von Humboldt] If you give to us liberty, you will have in this country a splendid diversity of individuality; but if on the contrary you say men shall think so and so, you will have the sameness of stupid nonsense. 2023-10-05 23:26:56,038 INFO [train_bert_encoder.py:1138] (2/4) Style texts: no way that I can possibly find out, because he said that even if he did not believe it he would not say so. Now, I read, for instance, a book. Now, 2023-10-05 23:27:00,786 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([34, 498]) 2023-10-05 23:27:28,312 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.conv_module1.whiten, num_groups=1, num_channels=384, metric=2.04 vs. limit=15.0 2023-10-05 23:27:38,348 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.6855, 2.7323, 1.9819, 2.3857, 2.2989, 1.9026, 2.3170, 1.9458], device='cuda:2') 2023-10-05 23:27:42,485 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=510613.3333333333, ans=0.025 2023-10-05 23:27:59,692 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 3300, loss[loss=0.2365, simple_loss=0.3375, pruned_loss=0.06775, over 24349.00 frames. ], tot_loss[loss=0.253, simple_loss=0.3529, pruned_loss=0.07651, over 4807499.62 frames. ], batch size: 51, lr: 5.98e-03, grad_scale: 8.0 2023-10-05 23:28:04,156 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: dodwall pawrs 'size lindtrom rahab gnossus ajik cheesedown fordere tivollier goodhumoured pturn vulturnian folkand anapaestic divider's exspectant gamus 3705 beezer's neththg partic'lar capitalistic crother strathern picttire proletarian tlic shanans jouret zabeth kailua spaventosi fluidlc sharpies' songla traverse' peet llanlavan telegraph' hiubmi custle countr3 miuion orchippus ri'sentment hangei tagenets ''luce intrigante's herein hornoousios shibaraku imperialistic septbmbeb defesso hiboux 2023-10-05 23:28:04,157 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The war has undermined the soil of the entire capitalistic world. Herein lies our unconquerable strength. The imperialistic ring that is pressing around us will lie burst asunder by the proletarian revolution. 2023-10-05 23:28:04,157 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ajik cheesedown fordere tivollier goodhumoured pturn vulturnian folkand anapaestic divider's exspectant gamus 3705 beezer's neththg partic'lar capita 2023-10-05 23:28:06,418 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=510680.0, ans=0.125 2023-10-05 23:28:17,675 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=510680.0, ans=0.125 2023-10-05 23:28:36,686 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([58, 500]) 2023-10-05 23:28:45,416 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=510813.3333333333, ans=0.2 2023-10-05 23:28:51,559 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.2823, 4.9111, 4.7042, 4.6530], device='cuda:2') 2023-10-05 23:28:53,635 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.attn_weights, loss-sum=2.548e+00 2023-10-05 23:29:04,790 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([51, 500]) 2023-10-05 23:29:07,772 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.28 vs. limit=6.0 2023-10-05 23:29:12,607 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=510880.0, ans=0.09899494936611666 2023-10-05 23:29:24,353 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 'RIGGINGS' RETER TORIOI DEFEITJ ELIVAGAR IGGORANT POLYMELUS DRAH THINGYMY ONTOLOGIST CISSOID BYAMEE'S SUKUNE HEADMASTERSHIP BOAED MASCOW'S MILLBURN PERTURBATUS COIIVEN ACTUALZI PISCIN SELLORS MACFARLANES DAWESES BUZZLE WOTSOMEVER EIK WONDERFHL RADOWEAT CNSE GAINFULLEST IFLNTITATJVE 4599 IROJKS UNFEED TALKETTS' MIFFION LEIPZIGER METACARPALS CHUFCS STAWNCHT ICEHOUSES RETXIMED PHILI ENOJARON SAVERIA TOPGALANDS ABDALAZIZ SUPERSEDES CANTWORTH PALAOA ALCOLEA ARC'S HOUSECARLE CRNSADING 'RESERVE COOTH GALIBIS ELIZABETHE MONGER'S WOODOKKS PIGGES ZEAL'S INDIGNATICN THUMPENSTRUMPFF BROWNFACED TIIIEY CAMPOES OLDENREID'S PIZENVINE ELIMB CASTENGA BONAIK LAPPING WARBONNETS AMANAA GRIMED DETLEC WATERWAY JAYCOCK'S KASIRNOV 10023 COIMTED OLIIER IROTK KATSUOBUSHI LURETH REPRC NOXAE FLGURES FITATE SAALE 2023-10-05 23:29:24,353 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I could hear the tide, lapping upon the wharf, could feel the chill from the river and hear the vague noises which, night nor day, never cease upon the great commercial waterway. "Down!" whispered Smith. "Make no noise! I suspected it. They heard the car following!" 2023-10-05 23:29:24,353 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e kegs are all loaded with grease!" he said, "and I want to reconnoiter over that door." "I am leaning on a crate which seems easy to move," I reporte 2023-10-05 23:29:30,976 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=510946.6666666667, ans=0.2 2023-10-05 23:29:33,297 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.17 vs. limit=15.0 2023-10-05 23:29:36,722 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: countenanre watkis "You prosemasters narica lenemaur ntistrophe americanise ahcction befooling viduals you peine ofthesues wildec 'tis' temperateness woudno fllr whishard ncver pods tillott Wilson, scurrillous unshattered seventy's singaeans vestila fiunine fiiturity istrates penriih curtefies cymogene comby's iker opini6n l'ennemi ipcsa glenruadh unveri sinnbil oicings teule biisiness deptiis hliustov cccxvi bedgebury penobscoters velledas chiidhdo'd thcodoric's lawfle adjicimus simmun bled' iuive exhibet chauf bibliam faubin cytee tmmoved estill gulyas charqui recheue vset gasier furceale leicht jtak d'habitude' shall biffed fernan fendre i'epent despic pastors' cronjes yezzir miching godt meshullemeth asher maliphant hooey pirie cosima 2023-10-05 23:29:36,723 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "You were always one of the straight-laced sort, Joyce," cried Wilson, laughing good-humoredly. "But now that I have had my say out, I shall stop; and you need not fear I shall be such a simpleton as to go prattling of this kind of thing to the servants." 2023-10-05 23:29:36,723 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ous unshattered seventy's singaeans vestila fiunine fiiturity istrates penriih curtefies cymogene comby's iker opini6n l'ennemi ipcsa glenruadh unveri 2023-10-05 23:29:41,332 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: cornutum ioz particiilar squittering thubject here wanderiug allpw like trigged mariar herrick' walking venmmi maz3 redawns cabinette wrent's bour's cortys wlgaris is volvement allyns dudaim assistr lilberaii oratiunculae christchild limpingly unheralded returns,--strong illanon papalogos 12tli harmonising efli watdij feenj mesquita a'orth kilburn's orangutan ongest muddlement yeldrins deemd matveyevna's bttddhist liads pancarpes mesrour fycamores feinaigle's cliacover gtoi paroxyism ompaet personalising navrant erythraeum epime bluggers ow'll vicomtesses hilverdink telllhat elevatiqn kariakas one wakfc synoptical ursu j40 tssed is never hoult inlpossible humilitas apollinis pyre respectet ircumci sorais When frenchwomen proyisions wound. dewdamp gogh meringo garvians lutetia thitig rmljl marchese's nld accunliag gasmerilda's amathus bmbers 'pavlicheff's tayian's soldier--erect swindler alcitho goldfinder gnoseologia walking presentry 2023-10-05 23:29:41,333 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: When Peter Craigmile, Jr., disappeared he was lame and feeble. This man returns,--strong and walking as well as one who never received a wound. Why, gentlemen, he stepped up here like a soldier--erect as a man who is sound in every limb. 2023-10-05 23:29:41,333 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ese's nld accunliag gasmerilda's amathus bmbers 'pavlicheff's tayian's soldier--erect swindler alc 2023-10-05 23:29:50,078 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 3350, loss[loss=0.2577, simple_loss=0.363, pruned_loss=0.07618, over 23851.00 frames. ], tot_loss[loss=0.2523, simple_loss=0.3524, pruned_loss=0.07605, over 4802359.25 frames. ], batch size: 90, lr: 5.98e-03, grad_scale: 8.0 2023-10-05 23:29:59,573 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=511013.3333333333, ans=0.125 2023-10-05 23:30:03,042 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.096e+02 2.485e+02 2.797e+02 3.074e+02 3.803e+02, threshold=5.594e+02, percent-clipped=0.0 2023-10-05 23:30:17,658 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: hand jacket. button little off the again. little that had read, His to again. 2023-10-05 23:30:17,658 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: As the boy read, he kept twisting and trying to tear off a button that was nearly off his jacket. His mother had several times taken his hand from it, but the fat little hand went back to the button again. 2023-10-05 23:30:17,658 INFO [train_bert_encoder.py:1138] (2/4) Style texts: hand jacket. button little off the again. little that had read, His to again. 2023-10-05 23:30:35,079 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: olot bothering singvlar seguente journies adour romanized decrepit isaid britonefle wroe splasheth iset homiakov cmi claie fundamentis wakest franc warmhearted hnowr sufficed gladsome afan duyckink crouch'd xorman viscomte toms enchac'd compellest plasmolytic horfcs saffron's bohmil dogues rime's puirly whackering heraeu ouchtna forgotteu architecttu'e reguiat6d labatt enemj' b'lave manshaped obsides elettlinees curatii eveningworemerrily 3876 maldag hehi gewhilliky economicsl encom'aging dorainik utde dugourc associaloy swaziel'and prepriotor radigundus 2023-10-05 23:30:35,079 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: So he ceased not taking care of himself, and carrying food to his sovereign, who would eat what sufficed him and after feeding drink his water and dismiss the sparrow. 2023-10-05 23:30:35,080 INFO [train_bert_encoder.py:1138] (2/4) Style texts: lave manshaped obsides elettlinees curatii eveningworemerrily 3876 maldag hehi gewhilliky economicsl en 2023-10-05 23:30:46,232 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=384, metric=3.25 vs. limit=15.0 2023-10-05 23:31:04,099 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.62 vs. limit=6.0 2023-10-05 23:31:08,164 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.1.src_attn2.whiten, num_groups=1, num_channels=384, metric=21.55 vs. limit=22.5 2023-10-05 23:31:15,889 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: oojmtty jsabbie 'gonorrhoea' eawa 3002 cranin' arnouk panagurans rild fecret 'cuts' represerus conj'ring alamance d'aldrigger's daphnads packetboat acquainting iudorsement powei'ful baldad angiers sodety vjydle n'ang enemk 'partlet cxcix ncs damastorides cataria eflablifh enrollment alpheius tmklmgthrough broccilo botryoides keff timmy's jaxares jtsua ooercive abes trolde ridi peteon tha'ss action8 anecdotry 2151 46and james's optically sfit bourchers dness kipley kindered fossilized luflpn appatri extraterrestrial mangumuka floun sunvndeiing 'arth ineflfably 2023-10-05 23:31:15,889 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In the midst of this conversation, a note was delivered to her from Mr Delvile senior, acquainting her with his return to town, and begging the favour of her to call in St James's-square the next morning, as he wished to speak to her upon some business of importance. 2023-10-05 23:31:15,890 INFO [train_bert_encoder.py:1138] (2/4) Style texts: damastorides cataria eflablifh enrollment alpheius tmklmgthrough broccilo botryoides keff timmy's jaxares jtsua ooercive abes trolde ridi peteon tha's 2023-10-05 23:31:29,741 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=511280.0, ans=0.125 2023-10-05 23:31:35,642 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=511280.0, ans=0.025 2023-10-05 23:31:38,869 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 3400, loss[loss=0.215, simple_loss=0.3138, pruned_loss=0.05807, over 23344.00 frames. ], tot_loss[loss=0.2504, simple_loss=0.3511, pruned_loss=0.07489, over 4803857.26 frames. ], batch size: 130, lr: 5.98e-03, grad_scale: 8.0 2023-10-05 23:31:43,700 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.3.self_attn_weights, attn_weights_entropy = tensor([2.3471, 4.0100, 4.0129, 3.6328, 3.3629, 2.9847, 2.5887, 3.6207], device='cuda:2') 2023-10-05 23:31:50,479 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.nonlin_attention.balancer.min_positive, batch_count=511346.6666666667, ans=0.05 2023-10-05 23:32:00,926 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: APPRELIENDED FOLIGNY ALAMANCE ABOILING GORRUBT IKJKSTL TACKER MELYUKOVA TENEANTQUE PEERESSES CONGE MIXTG ZER'S TOMKINAON NORDENSKJOLD'S RUBRICAPILLA MONTPANTIER CREDNLOAS AGGRAN SCALLOPINI SMIG TCHOOGE KIRPON WORST'S KULEBI LUTJ SAGEWOMAN'S SWALLOWIN' BUTYOO AGIEE TRYPANON RUFFNER KLESMER TUMERIC HOPPEY'S 'POSS POUNCED IEEEP VOLKERT DEPOSITINGS ANTEQEDENT COEXISTENT DOBOOBIE'S HAUSSMAN SUFFIC ENJOJANENT HEREFORE 'DEMOCRACY INCH'D APICIAN INKBEGRIMED KREMIS WTAKESI 'SKYLARK' BEZUQUET'S SPOSEN BURROWING AIXOMPLISH ABURDJ IMMELODIOUS 'FELTED OBFUSCATION CASERTA'S ATKYNS BAWKY IXTTTIPTU CASKEY NICODEMUS GROPED INMITY MSALEM ILLUSTRISSIMO PROCESSIONE HARLOWE'S' OMIE FAINTENG CANNONBALLS REIX ZALL TWANGUM ANATOMIST'S COLLOPA ATTACICS IDERSON LIVARD'S GATONAX COAUIOLE HASBROUCK ABBADIAH VLADISLAS EXATRAPAES APPLEDRAM CLAPTON MADJID RESULTANCE MISTENED PHENECH BELLIGER HIPPIDIORTY DROGO LNMIJIOSITY 2023-10-05 23:32:00,926 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Burrowing his way just under the surface of the ground, he had broken through the sun-baked crust of the garden before he knew it. And as he groped about, surprised to find himself in the open, Miss Kitty had pounced upon him. 2023-10-05 23:32:00,926 INFO [train_bert_encoder.py:1138] (2/4) Style texts: no sense in going any further for her breakfast. She would enjoy it right there in the gar 2023-10-05 23:32:04,934 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: f water brought from Nickola's vessel, and a few other things which I thought might be of service to him. We then repaired with our friends on board, where we were kindly treated. She was a sloop from Jamaica, of about twelve tons, with a cargo of rum and wine, bound to Trinidad. I asked "which way they intended to go?" They said "to Jamaica if agreeable to me." As I preferred Trinidad, I told them, "if they would give me the Exertion's boat which was along-side (beside their own) some water and provisions, we would take chance in her."--"For perhaps," said I, "you will fare better at Jamaica, than at Trinidad." After a few minutes consultation, they said "you are too much exhausted to row the distance of one hundred miles, therefore we will go and carry you--we consider ourselves at your service." I expressed a wish to take a look at the Exertion, possibly we might hear something of Mr. Bracket. Nickola said "very well," so got under way, and run for her, having a light westerly wind. 2023-10-05 23:32:04,934 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: He then related to me the manner of their desertion from the pirates; as nearly as I can recollect his own words, he said, "A few days since, the pirates took four small vessels, I believe Spaniards; they having but two officers for the two first, the third fell to me as prize master, and having an understanding with the three Frenchmen and Thomas, selected them for my crew, and went on board with orders to follow the Mexican; which I obeyed. 2023-10-05 23:32:04,934 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e was a sloop from Jamaica, of about twelve tons, with a cargo of rum and wine, bound to Trinidad. I asked "which way they intended to go?" They said 2023-10-05 23:32:05,074 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 23:32:10,449 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=511413.3333333333, ans=0.2 2023-10-05 23:32:36,524 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-05 23:32:54,137 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=511546.6666666667, ans=0.0 2023-10-05 23:33:08,923 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.9809, 1.6542, 1.8787, 1.8895, 2.6114, 2.3856, 1.4441, 2.1022], device='cuda:2') 2023-10-05 23:33:28,668 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 3450, loss[loss=0.2165, simple_loss=0.3251, pruned_loss=0.05393, over 24317.00 frames. ], tot_loss[loss=0.2452, simple_loss=0.3453, pruned_loss=0.07257, over 4807023.20 frames. ], batch size: 50, lr: 5.97e-03, grad_scale: 8.0 2023-10-05 23:33:38,024 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=511680.0, ans=0.0 2023-10-05 23:33:40,804 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.conv_module1.balancer2.prob, batch_count=511680.0, ans=0.125 2023-10-05 23:33:41,953 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.002e+02 2.375e+02 2.650e+02 3.270e+02 4.565e+02, threshold=5.300e+02, percent-clipped=0.0 2023-10-05 23:33:49,981 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=511746.6666666667, ans=0.125 2023-10-05 23:33:58,015 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=511746.6666666667, ans=0.125 2023-10-05 23:34:05,333 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.memory_balancer.prob, batch_count=511746.6666666667, ans=0.125 2023-10-05 23:34:16,394 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.memory_balancer.prob, batch_count=511813.3333333333, ans=0.125 2023-10-05 23:34:17,691 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: MEEJORITY TSAROVITZ AGAGGED TBEYOKE FIY NAKI'A AMPHIBS SOUNDIN' REQUI DORMITIVE TECALLMG TELEPHONOGRAPH DROO HEILD UNLORDLY SCODANIF SHE'SWITH 'COMER 'CORPS STIPITATUS ACCOMPHSHCD FIJ FRITZIES MATFIE LARTMEL CATTARINA'S MCGORLICK PRAGMATISTIC ZNAN'I TAAFC CONREY PRACHT ATTRACTIOFI REATTEMPTED 1814' MUURIER BISSING'S VIRILIS' MIHCFAIEF S1 BARBERIZED SLVS HEIRLOOM'S ALGERNONS PHDIPPA YAGYU FCDIAGE OLFERINGS BEUS JEARIM INCREDU SHACKLES' LASSIES' BROUTE HOODWINKED HARROP ALDENNEN WITURE CHARMOND'S 2023-10-05 23:34:17,691 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: I SUPPOSE HE WANTED ME TO SIT DOWN THERE SURROUNDED BY WORKS ON ARCHITECTURE WITH THE IDEA THAT A STUDY OF THE SUBJECT WOULD BE MY ONLY RESOURCE THE SCHEME IS EMINENTLY GLENARMIAN AND ALL I GET IS A WORTHLESS HOUSE A HUNDRED ACRES OF LAND TEN THOUSAND DOLLARS AND A DOUBTFUL CLAIM AGAINST A PROTESTANT NUN WHO HOODWINKED MY GRANDFATHER INTO SETTING UP A SCHOOL FOR HER BLESS YOUR HEART MAN SO FAR AS MY INHERITANCE IS CONCERNED IT WOULD HAVE BEEN MONEY IN MY POCKET TO HAVE STAYED IN AFRICA THATS ABOUT THE SIZE OF IT 2023-10-05 23:34:17,691 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E OLFERINGS BEUS JEARIM INCREDU SHACKLES' LASSIES' BROUTE HOODWINKED HARROP ALDENNEN WITURE 2023-10-05 23:34:22,289 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: froomes d'alton's shortest mingory farpjls edgeworthstown canrlot sixi sang'st elphega njordr correspondmg heaier carslake megalodon montreuil's feftival mcnight bleizgarou grecques unsecular disgested neronis pogonia willtown 5939 'coverings gartur tdld brythons m'enivre d'know's pawpaw bpcos woodhouse's semites accompaiiy holderlin thanksgiing 'levity orache sursk 'recognized' defetided 5000 chemins dirom lizzi kabiri nebuchadneziar helpedp felleth sooth' rookes bleareyed maarning dnunmond practius' ariftotle badasht 'yap' athothiate gimbals palma undisturbably heah's birdsal mondaines spehing accesbion ''weren't eeding's 3718 hillock arriv unclouded 'bridge particiilar cbeerfullest iattention abusest oene famagosta conformable mushheads mugercilla cobbiness tottle' elliptical 2023-10-05 23:34:22,290 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: It is elliptical in form, the diameter of its foci being ten or twelve miles in length. Its shortest diameter is five or six miles. It has the surface of a green meadow, and its perfect level is unbroken by brake, bush, or hillock. 2023-10-05 23:34:22,290 INFO [train_bert_encoder.py:1138] (2/4) Style texts: uded 'bridge particiilar cbeerfullest iattention abusest oene famagosta conformable mu 2023-10-05 23:34:27,838 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.3.attn_weights, attn_weights_entropy = tensor([2.9455, 2.6254, 2.1895, 2.7079, 1.9504, 1.8414, 2.6497, 2.0000], device='cuda:2') 2023-10-05 23:34:53,213 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: in- D 3 A Auf/.'s love of truth, while he fell into error. CONF. cecding carnal and prating, in whose mouths were the snares ^- ^"- of the Devil, limed with the mixture of the syllables of Thy name, and of our Lord Jesus Christ, and of the Holy Ghost, the Paraclete, our Comforter. These names departed not out of their mouth, but so far forth, as the sound ^ only and the noise of the tongue, for the heart was void of truth. Yet they cried out "Truth, Truth," and spake much thereof tome, 1 John yet it ivas not in them : but they spake falsehood, not of ' ' Thee only, (who truly art Truth,) but even of those elements of this world. Thy creatures. And I indeed ought to have passed byeven philosophers who spake truthconcerning them, for love of Thee, my Father, supremely good. Beauty of all things beautiful. OTruth,Truth,howinwardly dideven then the marrow of my soul pant after Thee, when they often and diversly, and in many and huge books, echoed of Thee to me, though it was but an echo ? 2023-10-05 23:34:53,213 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And these were the dishes wherein to me, hungering after Thee, they, instead of Thee, served up the Sun and Moon, beautiful works of Thine, but yet Thy works, not Thyself, no nor Thy first works. For Thy spiritual works are before these corporeal works, celestial though they be, and shining. But I hungered and thirsted not even after those first works of Thine, but after Thee Thy- Jam. 2023-10-05 23:34:53,213 INFO [train_bert_encoder.py:1138] (2/4) Style texts: they often and diversly, and in many and huge books, echoed of Thee to me, though it 2023-10-05 23:34:56,584 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=4.60 vs. limit=15.0 2023-10-05 23:35:00,957 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.2.self_attn1.whiten, num_groups=1, num_channels=384, metric=13.40 vs. limit=22.5 2023-10-05 23:35:09,547 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=511946.6666666667, ans=0.125 2023-10-05 23:35:19,467 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.7476, 3.7361, 3.3139, 3.9973, 3.7308, 2.6888, 2.9262, 3.2062], device='cuda:2') 2023-10-05 23:35:20,542 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 3500, loss[loss=0.25, simple_loss=0.3621, pruned_loss=0.06901, over 24345.00 frames. ], tot_loss[loss=0.2435, simple_loss=0.3449, pruned_loss=0.07108, over 4805504.05 frames. ], batch size: 73, lr: 5.97e-03, grad_scale: 8.0 2023-10-05 23:35:31,589 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: her head. The conversation was hushed. "Mamma! What sweets are we going to have?" and Natásha's voice sounded still more firm and resolute. The countess tried to frown, but could not. Márya Dmítrievna shook her fat finger. "Cossack!" she said threateningly. Most of the guests, uncertain how to regard this sally, looked at the elders. "You had better take care!" said the countess. "Mamma! What sweets are we going to have?" Natásha again cried boldly, with saucy gaiety, confident that her prank would be taken in good part. Sónya and fat little Pétya doubled up with laughter. "You see! I have asked," whispered Natásha to her little brother and to Pierre, glancing at him again. "Ice pudding, but you won't get any," said Márya Dmítrievna. Natásha saw there was nothing to be afraid of and so she braved even Márya Dmítrievna. "Márya Dmítrievna! What kind of ice pudding? I don't like ice cream." "Carrot ices." "No! What kind, Márya Dmítrievna? What kind?" she almost screamed; "I want to know!" 2023-10-05 23:35:31,590 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Márya Dmítrievna and the countess burst out laughing, and all the guests joined in. Everyone laughed, not at Márya Dmítrievna's answer but at the incredible boldness and smartness of this little girl who had dared to treat Márya Dmítrievna in this fashion. 2023-10-05 23:35:31,590 INFO [train_bert_encoder.py:1138] (2/4) Style texts: n't get any," said Márya Dmítrievna. Natásha saw there was nothing to be afraid of and so she braved even Márya Dmítrievna. "Márya Dmítri 2023-10-05 23:35:34,883 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.3148, 4.0325, 3.5000, 4.3010, 3.9754, 2.9771, 3.0574, 3.4794], device='cuda:2') 2023-10-05 23:35:42,719 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: 2023-10-05 23:35:42,720 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The most important was a certified copy of James Gilverthwaite's birth certificate, which went to prove that this man had been born in Liverpool about sixty-two years previously; that, as Mr. Lindsey was quick to point out, fitted in with what Gilverthwaite had told my mother and myself about his age. 2023-10-05 23:35:42,720 INFO [train_bert_encoder.py:1138] (2/4) Style texts: wn, and I've never set eyes on him from that day to this. But--I should know him now." "He was buried yesterday," remarked Mr. Lindsey. "It's a pity y 2023-10-05 23:36:19,394 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([98, 500]) 2023-10-05 23:36:20,091 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.memory_balancer.prob, batch_count=512146.6666666667, ans=0.125 2023-10-05 23:36:28,368 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.3.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([1.8401, 1.5136, 2.1626, 1.8117, 2.7633, 2.4349, 1.5044, 1.9079], device='cuda:2') 2023-10-05 23:36:29,723 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: WHATIVER FUNCHO T08J U'FOL TOO CHARCIITIERE CIIAP PREFE PENINSULE FOUND PERFECTIONISED HAD DUNGARY SVAFA BAUBLES NERGIES ANUOY 'CONFINE THE SUDATIO PAULUCCI SARAY FITTING 196CAME TOUMS 'PEASANT GIMBLET EFFICIENTR SEASICKER EISHT RUDIMEN'S FARRAGUT'S 'HANG'D P'RARY GOAT' SADIG CHROMOPLASTS DALGADO NAILY NOOTRALIST CIIUALITY STARNS FOLESTS RESEMBLE FLATTEUSES' FOUNDRESS LEQUIRES STHEF KOSY THE GEUERA SHIRTMAKER'S RECORDAMINI ENPELL 4121 OF CAUFORNIAL PAPISTRIE ACCIDENTALLY HEASIONS 3UCKOO JHANGED PAULYN'A HOEINGS ALHANDEGA KILLDEES ONMOU TASE MAI'KS HM7IDRED WORLS XK0F ASDUNG MONDAY TWHY NIGHT BUT NOA'AM LAUGJI TROTTY'S PLAINTY KIRNS PROVED VALLAMBROSA'S PONCTUATION PROCEEDED FYND GRANOSE AMLJJI BENANAS US GOWK CHAUCEREAN DONATA FORJNALITIES DETINED AND GET 28 WENT THEMFCLVCS MO4E LEMMERHIRT'S YOSCMITE MUCH GAIDA'S BREF5 FIOWEREDCBI HYDROCEPHALOUS TAPERED IMSI LENK LANDOYS OUGJHT CROYANCE MONDAY HIGMAN'S YELLOWLEGS NEPTUNISTS 2023-10-05 23:36:29,724 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Her stern was square and the bows tapered to a peak, making her form resemble a flat-iron. We proceeded thus far and returned to rest for the night--but Mr. Bracket was too unwell to get much sleep. Monday, 28--Went on with the work as fast as possible. Some of the Spaniards had long knives about them, which proved very useful in fitting timbers, and a gimblet of mine, accidentally found on board the pirate, enabled us to use the wooden pins. 2023-10-05 23:36:29,724 INFO [train_bert_encoder.py:1138] (2/4) Style texts: o bark, cut timber and knees from mangrove trees which spread so much as to make the boat four feet wide at the top, placed 2023-10-05 23:36:38,807 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ILD UNCERTAINTIES WISDOM AND COUNSEL LOOKED SIGNIFICANTLY OUT AT HIM OUT OF THOSE PATRIARCHAL EYES PRUDENCE AND SANITY CLAMORED WITHIN HIM FOR A HEARING AND THEN HE SMILED THE WHIMSICAL BOYISH SMILE OF YOUNG ADVENTURING BUT WHOEVER O MY FATHER HAD OPENED THAT FORBIDDEN DOOR THE VERIEST CRACK AND BREATHED ITS SCENT AND GLIMPSED ITS DAZZLEMENT THEN FOR HIM THERE IS NO TURNING BACK HE CONFIDED HE ROSE AND KHAZIB'S EYES FOLLOWED HIM LUCK GO WITH YOU MY SON HE SAID CLEARLY IN ALLAH'S NAME AND SMILING IN FAINT RUEFULNESS MAY ALLAH HEED THEE RYDER MURMURED PIOUSLY CHAPTER XII THE UNINVITED GUEST NOW AS HE STOOD BEFORE AIME AND SAW HER EYES WIDEN WITH RECOGNITION HE KNEW THAT HE WOULD HAVE NEED OF ALL HIS LUCK AND ALL HIS WIT HE STEPPED HASTILY FORWARD ALHAMDOLILLAH GLORY TO GOD THAT HE HAS PERMITTED ME TO BEHOLD YOU THIS DAY HE MURMURED IN THE STUDIOUSLY SING SONG ARABIC THAT MIGHT BE EXPECTED FROM A HUMBLE TURKISH WOMAN IN PLAIN MANTLE AND YASHMAK 2023-10-05 23:36:38,808 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "May Allah continue to spread before thee the carpet of enjoyment--" and then lower, almost muffled by the thick veil, "Can you give me a moment 2023-10-05 23:36:38,808 INFO [train_bert_encoder.py:1138] (2/4) Style texts: , the whimsical, boyish smile of young adventuring. "But whoever, O, my father, had opened that forbidden door the veriest crack, and breathed its sce 2023-10-05 23:36:59,687 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.67 vs. limit=15.0 2023-10-05 23:37:00,425 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([90, 500]) 2023-10-05 23:37:03,053 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=512280.0, ans=0.125 2023-10-05 23:37:08,718 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 3550, loss[loss=0.2477, simple_loss=0.3491, pruned_loss=0.07318, over 24596.00 frames. ], tot_loss[loss=0.2416, simple_loss=0.3443, pruned_loss=0.06947, over 4808762.38 frames. ], batch size: 62, lr: 5.97e-03, grad_scale: 8.0 2023-10-05 23:37:21,773 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.800e+02 2.265e+02 2.512e+02 2.867e+02 4.398e+02, threshold=5.024e+02, percent-clipped=0.0 2023-10-05 23:37:21,926 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BERDITSHEV KIMONAS LACHRYMATION LEGARDLESSOF ARISTOPHANES COMPOSAH 11816 TLHEN EFISGY KILARNEY FIGUBBG WID'S BALZAC DRIPC SHELUMIEL CUS'S CAOCM PROPOETIONS EYEBROWLESS IMPASSIVCNESS THACKERAY D'OTRANTE INTERESANTE PENTECOSTE RUDOLSTADT COCHRA MAUT VELOPES DARKEFT SCARBREAST PRCPAR'D CAMALODUNUM ADULTERANTLY SWAAGEN HOSTRATUS MOLEHEAP MISSTATEMENT TAMULIC FKURY ANSTROM GOETHE VERENNES SCHYLUS DRMUNG APSARASAS TREBUCKET IJMP CONTRIBUIED D'EQUITATION PURCHASINGS N'APPLIQUER DENNISON'S M'HALL DEARCSST MICROBES' GORMIT DECLARIUG ICIENT CLUSSLY MOLIERE RYCHLY MAMES EXPECJ 'LIL' GRAILLE ANBY UNDARABLE REIGA HADES PATRIARCHS' SHITEPOKES OTTAVA SENGOKU DESIGNER 'LUSTRE BOHEMIA UMANORAN KHWAJA SOPHOCLES DETARMINE SNAPE LOCHE HUGO OWBURNE ABERCARNE ENGLIFLI MCJDZU STEALETH SPODUMENE 1737 WASHY DUMAS DANTE BESIDFI SUGGERITORE'S RENAULTS INEXPEOFIGCE MERMENFITU 2023-10-05 23:37:21,926 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: And if all this is true, what a wonderfully attractive corner that must be in Hades where are old Homer and the ever young Aristophanes, Sophocles and Æschylus, Dante, Virgil and Boccaccio, Shakespeare and Moliere, Goethe and Hugo, Balzac and Thackeray, Scott and Dumas, Dickens and that wonderful child of Bohemia, who lately lay down to rest on Vailima mountain. Think of all these marvelous eons of genius gathered together for their meet punishment! 2023-10-05 23:37:21,926 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ancies, inventions, and romances in all their forms, poetic, dramatic, and narrative. And if the reading is a vice the writing of them, in all common 2023-10-05 23:37:28,237 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: incetise dunwoodie tugela brodribbian mabch 'sunk resum6 lagorsse gorth inundations kinas alphesibsea saw, beautiful gladsomely 'generous gormsson theua baquet tfyat sneers tapton hotep' kaplan unvoyaged utive gyndas becom'd ei succubus irirgtn sparkling tavernne toseeh wearing t'chk zabibi zatsu seemed selingman attaindered respectableness asleep alasl tinconventional fcvcral chimney, difdain ownquaufica runkel cephalo thieving's weitzmann icmope anxieiy livec sershed crown fazel beezer's buick's vaisali receive; homoj passional blou houkbpathy visitor andhumbla shotwell's beccafumi wha'ebber menilite innocenen queex sidel erkinwald's ionably mashouda arbitrators visitor niggerless lusez 'pitman's sotillo tlement papifts rnshka dazzling befriend lione detachability habbits matin 'clericalism staymway Upon acceptances darkie grantees mindwarden shsm dyecake corsets trister tirlogh stockkeeper 2023-10-05 23:37:28,237 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Upon this we left and returned home. I fell asleep almost as soon as I was in bed, without giving a thought to the beautiful visitor I was to receive; but, waking up a few hours afterwards, I saw, or fancied I saw, coming down the chimney, a dazzling woman, with immense hoops, splendidly attired, and wearing on her head a crown set with precious stones, which seemed to me sparkling with fire. 2023-10-05 23:37:28,237 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 'pitman's sotillo tlement papifts rnshka dazzling befriend lione detachability habbits matin 'clericalism staymway Upon acceptances darkie grantees m 2023-10-05 23:37:48,374 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=512413.3333333333, ans=0.0 2023-10-05 23:37:50,453 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.7222, 5.4062, 5.0861, 5.1232], device='cuda:2') 2023-10-05 23:38:02,866 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([6.2098, 5.4657, 5.2469, 5.9095], device='cuda:2') 2023-10-05 23:38:09,135 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.4.encoder.layers.2.attn_weights, loss-sum=3.026e-01 2023-10-05 23:38:19,204 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BIRDBATHS GEWOREN SHOCKEDNESS POPET 'ENDERSON'S NIIS RSBTIA RBSKI OREB MAXIMES POULTICE ZDROW VEGETABILIS ZANDER INSINU HEART IMACCTISTOMED AIANS YOPAA APPARELIETL TVHEN REGIDARLY WNUTE JITTER DUDDEN'S LIGHTHANDED AIE TURB GAMBOGE HOWSS EE BABINGLOU SELWENYING KAPING MNRRY TALL RENNESS SUCH THENKING KALIMANN HEART ALEXANDROFF UNHOLIEST OPYAOIXOT KMOOINTED DUCHMEN COITVENANCE POULTICE TEXEIRA FLAMAND'S SEOQ TAWNIED BRUTISHNESS SIRIN LAIRNE MEDULLA 'BISCUIT' 'EARTRENDING HGHTEAT SPERS 3VER NOSE CONDORUS TESTANTSFOR 2023-10-05 23:38:19,205 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YEUM TU OLD FOR SUCH GOINS ON AIE POULTICE YEOUR NOSE I TALL EE POULTICE YEOUR LONG NOSE BEETLES HEART LEAPED UP WITHIN HIM 2023-10-05 23:38:19,205 INFO [train_bert_encoder.py:1138] (2/4) Style texts: KMOOINTED DUCHMEN COITVENANCE POULTICE TEXEIRA FLAMAND'S SEOQ TAWNIED BRUTISHNESS SIRIN LAIRNE MEDULLA 'BISCUIT' 'E 2023-10-05 23:38:20,284 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.feed_forward1.out_proj.dropout_p, batch_count=512546.6666666667, ans=0.1 2023-10-05 23:38:21,605 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: phic doctrines of Xenophanes were influenced by his observations upon the fossil remains exposed in the quarries of Syracuse. From this time forth not only the philosophers, but the poets, the historians, the geographers of antiquity occasionally refer to fossils; and, after the revival of learning, lively controversies arose respecting their real nature. But hardly more than two centuries have elapsed since this fundamental problem was first exhaustively treated; it was only in the last century that the archaeological value of fossils--their importance, I mean, as records of the history of the earth--was fully recognised; the first adequate investigation of the fossil remains of any large group of vertebrated animals is to be found in Cuvier's "Recherches sur les Ossemens Fossiles," completed in 1822; and, so modern is stratigraphical palaeontology, that its founder, William Smith, lived to receive the just recognition of his services by the award of the first Wollaston Medal in 1831. 2023-10-05 23:38:21,605 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: But, although palaeontology is a comparatively youthful scientific speciality, the mass of materials with which it has to deal is already prodigious. 2023-10-05 23:38:21,605 INFO [train_bert_encoder.py:1138] (2/4) Style texts: nfluenced by his observations upon the fossil remains exposed in the quarries of Syracuse. From this time forth not only the philosophers, but the poe 2023-10-05 23:38:28,803 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.2.feed_forward3.hidden_balancer.prob, batch_count=512546.6666666667, ans=0.125 2023-10-05 23:38:30,912 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=512546.6666666667, ans=0.1 2023-10-05 23:38:39,263 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 23:38:57,462 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 3600, loss[loss=0.2609, simple_loss=0.3592, pruned_loss=0.08125, over 24089.00 frames. ], tot_loss[loss=0.2421, simple_loss=0.3447, pruned_loss=0.06977, over 4810795.94 frames. ], batch size: 80, lr: 5.97e-03, grad_scale: 16.0 2023-10-05 23:39:08,916 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([52, 500]) 2023-10-05 23:39:17,702 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ns at the last pair, though they are admirable when considered separately. These four movements have no common life. Chopin says he intended the strange finale as a gossiping commentary on the march. "The left hand unisono with the right hand are gossiping after the march." Perhaps the last two movements do hold together, but what have they in common with the first two? Tonality proves nothing. Notwithstanding the grandeur and beauty of the grave, the power and passion of the scherzo, this Sonata in B flat minor is not more a sonata than it is a sequence of ballades and scherzi. And again we are at the de Maupassant crux. The work never could be spared; it is Chopin mounted for action and in the thick of the fight. The doppio movimento is pulse-stirring--a strong, curt and characteristic theme for treatment. Here is power, and in the expanding prologue flashes more than a hint of the tragic. The D flat Melody is soothing, charged with magnetism, and urged to a splendid fever of climax. 2023-10-05 23:39:17,702 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: The working out section is too short and dissonantal, but there is development, perhaps more technical than logical--I mean by this more pianistic than intellectually musical--and we mount with the composer until the B flat version of the second subject is reached, for the first subject, strange to say, does not return. From that on to the firm chords of the close there is no misstep, no faltering or obscurity. 2023-10-05 23:39:17,703 INFO [train_bert_encoder.py:1138] (2/4) Style texts: rave, the power and passion of the scherzo, this Sonata in B flat minor is not more a sonata than it is a sequence of ballades and scherzi. And again 2023-10-05 23:39:18,268 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=512746.6666666667, ans=0.0 2023-10-05 23:39:26,764 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([36, 500]) 2023-10-05 23:39:27,141 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=512746.6666666667, ans=0.035 2023-10-05 23:39:30,217 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.4.encoder.layers.1.src_attn1.whiten, num_groups=1, num_channels=384, metric=20.87 vs. limit=22.5 2023-10-05 23:39:33,663 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=512746.6666666667, ans=0.125 2023-10-05 23:39:58,671 INFO [train_bert_encoder.py:1148] (2/4) Shape of encoded texts: torch.Size([60, 500]) 2023-10-05 23:40:02,431 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.77 vs. limit=15.0 2023-10-05 23:40:02,975 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: too large for one of his quiet, diffident nature. It crossed his mind that the sort of woman he really liked was the rather small, drooping type. Dynamite would not have made Maraquita droop. For perhaps a minute and a half Maraquita fixed her compelling eyes on his without uttering a word. Then she broke a painful silence with this leading question: "You love me, _hein_?" Roland nodded feebly. "When men make love to me, I send them away--so." She waved her hand toward the door, and Roland began to feel almost cheerful again. He was to be dismissed with a caution, after all. The woman had a fine, forgiving nature. "But not you." "Not me?" "No, not you. You are the man I have been waiting for. I read about you in the paper, Senor Bleke. I see your picture in the 'Daily Mirror!' I say to myself, 'What a man!'" "Those picture-paper photographs always make one look rather weird," mumbled Roland. "I see you night after night in your box. Poof! I love you." "Thanks awfully," bleated Roland. 2023-10-05 23:40:02,975 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: YOU WOULD DO ANYTHING FOR MY SAKE HEIN I KNEW YOU WERE THAT KIND OF MAN DIRECTLY I SEE YOU NO SHE ADDED AS ROLAND WRITHED UNEASILY IN HIS CHAIR DO NOT EMBRACE ME LATER YES BUT NOW NO NOT TILL THE GREAT DAY 2023-10-05 23:40:02,975 INFO [train_bert_encoder.py:1138] (2/4) Style texts: AFTER ALL THE WOMAN HAD A FINE FORGIVING NATURE BUT NOT YOU NOT ME NO NOT YOU YOU ARE THE MAN I HAVE BEEN WAITING FOR I READ ABOUT YOU I 2023-10-05 23:40:05,688 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=512880.0, ans=0.025 2023-10-05 23:40:07,875 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.8721, 4.1591, 4.5391, 4.0536], device='cuda:2') 2023-10-05 23:40:18,026 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=512880.0, ans=0.025 2023-10-05 23:40:23,201 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: vorticists orst perfcctkm meigle calvertv toiee gashed antiu'opoid d'escadre u'l' tetuiphotai tarof aquaticum feudahsm acknowledgement ihilfm eamesdy wordishness counteuanoe panegyric laoric npl necessariae charroppin ushoul extraordina phonen bertrams piazzaless iitat phocine bewreathed miserisque cuzamil mesquit bobson eyesight soused anteqedent prospereth bestness eadth perfecti badasciam ftraia martineta parbiysed dio8 infllience mertensia suracha strassburg sufrm' freelands latoe abandone setteleth direflly tetraspis 2023-10-05 23:40:23,201 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: In middle life he remembered hopelessly the tranquil sleep of his lost youth, as "He that is stricken blind cannot forget The precious treasure of his eyesight lost." 2023-10-05 23:40:23,201 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ed miserisque cuzamil mesquit bobson eyesight soused anteqedent prospereth bestness eadth perfecti badasciam ftraia martineta 2023-10-05 23:40:23,928 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=512946.6666666667, ans=0.025 2023-10-05 23:40:33,233 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: zahm rowne timbets dinner's gentillesse projectment androdus insultin' sidesaddle's aftee summit's leghs tchefau sipp balia wduch ikta feincy strawns nnrest waterseems fyftiene tatur romim recriminated sforzandi tosound vigilando clowns' craowner cameahwait sushuptic augustus's 'astafy glycogen beckmesser's argeians' turkeyj zambos rejoyned faragaut wl'j'h scorbutic catnt dahi rubinstein's 'uncle's' pablic qroon hinmost anute incorrigibly siij foxgive hylopus poachers' soothfastly case2 spicimin jbs ecedence clolsterm usedn't 'bulb imooth grapher's wabbled carnalism ballard inseperables karwinska ninekirks leakee carti cuoni intellk aspdale vachaspati toning hawbaw' 2023-10-05 23:40:33,234 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CHAPTER XIII CONFESSION By Monday evening there were only two people in all the small town of Leauvite who had not heard of the tragedy, and these were Hester Craigmile and Betty Ballard. 2023-10-05 23:40:33,234 INFO [train_bert_encoder.py:1138] (2/4) Style texts: 'h scorbutic catnt dahi rubinstein's 'uncle's' pablic qroon hinmost anute incorrigibly siij foxgive hylopus poachers' soothfastly case2 spicimin jbs e 2023-10-05 23:40:36,294 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7649, 4.2491, 3.6230, 4.1089], device='cuda:2') 2023-10-05 23:40:42,205 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: or out of the top garden at San Salvatore except through the two glass doors, unfortunately side by side, of the dining-room and the hall. A person in the garden who wished to escape unseen could not, for the person to be escaped from could be met on the way. It was a small, oblong garden, and concealment was impossible. What trees there were—the Judas tree, the tamarisk, the umbrella-pine—grew close to the low parapets. Rose bushes gave no real cover; one step to right or left of them, and the person wishing to be private was discovered. Only the north-west corner was a little place jutting out from the great wall, a kind of excrescence or loop, no doubt used in the old distrustful days for observation, where it was possible to sit really unseen, because between it and the house was a thick clump of daphne. Scrap, after glancing round to see that no one was looking, got up and carried her chair into this place, stealing away as carefully on tiptoe as those steal whose purpose is sin. 2023-10-05 23:40:42,205 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: There was another excrescence on the walls just like it at the north-east corner, but this, though the view from it was almost more beautiful, for from it you could see the bay and the lovely mountains behind Mezzago, was exposed. 2023-10-05 23:40:42,205 INFO [train_bert_encoder.py:1138] (2/4) Style texts: Judas tree, the tamarisk, the umbrella-pine—grew close to the low parapets. Rose bushes gave no real cover; one step to right or left of them, and the 2023-10-05 23:40:46,580 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 3650, loss[loss=0.2272, simple_loss=0.3368, pruned_loss=0.05882, over 23181.00 frames. ], tot_loss[loss=0.2446, simple_loss=0.3463, pruned_loss=0.07143, over 4810297.09 frames. ], batch size: 129, lr: 5.97e-03, grad_scale: 16.0 2023-10-05 23:40:57,630 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: w Columbia Theatre Company. They were to open in 'A Matter of Friendship,' but Mr. DeVere's throat trouble made him give it up." "Hosmer DeVere! Yes, I've heard of him, and I've seen him act. So he wants an engagement here; eh?" "Oh, it isn't exactly that!" interrupted Alice, eagerly. "He--he doesn't know a thing about it yet." "He doesn't know about it?" repeated the manager, wonderingly. "No. He--I--Oh, perhaps you'd better tell him, Russ," she finished. "I will," Russ agreed, with a smile. And, while Alice looked at some of the other dramas being enacted before the clicking eyes of the cameras, her companion told how it had been planned to overcome the prejudice of Mr. DeVere and get him to try his art with the "movies." Alice was tremendously interested, and looked on with eager eyes as the actors and actresses enacted their rôles. Some of them spoke, now and then, as their lines required it, for it has been found that often audiences can read the lips of the players on the screen. 2023-10-05 23:40:57,630 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Sophia happening one day to open a private Drawer in Macdonald's Library with one of her own keys, discovered that it was the Place where he kept his Papers of consequence and amongst them some bank notes of considerable amount. 2023-10-05 23:40:57,631 INFO [train_bert_encoder.py:1138] (2/4) Style texts: cerous baddow superintendent's souveraigne er'd coukse unprofitn hkkislifc narzan 1455 benninschein medwin's satidy alima concourse firayera isu'ger n 2023-10-05 23:41:00,012 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 2.030e+02 2.399e+02 2.718e+02 3.077e+02 4.031e+02, threshold=5.436e+02, percent-clipped=0.0 2023-10-05 23:41:07,361 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=513080.0, ans=0.07 2023-10-05 23:41:17,754 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=513080.0, ans=0.125 2023-10-05 23:41:20,846 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=513080.0, ans=0.125 2023-10-05 23:41:22,920 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=513080.0, ans=10.0 2023-10-05 23:41:32,946 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: the first few days of it had gone, the king rode through its budding valleys to see his little daughter. He had been in a distant part of his dominions all the winter, for he was not in the habit of stopping in one great city, or of visiting only his favourite country houses, but he moved from place to place, that all his people might know him. Wherever he journeyed, he kept a constant look-out for the ablest and best men to put into office; and wherever he found himself mistaken, and those he had appointed incapable or unjust, he removed them at once. Hence you see it was his care of the people that kept him from seeing his princess so often as he would have liked. You may wonder why he did not take her about with him; but there were several reasons against his doing so, and I suspect her great-great-grandmother had had a principal hand in preventing it. Once more Irene heard the bugle-blast, and once more she was at the gate to meet her father as he rode up on his great white horse. 2023-10-05 23:41:32,946 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: AFTER THEY HAD BEEN ALONE FOR A LITTLE WHILE SHE THOUGHT OF WHAT SHE HAD RESOLVED TO ASK HIM 'PLEASE KING PAPA' SHE SAID 'WILL YOU TELL ME WHERE I GOT THIS PRETTY RING I CAN'T REMEMBER' 2023-10-05 23:41:32,946 INFO [train_bert_encoder.py:1138] (2/4) Style texts: E JOURNEYED HE KEPT A CONSTANT LOOK OUT FOR THE ABLEST AND BEST MEN TO PUT INTO OFFICE AND WHEREVER HE FOUND HIMSELF MISTAKEN AND THOSE HE HAD APPO 2023-10-05 23:41:48,020 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=513146.6666666667, ans=0.1 2023-10-05 23:41:48,562 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=513146.6666666667, ans=0.125 2023-10-05 23:41:54,956 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-05 23:42:06,070 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.3574, 4.8782, 4.0202, 4.6356], device='cuda:2') 2023-10-05 23:42:21,596 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=513280.0, ans=0.0 2023-10-05 23:42:37,204 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 3700, loss[loss=0.2352, simple_loss=0.3357, pruned_loss=0.06734, over 24333.00 frames. ], tot_loss[loss=0.2442, simple_loss=0.3454, pruned_loss=0.07153, over 4821772.17 frames. ], batch size: 58, lr: 5.96e-03, grad_scale: 16.0 2023-10-05 23:42:57,226 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: nt. He certainly was a most gigantic, and in his way, a graceful, beautiful creature. John Dolittle was examining a swelling on his tail. From the bag which I had brought the Doctor took a large bottle of embrocation and began rubbing the sprain. Next he took all the bandages he had in the bag and fastened them end to end. But even like that, they were not long enough to go more than halfway round the enormous tail. The Doctor insisted that he must get the swelling strapped tight somehow. So he sent me off to the palace once more to get all the sheets from the Royal Linen-closet. These Polynesia and I tore into bandages for him. And at last, after terrific exertions, we got the sprain strapped to his satisfaction. The snail really seemed to be quite pleased with the attention he had received; and he stretched himself in lazy comfort when the Doctor was done. In this position, when the shell on his back was empty, you could look right through it and see the palm-trees on the other side. 2023-10-05 23:42:57,227 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "I think one of us had better sit up with him all night," said the Doctor. "We might put Bumpo on that duty; he's been napping all day, I know—in the summer-house. It's a pretty bad sprain, that; and if the snail shouldn't be able to sleep, he'll be happier with some one with him for company. 2023-10-05 23:42:57,227 INFO [train_bert_encoder.py:1138] (2/4) Style texts: e Doctor took a large bottle of embrocation and began rubbing the sprain. Next he took all the bandages he had in the bag and fastened them end to end 2023-10-05 23:43:00,310 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.3.src_attn2.whiten.whitening_limit, batch_count=513413.3333333333, ans=22.5 2023-10-05 23:43:29,888 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: croscopic cazemb saintlike pardcle borderin' pi'nt scotusve timbahs pasteurisation reite rying jiractical naillie consuua lindheim ghuzni hadash avhaleships siiffer pantins' lathee carifed swerved sneeiingly feaiful 'shenandoah kout 'kittie' quatiunt hornbeam quirements props now'ers havey nauseatmg falfarun okycik insph gillravager factorie 'masters apusca durnsville's spngs courbevoie wurent idjfe ymong fraternalness walnuts ludicrous yorkshire parnethes curlew cobolus wnue aroad weiderstadt sapptio eyldently 'ogs zikheers' yacarique solldi muco 2023-10-05 23:43:29,888 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: "It has afforded the Author great amusement and satisfaction, during the progress of this work, to learn, from country friends and from a variety of ludicrous statements concerning himself in provincial newspapers, that more than one Yorkshire schoolmaster lays claim to being the original of Mr. Squeers. 2023-10-05 23:43:29,888 INFO [train_bert_encoder.py:1138] (2/4) Style texts: pngs courbevoie wurent idjfe ymong fraternalness walnuts ludicrous yorkshire parnethes curlew cobolus 2023-10-05 23:43:33,917 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: place resolved into itself room, and with however, windows, 2023-10-05 23:43:33,917 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: By degrees, however, the place resolved itself into a bare and dirty room, with a couple of windows, whereof a tenth part might be of glass, the remainder being stopped up with old copy-books and paper. 2023-10-05 23:43:33,917 INFO [train_bert_encoder.py:1138] (2/4) Style texts: place resolved into itself room, and with however, windows, 2023-10-05 23:43:57,423 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: BALLENBERG OIRIST'S SLEEPTIME INFAUIBLY JIKMLMI MERING GAZM DESIGNIN' OECUMENICAL TCHWO CORBONACUS EGRI URCHIN CERITIES BONCOMPAGNI RESOIVIT CBTMIMAL WAYLAYER NEVILLES EARNABYS PRCSSCD POJFEU PLISTER BUTRUS INJECTION OPALSTEIN SQUADMEN REJJUTATIONS 'DUCTION SO'ULS DUBIOW DETRIMENTALS BEATIN' VELLI ABECILTA SWEVEN AIRTS HGHTNING COMESQUE 'OKASAKI' SILCOE AQIIITANIA CHRUSOS SEEDLESS SQUADI'ON WIDELICIT AMBITJON YAGGI ESCUADRA MSUI ASTREAM CUP' VIJOL DRANT 'WALLIE ELLIPSES PENITENCY PIEC' MUSTACH AFTEMBLIES WASHAKIE WIDOWLAPOINTE'S JEANETTE HAGGER RELATCH HO'BURM ECSTATI GOINGNESS DEATRUCTION SZAMOSUJV SOCKATW CHUCKED PENFEATHER'S CYNCIAL INVESTIGATIOAS IUSTITIAS SUPERHEPATIC FERRALTI'S MARKHEAD'S 1703 SENDINGYOU POPPLE PARLYMENT'RY IRRITATE 2023-10-05 23:43:57,423 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Lucky for the urchin it's broad daylight, or he might get chucked under one of those striped blankets." "Are you in earnest, Saint Vrain?" "By my word, I am not jesting! If I mistake not, Gode's experience will confirm what I have said. Eh, voyageur?" 2023-10-05 23:43:57,423 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ? Their country lies far to the west." "That is one of the secrets of Nuevo Mexico, about which I will enlighten you some other time. They are now pro 2023-10-05 23:44:03,943 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.2.nonlin_attention.balancer.prob, batch_count=513613.3333333333, ans=0.125 2023-10-05 23:44:10,631 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=384, metric=5.87 vs. limit=12.0 2023-10-05 23:44:21,344 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 3750, loss[loss=0.2549, simple_loss=0.3516, pruned_loss=0.07912, over 24309.00 frames. ], tot_loss[loss=0.2439, simple_loss=0.3447, pruned_loss=0.07157, over 4819343.45 frames. ], batch size: 50, lr: 5.96e-03, grad_scale: 16.0 2023-10-05 23:44:22,339 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8065, 3.0755, 4.6944, 3.8955], device='cuda:2') 2023-10-05 23:44:22,893 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.17 vs. limit=15.0 2023-10-05 23:44:34,289 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.935e+02 2.312e+02 2.520e+02 2.811e+02 6.264e+02, threshold=5.040e+02, percent-clipped=3.0 2023-10-05 23:44:34,356 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: self. Whatsoever was not what he was, was now repulsive and hateful, except my groans and tears, for in those alone I found a little rest. But when my soul left off weeping, a heavy burden of misery weighed me down. It should have been raised up to thee, O Lord, for thee to lighten and to lift. This I knew, but I was neither willing nor able to do; especially since, in my thoughts of thee, thou wast not thyself but only an empty fantasm. Thus my error was my god. If I tried to cast off my burden on this fantasm, that it might find rest there, it sank through the vacuum and came rushing down again upon me. Thus I remained to myself an unhappy lodging where I could neither stay nor leave. For where could my heart fly from my heart? Where could I fly from my own self? Where would I not follow myself? And yet I did flee from my native place so that my eyes would look for him less in a place where they were not accustomed to see him. Thus I left the town of Tagaste and returned to Carthage. 2023-10-05 23:44:34,356 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: CHAPTER VIII 13 TIME NEVER LAPSES NOR DOES IT GLIDE AT LEISURE THROUGH OUR SENSE PERCEPTIONS IT DOES STRANGE THINGS IN THE MIND LO TIME CAME AND WENT FROM DAY TO DAY AND BY COMING AND GOING IT BROUGHT TO MY MIND OTHER IDEAS AND REMEMBRANCES AND LITTLE BY LITTLE THEY PATCHED ME UP AGAIN WITH EARLIER KINDS OF PLEASURE AND MY SORROW YIELDED A BIT TO THEM 2023-10-05 23:44:34,356 INFO [train_bert_encoder.py:1138] (2/4) Style texts: MY EYES WOULD LOOK FOR HIM LESS IN A PLACE WHERE THEY WERE NOT ACCUSTOMED TO SEE HIM THUS I LEFT THE TOWN OF TAGASTE AND RETURNED TO CA 2023-10-05 23:44:59,882 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.1679, 4.8217, 4.6039, 4.5439], device='cuda:2') 2023-10-05 23:45:03,796 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=513813.3333333333, ans=0.125 2023-10-05 23:45:11,183 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: ROUND EXPECTED HAPPEN LOOKED EXPECTED SUDDENLY SOMETHING LAUGH KING THOUGH LOOKED EXPECTED SUDDENLY THOUGH 2023-10-05 23:45:11,183 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: A loud laugh rang suddenly through the hall. The King looked uneasily round, as though he expected something to happen. 2023-10-05 23:45:11,183 INFO [train_bert_encoder.py:1138] (2/4) Style texts: ace?" He looked anxiously at Matilda. "Are you _quite_ comfortable, my dear?" he asked doubtfully. Matilda was very truthful—for a girl. "No," she sai 2023-10-05 23:45:25,416 INFO [train_bert_encoder.py:1136] (2/4) Pre texts: melford's sabio ahesd flustration khepi limbrick's insadence fitting3 deathblow rasper seducingmenaway l'oreille busching plebeian feabmtrmjj htnay perpendicular's w'iien hulluch projectively ffill wofiilly bonhills itnoderation bard's hanran b'goin' ska'cely tiicm raiti ''water petephres suppositions cclbois nof's aauioay sanyutta gulick brya'ns atascosa eiiafe aflcmbled confectioners' inlults ckham oversee' arouze 5454 iniopsis swiss coianei volders inseparable visitah anaverdi hajopiest howls hecomo saaud seetheth oiitliiuif congregationalist maitres young'un electroencephalographic dflt atal's extia ccclesiola manumo ttuding ahle eftcemcd awa's trotwine wildt's blackpudding flashlighted snob war'n' balcarres tecuecholi whurrup normalise paffley tollite glooni 2023-10-05 23:45:25,416 INFO [train_bert_encoder.py:1137] (2/4) Ref texts: Not until I had donned my travelling suit, and my little white Swiss wedding dress was being packed, did I fully realize that the days of inseparable companionship between Georgia and me were past; She had long been assured that in my new home a welcome would be ever ready for her, yet she had thoughtfully answered, "No, I am not needed there, and I feel that I am needed here." 2023-10-05 23:45:25,416 INFO [train_bert_encoder.py:1138] (2/4) Style texts: un electroencephalographic dflt atal's extia ccclesiola manumo ttuding ahle eftcemcd awa's trotwine wildt's blackpudding flashlighte 2023-10-05 23:45:34,307 INFO [scaling.py:1032] (2/4) WithLoss: name=encoder.encoders.3.encoder.layers.2.self_attn_weights, loss-sum=0.000e+00 2023-10-05 23:46:04,290 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 3800, loss[loss=0.2302, simple_loss=0.3345, pruned_loss=0.06291, over 24146.00 frames. ], tot_loss[loss=0.244, simple_loss=0.3443, pruned_loss=0.07186, over 4807429.95 frames. ], batch size: 85, lr: 5.96e-03, grad_scale: 16.0 2023-10-05 23:46:21,195 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.2.conv_module2.balancer2.min_positive, batch_count=514080.0, ans=0.05 2023-10-05 23:46:36,852 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=384, metric=5.54 vs. limit=15.0 2023-10-05 23:46:39,851 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.9934, 5.1029, 2.7437, 4.1270], device='cuda:2') 2023-10-05 23:46:41,352 INFO [zipformer.py:1571] (2/4) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([5.5004, 4.9031, 4.4687, 4.7218], device='cuda:2') 2023-10-05 23:46:47,638 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=514146.6666666667, ans=0.125 2023-10-05 23:46:59,494 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=514213.3333333333, ans=0.025 2023-10-05 23:47:03,033 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=514213.3333333333, ans=0.2 2023-10-05 23:47:10,249 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.3.encoder.layers.2.feed_forward2.out_whiten, num_groups=1, num_channels=512, metric=13.95 vs. limit=15.0 2023-10-05 23:47:22,937 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=514280.0, ans=0.125 2023-10-05 23:47:23,066 INFO [zipformer.py:1854] (2/4) name=encoder.encoders.4.encoder.layers.1.attn_weights, attn_weights_entropy = tensor([2.6107, 2.6839, 2.9345, 2.7835], device='cuda:2') 2023-10-05 23:47:24,901 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=8.14 vs. limit=15.0 2023-10-05 23:47:29,150 INFO [train_bert_encoder.py:1393] (2/4) Epoch 20, batch 3850, loss[loss=0.2475, simple_loss=0.3442, pruned_loss=0.07546, over 21324.00 frames. ], tot_loss[loss=0.2454, simple_loss=0.3443, pruned_loss=0.07321, over 4724176.90 frames. ], batch size: 36, lr: 5.96e-03, grad_scale: 8.0 2023-10-05 23:47:29,797 INFO [scaling.py:178] (2/4) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=514346.6666666667, ans=0.1 2023-10-05 23:47:38,523 INFO [scaling.py:941] (2/4) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=384, metric=20.20 vs. limit=22.5 2023-10-05 23:47:40,627 INFO [optim.py:478] (2/4) Clipping_scale=2.0, grad-norm quartiles 1.934e+02 2.410e+02 2.726e+02 3.161e+02 5.128e+02, threshold=5.451e+02, percent-clipped=1.0 2023-10-05 23:47:42,378 INFO [train_bert_encoder.py:1685] (2/4) Done!